XTDB 2.x Early Access
I used XTDB on a side project (written in Clojure) a couple years ago and was blown away by something that I think isn't even really stressed as a headliner feature for XTDB. That was the ability to treat queries as first-class data. XTDB's datalog queries can be provided as just native Clojure data structures, and that allows you to build complex queries using code. What this means is that it's much more DRY than copy-pasting and editing SQL templates, there's no ORM gumming up your performance and expressivity as your queries get more complex, and there's no half-baked implementation of an API for query composition: the _programming language_ is your tool for query composition. I couldn't tell you if there are other ways of getting this, as I'm not really a software engineer anymore, but it was a jaw-dropping experience for me to see how easy it could be to write a database layer, and far and away the best experience I've ever had writing one in any programming language.
I suppose what's at the heart of this, more than anything, is that Datalog has a near-trivial syntax that can transparently be accommodated by two simple data structures: lists and hash tables. Not so with SQL, whose COBOL-y nature obscures its syntax and makes it less straightforward to represent in everyday data structures.
We use XTDB 1 and have a multi replica production instance with about 600GB of data at ~100 million transactions. The biggest problem we have is with minor versions bumping the underlying RocksDB i dex version, requiring a full reindex of the main "golden" data store taking days to weeks.
AFAIK this is not solved in XTDB 1. I'm hoping XTDB 2 provides a better upgrade path for modestly sized databases like ours.
> XTDB is a bitemporal and dynamic relational database for SQL
You have my attention. I actually raised an eyebrow reading that, last time this happened was when ChatGPT was released.
Temporal is a world of pain in SQL, bitemporal... nice.
edit: for those who haven't heard what bitemporal is, Wikipedia has an example: https://en.wikipedia.org/wiki/Temporal_database#Using_two_ax...
I don't want to distract from the main points in the slightest (this looks really cool) - but I'm just curious what is the business plan? The main landing page offers booking a free demo.. but I didn't really get where they start making money?
The whole thing looks really nice and polished. JUXT releases some really cool opensource stuff (I use tick all the time). But.. is there some catch? Like a PRO version or something? How are they paying the bills?
Thanks for all the work you guys do :))
XTDB is a very interesting database to me. Not only is it distributed, but the 'bi-temporality' which is mentioned on this page means that it has essentially a built-in audit trail mechanism. I'm working on writing an RDF library for Clojure, and am looking forward to seeing whether XTDB really is as good as an RDF datastore as I suspect it is!
OK, so cynicism warning here.
After seeing what seems like oodles of sqlite and rocksdb plus a bespoke cross-machine coordinator databases posted on HN, is this like the new employment guaranty architecture? Do any of these have actual ability over something like ScyllaDB (I'm taking cassandra out because of the GC pauses probably being a problem)
RocksDB in particular is a single node SSD-tuned log-structured merge tree "database" written in a non-GC language. ScyllaDB is the same, but comes with the cassandra-protocol scaling and multiple datacenters.
Sure, go ahead and market your whatever, but if you're just (IMO) repackaging a log structured merge tree engine, with various newly implemented distributed transaction / conflict resolution coordination schemes .... you should probably show some good due diligence on if they are, you know, somewhat correct.
The Jepsen test suite basically shows every distributed data processing engine as flawed the first couple times. So much so I don't really trust your new fangled engine unless you pass Jepsen to some degree.
Is there some comparison metrics (lies, big lies, and benchmarks and all that) showing why these semi-bespoke databases are being used versus other, say, more proven, schemes?
My understanding is the XTDB is either a fork of Datomic, or was inspired by it? I suppose it's timely, given the top story on HN right now is "Datomic is Free" [0]
Is there anyone who's familiar with both and would like to share their thoughts on the two?
undefined
undefined
What is a datalog?
undefined