Development Diary #2

Hot on the heels of our last XTDB site:/blog/dev-diary-jul-19/[dev diary] we have published site:/blog/xtdb-jdbc/[xtdb-jdbc], facilitating users to quickly get going without committing to a Kafka deployment model. As wrote about in the diary, the JDBC form of XTDB is aimed at that sweet spot, where users will already likely have a relational data-store setup in place, and the volumes of data may not be high enough to warrant the up-front Kafka investment.

This said, we’re still loving Kafka - Confluent’s managed cloud solution is worth considering as a good stepping stone, or even an end solution.

Kafka gives you a more authentic separation of concerns that you need with the database inside out approach, to handle vast amounts of data at scale. It is much more than just a message bus used for systems integration - it can be a golden store record of truth that is highly available and easily replayable.

To that end, the XTDB team has been cooking up a XTDB Kafka Connector for ease of plugging XTDB into an existing Kafka installation and for data to be streamed in to produce subsequent bitemporal graph queries. More details to follow.

Community / Unbundled

It’s been great to see so many people playing around with XTDB and the odd case-study emerging.

XTDB is unbundled and open source, and so we always hoped that people would start building out their own modules and different ways of using it. So far, we’ve seen XTDB built to use Active Objects as the event-log (mirroring the recent JDBC effort), Xodus as the node-local K/V store, and there have been experiments such as running on top of AWS SQS and using Pathom/Fulcro. There have been many other conversations about what underlying tools and platforms could support a running XTDB setup and so hopefully more community contributions will continue to blossom out.

On Tour

We’ve gone on tour to talk about XTDB. Håkan gave a talk at ClojuTRE on the Design and Implementaton of a Bitemporal Database. The last time he had visited Helsinki was for Assembly94.

(Because of the immutable nature of YouTube videos, you’ll note XTDB is still referred to as "Crux" in the video and the title.)

Figure 1. Håkan Råberg talking about the internals of XTDB

Johanna gave an XTDB tutorial at ClojuTRE, gracefully added to the schedule by our friends at Metosin.

Jeremy and I talked about Event Streaming Architectures and Temporal Databases at Strange Loop. Jeremy also ran an unsession that was well attended.

(Because of the immutable nature of YouTube videos, you’ll note XTDB is still referred to as "Crux" in the video and the title.)

Figure 2. Jeremy at Strange Loop 2019 talking about XTDB

The JUXT team have run training workshops at Heart of Clojure and we gave a talk to our friends at 'Scala in the City' hosted by Revolut at Canary Wharf.

Figure 3. Heart of Clojure XTDB Walkthrough

(Yes, once upon a time, XTDB went by the name of "Crux" instead.)

It’s been a fun journey since launch, but our thoughts now are turning to the medium and long term future.

The Future

We are growing the XTDB team with some exciting full-time additions and we are creating a new London based development center, that will be where future versions of XTDB core will be built from. When we formally start the BETA program in the next 2-3 months, we will do so with a global support capability including North America, Europe and Australia.

We have been looking long and hard at the roadmap and we’ve got exciting plans for XTDB and extending its range of functionality.

First though we will be entering a formal BETA program and we will inviting partners to join. As part of the BETA we want to see and understand how XTDB is performing in more production installations and of what improvements we will need to make.

If you want to simply get started with XTDB, checkout Joanna’s Space Adventure Tutorial.

Thanks for reading this journal. Come along and say hi: