Datomic solves reference traversal in databases
Sql doesn't have references, only scalars and tables. So the SQL programmer has three choices:
- don't program with references - use only scalars in our biz logic,
- map between object references and scalar relations in our heads,
- abstract the mapping with an Object Relational Mapper
ORMs give full graph capabilities, queries and reference traversal, at a cost. You must choose between:
- many round trips to the database
- deeply nested JOINs
Both perform ever worse as application gets bigger.
So the abstraction leaks, we must drop underneath it and hand-code optimizations. What can we batch to save round trips? Or introduce object caches to skip round trips entirely. Now we are a distributed system, so we need a programmer to hand-tune where we are in the CAP spectrum. How fast do the reads need to be? How much inconsistency can we tolerate?
These decisions become essential to the functioning of the system, and manifest in both server code coordinating with database, and UI code coordinating with server. This is the backend-for-frontend pattern, is O(n) growth of boilerplate that is resistant to abstraction.
This strain also explains the failure of REST:
REST does a shitty job of efficiently expressing relational data. I mean REST has its place. For example, it has very predictable performance and well-known cache characteristics. The problem is when you want to fetch data in repeated rounds, or when you want to fetch data that isn't expressed well as a hierarchy (think a graph with cycles -- not uncommon). That's where it breaks down. I think you can get pretty far with batched REST, but I'd like to see some way to query graphs in an easier way."
- Pete Hunt, Facebook, 2014 April
I think these intractable scaling problems are the deep reason that people hate ORMs.
Mongo's thesis is, we can do references in the db, and scale them, if we store trees instead of graphs. So for small apps, the boilerplate vanishes. But for non-toy apps, constraining state to be shaped like trees turns out to be hugely limiting. So back to SQL ORMs we go.
Datomic's thesis is, we can do references as proper graphs, and query them, and do ACID, and not lose anything critical as compared with sql.