Hyperfiddle vs GraphQL (notes on Lacina launch post)
This is a super-draft note post, just random notes.
My notes on Open Sourcing Lacinia, our GraphQL Library for Clojure (2017), emphasis added.
Used by Walmart at Black Friday scale
When you open your Walmart store receipts on your phone or visit samsclub.com and view your receipts on the web, you’re being served by Lacinia. If you use Walmart Grocery, you’re using Lacinia.provide Walmart and Sam’s Club customers with instant access to their entire history of in-store receipts, directly from their iOS/Android smartphone or web browserWe manage a real-time feed of every purchase and return from over five thousand stores to the tune of 500 receipts per second and up to quadruple that number on Black Friday and throughout the holiday season.
Data is graph-shaped, and in Cassandra
we have a sizable Cassandra database of customers, receipts, and the associations linking them. Our entire server-side stack is written in Clojure, and built to service that incoming data feed efficiently and reliably.
Coupling between common service APIs and different applications consuming them
offer our immense amount of data to lots of groups within the company. Each has its own needs and concerns ...
- The mobile application teams only want the fields required to populate a concise view
- websites provide a richer shopping experience ... working with more viewing space and want to ask for more detailed information
- one or two core questions they need answered, query for just those specific parameters. Examples include Savings Catcher, which credits customers for purchased items if they have a lower price at a competitor’s storeEach of these teams has vastly different requirements on what subset of transaction data it needs to access. Making our data easily consumable, especially since we’re a small group of engineers, is a core challenge that a traditional REST API structure wasn’t solving for us.
Over time, we found ourselves in the unenviable position of maintaining, extending, and documenting a collection of APIs, each initially built for a specific use case. Each of these API used a different HTTP stack, was configured differently, and had its own ad hoc conventions for URL schemes, query parameters, and so forth, as inspired by the practices at the time they were built. Newer services used Pedestal and Component; older services used Ring handlers and scattered mutable state.We want to be free to evolve and improve our APIs without complex versioning schemes and complicated migration plans. We need to be able to confidently make changes to supporting code without stressing out about undiscovered side-effects in far-flung services. As a back-end team, we decided to stop making product decisions for product teams. Instead, we have empowered each team to query our system in ways that best support the specific product they’re building.
A key value prop for Walmart is GraphQL works on any database, since it's a spec.
How would Lacina fit in with Hyperfiddle? We'd host a Lacina adapter, so GraphQL tools work on it. But we already have that for Datomic, Lacina is gonna be worse. Lacina is for querying non-datomic databases. But these systems are not isomorphic. What is missing? Essentially, the datalog query, i think. Where are the API endpoints then?
We designed our schemas around our data models and clients can simply ask for what they need. Of course, we can’t know all possible use-cases, but when a client tells us they can’t query for what they want, we can can typically add the missing field or relationship, and deploy it in a matter of minutes. Often, our internal data model already includes the information, and it’s only a matter of updating the schema to expose the new field. Because clients control what data they see, adding new fields and types to an existing schema is always safe and always backwards compatible.
Hyperfiddle does this better, because Datomic is simpler than GraphQL, there aren't any types. In what way might it be worse? What if you want a derived field? (Use Datalog is a good start for this, so hyperfiddle would be better). Since there's no GraphQL, its just Datomic, you get direct access to the datastore. If it's captured, you can query it. So Hyperfiddle is strictly better (though it creates dependencies on your datastore).
Not only that, everything in the schema is fully discoverable and documented through introspection.
Hyperfiddle is too
We serve an instance of the GraphiQL (with an i!) web-based IDE: this allows developers on other teams to interactively build and execute queries. Lacinia automatically implements the introspection parts of the GraphQL specification; all we have to do is include user-facing documentation on fields and types.
Hyperfiddle natively does docs
Previously, when assisting other teams, we would throw around ad hoc curl commands, and it was always a challenge reproducing our client’s problems. Today, we pass around a GraphiQL link which can include the entire query — instant reproducibility.onboarding a new client to our service is just a matter of giving them the URL for GraphiQL, and they can quickly and interactively sketch out their queries and learn about all the fields and types through the interface. If they have problems or questions, they can send us their problematic queries.
Hyperfiddle does this better, because you can debug it actually inside the application, since application URL equals fiddle URL since the app is a fiddle.
A surprising offshoot of this is how much I, as a developer of the service, have come to rely on GraphiQL for my own daily workflow: it’s faster and easier to build a GraphQL query than it is to access the underlying Cassandra database directly. GraphiQL, on top of GraphQL, constitutes an exceptionally powerful tool.
Hyperfiddle does this better (why?) Hyperfiddle is literally built in Hyperfiddle