Hyperfiddle 2017 Year in Review

by Dustin Getz, http://www.hyperfiddle.net/

Here are a bunch of screenshots was the product evolved in 2017.

Jan 1, 2017 – more of a tech demo than a prototype

March

Things are interactive enough to be useful, sort of. We tell YC that we'll have users in like two months. LOL

Cross database queries (color indicates database)

Cross database forms–What does that even mean? We think it doesn't mean anything, because it's a database join rather than a single entity, you can't edit it transactionally; this is a rare cirumstance for readonly dashboards only

Load up them databases. Semantic web anyone?

April

Hyperfiddle has a git-like staging area, which it turns out is a critical primitive. Datomic's git-like time-traveling primitives mean we can apply transactions locally and see if they work, without actually writing through to the database.

This gif is overly confusing but it's the only one I have from this period

An early look at Hyperfiddle-in-Hyperfiddle. We are fully bootstrapped at this point and Hyperfiddle's UI is fully data driven and editable live (like Smalltalk). No more database wipes; delete the fixtures; delete the bootstrapping code; its self-hosting. We really work like this, every day.

Links can navigate entity graph, navigage to another query, or form a custom datomic transaction. Buttons have a clojure snippet to build up a transaction in code. 99% of these fit a few simple categories (create-new, create-new-managed-child, retract-entity) and these formulas are now auto-generated. But before we figured all that out, we had to code it by hand.

Hover to see the link's query params, as evaluated

Trying to figure out how to make a form that edits a component entity

Create-new sometimes needs to manage a parent-child ref (This stuff is all automatic now)

May

Many of the link formula-functions and transaction-functions are automatic now. Many of the links are now buttons, which means there's a transaction-function attached (as opposed to navigation of the entity graph)

June

www.dustingetz.com is made in hyperfiddle now, this is our first test of prod traffic.

Popovers are "branches", this is a big deal

The problem is that any staged tranction must always be valid to hydrate through datomic, or the app crashes and you can't fix it. So to do a managed parent-child ref (create new entity then link to it, atomically), you have to stage the new entity first, so you can link to it, otherwise there is a dangling tempid reference.

This breaks our performance, so Karl migrates us to reagent reactions, and now the UI is beginning to be almost responsive enough to use.

Making an entity navigation look like a radio. The radio changes a link, the link is embedded (like an iframe), toggling the radio changes the iframe

July

We started writing other researchers and people we think might be interested in this stuff. Most people don't write back, a couple poo-poo us. Other Datomic researchers are receptive. Our message needs a lot of work, and to be honest we don't even understand what we're building yet, not deeply. But that doesn't matter.

Interactive error reporting (checked query params)

Shadow links—Override a default system link to e.g. hide it. You shadow the fake link with a real database record that matches it

Using Hyperfiddle to generate migration transactions. That reagent code is building up datomic transactions and pprint'ing to a :pre

Migrating Hyperfiddle-in-Hyperfiddle by hand. Thread the needle to migrate the app in atomic units that don't break the app :)

August

We apply early to YC and tell them we'll have 100 users in 8 weeks. LOL

Starting to care about onboarding and user experience

Combo boxes modeled as hypermedia, live interactive

What's in a select option, really?

Above and below are identical, except select renderer toggled off

Once you embed the query, styling it as select options is a matter of CSS and HTML markup only

Dynamic scope is becoming important for advanced UIs. Ctx value is how the system keeps the right values in scope (e.g. which row we are rendering).

Navigating around the www blog and blog navigation

Youtube recording of us doing actual work in Hyperfiddle-in-Hyperfiddle (click for video)

September 2017 (ClojureNYC)

Conj talk is rejected, but ClojureNYC takes us. Things starting to look polished

Pink Boxes highlight recursive UI components, makes it easy to talk about

Some error reporting

TodoMVC and it doesn't suck, and inline docstrings are really cool

October – Hyperfiddle now can be used with pre-existing Datomic databases

Hyperfiddles are literate programs! Docstrings used everywhere. These docstrings are reflected out of the database (:db/doc)

Configurable domain/environment—use your own datomic databases. All the hardcoded ops stuff getting refactored out. In theory, you can selfhost Hyperfiddle now against your own Datomic databases.

This is how you add Datomic connections to your Hyperfiddle environment

Clojure Conj — Datomic Cloud announced

We show Stu what we're working on and start to figure out how Datomic licensing is going to work.

We give a lightning talk. Dustin decides to go live audience Q&A, miscalibrates the audience and starts talking about the object/relational impedance mismatch, the audience is lost and the Q&A is a massacre. A YCombinator ex-partner is present and watching us get rekt. Oh well. Time to work on our messaging.

November

Time basis on all requests means all HTTP responses served Cache-Control: immutable. Our CDN is Hyperfiddle-aware integrates this time-basis. Which is really important, because without this optimization, hyperfiddle is currently too slow when there are concurrent users. It also means our blog is equal to a static site served from github pages.

This is a dynamic API response served from disk because the time-basis hasn't changed

Here is the time basis of this request, encoded into the route as base64

Hyperfiddle performance pattern*

All requests have a time basis, all responses are immutable
Immutable responses cached everywhere (disk, CDN, gateway)
Component-local requests have component-local time basis, optimized for cache hits
Component-local html fragments (immutable) cached in CDN
Automatic request partitioning and batching, optimized for cache hits
CDN/Gateway lambda stitches SSR response from cached component-local html fragments
Server anticipates future requests to pre-warm caches
Client javascript optional; page interactive before javascript downloaded
Streamy http infrastructure

*Not all of this is implemented yet

December

Full datalog syntax demonstrated in hyperblog

Beta starts Q1 2018 http://www.hyperfiddle.net/