Datomic Ion launch day questions answered by Rich

Ion functions vs Client API vs Entity API

dustingetz: @richhickey Does ion auto scaling apps essentially supersede the client model? Meaning there is no datomic client in play? Just a client-like local api
richhickey: @dustingetz right, client-free apps totally possible
dustingetz: Idiomatic, right?
richhickey: well, there's zero reason for clients inside ions, but you may have legacy architecture that dictates use of clients. They're not deprecated or anything
I think many apps can be written as ions
and of course you can mix and match
dustingetz: Are there technical reasons for no entity API or is it just historical at this point, and is the ion "local client api" equivalent in power to the entity api
richhickey: @dustingetz there are semantic differences with client (vs peer) that yield 'no entity'. The 'local' API of ions matches the client API so people can do local REPL dev and testing (against cloud). So semantically, it's compatible with 'over wires'. That said, the perf of the same API in local mode within ions trounces clients. but keeping wire semantics means if you need to make an architectural shift that requires moving some parts to clients you're not screwed
thus 'client' is the only API of cloud, ion or out
reduceclj: where does the performance gain comes from? eliminating the network calls?
richhickey: @reduceclj right, no wires, same process

Which query style is more idiomatic: Datalog functions or Ion functions?

dustingetz: Remote client requesting query for Ion cluster
Ion application local query
if I am an Ion app, I can do both, is the former considered idiomatic? Because it is equivalent and works over wires
reduceclj: the former has the overhead of the network, which of the two you use depends on your needs/architecture
richhickey: @dustingetz the portability of the former seems like a flexibility win, but you may not care
dustingetz: @richhickey I like the former but that
thats a pretty wild "partial" in there
you would consider that idiomatic? Im ok with that if you are
richhickey: idioms take time to develop
these are some brand new power tools, would hate to pour concrete advice :slightly_smiling_face:
dustingetz: ok:)

Foreign datastore interop

val_waeselynck: To what extent is it reasonable to query other data stores (e.g a remote SQL or ElasticSearch server) from ions?
richhickey: @val_waeselynck you can do anything as long as you give the datomic cluster node's role the necessary permissions
dustingetz: The ion will autoscale though and the foreign store will not
you'd need a queue
richhickey: your instances, your VPC
stuarthalloway: @dustingetz not everyone scales or needs to
but in any case the important point is, as @richhickey said, it is your instances, use them as you will
val_waeselynck: @richhickey thanks, it may be a good idea to make that obvious in the docs - it helps assess the power / limitations, especially since we've been accustomed to "be careful of the code Datomic runs for you" with tx fns
val_waeselynck: This is very exciting - it seems to give you the getting started experience of Firebase or Lambda with the scalability of advanced hand-rolled systems (edited)
richhickey: @val_waeselynck that still applies, you can hold up your txes

Do foreign datastore writes inside txfns slow down the transactor?

dustingetz: If I write to a durable kv store from a transactor fn and it takes 1ms to return, will this in practice slow down transactor throughput? Or are transactions processed in parallel up until the dynamo conditional put which i understand to be batched
richhickey: it's going to be serial (edited)
dustingetz: Oh I see, because if there is a cas in the same tx, that needs a dbval
richhickey: because e.g. tx fns can query and expect to see prior txes
yes, stuff like that
but remember you will likely be initiating your txes from ions, so there's another opportunity there for coordinated (if not transactional) work
dustingetz: I dont understand that, can you give me another hint
richhickey: put in elasticsearch, transact
means it might be in elasticsearch but tx fails, but not holding up txes
dustingetz: Oh ok, so if the other store supported two phase commit that might work too
richhickey: or whatever, cleanup on tx fail
you are likely retrying
dustingetz: thanks
reduceclj: why web ions and not just a ring app in a lambda function?
richhickey: the point is, you will be in your VPC, running in a role, able to do cool things
@reduceclj a) ease, b) not having to run in lambda execution container, c) running in db context vs making a client call, d) speed
the important value prop is API Gateway. Putting a lambda behind it is just one option (hint)
reduceclj: ok, pretty cool

Why is clojure.core/eval blacklisted in datalog evaluation?

dustingetz: @richhickey Since it is our instance, why is eval blacklisted in :where clause evaluation
Or is/could that be different in Ion (edited)
richhickey: there is an :accept list in ion, put stuff there to allow it
dustingetz: clojure.core/eval was excluded from cloud for security? security of your own jars or something
richhickey: nothing will run not on the list
i.e. nothing will be an entry point in tx/query/lambda
pants on by default
dustingetz: lol
richhickey: people inadvertently expose clients, query etc
dustingetz: Oh, ok makes sense