Does/will Noms have proper support for multiuser sync? That’s an issue with CouchDB.
I mean: instead of syncing whole the database, only syncing the parts that a user has access to, and being able to define those accesses. The standard use case for consumer apps.
Noms syncs at the level of a "value" not the entire database. Since a Noms database is a tree of values, you can sync the entire db, but you don't have to. You can also sync a single list.
So yes, in principle, we can definitely do this. In practice we are missing some conveniences that would make it a really easy drop-in feature.
There's quite a dramatic claim on the website, "merge [...] changes efficiently and correctly days, weeks, or years later." How does that work? For example if you have two records saying userid 3's name is "ann" and userid 3's name is "jane", I don't see how you could merge those without extra information or human input.
The claim on the website is not meant to suggest that any two changes can be automatically merged. I will try to clarify that.
The world contains logical conflicts because physical constraints mean that processes can operate disconnected from each other. No database can wave that away.
Noms will automatically, efficiently, and correctly merge changes that don't logically conflict. Which is a pretty cool and unique property in a database.
If any conflicts are found, there is a callback to user software to perform a resolution.
IBM Domino (aka Lotus Notes) has been automatically, efficiently, and correctly merging changes that don't logically conflict since 1989. How is the functionality in Noms unique?
It's a very hard problem to solve. The best you can really manage is to automatically merge non-conflicting changes, and defining what is a conflicting change will require some extra knowledge if you have multiple tables with relations between them.
For extra fun try doing this with geographic data and try merging geometry changes correctly.
Well, hopefully you're not just using an incrementing user id field. Twitter's Snowflake format is much closer to "definitely unique". So then you have a very simple set, which you could very simply merge with other sets. So that might help you get started with merging those pieces of data.
But I agree, there are certain kinds of data that you can't so efficiently merge. Your best bet is to try to adapt it into some sort of known, proven CRDT. (Dunno how you do that with this database, haven't really read up on it.)
More specifically, what isolation guarantee do you make? Are writes linearizable? What consistency guarantees do you claim? How well do you think you will do when reviewed by aphyr (https://aphyr.com)?
You should understand that Noms is doing something quite a lot simpler than the systems that are usually reviewed on Aphyr.
Noms doesn't manage its own storage - it relies on an underlying key/value store that must provide strongly consistent reads for at least one key. In other words, we delegate most of the hard part to somebody else.
With that all said...
Currently our intent is that:
- Transactions that read and write from a single dataset have strong serializability
- Transactions that read from multiple datasets and write to a single dataset have snapshot isolation
- Transactions that write to multiple datasets aren't possible
In the future, we will probably allow additional configuration, such that, e.g., one could choose snapshot isolation within a dataset for additional concurrency, or strong serializability for transactions that span datasets.
How do you deal with access control? I can imagine that it would be necessary in many applications to have parts of the database only accessible to some people, and that access rules could be complicated and also depend on values in the database itself.
We want to. It would be nice if we could just use cgo but it's not complete. So we need to build some kind of simplified API to Noms that can be exported via cgo.
In the meantime, as long as I've got your attention, here's a few new stuffs we've been working on since last time Noms was discussed here in August:
- A prototype query language, and a demo of how to create indexes in Noms: https://www.youtube.com/watch?v=fv6_T5yaWns
- Support for merging concurrent (and potentially conflicting) changes: https://www.youtube.com/watch?v=--7dgoJBdjU