Hacker Newsnew | past | comments | ask | show | jobs | submitlogin

I'm sad this project fizzled: https://github.com/losfair/mvsqlite

It had the fascinating property of being a full multi-writer SQL engine when using Page Level Conflict Checking. Even more fascinating: every single client itself becomes a writer.



The way they got multi-writer working is a bit too adventurous for me.

It involves fiddling the database header to ensure you can keep track of the read-set from the VFS layer, by convincing SQLite to drop its page cache at the start of every transaction.

Other than that, a MVCC VFS is relatively straightforward. Here's an in-memory experiment if a few hundred lines of Go: https://github.com/ncruces/go-sqlite3/blob/v0.29.0/vfs/mvcc/...

I haven't settled on what's a good storage layer for something like this.


Does that just do Database Level Conflict Checking?


Yes. It acts exactly like WAL mode in that regard. Readers operate on a snapshot and are never blocked, writers block each other. Upgrading a reader to a writer fails immediately if there is a concurrent writer, however BEGIN IMMEDIATE transactions never do.

One of the issues I have with the mvsqlite approach (beyond not being able to convince myself that it's correct — I'm sure it is I'm just unsure I could replicate it faithfully) is that I don't think the VFS gets any peak at the BEGIN CONCURRENT syntax, so suddenly all your transactions can fail because of write-write conflicts. It's not opt-in.


But for scale you don't want multiple writers. You want a single writer so that you can batch.




Guidelines | FAQ | Lists | API | Security | Legal | Apply to YC | Contact

Search: