database HA and sync of the meta data in sqlite using litestream

[gedw99]

The only weak point of JuiceFS seems to be the database.

it looks like it current uses github.com/mattn/go-sqlite3

Litestream, https://litestream.io/, is a golang sqlite that is based on go-sqlite3 and extends it with real time backup to S3.

Code: https://github.com/benbjohnson/litestream

Apache LIC: https://github.com/benbjohnson/litestream/blob/main/LICENSE

Supports replication: benbjohnson/litestream@a090706

Soon it will support load balancing in that when you call the db for a READ it will be directed to the nearest Read only db instance and for a write it will be forwarded to the origin write only db. Issue is here: benbjohnson/litestream#323

[gedw99]

Litestream uses Object storage as its backend also btw. It does this for every change to the DB.

So is is a nice synergy with JuiceJS

[davies]

After a quick look at litestream, it looks good as a near-realtime backup solution, thanks for sharing!

[gedw99]

@SandyXSD Just wanted to get eyeballs on this idea

[solracsf]

Thanks for sharing @gedw99 didn't knew this. 👍

[legout]

Any updates on this? Is litestream already supported by juicefs?

[hstock]

I'm not a juicefs dev but to my understanding it should work without modifications to juicefs. According to https://juicefs.com/docs/community/security/trash slices are also kept for trash-days period, so any database snapshot from that period restored through litestream should be usable. I'll get back after some more testing...