On 2020-07-10 at 15:41 +0200, Daniel Jagszent wrote: > > > > Ah yes, compression and probably encryption will indeed preclude > > any > > sort of partial block caching. An implementation will have to be > > limited to plain uncompressed blocks, which is okay for my use-case > > though (borg provides its own encryption and compression anyway). > > [...] > Compression and encryption are integral parts of S3QL and I would > argue > that disabling them is only an edge case. > I might be wrong but I think Nikolaus (maintainer of S3QL) will not > accept such a huge change into S3QL that is only beneficial for an > edge > case.
I see, but I was rather asking about the technical side of things, whether it sounds sane or I'm missing something. Provided I do work on partial cache in the end, if Nikolaus decides to accept that work into upstream -- great, if not -- I will carry it in a downstream fork. (I will probably need a fork anyway, as Nikolaus has apparently rejected a specific optimization in the B2 backend, absence of which makes my s3ql hit a certain API rate limit very often.) > > > Since you do not need compression and encryption (handled by borg > anyways) have you tried something with less abstractions? > Does https://github.com/kahing/goofys work for you? (maybe stack a > https://github.com/kahing/catfs on top) Yes, but no. S3QL is the only S3 filesystem I'm aware of that provides consistency guarantees in some form. goofys appears to work, but it only does work by virtue of having a single thread and a single backend connection (consequently, it ends up being very slow), whereas catfs has too many "will eat your data" disclaimers for my liking. -- Ivan Shapovalov / intelfx / -- You received this message because you are subscribed to the Google Groups "s3ql" group. To unsubscribe from this group and stop receiving emails from it, send an email to s3ql+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/s3ql/ba3eb217afdb3e1547d954d1a891043e4db12e8d.camel%40intelfx.name.
signature.asc
Description: This is a digitally signed message part
