On Sunday 20 December 2015 20:16:28 Stefan Bruens wrote: > On Sunday 20 December 2015 17:49:37 Jos Poortvliet wrote: > > You shouldn't run ownCloud on a SD card anyhow - these cards are heavily > > optimized for sequential read and write performance, sporting block sizes > > in the megabytes. Storing a database on that with its random write > > patterns reads to massive write amplification: for every tiny write the > > SD card writes a full block of several MB's, burning through the lifetime > > write capacity of the card in months, weeks or even days if you use it > > heavily. > Jos, can you please stop these unwarranted claims. > > While it is true that a small write may lead to a much larger write, this is > in general not the case. NAND flashes today have pages of 2kByte to 8kByte, > thats the smallest amount you can write. Everytime you "overwrite" a > logical block of data, it is just marked as stale data. > > The larger erase blocks only comes into play on garbage collection. Garbage > collection happens if > - there are no erase blocks with unused (neither used nor stale) pages, or > - occasionaly, during idle periods > Garbage collection happens more frequently if the media is filled up. > > Although SD cards *are* actually much slower on random access than on > sequential access, these are not worse than HDDs. HDDs have access times of > ~10ms, so you have ~100 Ops/s. SD write access is about the same, random > read access is about one order of magnitude faster. > > The RPi is also limited by its small memory. As soon as you have to move > dirty memory pages to storage or drop data from caches, the USB disk is a > bottleneck. > > For the matter of write amplification, you should differentiate between > write amplification caused by: > (1) the database > (2) the file system > (3) the flash translation layer (i.e. inside the storage media) > > In fact (1) can alone lead to a write amplification by a factor of two. > SQLite in its default journal mode writes everything to the journal *and* > to the database file. SQLite since 3.7.0 knows a Write-Ahead-Log (WAL) mode > where changes from the journal are only written back to the database during > checkpointing, coalescing many small writes into larger ones. Other > databases use similar schemes to SQLites WAL. > > (2) may be problematic some filesystems not used properly. For e.g. BTRFS > you should make sure COW is disabled for database files. > > (3) may be an issue if you have old, bad, cheap SD cards. You can not rule > this out in general, but if the card has a somewhat decent random write > performance (~1MByte/s), the card needs to have a proper FTL. > > IMHO is is a very good idea to put both the OS and the database on SD card. > Even a small (8GByte) SD will be largely unused, which leaves plenty of > space for flash wear leveling end ensures seldom run of the garbage > collector.
The SD card which comes with the prototype is 4GB and I've found plenty of reports on the internet about breaking those in 1-3 months of usage with a Pi. I myself have lost a USB stick of 8GB in 2 days (admittedly of continuous data writing, I did use it for performance testing). We could go for larger SD cards like 8 or 16 GB to give more room for wear leveling and we can use F2FS as filesystem perhaps. With good use of caching and no logging and read-only root filesystem, perhaps it is quite safe. I'd still want to have a backup of the database to the hard drive once a day or something in place, though... And an automated way to restore the operating system and DB to a new SD card if things break, then. I still don't feel comfortable with it, but I'm no expert either. How sure are you that it's safe? Sure enough to commit your private data to it? /J > Kind regards, > > Stefan -- Disclaimer: Everything I do and say is based on my view of the world today. I am not responsible for changes in the world, nor my view on it. Everything I say is meant in a positive and friendly way, unless explicitly stated otherwise. find me on blog.jospoortvliet.com
signature.asc
Description: This is a digitally signed message part.
_______________________________________________ Devel mailing list [email protected] http://mailman.owncloud.org/mailman/listinfo/devel
