Jos Poortvliet - 22:36 17.12.15 wrote: > On Thursday 17 December 2015 22:07:34 Michal Hrusecky wrote: > > Hi, > > > > while discussion is going on, I have a few questions regarding everyones > > experience with running similar setup to what is proposed in WD > > Challenge. I have at home BananaPi with external USB harddrive running > > ownCloud now and while installing it I did few decisions that might be > > relevant to the challenge as well, so I was wondering whether you have > > similar experience/setup or something else (will help us shape the image > > and improve our home setups). > > > > Decisions I made: > > ----------------- > > > > 1) Database on SD card > > > > I have whole system and even the ownCloud database on SD card. Reasoning > > behind it is that even though it is slow, my externel harddrive is put > > asleep most of the time and spinning it up would take quite some time. > > So I prefer to get immediately something responding slowly than > > something that needs to wake up and than performs. And I don't want to > > have it spinning all the time as that is the point of going for ARM > > board - saving up some energy. > > Yeah, but it will kill the SD card in a few weeks ;-)
Depends... Mine is running Tumbleweed fine for about a year. And yes, for end users dying SD cards might be an issue. But as SD cards are cheap, if it will backup itself once in a while on HDD and would be easy to recover from the issue... > These SD Cards SUCK at doing tiny writes - they have huge blocks > (think 4MB or so) so every time you write, like 1KB of data, it writes > 4 MB. Slow AND it will exhaust the number of writes the card can > sustain in no time. SD cards are made for photo's, not databases :( Yep, SQLite mimics pictures better than MySQL I would guess :-) But in general, I would like to avoid spinning HDD as much as possible to save energy and noise. But yes, performance is an issue with SD card. > > 2) SQLite instead of MySQL > > > > SQLite has plenty of issues, but I ended up using it instead of MySQL as > > I have a feeling that on SD card it is faster. But didn't run any > > benchmark and didn't tuned up MySQL much. On real drive, MySQL should > > beat SQLite in performance without any problem, but I had the feeling > > that writes/reads were much more scattered all around and thus slower > > than SQLite when running on slow weird storage like SD card. Did anybody > > did some benchmark? What is your experience? I actually migrated from > > SQLite to MySQL and than back, but maybe I should have spend more time > > tunning MySQL. > > Yeah, SQLite is cool - but not very good with ownCloud, really. We can > try, yes, but then we might need to do some patching in ownCloud to > make it more efficient with SQLite. > > That's the same as with NGNIX, btw, it works but there are rough areas > so we might need to patch ownCloud. Then again, that is not a big > problem, I have heard that ownCloud is open source so we can probably > get those patches upstream :P > > So - don't listen too much to grumpy me, give it a try. But then a > serious try: put the 1TB drive full of pictures and documents, then > create 10 users, let 5 run the sync client and let 2 others view > pictures. > > If that works fast, you're ready for release! ;-) Yep, sounds like good stress test :-) In general, I think it doesn't make sense to use SQLite on HDD, but we will see. > > My performance boosting TODO: > > ----------------------------- > > > > 1) Copy database to tmpfs on start and sync it back regularly to keep > > persistence, gain speed but end up with possibility of loosing last X > > minutes of history. Would require quite some scripting which I didn't > > get to yet, but wondering whether somebody does have something like that > > already and what people think about it :-) > > THAT is a great trick, yes, and that might work very well. Of course > we need to think about losing data but if only the DB is updated and > not the file system, it is probably fine. Yep, the only issue might be stuff like calendar/contacts. Magic last worlds, let's make it configurable and shift blame to the user :-D But seriously, might be a way to avoid spinning disk while keeping up the performance. With MySQL we can probably even split it per table and some tables put directly on the disk (contacts/calender) and others in memory (caches). > > 2) Use squashfs + overlayfs for OS. Found out that reads from SD card > > are quite slow and as I don't want to have my HDD spinning all the time, > > sounds like some of the reads from OS can be speed up by using squashfs > > and thus compressed filesystem. Would require special mounting on start > > but could safe both space and time. Not for data, but should allow to > > fit whole OS on small SD, maybe even multiple times and could improve > > performance over direct SD card. Did some basic unfair testing and > > looked like reading files from squashfs on top of SD card is faster than > > reading from SD card directly. > > Our idea was to use the SD card exclusively as a boot loader, do > everything from the hard drive... Because of the damage I talked about > earlier. I understand, not sure how much agree :-) _______________________________________________ Devel mailing list [email protected] http://mailman.owncloud.org/mailman/listinfo/devel
