On 20/02/2013 07:36, Jan Stary wrote:
On Tue, Feb 19, 2013 at 00:35, Keith wrote:
Q. How do I make the default web folder /var/www/ capable of holding
millions of files (say 50GB worth of small 2kb-12kb files) so that I
won't get inode issues ?
newfs defaults to -f 2k and -b 16k which is fine if you
know in advance you will hold 2k-12k files. As for inodes,
the default of -i is to create an inode for every 4 frags,
that is 8192 bytes. So on a 50G filesystem this should
give you over 6.1 millon inodes. What does df -hi say?
But first of all, fix your crappy app to not do that.
Hi, thanks for the info. Yesterday I did a backup, format, restore of
the /var/www partition although to be honest I wasn't really sure what i
was doing with regards to the newfs command. I tried running "newfs
-i"with different values and settled on "newfs -i 1 /var/www" as it
seemed at the time to makes the make the most inodes and that was just
based on how much output was generated while newfs was running.
# df -hi
Filesystem Size Used Avail Capacity iused ifree %iused
Mounted on
/dev/sd0a 1005M 135M 819M 14% 3272 152630 2% /
/dev/sd0k 1005M 2.0K 955M 0% 1 155901 0% /home
/dev/sd0n 21.0G 2.0K 20.0G 0% 1 2832253 0% /scratch
/dev/sd0d 3.9G 14.0K 3.7G 0% 21 545641 0% /tmp
/dev/sd0f 2.0G 461M 1.4G 24% 13537 272285 5% /usr
/dev/sd0g 1005M 193M 762M 20% 9547 146355 6% /usr/X11R6
/dev/sd0h 6.8G 2.0G 4.5G 31% 41346 868092 5% /usr/local
/dev/sd0j 2.0G 2.0K 1.9G 0% 1 285821 0% /usr/obj
/dev/sd0i 1.9G 2.0K 1.8G 0% 1 285821 0% /usr/src
/dev/sd0e 6.3G 37.2M 6.0G 1% 740 856730 0% /var
/dev/sd0m 1001M 6.5M 944M 1% 53 155849 0% /var/log
/dev/sd0l 4.7G 1.2G 3.3G 26% 449170 2206316 17% /var/www
/dev/sd1a 1.8T 1.6T 147G 92% 720111 60427023 1%
/mnt/Media2TB
/dev/sd2a 55.0G 11.3G 41.0G 22% 208 7353262 0% /var/mysql
The above "df -hi" output was done today after the wiped the app and
started it again from scratch. It had been running for about 12 hours
and there was about 450,000 files. How many files do you think I'll be
able to store with this number of inodes ? I'd never used dump or
restore before and was supprised as how easy it was to backup, format
and restore the files so that will come in handy if I need to move this
partition later to a larger disk. I'll think I will just have to keep an
eye on my inodes until I get a feel for how many I need.
I don't know how to fix the app or why the developers decided to make so
many files on disk so I asked in their chat room........
<Keef>: I don't know how many files I had at the time that I was getting
issues probably about 1/2 million but I have since wiped the partition
and reformatted with more inodes but.... I ended up asking for help in
with my inode problem on a OpenBSD mailing lists and they were asking
why the newznab app wrote the files to disk in the first place. So I
thought I'd should ask here...
<ll>: do you want 20GB of files in your db?
<forkless>: i know i dont
<ll>: nor i
<ll>: and thats the reason realy
<Safra>: lol
<Safra>: Then you will get "why is my nzbfiles table corrupt"?
<Safra>: =p
<Safra>: I cant download anything?
<Safra>: lol
<Safra>: "fix it for me NOW"
<Safra>: =p
<forkless>: then the next step will be why arent the cover in the db either
<forkless>: and before you know it your db is 100GB
<Keef>: So how many files do the typical newnab users end up having and
how much space should I partition up for ?
<forkless>: i've only got a 120k releases or so but i dont nearly index all
<forkless>: i guess depends on your needs
I guess they have a good point as they have to support the app.
Cheers
Keith
