Re: Get total size of all files in directory using unit Bytes?
Or with subdirectories find . -type f -ls | awk '{sum += $7} END {print sum}'
Re: Get total size of all files in directory using unit Bytes?
On Mon, Mar 04, 2013 at 12:54, Paul de Weerd wrote: > On Mon, Mar 04, 2013 at 12:32:32PM +0100, Paolo Aglialoro wrote: > | Great one! > | How to put that nice expression into an alias without console complaining > | when executed? I may be an oddball here, but I prefer just making little shell scripts and putting them in ~/bin for stuff like this. Advantages are you can get a little more complicated and when you inevitably need to edit the script later, it automatically updates in every shell without having to re-source .profile. > I still wonder why people want to know this (seemingly useless) value. > What does it even mean ? I forget now why I made that script, but I did it for a reason. :) I think I was trying to determine which of two directories had more data in it. du is good for telling you how much space you need, but doesn't actually tell you how much stuff you have.
Re: Get total size of all files in directory using unit Bytes?
On Mon, Mar 04, 2013 at 11:02:47PM +0800, f5b wrote: > At??2013-03-04??20:13:46,"Paul??de??Weerd"wrote: > >Exactly!So??what??is??the??point??in??summing??up??the??sizes??of??a??bunch??of > >files???I??am??197??cm??tall,??my??house??number??is??34,??my??zipcode??is??1318, > >I??have??2??brothers??and??1??sister??..??sum??is??1552.Great,??but??now??what??? > > > >This??total??value??does??not??correspond??to??anything??tangible??(as??far??as??I > >can??see,??at??least??..??hence??me??asking).It's??no??indication??of??how??much > >storage??space??is??needed??to??store??these??files,??it's??no??indication??of > >how??large??an??archive??would??be??containing??these??files,??it's??of??no??real > >use??(again,??afaics)??except??for??knowing??what??the??filesize??would??be??of > >cat??*??>??/tmp/newfile??(which??would??be??pointless??in??most??cases??I??guess). > > > >Why??do??people??care??? > > > > > Maybe because we come from Windows system. > In Windows, sum files' size by "Byte" is a simple quick way to check if > thousands of files are > > modified/sync/same, although not accurate. You must be kidding, right? This test both gives false positives and false negatives. -Otto > > In OpenBSD, Command ls or du can't do this directly. > > For example > # pwd > /home/test > # ls -l > total 8 > -rw-r--r-- 1 root wheel 2 Mar 3 23:29 a.txt > -rw-r--r-- 1 root wheel 3 Mar 3 23:29 b.txt > # du -sh > 6.0K. > # du -s > 12 . > # echo a >>b.txt > # ls -l > total 8 > -rw-r--r-- 1 root wheel 2 Mar 3 23:29 a.txt > -rw-r--r-- 1 root wheel 5 Mar 4 21:45 b.txt > # du -sh > 6.0K. > # du -s > 12 . > > You see? ls and du never know this directory's files(withtout subdirectory) > have been changed, but file sizes are changed from 5 to 7, so "sum" knows and > Tedu's shell script is my friend. > > Tedu's filesizes script. > ~/bin> cat filesizes > #!/bin/sh > ls -l $@ | awk '{sum += $5} END { print sum }' > > Would function like this script merge to ls' options or other command to > OpenBSD base?
Re: Get total size of all files in directory using unit Bytes?
On 4 Mar 2013, at 10:02, f5b wrote: > Maybe because we come from Windows system. > In Windows, sum files' size by "Byte" is a simple quick way to check if > thousands of files are > > modified/sync/same, although not accurate. openssl {md5|sha1|...} *
Re: Get total size of all files in directory using unit Bytes?
At 2013-03-04 20:13:46,"Paul de Weerd" wrote: >Exactly! So what is the point in summing up the sizes of a bunch of >files ? I am 197 cm tall, my house number is 34, my zipcode is 1318, >I have 2 brothers and 1 sister .. sum is 1552. Great, but now what ? > >This total value does not correspond to anything tangible (as far as I >can see, at least .. hence me asking). It's no indication of how much >storage space is needed to store these files, it's no indication of >how large an archive would be containing these files, it's of no real >use (again, afaics) except for knowing what the filesize would be of >cat * > /tmp/newfile (which would be pointless in most cases I guess). > >Why do people care ? > Maybe because we come from Windows system. In Windows, sum files' size by "Byte" is a simple quick way to check if thousands of files are modified/sync/same, although not accurate. In OpenBSD, Command ls or du can't do this directly. For example # pwd /home/test # ls -l total 8 -rw-r--r-- 1 root wheel 2 Mar 3 23:29 a.txt -rw-r--r-- 1 root wheel 3 Mar 3 23:29 b.txt # du -sh 6.0K. # du -s 12 . # echo a >>b.txt # ls -l total 8 -rw-r--r-- 1 root wheel 2 Mar 3 23:29 a.txt -rw-r--r-- 1 root wheel 5 Mar 4 21:45 b.txt # du -sh 6.0K. # du -s 12 . You see? ls and du never know this directory's files(withtout subdirectory) have been changed, but file sizes are changed from 5 to 7, so "sum" knows and Tedu's shell script is my friend. Tedu's filesizes script. ~/bin> cat filesizes #!/bin/sh ls -l $@ | awk '{sum += $5} END { print sum }' Would function like this script merge to ls' options or other command to OpenBSD base?
Re: Get total size of all files in directory using unit Bytes?
On Mon, Mar 04, 2013 at 12:57:05PM +0100, Otto Moerbeek wrote: | On Mon, Mar 04, 2013 at 12:54:49PM +0100, Paul de Weerd wrote: | | > On Mon, Mar 04, 2013 at 12:32:32PM +0100, Paolo Aglialoro wrote: | > | Great one! | > | How to put that nice expression into an alias without console complaining | > | when executed? | > | | > | | > | # ls -l | awk '{ SUM += $5 } END { print SUM }' | > | 569047 | > | | > | # alias tot="ls -l | awk '{ SUM += $5 } END { print SUM }'" | > | # tot | > | awk: syntax error at source line 1 | > | context is | > | { SUM += >>> } <<< | > | awk: illegal statement at source line 1 | > | > Escape the "$" in the awk expression: | > | > [weerd@despair] $ alias tot="ls -l | awk '{SUM+=\$5} END {print SUM}'" | > [weerd@despair] $ tot | > 20 | > | > I still wonder why people want to know this (seemingly useless) value. | > What does it even mean ? | > | > [weerd@despair] $ mkdir /tmp/test | > [weerd@despair] $ cd /tmp/test | > [weerd@despair] $ dd if=/dev/zero of=a bs=1024 seek=2048 count=1 | > 1+0 records in | > 1+0 records out | > 1024 bytes transferred in 0.000 secs (3413 bytes/sec) | > [weerd@despair] $ ln a b | > [weerd@despair] $ ls -l | > total 128 | > -rw-r--r-- 2 weerd wheel 2098176 Mar 4 12:49 a | > -rw-r--r-- 2 weerd wheel 2098176 Mar 4 12:49 b | > [weerd@despair] $ tot | > 4196352 | > [weerd@despair] $ du -csh a b | > 32.0K a | > 32.0K total | > | > | > (note the lie ls(1) spreads here) | > | > | Btw, for higher readability, it would also be great to put periods in the | > | resulting output like: 1.264.691 | > | > Well, that should be easy enough to add yourself :) Left as an | > exercise to the reader... | > | > Cheers, | > | > Paul 'WEiRD' de Weerd | | But remember, file size and disk usage are two different things, Exactly! So what is the point in summing up the sizes of a bunch of files ? I am 197 cm tall, my house number is 34, my zipcode is 1318, I have 2 brothers and 1 sister .. sum is 1552. Great, but now what ? This total value does not correspond to anything tangible (as far as I can see, at least .. hence me asking). It's no indication of how much storage space is needed to store these files, it's no indication of how large an archive would be containing these files, it's of no real use (again, afaics) except for knowing what the filesize would be of cat * > /tmp/newfile (which would be pointless in most cases I guess). Why do people care ? Paul 'WEiRD' de Weerd -- >[<++>-]<+++.>+++[<-->-]<.>+++[<+ +++>-]<.>++[<>-]<+.--.[-] http://www.weirdnet.nl/
Re: Get total size of all files in directory using unit Bytes?
On Mon, Mar 04, 2013 at 12:54:49PM +0100, Paul de Weerd wrote: > On Mon, Mar 04, 2013 at 12:32:32PM +0100, Paolo Aglialoro wrote: > | Great one! > | How to put that nice expression into an alias without console complaining > | when executed? > | > | > | # ls -l | awk '{ SUM += $5 } END { print SUM }' > | 569047 > | > | # alias tot="ls -l | awk '{ SUM += $5 } END { print SUM }'" > | # tot > | awk: syntax error at source line 1 > | context is > | { SUM += >>> } <<< > | awk: illegal statement at source line 1 > > Escape the "$" in the awk expression: > > [weerd@despair] $ alias tot="ls -l | awk '{SUM+=\$5} END {print SUM}'" > [weerd@despair] $ tot > 20 > > I still wonder why people want to know this (seemingly useless) value. > What does it even mean ? > > [weerd@despair] $ mkdir /tmp/test > [weerd@despair] $ cd /tmp/test > [weerd@despair] $ dd if=/dev/zero of=a bs=1024 seek=2048 count=1 > 1+0 records in > 1+0 records out > 1024 bytes transferred in 0.000 secs (3413 bytes/sec) > [weerd@despair] $ ln a b > [weerd@despair] $ ls -l > total 128 > -rw-r--r-- 2 weerd wheel 2098176 Mar 4 12:49 a > -rw-r--r-- 2 weerd wheel 2098176 Mar 4 12:49 b > [weerd@despair] $ tot > 4196352 > [weerd@despair] $ du -csh a b > 32.0K a > 32.0K total > > > (note the lie ls(1) spreads here) > > | Btw, for higher readability, it would also be great to put periods in the > | resulting output like: 1.264.691 > > Well, that should be easy enough to add yourself :) Left as an > exercise to the reader... > > Cheers, > > Paul 'WEiRD' de Weerd But remember, file size and disk usage are two different things, -Otto
Re: Get total size of all files in directory using unit Bytes?
On Mon, Mar 04, 2013 at 12:32:32PM +0100, Paolo Aglialoro wrote: | Great one! | How to put that nice expression into an alias without console complaining | when executed? | | | # ls -l | awk '{ SUM += $5 } END { print SUM }' | 569047 | | # alias tot="ls -l | awk '{ SUM += $5 } END { print SUM }'" | # tot | awk: syntax error at source line 1 | context is | { SUM += >>> } <<< | awk: illegal statement at source line 1 Escape the "$" in the awk expression: [weerd@despair] $ alias tot="ls -l | awk '{SUM+=\$5} END {print SUM}'" [weerd@despair] $ tot 20 I still wonder why people want to know this (seemingly useless) value. What does it even mean ? [weerd@despair] $ mkdir /tmp/test [weerd@despair] $ cd /tmp/test [weerd@despair] $ dd if=/dev/zero of=a bs=1024 seek=2048 count=1 1+0 records in 1+0 records out 1024 bytes transferred in 0.000 secs (3413 bytes/sec) [weerd@despair] $ ln a b [weerd@despair] $ ls -l total 128 -rw-r--r-- 2 weerd wheel 2098176 Mar 4 12:49 a -rw-r--r-- 2 weerd wheel 2098176 Mar 4 12:49 b [weerd@despair] $ tot 4196352 [weerd@despair] $ du -csh a b 32.0K a 32.0K total (note the lie ls(1) spreads here) | Btw, for higher readability, it would also be great to put periods in the | resulting output like: 1.264.691 Well, that should be easy enough to add yourself :) Left as an exercise to the reader... Cheers, Paul 'WEiRD' de Weerd | Thanks :) | | | On Sun, Mar 3, 2013 at 11:16 PM, Ted Unangst wrote: | | > On Sun, Mar 03, 2013 at 22:02, Paul de Weerd wrote: | > > [weerd@despair] $ ls -l /tmp/test/* | awk '{SUM+=$5} END {print SUM}' | > | > heh. :) | > | > ~/bin> cat filesizes | > #!/bin/sh | > ls -l $@ | awk '{sum += $5} END { print sum }' | -- >[<++>-]<+++.>+++[<-->-]<.>+++[<+ +++>-]<.>++[<>-]<+.--.[-] http://www.weirdnet.nl/
Re: Get total size of all files in directory using unit Bytes?
On Mon, Mar 04, 2013 at 12:32:32PM +0100, Paolo Aglialoro wrote: > Great one! > How to put that nice expression into an alias without console complaining > when executed? A shell function instead of an alias? jirib
Re: Get total size of all files in directory using unit Bytes?
Great one! How to put that nice expression into an alias without console complaining when executed? # ls -l | awk '{ SUM += $5 } END { print SUM }' 569047 # alias tot="ls -l | awk '{ SUM += $5 } END { print SUM }'" # tot awk: syntax error at source line 1 context is { SUM += >>> } <<< awk: illegal statement at source line 1 Btw, for higher readability, it would also be great to put periods in the resulting output like: 1.264.691 Thanks :) On Sun, Mar 3, 2013 at 11:16 PM, Ted Unangst wrote: > On Sun, Mar 03, 2013 at 22:02, Paul de Weerd wrote: > > [weerd@despair] $ ls -l /tmp/test/* | awk '{SUM+=$5} END {print SUM}' > > heh. :) > > ~/bin> cat filesizes > #!/bin/sh > ls -l $@ | awk '{sum += $5} END { print sum }'
Re: Get total size of all files in directory using unit Bytes?
On Sun, Mar 03, 2013 at 22:02, Paul de Weerd wrote: > [weerd@despair] $ ls -l /tmp/test/* | awk '{SUM+=$5} END {print SUM}' heh. :) ~/bin> cat filesizes #!/bin/sh ls -l $@ | awk '{sum += $5} END { print sum }'
Re: Get total size of all files in directory using unit Bytes?
Not really an answer to your question, I know, but what would such a number mean ? What use do you have for it ? On Mon, Mar 04, 2013 at 12:01:25AM +0800, f5b wrote: | for example | | 1. | there is only two file in /home/test/ | # ls -l /home/test/ | total 8 | -rw-r--r-- 1 root wheel 2 Mar 3 23:29 a.txt | -rw-r--r-- 1 root wheel 3 Mar 3 23:29 b.txt | | So the total size of all files ( a.txt + b.txt ) should be 5 Bytes. | How to get total size ( 5 Bytes ) directly but not the 8 Bytes. a.txt is 2 bytes long, but occupies the minimum allocation unit of your filesystem (2 kilobytes, in your case). The "total 8" is a reference to the total number of sectors (512 bytes low level disk allocation units) in use (2 files of 2 kilobytes use 4 kilobytes in total and 4 kilobytes of storage require 8 physical disk sectors). | 2. | # man du | says, | -h "Human-readable" output. Use unit suffixes: Byte, Kilobyte, | Megabyte, Gigabyte, Terabyte, Petabyte, Exabyte in order to | reduce the number of digits to four or less. | | but | # du -h /home/test | 6.0K/home/test | | How to let unit suffixes: Byte display (come out) but not Kilobyte? Note that du is reporting the correct size. 2K for a.txt, 2K for b.txt and 2K for the directory itself for a total of 6K. If you want du to show you a number in bytes, touch a non-existing file (i.e. create an empty file): [weerd@despair] $ touch /tmp/X [weerd@despair] $ du -sh /tmp/X 0B /tmp/X | just like | # du -b /home/test ( option "-b" not exist) | 5B /home/test /home/test is not 5 bytes. It's way bigger. As an upside, it won't cost you any extra diskspace to grow a.txt with 10 more bytes. Up until 2048 bytes, the space allocated to this file on the filesystem does not change. If you really want to know how many bytes are stored in a set of files you could go [weerd@despair] $ mkdir /tmp/test [weerd@despair] $ cd /tmp/test [weerd@despair] $ echo a > a.txt [weerd@despair] $ echo bb > b.txt [weerd@despair] $ ls -l total 4 -rw-r--r-- 1 weerd wheel 2 Mar 3 21:54 a.txt -rw-r--r-- 1 weerd wheel 3 Mar 3 21:54 b.txt [weerd@despair] $ ls -l /tmp/test/* | awk '{SUM+=$5} END {print SUM}' 5 [weerd@despair] $ cat * | wc -c 5 The first approach iterates over the output of ls(1) and uses awk to sum up the filesizes of all files in the listing. The second solutions simply reads all files and writes them to wc(1) which then shows a count of the number of bytes it read. Note that neither of these solutions take hardlinks into account (or subdirectories, or other fancy stuff). Also note that I have a smaller minimum allocation unit on my /tmp partition, since the two files consume 4 sectors worth of storage together. You may want to read up on filesystem design. Cheers, Pau 'WEiRD' de Weerd -- >[<++>-]<+++.>+++[<-->-]<.>+++[<+ +++>-]<.>++[<>-]<+.--.[-] http://www.weirdnet.nl/
Get total size of all files in directory using unit Bytes?
for example 1. there is only two file in /home/test/ # ls -l /home/test/ total 8 -rw-r--r-- 1 root wheel 2 Mar 3 23:29 a.txt -rw-r--r-- 1 root wheel 3 Mar 3 23:29 b.txt So the total size of all files ( a.txt + b.txt ) should be 5 Bytes. How to get total size ( 5 Bytes ) directly but not the 8 Bytes. 2. # man du says, -h "Human-readable" output. Use unit suffixes: Byte, Kilobyte, Megabyte, Gigabyte, Terabyte, Petabyte, Exabyte in order to reduce the number of digits to four or less. but # du -h /home/test 6.0K/home/test How to let unit suffixes: Byte display (come out) but not Kilobyte? just like # du -b /home/test ( option "-b" not exist) 5B /home/test