Getting file sizes
Newbie question: How can I get the total size, in K, of all files in a directory that match a pattern? For example, I have a dir with ~5000 files, I would like to know the total size of the ~1000 files matching *.txt. On RHEL and bash, if it matters... Thanks, Kent ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Getting file sizes
On Mon, 2007-10-22 at 09:11 -0400, Kent Johnson wrote: Newbie question: How can I get the total size, in K, of all files in a directory that match a pattern? For example, I have a dir with ~5000 files, I would like to know the total size of the ~1000 files matching *.txt. du -c *.txt | tail -1 (That's -(one), not -(ell), meaning, you only want the last line of output from du.) du prints out the sizes of each of the matching files; '-c' means you want a total, too; piping the output through tail -1 picks out just the last line with the total. -- Stephen Ryan Dartware, LLC ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Getting file sizes
On Monday 22 October 2007 09:11, Kent Johnson wrote: Newbie question: How can I get the total size, in K, of all files in a directory that match a pattern? For example, I have a dir with ~5000 files, I would like to know the total size of the ~1000 files matching *.txt. Ah! Perhaps I actually know an answer to this one. (Very rare) Go to directory of interest and try du -shc *.txt Jim Kuzdrall ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Getting file sizes
More than you asked for, but here's a command that reports total space occupied by all files with names ending in .jpg, recursively from the current directory (but not crossing mount points) and which is also a gratuitous example of the Process Substitution facility mentioned in a previous thread: du -c -h --files0-from=(find . -xdev -type f -name *.jpg -print0 2/dev/null) | tail -1 ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Getting file sizes
Kent Johnson wrote: Newbie question: How can I get the total size, in K, of all files in a directory that match a pattern? For example, I have a dir with ~5000 files, I would like to know the total size of the ~1000 files matching *.txt. On RHEL and bash, if it matters... Thanks, Kent ___ To get the result in K, specify: du -c --block-size=1024 *.txt or your choice of what you think K means ;) man du tells more. -- Ted Roche Ted Roche Associates, LLC http://www.tedroche.com ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Getting file sizes
Jim Kuzdrall wrote: On Monday 22 October 2007 09:11, Kent Johnson wrote: How can I get the total size, in K, of all files in a directory that match a pattern? For example, I have a dir with ~5000 files, I would like to know the total size of the ~1000 files matching *.txt. Ah! Perhaps I actually know an answer to this one. (Very rare) Go to directory of interest and try du -shc *.txt That still lists each file individually, it needs to pipe to tail as Stephen suggested. Kent ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Getting file sizes
Ooops - that --files0-from= option is apparently new enough (my du version is 5.97) that it's probably not widely available. My home system has it, but my work systems don't... -/ ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Getting file sizes
Kent Johnson [EMAIL PROTECTED] writes: Newbie question: How can I get the total size, in K, of all files in a directory that match a pattern? Stephen Ryan [EMAIL PROTECTED] writes: du -c *.txt | tail -1 du prints out the sizes of each of the matching files; '-c' means you want a total, too; piping the output through tail -1 picks out just the last line with the total. Hmmm, I wouldn't have chosen 'tail -1'. My instinct would have been to 'grep -i total', which is both more typing, and not as accurate (what if there was a filename containing the string 'total'?). Michael ODonnell [EMAIL PROTECTED] writes: du -c -h --files0-from=(find . -xdev -type f -name *.jpg -print0 \ 2/dev/null) | tail -1 Hmm, again, certainly not my fist instinct :) I almost *never* think to redirect stdin this way for some reason. Had I come up with the answer, I probably would have written it more like: find . -type f -name \*.muse -print0 | du -c --files0-from=- | tail -1 Which yields the same answer. I think I like mod's better :) Ted Roche [EMAIL PROTECTED] writes: To get the result in K, specify: du -c --block-size=1024 *.txt Hmmm, I would have just used -k, or actually, let it default to that and not specify any flag at all regarding size. or your choice of what you think K means ;) Though, it's very cool that you can specify exactly what you mean here. man du tells more. Indeed it does! What a great little thread! :) -- Seeya, Paul ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Getting file sizes
On 10/22/07, Stephen Ryan [EMAIL PROTECTED] wrote: On Mon, 2007-10-22 at 09:11 -0400, Kent Johnson wrote: Newbie question: How can I get the total size, in K, of all files in a directory that match a pattern? For example, I have a dir with ~5000 files, I would like to know the total size of the ~1000 files matching *.txt. du -c *.txt | tail -1 Since I know Kent has a Mac and this might be on his laptop, I'd like to add that this should really be: du -ck *.txt | tail -1 Although Linux (ie GNU) du defaults to outputting sizes in k, OS X does not. It counts blocks (512 byte blocks) and the -k option to du explicitly says I want output in k and GNU du honors this even though it's the default). For additional examples... Solaris 9 == 512 by default, FreeBSD 6 == 1024 by default, NetBSD 1.6.1 == 512 by default, but they all honor -k -Shawn (That's -(one), not -(ell), meaning, you only want the last line of output from du.) du prints out the sizes of each of the matching files; '-c' means you want a total, too; piping the output through tail -1 picks out just the last line with the total. -- Stephen Ryan Dartware, LLC ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/ ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Getting file sizes
Hmm, again, certainly not my fist instinct :) Paul, we embrace diversity here but that is *definitely* OT... ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Getting file sizes
On Monday 22 October 2007 09:36, Kent Johnson wrote: Jim Kuzdrall wrote: On Monday 22 October 2007 09:11, Kent Johnson wrote: How can I get the total size, in K, of all files in a directory that match a pattern? For example, I have a dir with ~5000 files, I would like to know the total size of the ~1000 files matching *.txt. Ah! Perhaps I actually know an answer to this one. (Very rare) Go to directory of interest and try du -shc *.txt That still lists each file individually, it needs to pipe to tail as Stephen suggested. I thought of that, but you just said you wanted the answer. So I gave you the engineering approach: simplest approximation of adequate accuracy; minimum time spent. (It takes less than two seconds to scroll the file names on the screen, and it does confirm what type of files are being counted.) However, next time I need that info (which I often do), I will try to remember the | tail -1 trick. Jim Kuzdrall ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Getting file sizes
On 10/22/07, Shawn K. O'Shea [EMAIL PROTECTED] wrote: Since I know Kent has a Mac and this might be on his laptop, I'd like to add that this should really be: du -ck *.txt | tail -1 Since we're on the subject, it should also be noted that du means *disk usage*. That means du is supposed to be aware of things like allocation overhead (a 3 byte might use 4096 bytes on disk, or whatever) and sparse files (files with holes in the middle, thus using *less* space on disk than the file size). The GNU variant, at least, has an option to report actual file sizes instead of disk usage. Which one you want depends on what you're looking for. -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Getting file sizes
Shawn K. O'Shea wrote: du -c *.txt | tail -1 Since I know Kent has a Mac and this might be on his laptop, I'd like to add that this should really be: du -ck *.txt | tail -1 No, this is a bona fide Linux question :-) it's a Webfaction account. But thanks for the note! Kent ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Getting file sizes
On 10/22/07, Michael ODonnell [EMAIL PROTECTED] wrote: Ooops - that --files0-from= option is apparently new enough ... that it's probably not widely available. find . -xdev -type f -name *.jpg -print0 2/dev/null | xargs -0 du -ch | tail -1 (untested) -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Brother, can you spare a couple of SCSI SCA disks?
Hello, world! GNHLUG has an Internet server (it runs this list, amongst other things). It has a pair of 18 GB disks (mirrored). That is kind of small, by today's standards -- especially if we want to start doing things like hosting videos of meetings. It was pointed out that we might have people on this list who have suitable hardware they would be willing to donate to the cause. So if anyone has such, or comes across such, please let me know. The one wrinkle is that they must be 3.5-inch, 1/3-height, SCSI, 80-pin SCA (single connector attachment) disks. That's all that will fit the 1U server we have. (Yes, I'm aware that SATA disks are cheap. Thing is, they're not cheap if you have to buy a new 1U server to put them in.) GNHLUG is a registered with the state of NH as a non-profit corporation. However, GNHLUG does *not* currently claim any status which would make donations tax-deductible. But you'd get a warm, fuzzy feeling inside. Plus a plug on the list and our website, if you want. Thanks for listening. -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Getting file sizes
On Monday, Oct 22nd 2007 at 10:17 -, quoth Ben Scott: =On 10/22/07, Shawn K. O'Shea [EMAIL PROTECTED] wrote: = Since I know Kent has a Mac and this might be on his laptop, I'd like = to add that this should really be: = du -ck *.txt | tail -1 = = Since we're on the subject, it should also be noted that du means =*disk usage*. That means du is supposed to be aware of things like =allocation overhead (a 3 byte might use 4096 bytes on disk, or =whatever) and sparse files (files with holes in the middle, thus =using *less* space on disk than the file size). = = The GNU variant, at least, has an option to report actual file sizes =instead of disk usage. = = Which one you want depends on what you're looking for. I'd just like to kibbutz one more subtlety: du reports disk usage as discussed above, but another way that you can get seemingly conflicting numbers is from sparse files, i.e., where the length of the file is large, but still contains little data. f = open ( newfile, O_CREAT | O_WRONLY ); lseek ( f, 10, SEEK_SET ); close ( f ); Bang. You now have a file that ls will report as 1 gig and yet still occupies almost no space on the dick. -- Time flies like the wind. Fruit flies like a banana. Stranger things have .0. happened but none stranger than this. Does your driver's license say Organ ..0 Donor?Black holes are where God divided by zero. Listen to me! We are all- 000 individuals! What if this weren't a hypothetical question? steveo at syslang.net ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Brother, can you spare a couple of SCSI SCA disks?
The one wrinkle is that they must be 3.5-inch, 1/3-height, SCSI, 80-pin SCA (single connector attachment) disks. That's all that will fit the 1U server we have. I have a pair of Seagate 36g that I believe fit the bill here, though their size is also nuthin' to write home about... -- especially if we want to start doing things like hosting videos of meetings. not fer nuthin', but may be want to consider a service such as YouTube (I know, flash, not foss) or GoogleVideo for these considerations? It would help with making such videos available to a much wider audience and not exponentially jump up GNHLUG's bandwidth at MV. Just a thought... -- ~ * ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Brother, can you spare a couple of SCSI SCA disks?
On 10/22/07, Star [EMAIL PROTECTED] wrote: The one wrinkle is that they must be 3.5-inch, 1/3-height, SCSI, I have a pair of Seagate 36g that I believe fit the bill here, though their size is also nuthin' to write home about... That would still be twice what we have now. So if you're willing to part with them for a price we can afford (i.e., free), that would be *sweet*. ... consider a service such as YouTube ... It's a near-certainty that we will be making use of such. But we may still end up hosting some stuff ourselves. handwave And as Ted notes, even for intermediate storage, we need more storage. -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Brother, can you spare a couple of SCSI SCA disks?
That would still be twice what we have now. So if you're willing to part with them for a price we can afford (i.e., free), that would be *sweet*. I'd be willing to let 'em go for Fifty Nothings apiece, me thinks... I'll double-check 'em tonight to make sure that they're the correct connectors and let you know. -- ~ * ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Brother, can you spare a couple of SCSI SCA disks?
On 10/22/07, Ben Scott [EMAIL PROTECTED] wrote: On 10/22/07, Star [EMAIL PROTECTED] wrote: The one wrinkle is that they must be 3.5-inch, 1/3-height, SCSI, What kind of hardware are these? Linux x86? SCA SCSI is kind of rare on x86 and bang/buck is low. If it's an option, how about a SATA PCI card SATA drives? ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Brother, can you spare a couple of SCSI SCA disks?
On 10/22/07, Tom Buskey [EMAIL PROTECTED] wrote: What kind of hardware are these? Linux x86? Linux x86. Actually, the hardware is x86 no matter what OS is running on it. ;-) Gory details are available: http://wiki.gnhlug.org/twiki2/bin/view/Organizational/ServerHardware SCA SCSI is kind of rare on x86 ... If you say so. I've been getting SCA on most of my x86 servers for something close to a decade now. :) I suppose if you compare vs the entire population of x86 boxes, including $200 Wal-Mart specials, then yah, SCA is rare. Of course, SCA is being displaced by SAS now, which has a yet another incompatible connector. Yay standards! ;) ... bang/buck is low. Divide by zero error. (The server was also free.) If it's an option, how about a SATA PCI card SATA drives? Thought about it. Thing is, I think the SCA backplane also serves to connect some of the chassis fans. If so, removing it (to make room for the SATA and power cables and connectors) would be a bad idea. Even if it is possible, we'd loose hot swap, and so disk changes would be a pain. Not that we can't take it out of the rack if we have to, but it'd be nice not to have to. That said, the chassis does have room for a PCI card, and my hacksaw still works, so it may be an option, should push come to shove. :) -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Brother, can you spare a couple of SCSI SCA disks?
On 10/22/07, Ben Scott [EMAIL PROTECTED] wrote: SCA SCSI is kind of rare on x86 ... If you say so. I've been getting SCA on most of my x86 servers for something close to a decade now. :) I suppose if you compare vs the entire population of x86 boxes, including $200 Wal-Mart specials, then yah, SCA is rare. The only place I'd seen SCA is on Sun boxes :-) But then, I haven't dug into too many x86 servers. Of course, SCA is being displaced by SAS now, which has a yet another incompatible connector. Yay standards! ;) ... bang/buck is low. Divide by zero error. (The server was also free.) If it's an option, how about a SATA PCI card SATA drives? Thought about it. Thing is, I think the SCA backplane also serves to connect some of the chassis fans. If so, removing it (to make room for the SATA and power cables and connectors) would be a bad idea. Even if it is possible, we'd loose hot swap, and so disk changes would be a pain. Not that we can't take it out of the rack if we have to, but it'd be nice not to have to. That said, the chassis does have room for a PCI card, and my hacksaw still works, so it may be an option, should push come to shove. :) Needs to be rack mounted then. I've used the following which might be acceptable for GNHLUG or not: Leave the OS disks on a RAID1 setup. Data on the following setup. 4 port SATA card 4 42 long SATA cables 4 short SATA extenders (from card to outside of box) external case with power supply (an old PC) 4 pin power adapter to 2 SATA power ends 4 SATA drives 4 short SATA extenders from drives to outside of case fans in case pulling air through drives There's a $20 4 port SATA card that works nice with Linux. There are IDE to SATA adapters for $20ish. They need a floppy power connector. I have a test setup at work with an external storage tower ($169). We could take up a collection for the parts drives. I have an extra IDE to SATA adapter and $20 toward the SATA card. Maybe someone has an old 1U server chassis w/ a power supply and fans? Maybe someone has an extra IDE or SATA drive? SATA is inherently hot plug. I've read about someone adding a SATA drive to an running linux box. It reminds me of doing it with external SCSI drives. -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/ ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: You really never need any other application...
Paul Lussier [EMAIL PROTECTED] writes: http://1010.co.uk/gneve.html Because, really, why would you *want* to? I wonder when we'll be able to save clips to the kill ring? --kevin -- GnuPG ID: B280F24E God, I loved that Pontiac. alumni.unh.edu!kdc -- Tom Waits ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Brother, can you spare a couple of SCSI SCA disks?
On 10/22/07, Tom Buskey [EMAIL PROTECTED] wrote: The only place I'd seen SCA is on Sun boxes Yah, we pee cee weenies have SCA too. ;-) Needs to be rack mounted then. Needs is a strong word, but MV Communications has been generously hosting us for free, and the fact that it only takes up 1U in their racks is, I'm sure, making that easier for them to do. Remember, in many datacenters, the most expensive commodity is space. Not power, not bandwidth, but the physical space the machine takes up. I've used the following which might be acceptable for GNHLUG or not ... Well, the non-zero price tag and larger than 1U size might be an issue. It's not that I've got anything against SATA, but we've got certain limitations due to what we're working with here. The world is not your living room. :-) If somebody wants to donate sweet new 1U server with dual 500 GB SATA disks, that's fine too ;-) SATA is inherently hot plug. But the drives won't be, without a hot plug backplane. (How you gonna hook up those cables if the drives are stuffed into a 1U pizza box chassis with no room to spare?) -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Brother, can you spare a couple of SCSI SCA disks?
Hmm, I always thought it was power, not space. Datacenter space varies wildly, but use $20/sqft/mo. A typical cabinet will take about 17sqft on average (that's not actual footprint, but allowing for aisleways, etc). So, $240/mo for the space. A 42U cabinet will generally hold about 35U of actual servers. So each billable U costs about $6.86/mo. An average 1U server draws anywhere from 2-5A of power (Ie: something running on a 300-500W poersupply). Considering the server runs 24/7, it's a constant draw. Rule of thumb in most places is AMPS x $10/mo for power. It varies from place to place, but that's worked for me locally. So, power is $20-$50/mo. Plus cooling costs. $6.86 for space. $20+/mo for power. Wholesale bandwidth is going for about $25-$40/Mb these days, and that can be heavily subscribed, so I agree that the bandwidth isn't the cost in most places (unless you want a dedicated pipe). On Oct 22, 2007, at 4:17 PM, Ben Scott wrote: Remember, in many datacenters, the most expensive commodity is space. Not power, not bandwidth, but the physical space the machine takes up. ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Brother, can you spare a couple of SCSI SCA disks?
On 10/22/07, Brian [EMAIL PROTECTED] wrote: Hmm, I always thought it was power, not space. many != all != most -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Brother, can you spare a couple of SCSI SCA disks?
On 10/22/07, Ben Scott [EMAIL PROTECTED] wrote: On 10/22/07, Tom Buskey [EMAIL PROTECTED] wrote: The only place I'd seen SCA is on Sun boxes Yah, we pee cee weenies have SCA too. ;-) As I should have said, I haven't looked into enough PCs :-) Needs to be rack mounted then. Needs is a strong word, but MV Communications has been generously hosting us for free, and the fact that it only takes up 1U in their racks is, I'm sure, making that easier for them to do. Needs to look fairly decent and not take up lots of space. Remember, in many datacenters, the most expensive commodity is space. Not power, not bandwidth, but the physical space the machine takes up. I've used the following which might be acceptable for GNHLUG or not ... Well, the non-zero price tag and larger than 1U size might be an issue. It's not that I've got anything against SATA, but we've got certain limitations due to what we're working with here. The world is not your living room. :-) Well, the setup I described could be an old 1U server case stacked on top of the current server. Ways to mount the disks into the chassis are left to the imagination. Early Google internal servers, Nyx.com systems, etc might be examples. It doesn't have to look that ugly. SATA is inherently hot plug. But the drives won't be, without a hot plug backplane. (How you gonna hook up those cables if the drives are stuffed into a 1U pizza box chassis with no room to spare?) Let me try an ASCI drawing.. (sorry for going HTML mode to get courier) serverDrive SATA case SATA Cable case SATA card -|-|= -|-|- =|-|- Disk SATASATA extenderextender 1) umount the disks 2) power off the drive case 3) unplug each the SATA cable from each SATA extender 4) swap out repair 5) reverse 4 through 1 Your current server stays the same with the addition of a 4 port SATA card and 4 SATA extenders from the card ports to the outside of the chassis (label each cable with the port :-) Your OS is on the 2 current SCA drives. Your data is on the external SATA disk in the other case. Think of it like external SCSI drives everyone used to (still?) use with much cheaper cables. You could even use an old external SCSI drive case w/ 1 disk, one extender. ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Brother, can you spare a couple of SCSI SCA disks?
OK. I've never seen a single datacenter where you were paying for space above anything else. Even if that's the unit of measure being sold. On Oct 22, 2007, at 5:08 PM, Ben Scott wrote: On 10/22/07, Brian [EMAIL PROTECTED] wrote: Hmm, I always thought it was power, not space. many != all != most -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/ ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Brother, can you spare a couple of SCSI SCA disks?
On 10/22/07, Tom Buskey [EMAIL PROTECTED] wrote: Needs to look fairly decent and not take up lots of space. Pretty much. Well, really, needs to not take lots of space, power, or bandwidth. SATA is inherently hot plug. 2) power off the drive case Just to be a wise-ass: In the above, there's a small wrinkle in your hot plug scenario... ;-) You could, of course, use two old PC cases (one drive in each), or, more sanely, just a couple of small external disk enclosures, which go for less than $25 these days. In any event, if someone wants to donate any of that, or money to buy same, I'm sure we'd be happy to take it. :) -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Brother, can you spare a couple of SCSI SCA disks?
On Mon, Oct 22, 2007 at 04:17:07PM -0400, Ben Scott wrote: If somebody wants to donate sweet new 1U server with dual 500 GB SATA disks, that's fine too ;-) Not new, and not really all that sweet, but I have a couple of old (PIII-era) 1U servers that I'd be willing to donate to the cause. If memory serves they've each got SCSI IDE internally and room for three or four drives. No hot-swap, though. They worked when I turned them off, but that was a few years ago now. I'll dig them out of the closet tonight and let you know the full specs. -- [EMAIL PROTECTED] OpenPGP KeyID 0x57C3430B Holder of Past Knowledge CS, O- From a certain point onward there is no longer any turning back. That is the point that must be reached. Franz Kafka ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Brother, can you spare a couple of SCSI SCA disks?
On 10/22/07, Ben Scott [EMAIL PROTECTED] wrote: On 10/22/07, Tom Buskey [EMAIL PROTECTED] wrote: Needs to look fairly decent and not take up lots of space. Pretty much. Well, really, needs to not take lots of space, power, or bandwidth. SATA is inherently hot plug. 2) power off the drive case Just to be a wise-ass: In the above, there's a small wrinkle in your hot plug scenario... ;-) Maybe I'm using the term wrong. What I mean is you can plug the SATA drive into the system while the system is running. You could power the disk up before or after plugging it in. You will have to umount the disks before unplugging. The important thing is the data disks can be swapped in/out without shutting down the rest of the system. You could, of course, use two old PC cases (one drive in each), or, more sanely, just a couple of small external disk enclosures, which go for less than $25 these days. Or a 1U server case with a PC power supply, and put all the drives in it. You'd power on/off the whole unit with all the disks at the same time. I've used external SCSI boxes that were similar. Gee, you have a SCSI controller. Just get a SCSI cable, drives and external chassis. You save $20 on the controller, but the cable is probably 3-5x the cost of the SATA cables. You could use an old PC for the case and power. The disks will cost lots more then SATA. I realize this is all a bit hackish/kludgey. But when you have no budget, you make do. I've set this up a number of times. In any event, if someone wants to donate any of that, or money to buy same, I'm sure we'd be happy to take it. :) I'm willing to throw $$ at the GNHLUG server farm. Let me know where to send a check. It's cheaper then 10 years of dues :-) I have external SCSI cases I can make available if you want to go SCSI instead of SATA. ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
[GNHLUG] PySIG this Thursday - Beautiful Soup - HTML scraping; actual code developed before your eyes
PySIGManchester, NH 25 October 2007 Kent Johnson: Beautiful Soup Us Ourselves: Python in Action PySIG -- New Hampshire Python Special Interest Group Amoskeag Business Incubator, Manchester, NH 25 October 2007 (4th Thursday) 7:00PM The monthly meeting of PySIG, the NH Python Special Interest Group, takes place on the fourth Thursday of the month, starting at 7:00 PM. Beginners' session precedes at 6:30 PM. (Bring a Python question!) Kent's Korner - Kent Johnson: Beautiful Soup Beautiful Soup is a Python HTML/XML parser designed for quick turnaround projects like screen-scraping. Three features make it powerful: 1. Beautiful Soup won't choke if you give it bad markup. It yields a parse tree that makes approximately as much sense as your original document. This is usually good enough to collect the data you need and run away. 2. Beautiful Soup provides a few simple methods and Pythonic idioms for navigating, searching, and modifying a parse tree: a toolkit fordissecting a document and extracting what you need. You don't have to create a custom parser for each application. 3. Beautiful Soup automatically converts incoming documents to Unicode and outgoing documents to UTF-8. You don't have to think about encodings, unless the document doesn't specify an encoding and Beautiful Soup can't autodetect one. Then you just have to specify the original encoding. Beautiful Soup parses anything you give it, and does the tree traversal stuff for you. You can tell it 'Find all the links', or 'Find all the links of class externalLink', or 'Find all the links whose urls match foo.com', or 'Find the table heading that's got bold text, then give me that text.' Valuable data that was once locked up in poorly-designed websites is now within your reach. Projects that would have taken hours take only minutes with Beautiful Soup. 1st-ever PySIG development sprint: we try to write Actual Code Per a Challenge from Ted Roche. Viz, Recently, I started messing with some of the data we have stored on GNHLUG.org, specifically, the Past Events page: http://wiki.gnhlug.org/twiki2/bin/view/Www/PastEvents Just like an end user, I had some questions, simple to ask, tough to answer. We have attendance data for most meetings since September of 2005. Given those dates to present, 1. What's the average monthly attendance at a GNHLUG event? 2. What's the average attendance over that period per group/chapter/SIG? 3. What's the most popular meeting? Most popular for each group? 4. What's the trends in attendance? Up, down? It seems like an interesting real-world problem: scrape a web page of questionable HTML, interpret dirty data (not all groups are groups, not all atttendance numbers are numbers), dump it into a database (or perhaps a spreadsheet?) and do the calculations. Pretty graphs get extra points. I think the results would be interesting, and the process of getting to the results interesting, too: presenting how you take on the problem, what tools you use, how much code is needed, would make a fun meeting not only inside the SIG, but to LUG meetings as well. --posted to PySIG mailing list 26 Sept 07 And the group thought so too... so here we go! We'll throw code up on the screen, develop the screenscraper Ted envisions (or as much of it as we can get done in the available time) and publish the results. A real world test of RAD, as empowered by Python (and its batteries-included libraries -- in this case Beautiful Soup). All are welcome. Come to help us code. Or come to laugh :) Plus: --- o Kent's Korner The Real Stuff! - Kent Johnson This month: Beautiful Soup (see above) Upcoming Kent's Korner topics: XML parsing (ElementTree) Profiling (timeit, prof) o Our usual roundtable of introductions, happenings, announcements o Gotcha contest - Got a favorite gotcha? Bring it and share... And of course, milk cookies. Cookies are assured, thanks to Janet. Milk also, thanks to Alex. --- 6:30 Beginners' QA 7:00 Welcome, Announcements - Bill Ted Alex 7:10 Milk Cookies - Alex Janet 7:10 Favorite-gotcha contest 7:15 Kent's Korner (Python Module of the Month) -
Re: Brother, can you spare a couple of SCSI SCA disks?
Bugger! My drives are not the 80-pin connectors :( Anyone else? -- ~ * ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
Re: Brother, can you spare a couple of SCSI SCA disks?
On 10/22/07, Tom Buskey [EMAIL PROTECTED] wrote: 2) power off the drive case Just to be a wise-ass: In the above, there's a small wrinkle in your hot plug scenario... ;-) Maybe I'm using the term wrong. What I mean is you can plug the SATA drive into the system while the system is running. Mainly, I was just being a wise-ass. :) In all seriousness, there's no industry-standard definition, so your usage is as valid as anyone's. But to me, hot plug means without having to power things off first. But either way, having to turn off both drives in the mirror set in order to swap a failed member makes the hot swap functionality a lot less useful. Hence the idea of two external enclosures; you power down the one enclosure but keep the other running. The important thing is the data disks can be swapped in/out without shutting down the rest of the system. Nobody really cares if the OS is up if the data it's supposed to be serving is unavailable. :) Gee, you have a SCSI controller. Just get a SCSI cable, drives and external chassis. Not a bad idea. Cables I've got. But I'd prefer not to impinge upon our host's generosity by depositing that kind of hacktacular setup if we don't have to. I've set this up a number of times. Me too./AOL Heck, having a case makes it a high-budget item. What about the ever-popular bare disk drive sitting on a table design? :) I have external SCSI cases I can make available if you want to go SCSI instead of SATA. Well, your enclosures combined with my cables and Star's disk drives might yield a not crappy, if somewhat space-wasteful, design. I think I even have a pass-through bracket, so we could put the disks on different SCSI channels and maintain hot-swap. But let's see what else washes ashore first. :-) -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss/
