Re: Hard Disk Failure
On 2/26/06, Jeff Kinz <[EMAIL PROTECTED]> wrote: >> Hard drive platters aren't "rusty" at all; they typically have a >> mirror finish. Perhaps that was a reference to the magnetic coatings > > Yes it was , hence the "quotes" "around" "the" "rust" Well, if you knew what the right term was, why didn't you use it? :-) > C'mon Ben, Do you really think the NSA is going to tell us if they shred > their drives? Then they wouldn't be able to smoke out parties trying > recover int. from the used hard drives sold by the gov't anymore... :-) I gather you're being facetious, but: You seem to be operating under the assumption that there is a "they". The "they" is "us". As it happens, I've had a couple of jobs where the sanitization of classified data storage was a concern of mine. The government security people are just that -- people, just like you and me. "They" have a job, and manuals, and things they like, and things they disagree with, and all that. You don't get a decoder ring or a secret handshake when you work with the NSA, just a lot more paperwork. :-/ But when it came to sanitization (rendering information unrecoverable by any known means), "they" were quite clear about what was required. I suppose it's possible there's some super-secret level of classification that's so secret "they" don't even tell "us" about it, but hey, maybe I know about that too and just aren't telling "you" about it. ;-) -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Hard Disk Failure
On Sun, Feb 26, 2006 at 07:31:28PM -0500, Ben Scott wrote: > On 2/25/06, Jeff Kinz <[EMAIL PROTECTED]> wrote: > > A long time ago, working on a contract for equipment destined to be > > delivered to the NSA I heard a rumor about the procedure that you had to > > go through to remove the hard drive from an NSA building: > > > > Basically the drive was disassembled, the "rust" was sanded from the > > platters, the platters were shredded, and all of the "rust" and shredded > > platter material were placed in a crucible and melted down.the drive can > > now be removed from the building. > > Hard drive platters aren't "rusty" at all; they typically have a > mirror finish. Perhaps that was a reference to the magnetic coatings Yes it was , hence the "quotes" "around" "the" "rust" > on the platters; while not iron, I can sorta see an analogy there. Sorta? Sorta? Just for that I'm not going to tell the story about the NSA secretary who moved her desk phone 11 inches closer to her. :P ! and lost her job. > Sanitizing procedures for disks containing TOP SECRET data require > one to obliterate the recording medium. For hard disks, that means > either sanding down the surfaces of the platters, or melting down the > entire platter. Shredding doesn't really enter the picture; it's not > required in addition to that, and is not sufficient by itself. C'mon Ben, Do you really think the NSA is going to tell us if they shred their drives? Then they wouldn't be able to smoke out parties trying recover int. from the used hard drives sold by the gov't anymore... :-) "Being paranoid, Its not just a job, Its a hell of a lot of fun! " (The obnoxious intelligence officer that dropped in on M*A*S*H) -- Jeff "Don't quote me" Kinz, Emergent Research, Hudson, MA. speech recognition software may have been used to screw up this e-mail Someone tell Monica that sometimes a cigar is just a cigar. ba-da bing! (Thank you, I'll be here all week, try the veal.) ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: using make to convert images to thumbs
On Sun, 2006-02-26 at 12:49 -0500, Dave Johnson wrote: > thumb/%: % > Why couldn't I see that? I must have a monkey-see-monkey-do sort of brain. I was just hung up on suffixes (which are both .jpg) and more exotic patterns. The Makefile works great. Thanks very much to all. -- Lloyd Kvam Venix Corp ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Hard Disk Failure
From: "Ben Scott" <[EMAIL PROTECTED]> Date: Sun, 26 Feb 2006 19:31:28 -0500 From what I understand, many drives have additional diagnostics and commands in addition to the standard "read block X" stuff. However, these commands are manufacturer-specific. Sometime during 2.4, the kernel gained an ability to talk to IDE devies in a "raw" mode. Though the kernel support exists, I haven't seen any tools which actually use it to talk to drives directly. :) Here's an excerpt from linux-2.4.20/Documentation/Configure.help: > Raw Access to Media > CONFIG_IDE_TASK_IOCTL > This is a direct raw access to the media. It is a complex but > elegant solution to test and validate the domain of the hardware and > perform below the driver data recover if needed. This is the most > basic form of media-forensics. > > If you are unsure, say N here. > > Use Taskfile I/O > CONFIG_IDE_TASKFILE_IO > This is the "Jewel" of the patch. It will go away and become the new > driver core. Since all the chipsets/host side hardware deal w/ their > exceptions in "their local code" currently, adoption of a > standardized data-transport is the only logical solution. > Additionally we packetize the requests and gain rapid performance and > a reduction in system latency. Additionally by using a memory struct > for the commands we can redirect to a MMIO host hardware in the next > generation of controllers, specifically second generation Ultra133 > and Serial ATA. > -- "So many people like XML because so few people understand Scheme." ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Hard Disk Failure
From: Bill McGonigle <[EMAIL PROTECTED]>
Date: Sun, 26 Feb 2006 15:52:34 -0500
On Feb 26, 2006, at 15:31, [EMAIL PROTECTED] wrote:
> okay, okay, so maybe my command can't copy the partitions
> BACKWARDS...but... *scratches head*
but often sectors that won't read forward will read backwards. I
don't understand the drive physics, but I've recovered drives with
I'm willing to bet money that's not true. The "drive physics" of
recovering data with dd_rescue are no different than any other fs
access (including using dd). dd_resuce *probably* just lseek(2)s
along the drive from back to front.
I'm sure the guys who write these programs understand dd well and more
about drives than we do.
To confirm my suspicion that the author of this program is in fact a
GUI baby, I just downloaded a copy of
11124beac799890c3048c9a5ccf4746b dd_rescue.c
The code uses open(2) to open the input file... which gains you
nothing over stock dd. I honestly can't think a reason to even run gcc
on this program. ?!
Dave
--
"So many people like XML because so few people understand Scheme."
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Hard Disk Failure
On 2/25/06, Jeff Kinz <[EMAIL PROTECTED]> wrote: > It would actually makes sense for the hard drive internals to be able to > deliver information about what the analog head reads were seeing to the > external interface because it would allow the manufacturer/developers, > or technicians to do really deep, meaningful diagnostics on a drive. From what I understand, many drives have additional diagnostics and commands in addition to the standard "read block X" stuff. However, these commands are manufacturer-specific. > A long time ago, working on a contract for equipment destined to be > delivered to the NSA I heard a rumor about the procedure that you had to > go through to remove the hard drive from an NSA building: > > Basically the drive was disassembled, the "rust" was sanded from the > platters, the platters were shredded, and all of the "rust" and shredded > platter material were placed in a crucible and melted down.the drive can > now be removed from the building. Hard drive platters aren't "rusty" at all; they typically have a mirror finish. Perhaps that was a reference to the magnetic coatings on the platters; while not iron, I can sorta see an analogy there. Sanitizing procedures for disks containing TOP SECRET data require one to obliterate the recording medium. For hard disks, that means either sanding down the surfaces of the platters, or melting down the entire platter. Shredding doesn't really enter the picture; it's not required in addition to that, and is not sufficient by itself. For lower levels of classified data, one used to be able to use certified degaussing or software tools. However, these are becoming less and less allowed these days -- both storage and recovery technology has become so sophisticated, and information densities so high, that it is difficult to be sure everything is gone. Physical destruction is becoming the universal requirement. Google for "clearing and sanitization matrix" to see some of the requirements that have been published in the past. -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Hard Disk Failure
> I don't understand the drive physics, but I've recovered drives with > dd_rhelp that dd couldn't salvage. dd_rhelp's main feature is that it recovers the "easy" stuff first, saving the "hard" work for last. Thus if the drive has only a few bad blocks, everything else gets copied first, and then it sits there retrying until it gets the last bits of data or you kill it. I dunno if dd_rescue just makes this easier, or if it could all be done with dd, or what. -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Unkillable processes?
On 2/25/06, Michael ODonnell <[EMAIL PROTECTED]> wrote: > The kernel has no notions such as "run level" and "single-user > mode" So, to the extent that frobbing your run level has > any effect at all on any wedged processes, it's a safe bet that > it's a distant side effect of all the other flailing that's going on. See also: Shotgun debugging. http://www.catb.org/jargon/html/S/shotgun-debugging.html "... the making of relatively undirected changes to software in the hope that a bug will be perturbed out of existence ..." :-) -- Ben ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Hard Disk Failure
On Feb 26, 2006, at 15:31, [EMAIL PROTECTED] wrote: okay, okay, so maybe my command can't copy the partitions BACKWARDS...but... *scratches head* but often sectors that won't read forward will read backwards. I don't understand the drive physics, but I've recovered drives with dd_rhelp that dd couldn't salvage. I'm sure the guys who write these programs understand dd well and more about drives than we do. -Bill - Bill McGonigle, Owner Work: 603.448.4440 BFC Computing, LLC Home: 603.448.1668 [EMAIL PROTECTED] Cell: 603.252.2606 http://www.bfccomputing.com/Page: 603.442.1833 Blog: http://blog.bfccomputing.com/ VCard: http://bfccomputing.com/vcard/bill.vcf ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Unkillable processes?
From: Jerry Feldman <[EMAIL PROTECTED]> Date: Sat, 25 Feb 2006 11:08:21 -0500 unkillable processes tend to be locked into an effective deadlock that is resolved when you get rid of some of the peers and ancestors.=20 Transitioning to single user mode also shuts down the network.=20 I think you may have found the reason why this method works here... However, there can be cases where a process is locked in an indefinite I/O wait. It is very rare when these occur and even single user mode won't clear it. The most common indefinite I/O wait that I I can easily get rm locked in uninterruptable sleep (state "D") by doing rm -rf /very/large/directory/ on a reiser3 partition. Control never returns to the shell and no signal on earth will remove the process. telinit 1/3, however, works *every* time. Hypothesis: rm could be deadlocking with other processes/states as mentioned above. -- "So many people like XML because so few people understand Scheme." ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: Hard Disk Failure
/me can't stand it when people who can't read man pages write new utils to do things that can be done with options to existing commands: % n=0 % dd if=/my/bad/drive of=/target/drive skip=$n seek=$n conv=sync,noerror okay, okay, so maybe my command can't copy the partitions BACKWARDS...but... *scratches head* -- "So many people like XML because so few people understand Scheme." ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: using make to convert images to thumbs
Dave Johnson writes: > Python writes: > > I have a client that laid out their images and thumbs into almost > > parallel directory structures. > > /img/thumb > > /x /img > > /y /x > > /*.jpg /y > > /*.jpg > > > > x and y are two digit directory names used to shorten the directory > > scans to retrieve files. I thought I could use a Makefile to automate > > the creation of thumbs, but this pattern has me stumped. > > > > I am going to just write a script to step through the source tree and > > check the corresponding time stamp unless someone offers a suggestion. > > (No they will not shuffle things around because they have lots of logic > > in place.) > > slightly improved clean rule: -- Dave .DEFAULT: all .DELETE_ON_ERROR: .PHONY: all clean IMAGES=$(wildcard img/*/*/*.jpg) THUMBS=$(patsubst %,thumb/%,$(IMAGES)) all: $(THUMBS) thumb/%: % @mkdir -p $(@D) convert $^ -scale 160x120 -quality 25 $@ clean: $(RM) -r thumb ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: using make to convert images to thumbs
Python writes: > I have a client that laid out their images and thumbs into almost > parallel directory structures. > /img/thumb > /x /img > /y /x > /*.jpg /y > /*.jpg > > x and y are two digit directory names used to shorten the directory > scans to retrieve files. I thought I could use a Makefile to automate > the creation of thumbs, but this pattern has me stumped. > > I am going to just write a script to step through the source tree and > check the corresponding time stamp unless someone offers a suggestion. > (No they will not shuffle things around because they have lots of logic > in place.) here you go: .DEFAULT: all .DELETE_ON_ERROR: .PHONY: all clean IMAGES=$(wildcard img/*/*/*.jpg) THUMBS=$(patsubst %,thumb/%,$(IMAGES)) all: $(THUMBS) thumb/%: % @mkdir -p $(@D) convert $^ -scale 160x120 -quality 25 $@ clean: $(RM) $(THUMBS) -- Dave ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: using make to convert images to thumbs
Not that using "make" won't work, but I don't see the advantage of the approaches being discussed versus a simple shell script. make is cool because it allows you to describe arbitrarily complex dependency relationships between various objects so that it can analyze those relationships to determine whether any actions are necessary to keep those relationships current. But it sorta looks like the approach just suggested simply launches an instance of bash for every target, with bash doing ALL the work, so why not just express that same logic in a simple shell script and dispense with make entirely? If you're really committed to using make, you might consider generating one giant rules file using a shell script that sweeps through the directory(s) in question and then feed those rules to make, maybe something like this: cd IMAGE_DIR for f in $( find . -type f ) do echo "$THUMB_DIR/$f:$IMAGE_DIR/$f" >> /tmp/makefile done make -f /tmp/makefile ...with maybe some kind of implicit rule along the lines of the .c.o rule to control the necessary action(s). Just a thought, FWIW, YMMV, IANAL, IMHO, etc... ___ gnhlug-discuss mailing list gnhlug-discuss@mail.gnhlug.org http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: using make to convert images to thumbs
Jason Stephenson wrote:
## Makefile example starts here. ##
IMG_BASE = /img
THM_BASE = /thumb/img
IMG_PROC = /path/to/image/processor
IMG_PROC_OPTS = # default options for image processor
TARGET = # undefined. define on command line
thumbs:
if test "${TARGET}" = "" ; then
echo "TARGET undefined"
else
${IMG_PROC} ${IMG_PROC_OPTS} ${IMG_BASE}/${TARGET} \
${THM_BASE}/${TARGET}
fi
## Makefile example ends here. ##
There are errors in the above Makefile example. One that would actually
run through make would like this:
## Makefile example starts here. ##
IMG_BASE = /img
THM_BASE = /thumb/img
IMG_PROC = /path/to/image/processor
IMG_PROC_OPTS = # default options for image processor
TARGET = # undefined. define on command line
thumbs:
if test "${TARGET}" == "" ; then \
echo "TARGET undefined" \
else \
${IMG_PROC} ${IMG_PROC_OPTS} ${IMG_BASE}/${TARGET} \
${THM_BASE}/${TARGET} \
fi
## Makefile example ends here. ##
I noticed them after I sent the email. Unfortunately, I don't pay as
much attention when typing code in an email than when actually doing
something that I'm going to try to use. ;)
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
Re: using make to convert images to thumbs
Python wrote:
I have a client that laid out their images and thumbs into almost
parallel directory structures.
/img/thumb
/x /img
/y /x
/*.jpg /y
/*.jpg
x and y are two digit directory names used to shorten the directory
scans to retrieve files. I thought I could use a Makefile to automate
the creation of thumbs, but this pattern has me stumped.
How so? It is very straightforward. I'd put the Makefile in the /img
directory or just above it. Your target to make the thumbnails would run
whatever program makes the thumbnails with the appropriate options.
If you needed different target directories, you could specify them in
the environment and on the command line. BSD make makes this trivial and
I'm pretty sure that GNU Make works about the same.
Here's an example:
The Makefile:
## Makefile example starts here. ##
IMG_BASE = /img
THM_BASE = /thumb/img
IMG_PROC = /path/to/image/processor
IMG_PROC_OPTS = # default options for image processor
TARGET = # undefined. define on command line
thumbs:
if test "${TARGET}" = "" ; then
echo "TARGET undefined"
else
${IMG_PROC} ${IMG_PROC_OPTS} ${IMG_BASE}/${TARGET} \
${THM_BASE}/${TARGET}
fi
## Makefile example ends here. ##
Then, on the command line, you might run:
$ make thumbs TARGET=x/y
The above should work with BSD make, and I'm 90% certain that it will
work with GNU make, as well. It might require a little massage to
actually do anything useful.
I am going to just write a script to step through the source tree and
check the corresponding time stamp unless someone offers a suggestion.
(No they will not shuffle things around because they have lots of logic
in place.)
That would work if you have it run periodically and just make thumbs for
new files, or files that have changed. My example above could be used to
do entire subdirectories at once.
If you know much about make, there are ways to make the above fancier
and more bulletproof. For instance, you might want to have IMG_PROC_OPTS
be added to what is specified in the environment rather just being
hardcoded in the Makefile or completely overridden by the command line
environment. This would make it easier to have slightly different
options on a given run.
Also, if you need to specify filenames or globs to the IMG_PROC, you can
do that as part of your target. My assumption above is that the IMG_PROC
can take directories as targets and processess everything in
them.--Perhaps it is a script.
Along the same lines, a fancier solution would run a shell loop over the
files in the TARGET directory, and process each one with IMG_PROC if you
don't have a script to do everything in a directory in one go.
HtH,
Jason
___
gnhlug-discuss mailing list
gnhlug-discuss@mail.gnhlug.org
http://mail.gnhlug.org/mailman/listinfo/gnhlug-discuss
