Re: trouble formatting 3TB Seagate external HDrives. need help
Chris Bannister cbannis...@slingshot.co.nz writes: On Fri, Jun 28, 2013 at 10:24:33PM +, Hendrik Boom wrote: What I do for every new disk before I use it is an exhaustive read/write check with badblocks. It reads and writes every block multiple times with various bitpatterns and random bitpatterns, and check that they can be read correctly. If there's anything wrong with the drive, I return it to the store for a replacement. And yes, I have had to return drives sometimes. Those turned out to have thousands of bad blocks, so good riddance. And they replace it, no questions asked? I've a feeling they'd want some sort of proof? The couple of times I've returned a drive after badblocks-reported failures, Seagate wanted a report from their diagnostic tool (basically equivalent to smartctl --all output) before they would issue a RMA number, and Western Digital didn't care. I've only returned drives directly to manufacturers. The stores where I live don't take returns, and direct you to the manufacturers themselves. -- regards, kushal -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/51d1185f.c402440a.2165.1...@mx.google.com
Re: trouble formatting 3TB Seagate external HDrives. need help
On Mon, 24 Jun 2013 09:55:46 -0600, paul condon wrote: I have two 3TB Seagate external HDs. They were purchased from different stores at slightly different times earlier this year, here in Colorado. I want them to have ext4 file systems on them, excepting if someone on this list can give a reason otherwise. I have googled and gotten a lot of hits, which indicate to me that this is a well known problem. Unfortunately, I have difficulty following the instructions, and all my efforts have not reached a successful conclusion. Now with further trys, it seems to me that stuff has been written onto the drives that needs to be wiped off because I get messages that from the disk utility in xfce4 that it won't overwrite a disk with data on it. So I want to use dd to wipe a complete drive. What I do for every new disk before I use it is an exhaustive read/write check with badblocks. It reads and writes every block multiple times with various bitpatterns and random bitpatterns, and check that they can be read correctly. If there's anything wrong with the drive, I return it to the store for a replacement. And yes, I have had to return drives sometimes. Those turned out to have thousands of bad blocks, so good riddance. The process does wipe the drive clean. And you'll find ouot if the drive is defective -- it's just possible that that's part of your problem. Be careful not to do this with the drive that contains your real OS and data by mistake. For drives larger than 2 terabytes, use gdisk instead of fdisk. Read the online documentation first; there are significant differences. In particular gdisk makes all the changes immediately instead of saving them up to write them when everything's done. So there's no leaing things alone by quitting at the end instead of writing, For this I have found the following: # dd if=/dev/zero of=/dev/sdg bs=1M In the above, I have already changed the HD device to sdg (from sda), but I wonder about bs=1M. Could the process go faster with a larger block size? What are the criteria for choosing a value for bs? And, how long should a 3T wipe take to complete? The job has been running for about 12 hours. Would it go faster with a different bs? Faster enough to make the waste of 12hrs running worthwhile? Is there some way to invoke an 'progress indicator' for dd? And, in general, is there a better way? TIA paul -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/kql2f1$kju$2...@ger.gmane.org
Re: trouble formatting 3TB Seagate external HDrives. need help
On Fri, Jun 28, 2013 at 10:24:33PM +, Hendrik Boom wrote: What I do for every new disk before I use it is an exhaustive read/write check with badblocks. It reads and writes every block multiple times with various bitpatterns and random bitpatterns, and check that they can be read correctly. If there's anything wrong with the drive, I return it to the store for a replacement. And yes, I have had to return drives sometimes. Those turned out to have thousands of bad blocks, so good riddance. And they replace it, no questions asked? I've a feeling they'd want some sort of proof? -- If you're not careful, the newspapers will have you hating the people who are being oppressed, and loving the people who are doing the oppressing. --- Malcolm X -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130629021359.GC17160@tal
trouble formatting 3TB Seagate external HDrives. need help
I have two 3TB Seagate external HDs. They were purchased from different stores at slightly different times earlier this year, here in Colorado. I want them to have ext4 file systems on them, excepting if someone on this list can give a reason otherwise. I have googled and gotten a lot of hits, which indicate to me that this is a well known problem. Unfortunately, I have difficulty following the instructions, and all my efforts have not reached a successful conclusion. Now with further trys, it seems to me that stuff has been written onto the drives that needs to be wiped off because I get messages that from the disk utility in xfce4 that it won't overwrite a disk with data on it. So I want to use dd to wipe a complete drive. For this I have found the following: # dd if=/dev/zero of=/dev/sdg bs=1M In the above, I have already changed the HD device to sdg (from sda), but I wonder about bs=1M. Could the process go faster with a larger block size? What are the criteria for choosing a value for bs? And, how long should a 3T wipe take to complete? The job has been running for about 12 hours. Would it go faster with a different bs? Faster enough to make the waste of 12hrs running worthwhile? Is there some way to invoke an 'progress indicator' for dd? And, in general, is there a better way? TIA paul -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/51c86c02.8060...@mesanetworks.net
Re: trouble formatting 3TB Seagate external HDrives. need help
paul condon: I have two 3TB Seagate external HDs. They were purchased from different stores at slightly different times earlier this year, here in Colorado. I want them to have ext4 file systems on them, excepting if someone on this list can give a reason otherwise. Ext4 is fine if you don't have any special needs. If you only use it for comparably big files (movies, music, photos), you might want to use mkfs.ext4 -T big. The -m option is probably interesting, too. I have googled and gotten a lot of hits, which indicate to me that this is a well known problem. Unfortunately, I have difficulty following the instructions, and all my efforts have not reached a successful conclusion. Now with further trys, it seems to me that stuff has been written onto the drives that needs to be wiped off because I get messages that from the disk utility in xfce4 that it won't overwrite a disk with data on it. We cannot help if you don't tell us what you tried and how it failed. But generally, overwriting the first megabyte is enough to make the disk look unpartitioned/unused. dd if=/dev/zero of=/dev/sdg bs=1M count=1 So I want to use dd to wipe a complete drive. Should not be necessary and can take a long time. 3TB is about 2,861,023MiB. If your disk could write 100MiB/s, it would still need almost eight hours to complete this task. Apparently, your disk is slower than that. For this I have found the following: # dd if=/dev/zero of=/dev/sdg bs=1M In the above, I have already changed the HD device to sdg (from sda), but I wonder about bs=1M. Could the process go faster with a larger block size? I doubt it, but you can try. You can add a parameter count=n like I did above to write only n times bytes (bs). Is there some way to invoke an 'progress indicator' for dd? Yes. See the man page: | Sending a USR1 signal to a running `dd' process makes it print I/O | statistics to standard error and then resume copying. | |$ dd if=/dev/zero of=/dev/null pid=$! |$ kill -USR1 $pid; sleep 1; kill $pid | |18335302+0 records in 18335302+0 records out 9387674624 |bytes (9.4 GB) copied, 34.6279 seconds, 271 MB/s And, in general, is there a better way? Just don't do that. :) J. -- In an ideal world I would cure poverty and go to the gym at least three days a week. [Agree] [Disagree] http://www.slowlydownward.com/NODATA/data_enter2.html signature.asc Description: Digital signature
Re: trouble formatting 3TB Seagate external HDrives. need help
On Mon, 24 Jun 2013, paul condon wrote: I have two 3TB Seagate external HDs. They were purchased from different stores at slightly different times earlier this year, here in Colorado. I want them to have ext4 file systems on them, excepting if someone on this list can give a reason otherwise. I Ext4 is a good option. have googled and gotten a lot of hits, which indicate to me that this is a well known problem. Unfortunately, I have difficulty following the instructions, and all my efforts have not reached a successful conclusion. Now with further trys, it seems to me that stuff has been written onto the drives that needs to be wiped off because I get messages that from the disk utility in xfce4 that it won't overwrite a disk with data on it. Note that old versions of fdisk, parted, etc can't handle 3TB drives properly. If you're running the latest release of Debian (Wheezy) you should be OK though -- if not, be careful. Some ancient SATA cards and quite a few USB adapters can't handle 3TB drives at all -- make sure that 'unformatted size' is reported correctly and you're probably OK. So I want to use dd to wipe a complete drive. For this I have found the following: # dd if=/dev/zero of=/dev/sdg bs=1M It's very unlikely that you need to wipe the full drive. You may just have to clear metadata at the front. This would do that: dd if=/dev/zero of=/dev/sdg bs=1M count=1 In the above, I have already changed the HD device to sdg (from sda), but I wonder about bs=1M. Could the process go faster with a larger block size? Not really. bs=1M is fine for what is being discussed. dd just pushes 'count' data blocks of size 'bs' through to its output (/dev/sdg in your example). You can hit performance issues if the chosen bs clashes with an internal cache and/or the native block size of the disk, but that requires small (kB'ish) block sizes. What are the criteria for choosing a value for bs? And, how long should a 3T wipe take to complete? The job has been running for about (3*1024*1024 MB)/(200 MB/sec) / (3600 sec/hour) = 4.3 hours 200 MB/sec may be optimistic. I get 180 MB/sec on my (mediocre) system. Note that if you're connected with USB2, then you're looking at 10 MB/sec and 86 hours... 12 hours. Would it go faster with a different bs? Faster enough to make the waste of 12hrs running worthwhile? Is there some way to invoke an 'progress indicator' for dd? And, in general, is there a better way? From 'man dd' (towards the end): Sending a USR1 signal to a running `dd' process makes it print I/O statistics to standard error and then resume copying. $ dd if=/dev/zero of=/dev/null pid=$! $ kill -USR1 $pid; sleep 1; kill $pid 18335302+0 records in 18335302+0 records out 9387674624 bytes (9.4 GB) copied, 34.6279 seconds, 271 MB/s -- Brad -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/20130624162518.ga21...@aether.jlab.org
Re: trouble formatting 3TB Seagate external HDrives. need help
On Mon, Jun 24, 2013 at 09:55:46AM -0600, paul condon wrote: I have two 3TB Seagate external HDs. They were purchased from different stores at slightly different times earlier this year, here in Colorado. I want them to have ext4 file systems on them, excepting if someone on this list can give a reason otherwise. I have googled and gotten a lot of hits, which indicate to me that this is a well known problem. Unfortunately, I have difficulty following the instructions, and all my efforts have not reached a successful conclusion. Now with further trys, it seems to me that stuff has been written onto the drives that needs to be wiped off because I get messages that from the disk utility in xfce4 that it won't overwrite a disk with data on it. So I want to use dd to wipe a complete drive. For this I have found the following: # dd if=/dev/zero of=/dev/sdg bs=1M In the above, I have already changed the HD device to sdg (from sda), but I wonder about bs=1M. Could the process go faster with a larger block size? Possibly. There exists a tool (https://github.com/sampablokuper/dd-opt) to help you find the fastest blocksize What are the criteria for choosing a value for bs? And, how long should a 3T wipe take to complete? The job has been running for about 12 hours. Would it go faster with a different bs? Faster enough to make the waste of 12hrs running worthwhile? Is there some way to invoke an 'progress indicator' for dd? You can send SIGUSR1 to a running dd and it will print current statistics to STDERR. term1$ pkill -USR1 dd term2$ dd if=/dev/zero ... 0+14 records in 0+14 records out 204 bytes (204 B) copied, 24.92 seconds, 0.0 kB/s Another alternative would be to run the data through pv: $ dd if=/dev/zero | pv | dd of=/dev/sdg bs=1M And, in general, is there a better way? You could try just overwriting the first few megabytes of the partition. If a mkfs is checking to see if there's already a filesystem in place, chances are it won't look VERY hard. Wiping out the start of the disk should make it look 'clean enough'. So add count=N (where N is a small number) to your dd command line. signature.asc Description: Digital signature
Re: trouble formatting 3TB Seagate external HDrives. need help
On 24/06/13 16:55, paul condon wrote: I have two 3TB Seagate external HDs. They were purchased from different stores at slightly different times earlier this year, here in Colorado. I want them to have ext4 file systems on them, excepting if someone on this list can give a reason otherwise. I have googled and gotten a lot of hits, which indicate to me that this is a well known problem. Unfortunately, I have difficulty following the instructions, and all my efforts have not reached a successful conclusion. Now with further trys, it seems to me that stuff has been written onto the drives that needs to be wiped off because I get messages that from the disk utility in xfce4 that it won't overwrite a disk with data on it. So I want to use dd to wipe a complete drive. For this I have found the following: # dd if=/dev/zero of=/dev/sdg bs=1M In the above, I have already changed the HD device to sdg (from sda), but I wonder about bs=1M. Could the process go faster with a larger block size? What are the criteria for choosing a value for bs? And, how long should a 3T wipe take to complete? The job has been running for about 12 hours. Would it go faster with a different bs? Faster enough to make the waste of 12hrs running worthwhile? Is there some way to invoke an 'progress indicator' for dd? And, in general, is there a better way? As people have already answered all your above questions, I'll just add to the last one. Install and use dcfldd. It uses the same syntax as dd, but with some extra security and progress features. If you add sizeprobe=of to the command list, it will use the size of the destination drive to show a regular progress report of MB transferred and %age complete. The only issue I've found with it is that it sometimes doesn't report 100% complete if the reporting interval isn't an exact fraction of the total size. It does still write the whole file though. -- Dom -- To UNSUBSCRIBE, email to debian-user-requ...@lists.debian.org with a subject of unsubscribe. Trouble? Contact listmas...@lists.debian.org Archive: http://lists.debian.org/51c87c34.3090...@rpdom.net
