Re: [gentoo-user] usb1 performance out of usb2 external HD
Actually, that's a good point. All that writing to the FAT would use those flash blocks a lot, which would indeed wear it out. I trashed a Crucial 1GB drive in a few weeks, until I found out about this change. I think this problem should find its way in the official Gentoo docs (if it isn't already there). It affects all recent Linux distros anyway. It's quite crucial -trashing usb flash drives is NOT good marketing for our beloved penguin. Anyone knows if on the LKML the devs gave some opinion about this side effect? m. -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] usb1 performance out of usb2 external HD
On Mon, 16 Jan 2006 20:06:50 -0700, Trenton Adams wrote: > Actually, that's a good point. All that writing to the FAT would use > those flash blocks a lot, which would indeed wear it out. I trashed a Crucial 1GB drive in a few weeks, until I found out about this change. > Windows does sync by default as well. But, it's sync is extremely > fast for some reason. It's not whether it does sync be default, that is down you your (auto)mounter, nothing to do with the kernel. The change is that in sync mode, the kernel used to update the FAT after each file, now it does it after each block. That's why it is so slow and destructive. -- Neil Bothwick Top Oxymorons Number 44: Advanced BASIC signature.asc Description: PGP signature
Re: [gentoo-user] usb1 performance out of usb2 external HD
On 1/16/06, b.n. <[EMAIL PROTECTED]> wrote: > > > I'm wondering if we should be reporting this to the kernel guys. I > > see this on *all* linux systems with kernel 2.6, but I'm not sure > > about 2.4. But, I think I'll try enabling some options that Richard > > mentioned first. > > Kernel guys know it very well. They recently implemented the sync option > for FAT filesystems, and this is turned on by default. > > Unluckly, sync is really OK for FAT partitions on an HD, but not for USB > devices. I hope they will realize soon that not only it degrades > performance of USB drives, but it also wears out flash memories really fast. Actually, that's a good point. All that writing to the FAT would use those flash blocks a lot, which would indeed wear it out. Windows does sync by default as well. But, it's sync is extremely fast for some reason. > > m. > -- > gentoo-user@gentoo.org mailing list > > -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] usb1 performance out of usb2 external HD
I'm wondering if we should be reporting this to the kernel guys. I see this on *all* linux systems with kernel 2.6, but I'm not sure about 2.4. But, I think I'll try enabling some options that Richard mentioned first. Kernel guys know it very well. They recently implemented the sync option for FAT filesystems, and this is turned on by default. Unluckly, sync is really OK for FAT partitions on an HD, but not for USB devices. I hope they will realize soon that not only it degrades performance of USB drives, but it also wears out flash memories really fast. m. -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] usb1 performance out of usb2 external HD
On 1/16/06, Iain Buchanan <[EMAIL PROTECTED]> wrote:
> On Mon, 2006-01-16 at 10:46 -0700, Trenton Adams wrote:
> > Are you mounting with the "sync" option, or no? Try it with, and
> > without. If you do it without, time what it takes to do the copy, and
> > unmount. unmount will do the actual sync.
>
> hey, that sped it up heaps!
>
> $ sudo mount -t vfat -o uid=iain,gid=users,async /dev/sdd6 /mnt/tmp/
> $ time { cp /usr/portage/distfiles/OOo_2.0.1_LinuxIntel_install.tar.gz
> /mnt/tmp/; sudo umount /mnt/tmp/; }
>
> real0m11.134s
>
> thats a huge increase (11s instead of 81s!), so how come hdparm still
> reports 9MB/s?
I'm wondering if we should be reporting this to the kernel guys. I
see this on *all* linux systems with kernel 2.6, but I'm not sure
about 2.4. But, I think I'll try enabling some options that Richard
mentioned first.
--
gentoo-user@gentoo.org mailing list
Re: [gentoo-user] usb1 performance out of usb2 external HD
On 1/16/06, Iain Buchanan <[EMAIL PROTECTED]> wrote:
> On Mon, 2006-01-16 at 10:46 -0700, Trenton Adams wrote:
> > Are you mounting with the "sync" option, or no? Try it with, and
> > without. If you do it without, time what it takes to do the copy, and
> > unmount. unmount will do the actual sync.
>
> hey, that sped it up heaps!
>
> $ sudo mount -t vfat -o uid=iain,gid=users,async /dev/sdd6 /mnt/tmp/
> $ time { cp /usr/portage/distfiles/OOo_2.0.1_LinuxIntel_install.tar.gz
> /mnt/tmp/; sudo umount /mnt/tmp/; }
>
> real0m11.134s
>
> thats a huge increase (11s instead of 81s!), so how come hdparm still
> reports 9MB/s?
>
> > I've noticed that linux in generall is extremely slow with USB devices
> > sometimes. My external USB2 (ATA 100) drive gets 27M/sec on my
> > machine. And with a relatively new driver, you should see speeds
> > about like that.
>
> Is that calculating the times yourself, or with hdparm?
Yes, calculating myself. And I don't know why hdparm would report
that other time.
>
> thanks,
> --
> Iain Buchanan
>
> I've had a perfectly wonderful evening. But this wasn't it.
> -- Groucho Marx
>
> --
> gentoo-user@gentoo.org mailing list
>
>
--
gentoo-user@gentoo.org mailing list
Re: [gentoo-user] usb1 performance out of usb2 external HD
On Mon, 2006-01-16 at 10:46 -0700, Trenton Adams wrote:
> Are you mounting with the "sync" option, or no? Try it with, and
> without. If you do it without, time what it takes to do the copy, and
> unmount. unmount will do the actual sync.
hey, that sped it up heaps!
$ sudo mount -t vfat -o uid=iain,gid=users,async /dev/sdd6 /mnt/tmp/
$ time { cp /usr/portage/distfiles/OOo_2.0.1_LinuxIntel_install.tar.gz
/mnt/tmp/; sudo umount /mnt/tmp/; }
real0m11.134s
thats a huge increase (11s instead of 81s!), so how come hdparm still
reports 9MB/s?
> I've noticed that linux in generall is extremely slow with USB devices
> sometimes. My external USB2 (ATA 100) drive gets 27M/sec on my
> machine. And with a relatively new driver, you should see speeds
> about like that.
Is that calculating the times yourself, or with hdparm?
thanks,
--
Iain Buchanan
I've had a perfectly wonderful evening. But this wasn't it.
-- Groucho Marx
--
gentoo-user@gentoo.org mailing list
Re: [gentoo-user] usb1 performance out of usb2 external HD
On Mon, 2006-01-16 at 07:21 -0700, Richard Fish wrote: > On 1/16/06, Iain Buchanan <[EMAIL PROTECTED]> wrote: > > On Mon, 2006-01-16 at 06:40 -0700, Richard Fish wrote: > > > On 1/15/06, Iain Buchanan <[EMAIL PROTECTED]> wrote: > > > > > > > > 9.6 is pretty slow! I should be able to get at least double that... > > > > actually, I should be able to get 4x that... > > No, you won't. The max I've ever seen any USB drive run at is about > 30MB/sec, even when the same drive installed internally will run at > 65MB/sec. My best 2.5" case and drive will pump about 25MB/sec. well, I was estimating, and I am expecting about 30+MB/s given a friend gets the same with the same drive. > Ok, I've never used one of these combination devices. I'm a little > concerned, because the maximum throughput of most media readers is > about 10MB/sec...so hoping this is not a limitation of the chipset. nope, windows does the transfer in about 10/15 seconds, (I can time it exactly if you're interested) making the speed (conservatively) about 7Mbytes/sec. > > relevant dmesg: > > Looks normal... > > > sdd: sdd1 sdd2 sdd3 sdd4 < sdd5 sdd6 sdd7 sdd8 sdd9 sdd10 sdd11 > > > Damn, and I thought I made a lot of volumes ;-> yeah, its the way it has to be unfortunately. This will eventually become my internal HD, so I need it to dual boot, plus I added a few more partitions for linux (/ /boot /usr /home) plus a couple for windows - one for play and one for work... anyway. > > $ zgrep USB /proc/config.gz | grep -v "^#" > I think you need to turn on some of the options under USB Mass Storage > support. Particularly > > CONFIG_USB_STORAGE_FREECOM=y > > but I don't think there is any harm in turning all of them on...it is > very likely that one of these will give you the best performance. ok I'll try, thanks. -- Iain Buchanan Superior ability breeds superior ambition. -- Spock, "Space Seed", stardate 3141.9 -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] usb1 performance out of usb2 external HD
Are you mounting with the "sync" option, or no? Try it with, and without. If you do it without, time what it takes to do the copy, and unmount. unmount will do the actual sync. I've noticed that linux in generall is extremely slow with USB devices sometimes. My external USB2 (ATA 100) drive gets 27M/sec on my machine. And with a relatively new driver, you should see speeds about like that. On 1/15/06, Iain Buchanan <[EMAIL PROTECTED]> wrote: > Hi all, > > I have a 2.5in usb HD and 2 external usb2 cases. I've been getting > pretty average performance with a 7200 rpm drive in both cases, but I > just thought this was the way it is. > > However, I just did a test: > $ time > cp /usr/portage/distfiles/OOo_2.0.1_LinuxIntel_install.tar.gz > /mnt/usb-storage/device-0/partition-5/ > > real1m20.903s > > which is pretty slow. I copied the same file in winblows (without > changing usb ports or anything) and it copies in about 15 seconds. > > I noticed I had uhci-hcd loaded (the usb 1 module?) so I unloaded it, > and my keyboard stopped working!!! So using the laptop keyboard, I > replugged the HD (now without uhci-hcd) and it still goes slow... > > $ sudo hdparm -tT /dev/sdd > > /dev/sdd: > Timing cached reads: 3640 MB in 2.00 seconds = 1820.28 MB/sec > Timing buffered disk reads: 30 MB in 3.13 seconds = 9.60 MB/sec > > 9.6 is pretty slow! I should be able to get at least double that... > > And as I've said, I know this port is usb 2.0. > > Any hints? > thanks, > -- > Iain Buchanan > > IN MY OPINION anyone interested in improving himself should not rule out > becoming pure energy. > -- Jack Handley, The New Mexican, 1988. > > -- > gentoo-user@gentoo.org mailing list > > -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] usb1 performance out of usb2 external HD
On 1/16/06, Iain Buchanan <[EMAIL PROTECTED]> wrote: > On Mon, 2006-01-16 at 06:40 -0700, Richard Fish wrote: > > On 1/15/06, Iain Buchanan <[EMAIL PROTECTED]> wrote: > > > Hi all, > > > $ sudo hdparm -tT /dev/sdd > > > > > > /dev/sdd: > > > Timing cached reads: 3640 MB in 2.00 seconds = 1820.28 MB/sec > > > Timing buffered disk reads: 30 MB in 3.13 seconds = 9.60 MB/sec > > > > > > 9.6 is pretty slow! I should be able to get at least double that... > > actually, I should be able to get 4x that... No, you won't. The max I've ever seen any USB drive run at is about 30MB/sec, even when the same drive installed internally will run at 65MB/sec. My best 2.5" case and drive will pump about 25MB/sec. > > Please post the dmesg output from plugging in the drive. > > Note, there are 4 LUNS on this device, the first 3 are card readers ,so > they show with no media (no drive or partitions), the fourth (sdd) is > the ide drive. Ok, I've never used one of these combination devices. I'm a little concerned, because the maximum throughput of most media readers is about 10MB/sec...so hoping this is not a limitation of the chipset. > relevant dmesg: Looks normal... > sdd: sdd1 sdd2 sdd3 sdd4 < sdd5 sdd6 sdd7 sdd8 sdd9 sdd10 sdd11 > Damn, and I thought I made a lot of volumes ;-> > $ zgrep USB /proc/config.gz | grep -v "^#" > CONFIG_USB_IRDA=m > CONFIG_BT_HCIUSB=m > CONFIG_BT_HCIUSB_SCO=y > CONFIG_BT_HCIBFUSB=m > CONFIG_USB_ARCH_HAS_HCD=y > CONFIG_USB_ARCH_HAS_OHCI=y > CONFIG_USB=m > CONFIG_USB_DEVICEFS=y > CONFIG_USB_EHCI_HCD=m > CONFIG_USB_UHCI_HCD=m > CONFIG_USB_PRINTER=m > CONFIG_USB_STORAGE=m I think you need to turn on some of the options under USB Mass Storage support. Particularly CONFIG_USB_STORAGE_FREECOM=y but I don't think there is any harm in turning all of them on...it is very likely that one of these will give you the best performance. -Richard -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] usb1 performance out of usb2 external HD
On Mon, 2006-01-16 at 06:40 -0700, Richard Fish wrote: > On 1/15/06, Iain Buchanan <[EMAIL PROTECTED]> wrote: > > Hi all, > > $ sudo hdparm -tT /dev/sdd > > > > /dev/sdd: > > Timing cached reads: 3640 MB in 2.00 seconds = 1820.28 MB/sec > > Timing buffered disk reads: 30 MB in 3.13 seconds = 9.60 MB/sec > > > > 9.6 is pretty slow! I should be able to get at least double that... actually, I should be able to get 4x that... > Please post the dmesg output from plugging in the drive. Note, there are 4 LUNS on this device, the first 3 are card readers ,so they show with no media (no drive or partitions), the fourth (sdd) is the ide drive. relevant dmesg: usb 5-4: new high speed USB device using ehci_hcd and address 6 scsi2 : SCSI emulation for USB Mass Storage devices usb-storage: device found at 6 usb-storage: waiting for device to settle before scanning Vendor: Model: VP6230Rev: 1.07 Type: Direct-Access ANSI SCSI revision: 00 sd 2:0:0:0: Attached scsi removable disk sda sd 2:0:0:0: Attached scsi generic sg0 type 0 Vendor: Model: VP6230Rev: 1.07 Type: Direct-Access ANSI SCSI revision: 00 sd 2:0:0:1: Attached scsi removable disk sdb sd 2:0:0:1: Attached scsi generic sg1 type 0 Vendor: Model: VP6230Rev: 1.07 Type: Direct-Access ANSI SCSI revision: 00 sd 2:0:0:2: Attached scsi removable disk sdc sd 2:0:0:2: Attached scsi generic sg2 type 0 Vendor: Model: VP6230Rev: 1.07 Type: Direct-Access ANSI SCSI revision: 00 SCSI device sdd: 195371568 512-byte hdwr sectors (100030 MB) sdd: assuming drive cache: write through SCSI device sdd: 195371568 512-byte hdwr sectors (100030 MB) sdd: assuming drive cache: write through sdd: sdd1 sdd2 sdd3 sdd4 < sdd5 sdd6 sdd7 sdd8 sdd9 sdd10 sdd11 > sd 2:0:0:3: Attached scsi disk sdd sd 2:0:0:3: Attached scsi generic sg3 type 0 usb-storage: device scan complete kjournald starting. Commit interval 5 seconds EXT3 FS on sdd1, internal journal EXT3-fs: recovery complete. EXT3-fs: mounted filesystem with ordered data mode. NTFS volume version 3.1. kjournald starting. Commit interval 5 seconds EXT3 FS on sdd3, internal journal EXT3-fs: recovery complete. EXT3-fs: mounted filesystem with ordered data mode. NTFS volume version 3.1. kjournald starting. Commit interval 5 seconds EXT3 FS on sdd7, internal journal EXT3-fs: recovery complete. EXT3-fs: mounted filesystem with ordered data mode. kjournald starting. Commit interval 5 seconds EXT3 FS on sdd8, internal journal EXT3-fs: recovery complete. EXT3-fs: mounted filesystem with ordered data mode. kjournald starting. Commit interval 5 seconds EXT3 FS on sdd9, internal journal EXT3-fs: recovery complete. EXT3-fs: mounted filesystem with ordered data mode. kjournald starting. Commit interval 5 seconds EXT3 FS on sdd10, internal journal EXT3-fs: recovery complete. EXT3-fs: mounted filesystem with ordered data mode. kjournald starting. Commit interval 5 seconds EXT3 FS on sdd11, internal journal EXT3-fs: recovery complete. EXT3-fs: mounted filesystem with ordered data mode. > Also the > output of "zgrep USB /proc/config.gz | grep -v "^#" (or use grep & > /usr/src/linux/.config if you don't have /proc/config.gz). $ zgrep USB /proc/config.gz | grep -v "^#" CONFIG_USB_IRDA=m CONFIG_BT_HCIUSB=m CONFIG_BT_HCIUSB_SCO=y CONFIG_BT_HCIBFUSB=m CONFIG_USB_ARCH_HAS_HCD=y CONFIG_USB_ARCH_HAS_OHCI=y CONFIG_USB=m CONFIG_USB_DEVICEFS=y CONFIG_USB_EHCI_HCD=m CONFIG_USB_UHCI_HCD=m CONFIG_USB_PRINTER=m CONFIG_USB_STORAGE=m CONFIG_USB_HID=m CONFIG_USB_HIDINPUT=y CONFIG_USB_HIDDEV=y CONFIG_USB_PEGASUS=m CONFIG_USB_SERIAL=m CONFIG_USB_SERIAL_GENERIC=y CONFIG_USB_SERIAL_PL2303=m CONFIG_USB_SISUSBVGA=m thanks for the help, Iain. -- Iain Buchanan (Never thought I'd be telling Malcolm and Ilya the same thing... :-) -- Larry Wall in <[EMAIL PROTECTED]> -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] usb1 performance out of usb2 external HD
On 1/15/06, Iain Buchanan <[EMAIL PROTECTED]> wrote: > Hi all, > $ sudo hdparm -tT /dev/sdd > > /dev/sdd: > Timing cached reads: 3640 MB in 2.00 seconds = 1820.28 MB/sec > Timing buffered disk reads: 30 MB in 3.13 seconds = 9.60 MB/sec > > 9.6 is pretty slow! I should be able to get at least double that... Please post the dmesg output from plugging in the drive. Also the output of "zgrep USB /proc/config.gz | grep -v "^#" (or use grep & /usr/src/linux/.config if you don't have /proc/config.gz). > And as I've said, I know this port is usb 2.0. Yes, true 1.1 speed would be about 1MB/sec. -Richard -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] usb1 performance out of usb2 external HD
On Mon, 2006-01-16 at 02:17 -0500, Willie Wong wrote: > On Mon, Jan 16, 2006 at 04:08:58PM +0930, Penguin Lover Iain Buchanan > squawked: > > I noticed I had uhci-hcd loaded (the usb 1 module?) so I unloaded it, > > and my keyboard stopped working!!! So using the laptop keyboard, I > > replugged the HD (now without uhci-hcd) and it still goes slow... > > uhci-hcd is the driver for the intel USB host controller, so if you > unload that, it is natural that it killed your (presumeably USB) > keyboard. > > the driver for USB2.0, incidentally, should be ehci-hcd, and according > to the kernel docs, if you have ehci built (grep USB_EHCI > /usr/src/linux/.config), it will use USB2.0 on enabled devices. > > So, the question is, did you compile your kernel with EHCI support? Yes, I should have mentioned that: I'm using ehci. I unloaded uhci. I'm not using the old slow usb-block-device driver either. thanks, -- Iain Buchanan Try not. Do. Or do not. There is no try. -- gentoo-user@gentoo.org mailing list
Re: [gentoo-user] usb1 performance out of usb2 external HD
On Mon, Jan 16, 2006 at 04:08:58PM +0930, Penguin Lover Iain Buchanan squawked: > I noticed I had uhci-hcd loaded (the usb 1 module?) so I unloaded it, > and my keyboard stopped working!!! So using the laptop keyboard, I > replugged the HD (now without uhci-hcd) and it still goes slow... > uhci-hcd is the driver for the intel USB host controller, so if you unload that, it is natural that it killed your (presumeably USB) keyboard. the driver for USB2.0, incidentally, should be ehci-hcd, and according to the kernel docs, if you have ehci built (grep USB_EHCI /usr/src/linux/.config), it will use USB2.0 on enabled devices. So, the question is, did you compile your kernel with EHCI support? HTH W -- An engineer is walking along and runs across a talking frog. The frog says "I am a princess, and if you kiss me, I will turn back into a princess and be your girlfriend." Then engineer mutters "Neat", picks up the frog, puts it in his back pocket, and continues walking. The frog kicked the engineer, prompting him to pull the frog out. The frog said, "My daddy is rich, and if you kiss me, he will reward you handsomely." The engineer shrugged his shoulders and replaced the frog in his pocket. A couple minutes later, the frog kicked him again. This time the frog asked, "Why won't you kiss me? I will make you rich and be your girlfriend." The engineer replied "Look, lady. I am an engineer and have no use for a girlfriend. But a talking frog is cool!" Sortir en Pantoufles: up 64 days, 23:34 -- gentoo-user@gentoo.org mailing list
