Low NTFS read performance
I can't get an Ultrium-2 LTO drive to stream, and (I think) I've traced the problem to the read performance of the USB2-attached NTFS disk, and specifically the NTFS filesystem. I'm reading a single 190GB file, and the throughput I'm getting is 5.4MB/s: $ dd if=ad2c.dump of=/dev/null bs=1M ^Τ load: 0.04 cmd: dd 1434 [biord] 0.00u 4.78s 2% 1672k 610+0 records in 610+0 records out 639631360 bytes transferred in 117.937613 secs (5423472 bytes/sec) The is an old but relatively fast machine CPU: Intel(R) Pentium(R) 4 CPU 2.40GHz (2407.18-MHz 686-class CPU) running 7.0-RC1: $ uname -a FreeBSD icarian.dmst.aueb.gr 7.0-RC1 FreeBSD 7.0-RC1 #0: Mon Dec 24 12:18:24 UTC 2007 [EMAIL PROTECTED]:/usr/obj/usr/src/sys/GENERIC i386 and the load during the read remains comfortably low: $ uptime 6:34PM up 3:50, 3 users, load averages: 0.09, 0.04, 0.04 Reading from the raw device tripples the performance: # dd if=/dev/da0s1 bs=1m of=/dev/null 533725184 bytes transferred in 34.777460 secs (15346871 bytes/sec) bringing it on par with what I get from Windows (on a different machine): F:\>dd if=other.20051007.tgz of=/dev/null bs=1M 1231030919 bytes (1.2 GB) copied, 82.845 s, 14.9 MB/s These are some (vaguely) relevant parts of dmesg: ehci0: [GIANT-LOCKED] ehci0: [ITHREAD] usb2: EHCI version 0.95 usb2: companion controllers, 3 ports each: usb0 usb1 usb2: on ehci0 usb2: USB revision 2.0 uhub2: on usb2 uhub2: 5 ports with 5 removable, self powered [...] ad0: 76319MB at ata0-master UDMA100 acd0: DVDROM at ata1-master UDMA33 acd1: DVDR at ata1-slave UDMA33 da0 at umass-sim0 bus 0 target 0 lun 0 da0: Fixed Direct Access SCSI-0 device da0: 40.000MB/s transfers da0: 305245MB (625142448 512 byte sectors: 255H 63S/T 38913C) GEOM_LABEL: Label for provider da0s1 is ntfs/Backup. Trying to mount root from ufs:/dev/ad0s2a GEOM_LABEL: Label ntfs/Backup removed. GEOM_LABEL: Label for provider da0s1 is ntfs/Backup. I'd appreciate any suggestions you may have. Diomidis Spinellis - http://www.spinellis.gr ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: scp -c none (was Re: NFS client slow on amd64 6.2-PRERELEASE #2)
Hans Lambermont wrote:
Diomidis Spinellis wrote:
You can also use ports/net/socketpipe. For example you can copy a
directory with:
socketpipe -b -i { tar cf - directory } -l { ssh remotehost } -r
{ tar xvf - }
Just curious, what is the advantage of this approach to the following :
tar cf - directory | ( ssh remotehost 'cd targetdir && tar xpf -' )
Something with buffering perhaps ?
The advantage is a performance gain, especially on slower machines. You
don't encrypt/decrypt the data, and you don't have other processes
read/writing the data. However, because the data is not encrypted, and
can therefore be read and modified on transfer, the approach is only
suitable for closed, secure networks.
With "tar cf - dir | ssh tar xf -" your data travels as follows:
tar sshsshd tar
|^ |^ |^
|| || ||
v| v| v|
++ +-+
|local kernel|>|remote kernel|
++ +-+
With socketpipe, after the initial plumbing, which is performed using
ssh, the data travels as follows:
tar tar
| ^
| |
v |
++ +-+
|local kernel|>|remote kernel|
++ +-+
The two tar processes directly write to and read from a network socket
(hence the program's name).
Diomidis - dds@ - http://www.spinellis.gr
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: scp -c none (was Re: NFS client slow on amd64 6.2-PRERELEASE #2)
Oliver Fromme wrote:
Jeremy Chadwick <[EMAIL PROTECTED]> wrote:
> [...]
> It's really too bad the OpenBSD guys refuse to
> incorporate the HP (high-performance) patches into OpenSSH, and
> being able to say "-c none" would *really* help when it comes to
> benchmarking network I/O via scp
Here's a patch for FreeBSD:
http://www.secnetix.de/~olli/FreeBSD/openssh-cipher-none.patch
Go to /usr/src/crypto/openssh, then apply the patch and
rebuild libssh, ssh and sshd. Then you can use "-c none".
I use "scp -c none" a lot within my internal network to
transfer files between slow boxes. Encryption isn't really
required there, but I can still use all of ssh'd features
such as .ssh/authorized_keys, aliases via .ssh/config etc.
I considered submitting the patch for official inclusion,
but the OpenSSH people would reject it because they call
it "insecure", and the FreeBSD people would reject it
because they say the patch should be submitted to the
OpenSSH people. *sigh* :-(
You can also use ports/net/socketpipe. For example you can copy a
directory with:
socketpipe -b -i { tar cf - directory } -l { ssh remotehost } -r
{ tar xvf - }
I use it for exactly the same purpose: copies between slow machines
within my internal network.
Diomidis - dds@
___
freebsd-stable@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/freebsd-stable
To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: reading process memory
Tofik Suleymanov wrote: Diomidis Spinellis wrote: Tofik Suleymanov wrote: The only way you're going to be able to read another processes address space is in the kernel.Even a process running as root is not able to read another process's data. Incorrect; see this example: $ sed -e 's/this/that/' & [1] 87345 $ /bin/su Password: # dd if=/proc/87345/mem conv=noerror 2> /dev/null | strings [...] @(#)compile.c 8.1 (Berkeley) 6/6/93 [...] RE error: %s RuneMagiNONE /this/that/ "s/this/that/ s/this/that/ this that that I followed instructions in your email, but had no success of getting simmilar results. When trying to read from mem file of particular process i get error messages from dd: (many of this records populate the screen) 0 bytes transferred in 6.393733 secs (0 bytes/sec) dd: /proc/13150/mem: Bad address dd: /proc/13150/mem: Bad address 0+0 records in 0+0 records out 0 bytes transferred in 6.393795 secs (0 bytes/sec) while pid 13510 exists: paranoia# ps ax |grep 13150 13150 p1 T 0:00.00 sed -e s/this/that/g paranoia# man 5 procfs says: mem The complete virtual memory image of the process. Only those address which exist in the process can be accessed. Reads and writes to this file modify the process. Writes to the text seg- ment remain private to the process. map A map of the process' virtual memory. I wonder why i cannot just dd data from mem ? Not all areas of the process's memory are accessible. This is why I set the conv=noerr option to dd (rather than run strings directly on mem), and also redirected the dd's standard error output to /dev/null. Your root's shell (probably tcsh) failed to do that. (Tcsh doesn't offer a way to redirect just the error output). Run sh after the su command to have this facility at your disposal. Diomidis - http://www.spinellis.gr ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: reading process memory
Tofik Suleymanov wrote: The only way you're going to be able to read another processes address space is in the kernel.Even a process running as root is not able to read another process's data. Incorrect; see this example: $ sed -e 's/this/that/' & [1] 87345 $ /bin/su Password: # dd if=/proc/87345/mem conv=noerror 2> /dev/null | strings [...] @(#)compile.c 8.1 (Berkeley) 6/6/93 [...] RE error: %s RuneMagiNONE /this/that/ "s/this/that/ s/this/that/ this that that Diomidis - http://www.spinellis.gr ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
vm_fault on 8GB RAM AMD-64 with 5.4-RC3 and 5.4-STABLE
e 3.0 on pci3 pci0: at device 11.1 (no driver attached) atkbdc0: port 0x64,0x60 irq 1 on acpi0 atkbd0: flags 0x1 irq 1 on atkbdc0 kbd0 at atkbd0 psm0: irq 12 on atkbdc0 psm0: model IntelliMouse, device ID 3 sio0: <16550A-compatible COM port> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0 sio0: type 16550A sio1: <16550A-compatible COM port> port 0x2f8-0x2ff irq 3 on acpi0 sio1: type 16550A fdc0: port 0x3f7,0x3f0-0x3f5 irq 6 drq 2 on acpi0 ppc0: port 0x378-0x37f irq 7 on acpi0 ppc0: Generic chipset (NIBBLE-only) in COMPATIBLE mode ppbus0: on ppc0 plip0: on ppbus0 lpt0: on ppbus0 lpt0: Interrupt-driven port ppi0: on ppbus0 orm0: at iomem 0xcd000-0xce7ff,0xcc800-0xccfff,0xc8000-0xcc7ff,0xc-0xc7fff on isa0 sc0: at flags 0x100 on isa0 sc0: VGA <16 virtual consoles, flags=0x300> vga0: at port 0x3c0-0x3df iomem 0xa-0xb on isa0 Timecounters tick every 1.000 msec acd0: CDROM at ata1-master PIO4 Waiting 15 seconds for SCSI devices to settle da0 at iir0 bus 2 target 0 lun 0 da0: Fixed Direct Access SCSI-2 device da0: Tagged Queueing Enabled da0: 286063MB (585858420 512 byte sectors: 255H 63S/T 36468C) SMP: AP CPU #1 Launched! Mounting root from ufs:/dev/da0s1a -- Diomidis Spinellis - dds@ - http://www.spinellis.gr ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Mounting a powered-down HDD renders system unusable
Mike Harding wrote: On a 5.4-pre system as of the last few days, If I mount a spun down hard disk (one I am using only for backup): This does not answer your question, but may help you solve the problem. Try detaching the disk after you are done using it, and attaching it again before you mount it. I use this procedure for my backup disk. It has the advantage of allowing me to physically isolate the disk from the computer. The disk lives on a detachable disk drawer-style cabinet; by pulling the disk drawer out an inch I severe the electrical connections between the disk and the computer. The idea is that even if the computer gets electrically fried by lightning, the disk will survive. Thus, the procedure I use is: atacontrol attach 0 mount /backup # Perform backup umount /backup atacontrol detach 0 -- Diomidis Spinellis - http://www.spinellis.gr ___ freebsd-stable@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
Re: Different snapshot sizes.
David Gilbert wrote: > and progressively delete them such that the oldest > snapshot we keep is about 30 days old. Off topic comment: consider using ports/sysutils/fileprune for this task. Diomidis Spinellis ___ [EMAIL PROTECTED] mailing list http://lists.freebsd.org/mailman/listinfo/freebsd-stable To unsubscribe, send any mail to "[EMAIL PROTECTED]"
