Re: write alignment matters?

2021-11-08 Thread Reinoud Zandijk
On Fri, Jun 24, 2011 at 04:39:13PM +, David Holland wrote: > On Fri, Jun 24, 2011 at 11:33:51AM -0400, der Mouse wrote: > > >> Oh, I was talking about current NetBSD where block devices are a > > >> second class citizen, soon to be abolished if someone finds enough > > >> round tuits. > >

Re: write alignment matters?

2011-06-28 Thread David Laight
On Mon, Jun 27, 2011 at 05:55:50PM +, David Holland wrote: RT-11 did not require explictly unmounting floppies before changing them. IIRC RT-11 fs could go wrong (docs suggested using TECO to read the raw disk and recover the file data from the adjacent sectors that contained it) I don't

Re: write alignment matters?

2011-06-28 Thread Paul Koning
On Jun 28, 2011, at 12:53 PM, David Laight wrote: On Mon, Jun 27, 2011 at 05:55:50PM +, David Holland wrote: RT-11 did not require explictly unmounting floppies before changing them. IIRC RT-11 fs could go wrong (docs suggested using TECO to read the raw disk and recover the file data

Re: write alignment matters?

2011-06-27 Thread Eduardo Horvath
On Sat, 25 Jun 2011, der Mouse wrote: That what it is reasonable for a disk to do consensus *is* the interface spec I was talking about, not the de-jure non-spec of you get whatever the device (via its driver) feels like giving you. That's sort of the point. If you want what it is reasonable

Re: write alignment matters?

2011-06-27 Thread der Mouse
That what it is reasonable for a disk to do consensus *is* the interface spec I was talking about, not the de-jure non-spec of you get whatever the device (via its driver) feels like giving you. That's sort of the point. If you want what it is reasonable for a disk to do you should be using

Re: write alignment matters?

2011-06-27 Thread David Holland
On Sat, Jun 25, 2011 at 09:29:57PM +0700, Robert Elz wrote: | At least for NetBSD, that's never been true. The most glaring | problem is that there's no protection against causing the same | underlying disk blocks to be multiply cached by accessing the | buffer cache with a different

Re: write alignment matters?

2011-06-27 Thread David Holland
On Sat, Jun 25, 2011 at 08:57:30PM +0200, Johnny Billquist wrote: I might be confused here. I thought that if you accessed the block device, you were restricted to blocks. So you can in fact not seek to an arbitrary byte, nor read an arbitrary length, like for a normal file, but instead

Re: write alignment matters?

2011-06-27 Thread Erik Fair
On Jun 27, 2011, at 10:59 , David Holland wrote: On Sat, Jun 25, 2011 at 08:57:30PM +0200, Johnny Billquist wrote: I might be confused here. I thought that if you accessed the block device, you were restricted to blocks. So you can in fact not seek to an arbitrary byte, nor read an arbitrary

Re: write alignment matters?

2011-06-27 Thread Dennis Ferguson
On 27 Jun 2011, at 10:27 , der Mouse wrote: That what it is reasonable for a disk to do consensus *is* the interface spec I was talking about, not the de-jure non-spec of you get whatever the device (via its driver) feels like giving you. That's sort of the point. If you want what it is

Re: write alignment matters?

2011-06-27 Thread Paul Koning
On Jun 27, 2011, at 1:55 PM, David Holland wrote: ... It took the invention of that paragon of OS's - DOS - to teach the populace that simply pulling the device/media was an acceptable operating procedure. That's hardly fair. Until the Mac appeared in 1984 every small computer that had

Re: write alignment matters?

2011-06-27 Thread Johnny Billquist
On 06/27/11 21:02, Dennis Ferguson wrote: On 27 Jun 2011, at 10:27 , der Mouse wrote: That what it is reasonable for a disk to do consensus *is* the interface spec I was talking about, not the de-jure non-spec of you get whatever the device (via its driver) feels like giving you. That's

Re: write alignment matters?

2011-06-25 Thread Robert Elz
Date:Fri, 24 Jun 2011 20:26:19 -0400 (EDT) From:der Mouse mo...@rodents-montreal.org Message-ID: 201106250026.uaa12...@sparkle.rodents-montreal.org | This would mean that raw devices as interfaces to disks are essentially | useless. Not at all, as history has

Accessing raw devices (was: write alignment matters?)

2011-06-25 Thread Edgar Fuß
the code would need to determine the requirements and adapt to the particular device it was being used on now But how would you achieve that? Given your analogy to accessing terminals, you would need an equivalent of termcap. Or, more reasonably, ioctls telling you the constraints (alignment,

Re: write alignment matters?

2011-06-25 Thread Johnny Billquist
On 2011-06-25 16:29, Robert Elz wrote: Date:Fri, 24 Jun 2011 20:26:19 -0400 (EDT) From:der Mousemo...@rodents-montreal.org Message-ID:201106250026.uaa12...@sparkle.rodents-montreal.org mo...@rodents-montreal.org said: | Yes, so it keeps being said. It would

Re: Accessing raw devices (was: write alignment matters?)

2011-06-25 Thread Robert Elz
Date:Sat, 25 Jun 2011 17:18:59 +0200 From:=?iso-8859-1?Q?Edgar_Fu=DF?= e...@math.uni-bonn.de Message-ID: 2f7d0c42-31b8-49e6-9f6d-361459851...@math.uni-bonn.de | Given your analogy to accessing terminals, you would need an equivalent | of termcap. Or, more

Re: write alignment matters?

2011-06-25 Thread Paul Koning
On Jun 25, 2011, at 10:29 AM, Robert Elz wrote: ... I suppose that's true in theory, just as we don't know that the next such device might not have 739 byte sectors, or rotate backwards, or ... However, in the real world, the manufacturers don't make products that they can't sell,

Re: write alignment matters?

2011-06-25 Thread der Mouse
This would mean that raw devices as interfaces to disks are essentially useless. Not at all, as history has proven, as that's what the rule has always been. Only because [I]n the real world, the manufacturers don't make products that they can't sell, and people don't buy products that

Re: write alignment matters?

2011-06-24 Thread Michael van Elst
s...@cs.columbia.edu (Steven Bellovin) writes: The point is that when dealing with raw devices, you take what the hardware gives you. 6th Edition could have detected this and copied the user data into a properly-aligned buffer, with the corresponding performance hit. Instead, it said this is the

Re: write alignment matters?

2011-06-24 Thread Erik Fair
On Jun 23, 2011, at 23:19 , Michael van Elst wrote: s...@cs.columbia.edu (Steven Bellovin) writes: The point is that when dealing with raw devices, you take what the hardware gives you. 6th Edition could have detected this and copied the user data into a properly-aligned buffer, with the

Re: write alignment matters?

2011-06-24 Thread Michael van Elst
f...@netbsd.org (Erik Fair) writes: On Jun 23, 2011, at 23:19 , Michael van Elst wrote: s...@cs.columbia.edu (Steven Bellovin) writes: The point is that when dealing with raw devices, you take what the hardware gives you. 6th Edition could have detected this and copied the user data into

Re: write alignment matters?

2011-06-24 Thread Robert Elz
Date:Fri, 24 Jun 2011 00:09:58 -0700 From:Erik Fair f...@netbsd.org Message-ID: d4b3cb6f-ecbc-4db6-97c2-031be0d26...@netbsd.org | Oh, and as noted before, through the block interface, you could do | arbitrary (up to ... was it 64KB? Just under that, the count arg

Re: write alignment matters?

2011-06-24 Thread Thor Lancelot Simon
On Fri, Jun 24, 2011 at 07:50:35AM +, Michael van Elst wrote: Similar to raw devices in performance, I/O caching, and without the hardware constraints regarding block sizes or alignment. I.e. something that lets you treat disk contents like a regular file. At least for NetBSD, that's

Re: write alignment matters?

2011-06-24 Thread David Holland
On Fri, Jun 24, 2011 at 11:33:51AM -0400, der Mouse wrote: Oh, I was talking about current NetBSD where block devices are a second class citizen, soon to be abolished if someone finds enough round tuits. Yes, so it keeps being said. It would truly be a pity to see that happen.

Re: write alignment matters?

2011-06-24 Thread David Laight
On Fri, Jun 24, 2011 at 12:09:58AM -0700, Erik Fair wrote: Of course, if you want to talk about really slow I/O with massive latency, let's talk about random access to DECtapes ... Apparantly it was useful to put a bad block at the end of every tape track so that otherwise sequential data

Re: write alignment matters?

2011-06-24 Thread Michael van Elst
On Fri, Jun 24, 2011 at 12:02:05PM -0400, Thor Lancelot Simon wrote: On Fri, Jun 24, 2011 at 07:50:35AM +, Michael van Elst wrote: Similar to raw devices in performance, I/O caching, and without the hardware constraints regarding block sizes or alignment. I.e. something that lets you

Re: write alignment matters?

2011-06-24 Thread der Mouse
[...], there's a lot of history behind the notion that userland alignment of write() buffers affects, at most, performance, to the point where I consider it part of the interface. Not on access to raw devices it isn't, and never was - what Erik Fair said [...] was 100% correct - if you're

Re: write alignment matters?

2011-06-23 Thread Thor Lancelot Simon
On Thu, Jun 23, 2011 at 03:36:25PM +0700, Robert Elz wrote: Date:Wed, 22 Jun 2011 19:30:55 -0400 (EDT) From:der Mouse mo...@rodents-montreal.org Message-ID: 201106222330.taa28...@sparkle.rodents-montreal.org | But the interface is much older than that, and,

Re: write alignment matters?

2011-06-23 Thread Matt Thomas
On Jun 23, 2011, at 4:41 AM, Thor Lancelot Simon wrote: On Thu, Jun 23, 2011 at 03:36:25PM +0700, Robert Elz wrote: Date:Wed, 22 Jun 2011 19:30:55 -0400 (EDT) From:der Mouse mo...@rodents-montreal.org Message-ID: 201106222330.taa28...@sparkle.rodents-montreal.org

Re: write alignment matters?

2011-06-23 Thread Thor Lancelot Simon
On Thu, Jun 23, 2011 at 06:36:31AM -0700, Matt Thomas wrote: On Jun 23, 2011, at 4:41 AM, Thor Lancelot Simon wrote: On Thu, Jun 23, 2011 at 03:36:25PM +0700, Robert Elz wrote: Date:Wed, 22 Jun 2011 19:30:55 -0400 (EDT) From:der Mouse mo...@rodents-montreal.org

Re: write alignment matters?

2011-06-23 Thread Steven Bellovin
On Jun 23, 2011, at 4:36 25AM, Robert Elz wrote: Date:Wed, 22 Jun 2011 19:30:55 -0400 (EDT) From:der Mouse mo...@rodents-montreal.org Message-ID: 201106222330.taa28...@sparkle.rodents-montreal.org | But the interface is much older than that, and, even if it's not

Re: write alignment matters?

2011-06-23 Thread Johnny Billquist
On 2011-06-23 23:05, Steven Bellovin wrote: On Jun 23, 2011, at 4:36 25AM, Robert Elz wrote: Date:Wed, 22 Jun 2011 19:30:55 -0400 (EDT) From:der Mousemo...@rodents-montreal.org Message-ID:201106222330.taa28...@sparkle.rodents-montreal.org | But the interface is

Re: write alignment matters?

2011-06-23 Thread David Holland
On Fri, Jun 24, 2011 at 01:43:34AM +0200, Johnny Billquist wrote: In other words, Erik is right, at least if we're talking historically. Of course, at least there it's documented. (I took a quick glance at the code, too -- it did appear to check for erroneous parameters, though I think it

Re: write alignment matters?

2011-06-23 Thread Steven Bellovin
On Jun 23, 2011, at 7:43 34PM, Johnny Billquist wrote: On 2011-06-23 23:05, Steven Bellovin wrote: On Jun 23, 2011, at 4:36 25AM, Robert Elz wrote: Date:Wed, 22 Jun 2011 19:30:55 -0400 (EDT) From:der Mousemo...@rodents-montreal.org

Re: write alignment matters?

2011-06-23 Thread Johnny Billquist
On 2011-06-24 02:15, Steven Bellovin wrote: On Jun 23, 2011, at 7:43 34PM, Johnny Billquist wrote: On 2011-06-23 23:05, Steven Bellovin wrote: On Jun 23, 2011, at 4:36 25AM, Robert Elz wrote: Date:Wed, 22 Jun 2011 19:30:55 -0400 (EDT) From:der

Re: write alignment matters?

2011-06-22 Thread Manuel Bouyer
On Tue, Jun 21, 2011 at 08:57:33PM -0600, Michael L. Hitch wrote: On Tue, 21 Jun 2011, der Mouse wrote: It's a 53c8xx: we load the firmware into it from our own driver. I just had a look at esiop.ss, and it looks high-level enough that it's plausible to me that the problem is with

Re: write alignment matters?

2011-06-22 Thread Erik Fair
On Jun 22, 2011, at 08:28 , der Mouse wrote: The issue for me is not that the hardware does or doesn't have alignment restrictions. It's that they show through to userland (and in a very peculiar way). As someone mentioned upthread, it's possible what's going on is that this hardware has

Re: write alignment matters?

2011-06-22 Thread Manuel Bouyer
On Wed, Jun 22, 2011 at 11:28:37AM -0400, der Mouse wrote: [...disk buffer alignment issues...] I suspect most disk controllers will have issues if the buffer is not aligned on at [least] 16bits. Perhaps. In such cases, something else in the I/O stack - the device driver, most likely,

write alignment matters?

2011-06-21 Thread der Mouse
I've just run into something (on 4.0.1) which looks to me as though write() buffer alignment matters. This sounds to me like a bug, but it appears to have something to do with the hardware, and I'd appreciate any thoughts on how I might best track it down. I tried to run one of my tools (a disk

Re: write alignment matters?

2011-06-21 Thread Eduardo Horvath
On Tue, 21 Jun 2011, der Mouse wrote: It does, however, appear to have something to do with the hardware (personally, I suspect the disk driver); if I try it on another 4.0.1 machine on vnd0d backed by an ordinary file, it doesn't misbehave, and if I try it on a real disk partition on that

Re: write alignment matters?

2011-06-21 Thread der Mouse
It does, however, appear to have something to do with the hardware [...] It's most likely the controller hardware (or firmware). The driver doesn't seem at all interested in the data alignment, and I don't recall other parts of the I/O stack caring. That sounds reasonable. To verify you

Re: write alignment matters?

2011-06-21 Thread David Laight
On Tue, Jun 21, 2011 at 02:16:18AM -0400, der Mouse wrote: I've just run into something (on 4.0.1) which looks to me as though write() buffer alignment matters. This sounds to me like a bug, but it appears to have something to do with the hardware, and I'd appreciate any thoughts on how I

Re: write alignment matters?

2011-06-21 Thread Thor Lancelot Simon
On Tue, Jun 21, 2011 at 12:03:10PM -0400, der Mouse wrote: Actually, I got the machine from a friend-of-a-friend who was a pretty hardcore SCSI geek in a former life (he now repairs organs, the musical kind). He would probably be the right person to ask about this; it's not totally

Re: write alignment matters?

2011-06-21 Thread der Mouse
It's a 53c8xx: we load the firmware into it from our own driver. I just had a look at esiop.ss, and it looks high-level enough that it's plausible to me that the problem is with whatever's backing that code, whether silicon or ROMed firmware or whatever. I've sent an email to the relevant

Re: write alignment matters?

2011-06-21 Thread Michael L. Hitch
On Tue, 21 Jun 2011, der Mouse wrote: It's a 53c8xx: we load the firmware into it from our own driver. I just had a look at esiop.ss, and it looks high-level enough that it's plausible to me that the problem is with whatever's backing that code, whether silicon or ROMed firmware or whatever.

Re: write alignment matters?

2011-06-21 Thread der Mouse
It's a 53c8xx: we load the firmware into it from our own driver. I just had a look at esiop.ss, and it looks high-level enough that it's plausible to me that the problem is with whatever's backing that code, whether silicon or ROMed firmware or whatever. I just verified that I see the same