Re: making GNU ls -i (--inode) work around the linux readdir bug

[Wayne Pollock]

Jim Meyering wrote:

Phillip Susi <[EMAIL PROTECTED]> wrote:

The new POSIX standard verbiage you
pointed out only _hints_ that that it is incorrect behavior, with no
justification for that position.


I know ls didn't use to work this (buggy!) way on old versions of
Linux.  I don't think it did on AIX either, but my memory isn't
that clear.  I reported the problem because when teaching about
filesystems I projected "ls -l /" to the class but the wrong
inodes appeared.  They didn't used to!  Worse, the students
said they saw different inode numbers than shown on the projection.
Turns out that I had unaliased ls (to turn off the color option as
that doesn't project well).  So I was running "/bin/ls -i /" but
the students were running "/bin/ls -i --color=auto /".

To me, it is a bug if adding color to the output changes the
reported inode numbers.  You must, for the sake of consistency,
get the same values if running ls -i with or without other
options such as -l, --color, -F, etc.

[I didn't report it earlier but "/bin/ls -iF" shows the incorrect
(underlying) inode numbers, even though it seems likely that
Gnu ls should do the same system/library calls for -F as for
"--color=auto".  Hope the Gnu maintainer checks into that.]

How can either ls or readdir be considered correct when
the output is so inconsistent?  What behavior do you expect from
backup scripts (and similar tools) that use find (or readdir)?
It seems clear to me that returning the underlying inode numbers
must result in having the wrong files backed up in this case.

If some folk feel the underlying numbers reported by the
(buggy; sorry but I don't know what other adjective to use) readdir,
then for the sake of consistency stat must be considered buggy
for reporting the root inode number of mount points.  "Consistency
first, and performance be damned!"

From my (admittedly inexpert) point of view the inode of the underlying
filesystem directory is useless except to find "hidden" files.  So
the stat behavior, even if useless without a device number too, is
the better value to return in all cases.

It might be argued which value is better, but inconsistent values
(and perhaps unpredictable values without reading the source code)
are clearly wrong.

The POSIX standard is correct in my opinion; glibc is currently buggy.
I would claim (and I guess I did) that this behavior is buggy even
if every Unix system in the universe always worked this way.
A long standing bug is still a bug.

-Wayne Pollock


___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

Re: making GNU ls -i (--inode) work around the linux readdir bug

[Jim Meyering]

Phillip Susi <[EMAIL PROTECTED]> wrote:
> The new POSIX standard verbiage you
> pointed out only _hints_ that that it is incorrect behavior, with no
> justification for that position.

>From what I've read, POSIX does not specify this.
If you know of wording that is more precise, please post a quote.

As far as I've heard, only one application would be affected
adversely by this change (extra stat calls would hurt performance),
and that application, "magicmirror" is not widely used -- since I
found no reference to it in the first few pages of search results.

If you know of other applications that run "ls -i" and depend on
the post-coreutils-6.0 behavior of not stat'ing some files,
please let me know.


___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

Re: making GNU ls -i (--inode) work around the linux readdir bug

[Phillip Susi]

Jim Meyering wrote:

The change I expect to implement does not going against POSIX.
On the contrary, the standard actually says the current readdir
behavior is buggy.  See my previous reference to a quote from
readdir's rationale.


"Going against the standard behavior" means differing in behavior that 
all other implementations have.  The new POSIX standard verbiage you 
pointed out only _hints_ that that it is incorrect behavior, with no 
justification for that position.




___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

Re: making GNU ls -i (--inode) work around the linux readdir bug

[Jim Meyering]

Phillip Susi <[EMAIL PROTECTED]> wrote:

> Jim Meyering wrote:
>> Ultimately, neither POSIX nor any other official standard defines what
>> is "right" for coreutils.  POSIX usually serves as a fine reference, but
>> I don't follow it blindly.  In rare cases I've had a well-considered
>> disagreement with some aspect of a standard, and I have implemented
>> default behavior that technically makes an application non-conforming.
>> The standard evolves, too.
>
> Going against the standard behavior is not strictly anathema, true,
> but you had better have a good reason for it.  This 'fix' gains
> NOTHING since any application ( whether it exists now or conceivably
> could in the future ) that depends on your preferred behavior is
> already inherently broken.  With nothing to gain, and both conformance
> and performance to loose, this fix seems to be bad form.

The change I expect to implement does not going against POSIX.
On the contrary, the standard actually says the current readdir
behavior is buggy.  See my previous reference to a quote from
readdir's rationale.


___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

Re: making GNU ls -i (--inode) work around the linux readdir bug

[Phillip Susi]

Jim Meyering wrote:

Ultimately, neither POSIX nor any other official standard defines what
is "right" for coreutils.  POSIX usually serves as a fine reference, but
I don't follow it blindly.  In rare cases I've had a well-considered
disagreement with some aspect of a standard, and I have implemented
default behavior that technically makes an application non-conforming.
The standard evolves, too.


Going against the standard behavior is not strictly anathema, true, but 
you had better have a good reason for it.  This 'fix' gains NOTHING 
since any application ( whether it exists now or conceivably could in 
the future ) that depends on your preferred behavior is already 
inherently broken.  With nothing to gain, and both conformance and 
performance to loose, this fix seems to be bad form.



Furthermore it _is_ right even in absolute terms.


Then we'll have to agree to disagree.


So far your well-considered disagreement with the standard seems to 
consist solely of "it is confusing that the two inum's don't match". 
That seems a fairly weak argument for technical correctness.  While I 
agree that your way makes more sense, that alone does not outweigh both 
performance and conformance.  There are plenty of things that don't seem 
to make sense at first glance, but we still do them with good reason.



When ls -i prints an inode number not equal to the one lstat would
produce, I consider it inaccurate.
ls -ldi DIR prints an accurate inode number for DIR.
ls -i DIR/.. |grep DIR fails to do so when DIR is a mount point
(but works on Cygwin)


I would argue that it is not inaccurate, since as far as correct 
operation of the machine is concerned, it has no effect.  The perceived 
inaccuracy is only in your own mind while you incorrectly attempt to 
assign meaning to the value which has none.  To justify the cost of 
conformance and performance, something a bit more substantial than 
making a human feel better needs to be achieved.



There are *no existing programs* and *no plausible correct programs*
which depend on your new behaviour.


Easy to say.  Even if it were known to be true,
imho, it's irrelevant, since your definition of "correct"
presupposes the broken behavior.


You seem to have this backwards.  Ian's definition of correct 
presupposes that the inums are useless in the presence of mount points 
in the first place, which they are without a dnum.  It is your 
definition of correct that is founded solely on what you feel the 
behavior should be, which by all logical measures, is broken.


You have no logical reason for arguing that your way is "correct" other 
than that you find the other way unsettling.



Not yet.  But I haven't done a complete survey, either.
Maybe this exposure will prod the affected systems to provide
a readdir-like interface with always-lstat-consistent d_ino values.


I'd bet money that Linux won't since there zero reason to do so, and 
several reasons not to.





___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

Re: making GNU ls -i (--inode) work around the linux readdir bug

[Paul Eggert]

Ian Jackson <[EMAIL PROTECTED]> writes:

> _Why_ is it considered a bug ?  Is it just that the members of the
> relevant committee didn't understand what d_ino was for

I don't think that is likely; the folks involved in that discussion
list are cognizant of the issues.

> Were there any examples of applications that are broken near a
> mountpoint with the traditional behaviour but correct with the new
> behaviour ?

I don't recall specific applications being discussed either way.

> Do you have a reference ?

Here's one, from me:



The subject has come up at other times on that mailing list, but with
the above URL perhaps you can search the list and track the other
instances down.


___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

Re: is hard-coding the group name, "tty", portable enough for who -a?

[Piotr Gackiewicz]

On Mon, 7 Jul 2008, James Youngman wrote:

> How about just comparing the GID of the other user's terminal against
> the GID of the terminal whose name would have been printed if we'd run
> "tty"?

Actually, my patch with checking terminals group against "tty" was based on
permissions of "write" and "wall" packaged in fedora 8 (and probably most of
modern linuxes):

-bash-3.2# ls -l /usr/bin/write /usr/bin/wall
-r-xr-sr-x 1 root tty  8308 paź 10  2007 /usr/bin/wall
-rwxr-sr-x 1 root tty 10056 lip  7 12:13 /usr/bin/write

Another possibility is to set same permissions on "who" as on these tools.

Regards,

-- 
Piotr Gackiewicz___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

Re: making GNU ls -i (--inode) work around the linux readdir bug

[Jim Meyering]

Ian Jackson <[EMAIL PROTECTED]> wrote:
> Jim Meyering writes ("Re: making GNU ls -i (--inode) work around the linux 
> readdir bug"):
>> Ian Jackson <[EMAIL PROTECTED]> wrote:
>> > That is all systems.  All UN*X systems since the dawn of time have
>> > behaved this way.
>>
>> Just because everyone does it doesn't make it right.
>
> In fact, since you yourself are referring to standards documents
> (which are supposed to document existing practice) to prove your
> point, yes, it does!

Ultimately, neither POSIX nor any other official standard defines what
is "right" for coreutils.  POSIX usually serves as a fine reference, but
I don't follow it blindly.  In rare cases I've had a well-considered
disagreement with some aspect of a standard, and I have implemented
default behavior that technically makes an application non-conforming.
The standard evolves, too.

> Furthermore it _is_ right even in absolute terms.

Then we'll have to agree to disagree.

> You have failed
> _again_ to respond to my point about the device number.  Let me repeat it:

No need.  I addressed it in what you quoted below.
You failed to make the connection.

...
>> Besides, according to this,
>> http://bugs.debian.org/cgi-bin/bugreport.cgi?bug=369822#60
>> at least Cygwin 1.5.20 provides a readdir function that works
>> the way I expect.
>
> I can't believe it!
> You're holding up Cygwin as an example of what the UN*X API should be!

Yes.

>> >> I think correctness is important enough to sacrifice
>> >> the optimization in this unusual corner-case usage of ls.
>> >
>> > This is the whole _point_ of  ls -i  !
>>
>> Being fast and inaccurate was never the point of ls -i.
>
> It's not `inaccurate'.  It's perfectly accurate.

When ls -i prints an inode number not equal to the one lstat would
produce, I consider it inaccurate.
ls -ldi DIR prints an accurate inode number for DIR.
ls -i DIR/.. |grep DIR fails to do so when DIR is a mount point
(but works on Cygwin)

> Any existing correct
> program will behave correctly with the traditional ls -i.
...

> There are *no existing programs* and *no plausible correct programs*
> which depend on your new behaviour.

Easy to say.  Even if it were known to be true,
imho, it's irrelevant, since your definition of "correct"
presupposes the broken behavior.

> Why break existing software for
> no benefit ?

"break" is an exaggeration, and for all I know, the only tool
that's affected is your (never-published?) magicmirror script.
Since any tool using pre-coreutils-6.0 ls will experience the
same behavior (wrt performance), I'm not too concerned.
Many distributions are still using coreutils-5.9x and earlier.

>> Or better still, maybe someone will fix Linux's getdents (the syscall
>> behind readdir) to do the right thing even in the presence
>> of mount points.
>
> This would be slow for many of the same reasons (although maybe not
> _as_ slow as doing stat for each entry).  I hope and trust that kernel
> developers are more aware of the proper behaviour of the API than
> implied by your suggestion.
>
> Can you name _one_ UN*X system (Cygwin does _not_ count) which behaves
> the way you think is correct ?

Not yet.  But I haven't done a complete survey, either.
Maybe this exposure will prod the affected systems to provide
a readdir-like interface with always-lstat-consistent d_ino values.

>> I insist: it is a bug.
>> If I weren't convinced I wouldn't be spending time on it, now.
>
> If there is nothing I can say to change your mind then why are we
> having this conversation ?

I've changed my mind before.
Still waiting for a convincing argument.
It'd certainly be easier to leave things the way they are.

...
>> Sure, but how can ls (in general, and efficiently) know whether
>> the device number is relevant?
>
> The _caller_ knows that the device number is _irrelevant_.

I don't want to limit `ls -i's correctness to that particular use case.

> Because
> otherwise the results from  ls -i  are useless.  So if the caller
> knows that the device number is irrelevant there is no point going to
> any effort to supply it.
>
> The caller communicates this fact (that the device number is
> irrelevant) to ls by passing the `-i' option.  This is a safe

This is neither common practice nor documented behavior.

> assumption by ls because no correct program could use the output from
> ls -i unless the device number were irrelevant.
...


___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

[bug #23798] Proposal to add some new free & open file formats to src/dircolors.hin

[INVALID.NOREPLY]

URL:
  

 Summary: Proposal to add some new free & open file formats
to src/dircolors.hin
 Project: GNU Core Utilities
Submitted by: jemm
Submitted on: Tuesday 07/08/2008 at 10:34
Category: None
Severity: 3 - Normal
  Item Group: None
  Status: None
 Privacy: Public
 Assigned to: None
 Open/Closed: Open
 Discussion Lock: Any

___

Details:

With reference to this Xiph.org (the organisation behind free video and audio
codecs) document http://wiki.xiph.org/index.php/MIME_Types_and_File_Extensions
I'd propose to add the following lines to src/dircolors.hin:

# the list of image & video formats
.axv 01;35
.anx 01;35
.ogv 01;35
.ogx 01;35

# the list of audio formats
.axa 00:;36
.oga 00:;36
.spx 00:;36
.xspf 00:;36




___

Reply to this item at:

  

___
  Message sent via/by Savannah
  http://savannah.gnu.org/



___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils

Re: making GNU ls -i (--inode) work around the linux readdir bug

[Ian Jackson]

Paul Eggert writes ("Re: making GNU ls -i (--inode) work around the linux 
readdir bug"):
> Tony Finch <[EMAIL PROTECTED]> writes:
> > Also, readdir(3) is not the only part of POSIX that needs clarifying.
> 
> I participated in the discussion that resulted in this new d_ino
> wording being added to POSIX, and my recollection is that the common
> behavior where readdir returns the inode number of the underlying
> mount point is now considered to be a bug,

_Why_ is it considered a bug ?  Is it just that the members of the
relevant committee didn't understand what d_ino was for, and its
inherent limitations in the absence of a the corresponding dev ?

Were there any examples of applications that are broken near a
mountpoint with the traditional behaviour but correct with the new
behaviour ?

> If there's any further question about this it might help to track down
> the relevant Austin Group discussion.

Do you have a reference ?

Ian.


___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils