Re: [PATCH]: ls: add --user-format option for user defined format
Hey, just chiming in, since Jim alerted me to this thread after I recently filed http://bugs.debian.org/505979. I think, printf-like functionality would be very useful to have for very much the same reasons quoted in this thread, so I won't repeat them here. The issue Jim raises -- bloat -- is surely a valid one, but I don't think there's a way around it. Bloat comes with two immediate dangers: larger size, which isn't really a problem given today's computation power and storage sizes, and manageability/security. printf-like functionality in ls is a worthwhile feature, I don't think there's a debate about that. The question is how to get there. I think it's either bloating the code and making it work, then slowly fading out other options which can be nicely implemented with the new printf-code, or the other way around: rewriting most options from scratch to use a common printf-infrastructure. The latter option surely allows for better design, but it's also guaranteed to take longer and introduce new bugs. The first option, on the other hand, is based on minimal changes, which are a more sustainable way forward, given the age and maturity of ls, I think. In my (possibly not-so) humble opinion, this is separate from fts tree traversal. -- .''`. martin f. krafft [EMAIL PROTECTED] Related projects: : :' : proud Debian developer http://debiansystem.info `. `'` http://people.debian.org/~madduckhttp://vcs-pkg.org `- Debian - when you have better things to do than fixing systems the strength of women comes from the fact that psychology cannot explain us. men can be analyzed, women merely adored. -- oscar wilde digital_signature_gpg.asc Description: Digital signature (see http://martin-krafft.net/gpg/) ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [PATCH]: ls: add --user-format option for user defined format
Paul Eggert [EMAIL PROTECTED] wrote:
Jim Meyering [EMAIL PROTECTED] writes:
- is it worthwhile to add a --printf option to ls?
I don't like the --user-format name)
- if so, should it use use a find -printf-compatible format string
or one compatible to stat --printf? Either way, it'll need a few
extensions.
I think it's worthwhile (as long as someone else does it :-) but I'm
afraid that neither find's nor stat's printf options are up to ls's main
task, which is to generate nice output for humans.
ls is pretty good about lining up columns, even when the column data is
unexpectedly wide, whereas 'find' and 'stat' are not. I would expect
that a -printf option to 'ls' would normally autoadjust column widths,
but neither 'find' nor 'stat' does that now.
Thanks for replying.
So far, enough of the feedback has been in favor
of adding the option that I think we can agree that *some*
new option will be added. Now only the form is up for debate.
I've been thinking about being able to specify the field
width for such columns. We may have to use a special character
to say use the automatically-computed minimum. E.g., if %p
denotes the name, a format of %p would print just the name,
with no padding, while %20p would use a 20-column-wide field,
and [EMAIL PROTECTED] would use the minimum width for all strings considered.
I don't see a strong need to maintain absolute compatibility with find
-printf or with stat -printf. Obviously we shouldn't be gratuitously
compatible, but we might as well do it 'right' for 'ls', propagating any
improvements to 'stat'. (find -printf is not standardized by POSIX, so
there's not a huge portability issue there.)
I just now looked at the documentation for stat -printf and find
-printf. The stat documentation is easier to read, and has some
features that I'd expect people would find useful (e.g., quoting file
names with funny characters). The find documentation has some fancy
features for time stamp formatting, which I couldn't figure out from the
documentation; I suggest cleaning that up in 'ls', for example, by
having a --time-format=FORMAT option that specifies the time format in a
strftime-like way.
Actually, that's nearly what find's format specifiers let you do.
But find's approach is more general (albeit more obscure) in that it
lets you format atime, ctime, and mtime independently. For example,
to print the atime in seconds-since-the-epoch (with fractional part),
ctime's %F (date-only), and the mtime's %T (time-only):
$ find . -maxdepth 0 -printf '%p %A@ %CF %TT\n'
. 1204905873.00 2008-11-21 18:08:35.00
note that find has to use something other than 's' for
seconds-since-epoch, because this is a printf format string
that must honor %s (argument string) substitutions.
Obviously, ls wouldn't have that constraint, so it _could_ use 's'.
The %A %T %C prefixes are fine if you're just doing one directive
at a time, but for general use, I find that notation too obtuse.
Something like '%{name} %{atime:%s.%N} %{ctime:%F} %{mtime:%T}\n'
seems cleaner.
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [PATCH]: ls: add --user-format option for user defined format
Pádraig Brady wrote: Ondřej Vašík wrote: Let's do some summary, feel free to add/comment items if you have something not mentioned here: What patched ls --user-format can and upstream find -printf not: 1) colored files by LS_COLORS or automatically if requested 2) could be used simply as ls alias for normal users - as they could still specify additional ls options like time-style, quoting-style, sorting style, blocksize, units and other things without modifying format string 6) Automated column width computation (in find -printf you have to hardcode the column width in %N.NX syntax yourself, otherwise there is no defined human-readable column structure.) I guess points #1, #2, #6 are the most important things, as #1 and #2 makes the output more user friendly and #6 generally readable by human Right. As I see it ls output is tuned for human consumption, while find is tuned for further consumption by other scripts/utils. Exactly - I have the same opinion and I wrote it in the previous email ... and as humans do differ and they could have different wishes about format of output, I would expect something to tune output in ls - without processing through awk/sed/whatever utilities. In my experience I've only needed to tweak output like this to ease the subsequent processing in scripts/utils. I.E. I've never needed it in ls, I'm quite sure that you (and Jim) never needed that option. Otherwise it would had been already implemented. In my case it is slightly different. I never needed that option, but I sometimes wished to have it. For me is ls -l too verbose and other ls formats too simple. I would like to have something like with format colored_name_with_link (owner:group:filetype:octalmode) Human-readable_size - so e.g. bar (bar :root:-:0664) 24K baz (baz :root:-:0664) 100K bazlink - ./baz (root:root:l:0777)5 foo (foo :foo :-:0664) 10K That would be enough for me in most cases (octal form is enough for me, in most cases I don't need ACL char, number of links and mostly not even the time). Now I have to parse things out of very verbose long format (and every user has to parse it). Easy, but I wish to have another choice - and this choice could be --user-format option. I'm not sure it's worth duplicating this in ls (adding a lot of interface for all users to parse and most ignore). You are right, it is a lot of interface for all users to parse. So probably user-format description should be removed from --help completely - and there should be only pointer that format is described in info documentation. Greetings, Ondrej signature.asc Description: Toto je digitálně podepsaná část zprávy ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [PATCH]: ls: add --user-format option for user defined format
Ondřej Vašík [EMAIL PROTECTED] wrote: Pádraig Brady wrote: Ondřej Vašík wrote: Let's do some summary, feel free to add/comment items if you have something not mentioned here: What patched ls --user-format can and upstream find -printf not: 1) colored files by LS_COLORS or automatically if requested 2) could be used simply as ls alias for normal users - as they could still specify additional ls options like time-style, quoting-style, sorting style, blocksize, units and other things without modifying format string 6) Automated column width computation (in find -printf you have to hardcode the column width in %N.NX syntax yourself, otherwise there is no defined human-readable column structure.) I guess points #1, #2, #6 are the most important things, as #1 and #2 makes the output more user friendly and #6 generally readable by human Right. As I see it ls output is tuned for human consumption, while find is tuned for further consumption by other scripts/utils. Exactly - I have the same opinion and I wrote it in the previous email ... and as humans do differ and they could have different wishes about format of output, I would expect something to tune output in ls - without processing through awk/sed/whatever utilities. In my experience I've only needed to tweak output like this to ease the subsequent processing in scripts/utils. I.E. I've never needed it in ls, I'm quite sure that you (and Jim) never needed that option. Otherwise it would had been already implemented. In my case it is slightly different. I never needed that option, but I sometimes wished to have it. For me Actually, ever since find got its -printf option, I've thought of adding the same to ls. But the size of the code addition as well as the logistics (this was before gnulib) were off-putting, not to mention the fact that this is ls, after all. That combined to make the overall cost/benefit ratio appear way too high. Here are my questions: - is it worthwhile to add a --printf option to ls? I don't like the --user-format name) - if so, should it use use a find -printf-compatible format string or one compatible to stat --printf? Either way, it'll need a few extensions. I'm still on the fence. On the one hand, I don't like to bloat ls further, even if it ends up using code that's shared with GNU find. On the other, I understand and sympathize with the desire to make ls output more useful/readable. Finally, if investing in ls, I'd rather invest in converting it to use fts for its hierarchy traversal. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [PATCH]: ls: add --user-format option for user defined format
Jim Meyering wrote: Actually, ever since find got its -printf option, I've thought of adding the same to ls. But the size of the code addition as well as the logistics (this was before gnulib) were off-putting, not to mention the fact that this is ls, after all. That combined to make the overall cost/benefit ratio appear way too high. Here are my questions: - is it worthwhile to add a --printf option to ls? I don't like the --user-format name) - if so, should it use use a find -printf-compatible format string or one compatible to stat --printf? Either way, it'll need a few extensions. So there would be 3 large interfaces to try and find commonalities in and maintain. I'm still on the fence. On the one hand, I don't like to bloat ls further, even if it ends up using code that's shared with GNU find. On the other, I understand and sympathize with the desire to make ls output more useful/readable. Finally, if investing in ls, I'd rather invest in converting it to use fts for its hierarchy traversal. I'm on the fence too, but more on the bottom rung. I don't think the interface pollution cost to the benefit gained is worth it. ls -l has good defaults for me. Infrequently I want other info which other options to ls, or stat provide already. I'm also reminded of http://examples.oreilly.com/upt2/#sls The main use case there is for processing by scripts, which find --printf is sufficient for now. Going forward I would add extra formats to `find --printf`, with a view to keeping them in sync with `stat --printf`. cheers, Pádraig. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [PATCH]: ls: add --user-format option for user defined format
Jim Meyering wrote:
- is it worthwhile to add a --printf option to ls?
I don't like the --user-format name)
You know my opinion ... I guess it is worthwhile, as the consumer of
ls output is generally human and human wishes do differ. I'm ok with
--printf option, you are right - it would be better to keep conventions.
- if so, should it use use a find -printf-compatible format string
or one compatible to stat --printf? Either way, it'll need a few
extensions.
I guess it should use find -printf style with some extensions. As ls
doesn't change much, I guess there will be not so many changes in the
interface in future. Some extensions would be necessary, some things,
maybe they could be mentioned explicitly in findutils and ls. Anyway it
is bad that there are already two different tools and interfaces (stat
--printf and find --printf) for exactly the same thing... as both are
usable only in scripts. Adding option to ls means human readability.
Additionally it would be good to unify stat --printf and find -printf
format at least for the %{filename} style format which was proposed as
possible future extension.
I'm still on the fence. On the one hand, I don't like to bloat
ls further, even if it ends up using code that's shared with GNU find.
On the other, I understand and sympathize with the desire to make ls
output more useful/readable.
There were already some wishes/requests to add that feature to ls.
Additionally - as I said, if the implementation of ls --printf option
would be improved (and I think it could be improved easily), it could
even simplify the ls.c code, as format options could be replaced just by
--printf format strings - so functions like print_with_commas,
print_horizontal, print_many_per_line and others could be removed
completely.
As Pádraig mentioned there are some implementations of 'ls-like'
utilities doing the same (sls, limo and maybe others) - always
mentioning that the utility is add-on to ls - simply because users
expect that feature from ls. And I guess it would be better to have it
centralized in ls...
Finally, if investing in ls, I'd rather invest in converting it to use
fts for its hierarchy traversal.
As the base code for ls --printf option (almost) exists, I hope I'll
be able to change the options (and to amend documentation) quite quickly
if the option with the objections from the discussion (at the moment I
guess verbosity on --help, different format from find -printf, name of
that option) will be generally accepted. Changing ls to use fts seems to
take much more time for me at the moment. But good TODO task...
Greetings,
Ondřej
signature.asc
Description: Toto je digitálně podepsaná část zprávy
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [PATCH]: ls: add --user-format option for user defined format
Jim Meyering wrote:
Back when I thought it might be worthwhile, I would have required a
find-compatible format string. One thing find has, but that stat lacks,
is a complete range of date/time-formatting options. i.e., you can
print the date-only portion of the atime:
$ find / -maxdepth 0 -printf '%p %AF\n'
/ 2008-10-04
or print both date and time with a T between them:
$ find / -maxdepth 0 -printf '%p %AFT%AT\n'
/ 2008-10-04T04:02:02.00
You could do that easily as you have ls
--time-style=+timestyleformat for specifying timestamp...
So just ls --time-style=+%F --user-format=format
and ls --time-style=+%FT%T --user-format=format
It has limitation of one user-defined time-style per format, but I can't
imagine use case of having more than one timestamp per entry (maybe to
have timestamp of last content modification and year of inode change,
but this one example could be done by column-width reduction to 4)
Have you considered adding SELinux-related format directives to GNU find?
It's not only about those SELinux-related things. But I'm ok with doing
some extends to find -printf later.
Let's do some summary, feel free to add/comment items if you have
something not mentioned here:
What patched ls --user-format can and upstream find -printf not:
1) colored files by LS_COLORS or automatically if requested
2) could be used simply as ls alias for normal users - as they could
still specify additional ls options like time-style, quoting-style,
sorting style, blocksize, units and other things without modifying
format string
3) SELinux context and it's parts
4) ACL/SELinux special char ('+'/'.')
5) Displaying only parts of access rights mode string
6) Automated column width computation (in find -printf you have to
hardcode the column width in %N.NX syntax yourself, otherwise there is
no defined human-readable column structure.)
7) Automated handling of recent/nonrecent date/time structure like is
done in common ls -l listing
8) Printing file size in human readable mode
9) Printing total blocksize/size in bytes of the filesets
10) Printing filename with destination symlink if exists
(name - symlink format)
I guess points #1, #2, #6 are the most important things, as #1 and #2
makes the output more user friendly and #6 generally readable by human
without additional processing or reserving more than necessary column
width. Some things mentioned here could/should be added to find -printf
quite easily, #2, #6 and #9 difficult/impossible without huge
performance impact (and is total nonsense to add them to find, should be
handled by ls and justify the fact that ls --user-format as NOT
duplicating of find -printf functionality).
What upstream find -printf can and patched ls --user-format not:
1) Specifying subsections of displayed time more than once
2) More than one entry per line (no new line after entry by default)
3) Displaying depth (I guess not necessary at the ls)
4) \c in format (flush output immediately)
5) Device number on which file exists
6) File system type
Missing things #1, #2 could be easily added/changed (#2 oneliner and
will extend possible format strings, #1 means additional parsing or
sharing with find -printf (maybe via new gnulib module)). Point #3 is
not the case of coreutils. I'm not sure about usefulness of #4. Things
mentioned as #5 and #6 can't be printed by current ls, in general it
could be added.
For me:
ls is utility which should be able to list content of the directory(ies)
(and file informations) in HUMAN READABLE form - so within columns, with
colors, with additional user-modifiable enhancements. This new
--user-format should allow the user to create simple UNIVERSAL alias for
just viewing directory content in requested form without additional
processing.
find is utility which with -printf option should be able to list content
of selected set of files in form suitable for FURTHER PROCESSING. So you
could use e.g. sort, wc, awk and other tools on it to make it better
readable for humans or you can store the output for further user. Some
things which could be EASILY handled by ls could not be solved in find,
as it uses completely different approach.
find in comparision with ls much more powerful, but also comparatively
harder to use. ls is light-weight, more handy for casual use.
Possible solutions:
1) To improve --user-format patch to comply completely with printf
format options, to reduce the code by factorization of some parts
to gnulib, to simplify main print_user_format function by making
small subfunctions. Some sections of code might be worth of sharing
with find -printf (e.g. time/date parsing definitions)
Will take quite long of work/time which could be done later when
using solution #2...
2) To make --user-format similar to find -printf - to comply with its
syntax (but not extend some redundant sections - no need to duplicate
time-style option functionality), to add some things to find -printf
and to reduce
Re: [PATCH]: ls: add --user-format option for user defined format
Ondřej Vašík wrote: Let's do some summary, feel free to add/comment items if you have something not mentioned here: What patched ls --user-format can and upstream find -printf not: 1) colored files by LS_COLORS or automatically if requested 2) could be used simply as ls alias for normal users - as they could still specify additional ls options like time-style, quoting-style, sorting style, blocksize, units and other things without modifying format string 6) Automated column width computation (in find -printf you have to hardcode the column width in %N.NX syntax yourself, otherwise there is no defined human-readable column structure.) I guess points #1, #2, #6 are the most important things, as #1 and #2 makes the output more user friendly and #6 generally readable by human Right. As I see it ls output is tuned for human consumption, while find is tuned for further consumption by other scripts/utils. In my experience I've only needed to tweak output like this to ease the subsequent processing in scripts/utils. I.E. I've never needed it in ls, and I'm not sure it's worth duplicating this in ls (adding a lot of interface for all users to parse and most ignore). Pádraig. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [PATCH]: ls: add --user-format option for user defined format
On Fri, Nov 7, 2008 at 8:39 AM, Jim Meyering [EMAIL PROTECTED] wrote:
Have you considered adding SELinux-related format directives to GNU find?
I would likely accept such patches into find, subject to a GNU
copyright assignment of course.
More generally, a more flexible syntax for find's -printf formats
might be useful. Because of the limited number of format directive
letters remaining unused in find, I have been thinking about
directives based on names. This might look like %20.18{pathname}s,
%{filemode}o %{filemode}d and so on. It's inappropriate to
complicate the format directives completely to the point where find
has an embedded string formatting language though (as well as its
embedded filesystem predicate language). If somebody did this work it
would probably make sense to refactor the code a bit such that the
formatting implementation could be lifted out as a module and re-used
elsewhere.
If you're interested in pursuing this (either the SELinux format
enhancements or the more ambitious change), please email
[EMAIL PROTECTED]
Thanks.
James.
___
Bug-coreutils mailing list
Bug-coreutils@gnu.org
http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [PATCH]: ls: add --user-format option for user defined format
Ondřej Vašík [EMAIL PROTECTED] wrote: Pádraig Brady wrote: Thanks for doing that. However I thought that item had been removed from the TODO: http://lists.gnu.org/archive/html/bug-coreutils/2008-02/msg00115.html This is a large patch to do essentially what `find -printf` already does. Sorry, haven't seen that thread before. I guess there are some things which you can't do with find -printf and which you can do with that --user-format ls extension. I would say there are few reasons why to use ls --user-format instead of find -printf. 1) It has scalable column width (even possible to reduce width) 2) It has some options unavailable in findutils e.g. for SELinux context handling and filemode splitting/handling. 3) It is based on the informations you have completed by ls - so it could be considered more reliable, e.g. it could have EXACTLY the same output as ls -l otherwise have. 4) Is more frequent to use ls than find -printf for listing files, you could get colored filenames output with column. I agree that the patch is quite large, anyway what I have heard from the guys around me, it would be useful to have that thing in ls. I tried to not impact the rest of the ls code, so the changes to the main ls code are pretty minimal (only some counting (and even that only for user_format selected) and factorization of one function in print_long_format). Hi Ondřej, I wish you had mentioned your plans before starting work on that. Please heed the warning at the top of the TODO file, especially when considering adding options to ls. Here it is: Beware: some are quite old and no longer valid. To avoid wasting your time by duplicating work or by working on a task that is no longer pertinent, please search the mailing list and post your intent before embarking on a big project. I did not remove that TODO item on a whim. Back when I thought it might be worthwhile, I would have required a find-compatible format string. One thing find has, but that stat lacks, is a complete range of date/time-formatting options. i.e., you can print the date-only portion of the atime: $ find / -maxdepth 0 -printf '%p %AF\n' / 2008-10-04 or print both date and time with a T between them: $ find / -maxdepth 0 -printf '%p %AFT%AT\n' / 2008-10-04T04:02:02.00 Have you considered adding SELinux-related format directives to GNU find? ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [PATCH]: ls: add --user-format option for user defined format
Pádraig Brady wrote: Thanks for doing that. However I thought that item had been removed from the TODO: http://lists.gnu.org/archive/html/bug-coreutils/2008-02/msg00115.html This is a large patch to do essentially what `find -printf` already does. Sorry, haven't seen that thread before. I guess there are some things which you can't do with find -printf and which you can do with that --user-format ls extension. I would say there are few reasons why to use ls --user-format instead of find -printf. 1) It has scalable column width (even possible to reduce width) 2) It has some options unavailable in findutils e.g. for SELinux context handling and filemode splitting/handling. 3) It is based on the informations you have completed by ls - so it could be considered more reliable, e.g. it could have EXACTLY the same output as ls -l otherwise have. 4) Is more frequent to use ls than find -printf for listing files, you could get colored filenames output with column. I agree that the patch is quite large, anyway what I have heard from the guys around me, it would be useful to have that thing in ls. I tried to not impact the rest of the ls code, so the changes to the main ls code are pretty minimal (only some counting (and even that only for user_format selected) and factorization of one function in print_long_format). Please consider those things before throwing that patch away. Greetings, Ondřej signature.asc Description: Toto je digitálně podepsaná část zprávy ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
Re: [PATCH]: ls: add --user-format option for user defined format
Ondřej Vašík wrote: Hello, as it was in TODO file and as I guess it could be useful, I made --user-format long option enhancement to ls. and added documentation for it to --help and info pages. This is base approach, I guess some reduction of common sections should be done (print_escape_char() and similar functions are used in more programs), but I tried to reduce that patch - will do that later. This long option was tested by few friends, so format seems to work properly (at least on GNU/Linux machines), it's based on stat's user-defined format parser. There are some new features added into that user-format - e.g. displaying mode as octal number, splitting mode string to separate section, choosing column width for entries. It could be easily extended when needed. Test to compare specific user-format output with ls -l was also added. Thanks for doing that. However I thought that item had been removed from the TODO: http://lists.gnu.org/archive/html/bug-coreutils/2008-02/msg00115.html This is a large patch to do essentially what `find -printf` already does. Pádraig. ___ Bug-coreutils mailing list Bug-coreutils@gnu.org http://lists.gnu.org/mailman/listinfo/bug-coreutils
