Re: svn commit: r284198 - head/bin/ls
On Sat, Jun 13, 2015 at 9:29 PM, Steve Kargl s...@troutmask.apl.washington.edu wrote: On Sat, Jun 13, 2015 at 05:40:59PM -0700, Craig Rodrigues wrote: On Sat, Jun 13, 2015 at 5:26 PM, Steve Kargl s...@troutmask.apl.washington.edu wrote: Given the horrid state of the manpages, which I showed in March, one can only wonder about the internals of the libxo itself. Are you talking about this comment you made? https://lists.freebsd.org/pipermail/freebsd-current/2015-March/054899.html I can't make heads or tails of what you wrote, other than you seemed very angry. I wasn't very angry. I'm simply pointing out that the libxo manpages, which should document what libxo is/does, are horrible documentation. If the quality of the manpages matches the quality of library, and the brokeness that we have been witnesses bears this out, should be questioned. % cd src/contrib/libxo/libxo % grep Nd *.3 | grep formatted xo_attr.3:.Nd emit formatted output based on format string and arguments xo_create.3:.Nd emit formatted output based on format string and arguments xo_emit.3:.Nd emit formatted output based on format string and arguments xo_finish.3:.Nd emit formatted output based on format string and arguments xo_flush.3:.Nd emit formatted output based on format string and arguments xo_open_list.3:.Nd emit formatted output based on format string and arguments xo_set_allocator.3:.Nd emit formatted output based on format string and arguments xo_set_flags.3:.Nd emit formatted output based on format string and arguments xo_set_info.3:.Nd emit formatted output based on format string and arguments xo_set_style.3:.Nd emit formatted output based on format string and arguments xo_set_writer.3:.Nd emit formatted output based on format string and arguments Do you really believe that the Nd entries for these manpages are correct? I opened this bug report against libxo: https://github.com/Juniper/libxo/issues/43 Phil Shafer fixed it and closed that bug report out. Hopefully these fixes will make it into FreeBSD after the next libxo import. -- Craig ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On 06/15/15 at 12:49P, Bryan Drewery wrote: On 6/13/15 9:29 PM, Steve Kargl wrote: On Sat, Jun 13, 2015 at 05:40:59PM -0700, Craig Rodrigues wrote: On Sat, Jun 13, 2015 at 5:26 PM, Steve Kargl s...@troutmask.apl.washington.edu wrote: Given the horrid state of the manpages, which I showed in March, one can only wonder about the internals of the libxo itself. Are you talking about this comment you made? https://lists.freebsd.org/pipermail/freebsd-current/2015-March/054899.html I can't make heads or tails of what you wrote, other than you seemed very angry. I wasn't very angry. I'm simply pointing out that the libxo manpages, which should document what libxo is/does, are horrible documentation. If the quality of the manpages matches the quality of library, and the brokeness that we have been witnesses bears this out, should be questioned. % cd src/contrib/libxo/libxo % grep Nd *.3 | grep formatted xo_attr.3:.Nd emit formatted output based on format string and arguments xo_create.3:.Nd emit formatted output based on format string and arguments xo_emit.3:.Nd emit formatted output based on format string and arguments xo_finish.3:.Nd emit formatted output based on format string and arguments xo_flush.3:.Nd emit formatted output based on format string and arguments xo_open_list.3:.Nd emit formatted output based on format string and arguments xo_set_allocator.3:.Nd emit formatted output based on format string and arguments xo_set_flags.3:.Nd emit formatted output based on format string and arguments xo_set_info.3:.Nd emit formatted output based on format string and arguments xo_set_style.3:.Nd emit formatted output based on format string and arguments xo_set_writer.3:.Nd emit formatted output based on format string and arguments Do you really believe that the Nd entries for these manpages are correct? I also found that from simple 'man ls' (etc) there is no real mention of what --libxo even is or how it works. Following the manpage cross-references leads me to have to go to a webpage to see what params --libxo even takes. The --libxo flag needs much more documentation in these manpages. Sorry if I've missed it but has any of the libxo proponents signed up to fix the documentation part? Cheers, Hiren pgpzpsibZAppH.pgp Description: PGP signature
Re: svn commit: r284198 - head/bin/ls
On Wed, Jun 17, 2015 at 12:28 PM, hiren panchasara hi...@strugglingcoder.info wrote: Sorry if I've missed it but has any of the libxo proponents signed up to fix the documentation part? I'm not signing up to fix the documentation for libxo, but I see bugs, I will file them upstream. I just filed this one based on Steve Kargl's e-mail: https://github.com/Juniper/libxo/issues/43 It would be helpful if folks could file bugs and submit patches to the maintainer via Github if they see problems in the library itself. -- Crai ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Jun 17, 2015, at 12:28 PM, hiren panchasara hi...@strugglingcoder.info wrote: I also found that from simple 'man ls' (etc) there is no real mention of what --libxo even is or how it works. Following the manpage cross-references leads me to have to go to a webpage to see what params --libxo even takes. The --libxo flag needs much more documentation in these manpages. Sorry if I've missed it but has any of the libxo proponents signed up to fix the documentation part? This already been relayed to Phil and said he’ll fix it. -- Marcel Moolenaar mar...@xcllnt.net signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r284198 - head/bin/ls
On 15/06/2015 19:49, Warner Losh wrote: I’ve yet to see why ls —libxo is better than a separate program articulated anywhere other than libxo all the things.” Having a clear statement about why it is needed, why changing it vs having a separate program, etc would help. But is seems overly gratuitous with little benefit. +1, I don't see how libxo-ized ls(1) adds value. e.g. in Python, one can use pathlib and scandir to walk arbitrarily wide and deep hierarchies, much as 'file ... | xargs foo'. It has even (with hard work by koobs) supported FreeBSD's stat.st_flags since 2.3. So anything ls(1) can do, Python could do anyway. ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On 15/06/2015 5:19 AM, Garrett Cooper wrote: Next time someone else converts ANYTHING to libxo -- write tests FIRST to make sure you're not breaking legacy behavior. If you need help figuring out how to do that, I'll be more than happy to document it on a wiki page, with simple, concise directions. Please do so anyway, these kinds of things are great (simple) ways for people inside and outside the project to start contributing to. *Then* perhaps we can hook in some code coverage checks into our CI, publish the numbers and go from there. Koobs ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On 6/15/15 1:34 PM, Garrett Cooper wrote: On Jun 14, 2015, at 20:53, Julian Elischer jul...@freebsd.org wrote: On 6/14/15 10:48 AM, Adrian Chadd wrote: On 13 June 2015 at 18:22, Craig Rodrigues rodr...@freebsd.org wrote: On Sat, Jun 13, 2015 at 6:00 PM, Adrian Chadd adr...@freebsd.org wrote: I guarantee that no matter what you've worked on, there's approximately five orders of magnitude of shipping devices whose entire storage space can be measured in 1 digit megabytes. Each year. (And yes - there's an appreciable set of them for which freebsd boots, runs and keeps running on them.0 You can buy em too, some of them even under $60. Can FreeBSD now not run on these systems because of libxo? It's a tight squeeze as it is. Running in 8MB of flash (even if it's compressed) is still an exercise in what can you cut out. My point isn't that it isn't running because of libxo; my point is that arguing about embedded involving lots of storage is woefully incorrect and will continue to be until those gigabytes of storage are available for a penny. Which yes, I'm guessing will happen in my career - but it's also quite likely code bloat will continue to chase that upward. do we have a WITHOUT_LIBXO option on sources? I believe we should.. +1. I would be more than happy to implement it by stubbing out the majority of the macros to something less invasive, but it might be a bit before I do that. Thanks, but that wouldn't remove the bloat within the apps.. just make it use null calls. ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Mon, Jun 15, 2015 at 6:46 PM, Julian Elischer jul...@freebsd.org wrote: On 6/15/15 1:34 PM, Garrett Cooper wrote: On Jun 14, 2015, at 20:53, Julian Elischer jul...@freebsd.org wrote: On 6/14/15 10:48 AM, Adrian Chadd wrote: On 13 June 2015 at 18:22, Craig Rodrigues rodr...@freebsd.org wrote: On Sat, Jun 13, 2015 at 6:00 PM, Adrian Chadd adr...@freebsd.org wrote: I guarantee that no matter what you've worked on, there's approximately five orders of magnitude of shipping devices whose entire storage space can be measured in 1 digit megabytes. Each year. (And yes - there's an appreciable set of them for which freebsd boots, runs and keeps running on them.0 You can buy em too, some of them even under $60. Can FreeBSD now not run on these systems because of libxo? It's a tight squeeze as it is. Running in 8MB of flash (even if it's compressed) is still an exercise in what can you cut out. My point isn't that it isn't running because of libxo; my point is that arguing about embedded involving lots of storage is woefully incorrect and will continue to be until those gigabytes of storage are available for a penny. Which yes, I'm guessing will happen in my career - but it's also quite likely code bloat will continue to chase that upward. do we have a WITHOUT_LIBXO option on sources? I believe we should.. +1. I would be more than happy to implement it by stubbing out the majority of the macros to something less invasive, but it might be a bit before I do that. Thanks, but that wouldn't remove the bloat within the apps.. just make it use null calls. That was the idea. Make it like libnxo or something, a dummy library like libnetbsd with weak symbols that can be optimized away by the compiler... ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On 6/13/15 9:29 PM, Steve Kargl wrote: On Sat, Jun 13, 2015 at 05:40:59PM -0700, Craig Rodrigues wrote: On Sat, Jun 13, 2015 at 5:26 PM, Steve Kargl s...@troutmask.apl.washington.edu wrote: Given the horrid state of the manpages, which I showed in March, one can only wonder about the internals of the libxo itself. Are you talking about this comment you made? https://lists.freebsd.org/pipermail/freebsd-current/2015-March/054899.html I can't make heads or tails of what you wrote, other than you seemed very angry. I wasn't very angry. I'm simply pointing out that the libxo manpages, which should document what libxo is/does, are horrible documentation. If the quality of the manpages matches the quality of library, and the brokeness that we have been witnesses bears this out, should be questioned. % cd src/contrib/libxo/libxo % grep Nd *.3 | grep formatted xo_attr.3:.Nd emit formatted output based on format string and arguments xo_create.3:.Nd emit formatted output based on format string and arguments xo_emit.3:.Nd emit formatted output based on format string and arguments xo_finish.3:.Nd emit formatted output based on format string and arguments xo_flush.3:.Nd emit formatted output based on format string and arguments xo_open_list.3:.Nd emit formatted output based on format string and arguments xo_set_allocator.3:.Nd emit formatted output based on format string and arguments xo_set_flags.3:.Nd emit formatted output based on format string and arguments xo_set_info.3:.Nd emit formatted output based on format string and arguments xo_set_style.3:.Nd emit formatted output based on format string and arguments xo_set_writer.3:.Nd emit formatted output based on format string and arguments Do you really believe that the Nd entries for these manpages are correct? I also found that from simple 'man ls' (etc) there is no real mention of what --libxo even is or how it works. Following the manpage cross-references leads me to have to go to a webpage to see what params --libxo even takes. The --libxo flag needs much more documentation in these manpages. -- Regards, Bryan Drewery ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On 6/14/15 3:17 AM, Garrett Cooper wrote: On Jun 14, 2015, at 3:00, Slawa Olhovchenkov s...@zxy.spb.ru wrote: On Sat, Jun 13, 2015 at 05:13:31PM -0700, Craig Rodrigues wrote: The people I talk to use scripting languages like Python or Ruby, and devops frameworks like Ansible, Saltstack, Puppet, and Chef. They may do some quick prototyping and UI work with Javascript and HTML/CSS. Being able to generate JSON directly from system-level tools, and then analyze that in a Python script, You need JSON from system-level tools for analyze that in a Python script? Realy? Not plain text? or tab/space separated? Having written a bunch of tools that parse plaintext, it’s a pain in the rear. It’s far easier to have JSON and a schema for working with that JSON when developing tools to parse things out. Programmers are inherently lazy — the more we have to make them work, the more pushback we’re going to get from them as far as integrating FreeBSD’s concerned. Thanks! For 'w' it makes sense, for 'ls' why? Most, all?, scripting languages have globbing and file listing functions already so there's no need to run /bin/ls and parse it. -- Regards, Bryan Drewery ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Mon, Jun 15, 2015 at 10:36:12AM -0600, Warner Losh wrote:
On Jun 14, 2015, at 12:53 PM, Adrian Chadd adr...@freebsd.org wrote:
I do like how zero percent of the comments are hey, maybe we need
unit tests that run these tools and ensure they output the right
stuff. If this were ${WORK} and I were ${BOSS}, I'd have asked the
libxo developers to include unit tests before/after for each thing
they broke, so we don't have a repeat of this kind of thing. But, this
apparently isn't ${WORK} and I definitely don't want to be anyones
boss, so..
gstat still produces the right output. It simply has been broken in that
you can't just hit 'q' anymore. So while necessary, it wouldn't be
sufficient.
I am about initial gstat behavior: you can't be do `gstat
output_log`, wait hour, press ^C and analyse output -- gstat exit
after one ineration. I am talk all utilites have different behavior --
some exits by pressing 'q', some only by ^C (systat), some can be infinitly
output to file|pipe, some -- not. Some have looped mode, some -- not,
some can handle multiple obejcts (gstat, iostat), some -- not (netstat
-nbI).
And more: ix show network statics in netmap mode, igb -- don't show.
No unification. This is more vital, I think.
And no tools for collect correlation data -- for example collect at
same time CPU load by core, i/o load by disk and network load by
inetrface. You may talk about Graphite, but graffit is crap:
http://grisha.org/blog/2015/05/04/recording-time-series/
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Jun 15, 2015, at 10:54 AM, Bryan Drewery bdrew...@freebsd.org wrote: On 6/14/15 3:17 AM, Garrett Cooper wrote: On Jun 14, 2015, at 3:00, Slawa Olhovchenkov s...@zxy.spb.ru wrote: On Sat, Jun 13, 2015 at 05:13:31PM -0700, Craig Rodrigues wrote: The people I talk to use scripting languages like Python or Ruby, and devops frameworks like Ansible, Saltstack, Puppet, and Chef. They may do some quick prototyping and UI work with Javascript and HTML/CSS. Being able to generate JSON directly from system-level tools, and then analyze that in a Python script, You need JSON from system-level tools for analyze that in a Python script? Realy? Not plain text? or tab/space separated? Having written a bunch of tools that parse plaintext, it’s a pain in the rear. It’s far easier to have JSON and a schema for working with that JSON when developing tools to parse things out. Programmers are inherently lazy — the more we have to make them work, the more pushback we’re going to get from them as far as integrating FreeBSD’s concerned. Thanks! For 'w' it makes sense, for 'ls' why? Most, all?, scripting languages have globbing and file listing functions already so there's no need to run /bin/ls and parse it. I’ve yet to see why ls —libxo is better than a separate program articulated anywhere other than libxo all the things.” Having a clear statement about why it is needed, why changing it vs having a separate program, etc would help. But is seems overly gratuitous with little benefit. Warner signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r284198 - head/bin/ls
On Mon, Jun 15, 2015 at 10:24:24AM -0700, Craig Rodrigues wrote: Having more tools use libxo will definitely make it easier to write tools like Eagleeye. Having more tools use libxo means having more broken tools. % w --libxo w: missing libxo option OK. Let's check the manpage. % man w SYNOPSIS w [--libxo] [-dhin] [-M core] [-N system] [user ...] --libxo doesn't take an optarg. The only other use of libxo in w(1) is the libxo(3) cross-reference. % man 3 libxo (search on --libxo) Output can then be generated in various style, using the --libxo option. The only appearance of --libxo is in the above sentence. Great. So, what are these fictitious optargs and why are they not documented? -- Steve ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Jun 14, 2015, at 12:53 PM, Adrian Chadd adr...@freebsd.org wrote:
I do like how zero percent of the comments are hey, maybe we need
unit tests that run these tools and ensure they output the right
stuff. If this were ${WORK} and I were ${BOSS}, I'd have asked the
libxo developers to include unit tests before/after for each thing
they broke, so we don't have a repeat of this kind of thing. But, this
apparently isn't ${WORK} and I definitely don't want to be anyones
boss, so..
gstat still produces the right output. It simply has been broken in that
you can’t just hit ‘q’ anymore. So while necessary, it wouldn’t be
sufficient.
Warner
signature.asc
Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r284198 - head/bin/ls
On Mon, Jun 15, 2015 at 10:12 AM, Slawa Olhovchenkov s...@zxy.spb.ru wrote: No unification. This is more vital, I think. And no tools for collect correlation data -- for example collect at same time CPU load by core, i/o load by disk and network load by inetrface. You may talk about Graphite, but graffit is crap: http://grisha.org/blog/2015/05/04/recording-time-series/ A few years ago, Alfred Perlstein wrote a tool called Eagleeye: https://github.com/splbio/eagleeye This tool did a few things: - ran various tools like netstat, vmstat, gstat - parsed the output of the various tools - produce web based reports for doing correlation of the data This was very useful for performance analysis. Since the code for Eagleeye is on GitHub, maybe some FreeBSD coders who have interest can update Eagleeye to produce output from the system with libxo. Having more tools use libxo will definitely make it easier to write tools like Eagleeye. -- Craig -- Craig ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On 6/13/15 8:40 PM, Craig Rodrigues wrote: For people building embedded products these days, storage of gigabytes and even terabytes is often available, I wish this was true. I recently was feeling good about finding a few hundred KB here and there to save space on an embedded system. -- Regards, Bryan Drewery ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Jun 14, 2015, at 3:00, Slawa Olhovchenkov s...@zxy.spb.ru wrote: On Sat, Jun 13, 2015 at 05:13:31PM -0700, Craig Rodrigues wrote: The people I talk to use scripting languages like Python or Ruby, and devops frameworks like Ansible, Saltstack, Puppet, and Chef. They may do some quick prototyping and UI work with Javascript and HTML/CSS. Being able to generate JSON directly from system-level tools, and then analyze that in a Python script, You need JSON from system-level tools for analyze that in a Python script? Realy? Not plain text? or tab/space separated? Having written a bunch of tools that parse plaintext, it’s a pain in the rear. It’s far easier to have JSON and a schema for working with that JSON when developing tools to parse things out. Programmers are inherently lazy — the more we have to make them work, the more pushback we’re going to get from them as far as integrating FreeBSD’s concerned. Thanks! signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r284198 - head/bin/ls
On Jun 13, 2015, at 23:09, Steve Kargl s...@troutmask.apl.washington.edu wrote: On Sat, Jun 13, 2015 at 06:45:57PM -0700, Craig Rodrigues wrote: On Sat, Jun 13, 2015 at 6:29 PM, Steve Kargl s...@troutmask.apl.washington.edu wrote: Are you talking about this comment you made? https://lists.freebsd.org/pipermail/freebsd-current/2015-March/054899.html I can't make heads or tails of what you wrote, other than you seemed very angry. I wasn't very angry. It's hard for me to tell, really, since you submitted patches with FUBAR in them, so you seemed pretty angry in that e-mail. FUBAR is as good as what is already there. Simply drawing attention to the people responsible for libxo's inclusion in FreeBSD that they need to fix it. I see the benefit in libxo, but I see the argument in it being unnecessary bloat as well. Do you really believe that the Nd entries for these manpages are correct? I'm not an expert on the mdoc format, so I couldn't tell you. If you can think of some patches to fix things, in the man pages, would you be able to submit the patches to Phil, and have them incorporated into the software to make it better? There isn't a public mailing list or other mechanism to submit a patch. One needs to sign up for an account. I have way too many accounts on way too many systems to sign up for a new account to send in a single patch. Github is rather straightforward, but yes.. the workflow’s a bit different: 1. Fork project. 2. Create a branch for a “topic” (i.e. fix/enhance X). 3. Open a pull request (which will open an issue). libxo is maintained at https://github.com/Juniper/libxo . I don't know if you are willing/able to submit patches to the libxo project on Github, to help fix things. That would be great if you could do that and help out. It seems that libxo developers have no interest in FreeBSD documentation. Not entirely true. They’re willing to work on the versioning problem I noted in the above issue. https://github.com/Juniper/libxo/issues/13 Those that imported/maintain libxo within FreeBSD should fix the documentation. The problem was with the manpage to github link referencing -- which can change. FWIW I’m happy they have documentation, unlike our current defacto toolchain in FreeBSD (man clang makes me sad… I keep gcc installed just so I can refer to something halfway decent): -O0 Means no optimization: this level compiles the fastest and generates the most debuggable code. -O1 Somewhere between -O0 and -O2. -O2 Moderate level of optimization which enables most optimizations. Really clang? I didn’t realize that 1 was between 0 and 2... Thanks. signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r284198 - head/bin/ls
On Sat, Jun 13, 2015 at 05:13:31PM -0700, Craig Rodrigues wrote: The people I talk to use scripting languages like Python or Ruby, and devops frameworks like Ansible, Saltstack, Puppet, and Chef. They may do some quick prototyping and UI work with Javascript and HTML/CSS. Being able to generate JSON directly from system-level tools, and then analyze that in a Python script, You need JSON from system-level tools for analyze that in a Python script? Realy? Not plain text? or tab/space separated? ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On 14/06/2015 11:00, Slawa Olhovchenkov wrote: On Sat, Jun 13, 2015 at 05:13:31PM -0700, Craig Rodrigues wrote: The people I talk to use scripting languages like Python or Ruby, and devops frameworks like Ansible, Saltstack, Puppet, and Chef. They may do some quick prototyping and UI work with Javascript and HTML/CSS. Being able to generate JSON directly from system-level tools, and then analyze that in a Python script, You need JSON from system-level tools for analyze that in a Python script? Realy? Not plain text? or tab/space separated? So, I am broadly in favour of keeping libxo -- providing the problems with its introduction are fixed. I'm not even thinking of the DevOps space, but the RD space, where Python and R are king. Man pages are small beer and can be dealt with easily. Control flow and regressions from previous functionality are another issue, and I am not for a moment going to duck out and suggest those responsible for introducing it aren't also responsible for fixing these specific issues. But I have yet to see a coherent argument here -- size(1) numbers, RSS figures etc. -- about how it allegedly adds bloat. Most of what I've seen so far is POLA, NIH resistance, and hand-wavery. If anything it helps to future proof the code as it stands, and make it easier for us to actually engineer the system for performance. I tend to look upon counter-arguments against that last point as The more we obfuscate, the harder it is to get found out that the code isn't actually that good. So, if you read my previous response to this thread, I've clearly pointed out that: there are specific problems in parsing the output of these system tools as they stand. If you don't believe this, you can have my yesterday morning/afternoon, where I was post-processing the output of 4000 individual vmstat -z and vmstat -m reports. Another 1000 this morning, with more to follow. Oh boy, I'd say to myself, I wish I had libxo! This argument is not limited to base system utilities. For example: iperf 3.x has had a similar reworking of its reporting format to include JSON. This is a massive improvement over iperf 2.x, which does not even clearly document its CSV fields; you have to read the source for that. JSON actually tags each field. This reduces the time from experiment to analysed result significantly, just because I can easily see what each god damned number means. Given, you need to read the source to understand why its naive sequencing algorithm breaks in multipath networking scenarios, but one should not have to do this just to get basic throughput results tabulated. ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Sun, Jun 14, 2015 at 11:22:03AM +0100, Bruce Simpson wrote: On 14/06/2015 11:00, Slawa Olhovchenkov wrote: On Sat, Jun 13, 2015 at 05:13:31PM -0700, Craig Rodrigues wrote: The people I talk to use scripting languages like Python or Ruby, and devops frameworks like Ansible, Saltstack, Puppet, and Chef. They may do some quick prototyping and UI work with Javascript and HTML/CSS. Being able to generate JSON directly from system-level tools, and then analyze that in a Python script, You need JSON from system-level tools for analyze that in a Python script? Realy? Not plain text? or tab/space separated? So, I am broadly in favour of keeping libxo -- providing the problems with its introduction are fixed. I'm not even thinking of the DevOps space, but the RD space, where Python and R are king. Man pages are small beer and can be dealt with easily. Control flow and regressions from previous functionality are another issue, and I am not for a moment going to duck out and suggest those responsible for introducing it aren't also responsible for fixing these specific issues. But I have yet to see a coherent argument here -- size(1) numbers, RSS figures etc. -- about how it allegedly adds bloat. Most of what I've seen so far is POLA, NIH resistance, and hand-wavery. If anything it helps to future proof the code as it stands, and make it easier for us to actually engineer the system for performance. I tend to look upon counter-arguments against that last point as The more we obfuscate, the harder it is to get found out that the code isn't actually that good. So, if you read my previous response to this thread, I've clearly pointed out that: there are specific problems in parsing the output of these system tools as they stand. If you don't believe this, you can have my yesterday morning/afternoon, where I was post-processing the output of 4000 individual vmstat -z and vmstat -m reports. Another 1000 this morning, with more to follow. Oh boy, I'd say to myself, I wish I had libxo! This argument is not limited to base system utilities. For example: iperf 3.x has had a similar reworking of its reporting format to include JSON. This is a massive improvement over iperf 2.x, which does not even clearly document its CSV fields; you have to read the source for that. JSON actually tags each field. This reduces the time from experiment to analysed result significantly, just because I can easily see what each god damned number means. Given, you need to read the source to understand why its naive sequencing algorithm breaks in multipath networking scenarios, but one should not have to do this just to get basic throughput results tabulated. Post-processing massive outputs and individual outputs is different task. Anoder one -- one-line scripts. I am don't like some aspects of ls/find outputs (for example -- date/time format changed depends of age). Some utilites already have key for script-processing (zfs -H, for example). I am don't see how libxo integrated to ls, but I see that in other utilites format changed options don't tigh integrated: # zpool list -v NAME SIZE ALLOC FREE EXPANDSZ FRAGCAP DEDUP HEALTH ALTROOT tank 46.2T 37.3M 46.2T - 0% 0% 1.00x ONLINE - mirror 2.72T 2.15M 2.72T - 0% 0% da2 - - - - - - da0 - - - - - - # zpool list -v -p NAME SIZE ALLOC FREE EXPANDSZ FRAGCAP DEDUP HEALTH ALTROOT tank 50818053046272 39088128 50818013958144 - 0% 0 1.00x ONLINE - mirror 2.72T 2.15M 2.72T - 0% 0% da2 - - - - - - da0 - - - - - - only 'total' format changed, not components. And this is more complex problem for scripting/automating that JSON/plaint text. Also, I am don't like pyton. I am use perl. For JSON parsing I am need additional modules. I am can't easy use JSON output in one-lines shell pipes -- system utilites (like sort, awk, cut and etc.) don't understand and parsed JSON. But understand tab/space/null separated stream. vmstat have slight different problem -- no easy filtering. gstat can't output in file/stream in looped mode. All this problems don't resolved by JSON output. I am don't talk about need or don't need libxo, good or bad code in libxo. I am just about needing json output for system utilites. I am see tons of utilites. Some utiltes born in FreeBSD. Some born in relative projects (NetBSD, OpenBSD). Some GNU. And etc. Who do JSONification all of this? Who ported to libxo ZFS utilites? clang oputput? Or some utilites will be JSON, some -- not? As I see, JSON output good only for Ansible and direct output to web interface. ___
Re: svn commit: r284198 - head/bin/ls
On Sat, Jun 13, 2015 at 06:45:57PM -0700, Craig Rodrigues wrote: On Sat, Jun 13, 2015 at 6:29 PM, Steve Kargl s...@troutmask.apl.washington.edu wrote: Are you talking about this comment you made? https://lists.freebsd.org/pipermail/freebsd-current/2015-March/054899.html I can't make heads or tails of what you wrote, other than you seemed very angry. I wasn't very angry. It's hard for me to tell, really, since you submitted patches with FUBAR in them, so you seemed pretty angry in that e-mail. FUBAR is as good as what is already there. Simply drawing attention to the people responsible for libxo's inclusion in FreeBSD that they need to fix it. Do you really believe that the Nd entries for these manpages are correct? I'm not an expert on the mdoc format, so I couldn't tell you. If you can think of some patches to fix things, in the man pages, would you be able to submit the patches to Phil, and have them incorporated into the software to make it better? There isn't a public mailing list or other mechanism to submit a patch. One needs to sign up for an account. I have way too many accounts on way too many systems to sign up for a new account to send in a single patch. libxo is maintained at https://github.com/Juniper/libxo . I don't know if you are willing/able to submit patches to the libxo project on Github, to help fix things. That would be great if you could do that and help out. It seems that libxo developers have no interest in FreeBSD documentation. https://github.com/Juniper/libxo/issues/13 Those that imported/maintain libxo within FreeBSD should fix the documentation. -- Steve ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Sun, Jun 14, 2015 at 11:22:03AM +0100, Bruce Simpson wrote: But I have yet to see a coherent argument here -- size(1) numbers, RSS figures etc. -- about how it allegedly adds bloat. Most of what I've seen so far is POLA, NIH resistance, and hand-wavery. It is not alleged. I actaully measured the bloat libxo caused when w(1) was converted. https://lists.freebsd.org/pipermail/freebsd-current/2015-March/054917.html Here's the bloat with ls(1) % ldd /bin/ls /bin/ls: libutil.so.9 = /lib/libutil.so.9 (0x2807d000) libncursesw.so.8 = /lib/libncursesw.so.8 (0x2808f000) libc.so.7 = /lib/libc.so.7 (0x280db000) % ll /bin/ls -r-xr-xr-x 1 root wheel - 28568 Jun 7 21:01 /bin/ls* % ldd /bin/ls /bin/ls: libutil.so.9 = /lib/libutil.so.9 (0x2807d000) libncursesw.so.8 = /lib/libncursesw.so.8 (0x2808f000) libxo.so.0 = /lib/libxo.so.0 (0x280db000) libc.so.7 = /lib/libc.so.7 (0x280ec000) laptop-kargl:kargl[204] ll /bin/ls -r-xr-xr-x 1 root wheel - 31376 Jun 14 10:06 /bin/ls* -- Steve ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Jun 13, 2015, at 8:40 PM, Craig Rodrigues rodr...@freebsd.org wrote: On Sat, Jun 13, 2015 at 5:26 PM, Steve Kargl s...@troutmask.apl.washington.edu wrote: On Sat, Jun 13, 2015 at 05:13:31PM -0700, Craig Rodrigues wrote: For people who are trying to build FreeBSD-based embedded products with modern web UI's, this is *really* useful. Given the bloat caused by libxo, which I showed in March, I don't see how people working on embedded products could be thrilled with this. Steve, For people building embedded products these days, storage of gigabytes and even terabytes is often available, Often, but not always. The bloat is another issue with libxo. It’s a problem for the embedded routers that Adrian has been pushing. It is a problem with many of the embedded boards I have. It isn’t too big of a problem with RPi and the like because SD cards are a lot easier to upgrade than chip-down NAND Flash parts. so the space increase that libxo provides is not that big a problem, at least for the last few products that I have worked on in the past few years. You are working on fairly big embedded. Warner signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r284198 - head/bin/ls
On 14/06/2015 18:10, Steve Kargl wrote: On Sun, Jun 14, 2015 at 11:22:03AM +0100, Bruce Simpson wrote: But I have yet to see a coherent argument here -- size(1) numbers, RSS figures etc. -- about how it allegedly adds bloat. Most of what I've seen so far is POLA, NIH resistance, and hand-wavery. It is not alleged. I actaully measured the bloat libxo caused when w(1) was converted. ... Here's the bloat with ls(1) ... Steve, that's still less than one 4KB page. OK, so I find it difficult to believe -- in this day and age of pipeline-saving CMOV instructions -- that the overhead is as large as ~2800 bytes, where I would have expected roughly half that. But not knowing what compile options you used, or having information about sizes (and working sets - just listing file sizes is hand waving) across the libxo modified binaries, this still doesn't give a complete picture of the relative cost of the feature. However, that's still a very modest increase, considering the architectural scope of the libxo change and what it provides. Warner suggests that for some targets it is too much, and he might have a point. But if you look at That Other Operating System, this is generally dealt with there by deploying something like BusyBox instead. I can understand why we don't want to go down that road -- in my experience, the choice of BusyBox sacrifices too much usability -- and would support a WITH_LIBXO for that reason alone. The extra bytes might even disappear in the noise after crunchgen. I think it is also fair that the people who advocated for this in the beginning (not I, though I support it in principle) and did the work (not I either, ENOTIME) should have done this work up front. I've had to do it to justify similar changes in other projects. ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
We have busybox already - it's called bsdbox, and it's in base, and
we're already using it, and yes, it does pull in libxo. :)
I'm not worried about the size increase of libxo. That's the wrong
thing to focus on. If it gets much more bloated then I'll poke people
with the big what are you thinking stick. The issues people have
aren't size related, they're this API needs improvement and we have
bugs introduced into tools.
I do like how zero percent of the comments are hey, maybe we need
unit tests that run these tools and ensure they output the right
stuff. If this were ${WORK} and I were ${BOSS}, I'd have asked the
libxo developers to include unit tests before/after for each thing
they broke, so we don't have a repeat of this kind of thing. But, this
apparently isn't ${WORK} and I definitely don't want to be anyones
boss, so..
-a
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Sun, Jun 14, 2015 at 07:46:05PM +0100, Bruce Simpson wrote: On 14/06/2015 18:10, Steve Kargl wrote: On Sun, Jun 14, 2015 at 11:22:03AM +0100, Bruce Simpson wrote: But I have yet to see a coherent argument here -- size(1) numbers, RSS figures etc. -- about how it allegedly adds bloat. Most of what I've seen so far is POLA, NIH resistance, and hand-wavery. It is not alleged. I actaully measured the bloat libxo caused when w(1) was converted. ... Here's the bloat with ls(1) ... Steve, that's still less than one 4KB page. 4KB for dynamic linking. 38KB for static linking. How many utilities are to be converted? 4KB (or 38KB) can add up quickly. But not knowing what compile options you used, or having information about sizes (and working sets - just listing file sizes is hand waving) across the libxo modified binaries, this still doesn't give a complete picture of the relative cost of the feature. Default 'make buildworld' options with CPUTYPE?=core2. So, -O2. I also tested with CFLAGS+=-static in w(1) and ls(1)'s Makefile. The numbers that I showed are from 'make buildworld'. The only difference is w(1) and ls(1) with/without the libxo change. I don't need to cook the numbers to fabricate some issue. Choose your favorite CFLAGS. Use svn to get w(1) and ls(1) with/without libxo. It's not difficult. -- Steve ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On 14 June 2015 at 10:10, Steve Kargl s...@troutmask.apl.washington.edu wrote: On Sun, Jun 14, 2015 at 11:22:03AM +0100, Bruce Simpson wrote: But I have yet to see a coherent argument here -- size(1) numbers, RSS figures etc. -- about how it allegedly adds bloat. Most of what I've seen so far is POLA, NIH resistance, and hand-wavery. It is not alleged. I actaully measured the bloat libxo caused when w(1) was converted. https://lists.freebsd.org/pipermail/freebsd-current/2015-March/054917.html Here's the bloat with ls(1) % ldd /bin/ls /bin/ls: libutil.so.9 = /lib/libutil.so.9 (0x2807d000) libncursesw.so.8 = /lib/libncursesw.so.8 (0x2808f000) libc.so.7 = /lib/libc.so.7 (0x280db000) % ll /bin/ls -r-xr-xr-x 1 root wheel - 28568 Jun 7 21:01 /bin/ls* % ldd /bin/ls /bin/ls: libutil.so.9 = /lib/libutil.so.9 (0x2807d000) libncursesw.so.8 = /lib/libncursesw.so.8 (0x2808f000) libxo.so.0 = /lib/libxo.so.0 (0x280db000) libc.so.7 = /lib/libc.so.7 (0x280ec000) laptop-kargl:kargl[204] ll /bin/ls -r-xr-xr-x 1 root wheel - 31376 Jun 14 10:06 /bin/ls* Plus 60k for the shared library. I don't mind that too much right now - it's around an extra 4k of overhead per binary right now. -adrian ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Jun 13, 2015, at 9:22 PM, Craig Rodrigues rodr...@freebsd.org wrote: On Sat, Jun 13, 2015 at 6:00 PM, Adrian Chadd adr...@freebsd.org wrote: I guarantee that no matter what you've worked on, there's approximately five orders of magnitude of shipping devices whose entire storage space can be measured in 1 digit megabytes. Each year. (And yes - there's an appreciable set of them for which freebsd boots, runs and keeps running on them.0 You can buy em too, some of them even under $60. Can FreeBSD now not run on these systems because of libxo? Yes. The root image sizes for a reasonable subset of the system now exceeds the flash that I have on my embedded arm boards. I have to spill over onto the SD card if I want to make progress. Some trimming could be done, but 11 images are quite a bit bigger than 10 images for the same software. Warner signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r284198 - head/bin/ls
On 14 June 2015 at 12:25, Craig Rodrigues rodr...@freebsd.org wrote: On Sun, Jun 14, 2015 at 11:53 AM, Adrian Chadd adr...@freebsd.org wrote: I do like how zero percent of the comments are hey, maybe we need unit tests that run these tools and ensure they output the right stuff. Actually you are wrong on that point. See: https://lists.freebsd.org/pipermail/svn-src-all/2015-June/105753.html -- Craig Woo! Glad to be wrong. -a ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Jun 14, 2015, at 11:53, Adrian Chadd adr...@freebsd.org wrote:
We have busybox already - it's called bsdbox, and it's in base, and
we're already using it, and yes, it does pull in libxo. :)
I'm not worried about the size increase of libxo. That's the wrong
thing to focus on. If it gets much more bloated then I'll poke people
with the big what are you thinking stick. The issues people have
aren't size related, they're this API needs improvement and we have
bugs introduced into tools.
I do like how zero percent of the comments are hey, maybe we need
unit tests that run these tools and ensure they output the right
stuff. If this were ${WORK} and I were ${BOSS}, I'd have asked the
libxo developers to include unit tests before/after for each thing
they broke, so we don't have a repeat of this kind of thing. But, this
apparently isn't ${WORK} and I definitely don't want to be anyones
boss, so..
Ugh. I'll write the tests ;(...
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On 14 June 2015 at 12:02, Garrett Cooper yaneurab...@gmail.com wrote:
On Jun 14, 2015, at 11:53, Adrian Chadd adr...@freebsd.org wrote:
We have busybox already - it's called bsdbox, and it's in base, and
we're already using it, and yes, it does pull in libxo. :)
I'm not worried about the size increase of libxo. That's the wrong
thing to focus on. If it gets much more bloated then I'll poke people
with the big what are you thinking stick. The issues people have
aren't size related, they're this API needs improvement and we have
bugs introduced into tools.
I do like how zero percent of the comments are hey, maybe we need
unit tests that run these tools and ensure they output the right
stuff. If this were ${WORK} and I were ${BOSS}, I'd have asked the
libxo developers to include unit tests before/after for each thing
they broke, so we don't have a repeat of this kind of thing. But, this
apparently isn't ${WORK} and I definitely don't want to be anyones
boss, so..
Ugh. I'll write the tests ;(...
I'm not your boss!
(but if you go and do it - great! thankyou! This is what should've
been neckbearded, not the rest of the thread!)
-a
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Jun 14, 2015, at 12:07, Adrian Chadd adr...@freebsd.org wrote:
On 14 June 2015 at 12:02, Garrett Cooper yaneurab...@gmail.com wrote:
On Jun 14, 2015, at 11:53, Adrian Chadd adr...@freebsd.org wrote:
We have busybox already - it's called bsdbox, and it's in base, and
we're already using it, and yes, it does pull in libxo. :)
I'm not worried about the size increase of libxo. That's the wrong
thing to focus on. If it gets much more bloated then I'll poke people
with the big what are you thinking stick. The issues people have
aren't size related, they're this API needs improvement and we have
bugs introduced into tools.
I do like how zero percent of the comments are hey, maybe we need
unit tests that run these tools and ensure they output the right
stuff. If this were ${WORK} and I were ${BOSS}, I'd have asked the
libxo developers to include unit tests before/after for each thing
they broke, so we don't have a repeat of this kind of thing. But, this
apparently isn't ${WORK} and I definitely don't want to be anyones
boss, so..
Ugh. I'll write the tests ;(...
I'm not your boss!
(but if you go and do it - great! thankyou! This is what should've
been neckbearded, not the rest of the thread!)
A part of it, yeah.. Not having manpqges still makes it annoying when I have to
figure out how things work ;(.
Next time someone else converts ANYTHING to libxo -- write tests FIRST to make
sure you're not breaking legacy behavior. If you need help figuring out how to
do that, I'll be more than happy to document it on a wiki page, with simple,
concise directions.
Thanks!
___
svn-src-head@freebsd.org mailing list
http://lists.freebsd.org/mailman/listinfo/svn-src-head
To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Jun 14, 2015, at 20:53, Julian Elischer jul...@freebsd.org wrote: On 6/14/15 10:48 AM, Adrian Chadd wrote: On 13 June 2015 at 18:22, Craig Rodrigues rodr...@freebsd.org wrote: On Sat, Jun 13, 2015 at 6:00 PM, Adrian Chadd adr...@freebsd.org wrote: I guarantee that no matter what you've worked on, there's approximately five orders of magnitude of shipping devices whose entire storage space can be measured in 1 digit megabytes. Each year. (And yes - there's an appreciable set of them for which freebsd boots, runs and keeps running on them.0 You can buy em too, some of them even under $60. Can FreeBSD now not run on these systems because of libxo? It's a tight squeeze as it is. Running in 8MB of flash (even if it's compressed) is still an exercise in what can you cut out. My point isn't that it isn't running because of libxo; my point is that arguing about embedded involving lots of storage is woefully incorrect and will continue to be until those gigabytes of storage are available for a penny. Which yes, I'm guessing will happen in my career - but it's also quite likely code bloat will continue to chase that upward. do we have a WITHOUT_LIBXO option on sources? I believe we should.. +1. I would be more than happy to implement it by stubbing out the majority of the macros to something less invasive, but it might be a bit before I do that. Thanks, signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r284198 - head/bin/ls
On 6/14/15 10:48 AM, Adrian Chadd wrote: On 13 June 2015 at 18:22, Craig Rodrigues rodr...@freebsd.org wrote: On Sat, Jun 13, 2015 at 6:00 PM, Adrian Chadd adr...@freebsd.org wrote: I guarantee that no matter what you've worked on, there's approximately five orders of magnitude of shipping devices whose entire storage space can be measured in 1 digit megabytes. Each year. (And yes - there's an appreciable set of them for which freebsd boots, runs and keeps running on them.0 You can buy em too, some of them even under $60. Can FreeBSD now not run on these systems because of libxo? It's a tight squeeze as it is. Running in 8MB of flash (even if it's compressed) is still an exercise in what can you cut out. My point isn't that it isn't running because of libxo; my point is that arguing about embedded involving lots of storage is woefully incorrect and will continue to be until those gigabytes of storage are available for a penny. Which yes, I'm guessing will happen in my career - but it's also quite likely code bloat will continue to chase that upward. do we have a WITHOUT_LIBXO option on sources? I believe we should.. -adrian ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On 6/14/15 2:40 AM, Marcel Moolenaar wrote: On Jun 13, 2015, at 1:19 PM, Julian Elischer jul...@freebsd.org wrote: On 6/13/15 11:38 PM, David Chisnall wrote: On 13 Jun 2015, at 11:17, Ian Lepore i...@freebsd.org wrote: If you would have told me a year ago that you had a simple scheme that could make 30 years of experience maintaining code for unix-like systems completely worthless I would have been skeptical, but it seems we're well on our way. There is a lot of heckling and unhelpful hyperbole in this thread. Reading the xo_emit format strings takes a little bit of getting used to, but the same is true of printf - it’s just that we’re already used to printf. The structured parts (xo_open_container, xo_close_container and friends) are clear and descriptive. The changes are fairly invasive, but the benefits are also very large for anyone who is wanting to automate administration of FreeBSD systems. If you have suggestions for how the libxo APIs could be improved, then please let us know - Phil is very reception to suggestions but objections along the lines of ‘it’s not what I’m used to and changes sometimes break things so we should never have changes’ are not helpful. David I made a suggestion for an alternate path in the previous thread. https://lists.freebsd.org/pipermail/freebsd-current/2015-March/054855.html but I have a job and kids so I can't object if I'm not listened to.. (no time to actually follow my own advice and produce working code.) Not wanting to change all programs and instead write grammars for all programs seems like a worse solution. The scope is the same (same number of programs), but since the grammars and programs are two distinct entities, it’s actually fairly hard to make sure both are changed at the same time when so needed. It’s also not at all a given that screen scrubbing is always easy enough to do that it isn’t causing some sort of problems in specific situations. If one wants to output JSON, XML and HTML you find that screen scrubbing doesn’t even give you all the information you need. It’s very natural to come to the conclusion that it’s easier to get the data from the source and skip the entire non-lossless translation phase. But once you have the framework for handling grammars you can make grammars for OTHER utilities that are not part of FreeBSD, and will NEVER EVER EVER have libxo support. And as a bonus you leave the existing programs alone entirely. I believe that in the end if you want to follow the path of automatic harvesting of data, you'll need the grammar handler anyway because all the 3rd party apps still require handling. It's a tricky task but It would be a really fascinating project. Especially tools that would look at existing output and allow the user to interactively identify tokens and landmarks of interest and generate grammars. We did this 20 years ago with scanned images.. So I'm sure it be done with ascii text. Come to think of it, there must be some out there already.. -- Marcel Moolenaar mar...@xcllnt.net ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Sun, Jun 14, 2015 at 11:53 AM, Adrian Chadd adr...@freebsd.org wrote: I do like how zero percent of the comments are hey, maybe we need unit tests that run these tools and ensure they output the right stuff. Actually you are wrong on that point. See: https://lists.freebsd.org/pipermail/svn-src-all/2015-June/105753.html -- Craig ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Sat, Jun 13, 2015 at 05:40:59PM -0700, Craig Rodrigues wrote: On Sat, Jun 13, 2015 at 5:26 PM, Steve Kargl s...@troutmask.apl.washington.edu wrote: Given the horrid state of the manpages, which I showed in March, one can only wonder about the internals of the libxo itself. Are you talking about this comment you made? https://lists.freebsd.org/pipermail/freebsd-current/2015-March/054899.html I can't make heads or tails of what you wrote, other than you seemed very angry. I wasn't very angry. I'm simply pointing out that the libxo manpages, which should document what libxo is/does, are horrible documentation. If the quality of the manpages matches the quality of library, and the brokeness that we have been witnesses bears this out, should be questioned. % cd src/contrib/libxo/libxo % grep Nd *.3 | grep formatted xo_attr.3:.Nd emit formatted output based on format string and arguments xo_create.3:.Nd emit formatted output based on format string and arguments xo_emit.3:.Nd emit formatted output based on format string and arguments xo_finish.3:.Nd emit formatted output based on format string and arguments xo_flush.3:.Nd emit formatted output based on format string and arguments xo_open_list.3:.Nd emit formatted output based on format string and arguments xo_set_allocator.3:.Nd emit formatted output based on format string and arguments xo_set_flags.3:.Nd emit formatted output based on format string and arguments xo_set_info.3:.Nd emit formatted output based on format string and arguments xo_set_style.3:.Nd emit formatted output based on format string and arguments xo_set_writer.3:.Nd emit formatted output based on format string and arguments Do you really believe that the Nd entries for these manpages are correct? -- Steve ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Sat, Jun 13, 2015 at 6:00 PM, Adrian Chadd adr...@freebsd.org wrote: I guarantee that no matter what you've worked on, there's approximately five orders of magnitude of shipping devices whose entire storage space can be measured in 1 digit megabytes. Each year. (And yes - there's an appreciable set of them for which freebsd boots, runs and keeps running on them.0 You can buy em too, some of them even under $60. Can FreeBSD now not run on these systems because of libxo? -- Craig ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
Hey Marcel, I hope that the current discussion thread doesn't dishearten you and Phil *too* much, and that you continue with the libxo work in FreeBSD. I think libxo is cool stuff. I've talked to a few people who are not FreeBSD users (they do Linux mostly), and they also think that libxo is pretty cool, because it makes it easier to obtain system-level information from FreeBSD, and analyze and present the info using the tools of the trade. The people I talk to use scripting languages like Python or Ruby, and devops frameworks like Ansible, Saltstack, Puppet, and Chef. They may do some quick prototyping and UI work with Javascript and HTML/CSS. Being able to generate JSON directly from system-level tools, and then analyze that in a Python script, or splash it on a Javascript/HTML 5 web page quite easily is really cool. For people who are trying to build FreeBSD-based embedded products with modern web UI's, this is *really* useful. It's unfortunate that some libxo commits broke some stuff. Are there ATF-style tests in either C or Bourne shell that we could add to the system as more utilities are converted over? It's not perfect, and a lot of the existing utilities do not have existing tests, so this might be tough, but anything we do which can help would be nice. Thanks for your work. -- Craig ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Sat, Jun 13, 2015 at 05:13:31PM -0700, Craig Rodrigues wrote: For people who are trying to build FreeBSD-based embedded products with modern web UI's, this is *really* useful. Given the bloat caused by libxo, which I showed in March, I don't see how people working on embedded products could be thrilled with this. Given the horrid state of the manpages, which I showed in March, one can only wonder about the internals of the libxo itself. -- Steve ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Sat, Jun 13, 2015 at 5:26 PM, Steve Kargl s...@troutmask.apl.washington.edu wrote: On Sat, Jun 13, 2015 at 05:13:31PM -0700, Craig Rodrigues wrote: For people who are trying to build FreeBSD-based embedded products with modern web UI's, this is *really* useful. Given the bloat caused by libxo, which I showed in March, I don't see how people working on embedded products could be thrilled with this. Steve, For people building embedded products these days, storage of gigabytes and even terabytes is often available, so the space increase that libxo provides is not that big a problem, at least for the last few products that I have worked on in the past few years. Given the horrid state of the manpages, which I showed in March, one can only wonder about the internals of the libxo itself. Are you talking about this comment you made? https://lists.freebsd.org/pipermail/freebsd-current/2015-March/054899.html I can't make heads or tails of what you wrote, other than you seemed very angry. -- Craig ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On 13 June 2015 at 17:58, Adrian Chadd adr...@freebsd.org wrote: Hi craig, I guarantee that no matter what you've worked on, there's approximately five orders of magnitude of shipping devices whose entire storage space can be measured in 1 digit megabytes. Each year. (And yes - there's an appreciable set of them for which freebsd boots, runs and keeps running on them.0 You can buy em too, some of them even under $60. -adrian ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Sat, Jun 13, 2015 at 6:29 PM, Steve Kargl s...@troutmask.apl.washington.edu wrote: Are you talking about this comment you made? https://lists.freebsd.org/pipermail/freebsd-current/2015-March/054899.html I can't make heads or tails of what you wrote, other than you seemed very angry. I wasn't very angry. It's hard for me to tell, really, since you submitted patches with FUBAR in them, so you seemed pretty angry in that e-mail. I'm simply pointing out that the libxo manpages, which should document what libxo is/does, are horrible documentation. If the quality of the manpages matches the quality of library, and the brokeness that we have been witnesses bears this out, should be questioned. I can't comment on the quality of the code, since I haven't looked at it in detail. I have read the discussions on arch@, and understand what libxo is trying to accomplish, and am eager to have this because I will definitely use it in stuff I am working on. I have also worked with Marcel and Phil, and respect them both quite a bit. It's a shame that some of the libxo commits broke some things, but if I was doing this type of work, I would probably make some mistakes and break things too, since this is low-level stuff covering many utilities in Unix. Hopefully, we can fix these things can quickly, and we move on. Do you really believe that the Nd entries for these manpages are correct? I'm not an expert on the mdoc format, so I couldn't tell you. If you can think of some patches to fix things, in the man pages, would you be able to submit the patches to Phil, and have them incorporated into the software to make it better? libxo is maintained at https://github.com/Juniper/libxo . I don't know if you are willing/able to submit patches to the libxo project on Github, to help fix things. That would be great if you could do that and help out. -- Craig ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On 06/13/2015 15:16, Gleb Smirnoff wrote: On Sat, Jun 13, 2015 at 11:38:11AM -0400, David Chisnall wrote: D If you would have told me a year ago that you had a simple scheme that D could make 30 years of experience maintaining code for unix-like systems D completely worthless I would have been skeptical, but it seems we're D well on our way. D D There is a lot of heckling and unhelpful hyperbole in this thread. Reading the xo_emit format strings takes a little bit of getting used to, but the same is true of printf - it’s just that we’re already used to printf. The structured parts (xo_open_container, xo_close_container and friends) are clear and descriptive. The changes are fairly invasive, but the benefits are also very large for anyone who is wanting to automate administration of FreeBSD systems. D D If you have suggestions for how the libxo APIs could be improved, then please let us know - Phil is very reception to suggestions but objections along the lines of ‘it’s not what I’m used to and changes sometimes break things so we should never have changes’ are not helpful. I would agree with David. After xo_emit format is learned, reading sources of converted programs isn't a big deal. All the problems with converted utilities are due to very bad quality of initial conversion commits. Can this whole conversation be moved out of the commits list and onto arch@ where it belongs? Best, George ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
Hi craig, I guarantee that no matter what you've worked on, there's approximately five orders of magnitude of shipping devices whose entire storage space can be measured in 1 digit megabytes. Each year. -adrian ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On 13 June 2015 at 18:22, Craig Rodrigues rodr...@freebsd.org wrote: On Sat, Jun 13, 2015 at 6:00 PM, Adrian Chadd adr...@freebsd.org wrote: I guarantee that no matter what you've worked on, there's approximately five orders of magnitude of shipping devices whose entire storage space can be measured in 1 digit megabytes. Each year. (And yes - there's an appreciable set of them for which freebsd boots, runs and keeps running on them.0 You can buy em too, some of them even under $60. Can FreeBSD now not run on these systems because of libxo? It's a tight squeeze as it is. Running in 8MB of flash (even if it's compressed) is still an exercise in what can you cut out. My point isn't that it isn't running because of libxo; my point is that arguing about embedded involving lots of storage is woefully incorrect and will continue to be until those gigabytes of storage are available for a penny. Which yes, I'm guessing will happen in my career - but it's also quite likely code bloat will continue to chase that upward. -adrian ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Jun 13, 2015, at 10:26 AM, Julian Elischer jul...@freebsd.org wrote: On 6/13/15 10:49 AM, Steve Kargl wrote: On Fri, Jun 12, 2015 at 08:43:09PM -0400, Alexander Kabaev wrote: On Wed, 10 Jun 2015 01:27:39 + (UTC) Marcel Moolenaar mar...@freebsd.org wrote: Author: marcel Date: Wed Jun 10 01:27:38 2015 New Revision: 284198 URL: https://svnweb.freebsd.org/changeset/base/284198 Log: Convert ls(1) to use libxo(3). Obtained from:Phil Shafer p...@juniper.net Sponsored by:Juniper Networks, Inc. SKIP This broke all code that pipes output of the ls command to pipeline, such as 'ls | wc -l'. ls never exits and never output anything. Is there any purpose to libxo other than breaking stuff, which it achieves so splendidly? -1 for libxo, which also makes code almost unreadable. +1 of the -1 my personal vote is to revert all libxo changes and banish it from /usr/src. not the way to solve the problem in question. It isn’t even wrong…. I think that we shouldn’t integrate any more libxo stuff until all the known bugs in the stuff that’s already been converted is fixed. For example, gstat’s ‘q’ function now needs a bleeping carriage return before it will quit. That’s insane. And the twisty maze of modifications has made it rather an uber-pita to figure out WTF I need to do to un-F this up. But back to the topic at hand. libxo for ls? Really? WTF were you thinking? I know the cat -v paper is a bit of an extreme viewpoint, but all the libxo integration can be used a poster child for Pike’s worries… Warner signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r284198 - head/bin/ls
On 6/13/15 10:49 AM, Steve Kargl wrote: On Fri, Jun 12, 2015 at 08:43:09PM -0400, Alexander Kabaev wrote: On Wed, 10 Jun 2015 01:27:39 + (UTC) Marcel Moolenaar mar...@freebsd.org wrote: Author: marcel Date: Wed Jun 10 01:27:38 2015 New Revision: 284198 URL: https://svnweb.freebsd.org/changeset/base/284198 Log: Convert ls(1) to use libxo(3). Obtained from: Phil Shafer p...@juniper.net Sponsored by:Juniper Networks, Inc. SKIP This broke all code that pipes output of the ls command to pipeline, such as 'ls | wc -l'. ls never exits and never output anything. Is there any purpose to libxo other than breaking stuff, which it achieves so splendidly? -1 for libxo, which also makes code almost unreadable. +1 of the -1 my personal vote is to revert all libxo changes and banish it from /usr/src. not the way to solve the problem in question. ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Sat, 2015-06-13 at 11:38 -0400, David Chisnall wrote: On 13 Jun 2015, at 11:17, Ian Lepore i...@freebsd.org wrote: If you would have told me a year ago that you had a simple scheme that could make 30 years of experience maintaining code for unix-like systems completely worthless I would have been skeptical, but it seems we're well on our way. There is a lot of heckling and unhelpful hyperbole in this thread. Reading the xo_emit format strings takes a little bit of getting used to, but the same is true of printf - it’s just that we’re already used to printf. The structured parts (xo_open_container, xo_close_container and friends) are clear and descriptive. The changes are fairly invasive, but the benefits are also very large for anyone who is wanting to automate administration of FreeBSD systems. If you have suggestions for how the libxo APIs could be improved, then please let us know - Phil is very reception to suggestions but objections along the lines of ‘it’s not what I’m used to and changes sometimes break things so we should never have changes’ are not helpful. This is a piece of crap that needs to be excised from the system and done a different way is useful input whether you agree with it or not. The idea that someone does not have the right to point out a huge mistake simply because they don't have a patchset in hand is pure BS. But, this is what you get when a disagreement about design is solved by someone pointing out that project policy has always been he who commits first wins the design discussion and that's pretty much what happened when all of this was being discussed. -- Ian ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
I think the experience so far has been ok, this way isn't working well, maybe it's time we made all our statistics gathering binaries thin layers on top of .so 's that implement the variation in output. It's been valuable to (re)learn that, but .. -adrian ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On 13 Jun 2015, at 11:17, Ian Lepore i...@freebsd.org wrote: If you would have told me a year ago that you had a simple scheme that could make 30 years of experience maintaining code for unix-like systems completely worthless I would have been skeptical, but it seems we're well on our way. There is a lot of heckling and unhelpful hyperbole in this thread. Reading the xo_emit format strings takes a little bit of getting used to, but the same is true of printf - it’s just that we’re already used to printf. The structured parts (xo_open_container, xo_close_container and friends) are clear and descriptive. The changes are fairly invasive, but the benefits are also very large for anyone who is wanting to automate administration of FreeBSD systems. If you have suggestions for how the libxo APIs could be improved, then please let us know - Phil is very reception to suggestions but objections along the lines of ‘it’s not what I’m used to and changes sometimes break things so we should never have changes’ are not helpful. David ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Jun 13, 2015, at 11:47 AM, Ian Lepore i...@freebsd.org wrote: On Sat, 2015-06-13 at 11:38 -0400, David Chisnall wrote: On 13 Jun 2015, at 11:17, Ian Lepore i...@freebsd.org wrote: If you would have told me a year ago that you had a simple scheme that could make 30 years of experience maintaining code for unix-like systems completely worthless I would have been skeptical, but it seems we're well on our way. There is a lot of heckling and unhelpful hyperbole in this thread. Reading the xo_emit format strings takes a little bit of getting used to, but the same is true of printf - it’s just that we’re already used to printf. The structured parts (xo_open_container, xo_close_container and friends) are clear and descriptive. The changes are fairly invasive, but the benefits are also very large for anyone who is wanting to automate administration of FreeBSD systems. If you have suggestions for how the libxo APIs could be improved, then please let us know - Phil is very reception to suggestions but objections along the lines of ‘it’s not what I’m used to and changes sometimes break things so we should never have changes’ are not helpful. This is a piece of crap that needs to be excised from the system and done a different way is useful input whether you agree with it or not. Actually: no. Not only does one not demonstrate an understanding of the problem by calling it “crap” and thus leaving the recipient to wonder whether it’s worth his or her time to even respond; the sentence also lack a concrete suggestion and, last but not least, is utter after this was all discussed on arch@, making it very much one of “too little, too late”. So, not useful at all. -- Marcel Moolenaar mar...@xcllnt.net signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r284198 - head/bin/ls
On Jun 13, 2015, at 12:35 PM, Adrian Chadd adr...@freebsd.org wrote: Hi, I think we're at the point now where it's worth doing that re-evaluation. I don't think it's worth backing everything out; just whether the current approach of overriding printing the way it's done is the right way. So, how about that happens nowish before more things are converted? What do you suggest we do instead? -- Marcel Moolenaar mar...@xcllnt.net signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r284198 - head/bin/ls
On 13/06/2015 16:38, David Chisnall wrote: On 13 Jun 2015, at 11:17, Ian Lepore i...@freebsd.org wrote: If you would have told me a year ago that you had a simple scheme that could make 30 years of experience maintaining code for unix-like systems completely worthless I would have been skeptical, but it seems we're well on our way. There is a lot of heckling and unhelpful hyperbole in this thread. Agree -- teething trouble. I am trying to push something out right now based on 8.x, and all the little gotchas (e.g. vmstat -z not using a uniform delimiter set) are like little landmines on the highway in front of my Pursuit Special. Personally I prefer the new formats, they're less ambiguous, and closer to the analytics where they actually get used. ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Fri, 2015-06-12 at 19:49 -0700, Steve Kargl wrote: On Fri, Jun 12, 2015 at 08:43:09PM -0400, Alexander Kabaev wrote: On Wed, 10 Jun 2015 01:27:39 + (UTC) Marcel Moolenaar mar...@freebsd.org wrote: Author: marcel Date: Wed Jun 10 01:27:38 2015 New Revision: 284198 URL: https://svnweb.freebsd.org/changeset/base/284198 Log: Convert ls(1) to use libxo(3). Obtained from: Phil Shafer p...@juniper.net Sponsored by: Juniper Networks, Inc. SKIP This broke all code that pipes output of the ls command to pipeline, such as 'ls | wc -l'. ls never exits and never output anything. Is there any purpose to libxo other than breaking stuff, which it achieves so splendidly? -1 for libxo, which also makes code almost unreadable. s/unreadable/unmaintainable/ If you would have told me a year ago that you had a simple scheme that could make 30 years of experience maintaining code for unix-like systems completely worthless I would have been skeptical, but it seems we're well on our way. -- Ian ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
Hi, I think we're at the point now where it's worth doing that re-evaluation. I don't think it's worth backing everything out; just whether the current approach of overriding printing the way it's done is the right way. So, how about that happens nowish before more things are converted? -adrian ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On 6/13/15 11:38 PM, David Chisnall wrote: On 13 Jun 2015, at 11:17, Ian Lepore i...@freebsd.org wrote: If you would have told me a year ago that you had a simple scheme that could make 30 years of experience maintaining code for unix-like systems completely worthless I would have been skeptical, but it seems we're well on our way. There is a lot of heckling and unhelpful hyperbole in this thread. Reading the xo_emit format strings takes a little bit of getting used to, but the same is true of printf - it’s just that we’re already used to printf. The structured parts (xo_open_container, xo_close_container and friends) are clear and descriptive. The changes are fairly invasive, but the benefits are also very large for anyone who is wanting to automate administration of FreeBSD systems. If you have suggestions for how the libxo APIs could be improved, then please let us know - Phil is very reception to suggestions but objections along the lines of ‘it’s not what I’m used to and changes sometimes break things so we should never have changes’ are not helpful. David I made a suggestion for an alternate path in the previous thread. https://lists.freebsd.org/pipermail/freebsd-current/2015-March/054855.html but I have a job and kids so I can't object if I'm not listened to.. (no time to actually follow my own advice and produce working code.) ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On 13 June 2015 at 10:06, Marcel Moolenaar mar...@xcllnt.net wrote: On Jun 13, 2015, at 12:35 PM, Adrian Chadd adr...@freebsd.org wrote: Hi, I think we're at the point now where it's worth doing that re-evaluation. I don't think it's worth backing everything out; just whether the current approach of overriding printing the way it's done is the right way. So, how about that happens nowish before more things are converted? What do you suggest we do instead? So a lot of these things are table driven. Having manual printing for tabular data is plain stupid. The libbsdstat library for doing basic statistics output with now and time series is what I'm playing with right now. It's only used by sam's wifi utilities, but I'm going to try and extend it for other utilities too (like netstat, vmstat, etc style output.) That way what's expressed in code is organised as such: * a bit of code fetches statistics * a bit of code sets up what the name of each field is, and what the units are * a bit of code handles any odd corner cases with data representations * libbsdstat takes care of recording the samples into the time-series or 'now' section, figuring out which fields need to be printed in which order, what the formatting is, etc. * .. I'm extending it to print out json for its table outputs versus just plain text. Having arbitrary formatting, arbitrary printing, random places where statistics are fetched, etc is actually the terrible problem that we could do better, without losing our minds by overcomplicating it with C++, templating, grammars, etc - and ending up with what look like five layers of nested java classes that do Setter(ToString(Getter()) in layers. -adrian ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Sat, Jun 13, 2015 at 11:38:11AM -0400, David Chisnall wrote: D If you would have told me a year ago that you had a simple scheme that D could make 30 years of experience maintaining code for unix-like systems D completely worthless I would have been skeptical, but it seems we're D well on our way. D D There is a lot of heckling and unhelpful hyperbole in this thread. Reading the xo_emit format strings takes a little bit of getting used to, but the same is true of printf - it’s just that we’re already used to printf. The structured parts (xo_open_container, xo_close_container and friends) are clear and descriptive. The changes are fairly invasive, but the benefits are also very large for anyone who is wanting to automate administration of FreeBSD systems. D D If you have suggestions for how the libxo APIs could be improved, then please let us know - Phil is very reception to suggestions but objections along the lines of ‘it’s not what I’m used to and changes sometimes break things so we should never have changes’ are not helpful. I would agree with David. After xo_emit format is learned, reading sources of converted programs isn't a big deal. All the problems with converted utilities are due to very bad quality of initial conversion commits. -- Totus tuus, Glebius. ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Jun 13, 2015, at 12:25 PM, Marcel Moolenaar mar...@xcllnt.net wrote: On Jun 13, 2015, at 11:47 AM, Ian Lepore i...@freebsd.org wrote: On Sat, 2015-06-13 at 11:38 -0400, David Chisnall wrote: On 13 Jun 2015, at 11:17, Ian Lepore i...@freebsd.org wrote: If you would have told me a year ago that you had a simple scheme that could make 30 years of experience maintaining code for unix-like systems completely worthless I would have been skeptical, but it seems we're well on our way. There is a lot of heckling and unhelpful hyperbole in this thread. Reading the xo_emit format strings takes a little bit of getting used to, but the same is true of printf - it’s just that we’re already used to printf. The structured parts (xo_open_container, xo_close_container and friends) are clear and descriptive. The changes are fairly invasive, but the benefits are also very large for anyone who is wanting to automate administration of FreeBSD systems. If you have suggestions for how the libxo APIs could be improved, then please let us know - Phil is very reception to suggestions but objections along the lines of ‘it’s not what I’m used to and changes sometimes break things so we should never have changes’ are not helpful. This is a piece of crap that needs to be excised from the system and done a different way is useful input whether you agree with it or not. Actually: no. Not only does one not demonstrate an understanding of the problem by calling it “crap” and thus leaving the recipient to wonder whether it’s worth his or her time to even respond; the sentence also lack a concrete suggestion and, last but not least, is utter after this was all discussed on arch@, making it very much one of “too little, too late”. So, not useful at all. My complaints have been specific: libxo conversion broke things, but didn’t fix them before going on to convert more things (which broke more things). This suggests a lack of competent testing as a standard operating procedure and pointing it out is helpful. And specifically about ls: it was already way overloaded. Overloading it further seemed to be unwise: a new program would have been better since it is a thin interface to fts(3). I didn’t recall seeing a specific discussion about ls, but the original thread in arch grew to be quite large and maybe I missed something. While I dislike libxo in general, I do understand why it is being done. I see the use in general, and the benefits. I have nothing better to offer. I object to the execution in small aspects. Warner signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r284198 - head/bin/ls
On Jun 13, 2015, at 1:19 PM, Julian Elischer jul...@freebsd.org wrote: On 6/13/15 11:38 PM, David Chisnall wrote: On 13 Jun 2015, at 11:17, Ian Lepore i...@freebsd.org wrote: If you would have told me a year ago that you had a simple scheme that could make 30 years of experience maintaining code for unix-like systems completely worthless I would have been skeptical, but it seems we're well on our way. There is a lot of heckling and unhelpful hyperbole in this thread. Reading the xo_emit format strings takes a little bit of getting used to, but the same is true of printf - it’s just that we’re already used to printf. The structured parts (xo_open_container, xo_close_container and friends) are clear and descriptive. The changes are fairly invasive, but the benefits are also very large for anyone who is wanting to automate administration of FreeBSD systems. If you have suggestions for how the libxo APIs could be improved, then please let us know - Phil is very reception to suggestions but objections along the lines of ‘it’s not what I’m used to and changes sometimes break things so we should never have changes’ are not helpful. David I made a suggestion for an alternate path in the previous thread. https://lists.freebsd.org/pipermail/freebsd-current/2015-March/054855.html but I have a job and kids so I can't object if I'm not listened to.. (no time to actually follow my own advice and produce working code.) Not wanting to change all programs and instead write grammars for all programs seems like a worse solution. The scope is the same (same number of programs), but since the grammars and programs are two distinct entities, it’s actually fairly hard to make sure both are changed at the same time when so needed. It’s also not at all a given that screen scrubbing is always easy enough to do that it isn’t causing some sort of problems in specific situations. If one wants to output JSON, XML and HTML you find that screen scrubbing doesn’t even give you all the information you need. It’s very natural to come to the conclusion that it’s easier to get the data from the source and skip the entire non-lossless translation phase. -- Marcel Moolenaar mar...@xcllnt.net signature.asc Description: Message signed with OpenPGP using GPGMail
Re: svn commit: r284198 - head/bin/ls
On Sun, Jun 14, 2015 at 01:19:27AM +0800, Julian Elischer wrote: On 6/13/15 11:38 PM, David Chisnall wrote: On 13 Jun 2015, at 11:17, Ian Lepore i...@freebsd.org wrote: If you would have told me a year ago that you had a simple scheme that could make 30 years of experience maintaining code for unix-like systems completely worthless I would have been skeptical, but it seems we're well on our way. There is a lot of heckling and unhelpful hyperbole in this thread. Reading the xo_emit format strings takes a little bit of getting used to, but the same is true of printf - it???s just that we???re already used to printf. The structured parts (xo_open_container, xo_close_container and friends) are clear and descriptive. The changes are fairly invasive, but the benefits are also very large for anyone who is wanting to automate administration of FreeBSD systems. If you have suggestions for how the libxo APIs could be improved, then please let us know - Phil is very reception to suggestions but objections along the lines of ???it???s not what I???m used to and changes sometimes break things so we should never have changes??? are not helpful. David I made a suggestion for an alternate path in the previous thread. https://lists.freebsd.org/pipermail/freebsd-current/2015-March/054855.html but I have a job and kids so I can't object if I'm not listened to.. (no time to actually follow my own advice and produce working code.) I also pointed out the bloat https://lists.freebsd.org/pipermail/freebsd-current/2015-March/054917.html and the poor documentation https://lists.freebsd.org/pipermail/freebsd-current/2015-March/054899.html -- Steve ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On 13.06.2015 17:26, Julian Elischer wrote: On 6/13/15 10:49 AM, Steve Kargl wrote: On Fri, Jun 12, 2015 at 08:43:09PM -0400, Alexander Kabaev wrote: On Wed, 10 Jun 2015 01:27:39 + (UTC) Marcel Moolenaar mar...@freebsd.org wrote: Author: marcel Date: Wed Jun 10 01:27:38 2015 New Revision: 284198 URL: https://svnweb.freebsd.org/changeset/base/284198 Log: Convert ls(1) to use libxo(3). Obtained from:Phil Shafer p...@juniper.net Sponsored by:Juniper Networks, Inc. SKIP This broke all code that pipes output of the ls command to pipeline, such as 'ls | wc -l'. ls never exits and never output anything. Is there any purpose to libxo other than breaking stuff, which it achieves so splendidly? -1 for libxo, which also makes code almost unreadable. +1 of the -1 my personal vote is to revert all libxo changes and banish it from /usr/src. I already express my opinion in another thread, so just repeat it here. When libxo starts to break things at very early stage, I perceive the rest of the way to hell. Proper way to do this thing is to back out all changes and write completely separate templates-based parser - xml/json/etc. writer. If somebody love libxo code, use it in that parser/writer. -- http://ache.vniz.net/ -- http://vniz.net/ ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Fri, 12 Jun 2015 20:43:09 -0400 Alexander Kabaev kab...@gmail.com wrote: On Wed, 10 Jun 2015 01:27:39 + (UTC) Marcel Moolenaar mar...@freebsd.org wrote: Author: marcel Date: Wed Jun 10 01:27:38 2015 New Revision: 284198 URL: https://svnweb.freebsd.org/changeset/base/284198 Log: Convert ls(1) to use libxo(3). Obtained from:Phil Shafer p...@juniper.net Sponsored by: Juniper Networks, Inc. SKIP This broke all code that pipes output of the ls command to pipeline, such as 'ls | wc -l'. ls never exits and never output anything. Is there any purpose to libxo other than breaking stuff, which it achieves so splendidly? Just to clarify, this happens because libxo cannot display file names in encodings current locate cannot handle. xo_format_string_direct function then spins indefinitely trying to call xo_failure(xop, invalid mbs char: %02hhx, *cp) over and over, which, of course, produces nothing and does not advance the cp pointer either, resulting in apparent ls hang. -- Alexander Kabaev pgpiuI2Xmlwdd.pgp Description: OpenPGP digital signature
Re: svn commit: r284198 - head/bin/ls
On Wed, 10 Jun 2015 01:27:39 + (UTC) Marcel Moolenaar mar...@freebsd.org wrote: Author: marcel Date: Wed Jun 10 01:27:38 2015 New Revision: 284198 URL: https://svnweb.freebsd.org/changeset/base/284198 Log: Convert ls(1) to use libxo(3). Obtained from: Phil Shafer p...@juniper.net Sponsored by: Juniper Networks, Inc. SKIP This broke all code that pipes output of the ls command to pipeline, such as 'ls | wc -l'. ls never exits and never output anything. Is there any purpose to libxo other than breaking stuff, which it achieves so splendidly? -- Alexander Kabaev pgpAWcz46WIBs.pgp Description: OpenPGP digital signature
Re: svn commit: r284198 - head/bin/ls
On Fri, Jun 12, 2015 at 08:43:09PM -0400, Alexander Kabaev wrote: On Wed, 10 Jun 2015 01:27:39 + (UTC) Marcel Moolenaar mar...@freebsd.org wrote: Author: marcel Date: Wed Jun 10 01:27:38 2015 New Revision: 284198 URL: https://svnweb.freebsd.org/changeset/base/284198 Log: Convert ls(1) to use libxo(3). Obtained from:Phil Shafer p...@juniper.net Sponsored by: Juniper Networks, Inc. SKIP This broke all code that pipes output of the ls command to pipeline, such as 'ls | wc -l'. ls never exits and never output anything. Is there any purpose to libxo other than breaking stuff, which it achieves so splendidly? -1 for libxo, which also makes code almost unreadable. -- Steve ___ svn-src-head@freebsd.org mailing list http://lists.freebsd.org/mailman/listinfo/svn-src-head To unsubscribe, send any mail to svn-src-head-unsubscr...@freebsd.org
Re: svn commit: r284198 - head/bin/ls
On Jun 9, 2015, at 18:27, Marcel Moolenaar mar...@freebsd.org wrote: Author: marcel Date: Wed Jun 10 01:27:38 2015 New Revision: 284198 URL: https://svnweb.freebsd.org/changeset/base/284198 Log: Convert ls(1) to use libxo(3). Obtained from: Phil Shafer p...@juniper.net Sponsored by:Juniper Networks, Inc. Modified: head/bin/ls/Makefile head/bin/ls/extern.h head/bin/ls/ls.1 head/bin/ls/ls.c head/bin/ls/print.c head/bin/ls/util.c Hi, This broke the build with libexec/ftpd because the Makefile doesn’t reference libxo: https://jenkins.freebsd.org/job/FreeBSD_HEAD/2847/console . Thanks, -NGie signature.asc Description: Message signed with OpenPGP using GPGMail
