Re: new: yq (jq wrapper for YAML/XML documents)

2019-11-05 Thread Klemens Nanni
On Tue, Nov 05, 2019 at 08:59:56AM -0500, Kurt Mosiejczuk wrote:
> I think I was confused by the stderr spray by the tests. They *look* like
> they fail, but the exit code is 0. I suppose it might be worth it to put a
> comment in the Makefile mentioning the messiness so others don't think the
> tests are failing when they aren't.
Comment added.

> I might tweak DESCR slightly on the subject of xq. yq is the package name
> and the name of an executable from the package. The description of xq in
> there currently is accurate, but looks like it is part of the DESCR from
> another package.
I see what you mean but fail to find the right wording to fix this, so
I've import with DESCR as is.

Thanks



Re: new: yq (jq wrapper for YAML/XML documents)

2019-11-03 Thread Klemens Nanni
On Fri, Oct 04, 2019 at 01:44:10PM +0200, Klemens Nanni wrote:
> rsadowski noted that DESCR is too lenghty and upon second read I agree
> that my shameless dump of the project's README is too much.
> 
> Nonetheless I'd like to provide more information than
> 
>   yq is a command line processor for Yaml/Xml documents,
>   similar to jq for Json.
> 
> for the user's convenience, so here's a second attempt that reduces
> DESCR into what I think is still relevant;  no other changes in the new
> tarball except a whitespace fix in Makefile.
> 
> Information for inst:yq-2.7.2
> 
> Comment:
> jq wrapper for YAML/XML documents
> 
> Description:
> yq takes YAML input, converts it to JSON, and filters it through jq(1).
> 
> No output conversion is done by default.  Command line arguments are 
> forwarded.
> yq exits 1 on YAML parsing errors, otherwise jq's exit code is forwarded.
> 
> YAML tags in the input are ignored (any nested data is treated as untagged).
> Key order is preserved.
> 
> xq behaves like yq but takes XML input.  Multiple XML documents can be passed
> in separate files/streams.  Entity expansion and DTD resolution is disabled to
> avoid XML parsing vulnerabilities.
> 
> Maintainer: Klemens Nanni 
> 
> WWW: https://yq.readthedocs.io/
kmos reported different (failing?) test results but I could not
reproduce any of it;  see attached the same tarball again as well as my
test.log.

Feedback?


yq.tgz
Description: Binary data
/usr/local/bin/python3.7 /usr/ports/pobj/yq-2.7.2/yq-2.7.2/test/test.py
.[
  "2016-12-20T22:07:36"
]
"2016-12-20"
.{}
..40333
x{}
."bar"
.{
  "foo": null
}
..jq - commandline JSON processor [version 1.6]

Usage:  jq [options]  [file...]
jq [options] --args  [strings...]
jq [options] --jsonargs  [JSON_TEXTS...]

jq is a tool for processing JSON inputs, applying the given filter to
its JSON text inputs and producing the filter's results as JSON on
standard output.

The simplest filter is ., which copies jq's input to its output
unmodified (except for formatting, but note that IEEE754 is used
for number representation internally, with all that that implies).

For more advanced filters see the jq(1) manpage ("man jq")
and/or https://stedolan.github.io/jq

Example:

$ echo '{"foo": 0}' | jq .
{
"foo": 0
}

Some of the options include:
  -c   compact instead of pretty-printed output;
  -n   use `null` as the single input value;
  -e   set the exit status code based on the output;
  -s   read (slurp) all inputs into an array; apply filter to it;
  -r   output raw strings, not JSON texts;
  -R   read raw strings, not JSON texts;
  -C   colorize JSON;
  -M   monochrome (don't colorize JSON);
  -S   sort keys of objects on output;
  --tabuse tabs for indentation;
  --arg a vset variable $a to value ;
  --argjson a vset variable $a to JSON value ;
  --slurpfile a f  set variable $a to an array of JSON texts read from ;
  --rawfile a fset variable $a to a string consisting of the contents of 
;
  --args   remaining arguments are string arguments, not files;
  --jsonargs   remaining arguments are JSON arguments, not files;
  --   terminates argument processing;

Named arguments are also available as $ARGS.named[], while
positional arguments are available as $ARGS.positional[].

See the manpage for more options.
jq - commandline JSON processor [version 1.6]

Usage:  jq [options]  [file...]
jq [options] --args  [strings...]
jq [options] --jsonargs  [JSON_TEXTS...]

jq is a tool for processing JSON inputs, applying the given filter to
its JSON text inputs and producing the filter's results as JSON on
standard output.

The simplest filter is ., which copies jq's input to its output
unmodified (except for formatting, but note that IEEE754 is used
for number representation internally, with all that that implies).

For more advanced filters see the jq(1) manpage ("man jq")
and/or https://stedolan.github.io/jq

Example:

$ echo '{"foo": 0}' | jq .
{
"foo": 0
}

Some of the options include:
  -c   compact instead of pretty-printed output;
  -n   use `null` as the single input value;
  -e   set the exit status code based on the output;
  -s   read (slurp) all inputs into an array; apply filter to it;
  -r   output raw strings, not JSON texts;
  -R   read raw strings, not JSON texts;
  -C   colorize JSON;
  -M   monochrome (don't colorize JSON);
  -S   sort keys of objects on output;
  --tabuse tabs for indentation;
  --arg a vset variable $a to value ;
  --argjson a vset variable $a to JSON value ;
  --slurpfile a f  set variable $a to an array of JSON texts read from ;
  --rawfile a fset variable 

Re: new: yq (jq wrapper for YAML/XML documents)

2019-10-04 Thread Klemens Nanni
rsadowski noted that DESCR is too lenghty and upon second read I agree
that my shameless dump of the project's README is too much.

Nonetheless I'd like to provide more information than

yq is a command line processor for Yaml/Xml documents,
similar to jq for Json.

for the user's convenience, so here's a second attempt that reduces
DESCR into what I think is still relevant;  no other changes in the new
tarball except a whitespace fix in Makefile.

Information for inst:yq-2.7.2

Comment:
jq wrapper for YAML/XML documents

Description:
yq takes YAML input, converts it to JSON, and filters it through jq(1).

No output conversion is done by default.  Command line arguments are forwarded.
yq exits 1 on YAML parsing errors, otherwise jq's exit code is forwarded.

YAML tags in the input are ignored (any nested data is treated as untagged).
Key order is preserved.

xq behaves like yq but takes XML input.  Multiple XML documents can be passed
in separate files/streams.  Entity expansion and DTD resolution is disabled to
avoid XML parsing vulnerabilities.

Maintainer: Klemens Nanni 

WWW: https://yq.readthedocs.io/


OK?


yq.tgz
Description: Binary data


new: yq (jq wrapper for YAML/XML documents)

2019-10-04 Thread Klemens Nanni
Here's a simple python port that I use a lot at work which could be
handy for others as well, hence I'd like to see it ship with 6.6.

Information for inst:yq-2.7.2

Comment:
jq wrapper for YAML/XML documents

Description:
yq takes YAML input, converts it to JSON, and pipes it to jq:
cat input.yml | yq .foo.bar

Like in jq, you can also specify input filename(s) as arguments:
yq .foo.bar input.yml

By default, no conversion of jq output is done. Use the --yaml-output/-y
argument to convert it back into YAML:
cat input.yml | yq -y .foo.bar

Use the --width/-w argument to pass the line wrap width for string literals.
All other command line arguments are forwarded to jq.  yq forwards the exit
code jq produced, unless there was an error in YAML parsing, in which case
the exit code is 1. See the jq manual for more details on jq features and
options.

YAML tags in the input are ignored (any nested data is treated as untagged).
Key order is preserved.

XML support yq also supports XML. The yq package installs an executable, xq,
which transcodes XML to JSON using xmltodict and pipes it to jq. Roundtrip
transcoding is available with the xq --xml-output/xq -x option. Multiple XML
documents can be passed in separate files/streams as xq a.xml b.xml. Entity
expansion and DTD resolution is disabled to avoid XML parsing vulnerabilities.

Maintainer: Klemens Nanni 

WWW: https://yq.readthedocs.io/


Note that this is not the same as https://github.com/mikefarah/yq/, some
pure Go implementation.  I deliberately picked the Python implementation
at https://github.com/kislyuk/yq/ because it reuses the excellent jq(1)
which really just works.

Regress passes all tests, but only after doing it by hand in do-test:

Ran 12 tests in 2.522s
OK (expected failures=1)

Without it, Python fails with an ImportError somewhere in its own code;
for now I do not bother ivestigating this further since I can still run
tests successfully and the program itself just works in normal usage.

Feedback? OK?


yq.tgz
Description: Binary data