On Sun, Jan 7, 2018 at 11:33 PM, Peter Faller <pgfal...@gmail.com> wrote:
> I'm trying to preprocess some version 3.x catalogs to eliminate some false
> positives reported by 'puppet catalog diff'. I copied some code from the
> catalog diff source, and have ended up with (extraneous details omitted):
>
> #! /usr/bin/env ruby
> require 'puppet/external/pson/pure'
> require 'fileutils'
>
> r = ARGV[0]
> unless File.exist?(r)
> raise "Cannot find resources in #{r}"
> end
>
> puts "Processing file: #{r}"
> case File.extname(r)
> when '.yaml'
> hash = YAML.load(File.read(r))
> when '.marshal'
> hash = Marshal.load(File.read(r))
> when '.pson'
> hash = PSON.load(File.read(r))
> when '.json'
> hash = PSON.load(File.read(r))
> else
> raise "Provide catalog with the appropriate file extension, valid
> extensions are pson, yaml and marshal"
> end
>
> puts hash
>
> I have two test files that 'puppet catalog diff' can read successfully,
> with the only difference being the presence of '\n':
>
> [root@tstpuppet01 ~]# diff test_file1.pson test_file2.pson 13c13 <
> "content": "#!/bin/sh\nPRETEND_NCM_SSH_TRUST_IS_VALID=YES\n", --- >
> "content": "#!/bin/shPRETEND_NCM_SSH_TRUST_IS_VALID=YES",
>
> 'puppet catalog diff' can read both files without problems.
>
> When I try to read these files with the code above, I get the following:
>
> [root@tstpuppet01 ~]# /usr/bin/ruby
> -I/opt/puppetlabs/puppet/lib/ruby/vendor_ruby
> ./preprocess.rb test_file1.pson Using pure library for PSON. Processing
> file: test_file1.pson Exception `NoMethodError' at
> /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/
> external/pson/pure/parser.rb:183 - undefined method `[]' for nil:NilClass
> Exception `PSON::GeneratorError' at /opt/puppetlabs/puppet/lib/
> ruby/vendor_ruby/puppet/external/pson/pure/parser.rb:201 - Caught
> NoMethodError: undefined method `[]' for nil:NilClass
> /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/
> external/pson/pure/parser.rb:183:in `block in parse_string': Caught
> NoMethodError: undefined method `[]' for nil:NilClass
> (PSON::GeneratorError) from /opt/puppetlabs/puppet/lib/
> ruby/vendor_ruby/puppet/external/pson/pure/parser.rb:182:in `gsub' from
> /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/
> external/pson/pure/parser.rb:182:in `parse_string' from
> /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/
> external/pson/pure/parser.rb:216:in `parse_value' from
> /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/
> external/pson/pure/parser.rb:280:in `parse_object' from
> /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/
> external/pson/pure/parser.rb:225:in `parse_value' from
> /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/
> external/pson/pure/parser.rb:280:in `parse_object' from
> /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/
> external/pson/pure/parser.rb:225:in `parse_value' from
> /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/
> external/pson/pure/parser.rb:246:in `parse_array' from
> /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/
> external/pson/pure/parser.rb:220:in `parse_value' from
> /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/
> external/pson/pure/parser.rb:280:in `parse_object' from
> /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/
> external/pson/pure/parser.rb:225:in `parse_value' from
> /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/
> external/pson/pure/parser.rb:280:in `parse_object' from
> /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/
> external/pson/pure/parser.rb:92:in `parse' from
> /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/external/pson/common.rb:125:in
> `parse' from
> /opt/puppetlabs/puppet/lib/ruby/vendor_ruby/puppet/external/pson/common.rb:259:in
> `load' from ./preprocess.rb:21:in `<main>'
>
> The file without '\n' reads OK:
>
> [root@tstpuppet01 ~]# /usr/bin/ruby
> -I/opt/puppetlabs/puppet/lib/ruby/vendor_ruby
> ./preprocess.rb test_file2.pson
> Using pure library for PSON.
> Processing file: test_file2.pson
> {"document_type"=>"Catalog", "data"=>{"resources"=>[{"type"=>"File",
> "title"=>"/etc/opt/rim/develop.sh", "tags"=>["file", "class", "rimcdm"],
> "file"=>"/etc/puppet/environments/trunk_9_4_0_0/
> modules/rimcdm/manifests/init.pp", "line"=>219, "exported"=>false,
> "parameters"=>{"content"=>"#!/bin/shPRETEND_NCM_SSH_TRUST_IS_VALID=YES",
> "backup"=>"main"}}], "edges"=>[{"source"=>"Stage[main]",
> "target"=>"Class[Settings]"}], "classes"=>["settings"]},
> "metadata"=>{"api_version"=>1}}
>
> Can anyone spot the difference between what 'puppet catalog diff' does to
> read PSON, and what the code above does?
>
> --
> You received this message because you are subscribed to the Google Groups
> "Puppet Users" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to puppet-users+unsubscr...@googlegroups.com.
> To view this discussion on the web visit https://groups.google.com/d/
> msgid/puppet-users/2503d182-0c13-4543-ab9f-08f5852abb5c%40googlegroups.com
> <https://groups.google.com/d/msgid/puppet-users/2503d182-0c13-4543-ab9f-08f5852abb5c%40googlegroups.com?utm_medium=email&utm_source=footer>
> .
> For more options, visit https://groups.google.com/d/optout.
>
I can reproduce the issue using a JSON snippet newlines.pson containing:
{
"content": "#!/bin/sh\nPRETEND_NCM_SSH_TRUST_IS_VALID=YES\n"
}
For some reason $MATCH is nil, which is bizarre[1]. Adding `require
'puppet'` to your script fixes the issue for me, or if I change puppet to
use the yielded parameter 'c' instead of '$MATCH'.
$ bx ruby preprocess.rb newlines.pson
Processing file: newlines.pson
/Users/josh/work/puppet/lib/puppet/external/pson/pure/parser.rb:183:in
`block in parse_string': Caught NoMethodError: undefined method `[]' for
nil:NilClass (PSON::GeneratorError)
from /Users/josh/work/puppet/lib/puppet/external/pson/pure/parser.rb:182:in
`gsub'
from /Users/josh/work/puppet/lib/puppet/external/pson/pure/parser.rb:182:in
`parse_string'
from /Users/josh/work/puppet/lib/puppet/external/pson/pure/parser.rb:216:in
`parse_value'
from /Users/josh/work/puppet/lib/puppet/external/pson/pure/parser.rb:280:in
`parse_object'
from /Users/josh/work/puppet/lib/puppet/external/pson/pure/parser.rb:92:in
`parse'
from /Users/josh/work/puppet/lib/puppet/external/pson/common.rb:121:in
`parse'
from /Users/josh/work/puppet/lib/puppet/external/pson/common.rb:255:in
`load'
from preprocess.rb:18:in `<main>'
$ git stash pop
On branch 4.10.x
Your branch is up to date with 'upstream/4.10.x'.
Changes not staged for commit:
(use "git add <file>..." to update what will be committed)
(use "git checkout -- <file>..." to discard changes in working directory)
modified: lib/puppet/external/pson/pure/parser.rb
no changes added to commit (use "git add" and/or "git commit -a")
Dropped refs/stash@{0} (6ef5ec0fb8027f9957e9c5f55856389db79258cb)
$ PAGER= git diff
diff --git a/lib/puppet/external/pson/pure/parser.rb
b/lib/puppet/external/pson/pure/parser.rb
index cf607afc0a..58fa165b9d 100644
--- a/lib/puppet/external/pson/pure/parser.rb
+++ b/lib/puppet/external/pson/pure/parser.rb
@@ -180,7 +180,7 @@ module PSON
if scan(STRING)
return '' if self[1].empty?
string =
self[1].gsub(%r{(?:\\[\\bfnrt"/]|(?:\\u(?:[A-Fa-f\d]{4}))+|\\[\x20-\xff])}n)
do |c|
- if u = UNESCAPE_MAP[$MATCH[1]]
+ if u = UNESCAPE_MAP[c[1]]
u
else # \uXXXX
bytes = ''
$ bx ruby preprocess.rb newlines.pson
Processing file: newlines.pson
{"content"=>"#!/bin/sh\nPRETEND_NCM_SSH_TRUST_IS_VALID=YES\n"}
Note bx is an alias for 'bundle exec'
Josh
[1]
https://github.com/puppetlabs/puppet/blob/4.10.9/lib/puppet/external/pson/pure/parser.rb#L183
--
Josh Cooper | Software Engineer
j...@puppet.com | @coopjn
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to puppet-users+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/puppet-users/CA%2Bu97uk8O_9%3D-2YNLjJj5e85A3%3Dth%3D6Zct8g1QT5bejKoGuuMQ%40mail.gmail.com.
For more options, visit https://groups.google.com/d/optout.
