[Puppet Users] Puppetdb source install on Solaris. Agents complain about "invalid encoding ("UTF-8//IGNORE", "UTF-8")"
Hi all,
I'm having an issue on the master I'm using in my staging
environment with getting storeconfigs to work with puppetdb.
I should say up front that my staging environment is probably a little
uncommon. I'm running OpenIndiana which is a distribution of the open
source fork of OpenSolaris.
I installed puppet (3.0), facter and hiera using `gem` on this system
and have gotten the master working, servicing some test clients without
too much trouble but I'm having some real trouble getting puppetdb
integrated with my master.
Puppetdb is running on the puppet master and I can access the dashboard
on the default port of 8080 without issue,
I've followed all the instructions at
http://docs.puppetlabs.com/puppetdb/1/connect_puppet_master.html quite
closely but if I have storeconfigs set to true and puppetdb as my
backend I get the following output from a puppet agent --test:
Info: Retrieving plugin
Info: Loading facts in /var/puppet/lib/facter/last_run.rb
Info: Loading facts in /var/puppet/lib/facter/puppet_vardir.rb
Info: Loading facts in /var/puppet/lib/facter/pe_version.rb
Info: Loading facts in /var/puppet/lib/facter/facter_dot_d.rb
Info: Loading facts in /var/puppet/lib/facter/root_home.rb
Info: Loading facts in /var/puppet/lib/facter/puppi_projects.rb
Error: Could not retrieve catalog from remote server: Error 400 on
SERVER: invalid encoding ("UTF-8//IGNORE", "UTF-8") Warning: Not using
cache on failed catalog Error: Could not retrieve catalog; skipping run
I've tried Googling but haven't turned up anyone else reporting the
same exact issue. I checked out puppetdb using git and got *most* of
the way through building with Rake (it stopped while building some of
the Debian-specific stuff) -- I got the jar file I'm using from there
and I copied the files I believe I was supposed to from puppetdb's
puppet/lib/puppet into
"/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/" on this system.
Verbose output from the copy command follows...
root@atropos:~/puppetdb/puppet/lib# cp -Rav
puppet/ /var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/ `puppet/'
-> `/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet'
`puppet/indirector' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/indirector'
`puppet/indirector/catalog' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/indirector/catalog'
`puppet/indirector/catalog/puppetdb.rb' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/indirector/catalog/puppetdb.rb'
`puppet/indirector/facts' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/indirector/facts'
`puppet/indirector/facts/puppetdb.rb' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/indirector/facts/puppetdb.rb'
`puppet/indirector/node' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/indirector/node'
`puppet/indirector/node/puppetdb.rb' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/indirector/node/puppetdb.rb'
`puppet/indirector/resource' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/indirector/resource'
`puppet/indirector/resource/puppetdb.rb' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/indirector/resource/puppetdb.rb'
`puppet/util' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/util'
`puppet/util/puppetdb' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/util/puppetdb'
`puppet/util/puppetdb/char_encoding.rb' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/util/puppetdb/char_encoding.rb'
`puppet/util/puppetdb/report_helper.rb' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/util/puppetdb/report_helper.rb'
`puppet/util/puppetdb.rb' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/util/puppetdb.rb'
`puppet/face' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/face'
`puppet/face/node' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/face/node'
`puppet/face/node/status.rb' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/face/node/status.rb'
`puppet/face/node/deactivate.rb' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/face/node/deactivate.rb'
`puppet/reports' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/reports'
`puppet/reports/puppetdb.rb' ->
`/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/puppet/reports/puppetdb.rb'
I dropped all the relevant config file contents into
http://pastebin.com/SfLEv5zM because it'd be pretty long inline.
Thanks in advance,
Ryan
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To view this discussion on the web visit
https://groups.google.com/d/msg/puppet-users/-/MDBRcAV_zQEJ.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Puppetdb source install on Solaris. Agents complain about "invalid encoding ("UTF-8//IGNORE", "UTF-8")"
On Tue, Nov 27, 2012 at 9:27 PM, Ryan Cunningham <
ryan.cunningham.xy...@gmail.com> wrote:
> Hi all,
>
> I'm having an issue on the master I'm using in my staging
> environment with getting storeconfigs to work with puppetdb.
>
> I should say up front that my staging environment is probably a little
> uncommon. I'm running OpenIndiana which is a distribution of the open
> source fork of OpenSolaris.
>
> I installed puppet (3.0), facter and hiera using `gem` on this system
> and have gotten the master working, servicing some test clients without
> too much trouble but I'm having some real trouble getting puppetdb
> integrated with my master.
>
> Puppetdb is running on the puppet master and I can access the dashboard
> on the default port of 8080 without issue,
>
> I've followed all the instructions at
> http://docs.puppetlabs.com/puppetdb/1/connect_puppet_master.html quite
> closely but if I have storeconfigs set to true and puppetdb as my
> backend I get the following output from a puppet agent --test:
>
>
> Info: Retrieving plugin
> Info: Loading facts in /var/puppet/lib/facter/last_run.rb
> Info: Loading facts in /var/puppet/lib/facter/puppet_vardir.rb
> Info: Loading facts in /var/puppet/lib/facter/pe_version.rb
> Info: Loading facts in /var/puppet/lib/facter/facter_dot_d.rb
> Info: Loading facts in /var/puppet/lib/facter/root_home.rb
> Info: Loading facts in /var/puppet/lib/facter/puppi_projects.rb
> Error: Could not retrieve catalog from remote server: Error 400 on
> SERVER: invalid encoding ("UTF-8//IGNORE", "UTF-8") Warning: Not using
> cache on failed catalog Error: Could not retrieve catalog; skipping run
>
> I've tried Googling but haven't turned up anyone else reporting the
> same exact issue. I checked out puppetdb using git and got *most* of
> the way through building with Rake (it stopped while building some of
> the Debian-specific stuff) -- I got the jar file I'm using from there
> and I copied the files I believe I was supposed to from puppetdb's
> puppet/lib/puppet into
> "/var/ruby/1.8/gem_home/gems/puppet-3.0.1/lib/puppet/" on this system.
> Verbose output from the copy command follows...
>
>
The code in question is trying to transcode your catalog to UTF-8 before
sending it to PuppetDB (because the wire format is JSON, and JSON is
UTF-8). On Ruby 1.8, we rely on the "iconv" Ruby library, which is
basically just a pass-through to libiconv. The error message seems to
indicate that your libiconv doesn't know about UTF-8 which is
strange...though I admit to not knowing exactly how (and with what flags)
Ruby or iconv is compiled for OpenIndiana.
Using the same Ruby you run your puppetmasters with, can you do:
ruby -e "require 'iconv'; puts Iconv.list.sort"
That should dump out the list of available encodings. That should help us
at least more properly triangulate the issue.
deepak
--
You received this message because you are subscribed to the Google Groups
"Puppet Users" group.
To post to this group, send email to puppet-users@googlegroups.com.
To unsubscribe from this group, send email to
puppet-users+unsubscr...@googlegroups.com.
For more options, visit this group at
http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Puppetdb source install on Solaris. Agents complain about "invalid encoding ("UTF-8//IGNORE", "UTF-8")"
Deepak,
Thanks for your insight into the error.
It looks like the iconv library in the version of Ruby provided by the
OpenIndiana repo may be too old, lack a required method or have an
incompatible version of the method being used to transform the contents of
the catalog.
root@atropos:~# ruby -r iconv -ve 'pp Iconv.list'
ruby 1.8.7 (2009-06-12 patchlevel 174) [i386-solaris2.11]
-e:1: undefined method `list' for Iconv:Class (NoMethodError)
Based on that I'll give all this another go on a fresh VM with Ruby 1.9
built from source unless there's some kind of "easy button" fix.
I'm sure there's comparatively little demand for compatibility with this
particular offshoot of SunOS but if you're at all interested I managed to
Google my way through printing Config::CONFIG with rbconfig (learning Ruby
is still on my to-do list)
=> {"PACKAGE_VERSION"=>""
"INSTALL_SCRIPT"=>"/usr/bin/ginstall -c"
"DEFS"=>"-D_FILE_OFFSET_BITS=64"
"GREP"=>"/usr/bin/ggrep"
"sharedstatedir"=>"/usr/ruby/1.8/com"
"COMMON_LIBS"=>""
"EXPORT_PREFIX"=>""
"AR"=>"ar"
"target_alias"=>""
"PACKAGE_STRING"=>""
"host_os"=>"solaris2.11"
"RDOCTARGET"=>"install-doc"
"MANTYPE"=>"man"
"build"=>"i386-pc-solaris2.11"
"AS"=>"as"
"DLDFLAGS"=>""
"build_os"=>"solaris2.11"
"PATH_SEPARATOR"=>":"
"PACKAGE_NAME"=>""
"LIBRUBYARG"=>"-R /usr/ruby/1.8/lib -L/usr/ruby/1.8/lib -lruby"
"RUBY_INSTALL_NAME"=>"ruby"
"LN_S"=>"ln -s"
"EXTSTATIC"=>""
"EXEEXT"=>""
"mandir"=>"/usr/ruby/1.8/share/man"
"YACC"=>"bison -y"
"LIBPATHENV"=>"LD_LIBRARY_PATH"
"oldincludedir"=>"/usr/include"
"psdir"=>"/usr/ruby/1.8/share/doc/$(PACKAGE)"
"sitearch"=>"i386-solaris2.11"
"DLEXT2"=>""
"LIBS"=>"-lrt -ldl -lcrypt -lm "
"bindir"=>"/usr/ruby/1.8/bin"
"localedir"=>"/usr/ruby/1.8/share/locale"
"rubyw_install_name"=>""
"RUBY_SO_NAME"=>"ruby"
"OBJDUMP"=>""
"ECHO_C"=>"c"
"optflags"=>""
"arch"=>"i386-solaris2.11"
"NROFF"=>"/usr/bin/nroff"
"libexecdir"=>"/usr/ruby/1.8/libexec"
"CC"=>"/usr/sfw/bin/gcc"
"cflags"=>" "
"OUTFLAG"=>"-o "
"STATIC"=>""
"PREP"=>"miniruby"
"build_vendor"=>"pc"
"debugflags"=>""
"htmldir"=>"/usr/ruby/1.8/share/doc/$(PACKAGE)"
"sitedir"=>"/usr/ruby/1.8/lib/ruby/site_ruby"
"YFLAGS"=>""
"MAKEFILES"=>"Makefile"
"STRIP"=>"strip"
"MINOR"=>"8"
"LIBRUBYARG_SHARED"=>"-R /usr/ruby/1.8/lib -L/usr/ruby/1.8/lib -lruby"
"MAKEDIRS"=>"mkdir -p"
"LIBRUBY_LDSHARED"=>"ld -G"
"LDFLAGS"=>"-L/usr/sfw/lib -R/usr/sfw/lib"
"vendorarchdir"=>"/usr/ruby/1.8/lib/ruby/vendor_ruby/1.8/i386-solaris2.11"
"includedir"=>"/usr/ruby/1.8/include"
"infodir"=>"/usr/ruby/1.8/share/info"
"NM"=>""
"target"=>"i386-pc-solaris2.11"
"ENABLE_SHARED"=>"yes"
"DLLWRAP"=>""
"vendorlibdir"=>"/usr/ruby/1.8/lib/ruby/vendor_ruby/1.8"
"CFLAGS"=>"-g -O3 -fPIC"
"host"=>"i386-pc-solaris2.11"
"INSTALL_DATA"=>"/usr/bin/ginstall -c -m 644"
"INSTALL_PROGRAM"=>"/usr/bin/ginstall -c"
"DLEXT"=>"so"
"PACKAGE_BUGREPORT"=>""
"exec_prefix"=>"/usr/ruby/1.8"
"sysconfdir"=>"/usr/ruby/1.8/etc"
"TEENY"=>"7"
"CPP"=>"/usr/sfw/bin/gcc -E"
"ALLOCA"=>""
"LIBEXT"=>"a"
"libdir"=>"/usr/ruby/1.8/lib"
"LIBRUBY"=>"libruby.so.1"
"build_alias"=>""
"prefix"=>"/usr/ruby/1.8"
"target_vendor"=>"pc"
"target_cpu"=>"i386"
"ECHO_N"=>""
"rubylibdir"=>"/usr/ruby/1.8/lib/ruby/1.8"
"sbindir"=>"/usr/ruby/1.8/sbin"
"docdir"=>"/usr/ruby/1.8/share/doc/$(PACKAGE)"
"dvidir"=>"/usr/ruby/1.8/share/doc/$(PACKAGE)"
"host_vendor"=>"pc"
"TRY_LINK"=>""
"SOLIBS"=>"-lrt -ldl -lcrypt -lm "
"configure_args"=>" --prefix=/usr/ruby/1.8 --enable-dtrace --enable-shared
--enable-install-doc --disable-option-checking --with-openssl
--with-tk-dir=/usr --with-curses-dir=/usr CC=/opt/SUNWspro/bin/cc
CFLAGS=-xO4 -xbuiltin=%all -xinline=%auto -xprefetch=auto -xdepend
-xtarget=generic
-I/home/jt/OI-151A-STABLE/151A-PRESTABLE7/newbuilds/sfw/proto/root_i386/usr/include
LDFLAGS=-L/home/jt/OI-151A-STABLE/151A-PRESTABLE7/newbuilds/sfw/proto/root_i386/usr/lib
-M
/home/jt/OI-151A-STABLE/151A-PRESTABLE7/newbuilds/sfw/usr/src/cmd/mapfile_noexstk
-R/usr/sfw/lib CPPFLAGS=-DTEXT_DOMAIN=\"\"
-I/home/jt/OI-151A-STABLE/151A-PRESTABLE7/newbuilds/sfw/proto/root_i386/usr/include
-I/home/jt/OI-151A-STABLE/151A-PRESTABLE7/newbuilds/sfw/proto/root_i386/usr/sfw/include
-I/home/jt/OI-151A-STABLE/151A-PRESTABLE7/newbuilds/sfw/proto/root_i386/usr/include
-I/home/jt/OI-151A-STABLE/151A-PRESTABLE7/newbuilds/sfw/proto/root_i386/usr/include"
"vendordir"=>"/usr/ruby/1.8/lib/ruby/vendor_ruby"
"target_os"=>"solaris2.11"
"GNU_LD"=>"no"
"CP"=>"cp"
"EXTOUT"=>".ext"
"MAINLIBS"=>""
"SET_MAKE"=>""
"datarootdir"=>"/usr/ruby/1.8/share"
"pdfdir"=>"/usr/ruby/1.8/share/doc/$(PACKAGE)"
"LIBRUBYARG_STATIC"=>"-lruby-static"
"RUBYW_INSTALL_NAME"=>""
"WINDRES"=>""
"archdir"=>"/usr/ruby/1.8/lib/ruby/1.8/i386-solaris2.11"
"ruby_version"=>"1.8"
"LINK_SO"=>""
"ECHO_T"=>""
"RPATHFLAG"=>" -R%1$-s"
"SHELL"=>"/bin/sh"
"LDSHARED"=>"/usr/sfw/bin/gcc -G"
"RANLIB"=>"ranlib"
"DLDLIBS"=>" -lc"
"build_cpu"=>"i386"
"host_alias"
Re: [Puppet Users] Puppetdb source install on Solaris. Agents complain about "invalid encoding ("UTF-8//IGNORE", "UTF-8")"
In regard to: Re: [Puppet Users] Puppetdb source install on Solaris. Agents...: It looks like the iconv library in the version of Ruby provided by the OpenIndiana repo may be too old, lack a required method or have an incompatible version of the method being used to transform the contents of the catalog. I don't think it's an issue with your ruby Iconv. root@atropos:~# ruby -r iconv -ve 'pp Iconv.list' ruby 1.8.7 (2009-06-12 patchlevel 174) [i386-solaris2.11] -e:1: undefined method `list' for Iconv:Class (NoMethodError) I get the exact same error when I run Deepak's suggested command on RHEL 6, which includes ruby 1.8.7: $ ruby -e "require 'iconv'; puts Iconv.list.sort" -e:1: undefined method `list' for Iconv:Class (NoMethodError) If I run that on Solaris 10 with ruby 1.9.3 p327 compiled from source, I get: $ ruby -e "require 'iconv'; puts Iconv.list.sort" /local/lib/64/ruby/1.9.1/rubygems/custom_require.rb:36:in `require': iconv will be deprecated in the future, use String#encode instead. -e:1:in `list': list() function is unimplemented on this machine (NotImplementedError) from -e:1:in `' (learning Ruby is still on my to-do list) Same for me, and it has presented a slight barrier to entry for becoming really comfortable with puppet. ruby -e "require 'iconv'; puts Iconv.list.sort" That should dump out the list of available encodings. That should help us at least more properly triangulate the issue. Tim -- Tim Mooney tim.moo...@ndsu.edu Enterprise Computing & Infrastructure 701-231-1076 (Voice) Room 242-J6, IACC Building 701-231-8541 (Fax) North Dakota State University, Fargo, ND 58105-5164 -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
Re: [Puppet Users] Puppetdb source install on Solaris. Agents complain about "invalid encoding ("UTF-8//IGNORE", "UTF-8")"
> I get the exact same error when I run Deepak's suggested command on > RHEL 6, which includes ruby 1.8.7: > Yeeeah. Or at least if there is an issue that's not the way to figure it out. Thanks for taking time to sanity check the diagnostic command. When I asked about the same issue in #puppet someone elses' best guess was also that it could be something to do with Iconv. Creating a new zone and installing everything on 1.9.x isn't a prohibitive amount of work just to see. (learning Ruby is still on my to-do list) >> > > Same for me, and it has presented a slight barrier to entry for becoming > really comfortable with puppet. It could be time for me to move it up in priority. At least to the point where I can effectively dig around in a module's definition and grok the portion of Puppet where this is choking. The good thing about installing from source on SunOS is that you tend to learn quite a few more things before you get the software working the way you want than if you'd just installed an rpm. -- You received this message because you are subscribed to the Google Groups "Puppet Users" group. To post to this group, send email to puppet-users@googlegroups.com. To unsubscribe from this group, send email to puppet-users+unsubscr...@googlegroups.com. For more options, visit this group at http://groups.google.com/group/puppet-users?hl=en.
