This is a proposed bundeling of the pure ruby json library to address the various cross version serliaization bugs (#2615, et al).
This is not a final patch, but it works and I wanted to circulate it for discussion. See the end of this commit message for areas I'd like to address prior to inclusion. This patch adds a subset of the files from the json_pure gem to lib/puppet/external/json so that we will always have json available. However, it still uses the "feature" system to manage the json, because rails (which is also a "feature") likes to take over json processing when it is loaded. In addition the pure ruby json gem as distibuted is setup to defer to the compiled version if it is installed. For this reason the files that have been brought over have been modified slightly to always and only use the bundled version. Finaly, the behavior of the pure-ruby version is slightly different in that it raises an exception if json_create returns nil. This required minor modifications to some tests. Open issues: * This is the lowest-overall-ugliness integration I could find, but I'm open to suggestion on how to improve it. Bear in mind that a working solution has to function in environments with any combination of gems (including rails) and should always use the bundled library. Note also that we DON'T want to be using the C code as supporting that in heterogenious environments is beyond the scope. * The existing patch was failing some rake spec tests about one time out of ten when run with the non-pure json gem installed and then began passing consistantly. The failures were caused by it trying to use code from the compiled version which has a slightly different call signature. The problem appeared to be a test isolation issue, it only happened if some test or tests were run before the json test but I was not able to track it down further before it stopped happening. * We should probably incude some subset of the read-me files in the lib/external/json directory though (since we are GPL and providing the source) we would be in a defensible position to omit them. * There is a diagnostic rescue clause in the json feature code that should come out for the final patch. Signed-off-by: Markus Roberts <[email protected]> --- lib/puppet/external/json/common.rb | 356 +++++++++++++++++++++++ lib/puppet/external/json/pure.rb | 77 +++++ lib/puppet/external/json/pure/generator.rb | 429 ++++++++++++++++++++++++++++ lib/puppet/external/json/pure/parser.rb | 269 +++++++++++++++++ lib/puppet/external/json/version.rb | 8 + lib/puppet/feature/json.rb | 16 + lib/puppet/feature/rails.rb | 5 - spec/unit/relationship.rb | 2 +- spec/unit/resource/catalog.rb | 2 +- 9 files changed, 1157 insertions(+), 7 deletions(-) create mode 100644 lib/puppet/external/json/common.rb create mode 100644 lib/puppet/external/json/pure.rb create mode 100644 lib/puppet/external/json/pure/generator.rb create mode 100644 lib/puppet/external/json/pure/parser.rb create mode 100644 lib/puppet/external/json/version.rb diff --git a/lib/puppet/external/json/common.rb b/lib/puppet/external/json/common.rb new file mode 100644 index 0000000..bae6508 --- /dev/null +++ b/lib/puppet/external/json/common.rb @@ -0,0 +1,356 @@ +require 'puppet/external/json/version' + +module JSON + class << self + # If _object_ is string-like parse the string and return the parsed result + # as a Ruby data structure. Otherwise generate a JSON text from the Ruby + # data structure object and return it. + # + # The _opts_ argument is passed through to generate/parse respectively, see + # generate and parse for their documentation. + def [](object, opts = {}) + if object.respond_to? :to_str + JSON.parse(object.to_str, opts => {}) + else + JSON.generate(object, opts => {}) + end + end + + # Returns the JSON parser class, that is used by JSON. This might be either + # JSON::Ext::Parser or JSON::Pure::Parser. + attr_reader :parser + + # Set the JSON parser class _parser_ to be used by JSON. + def parser=(parser) # :nodoc: + @parser = parser + remove_const :Parser if const_defined? :Parser + const_set :Parser, parser + end + + # Return the constant located at _path_. The format of _path_ has to be + # either ::A::B::C or A::B::C. In any case A has to be located at the top + # level (absolute namespace path?). If there doesn't exist a constant at + # the given path, an ArgumentError is raised. + def deep_const_get(path) # :nodoc: + path = path.to_s + path.split(/::/).inject(Object) do |p, c| + case + when c.empty? then p + when p.const_defined?(c) then p.const_get(c) + else raise ArgumentError, "can't find const #{path}" + end + end + end + + # Set the module _generator_ to be used by JSON. + def generator=(generator) # :nodoc: + @generator = generator + generator_methods = generator::GeneratorMethods + for const in generator_methods.constants + klass = deep_const_get(const) + modul = generator_methods.const_get(const) + klass.class_eval do + instance_methods(false).each do |m| + m.to_s == 'to_json' and remove_method m + end + include modul + end + end + self.state = generator::State + const_set :State, self.state + end + + # Returns the JSON generator modul, that is used by JSON. This might be + # either JSON::Ext::Generator or JSON::Pure::Generator. + attr_reader :generator + + # Returns the JSON generator state class, that is used by JSON. This might + # be either JSON::Ext::Generator::State or JSON::Pure::Generator::State. + attr_accessor :state + + # This is create identifier, that is used to decide, if the _json_create_ + # hook of a class should be called. It defaults to 'json_class'. + attr_accessor :create_id + end + self.create_id = 'json_class' + + NaN = (-1.0) ** 0.5 + + Infinity = 1.0/0 + + MinusInfinity = -Infinity + + # The base exception for JSON errors. + class JSONError < StandardError; end + + # This exception is raised, if a parser error occurs. + class ParserError < JSONError; end + + # This exception is raised, if the nesting of parsed datastructures is too + # deep. + class NestingError < ParserError; end + + # This exception is raised, if a generator or unparser error occurs. + class GeneratorError < JSONError; end + # For backwards compatibility + UnparserError = GeneratorError + + # If a circular data structure is encountered while unparsing + # this exception is raised. + class CircularDatastructure < GeneratorError; end + + # This exception is raised, if the required unicode support is missing on the + # system. Usually this means, that the iconv library is not installed. + class MissingUnicodeSupport < JSONError; end + + module_function + + # Parse the JSON string _source_ into a Ruby data structure and return it. + # + # _opts_ can have the following + # keys: + # * *max_nesting*: The maximum depth of nesting allowed in the parsed data + # structures. Disable depth checking with :max_nesting => false, it defaults + # to 19. + # * *allow_nan*: If set to true, allow NaN, Infinity and -Infinity in + # defiance of RFC 4627 to be parsed by the Parser. This option defaults + # to false. + # * *create_additions*: If set to false, the Parser doesn't create + # additions even if a matchin class and create_id was found. This option + # defaults to true. + def parse(source, opts = {}) + JSON.parser.new(source, opts).parse + end + + # Parse the JSON string _source_ into a Ruby data structure and return it. + # The bang version of the parse method, defaults to the more dangerous values + # for the _opts_ hash, so be sure only to parse trusted _source_ strings. + # + # _opts_ can have the following keys: + # * *max_nesting*: The maximum depth of nesting allowed in the parsed data + # structures. Enable depth checking with :max_nesting => anInteger. The parse! + # methods defaults to not doing max depth checking: This can be dangerous, + # if someone wants to fill up your stack. + # * *allow_nan*: If set to true, allow NaN, Infinity, and -Infinity in + # defiance of RFC 4627 to be parsed by the Parser. This option defaults + # to true. + # * *create_additions*: If set to false, the Parser doesn't create + # additions even if a matchin class and create_id was found. This option + # defaults to true. + def parse!(source, opts = {}) + opts = { + :max_nesting => false, + :allow_nan => true + }.update(opts) + JSON.parser.new(source, opts).parse + end + + # Unparse the Ruby data structure _obj_ into a single line JSON string and + # return it. _state_ is + # * a JSON::State object, + # * or a Hash like object (responding to to_hash), + # * an object convertible into a hash by a to_h method, + # that is used as or to configure a State object. + # + # It defaults to a state object, that creates the shortest possible JSON text + # in one line, checks for circular data structures and doesn't allow NaN, + # Infinity, and -Infinity. + # + # A _state_ hash can have the following keys: + # * *indent*: a string used to indent levels (default: ''), + # * *space*: a string that is put after, a : or , delimiter (default: ''), + # * *space_before*: a string that is put before a : pair delimiter (default: ''), + # * *object_nl*: a string that is put at the end of a JSON object (default: ''), + # * *array_nl*: a string that is put at the end of a JSON array (default: ''), + # * *check_circular*: true if checking for circular data structures + # should be done (the default), false otherwise. + # * *allow_nan*: true if NaN, Infinity, and -Infinity should be + # generated, otherwise an exception is thrown, if these values are + # encountered. This options defaults to false. + # * *max_nesting*: The maximum depth of nesting allowed in the data + # structures from which JSON is to be generated. Disable depth checking + # with :max_nesting => false, it defaults to 19. + # + # See also the fast_generate for the fastest creation method with the least + # amount of sanity checks, and the pretty_generate method for some + # defaults for a pretty output. + def generate(obj, state = nil) + if state + state = State.from_state(state) + else + state = State.new + end + obj.to_json(state) + end + + # :stopdoc: + # I want to deprecate these later, so I'll first be silent about them, and + # later delete them. + alias unparse generate + module_function :unparse + # :startdoc: + + # Unparse the Ruby data structure _obj_ into a single line JSON string and + # return it. This method disables the checks for circles in Ruby objects, and + # also generates NaN, Infinity, and, -Infinity float values. + # + # *WARNING*: Be careful not to pass any Ruby data structures with circles as + # _obj_ argument, because this will cause JSON to go into an infinite loop. + def fast_generate(obj) + obj.to_json(nil) + end + + # :stopdoc: + # I want to deprecate these later, so I'll first be silent about them, and later delete them. + alias fast_unparse fast_generate + module_function :fast_unparse + # :startdoc: + + # Unparse the Ruby data structure _obj_ into a JSON string and return it. The + # returned string is a prettier form of the string returned by #unparse. + # + # The _opts_ argument can be used to configure the generator, see the + # generate method for a more detailed explanation. + def pretty_generate(obj, opts = nil) + state = JSON.state.new( + :indent => ' ', + :space => ' ', + :object_nl => "\n", + :array_nl => "\n", + :check_circular => true + ) + if opts + if opts.respond_to? :to_hash + opts = opts.to_hash + elsif opts.respond_to? :to_h + opts = opts.to_h + else + raise TypeError, "can't convert #{opts.class} into Hash" + end + state.configure(opts) + end + obj.to_json(state) + end + + # :stopdoc: + # I want to deprecate these later, so I'll first be silent about them, and later delete them. + alias pretty_unparse pretty_generate + module_function :pretty_unparse + # :startdoc: + + # Load a ruby data structure from a JSON _source_ and return it. A source can + # either be a string-like object, an IO like object, or an object responding + # to the read method. If _proc_ was given, it will be called with any nested + # Ruby object as an argument recursively in depth first order. + # + # This method is part of the implementation of the load/dump interface of + # Marshal and YAML. + def load(source, proc = nil) + if source.respond_to? :to_str + source = source.to_str + elsif source.respond_to? :to_io + source = source.to_io.read + else + source = source.read + end + result = parse(source, :max_nesting => false, :allow_nan => true) + recurse_proc(result, &proc) if proc + result + end + + def recurse_proc(result, &proc) + case result + when Array + result.each { |x| recurse_proc x, &proc } + proc.call result + when Hash + result.each { |x, y| recurse_proc x, &proc; recurse_proc y, &proc } + proc.call result + else + proc.call result + end + end + private :recurse_proc + module_function :recurse_proc + + alias restore load + module_function :restore + + # Dumps _obj_ as a JSON string, i.e. calls generate on the object and returns + # the result. + # + # If anIO (an IO like object or an object that responds to the write method) + # was given, the resulting JSON is written to it. + # + # If the number of nested arrays or objects exceeds _limit_ an ArgumentError + # exception is raised. This argument is similar (but not exactly the + # same!) to the _limit_ argument in Marshal.dump. + # + # This method is part of the implementation of the load/dump interface of + # Marshal and YAML. + def dump(obj, anIO = nil, limit = nil) + if anIO and limit.nil? + anIO = anIO.to_io if anIO.respond_to?(:to_io) + unless anIO.respond_to?(:write) + limit = anIO + anIO = nil + end + end + limit ||= 0 + result = generate(obj, :allow_nan => true, :max_nesting => limit) + if anIO + anIO.write result + anIO + else + result + end + rescue JSON::NestingError + raise ArgumentError, "exceed depth limit" + end +end + +module ::Kernel + private + + # Outputs _objs_ to STDOUT as JSON strings in the shortest form, that is in + # one line. + def j(*objs) + objs.each do |obj| + puts JSON::generate(obj, :allow_nan => true, :max_nesting => false) + end + nil + end + + # Ouputs _objs_ to STDOUT as JSON strings in a pretty format, with + # indentation and over many lines. + def jj(*objs) + objs.each do |obj| + puts JSON::pretty_generate(obj, :allow_nan => true, :max_nesting => false) + end + nil + end + + # If _object_ is string-like parse the string and return the parsed result as + # a Ruby data structure. Otherwise generate a JSON text from the Ruby data + # structure object and return it. + # + # The _opts_ argument is passed through to generate/parse respectively, see + # generate and parse for their documentation. + def JSON(object, opts = {}) + if object.respond_to? :to_str + JSON.parse(object.to_str, opts) + else + JSON.generate(object, opts) + end + end +end + +class ::Class + # Returns true, if this class can be used to create an instance + # from a serialised JSON string. The class has to implement a class + # method _json_create_ that expects a hash as first parameter, which includes + # the required data. + def json_creatable? + respond_to?(:json_create) + end +end diff --git a/lib/puppet/external/json/pure.rb b/lib/puppet/external/json/pure.rb new file mode 100644 index 0000000..7f6439b --- /dev/null +++ b/lib/puppet/external/json/pure.rb @@ -0,0 +1,77 @@ +require 'puppet/external/json/common' +require 'puppet/external/json/pure/parser' +require 'puppet/external/json/pure/generator' + +module JSON + begin + require 'iconv' + # An iconv instance to convert from UTF8 to UTF16 Big Endian. + UTF16toUTF8 = Iconv.new('utf-8', 'utf-16be') # :nodoc: + # An iconv instance to convert from UTF16 Big Endian to UTF8. + UTF8toUTF16 = Iconv.new('utf-16be', 'utf-8') # :nodoc: + UTF8toUTF16.iconv('no bom') + rescue LoadError + raise MissingUnicodeSupport, + "iconv couldn't be loaded, which is required for UTF-8/UTF-16 conversions" + rescue Errno::EINVAL, Iconv::InvalidEncoding + # Iconv doesn't support big endian utf-16. Let's try to hack this manually + # into the converters. + begin + old_verbose, $VERBSOSE = $VERBOSE, nil + # An iconv instance to convert from UTF8 to UTF16 Big Endian. + UTF16toUTF8 = Iconv.new('utf-8', 'utf-16') # :nodoc: + # An iconv instance to convert from UTF16 Big Endian to UTF8. + UTF8toUTF16 = Iconv.new('utf-16', 'utf-8') # :nodoc: + UTF8toUTF16.iconv('no bom') + if UTF8toUTF16.iconv("\xe2\x82\xac") == "\xac\x20" + swapper = Class.new do + def initialize(iconv) # :nodoc: + @iconv = iconv + end + + def iconv(string) # :nodoc: + result = @iconv.iconv(string) + JSON.swap!(result) + end + end + UTF8toUTF16 = swapper.new(UTF8toUTF16) # :nodoc: + end + if UTF16toUTF8.iconv("\xac\x20") == "\xe2\x82\xac" + swapper = Class.new do + def initialize(iconv) # :nodoc: + @iconv = iconv + end + + def iconv(string) # :nodoc: + string = JSON.swap!(string.dup) + @iconv.iconv(string) + end + end + UTF16toUTF8 = swapper.new(UTF16toUTF8) # :nodoc: + end + rescue Errno::EINVAL, Iconv::InvalidEncoding + raise MissingUnicodeSupport, "iconv doesn't seem to support UTF-8/UTF-16 conversions" + ensure + $VERBOSE = old_verbose + end + end + + # Swap consecutive bytes of _string_ in place. + def self.swap!(string) # :nodoc: + 0.upto(string.size / 2) do |i| + break unless string[2 * i + 1] + string[2 * i], string[2 * i + 1] = string[2 * i + 1], string[2 * i] + end + string + end + + # This module holds all the modules/classes that implement JSON's + # functionality in pure ruby. + module Pure + $DEBUG and warn "Using pure library for JSON." + JSON.parser = Parser + JSON.generator = Generator + end + + JSON_LOADED = true +end diff --git a/lib/puppet/external/json/pure/generator.rb b/lib/puppet/external/json/pure/generator.rb new file mode 100644 index 0000000..d51316e --- /dev/null +++ b/lib/puppet/external/json/pure/generator.rb @@ -0,0 +1,429 @@ +module JSON + MAP = { + "\x0" => '\u0000', + "\x1" => '\u0001', + "\x2" => '\u0002', + "\x3" => '\u0003', + "\x4" => '\u0004', + "\x5" => '\u0005', + "\x6" => '\u0006', + "\x7" => '\u0007', + "\b" => '\b', + "\t" => '\t', + "\n" => '\n', + "\xb" => '\u000b', + "\f" => '\f', + "\r" => '\r', + "\xe" => '\u000e', + "\xf" => '\u000f', + "\x10" => '\u0010', + "\x11" => '\u0011', + "\x12" => '\u0012', + "\x13" => '\u0013', + "\x14" => '\u0014', + "\x15" => '\u0015', + "\x16" => '\u0016', + "\x17" => '\u0017', + "\x18" => '\u0018', + "\x19" => '\u0019', + "\x1a" => '\u001a', + "\x1b" => '\u001b', + "\x1c" => '\u001c', + "\x1d" => '\u001d', + "\x1e" => '\u001e', + "\x1f" => '\u001f', + '"' => '\"', + '\\' => '\\\\', + } # :nodoc: + + # Convert a UTF8 encoded Ruby string _string_ to a JSON string, encoded with + # UTF16 big endian characters as \u????, and return it. + if String.method_defined?(:force_encoding) + def utf8_to_json(string) # :nodoc: + string = string.dup + string << '' # XXX workaround: avoid buffer sharing + string.force_encoding(Encoding::ASCII_8BIT) + string.gsub!(/["\\\x0-\x1f]/) { MAP[$&] } + string.gsub!(/( + (?: + [\xc2-\xdf][\x80-\xbf] | + [\xe0-\xef][\x80-\xbf]{2} | + [\xf0-\xf4][\x80-\xbf]{3} + )+ | + [\x80-\xc1\xf5-\xff] # invalid + )/nx) { |c| + c.size == 1 and raise GeneratorError, "invalid utf8 byte: '#{c}'" + s = JSON::UTF8toUTF16.iconv(c).unpack('H*')[0] + s.gsub!(/.{4}/n, '\\\\u\&') + } + string.force_encoding(Encoding::UTF_8) + string + rescue Iconv::Failure => e + raise GeneratorError, "Caught #{e.class}: #{e}" + end + else + def utf8_to_json(string) # :nodoc: + string = string.gsub(/["\\\x0-\x1f]/) { MAP[$&] } + string.gsub!(/( + (?: + [\xc2-\xdf][\x80-\xbf] | + [\xe0-\xef][\x80-\xbf]{2} | + [\xf0-\xf4][\x80-\xbf]{3} + )+ | + [\x80-\xc1\xf5-\xff] # invalid + )/nx) { |c| + c.size == 1 and raise GeneratorError, "invalid utf8 byte: '#{c}'" + s = JSON::UTF8toUTF16.iconv(c).unpack('H*')[0] + s.gsub!(/.{4}/n, '\\\\u\&') + } + string + rescue Iconv::Failure => e + raise GeneratorError, "Caught #{e.class}: #{e}" + end + end + module_function :utf8_to_json + + module Pure + module Generator + # This class is used to create State instances, that are use to hold data + # while generating a JSON text from a a Ruby data structure. + class State + # Creates a State object from _opts_, which ought to be Hash to create + # a new State instance configured by _opts_, something else to create + # an unconfigured instance. If _opts_ is a State object, it is just + # returned. + def self.from_state(opts) + case opts + when self + opts + when Hash + new(opts) + else + new + end + end + + # Instantiates a new State object, configured by _opts_. + # + # _opts_ can have the following keys: + # + # * *indent*: a string used to indent levels (default: ''), + # * *space*: a string that is put after, a : or , delimiter (default: ''), + # * *space_before*: a string that is put before a : pair delimiter (default: ''), + # * *object_nl*: a string that is put at the end of a JSON object (default: ''), + # * *array_nl*: a string that is put at the end of a JSON array (default: ''), + # * *check_circular*: true if checking for circular data structures + # should be done (the default), false otherwise. + # * *check_circular*: true if checking for circular data structures + # should be done, false (the default) otherwise. + # * *allow_nan*: true if NaN, Infinity, and -Infinity should be + # generated, otherwise an exception is thrown, if these values are + # encountered. This options defaults to false. + def initialize(opts = {}) + @seen = {} + @indent = '' + @space = '' + @space_before = '' + @object_nl = '' + @array_nl = '' + @check_circular = true + @allow_nan = false + configure opts + end + + # This string is used to indent levels in the JSON text. + attr_accessor :indent + + # This string is used to insert a space between the tokens in a JSON + # string. + attr_accessor :space + + # This string is used to insert a space before the ':' in JSON objects. + attr_accessor :space_before + + # This string is put at the end of a line that holds a JSON object (or + # Hash). + attr_accessor :object_nl + + # This string is put at the end of a line that holds a JSON array. + attr_accessor :array_nl + + # This integer returns the maximum level of data structure nesting in + # the generated JSON, max_nesting = 0 if no maximum is checked. + attr_accessor :max_nesting + + def check_max_nesting(depth) # :nodoc: + return if @max_nesting.zero? + current_nesting = depth + 1 + current_nesting > @max_nesting and + raise NestingError, "nesting of #{current_nesting} is too deep" + end + + # Returns true, if circular data structures should be checked, + # otherwise returns false. + def check_circular? + @check_circular + end + + # Returns true if NaN, Infinity, and -Infinity should be considered as + # valid JSON and output. + def allow_nan? + @allow_nan + end + + # Returns _true_, if _object_ was already seen during this generating + # run. + def seen?(object) + @seen.key?(object.__id__) + end + + # Remember _object_, to find out if it was already encountered (if a + # cyclic data structure is if a cyclic data structure is rendered). + def remember(object) + @seen[object.__id__] = true + end + + # Forget _object_ for this generating run. + def forget(object) + @seen.delete object.__id__ + end + + # Configure this State instance with the Hash _opts_, and return + # itself. + def configure(opts) + @indent = opts[:indent] if opts.key?(:indent) + @space = opts[:space] if opts.key?(:space) + @space_before = opts[:space_before] if opts.key?(:space_before) + @object_nl = opts[:object_nl] if opts.key?(:object_nl) + @array_nl = opts[:array_nl] if opts.key?(:array_nl) + @check_circular = !!opts[:check_circular] if opts.key?(:check_circular) + @allow_nan = !!opts[:allow_nan] if opts.key?(:allow_nan) + if !opts.key?(:max_nesting) # defaults to 19 + @max_nesting = 19 + elsif opts[:max_nesting] + @max_nesting = opts[:max_nesting] + else + @max_nesting = 0 + end + self + end + + # Returns the configuration instance variables as a hash, that can be + # passed to the configure method. + def to_h + result = {} + for iv in %w[indent space space_before object_nl array_nl check_circular allow_nan max_nesting] + result[iv.intern] = instance_variable_get("@#{iv}") + end + result + end + end + + module GeneratorMethods + module Object + # Converts this object to a string (calling #to_s), converts + # it to a JSON string, and returns the result. This is a fallback, if no + # special method #to_json was defined for some object. + def to_json(*) to_s.to_json end + end + + module Hash + # Returns a JSON string containing a JSON object, that is unparsed from + # this Hash instance. + # _state_ is a JSON::State object, that can also be used to configure the + # produced JSON string output further. + # _depth_ is used to find out nesting depth, to indent accordingly. + def to_json(state = nil, depth = 0, *) + if state + state = JSON.state.from_state(state) + state.check_max_nesting(depth) + json_check_circular(state) { json_transform(state, depth) } + else + json_transform(state, depth) + end + end + + private + + def json_check_circular(state) + if state and state.check_circular? + state.seen?(self) and raise JSON::CircularDatastructure, + "circular data structures not supported!" + state.remember self + end + yield + ensure + state and state.forget self + end + + def json_shift(state, depth) + state and not state.object_nl.empty? or return '' + state.indent * depth + end + + def json_transform(state, depth) + delim = ',' + if state + delim << state.object_nl + result = '{' + result << state.object_nl + result << map { |key,value| + s = json_shift(state, depth + 1) + s << key.to_s.to_json(state, depth + 1) + s << state.space_before + s << ':' + s << state.space + s << value.to_json(state, depth + 1) + }.join(delim) + result << state.object_nl + result << json_shift(state, depth) + result << '}' + else + result = '{' + result << map { |key,value| + key.to_s.to_json << ':' << value.to_json + }.join(delim) + result << '}' + end + result + end + end + + module Array + # Returns a JSON string containing a JSON array, that is unparsed from + # this Array instance. + # _state_ is a JSON::State object, that can also be used to configure the + # produced JSON string output further. + # _depth_ is used to find out nesting depth, to indent accordingly. + def to_json(state = nil, depth = 0, *) + if state + state = JSON.state.from_state(state) + state.check_max_nesting(depth) + json_check_circular(state) { json_transform(state, depth) } + else + json_transform(state, depth) + end + end + + private + + def json_check_circular(state) + if state and state.check_circular? + state.seen?(self) and raise JSON::CircularDatastructure, + "circular data structures not supported!" + state.remember self + end + yield + ensure + state and state.forget self + end + + def json_shift(state, depth) + state and not state.array_nl.empty? or return '' + state.indent * depth + end + + def json_transform(state, depth) + delim = ',' + if state + delim << state.array_nl + result = '[' + result << state.array_nl + result << map { |value| + json_shift(state, depth + 1) << value.to_json(state, depth + 1) + }.join(delim) + result << state.array_nl + result << json_shift(state, depth) + result << ']' + else + '[' << map { |value| value.to_json }.join(delim) << ']' + end + end + end + + module Integer + # Returns a JSON string representation for this Integer number. + def to_json(*) to_s end + end + + module Float + # Returns a JSON string representation for this Float number. + def to_json(state = nil, *) + case + when infinite? + if !state || state.allow_nan? + to_s + else + raise GeneratorError, "#{self} not allowed in JSON" + end + when nan? + if !state || state.allow_nan? + to_s + else + raise GeneratorError, "#{self} not allowed in JSON" + end + else + to_s + end + end + end + + module String + # This string should be encoded with UTF-8 A call to this method + # returns a JSON string encoded with UTF16 big endian characters as + # \u????. + def to_json(*) + '"' << JSON.utf8_to_json(self) << '"' + end + + # Module that holds the extinding methods if, the String module is + # included. + module Extend + # Raw Strings are JSON Objects (the raw bytes are stored in an array for the + # key "raw"). The Ruby String can be created by this module method. + def json_create(o) + o['raw'].pack('C*') + end + end + + # Extends _modul_ with the String::Extend module. + def self.included(modul) + modul.extend Extend + end + + # This method creates a raw object hash, that can be nested into + # other data structures and will be unparsed as a raw string. This + # method should be used, if you want to convert raw strings to JSON + # instead of UTF-8 strings, e. g. binary data. + def to_json_raw_object + { + JSON.create_id => self.class.name, + 'raw' => self.unpack('C*'), + } + end + + # This method creates a JSON text from the result of + # a call to to_json_raw_object of this String. + def to_json_raw(*args) + to_json_raw_object.to_json(*args) + end + end + + module TrueClass + # Returns a JSON string for true: 'true'. + def to_json(*) 'true' end + end + + module FalseClass + # Returns a JSON string for false: 'false'. + def to_json(*) 'false' end + end + + module NilClass + # Returns a JSON string for nil: 'null'. + def to_json(*) 'null' end + end + end + end + end +end diff --git a/lib/puppet/external/json/pure/parser.rb b/lib/puppet/external/json/pure/parser.rb new file mode 100644 index 0000000..7e8fe08 --- /dev/null +++ b/lib/puppet/external/json/pure/parser.rb @@ -0,0 +1,269 @@ +require 'strscan' + +module JSON + module Pure + # This class implements the JSON parser that is used to parse a JSON string + # into a Ruby data structure. + class Parser < StringScanner + STRING = /" ((?:[^\x0-\x1f"\\] | + # escaped special characters: + \\["\\\/bfnrt] | + \\u[0-9a-fA-F]{4} | + # match all but escaped special characters: + \\[\x20-\x21\x23-\x2e\x30-\x5b\x5d-\x61\x63-\x65\x67-\x6d\x6f-\x71\x73\x75-\xff])*) + "/nx + INTEGER = /(-?0|-?[1-9]\d*)/ + FLOAT = /(-? + (?:0|[1-9]\d*) + (?: + \.\d+(?i:e[+-]?\d+) | + \.\d+ | + (?i:e[+-]?\d+) + ) + )/x + NAN = /NaN/ + INFINITY = /Infinity/ + MINUS_INFINITY = /-Infinity/ + OBJECT_OPEN = /\{/ + OBJECT_CLOSE = /\}/ + ARRAY_OPEN = /\[/ + ARRAY_CLOSE = /\]/ + PAIR_DELIMITER = /:/ + COLLECTION_DELIMITER = /,/ + TRUE = /true/ + FALSE = /false/ + NULL = /null/ + IGNORE = %r( + (?: + //[^\n\r]*[\n\r]| # line comments + /\* # c-style comments + (?: + [^*/]| # normal chars + /[^*]| # slashes that do not start a nested comment + \*[^/]| # asterisks that do not end this comment + /(?=\*/) # single slash before this comment's end + )* + \*/ # the End of this comment + |[ \t\r\n]+ # whitespaces: space, horicontal tab, lf, cr + )+ + )mx + + UNPARSED = Object.new + + # Creates a new JSON::Pure::Parser instance for the string _source_. + # + # It will be configured by the _opts_ hash. _opts_ can have the following + # keys: + # * *max_nesting*: The maximum depth of nesting allowed in the parsed data + # structures. Disable depth checking with :max_nesting => false|nil|0, + # it defaults to 19. + # * *allow_nan*: If set to true, allow NaN, Infinity and -Infinity in + # defiance of RFC 4627 to be parsed by the Parser. This option defaults + # to false. + # * *create_additions*: If set to false, the Parser doesn't create + # additions even if a matchin class and create_id was found. This option + # defaults to true. + # * *object_class*: Defaults to Hash + # * *array_class*: Defaults to Array + def initialize(source, opts = {}) + super + if !opts.key?(:max_nesting) # defaults to 19 + @max_nesting = 19 + elsif opts[:max_nesting] + @max_nesting = opts[:max_nesting] + else + @max_nesting = 0 + end + @allow_nan = !!opts[:allow_nan] + ca = true + ca = opts[:create_additions] if opts.key?(:create_additions) + @create_id = ca ? JSON.create_id : nil + @object_class = opts[:object_class] || Hash + @array_class = opts[:array_class] || Array + end + + alias source string + + # Parses the current JSON string _source_ and returns the complete data + # structure as a result. + def parse + reset + obj = nil + until eos? + case + when scan(OBJECT_OPEN) + obj and raise ParserError, "source '#{peek(20)}' not in JSON!" + @current_nesting = 1 + obj = parse_object + when scan(ARRAY_OPEN) + obj and raise ParserError, "source '#{peek(20)}' not in JSON!" + @current_nesting = 1 + obj = parse_array + when skip(IGNORE) + ; + else + raise ParserError, "source '#{peek(20)}' not in JSON!" + end + end + obj or raise ParserError, "source did not contain any JSON!" + obj + end + + private + + # Unescape characters in strings. + UNESCAPE_MAP = Hash.new { |h, k| h[k] = k.chr } + UNESCAPE_MAP.update({ + ?" => '"', + ?\\ => '\\', + ?/ => '/', + ?b => "\b", + ?f => "\f", + ?n => "\n", + ?r => "\r", + ?t => "\t", + ?u => nil, + }) + + def parse_string + if scan(STRING) + return '' if self[1].empty? + string = self[1].gsub(%r((?:\\[\\bfnrt"/]|(?:\\u(?:[A-Fa-f\d]{4}))+|\\[\x20-\xff]))n) do |c| + if u = UNESCAPE_MAP[$&[1]] + u + else # \uXXXX + bytes = '' + i = 0 + while c[6 * i] == ?\\ && c[6 * i + 1] == ?u + bytes << c[6 * i + 2, 2].to_i(16) << c[6 * i + 4, 2].to_i(16) + i += 1 + end + JSON::UTF16toUTF8.iconv(bytes) + end + end + if string.respond_to?(:force_encoding) + string.force_encoding(Encoding::UTF_8) + end + string + else + UNPARSED + end + rescue Iconv::Failure => e + raise GeneratorError, "Caught #{e.class}: #{e}" + end + + def parse_value + case + when scan(FLOAT) + Float(self[1]) + when scan(INTEGER) + Integer(self[1]) + when scan(TRUE) + true + when scan(FALSE) + false + when scan(NULL) + nil + when (string = parse_string) != UNPARSED + string + when scan(ARRAY_OPEN) + @current_nesting += 1 + ary = parse_array + @current_nesting -= 1 + ary + when scan(OBJECT_OPEN) + @current_nesting += 1 + obj = parse_object + @current_nesting -= 1 + obj + when @allow_nan && scan(NAN) + NaN + when @allow_nan && scan(INFINITY) + Infinity + when @allow_nan && scan(MINUS_INFINITY) + MinusInfinity + else + UNPARSED + end + end + + def parse_array + raise NestingError, "nesting of #...@current_nesting is too deep" if + @max_nesting.nonzero? && @current_nesting > @max_nesting + result = @array_class.new + delim = false + until eos? + case + when (value = parse_value) != UNPARSED + delim = false + result << value + skip(IGNORE) + if scan(COLLECTION_DELIMITER) + delim = true + elsif match?(ARRAY_CLOSE) + ; + else + raise ParserError, "expected ',' or ']' in array at '#{peek(20)}'!" + end + when scan(ARRAY_CLOSE) + if delim + raise ParserError, "expected next element in array at '#{peek(20)}'!" + end + break + when skip(IGNORE) + ; + else + raise ParserError, "unexpected token in array at '#{peek(20)}'!" + end + end + result + end + + def parse_object + raise NestingError, "nesting of #...@current_nesting is too deep" if + @max_nesting.nonzero? && @current_nesting > @max_nesting + result = @object_class.new + delim = false + until eos? + case + when (string = parse_string) != UNPARSED + skip(IGNORE) + unless scan(PAIR_DELIMITER) + raise ParserError, "expected ':' in object at '#{peek(20)}'!" + end + skip(IGNORE) + unless (value = parse_value).equal? UNPARSED + result[string] = value + delim = false + skip(IGNORE) + if scan(COLLECTION_DELIMITER) + delim = true + elsif match?(OBJECT_CLOSE) + ; + else + raise ParserError, "expected ',' or '}' in object at '#{peek(20)}'!" + end + else + raise ParserError, "expected value in object at '#{peek(20)}'!" + end + when scan(OBJECT_CLOSE) + if delim + raise ParserError, "expected next name, value pair in object at '#{peek(20)}'!" + end + if @create_id and klassname = resu...@create_id] + klass = JSON.deep_const_get klassname + break unless klass and klass.json_creatable? + result = klass.json_create(result) + end + break + when skip(IGNORE) + ; + else + raise ParserError, "unexpected token in object at '#{peek(20)}'!" + end + end + result + end + end + end +end diff --git a/lib/puppet/external/json/version.rb b/lib/puppet/external/json/version.rb new file mode 100644 index 0000000..788ca2e --- /dev/null +++ b/lib/puppet/external/json/version.rb @@ -0,0 +1,8 @@ +module JSON + # JSON version + VERSION = '1.1.9' + VERSION_ARRAY = VERSION.split(/\./).map { |x| x.to_i } # :nodoc: + VERSION_MAJOR = VERSION_ARRAY[0] # :nodoc: + VERSION_MINOR = VERSION_ARRAY[1] # :nodoc: + VERSION_BUILD = VERSION_ARRAY[2] # :nodoc: +end diff --git a/lib/puppet/feature/json.rb b/lib/puppet/feature/json.rb index 1f3845c..d2389cc 100644 --- a/lib/puppet/feature/json.rb +++ b/lib/puppet/feature/json.rb @@ -1,2 +1,18 @@ + # This is here so it autoloads correctly. +# We have JSON available +# This is stupid - Rails breaks json compatibility if we load json before +# rails, therefore we load rails and then json. Dumb, mother-dumb. Puppet.features.rails? + +Puppet.features.add(:json) do + begin + require 'puppet/external/json/common' + require 'puppet/external/json/version' + require 'puppet/external/json/pure' + rescue Exception => details + p details + print [details.backtrace].join("\n") + end + true +end diff --git a/lib/puppet/feature/rails.rb b/lib/puppet/feature/rails.rb index 87b408c..a290072 100644 --- a/lib/puppet/feature/rails.rb +++ b/lib/puppet/feature/rails.rb @@ -35,8 +35,3 @@ Puppet.features.add(:rails) do true end end - -# We have JSON available -# This is stupid - Rails breaks json compatibility if we load json before -# rails, therefore we load rails and then json. Dumb, mother-dumb. -Puppet.features.add(:json, :libs => ["json"]) diff --git a/spec/unit/relationship.rb b/spec/unit/relationship.rb index fd7e0aa..f202fd9 100755 --- a/spec/unit/relationship.rb +++ b/spec/unit/relationship.rb @@ -163,7 +163,7 @@ describe Puppet::Relationship, "when converting to json" do end def json_output_should - @edge.class.expects(:json_create).with { |hash| yield hash } + @edge.class.expects(:json_create).with { |hash| yield hash }.returns(:something) end # LAK:NOTE For all of these tests, we convert back to the edge so we can diff --git a/spec/unit/resource/catalog.rb b/spec/unit/resource/catalog.rb index af399aa..cef3224 100755 --- a/spec/unit/resource/catalog.rb +++ b/spec/unit/resource/catalog.rb @@ -883,7 +883,7 @@ describe Puppet::Resource::Catalog, "when converting to json" do end def json_output_should - @catalog.class.expects(:json_create).with { |hash| yield hash } + @catalog.class.expects(:json_create).with { |hash| yield hash }.returns(:something) end # LAK:NOTE For all of these tests, we convert back to the resource so we can -- 1.6.4 --~--~---------~--~----~------------~-------~--~----~ You received this message because you are subscribed to the Google Groups "Puppet Developers" group. To post to this group, send email to [email protected] To unsubscribe from this group, send email to [email protected] For more options, visit this group at http://groups.google.com/group/puppet-dev?hl=en -~----------~----~----~----~------~----~------~--~---
