Jira (PUP-8725) Agent Functions - Check Puppet Type Specifications against PDB

[Rob Browning (JIRA)]

Title: Message Title

Rob Browning commented on PUP-8725   Re: Agent Functions - Check Puppet Type Specifications against PDB For reference, here's a copy of the initial summary: #+TITLE: initial summary   Under the assumption that once rich data has been enabled, puppet rich data values will be submitted in puppetdb commands for storage (perhaps most likely as resource parameter values), I've been investigating how compatible the rich data types are with puppetdb's current jsonb-based storage of say resource parameters and fact values.   The primary question is whether/how puppetdb might have to change before the platform 6 release.   **** background - [[https://github.com/puppetlabs/puppet-specifications/blob/master/language/types_values_variables.md][puppet types]] - [[https://www.postgresql.org/docs/9.6/static/datatype-json.html][jsonb types]] (particularly table 8-23 and the description above it) - [[https://puppet.com/docs/puppetdb/5.2/api/query/v4/resources.html][puppetdb resource queries]]   **** rich data and postgresql type correspondance   - string :: specified as up to 2^31-1 unicode characters in puppet; maps to ~text~ in pg, which is limited to about ~1GB of storage, irrespective of the number of characters. Unicode null (\u0000) is forbidden in pg, so if we try to pass puppet strings straight through, I expect that any containing null will cause the entire command (value) to eventually be rejected (discarded to the DLO).   Perhaps also worth noting that pg doesn't support null in *any* strings, not just those within jsonb, and so this is already an issue "everywhere". I believe the terminus may still do some mangling to "avoid" the problem at the moment, but no matter what it does, since 5.1, puppedb universally replaces unicode \u0000 in incoming json with the replacement character \ufffd (cf. [[https://tickets.puppetlabs.com/browse/PDB-3648][PDB-3648]]).   - integer :: i64 in puppet; maps to ~numeric~ in pg jsonb, with up to 131072 digits before the decimal point; up to 16383 digits after the decimal point. This looks like it may be fine, though any numeric operations will be *much* slower than the more restricted pg types like ~integer~, ~double precision~, etc.   - float :: native double in puppet; maps to numeric in pg (see ~integer~ above) which forbids NaN, +Inf and -Inf. As long as those three values can't appear in rich data ([[https://github.com/puppetlabs/puppet-specifications/blob/master/language/types_values_variables.md#float-from-to][perhaps implied here]]) then puppetdb should be able to handle puppet floats. If they can appear, then we have a problem similar to that with unicode null mentioned above.   - undef :: does this appear as an actual value, and if so, how? I see that it is allowed as a hash key, so presumably it can appear.   - default :: does this appear as an actual value, and if so, how?   - boolean :: compatible with pg boolean   - array :: unlimited heterogeneous zero-indexed array in puppet; maps to possibly the same in pg, plus or minus the potential ~1GB jsonb overall doc size restriction.   - hash :: puppet specifies that key order is preserved, but pg jsonb explicitly does not preserve order. Does this require accommodation?   - everything-else :: presumably composed of the types above? If so, then all the restrictions above apply. For example, [[https://github.com/puppetlabs/puppet-specifications/blob/master/language/types_values_variables.md#timespan-from-to][Timespan]] cannot use the floating point infinities to represent open-ended ranges and still be compatible with pg jsonb.   Add Comment   This message was sent by Atlassian JIRA (v7.7.1#77002-sha1:e75ca93) -- You received this message because you are subscribed to the Google Groups "Puppet Bugs" group. To unsubscribe from this group and stop receiving emails from it, send an email to puppet-bugs+unsubscr...@googlegroups.com. To post to this group, send email to puppet-bugs@googlegroups.com. Visit this group at https://groups.google.com/group/puppet-bugs. For more options, visit https://groups.google.com/d/optout.