This is a bit orthogonal, but in JsonSchemaConverter I use JSONSchemas to indicate whether a JSON number should be deserialized as an integer or a decimal <https://github.com/ottomata/kafka-connect-jsonschema/blob/master/src/main/java/org/wikimedia/kafka/connect/jsonschema/JsonSchemaConverter.java#L251-L261>. Not everyone is going to have JSONSchemas available when converting, but if you do, it is an easy way to support JSON numbers as decimals.
Carry on! :) On Thu, Jul 25, 2019 at 9:12 AM Andy Coates <a...@confluent.io> wrote: > Hi Almog, > > Like the KIP - I think being able to support decimals in JSON in the same > way most other systems do is a great improvement. > > It's not 100% clear to me from the KIP what the current format is. Is the > text decimal a base16 encoded number, or is it base16 encoded binary form > of the number? (I've not tried to get my head around if these two are even > different!) > > One suggestion I have is to change the proposed new config to only affect > decimals stored as text, i.e. to switch between the current base16 and the > more common base10. Then add another config to the serialzier only that > controls if decimals should be serialized as text or numeric. The benefit > of this approach is it allows us to enhance the deserializer to > automatically handle numeric decimals even without any config having to be > set, i.e. default config in the deserializer would be able to handle > numeric decimals. Of course, this is a two edged sword: this would make > the deserializer work out of the box with numeric decimals, (yay!), but > would be a breaking change in that things that previously failed would > suddenly start deserializing. This is a price I'm willing to pay. > > IMHO, we should then plan to switch the default of decimal serialization to > numeric, and text serialization to base 10 in the next major release. > (With upgrade notes to match). Though I know this is more contentious, I > think it moves us forward in a much more standard way that the current > encoding of decimals. > > On Tue, 25 Jun 2019 at 01:03, Almog Gavra <al...@confluent.io> wrote: > > > Hi Everyone! > > > > Kicking off discussion for a new KIP: > > > > > https://cwiki.apache.org/confluence/display/KAFKA/KIP-481%3A+SerDe+Improvements+for+Connect+Decimal+type+in+JSON > > > > For those who are interested, I have a prototype implementation that > helped > > guide my design: https://github.com/agavra/kafka/pull/1 > > > > Cheers, > > Almog > > >
