[
https://issues.apache.org/jira/browse/NIFI-12456?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=18074434#comment-18074434
]
Daniel Stieglitz edited comment on NIFI-12456 at 4/17/26 9:22 PM:
------------------------------------------------------------------
[~markap14] I have been researching this to see what needs to be implemented. I
am just trying to understand which features you had in mind which should be
implemented. In NIFI-12697, the following were implemented
||Class||Features||
|AbstractJsonRowRecordReader|com.fasterxml.jackson.core.JsonParser.Feature.USE_FAST_DOUBLE_PARSER
com.fasterxml.jackson.core.JsonParser.Feature.USE_FAST_BIG_NUMBER_PARSER|
|WriteJsonResult|com.fasterxml.jackson.core.JsonGenerator.Feature.WRITE_BIGDECIMAL_AS_PLAIN|
The Jackson Javadocs indicate most members in
com.fasterxml.jackson.core.JsonParser.Feature are deprecated, if not deprecated
are encouraged to use the equivalents in
com.fasterxml.jackson.core.json.JsonReadFeature or are not deprecated and
unique to this class.
The following unique to com.fasterxml.jackson.core.JsonParser.Feature which I
did not think need to be considered as I believe are low level details or may
incur a increased processing time and the defaults are sufficient:
* AUTO_CLOSE_SOURCE
* CLEAR_CURRENT_TOKEN_ON_CLOSE
* IGNORE_UNDEFINED
* INCLUDE_SOURCE_IN_LOCATION
* STRICT_DUPLICATE_DETECTION
Possible features to implement from
com.fasterxml.jackson.core.json.JsonReadFeature are:
* ALLOW_JAVA_COMMENTS
* ALLOW_LEADING_PLUS_SIGN_FOR_NUMBERS
* ALLOW_LEADING_ZEROS_FOR_NUMBERS
* ALLOW_MISSING_VALUES
* ALLOW_NON_NUMERIC_NUMBERS
* ALLOW_SINGLE_QUOTES
* ALLOW_TRAILING_COMMA
* ALLOW_TRAILING_DECIMAL_POINT_FOR_NUMBERS
* ALLOW_UNQUOTED_FIELD_NAMES
* ALLOW_YAML_COMMENTS
Those not considered are:
* ALLOW_RS_CONTROL_CHAR
* ALLOW_UNESCAPED_CONTROL_CHARS
The Jackson Javadocs indicate most members in
com.fasterxml.jackson.core.JsonGenerator.Feature are deprecated, if not
deprecated are encouraged to use the equivalents in
com.fasterxml.jackson.core.json.JsonWriteFeature or are not deprecated and
unique to this class.
Those unique to com.fasterxml.jackson.core.JsonGenerator.Feature which I did
not think need to be considered as I believe are low level details or may incur
a increased processing time and the defaults are sufficient are:
* AUTO_CLOSE_JSON_CONTENT
* AUTO_CLOSE_TARGET
* FLUSH_PASSED_TO_STREAM
* IGNORE_UNKNOWN
* STRICT_DUPLICATE_DETECTION
Possible features to implement from
com.fasterxml.jackson.core.json.JsonWriteFeature are:
* ESCAPE_FORWARD_SLASHES
* ESCAPE_NON_ASCII ( a feature I added to the JoltTransformJSON processor in
NIFI-9369)
* QUOTE_FIELD_NAMES
* WRITE_HEX_UPPER_CASE
* WRITE_NAN_AS_STRINGS
* WRITE_NUMBERS_AS_STRINGS
Those not considered are:
* COMBINE_UNICODE_SURROGATES_IN_UTF8
Please let me know whether you concur with analysis. Thanks!
was (Author: JIRAUSER294662):
[~markap14] I have been researching this to see what needs to be implemented. I
am just trying to understand which features you had in mind which should be
implemented. In NIFI-12697, the following were implemented
||Class||Features||
|AbstractJsonRowRecordReader|com.fasterxml.jackson.core.JsonParser.Feature.USE_FAST_DOUBLE_PARSER
com.fasterxml.jackson.core.JsonParser.Feature.USE_FAST_BIG_NUMBER_PARSER|
|WriteJsonResult|com.fasterxml.jackson.core.JsonGenerator.Feature.WRITE_BIGDECIMAL_AS_PLAIN|
The Jackson Javadocs indicate most members in
com.fasterxml.jackson.core.JsonParser.Feature are deprecated, if not deprecated
are encouraged to use the equivalents in
com.fasterxml.jackson.core.json.JsonReadFeature or are not deprecated and
unique to this class.
The following unique to com.fasterxml.jackson.core.JsonParser.Feature which I
did not think need to be considered as I believe are low level details or may
incur a increased processing time and the defaults are sufficient:
* AUTO_CLOSE_SOURCE
* CLEAR_CURRENT_TOKEN_ON_CLOSE
* IGNORE_UNDEFINED
* INCLUDE_SOURCE_IN_LOCATION
* STRICT_DUPLICATE_DETECTION
Possible features to implement from
com.fasterxml.jackson.core.json.JsonReadFeature are:
* ALLOW_JAVA_COMMENTS
* ALLOW_LEADING_PLUS_SIGN_FOR_NUMBERS
* ALLOW_LEADING_ZEROS_FOR_NUMBERS
* ALLOW_MISSING_VALUES
* ALLOW_NON_NUMERIC_NUMBERS
* ALLOW_SINGLE_QUOTES
* ALLOW_TRAILING_COMMA
* ALLOW_TRAILING_DECIMAL_POINT_FOR_NUMBERS
* ALLOW_UNQUOTED_FIELD_NAMES
* ALLOW_YAML_COMMENTS
Those not considered are:
* ALLOW_RS_CONTROL_CHAR
* ALLOW_UNESCAPED_CONTROL_CHARS
The Jackson Javadocs indicate most members in
com.fasterxml.jackson.core.JsonGenerator.Feature are deprecated, if not
deprecated are encouraged to use the equivalents in
com.fasterxml.jackson.core.json.JsonWriteFeature or are not deprecated and
unique to this class.
Those unique to com.fasterxml.jackson.core.JsonGenerator.Feature which I did
not think need to be considered as I believe are low level details or may incur
a increased processing time and the defaults are sufficient are:
* AUTO_CLOSE_JSON_CONTENT
* AUTO_CLOSE_TARGET
* FLUSH_PASSED_TO_STREAM
* IGNORE_UNKNOWN
* STRICT_DUPLICATE_DETECTION
Possible features to implement from
com.fasterxml.jackson.core.json.JsonWriteFeature are:
* ESCAPE_FORWARD_SLASHES
* ESCAPE_NON_ASCII ( a feature I added to the JoltTransformJSON processor in
NIFI-9369)
* QUOTE_FIELD_NAMES
* WRITE_HEX_UPPER_CASE
* WRITE_NAN_AS_STRINGS
* WRITE_NUMBERS_AS_STRINGS
Those not considered are:
* COMBINE_UNICODE_SURROGATES_IN_UTF8
Please let me know whether you concur with analysis. Thanks!
> Improve leniency of JSON readers and flexibility of JSON Writer
> ---------------------------------------------------------------
>
> Key: NIFI-12456
> URL: https://issues.apache.org/jira/browse/NIFI-12456
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Extensions
> Reporter: Mark Payne
> Priority: Major
>
> Currently, we adhere to the JSON specification fairly strictly, with the
> exception of allowing for "JSON Lines" / ndjson / ldjson.
> However, the Jackson library allows for several {{Features}} that we do not
> expose, which may be helpful for handling data that does not strictly adhere
> to the schema, or where there are preferences in serialization.
> For example, {{JsonParser.Feature}} allows for the ability to allow comments
> in JSON (to include lines beginning with {{{}//{}}}, {{{}/*{}}}, and "YAML
> Style" comments (#)). Additionally, it allows for single-quotes for field
> names or no quoting at all. While these do not adhere to the specification,
> they are common enough for the parser to support them, and we should too.
> Similarly, on the serialization side, we have had requests to support writing
> decimal values without use of scientific notation, which can be achieved by
> enabling the {{WRITE_BIGDECIMAL_AS_PLAIN}} feature.
> We should expose these options on the JsonTreeReader and the JSON Writer. I
> don't know of any downside to enabling the leniency / non-standard options,
> so it probably makes sense to simply enable them all by default. Though there
> is argument for introducing a new "Parsing Leniency" option that allows the
> user to disable these features.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
