markap14 commented on a change in pull request #4934:
URL: https://github.com/apache/nifi/pull/4934#discussion_r604177468
##########
File path:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/json/AbstractJsonRowRecordReader.java
##########
@@ -176,6 +177,30 @@ protected Object getRawNodeValue(final JsonNode fieldNode,
final DataType dataTy
if (dataType != null && dataType.getFieldType() ==
RecordFieldType.ARRAY) {
final ArrayDataType arrayDataType = (ArrayDataType) dataType;
elementDataType = arrayDataType.getElementType();
+ } else if (dataType != null && dataType.getFieldType() ==
RecordFieldType.CHOICE) {
+ List<DataType> possibleSubTypes =
((ChoiceDataType)dataType).getPossibleSubTypes();
+
+ for (DataType possibleSubType : possibleSubTypes) {
+ if (possibleSubType.getFieldType() ==
RecordFieldType.ARRAY) {
+ ArrayDataType possibleArrayDataType =
(ArrayDataType)possibleSubType;
+ DataType possibleElementType =
possibleArrayDataType.getElementType();
+
+ final Object[] possibleArrayElements = new
Object[numElements];
+ int elementCounter = 0;
+ for (final JsonNode node : arrayNode) {
+ final Object value = getRawNodeValue(node,
possibleElementType, fieldName);
+ possibleArrayElements[elementCounter++] = value;
+ }
+
+ if
(DataTypeUtils.isArrayTypeCompatible(possibleArrayElements,
possibleElementType, true)) {
+ return possibleArrayElements;
+ }
+ }
+ }
+
+ logger.warn("Couldn't find proper schema for '{}'. This could
lead to data loss as fields might end up missing in the output!", fieldName);
Review comment:
I don't think this is an appropriate warning. In this situation, the
JSON has an array element. It is up to the caller of this method to determine
what to do with the elements that are returned. If the elements are iterated
over only by looking at the schema, then it is possible that these elements
could be skipped - and that is intentional, not cause for a warning. It's also
possible, as may be the case in ValidateRecord that the elements will not be
skipped. This is the same as the behavior before this PR, and the same behavior
that exists throughout most of the record handling. At most, I think this needs
to be a DEBUG level log message.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
