[
https://issues.apache.org/jira/browse/NIFI-5491?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16570594#comment-16570594
]
ASF GitHub Bot commented on NIFI-5491:
--------------------------------------
Github user markap14 commented on a diff in the pull request:
https://github.com/apache/nifi/pull/2938#discussion_r207981236
--- Diff:
nifi-nar-bundles/nifi-hive-bundle/nifi-hive3-processors/src/main/java/org/apache/hive/streaming/NiFiRecordSerDe.java
---
@@ -227,8 +236,32 @@ private Object extractCurrentField(Record record,
RecordField field, TypeInfo fi
val =
DataTypeUtils.convertRecordFieldtoObject(record.getValue(fieldName),
field.getDataType());
break;
case STRUCT:
- val =
DataTypeUtils.convertRecordFieldtoObject(record.getValue(fieldName),
field.getDataType());
- break;
+ // For some reason the Hive StandardStructObjectInspector
expects the object corresponding to a "struct" to be an array or List rather
than a Map.
+ // Do the conversion here, calling extractCurrentField
recursively to traverse any nested structs.
+ Record r = (Record) record.getValue(fieldName);
+ if (r == null) {
+ return null;
+ }
+ try {
+ RecordSchema recordSchema = r.getSchema();
+ List<RecordField> recordFields =
recordSchema.getFields();
+ if (recordFields == null || recordFields.isEmpty()) {
+ return new ArrayList<>(0);
--- End diff --
should just return Collections.emptyList()
> PutHive3Streaming incorrectly handles bytes, shorts, and nested structs
> -----------------------------------------------------------------------
>
> Key: NIFI-5491
> URL: https://issues.apache.org/jira/browse/NIFI-5491
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Reporter: Matt Burgess
> Assignee: Matt Burgess
> Priority: Major
>
> When trying to insert a record into a Hive table using PutHive3Streaming, if
> the table contains columns of types byte, short, or struct, then an error
> occurs and the records cannot be written.
> This is due to a mismatch between the data types used in the NiFi Record API
> and the Hive ORC writer and StructObjectInspector. For byte and short, NiFi
> currently maintains an Integer value but Hive expects a Byte or Short,
> respectively. For structs, NiFi maintains a Map value but Hive expects a List
> or array.
> NiFiRecordSerDe should handle the conversion of values for use by Hive
> Streaming.
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
