[jira] [Resolved] (NIFI-7891) Allow for Defaults to be set by Avro writers
[
https://issues.apache.org/jira/browse/NIFI-7891?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Thomsen resolved NIFI-7891.
Resolution: Won't Do
As mentioned on GitHub, the contribution is appreciated, but we have to pass
because it would introduce a lot of potentially disruptive behavior to users of
schema inference.
> Allow for Defaults to be set by Avro writers
>
>
> Key: NIFI-7891
> URL: https://issues.apache.org/jira/browse/NIFI-7891
> Project: Apache NiFi
> Issue Type: New Feature
> Components: Extensions
>Reporter: Alasdair Brown
>Assignee: Alasdair Brown
>Priority: Minor
> Time Spent: 40m
> Remaining Estimate: 0h
>
> This would be an improvement to Avro writer services. Allowing a set of
> default values to be given based on the data type when infering the schema.
>
> For example, you could have an additional property per type (e.g. Int,
> String, etc.) where the value of the property would be used as the default
> value. However, this would add a lot of properties Possibly you could
> provide a JSON set of types -> defaults, e.g.
> {code:java}
> [
>{ "type":"string", "default":"test"},
>{ "type":"int","default":1},
> ]{code}
> Thus, for any field inferred as type String, it's embedded Avro schema
> contains a default value entry for that field.
> This would then provide flexibility where NULL values are present - the
> current behaviour only supports adding NULL to the allowable types - this
> would be a very useful alternative.
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] MikeThomsen closed pull request #4591: NIFI-7891 Allow for Defaults to be set by Avro writers
MikeThomsen closed pull request #4591: URL: https://github.com/apache/nifi/pull/4591 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] MikeThomsen commented on pull request #4648: NIFI-7981 - add support for enum type in avro schema
MikeThomsen commented on pull request #4648: URL: https://github.com/apache/nifi/pull/4648#issuecomment-722721023 @markap14 has any work, even just planning, started on restricted types? I agree with your points, and if there are plans to proceed on such a feature set it would make sense to back this type out and plan to massage enum functionality into that. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on pull request #4648: NIFI-7981 - add support for enum type in avro schema
markap14 commented on pull request #4648: URL: https://github.com/apache/nifi/pull/4648#issuecomment-722710471 I'm personally not a fan of introducing Enum here as a datatype. I avoided introducing that as one of the data types when originally creating the Record API. What I think makes far more sense is a more general notion of restrictions of types. Consider, for example, how XSD allows you to add a restriction to a base type. We want to eventually support specifying schemas using XSD, and we'll need to support this notion in order to make that happen. I would certainly favor introducing that notion now over introducing an Enum. Then, when converting an Avro schema into a RecordSchema, we could do so as a restriction on the String type. This provides far more power and flexibility when using schemas, such as specifying that an int must be in the range of 1-100, and this then becomes extremely powerful when coupled with something like ValidateRecord. Adding an enum at this time, IMO, makes this a bit awkward, because it is then a specific type whose purpose is to introduce a restriction on another type (I.e., it is basically a Restriction on a String type). So I think it kind of makes things confusing if our API provides two mechanisms for specifying an enumeration: using an enum type, or specifying a restriction on a String type... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on a change in pull request #4648: NIFI-7981 - add support for enum type in avro schema
markap14 commented on a change in pull request #4648:
URL: https://github.com/apache/nifi/pull/4648#discussion_r518437335
##
File path:
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
##
@@ -1025,6 +1029,17 @@ public static boolean isStringTypeCompatible(final
Object value) {
return value != null;
}
+public static boolean isEnumTypeCompatible(final Object value, final
EnumDataType enumType) {
+return enumType.getEnums() != null &&
enumType.getEnums().contains(value);
+}
+
+private static Object toEnum(Object value, EnumDataType dataType, String
fieldName) {
+if(dataType.getEnums() != null && dataType.getEnums().contains(value))
{
Review comment:
We should be very careful here - there is no spec. We absolutely do not
- and should not - follow the Avro spec. We should do what makes sense for us.
Personally I am not a fan of introducing Enums at all. I very intentionally
didn't add those in as a data type when we created the initial set. More on
that in a general comment below :)
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] dependabot[bot] opened a new pull request #4649: Bump jetty.version from 9.4.26.v20200117 to 9.4.34.v20201102
dependabot[bot] opened a new pull request #4649: URL: https://github.com/apache/nifi/pull/4649 Bumps `jetty.version` from 9.4.26.v20200117 to 9.4.34.v20201102. Updates `jetty-server` from 9.4.26.v20200117 to 9.4.34.v20201102 Release notes Sourced from https://github.com/eclipse/jetty.project/releases;>jetty-server's releases. 9.4.34.v20201102 Bugs https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5320;>#5320 : Using WebSocketClient with jetty-websocket-httpclient.xml in a Jetty web application causes ClassCastException https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5521;>#5521 : ResourceCollection NPE in list() https://github-redirect.dependabot.com/eclipse/jetty.project/issues/;># : NPE for servlet with no mapping Enhancements https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5488;>#5488 : jetty-dir.css not found when using JPMS https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5498;>#5498 : ServletHolder lifecycle correctness https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5535;>#5535 : Support regex in SslContextFactory include/exclude of protocols 9.4.33.v20201020 Changes https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5022;>#5022 : Cleanup ServletHandler, specifically with respect to making filter chains more extensible https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5368;>#5368 : WebSocket text event execute in same thread as running binary event and destroy Threadlocal https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5378;>#5378 : Filter/Servlet/Listener Holders are not started if added during STARTING state. https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5409;>#5409 : HttpClient fails intermittently with Invalid response state TRANSIENT https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5417;>#5417 : Badly configured HttpConfiguration.securePort can lead to wrong port produced by ForwardedHeader https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5443;>#5443 : Request without Host header fails with NullPointerException in ForwardedRequestCustomizer https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5451;>#5451 : Improve Working Directory creation https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5454;>#5454 : Request error context is not reset https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5475;>#5475 : Update to spifly 1.3.2 and asm 9 https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5480;>#5480 : NPE from WebInfConfiguration.deconfigure during WebAppContext shutdown 9.4.32.v20200930 Changelog https://github-redirect.dependabot.com/eclipse/jetty.project/issues/2796;>#2796 : HTTP/2 max local stream count exceeded when request fails https://github-redirect.dependabot.com/eclipse/jetty.project/issues/3766;>#3766 : Introduce HTTP/2 API to batch frames https://github-redirect.dependabot.com/eclipse/jetty.project/issues/3916;>#3916 : multipart/byterange output is invalid to RFC7233 https://github-redirect.dependabot.com/eclipse/jetty.project/issues/4809;>#4809 : Set a max number of requests per connection https://github-redirect.dependabot.com/eclipse/jetty.project/issues/4824;>#4824 : WebSocket server outgoing message queue memory growth https://github-redirect.dependabot.com/eclipse/jetty.project/issues/4888;>#4888 : Request getSession() method throws IllegalStateException when Session exists https://github-redirect.dependabot.com/eclipse/jetty.project/issues/4954;>#4954 : Simplify ability to get Byte Counts about requests https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5032;>#5032 : Introduce Listeners to aid in tracking timings within ServletContext and WebApp https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5079;>#5079 : :authority header for IPv6 address in HTTP/2 not having square brackets https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5103;>#5103 : Proxy sets protocol version to 2.0 instead of 1.1 when accessed from H2 client https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5104;>#5104 : AbstractProxyServlet include incorrect protocol version in Via header when accessed over H2 https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5105;>#5105 : Graceful shutdown does not wait for resumed requests https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5108;>#5108 : Improve SessionTracker scalability https://github-redirect.dependabot.com/eclipse/jetty.project/issues/5121;>#5121 : Add if (LOG.isDebugEnabled()) in CompressExtension.decompress
[jira] [Updated] (NIFI-7981) ConvertRecord cannot handle open enum in Avro
[
https://issues.apache.org/jira/browse/NIFI-7981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mike Thomsen updated NIFI-7981:
---
Fix Version/s: 1.13.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> ConvertRecord cannot handle open enum in Avro
> -
>
> Key: NIFI-7981
> URL: https://issues.apache.org/jira/browse/NIFI-7981
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.12.1
>Reporter: Christophe Monnet
>Assignee: Pierre Villard
>Priority: Minor
> Fix For: 1.13.0
>
> Attachments: record_enum.xml
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> This avro schema is using the open enum pattern:
> {code:json}
> {
> "namespace": "acme",
> "name": "openEnum",
> "type": "record",
> "fields": [{
> "name": "personType",
> "type": ["null", {
> "type": "enum",
> "name": "personType_enum",
> "symbols": [
> "DRIVER",
> "TECHNICIAN"
> ]
> }, "string"],
> "default": null
> }
> ]
> }
> {code}
> The *personType* can be either null, an enum, or any string. This brings
> forward-compatibility.
> Using ConvertRecord with
> {"personType":"DRIVER"}
> as a payload, it works.
> But with
> {"personType":"DUDE"}
> it fails with {{java.lang.NullPointerException: null of acme.personType_enum
> of union in field personType of acme.openEnum}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4648: NIFI-7981 - add support for enum type in avro schema
asfgit closed pull request #4648: URL: https://github.com/apache/nifi/pull/4648 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-7981) ConvertRecord cannot handle open enum in Avro
[
https://issues.apache.org/jira/browse/NIFI-7981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17227076#comment-17227076
]
ASF subversion and git services commented on NIFI-7981:
---
Commit 14ec02f21dde3489e02b94c204eb9b9e24f3c665 in nifi's branch
refs/heads/main from Pierre Villard
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=14ec02f ]
NIFI-7981 - add support for enum type in avro schema
This closes #4648
Signed-off-by: Mike Thomsen
> ConvertRecord cannot handle open enum in Avro
> -
>
> Key: NIFI-7981
> URL: https://issues.apache.org/jira/browse/NIFI-7981
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.12.1
>Reporter: Christophe Monnet
>Assignee: Pierre Villard
>Priority: Minor
> Attachments: record_enum.xml
>
> Time Spent: 1h
> Remaining Estimate: 0h
>
> This avro schema is using the open enum pattern:
> {code:json}
> {
> "namespace": "acme",
> "name": "openEnum",
> "type": "record",
> "fields": [{
> "name": "personType",
> "type": ["null", {
> "type": "enum",
> "name": "personType_enum",
> "symbols": [
> "DRIVER",
> "TECHNICIAN"
> ]
> }, "string"],
> "default": null
> }
> ]
> }
> {code}
> The *personType* can be either null, an enum, or any string. This brings
> forward-compatibility.
> Using ConvertRecord with
> {"personType":"DRIVER"}
> as a payload, it works.
> But with
> {"personType":"DUDE"}
> it fails with {{java.lang.NullPointerException: null of acme.personType_enum
> of union in field personType of acme.openEnum}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] MikeThomsen commented on pull request #4648: NIFI-7981 - add support for enum type in avro schema
MikeThomsen commented on pull request #4648: URL: https://github.com/apache/nifi/pull/4648#issuecomment-722700884 Ran it with a live flow, and it worked as expected with my enum test schema (with both good and bad json inputs). This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-7985) ValidateRecord does not validate the type enum
[
https://issues.apache.org/jira/browse/NIFI-7985?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17227036#comment-17227036
]
Pierre Villard commented on NIFI-7985:
--
This would be fixed by the PR submitted on NIFI-7981.
> ValidateRecord does not validate the type enum
> --
>
> Key: NIFI-7985
> URL: https://issues.apache.org/jira/browse/NIFI-7985
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.9.2, 1.12.1
>Reporter: Raquel Benito Barbero
>Priority: Major
> Attachments: testIssueEnums.xml
>
>
> ValidateRecord processor does not validate correctly the enum data type
> against a flow file with JSON content. If an AVRO schema is placed with an
> enum it does not check the content of the field, just the type.
> Example:
> We have json like this:
> {"test":"test"}
>
> Then whe have the next AVRO schema:
> {
> "name": "MyClass",
> "type": "record",
> "namespace": "com.test",
> "fields": [
> {
> "name": "test",
> "type": {
> "type": "enum",
> "name": "status",
> "symbols": [
> "OK",
> "Failure"
> ]
> }
> }
> ]
> }
>
> Eventhough the word "test" is not included in one of the options for this
> field (that should be an enum type)The flowfile passes as valid. It only
> checks if it is the same type than the options inside the enum (In this case
> String)
> I made a simple template in case you want to test it.
>
> I've tested it in versions 1.9.2 and 1.12.1 and the same behavior is
> observed.
>
>
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-5536) Add EL support for password on AMQP processors
[ https://issues.apache.org/jira/browse/NIFI-5536?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17227034#comment-17227034 ] Matt Gregory commented on NIFI-5536: Any chance of reviving this one since 1.10.0? I believe the Parameter Contexts feature addresses some of the sensitive value management concerns? We do the exact same thing as the creator which works wonderfully with, for example, RedisConnectionPoolService. Seems like a one-liner in AbstractAMQPProcessor. > Add EL support for password on AMQP processors > -- > > Key: NIFI-5536 > URL: https://issues.apache.org/jira/browse/NIFI-5536 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Corey Fritz >Priority: Major > > NIFI-5489 added EL support to the host, port, virtual host, and user > properties of AMQP processors. Not sure why password was not included. We > have a use case where sensitive values (passwords) are set as environment > variables on our Docker containers and then those variables are referenced by > name using EL expressions in our processors and controller services. Flow > authors then have no need or means to know what those sensitive values are. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] MikeThomsen commented on a change in pull request #4648: NIFI-7981 - add support for enum type in avro schema
MikeThomsen commented on a change in pull request #4648:
URL: https://github.com/apache/nifi/pull/4648#discussion_r518407403
##
File path:
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
##
@@ -1025,6 +1029,17 @@ public static boolean isStringTypeCompatible(final
Object value) {
return value != null;
}
+public static boolean isEnumTypeCompatible(final Object value, final
EnumDataType enumType) {
+return enumType.getEnums() != null &&
enumType.getEnums().contains(value);
+}
+
+private static Object toEnum(Object value, EnumDataType dataType, String
fieldName) {
+if(dataType.getEnums() != null && dataType.getEnums().contains(value))
{
Review comment:
Yeah, I know the spec only allows strings, but I was wondering if wanted
to allow boolean values to be converted to strings here.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on a change in pull request #4648: NIFI-7981 - add support for enum type in avro schema
pvillard31 commented on a change in pull request #4648:
URL: https://github.com/apache/nifi/pull/4648#discussion_r518405917
##
File path:
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
##
@@ -1025,6 +1029,17 @@ public static boolean isStringTypeCompatible(final
Object value) {
return value != null;
}
+public static boolean isEnumTypeCompatible(final Object value, final
EnumDataType enumType) {
+return enumType.getEnums() != null &&
enumType.getEnums().contains(value);
+}
+
+private static Object toEnum(Object value, EnumDataType dataType, String
fieldName) {
+if(dataType.getEnums() != null && dataType.getEnums().contains(value))
{
Review comment:
I assumed that we can only have strings:
http://avro.apache.org/docs/current/spec.html#Enums
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] pvillard31 commented on a change in pull request #4648: NIFI-7981 - add support for enum type in avro schema
pvillard31 commented on a change in pull request #4648:
URL: https://github.com/apache/nifi/pull/4648#discussion_r518405814
##
File path:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/json/JsonTreeRowRecordReader.java
##
@@ -84,22 +84,6 @@ private JsonNode getChildNode(final JsonNode jsonNode, final
RecordField field)
return null;
}
-private boolean isDateTimeTimestampType(final RecordField field) {
Review comment:
yes
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] MikeThomsen commented on a change in pull request #4648: NIFI-7981 - add support for enum type in avro schema
MikeThomsen commented on a change in pull request #4648:
URL: https://github.com/apache/nifi/pull/4648#discussion_r518366732
##
File path:
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/type/EnumDataType.java
##
@@ -0,0 +1,62 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.serialization.record.type;
+
+import org.apache.nifi.serialization.record.DataType;
+import org.apache.nifi.serialization.record.RecordFieldType;
+
+import java.util.List;
+import java.util.Objects;
+
+public class EnumDataType extends DataType {
+
+private final List enums;
Review comment:
Would it make more sense to call this `symbols`, which is the
terminology that Avro uses?
##
File path:
nifi-commons/nifi-record/src/main/java/org/apache/nifi/serialization/record/util/DataTypeUtils.java
##
@@ -1025,6 +1029,17 @@ public static boolean isStringTypeCompatible(final
Object value) {
return value != null;
}
+public static boolean isEnumTypeCompatible(final Object value, final
EnumDataType enumType) {
+return enumType.getEnums() != null &&
enumType.getEnums().contains(value);
+}
+
+private static Object toEnum(Object value, EnumDataType dataType, String
fieldName) {
+if(dataType.getEnums() != null && dataType.getEnums().contains(value))
{
Review comment:
Are you assuming only conversion from strings here? Like not even
allowing people to convert booleans to TRUE and FALSE?
##
File path:
nifi-nar-bundles/nifi-standard-services/nifi-record-serialization-services-bundle/nifi-record-serialization-services/src/main/java/org/apache/nifi/json/JsonTreeRowRecordReader.java
##
@@ -84,22 +84,6 @@ private JsonNode getChildNode(final JsonNode jsonNode, final
RecordField field)
return null;
}
-private boolean isDateTimeTimestampType(final RecordField field) {
Review comment:
Was this just unused code?
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[jira] [Updated] (NIFI-7981) ConvertRecord cannot handle open enum in Avro
[
https://issues.apache.org/jira/browse/NIFI-7981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-7981:
-
Status: Patch Available (was: Open)
> ConvertRecord cannot handle open enum in Avro
> -
>
> Key: NIFI-7981
> URL: https://issues.apache.org/jira/browse/NIFI-7981
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.12.1
>Reporter: Christophe Monnet
>Assignee: Pierre Villard
>Priority: Minor
> Attachments: record_enum.xml
>
> Time Spent: 10m
> Remaining Estimate: 0h
>
> This avro schema is using the open enum pattern:
> {code:json}
> {
> "namespace": "acme",
> "name": "openEnum",
> "type": "record",
> "fields": [{
> "name": "personType",
> "type": ["null", {
> "type": "enum",
> "name": "personType_enum",
> "symbols": [
> "DRIVER",
> "TECHNICIAN"
> ]
> }, "string"],
> "default": null
> }
> ]
> }
> {code}
> The *personType* can be either null, an enum, or any string. This brings
> forward-compatibility.
> Using ConvertRecord with
> {"personType":"DRIVER"}
> as a payload, it works.
> But with
> {"personType":"DUDE"}
> it fails with {{java.lang.NullPointerException: null of acme.personType_enum
> of union in field personType of acme.openEnum}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[GitHub] [nifi] pvillard31 opened a new pull request #4648: NIFI-7981 - add support for enum type in avro schema
pvillard31 opened a new pull request #4648:
URL: https://github.com/apache/nifi/pull/4648
Thank you for submitting a contribution to Apache NiFi.
Please provide a short description of the PR here:
Description of PR
_Enables X functionality; fixes bug NIFI-._
In order to streamline the review of the contribution we ask you
to ensure the following steps have been taken:
### For all changes:
- [ ] Is there a JIRA ticket associated with this PR? Is it referenced
in the commit message?
- [ ] Does your PR title start with **NIFI-** where is the JIRA
number you are trying to resolve? Pay particular attention to the hyphen "-"
character.
- [ ] Has your PR been rebased against the latest commit within the target
branch (typically `main`)?
- [ ] Is your initial contribution a single, squashed commit? _Additional
commits in response to PR reviewer feedback should be made on this branch and
pushed to allow change tracking. Do not `squash` or use `--force` when pushing
to allow for clean monitoring of changes._
### For code changes:
- [ ] Have you ensured that the full suite of tests is executed via `mvn
-Pcontrib-check clean install` at the root `nifi` folder?
- [ ] Have you written or updated unit tests to verify your changes?
- [ ] Have you verified that the full build is successful on JDK 8?
- [ ] Have you verified that the full build is successful on JDK 11?
- [ ] If adding new dependencies to the code, are these dependencies
licensed in a way that is compatible for inclusion under [ASF
2.0](http://www.apache.org/legal/resolved.html#category-a)?
- [ ] If applicable, have you updated the `LICENSE` file, including the main
`LICENSE` file under `nifi-assembly`?
- [ ] If applicable, have you updated the `NOTICE` file, including the main
`NOTICE` file found under `nifi-assembly`?
- [ ] If adding new Properties, have you added `.displayName` in addition to
.name (programmatic access) for each of the new properties?
### For documentation related changes:
- [ ] Have you ensured that format looks appropriate for the output in which
it is rendered?
### Note:
Please ensure that once the PR is submitted, you check GitHub Actions CI for
build issues and submit an update to your PR as soon as possible.
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
[GitHub] [nifi] czobrisky commented on pull request #4554: NIFI-7842 Return Lists when multiple records are returned to the restlookupservice
czobrisky commented on pull request #4554: URL: https://github.com/apache/nifi/pull/4554#issuecomment-722554411 +1 Looks good to me. Commits need to be squashed. Is there concern about backwards compatibility? I'd say the previous implementation was a bug due to how it could handle an array response but would just pop the result, but this may break some people using the current functionality since it is a change to that. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-7985) ValidateRecord does not validate the type enum
Raquel Benito Barbero created NIFI-7985:
---
Summary: ValidateRecord does not validate the type enum
Key: NIFI-7985
URL: https://issues.apache.org/jira/browse/NIFI-7985
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.12.1, 1.9.2
Reporter: Raquel Benito Barbero
Attachments: testIssueEnums.xml
ValidateRecord processor does not validate correctly the enum data type against
a flow file with JSON content. If an AVRO schema is placed with an enum it does
not check the content of the field, just the type.
Example:
We have json like this:
{"test":"test"}
Then whe have the next AVRO schema:
{
"name": "MyClass",
"type": "record",
"namespace": "com.test",
"fields": [
{
"name": "test",
"type": {
"type": "enum",
"name": "status",
"symbols": [
"OK",
"Failure"
]
}
}
]
}
Eventhough the word "test" is not included in one of the options for this field
(that should be an enum type)The flowfile passes as valid. It only checks if it
is the same type than the options inside the enum (In this case String)
I made a simple template in case you want to test it.
I've tested it in versions 1.9.2 and 1.12.1 and the same behavior is observed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (NIFI-7981) ConvertRecord cannot handle open enum in Avro
[
https://issues.apache.org/jira/browse/NIFI-7981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard reassigned NIFI-7981:
Assignee: Pierre Villard
> ConvertRecord cannot handle open enum in Avro
> -
>
> Key: NIFI-7981
> URL: https://issues.apache.org/jira/browse/NIFI-7981
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.12.1
>Reporter: Christophe Monnet
>Assignee: Pierre Villard
>Priority: Minor
> Attachments: record_enum.xml
>
>
> This avro schema is using the open enum pattern:
> {code:json}
> {
> "namespace": "acme",
> "name": "openEnum",
> "type": "record",
> "fields": [{
> "name": "personType",
> "type": ["null", {
> "type": "enum",
> "name": "personType_enum",
> "symbols": [
> "DRIVER",
> "TECHNICIAN"
> ]
> }, "string"],
> "default": null
> }
> ]
> }
> {code}
> The *personType* can be either null, an enum, or any string. This brings
> forward-compatibility.
> Using ConvertRecord with
> {"personType":"DRIVER"}
> as a payload, it works.
> But with
> {"personType":"DUDE"}
> it fails with {{java.lang.NullPointerException: null of acme.personType_enum
> of union in field personType of acme.openEnum}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Commented] (NIFI-7981) ConvertRecord cannot handle open enum in Avro
[
https://issues.apache.org/jira/browse/NIFI-7981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17226878#comment-17226878
]
Pierre Villard commented on NIFI-7981:
--
Correction: this can be reproduced using an Avro writer.
{noformat}
[pool-1-thread-1] ERROR org.apache.nifi.processors.standard.ConvertRecord -
ConvertRecord[id=7a06f324-13d1-40da-9e17-00d902dfe958] Failed to process
FlowFile[0,person_bad_enum.json,101B]; will route to failure:
org.apache.avro.file.DataFileWriter$AppendWriteException:
java.lang.NullPointerException: null of nifi.statusEnum of union in field
status of nifi.personWithNameRecord
org.apache.avro.file.DataFileWriter$AppendWriteException:
java.lang.NullPointerException: null of nifi.statusEnum of union in field
status of nifi.personWithNameRecord
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:308)
at
org.apache.nifi.avro.WriteAvroResultWithSchema.writeRecord(WriteAvroResultWithSchema.java:61)
at
org.apache.nifi.serialization.AbstractRecordSetWriter.write(AbstractRecordSetWriter.java:59)
at
org.apache.nifi.processors.standard.AbstractRecordProcessor$1.process(AbstractRecordProcessor.java:153)
at
org.apache.nifi.util.MockProcessSession.write(MockProcessSession.java:925)
at
org.apache.nifi.util.MockProcessSession.write(MockProcessSession.java:1)
at
org.apache.nifi.processors.standard.AbstractRecordProcessor.onTrigger(AbstractRecordProcessor.java:122)
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at
org.apache.nifi.util.StandardProcessorTestRunner$RunProcessor.call(StandardProcessorTestRunner.java:270)
at
org.apache.nifi.util.StandardProcessorTestRunner$RunProcessor.call(StandardProcessorTestRunner.java:1)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
Caused by: java.lang.NullPointerException: null of nifi.statusEnum of union in
field status of nifi.personWithNameRecord
at
org.apache.avro.generic.GenericDatumWriter.npe(GenericDatumWriter.java:132)
at
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:126)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:60)
at org.apache.avro.file.DataFileWriter.append(DataFileWriter.java:302)
... 15 more
Caused by: java.lang.NullPointerException
at org.apache.avro.Schema$EnumSchema.getEnumOrdinal(Schema.java:755)
at
org.apache.avro.generic.GenericDatumWriter.writeEnum(GenericDatumWriter.java:165)
at
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:106)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
at
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:112)
at
org.apache.avro.generic.GenericDatumWriter.write(GenericDatumWriter.java:73)
at
org.apache.avro.generic.GenericDatumWriter.writeField(GenericDatumWriter.java:153)
at
org.apache.avro.generic.GenericDatumWriter.writeRecord(GenericDatumWriter.java:143)
at
org.apache.avro.generic.GenericDatumWriter.writeWithoutConversion(GenericDatumWriter.java:105)
... 18 more{noformat}
> ConvertRecord cannot handle open enum in Avro
> -
>
> Key: NIFI-7981
> URL: https://issues.apache.org/jira/browse/NIFI-7981
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.12.1
>Reporter: Christophe Monnet
>Priority: Minor
> Attachments: record_enum.xml
>
>
> This avro schema is using the open enum pattern:
> {code:json}
> {
> "namespace": "acme",
> "name": "openEnum",
> "type": "record",
> "fields": [{
> "name": "personType",
> "type": ["null", {
> "type": "enum",
> "name": "personType_enum",
> "symbols": [
> "DRIVER",
> "TECHNICIAN"
> ]
> }, "string"],
> "default": null
> }
> ]
> }
> {code}
> The *personType* can be either null, an enum, or any string. This brings
>
[jira] [Commented] (NIFI-7983) Several SELECTs does not return correct values
[
https://issues.apache.org/jira/browse/NIFI-7983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17226877#comment-17226877
]
Pierre Villard commented on NIFI-7983:
--
What schema do you use? It sounds like you're not giving the same column name
across your queries.
> Several SELECTs does not return correct values
> --
>
> Key: NIFI-7983
> URL: https://issues.apache.org/jira/browse/NIFI-7983
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.12.1
>Reporter: Michal Šunka
>Priority: Blocker
> Attachments: NiFi_SELECTs.jpg
>
>
> If in ExecuteSQLRecord processor I use multiple SELECTs as in the following
> code (note, it all is in single instance of processor), the results are
> weird. First SELECT (1, 'Yeees') comes through okay as it should, but second
> and third seems to be processed (additional flowfiles are created), but these
> are _empty_.
> {code:java}
> SELECT 1 as 'Yeees'
> SELECT 0 as 'Noo'
> DECLARE @q int
> SET @q = 0
> SELECT @q AS 'qqq'
> {code}
>
> The use case is to call stored procedure returning resultset and then select
> the returnCode:
> {code:java}
> DECLARE @retCode int
> EXEC @retCode = stored_procedure_giving_resultset
>
> SELECT @retCode AS 'retCode_run'
> {code}
> I am using NiFi 1.12.1, DB is SAP ASE 16.0, db driver is jConnect (TM) for
> JDBC(TM)/16.0 SP02 PL06 (Build 27337)/P/EBF26829/JDK 1.6.0/jdbcmain/OPT/Wed
> Mar 15 03:15:38 PDT 2017 and trivially configured JSON recordset writer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-7984) Add Run Duration to FetchHBaseRow processor in NiFi
Rahul Soni created NIFI-7984: Summary: Add Run Duration to FetchHBaseRow processor in NiFi Key: NIFI-7984 URL: https://issues.apache.org/jira/browse/NIFI-7984 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Rahul Soni Today, we don't have an option to configure a Run Duration for the Fetch HBase Processor. As a result, in order to yield a higher throughput, a user may end up giving more threads to this processor. This, as we know, is not the best practice and can result in thread starvation for other processors/processes. I, hereby, would suggest adding a Run Duration configuration option to the FetchHbaseRow processor to utilize batching for better performance. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7444) Add "retry" relation to the Convert Record Processor
[ https://issues.apache.org/jira/browse/NIFI-7444?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17226838#comment-17226838 ] Rahul Soni commented on NIFI-7444: -- I agree to you POV [~markap14]. In my opinion this is a big improvement that we should make to NiFi processors, wherever applicable. I have now seen a few clients who have struggled with this implementation, or rather lack of it. Would be more than happy to discuss it further. > Add "retry" relation to the Convert Record Processor > > > Key: NIFI-7444 > URL: https://issues.apache.org/jira/browse/NIFI-7444 > Project: Apache NiFi > Issue Type: New Feature > Components: Configuration >Reporter: Rahul Soni >Priority: Minor > > Follows the case for adding a retry relationship and redirect the "failed" > conversion to this relationship in certain scenarios with one of those > discussed below. > Today we have either success or failure relationship for ConvertRecord > processor. It works well until the Record Reader/Writer is using Schema > Registry and then we reach a point where Schema Registry is unavailable due > to any reason. Now the records which are supposed to be converted from one > format to another are routed to failure because “schema” is not found for > those flow files. > One can re-attempt those flow files in one way or another but the issue is > it’s not effective to retry a flow file any number of times which doesn’t > have a schema available, actually. On the flip side, we can have a counter > that the flow file should be tried n number of times before being redirected > to the failure relationship but then again we are on the mercy of Schema > Registry to be up within the configured period of re-attempts before we > potentially "lose" the flow file by redirecting it to the failed relationship. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7444) Add "retry" relation to the Convert Record Processor
[ https://issues.apache.org/jira/browse/NIFI-7444?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rahul Soni updated NIFI-7444: - Priority: Major (was: Minor) > Add "retry" relation to the Convert Record Processor > > > Key: NIFI-7444 > URL: https://issues.apache.org/jira/browse/NIFI-7444 > Project: Apache NiFi > Issue Type: New Feature > Components: Configuration >Reporter: Rahul Soni >Priority: Major > > Follows the case for adding a retry relationship and redirect the "failed" > conversion to this relationship in certain scenarios with one of those > discussed below. > Today we have either success or failure relationship for ConvertRecord > processor. It works well until the Record Reader/Writer is using Schema > Registry and then we reach a point where Schema Registry is unavailable due > to any reason. Now the records which are supposed to be converted from one > format to another are routed to failure because “schema” is not found for > those flow files. > One can re-attempt those flow files in one way or another but the issue is > it’s not effective to retry a flow file any number of times which doesn’t > have a schema available, actually. On the flip side, we can have a counter > that the flow file should be tried n number of times before being redirected > to the failure relationship but then again we are on the mercy of Schema > Registry to be up within the configured period of re-attempts before we > potentially "lose" the flow file by redirecting it to the failed relationship. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7983) Several SELECTs does not return correct values
[
https://issues.apache.org/jira/browse/NIFI-7983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michal Šunka updated NIFI-7983:
---
Description:
If in ExecuteSQLRecord processor I use multiple SELECTs as in the following
code (note, it all is in single instance of processor), the results are weird.
First SELECT (1, 'Yeees') comes through okay as it should, but second and third
seems to be processed (additional flowfiles are created), but these are _empty_.
{code:java}
SELECT 1 as 'Yeees'
SELECT 0 as 'Noo'
DECLARE @q int
SET @q = 0
SELECT @q AS 'qqq'
{code}
The use case is to call stored procedure returning resultset and then select
the returnCode:
{code:java}
DECLARE @retCode int
EXEC @retCode = stored_procedure_giving_resultset
SELECT @retCode AS 'retCode_run'
{code}
I am using NiFi 1.12.1, DB is SAP ASE 16.0, db driver is jConnect (TM) for
JDBC(TM)/16.0 SP02 PL06 (Build 27337)/P/EBF26829/JDK 1.6.0/jdbcmain/OPT/Wed Mar
15 03:15:38 PDT 2017 and trivially configured JSON recordset writer.
was:
If in ExecuteSQLRecord processor I use multiple SELECTs as in following (note,
it all is in single instance of processor), the results are weird. First SELECT
(1, 'Yeees') comes through okay as it should, but second and third seems to be
processed (additional flowfiles are created), but these are _empty_.
{code:java}
SELECT 1 as 'Yeees'
SELECT 0 as 'Noo'
DECLARE @q int
SET @q = 0
SELECT @q AS 'qqq'
{code}
The use case is to call stored procedure returning resultset and then selecr
the returncode:
{code:java}
DECLARE @retCode int
EXEC @retCode = stored_procedure_giving_resultset
SELECT @retCode AS 'retCode_run'
{code}
I am using NiFi 1.12.1, DB is SAP ASE 16.0, db driver is jConnect (TM) for
JDBC(TM)/16.0 SP02 PL06 (Build 27337)/P/EBF26829/JDK 1.6.0/jdbcmain/OPT/Wed Mar
15 03:15:38 PDT 2017 and trivially configured JSON recordset writer.
> Several SELECTs does not return correct values
> --
>
> Key: NIFI-7983
> URL: https://issues.apache.org/jira/browse/NIFI-7983
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.12.1
>Reporter: Michal Šunka
>Priority: Blocker
> Attachments: NiFi_SELECTs.jpg
>
>
> If in ExecuteSQLRecord processor I use multiple SELECTs as in the following
> code (note, it all is in single instance of processor), the results are
> weird. First SELECT (1, 'Yeees') comes through okay as it should, but second
> and third seems to be processed (additional flowfiles are created), but these
> are _empty_.
> {code:java}
> SELECT 1 as 'Yeees'
> SELECT 0 as 'Noo'
> DECLARE @q int
> SET @q = 0
> SELECT @q AS 'qqq'
> {code}
>
> The use case is to call stored procedure returning resultset and then select
> the returnCode:
> {code:java}
> DECLARE @retCode int
> EXEC @retCode = stored_procedure_giving_resultset
>
> SELECT @retCode AS 'retCode_run'
> {code}
> I am using NiFi 1.12.1, DB is SAP ASE 16.0, db driver is jConnect (TM) for
> JDBC(TM)/16.0 SP02 PL06 (Build 27337)/P/EBF26829/JDK 1.6.0/jdbcmain/OPT/Wed
> Mar 15 03:15:38 PDT 2017 and trivially configured JSON recordset writer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-7983) Several SELECTs does not return correct values
[
https://issues.apache.org/jira/browse/NIFI-7983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Michal Šunka updated NIFI-7983:
---
Description:
If in ExecuteSQLRecord processor I use multiple SELECTs as in following (note,
it all is in single instance of processor), the results are weird. First SELECT
(1, 'Yeees') comes through okay as it should, but second and third seems to be
processed (additional flowfiles are created), but these are _empty_.
{code:java}
SELECT 1 as 'Yeees'
SELECT 0 as 'Noo'
DECLARE @q int
SET @q = 0
SELECT @q AS 'qqq'
{code}
The use case is to call stored procedure returning resultset and then selecr
the returncode:
{code:java}
DECLARE @retCode int
EXEC @retCode = stored_procedure_giving_resultset
SELECT @retCode AS 'retCode_run'
{code}
I am using NiFi 1.12.1, DB is SAP ASE 16.0, db driver is jConnect (TM) for
JDBC(TM)/16.0 SP02 PL06 (Build 27337)/P/EBF26829/JDK 1.6.0/jdbcmain/OPT/Wed Mar
15 03:15:38 PDT 2017 and trivially configured JSON recordset writer.
was:
If in ExecuteSQLRecord processor I use multiple SELECTs as in following (note,
it all is in single instance of processor), the results are weird. First SELECT
(1, 'Yeees') comes through okay as it should, but second and third seems to be
processed (additional flowfiles are created), but these are _empty_.
{code:java}
SELECT 1 as 'Yeees'
SELECT 0 as 'Noo'
DECLARE @q int
SET @q = 0
SELECT @q AS 'qqq'
{code}
The use case is to call stored procedure returning resultset and then selecr
the returncode:
{code:java}
DECLARE @retCode int
EXEC @retCode = stored_procedure_giving_resultset
SELECT @retCode AS 'retCode_run'
{code}
I am using NiFi 12.1, DB is SAP ASE 16.0, db driver is jConnect (TM) for
JDBC(TM)/16.0 SP02 PL06 (Build 27337)/P/EBF26829/JDK 1.6.0/jdbcmain/OPT/Wed Mar
15 03:15:38 PDT 2017 and trivially configured JSON recordset writer.
> Several SELECTs does not return correct values
> --
>
> Key: NIFI-7983
> URL: https://issues.apache.org/jira/browse/NIFI-7983
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.12.1
>Reporter: Michal Šunka
>Priority: Blocker
> Attachments: NiFi_SELECTs.jpg
>
>
> If in ExecuteSQLRecord processor I use multiple SELECTs as in following
> (note, it all is in single instance of processor), the results are weird.
> First SELECT (1, 'Yeees') comes through okay as it should, but second and
> third seems to be processed (additional flowfiles are created), but these are
> _empty_.
> {code:java}
> SELECT 1 as 'Yeees'
> SELECT 0 as 'Noo'
> DECLARE @q int
> SET @q = 0
> SELECT @q AS 'qqq'
> {code}
> The use case is to call stored procedure returning resultset and then selecr
> the returncode:
> {code:java}
> DECLARE @retCode int
> EXEC @retCode = stored_procedure_giving_resultset
>
> SELECT @retCode AS 'retCode_run'
> {code}
> I am using NiFi 1.12.1, DB is SAP ASE 16.0, db driver is jConnect (TM) for
> JDBC(TM)/16.0 SP02 PL06 (Build 27337)/P/EBF26829/JDK 1.6.0/jdbcmain/OPT/Wed
> Mar 15 03:15:38 PDT 2017 and trivially configured JSON recordset writer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-7983) Several SELECTs does not return correct values
Michal Šunka created NIFI-7983:
--
Summary: Several SELECTs does not return correct values
Key: NIFI-7983
URL: https://issues.apache.org/jira/browse/NIFI-7983
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.12.1
Reporter: Michal Šunka
Attachments: NiFi_SELECTs.jpg
If in ExecuteSQLRecord processor I use multiple SELECTs as in following (note,
it all is in single instance of processor), the results are weird. First SELECT
(1, 'Yeees') comes through okay as it should, but second and third seems to be
processed (additional flowfiles are created), but these are _empty_.
{code:java}
SELECT 1 as 'Yeees'
SELECT 0 as 'Noo'
DECLARE @q int
SET @q = 0
SELECT @q AS 'qqq'
{code}
The use case is to call stored procedure returning resultset and then selecr
the returncode:
{code:java}
DECLARE @retCode int
EXEC @retCode = stored_procedure_giving_resultset
SELECT @retCode AS 'retCode_run'
{code}
I am using NiFi 12.1, DB is SAP ASE 16.0, db driver is jConnect (TM) for
JDBC(TM)/16.0 SP02 PL06 (Build 27337)/P/EBF26829/JDK 1.6.0/jdbcmain/OPT/Wed Mar
15 03:15:38 PDT 2017 and trivially configured JSON recordset writer.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-7982) Add Run Duration option to FlattanJSON processor
Rahul Soni created NIFI-7982: Summary: Add Run Duration option to FlattanJSON processor Key: NIFI-7982 URL: https://issues.apache.org/jira/browse/NIFI-7982 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Rahul Soni FlattenJSON processor doesn't have a run duration today. As a result, the flows using it sometimes needs to assign more threads to the processor for a higher throughput, which is not an ideal scenario. In order to yield maximum throughput from FlattenJSON processor, I would propose adding Run Duration option to the scheduling section. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7981) ConvertRecord cannot handle open enum in Avro
[
https://issues.apache.org/jira/browse/NIFI-7981?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17226721#comment-17226721
]
Pierre Villard commented on NIFI-7981:
--
I've not been able to reproduce this issue using the main branch of NiFi. If it
happened on 1.12.1, then a recent commit fixed the issue and this will be
solved in 1.13.0 when we release it.
Template to test: [^record_enum.xml]
Going to open a pull request with a unit test so that we cover enums in our
test suite anyway.
> ConvertRecord cannot handle open enum in Avro
> -
>
> Key: NIFI-7981
> URL: https://issues.apache.org/jira/browse/NIFI-7981
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.12.1
>Reporter: Christophe Monnet
>Priority: Minor
> Attachments: record_enum.xml
>
>
> This avro schema is using the open enum pattern:
> {code:json}
> {
> "namespace": "acme",
> "name": "openEnum",
> "type": "record",
> "fields": [{
> "name": "personType",
> "type": ["null", {
> "type": "enum",
> "name": "personType_enum",
> "symbols": [
> "DRIVER",
> "TECHNICIAN"
> ]
> }, "string"],
> "default": null
> }
> ]
> }
> {code}
> The *personType* can be either null, an enum, or any string. This brings
> forward-compatibility.
> Using ConvertRecord with
> {"personType":"DRIVER"}
> as a payload, it works.
> But with
> {"personType":"DUDE"}
> it fails with {{java.lang.NullPointerException: null of acme.personType_enum
> of union in field personType of acme.openEnum}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-7981) ConvertRecord cannot handle open enum in Avro
[
https://issues.apache.org/jira/browse/NIFI-7981?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-7981:
-
Attachment: record_enum.xml
> ConvertRecord cannot handle open enum in Avro
> -
>
> Key: NIFI-7981
> URL: https://issues.apache.org/jira/browse/NIFI-7981
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.12.1
>Reporter: Christophe Monnet
>Priority: Minor
> Attachments: record_enum.xml
>
>
> This avro schema is using the open enum pattern:
> {code:json}
> {
> "namespace": "acme",
> "name": "openEnum",
> "type": "record",
> "fields": [{
> "name": "personType",
> "type": ["null", {
> "type": "enum",
> "name": "personType_enum",
> "symbols": [
> "DRIVER",
> "TECHNICIAN"
> ]
> }, "string"],
> "default": null
> }
> ]
> }
> {code}
> The *personType* can be either null, an enum, or any string. This brings
> forward-compatibility.
> Using ConvertRecord with
> {"personType":"DRIVER"}
> as a payload, it works.
> But with
> {"personType":"DUDE"}
> it fails with {{java.lang.NullPointerException: null of acme.personType_enum
> of union in field personType of acme.openEnum}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (NIFI-7949) Add tag HCFS to HDFS Processors
[ https://issues.apache.org/jira/browse/NIFI-7949?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7949: - Fix Version/s: 1.13.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Add tag HCFS to HDFS Processors > --- > > Key: NIFI-7949 > URL: https://issues.apache.org/jira/browse/NIFI-7949 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.12.1 >Reporter: Siyao Meng >Priority: Major > Fix For: 1.13.0 > > Time Spent: 20m > Remaining Estimate: 0h > > Adding tag "HCFS" to all HDFS Processors: > - DeleteHDFS > - FetchHDFS > - GetHDFS > - GetHDFSFileInfo > - GetHDFSSequenceFile > - ListHDFS > - MoveHDFS > - PutHDFS > --- > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-hadoop-nar/1.12.1/org.apache.nifi.processors.hadoop.PutHDFS/index.html > The PutHDFS processor can actually write to any [Hadoop Compatible File > System > (HCFS)|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/filesystem/introduction.html], > where HDFS is merely one of them. > Especially with the emergence of Hadoop Ozone and others, the name of the > processor might be a bit inaccurate and misleading. > What do you think? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4633: NIFI-7949: Add tag HCFS to HDFS Processors
asfgit closed pull request #4633: URL: https://github.com/apache/nifi/pull/4633 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-7949) Add tag HCFS to HDFS Processors
[ https://issues.apache.org/jira/browse/NIFI-7949?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17226687#comment-17226687 ] ASF subversion and git services commented on NIFI-7949: --- Commit 9a3a659c441f7d4d61f7b526777f1a2def9198ac in nifi's branch refs/heads/main from Siyao Meng [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=9a3a659 ] NIFI-7949: Add tag HCFS to HDFS Processors Signed-off-by: Pierre Villard This closes #4633. > Add tag HCFS to HDFS Processors > --- > > Key: NIFI-7949 > URL: https://issues.apache.org/jira/browse/NIFI-7949 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.12.1 >Reporter: Siyao Meng >Priority: Major > Time Spent: 10m > Remaining Estimate: 0h > > Adding tag "HCFS" to all HDFS Processors: > - DeleteHDFS > - FetchHDFS > - GetHDFS > - GetHDFSFileInfo > - GetHDFSSequenceFile > - ListHDFS > - MoveHDFS > - PutHDFS > --- > https://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-hadoop-nar/1.12.1/org.apache.nifi.processors.hadoop.PutHDFS/index.html > The PutHDFS processor can actually write to any [Hadoop Compatible File > System > (HCFS)|https://hadoop.apache.org/docs/stable/hadoop-project-dist/hadoop-common/filesystem/introduction.html], > where HDFS is merely one of them. > Especially with the emergence of Hadoop Ozone and others, the name of the > processor might be a bit inaccurate and misleading. > What do you think? -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7974) Upgrading calcite, hbase, geo2ip deps
[ https://issues.apache.org/jira/browse/NIFI-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17226685#comment-17226685 ] Pierre Villard commented on NIFI-7974: -- [~mtien] - can you have a look when you get the chance? > Upgrading calcite, hbase, geo2ip deps > - > > Key: NIFI-7974 > URL: https://issues.apache.org/jira/browse/NIFI-7974 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 1.13.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Upgrading calcite, hbase, geo2ip deps -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7974) Upgrading calcite, hbase, geo2ip deps
[ https://issues.apache.org/jira/browse/NIFI-7974?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-7974: - Status: Patch Available (was: Open) > Upgrading calcite, hbase, geo2ip deps > - > > Key: NIFI-7974 > URL: https://issues.apache.org/jira/browse/NIFI-7974 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Major > Fix For: 1.13.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Upgrading calcite, hbase, geo2ip deps -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7968) PutHDFS/PutParquet fail to write to Kerberized HDFS with KMS enabled
[
https://issues.apache.org/jira/browse/NIFI-7968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pierre Villard updated NIFI-7968:
-
Fix Version/s: 1.13.0
Resolution: Fixed
Status: Resolved (was: Patch Available)
> PutHDFS/PutParquet fail to write to Kerberized HDFS with KMS enabled
>
>
> Key: NIFI-7968
> URL: https://issues.apache.org/jira/browse/NIFI-7968
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.12.0, 1.12.1
>Reporter: Bryan Bende
>Assignee: Bryan Bende
>Priority: Minor
> Fix For: 1.13.0
>
> Time Spent: 20m
> Remaining Estimate: 0h
>
> From apache slack...
> {color:#1d1c1d}My PutHDFS and PutParquet processors are configured to use a
> KeytabCredentialsService. I've confirmed that that service is configured
> correctly. The server also has the latest core-site and hdfs-site XML
> configuration files from the HDFS cluster. However, whenever either of those
> processors run, we receive the attached error message.{color}
> {code:java}
> 2020-10-13 21:37:33,547 WARN [Timer-Driven Process Thread-100]
> o.a.h.c.k.k.LoadBalancingKMSClientProvider KMS provider at [https:// SERVER>:9393/kms/v1/] threw an IOException:java.io.IOException:
> org.apache.hadoop.security.authentication.client.AuthenticationException:
> Error while authenticating with endpoint: https:// SERVER>:9393/kms/v1/keyversion/keyname/_eek?eek_op=decryptat
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:525)
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.decryptEncryptedKey(KMSClientProvider.java:826)
> at
> org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:351)
> at
> org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:347)
> at
> org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.doOp(LoadBalancingKMSClientProvider.java:172)
> at
> org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.decryptEncryptedKey(LoadBalancingKMSClientProvider.java:347)
> at
> org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.decryptEncryptedKey(KeyProviderCryptoExtension.java:532)
> at
> org.apache.hadoop.hdfs.HdfsKMSUtil.decryptEncryptedDataEncryptionKey(HdfsKMSUtil.java:206)
> at
> org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:966)
> at
> org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:947)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:533)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:527)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:541)
> at
> org.apache.nifi.processors.hadoop.PutHDFS$1$1.process(PutHDFS.java:337)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2324)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2292)
> at org.apache.nifi.processors.hadoop.PutHDFS$1.run(PutHDFS.java:320)
> at java.security.AccessController.doPrivileged(Native Method)at
> javax.security.auth.Subject.doAs(Subject.java:360)at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710)
> at
> org.apache.nifi.processors.hadoop.PutHDFS.onTrigger(PutHDFS.java:250)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1174)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
> at
>
[GitHub] [nifi] asfgit closed pull request #4643: NIFI-7968 Ensure the static login user is set in UserGroupInformation…
asfgit closed pull request #4643: URL: https://github.com/apache/nifi/pull/4643 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-7968) PutHDFS/PutParquet fail to write to Kerberized HDFS with KMS enabled
[
https://issues.apache.org/jira/browse/NIFI-7968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17226684#comment-17226684
]
ASF subversion and git services commented on NIFI-7968:
---
Commit 1366d017b8cbd684308f49a6875d65bf7c61c9d6 in nifi's branch
refs/heads/main from Bryan Bende
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=1366d01 ]
NIFI-7968 Ensure the status login user is set in UserGroupInformation after
creating a UGI from a Subject
Signed-off-by: Pierre Villard
This closes #4643.
> PutHDFS/PutParquet fail to write to Kerberized HDFS with KMS enabled
>
>
> Key: NIFI-7968
> URL: https://issues.apache.org/jira/browse/NIFI-7968
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.12.0, 1.12.1
>Reporter: Bryan Bende
>Assignee: Bryan Bende
>Priority: Minor
> Time Spent: 10m
> Remaining Estimate: 0h
>
> From apache slack...
> {color:#1d1c1d}My PutHDFS and PutParquet processors are configured to use a
> KeytabCredentialsService. I've confirmed that that service is configured
> correctly. The server also has the latest core-site and hdfs-site XML
> configuration files from the HDFS cluster. However, whenever either of those
> processors run, we receive the attached error message.{color}
> {code:java}
> 2020-10-13 21:37:33,547 WARN [Timer-Driven Process Thread-100]
> o.a.h.c.k.k.LoadBalancingKMSClientProvider KMS provider at [https:// SERVER>:9393/kms/v1/] threw an IOException:java.io.IOException:
> org.apache.hadoop.security.authentication.client.AuthenticationException:
> Error while authenticating with endpoint: https:// SERVER>:9393/kms/v1/keyversion/keyname/_eek?eek_op=decryptat
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.createConnection(KMSClientProvider.java:525)
> at
> org.apache.hadoop.crypto.key.kms.KMSClientProvider.decryptEncryptedKey(KMSClientProvider.java:826)
> at
> org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:351)
> at
> org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider$5.call(LoadBalancingKMSClientProvider.java:347)
> at
> org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.doOp(LoadBalancingKMSClientProvider.java:172)
> at
> org.apache.hadoop.crypto.key.kms.LoadBalancingKMSClientProvider.decryptEncryptedKey(LoadBalancingKMSClientProvider.java:347)
> at
> org.apache.hadoop.crypto.key.KeyProviderCryptoExtension.decryptEncryptedKey(KeyProviderCryptoExtension.java:532)
> at
> org.apache.hadoop.hdfs.HdfsKMSUtil.decryptEncryptedDataEncryptionKey(HdfsKMSUtil.java:206)
> at
> org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:966)
> at
> org.apache.hadoop.hdfs.DFSClient.createWrappedOutputStream(DFSClient.java:947)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:533)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem$8.doCall(DistributedFileSystem.java:527)
> at
> org.apache.hadoop.fs.FileSystemLinkResolver.resolve(FileSystemLinkResolver.java:81)
> at
> org.apache.hadoop.hdfs.DistributedFileSystem.create(DistributedFileSystem.java:541)
> at
> org.apache.nifi.processors.hadoop.PutHDFS$1$1.process(PutHDFS.java:337)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2324)
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:2292)
> at org.apache.nifi.processors.hadoop.PutHDFS$1.run(PutHDFS.java:320)
> at java.security.AccessController.doPrivileged(Native Method)at
> javax.security.auth.Subject.doAs(Subject.java:360)at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1710)
> at
> org.apache.nifi.processors.hadoop.PutHDFS.onTrigger(PutHDFS.java:250)
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1174)
> at
> org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:213)
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:117)
> at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
> at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
>
[jira] [Commented] (NIFI-7790) XML record reader - failure on well-formed XML
[
https://issues.apache.org/jira/browse/NIFI-7790?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17226647#comment-17226647
]
Pierre Villard commented on NIFI-7790:
--
Yeah we definitely want to support more than just Avro for defining schemas...
Supporting XSDs would provide a lot of features that we don't have with Avro
schemas. This would be a large effort though and not sure someone is working on
this at the moment.
I'm not aware of an automated way of converting XSDs to Avro schemas, let us
know if what you found is working.
> XML record reader - failure on well-formed XML
> --
>
> Key: NIFI-7790
> URL: https://issues.apache.org/jira/browse/NIFI-7790
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.11.4
>Reporter: Pierre Gramme
>Priority: Major
> Labels: records, xml
> Attachments: bug-parse-xml.xml
>
>
> I am using ConvertRecord in order to parse XML flowfiles to Avro, with the
> Infer Schema strategy. Some input flowfiles are sent to the failure output
> queue whereas they are well-formed:
> {code:java}
>
>
>
>
> Neil Gaiman
>
>
>
>
> Hachette
>
>
>
> {code}
> Note the use of authors/item/name on one side, and
> editors/item/commercialName on the other side.
> On the other hand, this gets correctly parsed:
> {code:java}
>
>
>
>
> Neil Gaiman
>
>
>
>
> Hachette
>
>
>
> {code}
> See the attached template for minimal reproducible example.
>
> My interpretation is that the failure in the first case is due to 2
> independent XML node types having the same name ( in this case) but
> having different types and occurring in different parents with different
> types. In the second case, both 's actually have the same node type. I
> didn't use any Schema Inference Cache, so both item types should be inferred
> independently.
> Since the first document is legal XML (an XSD could be written for it) and
> can also be represented in Avro, its conversion shouldn't fail.
> I'll be happy to provide more details if needed.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (NIFI-7981) ConvertRecord cannot handle open enum in Avro
Christophe Monnet created NIFI-7981:
---
Summary: ConvertRecord cannot handle open enum in Avro
Key: NIFI-7981
URL: https://issues.apache.org/jira/browse/NIFI-7981
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 1.12.1
Reporter: Christophe Monnet
This avro schema is using the open enum pattern:
{code:json}
{
"namespace": "acme",
"name": "openEnum",
"type": "record",
"fields": [{
"name": "personType",
"type": ["null", {
"type": "enum",
"name": "personType_enum",
"symbols": [
"DRIVER",
"TECHNICIAN"
]
}, "string"],
"default": null
}
]
}
{code}
The *personType* can be either null, an enum, or any string. This brings
forward-compatibility.
Using ConvertRecord with
{"personType":"DRIVER"}
as a payload, it works.
But with
{"personType":"DUDE"}
it fails with {{java.lang.NullPointerException: null of acme.personType_enum of
union in field personType of acme.openEnum}}
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
