[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085168#comment-16085168 ] ASF GitHub Bot commented on NIFI-1613: -- Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/1976 @mattyb149 I confirmed that changes are merged, closing. Thanks for reviewing! > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > Fix For: 1.4.0 > > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16085169#comment-16085169 ] ASF GitHub Bot commented on NIFI-1613: -- Github user ijokarumawak closed the pull request at: https://github.com/apache/nifi/pull/1976 > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > Fix For: 1.4.0 > > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084701#comment-16084701 ] ASF subversion and git services commented on NIFI-1613: --- Commit 3844a821f18e6a76e44e289c923be333e49d5251 in nifi's branch refs/heads/master from [~ijokarumawak] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=3844a82 ] NIFI-1613: ConvertJSONToSQL truncates numeric value wrongly. - Changed boolean value conversion to use Boolean.valueOf. - Updated comments in source code to reflect current situation more clearly. - Updated tests those have been added since the original commits were made. > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > Fix For: 1.4.0 > > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084715#comment-16084715 ] ASF GitHub Bot commented on NIFI-1613: -- Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/1976 @ijokarumawak For some reason my rebase/squash didn't work so it also didn't close this PR. But it has been merged, can you close this PR? Please and thanks! > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > Fix For: 1.4.0 > > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084702#comment-16084702 ] ASF subversion and git services commented on NIFI-1613: --- Commit 8acee02393f9557b9679038b933ba49705984cf8 in nifi's branch refs/heads/master from [~ijokarumawak] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=8acee02 ] NIFI-1613 - Truncate text data types only. - Added conversion from a boolean to number. > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > Fix For: 1.4.0 > > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084705#comment-16084705 ] ASF GitHub Bot commented on NIFI-1613: -- Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/1976 +1 LGTM, tested with MySQL, Oracle, and Postgres (and ran unit tests which use Derby). Thanks for the fix/improvement @ijokarumawak and @ToivoAdams! Merging to master > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > Fix For: 1.4.0 > > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16084700#comment-16084700 ] ASF subversion and git services commented on NIFI-1613: --- Commit 3b2e43b75c80be854cc854c3941e882c794c1d76 in nifi's branch refs/heads/master from [~Toivo Adams] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=3b2e43b ] NIFI-1613 Initial version, try to improve conversion for different SQL types. New test and refactored existing test to reuse DBCP service. nifi-1613 Adding numeric and Date/time types conversion and test. > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > Fix For: 1.4.0 > > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16083172#comment-16083172 ] ASF GitHub Bot commented on NIFI-1613: -- Github user ijokarumawak commented on a diff in the pull request: https://github.com/apache/nifi/pull/1976#discussion_r126833370 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml --- @@ -467,6 +467,7 @@ src/test/resources/TestUpdateRecord/output/person-with-firstname.json src/test/resources/TestUpdateRecord/output/person-with-firstname-lastname.json src/test/resources/TestUpdateRecord/output/person-with-capital-lastname.json + src/test/resources/TestUpdateRecord/output/person-with-bool.json --- End diff -- Thanks @mattyb149 , fixed this and confirmed RAT check passed locally. > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16082135#comment-16082135 ] ASF GitHub Bot commented on NIFI-1613: -- Github user mattyb149 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1976#discussion_r126673168 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml --- @@ -467,6 +467,7 @@ src/test/resources/TestUpdateRecord/output/person-with-firstname.json src/test/resources/TestUpdateRecord/output/person-with-firstname-lastname.json src/test/resources/TestUpdateRecord/output/person-with-capital-lastname.json + src/test/resources/TestUpdateRecord/output/person-with-bool.json --- End diff -- This should be in the TestConvertJSONToSQL section rather than TestUpdateRecord, it causes a RAT failure > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16080009#comment-16080009 ] ASF GitHub Bot commented on NIFI-1613: -- Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/1976 @mattyb149 Thanks for your feedback. I added a conversion for numeric columns if the input JSON field is a boolean (true -> 0, false -> 1). Now boolean JSON value can be mapped to Oracle smoothly, this should make most users happier. Also, made automatic data truncation more defensive. When I looked at the data types, I sensed only string data types should be truncated and it's safer than specifying every types not to truncate. > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16078932#comment-16078932 ] ASF GitHub Bot commented on NIFI-1613: -- Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/1976 The code changes look good; I have a question in general about expected behavior when the database doesn't explicitly support boolean types (such as Oracle SQL). Let's say I have a JSON object `{"id": 1, "b": true}` and a table with column "id" of type INT and "b" of type NUMBER(1). Do we need to support this case, or if the target doesn't have a BOOLEAN type, should the onus be on the flow designer to change the values accordingly (such as with JoltTransformJSON)? Also should we include BIT and/or other numeric types in the switch statement? > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16074342#comment-16074342 ] ASF GitHub Bot commented on NIFI-1613: -- Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/293 I've taken over the proposed change into #1976 > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.4.14#64029)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16074337#comment-16074337 ] ASF GitHub Bot commented on NIFI-1613: -- Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/1976 This PR contains commits from #293 which has been closed due to being inactive, but the change was reasonable. I've cherry picked commits from the PR, the 1st commit in this PR is a squashed ones those are made by @ToivoAdams . Thanks for your contribution Toivo! I also added the 2nd commit, in order to update the proposed change with the latest codebase, as well as few modification. We now have JsonTreeReader and PutDatabaseRecord those can handle type coercing better, so I think it's not necessary to handle DATE/TIME conversion at ConvertJSONToSQL. However, without this fix, it can truncate numeric values wrongly and lose part of user data, and that should be fixed at least. This PR is ready for review, thanks! > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlas
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16074331#comment-16074331 ] ASF GitHub Bot commented on NIFI-1613: -- GitHub user ijokarumawak opened a pull request: https://github.com/apache/nifi/pull/1976 NIFI-1613: Make use of column type correctly at ConvertJSONToSQL Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [ ] Is your initial contribution a single, squashed commit? ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [X] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijokarumawak/nifi nifi-1613 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1976.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1976 commit 07ed78cd130c632f63e1357d2407f946e6f5f45a Author: Toivo Adams Date: 2016-03-20T19:13:15Z NIFI-1613 Initial version, try to improve conversion for different SQL types. New test and refactored existing test to reuse DBCP service. nifi-1613 Adding numeric and Date/time types conversion and test. commit 81e8391e3bc8e26413c0f1e38a41bc01b88159e1 Author: Koji Kawamura Date: 2017-07-05T05:49:32Z NIFI-1613: ConvertJSONToSQL truncates numeric value wrongly. - Changed boolean value conversion to use Boolean.valueOf. - Updated comments in source code to reflect current situation more clearly. - Updated tests those have been added since the original commits were made. > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.jav
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963091#comment-15963091 ] ASF GitHub Bot commented on NIFI-1613: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/293 > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15866364#comment-15866364 ] ASF GitHub Bot commented on NIFI-1613: -- Github user joewitt commented on the issue: https://github.com/apache/nifi/pull/293 I plan to close out this PR in a stale PR sweep soon. > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15582418#comment-15582418 ] Carlos Manuel António Fernandes commented on NIFI-1613: --- Toivo, On ConvertJSONToSQL, all Oracle Dates and Timestamps are truncated because : if (colSize != null && fieldValue.length() > colSize) { fieldValue = fieldValue.substring(0, colSize); } The Dates are size 7, timestamps are size 11 (Oracle internal representation) not Json representation which are 10 for dates ('2016-01-01') and 26 for timestamps ('2015-02-11 18:10:20.402965' ) For Dates, in putSQL, after convertJSONToSQL, you are right, is expected a Long numeric Type and the value passed is a String. For Timestamps will work because if the Type isnt a Long (and isnt) they area parsed with "("-MM-dd HH:mm:ss.SSS") Carlos > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15545519#comment-15545519 ] Toivo Adams commented on NIFI-1613: --- Hi @trixpan If I remember correctly, Boolean and Numeric types handling is improved. But DATE, TIME and TIMESTAMP handling is still problematic. (when used together with PutSQL processor) PutSQL expects Long numeric types. But JSON usually uses ISO 8601, for example: 2012-04-23T18:25:43.511Z for dates. This means PutSql will fail. Maybe PutSQL should be changed? But change may break current PutSQL behavior. Workaround maybe change PutSQL in the way there: 1. We first try to convert numeric value. 2. If conversion fails we try to convert String value. This way we can preserve current behaviour. Thanks Toivo > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15543807#comment-15543807 ] ASF GitHub Bot commented on NIFI-1613: -- Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/293 @ToivoAdams is this ready for review? > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431579#comment-15431579 ] Charles Bryan Clifford commented on NIFI-1613: -- As first pointed out by Aaron, ConvertJsonToSQL is the likely source of the in-coming flow file attribute entry value truncation. > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.TypeMismatchException: ERROR 203 > (22005): Type mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.exception.SQLExceptionCode$1.newException(SQLExceptionCode.java:71) > ~[na:na] > at > org.apache.phoenix.exception.SQLExceptionInfo.buildException(SQLExceptionInfo.java:145) > ~[na:na] > ... 20 common frames omitted > {noformat} -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-1613) ConvertJSONToSQL Drops Type Information
[ https://issues.apache.org/jira/browse/NIFI-1613?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15431479#comment-15431479 ] Charles Bryan Clifford commented on NIFI-1613: -- As regards the database server data types and Java data type conflicts related to PutSQL, these appear to originate in each of these methods: private void setParameters(final PreparedStatement stmt, final Map attributes) throws SQLException {...} private void setParameter(final PreparedStatement stmt, final String attrName, final int parameterIndex, final String parameterValue, final int jdbcType) throws SQLException, ParseException {...} Each is implemented under the assumption that physical data types used by a database service conform 100% to similarly named data types implemented in Java. Unfortunately, SQL data types, like SQL syntax, are non-standard standards Lol. It appears that each database service data type (based upon information contained in a Connection's ResultSetMetadata) needs to be mapped to a supporting Java data type. My guess, right now, is that the FlowFile's attributes' entry values (which conform to the database service data type definition) are being truncated (into an incorrect Java data type) before they reach PutSQL.setParameter() or PutSQL.setParameters(). Perhaps, these issues with PutSQL are just a symptom related to how the in-coming FlowFile's attribute entry values are being instantiated. > ConvertJSONToSQL Drops Type Information > --- > > Key: NIFI-1613 > URL: https://issues.apache.org/jira/browse/NIFI-1613 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.4.1, 0.5.1 > Environment: Ubuntu 14.04 LTS >Reporter: Aaron Stephens >Assignee: Toivo Adams > Labels: ConvertJSONToSQL, Phoenix, SQL > > It appears that the ConvertJSONToSQL processor is turning Boolean (and > possibly Integer and Float) values into Strings. This is okay for some > drivers (like PostgreSQL) which can coerce a String back into a Boolean, but > it causes issues for others (specifically Phoenix in my case). > {noformat} > org.apache.phoenix.schema.ConstraintViolationException: > org.apache.phoenix.schema.TypeMismatchException: ERROR 203 (22005): Type > mismatch. VARCHAR cannot be coerced to BOOLEAN > at > org.apache.phoenix.schema.types.PDataType.throwConstraintViolationException(PDataType.java:282) > ~[na:na] > at > org.apache.phoenix.schema.types.PBoolean.toObject(PBoolean.java:136) ~[na:na] > at > org.apache.phoenix.jdbc.PhoenixPreparedStatement.setObject(PhoenixPreparedStatement.java:442) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.commons.dbcp.DelegatingPreparedStatement.setObject(DelegatingPreparedStatement.java:166) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameter(PutSQL.java:728) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.setParameters(PutSQL.java:606) > ~[na:na] > at > org.apache.nifi.processors.standard.PutSQL.onTrigger(PutSQL.java:223) ~[na:na] > at > org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27) > ~[nifi-api-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1146) > ~[nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:139) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:49) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:119) > [nifi-framework-core-0.4.1.jar:0.4.1] > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > [na:1.7.0_79] > at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:304) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:178) > [na:1.7.0_79] > at > java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > [na:1.7.0_79] > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > [na:1.7.0_79] > at java.lang.Thread.run(Thread.java:745) [na:1.7.0_79] > Caused by: org.apache.phoenix.schema.