[jira] [Commented] (NIFI-4174) GenerateTableFetch does not work with oracle on Nifi 1.2
[
https://issues.apache.org/jira/browse/NIFI-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16090960#comment-16090960
]
Koji Kawamura commented on NIFI-4174:
-
[~jomach] ExecuteSQL or PutSQL are generally used to execute a sql statement.
Probably using SplitText or SplitContent before Execute/PutSQL would work if
incoming data contains multiple lines of SQL.
> GenerateTableFetch does not work with oracle on Nifi 1.2
>
>
> Key: NIFI-4174
> URL: https://issues.apache.org/jira/browse/NIFI-4174
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Jorge Machado
>Priority: Minor
>
> I'm trying to extract some data from a oracle DB.
> I'm getting :
> {code:java}
> 2017-07-11 16:19:29,612 WARN [StandardProcessScheduler Thread-7]
> o.a.n.controller.StandardProcessorNode Timed out while waiting for
> OnScheduled of 'GenerateTableFetch' processor to finish. An attempt is made
> to cancel the task via Thread.interrupt(). However it does not guarantee that
> the task will be canceled since the code inside current OnScheduled operation
> may have been written to ignore interrupts which may result in a runaway
> thread. This could lead to more issues, eventually requiring NiFi to be
> restarted. This is usually a bug in the target Processor
> 'GenerateTableFetch[id=f08a3acd-ac7e-17d7-598b-8f9720fd92d4]' that needs to
> be documented, reported and eventually fixed.
> 2017-07-11 16:19:29,612 ERROR [StandardProcessScheduler Thread-7]
> o.a.n.p.standard.GenerateTableFetch
> GenerateTableFetch[id=f08a3acd-ac7e-17d7-598b-8f9720fd92d4]
> GenerateTableFetch[id=f08a3acd-ac7e-17d7-598b-8f9720fd92d4] failed to invoke
> @OnScheduled method due to java.lang.RuntimeException: Timed out while
> executing one of processor's OnScheduled task.; processor will not be
> scheduled to run for 30 seconds: java.lang.RuntimeException: Timed out while
> executing one of processor's OnScheduled task.
> java.lang.RuntimeException: Timed out while executing one of processor's
> OnScheduled task.
> at
> org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1480)
> at
> org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:102)
> at
> org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1303)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.util.concurrent.TimeoutException: null
> at java.util.concurrent.FutureTask.get(FutureTask.java:205)
> at
> org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1465)
> ... 9 common frames omitted
> 2017-07-11 16:19:29,613 ERROR [StandardProcessScheduler Thread-7]
> o.a.n.controller.StandardProcessorNode Failed to invoke @OnScheduled method
> due to java.lang.RuntimeException: Timed out while executing one of
> processor's OnScheduled task.
> java.lang.RuntimeException: Timed out while executing one of processor's
> OnScheduled task.
> at
> org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1480)
> at
> org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:102)
> at
> org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1303)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.util.concurrent.TimeoutException: null
> at java.util.concurrent.FutureTask.get(FutureTask.java:205)
> at
>
[jira] [Commented] (NIFI-4174) GenerateTableFetch does not work with oracle on Nifi 1.2
[
https://issues.apache.org/jira/browse/NIFI-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16085196#comment-16085196
]
Jorge Machado commented on NIFI-4174:
-
Yeah, you are right.
One question: Is there a processor that can just execute a list of sql
statements. ? that would be nice
> GenerateTableFetch does not work with oracle on Nifi 1.2
>
>
> Key: NIFI-4174
> URL: https://issues.apache.org/jira/browse/NIFI-4174
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Jorge Machado
>Priority: Minor
>
> I'm trying to extract some data from a oracle DB.
> I'm getting :
> {code:java}
> 2017-07-11 16:19:29,612 WARN [StandardProcessScheduler Thread-7]
> o.a.n.controller.StandardProcessorNode Timed out while waiting for
> OnScheduled of 'GenerateTableFetch' processor to finish. An attempt is made
> to cancel the task via Thread.interrupt(). However it does not guarantee that
> the task will be canceled since the code inside current OnScheduled operation
> may have been written to ignore interrupts which may result in a runaway
> thread. This could lead to more issues, eventually requiring NiFi to be
> restarted. This is usually a bug in the target Processor
> 'GenerateTableFetch[id=f08a3acd-ac7e-17d7-598b-8f9720fd92d4]' that needs to
> be documented, reported and eventually fixed.
> 2017-07-11 16:19:29,612 ERROR [StandardProcessScheduler Thread-7]
> o.a.n.p.standard.GenerateTableFetch
> GenerateTableFetch[id=f08a3acd-ac7e-17d7-598b-8f9720fd92d4]
> GenerateTableFetch[id=f08a3acd-ac7e-17d7-598b-8f9720fd92d4] failed to invoke
> @OnScheduled method due to java.lang.RuntimeException: Timed out while
> executing one of processor's OnScheduled task.; processor will not be
> scheduled to run for 30 seconds: java.lang.RuntimeException: Timed out while
> executing one of processor's OnScheduled task.
> java.lang.RuntimeException: Timed out while executing one of processor's
> OnScheduled task.
> at
> org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1480)
> at
> org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:102)
> at
> org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1303)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.util.concurrent.TimeoutException: null
> at java.util.concurrent.FutureTask.get(FutureTask.java:205)
> at
> org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1465)
> ... 9 common frames omitted
> 2017-07-11 16:19:29,613 ERROR [StandardProcessScheduler Thread-7]
> o.a.n.controller.StandardProcessorNode Failed to invoke @OnScheduled method
> due to java.lang.RuntimeException: Timed out while executing one of
> processor's OnScheduled task.
> java.lang.RuntimeException: Timed out while executing one of processor's
> OnScheduled task.
> at
> org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1480)
> at
> org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:102)
> at
> org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1303)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.util.concurrent.TimeoutException: null
> at java.util.concurrent.FutureTask.get(FutureTask.java:205)
> at
>
[jira] [Commented] (NIFI-4174) GenerateTableFetch does not work with oracle on Nifi 1.2
[
https://issues.apache.org/jira/browse/NIFI-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083644#comment-16083644
]
Koji Kawamura commented on NIFI-4174:
-
There are two things, one is connection timeout setting, and the other is the
error message.
In your case, OracleDriver kept waiting to make a connection and NiFi framework
decided to time it out. In this case I don't think we can produce better error
message. By default, connection timeout is not set and a driver keeps waiting.
In order to set connection timeout, we need to set different JDBC properties
for different drivers:
For [Oracle Thin
driver|http://docs.oracle.com/cd/E18283_01/appdev.112/e13995/constant-values.html#oracle_jdbc_OracleConnection_CONNECTION_PROPERTY_THIN_READ_TIMEOUT],
use "oracle.net.CONNECT_TIMEOUT",
[MySQL|https://dev.mysql.com/doc/connector-j/5.1/en/connector-j-reference-configuration-properties.html]
uses "connectTimeout",
[PostgreSQL|https://jdbc.postgresql.org/documentation/91/connect.html] seems
using "loginTimeout".
By setting these JDBC property by 'User Defined property' (Dynamic processor
property), a driver can throw an Exception when it's timeout.
Even though, since NiFi framework uses async call and the root cause is deep in
a exception that NiFi catches, we need to change NiFi framework code to utilize
the meaningful exception message to show as a bulletin message.
By setting Oracle timeout, I got following stacktrace:
{code}
2017-07-12 17:06:09,483 ERROR [StandardProcessScheduler Thread-1]
o.a.n.controller.StandardProcessorNode Failed to invoke @OnScheduled method due
to java.lang.RuntimeException: Failed while executing one
of processor's OnScheduled task.
java.lang.RuntimeException: Failed while executing one of processor's
OnScheduled task.
at
org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1482)
at
org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:102)
at
org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1303)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.util.concurrent.ExecutionException:
java.lang.reflect.InvocationTargetException
at java.util.concurrent.FutureTask.report(FutureTask.java:122)
at java.util.concurrent.FutureTask.get(FutureTask.java:206)
at
org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1465)
... 9 common frames omitted
Caused by: java.lang.reflect.InvocationTargetException: null
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:137)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:125)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotations(ReflectionUtils.java:70)
at
org.apache.nifi.util.ReflectionUtils.invokeMethodsWithAnnotation(ReflectionUtils.java:47)
at
org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1307)
at
org.apache.nifi.controller.StandardProcessorNode$1$1.call(StandardProcessorNode.java:1303)
... 6 common frames omitted
Caused by: org.apache.nifi.processor.exception.ProcessException:
org.apache.commons.dbcp.SQLNestedException: Cannot create
PoolableConnectionFactory (IO Error: The Network Adapter could not establish
the connection)
at
org.apache.nifi.dbcp.DBCPConnectionPool.getConnection(DBCPConnectionPool.java:275)
at sun.reflect.GeneratedMethodAccessor434.invoke(Unknown Source)
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at
org.apache.nifi.controller.service.StandardControllerServiceInvocationHandler.invoke(StandardControllerServiceInvocationHandler.java:89)
at
[jira] [Commented] (NIFI-4174) GenerateTableFetch does not work with oracle on Nifi 1.2
[
https://issues.apache.org/jira/browse/NIFI-4174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16083577#comment-16083577
]
Jorge Machado commented on NIFI-4174:
-
After some tests I found out that I don't have connection to the DB.
This should throw an error like: Cannot connect to database instead of
schedulers error.
> GenerateTableFetch does not work with oracle on Nifi 1.2
>
>
> Key: NIFI-4174
> URL: https://issues.apache.org/jira/browse/NIFI-4174
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.2.0
>Reporter: Jorge Machado
>Priority: Minor
>
> I'm trying to extract some data from a oracle DB.
> I'm getting :
> {code:java}
> 2017-07-11 16:19:29,612 WARN [StandardProcessScheduler Thread-7]
> o.a.n.controller.StandardProcessorNode Timed out while waiting for
> OnScheduled of 'GenerateTableFetch' processor to finish. An attempt is made
> to cancel the task via Thread.interrupt(). However it does not guarantee that
> the task will be canceled since the code inside current OnScheduled operation
> may have been written to ignore interrupts which may result in a runaway
> thread. This could lead to more issues, eventually requiring NiFi to be
> restarted. This is usually a bug in the target Processor
> 'GenerateTableFetch[id=f08a3acd-ac7e-17d7-598b-8f9720fd92d4]' that needs to
> be documented, reported and eventually fixed.
> 2017-07-11 16:19:29,612 ERROR [StandardProcessScheduler Thread-7]
> o.a.n.p.standard.GenerateTableFetch
> GenerateTableFetch[id=f08a3acd-ac7e-17d7-598b-8f9720fd92d4]
> GenerateTableFetch[id=f08a3acd-ac7e-17d7-598b-8f9720fd92d4] failed to invoke
> @OnScheduled method due to java.lang.RuntimeException: Timed out while
> executing one of processor's OnScheduled task.; processor will not be
> scheduled to run for 30 seconds: java.lang.RuntimeException: Timed out while
> executing one of processor's OnScheduled task.
> java.lang.RuntimeException: Timed out while executing one of processor's
> OnScheduled task.
> at
> org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1480)
> at
> org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:102)
> at
> org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1303)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.util.concurrent.TimeoutException: null
> at java.util.concurrent.FutureTask.get(FutureTask.java:205)
> at
> org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1465)
> ... 9 common frames omitted
> 2017-07-11 16:19:29,613 ERROR [StandardProcessScheduler Thread-7]
> o.a.n.controller.StandardProcessorNode Failed to invoke @OnScheduled method
> due to java.lang.RuntimeException: Timed out while executing one of
> processor's OnScheduled task.
> java.lang.RuntimeException: Timed out while executing one of processor's
> OnScheduled task.
> at
> org.apache.nifi.controller.StandardProcessorNode.invokeTaskAsCancelableFuture(StandardProcessorNode.java:1480)
> at
> org.apache.nifi.controller.StandardProcessorNode.access$000(StandardProcessorNode.java:102)
> at
> org.apache.nifi.controller.StandardProcessorNode$1.run(StandardProcessorNode.java:1303)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
> at java.lang.Thread.run(Thread.java:748)
> Caused by: java.util.concurrent.TimeoutException: null
> at java.util.concurrent.FutureTask.get(FutureTask.java:205)
> at
>
