[jira] [Created] (NIFI-11335) Auto refresh for Parameter Providers

2023-03-23 Thread hipotures (Jira) 
hipotures created NIFI-11335:


 Summary: Auto refresh for Parameter Providers
 Key: NIFI-11335
 URL: https://issues.apache.org/jira/browse/NIFI-11335
 Project: Apache NiFi
  Issue Type: Improvement
Reporter: hipotures


{color:#775351}We are using Secret Manager/GCP for storing SA key. From 
{color:#775351}{color:#775351}{color:#775351}{color:#775351}GcpSecretManagerParameterProvider,
 parameters are used in GCP Credentials Provider Service. 
{color}{color}{color}{color}{color}{color:#775351}These SA keys are rotating 
every day. To activate new version of secret, we need manual refresh by _fetch 
parameters._ 
Idea: Autorefresh option to check new versions of secrets and automatic update 
processors/controllers/context if new version of parameter was detected.
{color}

 

{color:#775351}{color:#775351}{color:#775351}{color:#775351} {color}{color}   
{color}
{color}



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Created] (NIFI-10986) Failed to perform update flow request java.lang.IllegalStateException: The given connection is not currently registered for this Funnel

2022-12-15 Thread hipotures (Jira) 
hipotures created NIFI-10986:


 Summary:  Failed to perform update flow request  
java.lang.IllegalStateException: The given connection is not currently 
registered for this Funnel
 Key: NIFI-10986
 URL: https://issues.apache.org/jira/browse/NIFI-10986
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Affects Versions: 1.19.1, 1.18.0
 Environment: Debian
Reporter: hipotures


Synchro DEV-PROD using versioning flows by NiFi Registry (1.16.0).
{code:java}
2022-12-15 11:50:45,718 ERROR [Process Group Update Thread-1] 
o.apache.nifi.web.api.FlowUpdateResource Failed to perform update flow request 
java.lang.IllegalStateException: The given connection is not currently 
registered for this Funnel
    at 
org.apache.nifi.controller.StandardFunnel.removeConnection(StandardFunnel.java:224)
    at 
org.apache.nifi.connectable.StandardConnection.setDestination(StandardConnection.java:322)
    at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.restoreConnectionDestinations(StandardVersionedComponentSynchronizer.java:706)
    at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:421)
    at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:537)
    at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:411)
    at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:537)
    at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:411)
    at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:537)
    at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:411)
    at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:255)
    at 
org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:550)
    at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:253)
    at 
org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3970)
    at 
org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3950)
    at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:435)
    at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
    at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
    at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
    at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
    at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
    at 
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
    at 
org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313)
    at sun.reflect.GeneratedMethodAccessor1054.invoke(Unknown Source)
    at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
    at java.lang.reflect.Method.invoke(Method.java:498)
    at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
    at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
    at 
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
    at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
    at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
    at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
    at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
    at 
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
    at 
org.apach

[jira] [Created] (NIFI-10985) Upgrade 1.18.0 - > 1.19.1 / many local changes

2022-12-15 Thread hipotures (Jira) 
hipotures created NIFI-10985:


 Summary: Upgrade 1.18.0 - > 1.19.1 / many local changes
 Key: NIFI-10985
 URL: https://issues.apache.org/jira/browse/NIFI-10985
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Affects Versions: 1.19.1
 Environment: Debian Linux
Reporter: hipotures


We have flows synchronization (DEV->PROD) using NiFi Registry. DEV was upgraded 
5 days ago. Today we were trying synchronize new version flows DEV->PROD, but 
without success. Errror: _Failed to perform update flow request 
java.lang.IllegalStateException: The given connection is not currently 
registered for this Funnel_ (next ticket). We decide to upgrade PROD, but after 
upgrade versioning system show hundreds local changes. We've revert version to 
1.18 and there was no local changes. We are stuck here.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (NIFI-10972) PutBigQuery - Invalid project resource name projects/${Project}

2022-12-13 Thread hipotures (Jira) 


 [ 
https://issues.apache.org/jira/browse/NIFI-10972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hipotures updated NIFI-10972:
-
Description: 
Processor: *PutBigQuery* 
Parameter: *Project ID*
Value for Project ID: *${Project}*
Problem: *attributes are unevaluated*
Error log:
{code:java}
2022-12-13 10:40:18,955 ERROR [Timer-Driven Process Thread-9] 
o.a.n.p.gcp.bigquery.PutBigQuery 
PutBigQuery[id=121711dc-1182-1bfc-7f3d-6fb3bcfc7d0b] Processing halted: 
yielding [1 sec]
com.google.api.gax.rpc.InvalidArgumentException: 
io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Invalid project resource name 
projects/${Project}; Project id: ${Project}
    at 
com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:92)
    at 
com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:41)
    at 
com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:86)
    at 
com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:66)
    at 
com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
    at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:67)
    at 
com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1132)
    at 
com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)
    at 
com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1270)
    at 
com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1038)
    at 
com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:808)
    at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:563)
    at 
io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:533)
    at 
io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
    at 
io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
    at 
io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
    at 
com.google.api.gax.grpc.ChannelPool$ReleasingClientCall$1.onClose(ChannelPool.java:535)
    at 
io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:463)
    at 
io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:427)
    at 
io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:460)
    at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:562)
    at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
    at 
io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:743)
    at 
io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:722)
    at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
    at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
    at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:750)
    Suppressed: com.google.api.gax.rpc.AsyncTaskException: Asynchronous task 
failed
        at 
com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:57)
        at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112)
        at 
com.google.cloud.bigquery.storage.v1.BigQueryWriteClient.createWriteStream(BigQueryWriteClient.java:231)
        at 
org.apache.nifi.processors.gcp.bigquery.PutBigQuery.createWriteStream(PutBigQuery.java:380)
        at 
org.apache.nifi.processors.gcp.bigquery.PutBigQuery.onTrigger(PutBigQuery.java:207)
        at 
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
        at 
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1356)
        at 
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:246)
        at 
org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:59)
        at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
        at 
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.run(FutureTask.java:266)
        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
        at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
        ... 3 common frames omitted
Caused by: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Invalid project 
resource

[jira] [Created] (NIFI-10972) PutBigQuery - Invalid project resource name projects/${Project}

2022-12-13 Thread hipotures (Jira) 
hipotures created NIFI-10972:


 Summary: PutBigQuery - Invalid project resource name 
projects/${Project}
 Key: NIFI-10972
 URL: https://issues.apache.org/jira/browse/NIFI-10972
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Affects Versions: 1.19.1, 1.19.0
 Environment: Debian
Reporter: hipotures


Processor: *PutBigQuery* 
Parameter: *Project ID*
Value for Project ID: *${Project}*
Problem: *attributes are unevaluated*
Error log:
{code:java}
2022-12-13 10:40:18,983 WARN [Timer-Driven Process Thread-9] 
o.a.n.controller.tasks.ConnectableTask Processing halted: uncaught exception in 
Component [PutBigQuery[id=121711dc-1182-1bfc-7f3d-6fb3bcfc7d0b]]
com.google.api.gax.rpc.InvalidArgumentException: 
io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Invalid project resource name 
projects/${Project}; Project id: ${Project}
at 
com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:92)
at 
com.google.api.gax.rpc.ApiExceptionFactory.createException(ApiExceptionFactory.java:41)
at 
com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:86)
at 
com.google.api.gax.grpc.GrpcApiExceptionFactory.create(GrpcApiExceptionFactory.java:66)
at 
com.google.api.gax.grpc.GrpcExceptionCallable$ExceptionTransformingFuture.onFailure(GrpcExceptionCallable.java:97)
at com.google.api.core.ApiFutures$1.onFailure(ApiFutures.java:67)
at 
com.google.common.util.concurrent.Futures$CallbackListener.run(Futures.java:1132)
at 
com.google.common.util.concurrent.DirectExecutor.execute(DirectExecutor.java:31)
at 
com.google.common.util.concurrent.AbstractFuture.executeListener(AbstractFuture.java:1270)
at 
com.google.common.util.concurrent.AbstractFuture.complete(AbstractFuture.java:1038)
at 
com.google.common.util.concurrent.AbstractFuture.setException(AbstractFuture.java:808)
at io.grpc.stub.ClientCalls$GrpcFuture.setException(ClientCalls.java:563)
at io.grpc.stub.ClientCalls$UnaryStreamToFuture.onClose(ClientCalls.java:533)
at 
io.grpc.PartialForwardingClientCallListener.onClose(PartialForwardingClientCallListener.java:39)
at 
io.grpc.ForwardingClientCallListener.onClose(ForwardingClientCallListener.java:23)
at 
io.grpc.ForwardingClientCallListener$SimpleForwardingClientCallListener.onClose(ForwardingClientCallListener.java:40)
at 
com.google.api.gax.grpc.ChannelPool$ReleasingClientCall$1.onClose(ChannelPool.java:535)
at 
io.grpc.internal.DelayedClientCall$DelayedListener$3.run(DelayedClientCall.java:463)
at 
io.grpc.internal.DelayedClientCall$DelayedListener.delayOrExecute(DelayedClientCall.java:427)
at 
io.grpc.internal.DelayedClientCall$DelayedListener.onClose(DelayedClientCall.java:460)
at io.grpc.internal.ClientCallImpl.closeObserver(ClientCallImpl.java:562)
at io.grpc.internal.ClientCallImpl.access$300(ClientCallImpl.java:70)
at 
io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInternal(ClientCallImpl.java:743)
at 
io.grpc.internal.ClientCallImpl$ClientStreamListenerImpl$1StreamClosed.runInContext(ClientCallImpl.java:722)
at io.grpc.internal.ContextRunnable.run(ContextRunnable.java:37)
at io.grpc.internal.SerializingExecutor.run(SerializingExecutor.java:133)
at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:750)
Suppressed: com.google.api.gax.rpc.AsyncTaskException: Asynchronous task failed
at 
com.google.api.gax.rpc.ApiExceptions.callAndTranslateApiException(ApiExceptions.java:57)
at com.google.api.gax.rpc.UnaryCallable.call(UnaryCallable.java:112)
at 
com.google.cloud.bigquery.storage.v1.BigQueryWriteClient.createWriteStream(BigQueryWriteClient.java:231)
at 
org.apache.nifi.processors.gcp.bigquery.PutBigQuery.createWriteStream(PutBigQuery.java:380)
at 
org.apache.nifi.processors.gcp.bigquery.PutBigQuery.onTrigger(PutBigQuery.java:207)
at 
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
at 
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1356)
at 
org.apache.nifi.controller.tasks.ConnectableTask.invoke(ConnectableTask.java:246)
at 
org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.lambda$doScheduleOnce$0(AbstractTimeBasedSchedulingAgent.java:59)
at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
... 3 common frames omitted
Caused by: io.grpc.StatusRuntimeException: INVALID_ARGUMENT: Inv

[jira] [Comment Edited] (NIFI-10453) Cannot set AnnotationData while processor is running

2022-09-21 Thread hipotures (Jira) 


[ 
https://issues.apache.org/jira/browse/NIFI-10453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17607785#comment-17607785
 ] 

hipotures edited comment on NIFI-10453 at 9/21/22 2:28 PM:
---

[~mattyb149] Thank you for Your response. Only root Process Group is versioned, 
in this root group we have 30-40 "universal" controllers for sub processes 
groups (~30 groups, ~500 processors). Update is from NiFi Registry. 


was (Author: hipotures):
[~mattyb149] Thank you for Your response. Only root Process Group is versioned, 
in this root group we have 30-40 "universal" controllers for sub processes 
groups. Update is from NiFi Registry. 

> Cannot set AnnotationData while processor is running
> 
>
> Key: NIFI-10453
> URL: https://issues.apache.org/jira/browse/NIFI-10453
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Flow Versioning
>Affects Versions: 1.17.0, 1.16.3
> Environment: Linux debian
>Reporter: hipotures
>Priority: Major
>
> *Action*: Change version
> *Problem*: Can't change to newer version of flow
> *Web UI error popup*:
> Change Version
> Failed to perform update flow request due to Cannot set AnnotationData while 
> processor is running
> *Description*: Updated flow is in state STOPPED. Controller is active (this 
> controller is shared between many other flows).
> *Workaround*: Stop all flows and controllers.
> *Error Log*:
> {code:java}
> 2022-09-07 10:43:05,380 ERROR [Process Group Update Thread-1] 
> o.apache.nifi.web.api.FlowUpdateResource Failed to perform update flow 
> request 
> java.lang.IllegalStateException: Cannot set AnnotationData while processor is 
> running
>   at 
> org.apache.nifi.controller.StandardProcessorNode.setAnnotationData(StandardProcessorNode.java:1370)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.updateProcessor(StandardVersionedComponentSynchronizer.java:2529)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeProcessors(StandardVersionedComponentSynchronizer.java:880)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:407)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:489)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:401)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:489)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:401)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:248)
>   at 
> org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:464)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:246)
>   at 
> org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3825)
>   at 
> org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3805)
>   at 
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:431)
>   at 
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
>   at 
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
>   at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
>   at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>   at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
>   at 
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
>   at 
> org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313)
>   at sun.reflect.GeneratedMethodAccessor1323.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractA

[jira] [Commented] (NIFI-10453) Cannot set AnnotationData while processor is running

2022-09-21 Thread hipotures (Jira) 


[ 
https://issues.apache.org/jira/browse/NIFI-10453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17607785#comment-17607785
 ] 

hipotures commented on NIFI-10453:
--

[~mattyb149] Thank you for Your response. Only root Process Group is versioned, 
in this root group we have 30-40 "universal" controllers for sub processes 
groups. Update is from NiFi Registry. 

> Cannot set AnnotationData while processor is running
> 
>
> Key: NIFI-10453
> URL: https://issues.apache.org/jira/browse/NIFI-10453
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Flow Versioning
>Affects Versions: 1.17.0, 1.16.3
> Environment: Linux debian
>Reporter: hipotures
>Priority: Major
>
> *Action*: Change version
> *Problem*: Can't change to newer version of flow
> *Web UI error popup*:
> Change Version
> Failed to perform update flow request due to Cannot set AnnotationData while 
> processor is running
> *Description*: Updated flow is in state STOPPED. Controller is active (this 
> controller is shared between many other flows).
> *Workaround*: Stop all flows and controllers.
> *Error Log*:
> {code:java}
> 2022-09-07 10:43:05,380 ERROR [Process Group Update Thread-1] 
> o.apache.nifi.web.api.FlowUpdateResource Failed to perform update flow 
> request 
> java.lang.IllegalStateException: Cannot set AnnotationData while processor is 
> running
>   at 
> org.apache.nifi.controller.StandardProcessorNode.setAnnotationData(StandardProcessorNode.java:1370)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.updateProcessor(StandardVersionedComponentSynchronizer.java:2529)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeProcessors(StandardVersionedComponentSynchronizer.java:880)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:407)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:489)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:401)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:489)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:401)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:248)
>   at 
> org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:464)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:246)
>   at 
> org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3825)
>   at 
> org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3805)
>   at 
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:431)
>   at 
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
>   at 
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
>   at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
>   at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>   at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
>   at 
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
>   at 
> org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313)
>   at sun.reflect.GeneratedMethodAccessor1323.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
>   at 
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
>   at 
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
>   at 
> org.springframework.aop.framework.Reflective

[jira] [Commented] (NIFI-3606) MonitorAcitivity to track activity based on attribute

2022-09-21 Thread hipotures (Jira) 


[ 
https://issues.apache.org/jira/browse/NIFI-3606?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1760#comment-1760
 ] 

hipotures commented on NIFI-3606:
-

This feature really help monitoring flows.

> MonitorAcitivity to track activity based on attribute
> -
>
> Key: NIFI-3606
> URL: https://issues.apache.org/jira/browse/NIFI-3606
> Project: Apache NiFi
>  Issue Type: Improvement
>Reporter: Nicholas Carenza
>Priority: Minor
>
> Just like ControlRate can throttle based on attribute values, MonitorActivity 
> should be able to monitor based on attribute values.



--
This message was sent by Atlassian Jira
(v8.20.10#820010)

[jira] [Updated] (NIFI-10453) Cannot set AnnotationData while processor is running

2022-09-07 Thread hipotures (Jira) 


 [ 
https://issues.apache.org/jira/browse/NIFI-10453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hipotures updated NIFI-10453:
-
Affects Version/s: 1.16.3

> Cannot set AnnotationData while processor is running
> 
>
> Key: NIFI-10453
> URL: https://issues.apache.org/jira/browse/NIFI-10453
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Flow Versioning
>Affects Versions: 1.17.0, 1.16.3
> Environment: Linux debian
>Reporter: hipotures
>Priority: Major
>
> *Action*: Change version
> *Problem*: Can't change to newer version of flow
> *Web UI error popup*:
> Change Version
> Failed to perform update flow request due to Cannot set AnnotationData while 
> processor is running
> *Description*: Updated flow is in state STOPPED. Controller is active (this 
> controller is shared between many other flows).
> *Workaround*: Stop all flows and controllers.
> *Error Log*:
> {code:java}
> 2022-09-07 10:43:05,380 ERROR [Process Group Update Thread-1] 
> o.apache.nifi.web.api.FlowUpdateResource Failed to perform update flow 
> request 
> java.lang.IllegalStateException: Cannot set AnnotationData while processor is 
> running
>   at 
> org.apache.nifi.controller.StandardProcessorNode.setAnnotationData(StandardProcessorNode.java:1370)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.updateProcessor(StandardVersionedComponentSynchronizer.java:2529)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeProcessors(StandardVersionedComponentSynchronizer.java:880)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:407)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:489)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:401)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:489)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:401)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:248)
>   at 
> org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:464)
>   at 
> org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:246)
>   at 
> org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3825)
>   at 
> org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3805)
>   at 
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:431)
>   at 
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
>   at 
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
>   at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
>   at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
>   at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
>   at 
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
>   at 
> org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313)
>   at sun.reflect.GeneratedMethodAccessor1323.invoke(Unknown Source)
>   at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
>   at java.lang.reflect.Method.invoke(Method.java:498)
>   at 
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
>   at 
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
>   at 
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
>   at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
>   at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
>   at 
> org.springframework.aop.intercep

[jira] [Updated] (NIFI-10453) Cannot set AnnotationData while processor is running

2022-09-07 Thread hipotures (Jira) 


 [ 
https://issues.apache.org/jira/browse/NIFI-10453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hipotures updated NIFI-10453:
-
Description: 
*Action*: Change version
*Problem*: Can't change to newer version of flow

*Web UI error popup*:
Change Version
Failed to perform update flow request due to Cannot set AnnotationData while 
processor is running

*Description*: Updated flow is in state STOPPED. Controller is active (this 
controller is shared between many other flows).

*Workaround*: Stop all flows and controllers.

*Error Log*:

{code:java}
2022-09-07 10:43:05,380 ERROR [Process Group Update Thread-1] 
o.apache.nifi.web.api.FlowUpdateResource Failed to perform update flow request 
java.lang.IllegalStateException: Cannot set AnnotationData while processor is 
running
at 
org.apache.nifi.controller.StandardProcessorNode.setAnnotationData(StandardProcessorNode.java:1370)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.updateProcessor(StandardVersionedComponentSynchronizer.java:2529)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeProcessors(StandardVersionedComponentSynchronizer.java:880)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:407)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:489)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:401)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:489)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:401)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:248)
at 
org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:464)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:246)
at 
org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3825)
at 
org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3805)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:431)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
at 
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at 
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
at 
org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313)
at sun.reflect.GeneratedMethodAccessor1323.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
at 
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at 
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$f6deb592.updateProcessGroupFlow()

[jira] [Updated] (NIFI-10453) Cannot set AnnotationData while processor is running

2022-09-07 Thread hipotures (Jira) 


 [ 
https://issues.apache.org/jira/browse/NIFI-10453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hipotures updated NIFI-10453:
-
Description: 
Action: Change version
Problem: Can't change to newer version of flow

Web UI error popup:
Change Version
Failed to perform update flow request due to Cannot set AnnotationData while 
processor is running

Description: Updated flow is in state STOPPED. Controller is active (this 
controller is shared between many other flows).

Workaround: Stop all flows and controllers.

Error Log:

{code:java}
2022-09-07 10:43:05,380 ERROR [Process Group Update Thread-1] 
o.apache.nifi.web.api.FlowUpdateResource Failed to perform update flow request 
java.lang.IllegalStateException: Cannot set AnnotationData while processor is 
running
at 
org.apache.nifi.controller.StandardProcessorNode.setAnnotationData(StandardProcessorNode.java:1370)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.updateProcessor(StandardVersionedComponentSynchronizer.java:2529)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeProcessors(StandardVersionedComponentSynchronizer.java:880)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:407)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:489)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:401)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:489)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:401)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:248)
at 
org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:464)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:246)
at 
org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3825)
at 
org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3805)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:431)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
at 
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at 
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
at 
org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313)
at sun.reflect.GeneratedMethodAccessor1323.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
at 
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at 
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:708)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$f6deb592.updateProcessGroupFlow()
at 
org

[jira] [Created] (NIFI-10453) Cannot set AnnotationData while processor is running

2022-09-07 Thread hipotures (Jira) 
hipotures created NIFI-10453:


 Summary: Cannot set AnnotationData while processor is running
 Key: NIFI-10453
 URL: https://issues.apache.org/jira/browse/NIFI-10453
 Project: Apache NiFi
  Issue Type: Bug
  Components: Flow Versioning
Affects Versions: 1.17.0
 Environment: Linux debian
Reporter: hipotures


Action: Change version
Problem: Can't change to newer version of flow

Web UI error popup:
Change Version
Failed to perform update flow request due to Cannot set AnnotationData while 
processor is running

Description: Updated flow is in state STOPPED. Controller is active (this 
controller is shared between many other flows).

Workaround: Stop all flows and controllers.

Error Log:
2022-09-07 10:43:05,380 ERROR [Process Group Update Thread-1] 
o.apache.nifi.web.api.FlowUpdateResource Failed to perform update flow request 
java.lang.IllegalStateException: Cannot set AnnotationData while processor is 
running
at 
org.apache.nifi.controller.StandardProcessorNode.setAnnotationData(StandardProcessorNode.java:1370)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.updateProcessor(StandardVersionedComponentSynchronizer.java:2529)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeProcessors(StandardVersionedComponentSynchronizer.java:880)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:407)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:489)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:401)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronizeChildGroups(StandardVersionedComponentSynchronizer.java:489)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:401)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.lambda$synchronize$0(StandardVersionedComponentSynchronizer.java:248)
at 
org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:464)
at 
org.apache.nifi.flow.synchronization.StandardVersionedComponentSynchronizer.synchronize(StandardVersionedComponentSynchronizer.java:246)
at 
org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3825)
at 
org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3805)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:431)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
at 
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:218)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:793)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:163)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at 
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:89)
at 
org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313)
at sun.reflect.GeneratedMethodAccessor1323.invoke(Unknown Source)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:634)
at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:624)
at 
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:72)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:97)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:186)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.proceed(CglibAopProxy.java:763)
at 
org.springframework.aop.fra

[jira] [Comment Edited] (NIFI-10001) Nifi 1.16.1 issue when running

2022-05-09 Thread hipotures (Jira) 


[ 
https://issues.apache.org/jira/browse/NIFI-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17533749#comment-17533749
 ] 

hipotures edited comment on NIFI-10001 at 5/9/22 10:53 AM:
---

The same problem, upgrade 1.16.0 -> 1.16.1. Problem occure during start NiFi. 
After disabling this controller, error jump to next controller.


{code:java}
2022-05-04 15:03:49,147 ERROR [main] 
o.a.n.c.s.StandardControllerServiceProvider Failed to enable 
StandardControllerServiceNode[service=JsonRecordSetWriter[id=bd6355e9-0175-1000-aba5-eedcd0d6697a],
 name= JsonRecordSetWriter, active=true]
java.util.concurrent.TimeoutException: null
at 
java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1784)
at 
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
at 
org.apache.nifi.controller.service.StandardControllerServiceProvider.enableControllerServices(StandardControllerServiceProvider.java:209)
at 
org.apache.nifi.controller.serialization.VersionedFlowSynchronizer$FlowControllerComponentScheduler.enableNow(VersionedFlowSynchronizer.java:1022)
at 
org.apache.nifi.groups.AbstractComponentScheduler.resume(AbstractComponentScheduler.java:67)
at 
org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:402)
at 
org.apache.nifi.groups.StandardProcessGroupSynchronizer.lambda$synchronize$0(StandardProcessGroupSynchronizer.java:221)
at 
org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:462)
at 
org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:219)
at 
org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3832)
at 
org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:358)
at 
org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.sync(VersionedFlowSynchronizer.java:186)
at 
org.apache.nifi.controller.serialization.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:43)
at 
org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1524)
at 
org.apache.nifi.persistence.StandardFlowConfigurationDAO.load(StandardFlowConfigurationDAO.java:104)
at 
org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:815)
at 
org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:538)
at 
org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:67)
at 
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073)
at 
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)
at 
org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002)
at 
org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746)
at 
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
at 
org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
at 
org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
at 
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
at 
org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
at 
org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
at 
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
at org.eclipse.jetty.server.Server.start(Server.java:423)
at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
at 
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
at org.eclipse.jetty.server.Server.doStart(Server.java:387)
at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1008)
at org.apache.nifi.NiFi.(NiFi.java:170)
at org.apache.nifi.NiFi.(NiF

[jira] [Comment Edited] (NIFI-10001) Nifi 1.16.1 issue when running

2022-05-09 Thread hipotures (Jira) 


[ 
https://issues.apache.org/jira/browse/NIFI-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17533749#comment-17533749
 ] 

hipotures edited comment on NIFI-10001 at 5/9/22 10:51 AM:
---

The same problem, upgrade 1.16.0 -> 1.16.1. Problem occure during start NiFi. 
After disabling this controller, error jump to next controller.
{quote}{{2022-05-04 15:03:49,147 ERROR [main] 
o.a.n.c.s.StandardControllerServiceProvider Failed to enable 
StandardControllerServiceNode[service=JsonRecordSetWriter[id=bd6355e9-0175-1000-aba5-eedcd0d6697a],
 name= JsonRecordSetWriter, active=true]}}
{{java.util.concurrent.TimeoutException: null}}
{{        at 
java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1784)}}
{{        at 
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)}}
{{        at 
org.apache.nifi.controller.service.StandardControllerServiceProvider.enableControllerServices(StandardControllerServiceProvider.java:209)}}
{{        at 
org.apache.nifi.controller.serialization.VersionedFlowSynchronizer$FlowControllerComponentScheduler.enableNow(VersionedFlowSynchronizer.java:1022)}}
{{        at 
org.apache.nifi.groups.AbstractComponentScheduler.resume(AbstractComponentScheduler.java:67)}}
{{        at 
org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:402)}}
{{        at 
org.apache.nifi.groups.StandardProcessGroupSynchronizer.lambda$synchronize$0(StandardProcessGroupSynchronizer.java:221)}}
{{        at 
org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:462)}}
{{        at 
org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:219)}}
{{        at 
org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3832)}}
{{        at 
org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:358)}}
{{        at 
org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.sync(VersionedFlowSynchronizer.java:186)}}
{{        at 
org.apache.nifi.controller.serialization.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:43)}}
{{        at 
org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1524)}}
{{        at 
org.apache.nifi.persistence.StandardFlowConfigurationDAO.load(StandardFlowConfigurationDAO.java:104)}}
{{        at 
org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:815)}}
{{        at 
org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:538)}}
{{        at 
org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:67)}}
{{        at 
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073)}}
{{        at 
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)}}
{{        at 
org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002)}}
{{        at 
org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746)}}
{{        at 
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)}}
{{        at 
org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)}}
{{        at 
org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)}}
{{        at 
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)}}
{{        at 
org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)}}
{{        at 
org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)}}
{{        at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)}}
{{        at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)}}
{{        at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)}}
{{        at 
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)}}
{{        at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)}}
{{        at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)}}
{{        at org.eclipse.jetty.server.Server.start(Server.java:423)}}
{{        at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)}}
{{        at 
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)}}
{{        at org.eclipse.jetty.server.Server.doStart(Server.java:387)}}
{{        at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)}}
{{   

[jira] [Commented] (NIFI-10001) Nifi 1.16.1 issue when running

2022-05-09 Thread hipotures (Jira) 


[ 
https://issues.apache.org/jira/browse/NIFI-10001?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17533749#comment-17533749
 ] 

hipotures commented on NIFI-10001:
--

The same problem, upgrade 1.16.0 -> 1.16.1. Problem occure during start NiFi. 
After disabling this controller, error jump to next controller.
{quote}2022-05-04 15:03:49,147 ERROR [main] 
o.a.n.c.s.StandardControllerServiceProvider Failed to enable 
StandardControllerServiceNode[service=JsonRecordSetWriter[id=bd6355e9-0175-1000-aba5-eedcd0d6697a],
 name= JsonRecordSetWriter, active=true]
java.util.concurrent.TimeoutException: null
        at 
java.util.concurrent.CompletableFuture.timedGet(CompletableFuture.java:1784)
        at 
java.util.concurrent.CompletableFuture.get(CompletableFuture.java:1928)
        at 
org.apache.nifi.controller.service.StandardControllerServiceProvider.enableControllerServices(StandardControllerServiceProvider.java:209)
        at 
org.apache.nifi.controller.serialization.VersionedFlowSynchronizer$FlowControllerComponentScheduler.enableNow(VersionedFlowSynchronizer.java:1022)
        at 
org.apache.nifi.groups.AbstractComponentScheduler.resume(AbstractComponentScheduler.java:67)
        at 
org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:402)
        at 
org.apache.nifi.groups.StandardProcessGroupSynchronizer.lambda$synchronize$0(StandardProcessGroupSynchronizer.java:221)
        at 
org.apache.nifi.controller.flow.AbstractFlowManager.withParameterContextResolution(AbstractFlowManager.java:462)
        at 
org.apache.nifi.groups.StandardProcessGroupSynchronizer.synchronize(StandardProcessGroupSynchronizer.java:219)
        at 
org.apache.nifi.groups.StandardProcessGroup.synchronizeFlow(StandardProcessGroup.java:3832)
        at 
org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.synchronizeFlow(VersionedFlowSynchronizer.java:358)
        at 
org.apache.nifi.controller.serialization.VersionedFlowSynchronizer.sync(VersionedFlowSynchronizer.java:186)
        at 
org.apache.nifi.controller.serialization.StandardFlowSynchronizer.sync(StandardFlowSynchronizer.java:43)
        at 
org.apache.nifi.controller.FlowController.synchronize(FlowController.java:1524)
        at 
org.apache.nifi.persistence.StandardFlowConfigurationDAO.load(StandardFlowConfigurationDAO.java:104)
        at 
org.apache.nifi.controller.StandardFlowService.loadFromBytes(StandardFlowService.java:815)
        at 
org.apache.nifi.controller.StandardFlowService.load(StandardFlowService.java:538)
        at 
org.apache.nifi.web.contextlistener.ApplicationStartupContextListener.contextInitialized(ApplicationStartupContextListener.java:67)
        at 
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:1073)
        at 
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:572)
        at 
org.eclipse.jetty.server.handler.ContextHandler.contextInitialized(ContextHandler.java:1002)
        at 
org.eclipse.jetty.servlet.ServletHandler.initialize(ServletHandler.java:746)
        at 
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:379)
        at 
org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1449)
        at 
org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1414)
        at 
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:916)
        at 
org.eclipse.jetty.servlet.ServletContextHandler.doStart(ServletContextHandler.java:288)
        at 
org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:524)
        at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
        at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
        at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:117)
        at 
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
        at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
        at 
org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:169)
        at org.eclipse.jetty.server.Server.start(Server.java:423)
        at 
org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:110)
        at 
org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:97)
        at org.eclipse.jetty.server.Server.doStart(Server.java:387)
        at 
org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:73)
        at org.apache.nifi.web.server.JettyServer.start(JettyServer.java:1008)
        at org.apache.nifi.NiFi.(NiFi.java:170)
        at org.apache.nifi.NiFi.(NiFi.java:82)
        at org.apache.nifi.NiFi.main(NiFi.jav

[jira] [Created] (NIFI-9370) Run Once functionality with CRON driven scheduling

2021-11-09 Thread hipotures (Jira) 
hipotures created NIFI-9370:
---

 Summary: Run Once functionality with CRON driven scheduling
 Key: NIFI-9370
 URL: https://issues.apache.org/jira/browse/NIFI-9370
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core UI
Affects Versions: 1.15.0, 1.14.0
Reporter: hipotures


*Settings:*

Scheduling Strategy: CRON driven
Processor state: Stopped

*Action:* 

Run Once
**

*Workaround:* 

terminate processor state

 

2021-11-09 14:47:45,537 WARN [Timer-Driven Process Thread-4] 
o.a.n.controller.StandardProcessorNode Failed while shutting down processor 
GenerateFlowFile[id=04f41444-017d-1000-8ce8-e1d1037091d6]
java.lang.IllegalStateException: Cannot unschedule 
GenerateFlowFile[id=04f41444-017d-1000-8ce8-e1d1037091d6] because it was not 
scheduled to run
    at 
org.apache.nifi.controller.scheduling.QuartzSchedulingAgent.unschedule(QuartzSchedulingAgent.java:178)
    at 
org.apache.nifi.controller.scheduling.QuartzSchedulingAgent.doUnschedule(QuartzSchedulingAgent.java:167)
    at 
org.apache.nifi.controller.scheduling.AbstractSchedulingAgent.unschedule(AbstractSchedulingAgent.java:61)
    at 
org.apache.nifi.controller.scheduling.AbstractTimeBasedSchedulingAgent.unschedule(AbstractTimeBasedSchedulingAgent.java:36)
    at 
org.apache.nifi.controller.StandardProcessorNode$2.run(StandardProcessorNode.java:1790)
    at org.apache.nifi.engine.FlowEngine$2.run(FlowEngine.java:110)
    at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
    at java.util.concurrent.FutureTask.run(FutureTask.java:266)
    at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
    at 
java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
    at 
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
    at 
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
    at java.lang.Thread.run(Thread.java:748)



--
This message was sent by Atlassian Jira
(v8.20.1#820001)

[jira] [Created] (NIFI-8077) ConvertRecord Json to Avro with empty array

2020-12-07 Thread hipotures (Jira) 
hipotures created NIFI-8077:
---

 Summary: ConvertRecord Json to Avro with empty array
 Key: NIFI-8077
 URL: https://issues.apache.org/jira/browse/NIFI-8077
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Affects Versions: 1.12.1
 Environment: 18.04.1-Ubuntu
Reporter: hipotures


ConvertRecord processor with settings
   Record Reader: JsonTreeReader (with Infer Schema)
   Record Writer: AvroRecordSetWriter (with Embed Avro Schema)

Error:
{noformat}
PutBigQueryBatch[id=dc0f47e9-0175-1000-f086-85c2c62eea10] Provided Schema does 
not match Table nfz_agreements.agreement. Field plans has changed type from 
RECORD to STRING{noformat}

First source JSON (part):
{noformat}
(...)
"plans":[
 {
 "id":"8abb0ba3-f0e5-4720-70d3-bfef74c16932",
 "type":"agreement-plan",
 "attributes":{
 "product_code":"02..078.02",
 "product_name":"BADANIA ENDOSKOPOWE PRZEWODU POKARMOWEGO - GASTROSKOPIA",
 "order":1,
 "unit_count":167430.0,
 "price":155709.9,
 "avg_price":0.93
 },
 "links":{
 
"related":"https://api.nfz.gov.pl/app-umw-api/plans/8abb0ba3-f0e5-4720-70d3-bfef74c16932?format=json";
 }
 }
],
(...){noformat}
 


Second source JSON (part):
{noformat}
(...)
"plans" : [ ],
(...){noformat}
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Updated] (NIFI-8075) ConvertRecord Json to Avro with null values

2020-12-07 Thread hipotures (Jira) 


 [ 
https://issues.apache.org/jira/browse/NIFI-8075?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hipotures updated NIFI-8075:

Description: 
ConvertRecord processor with settings
   Record Reader: JsonTreeReader (with Infer Schema)
   Record Writer: AvroRecordSetWriter (with Embed Avro Schema)
 change non {color:#00}schema field names{color} avro (e.g. with "-", "@") 
to fields with underscore, but with NULL values.

Example below.

Input JSON:
{noformat}
{
   "id":"6e07e04d-d644-bf76-f30f-ce648da478e0",
   "type":"agreement",
   "attributes":{
  "code":"16-00-04332-20-01",
  "origin-code":"07-19-305",
  "service-type":"07",
  "service-name":"Leczenie stomatologiczne",
  "amount":112500.0,
  "updated-at":"2019-12-13T08:39:20",
  "provider-code":"160004332",
  "provider-name":"ISP STOMATOLOGICZNA JANUSZ KUBRAK",
  "provider-place":"SZCZECIN",
  "year":2020,
  "branch":"16"
   },
   "links":{
  
"related":"https://api.nfz.gov.pl/app-umw-api/agreements/6e07e04d-d644-bf76-f30f-ce648da478e0?format=json";
   }
}{noformat}
Output Avro data:
{code:java}
[
   {
  "id":"6e07e04d-d644-bf76-f30f-ce648da478e0",
  "type":"agreement",
  "attributes":{
 "code":"16-00-04332-20-01",
 "origin_code":null,
 "service_type":null,
 "service_name":null,
 "amount":112500.0,
 "updated_at":null,
 "provider_code":null,
 "provider_name":null,
 "provider_place":null,
 "year":2020,
 "branch":"16"
  },
  "links":{
 
"related":"https://api.nfz.gov.pl/app-umw-api/agreements/6e07e04d-d644-bf76-f30f-ce648da478e0?format=json";
  }
   }
]{code}
Output Avro schema:
{noformat}
{
   "type":"record",
   "name":"nifiRecord",
   "namespace":"org.apache.nifi",
   "fields":[
  {
 "name":"id",
 "type":[
"null",
"string"
 ]
  },
  {
 "name":"type",
 "type":[
"null",
"string"
 ]
  },
  {
 "name":"attributes",
 "type":[
"null",
{
   "type":"record",
   "name":"attributesType",
   "fields":[
  {
 "name":"code",
 "type":[
"null",
"string"
 ]
  },
  {
 "name":"origin_code",
 "type":[
"null",
"string"
 ],
 "aliases":[
"origin-code"
 ]
  },
  {
 "name":"service_type",
 "type":[
"null",
"string"
 ],
 "aliases":[
"service-type"
 ]
  },
  {
 "name":"service_name",
 "type":[
"null",
"string"
 ],
 "aliases":[
"service-name"
 ]
  },
  {
 "name":"amount",
 "type":[
"null",
"double"
 ]
  },
  {
 "name":"updated_at",
 "type":[
"null",
"string"
 ],
 "aliases":[
"updated-at"
 ]
  },
  {
 "name":"provider_code",
 "type":[
"null",
"string"
 ],
 "aliases":[
"provider-code"
 ]
  },
  {
 "name":"provider_name",
 "type":[
"null",
"string"
 ],
 "aliases":[
"provider-name"
 ]
  },
  {
 "name":"provider_place",
 "type":[
"null",
"string"
 ],
 "aliases":[
"provider-place"
 ]
  },
  {
 "name":"year",
 "type":[
"null",
"long"
 ]
  },
  {

[jira] [Updated] (NIFI-8076) ConvertRecord - Can't redefine

2020-12-07 Thread hipotures (Jira) 


 [ 
https://issues.apache.org/jira/browse/NIFI-8076?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

hipotures updated NIFI-8076:

Description: 
{color:#172b4d}ConvertRecord processor with settings{color}
 {color:#172b4d}  Record Reader: JsonTreeReader (with Infer Schema){color}
 {color:#172b4d}  Record Writer: AvroRecordSetWriter (with Embed Avro 
Schema){color}
 Problem with two fields with name *attributes*.
{noformat}
ConvertRecord[id=dc0f47e8-0175-1000-9f2c-184ff602c868] Failed to process 
StandardFlowFileRecord[uuid=5d1699a9-cde5-472b-a78b-79a155f3c4b2,claim=StandardContentClaim
 
 [resourceClaim=StandardResourceClaim[id=1607356761339-4176, container=default, 
section=80], offset=454572, 
length=7824],offset=4958,name=e4a67bb4-68b5-4ca7-8967-6f68f94e9874,size=865]; 
 will route to failure: Can't redefine: org.apache.nifi.attributesType{noformat}
Example below (paths *$.agreement.attributes* and *$.plans[0].attributes*)

Input JSON
{code:java}
{
   "agreement":{
  "id":"6e07e04d-d644-bf76-f30f-ce648da478e0",
  "type":"agreement",
  "attributes":{
 "code":"16-00-04332-20-01",
 "origin_code":"07-19-305",
 "service_type":"07",
 "service_name":"Leczenie stomatologiczne",
 "amount":112500.0,
 "updated_at":"2019-12-13T08:39:20",
 "provider_code":"160004332",
 "provider_name":"ISP STOMATOLOGICZNA JANUSZ KUBRAK",
 "provider_place":"SZCZECIN",
 "year":2020,
 "branch":"16"
  },
  "links":null
   },
   "plans":[
  {
 "id":"e040eca0-ab80-5823-8ab7-0333d2063159",
 "type":"agreement-plan",
 "attributes":{
"product_code":"07..214.02",
"product_name":"ŚWIADCZENIA PROTETYKI STOMATOLOGICZNEJ",
"order":1,
"unit_count":9.0,
"price":112500.0,
"avg_price":1.25
 },
 "links":{

"related":"https://api.nfz.gov.pl/app-umw-api/plans/e040eca0-ab80-5823-8ab7-0333d2063159?format=json";
 }
  }
   ],
   "orthopedic_supplies":[
  
   ]
}{code}
Workaround: JoltTransformationJSON shift:
{code:java}
{
   "agreement":{
  "attributes":"agreement.attr",
  "*":"agreement.&"
   },
   "*":"&"
}{code}
 

  was:
{color:#172b4d}ConvertRecord processor with settings{color}
{color:#172b4d}  Record Reader: JsonTreeReader (with Infer Schema){color}
{color:#172b4d}  Record Writer: AvroRecordSetWriter (with Embed Avro 
Schema){color}
Problem with two fields with name *attributes*.
{noformat}
ConvertRecord[id=dc0f47e8-0175-1000-9f2c-184ff602c868] Failed to process 
StandardFlowFileRecord[uuid=5d1699a9-cde5-472b-a78b-79a155f3c4b2,claim=StandardContentClaim
 
 [resourceClaim=StandardResourceClaim[id=1607356761339-4176, container=default, 
section=80], offset=454572, 
length=7824],offset=4958,name=e4a67bb4-68b5-4ca7-8967-6f68f94e9874,size=865]; 
 will route to failure: Can't redefine: org.apache.nifi.attributesType{noformat}

Example below (paths *$.agreement.attributes* and *$.plans[0].attributes*)

Input JSON
{code:java}
{
 "agreement" : {
 "id" : "6e07e04d-d644-bf76-f30f-ce648da478e0",
 "type" : "agreement",
 "attributes" : {
 "code" : "16-00-04332-20-01",
 "origin_code" : "07-19-305",
 "service_type" : "07",
 "service_name" : "Leczenie stomatologiczne",
 "amount" : 112500.0,
 "updated_at" : "2019-12-13T08:39:20",
 "provider_code" : "160004332",
 "provider_name" : "ISP STOMATOLOGICZNA JANUSZ KUBRAK",
 "provider_place" : "SZCZECIN",
 "year" : 2020,
 "branch" : "16"
 },
 "links" : null
 },
 "plans" : [ {
 "id" : "e040eca0-ab80-5823-8ab7-0333d2063159",
 "type" : "agreement-plan",
 "attributes" : {
 "product_code" : "07..214.02",
 "product_name" : "ŚWIADCZENIA PROTETYKI STOMATOLOGICZNEJ",
 "order" : 1,
 "unit_count" : 9.0,
 "price" : 112500.0,
 "avg_price" : 1.25
 },
 "links" : {
 "related" : 
"https://api.nfz.gov.pl/app-umw-api/plans/e040eca0-ab80-5823-8ab7-0333d2063159?format=json";
 }
 } ],
 "orthopedic_supplies" : [ ]
}{code}


Workaround: JoltTransformationJSON shift:
{code:java}
{
 "agreement": {
 "attributes": "agreement.attr",
 "*": "agreement.&"
 },
 "*": "&"
}{code}
 


> ConvertRecord - Can't redefine
> --
>
> Key: NIFI-8076
> URL: https://issues.apache.org/jira/browse/NIFI-8076
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.12.1
> Environment: 18.04.1-Ubuntu
>Reporter: hipotures
>Priority: Major
>
> {color:#172b4d}ConvertRecord processor with settings{color}
>  {color:#172b4d}  Record Reader: JsonTreeReader (with Infer Schema){color}
>  {color:#172b4d}  Record Writer: AvroRecordSetWriter (with Embed Avro 
> Schema){color}
>  Problem with two fields with name *attributes*.
> {noformat}
> ConvertRecord[id=dc0f47e8-0175-1000-9f2c-184ff602c868] Failed to

[jira] [Created] (NIFI-8076) ConvertRecord - Can't redefine

2020-12-07 Thread hipotures (Jira) 
hipotures created NIFI-8076:
---

 Summary: ConvertRecord - Can't redefine
 Key: NIFI-8076
 URL: https://issues.apache.org/jira/browse/NIFI-8076
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Affects Versions: 1.12.1
 Environment: 18.04.1-Ubuntu
Reporter: hipotures


{color:#172b4d}ConvertRecord processor with settings{color}
{color:#172b4d}  Record Reader: JsonTreeReader (with Infer Schema){color}
{color:#172b4d}  Record Writer: AvroRecordSetWriter (with Embed Avro 
Schema){color}
Problem with two fields with name *attributes*.
{noformat}
ConvertRecord[id=dc0f47e8-0175-1000-9f2c-184ff602c868] Failed to process 
StandardFlowFileRecord[uuid=5d1699a9-cde5-472b-a78b-79a155f3c4b2,claim=StandardContentClaim
 
 [resourceClaim=StandardResourceClaim[id=1607356761339-4176, container=default, 
section=80], offset=454572, 
length=7824],offset=4958,name=e4a67bb4-68b5-4ca7-8967-6f68f94e9874,size=865]; 
 will route to failure: Can't redefine: org.apache.nifi.attributesType{noformat}

Example below (paths *$.agreement.attributes* and *$.plans[0].attributes*)

Input JSON
{code:java}
{
 "agreement" : {
 "id" : "6e07e04d-d644-bf76-f30f-ce648da478e0",
 "type" : "agreement",
 "attributes" : {
 "code" : "16-00-04332-20-01",
 "origin_code" : "07-19-305",
 "service_type" : "07",
 "service_name" : "Leczenie stomatologiczne",
 "amount" : 112500.0,
 "updated_at" : "2019-12-13T08:39:20",
 "provider_code" : "160004332",
 "provider_name" : "ISP STOMATOLOGICZNA JANUSZ KUBRAK",
 "provider_place" : "SZCZECIN",
 "year" : 2020,
 "branch" : "16"
 },
 "links" : null
 },
 "plans" : [ {
 "id" : "e040eca0-ab80-5823-8ab7-0333d2063159",
 "type" : "agreement-plan",
 "attributes" : {
 "product_code" : "07..214.02",
 "product_name" : "ŚWIADCZENIA PROTETYKI STOMATOLOGICZNEJ",
 "order" : 1,
 "unit_count" : 9.0,
 "price" : 112500.0,
 "avg_price" : 1.25
 },
 "links" : {
 "related" : 
"https://api.nfz.gov.pl/app-umw-api/plans/e040eca0-ab80-5823-8ab7-0333d2063159?format=json";
 }
 } ],
 "orthopedic_supplies" : [ ]
}{code}


Workaround: JoltTransformationJSON shift:
{code:java}
{
 "agreement": {
 "attributes": "agreement.attr",
 "*": "agreement.&"
 },
 "*": "&"
}{code}
 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)

[jira] [Created] (NIFI-8075) ConvertRecord Json to Avro with null values

2020-12-07 Thread hipotures (Jira) 
hipotures created NIFI-8075:
---

 Summary: ConvertRecord Json to Avro with null values
 Key: NIFI-8075
 URL: https://issues.apache.org/jira/browse/NIFI-8075
 Project: Apache NiFi
  Issue Type: Bug
  Components: Core Framework
Affects Versions: 1.12.1
 Environment: 18.04.1-Ubuntu
Reporter: hipotures


ConvertRecord processor with settings
  Record Reader: JsonTreeReader (with Infer Schema)
  Record Writer: AvroRecordSetWriter (with Embed Avro Schema)
change non {color:#00}schema field names{color} avro (e.g. with "-", "@") 
to fields with underscore, but with NULL values.

Example below.

Input JSON:
{noformat}
{
 "id" : "6e07e04d-d644-bf76-f30f-ce648da478e0",
 "type" : "agreement",
 "attributes" : {
 "code" : "16-00-04332-20-01",
 "origin-code" : "07-19-305",
 "service-type" : "07",
 "service-name" : "Leczenie stomatologiczne",
 "amount" : 112500.0,
 "updated-at" : "2019-12-13T08:39:20",
 "provider-code" : "160004332",
 "provider-name" : "ISP STOMATOLOGICZNA JANUSZ KUBRAK",
 "provider-place" : "SZCZECIN",
 "year" : 2020,
 "branch" : "16"
 },
 "links" : {
 "related" : 
"https://api.nfz.gov.pl/app-umw-api/agreements/6e07e04d-d644-bf76-f30f-ce648da478e0?format=json";
 }
}{noformat}

Output Avro data:
{code:java}
[ {
 "id" : "6e07e04d-d644-bf76-f30f-ce648da478e0",
 "type" : "agreement",
 "attributes" : {
 "code" : "16-00-04332-20-01",
 "origin_code" : null,
 "service_type" : null,
 "service_name" : null,
 "amount" : 112500.0,
 "updated_at" : null,
 "provider_code" : null,
 "provider_name" : null,
 "provider_place" : null,
 "year" : 2020,
 "branch" : "16"
 },
 "links" : {
 "related" : 
"https://api.nfz.gov.pl/app-umw-api/agreements/6e07e04d-d644-bf76-f30f-ce648da478e0?format=json";
 }
} ]{code}

Output Avro schema:
{noformat}
{
 "type":"record",
 "name":"nifiRecord",
 "namespace":"org.apache.nifi",
 "fields":[
 {
 "name":"id",
 "type":[
 "null",
 "string"
 ]
 },
 {
 "name":"type",
 "type":[
 "null",
 "string"
 ]
 },
 {
 "name":"attributes",
 "type":[
 "null",
 {
 "type":"record",
 "name":"attributesType",
 "fields":[
 {
 "name":"code",
 "type":[
 "null",
 "string"
 ]
 },
 {
 "name":"origin_code",
 "type":[
 "null",
 "string"
 ],
 "aliases":[
 "origin-code"
 ]
 },
 {
 "name":"service_type",
 "type":[
 "null",
 "string"
 ],
 "aliases":[
 "service-type"
 ]
 },
 {
 "name":"service_name",
 "type":[
 "null",
 "string"
 ],
 "aliases":[
 "service-name"
 ]
 },
 {
 "name":"amount",
 "type":[
 "null",
 "double"
 ]
 },
 {
 "name":"updated_at",
 "type":[
 "null",
 "string"
 ],
 "aliases":[
 "updated-at"
 ]
 },
 {
 "name":"provider_code",
 "type":[
 "null",
 "string"
 ],
 "aliases":[
 "provider-code"
 ]
 },
 {
 "name":"provider_name",
 "type":[
 "null",
 "string"
 ],
 "aliases":[
 "provider-name"
 ]
 },
 {
 "name":"provider_place",
 "type":[
 "null",
 "string"
 ],
 "aliases":[
 "provider-place"
 ]
 },
 {
 "name":"year",
 "type":[
 "null",
 "long"
 ]
 },
 {
 "name":"branch",
 "type":[
 "null",
 "string"
 ]
 }
 ]
 }
 ]
 },
 {
 "name":"links",
 "type":[
 "null",
 {
 "type":"record",
 "name":"linksType",
 "fields":[
 {
 "name":"related",
 "type":[
 "null",
 "string"
 ]
 }
 ]
 }
 ]
 }
 ]
}{noformat}
 

Workaround: for "static" JSONs - ReplaceTextWithMapping processor before 
ConvertRecord.

 

 

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)