[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync

2020-07-08 Thread GitBox


hunyadi-dev commented on a change in pull request #835:
URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r452001585



##
File path: extensions/http-curl/tests/HTTPIntegrationBase.h
##
@@ -135,6 +135,9 @@ class VerifyC2Describe : public VerifyC2Base {
   }
 
   void runAssertions() override {
+// This class is never used for running assertions, but we are forced to 
wait for DescribeManifestHandler to verifyJsonHasAgentManifest
+// if we were to log something on finished verification, we could poll on 
finding it 
+std::this_thread::sleep_for(std::chrono::milliseconds(wait_time_));

Review comment:
   We no longer have `waitToVerifyProcessor()` though...





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (NIFI-7622) updateParameterContext NullPointerException on StandardProcessGroup.java:4220

2020-07-08 Thread Eric Secules (Jira)


 [ 
https://issues.apache.org/jira/browse/NIFI-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Secules updated NIFI-7622:
---
Description: 
I am getting this null pointer exception when importing one versioned flow from 
a registry which contains another versioned flow and each uses different 
parameter contexts.

I am using version 1.11.4 of NiFi

This error is blocking me from developing flows which comprise common 
components and project-specific components that use their own parameter 
contexts.

Attached a screen cap of the state of the stack at 
StandardProcessGroup.java:4220
{code:java}
2020-07-09 05:33:10,414 ERROR [NiFi Web Server-86] 
o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: 
java.lang.NullPointerException. Returning Internal Server Error response.
java.lang.NullPointerException: null
at 
org.apache.nifi.groups.StandardProcessGroup.updateParameterContext(StandardProcessGroup.java:4220)
at 
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3754)
at 
org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324)
at 
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876)
at 
org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324)
at 
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876)
at 
org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324)
at 
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876)
at 
org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3653)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:408)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
at 
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at 
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84)
at 
org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627)
at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:616)
at 
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at 
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$202f172f.updateProcessGroupFlow()
at 
org.apache.nifi.web.StandardNiFiServiceFacade$14.update(StandardNiFiServiceFacade.java:5044)
at 
org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:117)
at 
org.apache.nifi.web.StandardNiFiServiceFacade.updateProcessGroupContents(StandardNiFiServiceFacade.java:5040)
at 
org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
at 
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at 
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84)
at 
org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithWriteLock(NiFiServiceFacadeLock.java:179)
at 
org.apache.nifi.web.NiFiServiceFacadeLock.updateLock(NiFiServ

[jira] [Updated] (NIFI-7622) updateParameterContext NullPointerException on StandardProcessGroup.java:4220

2020-07-08 Thread Eric Secules (Jira)


 [ 
https://issues.apache.org/jira/browse/NIFI-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Secules updated NIFI-7622:
---
Description: 
I am getting this null pointer exception when importing one versioned flow 
which contains another versioned flow and each uses different parameter 
contexts.

I am using version 1.11.4 of NiFi

This error is blocking me from developing flows which comprise common 
components and project-specific components that use their own parameter 
contexts.

Attached a screen cap of the state of the stack at 
StandardProcessGroup.java:4220
{code:java}
2020-07-09 05:33:10,414 ERROR [NiFi Web Server-86] 
o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: 
java.lang.NullPointerException. Returning Internal Server Error response.
java.lang.NullPointerException: null
at 
org.apache.nifi.groups.StandardProcessGroup.updateParameterContext(StandardProcessGroup.java:4220)
at 
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3754)
at 
org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324)
at 
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876)
at 
org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324)
at 
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876)
at 
org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324)
at 
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876)
at 
org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3653)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:408)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
at 
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at 
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84)
at 
org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627)
at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:616)
at 
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at 
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$202f172f.updateProcessGroupFlow()
at 
org.apache.nifi.web.StandardNiFiServiceFacade$14.update(StandardNiFiServiceFacade.java:5044)
at 
org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:117)
at 
org.apache.nifi.web.StandardNiFiServiceFacade.updateProcessGroupContents(StandardNiFiServiceFacade.java:5040)
at 
org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
at 
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at 
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84)
at 
org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithWriteLock(NiFiServiceFacadeLock.java:179)
at 
org.apache.nifi.web.NiFiServiceFacadeLock.updateLock(NiFiServiceFacadeLock.ja

[jira] [Created] (NIFI-7623) Take unnecessary behavior out when using registry to update the flow

2020-07-08 Thread Hsin-Ying Lee (Jira)
Hsin-Ying Lee created NIFI-7623:
---

 Summary: Take unnecessary behavior out when using registry to 
update the flow
 Key: NIFI-7623
 URL: https://issues.apache.org/jira/browse/NIFI-7623
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework, Flow Versioning
Reporter: Hsin-Ying Lee


We tried to use the registry to change version/revert for a versioned PG, the 
changes only were processor position, color style, and connection bend points 
changes.

When the processor was doing the long time tasks, the version control would not 
complete until the processors stopped, even that is not necessary to stop.

If we tried to cancel the request, we also have to wait for the processors to 
stop and go back to start it manually.

 

I'm trying to fix this issue, and make a PR for this.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (NIFI-7622) updateParameterContext NullPointerException on StandardProcessGroup.java:4220

2020-07-08 Thread Eric Secules (Jira)


 [ 
https://issues.apache.org/jira/browse/NIFI-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Secules updated NIFI-7622:
---
Description: 
I am getting this null pointer exception when importing one versioned flow 
which contains another versioned flow and each uses different parameter 
contexts.

This error is blocking me from developing flows which comprise common 
components and project-specific components that use their own parameter 
contexts.

Attached a screen cap of the state of the stack at 
StandardProcessGroup.java:4220
{code:java}
2020-07-09 05:33:10,414 ERROR [NiFi Web Server-86] 
o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: 
java.lang.NullPointerException. Returning Internal Server Error response.
java.lang.NullPointerException: null
at 
org.apache.nifi.groups.StandardProcessGroup.updateParameterContext(StandardProcessGroup.java:4220)
at 
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3754)
at 
org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324)
at 
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876)
at 
org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324)
at 
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876)
at 
org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324)
at 
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876)
at 
org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3653)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:408)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
at 
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at 
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84)
at 
org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627)
at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:616)
at 
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at 
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$202f172f.updateProcessGroupFlow()
at 
org.apache.nifi.web.StandardNiFiServiceFacade$14.update(StandardNiFiServiceFacade.java:5044)
at 
org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:117)
at 
org.apache.nifi.web.StandardNiFiServiceFacade.updateProcessGroupContents(StandardNiFiServiceFacade.java:5040)
at 
org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
at 
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at 
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84)
at 
org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithWriteLock(NiFiServiceFacadeLock.java:179)
at 
org.apache.nifi.web.NiFiServiceFacadeLock.updateLock(NiFiServiceFacadeLock.java:66)
at sun.reflect.Gener

[jira] [Updated] (NIFI-7622) updateParameterContext NullPointerException on StandardProcessGroup.java:4220

2020-07-08 Thread Eric Secules (Jira)


 [ 
https://issues.apache.org/jira/browse/NIFI-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Eric Secules updated NIFI-7622:
---
Attachment: Screen Shot 2020-07-08 at 11.04.37 PM.png

> updateParameterContext NullPointerException on StandardProcessGroup.java:4220
> -
>
> Key: NIFI-7622
> URL: https://issues.apache.org/jira/browse/NIFI-7622
> Project: Apache NiFi
>  Issue Type: Bug
>Affects Versions: 1.11.4
>Reporter: Eric Secules
>Priority: Major
> Attachments: Screen Shot 2020-07-08 at 11.04.37 PM.png
>
>
> I am getting this null pointer exception when importing one versioned flow 
> which contains another versioned flow and each uses different parameter 
> contexts.
> This error is blocking me from developing flows which comprise common 
> components and project-specific components that use their own parameter 
> contexts.
> {code:java}
> 2020-07-09 05:33:10,414 ERROR [NiFi Web Server-86] 
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: 
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at 
> org.apache.nifi.groups.StandardProcessGroup.updateParameterContext(StandardProcessGroup.java:4220)
> at 
> org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3754)
> at 
> org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324)
> at 
> org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876)
> at 
> org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324)
> at 
> org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876)
> at 
> org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324)
> at 
> org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876)
> at 
> org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3653)
> at 
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:408)
> at 
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
> at 
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> at 
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
> at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> at 
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84)
> at 
> org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313)
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> at 
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> at 
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:498)
> at 
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627)
> at 
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:616)
> at 
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
> at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at 
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> at 
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> at 
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
> at 
> org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$202f172f.updateProcessGroupFlow()
> at 
> org.apache.nifi.web.StandardNiFiServiceFacade$14.update(StandardNiFiServiceFacade.java:5044)
> at 
> org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:117)
> at 
> org.apache.nifi.web.StandardNiFiServiceFacade.updateProcessGroupContents(StandardNiFiServiceFacade.java:5040)
> at 
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> at 
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> 

[jira] [Created] (NIFI-7622) updateParameterContext NullPointerException on StandardProcessGroup.java:4220

2020-07-08 Thread Eric Secules (Jira)
Eric Secules created NIFI-7622:
--

 Summary: updateParameterContext NullPointerException on 
StandardProcessGroup.java:4220
 Key: NIFI-7622
 URL: https://issues.apache.org/jira/browse/NIFI-7622
 Project: Apache NiFi
  Issue Type: Bug
Affects Versions: 1.11.4
Reporter: Eric Secules


I am getting this null pointer exception when importing one versioned flow 
which contains another versioned flow and each uses different parameter 
contexts.

This error is blocking me from developing flows which comprise common 
components and project-specific components that use their own parameter 
contexts.
{code:java}
2020-07-09 05:33:10,414 ERROR [NiFi Web Server-86] 
o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: 
java.lang.NullPointerException. Returning Internal Server Error response.
java.lang.NullPointerException: null
at 
org.apache.nifi.groups.StandardProcessGroup.updateParameterContext(StandardProcessGroup.java:4220)
at 
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3754)
at 
org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324)
at 
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876)
at 
org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324)
at 
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876)
at 
org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324)
at 
org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876)
at 
org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3653)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:408)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke()
at 
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at 
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84)
at 
org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:498)
at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627)
at 
org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:616)
at 
org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at 
org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at 
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671)
at 
org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$202f172f.updateProcessGroupFlow()
at 
org.apache.nifi.web.StandardNiFiServiceFacade$14.update(StandardNiFiServiceFacade.java:5044)
at 
org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:117)
at 
org.apache.nifi.web.StandardNiFiServiceFacade.updateProcessGroupContents(StandardNiFiServiceFacade.java:5040)
at 
org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
at 
org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at 
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736)
at 
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at 
org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84)
at 
org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithWriteLock(NiFiServiceFacadeLock.java:179)
at 
org.apach

[GitHub] [nifi] alopresto opened a new pull request #4396: NIFI-7621 & NIFI-7614 Updated terminology throughout codebase and doc…

2020-07-08 Thread GitBox


alopresto opened a new pull request #4396:
URL: https://github.com/apache/nifi/pull/4396


   …umentation.
   
   Signed-off-by: Andy LoPresto 
   
   Thank you for submitting a contribution to Apache NiFi.
   
   Please provide a short description of the PR here:
   
    Description of PR
   
   _Changes legacy terminology around keys, clustering, and integrations where 
new terms and transition for users are available._
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [x] Is there a JIRA ticket associated with this PR? Is it referenced 
in the commit message?
   
   - [x] Does your PR title start with **NIFI-** where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [x] Has your PR been rebased against the latest commit within the target 
branch (typically `master`)?
   
   - [x] Is your initial contribution a single, squashed commit? _Additional 
commits in response to PR reviewer feedback should be made on this branch and 
pushed to allow change tracking. Do not `squash` or use `--force` when pushing 
to allow for clean monitoring of changes._
   
   ### For code changes:
   - [x] Have you ensured that the full suite of tests is executed via `mvn 
-Pcontrib-check clean install` at the root `nifi` folder?
   - [x] Have you written or updated unit tests to verify your changes?
   - [ ] Have you verified that the full build is successful on JDK 8?
   - [x] Have you verified that the full build is successful on JDK 11?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
   - [ ] If applicable, have you updated the `LICENSE` file, including the main 
`LICENSE` file under `nifi-assembly`?
   - [ ] If applicable, have you updated the `NOTICE` file, including the main 
`NOTICE` file found under `nifi-assembly`?
   - [ ] If adding new Properties, have you added `.displayName` in addition to 
.name (programmatic access) for each of the new properties?
   
   ### For documentation related changes:
   - [x] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check GitHub Actions CI for 
build issues and submit an update to your PR as soon as possible.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Closed] (NIFI-7606) Provide "NIFI_HOME()" function in Expression Language

2020-07-08 Thread ZhangCheng (Jira)


 [ 
https://issues.apache.org/jira/browse/NIFI-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhangCheng closed NIFI-7606.


Abandoned

> Provide "NIFI_HOME()" function in Expression Language
> -
>
> Key: NIFI-7606
> URL: https://issues.apache.org/jira/browse/NIFI-7606
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: ZhangCheng
>Assignee: ZhangCheng
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Usually,  I make a file '$NIFI_HOME/jdbc' to hold some jars of driivers. I 
> find it's useful to provide a function to get the path NIFI installed, and we 
> will use this such as `$(NIFI_HOME):append('/jdbc/ojdbc8.jar'))` in 
> `DbcpConnectionPoll` property. I think it will be useful for other scenes 
> need to indicate file path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (NIFI-7606) Provide "NIFI_HOME()" function in Expression Language

2020-07-08 Thread ZhangCheng (Jira)


 [ 
https://issues.apache.org/jira/browse/NIFI-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhangCheng resolved NIFI-7606.
--
Resolution: Abandoned

It's not necessary.  ${NIFI_HOME} is useful

> Provide "NIFI_HOME()" function in Expression Language
> -
>
> Key: NIFI-7606
> URL: https://issues.apache.org/jira/browse/NIFI-7606
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: ZhangCheng
>Assignee: ZhangCheng
>Priority: Major
>  Time Spent: 50m
>  Remaining Estimate: 0h
>
> Usually,  I make a file '$NIFI_HOME/jdbc' to hold some jars of driivers. I 
> find it's useful to provide a function to get the path NIFI installed, and we 
> will use this such as `$(NIFI_HOME):append('/jdbc/ojdbc8.jar'))` in 
> `DbcpConnectionPoll` property. I think it will be useful for other scenes 
> need to indicate file path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [nifi] KuKuDeCheng commented on pull request #4393: NIFI-7606:Provide "NIFI_HOME()" function in Expression Language to ob…

2020-07-08 Thread GitBox


KuKuDeCheng commented on pull request #4393:
URL: https://github.com/apache/nifi/pull/4393#issuecomment-655846568


   > Thanks, but we can't accept contributions like this without test coverage. 
Once you add that we can proceed with a review.
   
   @MikeThomsen I find that it's not necessary. Thank you very much for review 
and I will close this PR



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi] KuKuDeCheng closed pull request #4393: NIFI-7606:Provide "NIFI_HOME()" function in Expression Language to ob…

2020-07-08 Thread GitBox


KuKuDeCheng closed pull request #4393:
URL: https://github.com/apache/nifi/pull/4393


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi] KuKuDeCheng commented on pull request #4393: NIFI-7606:Provide "NIFI_HOME()" function in Expression Language to ob…

2020-07-08 Thread GitBox


KuKuDeCheng commented on pull request #4393:
URL: https://github.com/apache/nifi/pull/4393#issuecomment-655845996


   > I don't think this function is necessary. It's simply exposing an 
environment variable, but all environment variables already are available via 
Expression Language. You can simply use `${NIFI_HOME}`
   
   @markap14 It's useful!Thank you very much.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (NIFI-7610) Edit README.md for dockerhub to reflect 1.12.0 release

2020-07-08 Thread Chad Zobrisky (Jira)


 [ 
https://issues.apache.org/jira/browse/NIFI-7610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chad Zobrisky updated NIFI-7610:

Status: Patch Available  (was: Open)

> Edit README.md for dockerhub to reflect 1.12.0 release
> --
>
> Key: NIFI-7610
> URL: https://issues.apache.org/jira/browse/NIFI-7610
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Affects Versions: 1.11.4
>Reporter: Chad Zobrisky
>Priority: Minor
> Fix For: 1.12.0
>
>  Time Spent: 20m
>  Remaining Estimate: 0h
>
> The readme.md for dockerhub still refers to 1.8.0. This Jira ticket is to 
> reflect the latest version in main, 1.12.0 as the latest dockerhub image.
>  
> Ref: 
> [https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/README.md]
> dockerhub: [https://hub.docker.com/r/apache/nifi/]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (NIFI-7621) Remove legacy terminology around branch names from GitHub messaging and developer documentation

2020-07-08 Thread Andy LoPresto (Jira)
Andy LoPresto created NIFI-7621:
---

 Summary: Remove legacy terminology around branch names from GitHub 
messaging and developer documentation
 Key: NIFI-7621
 URL: https://issues.apache.org/jira/browse/NIFI-7621
 Project: Apache NiFi
  Issue Type: Task
  Components: Documentation & Website, Tools and Build
Affects Versions: 1.11.4
Reporter: Andy LoPresto
Assignee: Andy LoPresto


Change references to {{master}} branch to reflect new name {{main}}. 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (NIFI-7619) Resolve failing unit test due to timeout in CI/CD environment

2020-07-08 Thread Andy LoPresto (Jira)


 [ 
https://issues.apache.org/jira/browse/NIFI-7619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy LoPresto resolved NIFI-7619.
-
Fix Version/s: 1.12.0
   Resolution: Duplicate

Jira network instability caused creation of duplicate tickets. 

> Resolve failing unit test due to timeout in CI/CD environment
> -
>
> Key: NIFI-7619
> URL: https://issues.apache.org/jira/browse/NIFI-7619
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.11.4
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
>  Labels: ci-failure, site-to-site, unit-test
> Fix For: 1.12.0
>
>
> There is a test failure on MacOS JP 1.8 for GitHub Actions:
> {code}
> [ERROR] Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
> 42.397 s <<< FAILURE! - in org.apache.nifi.remote.client.http.TestHttpClient
> [ERROR] 
> testReceiveTimeoutAfterDataExchange(org.apache.nifi.remote.client.http.TestHttpClient)
>   Time elapsed: 21.275 s  <<< FAILURE!
> java.lang.AssertionError
>   at 
> org.apache.nifi.remote.client.http.TestHttpClient.testReceiveTimeoutAfterDataExchange(TestHttpClient.java:1384)
> [WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 
> 0.454 s - in org.apache.nifi.remote.client.socket.TestSiteToSiteClient
> [ERROR] Failures: 
> [ERROR]   TestHttpClient.testReceiveTimeoutAfterDataExchange:1384
> [ERROR] Tests run: 92, Failures: 1, Errors: 0, Skipped: 2
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (NIFI-7618) Resolve failing unit test due to timeout in CI/CD environment

2020-07-08 Thread Andy LoPresto (Jira)


 [ 
https://issues.apache.org/jira/browse/NIFI-7618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy LoPresto resolved NIFI-7618.
-
Fix Version/s: 1.12.0
   Resolution: Duplicate

Jira network instability caused creation of duplicate tickets. 

> Resolve failing unit test due to timeout in CI/CD environment
> -
>
> Key: NIFI-7618
> URL: https://issues.apache.org/jira/browse/NIFI-7618
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.11.4
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
>  Labels: ci-failure, site-to-site, unit-test
> Fix For: 1.12.0
>
>
> There is a test failure on MacOS JP 1.8 for GitHub Actions:
> {code}
> [ERROR] Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
> 42.397 s <<< FAILURE! - in org.apache.nifi.remote.client.http.TestHttpClient
> [ERROR] 
> testReceiveTimeoutAfterDataExchange(org.apache.nifi.remote.client.http.TestHttpClient)
>   Time elapsed: 21.275 s  <<< FAILURE!
> java.lang.AssertionError
>   at 
> org.apache.nifi.remote.client.http.TestHttpClient.testReceiveTimeoutAfterDataExchange(TestHttpClient.java:1384)
> [WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 
> 0.454 s - in org.apache.nifi.remote.client.socket.TestSiteToSiteClient
> [ERROR] Failures: 
> [ERROR]   TestHttpClient.testReceiveTimeoutAfterDataExchange:1384
> [ERROR] Tests run: 92, Failures: 1, Errors: 0, Skipped: 2
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (NIFI-7617) Resolve failing unit test due to timeout in CI/CD environment

2020-07-08 Thread Andy LoPresto (Jira)


 [ 
https://issues.apache.org/jira/browse/NIFI-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andy LoPresto resolved NIFI-7617.
-
Fix Version/s: 1.12.0
   Resolution: Duplicate

Jira network instability caused creation of duplicate tickets. 

> Resolve failing unit test due to timeout in CI/CD environment
> -
>
> Key: NIFI-7617
> URL: https://issues.apache.org/jira/browse/NIFI-7617
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Tools and Build
>Affects Versions: 1.11.4
>Reporter: Andy LoPresto
>Assignee: Andy LoPresto
>Priority: Major
>  Labels: ci-failure, site-to-site, unit-test
> Fix For: 1.12.0
>
>
> There is a test failure on MacOS JP 1.8 for GitHub Actions:
> {code}
> [ERROR] Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 
> 42.397 s <<< FAILURE! - in org.apache.nifi.remote.client.http.TestHttpClient
> [ERROR] 
> testReceiveTimeoutAfterDataExchange(org.apache.nifi.remote.client.http.TestHttpClient)
>   Time elapsed: 21.275 s  <<< FAILURE!
> java.lang.AssertionError
>   at 
> org.apache.nifi.remote.client.http.TestHttpClient.testReceiveTimeoutAfterDataExchange(TestHttpClient.java:1384)
> [WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 
> 0.454 s - in org.apache.nifi.remote.client.socket.TestSiteToSiteClient
> [ERROR] Failures: 
> [ERROR]   TestHttpClient.testReceiveTimeoutAfterDataExchange:1384
> [ERROR] Tests run: 92, Failures: 1, Errors: 0, Skipped: 2
> {code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [nifi] alopresto opened a new pull request #4395: NIFI-7620 Fixed failing unit test due to timeout.

2020-07-08 Thread GitBox


alopresto opened a new pull request #4395:
URL: https://github.com/apache/nifi/pull/4395


   Thank you for submitting a contribution to Apache NiFi.
   
   Please provide a short description of the PR here:
   
    Description of PR
   
   _Fixes a failing unit test in one of the CI/CD builds._
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [x] Is there a JIRA ticket associated with this PR? Is it referenced 
in the commit message?
   
   - [x] Does your PR title start with **NIFI-** where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [x] Has your PR been rebased against the latest commit within the target 
branch (typically `master`)?
   
   - [x] Is your initial contribution a single, squashed commit? _Additional 
commits in response to PR reviewer feedback should be made on this branch and 
pushed to allow change tracking. Do not `squash` or use `--force` when pushing 
to allow for clean monitoring of changes._
   
   ### For code changes:
   - [x] Have you ensured that the full suite of tests is executed via `mvn 
-Pcontrib-check clean install` at the root `nifi` folder?
   - [x] Have you written or updated unit tests to verify your changes?
   - [ ] Have you verified that the full build is successful on JDK 8?
   - [ ] Have you verified that the full build is successful on JDK 11?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
   - [ ] If applicable, have you updated the `LICENSE` file, including the main 
`LICENSE` file under `nifi-assembly`?
   - [ ] If applicable, have you updated the `NOTICE` file, including the main 
`NOTICE` file found under `nifi-assembly`?
   - [ ] If adding new Properties, have you added `.displayName` in addition to 
.name (programmatic access) for each of the new properties?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check GitHub Actions CI for 
build issues and submit an update to your PR as soon as possible.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Created] (NIFI-7620) Resolve failing unit test due to timeout in CI/CD environment

2020-07-08 Thread Andy LoPresto (Jira)
Andy LoPresto created NIFI-7620:
---

 Summary: Resolve failing unit test due to timeout in CI/CD 
environment
 Key: NIFI-7620
 URL: https://issues.apache.org/jira/browse/NIFI-7620
 Project: Apache NiFi
  Issue Type: Bug
  Components: Tools and Build
Affects Versions: 1.11.4
Reporter: Andy LoPresto
Assignee: Andy LoPresto


There is a test failure on MacOS JP 1.8 for GitHub Actions:

{code}
[ERROR] Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 42.397 
s <<< FAILURE! - in org.apache.nifi.remote.client.http.TestHttpClient
[ERROR] 
testReceiveTimeoutAfterDataExchange(org.apache.nifi.remote.client.http.TestHttpClient)
  Time elapsed: 21.275 s  <<< FAILURE!
java.lang.AssertionError
at 
org.apache.nifi.remote.client.http.TestHttpClient.testReceiveTimeoutAfterDataExchange(TestHttpClient.java:1384)

[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0.454 
s - in org.apache.nifi.remote.client.socket.TestSiteToSiteClient
[ERROR] Failures: 
[ERROR]   TestHttpClient.testReceiveTimeoutAfterDataExchange:1384
[ERROR] Tests run: 92, Failures: 1, Errors: 0, Skipped: 2
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (NIFI-7619) Resolve failing unit test due to timeout in CI/CD environment

2020-07-08 Thread Andy LoPresto (Jira)
Andy LoPresto created NIFI-7619:
---

 Summary: Resolve failing unit test due to timeout in CI/CD 
environment
 Key: NIFI-7619
 URL: https://issues.apache.org/jira/browse/NIFI-7619
 Project: Apache NiFi
  Issue Type: Bug
  Components: Tools and Build
Affects Versions: 1.11.4
Reporter: Andy LoPresto
Assignee: Andy LoPresto


There is a test failure on MacOS JP 1.8 for GitHub Actions:

{code}
[ERROR] Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 42.397 
s <<< FAILURE! - in org.apache.nifi.remote.client.http.TestHttpClient
[ERROR] 
testReceiveTimeoutAfterDataExchange(org.apache.nifi.remote.client.http.TestHttpClient)
  Time elapsed: 21.275 s  <<< FAILURE!
java.lang.AssertionError
at 
org.apache.nifi.remote.client.http.TestHttpClient.testReceiveTimeoutAfterDataExchange(TestHttpClient.java:1384)

[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0.454 
s - in org.apache.nifi.remote.client.socket.TestSiteToSiteClient
[ERROR] Failures: 
[ERROR]   TestHttpClient.testReceiveTimeoutAfterDataExchange:1384
[ERROR] Tests run: 92, Failures: 1, Errors: 0, Skipped: 2
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (NIFI-7618) Resolve failing unit test due to timeout in CI/CD environment

2020-07-08 Thread Andy LoPresto (Jira)
Andy LoPresto created NIFI-7618:
---

 Summary: Resolve failing unit test due to timeout in CI/CD 
environment
 Key: NIFI-7618
 URL: https://issues.apache.org/jira/browse/NIFI-7618
 Project: Apache NiFi
  Issue Type: Bug
  Components: Tools and Build
Affects Versions: 1.11.4
Reporter: Andy LoPresto
Assignee: Andy LoPresto


There is a test failure on MacOS JP 1.8 for GitHub Actions:

{code}
[ERROR] Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 42.397 
s <<< FAILURE! - in org.apache.nifi.remote.client.http.TestHttpClient
[ERROR] 
testReceiveTimeoutAfterDataExchange(org.apache.nifi.remote.client.http.TestHttpClient)
  Time elapsed: 21.275 s  <<< FAILURE!
java.lang.AssertionError
at 
org.apache.nifi.remote.client.http.TestHttpClient.testReceiveTimeoutAfterDataExchange(TestHttpClient.java:1384)

[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0.454 
s - in org.apache.nifi.remote.client.socket.TestSiteToSiteClient
[ERROR] Failures: 
[ERROR]   TestHttpClient.testReceiveTimeoutAfterDataExchange:1384
[ERROR] Tests run: 92, Failures: 1, Errors: 0, Skipped: 2
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (NIFI-7617) Resolve failing unit test due to timeout in CI/CD environment

2020-07-08 Thread Andy LoPresto (Jira)
Andy LoPresto created NIFI-7617:
---

 Summary: Resolve failing unit test due to timeout in CI/CD 
environment
 Key: NIFI-7617
 URL: https://issues.apache.org/jira/browse/NIFI-7617
 Project: Apache NiFi
  Issue Type: Bug
  Components: Tools and Build
Affects Versions: 1.11.4
Reporter: Andy LoPresto
Assignee: Andy LoPresto


There is a test failure on MacOS JP 1.8 for GitHub Actions:

{code}
[ERROR] Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 42.397 
s <<< FAILURE! - in org.apache.nifi.remote.client.http.TestHttpClient
[ERROR] 
testReceiveTimeoutAfterDataExchange(org.apache.nifi.remote.client.http.TestHttpClient)
  Time elapsed: 21.275 s  <<< FAILURE!
java.lang.AssertionError
at 
org.apache.nifi.remote.client.http.TestHttpClient.testReceiveTimeoutAfterDataExchange(TestHttpClient.java:1384)

[WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0.454 
s - in org.apache.nifi.remote.client.socket.TestSiteToSiteClient
[ERROR] Failures: 
[ERROR]   TestHttpClient.testReceiveTimeoutAfterDataExchange:1384
[ERROR] Tests run: 92, Failures: 1, Errors: 0, Skipped: 2
{code}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [nifi] turcsanyip commented on a change in pull request #4370: NIFI-6128 UnpackContent: Store unpacked file data

2020-07-08 Thread GitBox


turcsanyip commented on a change in pull request #4370:
URL: https://github.com/apache/nifi/pull/4370#discussion_r451834785



##
File path: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/FileInfo.java
##
@@ -217,6 +217,20 @@ public Builder group(String group) {
 }
 }
 
+public static String permissionToString(int fileModeOctal) {
+if(fileModeOctal > 0777 || fileModeOctal < 00) {
+throw new RuntimeException("Invalid permission numerals");
+}
+char[] PERM = "xwrxwrxwr".toCharArray();

Review comment:
   I meant it to be a constant, that's why the uppercase.

##
File path: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/UnpackContent.java
##
@@ -110,6 +118,14 @@
 
 public static final String OCTET_STREAM = "application/octet-stream";
 
+public static final String FILE_LAST_MODIFIED_TIME_ATTRIBUTE = 
"file.lastModifiedTime";
+public static final String FILE_OWNER_ATTRIBUTE = "file.owner";
+public static final String FILE_GROUP_ATTRIBUTE = "file.group";
+public static final String FILE_PERMISSIONS_ATTRIBUTE = "file.permissions";
+
+public static final String FILE_MODIFIED_DATE_ATTR_FORMAT = 
"-MM-dd'T'HH:mm:ssZ";
+public static final DateTimeFormatter dateTimeFormatter = 
DateTimeFormatter.ofPattern(FILE_MODIFIED_DATE_ATTR_FORMAT).withZone(ZoneId.systemDefault());

Review comment:
   Please use uppercase for constant names.

##
File path: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/FileInfo.java
##
@@ -217,6 +217,20 @@ public Builder group(String group) {
 }
 }
 
+public static String permissionToString(int fileModeOctal) {
+if(fileModeOctal > 0777 || fileModeOctal < 00) {

Review comment:
   Formatting: `if (`

##
File path: 
nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/FileInfo.java
##
@@ -217,6 +217,20 @@ public Builder group(String group) {
 }
 }
 
+public static String permissionToString(int fileModeOctal) {

Review comment:
   As it is a pure function without side effects, some unit tests could be 
added easily.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Created] (NIFI-7616) PrometheusReportingTask: Wrong number of processor active threads reported ?

2020-07-08 Thread Benjamin Charron (Jira)
Benjamin Charron created NIFI-7616:
--

 Summary: PrometheusReportingTask: Wrong number of processor active 
threads reported ?
 Key: NIFI-7616
 URL: https://issues.apache.org/jira/browse/NIFI-7616
 Project: Apache NiFi
  Issue Type: Bug
  Components: Extensions
Affects Versions: 1.9.2
Reporter: Benjamin Charron


The number of _processor_ active threads reported by prometheus 
("nifi_amount_threads_active") seem off.

I think "{{status.getActiveThreadCount()}}" should be 
"{{processorStatus.getActiveThreadCount()}}" at 
[PrometheusMetricsUtil.java:193|https://github.com/apache/nifi/blob/aa741cc5967f62c3c38c2a47e712b7faa6fe19ff/nifi-nar-bundles/nifi-extension-utils/nifi-prometheus-utils/src/main/java/org/apache/nifi/prometheus/util/PrometheusMetricsUtil.java#L193]
 and :194

 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (MINIFICPP-1284) TailFile: Add Record Limit

2020-07-08 Thread Andrew Christianson (Jira)
Andrew Christianson created MINIFICPP-1284:
--

 Summary: TailFile: Add Record Limit
 Key: MINIFICPP-1284
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1284
 Project: Apache NiFi MiNiFi C++
  Issue Type: Improvement
Reporter: Andrew Christianson
Assignee: Andrew Christianson


{{TailFile}} currently will create an unlimited number of new FlowFiles per 
invocation of {{onTrigger}}. This can quickly fill up queues/repositories 
within minificpp. To allow operators to control the rate that data is ingested 
into minificpp, we need a {{Record Limit}} property which will limit the number 
of new FlowFiles created per tailed file per {{onTrigger}} invocation.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (NIFI-7572) Add a ScriptedTransformRecord processor

2020-07-08 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/NIFI-7572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153967#comment-17153967
 ] 

ASF subversion and git services commented on NIFI-7572:
---

Commit 6cece9cad746c39ada327d064e67e52dd8b7d959 in nifi's branch 
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6cece9c ]

NIFI-7572: Added ScriptedTransformRecord processor. Addressed a couple of minor 
bugs in mock classes that were encountered during testing.
NIFI-7572: Addressed review feedback
NIFI-7572: Fixed bug that resulted in constantly recompiling Jython script. 
Updated documentation showing performance difference. Fixed problematic unit 
tests for TestResizeImage

This closes #4374

Signed-off-by: Mike Thomsen 


> Add a ScriptedTransformRecord processor
> ---
>
> Key: NIFI-7572
> URL: https://issues.apache.org/jira/browse/NIFI-7572
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> NiFi has started to put a heavier emphasis on Record-oriented processors, as 
> they provide many benefits including better performance and a better UX over 
> their purely byte-oriented counterparts. It is common to see users wanting to 
> transform a Record in some very specific way, but NiFi doesn't make this as 
> easy as it should. There are methods using ExecuteScript, 
> InvokedScriptedProcessor, ScriptedRecordWriter, and ScriptedRecordReader for 
> instance.
> But each of these requires that the Script writer understand a lot about NiFi 
> and how to expose properties, create Property Descriptors, etc. and for 
> fairly simple transformation we end up with scripts where the logic takes 
> fewer lines of code than the boilerplate.
> We should expose a Processor that allows a user to write a script that takes 
> a Record and transforms that Record in some way. The processor should be 
> configured with the following:
>  * Record Reader (required)
>  * Record Writer (required)
>  * Script Language (required)
>  * Script Body or Script File (one and only one of these required)
> The script should implement a single method along the lines of:
> {code:java}
> Record transform(Record input) throws Exception; {code}
> If the script returns null, the input Record should be dropped. Otherwise, 
> whatever Record is returned should be written to the Record Writer.
> The processor should have two relationships: "success" and "failure."
> The script should not be allowed to expose any properties or define any 
> relationships. The point is to keep the script focused purely on processing 
> the record itself.
> It's not entirely clear to me how easy the Record API works with some of the 
> scripting languages. The Record object does expose a method named toMap() 
> that returns a Map containing the underlying key/value pairs. 
> However, the values in that Map may themselves be Records. It might make 
> sense to expose a new method toNormalizedMap() or something along those lines 
> that would return a Map where the values have been 
> recursively normalized, in much the same way that we do for 
> JoltTransformRecord. This would perhaps allow for cleaner syntax, but I'm not 
> a scripting expert so I can't say for sure whether such a method is necessary.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (NIFI-7572) Add a ScriptedTransformRecord processor

2020-07-08 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/NIFI-7572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153968#comment-17153968
 ] 

ASF subversion and git services commented on NIFI-7572:
---

Commit 6cece9cad746c39ada327d064e67e52dd8b7d959 in nifi's branch 
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6cece9c ]

NIFI-7572: Added ScriptedTransformRecord processor. Addressed a couple of minor 
bugs in mock classes that were encountered during testing.
NIFI-7572: Addressed review feedback
NIFI-7572: Fixed bug that resulted in constantly recompiling Jython script. 
Updated documentation showing performance difference. Fixed problematic unit 
tests for TestResizeImage

This closes #4374

Signed-off-by: Mike Thomsen 


> Add a ScriptedTransformRecord processor
> ---
>
> Key: NIFI-7572
> URL: https://issues.apache.org/jira/browse/NIFI-7572
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> NiFi has started to put a heavier emphasis on Record-oriented processors, as 
> they provide many benefits including better performance and a better UX over 
> their purely byte-oriented counterparts. It is common to see users wanting to 
> transform a Record in some very specific way, but NiFi doesn't make this as 
> easy as it should. There are methods using ExecuteScript, 
> InvokedScriptedProcessor, ScriptedRecordWriter, and ScriptedRecordReader for 
> instance.
> But each of these requires that the Script writer understand a lot about NiFi 
> and how to expose properties, create Property Descriptors, etc. and for 
> fairly simple transformation we end up with scripts where the logic takes 
> fewer lines of code than the boilerplate.
> We should expose a Processor that allows a user to write a script that takes 
> a Record and transforms that Record in some way. The processor should be 
> configured with the following:
>  * Record Reader (required)
>  * Record Writer (required)
>  * Script Language (required)
>  * Script Body or Script File (one and only one of these required)
> The script should implement a single method along the lines of:
> {code:java}
> Record transform(Record input) throws Exception; {code}
> If the script returns null, the input Record should be dropped. Otherwise, 
> whatever Record is returned should be written to the Record Writer.
> The processor should have two relationships: "success" and "failure."
> The script should not be allowed to expose any properties or define any 
> relationships. The point is to keep the script focused purely on processing 
> the record itself.
> It's not entirely clear to me how easy the Record API works with some of the 
> scripting languages. The Record object does expose a method named toMap() 
> that returns a Map containing the underlying key/value pairs. 
> However, the values in that Map may themselves be Records. It might make 
> sense to expose a new method toNormalizedMap() or something along those lines 
> that would return a Map where the values have been 
> recursively normalized, in much the same way that we do for 
> JoltTransformRecord. This would perhaps allow for cleaner syntax, but I'm not 
> a scripting expert so I can't say for sure whether such a method is necessary.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Updated] (NIFI-7572) Add a ScriptedTransformRecord processor

2020-07-08 Thread Mike Thomsen (Jira)


 [ 
https://issues.apache.org/jira/browse/NIFI-7572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Mike Thomsen updated NIFI-7572:
---
Fix Version/s: 1.12.0
   Resolution: Fixed
   Status: Resolved  (was: Patch Available)

> Add a ScriptedTransformRecord processor
> ---
>
> Key: NIFI-7572
> URL: https://issues.apache.org/jira/browse/NIFI-7572
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
> Fix For: 1.12.0
>
>  Time Spent: 3h 40m
>  Remaining Estimate: 0h
>
> NiFi has started to put a heavier emphasis on Record-oriented processors, as 
> they provide many benefits including better performance and a better UX over 
> their purely byte-oriented counterparts. It is common to see users wanting to 
> transform a Record in some very specific way, but NiFi doesn't make this as 
> easy as it should. There are methods using ExecuteScript, 
> InvokedScriptedProcessor, ScriptedRecordWriter, and ScriptedRecordReader for 
> instance.
> But each of these requires that the Script writer understand a lot about NiFi 
> and how to expose properties, create Property Descriptors, etc. and for 
> fairly simple transformation we end up with scripts where the logic takes 
> fewer lines of code than the boilerplate.
> We should expose a Processor that allows a user to write a script that takes 
> a Record and transforms that Record in some way. The processor should be 
> configured with the following:
>  * Record Reader (required)
>  * Record Writer (required)
>  * Script Language (required)
>  * Script Body or Script File (one and only one of these required)
> The script should implement a single method along the lines of:
> {code:java}
> Record transform(Record input) throws Exception; {code}
> If the script returns null, the input Record should be dropped. Otherwise, 
> whatever Record is returned should be written to the Record Writer.
> The processor should have two relationships: "success" and "failure."
> The script should not be allowed to expose any properties or define any 
> relationships. The point is to keep the script focused purely on processing 
> the record itself.
> It's not entirely clear to me how easy the Record API works with some of the 
> scripting languages. The Record object does expose a method named toMap() 
> that returns a Map containing the underlying key/value pairs. 
> However, the values in that Map may themselves be Records. It might make 
> sense to expose a new method toNormalizedMap() or something along those lines 
> that would return a Map where the values have been 
> recursively normalized, in much the same way that we do for 
> JoltTransformRecord. This would perhaps allow for cleaner syntax, but I'm not 
> a scripting expert so I can't say for sure whether such a method is necessary.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Commented] (NIFI-7572) Add a ScriptedTransformRecord processor

2020-07-08 Thread ASF subversion and git services (Jira)


[ 
https://issues.apache.org/jira/browse/NIFI-7572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153966#comment-17153966
 ] 

ASF subversion and git services commented on NIFI-7572:
---

Commit 6cece9cad746c39ada327d064e67e52dd8b7d959 in nifi's branch 
refs/heads/main from Mark Payne
[ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6cece9c ]

NIFI-7572: Added ScriptedTransformRecord processor. Addressed a couple of minor 
bugs in mock classes that were encountered during testing.
NIFI-7572: Addressed review feedback
NIFI-7572: Fixed bug that resulted in constantly recompiling Jython script. 
Updated documentation showing performance difference. Fixed problematic unit 
tests for TestResizeImage

This closes #4374

Signed-off-by: Mike Thomsen 


> Add a ScriptedTransformRecord processor
> ---
>
> Key: NIFI-7572
> URL: https://issues.apache.org/jira/browse/NIFI-7572
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: Mark Payne
>Assignee: Mark Payne
>Priority: Major
>  Time Spent: 3.5h
>  Remaining Estimate: 0h
>
> NiFi has started to put a heavier emphasis on Record-oriented processors, as 
> they provide many benefits including better performance and a better UX over 
> their purely byte-oriented counterparts. It is common to see users wanting to 
> transform a Record in some very specific way, but NiFi doesn't make this as 
> easy as it should. There are methods using ExecuteScript, 
> InvokedScriptedProcessor, ScriptedRecordWriter, and ScriptedRecordReader for 
> instance.
> But each of these requires that the Script writer understand a lot about NiFi 
> and how to expose properties, create Property Descriptors, etc. and for 
> fairly simple transformation we end up with scripts where the logic takes 
> fewer lines of code than the boilerplate.
> We should expose a Processor that allows a user to write a script that takes 
> a Record and transforms that Record in some way. The processor should be 
> configured with the following:
>  * Record Reader (required)
>  * Record Writer (required)
>  * Script Language (required)
>  * Script Body or Script File (one and only one of these required)
> The script should implement a single method along the lines of:
> {code:java}
> Record transform(Record input) throws Exception; {code}
> If the script returns null, the input Record should be dropped. Otherwise, 
> whatever Record is returned should be written to the Record Writer.
> The processor should have two relationships: "success" and "failure."
> The script should not be allowed to expose any properties or define any 
> relationships. The point is to keep the script focused purely on processing 
> the record itself.
> It's not entirely clear to me how easy the Record API works with some of the 
> scripting languages. The Record object does expose a method named toMap() 
> that returns a Map containing the underlying key/value pairs. 
> However, the values in that Map may themselves be Records. It might make 
> sense to expose a new method toNormalizedMap() or something along those lines 
> that would return a Map where the values have been 
> recursively normalized, in much the same way that we do for 
> JoltTransformRecord. This would perhaps allow for cleaner syntax, but I'm not 
> a scripting expert so I can't say for sure whether such a method is necessary.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [nifi] asfgit closed pull request #4374: NIFI-7572: Added ScriptedTransformRecord processor. Addressed a coupl…

2020-07-08 Thread GitBox


asfgit closed pull request #4374:
URL: https://github.com/apache/nifi/pull/4374


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi] MikeThomsen commented on pull request #4374: NIFI-7572: Added ScriptedTransformRecord processor. Addressed a coupl…

2020-07-08 Thread GitBox


MikeThomsen commented on pull request #4374:
URL: https://github.com/apache/nifi/pull/4374#issuecomment-655721235


   @mattyb149 I'll do it.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Commented] (NIFI-7583) Update User Guide to explain how swapping affects queue prioritization

2020-07-08 Thread Ryan Hendrickson (Jira)


[ 
https://issues.apache.org/jira/browse/NIFI-7583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153857#comment-17153857
 ] 

Ryan Hendrickson commented on NIFI-7583:


   For the documentation, I'd ask that you consider adding something to the 
"Configure Connection -> Available Prioritizer" Help Icon.. Right now it says 
"Available prioritizers that could reprioritize FlowFiles in this work queue."  
And the Selected Prioritizers says: "Prioritizers that have been selected to 
prioritize FlowFiles in this work queue."  Something to notify users of the 
behavior and NiFi attribute that controls it would be helpful.


   For a long term improvement..  What about providing an override?.. Similar 
to the box for "Size Threshold" and "Back Pressure Object Threshold", could 
there be one with "Priority Queue Size", set to default to 20,000, but could be 
modified on a connection-by-connection basis?  That'd help mitigate what I'm 
afraid is going to happen when we jacked the size from 20,000 to 200,000 for 
the entire server instead of the single queue.

> Update User Guide to explain how swapping affects queue prioritization
> --
>
> Key: NIFI-7583
> URL: https://issues.apache.org/jira/browse/NIFI-7583
> Project: Apache NiFi
>  Issue Type: Task
>  Components: Configuration
>Reporter: Mark Payne
>Priority: Major
>




--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [nifi] czobrisky commented on pull request #4394: NIFI-7610 Updated dockerhub readme to latest nifi version

2020-07-08 Thread GitBox


czobrisky commented on pull request #4394:
URL: https://github.com/apache/nifi/pull/4394#issuecomment-655695236


   Just updated the Readme for dockerhub, this should probably be part of the 
release guide when doing the DockerImage.txt and Dockerfile version update, 
just to remove any confusion about the most recent version in Docker Hub. If 
the Release Management should be updated, I don't mind doing that as well.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi] czobrisky opened a new pull request #4394: NIFI-7610 Updated dockerhub readme to latest nifi version

2020-07-08 Thread GitBox


czobrisky opened a new pull request #4394:
URL: https://github.com/apache/nifi/pull/4394


   Thank you for submitting a contribution to Apache NiFi.
   
   Please provide a short description of the PR here:
   
   Updates Docker Hub Readme to be the latest nifi version, 1.12.0
   
   Improves Docker hub documentation: NIFI-7610
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [x] Is there a JIRA ticket associated with this PR? Is it referenced 
in the commit message?
   
   - [x] Does your PR title start with **NIFI-** where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [x] Has your PR been rebased against the latest commit within the target 
branch (typically `master`)?
   
   - [x] Is your initial contribution a single, squashed commit? _Additional 
commits in response to PR reviewer feedback should be made on this branch and 
pushed to allow change tracking. Do not `squash` or use `--force` when pushing 
to allow for clean monitoring of changes._
   
   ### For code changes:
   - [ ] Have you ensured that the full suite of tests is executed via `mvn 
-Pcontrib-check clean install` at the root `nifi` folder?
   - [ ] Have you written or updated unit tests to verify your changes?
   - [ ] Have you verified that the full build is successful on JDK 8?
   - [ ] Have you verified that the full build is successful on JDK 11?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
   - [ ] If applicable, have you updated the `LICENSE` file, including the main 
`LICENSE` file under `nifi-assembly`?
   - [ ] If applicable, have you updated the `NOTICE` file, including the main 
`NOTICE` file found under `nifi-assembly`?
   - [ ] If adding new Properties, have you added `.displayName` in addition to 
.name (programmatic access) for each of the new properties?
   
   ### For documentation related changes:
   - [x] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check GitHub Actions CI for 
build issues and submit an update to your PR as soon as possible.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi] mattyb149 commented on pull request #4350: NIFI-6934 In PutDatabaseRecord added Postgres UPSERT support

2020-07-08 Thread GitBox


mattyb149 commented on pull request #4350:
URL: https://github.com/apache/nifi/pull/4350#issuecomment-655679853


   +1 LGTM, ran contrib-check and tested a few scenarios. @simonbence 
@turcsanyip Are your reviews complete? If so I can merge to main. Thanks for 
this improvement! It will also make it easier to do DB-specific things now that 
we have the Database Type property in there.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi-minifi-cpp] arpadboda commented on pull request #807: MINIFICPP-1228 - Resource ownership refactor + flowFile-owning processors.

2020-07-08 Thread GitBox


arpadboda commented on pull request #807:
URL: https://github.com/apache/nifi-minifi-cpp/pull/807#issuecomment-655668294


   @adebreceni 
   The is a CI failure, might be related:
   ```
   
/home/travis/build/apache/nifi-minifi-cpp/extensions/sftp/tests/FetchSFTPTests.cpp:274:
 FAILED:
 {Unknown expression after the reported line}
   due to unexpected exception with message:
 File Operation: No Content Claim existed for read
   ```
   
   It can be an issue in that given test or in the code of that processor your 
changes just revealed. 
   Please take a look!



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi-minifi-cpp] arpadboda edited a comment on pull request #807: MINIFICPP-1228 - Resource ownership refactor + flowFile-owning processors.

2020-07-08 Thread GitBox


arpadboda edited a comment on pull request #807:
URL: https://github.com/apache/nifi-minifi-cpp/pull/807#issuecomment-655668294


   @adebreceni 
   There is a CI failure, might be related:
   ```
   
/home/travis/build/apache/nifi-minifi-cpp/extensions/sftp/tests/FetchSFTPTests.cpp:274:
 FAILED:
 {Unknown expression after the reported line}
   due to unexpected exception with message:
 File Operation: No Content Claim existed for read
   ```
   
   It can be an issue in that given test or in the code of that processor your 
changes just revealed. 
   Please take a look!
   
   Ps.: this testcase is pretty stable otherwise. 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Created] (MINIFICPP-1283) improve terminology around variable registry

2020-07-08 Thread Tony Kurc (Jira)
Tony Kurc created MINIFICPP-1283:


 Summary: improve terminology around variable registry 
 Key: MINIFICPP-1283
 URL: https://issues.apache.org/jira/browse/MINIFICPP-1283
 Project: Apache NiFi MiNiFi C++
  Issue Type: Improvement
Affects Versions: 0.7.0
Reporter: Tony Kurc
Assignee: Tony Kurc


swap whitelist/blacklist with allow/deny list



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (NIFI-7615) Improve terminology in wali

2020-07-08 Thread Tony Kurc (Jira)
Tony Kurc created NIFI-7615:
---

 Summary: Improve terminology in wali 
 Key: NIFI-7615
 URL: https://issues.apache.org/jira/browse/NIFI-7615
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Core Framework
Affects Versions: 1.11.4
Reporter: Tony Kurc


Blacklist in wali should be renamed Blocklist



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [nifi-registry] alopresto opened a new pull request #285: NIFIREG-400 Upgraded dependency versions.

2020-07-08 Thread GitBox


alopresto opened a new pull request #285:
URL: https://github.com/apache/nifi-registry/pull/285


   Upgraded the jersey glassfish dependency version and ran a full build to 
verify. 



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Created] (NIFI-7614) Improve terminology around top-level key management for sensitive configuration values

2020-07-08 Thread Andy LoPresto (Jira)
Andy LoPresto created NIFI-7614:
---

 Summary: Improve terminology around top-level key management for 
sensitive configuration values
 Key: NIFI-7614
 URL: https://issues.apache.org/jira/browse/NIFI-7614
 Project: Apache NiFi
  Issue Type: Improvement
  Components: Configuration, Configuration Management, Core Framework
Affects Versions: 1.11.4
Reporter: Andy LoPresto
Assignee: Andy LoPresto


The top-level key and associated handling stored in {{bootstrap.conf}} and used 
to encrypt the sensitive properties in {{nifi.properties}}, etc. should be 
renamed from "master". 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Created] (NIFIREG-400) Upgrade dependencies

2020-07-08 Thread Andy LoPresto (Jira)
Andy LoPresto created NIFIREG-400:
-

 Summary: Upgrade dependencies
 Key: NIFIREG-400
 URL: https://issues.apache.org/jira/browse/NIFIREG-400
 Project: NiFi Registry
  Issue Type: Improvement
Affects Versions: 0.6.0
Reporter: Andy LoPresto
Assignee: Andy LoPresto


Some dependencies can be upgraded (jquery, jackson-databind). 



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [nifi] mattyb149 commented on pull request #4374: NIFI-7572: Added ScriptedTransformRecord processor. Addressed a coupl…

2020-07-08 Thread GitBox


mattyb149 commented on pull request #4374:
URL: https://github.com/apache/nifi/pull/4374#issuecomment-655583404


   +1 LGTM tested a few scenarios, @MikeThomsen are you good with me merging?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync

2020-07-08 Thread GitBox


fgerlits commented on a change in pull request #835:
URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r451599406



##
File path: extensions/http-curl/tests/HTTPIntegrationBase.h
##
@@ -135,6 +135,9 @@ class VerifyC2Describe : public VerifyC2Base {
   }
 
   void runAssertions() override {
+// This class is never used for running assertions, but we are forced to 
wait for DescribeManifestHandler to verifyJsonHasAgentManifest
+// if we were to log something on finished verification, we could poll on 
finding it 
+std::this_thread::sleep_for(std::chrono::milliseconds(wait_time_));

Review comment:
   Can the sleep go into `waitToVerifyProcessor()` and `runAssertion()` be 
empty in this case?  I think that would say in code what you say in the comment 
above.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync

2020-07-08 Thread GitBox


fgerlits commented on a change in pull request #835:
URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r451597749



##
File path: extensions/http-curl/tests/VerifyInvokeHTTPTest.cpp
##
@@ -93,47 +95,55 @@ class VerifyInvokeHTTP : public CoapIntegrationBase {
 flowController_->unload();
 flowController_->stopC2();
 
-runAssertions();
 cleanup();
   }
 };
 
 class VerifyInvokeHTTPOKResponse : public VerifyInvokeHTTP {
-public:
-  virtual void runAssertions() override {
-
assert(LogTestController::getInstance().contains("key:invokehttp.status.code 
value:201"));
-assert(LogTestController::getInstance().contains("response code 201"));
+ public:
+  void runAssertions() override {
+using org::apache::nifi::minifi::utils::verifyLogLinePresenceInPollTime;
+assert(verifyLogLinePresenceInPollTime(std::chrono::seconds(6),

Review comment:
   👍 





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Resolved] (MINIFICPP-1282) BUILD_BYPRODUCTS incorrectly set to lib64

2020-07-08 Thread Andrew Christianson (Jira)


 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Andrew Christianson resolved MINIFICPP-1282.

Resolution: Fixed

> BUILD_BYPRODUCTS incorrectly set to lib64
> -
>
> Key: MINIFICPP-1282
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1282
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Andrew Christianson
>Assignee: Andrew Christianson
>Priority: Major
>  Time Spent: 40m
>  Remaining Estimate: 0h
>
> On updated arch linux as of 2020-07-07, getting the following build error:
> {code:java}
> make[2]: *** No rule to make target 
> 'thirdparty/rocksdb-install/lib64/librocksdb.a', needed by 'main/minifi'.  
> Stop.
> {code}
> Looking at the directory, the library was built, but its in {{lib/}}:
> {noformat}
> $ ls thirdparty/rocksdb-install/lib64/
> ls: cannot access 'thirdparty/rocksdb-install/lib64/': No such file or 
> directory
> $ ls thirdparty/rocksdb-install/lib/
> cmake  librocksdb.a
> {noformat}



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync

2020-07-08 Thread GitBox


hunyadi-dev commented on a change in pull request #835:
URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r451561893



##
File path: extensions/http-curl/tests/VerifyInvokeHTTPTest.cpp
##
@@ -93,47 +95,55 @@ class VerifyInvokeHTTP : public CoapIntegrationBase {
 flowController_->unload();
 flowController_->stopC2();
 
-runAssertions();
 cleanup();
   }
 };
 
 class VerifyInvokeHTTPOKResponse : public VerifyInvokeHTTP {
-public:
-  virtual void runAssertions() override {
-
assert(LogTestController::getInstance().contains("key:invokehttp.status.code 
value:201"));
-assert(LogTestController::getInstance().contains("response code 201"));
+ public:
+  void runAssertions() override {
+using org::apache::nifi::minifi::utils::verifyLogLinePresenceInPollTime;
+assert(verifyLogLinePresenceInPollTime(std::chrono::seconds(6),

Review comment:
   This is a maximum value, and the test would probably run for far less 
time. `contains` was running for 3 seconds, but the `waitForVerifyProcessor` 
added an extra 6 seconds (line 35 of this file). I went with keeping the flat 6 
seconds, because this is seems like the time frame these logs should have been 
produced, and not the 3 seconds the log checking introduced.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi-minifi-cpp] achristianson commented on pull request #834: MINIFICPP-1282 Use GNUInstallDirs for more reliable determination of installed lib/lib64 directories

2020-07-08 Thread GitBox


achristianson commented on pull request #834:
URL: https://github.com/apache/nifi-minifi-cpp/pull/834#issuecomment-655534531


   Thanks for the review/merge.



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync

2020-07-08 Thread GitBox


hunyadi-dev commented on a change in pull request #835:
URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r451559061



##
File path: libminifi/test/0-9
##
@@ -0,0 +1 @@
+C API raNdOMcaSe test d4t4 th1s is!

Review comment:
   Oh, thanks for spotting it.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync

2020-07-08 Thread GitBox


hunyadi-dev commented on a change in pull request #835:
URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r451558272



##
File path: extensions/http-curl/tests/HTTPIntegrationBase.h
##
@@ -135,6 +135,9 @@ class VerifyC2Describe : public VerifyC2Base {
   }
 
   void runAssertions() override {
+// This class is never used for running assertions, but we are forced to 
wait for DescribeManifestHandler to verifyJsonHasAgentManifest
+// if we were to log something on finished verification, we could poll on 
finding it 
+std::this_thread::sleep_for(std::chrono::milliseconds(wait_time_));

Review comment:
   It is the same as before. Maybe I can rename `runAssertions` to 
something more appropriate, if you have suggestions.

##
File path: extensions/http-curl/tests/HTTPIntegrationBase.h
##
@@ -135,6 +135,9 @@ class VerifyC2Describe : public VerifyC2Base {
   }
 
   void runAssertions() override {
+// This class is never used for running assertions, but we are forced to 
wait for DescribeManifestHandler to verifyJsonHasAgentManifest
+// if we were to log something on finished verification, we could poll on 
finding it 
+std::this_thread::sleep_for(std::chrono::milliseconds(wait_time_));

Review comment:
   It is the same as before. Maybe I can rename `runAssertions` globally to 
something more appropriate, if you have suggestions.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi] markap14 commented on pull request #4393: NIFI-7606:Provide "NIFI_HOME()" function in Expression Language to ob…

2020-07-08 Thread GitBox


markap14 commented on pull request #4393:
URL: https://github.com/apache/nifi/pull/4393#issuecomment-655526041


   I don't think this function is necessary. It's simply exposing an 
environment variable, but all environment variables already are available via 
Expression Language. You can simply use `${NIFI_HOME}`



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #834: MINIFICPP-1282 Use GNUInstallDirs for more reliable determination of installed lib/lib64 directories

2020-07-08 Thread GitBox


arpadboda closed pull request #834:
URL: https://github.com/apache/nifi-minifi-cpp/pull/834


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (NIFI-7610) Edit README.md for dockerhub to reflect 1.12.0 release

2020-07-08 Thread Chad Zobrisky (Jira)


 [ 
https://issues.apache.org/jira/browse/NIFI-7610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Chad Zobrisky updated NIFI-7610:

Summary: Edit README.md for dockerhub to reflect 1.12.0 release  (was: Edit 
README.md for dockerhub to reflect 1.11.4 image)

> Edit README.md for dockerhub to reflect 1.12.0 release
> --
>
> Key: NIFI-7610
> URL: https://issues.apache.org/jira/browse/NIFI-7610
> Project: Apache NiFi
>  Issue Type: Improvement
>  Components: Documentation & Website
>Affects Versions: 1.11.4
>Reporter: Chad Zobrisky
>Priority: Minor
> Fix For: 1.12.0
>
>
> The readme.md for dockerhub still refers to 1.8.0. This Jira ticket is to 
> reflect the latest version in main, 1.12.0 as the latest dockerhub image.
>  
> Ref: 
> [https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/README.md]
> dockerhub: [https://hub.docker.com/r/apache/nifi/]



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [nifi] MikeThomsen commented on a change in pull request #4374: NIFI-7572: Added ScriptedTransformRecord processor. Addressed a coupl…

2020-07-08 Thread GitBox


MikeThomsen commented on a change in pull request #4374:
URL: https://github.com/apache/nifi/pull/4374#discussion_r451524437



##
File path: 
nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptedTransformRecord.java
##
@@ -0,0 +1,398 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.script;
+
+import org.apache.commons.io.IOUtils;
+import org.apache.nifi.annotation.behavior.EventDriven;
+import org.apache.nifi.annotation.behavior.InputRequirement;
+import org.apache.nifi.annotation.behavior.Restricted;
+import org.apache.nifi.annotation.behavior.Restriction;
+import org.apache.nifi.annotation.behavior.SideEffectFree;
+import org.apache.nifi.annotation.behavior.SupportsBatching;
+import org.apache.nifi.annotation.behavior.WritesAttribute;
+import org.apache.nifi.annotation.behavior.WritesAttributes;
+import org.apache.nifi.annotation.documentation.CapabilityDescription;
+import org.apache.nifi.annotation.documentation.SeeAlso;
+import org.apache.nifi.annotation.documentation.Tags;
+import org.apache.nifi.annotation.lifecycle.OnScheduled;
+import org.apache.nifi.components.PropertyDescriptor;
+import org.apache.nifi.components.PropertyDescriptor.Builder;
+import org.apache.nifi.components.RequiredPermission;
+import org.apache.nifi.components.ValidationContext;
+import org.apache.nifi.components.ValidationResult;
+import org.apache.nifi.flowfile.FlowFile;
+import org.apache.nifi.processor.AbstractProcessor;
+import org.apache.nifi.processor.ProcessContext;
+import org.apache.nifi.processor.ProcessSession;
+import org.apache.nifi.processor.Relationship;
+import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.schema.access.SchemaNotFoundException;
+import org.apache.nifi.script.ScriptingComponentHelper;
+import org.apache.nifi.script.ScriptingComponentUtils;
+import org.apache.nifi.search.SearchContext;
+import org.apache.nifi.search.SearchResult;
+import org.apache.nifi.search.Searchable;
+import org.apache.nifi.serialization.MalformedRecordException;
+import org.apache.nifi.serialization.RecordReader;
+import org.apache.nifi.serialization.RecordReaderFactory;
+import org.apache.nifi.serialization.RecordSetWriter;
+import org.apache.nifi.serialization.RecordSetWriterFactory;
+import org.apache.nifi.serialization.WriteResult;
+import org.apache.nifi.serialization.record.Record;
+import org.python.jsr223.PyScriptEngine;
+
+import javax.script.Bindings;
+import javax.script.CompiledScript;
+import javax.script.ScriptContext;
+import javax.script.ScriptEngine;
+import javax.script.ScriptException;
+import javax.script.SimpleBindings;
+import java.io.FileInputStream;
+import java.io.IOException;
+import java.nio.charset.Charset;
+import java.util.Arrays;
+import java.util.Collection;
+import java.util.Collections;
+import java.util.HashMap;
+import java.util.HashSet;
+import java.util.List;
+import java.util.Map;
+import java.util.Set;
+import java.util.concurrent.atomic.AtomicLong;
+
+@EventDriven
+@SupportsBatching
+@SideEffectFree
+@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED)
+@Tags({"record", "transform", "script", "groovy", "jython", "python", 
"update", "modify", "filter"})
+@Restricted(restrictions = {
+@Restriction(requiredPermission = RequiredPermission.EXECUTE_CODE,
+explanation = "Provides operator the ability to execute arbitrary code 
assuming all permissions that NiFi has.")
+})
+@WritesAttributes({
+@WritesAttribute(attribute = "mime.type", description = "Sets the 
mime.type attribute to the MIME Type specified by the Record Writer"),
+@WritesAttribute(attribute = "record.count", description = "The number of 
records in the FlowFile"),
+@WritesAttribute(attribute = "record.error.message", description = "This 
attribute provides on failure the error message encountered by the Reader or 
Writer.")
+})
+@CapabilityDescription("Provides the ability to evaluate a simple script 
against each record in an incoming FlowFile. The script may transform the 
record in some way, filter the record, or fork " +
+"additional records. See Processor's Additional D

[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync

2020-07-08 Thread GitBox


fgerlits commented on a change in pull request #835:
URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r451477045



##
File path: extensions/http-curl/tests/HTTPIntegrationBase.h
##
@@ -135,6 +135,9 @@ class VerifyC2Describe : public VerifyC2Base {
   }
 
   void runAssertions() override {
+// This class is never used for running assertions, but we are forced to 
wait for DescribeManifestHandler to verifyJsonHasAgentManifest
+// if we were to log something on finished verification, we could poll on 
finding it 
+std::this_thread::sleep_for(std::chrono::milliseconds(wait_time_));

Review comment:
   A `runAssertions()` function that doesn't run assertions looks weird (as 
you clearly felt, too, hence the comment).  It may be better to keep 
`waitToVerifyProcessor()` and `runAssertions()` separate, but make their 
default implementations no-op.

##
File path: libminifi/test/0-9
##
@@ -0,0 +1 @@
+C API raNdOMcaSe test d4t4 th1s is!

Review comment:
   garbage left over by a test run

##
File path: libminifi/include/utils/IntegrationTestUtils.h
##
@@ -0,0 +1,65 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#pragma once
+
+#include 
+#include 
+
+#include "../../../libminifi/test/TestBase.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace utils {
+
+template 
+bool verifyEventHappenedInPollTime(const std::chrono::duration& 
wait_duration, Fun&& check) {

Review comment:
   You could replace the `becomesTrueWithinTimeout()` calls in 
`libminifi/test/unit/MinifiConcurrentQueueTests.cpp` with calls to this 
function, or make `becomesTrueWithinTimeout()` call 
`verifyEventHappenedInPollTime()`, as they are almost identical.

##
File path: extensions/http-curl/tests/VerifyInvokeHTTPTest.cpp
##
@@ -93,47 +95,55 @@ class VerifyInvokeHTTP : public CoapIntegrationBase {
 flowController_->unload();
 flowController_->stopC2();
 
-runAssertions();
 cleanup();
   }
 };
 
 class VerifyInvokeHTTPOKResponse : public VerifyInvokeHTTP {
-public:
-  virtual void runAssertions() override {
-
assert(LogTestController::getInstance().contains("key:invokehttp.status.code 
value:201"));
-assert(LogTestController::getInstance().contains("response code 201"));
+ public:
+  void runAssertions() override {
+using org::apache::nifi::minifi::utils::verifyLogLinePresenceInPollTime;
+assert(verifyLogLinePresenceInPollTime(std::chrono::seconds(6),

Review comment:
   Why did the timeouts in this test increase to 6 seconds (from the 
default 3 seconds which they were before)?





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Created] (NIFI-7613) Unpacked unique-named native libraries should be in the work directory

2020-07-08 Thread Peter Radden (Jira)
Peter Radden created NIFI-7613:
--

 Summary: Unpacked unique-named native libraries should be in the 
work directory
 Key: NIFI-7613
 URL: https://issues.apache.org/jira/browse/NIFI-7613
 Project: Apache NiFi
  Issue Type: Wish
  Components: Core Framework
Affects Versions: 1.11.4
Reporter: Peter Radden


In AbstractNativeLibHandlingClassLoader, it copies the libraries to the system 
temp directory using Files.createTempFile.

These should instead be within the work directory.

(Allowing them to be managed/cleaned up by NiFi, rather than relying on the 
system to clean up - this is seen to fill up storage on test system)



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor

2020-07-08 Thread GitBox


hunyadi-dev commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r451473323



##
File path: extensions/standard-processors/processors/RetryFlowFile.cpp
##
@@ -0,0 +1,183 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "RetryFlowFile.h"
+
+#include "core/PropertyValidation.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property 
RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry 
Attribute")
+->withDescription(
+"The name of the attribute that contains the current retry count for 
the FlowFile."
+"WARNING: If the name matches an attribute already on the FlowFile 
that does not contain a numerical value, "
+"the processor will either overwrite that attribute with '1' or fail 
based on configuration.")
+->withDefaultValue("flowfile.retries")
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property 
RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum 
Retries")
+->withDescription("The maximum number of times a FlowFile can be retried 
before being passed to the 'retries_exceeded' relationship.")
+->withDefaultValue(3)
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property 
RetryFlowFile::PenalizeRetries(core::PropertyBuilder::createProperty("Penalize 
Retries")
+  ->withDescription("If set to 'true', this Processor will penalize input 
FlowFiles before passing them to the 'retry' relationship. This does not apply 
to the 'retries_exceeded' relationship.")
+  ->withDefaultValue(true)
+  ->build());
+
+core::Property 
RetryFlowFile::FailOnNonNumericalOverwrite(core::PropertyBuilder::createProperty("Fail
 on Non-numerical Overwrite")
+->withDescription("If the FlowFile already has the attribute defined in 
'Retry Attribute' that is *not* a number, fail the FlowFile instead of 
resetting that value to '1'")
+->withDefaultValue(false)
+->build());
+
+core::Property 
RetryFlowFile::ReuseMode(core::PropertyBuilder::createProperty("Reuse Mode")
+->withDescription(
+"Defines how the Processor behaves if the retry FlowFile has a 
different retry UUID than "
+"the instance that received the FlowFile. This generally means that 
the attribute was "
+"not reset after being successfully retried by a previous instance of 
this processor.")
+->withAllowableValues({FAIL_ON_REUSE, WARN_ON_REUSE, 
RESET_REUSE})
+->withDefaultValue(FAIL_ON_REUSE)
+->build());
+
+core::Relationship RetryFlowFile::Retry("retry",
+  "Input FlowFile has not exceeded the configured maximum retry count, pass 
this relationship back to the input Processor to create a limited feedback 
loop.");
+core::Relationship RetryFlowFile::RetriesExceeded("retries_exceeded",
+  "Input FlowFile has exceeded the configured maximum retry count, do not pass 
this relationship back to the input Processor to terminate the limited feedback 
loop.");
+core::Relationship RetryFlowFile::Failure("failure",
+"The processor is configured such that a non-numerical value on 'Retry 
Attribute' results in a failure instead of resetting "
+"that value to '1'. This will immediately terminate the limited feedback 
loop. Might also include when 'Maximum Retries' contains "
+" attribute expression language that does not resolve to an Integer.");
+
+void RetryFlowFile::initialize() {
+  setSupportedProperties({
+RetryAttribute,
+MaximumRetries,
+PenalizeRetries,
+FailOnNonNumericalOverwrite,
+ReuseMode,
+  });
+  setSupportedRelationships({
+Retry,
+RetriesExceeded,
+Failure,
+  });
+}
+
+void RetryFlowFile::onSchedule(core::ProcessContext* context, 
core::ProcessSessionFactory* /* sessionFactory */) {
+  context->getProperty(RetryAttribute.getName(), retry_attribute_);
+  context->getProperty(MaximumRetries.getName(), maximum_retries_);
+  context->getProperty(PenalizeRetries.getName(), penalize_retries_);
+  context->getProperty(FailOnNonNumericalOverwrite.getName(), 
fail_on_non_numerical_overwrite_);
+  context->getProperty(ReuseMode.getName(), reuse_

[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor

2020-07-08 Thread GitBox


hunyadi-dev commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r451465600



##
File path: extensions/standard-processors/processors/RetryFlowFile.cpp
##
@@ -0,0 +1,183 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "RetryFlowFile.h"
+
+#include "core/PropertyValidation.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property 
RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry 
Attribute")
+->withDescription(
+"The name of the attribute that contains the current retry count for 
the FlowFile."
+"WARNING: If the name matches an attribute already on the FlowFile 
that does not contain a numerical value, "
+"the processor will either overwrite that attribute with '1' or fail 
based on configuration.")
+->withDefaultValue("flowfile.retries")
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property 
RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum 
Retries")
+->withDescription("The maximum number of times a FlowFile can be retried 
before being passed to the 'retries_exceeded' relationship.")
+->withDefaultValue(3)
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property 
RetryFlowFile::PenalizeRetries(core::PropertyBuilder::createProperty("Penalize 
Retries")
+  ->withDescription("If set to 'true', this Processor will penalize input 
FlowFiles before passing them to the 'retry' relationship. This does not apply 
to the 'retries_exceeded' relationship.")
+  ->withDefaultValue(true)
+  ->build());
+
+core::Property 
RetryFlowFile::FailOnNonNumericalOverwrite(core::PropertyBuilder::createProperty("Fail
 on Non-numerical Overwrite")
+->withDescription("If the FlowFile already has the attribute defined in 
'Retry Attribute' that is *not* a number, fail the FlowFile instead of 
resetting that value to '1'")
+->withDefaultValue(false)
+->build());
+
+core::Property 
RetryFlowFile::ReuseMode(core::PropertyBuilder::createProperty("Reuse Mode")
+->withDescription(
+"Defines how the Processor behaves if the retry FlowFile has a 
different retry UUID than "
+"the instance that received the FlowFile. This generally means that 
the attribute was "
+"not reset after being successfully retried by a previous instance of 
this processor.")
+->withAllowableValues({FAIL_ON_REUSE, WARN_ON_REUSE, 
RESET_REUSE})
+->withDefaultValue(FAIL_ON_REUSE)
+->build());
+
+core::Relationship RetryFlowFile::Retry("retry",
+  "Input FlowFile has not exceeded the configured maximum retry count, pass 
this relationship back to the input Processor to create a limited feedback 
loop.");
+core::Relationship RetryFlowFile::RetriesExceeded("retries_exceeded",
+  "Input FlowFile has exceeded the configured maximum retry count, do not pass 
this relationship back to the input Processor to terminate the limited feedback 
loop.");
+core::Relationship RetryFlowFile::Failure("failure",
+"The processor is configured such that a non-numerical value on 'Retry 
Attribute' results in a failure instead of resetting "
+"that value to '1'. This will immediately terminate the limited feedback 
loop. Might also include when 'Maximum Retries' contains "
+" attribute expression language that does not resolve to an Integer.");
+
+void RetryFlowFile::initialize() {
+  setSupportedProperties({
+RetryAttribute,
+MaximumRetries,
+PenalizeRetries,
+FailOnNonNumericalOverwrite,
+ReuseMode,
+  });
+  setSupportedRelationships({
+Retry,
+RetriesExceeded,
+Failure,
+  });
+}
+
+void RetryFlowFile::onSchedule(core::ProcessContext* context, 
core::ProcessSessionFactory* /* sessionFactory */) {
+  context->getProperty(RetryAttribute.getName(), retry_attribute_);
+  context->getProperty(MaximumRetries.getName(), maximum_retries_);
+  context->getProperty(PenalizeRetries.getName(), penalize_retries_);
+  context->getProperty(FailOnNonNumericalOverwrite.getName(), 
fail_on_non_numerical_overwrite_);
+  context->getProperty(ReuseMode.getName(), reuse_

[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor

2020-07-08 Thread GitBox


hunyadi-dev commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r451462518



##
File path: extensions/standard-processors/processors/RetryFlowFile.cpp
##
@@ -0,0 +1,183 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "RetryFlowFile.h"
+
+#include "core/PropertyValidation.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property 
RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry 
Attribute")
+->withDescription(
+"The name of the attribute that contains the current retry count for 
the FlowFile."
+"WARNING: If the name matches an attribute already on the FlowFile 
that does not contain a numerical value, "
+"the processor will either overwrite that attribute with '1' or fail 
based on configuration.")
+->withDefaultValue("flowfile.retries")
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property 
RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum 
Retries")
+->withDescription("The maximum number of times a FlowFile can be retried 
before being passed to the 'retries_exceeded' relationship.")
+->withDefaultValue(3)
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property 
RetryFlowFile::PenalizeRetries(core::PropertyBuilder::createProperty("Penalize 
Retries")
+  ->withDescription("If set to 'true', this Processor will penalize input 
FlowFiles before passing them to the 'retry' relationship. This does not apply 
to the 'retries_exceeded' relationship.")
+  ->withDefaultValue(true)
+  ->build());
+
+core::Property 
RetryFlowFile::FailOnNonNumericalOverwrite(core::PropertyBuilder::createProperty("Fail
 on Non-numerical Overwrite")
+->withDescription("If the FlowFile already has the attribute defined in 
'Retry Attribute' that is *not* a number, fail the FlowFile instead of 
resetting that value to '1'")
+->withDefaultValue(false)
+->build());
+
+core::Property 
RetryFlowFile::ReuseMode(core::PropertyBuilder::createProperty("Reuse Mode")
+->withDescription(
+"Defines how the Processor behaves if the retry FlowFile has a 
different retry UUID than "
+"the instance that received the FlowFile. This generally means that 
the attribute was "
+"not reset after being successfully retried by a previous instance of 
this processor.")
+->withAllowableValues({FAIL_ON_REUSE, WARN_ON_REUSE, 
RESET_REUSE})
+->withDefaultValue(FAIL_ON_REUSE)
+->build());
+
+core::Relationship RetryFlowFile::Retry("retry",
+  "Input FlowFile has not exceeded the configured maximum retry count, pass 
this relationship back to the input Processor to create a limited feedback 
loop.");
+core::Relationship RetryFlowFile::RetriesExceeded("retries_exceeded",
+  "Input FlowFile has exceeded the configured maximum retry count, do not pass 
this relationship back to the input Processor to terminate the limited feedback 
loop.");
+core::Relationship RetryFlowFile::Failure("failure",
+"The processor is configured such that a non-numerical value on 'Retry 
Attribute' results in a failure instead of resetting "
+"that value to '1'. This will immediately terminate the limited feedback 
loop. Might also include when 'Maximum Retries' contains "
+" attribute expression language that does not resolve to an Integer.");
+
+void RetryFlowFile::initialize() {
+  setSupportedProperties({
+RetryAttribute,
+MaximumRetries,
+PenalizeRetries,
+FailOnNonNumericalOverwrite,
+ReuseMode,
+  });
+  setSupportedRelationships({
+Retry,
+RetriesExceeded,
+Failure,
+  });
+}
+
+void RetryFlowFile::onSchedule(core::ProcessContext* context, 
core::ProcessSessionFactory* /* sessionFactory */) {
+  context->getProperty(RetryAttribute.getName(), retry_attribute_);
+  context->getProperty(MaximumRetries.getName(), maximum_retries_);
+  context->getProperty(PenalizeRetries.getName(), penalize_retries_);
+  context->getProperty(FailOnNonNumericalOverwrite.getName(), 
fail_on_non_numerical_overwrite_);
+  context->getProperty(ReuseMode.getName(), reuse_

[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor

2020-07-08 Thread GitBox


hunyadi-dev commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r451460991



##
File path: extensions/standard-processors/processors/RetryFlowFile.cpp
##
@@ -0,0 +1,183 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "RetryFlowFile.h"
+
+#include "core/PropertyValidation.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property 
RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry 
Attribute")
+->withDescription(
+"The name of the attribute that contains the current retry count for 
the FlowFile."
+"WARNING: If the name matches an attribute already on the FlowFile 
that does not contain a numerical value, "
+"the processor will either overwrite that attribute with '1' or fail 
based on configuration.")
+->withDefaultValue("flowfile.retries")
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property 
RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum 
Retries")
+->withDescription("The maximum number of times a FlowFile can be retried 
before being passed to the 'retries_exceeded' relationship.")
+->withDefaultValue(3)
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property 
RetryFlowFile::PenalizeRetries(core::PropertyBuilder::createProperty("Penalize 
Retries")
+  ->withDescription("If set to 'true', this Processor will penalize input 
FlowFiles before passing them to the 'retry' relationship. This does not apply 
to the 'retries_exceeded' relationship.")
+  ->withDefaultValue(true)
+  ->build());
+
+core::Property 
RetryFlowFile::FailOnNonNumericalOverwrite(core::PropertyBuilder::createProperty("Fail
 on Non-numerical Overwrite")
+->withDescription("If the FlowFile already has the attribute defined in 
'Retry Attribute' that is *not* a number, fail the FlowFile instead of 
resetting that value to '1'")
+->withDefaultValue(false)
+->build());
+
+core::Property 
RetryFlowFile::ReuseMode(core::PropertyBuilder::createProperty("Reuse Mode")
+->withDescription(
+"Defines how the Processor behaves if the retry FlowFile has a 
different retry UUID than "
+"the instance that received the FlowFile. This generally means that 
the attribute was "
+"not reset after being successfully retried by a previous instance of 
this processor.")
+->withAllowableValues({FAIL_ON_REUSE, WARN_ON_REUSE, 
RESET_REUSE})
+->withDefaultValue(FAIL_ON_REUSE)

Review comment:
   [Same as 
above.](http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.RetryFlowFile/index.html)

##
File path: extensions/standard-processors/processors/RetryFlowFile.cpp
##
@@ -0,0 +1,183 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "RetryFlowFile.h"
+
+#include "core/PropertyValidation.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property 
RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry 
Attribute")
+->withDescription(
+"The name of the attribute that contains the current retry count for 
the FlowFile."
+"WARNING: If the name matches an attribute already on the FlowFile 
that does not contain a num

[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor

2020-07-08 Thread GitBox


hunyadi-dev commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r451460587



##
File path: extensions/standard-processors/processors/RetryFlowFile.cpp
##
@@ -0,0 +1,183 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "RetryFlowFile.h"
+
+#include "core/PropertyValidation.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property 
RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry 
Attribute")
+->withDescription(
+"The name of the attribute that contains the current retry count for 
the FlowFile."
+"WARNING: If the name matches an attribute already on the FlowFile 
that does not contain a numerical value, "
+"the processor will either overwrite that attribute with '1' or fail 
based on configuration.")
+->withDefaultValue("flowfile.retries")

Review comment:
   The NiFi implementation has this default value set:
   
http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.RetryFlowFile/index.html

##
File path: extensions/standard-processors/processors/RetryFlowFile.cpp
##
@@ -0,0 +1,183 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "RetryFlowFile.h"
+
+#include "core/PropertyValidation.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property 
RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry 
Attribute")
+->withDescription(
+"The name of the attribute that contains the current retry count for 
the FlowFile."
+"WARNING: If the name matches an attribute already on the FlowFile 
that does not contain a numerical value, "
+"the processor will either overwrite that attribute with '1' or fail 
based on configuration.")
+->withDefaultValue("flowfile.retries")
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property 
RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum 
Retries")
+->withDescription("The maximum number of times a FlowFile can be retried 
before being passed to the 'retries_exceeded' relationship.")
+->withDefaultValue(3)

Review comment:
   Same as above.





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi-minifi-cpp] hunyadi-dev opened a new pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync

2020-07-08 Thread GitBox


hunyadi-dev opened a new pull request #835:
URL: https://github.com/apache/nifi-minifi-cpp/pull/835


   **Decreases the avg. test execution time by 57% (from 326 seconds to 139 
seconds).**
   
   Measurements on test times of master (on MacBook Pro 16-inch, 2019):
   ```
   ➜ for repetition in {1..5}; do; time ninja test 2>&1 >> test_logs.txt; done;
   ninja test 2>&1 >> test_logs.txt  46.05s user 19.32s system 18% cpu 5:58.81 
total
   ninja test 2>&1 >> test_logs.txt  45.81s user 13.33s system 18% cpu 5:16.95 
total
   ninja test 2>&1 >> test_logs.txt  46.10s user 13.22s system 18% cpu 5:18.84 
total
   ninja test 2>&1 >> test_logs.txt  46.08s user 13.39s system 18% cpu 5:17.21 
total
   ninja test 2>&1 >> test_logs.txt  46.16s user 13.51s system 18% cpu 5:16.61 
total
   ➜ grep "Total Test time" test_logs.txt
   Total Test time (real) = 358.74 sec
   Total Test time (real) = 316.88 sec
   Total Test time (real) = 318.78 sec
   Total Test time (real) = 317.14 sec
   Total Test time (real) = 316.57 sec
   ```
   On the source branch of this PR:
   ```
   ➜ for repetition in {1..5}; do; time ninja test 2>&1 >> test_logs.txt; done;
   ninja test 2>&1 >> test_logs.txt  24.04s user 9.62s system 24% cpu 2:19.84 
total
   ninja test 2>&1 >> test_logs.txt  23.98s user 9.17s system 24% cpu 2:17.74 
total
   ninja test 2>&1 >> test_logs.txt  23.89s user 9.44s system 24% cpu 2:18.17 
total
   ninja test 2>&1 >> test_logs.txt  23.82s user 9.53s system 24% cpu 2:18.72 
total
   ninja test 2>&1 >> test_logs.txt  24.04s user 9.56s system 23% cpu 2:20.81 
total
   ➜ grep "Total Test time" test_logs.txt
   Total Test time (real) = 139.77 sec
   Total Test time (real) = 137.68 sec
   Total Test time (real) = 138.10 sec
   Total Test time (real) = 138.66 sec
   Total Test time (real) = 140.75 sec
   ```



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi-minifi-cpp] arpadboda commented on pull request #834: MINIFICPP-1282 Use GNUInstallDirs for more reliable determination of installed lib/lib64 directories

2020-07-08 Thread GitBox


arpadboda commented on pull request #834:
URL: https://github.com/apache/nifi-minifi-cpp/pull/834#issuecomment-655434518


   Looks good to me.
   @szaszm do you have any concerns?



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Created] (NIFI-7612) use processor (ComsumeKafka_2_0) , Error commit kafka offset

2020-07-08 Thread JosieMai (Jira)
JosieMai created NIFI-7612:
--

 Summary: use processor (ComsumeKafka_2_0) , Error commit kafka 
offset
 Key: NIFI-7612
 URL: https://issues.apache.org/jira/browse/NIFI-7612
 Project: Apache NiFi
  Issue Type: Epic
  Components: Extensions
Affects Versions: 1.10.0
 Environment: CentOS Linux release 7.7.1908 (Core)

Reporter: JosieMai


at java.lang.Thread.run(Thread.java:748) at 
java.lang.Thread.run(Thread.java:748)2020-07-08 14:45:33,977 ERROR 
[Timer-Driven Process Thread-113] o.a.n.p.s.a.AttributeRollingWindow 
AttributeRollingWindow[id=2d2734ab-0173-1000--8c93fb57] Ran into an 
error while processing 
StandardFlowFileRecord[uuid=edac77c1-b519-4ddd-8b40-bfd9c59c376a,claim=StandardContentClaim
 [resourceClaim=StandardResourceClaim[id=1594187029544-1920, container=default, 
section=896], offset=366216, 
length=669582],offset=392,name=a87943bc-b773-4690-8bfc-f0c7a7f9a786,size=406].: 
java.lang.NumberFormatException: For input string: 
"kafka.offset"java.lang.NumberFormatException: For input string: "kafka.offset"

...

Caused by: org.apache.calcite.runtime.CalciteContextException: From line 1, 
column 38 to line 1, column 49: Column 'kafka_offset' not found in any table



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[jira] [Resolved] (MINIFICPP-1253) Backpressure is applied to self-loops causing the processor to deadlock

2020-07-08 Thread Adam Debreceni (Jira)


 [ 
https://issues.apache.org/jira/browse/MINIFICPP-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Adam Debreceni resolved MINIFICPP-1253.
---
Resolution: Fixed

> Backpressure is applied to self-loops causing the processor to deadlock
> ---
>
> Key: MINIFICPP-1253
> URL: https://issues.apache.org/jira/browse/MINIFICPP-1253
> Project: Apache NiFi MiNiFi C++
>  Issue Type: Bug
>Reporter: Adam Debreceni
>Assignee: Adam Debreceni
>Priority: Major
>  Time Spent: 3h
>  Remaining Estimate: 0h
>
> When a self-loop gets full backpressure is applied to the source processor 
> which in this case also the destination processor, preventing the processor 
> from ever triggering.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)


[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #822: MINIFICPP-1253 - Do not hang on full cycles

2020-07-08 Thread GitBox


arpadboda closed pull request #822:
URL: https://github.com/apache/nifi-minifi-cpp/pull/822


   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #820: MINIFICPP-1183 - Cleanup C2 Update tests

2020-07-08 Thread GitBox


adamdebreceni commented on a change in pull request #820:
URL: https://github.com/apache/nifi-minifi-cpp/pull/820#discussion_r451404038



##
File path: extensions/http-curl/tests/HTTPHandlers.h
##
@@ -360,23 +359,19 @@ class DeleteTransactionResponder : public 
ServerAwareHandler {
   std::string response_code;
 };
 
-class HeartbeatHandler : public ServerAwareHandler {
- public:
-  explicit HeartbeatHandler(bool isSecure)
-  : isSecure(isSecure) {
-  }
-
-  std::string readPost(struct mg_connection *conn) {
-std::string response;
-int readBytes;
+std::string readPayload(struct mg_connection *conn) {
+  std::string response;
+  int readBytes;
 
-char buffer[1024];
-while ((readBytes = mg_read(conn, buffer, sizeof(buffer))) > 0) {
-  response.append(buffer, (readBytes / sizeof(char)));
-}
-return response;
+  char buffer[1024];
+  while ((readBytes = mg_read(conn, buffer, sizeof(buffer))) > 0) {
+response.append(buffer, (readBytes / sizeof(char)));

Review comment:
   I know it's old code but that division by `sizeof(char)` though 😏 





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Created] (NIFI-7611) NiFi fails to index provenance events

2020-07-08 Thread Michal W (Jira)
Michal W created NIFI-7611:
--

 Summary: NiFi fails to index provenance events
 Key: NIFI-7611
 URL: https://issues.apache.org/jira/browse/NIFI-7611
 Project: Apache NiFi
  Issue Type: Bug
Affects Versions: 1.11.4
 Environment: Microsoft Windows Server 2016 Standard - Intel Xeon Gold 
6140 CPU @ 2,30 GHz 8 processors, 32 GB RAM, total disk space 877 GB
Reporter: Michal W


Getting error "failed to index provenance events". Nifi.app log displays 
following information:

2020-07-08 09:00:00,406 ERROR [Index Provenance Events-4] 
o.a.n.p.index.lucene.EventIndexTask Failed to index Provenance Events

org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed

    at 
org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:681)

    at 
org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:695)

    at 
org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1281)

    at 
org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1257)

    at 
org.apache.nifi.provenance.lucene.LuceneEventIndexWriter.index(LuceneEventIndexWriter.java:70)

    at 
org.apache.nifi.provenance.index.lucene.EventIndexTask.index(EventIndexTask.java:202)

    at 
org.apache.nifi.provenance.index.lucene.EventIndexTask.run(EventIndexTask.java:113)

    at 
java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515)

    at 
java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264)

    at 
java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)

    at 
java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)

    at java.base/java.lang.Thread.run(Thread.java:834)

Caused by: java.nio.file.FileSystemException: 
E:\nifi-storage\provenance_repository\lucene-8-index-1593163985970\_11r.cfe: 
The process cannot access the file because it is being used by another process.

 

    at 
java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:92)

    at 
java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103)

    at 
java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108)

    at 
java.base/sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:120)

    at 
java.base/java.nio.channels.FileChannel.open(FileChannel.java:292)

    at 
java.base/java.nio.channels.FileChannel.open(FileChannel.java:345)

    at 
org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:238)

    at 
org.apache.lucene.store.Directory.openChecksumInput(Directory.java:157)

    at 
org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.readEntries(Lucene50CompoundReader.java:105)

    at 
org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.(Lucene50CompoundReader.java:69)

    at 
org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:70)

    at 
org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:100)

    at 
org.apache.lucene.index.SegmentReader.(SegmentReader.java:83)

    at 
org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:172)

    at 
org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:709)

    at 
org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4396)

    at 
org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4054)

    at 
org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:625)

    at 
org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:662)

 

Logs eventually grow over time and fill up the partition.

 

Configuration related to provenance repository:

 

# Provenance Repository Properties

nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository

nifi.provenance.repository.debug.frequency=1_000_000

nifi.provenance.repository.encryption.key.provider.implementation=

nifi.provenance.repository.encryption.key.provider.location=

nifi.provenance.repository.encryption.key.id=

nifi.provenance.repository.encryption.key=

 

# Persistent Provenance Repository Properties

nifi.provenance.repository.directory.default=E:\\nifi-storage\\provenance_repository

nifi.provenance.repository.directory.content1=F:\\nifi-storage\\provenance_repository

nifi.provenance.repository.max.storage.time=24 hours

# nifi.provenance.repository.max.storage.size

[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor

2020-07-08 Thread GitBox


adamdebreceni commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r451350379



##
File path: extensions/standard-processors/processors/RetryFlowFile.cpp
##
@@ -0,0 +1,183 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "RetryFlowFile.h"
+
+#include "core/PropertyValidation.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property 
RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry 
Attribute")
+->withDescription(
+"The name of the attribute that contains the current retry count for 
the FlowFile."
+"WARNING: If the name matches an attribute already on the FlowFile 
that does not contain a numerical value, "
+"the processor will either overwrite that attribute with '1' or fail 
based on configuration.")
+->withDefaultValue("flowfile.retries")

Review comment:
   it might be beneficial to make all new properties required by default, 
and if in the future it proves to be a pain point, we could always give a 
default based on real usage requirements, while if we would want to make 
something required later we would break previous flows





This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor

2020-07-08 Thread GitBox


adamdebreceni commented on a change in pull request #821:
URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r451340560



##
File path: extensions/standard-processors/processors/RetryFlowFile.cpp
##
@@ -0,0 +1,183 @@
+/**
+ *
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements.  See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License.  You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+#include "RetryFlowFile.h"
+
+#include "core/PropertyValidation.h"
+
+namespace org {
+namespace apache {
+namespace nifi {
+namespace minifi {
+namespace processors {
+
+core::Property 
RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry 
Attribute")
+->withDescription(
+"The name of the attribute that contains the current retry count for 
the FlowFile."
+"WARNING: If the name matches an attribute already on the FlowFile 
that does not contain a numerical value, "
+"the processor will either overwrite that attribute with '1' or fail 
based on configuration.")
+->withDefaultValue("flowfile.retries")
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property 
RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum 
Retries")
+->withDescription("The maximum number of times a FlowFile can be retried 
before being passed to the 'retries_exceeded' relationship.")
+->withDefaultValue(3)
+->supportsExpressionLanguage(true)
+->build());
+
+core::Property 
RetryFlowFile::PenalizeRetries(core::PropertyBuilder::createProperty("Penalize 
Retries")
+  ->withDescription("If set to 'true', this Processor will penalize input 
FlowFiles before passing them to the 'retry' relationship. This does not apply 
to the 'retries_exceeded' relationship.")
+  ->withDefaultValue(true)
+  ->build());
+
+core::Property 
RetryFlowFile::FailOnNonNumericalOverwrite(core::PropertyBuilder::createProperty("Fail
 on Non-numerical Overwrite")
+->withDescription("If the FlowFile already has the attribute defined in 
'Retry Attribute' that is *not* a number, fail the FlowFile instead of 
resetting that value to '1'")
+->withDefaultValue(false)
+->build());
+
+core::Property 
RetryFlowFile::ReuseMode(core::PropertyBuilder::createProperty("Reuse Mode")
+->withDescription(
+"Defines how the Processor behaves if the retry FlowFile has a 
different retry UUID than "
+"the instance that received the FlowFile. This generally means that 
the attribute was "
+"not reset after being successfully retried by a previous instance of 
this processor.")
+->withAllowableValues({FAIL_ON_REUSE, WARN_ON_REUSE, 
RESET_REUSE})
+->withDefaultValue(FAIL_ON_REUSE)
+->build());
+
+core::Relationship RetryFlowFile::Retry("retry",
+  "Input FlowFile has not exceeded the configured maximum retry count, pass 
this relationship back to the input Processor to create a limited feedback 
loop.");
+core::Relationship RetryFlowFile::RetriesExceeded("retries_exceeded",
+  "Input FlowFile has exceeded the configured maximum retry count, do not pass 
this relationship back to the input Processor to terminate the limited feedback 
loop.");
+core::Relationship RetryFlowFile::Failure("failure",
+"The processor is configured such that a non-numerical value on 'Retry 
Attribute' results in a failure instead of resetting "
+"that value to '1'. This will immediately terminate the limited feedback 
loop. Might also include when 'Maximum Retries' contains "
+" attribute expression language that does not resolve to an Integer.");
+
+void RetryFlowFile::initialize() {
+  setSupportedProperties({
+RetryAttribute,
+MaximumRetries,
+PenalizeRetries,
+FailOnNonNumericalOverwrite,
+ReuseMode,
+  });
+  setSupportedRelationships({
+Retry,
+RetriesExceeded,
+Failure,
+  });
+}
+
+void RetryFlowFile::onSchedule(core::ProcessContext* context, 
core::ProcessSessionFactory* /* sessionFactory */) {
+  context->getProperty(RetryAttribute.getName(), retry_attribute_);
+  context->getProperty(MaximumRetries.getName(), maximum_retries_);
+  context->getProperty(PenalizeRetries.getName(), penalize_retries_);
+  context->getProperty(FailOnNonNumericalOverwrite.getName(), 
fail_on_non_numerical_overwrite_);
+  context->getProperty(ReuseMode.getName(), reus

[GitHub] [nifi] KuKuDeCheng opened a new pull request #4393: NIFI-7606:Provide "NIFI_HOME()" function in Expression Language to ob…

2020-07-08 Thread GitBox


KuKuDeCheng opened a new pull request #4393:
URL: https://github.com/apache/nifi/pull/4393


   NIFI-7606:Provide "NIFI_HOME()" function in Expression Language to obtain 
the installation directory of NIFI
   
   Thank you for submitting a contribution to Apache NiFi.
   
   Please provide a short description of the PR here:
   
    Description of PR
   
   Usually, I make a file '$NIFI_HOME/jdbc' to hold some jars of drivers. I 
find it's useful to provide a function to get the installation directory of 
NIFI, and we will use this such as `${NIFI_HOME():append('/jdbc/ojdbc8.jar')}` 
in `DbcpConnectionPoll` property. I think it will be useful for other scenes 
need to indicate file directory.
   
   In order to streamline the review of the contribution we ask you
   to ensure the following steps have been taken:
   
   ### For all changes:
   - [ ] Is there a JIRA ticket associated with this PR? Is it referenced 
in the commit message?
   
   - [ ] Does your PR title start with **NIFI-** where  is the JIRA 
number you are trying to resolve? Pay particular attention to the hyphen "-" 
character.
   
   - [ ] Has your PR been rebased against the latest commit within the target 
branch (typically `master`)?
   
   - [ ] Is your initial contribution a single, squashed commit? _Additional 
commits in response to PR reviewer feedback should be made on this branch and 
pushed to allow change tracking. Do not `squash` or use `--force` when pushing 
to allow for clean monitoring of changes._
   
   ### For code changes:
   - [ ] Have you ensured that the full suite of tests is executed via `mvn 
-Pcontrib-check clean install` at the root `nifi` folder?
   - [ ] Have you written or updated unit tests to verify your changes?
   - [√ ] Have you verified that the full build is successful on JDK 8?
   - [ ] Have you verified that the full build is successful on JDK 11?
   - [ ] If adding new dependencies to the code, are these dependencies 
licensed in a way that is compatible for inclusion under [ASF 
2.0](http://www.apache.org/legal/resolved.html#category-a)? 
   - [ ] If applicable, have you updated the `LICENSE` file, including the main 
`LICENSE` file under `nifi-assembly`?
   - [ ] If applicable, have you updated the `NOTICE` file, including the main 
`NOTICE` file found under `nifi-assembly`?
   - [ ] If adding new Properties, have you added `.displayName` in addition to 
.name (programmatic access) for each of the new properties?
   
   ### For documentation related changes:
   - [ ] Have you ensured that format looks appropriate for the output in which 
it is rendered?
   
   ### Note:
   Please ensure that once the PR is submitted, you check GitHub Actions CI for 
build issues and submit an update to your PR as soon as possible.
   



This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org




[jira] [Updated] (NIFI-7606) Provide "NIFI_HOME()" function in Expression Language

2020-07-08 Thread ZhangCheng (Jira)


 [ 
https://issues.apache.org/jira/browse/NIFI-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

ZhangCheng updated NIFI-7606:
-
Description: Usually,  I make a file '$NIFI_HOME/jdbc' to hold some jars of 
driivers. I find it's useful to provide a function to get the path NIFI 
installed, and we will use this such as 
`$(NIFI_HOME):append('/jdbc/ojdbc8.jar'))` in `DbcpConnectionPoll` property. I 
think it will be useful for other scenes need to indicate file path.  (was: 
Usually,  I make a file '$NIFI_HOME/jdbc' to hold some jars of driivers. I find 
it's useful to provide a function to get the path NIFI installed, and we will 
use this such as `$(NIFI_HOME().append('/jdbc/ojdbc8.jar'))` in 
`DbcpConnectionPoll` property. I think it will be useful for other scenes need 
to indicate file path.)

> Provide "NIFI_HOME()" function in Expression Language
> -
>
> Key: NIFI-7606
> URL: https://issues.apache.org/jira/browse/NIFI-7606
> Project: Apache NiFi
>  Issue Type: New Feature
>  Components: Extensions
>Reporter: ZhangCheng
>Assignee: ZhangCheng
>Priority: Major
>
> Usually,  I make a file '$NIFI_HOME/jdbc' to hold some jars of driivers. I 
> find it's useful to provide a function to get the path NIFI installed, and we 
> will use this such as `$(NIFI_HOME):append('/jdbc/ojdbc8.jar'))` in 
> `DbcpConnectionPoll` property. I think it will be useful for other scenes 
> need to indicate file path.



--
This message was sent by Atlassian Jira
(v8.3.4#803005)