[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync
hunyadi-dev commented on a change in pull request #835: URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r452001585 ## File path: extensions/http-curl/tests/HTTPIntegrationBase.h ## @@ -135,6 +135,9 @@ class VerifyC2Describe : public VerifyC2Base { } void runAssertions() override { +// This class is never used for running assertions, but we are forced to wait for DescribeManifestHandler to verifyJsonHasAgentManifest +// if we were to log something on finished verification, we could poll on finding it +std::this_thread::sleep_for(std::chrono::milliseconds(wait_time_)); Review comment: We no longer have `waitToVerifyProcessor()` though... This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-7622) updateParameterContext NullPointerException on StandardProcessGroup.java:4220
[ https://issues.apache.org/jira/browse/NIFI-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Secules updated NIFI-7622: --- Description: I am getting this null pointer exception when importing one versioned flow from a registry which contains another versioned flow and each uses different parameter contexts. I am using version 1.11.4 of NiFi This error is blocking me from developing flows which comprise common components and project-specific components that use their own parameter contexts. Attached a screen cap of the state of the stack at StandardProcessGroup.java:4220 {code:java} 2020-07-09 05:33:10,414 ERROR [NiFi Web Server-86] o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: java.lang.NullPointerException. Returning Internal Server Error response. java.lang.NullPointerException: null at org.apache.nifi.groups.StandardProcessGroup.updateParameterContext(StandardProcessGroup.java:4220) at org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3754) at org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324) at org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876) at org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324) at org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876) at org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324) at org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876) at org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3653) at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:408) at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84) at org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:616) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671) at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$202f172f.updateProcessGroupFlow() at org.apache.nifi.web.StandardNiFiServiceFacade$14.update(StandardNiFiServiceFacade.java:5044) at org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:117) at org.apache.nifi.web.StandardNiFiServiceFacade.updateProcessGroupContents(StandardNiFiServiceFacade.java:5040) at org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84) at org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithWriteLock(NiFiServiceFacadeLock.java:179) at org.apache.nifi.web.NiFiServiceFacadeLock.updateLock(NiFiServ
[jira] [Updated] (NIFI-7622) updateParameterContext NullPointerException on StandardProcessGroup.java:4220
[ https://issues.apache.org/jira/browse/NIFI-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Secules updated NIFI-7622: --- Description: I am getting this null pointer exception when importing one versioned flow which contains another versioned flow and each uses different parameter contexts. I am using version 1.11.4 of NiFi This error is blocking me from developing flows which comprise common components and project-specific components that use their own parameter contexts. Attached a screen cap of the state of the stack at StandardProcessGroup.java:4220 {code:java} 2020-07-09 05:33:10,414 ERROR [NiFi Web Server-86] o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: java.lang.NullPointerException. Returning Internal Server Error response. java.lang.NullPointerException: null at org.apache.nifi.groups.StandardProcessGroup.updateParameterContext(StandardProcessGroup.java:4220) at org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3754) at org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324) at org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876) at org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324) at org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876) at org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324) at org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876) at org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3653) at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:408) at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84) at org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:616) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671) at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$202f172f.updateProcessGroupFlow() at org.apache.nifi.web.StandardNiFiServiceFacade$14.update(StandardNiFiServiceFacade.java:5044) at org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:117) at org.apache.nifi.web.StandardNiFiServiceFacade.updateProcessGroupContents(StandardNiFiServiceFacade.java:5040) at org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84) at org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithWriteLock(NiFiServiceFacadeLock.java:179) at org.apache.nifi.web.NiFiServiceFacadeLock.updateLock(NiFiServiceFacadeLock.ja
[jira] [Created] (NIFI-7623) Take unnecessary behavior out when using registry to update the flow
Hsin-Ying Lee created NIFI-7623: --- Summary: Take unnecessary behavior out when using registry to update the flow Key: NIFI-7623 URL: https://issues.apache.org/jira/browse/NIFI-7623 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Flow Versioning Reporter: Hsin-Ying Lee We tried to use the registry to change version/revert for a versioned PG, the changes only were processor position, color style, and connection bend points changes. When the processor was doing the long time tasks, the version control would not complete until the processors stopped, even that is not necessary to stop. If we tried to cancel the request, we also have to wait for the processors to stop and go back to start it manually. I'm trying to fix this issue, and make a PR for this. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7622) updateParameterContext NullPointerException on StandardProcessGroup.java:4220
[ https://issues.apache.org/jira/browse/NIFI-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Secules updated NIFI-7622: --- Description: I am getting this null pointer exception when importing one versioned flow which contains another versioned flow and each uses different parameter contexts. This error is blocking me from developing flows which comprise common components and project-specific components that use their own parameter contexts. Attached a screen cap of the state of the stack at StandardProcessGroup.java:4220 {code:java} 2020-07-09 05:33:10,414 ERROR [NiFi Web Server-86] o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: java.lang.NullPointerException. Returning Internal Server Error response. java.lang.NullPointerException: null at org.apache.nifi.groups.StandardProcessGroup.updateParameterContext(StandardProcessGroup.java:4220) at org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3754) at org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324) at org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876) at org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324) at org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876) at org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324) at org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876) at org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3653) at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:408) at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84) at org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:616) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671) at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$202f172f.updateProcessGroupFlow() at org.apache.nifi.web.StandardNiFiServiceFacade$14.update(StandardNiFiServiceFacade.java:5044) at org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:117) at org.apache.nifi.web.StandardNiFiServiceFacade.updateProcessGroupContents(StandardNiFiServiceFacade.java:5040) at org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84) at org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithWriteLock(NiFiServiceFacadeLock.java:179) at org.apache.nifi.web.NiFiServiceFacadeLock.updateLock(NiFiServiceFacadeLock.java:66) at sun.reflect.Gener
[jira] [Updated] (NIFI-7622) updateParameterContext NullPointerException on StandardProcessGroup.java:4220
[ https://issues.apache.org/jira/browse/NIFI-7622?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Eric Secules updated NIFI-7622: --- Attachment: Screen Shot 2020-07-08 at 11.04.37 PM.png > updateParameterContext NullPointerException on StandardProcessGroup.java:4220 > - > > Key: NIFI-7622 > URL: https://issues.apache.org/jira/browse/NIFI-7622 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.11.4 >Reporter: Eric Secules >Priority: Major > Attachments: Screen Shot 2020-07-08 at 11.04.37 PM.png > > > I am getting this null pointer exception when importing one versioned flow > which contains another versioned flow and each uses different parameter > contexts. > This error is blocking me from developing flows which comprise common > components and project-specific components that use their own parameter > contexts. > {code:java} > 2020-07-09 05:33:10,414 ERROR [NiFi Web Server-86] > o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: > java.lang.NullPointerException. Returning Internal Server Error response. > java.lang.NullPointerException: null > at > org.apache.nifi.groups.StandardProcessGroup.updateParameterContext(StandardProcessGroup.java:4220) > at > org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3754) > at > org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324) > at > org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876) > at > org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324) > at > org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876) > at > org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324) > at > org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876) > at > org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3653) > at > org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:408) > at > org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke() > at > org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > at > org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) > at > org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84) > at > org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:498) > at > org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627) > at > org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:616) > at > org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) > at > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) > at > org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671) > at > org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$202f172f.updateProcessGroupFlow() > at > org.apache.nifi.web.StandardNiFiServiceFacade$14.update(StandardNiFiServiceFacade.java:5044) > at > org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:117) > at > org.apache.nifi.web.StandardNiFiServiceFacade.updateProcessGroupContents(StandardNiFiServiceFacade.java:5040) > at > org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke() > at > org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) >
[jira] [Created] (NIFI-7622) updateParameterContext NullPointerException on StandardProcessGroup.java:4220
Eric Secules created NIFI-7622: -- Summary: updateParameterContext NullPointerException on StandardProcessGroup.java:4220 Key: NIFI-7622 URL: https://issues.apache.org/jira/browse/NIFI-7622 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.11.4 Reporter: Eric Secules I am getting this null pointer exception when importing one versioned flow which contains another versioned flow and each uses different parameter contexts. This error is blocking me from developing flows which comprise common components and project-specific components that use their own parameter contexts. {code:java} 2020-07-09 05:33:10,414 ERROR [NiFi Web Server-86] o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: java.lang.NullPointerException. Returning Internal Server Error response. java.lang.NullPointerException: null at org.apache.nifi.groups.StandardProcessGroup.updateParameterContext(StandardProcessGroup.java:4220) at org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3754) at org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324) at org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876) at org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324) at org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876) at org.apache.nifi.groups.StandardProcessGroup.addProcessGroup(StandardProcessGroup.java:4324) at org.apache.nifi.groups.StandardProcessGroup.updateProcessGroup(StandardProcessGroup.java:3876) at org.apache.nifi.groups.StandardProcessGroup.updateFlow(StandardProcessGroup.java:3653) at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO.updateProcessGroupFlow(StandardProcessGroupDAO.java:408) at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$FastClassBySpringCGLIB$$10a99b47.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84) at org.apache.nifi.audit.ProcessGroupAuditor.updateProcessGroupFlowAdvice(ProcessGroupAuditor.java:313) at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62) at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) at java.lang.reflect.Method.invoke(Method.java:498) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:627) at org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:616) at org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:70) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) at org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:671) at org.apache.nifi.web.dao.impl.StandardProcessGroupDAO$$EnhancerBySpringCGLIB$$202f172f.updateProcessGroupFlow() at org.apache.nifi.web.StandardNiFiServiceFacade$14.update(StandardNiFiServiceFacade.java:5044) at org.apache.nifi.web.revision.NaiveRevisionManager.updateRevision(NaiveRevisionManager.java:117) at org.apache.nifi.web.StandardNiFiServiceFacade.updateProcessGroupContents(StandardNiFiServiceFacade.java:5040) at org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke() at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) at org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:736) at org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) at org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:84) at org.apache.nifi.web.NiFiServiceFacadeLock.proceedWithWriteLock(NiFiServiceFacadeLock.java:179) at org.apach
[GitHub] [nifi] alopresto opened a new pull request #4396: NIFI-7621 & NIFI-7614 Updated terminology throughout codebase and doc…
alopresto opened a new pull request #4396: URL: https://github.com/apache/nifi/pull/4396 …umentation. Signed-off-by: Andy LoPresto Thank you for submitting a contribution to Apache NiFi. Please provide a short description of the PR here: Description of PR _Changes legacy terminology around keys, clustering, and integrations where new terms and transition for users are available._ In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically `master`)? - [x] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [x] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] Have you verified that the full build is successful on JDK 8? - [x] Have you verified that the full build is successful on JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Closed] (NIFI-7606) Provide "NIFI_HOME()" function in Expression Language
[ https://issues.apache.org/jira/browse/NIFI-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhangCheng closed NIFI-7606. Abandoned > Provide "NIFI_HOME()" function in Expression Language > - > > Key: NIFI-7606 > URL: https://issues.apache.org/jira/browse/NIFI-7606 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: ZhangCheng >Assignee: ZhangCheng >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > Usually, I make a file '$NIFI_HOME/jdbc' to hold some jars of driivers. I > find it's useful to provide a function to get the path NIFI installed, and we > will use this such as `$(NIFI_HOME):append('/jdbc/ojdbc8.jar'))` in > `DbcpConnectionPoll` property. I think it will be useful for other scenes > need to indicate file path. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-7606) Provide "NIFI_HOME()" function in Expression Language
[ https://issues.apache.org/jira/browse/NIFI-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhangCheng resolved NIFI-7606. -- Resolution: Abandoned It's not necessary. ${NIFI_HOME} is useful > Provide "NIFI_HOME()" function in Expression Language > - > > Key: NIFI-7606 > URL: https://issues.apache.org/jira/browse/NIFI-7606 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: ZhangCheng >Assignee: ZhangCheng >Priority: Major > Time Spent: 50m > Remaining Estimate: 0h > > Usually, I make a file '$NIFI_HOME/jdbc' to hold some jars of driivers. I > find it's useful to provide a function to get the path NIFI installed, and we > will use this such as `$(NIFI_HOME):append('/jdbc/ojdbc8.jar'))` in > `DbcpConnectionPoll` property. I think it will be useful for other scenes > need to indicate file path. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] KuKuDeCheng commented on pull request #4393: NIFI-7606:Provide "NIFI_HOME()" function in Expression Language to ob…
KuKuDeCheng commented on pull request #4393: URL: https://github.com/apache/nifi/pull/4393#issuecomment-655846568 > Thanks, but we can't accept contributions like this without test coverage. Once you add that we can proceed with a review. @MikeThomsen I find that it's not necessary. Thank you very much for review and I will close this PR This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] KuKuDeCheng closed pull request #4393: NIFI-7606:Provide "NIFI_HOME()" function in Expression Language to ob…
KuKuDeCheng closed pull request #4393: URL: https://github.com/apache/nifi/pull/4393 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] KuKuDeCheng commented on pull request #4393: NIFI-7606:Provide "NIFI_HOME()" function in Expression Language to ob…
KuKuDeCheng commented on pull request #4393: URL: https://github.com/apache/nifi/pull/4393#issuecomment-655845996 > I don't think this function is necessary. It's simply exposing an environment variable, but all environment variables already are available via Expression Language. You can simply use `${NIFI_HOME}` @markap14 It's useful!Thank you very much. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-7610) Edit README.md for dockerhub to reflect 1.12.0 release
[ https://issues.apache.org/jira/browse/NIFI-7610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chad Zobrisky updated NIFI-7610: Status: Patch Available (was: Open) > Edit README.md for dockerhub to reflect 1.12.0 release > -- > > Key: NIFI-7610 > URL: https://issues.apache.org/jira/browse/NIFI-7610 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Affects Versions: 1.11.4 >Reporter: Chad Zobrisky >Priority: Minor > Fix For: 1.12.0 > > Time Spent: 20m > Remaining Estimate: 0h > > The readme.md for dockerhub still refers to 1.8.0. This Jira ticket is to > reflect the latest version in main, 1.12.0 as the latest dockerhub image. > > Ref: > [https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/README.md] > dockerhub: [https://hub.docker.com/r/apache/nifi/] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7621) Remove legacy terminology around branch names from GitHub messaging and developer documentation
Andy LoPresto created NIFI-7621: --- Summary: Remove legacy terminology around branch names from GitHub messaging and developer documentation Key: NIFI-7621 URL: https://issues.apache.org/jira/browse/NIFI-7621 Project: Apache NiFi Issue Type: Task Components: Documentation & Website, Tools and Build Affects Versions: 1.11.4 Reporter: Andy LoPresto Assignee: Andy LoPresto Change references to {{master}} branch to reflect new name {{main}}. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-7619) Resolve failing unit test due to timeout in CI/CD environment
[ https://issues.apache.org/jira/browse/NIFI-7619?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto resolved NIFI-7619. - Fix Version/s: 1.12.0 Resolution: Duplicate Jira network instability caused creation of duplicate tickets. > Resolve failing unit test due to timeout in CI/CD environment > - > > Key: NIFI-7619 > URL: https://issues.apache.org/jira/browse/NIFI-7619 > Project: Apache NiFi > Issue Type: Bug > Components: Tools and Build >Affects Versions: 1.11.4 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Major > Labels: ci-failure, site-to-site, unit-test > Fix For: 1.12.0 > > > There is a test failure on MacOS JP 1.8 for GitHub Actions: > {code} > [ERROR] Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: > 42.397 s <<< FAILURE! - in org.apache.nifi.remote.client.http.TestHttpClient > [ERROR] > testReceiveTimeoutAfterDataExchange(org.apache.nifi.remote.client.http.TestHttpClient) > Time elapsed: 21.275 s <<< FAILURE! > java.lang.AssertionError > at > org.apache.nifi.remote.client.http.TestHttpClient.testReceiveTimeoutAfterDataExchange(TestHttpClient.java:1384) > [WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: > 0.454 s - in org.apache.nifi.remote.client.socket.TestSiteToSiteClient > [ERROR] Failures: > [ERROR] TestHttpClient.testReceiveTimeoutAfterDataExchange:1384 > [ERROR] Tests run: 92, Failures: 1, Errors: 0, Skipped: 2 > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-7618) Resolve failing unit test due to timeout in CI/CD environment
[ https://issues.apache.org/jira/browse/NIFI-7618?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto resolved NIFI-7618. - Fix Version/s: 1.12.0 Resolution: Duplicate Jira network instability caused creation of duplicate tickets. > Resolve failing unit test due to timeout in CI/CD environment > - > > Key: NIFI-7618 > URL: https://issues.apache.org/jira/browse/NIFI-7618 > Project: Apache NiFi > Issue Type: Bug > Components: Tools and Build >Affects Versions: 1.11.4 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Major > Labels: ci-failure, site-to-site, unit-test > Fix For: 1.12.0 > > > There is a test failure on MacOS JP 1.8 for GitHub Actions: > {code} > [ERROR] Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: > 42.397 s <<< FAILURE! - in org.apache.nifi.remote.client.http.TestHttpClient > [ERROR] > testReceiveTimeoutAfterDataExchange(org.apache.nifi.remote.client.http.TestHttpClient) > Time elapsed: 21.275 s <<< FAILURE! > java.lang.AssertionError > at > org.apache.nifi.remote.client.http.TestHttpClient.testReceiveTimeoutAfterDataExchange(TestHttpClient.java:1384) > [WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: > 0.454 s - in org.apache.nifi.remote.client.socket.TestSiteToSiteClient > [ERROR] Failures: > [ERROR] TestHttpClient.testReceiveTimeoutAfterDataExchange:1384 > [ERROR] Tests run: 92, Failures: 1, Errors: 0, Skipped: 2 > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (NIFI-7617) Resolve failing unit test due to timeout in CI/CD environment
[ https://issues.apache.org/jira/browse/NIFI-7617?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andy LoPresto resolved NIFI-7617. - Fix Version/s: 1.12.0 Resolution: Duplicate Jira network instability caused creation of duplicate tickets. > Resolve failing unit test due to timeout in CI/CD environment > - > > Key: NIFI-7617 > URL: https://issues.apache.org/jira/browse/NIFI-7617 > Project: Apache NiFi > Issue Type: Bug > Components: Tools and Build >Affects Versions: 1.11.4 >Reporter: Andy LoPresto >Assignee: Andy LoPresto >Priority: Major > Labels: ci-failure, site-to-site, unit-test > Fix For: 1.12.0 > > > There is a test failure on MacOS JP 1.8 for GitHub Actions: > {code} > [ERROR] Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: > 42.397 s <<< FAILURE! - in org.apache.nifi.remote.client.http.TestHttpClient > [ERROR] > testReceiveTimeoutAfterDataExchange(org.apache.nifi.remote.client.http.TestHttpClient) > Time elapsed: 21.275 s <<< FAILURE! > java.lang.AssertionError > at > org.apache.nifi.remote.client.http.TestHttpClient.testReceiveTimeoutAfterDataExchange(TestHttpClient.java:1384) > [WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: > 0.454 s - in org.apache.nifi.remote.client.socket.TestSiteToSiteClient > [ERROR] Failures: > [ERROR] TestHttpClient.testReceiveTimeoutAfterDataExchange:1384 > [ERROR] Tests run: 92, Failures: 1, Errors: 0, Skipped: 2 > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] alopresto opened a new pull request #4395: NIFI-7620 Fixed failing unit test due to timeout.
alopresto opened a new pull request #4395: URL: https://github.com/apache/nifi/pull/4395 Thank you for submitting a contribution to Apache NiFi. Please provide a short description of the PR here: Description of PR _Fixes a failing unit test in one of the CI/CD builds._ In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically `master`)? - [x] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [x] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] Have you verified that the full build is successful on JDK 8? - [ ] Have you verified that the full build is successful on JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-7620) Resolve failing unit test due to timeout in CI/CD environment
Andy LoPresto created NIFI-7620: --- Summary: Resolve failing unit test due to timeout in CI/CD environment Key: NIFI-7620 URL: https://issues.apache.org/jira/browse/NIFI-7620 Project: Apache NiFi Issue Type: Bug Components: Tools and Build Affects Versions: 1.11.4 Reporter: Andy LoPresto Assignee: Andy LoPresto There is a test failure on MacOS JP 1.8 for GitHub Actions: {code} [ERROR] Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 42.397 s <<< FAILURE! - in org.apache.nifi.remote.client.http.TestHttpClient [ERROR] testReceiveTimeoutAfterDataExchange(org.apache.nifi.remote.client.http.TestHttpClient) Time elapsed: 21.275 s <<< FAILURE! java.lang.AssertionError at org.apache.nifi.remote.client.http.TestHttpClient.testReceiveTimeoutAfterDataExchange(TestHttpClient.java:1384) [WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0.454 s - in org.apache.nifi.remote.client.socket.TestSiteToSiteClient [ERROR] Failures: [ERROR] TestHttpClient.testReceiveTimeoutAfterDataExchange:1384 [ERROR] Tests run: 92, Failures: 1, Errors: 0, Skipped: 2 {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7619) Resolve failing unit test due to timeout in CI/CD environment
Andy LoPresto created NIFI-7619: --- Summary: Resolve failing unit test due to timeout in CI/CD environment Key: NIFI-7619 URL: https://issues.apache.org/jira/browse/NIFI-7619 Project: Apache NiFi Issue Type: Bug Components: Tools and Build Affects Versions: 1.11.4 Reporter: Andy LoPresto Assignee: Andy LoPresto There is a test failure on MacOS JP 1.8 for GitHub Actions: {code} [ERROR] Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 42.397 s <<< FAILURE! - in org.apache.nifi.remote.client.http.TestHttpClient [ERROR] testReceiveTimeoutAfterDataExchange(org.apache.nifi.remote.client.http.TestHttpClient) Time elapsed: 21.275 s <<< FAILURE! java.lang.AssertionError at org.apache.nifi.remote.client.http.TestHttpClient.testReceiveTimeoutAfterDataExchange(TestHttpClient.java:1384) [WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0.454 s - in org.apache.nifi.remote.client.socket.TestSiteToSiteClient [ERROR] Failures: [ERROR] TestHttpClient.testReceiveTimeoutAfterDataExchange:1384 [ERROR] Tests run: 92, Failures: 1, Errors: 0, Skipped: 2 {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7618) Resolve failing unit test due to timeout in CI/CD environment
Andy LoPresto created NIFI-7618: --- Summary: Resolve failing unit test due to timeout in CI/CD environment Key: NIFI-7618 URL: https://issues.apache.org/jira/browse/NIFI-7618 Project: Apache NiFi Issue Type: Bug Components: Tools and Build Affects Versions: 1.11.4 Reporter: Andy LoPresto Assignee: Andy LoPresto There is a test failure on MacOS JP 1.8 for GitHub Actions: {code} [ERROR] Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 42.397 s <<< FAILURE! - in org.apache.nifi.remote.client.http.TestHttpClient [ERROR] testReceiveTimeoutAfterDataExchange(org.apache.nifi.remote.client.http.TestHttpClient) Time elapsed: 21.275 s <<< FAILURE! java.lang.AssertionError at org.apache.nifi.remote.client.http.TestHttpClient.testReceiveTimeoutAfterDataExchange(TestHttpClient.java:1384) [WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0.454 s - in org.apache.nifi.remote.client.socket.TestSiteToSiteClient [ERROR] Failures: [ERROR] TestHttpClient.testReceiveTimeoutAfterDataExchange:1384 [ERROR] Tests run: 92, Failures: 1, Errors: 0, Skipped: 2 {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7617) Resolve failing unit test due to timeout in CI/CD environment
Andy LoPresto created NIFI-7617: --- Summary: Resolve failing unit test due to timeout in CI/CD environment Key: NIFI-7617 URL: https://issues.apache.org/jira/browse/NIFI-7617 Project: Apache NiFi Issue Type: Bug Components: Tools and Build Affects Versions: 1.11.4 Reporter: Andy LoPresto Assignee: Andy LoPresto There is a test failure on MacOS JP 1.8 for GitHub Actions: {code} [ERROR] Tests run: 32, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 42.397 s <<< FAILURE! - in org.apache.nifi.remote.client.http.TestHttpClient [ERROR] testReceiveTimeoutAfterDataExchange(org.apache.nifi.remote.client.http.TestHttpClient) Time elapsed: 21.275 s <<< FAILURE! java.lang.AssertionError at org.apache.nifi.remote.client.http.TestHttpClient.testReceiveTimeoutAfterDataExchange(TestHttpClient.java:1384) [WARNING] Tests run: 5, Failures: 0, Errors: 0, Skipped: 2, Time elapsed: 0.454 s - in org.apache.nifi.remote.client.socket.TestSiteToSiteClient [ERROR] Failures: [ERROR] TestHttpClient.testReceiveTimeoutAfterDataExchange:1384 [ERROR] Tests run: 92, Failures: 1, Errors: 0, Skipped: 2 {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] turcsanyip commented on a change in pull request #4370: NIFI-6128 UnpackContent: Store unpacked file data
turcsanyip commented on a change in pull request #4370: URL: https://github.com/apache/nifi/pull/4370#discussion_r451834785 ## File path: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/FileInfo.java ## @@ -217,6 +217,20 @@ public Builder group(String group) { } } +public static String permissionToString(int fileModeOctal) { +if(fileModeOctal > 0777 || fileModeOctal < 00) { +throw new RuntimeException("Invalid permission numerals"); +} +char[] PERM = "xwrxwrxwr".toCharArray(); Review comment: I meant it to be a constant, that's why the uppercase. ## File path: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/UnpackContent.java ## @@ -110,6 +118,14 @@ public static final String OCTET_STREAM = "application/octet-stream"; +public static final String FILE_LAST_MODIFIED_TIME_ATTRIBUTE = "file.lastModifiedTime"; +public static final String FILE_OWNER_ATTRIBUTE = "file.owner"; +public static final String FILE_GROUP_ATTRIBUTE = "file.group"; +public static final String FILE_PERMISSIONS_ATTRIBUTE = "file.permissions"; + +public static final String FILE_MODIFIED_DATE_ATTR_FORMAT = "-MM-dd'T'HH:mm:ssZ"; +public static final DateTimeFormatter dateTimeFormatter = DateTimeFormatter.ofPattern(FILE_MODIFIED_DATE_ATTR_FORMAT).withZone(ZoneId.systemDefault()); Review comment: Please use uppercase for constant names. ## File path: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/FileInfo.java ## @@ -217,6 +217,20 @@ public Builder group(String group) { } } +public static String permissionToString(int fileModeOctal) { +if(fileModeOctal > 0777 || fileModeOctal < 00) { Review comment: Formatting: `if (` ## File path: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/util/FileInfo.java ## @@ -217,6 +217,20 @@ public Builder group(String group) { } } +public static String permissionToString(int fileModeOctal) { Review comment: As it is a pure function without side effects, some unit tests could be added easily. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-7616) PrometheusReportingTask: Wrong number of processor active threads reported ?
Benjamin Charron created NIFI-7616: -- Summary: PrometheusReportingTask: Wrong number of processor active threads reported ? Key: NIFI-7616 URL: https://issues.apache.org/jira/browse/NIFI-7616 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 1.9.2 Reporter: Benjamin Charron The number of _processor_ active threads reported by prometheus ("nifi_amount_threads_active") seem off. I think "{{status.getActiveThreadCount()}}" should be "{{processorStatus.getActiveThreadCount()}}" at [PrometheusMetricsUtil.java:193|https://github.com/apache/nifi/blob/aa741cc5967f62c3c38c2a47e712b7faa6fe19ff/nifi-nar-bundles/nifi-extension-utils/nifi-prometheus-utils/src/main/java/org/apache/nifi/prometheus/util/PrometheusMetricsUtil.java#L193] and :194 -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (MINIFICPP-1284) TailFile: Add Record Limit
Andrew Christianson created MINIFICPP-1284: -- Summary: TailFile: Add Record Limit Key: MINIFICPP-1284 URL: https://issues.apache.org/jira/browse/MINIFICPP-1284 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Reporter: Andrew Christianson Assignee: Andrew Christianson {{TailFile}} currently will create an unlimited number of new FlowFiles per invocation of {{onTrigger}}. This can quickly fill up queues/repositories within minificpp. To allow operators to control the rate that data is ingested into minificpp, we need a {{Record Limit}} property which will limit the number of new FlowFiles created per tailed file per {{onTrigger}} invocation. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7572) Add a ScriptedTransformRecord processor
[ https://issues.apache.org/jira/browse/NIFI-7572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153967#comment-17153967 ] ASF subversion and git services commented on NIFI-7572: --- Commit 6cece9cad746c39ada327d064e67e52dd8b7d959 in nifi's branch refs/heads/main from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6cece9c ] NIFI-7572: Added ScriptedTransformRecord processor. Addressed a couple of minor bugs in mock classes that were encountered during testing. NIFI-7572: Addressed review feedback NIFI-7572: Fixed bug that resulted in constantly recompiling Jython script. Updated documentation showing performance difference. Fixed problematic unit tests for TestResizeImage This closes #4374 Signed-off-by: Mike Thomsen > Add a ScriptedTransformRecord processor > --- > > Key: NIFI-7572 > URL: https://issues.apache.org/jira/browse/NIFI-7572 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 3.5h > Remaining Estimate: 0h > > NiFi has started to put a heavier emphasis on Record-oriented processors, as > they provide many benefits including better performance and a better UX over > their purely byte-oriented counterparts. It is common to see users wanting to > transform a Record in some very specific way, but NiFi doesn't make this as > easy as it should. There are methods using ExecuteScript, > InvokedScriptedProcessor, ScriptedRecordWriter, and ScriptedRecordReader for > instance. > But each of these requires that the Script writer understand a lot about NiFi > and how to expose properties, create Property Descriptors, etc. and for > fairly simple transformation we end up with scripts where the logic takes > fewer lines of code than the boilerplate. > We should expose a Processor that allows a user to write a script that takes > a Record and transforms that Record in some way. The processor should be > configured with the following: > * Record Reader (required) > * Record Writer (required) > * Script Language (required) > * Script Body or Script File (one and only one of these required) > The script should implement a single method along the lines of: > {code:java} > Record transform(Record input) throws Exception; {code} > If the script returns null, the input Record should be dropped. Otherwise, > whatever Record is returned should be written to the Record Writer. > The processor should have two relationships: "success" and "failure." > The script should not be allowed to expose any properties or define any > relationships. The point is to keep the script focused purely on processing > the record itself. > It's not entirely clear to me how easy the Record API works with some of the > scripting languages. The Record object does expose a method named toMap() > that returns a Map containing the underlying key/value pairs. > However, the values in that Map may themselves be Records. It might make > sense to expose a new method toNormalizedMap() or something along those lines > that would return a Map where the values have been > recursively normalized, in much the same way that we do for > JoltTransformRecord. This would perhaps allow for cleaner syntax, but I'm not > a scripting expert so I can't say for sure whether such a method is necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7572) Add a ScriptedTransformRecord processor
[ https://issues.apache.org/jira/browse/NIFI-7572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153968#comment-17153968 ] ASF subversion and git services commented on NIFI-7572: --- Commit 6cece9cad746c39ada327d064e67e52dd8b7d959 in nifi's branch refs/heads/main from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6cece9c ] NIFI-7572: Added ScriptedTransformRecord processor. Addressed a couple of minor bugs in mock classes that were encountered during testing. NIFI-7572: Addressed review feedback NIFI-7572: Fixed bug that resulted in constantly recompiling Jython script. Updated documentation showing performance difference. Fixed problematic unit tests for TestResizeImage This closes #4374 Signed-off-by: Mike Thomsen > Add a ScriptedTransformRecord processor > --- > > Key: NIFI-7572 > URL: https://issues.apache.org/jira/browse/NIFI-7572 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 3.5h > Remaining Estimate: 0h > > NiFi has started to put a heavier emphasis on Record-oriented processors, as > they provide many benefits including better performance and a better UX over > their purely byte-oriented counterparts. It is common to see users wanting to > transform a Record in some very specific way, but NiFi doesn't make this as > easy as it should. There are methods using ExecuteScript, > InvokedScriptedProcessor, ScriptedRecordWriter, and ScriptedRecordReader for > instance. > But each of these requires that the Script writer understand a lot about NiFi > and how to expose properties, create Property Descriptors, etc. and for > fairly simple transformation we end up with scripts where the logic takes > fewer lines of code than the boilerplate. > We should expose a Processor that allows a user to write a script that takes > a Record and transforms that Record in some way. The processor should be > configured with the following: > * Record Reader (required) > * Record Writer (required) > * Script Language (required) > * Script Body or Script File (one and only one of these required) > The script should implement a single method along the lines of: > {code:java} > Record transform(Record input) throws Exception; {code} > If the script returns null, the input Record should be dropped. Otherwise, > whatever Record is returned should be written to the Record Writer. > The processor should have two relationships: "success" and "failure." > The script should not be allowed to expose any properties or define any > relationships. The point is to keep the script focused purely on processing > the record itself. > It's not entirely clear to me how easy the Record API works with some of the > scripting languages. The Record object does expose a method named toMap() > that returns a Map containing the underlying key/value pairs. > However, the values in that Map may themselves be Records. It might make > sense to expose a new method toNormalizedMap() or something along those lines > that would return a Map where the values have been > recursively normalized, in much the same way that we do for > JoltTransformRecord. This would perhaps allow for cleaner syntax, but I'm not > a scripting expert so I can't say for sure whether such a method is necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (NIFI-7572) Add a ScriptedTransformRecord processor
[ https://issues.apache.org/jira/browse/NIFI-7572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mike Thomsen updated NIFI-7572: --- Fix Version/s: 1.12.0 Resolution: Fixed Status: Resolved (was: Patch Available) > Add a ScriptedTransformRecord processor > --- > > Key: NIFI-7572 > URL: https://issues.apache.org/jira/browse/NIFI-7572 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Fix For: 1.12.0 > > Time Spent: 3h 40m > Remaining Estimate: 0h > > NiFi has started to put a heavier emphasis on Record-oriented processors, as > they provide many benefits including better performance and a better UX over > their purely byte-oriented counterparts. It is common to see users wanting to > transform a Record in some very specific way, but NiFi doesn't make this as > easy as it should. There are methods using ExecuteScript, > InvokedScriptedProcessor, ScriptedRecordWriter, and ScriptedRecordReader for > instance. > But each of these requires that the Script writer understand a lot about NiFi > and how to expose properties, create Property Descriptors, etc. and for > fairly simple transformation we end up with scripts where the logic takes > fewer lines of code than the boilerplate. > We should expose a Processor that allows a user to write a script that takes > a Record and transforms that Record in some way. The processor should be > configured with the following: > * Record Reader (required) > * Record Writer (required) > * Script Language (required) > * Script Body or Script File (one and only one of these required) > The script should implement a single method along the lines of: > {code:java} > Record transform(Record input) throws Exception; {code} > If the script returns null, the input Record should be dropped. Otherwise, > whatever Record is returned should be written to the Record Writer. > The processor should have two relationships: "success" and "failure." > The script should not be allowed to expose any properties or define any > relationships. The point is to keep the script focused purely on processing > the record itself. > It's not entirely clear to me how easy the Record API works with some of the > scripting languages. The Record object does expose a method named toMap() > that returns a Map containing the underlying key/value pairs. > However, the values in that Map may themselves be Records. It might make > sense to expose a new method toNormalizedMap() or something along those lines > that would return a Map where the values have been > recursively normalized, in much the same way that we do for > JoltTransformRecord. This would perhaps allow for cleaner syntax, but I'm not > a scripting expert so I can't say for sure whether such a method is necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (NIFI-7572) Add a ScriptedTransformRecord processor
[ https://issues.apache.org/jira/browse/NIFI-7572?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153966#comment-17153966 ] ASF subversion and git services commented on NIFI-7572: --- Commit 6cece9cad746c39ada327d064e67e52dd8b7d959 in nifi's branch refs/heads/main from Mark Payne [ https://gitbox.apache.org/repos/asf?p=nifi.git;h=6cece9c ] NIFI-7572: Added ScriptedTransformRecord processor. Addressed a couple of minor bugs in mock classes that were encountered during testing. NIFI-7572: Addressed review feedback NIFI-7572: Fixed bug that resulted in constantly recompiling Jython script. Updated documentation showing performance difference. Fixed problematic unit tests for TestResizeImage This closes #4374 Signed-off-by: Mike Thomsen > Add a ScriptedTransformRecord processor > --- > > Key: NIFI-7572 > URL: https://issues.apache.org/jira/browse/NIFI-7572 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: Mark Payne >Assignee: Mark Payne >Priority: Major > Time Spent: 3.5h > Remaining Estimate: 0h > > NiFi has started to put a heavier emphasis on Record-oriented processors, as > they provide many benefits including better performance and a better UX over > their purely byte-oriented counterparts. It is common to see users wanting to > transform a Record in some very specific way, but NiFi doesn't make this as > easy as it should. There are methods using ExecuteScript, > InvokedScriptedProcessor, ScriptedRecordWriter, and ScriptedRecordReader for > instance. > But each of these requires that the Script writer understand a lot about NiFi > and how to expose properties, create Property Descriptors, etc. and for > fairly simple transformation we end up with scripts where the logic takes > fewer lines of code than the boilerplate. > We should expose a Processor that allows a user to write a script that takes > a Record and transforms that Record in some way. The processor should be > configured with the following: > * Record Reader (required) > * Record Writer (required) > * Script Language (required) > * Script Body or Script File (one and only one of these required) > The script should implement a single method along the lines of: > {code:java} > Record transform(Record input) throws Exception; {code} > If the script returns null, the input Record should be dropped. Otherwise, > whatever Record is returned should be written to the Record Writer. > The processor should have two relationships: "success" and "failure." > The script should not be allowed to expose any properties or define any > relationships. The point is to keep the script focused purely on processing > the record itself. > It's not entirely clear to me how easy the Record API works with some of the > scripting languages. The Record object does expose a method named toMap() > that returns a Map containing the underlying key/value pairs. > However, the values in that Map may themselves be Records. It might make > sense to expose a new method toNormalizedMap() or something along those lines > that would return a Map where the values have been > recursively normalized, in much the same way that we do for > JoltTransformRecord. This would perhaps allow for cleaner syntax, but I'm not > a scripting expert so I can't say for sure whether such a method is necessary. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] asfgit closed pull request #4374: NIFI-7572: Added ScriptedTransformRecord processor. Addressed a coupl…
asfgit closed pull request #4374: URL: https://github.com/apache/nifi/pull/4374 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] MikeThomsen commented on pull request #4374: NIFI-7572: Added ScriptedTransformRecord processor. Addressed a coupl…
MikeThomsen commented on pull request #4374: URL: https://github.com/apache/nifi/pull/4374#issuecomment-655721235 @mattyb149 I'll do it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Commented] (NIFI-7583) Update User Guide to explain how swapping affects queue prioritization
[ https://issues.apache.org/jira/browse/NIFI-7583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17153857#comment-17153857 ] Ryan Hendrickson commented on NIFI-7583: For the documentation, I'd ask that you consider adding something to the "Configure Connection -> Available Prioritizer" Help Icon.. Right now it says "Available prioritizers that could reprioritize FlowFiles in this work queue." And the Selected Prioritizers says: "Prioritizers that have been selected to prioritize FlowFiles in this work queue." Something to notify users of the behavior and NiFi attribute that controls it would be helpful. For a long term improvement.. What about providing an override?.. Similar to the box for "Size Threshold" and "Back Pressure Object Threshold", could there be one with "Priority Queue Size", set to default to 20,000, but could be modified on a connection-by-connection basis? That'd help mitigate what I'm afraid is going to happen when we jacked the size from 20,000 to 200,000 for the entire server instead of the single queue. > Update User Guide to explain how swapping affects queue prioritization > -- > > Key: NIFI-7583 > URL: https://issues.apache.org/jira/browse/NIFI-7583 > Project: Apache NiFi > Issue Type: Task > Components: Configuration >Reporter: Mark Payne >Priority: Major > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] czobrisky commented on pull request #4394: NIFI-7610 Updated dockerhub readme to latest nifi version
czobrisky commented on pull request #4394: URL: https://github.com/apache/nifi/pull/4394#issuecomment-655695236 Just updated the Readme for dockerhub, this should probably be part of the release guide when doing the DockerImage.txt and Dockerfile version update, just to remove any confusion about the most recent version in Docker Hub. If the Release Management should be updated, I don't mind doing that as well. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] czobrisky opened a new pull request #4394: NIFI-7610 Updated dockerhub readme to latest nifi version
czobrisky opened a new pull request #4394: URL: https://github.com/apache/nifi/pull/4394 Thank you for submitting a contribution to Apache NiFi. Please provide a short description of the PR here: Updates Docker Hub Readme to be the latest nifi version, 1.12.0 Improves Docker hub documentation: NIFI-7610 In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically `master`)? - [x] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] Have you verified that the full build is successful on JDK 8? - [ ] Have you verified that the full build is successful on JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [x] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] mattyb149 commented on pull request #4350: NIFI-6934 In PutDatabaseRecord added Postgres UPSERT support
mattyb149 commented on pull request #4350: URL: https://github.com/apache/nifi/pull/4350#issuecomment-655679853 +1 LGTM, ran contrib-check and tested a few scenarios. @simonbence @turcsanyip Are your reviews complete? If so I can merge to main. Thanks for this improvement! It will also make it easier to do DB-specific things now that we have the Database Type property in there. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda commented on pull request #807: MINIFICPP-1228 - Resource ownership refactor + flowFile-owning processors.
arpadboda commented on pull request #807: URL: https://github.com/apache/nifi-minifi-cpp/pull/807#issuecomment-655668294 @adebreceni The is a CI failure, might be related: ``` /home/travis/build/apache/nifi-minifi-cpp/extensions/sftp/tests/FetchSFTPTests.cpp:274: FAILED: {Unknown expression after the reported line} due to unexpected exception with message: File Operation: No Content Claim existed for read ``` It can be an issue in that given test or in the code of that processor your changes just revealed. Please take a look! This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda edited a comment on pull request #807: MINIFICPP-1228 - Resource ownership refactor + flowFile-owning processors.
arpadboda edited a comment on pull request #807: URL: https://github.com/apache/nifi-minifi-cpp/pull/807#issuecomment-655668294 @adebreceni There is a CI failure, might be related: ``` /home/travis/build/apache/nifi-minifi-cpp/extensions/sftp/tests/FetchSFTPTests.cpp:274: FAILED: {Unknown expression after the reported line} due to unexpected exception with message: File Operation: No Content Claim existed for read ``` It can be an issue in that given test or in the code of that processor your changes just revealed. Please take a look! Ps.: this testcase is pretty stable otherwise. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (MINIFICPP-1283) improve terminology around variable registry
Tony Kurc created MINIFICPP-1283: Summary: improve terminology around variable registry Key: MINIFICPP-1283 URL: https://issues.apache.org/jira/browse/MINIFICPP-1283 Project: Apache NiFi MiNiFi C++ Issue Type: Improvement Affects Versions: 0.7.0 Reporter: Tony Kurc Assignee: Tony Kurc swap whitelist/blacklist with allow/deny list -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFI-7615) Improve terminology in wali
Tony Kurc created NIFI-7615: --- Summary: Improve terminology in wali Key: NIFI-7615 URL: https://issues.apache.org/jira/browse/NIFI-7615 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Affects Versions: 1.11.4 Reporter: Tony Kurc Blacklist in wali should be renamed Blocklist -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-registry] alopresto opened a new pull request #285: NIFIREG-400 Upgraded dependency versions.
alopresto opened a new pull request #285: URL: https://github.com/apache/nifi-registry/pull/285 Upgraded the jersey glassfish dependency version and ran a full build to verify. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-7614) Improve terminology around top-level key management for sensitive configuration values
Andy LoPresto created NIFI-7614: --- Summary: Improve terminology around top-level key management for sensitive configuration values Key: NIFI-7614 URL: https://issues.apache.org/jira/browse/NIFI-7614 Project: Apache NiFi Issue Type: Improvement Components: Configuration, Configuration Management, Core Framework Affects Versions: 1.11.4 Reporter: Andy LoPresto Assignee: Andy LoPresto The top-level key and associated handling stored in {{bootstrap.conf}} and used to encrypt the sensitive properties in {{nifi.properties}}, etc. should be renamed from "master". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (NIFIREG-400) Upgrade dependencies
Andy LoPresto created NIFIREG-400: - Summary: Upgrade dependencies Key: NIFIREG-400 URL: https://issues.apache.org/jira/browse/NIFIREG-400 Project: NiFi Registry Issue Type: Improvement Affects Versions: 0.6.0 Reporter: Andy LoPresto Assignee: Andy LoPresto Some dependencies can be upgraded (jquery, jackson-databind). -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] mattyb149 commented on pull request #4374: NIFI-7572: Added ScriptedTransformRecord processor. Addressed a coupl…
mattyb149 commented on pull request #4374: URL: https://github.com/apache/nifi/pull/4374#issuecomment-655583404 +1 LGTM tested a few scenarios, @MikeThomsen are you good with me merging? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync
fgerlits commented on a change in pull request #835: URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r451599406 ## File path: extensions/http-curl/tests/HTTPIntegrationBase.h ## @@ -135,6 +135,9 @@ class VerifyC2Describe : public VerifyC2Base { } void runAssertions() override { +// This class is never used for running assertions, but we are forced to wait for DescribeManifestHandler to verifyJsonHasAgentManifest +// if we were to log something on finished verification, we could poll on finding it +std::this_thread::sleep_for(std::chrono::milliseconds(wait_time_)); Review comment: Can the sleep go into `waitToVerifyProcessor()` and `runAssertion()` be empty in this case? I think that would say in code what you say in the comment above. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync
fgerlits commented on a change in pull request #835: URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r451597749 ## File path: extensions/http-curl/tests/VerifyInvokeHTTPTest.cpp ## @@ -93,47 +95,55 @@ class VerifyInvokeHTTP : public CoapIntegrationBase { flowController_->unload(); flowController_->stopC2(); -runAssertions(); cleanup(); } }; class VerifyInvokeHTTPOKResponse : public VerifyInvokeHTTP { -public: - virtual void runAssertions() override { - assert(LogTestController::getInstance().contains("key:invokehttp.status.code value:201")); -assert(LogTestController::getInstance().contains("response code 201")); + public: + void runAssertions() override { +using org::apache::nifi::minifi::utils::verifyLogLinePresenceInPollTime; +assert(verifyLogLinePresenceInPollTime(std::chrono::seconds(6), Review comment: 👍 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Resolved] (MINIFICPP-1282) BUILD_BYPRODUCTS incorrectly set to lib64
[ https://issues.apache.org/jira/browse/MINIFICPP-1282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Christianson resolved MINIFICPP-1282. Resolution: Fixed > BUILD_BYPRODUCTS incorrectly set to lib64 > - > > Key: MINIFICPP-1282 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1282 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Andrew Christianson >Assignee: Andrew Christianson >Priority: Major > Time Spent: 40m > Remaining Estimate: 0h > > On updated arch linux as of 2020-07-07, getting the following build error: > {code:java} > make[2]: *** No rule to make target > 'thirdparty/rocksdb-install/lib64/librocksdb.a', needed by 'main/minifi'. > Stop. > {code} > Looking at the directory, the library was built, but its in {{lib/}}: > {noformat} > $ ls thirdparty/rocksdb-install/lib64/ > ls: cannot access 'thirdparty/rocksdb-install/lib64/': No such file or > directory > $ ls thirdparty/rocksdb-install/lib/ > cmake librocksdb.a > {noformat} -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync
hunyadi-dev commented on a change in pull request #835: URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r451561893 ## File path: extensions/http-curl/tests/VerifyInvokeHTTPTest.cpp ## @@ -93,47 +95,55 @@ class VerifyInvokeHTTP : public CoapIntegrationBase { flowController_->unload(); flowController_->stopC2(); -runAssertions(); cleanup(); } }; class VerifyInvokeHTTPOKResponse : public VerifyInvokeHTTP { -public: - virtual void runAssertions() override { - assert(LogTestController::getInstance().contains("key:invokehttp.status.code value:201")); -assert(LogTestController::getInstance().contains("response code 201")); + public: + void runAssertions() override { +using org::apache::nifi::minifi::utils::verifyLogLinePresenceInPollTime; +assert(verifyLogLinePresenceInPollTime(std::chrono::seconds(6), Review comment: This is a maximum value, and the test would probably run for far less time. `contains` was running for 3 seconds, but the `waitForVerifyProcessor` added an extra 6 seconds (line 35 of this file). I went with keeping the flat 6 seconds, because this is seems like the time frame these logs should have been produced, and not the 3 seconds the log checking introduced. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] achristianson commented on pull request #834: MINIFICPP-1282 Use GNUInstallDirs for more reliable determination of installed lib/lib64 directories
achristianson commented on pull request #834: URL: https://github.com/apache/nifi-minifi-cpp/pull/834#issuecomment-655534531 Thanks for the review/merge. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync
hunyadi-dev commented on a change in pull request #835: URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r451559061 ## File path: libminifi/test/0-9 ## @@ -0,0 +1 @@ +C API raNdOMcaSe test d4t4 th1s is! Review comment: Oh, thanks for spotting it. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync
hunyadi-dev commented on a change in pull request #835: URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r451558272 ## File path: extensions/http-curl/tests/HTTPIntegrationBase.h ## @@ -135,6 +135,9 @@ class VerifyC2Describe : public VerifyC2Base { } void runAssertions() override { +// This class is never used for running assertions, but we are forced to wait for DescribeManifestHandler to verifyJsonHasAgentManifest +// if we were to log something on finished verification, we could poll on finding it +std::this_thread::sleep_for(std::chrono::milliseconds(wait_time_)); Review comment: It is the same as before. Maybe I can rename `runAssertions` to something more appropriate, if you have suggestions. ## File path: extensions/http-curl/tests/HTTPIntegrationBase.h ## @@ -135,6 +135,9 @@ class VerifyC2Describe : public VerifyC2Base { } void runAssertions() override { +// This class is never used for running assertions, but we are forced to wait for DescribeManifestHandler to verifyJsonHasAgentManifest +// if we were to log something on finished verification, we could poll on finding it +std::this_thread::sleep_for(std::chrono::milliseconds(wait_time_)); Review comment: It is the same as before. Maybe I can rename `runAssertions` globally to something more appropriate, if you have suggestions. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi] markap14 commented on pull request #4393: NIFI-7606:Provide "NIFI_HOME()" function in Expression Language to ob…
markap14 commented on pull request #4393: URL: https://github.com/apache/nifi/pull/4393#issuecomment-655526041 I don't think this function is necessary. It's simply exposing an environment variable, but all environment variables already are available via Expression Language. You can simply use `${NIFI_HOME}` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #834: MINIFICPP-1282 Use GNUInstallDirs for more reliable determination of installed lib/lib64 directories
arpadboda closed pull request #834: URL: https://github.com/apache/nifi-minifi-cpp/pull/834 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-7610) Edit README.md for dockerhub to reflect 1.12.0 release
[ https://issues.apache.org/jira/browse/NIFI-7610?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Chad Zobrisky updated NIFI-7610: Summary: Edit README.md for dockerhub to reflect 1.12.0 release (was: Edit README.md for dockerhub to reflect 1.11.4 image) > Edit README.md for dockerhub to reflect 1.12.0 release > -- > > Key: NIFI-7610 > URL: https://issues.apache.org/jira/browse/NIFI-7610 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Affects Versions: 1.11.4 >Reporter: Chad Zobrisky >Priority: Minor > Fix For: 1.12.0 > > > The readme.md for dockerhub still refers to 1.8.0. This Jira ticket is to > reflect the latest version in main, 1.12.0 as the latest dockerhub image. > > Ref: > [https://github.com/apache/nifi/blob/master/nifi-docker/dockerhub/README.md] > dockerhub: [https://hub.docker.com/r/apache/nifi/] -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi] MikeThomsen commented on a change in pull request #4374: NIFI-7572: Added ScriptedTransformRecord processor. Addressed a coupl…
MikeThomsen commented on a change in pull request #4374: URL: https://github.com/apache/nifi/pull/4374#discussion_r451524437 ## File path: nifi-nar-bundles/nifi-scripting-bundle/nifi-scripting-processors/src/main/java/org/apache/nifi/processors/script/ScriptedTransformRecord.java ## @@ -0,0 +1,398 @@ +/* + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +package org.apache.nifi.processors.script; + +import org.apache.commons.io.IOUtils; +import org.apache.nifi.annotation.behavior.EventDriven; +import org.apache.nifi.annotation.behavior.InputRequirement; +import org.apache.nifi.annotation.behavior.Restricted; +import org.apache.nifi.annotation.behavior.Restriction; +import org.apache.nifi.annotation.behavior.SideEffectFree; +import org.apache.nifi.annotation.behavior.SupportsBatching; +import org.apache.nifi.annotation.behavior.WritesAttribute; +import org.apache.nifi.annotation.behavior.WritesAttributes; +import org.apache.nifi.annotation.documentation.CapabilityDescription; +import org.apache.nifi.annotation.documentation.SeeAlso; +import org.apache.nifi.annotation.documentation.Tags; +import org.apache.nifi.annotation.lifecycle.OnScheduled; +import org.apache.nifi.components.PropertyDescriptor; +import org.apache.nifi.components.PropertyDescriptor.Builder; +import org.apache.nifi.components.RequiredPermission; +import org.apache.nifi.components.ValidationContext; +import org.apache.nifi.components.ValidationResult; +import org.apache.nifi.flowfile.FlowFile; +import org.apache.nifi.processor.AbstractProcessor; +import org.apache.nifi.processor.ProcessContext; +import org.apache.nifi.processor.ProcessSession; +import org.apache.nifi.processor.Relationship; +import org.apache.nifi.processor.exception.ProcessException; +import org.apache.nifi.schema.access.SchemaNotFoundException; +import org.apache.nifi.script.ScriptingComponentHelper; +import org.apache.nifi.script.ScriptingComponentUtils; +import org.apache.nifi.search.SearchContext; +import org.apache.nifi.search.SearchResult; +import org.apache.nifi.search.Searchable; +import org.apache.nifi.serialization.MalformedRecordException; +import org.apache.nifi.serialization.RecordReader; +import org.apache.nifi.serialization.RecordReaderFactory; +import org.apache.nifi.serialization.RecordSetWriter; +import org.apache.nifi.serialization.RecordSetWriterFactory; +import org.apache.nifi.serialization.WriteResult; +import org.apache.nifi.serialization.record.Record; +import org.python.jsr223.PyScriptEngine; + +import javax.script.Bindings; +import javax.script.CompiledScript; +import javax.script.ScriptContext; +import javax.script.ScriptEngine; +import javax.script.ScriptException; +import javax.script.SimpleBindings; +import java.io.FileInputStream; +import java.io.IOException; +import java.nio.charset.Charset; +import java.util.Arrays; +import java.util.Collection; +import java.util.Collections; +import java.util.HashMap; +import java.util.HashSet; +import java.util.List; +import java.util.Map; +import java.util.Set; +import java.util.concurrent.atomic.AtomicLong; + +@EventDriven +@SupportsBatching +@SideEffectFree +@InputRequirement(InputRequirement.Requirement.INPUT_REQUIRED) +@Tags({"record", "transform", "script", "groovy", "jython", "python", "update", "modify", "filter"}) +@Restricted(restrictions = { +@Restriction(requiredPermission = RequiredPermission.EXECUTE_CODE, +explanation = "Provides operator the ability to execute arbitrary code assuming all permissions that NiFi has.") +}) +@WritesAttributes({ +@WritesAttribute(attribute = "mime.type", description = "Sets the mime.type attribute to the MIME Type specified by the Record Writer"), +@WritesAttribute(attribute = "record.count", description = "The number of records in the FlowFile"), +@WritesAttribute(attribute = "record.error.message", description = "This attribute provides on failure the error message encountered by the Reader or Writer.") +}) +@CapabilityDescription("Provides the ability to evaluate a simple script against each record in an incoming FlowFile. The script may transform the record in some way, filter the record, or fork " + +"additional records. See Processor's Additional D
[GitHub] [nifi-minifi-cpp] fgerlits commented on a change in pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync
fgerlits commented on a change in pull request #835: URL: https://github.com/apache/nifi-minifi-cpp/pull/835#discussion_r451477045 ## File path: extensions/http-curl/tests/HTTPIntegrationBase.h ## @@ -135,6 +135,9 @@ class VerifyC2Describe : public VerifyC2Base { } void runAssertions() override { +// This class is never used for running assertions, but we are forced to wait for DescribeManifestHandler to verifyJsonHasAgentManifest +// if we were to log something on finished verification, we could poll on finding it +std::this_thread::sleep_for(std::chrono::milliseconds(wait_time_)); Review comment: A `runAssertions()` function that doesn't run assertions looks weird (as you clearly felt, too, hence the comment). It may be better to keep `waitToVerifyProcessor()` and `runAssertions()` separate, but make their default implementations no-op. ## File path: libminifi/test/0-9 ## @@ -0,0 +1 @@ +C API raNdOMcaSe test d4t4 th1s is! Review comment: garbage left over by a test run ## File path: libminifi/include/utils/IntegrationTestUtils.h ## @@ -0,0 +1,65 @@ +/** + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#pragma once + +#include +#include + +#include "../../../libminifi/test/TestBase.h" + +namespace org { +namespace apache { +namespace nifi { +namespace minifi { +namespace utils { + +template +bool verifyEventHappenedInPollTime(const std::chrono::duration& wait_duration, Fun&& check) { Review comment: You could replace the `becomesTrueWithinTimeout()` calls in `libminifi/test/unit/MinifiConcurrentQueueTests.cpp` with calls to this function, or make `becomesTrueWithinTimeout()` call `verifyEventHappenedInPollTime()`, as they are almost identical. ## File path: extensions/http-curl/tests/VerifyInvokeHTTPTest.cpp ## @@ -93,47 +95,55 @@ class VerifyInvokeHTTP : public CoapIntegrationBase { flowController_->unload(); flowController_->stopC2(); -runAssertions(); cleanup(); } }; class VerifyInvokeHTTPOKResponse : public VerifyInvokeHTTP { -public: - virtual void runAssertions() override { - assert(LogTestController::getInstance().contains("key:invokehttp.status.code value:201")); -assert(LogTestController::getInstance().contains("response code 201")); + public: + void runAssertions() override { +using org::apache::nifi::minifi::utils::verifyLogLinePresenceInPollTime; +assert(verifyLogLinePresenceInPollTime(std::chrono::seconds(6), Review comment: Why did the timeouts in this test increase to 6 seconds (from the default 3 seconds which they were before)? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-7613) Unpacked unique-named native libraries should be in the work directory
Peter Radden created NIFI-7613: -- Summary: Unpacked unique-named native libraries should be in the work directory Key: NIFI-7613 URL: https://issues.apache.org/jira/browse/NIFI-7613 Project: Apache NiFi Issue Type: Wish Components: Core Framework Affects Versions: 1.11.4 Reporter: Peter Radden In AbstractNativeLibHandlingClassLoader, it copies the libraries to the system temp directory using Files.createTempFile. These should instead be within the work directory. (Allowing them to be managed/cleaned up by NiFi, rather than relying on the system to clean up - this is seen to fill up storage on test system) -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
hunyadi-dev commented on a change in pull request #821: URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r451473323 ## File path: extensions/standard-processors/processors/RetryFlowFile.cpp ## @@ -0,0 +1,183 @@ +/** + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "RetryFlowFile.h" + +#include "core/PropertyValidation.h" + +namespace org { +namespace apache { +namespace nifi { +namespace minifi { +namespace processors { + +core::Property RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry Attribute") +->withDescription( +"The name of the attribute that contains the current retry count for the FlowFile." +"WARNING: If the name matches an attribute already on the FlowFile that does not contain a numerical value, " +"the processor will either overwrite that attribute with '1' or fail based on configuration.") +->withDefaultValue("flowfile.retries") +->supportsExpressionLanguage(true) +->build()); + +core::Property RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum Retries") +->withDescription("The maximum number of times a FlowFile can be retried before being passed to the 'retries_exceeded' relationship.") +->withDefaultValue(3) +->supportsExpressionLanguage(true) +->build()); + +core::Property RetryFlowFile::PenalizeRetries(core::PropertyBuilder::createProperty("Penalize Retries") + ->withDescription("If set to 'true', this Processor will penalize input FlowFiles before passing them to the 'retry' relationship. This does not apply to the 'retries_exceeded' relationship.") + ->withDefaultValue(true) + ->build()); + +core::Property RetryFlowFile::FailOnNonNumericalOverwrite(core::PropertyBuilder::createProperty("Fail on Non-numerical Overwrite") +->withDescription("If the FlowFile already has the attribute defined in 'Retry Attribute' that is *not* a number, fail the FlowFile instead of resetting that value to '1'") +->withDefaultValue(false) +->build()); + +core::Property RetryFlowFile::ReuseMode(core::PropertyBuilder::createProperty("Reuse Mode") +->withDescription( +"Defines how the Processor behaves if the retry FlowFile has a different retry UUID than " +"the instance that received the FlowFile. This generally means that the attribute was " +"not reset after being successfully retried by a previous instance of this processor.") +->withAllowableValues({FAIL_ON_REUSE, WARN_ON_REUSE, RESET_REUSE}) +->withDefaultValue(FAIL_ON_REUSE) +->build()); + +core::Relationship RetryFlowFile::Retry("retry", + "Input FlowFile has not exceeded the configured maximum retry count, pass this relationship back to the input Processor to create a limited feedback loop."); +core::Relationship RetryFlowFile::RetriesExceeded("retries_exceeded", + "Input FlowFile has exceeded the configured maximum retry count, do not pass this relationship back to the input Processor to terminate the limited feedback loop."); +core::Relationship RetryFlowFile::Failure("failure", +"The processor is configured such that a non-numerical value on 'Retry Attribute' results in a failure instead of resetting " +"that value to '1'. This will immediately terminate the limited feedback loop. Might also include when 'Maximum Retries' contains " +" attribute expression language that does not resolve to an Integer."); + +void RetryFlowFile::initialize() { + setSupportedProperties({ +RetryAttribute, +MaximumRetries, +PenalizeRetries, +FailOnNonNumericalOverwrite, +ReuseMode, + }); + setSupportedRelationships({ +Retry, +RetriesExceeded, +Failure, + }); +} + +void RetryFlowFile::onSchedule(core::ProcessContext* context, core::ProcessSessionFactory* /* sessionFactory */) { + context->getProperty(RetryAttribute.getName(), retry_attribute_); + context->getProperty(MaximumRetries.getName(), maximum_retries_); + context->getProperty(PenalizeRetries.getName(), penalize_retries_); + context->getProperty(FailOnNonNumericalOverwrite.getName(), fail_on_non_numerical_overwrite_); + context->getProperty(ReuseMode.getName(), reuse_
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
hunyadi-dev commented on a change in pull request #821: URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r451465600 ## File path: extensions/standard-processors/processors/RetryFlowFile.cpp ## @@ -0,0 +1,183 @@ +/** + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "RetryFlowFile.h" + +#include "core/PropertyValidation.h" + +namespace org { +namespace apache { +namespace nifi { +namespace minifi { +namespace processors { + +core::Property RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry Attribute") +->withDescription( +"The name of the attribute that contains the current retry count for the FlowFile." +"WARNING: If the name matches an attribute already on the FlowFile that does not contain a numerical value, " +"the processor will either overwrite that attribute with '1' or fail based on configuration.") +->withDefaultValue("flowfile.retries") +->supportsExpressionLanguage(true) +->build()); + +core::Property RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum Retries") +->withDescription("The maximum number of times a FlowFile can be retried before being passed to the 'retries_exceeded' relationship.") +->withDefaultValue(3) +->supportsExpressionLanguage(true) +->build()); + +core::Property RetryFlowFile::PenalizeRetries(core::PropertyBuilder::createProperty("Penalize Retries") + ->withDescription("If set to 'true', this Processor will penalize input FlowFiles before passing them to the 'retry' relationship. This does not apply to the 'retries_exceeded' relationship.") + ->withDefaultValue(true) + ->build()); + +core::Property RetryFlowFile::FailOnNonNumericalOverwrite(core::PropertyBuilder::createProperty("Fail on Non-numerical Overwrite") +->withDescription("If the FlowFile already has the attribute defined in 'Retry Attribute' that is *not* a number, fail the FlowFile instead of resetting that value to '1'") +->withDefaultValue(false) +->build()); + +core::Property RetryFlowFile::ReuseMode(core::PropertyBuilder::createProperty("Reuse Mode") +->withDescription( +"Defines how the Processor behaves if the retry FlowFile has a different retry UUID than " +"the instance that received the FlowFile. This generally means that the attribute was " +"not reset after being successfully retried by a previous instance of this processor.") +->withAllowableValues({FAIL_ON_REUSE, WARN_ON_REUSE, RESET_REUSE}) +->withDefaultValue(FAIL_ON_REUSE) +->build()); + +core::Relationship RetryFlowFile::Retry("retry", + "Input FlowFile has not exceeded the configured maximum retry count, pass this relationship back to the input Processor to create a limited feedback loop."); +core::Relationship RetryFlowFile::RetriesExceeded("retries_exceeded", + "Input FlowFile has exceeded the configured maximum retry count, do not pass this relationship back to the input Processor to terminate the limited feedback loop."); +core::Relationship RetryFlowFile::Failure("failure", +"The processor is configured such that a non-numerical value on 'Retry Attribute' results in a failure instead of resetting " +"that value to '1'. This will immediately terminate the limited feedback loop. Might also include when 'Maximum Retries' contains " +" attribute expression language that does not resolve to an Integer."); + +void RetryFlowFile::initialize() { + setSupportedProperties({ +RetryAttribute, +MaximumRetries, +PenalizeRetries, +FailOnNonNumericalOverwrite, +ReuseMode, + }); + setSupportedRelationships({ +Retry, +RetriesExceeded, +Failure, + }); +} + +void RetryFlowFile::onSchedule(core::ProcessContext* context, core::ProcessSessionFactory* /* sessionFactory */) { + context->getProperty(RetryAttribute.getName(), retry_attribute_); + context->getProperty(MaximumRetries.getName(), maximum_retries_); + context->getProperty(PenalizeRetries.getName(), penalize_retries_); + context->getProperty(FailOnNonNumericalOverwrite.getName(), fail_on_non_numerical_overwrite_); + context->getProperty(ReuseMode.getName(), reuse_
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
hunyadi-dev commented on a change in pull request #821: URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r451462518 ## File path: extensions/standard-processors/processors/RetryFlowFile.cpp ## @@ -0,0 +1,183 @@ +/** + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "RetryFlowFile.h" + +#include "core/PropertyValidation.h" + +namespace org { +namespace apache { +namespace nifi { +namespace minifi { +namespace processors { + +core::Property RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry Attribute") +->withDescription( +"The name of the attribute that contains the current retry count for the FlowFile." +"WARNING: If the name matches an attribute already on the FlowFile that does not contain a numerical value, " +"the processor will either overwrite that attribute with '1' or fail based on configuration.") +->withDefaultValue("flowfile.retries") +->supportsExpressionLanguage(true) +->build()); + +core::Property RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum Retries") +->withDescription("The maximum number of times a FlowFile can be retried before being passed to the 'retries_exceeded' relationship.") +->withDefaultValue(3) +->supportsExpressionLanguage(true) +->build()); + +core::Property RetryFlowFile::PenalizeRetries(core::PropertyBuilder::createProperty("Penalize Retries") + ->withDescription("If set to 'true', this Processor will penalize input FlowFiles before passing them to the 'retry' relationship. This does not apply to the 'retries_exceeded' relationship.") + ->withDefaultValue(true) + ->build()); + +core::Property RetryFlowFile::FailOnNonNumericalOverwrite(core::PropertyBuilder::createProperty("Fail on Non-numerical Overwrite") +->withDescription("If the FlowFile already has the attribute defined in 'Retry Attribute' that is *not* a number, fail the FlowFile instead of resetting that value to '1'") +->withDefaultValue(false) +->build()); + +core::Property RetryFlowFile::ReuseMode(core::PropertyBuilder::createProperty("Reuse Mode") +->withDescription( +"Defines how the Processor behaves if the retry FlowFile has a different retry UUID than " +"the instance that received the FlowFile. This generally means that the attribute was " +"not reset after being successfully retried by a previous instance of this processor.") +->withAllowableValues({FAIL_ON_REUSE, WARN_ON_REUSE, RESET_REUSE}) +->withDefaultValue(FAIL_ON_REUSE) +->build()); + +core::Relationship RetryFlowFile::Retry("retry", + "Input FlowFile has not exceeded the configured maximum retry count, pass this relationship back to the input Processor to create a limited feedback loop."); +core::Relationship RetryFlowFile::RetriesExceeded("retries_exceeded", + "Input FlowFile has exceeded the configured maximum retry count, do not pass this relationship back to the input Processor to terminate the limited feedback loop."); +core::Relationship RetryFlowFile::Failure("failure", +"The processor is configured such that a non-numerical value on 'Retry Attribute' results in a failure instead of resetting " +"that value to '1'. This will immediately terminate the limited feedback loop. Might also include when 'Maximum Retries' contains " +" attribute expression language that does not resolve to an Integer."); + +void RetryFlowFile::initialize() { + setSupportedProperties({ +RetryAttribute, +MaximumRetries, +PenalizeRetries, +FailOnNonNumericalOverwrite, +ReuseMode, + }); + setSupportedRelationships({ +Retry, +RetriesExceeded, +Failure, + }); +} + +void RetryFlowFile::onSchedule(core::ProcessContext* context, core::ProcessSessionFactory* /* sessionFactory */) { + context->getProperty(RetryAttribute.getName(), retry_attribute_); + context->getProperty(MaximumRetries.getName(), maximum_retries_); + context->getProperty(PenalizeRetries.getName(), penalize_retries_); + context->getProperty(FailOnNonNumericalOverwrite.getName(), fail_on_non_numerical_overwrite_); + context->getProperty(ReuseMode.getName(), reuse_
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
hunyadi-dev commented on a change in pull request #821: URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r451460991 ## File path: extensions/standard-processors/processors/RetryFlowFile.cpp ## @@ -0,0 +1,183 @@ +/** + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "RetryFlowFile.h" + +#include "core/PropertyValidation.h" + +namespace org { +namespace apache { +namespace nifi { +namespace minifi { +namespace processors { + +core::Property RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry Attribute") +->withDescription( +"The name of the attribute that contains the current retry count for the FlowFile." +"WARNING: If the name matches an attribute already on the FlowFile that does not contain a numerical value, " +"the processor will either overwrite that attribute with '1' or fail based on configuration.") +->withDefaultValue("flowfile.retries") +->supportsExpressionLanguage(true) +->build()); + +core::Property RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum Retries") +->withDescription("The maximum number of times a FlowFile can be retried before being passed to the 'retries_exceeded' relationship.") +->withDefaultValue(3) +->supportsExpressionLanguage(true) +->build()); + +core::Property RetryFlowFile::PenalizeRetries(core::PropertyBuilder::createProperty("Penalize Retries") + ->withDescription("If set to 'true', this Processor will penalize input FlowFiles before passing them to the 'retry' relationship. This does not apply to the 'retries_exceeded' relationship.") + ->withDefaultValue(true) + ->build()); + +core::Property RetryFlowFile::FailOnNonNumericalOverwrite(core::PropertyBuilder::createProperty("Fail on Non-numerical Overwrite") +->withDescription("If the FlowFile already has the attribute defined in 'Retry Attribute' that is *not* a number, fail the FlowFile instead of resetting that value to '1'") +->withDefaultValue(false) +->build()); + +core::Property RetryFlowFile::ReuseMode(core::PropertyBuilder::createProperty("Reuse Mode") +->withDescription( +"Defines how the Processor behaves if the retry FlowFile has a different retry UUID than " +"the instance that received the FlowFile. This generally means that the attribute was " +"not reset after being successfully retried by a previous instance of this processor.") +->withAllowableValues({FAIL_ON_REUSE, WARN_ON_REUSE, RESET_REUSE}) +->withDefaultValue(FAIL_ON_REUSE) Review comment: [Same as above.](http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.RetryFlowFile/index.html) ## File path: extensions/standard-processors/processors/RetryFlowFile.cpp ## @@ -0,0 +1,183 @@ +/** + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "RetryFlowFile.h" + +#include "core/PropertyValidation.h" + +namespace org { +namespace apache { +namespace nifi { +namespace minifi { +namespace processors { + +core::Property RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry Attribute") +->withDescription( +"The name of the attribute that contains the current retry count for the FlowFile." +"WARNING: If the name matches an attribute already on the FlowFile that does not contain a num
[GitHub] [nifi-minifi-cpp] hunyadi-dev commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
hunyadi-dev commented on a change in pull request #821: URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r451460587 ## File path: extensions/standard-processors/processors/RetryFlowFile.cpp ## @@ -0,0 +1,183 @@ +/** + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "RetryFlowFile.h" + +#include "core/PropertyValidation.h" + +namespace org { +namespace apache { +namespace nifi { +namespace minifi { +namespace processors { + +core::Property RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry Attribute") +->withDescription( +"The name of the attribute that contains the current retry count for the FlowFile." +"WARNING: If the name matches an attribute already on the FlowFile that does not contain a numerical value, " +"the processor will either overwrite that attribute with '1' or fail based on configuration.") +->withDefaultValue("flowfile.retries") Review comment: The NiFi implementation has this default value set: http://nifi.apache.org/docs/nifi-docs/components/org.apache.nifi/nifi-standard-nar/1.11.4/org.apache.nifi.processors.standard.RetryFlowFile/index.html ## File path: extensions/standard-processors/processors/RetryFlowFile.cpp ## @@ -0,0 +1,183 @@ +/** + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "RetryFlowFile.h" + +#include "core/PropertyValidation.h" + +namespace org { +namespace apache { +namespace nifi { +namespace minifi { +namespace processors { + +core::Property RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry Attribute") +->withDescription( +"The name of the attribute that contains the current retry count for the FlowFile." +"WARNING: If the name matches an attribute already on the FlowFile that does not contain a numerical value, " +"the processor will either overwrite that attribute with '1' or fail based on configuration.") +->withDefaultValue("flowfile.retries") +->supportsExpressionLanguage(true) +->build()); + +core::Property RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum Retries") +->withDescription("The maximum number of times a FlowFile can be retried before being passed to the 'retries_exceeded' relationship.") +->withDefaultValue(3) Review comment: Same as above. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] hunyadi-dev opened a new pull request #835: MINIFICPP-1281 - Improve test performance by using event polling instead of sleep by sync
hunyadi-dev opened a new pull request #835: URL: https://github.com/apache/nifi-minifi-cpp/pull/835 **Decreases the avg. test execution time by 57% (from 326 seconds to 139 seconds).** Measurements on test times of master (on MacBook Pro 16-inch, 2019): ``` ➜ for repetition in {1..5}; do; time ninja test 2>&1 >> test_logs.txt; done; ninja test 2>&1 >> test_logs.txt 46.05s user 19.32s system 18% cpu 5:58.81 total ninja test 2>&1 >> test_logs.txt 45.81s user 13.33s system 18% cpu 5:16.95 total ninja test 2>&1 >> test_logs.txt 46.10s user 13.22s system 18% cpu 5:18.84 total ninja test 2>&1 >> test_logs.txt 46.08s user 13.39s system 18% cpu 5:17.21 total ninja test 2>&1 >> test_logs.txt 46.16s user 13.51s system 18% cpu 5:16.61 total ➜ grep "Total Test time" test_logs.txt Total Test time (real) = 358.74 sec Total Test time (real) = 316.88 sec Total Test time (real) = 318.78 sec Total Test time (real) = 317.14 sec Total Test time (real) = 316.57 sec ``` On the source branch of this PR: ``` ➜ for repetition in {1..5}; do; time ninja test 2>&1 >> test_logs.txt; done; ninja test 2>&1 >> test_logs.txt 24.04s user 9.62s system 24% cpu 2:19.84 total ninja test 2>&1 >> test_logs.txt 23.98s user 9.17s system 24% cpu 2:17.74 total ninja test 2>&1 >> test_logs.txt 23.89s user 9.44s system 24% cpu 2:18.17 total ninja test 2>&1 >> test_logs.txt 23.82s user 9.53s system 24% cpu 2:18.72 total ninja test 2>&1 >> test_logs.txt 24.04s user 9.56s system 23% cpu 2:20.81 total ➜ grep "Total Test time" test_logs.txt Total Test time (real) = 139.77 sec Total Test time (real) = 137.68 sec Total Test time (real) = 138.10 sec Total Test time (real) = 138.66 sec Total Test time (real) = 140.75 sec ``` This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] arpadboda commented on pull request #834: MINIFICPP-1282 Use GNUInstallDirs for more reliable determination of installed lib/lib64 directories
arpadboda commented on pull request #834: URL: https://github.com/apache/nifi-minifi-cpp/pull/834#issuecomment-655434518 Looks good to me. @szaszm do you have any concerns? This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-7612) use processor (ComsumeKafka_2_0) , Error commit kafka offset
JosieMai created NIFI-7612: -- Summary: use processor (ComsumeKafka_2_0) , Error commit kafka offset Key: NIFI-7612 URL: https://issues.apache.org/jira/browse/NIFI-7612 Project: Apache NiFi Issue Type: Epic Components: Extensions Affects Versions: 1.10.0 Environment: CentOS Linux release 7.7.1908 (Core) Reporter: JosieMai at java.lang.Thread.run(Thread.java:748) at java.lang.Thread.run(Thread.java:748)2020-07-08 14:45:33,977 ERROR [Timer-Driven Process Thread-113] o.a.n.p.s.a.AttributeRollingWindow AttributeRollingWindow[id=2d2734ab-0173-1000--8c93fb57] Ran into an error while processing StandardFlowFileRecord[uuid=edac77c1-b519-4ddd-8b40-bfd9c59c376a,claim=StandardContentClaim [resourceClaim=StandardResourceClaim[id=1594187029544-1920, container=default, section=896], offset=366216, length=669582],offset=392,name=a87943bc-b773-4690-8bfc-f0c7a7f9a786,size=406].: java.lang.NumberFormatException: For input string: "kafka.offset"java.lang.NumberFormatException: For input string: "kafka.offset" ... Caused by: org.apache.calcite.runtime.CalciteContextException: From line 1, column 38 to line 1, column 49: Column 'kafka_offset' not found in any table -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (MINIFICPP-1253) Backpressure is applied to self-loops causing the processor to deadlock
[ https://issues.apache.org/jira/browse/MINIFICPP-1253?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Adam Debreceni resolved MINIFICPP-1253. --- Resolution: Fixed > Backpressure is applied to self-loops causing the processor to deadlock > --- > > Key: MINIFICPP-1253 > URL: https://issues.apache.org/jira/browse/MINIFICPP-1253 > Project: Apache NiFi MiNiFi C++ > Issue Type: Bug >Reporter: Adam Debreceni >Assignee: Adam Debreceni >Priority: Major > Time Spent: 3h > Remaining Estimate: 0h > > When a self-loop gets full backpressure is applied to the source processor > which in this case also the destination processor, preventing the processor > from ever triggering. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[GitHub] [nifi-minifi-cpp] arpadboda closed pull request #822: MINIFICPP-1253 - Do not hang on full cycles
arpadboda closed pull request #822: URL: https://github.com/apache/nifi-minifi-cpp/pull/822 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #820: MINIFICPP-1183 - Cleanup C2 Update tests
adamdebreceni commented on a change in pull request #820: URL: https://github.com/apache/nifi-minifi-cpp/pull/820#discussion_r451404038 ## File path: extensions/http-curl/tests/HTTPHandlers.h ## @@ -360,23 +359,19 @@ class DeleteTransactionResponder : public ServerAwareHandler { std::string response_code; }; -class HeartbeatHandler : public ServerAwareHandler { - public: - explicit HeartbeatHandler(bool isSecure) - : isSecure(isSecure) { - } - - std::string readPost(struct mg_connection *conn) { -std::string response; -int readBytes; +std::string readPayload(struct mg_connection *conn) { + std::string response; + int readBytes; -char buffer[1024]; -while ((readBytes = mg_read(conn, buffer, sizeof(buffer))) > 0) { - response.append(buffer, (readBytes / sizeof(char))); -} -return response; + char buffer[1024]; + while ((readBytes = mg_read(conn, buffer, sizeof(buffer))) > 0) { +response.append(buffer, (readBytes / sizeof(char))); Review comment: I know it's old code but that division by `sizeof(char)` though 😏 This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Created] (NIFI-7611) NiFi fails to index provenance events
Michal W created NIFI-7611: -- Summary: NiFi fails to index provenance events Key: NIFI-7611 URL: https://issues.apache.org/jira/browse/NIFI-7611 Project: Apache NiFi Issue Type: Bug Affects Versions: 1.11.4 Environment: Microsoft Windows Server 2016 Standard - Intel Xeon Gold 6140 CPU @ 2,30 GHz 8 processors, 32 GB RAM, total disk space 877 GB Reporter: Michal W Getting error "failed to index provenance events". Nifi.app log displays following information: 2020-07-08 09:00:00,406 ERROR [Index Provenance Events-4] o.a.n.p.index.lucene.EventIndexTask Failed to index Provenance Events org.apache.lucene.store.AlreadyClosedException: this IndexWriter is closed at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:681) at org.apache.lucene.index.IndexWriter.ensureOpen(IndexWriter.java:695) at org.apache.lucene.index.IndexWriter.updateDocuments(IndexWriter.java:1281) at org.apache.lucene.index.IndexWriter.addDocuments(IndexWriter.java:1257) at org.apache.nifi.provenance.lucene.LuceneEventIndexWriter.index(LuceneEventIndexWriter.java:70) at org.apache.nifi.provenance.index.lucene.EventIndexTask.index(EventIndexTask.java:202) at org.apache.nifi.provenance.index.lucene.EventIndexTask.run(EventIndexTask.java:113) at java.base/java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:515) at java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) at java.base/java.lang.Thread.run(Thread.java:834) Caused by: java.nio.file.FileSystemException: E:\nifi-storage\provenance_repository\lucene-8-index-1593163985970\_11r.cfe: The process cannot access the file because it is being used by another process. at java.base/sun.nio.fs.WindowsException.translateToIOException(WindowsException.java:92) at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:103) at java.base/sun.nio.fs.WindowsException.rethrowAsIOException(WindowsException.java:108) at java.base/sun.nio.fs.WindowsFileSystemProvider.newFileChannel(WindowsFileSystemProvider.java:120) at java.base/java.nio.channels.FileChannel.open(FileChannel.java:292) at java.base/java.nio.channels.FileChannel.open(FileChannel.java:345) at org.apache.lucene.store.MMapDirectory.openInput(MMapDirectory.java:238) at org.apache.lucene.store.Directory.openChecksumInput(Directory.java:157) at org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.readEntries(Lucene50CompoundReader.java:105) at org.apache.lucene.codecs.lucene50.Lucene50CompoundReader.(Lucene50CompoundReader.java:69) at org.apache.lucene.codecs.lucene50.Lucene50CompoundFormat.getCompoundReader(Lucene50CompoundFormat.java:70) at org.apache.lucene.index.SegmentCoreReaders.(SegmentCoreReaders.java:100) at org.apache.lucene.index.SegmentReader.(SegmentReader.java:83) at org.apache.lucene.index.ReadersAndUpdates.getReader(ReadersAndUpdates.java:172) at org.apache.lucene.index.ReadersAndUpdates.getReaderForMerge(ReadersAndUpdates.java:709) at org.apache.lucene.index.IndexWriter.mergeMiddle(IndexWriter.java:4396) at org.apache.lucene.index.IndexWriter.merge(IndexWriter.java:4054) at org.apache.lucene.index.ConcurrentMergeScheduler.doMerge(ConcurrentMergeScheduler.java:625) at org.apache.lucene.index.ConcurrentMergeScheduler$MergeThread.run(ConcurrentMergeScheduler.java:662) Logs eventually grow over time and fill up the partition. Configuration related to provenance repository: # Provenance Repository Properties nifi.provenance.repository.implementation=org.apache.nifi.provenance.WriteAheadProvenanceRepository nifi.provenance.repository.debug.frequency=1_000_000 nifi.provenance.repository.encryption.key.provider.implementation= nifi.provenance.repository.encryption.key.provider.location= nifi.provenance.repository.encryption.key.id= nifi.provenance.repository.encryption.key= # Persistent Provenance Repository Properties nifi.provenance.repository.directory.default=E:\\nifi-storage\\provenance_repository nifi.provenance.repository.directory.content1=F:\\nifi-storage\\provenance_repository nifi.provenance.repository.max.storage.time=24 hours # nifi.provenance.repository.max.storage.size
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
adamdebreceni commented on a change in pull request #821: URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r451350379 ## File path: extensions/standard-processors/processors/RetryFlowFile.cpp ## @@ -0,0 +1,183 @@ +/** + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "RetryFlowFile.h" + +#include "core/PropertyValidation.h" + +namespace org { +namespace apache { +namespace nifi { +namespace minifi { +namespace processors { + +core::Property RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry Attribute") +->withDescription( +"The name of the attribute that contains the current retry count for the FlowFile." +"WARNING: If the name matches an attribute already on the FlowFile that does not contain a numerical value, " +"the processor will either overwrite that attribute with '1' or fail based on configuration.") +->withDefaultValue("flowfile.retries") Review comment: it might be beneficial to make all new properties required by default, and if in the future it proves to be a pain point, we could always give a default based on real usage requirements, while if we would want to make something required later we would break previous flows This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[GitHub] [nifi-minifi-cpp] adamdebreceni commented on a change in pull request #821: MINIFICPP-1251 - Implement and test RetryFlowFile processor
adamdebreceni commented on a change in pull request #821: URL: https://github.com/apache/nifi-minifi-cpp/pull/821#discussion_r451340560 ## File path: extensions/standard-processors/processors/RetryFlowFile.cpp ## @@ -0,0 +1,183 @@ +/** + * + * Licensed to the Apache Software Foundation (ASF) under one or more + * contributor license agreements. See the NOTICE file distributed with + * this work for additional information regarding copyright ownership. + * The ASF licenses this file to You under the Apache License, Version 2.0 + * (the "License"); you may not use this file except in compliance with + * the License. You may obtain a copy of the License at + * + * http://www.apache.org/licenses/LICENSE-2.0 + * + * Unless required by applicable law or agreed to in writing, software + * distributed under the License is distributed on an "AS IS" BASIS, + * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. + * See the License for the specific language governing permissions and + * limitations under the License. + */ + +#include "RetryFlowFile.h" + +#include "core/PropertyValidation.h" + +namespace org { +namespace apache { +namespace nifi { +namespace minifi { +namespace processors { + +core::Property RetryFlowFile::RetryAttribute(core::PropertyBuilder::createProperty("Retry Attribute") +->withDescription( +"The name of the attribute that contains the current retry count for the FlowFile." +"WARNING: If the name matches an attribute already on the FlowFile that does not contain a numerical value, " +"the processor will either overwrite that attribute with '1' or fail based on configuration.") +->withDefaultValue("flowfile.retries") +->supportsExpressionLanguage(true) +->build()); + +core::Property RetryFlowFile::MaximumRetries(core::PropertyBuilder::createProperty("Maximum Retries") +->withDescription("The maximum number of times a FlowFile can be retried before being passed to the 'retries_exceeded' relationship.") +->withDefaultValue(3) +->supportsExpressionLanguage(true) +->build()); + +core::Property RetryFlowFile::PenalizeRetries(core::PropertyBuilder::createProperty("Penalize Retries") + ->withDescription("If set to 'true', this Processor will penalize input FlowFiles before passing them to the 'retry' relationship. This does not apply to the 'retries_exceeded' relationship.") + ->withDefaultValue(true) + ->build()); + +core::Property RetryFlowFile::FailOnNonNumericalOverwrite(core::PropertyBuilder::createProperty("Fail on Non-numerical Overwrite") +->withDescription("If the FlowFile already has the attribute defined in 'Retry Attribute' that is *not* a number, fail the FlowFile instead of resetting that value to '1'") +->withDefaultValue(false) +->build()); + +core::Property RetryFlowFile::ReuseMode(core::PropertyBuilder::createProperty("Reuse Mode") +->withDescription( +"Defines how the Processor behaves if the retry FlowFile has a different retry UUID than " +"the instance that received the FlowFile. This generally means that the attribute was " +"not reset after being successfully retried by a previous instance of this processor.") +->withAllowableValues({FAIL_ON_REUSE, WARN_ON_REUSE, RESET_REUSE}) +->withDefaultValue(FAIL_ON_REUSE) +->build()); + +core::Relationship RetryFlowFile::Retry("retry", + "Input FlowFile has not exceeded the configured maximum retry count, pass this relationship back to the input Processor to create a limited feedback loop."); +core::Relationship RetryFlowFile::RetriesExceeded("retries_exceeded", + "Input FlowFile has exceeded the configured maximum retry count, do not pass this relationship back to the input Processor to terminate the limited feedback loop."); +core::Relationship RetryFlowFile::Failure("failure", +"The processor is configured such that a non-numerical value on 'Retry Attribute' results in a failure instead of resetting " +"that value to '1'. This will immediately terminate the limited feedback loop. Might also include when 'Maximum Retries' contains " +" attribute expression language that does not resolve to an Integer."); + +void RetryFlowFile::initialize() { + setSupportedProperties({ +RetryAttribute, +MaximumRetries, +PenalizeRetries, +FailOnNonNumericalOverwrite, +ReuseMode, + }); + setSupportedRelationships({ +Retry, +RetriesExceeded, +Failure, + }); +} + +void RetryFlowFile::onSchedule(core::ProcessContext* context, core::ProcessSessionFactory* /* sessionFactory */) { + context->getProperty(RetryAttribute.getName(), retry_attribute_); + context->getProperty(MaximumRetries.getName(), maximum_retries_); + context->getProperty(PenalizeRetries.getName(), penalize_retries_); + context->getProperty(FailOnNonNumericalOverwrite.getName(), fail_on_non_numerical_overwrite_); + context->getProperty(ReuseMode.getName(), reus
[GitHub] [nifi] KuKuDeCheng opened a new pull request #4393: NIFI-7606:Provide "NIFI_HOME()" function in Expression Language to ob…
KuKuDeCheng opened a new pull request #4393: URL: https://github.com/apache/nifi/pull/4393 NIFI-7606:Provide "NIFI_HOME()" function in Expression Language to obtain the installation directory of NIFI Thank you for submitting a contribution to Apache NiFi. Please provide a short description of the PR here: Description of PR Usually, I make a file '$NIFI_HOME/jdbc' to hold some jars of drivers. I find it's useful to provide a function to get the installation directory of NIFI, and we will use this such as `${NIFI_HOME():append('/jdbc/ojdbc8.jar')}` in `DbcpConnectionPoll` property. I think it will be useful for other scenes need to indicate file directory. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [ ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [ ] Does your PR title start with **NIFI-** where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [ ] Has your PR been rebased against the latest commit within the target branch (typically `master`)? - [ ] Is your initial contribution a single, squashed commit? _Additional commits in response to PR reviewer feedback should be made on this branch and pushed to allow change tracking. Do not `squash` or use `--force` when pushing to allow for clean monitoring of changes._ ### For code changes: - [ ] Have you ensured that the full suite of tests is executed via `mvn -Pcontrib-check clean install` at the root `nifi` folder? - [ ] Have you written or updated unit tests to verify your changes? - [√ ] Have you verified that the full build is successful on JDK 8? - [ ] Have you verified that the full build is successful on JDK 11? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the `LICENSE` file, including the main `LICENSE` file under `nifi-assembly`? - [ ] If applicable, have you updated the `NOTICE` file, including the main `NOTICE` file found under `nifi-assembly`? - [ ] If adding new Properties, have you added `.displayName` in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check GitHub Actions CI for build issues and submit an update to your PR as soon as possible. This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[jira] [Updated] (NIFI-7606) Provide "NIFI_HOME()" function in Expression Language
[ https://issues.apache.org/jira/browse/NIFI-7606?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] ZhangCheng updated NIFI-7606: - Description: Usually, I make a file '$NIFI_HOME/jdbc' to hold some jars of driivers. I find it's useful to provide a function to get the path NIFI installed, and we will use this such as `$(NIFI_HOME):append('/jdbc/ojdbc8.jar'))` in `DbcpConnectionPoll` property. I think it will be useful for other scenes need to indicate file path. (was: Usually, I make a file '$NIFI_HOME/jdbc' to hold some jars of driivers. I find it's useful to provide a function to get the path NIFI installed, and we will use this such as `$(NIFI_HOME().append('/jdbc/ojdbc8.jar'))` in `DbcpConnectionPoll` property. I think it will be useful for other scenes need to indicate file path.) > Provide "NIFI_HOME()" function in Expression Language > - > > Key: NIFI-7606 > URL: https://issues.apache.org/jira/browse/NIFI-7606 > Project: Apache NiFi > Issue Type: New Feature > Components: Extensions >Reporter: ZhangCheng >Assignee: ZhangCheng >Priority: Major > > Usually, I make a file '$NIFI_HOME/jdbc' to hold some jars of driivers. I > find it's useful to provide a function to get the path NIFI installed, and we > will use this such as `$(NIFI_HOME):append('/jdbc/ojdbc8.jar'))` in > `DbcpConnectionPoll` property. I think it will be useful for other scenes > need to indicate file path. -- This message was sent by Atlassian Jira (v8.3.4#803005)