[jira] [Commented] (NIFI-3408) Add an attribute for failure reason to InvokeHTTP in all cases to match docs
[ https://issues.apache.org/jira/browse/NIFI-3408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15855521#comment-15855521 ] ASF GitHub Bot commented on NIFI-3408: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1467 > Add an attribute for failure reason to InvokeHTTP in all cases to match docs > > > Key: NIFI-3408 > URL: https://issues.apache.org/jira/browse/NIFI-3408 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Priority: Minor > Fix For: 1.2.0 > > > Originally brought up on the mailing list[1], the InvokeHTTP docs state: > ``` > The original FlowFile will be routed on any type of connection failure, > timeout or general exception. It will have new attributes detailing the > request. > ``` > Adding attributes currently only happens though when a response is received > from the server, so not attributes if it fails to connect or the socket times > out. > These attributes should be added. > [1] > http://apache-nifi-users-list.2361937.n4.nabble.com/InvokeHTTP-and-SocketTimeoutException-td743.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Resolved] (NIFI-3408) Add an attribute for failure reason to InvokeHTTP in all cases to match docs
[ https://issues.apache.org/jira/browse/NIFI-3408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-3408. -- Resolution: Fixed Fix Version/s: 1.2.0 > Add an attribute for failure reason to InvokeHTTP in all cases to match docs > > > Key: NIFI-3408 > URL: https://issues.apache.org/jira/browse/NIFI-3408 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Priority: Minor > Fix For: 1.2.0 > > > Originally brought up on the mailing list[1], the InvokeHTTP docs state: > ``` > The original FlowFile will be routed on any type of connection failure, > timeout or general exception. It will have new attributes detailing the > request. > ``` > Adding attributes currently only happens though when a response is received > from the server, so not attributes if it fails to connect or the socket times > out. > These attributes should be added. > [1] > http://apache-nifi-users-list.2361937.n4.nabble.com/InvokeHTTP-and-SocketTimeoutException-td743.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3408) Add an attribute for failure reason to InvokeHTTP in all cases to match docs
[ https://issues.apache.org/jira/browse/NIFI-3408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15855520#comment-15855520 ] ASF subversion and git services commented on NIFI-3408: --- Commit 31b943d2c137f4ded254d4beb7c1d3268ce1bac8 in nifi's branch refs/heads/master from [~lanzani] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=31b943d ] NIFI-3408 Add exception class when InvokeHTTP fails This closes #1467. > Add an attribute for failure reason to InvokeHTTP in all cases to match docs > > > Key: NIFI-3408 > URL: https://issues.apache.org/jira/browse/NIFI-3408 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Priority: Minor > > Originally brought up on the mailing list[1], the InvokeHTTP docs state: > ``` > The original FlowFile will be routed on any type of connection failure, > timeout or general exception. It will have new attributes detailing the > request. > ``` > Adding attributes currently only happens though when a response is received > from the server, so not attributes if it fails to connect or the socket times > out. > These attributes should be added. > [1] > http://apache-nifi-users-list.2361937.n4.nabble.com/InvokeHTTP-and-SocketTimeoutException-td743.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1467: NIFI-3408 Add exception class when InvokeHTTP fails
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1467 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3414) Implement an EnforceOrder processor
[ https://issues.apache.org/jira/browse/NIFI-3414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15855371#comment-15855371 ] Koji Kawamura commented on NIFI-3414: - Thanks Matt for explaining detailed use-case. (I thought I wrote a reply last week but the comment wasn't saved properly..) I'm going to implement the processor, and add following capabilities as well: - To not update state storage too often, batch up input flow files max to 'Max Batch Count' and process those at once - Group incoming flow files by 'Group Identifier', and handle multiple groups - Add 'wait' queue to route flow files those didn't match with current order number, so that those leave from the incoming queue and make room for other flow files can be enqueued. Penalize those, too. - Track how long a flow file has been waiting. If it exceeds configured 'Timeout', routes it to 'timeout' relationship. Update the order number with the maximum + 1 of timeout flow files, so that if the skipped flow file arrives later, processor can detect it - If incoming file has an order number younger than the current, then route it to 'skipped' relationship > Implement an EnforceOrder processor > --- > > Key: NIFI-3414 > URL: https://issues.apache.org/jira/browse/NIFI-3414 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Matt Burgess >Assignee: Koji Kawamura > > For some flows, it is imperative that the flow files are processed in a > certain order. The PriorityAttributePrioritizer can be used on a connection > to ensure that flow files going through that connection are in priority > order, but depending on error-handling, branching, and other flow designs, it > is possible for flow files to get out-of-order. > I propose an EnforceOrder processor, which would be single-threaded and have > (at a minimum) the following properties: > 1) Order Attribute: This would be the name of a flow file attribute from > which the current value will be retrieved. > 2) Initial Value: This property specifies an initial value for the order. The > processor is stateful, however, so this property is only used when there is > no entry in the state map for current value. > The processor would store the Initial Value into the state map (if no state > map entry exists), then for each incoming flow file, it checks the value in > the Order Attribute against the current value. If the attribute value > matches the current value, the flow file is transferred to the "success" > relationship, and the current value is incremented in the state map. If the > attribute value does not match the current value, the session will be rolled > back. > Using this processor, along with a PriorityAttributePrioritizer on the > incoming connection, will allow for out-of-order flow files to have a sort of > "barrier", thereby guaranteeing that flow files transferred to the "success" > relationship are in the specified order. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3446) Make FlowConfigurationDAO Pluggable
[ https://issues.apache.org/jira/browse/NIFI-3446?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15855326#comment-15855326 ] ASF GitHub Bot commented on NIFI-3446: -- GitHub user patricker opened a pull request: https://github.com/apache/nifi/pull/1478 NIFI-3446 FlowConfigurationDAO Changes Allows for Flow Configuration (flow.xml.gz) to be pluggable. There should be no "functional" changes as a result of this code change. ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/patricker/nifi NIFI-3446 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1478.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1478 commit e346336161c6c5220661d2ae31e91391c40c54d3 Author: patricker Date: 2017-02-07T03:49:28Z FlowConfigurationDAO Changes > Make FlowConfigurationDAO Pluggable > --- > > Key: NIFI-3446 > URL: https://issues.apache.org/jira/browse/NIFI-3446 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Peter Wicks >Assignee: Peter Wicks > Fix For: 1.2.0 > > > FlowConfigurationDAO should be pluggable so that an alternate Flow > Configuration storage layer can be used. > This may be a multistep process, this ticket does the first step by moving > more of the functionality into FlowConfigurationDAO, allows users to override > (optional) properties in nifi.properties. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1478: NIFI-3446 FlowConfigurationDAO Changes
GitHub user patricker opened a pull request: https://github.com/apache/nifi/pull/1478 NIFI-3446 FlowConfigurationDAO Changes Allows for Flow Configuration (flow.xml.gz) to be pluggable. There should be no "functional" changes as a result of this code change. ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [ ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/patricker/nifi NIFI-3446 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1478.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1478 commit e346336161c6c5220661d2ae31e91391c40c54d3 Author: patricker Date: 2017-02-07T03:49:28Z FlowConfigurationDAO Changes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (NIFI-3446) Make FlowConfigurationDAO Pluggable
Peter Wicks created NIFI-3446: - Summary: Make FlowConfigurationDAO Pluggable Key: NIFI-3446 URL: https://issues.apache.org/jira/browse/NIFI-3446 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Affects Versions: 1.2.0 Reporter: Peter Wicks Assignee: Peter Wicks Fix For: 1.2.0 FlowConfigurationDAO should be pluggable so that an alternate Flow Configuration storage layer can be used. This may be a multistep process, this ticket does the first step by moving more of the functionality into FlowConfigurationDAO, allows users to override (optional) properties in nifi.properties. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1450: NIFI-3339b Add getDataSource() to DBCPService, second vers...
Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/1450 My use case is a third-party library that takes the same information you might put into a DBCPControllerService, either in the URL and/or the username/password, etc. I was hoping to avoid trying to parse the URL looking for some properties (hostname:port) and need access to other properties (username/password). I could just add these properties explicitly to my processor(s), especially if there'd be a lot of refactor involved with making such information available via DBCPService. Your getTransitUri() addition would be a good improvement IMO, I added something similar to HiveDBCPService (a getConnectionURL() method). If we added that to DBCPService we could remove it from HiveDBCPService (although I would still keep the class as a marker interface). I withdraw my use case from this discussion as I can accomplish it another way, and as the processor(s) I am working on are TriggerSerially and to be run on the Primary Node only, I doubt there's much to be gained by reusing a Controller Service anyway. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-1125) ERROR [NiFi Web Server-22655] o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: java.lang.NullPointerException
[ https://issues.apache.org/jira/browse/NIFI-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-1125: Status: Patch Available (was: Open) Thank you [~mpetronic] for reporting this issue with detailed description. I confirmed this issue still exists in the latest 1.2.0-SNAPSHOT as of this writing. I've submitted a PR to fix this and hopefully it will be merged soon. > ERROR [NiFi Web Server-22655] o.a.nifi.web.api.config.ThrowableMapper An > unexpected error has occurred: java.lang.NullPointerException > -- > > Key: NIFI-1125 > URL: https://issues.apache.org/jira/browse/NIFI-1125 > Project: Apache NiFi > Issue Type: Bug > Components: Configuration >Affects Versions: 0.4.0 > Environment: Centos 6.7 >Reporter: Mark Petronic >Assignee: Koji Kawamura >Priority: Minor > Attachments: StatsIngestFlow.xml > > > Running with a latest master branch build off commit > dbf0c7893fef964bfbb3a4c039c756396587ce12. > Steps to recreate: > 1. All processors stopped > 2. Add uuid to "Attributes to Send" in the InvokeHttp Processor > 3. Save and close config dialog > 4. Open same config dialog and remove uuid field so that is is "No value set" > 5. Apply changes and web UI will crash and indicate error has occurred > and below will seen in user log > 6. Hit F5 and browser reloads UI just fine > See attached template. > 2015-11-07 19:42:58,369 ERROR [NiFi Web Server-22655] > o.a.nifi.web.api.config.ThrowableMapper An unexpected error has > occurred: java.lang.NullPointerException. Returning Internal Server > Error response. > java.lang.NullPointerException: null > at > org.apache.nifi.processors.standard.InvokeHTTP.onPropertyModified(InvokeHTTP.java:121) > ~[na:na] > at > org.apache.nifi.controller.AbstractConfiguredComponent.removeProperty(AbstractConfiguredComponent.java:163) > ~[nifi-framework-core-api-0.3.1-SNAPSHOT.jar:0.3.1-SNAPSHOT] > at > org.apache.nifi.web.dao.impl.StandardProcessorDAO.configureProcessor(StandardProcessorDAO.java:174) > ~[classes/:na] > at > org.apache.nifi.web.dao.impl.StandardProcessorDAO.updateProcessor(StandardProcessorDAO.java:391) > ~[classes/:na] > at > org.apache.nifi.web.dao.impl.StandardProcessorDAO$$FastClassBySpringCGLIB$$779e089b.invoke() > ~[spring-core-4.1.6.RELEASE.jar:na] > at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > ~[spring-core-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:717) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.apache.nifi.audit.ProcessorAuditor.updateProcessorAdvice(ProcessorAuditor.java:120) > ~[classes/:na] > at sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source) ~[na:na] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > ~[na:1.8.0_51] > at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_51] > at > org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:653) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.apache.nifi.web.dao.impl.StandardProcessorDAO$$EnhancerBySpringCGLIB$$f8bfa279.updateProcessor() > ~[spring-core-4.1.6.RELEASE.jar:na] > at > org.apache.nifi.web.StandardNiFiServiceFacade$2.execute(StandardNiFiServiceFacade.java:412) > ~[classes/:0.3.1-SNAPSHOT] >
[jira] [Assigned] (NIFI-1125) ERROR [NiFi Web Server-22655] o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: java.lang.NullPointerException
[ https://issues.apache.org/jira/browse/NIFI-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura reassigned NIFI-1125: --- Assignee: Koji Kawamura > ERROR [NiFi Web Server-22655] o.a.nifi.web.api.config.ThrowableMapper An > unexpected error has occurred: java.lang.NullPointerException > -- > > Key: NIFI-1125 > URL: https://issues.apache.org/jira/browse/NIFI-1125 > Project: Apache NiFi > Issue Type: Bug > Components: Configuration >Affects Versions: 0.4.0 > Environment: Centos 6.7 >Reporter: Mark Petronic >Assignee: Koji Kawamura >Priority: Minor > Attachments: StatsIngestFlow.xml > > > Running with a latest master branch build off commit > dbf0c7893fef964bfbb3a4c039c756396587ce12. > Steps to recreate: > 1. All processors stopped > 2. Add uuid to "Attributes to Send" in the InvokeHttp Processor > 3. Save and close config dialog > 4. Open same config dialog and remove uuid field so that is is "No value set" > 5. Apply changes and web UI will crash and indicate error has occurred > and below will seen in user log > 6. Hit F5 and browser reloads UI just fine > See attached template. > 2015-11-07 19:42:58,369 ERROR [NiFi Web Server-22655] > o.a.nifi.web.api.config.ThrowableMapper An unexpected error has > occurred: java.lang.NullPointerException. Returning Internal Server > Error response. > java.lang.NullPointerException: null > at > org.apache.nifi.processors.standard.InvokeHTTP.onPropertyModified(InvokeHTTP.java:121) > ~[na:na] > at > org.apache.nifi.controller.AbstractConfiguredComponent.removeProperty(AbstractConfiguredComponent.java:163) > ~[nifi-framework-core-api-0.3.1-SNAPSHOT.jar:0.3.1-SNAPSHOT] > at > org.apache.nifi.web.dao.impl.StandardProcessorDAO.configureProcessor(StandardProcessorDAO.java:174) > ~[classes/:na] > at > org.apache.nifi.web.dao.impl.StandardProcessorDAO.updateProcessor(StandardProcessorDAO.java:391) > ~[classes/:na] > at > org.apache.nifi.web.dao.impl.StandardProcessorDAO$$FastClassBySpringCGLIB$$779e089b.invoke() > ~[spring-core-4.1.6.RELEASE.jar:na] > at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204) > ~[spring-core-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:717) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.apache.nifi.audit.ProcessorAuditor.updateProcessorAdvice(ProcessorAuditor.java:120) > ~[classes/:na] > at sun.reflect.GeneratedMethodAccessor176.invoke(Unknown Source) ~[na:na] > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > ~[na:1.8.0_51] > at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_51] > at > org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:168) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:653) > ~[spring-aop-4.1.6.RELEASE.jar:4.1.6.RELEASE] > at > org.apache.nifi.web.dao.impl.StandardProcessorDAO$$EnhancerBySpringCGLIB$$f8bfa279.updateProcessor() > ~[spring-core-4.1.6.RELEASE.jar:na] > at > org.apache.nifi.web.StandardNiFiServiceFacade$2.execute(StandardNiFiServiceFacade.java:412) > ~[classes/:0.3.1-SNAPSHOT] > at > org.apache.nifi.web.StandardOptimisticLockingManager.configureFlow(StandardOptimisticLockingManager.java:83) > ~[nifi-web-optimistic-locking-0.3.1-SNAPSHOT.jar:0.3.1-SNAPSHOT] > at > org.apache.nifi.web.StandardNiFiServiceFacade.upda
[jira] [Commented] (NIFI-1125) ERROR [NiFi Web Server-22655] o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred: java.lang.NullPointerException
[ https://issues.apache.org/jira/browse/NIFI-1125?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15855158#comment-15855158 ] ASF GitHub Bot commented on NIFI-1125: -- GitHub user ijokarumawak opened a pull request: https://github.com/apache/nifi/pull/1477 NIFI-1125 InvokeHTTP throws NullPointerException Added null check in onPropertyModified to avoid NPE. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijokarumawak/nifi nifi-1125 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1477.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1477 commit 3120cd20e4a8fe911c7e61ed96e10519d647be74 Author: Koji Kawamura Date: 2017-02-07T02:08:06Z NIFI-1125 InvokeHTTP throws NullPointerException Added null check in onPropertyModified to avoid NPE. > ERROR [NiFi Web Server-22655] o.a.nifi.web.api.config.ThrowableMapper An > unexpected error has occurred: java.lang.NullPointerException > -- > > Key: NIFI-1125 > URL: https://issues.apache.org/jira/browse/NIFI-1125 > Project: Apache NiFi > Issue Type: Bug > Components: Configuration >Affects Versions: 0.4.0 > Environment: Centos 6.7 >Reporter: Mark Petronic >Priority: Minor > Attachments: StatsIngestFlow.xml > > > Running with a latest master branch build off commit > dbf0c7893fef964bfbb3a4c039c756396587ce12. > Steps to recreate: > 1. All processors stopped > 2. Add uuid to "Attributes to Send" in the InvokeHttp Processor > 3. Save and close config dialog > 4. Open same config dialog and remove uuid field so that is is "No value set" > 5. Apply changes and web UI will crash and indicate error has occurred > and below will seen in user log > 6. Hit F5 and browser reloads UI just fine > See attached template. > 2015-11-07 19:42:58,369 ERROR [NiFi Web Server-22655] > o.a.nifi.web.api.config.ThrowableMapper An unexpected error has > occurred: java.lang.NullPointerException. Returning Internal Server > Error response. > java.lang.NullPointerException: null > at > org.apache.nifi.processors.standard.InvokeHTTP.onPropertyModified(InvokeHTTP.java:121) > ~[na:na] > at > org.apache.nifi.controller.AbstractConfiguredComponent.removeProperty(AbstractConfiguredComponent.java:163) > ~[nifi-framework-core-api-0.3.1-SNAPSHOT.jar:0.3.1-SNAPSHOT] > at > org.apache.nifi.web.dao.impl.StandardProcessorDAO.configureProcessor(StandardProcessorDAO.java:174) > ~[classes/:na] > at > org.apache.nifi.web.dao.impl.StandardProcessorDAO.updateProcessor(StandardProcessorDAO.java:391) > ~[classes/:na] > at > org.apache.nifi.web.dao.impl.StandardProcessorDAO$$FastClassBySpringCGLIB$$779e089b.invoke() > ~[spring-core-4.1.6.RELEASE.jar:na] > at org.springframework.cglib.pr
[GitHub] nifi pull request #1477: NIFI-1125 InvokeHTTP throws NullPointerException
GitHub user ijokarumawak opened a pull request: https://github.com/apache/nifi/pull/1477 NIFI-1125 InvokeHTTP throws NullPointerException Added null check in onPropertyModified to avoid NPE. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijokarumawak/nifi nifi-1125 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1477.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1477 commit 3120cd20e4a8fe911c7e61ed96e10519d647be74 Author: Koji Kawamura Date: 2017-02-07T02:08:06Z NIFI-1125 InvokeHTTP throws NullPointerException Added null check in onPropertyModified to avoid NPE. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3162) RPG proxy and Remote Group Port configuration changes should be audited
[ https://issues.apache.org/jira/browse/NIFI-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15855128#comment-15855128 ] Koji Kawamura commented on NIFI-3162: - For the issue of deleted component, I created another JIRA NIFI-3445 to track progress separately. This JIRA is solely for fixing RPG and RPG Port configuration audits. > RPG proxy and Remote Group Port configuration changes should be audited > --- > > Key: NIFI-3162 > URL: https://issues.apache.org/jira/browse/NIFI-3162 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Koji Kawamura > > Since NiFi 1.0.0, several configurations have been added to > RemoteProcessGroup such as Transport Protocol and Proxy settings. > Currently, configuration updates against these new settings are not audited. > In addition to these RemoteProcessGroup settings, RemoteProcessGroupPort > will have new configurations by NIFI-1202. > This JIRA ticket tracks the work of adding audit events for these new > settings. > h3. Investigate current behavior > In order to fix properly, I've tested to know what works and what doesn't. > Here is the result of current behavior (measured with 1.2.0-SNAPSHOT, but it > should be the same for versions after 1.0). > || Operation performed || Created Audit Type || Audited Operation || Need > fix? || > | Create RPG | RPG | Add | No | > | Enable transmission | RPG | Start | No | > | Disable transmission | RPG | Stop | No | > | Delete RPG | RPG? | Remove? | Different issue | > | Edit RPG config | RPG | Configure | Yes fix-1 | > | Enable/Disable individual remote port | (none) | (none) | Yes fix-2 | > | Edit individual remote port config | (none) | (none) | Yes fix-3 | > h3. Fix-1: Edit RPG config > Currently, this is partially audited for 'Communications Timeout' and 'Yield > Duration'. > We need to track edit for these as well: Transport Protocol, HTTP Proxy > Server Hostname, Port, User, Password > h3. Fix-2: Enable/Disable individual remote port > From 'Remote ports' context menu of a RPG, each remote port can be > enabled/disabled individually. Currently this operation is not audited. > There're two ways to fix this, one is using the existing REMOTE_PROCESS_GROUP > table in the nifi-flow-audit h2 database. Use 'Configure' Operation type with > 'Name=.transmission' and 'Value=enabled/disabled'. > Another way is to create a REMOTE_PROCESS_GROUP_PORT table, but this requires > some additional migration code to create the table. > I think the former approach is reasonable. > h3. Fix-3: Edit individual remote port config > Currently, user can configure 'Concurrent Tasks' and 'Compressed', but no > audit event is created for these operations. Same as Fix-2, we might be able > to use existing REMOTE_PROCESS_GROUP table, using 'Configure' Operation type > with 'Name=.' dot notation and > 'Value='. > h3. NiFi history shows audit record as 'Not authorized' if the component has > been removed > Once the target component (i.e RemoteProcessGroup) is removed from a flow, > its audit records as shown as 'Not authorized'. This can be problematic since > user won't be able to know who deleted the component. Created NIFI-3445 for > tracking this issue separately. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3162) RPG proxy and Remote Group Port configuration changes should be audited
[ https://issues.apache.org/jira/browse/NIFI-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-3162: Description: Since NiFi 1.0.0, several configurations have been added to RemoteProcessGroup such as Transport Protocol and Proxy settings. Currently, configuration updates against these new settings are not audited. In addition to these RemoteProcessGroup settings, RemoteProcessGroupPort will have new configurations by NIFI-1202. This JIRA ticket tracks the work of adding audit events for these new settings. h3. Investigate current behavior In order to fix properly, I've tested to know what works and what doesn't. Here is the result of current behavior (measured with 1.2.0-SNAPSHOT, but it should be the same for versions after 1.0). || Operation performed || Created Audit Type || Audited Operation || Need fix? || | Create RPG | RPG | Add | No | | Enable transmission | RPG | Start | No | | Disable transmission | RPG | Stop | No | | Delete RPG | RPG? | Remove? | Different issue | | Edit RPG config | RPG | Configure | Yes fix-1 | | Enable/Disable individual remote port | (none) | (none) | Yes fix-2 | | Edit individual remote port config | (none) | (none) | Yes fix-3 | h3. Fix-1: Edit RPG config Currently, this is partially audited for 'Communications Timeout' and 'Yield Duration'. We need to track edit for these as well: Transport Protocol, HTTP Proxy Server Hostname, Port, User, Password h3. Fix-2: Enable/Disable individual remote port >From 'Remote ports' context menu of a RPG, each remote port can be >enabled/disabled individually. Currently this operation is not audited. There're two ways to fix this, one is using the existing REMOTE_PROCESS_GROUP table in the nifi-flow-audit h2 database. Use 'Configure' Operation type with 'Name=.transmission' and 'Value=enabled/disabled'. Another way is to create a REMOTE_PROCESS_GROUP_PORT table, but this requires some additional migration code to create the table. I think the former approach is reasonable. h3. Fix-3: Edit individual remote port config Currently, user can configure 'Concurrent Tasks' and 'Compressed', but no audit event is created for these operations. Same as Fix-2, we might be able to use existing REMOTE_PROCESS_GROUP table, using 'Configure' Operation type with 'Name=.' dot notation and 'Value='. h3. NiFi history shows audit record as 'Not authorized' if the component has been removed Once the target component (i.e RemoteProcessGroup) is removed from a flow, its audit records as shown as 'Not authorized'. This can be problematic since user won't be able to know who deleted the component. Created NIFI-3445 for tracking this issue separately. was: Since NiFi 1.0.0, several configurations have been added to RemoteProcessGroup such as Transport Protocol and Proxy settings. Currently, configuration updates against these new settings are not audited. In addition to these RemoteProcessGroup settings, RemoteProcessGroupPort will have new configurations by NIFI-1202. This JIRA ticket tracks the work of adding audit events for these new settings. h3. Investigate current behavior In order to fix properly, I've tested to know what works and what doesn't. Here is the result of current behavior (measured with 1.2.0-SNAPSHOT, but it should be the same for versions after 1.0). || Operation performed || Created Audit Type || Audited Operation || Need fix? || | Create RPG | RPG | Add | No | | Enable transmission | RPG | Start | No | | Disable transmission | RPG | Stop | No | | Delete RPG | RPG? | Remove? | Different issue | | Edit RPG config | RPG | Configure | Yes fix-1 | | Enable/Disable individual remote port | (none) | (none) | Yes fix-2 | | Edit individual remote port config | (none) | (none) | Yes fix-3 | h3. Fix-1: Edit RPG config Currently, this is partially audited for 'Communications Timeout' and 'Yield Duration'. We need to track edit for these as well: Transport Protocol, HTTP Proxy Server Hostname, Port, User, Password h3. Fix-2: Enable/Disable individual remote port >From 'Remote ports' context menu of a RPG, each remote port can be >enabled/disabled individually. Currently this operation is not audited. There're two ways to fix this, one is using the existing REMOTE_PROCESS_GROUP table in the nifi-flow-audit h2 database. Use 'Configure' Operation type with 'Name=.transmission' and 'Value=enabled/disabled'. Another way is to create a REMOTE_PROCESS_GROUP_PORT table, but this requires some additional migration code to create the table. I think the former approach is reasonable. h3. Fix-3: Edit individual remote port config Currently, user can configure 'Concurrent Tasks' and 'Compressed', but no audit event is created for these operations. Same as Fix-2, we might be able to use existing REMOTE_PROCESS_GROUP table, using 'Configure' Operation type with 'Name=.' dot notation and 'Value='. h3. Ni
[jira] [Created] (NIFI-3445) Show more meaningful audit events for deleted components
Koji Kawamura created NIFI-3445: --- Summary: Show more meaningful audit events for deleted components Key: NIFI-3445 URL: https://issues.apache.org/jira/browse/NIFI-3445 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Core UI Affects Versions: 1.0.0 Reporter: Koji Kawamura Once a target component such as RemoteProcessGroup or Processor is removed from a flow, its audit records as shown as 'Not authorized'. This can be problematic since users won't be able to know who deleted the component. This is because we no longer have the policies for that component so we don't know who is allowed to read details about it. Possible solutions for changing this behavior are below: A. Retain all policies forever (or at least for some longer period of time when we would need to know about some action). B. Allow some events to always be accessible. Like the fact that an user deleted a component but just filter out any details about it when the user is unauthorized to read it or the component and its policies have been removed. Accessing provenance has similar issue. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3162) RPG proxy and Remote Group Port configuration changes should be audited
[ https://issues.apache.org/jira/browse/NIFI-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-3162: Status: Patch Available (was: In Progress) > RPG proxy and Remote Group Port configuration changes should be audited > --- > > Key: NIFI-3162 > URL: https://issues.apache.org/jira/browse/NIFI-3162 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Koji Kawamura > > Since NiFi 1.0.0, several configurations have been added to > RemoteProcessGroup such as Transport Protocol and Proxy settings. > Currently, configuration updates against these new settings are not audited. > In addition to these RemoteProcessGroup settings, RemoteProcessGroupPort > will have new configurations by NIFI-1202. > This JIRA ticket tracks the work of adding audit events for these new > settings. > h3. Investigate current behavior > In order to fix properly, I've tested to know what works and what doesn't. > Here is the result of current behavior (measured with 1.2.0-SNAPSHOT, but it > should be the same for versions after 1.0). > || Operation performed || Created Audit Type || Audited Operation || Need > fix? || > | Create RPG | RPG | Add | No | > | Enable transmission | RPG | Start | No | > | Disable transmission | RPG | Stop | No | > | Delete RPG | RPG? | Remove? | Different issue | > | Edit RPG config | RPG | Configure | Yes fix-1 | > | Enable/Disable individual remote port | (none) | (none) | Yes fix-2 | > | Edit individual remote port config | (none) | (none) | Yes fix-3 | > h3. Fix-1: Edit RPG config > Currently, this is partially audited for 'Communications Timeout' and 'Yield > Duration'. > We need to track edit for these as well: Transport Protocol, HTTP Proxy > Server Hostname, Port, User, Password > h3. Fix-2: Enable/Disable individual remote port > From 'Remote ports' context menu of a RPG, each remote port can be > enabled/disabled individually. Currently this operation is not audited. > There're two ways to fix this, one is using the existing REMOTE_PROCESS_GROUP > table in the nifi-flow-audit h2 database. Use 'Configure' Operation type with > 'Name=.transmission' and 'Value=enabled/disabled'. > Another way is to create a REMOTE_PROCESS_GROUP_PORT table, but this requires > some additional migration code to create the table. > I think the former approach is reasonable. > h3. Fix-3: Edit individual remote port config > Currently, user can configure 'Concurrent Tasks' and 'Compressed', but no > audit event is created for these operations. Same as Fix-2, we might be able > to use existing REMOTE_PROCESS_GROUP table, using 'Configure' Operation type > with 'Name=.' dot notation and > 'Value='. > h3. NiFi history shows audit record as 'Not authorized' if the component has > been removed > Once the target component (i.e RemoteProcessGroup) is removed from a flow, > its audit records as shown as 'Not authorized'. This can be problematic since > user won't be able to know who deleted the component. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3162) RPG proxy and Remote Group Port configuration changes should be audited
[ https://issues.apache.org/jira/browse/NIFI-3162?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15855104#comment-15855104 ] ASF GitHub Bot commented on NIFI-3162: -- GitHub user ijokarumawak opened a pull request: https://github.com/apache/nifi/pull/1476 NIFI-3162: Audit RPG and RPG port config changes - Added configure audits for Transport Protocol, HTTP Proxy Server Host, Port, User and Password in RemoteProcessGroup configuration - Added configure audits for enabling/disabling individual remote port - Added configure audits for Concurrent Tasks and Compressed in Remote Port configuration Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijokarumawak/nifi nifi-3162 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1476.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1476 commit 5ec7c64706c5891e56d92ea5dac9b9120bfc426b Author: Koji Kawamura Date: 2017-02-07T01:11:30Z NIFI-3162: Audit RPG and RPG port config changes - Added configure audits for Transport Protocol, HTTP Proxy Server Host, Port, User and Password in RemoteProcessGroup configuration - Added configure audits for enabling/disabling individual remote port - Added configure audits for Concurrent Tasks and Compressed in Remote Port configuration > RPG proxy and Remote Group Port configuration changes should be audited > --- > > Key: NIFI-3162 > URL: https://issues.apache.org/jira/browse/NIFI-3162 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Koji Kawamura >Assignee: Koji Kawamura > > Since NiFi 1.0.0, several configurations have been added to > RemoteProcessGroup such as Transport Protocol and Proxy settings. > Currently, configuration updates against these new settings are not audited. > In addition to these RemoteProcessGroup settings, RemoteProcessGroupPort > will have new configurations by NIFI-1202. > This JIRA ticket tracks the work of adding audit events for these new > settings. > h3. Investigate current behavior > In order to fix properly, I've tested to know what works and what doesn't. > Here is the result of current behavior (measured with 1.2.0-SNAPSHOT, but it > should be the same for versions after 1.0). > || Operation performed || Created Audit Type || Audited Operation || Need > fix? || > | Create RPG | RPG | Add | No | > | Enable transmission | RPG | Start | No | > | Disable transmission | RPG | Stop | No | > | Delete RPG | RPG? | Remove? | Different issue | > | Edit RPG config | RPG | Configure | Yes fix-1 | > | Enable/Disable individual remote port | (none) | (none) | Yes fix-2 | > | Edit individual remote port config | (none) | (none) | Yes fix-3 | > h3. Fix-1: Edit R
[GitHub] nifi pull request #1476: NIFI-3162: Audit RPG and RPG port config changes
GitHub user ijokarumawak opened a pull request: https://github.com/apache/nifi/pull/1476 NIFI-3162: Audit RPG and RPG port config changes - Added configure audits for Transport Protocol, HTTP Proxy Server Host, Port, User and Password in RemoteProcessGroup configuration - Added configure audits for enabling/disabling individual remote port - Added configure audits for Concurrent Tasks and Compressed in Remote Port configuration Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x] Is your initial contribution a single, squashed commit? ### For code changes: - [x] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijokarumawak/nifi nifi-3162 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1476.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1476 commit 5ec7c64706c5891e56d92ea5dac9b9120bfc426b Author: Koji Kawamura Date: 2017-02-07T01:11:30Z NIFI-3162: Audit RPG and RPG port config changes - Added configure audits for Transport Protocol, HTTP Proxy Server Host, Port, User and Password in RemoteProcessGroup configuration - Added configure audits for enabling/disabling individual remote port - Added configure audits for Concurrent Tasks and Compressed in Remote Port configuration --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1450: NIFI-3339b Add getDataSource() to DBCPService, second vers...
Github user ijokarumawak commented on the issue: https://github.com/apache/nifi/pull/1450 @mattyb149 Thanks for sharing your use case. That's interesting. Why does your processor need DBCPService's URL? For provenance? If so, I'd prefer creating another interface looks like below: ```java public interface ExternalSource { String getTransitUri(); } ``` Then add the interface to ControllerServices that interact with external source such as DBCPService, so that processors use those controller services can create more meaningful provenance events. For DBCPService, DatabaseConnectionURL would be legitimate for a return value of getTransitUri method. Or, if you need to use the same configuration among ControllerService and Processor, then VariableRegistry can be helpful. How do you think? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #46: MINIFI-188: Incorporate CATCH and example ...
Github user asfgit closed the pull request at: https://github.com/apache/nifi-minifi-cpp/pull/46 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #46: MINIFI-188: Incorporate CATCH and example tests
Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/46 Thanks for taking care of licensing. Will get this merged in! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-3213) ListFile always skips files with the latest timestamp in an iteration even if the files have existed a while ago
[ https://issues.apache.org/jira/browse/NIFI-3213?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-3213: Status: Patch Available (was: Open) > ListFile always skips files with the latest timestamp in an iteration even if > the files have existed a while ago > > > Key: NIFI-3213 > URL: https://issues.apache.org/jira/browse/NIFI-3213 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 0.7.1, 1.1.0, 0.6.1, 0.7.0, 0.5.1, 0.6.0, 0.5.0, 1.0.0 >Reporter: Koji Kawamura >Assignee: Koji Kawamura > > NIFI-1484 add few lines of code to avoid files to be emitted if those have > the latest timestamp within an iteration of listing, because it may still be > written at the same time. > While it doesn't affect much if ListFiles processor is scheduled with a short > period of time, such as few ms, but it does affect negatively if an user > scheduled it with longer run schedule such as "1 day" or with cron scheduler. > For example, user would expect to process list of files per daily basis. Even > if a file is saved few hours ago, the processor will skip this, because the > file has the latest timestamp within the iteration. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3429) ConvertJSONToSQL Does not work with table identifiers that need to be quoted
[ https://issues.apache.org/jira/browse/NIFI-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854883#comment-15854883 ] ASF GitHub Bot commented on NIFI-3429: -- Github user nickcarenza commented on a diff in the pull request: https://github.com/apache/nifi/pull/1464#discussion_r99702931 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ConvertJSONToSQL.java --- @@ -500,7 +521,11 @@ private String generateUpdate(final JsonNode rootNode, final Map final StringBuilder sqlBuilder = new StringBuilder(); int fieldCount = 0; -sqlBuilder.append("UPDATE ").append(tableName).append(" SET "); +sqlBuilder.append("UPDATE "); --- End diff -- @pvillard31 i think i got everything this time > ConvertJSONToSQL Does not work with table identifiers that need to be quoted > > > Key: NIFI-3429 > URL: https://issues.apache.org/jira/browse/NIFI-3429 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Nicholas Carenza >Priority: Minor > Fix For: 1.2.0 > > > I have a table identifer that needs to be quoted. If i supple the table > identifer unquoted in the properties, the processor can convert the json to > sql but the output sql doesn't have the table name quoted even if I select > the option Quote Identifiers because, true to the description, it only > applies to column identifiers. If I supply the table identifier quoted in the > properties, then the processor fails completely because it can't find the > table. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1464: NIFI-3429 optionally quoting all table identifiers
Github user nickcarenza commented on a diff in the pull request: https://github.com/apache/nifi/pull/1464#discussion_r99702931 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ConvertJSONToSQL.java --- @@ -500,7 +521,11 @@ private String generateUpdate(final JsonNode rootNode, final Map final StringBuilder sqlBuilder = new StringBuilder(); int fieldCount = 0; -sqlBuilder.append("UPDATE ").append(tableName).append(" SET "); +sqlBuilder.append("UPDATE "); --- End diff -- @pvillard31 i think i got everything this time --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-55) ListenHTTP shoudl log who the sender of a FlowFile bundle is if the HOLD expires
[ https://issues.apache.org/jira/browse/NIFI-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854849#comment-15854849 ] Joseph Witt commented on NIFI-55: - [~trixpan] i am staring to wonder if you ever sleep! We spoke a few hours ago and you said it was 2am. Now its not much later. Sleep is good my friend. Anyway, the motivation behind this is a busted producer basically. If we're not tracking who/what is putting these flakey holds in place we cannot help much with troubleshooting. But if we know then we can track/log/bulletin that and help someone pinpoint the offending sender. > ListenHTTP shoudl log who the sender of a FlowFile bundle is if the HOLD > expires > > > Key: NIFI-55 > URL: https://issues.apache.org/jira/browse/NIFI-55 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Joseph Witt >Assignee: Andre >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-55) ListenHTTP shoudl log who the sender of a FlowFile bundle is if the HOLD expires
[ https://issues.apache.org/jira/browse/NIFI-55?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854837#comment-15854837 ] Andre commented on NIFI-55: --- [~joewitt] would you mind explain a bit more what you had in mind when you put this in place? > ListenHTTP shoudl log who the sender of a FlowFile bundle is if the HOLD > expires > > > Key: NIFI-55 > URL: https://issues.apache.org/jira/browse/NIFI-55 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Joseph Witt >Assignee: Andre >Priority: Minor > -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3442) FTPTransfer tries to evaluate PORT as expression language, although property doesn't support EL
[ https://issues.apache.org/jira/browse/NIFI-3442?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andre updated NIFI-3442: Status: Patch Available (was: Open) > FTPTransfer tries to evaluate PORT as expression language, although property > doesn't support EL > --- > > Key: NIFI-3442 > URL: https://issues.apache.org/jira/browse/NIFI-3442 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.1.1, 0.7.1 >Reporter: Andre >Assignee: Andre > > while crafting test coverage for PutFTP I noticed the following error: > java.lang.AssertionError: java.lang.IllegalStateException: Attempting to > Evaluate Expressions but PropertyDescriptor[Port] indicates that the > Expression Language is not supported. If you realize that this is the case > and do not want this error to occur, it can be disabled by calling > TestRunner.setValidateExpressionUsage(false) > Root cause seems to be a disconnect between the property builder (EL = false) > and the attempt to evaluate the property as part of client.connect -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3434) nifi-assembly dependencies should allow users to skip the creation of ZIP and TAR.GZ packages
[ https://issues.apache.org/jira/browse/NIFI-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andre updated NIFI-3434: Status: Patch Available (was: Open) > nifi-assembly dependencies should allow users to skip the creation of ZIP and > TAR.GZ packages > - > > Key: NIFI-3434 > URL: https://issues.apache.org/jira/browse/NIFI-3434 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.1.1 >Reporter: Andre >Assignee: Andre > > devs, > Currently calling 'mvn clean install' creates a ZIP, a TAR.GZ and a > directory containing the same code. This leads to wasted disk space and a > lot of wasted disk writes (something that a lot of folks using SSDs prefer > to avoid). > Would anyone oppose the idea of moving the ZIP and TAR.GZ assemblies into a > "release" profile (or whatever name we agree to). This way we could > maintain the directory "format" (which I suspect most of us use during > development), while still providing a convenient way of creating the ZIP > and TAR.GZ packages. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3434) nifi-assembly dependencies should allow users to skip the creation of ZIP and TAR.GZ packages
[ https://issues.apache.org/jira/browse/NIFI-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854832#comment-15854832 ] ASF GitHub Bot commented on NIFI-3434: -- Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1472 @pvillard31 @apiri done. Please note I added a second commit as this helps keeping track of why the travis file was changed. > nifi-assembly dependencies should allow users to skip the creation of ZIP and > TAR.GZ packages > - > > Key: NIFI-3434 > URL: https://issues.apache.org/jira/browse/NIFI-3434 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.1.1 >Reporter: Andre >Assignee: Andre > > devs, > Currently calling 'mvn clean install' creates a ZIP, a TAR.GZ and a > directory containing the same code. This leads to wasted disk space and a > lot of wasted disk writes (something that a lot of folks using SSDs prefer > to avoid). > Would anyone oppose the idea of moving the ZIP and TAR.GZ assemblies into a > "release" profile (or whatever name we agree to). This way we could > maintain the directory "format" (which I suspect most of us use during > development), while still providing a convenient way of creating the ZIP > and TAR.GZ packages. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1472: NIFI-3434 - Introduce the ability to skip the packaging of...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1472 @pvillard31 @apiri done. Please note I added a second commit as this helps keeping track of why the travis file was changed. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #46: MINIFI-188: Incorporate CATCH and example tests
Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/46 @phrocker Thanks for cleaning up the typos. The changes all look good and builds are working on Linux for me. Boost license is perfectly fine as per http://www.apache.org/legal/resolved.html#category-a To that end, and apologize for overlooking this initially, but we will need to update the project LICENSE as we are including the source as part of the codebase. After that, I believe we should be good for merging. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #46: MINIFI-188: Incorporate CATCH and example ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/46#discussion_r99691327 --- Diff: libminifi/cmake/FindLeveldb.cmake --- @@ -0,0 +1,50 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless qrequired by applicable law or agreed to in writing, --- End diff -- I updated the commit to include a blurb about it so that it was squashed into a single commit. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #46: MINIFI-188: Incorporate CATCH and example ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/46#discussion_r99691182 --- Diff: libminifi/cmake/FindLeveldb.cmake --- @@ -0,0 +1,50 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless qrequired by applicable law or agreed to in writing, --- End diff -- Oh I already force pushed the change. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #46: MINIFI-188: Incorporate CATCH and example ...
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/46#discussion_r99690213 --- Diff: libminifi/cmake/FindLeveldb.cmake --- @@ -0,0 +1,50 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless qrequired by applicable law or agreed to in writing, --- End diff -- Actually, I can just take care of those items as part of the merge. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2809) Add processors for Google Cloud Storage Fetch/Put/Delete
[ https://issues.apache.org/jira/browse/NIFI-2809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854756#comment-15854756 ] Frank Maritato commented on NIFI-2809: -- [~philpot] Hi Andrew, I haven't had a chance to address the concerns in the last comment. Hopefully I can get to it this week and I will submit another patch. > Add processors for Google Cloud Storage Fetch/Put/Delete > > > Key: NIFI-2809 > URL: https://issues.apache.org/jira/browse/NIFI-2809 > Project: Apache NiFi > Issue Type: New Feature >Affects Versions: 1.1.0 >Reporter: Frank Maritato > Attachments: gcp_bundle.patch, gcs_bundle.patch, nifi-2809.patch > > > Creating a ticket to add nifi processors for Google Cloud Storage similar to > how the AWS S3 processors are done: > * FetchGCSObject > * PutGCSObject > * DeleteGCSObject > I have the code mostly written and will attach the patch file when complete. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi-minifi-cpp pull request #46: MINIFI-188: Incorporate CATCH and example ...
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/46#discussion_r99686329 --- Diff: libminifi/cmake/FindLeveldb.cmake --- @@ -0,0 +1,50 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless qrequired by applicable law or agreed to in writing, --- End diff -- Ah, gotcha. Good with this one or a follow-on. Your call! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #46: MINIFI-188: Incorporate CATCH and example ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/46#discussion_r99686039 --- Diff: libminifi/cmake/FindLeveldb.cmake --- @@ -0,0 +1,50 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless qrequired by applicable law or agreed to in writing, --- End diff -- I see this same typo in other files. I'll make that change in either this PR or the next. Apparently I propagated the typo. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp issue #46: MINIFI-188: Incorporate CATCH and example tests
Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/46 reviewing updates --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3429) ConvertJSONToSQL Does not work with table identifiers that need to be quoted
[ https://issues.apache.org/jira/browse/NIFI-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854727#comment-15854727 ] ASF GitHub Bot commented on NIFI-3429: -- Github user nickcarenza commented on a diff in the pull request: https://github.com/apache/nifi/pull/1464#discussion_r99683156 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ConvertJSONToSQL.java --- @@ -500,7 +521,11 @@ private String generateUpdate(final JsonNode rootNode, final Map final StringBuilder sqlBuilder = new StringBuilder(); int fieldCount = 0; -sqlBuilder.append("UPDATE ").append(tableName).append(" SET "); +sqlBuilder.append("UPDATE "); --- End diff -- ah yes, of course > ConvertJSONToSQL Does not work with table identifiers that need to be quoted > > > Key: NIFI-3429 > URL: https://issues.apache.org/jira/browse/NIFI-3429 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Nicholas Carenza >Priority: Minor > Fix For: 1.2.0 > > > I have a table identifer that needs to be quoted. If i supple the table > identifer unquoted in the properties, the processor can convert the json to > sql but the output sql doesn't have the table name quoted even if I select > the option Quote Identifiers because, true to the description, it only > applies to column identifiers. If I supply the table identifier quoted in the > properties, then the processor fails completely because it can't find the > table. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1464: NIFI-3429 optionally quoting all table identifiers
Github user nickcarenza commented on a diff in the pull request: https://github.com/apache/nifi/pull/1464#discussion_r99683156 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ConvertJSONToSQL.java --- @@ -500,7 +521,11 @@ private String generateUpdate(final JsonNode rootNode, final Map final StringBuilder sqlBuilder = new StringBuilder(); int fieldCount = 0; -sqlBuilder.append("UPDATE ").append(tableName).append(" SET "); +sqlBuilder.append("UPDATE "); --- End diff -- ah yes, of course --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3429) ConvertJSONToSQL Does not work with table identifiers that need to be quoted
[ https://issues.apache.org/jira/browse/NIFI-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854714#comment-15854714 ] ASF GitHub Bot commented on NIFI-3429: -- Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1464#discussion_r99680121 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ConvertJSONToSQL.java --- @@ -500,7 +521,11 @@ private String generateUpdate(final JsonNode rootNode, final Map final StringBuilder sqlBuilder = new StringBuilder(); int fieldCount = 0; -sqlBuilder.append("UPDATE ").append(tableName).append(" SET "); +sqlBuilder.append("UPDATE "); --- End diff -- Same comment applies here ;) And for columns quotes as well. > ConvertJSONToSQL Does not work with table identifiers that need to be quoted > > > Key: NIFI-3429 > URL: https://issues.apache.org/jira/browse/NIFI-3429 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Nicholas Carenza >Priority: Minor > Fix For: 1.2.0 > > > I have a table identifer that needs to be quoted. If i supple the table > identifer unquoted in the properties, the processor can convert the json to > sql but the output sql doesn't have the table name quoted even if I select > the option Quote Identifiers because, true to the description, it only > applies to column identifiers. If I supply the table identifier quoted in the > properties, then the processor fails completely because it can't find the > table. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1464: NIFI-3429 optionally quoting all table identifiers
Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1464#discussion_r99680121 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ConvertJSONToSQL.java --- @@ -500,7 +521,11 @@ private String generateUpdate(final JsonNode rootNode, final Map final StringBuilder sqlBuilder = new StringBuilder(); int fieldCount = 0; -sqlBuilder.append("UPDATE ").append(tableName).append(" SET "); +sqlBuilder.append("UPDATE "); --- End diff -- Same comment applies here ;) And for columns quotes as well. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3429) ConvertJSONToSQL Does not work with table identifiers that need to be quoted
[ https://issues.apache.org/jira/browse/NIFI-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854696#comment-15854696 ] ASF GitHub Bot commented on NIFI-3429: -- Github user nickcarenza commented on the issue: https://github.com/apache/nifi/pull/1464 @pvillard31 thanks for the review! Changes applied. > ConvertJSONToSQL Does not work with table identifiers that need to be quoted > > > Key: NIFI-3429 > URL: https://issues.apache.org/jira/browse/NIFI-3429 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Nicholas Carenza >Priority: Minor > Fix For: 1.2.0 > > > I have a table identifer that needs to be quoted. If i supple the table > identifer unquoted in the properties, the processor can convert the json to > sql but the output sql doesn't have the table name quoted even if I select > the option Quote Identifiers because, true to the description, it only > applies to column identifiers. If I supply the table identifier quoted in the > properties, then the processor fails completely because it can't find the > table. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1464: NIFI-3429 optionally quoting all table identifiers
Github user nickcarenza commented on the issue: https://github.com/apache/nifi/pull/1464 @pvillard31 thanks for the review! Changes applied. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Assigned] (NIFI-3362) Update FlowConfiguration.xsd TimePeriod to match FormatUtils
[ https://issues.apache.org/jira/browse/NIFI-3362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Moser reassigned NIFI-3362: --- Assignee: Michael Moser > Update FlowConfiguration.xsd TimePeriod to match FormatUtils > > > Key: NIFI-3362 > URL: https://issues.apache.org/jira/browse/NIFI-3362 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 0.7.1, 1.1.1 >Reporter: Michael Moser >Assignee: Michael Moser >Priority: Minor > > The framework FormatUtils time duration parser has been updated to support > new terms such as "weeks". But when using those new terms, we get ERROR > messages in the nifi-bootstrap.log. > cvc-pattern-valid: Value '4 weeks' is not facet-valid with respect to pattern > '\d+\s*(ns|nano|nanos|nanoseconds|ms|milli|millis|milliseconds|s|sec|secs|seconds|m|min|mins|minutes|h|hr|hrs|hours|d|day|days)' > for type 'TimePeriod'. > cvc-type.3.1.3: The value '4 weeks' of element 'flowFileExpiration' is not > valid. > FlowConfiguration.xsd TimePeriod pattern should be updated to match > FormatUtils. > Add to FlowConguration.xsd: > nanosecond > millisecond > second > minute > hour > w > wk > wks > week > weeks > Add to FormatUtils.java > nanosecond > millisecond -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3004) Improve StandardSSLContextService customValidate
[ https://issues.apache.org/jira/browse/NIFI-3004?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Moser updated NIFI-3004: Fix Version/s: 1.2.0 > Improve StandardSSLContextService customValidate > > > Key: NIFI-3004 > URL: https://issues.apache.org/jira/browse/NIFI-3004 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 >Reporter: Michael Moser >Assignee: Pierre Villard >Priority: Minor > Labels: newbie > Fix For: 1.2.0 > > > The StandardSSLContextService customValidate() method can be called many > times in one validation cycle, so it must be efficient. Currently, it calls > validateStore() on keystore and truststore, which opens each file and > verifies it can be read and the password is correct. If that works then it > calls verifySslConfig() which creates 3 sample SSLContext objects. > Improve this by remembering when customValidate() has been successful and > avoiding doing another full validation either for a very small amount of time > or until one of the Property values changes. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3429) ConvertJSONToSQL Does not work with table identifiers that need to be quoted
[ https://issues.apache.org/jira/browse/NIFI-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854600#comment-15854600 ] ASF GitHub Bot commented on NIFI-3429: -- Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1464#discussion_r99665513 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ConvertJSONToSQL.java --- @@ -420,7 +433,11 @@ private String generateInsert(final JsonNode rootNode, final Map final StringBuilder sqlBuilder = new StringBuilder(); int fieldCount = 0; -sqlBuilder.append("INSERT INTO ").append(tableName).append(" ("); +sqlBuilder.append("INSERT INTO "); +if (quoteTableName) sqlBuilder.append(schema.getQuotedIdentifierString()); --- End diff -- Quick remark: could you change to do it similarly to column names? Something like: java if(quoteTableName){ sqlBuilder.append(schema.getQuotedIdentifierString()) .append(tableName) .append(schema.getQuotedIdentifierString()); } else { sqlBuilder.append(tableName); } This way you avoid two if evaluations. Also, since you are going to make a change, could you update this same piece of code for column names to avoid the negation check in the if statement? To have: java if(escapeColumnNames){ sqlBuilder.append(schema.getQuotedIdentifierString()) .append(normalizedColName) .append(schema.getQuotedIdentifierString()); } else { sqlBuilder.append(normalizedColName); } I believe this is more efficient. Otherwise, LGTM. > ConvertJSONToSQL Does not work with table identifiers that need to be quoted > > > Key: NIFI-3429 > URL: https://issues.apache.org/jira/browse/NIFI-3429 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Nicholas Carenza >Priority: Minor > Fix For: 1.2.0 > > > I have a table identifer that needs to be quoted. If i supple the table > identifer unquoted in the properties, the processor can convert the json to > sql but the output sql doesn't have the table name quoted even if I select > the option Quote Identifiers because, true to the description, it only > applies to column identifiers. If I supply the table identifier quoted in the > properties, then the processor fails completely because it can't find the > table. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1464: NIFI-3429 automatically quoting all table identifie...
Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1464#discussion_r99665513 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ConvertJSONToSQL.java --- @@ -420,7 +433,11 @@ private String generateInsert(final JsonNode rootNode, final Map final StringBuilder sqlBuilder = new StringBuilder(); int fieldCount = 0; -sqlBuilder.append("INSERT INTO ").append(tableName).append(" ("); +sqlBuilder.append("INSERT INTO "); +if (quoteTableName) sqlBuilder.append(schema.getQuotedIdentifierString()); --- End diff -- Quick remark: could you change to do it similarly to column names? Something like: java if(quoteTableName){ sqlBuilder.append(schema.getQuotedIdentifierString()) .append(tableName) .append(schema.getQuotedIdentifierString()); } else { sqlBuilder.append(tableName); } This way you avoid two if evaluations. Also, since you are going to make a change, could you update this same piece of code for column names to avoid the negation check in the if statement? To have: java if(escapeColumnNames){ sqlBuilder.append(schema.getQuotedIdentifierString()) .append(normalizedColName) .append(schema.getQuotedIdentifierString()); } else { sqlBuilder.append(normalizedColName); } I believe this is more efficient. Otherwise, LGTM. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (NIFI-3444) PutKenesisFirehose Optional message demarcator
Nicholas Carenza created NIFI-3444: -- Summary: PutKenesisFirehose Optional message demarcator Key: NIFI-3444 URL: https://issues.apache.org/jira/browse/NIFI-3444 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Nicholas Carenza Priority: Trivial When using PutKenesisFirehose, flowfile content will be written as-is byte-for-byte to the firehose stream with no demarcation. For me at least, I expected there to be new lines in between but I can certainly understand this not being the default. I have seen many processors with options for message demarcating. Is that an option others might find useful on this processor and does it belong there? For now, I am using ReplaceText with '$' as the search and '' as the replace. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3055) StandardRecordWriter can throw UTFDataFormatException
[ https://issues.apache.org/jira/browse/NIFI-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854545#comment-15854545 ] ASF GitHub Bot commented on NIFI-3055: -- GitHub user jskora opened a pull request: https://github.com/apache/nifi/pull/1475 NIFI-3055 StandardRecordWriter Can Throw UTFDataFormatException (1.x) * Remove function based on JDK source. * Add new function to find bytes based on RFC3629. * Add field name to log entry when field is truncated. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x ] Is your initial contribution a single, squashed commit? ### For code changes: - [x ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jskora/nifi NIFI-3055-1.x-v4 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1475.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1475 commit 65310351bf07c410544c1d8e1acbba96de649c95 Author: Joe Skora Date: 2017-02-06T18:55:01Z NIFI-3055 StandardRecordWriter Can Throw UTFDataFormatException (1.x) * Remove function based on JDK source. * Add new function to find bytes based on RFC3629. * Add field name to log entry when field is truncated. > StandardRecordWriter can throw UTFDataFormatException > - > > Key: NIFI-3055 > URL: https://issues.apache.org/jira/browse/NIFI-3055 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.0.0, 0.7.1 >Reporter: Brandon DeVries >Assignee: Joe Skora >Priority: Blocker > Fix For: 0.8.0, 1.2.0 > > > StandardRecordWriter.writeRecord()\[1] uses DataOutputStream.writeUTF()\[2] > without checking the length of the value to be written. If this length is > greater than 65535 (2^16 - 1), you get a UTFDataFormatException "encoded > string too long..."\[3]. Ultimately, this can result in an > IllegalStateException\[4], -bringing a halt to the data flow- causing > PersistentProvenanceRepository "Unable to merge with other > Journal Files due to..." WARNings. > Several of the field values being written in this way are pre-defined, and > thus not likely an issue. However, the "details" field can be populated by a > processor, and can be of an arbitrary length. -Additionally, if the detail > filed is indexed (which I didn't investigate, but I'm sure is easy enough to > determine), then the length might be subject to the Lucene limit discussed in > NIFI-2787-. > \[1] > https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/StandardRecordWriter.java#L163-L173 > \[2] > http://docs.oracle.com/javase/7/docs/api/java/io/DataOutputStream.html#writeUTF%28java.lang.String%29 > \[3] > http://stackoverflow.com/questions/22741556/dataoutputstream-purpose-of-the-encoded-string-too-long-restriction > \[4] > https://github.com/apache/nifi/blob/5fd4a55791da27fdba577636ac985a294618328a
[GitHub] nifi pull request #1475: NIFI-3055 StandardRecordWriter Can Throw UTFDataFor...
GitHub user jskora opened a pull request: https://github.com/apache/nifi/pull/1475 NIFI-3055 StandardRecordWriter Can Throw UTFDataFormatException (1.x) * Remove function based on JDK source. * Add new function to find bytes based on RFC3629. * Add field name to log entry when field is truncated. Thank you for submitting a contribution to Apache NiFi. In order to streamline the review of the contribution we ask you to ensure the following steps have been taken: ### For all changes: - [x ] Is there a JIRA ticket associated with this PR? Is it referenced in the commit message? - [x ] Does your PR title start with NIFI- where is the JIRA number you are trying to resolve? Pay particular attention to the hyphen "-" character. - [x ] Has your PR been rebased against the latest commit within the target branch (typically master)? - [x ] Is your initial contribution a single, squashed commit? ### For code changes: - [x ] Have you ensured that the full suite of tests is executed via mvn -Pcontrib-check clean install at the root nifi folder? - [x ] Have you written or updated unit tests to verify your changes? - [ ] If adding new dependencies to the code, are these dependencies licensed in a way that is compatible for inclusion under [ASF 2.0](http://www.apache.org/legal/resolved.html#category-a)? - [ ] If applicable, have you updated the LICENSE file, including the main LICENSE file under nifi-assembly? - [ ] If applicable, have you updated the NOTICE file, including the main NOTICE file found under nifi-assembly? - [ ] If adding new Properties, have you added .displayName in addition to .name (programmatic access) for each of the new properties? ### For documentation related changes: - [ ] Have you ensured that format looks appropriate for the output in which it is rendered? ### Note: Please ensure that once the PR is submitted, you check travis-ci for build issues and submit an update to your PR as soon as possible. You can merge this pull request into a Git repository by running: $ git pull https://github.com/jskora/nifi NIFI-3055-1.x-v4 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1475.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1475 commit 65310351bf07c410544c1d8e1acbba96de649c95 Author: Joe Skora Date: 2017-02-06T18:55:01Z NIFI-3055 StandardRecordWriter Can Throw UTFDataFormatException (1.x) * Remove function based on JDK source. * Add new function to find bytes based on RFC3629. * Add field name to log entry when field is truncated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3438) Ability to link to a view in Nifi (process group, processor, etc...)
[ https://issues.apache.org/jira/browse/NIFI-3438?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854537#comment-15854537 ] Andrew Lim commented on NIFI-3438: -- NIFI-3035 is a similar ticket requesting deep-linking. > Ability to link to a view in Nifi (process group, processor, etc...) > > > Key: NIFI-3438 > URL: https://issues.apache.org/jira/browse/NIFI-3438 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Nicholas Carenza >Priority: Minor > > I would like to be able to share a link to a particular view in the Nifi UI. > For example: > #/ might just bring me to that processor, center it in view and > reset the zoom. If it is a process group, it might go to it an bring all > processors into view. > #//config might then also bring up the configuration for that > processor > #//config/properties might then also set the tab to 'properties' -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi-minifi-cpp pull request #46: MINIFI-188: Incorporate CATCH and example ...
Github user phrocker commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/46#discussion_r99655455 --- Diff: libminifi/cmake/FindLeveldb.cmake --- @@ -0,0 +1,50 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless qrequired by applicable law or agreed to in writing, --- End diff -- Thanks! I'll fix that. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3408) Add an attribute for failure reason to InvokeHTTP in all cases to match docs
[ https://issues.apache.org/jira/browse/NIFI-3408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854530#comment-15854530 ] ASF GitHub Bot commented on NIFI-3408: -- Github user gglanzani commented on the issue: https://github.com/apache/nifi/pull/1467 Hi @pvillard31 thanks for the review. I've incorporated the changes. I've renamed the two new attributes to `invoke.http.java.exception.{class,message}`. They reflect better the underlying methods. Note: I've not updated the test to test for the message. I think it would turn out to be a pretty verbose test BUT let me know in case you disagree :) > Add an attribute for failure reason to InvokeHTTP in all cases to match docs > > > Key: NIFI-3408 > URL: https://issues.apache.org/jira/browse/NIFI-3408 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Priority: Minor > > Originally brought up on the mailing list[1], the InvokeHTTP docs state: > ``` > The original FlowFile will be routed on any type of connection failure, > timeout or general exception. It will have new attributes detailing the > request. > ``` > Adding attributes currently only happens though when a response is received > from the server, so not attributes if it fails to connect or the socket times > out. > These attributes should be added. > [1] > http://apache-nifi-users-list.2361937.n4.nabble.com/InvokeHTTP-and-SocketTimeoutException-td743.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1467: NIFI-3408 Add exception class when InvokeHTTP fails
Github user gglanzani commented on the issue: https://github.com/apache/nifi/pull/1467 Hi @pvillard31 thanks for the review. I've incorporated the changes. I've renamed the two new attributes to `invoke.http.java.exception.{class,message}`. They reflect better the underlying methods. Note: I've not updated the test to test for the message. I think it would turn out to be a pretty verbose test BUT let me know in case you disagree :) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-3443) Add Hadoop controller service, allowing users to define hadoop configuration as a central and reusable resource.
[ https://issues.apache.org/jira/browse/NIFI-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Narayanan updated NIFI-3443: Summary: Add Hadoop controller service, allowing users to define hadoop configuration as a central and reusable resource. (was: Add ambari controller service, allowing users to point to a cluster in ambari for configuration ) > Add Hadoop controller service, allowing users to define hadoop configuration > as a central and reusable resource. > > > Key: NIFI-3443 > URL: https://issues.apache.org/jira/browse/NIFI-3443 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Karthik Narayanan >Assignee: Karthik Narayanan > > The improvement will enable users to define hadoop configuration as a > Controller Service. This will make the configuration available centrally and > also make it reusable across processors. This will also allow these > processors to take advantage of receiving configs from a managed environment, > ex. ambari. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3443) Add ambari controller service, allowing users to point to a cluster in ambari for configuration
[ https://issues.apache.org/jira/browse/NIFI-3443?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Karthik Narayanan updated NIFI-3443: Description: The improvement will enable users to define hadoop configuration as a Controller Service. This will make the configuration available centrally and also make it reusable across processors. This will also allow these processors to take advantage of receiving configs from a managed environment, ex. ambari. was: The improvement will NiFi HDFS,HBASE and HIVE processors to work with ambari. Users can create an ambaricontrollerservice, that will point to the ambari host, port and cluster. The Controller Service, will connect to ambari and download the required configuration using the rest api. This will allow users to centrally manage their hadoop configurations in NIFI and also makes it easy for them to get started with NIFI and Hadoop. > Add ambari controller service, allowing users to point to a cluster in ambari > for configuration > > > Key: NIFI-3443 > URL: https://issues.apache.org/jira/browse/NIFI-3443 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Karthik Narayanan >Assignee: Karthik Narayanan > > The improvement will enable users to define hadoop configuration as a > Controller Service. This will make the configuration available centrally and > also make it reusable across processors. This will also allow these > processors to take advantage of receiving configs from a managed environment, > ex. ambari. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3419) MergeContent should route to failure on ZipException
[ https://issues.apache.org/jira/browse/NIFI-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854512#comment-15854512 ] ASF subversion and git services commented on NIFI-3419: --- Commit 229b20f3954e66f3f8a5c373e08b887f1209a315 in nifi's branch refs/heads/master from Joe Gresock [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=229b20f ] NIFI-3419: Routing flow files causing ZipException to failure in MergeContent This closes #1454. > MergeContent should route to failure on ZipException > > > Key: NIFI-3419 > URL: https://issues.apache.org/jira/browse/NIFI-3419 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.1.1 >Reporter: Joseph Gresock >Assignee: Joseph Gresock > Fix For: 1.2.0 > > > In cases like duplicate entries using the Zip format in MergeContent, the > processor should route to failure instead of leaving the exception uncaught, > which causes flow files to administratively yield and never be processed. > 2017-01-30 11:11:39,106 ERROR [Timer-Driven Process Thread-142] > o.a.n.processors.standard.MergeContent > org.apache.nifi.processor.exception.ProcessException: IOException thrown from > MergeContent[id=a9476272-dab0-3d2f-acef-abf11f7d1b80]: > java.util.zip.ZipException: duplicate entry: > c8e206d2c658b14c386b8c8ffc8d3db88c082874.json > at > org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2348) > ~[na:na] > at > org.apache.nifi.processors.standard.MergeContent$ZipMerge.merge(MergeContent.java:826) > ~[nifi-standard-processors-1.1.1.jar:1.1.1] > at > org.apache.nifi.processors.standard.MergeContent.processBin(MergeContent.java:459) > ~[nifi-standard-processors-1.1.1.jar:1.1.1] > at > org.apache.nifi.processor.util.bin.BinFiles.processBins(BinFiles.java:231) > [nifi-processor-utils-1.1.1.jar:1.1.1] > at > org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:191) > [nifi-processor-utils-1.1.1.jar:1.1.1] > Caused by: java.util.zip.ZipException: duplicate entry: > c8e206d2c658b14c386b8c8ffc8d3db88c082874.json > at > java.util.zip.ZipOutputStream.putNextEntry(ZipOutputStream.java:232) > ~[na:1.8.0_111] > at > org.apache.nifi.processors.standard.MergeContent$ZipMerge$1.process(MergeContent.java:837) > ~[nifi-standard-processors-1.1.1.jar:1.1.1] > at > org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2329) > ~[na:na] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3419) MergeContent should route to failure on ZipException
[ https://issues.apache.org/jira/browse/NIFI-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3419: - Resolution: Fixed Status: Resolved (was: Patch Available) > MergeContent should route to failure on ZipException > > > Key: NIFI-3419 > URL: https://issues.apache.org/jira/browse/NIFI-3419 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.1.1 >Reporter: Joseph Gresock >Assignee: Joseph Gresock > Fix For: 1.2.0 > > > In cases like duplicate entries using the Zip format in MergeContent, the > processor should route to failure instead of leaving the exception uncaught, > which causes flow files to administratively yield and never be processed. > 2017-01-30 11:11:39,106 ERROR [Timer-Driven Process Thread-142] > o.a.n.processors.standard.MergeContent > org.apache.nifi.processor.exception.ProcessException: IOException thrown from > MergeContent[id=a9476272-dab0-3d2f-acef-abf11f7d1b80]: > java.util.zip.ZipException: duplicate entry: > c8e206d2c658b14c386b8c8ffc8d3db88c082874.json > at > org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2348) > ~[na:na] > at > org.apache.nifi.processors.standard.MergeContent$ZipMerge.merge(MergeContent.java:826) > ~[nifi-standard-processors-1.1.1.jar:1.1.1] > at > org.apache.nifi.processors.standard.MergeContent.processBin(MergeContent.java:459) > ~[nifi-standard-processors-1.1.1.jar:1.1.1] > at > org.apache.nifi.processor.util.bin.BinFiles.processBins(BinFiles.java:231) > [nifi-processor-utils-1.1.1.jar:1.1.1] > at > org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:191) > [nifi-processor-utils-1.1.1.jar:1.1.1] > Caused by: java.util.zip.ZipException: duplicate entry: > c8e206d2c658b14c386b8c8ffc8d3db88c082874.json > at > java.util.zip.ZipOutputStream.putNextEntry(ZipOutputStream.java:232) > ~[na:1.8.0_111] > at > org.apache.nifi.processors.standard.MergeContent$ZipMerge$1.process(MergeContent.java:837) > ~[nifi-standard-processors-1.1.1.jar:1.1.1] > at > org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2329) > ~[na:na] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3429) ConvertJSONToSQL Does not work with table identifiers that need to be quoted
[ https://issues.apache.org/jira/browse/NIFI-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Wicks updated NIFI-3429: -- Affects Version/s: 1.2.0 > ConvertJSONToSQL Does not work with table identifiers that need to be quoted > > > Key: NIFI-3429 > URL: https://issues.apache.org/jira/browse/NIFI-3429 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Nicholas Carenza >Priority: Minor > Fix For: 1.2.0 > > > I have a table identifer that needs to be quoted. If i supple the table > identifer unquoted in the properties, the processor can convert the json to > sql but the output sql doesn't have the table name quoted even if I select > the option Quote Identifiers because, true to the description, it only > applies to column identifiers. If I supply the table identifier quoted in the > properties, then the processor fails completely because it can't find the > table. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3429) ConvertJSONToSQL Does not work with table identifiers that need to be quoted
[ https://issues.apache.org/jira/browse/NIFI-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854514#comment-15854514 ] ASF GitHub Bot commented on NIFI-3429: -- Github user patricker commented on the issue: https://github.com/apache/nifi/pull/1464 @nickcarenza It looks good to me, but I don't have the power to do anything... I did go out to JIRA and tag your ticket as 1.2.0 so it will get reviewed sooner: https://issues.apache.org/jira/browse/NIFI-3429 > ConvertJSONToSQL Does not work with table identifiers that need to be quoted > > > Key: NIFI-3429 > URL: https://issues.apache.org/jira/browse/NIFI-3429 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Nicholas Carenza >Priority: Minor > Fix For: 1.2.0 > > > I have a table identifer that needs to be quoted. If i supple the table > identifer unquoted in the properties, the processor can convert the json to > sql but the output sql doesn't have the table name quoted even if I select > the option Quote Identifiers because, true to the description, it only > applies to column identifiers. If I supply the table identifier quoted in the > properties, then the processor fails completely because it can't find the > table. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3429) ConvertJSONToSQL Does not work with table identifiers that need to be quoted
[ https://issues.apache.org/jira/browse/NIFI-3429?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Peter Wicks updated NIFI-3429: -- Fix Version/s: 1.2.0 > ConvertJSONToSQL Does not work with table identifiers that need to be quoted > > > Key: NIFI-3429 > URL: https://issues.apache.org/jira/browse/NIFI-3429 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.2.0 >Reporter: Nicholas Carenza >Priority: Minor > Fix For: 1.2.0 > > > I have a table identifer that needs to be quoted. If i supple the table > identifer unquoted in the properties, the processor can convert the json to > sql but the output sql doesn't have the table name quoted even if I select > the option Quote Identifiers because, true to the description, it only > applies to column identifiers. If I supply the table identifier quoted in the > properties, then the processor fails completely because it can't find the > table. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1464: NIFI-3429 automatically quoting all table identifiers
Github user patricker commented on the issue: https://github.com/apache/nifi/pull/1464 @nickcarenza It looks good to me, but I don't have the power to do anything... I did go out to JIRA and tag your ticket as 1.2.0 so it will get reviewed sooner: https://issues.apache.org/jira/browse/NIFI-3429 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3419) MergeContent should route to failure on ZipException
[ https://issues.apache.org/jira/browse/NIFI-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854513#comment-15854513 ] ASF GitHub Bot commented on NIFI-3419: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1454 > MergeContent should route to failure on ZipException > > > Key: NIFI-3419 > URL: https://issues.apache.org/jira/browse/NIFI-3419 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.1.1 >Reporter: Joseph Gresock >Assignee: Joseph Gresock > Fix For: 1.2.0 > > > In cases like duplicate entries using the Zip format in MergeContent, the > processor should route to failure instead of leaving the exception uncaught, > which causes flow files to administratively yield and never be processed. > 2017-01-30 11:11:39,106 ERROR [Timer-Driven Process Thread-142] > o.a.n.processors.standard.MergeContent > org.apache.nifi.processor.exception.ProcessException: IOException thrown from > MergeContent[id=a9476272-dab0-3d2f-acef-abf11f7d1b80]: > java.util.zip.ZipException: duplicate entry: > c8e206d2c658b14c386b8c8ffc8d3db88c082874.json > at > org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2348) > ~[na:na] > at > org.apache.nifi.processors.standard.MergeContent$ZipMerge.merge(MergeContent.java:826) > ~[nifi-standard-processors-1.1.1.jar:1.1.1] > at > org.apache.nifi.processors.standard.MergeContent.processBin(MergeContent.java:459) > ~[nifi-standard-processors-1.1.1.jar:1.1.1] > at > org.apache.nifi.processor.util.bin.BinFiles.processBins(BinFiles.java:231) > [nifi-processor-utils-1.1.1.jar:1.1.1] > at > org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:191) > [nifi-processor-utils-1.1.1.jar:1.1.1] > Caused by: java.util.zip.ZipException: duplicate entry: > c8e206d2c658b14c386b8c8ffc8d3db88c082874.json > at > java.util.zip.ZipOutputStream.putNextEntry(ZipOutputStream.java:232) > ~[na:1.8.0_111] > at > org.apache.nifi.processors.standard.MergeContent$ZipMerge$1.process(MergeContent.java:837) > ~[nifi-standard-processors-1.1.1.jar:1.1.1] > at > org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2329) > ~[na:na] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1454: NIFI-3419: Routing flow files causing ZipException ...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1454 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3419) MergeContent should route to failure on ZipException
[ https://issues.apache.org/jira/browse/NIFI-3419?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854501#comment-15854501 ] ASF GitHub Bot commented on NIFI-3419: -- Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/1454 LGTM, merging. The same could apply to ``TarMerge`` but this can be done in another JIRA. > MergeContent should route to failure on ZipException > > > Key: NIFI-3419 > URL: https://issues.apache.org/jira/browse/NIFI-3419 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.1.1 >Reporter: Joseph Gresock >Assignee: Joseph Gresock > Fix For: 1.2.0 > > > In cases like duplicate entries using the Zip format in MergeContent, the > processor should route to failure instead of leaving the exception uncaught, > which causes flow files to administratively yield and never be processed. > 2017-01-30 11:11:39,106 ERROR [Timer-Driven Process Thread-142] > o.a.n.processors.standard.MergeContent > org.apache.nifi.processor.exception.ProcessException: IOException thrown from > MergeContent[id=a9476272-dab0-3d2f-acef-abf11f7d1b80]: > java.util.zip.ZipException: duplicate entry: > c8e206d2c658b14c386b8c8ffc8d3db88c082874.json > at > org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2348) > ~[na:na] > at > org.apache.nifi.processors.standard.MergeContent$ZipMerge.merge(MergeContent.java:826) > ~[nifi-standard-processors-1.1.1.jar:1.1.1] > at > org.apache.nifi.processors.standard.MergeContent.processBin(MergeContent.java:459) > ~[nifi-standard-processors-1.1.1.jar:1.1.1] > at > org.apache.nifi.processor.util.bin.BinFiles.processBins(BinFiles.java:231) > [nifi-processor-utils-1.1.1.jar:1.1.1] > at > org.apache.nifi.processor.util.bin.BinFiles.onTrigger(BinFiles.java:191) > [nifi-processor-utils-1.1.1.jar:1.1.1] > Caused by: java.util.zip.ZipException: duplicate entry: > c8e206d2c658b14c386b8c8ffc8d3db88c082874.json > at > java.util.zip.ZipOutputStream.putNextEntry(ZipOutputStream.java:232) > ~[na:1.8.0_111] > at > org.apache.nifi.processors.standard.MergeContent$ZipMerge$1.process(MergeContent.java:837) > ~[nifi-standard-processors-1.1.1.jar:1.1.1] > at > org.apache.nifi.controller.repository.StandardProcessSession.write(StandardProcessSession.java:2329) > ~[na:na] -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1454: NIFI-3419: Routing flow files causing ZipException to fail...
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/1454 LGTM, merging. The same could apply to ``TarMerge`` but this can be done in another JIRA. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi-minifi-cpp pull request #46: MINIFI-188: Incorporate CATCH and example ...
Github user apiri commented on a diff in the pull request: https://github.com/apache/nifi-minifi-cpp/pull/46#discussion_r99649610 --- Diff: libminifi/cmake/FindLeveldb.cmake --- @@ -0,0 +1,50 @@ +# Licensed to the Apache Software Foundation (ASF) under one +# or more contributor license agreements. See the NOTICE file +# distributed with this work for additional information +# regarding copyright ownership. The ASF licenses this file +# to you under the Apache License, Version 2.0 (the +# "License"); you may not use this file except in compliance +# with the License. You may obtain a copy of the License at +# +# http://www.apache.org/licenses/LICENSE-2.0 +# +# Unless qrequired by applicable law or agreed to in writing, --- End diff -- typo --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3408) Add an attribute for failure reason to InvokeHTTP in all cases to match docs
[ https://issues.apache.org/jira/browse/NIFI-3408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854488#comment-15854488 ] ASF GitHub Bot commented on NIFI-3408: -- Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1467#discussion_r99649592 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java --- @@ -118,6 +119,8 @@ public final static String REQUEST_URL = "invokehttp.request.url"; public final static String TRANSACTION_ID = "invokehttp.tx.id"; public final static String REMOTE_DN = "invokehttp.remote.dn"; +public final static String EXCEPTION_CLASS = "invokehttp.java.exception"; --- End diff -- What do you think about adding an attribute for exception message as well? Might be useful in addition to the class name? ``invokehttp.java.exception.message``? > Add an attribute for failure reason to InvokeHTTP in all cases to match docs > > > Key: NIFI-3408 > URL: https://issues.apache.org/jira/browse/NIFI-3408 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Priority: Minor > > Originally brought up on the mailing list[1], the InvokeHTTP docs state: > ``` > The original FlowFile will be routed on any type of connection failure, > timeout or general exception. It will have new attributes detailing the > request. > ``` > Adding attributes currently only happens though when a response is received > from the server, so not attributes if it fails to connect or the socket times > out. > These attributes should be added. > [1] > http://apache-nifi-users-list.2361937.n4.nabble.com/InvokeHTTP-and-SocketTimeoutException-td743.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1467: NIFI-3408 Add exception class when InvokeHTTP fails
Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1467#discussion_r99649592 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java --- @@ -118,6 +119,8 @@ public final static String REQUEST_URL = "invokehttp.request.url"; public final static String TRANSACTION_ID = "invokehttp.tx.id"; public final static String REMOTE_DN = "invokehttp.remote.dn"; +public final static String EXCEPTION_CLASS = "invokehttp.java.exception"; --- End diff -- What do you think about adding an attribute for exception message as well? Might be useful in addition to the class name? ``invokehttp.java.exception.message``? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3408) Add an attribute for failure reason to InvokeHTTP in all cases to match docs
[ https://issues.apache.org/jira/browse/NIFI-3408?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854489#comment-15854489 ] ASF GitHub Bot commented on NIFI-3408: -- Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1467#discussion_r99649060 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java --- @@ -753,6 +756,9 @@ public void onTrigger(ProcessContext context, ProcessSession session) throws Pro if (requestFlowFile != null) { logger.error("Routing to {} due to exception: {}", new Object[]{REL_FAILURE.getName(), e}, e); requestFlowFile = session.penalize(requestFlowFile); +String attributeKey = EXCEPTION_CLASS; +String attributeValue = e.getClass().getName(); +requestFlowFile = session.putAttribute(requestFlowFile, attributeKey, attributeValue); --- End diff -- Purely stylistic comment: could you simplify to java requestFlowFile = session.putAttribute(requestFlowFile, EXCEPTION_CLASS, e.getClass().getName()); > Add an attribute for failure reason to InvokeHTTP in all cases to match docs > > > Key: NIFI-3408 > URL: https://issues.apache.org/jira/browse/NIFI-3408 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Priority: Minor > > Originally brought up on the mailing list[1], the InvokeHTTP docs state: > ``` > The original FlowFile will be routed on any type of connection failure, > timeout or general exception. It will have new attributes detailing the > request. > ``` > Adding attributes currently only happens though when a response is received > from the server, so not attributes if it fails to connect or the socket times > out. > These attributes should be added. > [1] > http://apache-nifi-users-list.2361937.n4.nabble.com/InvokeHTTP-and-SocketTimeoutException-td743.html -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1467: NIFI-3408 Add exception class when InvokeHTTP fails
Github user pvillard31 commented on a diff in the pull request: https://github.com/apache/nifi/pull/1467#discussion_r99649060 --- Diff: nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/InvokeHTTP.java --- @@ -753,6 +756,9 @@ public void onTrigger(ProcessContext context, ProcessSession session) throws Pro if (requestFlowFile != null) { logger.error("Routing to {} due to exception: {}", new Object[]{REL_FAILURE.getName(), e}, e); requestFlowFile = session.penalize(requestFlowFile); +String attributeKey = EXCEPTION_CLASS; +String attributeValue = e.getClass().getName(); +requestFlowFile = session.putAttribute(requestFlowFile, attributeKey, attributeValue); --- End diff -- Purely stylistic comment: could you simplify to java requestFlowFile = session.putAttribute(requestFlowFile, EXCEPTION_CLASS, e.getClass().getName()); --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (NIFI-3437) KafkaConsumer pause/resume
[ https://issues.apache.org/jira/browse/NIFI-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt resolved NIFI-3437. --- Resolution: Information Provided > KafkaConsumer pause/resume > -- > > Key: NIFI-3437 > URL: https://issues.apache.org/jira/browse/NIFI-3437 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Nicholas Carenza >Priority: Minor > > Could the KafkaConsumer processor keep track of it's position so that if it > is stopped and started again it can attempt to resume consuming from that > position? > If I have to restart nifi or rename the consumer processor I want to make > sure I don't miss any messages. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3437) KafkaConsumer pause/resume
[ https://issues.apache.org/jira/browse/NIFI-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854485#comment-15854485 ] Joseph Witt commented on NIFI-3437: --- This statement "So when I stop and start the processor, as long as it has the same group id, Kafka will start sending messages starting from the last committed offset." Is accurate. I'll go ahead and close the ticket but should you find a problem please let us know. Thanks > KafkaConsumer pause/resume > -- > > Key: NIFI-3437 > URL: https://issues.apache.org/jira/browse/NIFI-3437 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Nicholas Carenza >Priority: Minor > > Could the KafkaConsumer processor keep track of it's position so that if it > is stopped and started again it can attempt to resume consuming from that > position? > If I have to restart nifi or rename the consumer processor I want to make > sure I don't miss any messages. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3437) KafkaConsumer pause/resume
[ https://issues.apache.org/jira/browse/NIFI-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854475#comment-15854475 ] Nicholas Carenza commented on NIFI-3437: [~joewitt] Apologies I am not very experienced with Kafka, or Nifi for that matter. I was trying to find something in the ConsumeKafka processor that kept track of state. I read up on the Kafka documentation a bit and I think I understand the processor works with Kafka to keep track of its offset. Let me see if I have this right: The processor commits its latest consumed offset to Kafka at least every MAX_UNCOMMITTED_TIME and it's Kafka that keeps a record based on the consumer's group id. So when I stop and start the processor, as long as it has the same group id, Kafka will start sending messages starting from the last committed offset. https://kafka.apache.org/documentation/#distributionimpl > KafkaConsumer pause/resume > -- > > Key: NIFI-3437 > URL: https://issues.apache.org/jira/browse/NIFI-3437 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Nicholas Carenza >Priority: Minor > > Could the KafkaConsumer processor keep track of it's position so that if it > is stopped and started again it can attempt to resume consuming from that > position? > If I have to restart nifi or rename the consumer processor I want to make > sure I don't miss any messages. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Created] (NIFI-3443) Add ambari controller service, allowing users to point to a cluster in ambari for configuration
Karthik Narayanan created NIFI-3443: --- Summary: Add ambari controller service, allowing users to point to a cluster in ambari for configuration Key: NIFI-3443 URL: https://issues.apache.org/jira/browse/NIFI-3443 Project: Apache NiFi Issue Type: Improvement Reporter: Karthik Narayanan Assignee: Karthik Narayanan The improvement will NiFi HDFS,HBASE and HIVE processors to work with ambari. Users can create an ambaricontrollerservice, that will point to the ambari host, port and cluster. The Controller Service, will connect to ambari and download the required configuration using the rest api. This will allow users to centrally manage their hadoop configurations in NIFI and also makes it easy for them to get started with NIFI and Hadoop. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Comment Edited] (NIFI-3437) KafkaConsumer pause/resume
[ https://issues.apache.org/jira/browse/NIFI-3437?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854475#comment-15854475 ] Nicholas Carenza edited comment on NIFI-3437 at 2/6/17 6:13 PM: [~joewitt] Apologies I am not very experienced with Kafka, or Nifi for that matter. I was trying to find something in the ConsumeKafka processor that kept track of state. I read up on the Kafka documentation a bit and I think I understand how the processor works with Kafka to keep track of its offset. Let me see if I have this right: The processor commits its latest consumed offset to Kafka at least every MAX_UNCOMMITTED_TIME and it's Kafka that keeps a record based on the consumer's group id. So when I stop and start the processor, as long as it has the same group id, Kafka will start sending messages starting from the last committed offset. https://kafka.apache.org/documentation/#distributionimpl was (Author: ncarenza): [~joewitt] Apologies I am not very experienced with Kafka, or Nifi for that matter. I was trying to find something in the ConsumeKafka processor that kept track of state. I read up on the Kafka documentation a bit and I think I understand the processor works with Kafka to keep track of its offset. Let me see if I have this right: The processor commits its latest consumed offset to Kafka at least every MAX_UNCOMMITTED_TIME and it's Kafka that keeps a record based on the consumer's group id. So when I stop and start the processor, as long as it has the same group id, Kafka will start sending messages starting from the last committed offset. https://kafka.apache.org/documentation/#distributionimpl > KafkaConsumer pause/resume > -- > > Key: NIFI-3437 > URL: https://issues.apache.org/jira/browse/NIFI-3437 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Nicholas Carenza >Priority: Minor > > Could the KafkaConsumer processor keep track of it's position so that if it > is stopped and started again it can attempt to resume consuming from that > position? > If I have to restart nifi or rename the consumer processor I want to make > sure I don't miss any messages. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi-minifi-cpp issue #46: MINIFI-188: Incorporate CATCH and example tests
Github user apiri commented on the issue: https://github.com/apache/nifi-minifi-cpp/pull/46 reviewing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3055) StandardRecordWriter can throw UTFDataFormatException
[ https://issues.apache.org/jira/browse/NIFI-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854446#comment-15854446 ] ASF GitHub Bot commented on NIFI-3055: -- Github user jskora closed the pull request at: https://github.com/apache/nifi/pull/1470 > StandardRecordWriter can throw UTFDataFormatException > - > > Key: NIFI-3055 > URL: https://issues.apache.org/jira/browse/NIFI-3055 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.0.0, 0.7.1 >Reporter: Brandon DeVries >Assignee: Joe Skora >Priority: Blocker > Fix For: 0.8.0, 1.2.0 > > > StandardRecordWriter.writeRecord()\[1] uses DataOutputStream.writeUTF()\[2] > without checking the length of the value to be written. If this length is > greater than 65535 (2^16 - 1), you get a UTFDataFormatException "encoded > string too long..."\[3]. Ultimately, this can result in an > IllegalStateException\[4], -bringing a halt to the data flow- causing > PersistentProvenanceRepository "Unable to merge with other > Journal Files due to..." WARNings. > Several of the field values being written in this way are pre-defined, and > thus not likely an issue. However, the "details" field can be populated by a > processor, and can be of an arbitrary length. -Additionally, if the detail > filed is indexed (which I didn't investigate, but I'm sure is easy enough to > determine), then the length might be subject to the Lucene limit discussed in > NIFI-2787-. > \[1] > https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/StandardRecordWriter.java#L163-L173 > \[2] > http://docs.oracle.com/javase/7/docs/api/java/io/DataOutputStream.html#writeUTF%28java.lang.String%29 > \[3] > http://stackoverflow.com/questions/22741556/dataoutputstream-purpose-of-the-encoded-string-too-long-restriction > \[4] > https://github.com/apache/nifi/blob/5fd4a55791da27fdba577636ac985a294618328a/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/PersistentProvenanceRepository.java#L754-L755 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3055) StandardRecordWriter can throw UTFDataFormatException
[ https://issues.apache.org/jira/browse/NIFI-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854445#comment-15854445 ] ASF GitHub Bot commented on NIFI-3055: -- Github user jskora commented on the issue: https://github.com/apache/nifi/pull/1470 Closing this PR, committed on [4f72e34](https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=4f72e3491f2372c8c45afb96a765c1f5cdd2f07d). > StandardRecordWriter can throw UTFDataFormatException > - > > Key: NIFI-3055 > URL: https://issues.apache.org/jira/browse/NIFI-3055 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.0.0, 0.7.1 >Reporter: Brandon DeVries >Assignee: Joe Skora >Priority: Blocker > Fix For: 0.8.0, 1.2.0 > > > StandardRecordWriter.writeRecord()\[1] uses DataOutputStream.writeUTF()\[2] > without checking the length of the value to be written. If this length is > greater than 65535 (2^16 - 1), you get a UTFDataFormatException "encoded > string too long..."\[3]. Ultimately, this can result in an > IllegalStateException\[4], -bringing a halt to the data flow- causing > PersistentProvenanceRepository "Unable to merge with other > Journal Files due to..." WARNings. > Several of the field values being written in this way are pre-defined, and > thus not likely an issue. However, the "details" field can be populated by a > processor, and can be of an arbitrary length. -Additionally, if the detail > filed is indexed (which I didn't investigate, but I'm sure is easy enough to > determine), then the length might be subject to the Lucene limit discussed in > NIFI-2787-. > \[1] > https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/StandardRecordWriter.java#L163-L173 > \[2] > http://docs.oracle.com/javase/7/docs/api/java/io/DataOutputStream.html#writeUTF%28java.lang.String%29 > \[3] > http://stackoverflow.com/questions/22741556/dataoutputstream-purpose-of-the-encoded-string-too-long-restriction > \[4] > https://github.com/apache/nifi/blob/5fd4a55791da27fdba577636ac985a294618328a/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/PersistentProvenanceRepository.java#L754-L755 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1470: NIFI-3055 StandardRecordWriter Can Throw UTFDataFor...
Github user jskora closed the pull request at: https://github.com/apache/nifi/pull/1470 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1470: NIFI-3055 StandardRecordWriter Can Throw UTFDataFormatExce...
Github user jskora commented on the issue: https://github.com/apache/nifi/pull/1470 Closing this PR, committed on [4f72e34](https://git-wip-us.apache.org/repos/asf?p=nifi.git;a=commit;h=4f72e3491f2372c8c45afb96a765c1f5cdd2f07d). --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3427) Data Provenance date/time sort ignores the date
[ https://issues.apache.org/jira/browse/NIFI-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854428#comment-15854428 ] ASF GitHub Bot commented on NIFI-3427: -- Github user jvwing commented on the issue: https://github.com/apache/nifi/pull/1465 Thanks, @pvillard31 > Data Provenance date/time sort ignores the date > --- > > Key: NIFI-3427 > URL: https://issues.apache.org/jira/browse/NIFI-3427 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.1.1 >Reporter: Justin Rittenhouse >Assignee: James Wing >Priority: Minor > Fix For: 1.2.0 > > Attachments: Screen Shot 2017-02-01 at 9.49.11 AM.png > > > The Data provenance display window that pops up doesn't properly sort by > date/time. It sorts by time, with events dated 11pm from January 31 > appearing before/after (depending on ascending or descending sort) events > dated February 1 before 11pm. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1465: NIFI-3427 Fix Provenance DateTime Sorting
Github user jvwing commented on the issue: https://github.com/apache/nifi/pull/1465 Thanks, @pvillard31 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3055) StandardRecordWriter can throw UTFDataFormatException
[ https://issues.apache.org/jira/browse/NIFI-3055?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854426#comment-15854426 ] ASF GitHub Bot commented on NIFI-3055: -- Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/1470 Even if I know there will be additional changes for this JIRA in a new PR, can you close this PR @jskora ? (not closed automatically in 0.x) > StandardRecordWriter can throw UTFDataFormatException > - > > Key: NIFI-3055 > URL: https://issues.apache.org/jira/browse/NIFI-3055 > Project: Apache NiFi > Issue Type: Bug >Affects Versions: 1.0.0, 0.7.1 >Reporter: Brandon DeVries >Assignee: Joe Skora >Priority: Blocker > Fix For: 0.8.0, 1.2.0 > > > StandardRecordWriter.writeRecord()\[1] uses DataOutputStream.writeUTF()\[2] > without checking the length of the value to be written. If this length is > greater than 65535 (2^16 - 1), you get a UTFDataFormatException "encoded > string too long..."\[3]. Ultimately, this can result in an > IllegalStateException\[4], -bringing a halt to the data flow- causing > PersistentProvenanceRepository "Unable to merge with other > Journal Files due to..." WARNings. > Several of the field values being written in this way are pre-defined, and > thus not likely an issue. However, the "details" field can be populated by a > processor, and can be of an arbitrary length. -Additionally, if the detail > filed is indexed (which I didn't investigate, but I'm sure is easy enough to > determine), then the length might be subject to the Lucene limit discussed in > NIFI-2787-. > \[1] > https://github.com/apache/nifi/blob/master/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/StandardRecordWriter.java#L163-L173 > \[2] > http://docs.oracle.com/javase/7/docs/api/java/io/DataOutputStream.html#writeUTF%28java.lang.String%29 > \[3] > http://stackoverflow.com/questions/22741556/dataoutputstream-purpose-of-the-encoded-string-too-long-restriction > \[4] > https://github.com/apache/nifi/blob/5fd4a55791da27fdba577636ac985a294618328a/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/src/main/java/org/apache/nifi/provenance/PersistentProvenanceRepository.java#L754-L755 -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1470: NIFI-3055 StandardRecordWriter Can Throw UTFDataFormatExce...
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/1470 Even if I know there will be additional changes for this JIRA in a new PR, can you close this PR @jskora ? (not closed automatically in 0.x) --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Resolved] (NIFI-3430) PutSQL Errors attempting to coerce date string to timestamp
[ https://issues.apache.org/jira/browse/NIFI-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard resolved NIFI-3430. -- Resolution: Fixed Fix Version/s: 1.2.0 > PutSQL Errors attempting to coerce date string to timestamp > --- > > Key: NIFI-3430 > URL: https://issues.apache.org/jira/browse/NIFI-3430 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Nicholas Carenza >Priority: Minor > Labels: convertjsontosql, putsql > Fix For: 1.2.0 > > > I am attempting to write a flat json object to my postgresql database using > ConvertJSONToSQL followed by PutSQL. My json has a string date field in ISO > 8601 format which postgres can accept as a timestamp. The column in the > database is of type 'timestamp without time zone'. > org.apache.nifi.processor.exception.ProcessException: The value of the > sql.args.33.value is '2017-02-01T23:12:21+00:00', which cannot be converted > to a timestamp > Is PutSQL trying to do some preprocessing here that it should just let the > database handle? > select cast('2017-02-01T23:12:21+00:00' as timestamp without time zone); > timestamp > - > 2017-02-01 23:12:21 > (1 row) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3430) PutSQL Errors attempting to coerce date string to timestamp
[ https://issues.apache.org/jira/browse/NIFI-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854422#comment-15854422 ] ASF GitHub Bot commented on NIFI-3430: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1468 > PutSQL Errors attempting to coerce date string to timestamp > --- > > Key: NIFI-3430 > URL: https://issues.apache.org/jira/browse/NIFI-3430 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Nicholas Carenza >Priority: Minor > Labels: convertjsontosql, putsql > > I am attempting to write a flat json object to my postgresql database using > ConvertJSONToSQL followed by PutSQL. My json has a string date field in ISO > 8601 format which postgres can accept as a timestamp. The column in the > database is of type 'timestamp without time zone'. > org.apache.nifi.processor.exception.ProcessException: The value of the > sql.args.33.value is '2017-02-01T23:12:21+00:00', which cannot be converted > to a timestamp > Is PutSQL trying to do some preprocessing here that it should just let the > database handle? > select cast('2017-02-01T23:12:21+00:00' as timestamp without time zone); > timestamp > - > 2017-02-01 23:12:21 > (1 row) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Updated] (NIFI-3430) PutSQL Errors attempting to coerce date string to timestamp
[ https://issues.apache.org/jira/browse/NIFI-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3430: - Component/s: (was: Core Framework) Extensions > PutSQL Errors attempting to coerce date string to timestamp > --- > > Key: NIFI-3430 > URL: https://issues.apache.org/jira/browse/NIFI-3430 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Reporter: Nicholas Carenza >Priority: Minor > Labels: convertjsontosql, putsql > > I am attempting to write a flat json object to my postgresql database using > ConvertJSONToSQL followed by PutSQL. My json has a string date field in ISO > 8601 format which postgres can accept as a timestamp. The column in the > database is of type 'timestamp without time zone'. > org.apache.nifi.processor.exception.ProcessException: The value of the > sql.args.33.value is '2017-02-01T23:12:21+00:00', which cannot be converted > to a timestamp > Is PutSQL trying to do some preprocessing here that it should just let the > database handle? > select cast('2017-02-01T23:12:21+00:00' as timestamp without time zone); > timestamp > - > 2017-02-01 23:12:21 > (1 row) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1468: NIFI-3430 adds support for date format property to ...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1468 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3430) PutSQL Errors attempting to coerce date string to timestamp
[ https://issues.apache.org/jira/browse/NIFI-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854421#comment-15854421 ] ASF subversion and git services commented on NIFI-3430: --- Commit 6466931c236aa29bfd95da63e0e6588fca420f97 in nifi's branch refs/heads/master from Nick Carenza [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=6466931 ] NIFI-3430 - added support for specifying sql.arg.N.format for timestamp fields according to java.time.format.DateTimeFormatter This closes #1468. > PutSQL Errors attempting to coerce date string to timestamp > --- > > Key: NIFI-3430 > URL: https://issues.apache.org/jira/browse/NIFI-3430 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Nicholas Carenza >Priority: Minor > Labels: convertjsontosql, putsql > > I am attempting to write a flat json object to my postgresql database using > ConvertJSONToSQL followed by PutSQL. My json has a string date field in ISO > 8601 format which postgres can accept as a timestamp. The column in the > database is of type 'timestamp without time zone'. > org.apache.nifi.processor.exception.ProcessException: The value of the > sql.args.33.value is '2017-02-01T23:12:21+00:00', which cannot be converted > to a timestamp > Is PutSQL trying to do some preprocessing here that it should just let the > database handle? > select cast('2017-02-01T23:12:21+00:00' as timestamp without time zone); > timestamp > - > 2017-02-01 23:12:21 > (1 row) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3430) PutSQL Errors attempting to coerce date string to timestamp
[ https://issues.apache.org/jira/browse/NIFI-3430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854419#comment-15854419 ] ASF GitHub Bot commented on NIFI-3430: -- Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/1468 LGTM, I'll take care of merging @trixpan ;) Thanks @nickcarenza > PutSQL Errors attempting to coerce date string to timestamp > --- > > Key: NIFI-3430 > URL: https://issues.apache.org/jira/browse/NIFI-3430 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Reporter: Nicholas Carenza >Priority: Minor > Labels: convertjsontosql, putsql > > I am attempting to write a flat json object to my postgresql database using > ConvertJSONToSQL followed by PutSQL. My json has a string date field in ISO > 8601 format which postgres can accept as a timestamp. The column in the > database is of type 'timestamp without time zone'. > org.apache.nifi.processor.exception.ProcessException: The value of the > sql.args.33.value is '2017-02-01T23:12:21+00:00', which cannot be converted > to a timestamp > Is PutSQL trying to do some preprocessing here that it should just let the > database handle? > select cast('2017-02-01T23:12:21+00:00' as timestamp without time zone); > timestamp > - > 2017-02-01 23:12:21 > (1 row) -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1468: NIFI-3430 adds support for date format property to parse i...
Github user pvillard31 commented on the issue: https://github.com/apache/nifi/pull/1468 LGTM, I'll take care of merging @trixpan ;) Thanks @nickcarenza --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-3434) nifi-assembly dependencies should allow users to skip the creation of ZIP and TAR.GZ packages
[ https://issues.apache.org/jira/browse/NIFI-3434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854390#comment-15854390 ] ASF GitHub Bot commented on NIFI-3434: -- Github user apiri commented on the issue: https://github.com/apache/nifi/pull/1472 @pvillard31 Yes, please! > nifi-assembly dependencies should allow users to skip the creation of ZIP and > TAR.GZ packages > - > > Key: NIFI-3434 > URL: https://issues.apache.org/jira/browse/NIFI-3434 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.1.1 >Reporter: Andre >Assignee: Andre > > devs, > Currently calling 'mvn clean install' creates a ZIP, a TAR.GZ and a > directory containing the same code. This leads to wasted disk space and a > lot of wasted disk writes (something that a lot of folks using SSDs prefer > to avoid). > Would anyone oppose the idea of moving the ZIP and TAR.GZ assemblies into a > "release" profile (or whatever name we agree to). This way we could > maintain the directory "format" (which I suspect most of us use during > development), while still providing a convenient way of creating the ZIP > and TAR.GZ packages. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi issue #1472: NIFI-3434 - Introduce the ability to skip the packaging of...
Github user apiri commented on the issue: https://github.com/apache/nifi/pull/1472 @pvillard31 Yes, please! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-3427) Data Provenance date/time sort ignores the date
[ https://issues.apache.org/jira/browse/NIFI-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pierre Villard updated NIFI-3427: - Resolution: Fixed Fix Version/s: 1.2.0 Status: Resolved (was: Patch Available) > Data Provenance date/time sort ignores the date > --- > > Key: NIFI-3427 > URL: https://issues.apache.org/jira/browse/NIFI-3427 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.1.1 >Reporter: Justin Rittenhouse >Assignee: James Wing >Priority: Minor > Fix For: 1.2.0 > > Attachments: Screen Shot 2017-02-01 at 9.49.11 AM.png > > > The Data provenance display window that pops up doesn't properly sort by > date/time. It sorts by time, with events dated 11pm from January 31 > appearing before/after (depending on ascending or descending sort) events > dated February 1 before 11pm. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3427) Data Provenance date/time sort ignores the date
[ https://issues.apache.org/jira/browse/NIFI-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854376#comment-15854376 ] ASF GitHub Bot commented on NIFI-3427: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1465 > Data Provenance date/time sort ignores the date > --- > > Key: NIFI-3427 > URL: https://issues.apache.org/jira/browse/NIFI-3427 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.1.1 >Reporter: Justin Rittenhouse >Assignee: James Wing >Priority: Minor > Attachments: Screen Shot 2017-02-01 at 9.49.11 AM.png > > > The Data provenance display window that pops up doesn't properly sort by > date/time. It sorts by time, with events dated 11pm from January 31 > appearing before/after (depending on ascending or descending sort) events > dated February 1 before 11pm. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[jira] [Commented] (NIFI-3427) Data Provenance date/time sort ignores the date
[ https://issues.apache.org/jira/browse/NIFI-3427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15854375#comment-15854375 ] ASF subversion and git services commented on NIFI-3427: --- Commit e4cee7ee0d9ee4dfe3dcd2ba99f7bd1f6b3d38e4 in nifi's branch refs/heads/master from [~jameswing] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=e4cee7e ] NIFI-3427 Fix Provenance DateTime Sorting This closes #1465. > Data Provenance date/time sort ignores the date > --- > > Key: NIFI-3427 > URL: https://issues.apache.org/jira/browse/NIFI-3427 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.1.1 >Reporter: Justin Rittenhouse >Assignee: James Wing >Priority: Minor > Attachments: Screen Shot 2017-02-01 at 9.49.11 AM.png > > > The Data provenance display window that pops up doesn't properly sort by > date/time. It sorts by time, with events dated 11pm from January 31 > appearing before/after (depending on ascending or descending sort) events > dated February 1 before 11pm. -- This message was sent by Atlassian JIRA (v6.3.15#6346)
[GitHub] nifi pull request #1465: NIFI-3427 Fix Provenance DateTime Sorting
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1465 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---