[GitHub] nifi pull request #872: NIFI-2574 checkpoint commit to allow 1831 progress
GitHub user joewitt opened a pull request: https://github.com/apache/nifi/pull/872 NIFI-2574 checkpoint commit to allow 1831 progress Builds fine. Just cannot run tests. mvn clean install -DskipTests But this lets NIFI-1831 work continue while i clean up NiFiProperties static singleton usage Will remove getInstance() method altogether and will remove BOGUS_INSTANCE and will not extend Properties You can merge this pull request into a Git repository by running: $ git pull https://github.com/joewitt/incubator-nifi NIFI-2574 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/872.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #872 commit c3f406de785eb2dcdc1c0ab9a8b574b3ddac1f4a Author: joewitt Date: 2016-08-16T03:18:47Z NIFI-2574 checkpoint commit to allow 1831 progress --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2574) Update NiFiProperties usage to a context based accessor rather than static singleton
[ https://issues.apache.org/jira/browse/NIFI-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422108#comment-15422108 ] ASF GitHub Bot commented on NIFI-2574: -- GitHub user joewitt opened a pull request: https://github.com/apache/nifi/pull/872 NIFI-2574 checkpoint commit to allow 1831 progress Builds fine. Just cannot run tests. mvn clean install -DskipTests But this lets NIFI-1831 work continue while i clean up NiFiProperties static singleton usage Will remove getInstance() method altogether and will remove BOGUS_INSTANCE and will not extend Properties You can merge this pull request into a Git repository by running: $ git pull https://github.com/joewitt/incubator-nifi NIFI-2574 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/872.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #872 commit c3f406de785eb2dcdc1c0ab9a8b574b3ddac1f4a Author: joewitt Date: 2016-08-16T03:18:47Z NIFI-2574 checkpoint commit to allow 1831 progress > Update NiFiProperties usage to a context based accessor rather than static > singleton > > > Key: NIFI-2574 > URL: https://issues.apache.org/jira/browse/NIFI-2574 > Project: Apache NiFi > Issue Type: Sub-task > Components: Configuration, Core Framework >Reporter: Joseph Witt >Assignee: Joseph Witt > Fix For: 1.0.0 > > > This is a key step to move our longstanding odd static singleton usage of > NiFiProperties which enables a loader to exist which will encapsulate all > logic for dealing with encrypted properties as is the goal of the parent task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2576) PutSQL assumes all Timestamps are provided in Epoch
[ https://issues.apache.org/jira/browse/NIFI-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422106#comment-15422106 ] ASF GitHub Bot commented on NIFI-2576: -- Github user patricker commented on the issue: https://github.com/apache/nifi/pull/869 I can go for that. Updated. > PutSQL assumes all Timestamps are provided in Epoch > --- > > Key: NIFI-2576 > URL: https://issues.apache.org/jira/browse/NIFI-2576 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Peter Wicks > Original Estimate: 1h > Remaining Estimate: 1h > > When PutSQL sees a TIMESTAMP data type it assumes that it's being provided as > a Long in Epoch format. > This doesn't make much sense since the Query Database tools that return Avro > return timestamps as strings; and thus following the Avro->JSON->JSON To SQL > Route leads to timestamps as being strings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #869: NIFI-2576 - PutSQL assumes all Timestamps are provided in E...
Github user patricker commented on the issue: https://github.com/apache/nifi/pull/869 I can go for that. Updated. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Reopened] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build
[ https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt reopened NIFI-2519: --- Assignee: Joseph Witt (was: Oleg Zhurakousky) [~trixpan] please avoid putting a PR against a closed ticket. Could easily get missed/lost. Best to make a new JIRA or reopen. I've just reopened this one since the PR is already there > TestListenSMTP ValidEmail fails during parallel build > - > > Key: NIFI-2519 > URL: https://issues.apache.org/jira/browse/NIFI-2519 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Joseph Witt >Assignee: Joseph Witt > Fix For: 1.0.0 > > Attachments: > 0001-NIFI-2519-fixed-max-connections-and-max-size-enforce.patch > > > While running a full NiFi parallel build received the following. So there is > some test issue at least that is impacting build stability. > [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ > nifi-email-processors --- > [INFO] Changes detected - recompiling the module! > [INFO] Compiling 4 source files to > /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes > [WARNING] > /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24] > [deprecation] stop() in Thread has been deprecated > [WARNING] > /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24] > [deprecation] stop() in Thread has been deprecated > [WARNING] > /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24] > [deprecation] stop() in Thread has been deprecated > [INFO] > [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @ > nifi-email-processors --- > [INFO] Changes detected - recompiling the module! > [INFO] Nothing to compile - all classes are up to date > [INFO] > [INFO] --- maven-surefire-plugin:2.18:test (default-test) @ > nifi-email-processors --- > [INFO] Surefire report directory: > /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports > [INFO] Using configured provider > org.apache.maven.surefire.junit4.JUnit4Provider > --- > T E S T S > --- > Running org.apache.nifi.processors.email.TestListenSMTP > Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<< > FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP > ValidEmail(org.apache.nifi.processors.email.TestListenSMTP) Time elapsed: > 0.038 sec <<< FAILURE! > java.lang.AssertionError: Sending email failed > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.assertTrue(Assert.java:41) > at org.junit.Assert.assertFalse(Assert.java:64) > at > org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188) > Running org.apache.nifi.processors.email.TestExtractEmailAttachments > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in > org.apache.nifi.processors.email.TestExtractEmailAttachments > Running org.apache.nifi.processors.email.TestExtractEmailHeaders > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec - > in org.apache.nifi.processors.email.TestExtractEmailHeaders > Results : > Failed tests: > TestListenSMTP.ValidEmail:188 Sending email failed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build
[ https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422095#comment-15422095 ] ASF GitHub Bot commented on NIFI-2519: -- GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/871 NIFI-2519 - ListenSMTP addresses mismatch between smtp.src attribute … …documentation and attribute.put You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-2519 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/871.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #871 commit 47e4ca973c4d13be8452d09e67dd04d3493e4022 Author: Andre F de Miranda Date: 2016-08-15T23:13:39Z NIFI-2519 - ListenSMTP addresses mismatch between smtp.src attribute documentation and attribute.put > TestListenSMTP ValidEmail fails during parallel build > - > > Key: NIFI-2519 > URL: https://issues.apache.org/jira/browse/NIFI-2519 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Reporter: Joseph Witt >Assignee: Oleg Zhurakousky > Fix For: 1.0.0 > > Attachments: > 0001-NIFI-2519-fixed-max-connections-and-max-size-enforce.patch > > > While running a full NiFi parallel build received the following. So there is > some test issue at least that is impacting build stability. > [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @ > nifi-email-processors --- > [INFO] Changes detected - recompiling the module! > [INFO] Compiling 4 source files to > /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes > [WARNING] > /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24] > [deprecation] stop() in Thread has been deprecated > [WARNING] > /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24] > [deprecation] stop() in Thread has been deprecated > [WARNING] > /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24] > [deprecation] stop() in Thread has been deprecated > [INFO] > [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @ > nifi-email-processors --- > [INFO] Changes detected - recompiling the module! > [INFO] Nothing to compile - all classes are up to date > [INFO] > [INFO] --- maven-surefire-plugin:2.18:test (default-test) @ > nifi-email-processors --- > [INFO] Surefire report directory: > /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports > [INFO] Using configured provider > org.apache.maven.surefire.junit4.JUnit4Provider > --- > T E S T S > --- > Running org.apache.nifi.processors.email.TestListenSMTP > Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<< > FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP > ValidEmail(org.apache.nifi.processors.email.TestListenSMTP) Time elapsed: > 0.038 sec <<< FAILURE! > java.lang.AssertionError: Sending email failed > at org.junit.Assert.fail(Assert.java:88) > at org.junit.Assert.assertTrue(Assert.java:41) > at org.junit.Assert.assertFalse(Assert.java:64) > at > org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188) > Running org.apache.nifi.processors.email.TestExtractEmailAttachments > Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in > org.apache.nifi.processors.email.TestExtractEmailAttachments > Running org.apache.nifi.processors.email.TestExtractEmailHeaders > Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec - > in org.apache.nifi.processors.email.TestExtractEmailHeaders > Results : > Failed tests: > TestListenSMTP.ValidEmail:188 Sending email failed -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #871: NIFI-2519 - ListenSMTP addresses mismatch between sm...
GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/871 NIFI-2519 - ListenSMTP addresses mismatch between smtp.src attribute ⦠â¦documentation and attribute.put You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-2519 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/871.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #871 commit 47e4ca973c4d13be8452d09e67dd04d3493e4022 Author: Andre F de Miranda Date: 2016-08-15T23:13:39Z NIFI-2519 - ListenSMTP addresses mismatch between smtp.src attribute documentation and attribute.put --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build
[
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422093#comment-15422093
]
Andre commented on NIFI-2519:
-
So it turns out to be master still has the issue:
Code writes
{code}
attributes.put("smtp.remote.addr", context.getRemoteAddress().toString());
{code}
Resulting on the data to be present but under a different name.
Pushed a small PR to:
1. rename attribute.put
2. remove the leading {{/}} as it is a Java {{toString()}} thing more than a
proper IP notation.
3. reword property description property (as recording source IP and port is
good practice)
> TestListenSMTP ValidEmail fails during parallel build
> -
>
> Key: NIFI-2519
> URL: https://issues.apache.org/jira/browse/NIFI-2519
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0
>
> Attachments:
> 0001-NIFI-2519-fixed-max-connections-and-max-size-enforce.patch
>
>
> While running a full NiFi parallel build received the following. So there is
> some test issue at least that is impacting build stability.
> [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 4 source files to
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes
> [WARNING]
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24]
> [deprecation] stop() in Thread has been deprecated
> [WARNING]
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24]
> [deprecation] stop() in Thread has been deprecated
> [WARNING]
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24]
> [deprecation] stop() in Thread has been deprecated
> [INFO]
> [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO]
> [INFO] --- maven-surefire-plugin:2.18:test (default-test) @
> nifi-email-processors ---
> [INFO] Surefire report directory:
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports
> [INFO] Using configured provider
> org.apache.maven.surefire.junit4.JUnit4Provider
> ---
> T E S T S
> ---
> Running org.apache.nifi.processors.email.TestListenSMTP
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<<
> FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP
> ValidEmail(org.apache.nifi.processors.email.TestListenSMTP) Time elapsed:
> 0.038 sec <<< FAILURE!
> java.lang.AssertionError: Sending email failed
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188)
> Running org.apache.nifi.processors.email.TestExtractEmailAttachments
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in
> org.apache.nifi.processors.email.TestExtractEmailAttachments
> Running org.apache.nifi.processors.email.TestExtractEmailHeaders
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec -
> in org.apache.nifi.processors.email.TestExtractEmailHeaders
> Results :
> Failed tests:
> TestListenSMTP.ValidEmail:188 Sending email failed
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build
[
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422048#comment-15422048
]
Andre edited comment on NIFI-2519 at 8/16/16 2:08 AM:
--
---[~joewitt] I was migrating some real workloads from my dev version to the
new master and noted that {{smtp.src}} is not being written to the flowfile.---
Forget it. Wrong branch.
was (Author: trixpan):
[~joewitt] I was migrating some real workloads from my dev version to the new
master and noted that {{smtp.src}} is not being written to the flowfile.
> TestListenSMTP ValidEmail fails during parallel build
> -
>
> Key: NIFI-2519
> URL: https://issues.apache.org/jira/browse/NIFI-2519
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0
>
> Attachments:
> 0001-NIFI-2519-fixed-max-connections-and-max-size-enforce.patch
>
>
> While running a full NiFi parallel build received the following. So there is
> some test issue at least that is impacting build stability.
> [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 4 source files to
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes
> [WARNING]
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24]
> [deprecation] stop() in Thread has been deprecated
> [WARNING]
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24]
> [deprecation] stop() in Thread has been deprecated
> [WARNING]
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24]
> [deprecation] stop() in Thread has been deprecated
> [INFO]
> [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO]
> [INFO] --- maven-surefire-plugin:2.18:test (default-test) @
> nifi-email-processors ---
> [INFO] Surefire report directory:
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports
> [INFO] Using configured provider
> org.apache.maven.surefire.junit4.JUnit4Provider
> ---
> T E S T S
> ---
> Running org.apache.nifi.processors.email.TestListenSMTP
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<<
> FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP
> ValidEmail(org.apache.nifi.processors.email.TestListenSMTP) Time elapsed:
> 0.038 sec <<< FAILURE!
> java.lang.AssertionError: Sending email failed
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188)
> Running org.apache.nifi.processors.email.TestExtractEmailAttachments
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in
> org.apache.nifi.processors.email.TestExtractEmailAttachments
> Running org.apache.nifi.processors.email.TestExtractEmailHeaders
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec -
> in org.apache.nifi.processors.email.TestExtractEmailHeaders
> Results :
> Failed tests:
> TestListenSMTP.ValidEmail:188 Sending email failed
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2519) TestListenSMTP ValidEmail fails during parallel build
[
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422048#comment-15422048
]
Andre commented on NIFI-2519:
-
[~joewitt] I was migrating some real workloads from my dev version to the new
master and noted that {{smtp.src}} is not being written to the flowfile.
> TestListenSMTP ValidEmail fails during parallel build
> -
>
> Key: NIFI-2519
> URL: https://issues.apache.org/jira/browse/NIFI-2519
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
> Fix For: 1.0.0
>
> Attachments:
> 0001-NIFI-2519-fixed-max-connections-and-max-size-enforce.patch
>
>
> While running a full NiFi parallel build received the following. So there is
> some test issue at least that is impacting build stability.
> [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 4 source files to
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes
> [WARNING]
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24]
> [deprecation] stop() in Thread has been deprecated
> [WARNING]
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24]
> [deprecation] stop() in Thread has been deprecated
> [WARNING]
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24]
> [deprecation] stop() in Thread has been deprecated
> [INFO]
> [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO]
> [INFO] --- maven-surefire-plugin:2.18:test (default-test) @
> nifi-email-processors ---
> [INFO] Surefire report directory:
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports
> [INFO] Using configured provider
> org.apache.maven.surefire.junit4.JUnit4Provider
> ---
> T E S T S
> ---
> Running org.apache.nifi.processors.email.TestListenSMTP
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<<
> FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP
> ValidEmail(org.apache.nifi.processors.email.TestListenSMTP) Time elapsed:
> 0.038 sec <<< FAILURE!
> java.lang.AssertionError: Sending email failed
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188)
> Running org.apache.nifi.processors.email.TestExtractEmailAttachments
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in
> org.apache.nifi.processors.email.TestExtractEmailAttachments
> Running org.apache.nifi.processors.email.TestExtractEmailHeaders
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec -
> in org.apache.nifi.processors.email.TestExtractEmailHeaders
> Results :
> Failed tests:
> TestListenSMTP.ValidEmail:188 Sending email failed
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] nifi issue #869: NIFI-2576 - PutSQL assumes all Timestamps are provided in E...
Github user bbende commented on the issue: https://github.com/apache/nifi/pull/869 @patricker thanks for the contribution! Definitely makes sense to support both formats... What do you think about using a regex to determine if it is a long, and if so then parseLong, otherwise attempt to parse a date? (could be the other way around, but I was thinking the regex for a long is easiest) I'm thinking of the case where someone is always sending in longs and it is continuously throwing/catching the parse exception, that may end up being expensive. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2576) PutSQL assumes all Timestamps are provided in Epoch
[ https://issues.apache.org/jira/browse/NIFI-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15422018#comment-15422018 ] ASF GitHub Bot commented on NIFI-2576: -- Github user bbende commented on the issue: https://github.com/apache/nifi/pull/869 @patricker thanks for the contribution! Definitely makes sense to support both formats... What do you think about using a regex to determine if it is a long, and if so then parseLong, otherwise attempt to parse a date? (could be the other way around, but I was thinking the regex for a long is easiest) I'm thinking of the case where someone is always sending in longs and it is continuously throwing/catching the parse exception, that may end up being expensive. > PutSQL assumes all Timestamps are provided in Epoch > --- > > Key: NIFI-2576 > URL: https://issues.apache.org/jira/browse/NIFI-2576 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Peter Wicks > Original Estimate: 1h > Remaining Estimate: 1h > > When PutSQL sees a TIMESTAMP data type it assumes that it's being provided as > a Long in Epoch format. > This doesn't make much sense since the Query Database tools that return Avro > return timestamps as strings; and thus following the Avro->JSON->JSON To SQL > Route leads to timestamps as being strings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #866: NIFI-2566: Refactoring to improve robustness of cluster
Github user JPercivall commented on the issue: https://github.com/apache/nifi/pull/866 @markap14, I'm having trouble running a standalone instance with this PR. I get the at I can't start the FlowController due to (very long message, took the end): `Caused by: java.lang.IllegalStateException: The 'nifi.zookeeper.connect.string' property is not set in nifi.properties at org.apache.nifi.controller.cluster.ZooKeeperClientConfig.createConfig(ZooKeeperClientConfig.java:76) ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT] at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.(CuratorLeaderElectionManager.java:61) ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_74] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_74] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_74] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_74] at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:147) ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE] ... 52 common frames omitted` --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2566) Clustered Nodes can become out of sync regarding which node is coordinator
[ https://issues.apache.org/jira/browse/NIFI-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421911#comment-15421911 ] ASF GitHub Bot commented on NIFI-2566: -- Github user JPercivall commented on the issue: https://github.com/apache/nifi/pull/866 @markap14, I'm having trouble running a standalone instance with this PR. I get the at I can't start the FlowController due to (very long message, took the end): `Caused by: java.lang.IllegalStateException: The 'nifi.zookeeper.connect.string' property is not set in nifi.properties at org.apache.nifi.controller.cluster.ZooKeeperClientConfig.createConfig(ZooKeeperClientConfig.java:76) ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT] at org.apache.nifi.controller.leader.election.CuratorLeaderElectionManager.(CuratorLeaderElectionManager.java:61) ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT] at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) ~[na:1.8.0_74] at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62) ~[na:1.8.0_74] at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) ~[na:1.8.0_74] at java.lang.reflect.Constructor.newInstance(Constructor.java:423) ~[na:1.8.0_74] at org.springframework.beans.BeanUtils.instantiateClass(BeanUtils.java:147) ~[spring-beans-4.2.4.RELEASE.jar:4.2.4.RELEASE] ... 52 common frames omitted` > Clustered Nodes can become out of sync regarding which node is coordinator > -- > > Key: NIFI-2566 > URL: https://issues.apache.org/jira/browse/NIFI-2566 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.0.0 > > > Occasionally, I will see the UI telling me that no Cluster Coordinator has > been elected. However, I can see in the logs that the node is sending > heartbeats to the coordinator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2577) Increase default ORC stripe size to 64 MB in ConvertAvroToORC
[ https://issues.apache.org/jira/browse/NIFI-2577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-2577: --- Status: Patch Available (was: In Progress) > Increase default ORC stripe size to 64 MB in ConvertAvroToORC > - > > Key: NIFI-2577 > URL: https://issues.apache.org/jira/browse/NIFI-2577 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.0.0 > > > The current default stripe size for the ConvertAvroToORC processor is 100KB > which would cause performance issues for the normal use case. Suggestion is > to increase it to 64MB, which is what the default (for Apache ORC) is > currently specified as: > https://orc.apache.org/docs/hive-config.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2577) Increase default ORC stripe size to 64 MB in ConvertAvroToORC
[ https://issues.apache.org/jira/browse/NIFI-2577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421906#comment-15421906 ] ASF GitHub Bot commented on NIFI-2577: -- GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/870 NIFI-2577: Increased default stripe size in ConvertAvroToORC to 64MB This one's pretty straightforward ;) You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-2577 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/870.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #870 commit b6cbbf7f75d5c03174aaae60b0260518da103d04 Author: Matt Burgess Date: 2016-08-15T23:50:20Z NIFI-2577: Increased default stripe size in ConvertAvroToORC to 64MB > Increase default ORC stripe size to 64 MB in ConvertAvroToORC > - > > Key: NIFI-2577 > URL: https://issues.apache.org/jira/browse/NIFI-2577 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.0.0 > > > The current default stripe size for the ConvertAvroToORC processor is 100KB > which would cause performance issues for the normal use case. Suggestion is > to increase it to 64MB, which is what the default (for Apache ORC) is > currently specified as: > https://orc.apache.org/docs/hive-config.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #870: NIFI-2577: Increased default stripe size in ConvertA...
GitHub user mattyb149 opened a pull request: https://github.com/apache/nifi/pull/870 NIFI-2577: Increased default stripe size in ConvertAvroToORC to 64MB This one's pretty straightforward ;) You can merge this pull request into a Git repository by running: $ git pull https://github.com/mattyb149/nifi NIFI-2577 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/870.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #870 commit b6cbbf7f75d5c03174aaae60b0260518da103d04 Author: Matt Burgess Date: 2016-08-15T23:50:20Z NIFI-2577: Increased default stripe size in ConvertAvroToORC to 64MB --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-2577) Increase default ORC stripe size to 64 MB in ConvertAvroToORC
[ https://issues.apache.org/jira/browse/NIFI-2577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Burgess updated NIFI-2577: --- Description: The current default stripe size for the ConvertAvroToORC processor is 100KB which would cause performance issues for the normal use case. Suggestion is to increase it to 64MB, which is what the default (for Apache ORC) is currently specified as: https://orc.apache.org/docs/hive-config.html was:The current default stripe size for the ConvertAvroToORC processor is 100KB which would cause performance issues for the normal use case. Suggestion is to increase it to 64MB, which is what the default (for Apache ORC) is currently specified as. > Increase default ORC stripe size to 64 MB in ConvertAvroToORC > - > > Key: NIFI-2577 > URL: https://issues.apache.org/jira/browse/NIFI-2577 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.0.0 > > > The current default stripe size for the ConvertAvroToORC processor is 100KB > which would cause performance issues for the normal use case. Suggestion is > to increase it to 64MB, which is what the default (for Apache ORC) is > currently specified as: > https://orc.apache.org/docs/hive-config.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2577) Increase default ORC stripe size to 64 MB in ConvertAvroToORC
Matt Burgess created NIFI-2577: -- Summary: Increase default ORC stripe size to 64 MB in ConvertAvroToORC Key: NIFI-2577 URL: https://issues.apache.org/jira/browse/NIFI-2577 Project: Apache NiFi Issue Type: Improvement Affects Versions: 1.0.0 Reporter: Matt Burgess Assignee: Matt Burgess Fix For: 1.0.0 The current default stripe size for the ConvertAvroToORC processor is 100KB which would cause performance issues for the normal use case. Suggestion is to increase it to 64MB, which is what the default (for Apache ORC) is currently specified as. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2410) Document controller service display
[ https://issues.apache.org/jira/browse/NIFI-2410?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421863#comment-15421863 ] Sarah Olson commented on NIFI-2410: --- Adding comment from 2515 for additional context: Controller services work as anticipated but the way they are structured is a bit of a departure from how they used to work. Would be helpful to have docs that inform users of how each is structured. It seems that Flow Controller level controller services are only there for other controller services and/or reporting tasks. These are separate from controller services that are for an associated process group (or parent) which are nested. The access/visibility of controller services seems a bit confusing to me and don't see any docs that clarify. It seems that for nested process groups, I can view all controller services in parent process groups. The configuration on the "Operate" palette seems to control these which seems to be completely disjoint from the flow level controller settings, which are not listed in any of the nested process groups. I would imagine the flow level controller settings would also be present in each of the child PGs regardless of nesting level but these do not seem to be visible to any of my components. Are these just to support reporting tasks? Would reporting tasks also have a similar hierarchy where they could also be contained within a given PG chain? > Document controller service display > --- > > Key: NIFI-2410 > URL: https://issues.apache.org/jira/browse/NIFI-2410 > Project: Apache NiFi > Issue Type: Task > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Haimo Liu >Assignee: Sarah Olson >Priority: Critical > Fix For: 1.0.0 > > Attachments: Screen Shot 2016-07-26 at 10.36.08 PM.png, Screen Shot > 2016-07-26 at 10.57.19 PM.png, Screen Shot 2016-07-26 at 11.05.42 PM.png > > > NIFI 1.0 snapshot, 7/26 build, didn't turn on internal authorization/policy > management, running in a clustered mode (3-node). > Group level CS display seems inconsistent > 1. hamburger menu -> Controller Settings -> Controller Services, can see one > CS (no matter which PG i am in, can always see this single CS, > "DBCPConnectionPool", see attached image) this CS is only referenceable by > Reporting Tasks, NOT by any processors > 2. Select a PG where I have the GenerateTableFetch processor, click the > "Configuration" icon in "Operate panel", can see five different controller > services here, these are all PG level CS. Select nothing (click the canvas), > click the "Configuration" icon in "Operate panel", can see two different > controller services here, these are root level CS > in NIFI 1.0, we need to document the difference between CS that are > referenceable by processors. vs that are only referenceable by reporting tasks -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-2514) Update documentation on handling of Controller Services
[
https://issues.apache.org/jira/browse/NIFI-2514?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sarah Olson resolved NIFI-2514.
---
Resolution: Duplicate
> Update documentation on handling of Controller Services
> ---
>
> Key: NIFI-2514
> URL: https://issues.apache.org/jira/browse/NIFI-2514
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Documentation & Website
>Affects Versions: 1.0.0-Beta
>Reporter: Aldrin Piri
>Assignee: Sarah Olson
> Fix For: 1.0.0
>
>
> Controller services work as anticipated but the way they are structured is a
> bit of a departure from how they used to work. Would be helpful to have docs
> that inform users of how each is structured.
> It seems that Flow Controller level controller services are only there for
> other controller services and/or reporting tasks.
> These are separate from controller services that are for an associated
> process group (or parent) which are nested.
> {quote}
> The access/visibility of controller services seems a bit confusing to me and
> don't see any docs that clarify. It seems that for nested process groups, I
> can view all controller services in parent process groups. The configuration
> on the "Operate" palette seems to control these which seems to be completely
> disjoint from the flow level controller settings, which are not listed in any
> of the nested process groups. I would imagine the flow level controller
> settings would also be present in each of the child PGs regardless of nesting
> level but these do not seem to be visible to any of my components. Are these
> just to support reporting tasks? Would reporting tasks also have a similar
> hierarchy where they could also be contained within a given PG chain?
> {quote}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-1833) Add support for Azure Blob Storage and Table Storage
[ https://issues.apache.org/jira/browse/NIFI-1833?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421831#comment-15421831 ] ASF GitHub Bot commented on NIFI-1833: -- Github user simonellistonball commented on the issue: https://github.com/apache/nifi/pull/399 Latest commit should remove some of the duplicate nar circular dependency issues. It's not the cleanest way to implement this, but that really wants a common cloud util library around the way cases like listing are handles. That feels like the scope for a much greater refactoring. > Add support for Azure Blob Storage and Table Storage > > > Key: NIFI-1833 > URL: https://issues.apache.org/jira/browse/NIFI-1833 > Project: Apache NiFi > Issue Type: Improvement > Components: Extensions >Affects Versions: 0.6.1 >Reporter: Simon Elliston Ball >Priority: Minor > > It would be useful to have an Azure equivalent of the current S3 capability. > Azure also provides a Table storage mechanism, providing simple key value > storage. Since the Azure SDKs are Apache Licensed, this should be reasonably > straightforward. A first cut is available as an addition to the existing > azure bundle. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #399: NIFI-1833 - Adding Azure Storage processors
Github user simonellistonball commented on the issue: https://github.com/apache/nifi/pull/399 Latest commit should remove some of the duplicate nar circular dependency issues. It's not the cleanest way to implement this, but that really wants a common cloud util library around the way cases like listing are handles. That feels like the scope for a much greater refactoring. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2451) Need to update Encryption Configuration content in Admin guide for 1.0 changes
[ https://issues.apache.org/jira/browse/NIFI-2451?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421813#comment-15421813 ] ASF GitHub Bot commented on NIFI-2451: -- Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/855 This is going to change (potentially substantially) with the new `NiFiProperties` refactor going on because of PR 834. I will update as necessary. > Need to update Encryption Configuration content in Admin guide for 1.0 changes > -- > > Key: NIFI-2451 > URL: https://issues.apache.org/jira/browse/NIFI-2451 > Project: Apache NiFi > Issue Type: Bug > Components: Documentation & Website >Affects Versions: 1.0.0 >Reporter: Andrew Lim >Assignee: Andrew Lim > Fix For: 1.0.0 > > > Will need to document changes in Admin Guide for: > NIFI-1831 for Encryption Configuration -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #855: NIFI-2451 Update Admin guide for encrypt-config command uti...
Github user alopresto commented on the issue: https://github.com/apache/nifi/pull/855 This is going to change (potentially substantially) with the new `NiFiProperties` refactor going on because of PR 834. I will update as necessary. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #869: NIFI-2576 - PutSQL assumes all Timestamps are provid...
GitHub user patricker opened a pull request: https://github.com/apache/nifi/pull/869 NIFI-2576 - PutSQL assumes all Timestamps are provided in Epoch This change allows for either epoch timestamps or "-MM-dd hh:mm:ss.SSS" format; which is the format you get when you call ResultSet.getString on a JDBC Timestamp column. Also include a unit test for PutSQL to test both flavors of Timestamps. You can merge this pull request into a Git repository by running: $ git pull https://github.com/patricker/nifi master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/869.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #869 commit 92eb864c96dc3096c28f6d3a2e13402d7bc0a292 Author: Peter Wicks Date: 2016-08-15T16:49:37Z NIFI-2356 commit b28806b52f0a7c7d35e2713db0495c9af6a55641 Author: Peter Wicks Date: 2016-08-15T17:47:46Z Sync commit ab0568341c5639998f1a292311a58069ead21e7a Author: Peter Wicks Date: 2016-08-15T21:50:47Z NIFI-2576 commit 1c7ea31346ce1dbd80b2fd392be1d2f4dfccbabe Author: Peter Wicks Date: 2016-08-15T21:52:45Z Merge remote-tracking branch 'upstream/master' commit 4ce80fa69ac8524756818dd6d530954985b430c5 Author: Peter Wicks Date: 2016-08-15T22:00:31Z NIFI-2576 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2576) PutSQL assumes all Timestamps are provided in Epoch
[ https://issues.apache.org/jira/browse/NIFI-2576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421757#comment-15421757 ] ASF GitHub Bot commented on NIFI-2576: -- GitHub user patricker opened a pull request: https://github.com/apache/nifi/pull/869 NIFI-2576 - PutSQL assumes all Timestamps are provided in Epoch This change allows for either epoch timestamps or "-MM-dd hh:mm:ss.SSS" format; which is the format you get when you call ResultSet.getString on a JDBC Timestamp column. Also include a unit test for PutSQL to test both flavors of Timestamps. You can merge this pull request into a Git repository by running: $ git pull https://github.com/patricker/nifi master Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/869.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #869 commit 92eb864c96dc3096c28f6d3a2e13402d7bc0a292 Author: Peter Wicks Date: 2016-08-15T16:49:37Z NIFI-2356 commit b28806b52f0a7c7d35e2713db0495c9af6a55641 Author: Peter Wicks Date: 2016-08-15T17:47:46Z Sync commit ab0568341c5639998f1a292311a58069ead21e7a Author: Peter Wicks Date: 2016-08-15T21:50:47Z NIFI-2576 commit 1c7ea31346ce1dbd80b2fd392be1d2f4dfccbabe Author: Peter Wicks Date: 2016-08-15T21:52:45Z Merge remote-tracking branch 'upstream/master' commit 4ce80fa69ac8524756818dd6d530954985b430c5 Author: Peter Wicks Date: 2016-08-15T22:00:31Z NIFI-2576 > PutSQL assumes all Timestamps are provided in Epoch > --- > > Key: NIFI-2576 > URL: https://issues.apache.org/jira/browse/NIFI-2576 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 >Reporter: Peter Wicks > Original Estimate: 1h > Remaining Estimate: 1h > > When PutSQL sees a TIMESTAMP data type it assumes that it's being provided as > a Long in Epoch format. > This doesn't make much sense since the Query Database tools that return Avro > return timestamps as strings; and thus following the Avro->JSON->JSON To SQL > Route leads to timestamps as being strings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #868: NIFI-2556: Updated User Guide with informaiton about...
GitHub user thesolson opened a pull request: https://github.com/apache/nifi/pull/868 NIFI-2556: Updated User Guide with informaiton about custom properties You can merge this pull request into a Git repository by running: $ git pull https://github.com/thesolson/nifi NIFI-2556 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/868.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #868 commit a13a5eb2189431005178cfb1090995e7a286e29a Author: Sarah Olson Date: 2016-08-15T21:31:58Z NIFI-2556: Updated User Guide with informaiton about custom properties --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2556) User Guide Update: Considerations for creating custom properties
[ https://issues.apache.org/jira/browse/NIFI-2556?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421745#comment-15421745 ] ASF GitHub Bot commented on NIFI-2556: -- GitHub user thesolson opened a pull request: https://github.com/apache/nifi/pull/868 NIFI-2556: Updated User Guide with informaiton about custom properties You can merge this pull request into a Git repository by running: $ git pull https://github.com/thesolson/nifi NIFI-2556 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/868.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #868 commit a13a5eb2189431005178cfb1090995e7a286e29a Author: Sarah Olson Date: 2016-08-15T21:31:58Z NIFI-2556: Updated User Guide with informaiton about custom properties > User Guide Update: Considerations for creating custom properties > > > Key: NIFI-2556 > URL: https://issues.apache.org/jira/browse/NIFI-2556 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Sarah Olson >Assignee: Sarah Olson > Fix For: 1.0.0 > > > Update the User Guide with information about creating custom properties. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2576) PutSQL assumes all Timestamps are provided in Epoch
Peter Wicks created NIFI-2576: - Summary: PutSQL assumes all Timestamps are provided in Epoch Key: NIFI-2576 URL: https://issues.apache.org/jira/browse/NIFI-2576 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 1.0.0 Reporter: Peter Wicks When PutSQL sees a TIMESTAMP data type it assumes that it's being provided as a Long in Epoch format. This doesn't make much sense since the Query Database tools that return Avro return timestamps as strings; and thus following the Avro->JSON->JSON To SQL Route leads to timestamps as being strings. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2575) HiveQL Processors Fail due to invalid JDBC URI resolution when using Zookeeper URI
[
https://issues.apache.org/jira/browse/NIFI-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yolanda M. Davis updated NIFI-2575:
---
Fix Version/s: (was: 1.0.0)
> HiveQL Processors Fail due to invalid JDBC URI resolution when using
> Zookeeper URI
> --
>
> Key: NIFI-2575
> URL: https://issues.apache.org/jira/browse/NIFI-2575
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Yolanda M. Davis
>
> When configuring a HiveQL processor using the Zookeeper URL (e.g.
> jdbc:hive2://ydavis-hdp-nifi-test-3.openstacklocal:2181,ydavis-hdp-nifi-test-1.openstacklocal:2181,ydavis-hdp-nifi-test-2.openstacklocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2),
> it appears that the JDBC driver does not properly build the the uri in the
> expected format. This is because HS2 is storing JDBC parameters in ZK
> (https://issues.apache.org/jira/browse/HIVE-11581) and it is expecting the
> driver to be able to parse and use those values to configure the connection.
> However it appears the driver is expecting zookeeper to simply return the
> host:port and subsequently building an invalid URI.
> This problem has result in two variation of errors. The following was
> experienced by [~mattyb149]
> {noformat}
> 2016-08-15 12:45:12,918 INFO [Timer-Driven Process Thread-2]
> org.apache.hive.jdbc.Utils Resolved authority:
> hive.server2.authentication=KERBEROS;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=hdp-cluster-2-2.novalocal;hive.server2.thrift.port=1;hive.server2.use.SSL=false;hive.server2.authentication.kerberos.principal=hive/_h...@hdf.com
> 2016-08-15 12:45:13,835 INFO [Timer-Driven Process Thread-2]
> org.apache.hive.jdbc.HiveConnection Will try to open client transport with
> JDBC Uri:
> jdbc:hive2://hive.server2.authentication=KERBEROS;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=hdp-cluster-2-2.novalocal;hive.server2.thrift.port=1;hive.server2.use.SSL=false;hive.server2.authentication.kerberos.principal=hive/_h...@hdf.com/default;principal=hive/_h...@hdf.com;serviceDiscoveryMode=zookeeper;zooKeeperNamespace=hiveserver2
> 2016-08-15 12:45:13,835 INFO [Timer-Driven Process Thread-2]
> org.apache.hive.jdbc.HiveConnection Could not open client transport with JDBC
> Uri:
> jdbc:hive2://hive.server2.authentication=KERBEROS;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=hdp-cluster-2-2.novalocal;hive.server2.thrift.port=1;hive.server2.use.SSL=false;hive.server2.authentication.kerberos.principal=hive/_h...@hdf.com/default;principal=hive/_h...@hdf.com;serviceDiscoveryMode=zookeeper;zooKeeperNamespace=hiveserver2
> 2016-08-15 12:45:13,836 INFO [Timer-Driven Process Thread-2]
> o.a.c.f.imps.CuratorFrameworkImpl Starting
> 2016-08-15 12:45:14,064 INFO [Timer-Driven Process Thread-2-EventThread]
> o.a.c.f.state.ConnectionStateManager State change: CONNECTED
> 2016-08-15 12:45:14,182 INFO [Curator-Framework-0]
> o.a.c.f.imps.CuratorFrameworkImpl backgroundOperationsLoop exiting
> 2016-08-15 12:45:14,337 ERROR [Timer-Driven Process Thread-2]
> o.a.nifi.processors.hive.SelectHiveQL
> SelectHiveQL[id=7aaffd71-0156-1000-d962-8102c06b23df]
> SelectHiveQL[id=7aaffd71-0156-1000-d962-8102c06b23df] failed to process due
> to java.lang.reflect.UndeclaredThrowableException; rolling back session:
> java.lang.reflect.UndeclaredThrowableException
> 2016-08-15 12:45:14,346 ERROR [Timer-Driven Process Thread-2]
> o.a.nifi.processors.hive.SelectHiveQL
> java.lang.reflect.UndeclaredThrowableException: null
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> ~[na:na]
> at
> org.apache.nifi.dbcp.hive.HiveConnectionPool.getConnection(HiveConnectionPool.java:255)
> ~[na:na]
> at sun.reflect.GeneratedMethodAccessor331.invoke(Unknown
> Source) ~[na:na]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_65]
> at java.lang.reflect.Method.invoke(Method.java:497)
> ~[na:1.8.0_65]
> at
> org.apache.nifi.controller.service.StandardControllerServiceProvider$1.invoke(StandardControllerServiceProvider.java:174)
> ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy81.getConnection(Unknown Source) ~[na:na]
> at
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:158)
> ~[na:na]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> ~[nifi-api-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
>
[jira] [Updated] (NIFI-2534) Error when stopping a process group the user doesn't have any permission for
[
https://issues.apache.org/jira/browse/NIFI-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Scott Aslan updated NIFI-2534:
--
Status: Patch Available (was: In Progress)
> Error when stopping a process group the user doesn't have any permission for
>
>
> Key: NIFI-2534
> URL: https://issues.apache.org/jira/browse/NIFI-2534
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Bryan Bende
>Assignee: Scott Aslan
>Priority: Minor
> Fix For: 1.0.0
>
>
> I created a process group in a secure instance, then overrode the policy for
> "view component" such that no user had view component for the group (left the
> modify component inheriting from parent). Then right-clicked the process
> group and chose stop, and the circle in the header stayed spinning.
> In Chrome Dev tools the error showed on line 1090 of nf-process-group.js
> if (processGroupMap.has(processGroup.id)) {
> Saying "id" is undefined.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2534) Error when stopping a process group the user doesn't have any permission for
[
https://issues.apache.org/jira/browse/NIFI-2534?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421670#comment-15421670
]
ASF GitHub Bot commented on NIFI-2534:
--
GitHub user scottyaslan opened a pull request:
https://github.com/apache/nifi/pull/867
[NIFI-2534] Fix JS error when stopping a process group
You can merge this pull request into a Git repository by running:
$ git pull https://github.com/scottyaslan/nifi responsiveDevBranch
Alternatively you can review and apply these changes as the patch at:
https://github.com/apache/nifi/pull/867.patch
To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:
This closes #867
commit 318a60cb5a32529b4502a249c5d435b274e31178
Author: Scott Aslan
Date: 2016-08-15T21:04:28Z
[NIFI-2534] Fix JS error when stopping a process group
> Error when stopping a process group the user doesn't have any permission for
>
>
> Key: NIFI-2534
> URL: https://issues.apache.org/jira/browse/NIFI-2534
> Project: Apache NiFi
> Issue Type: Bug
>Affects Versions: 1.0.0
>Reporter: Bryan Bende
>Assignee: Scott Aslan
>Priority: Minor
> Fix For: 1.0.0
>
>
> I created a process group in a secure instance, then overrode the policy for
> "view component" such that no user had view component for the group (left the
> modify component inheriting from parent). Then right-clicked the process
> group and chose stop, and the circle in the header stayed spinning.
> In Chrome Dev tools the error showed on line 1090 of nf-process-group.js
> if (processGroupMap.has(processGroup.id)) {
> Saying "id" is undefined.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] nifi pull request #867: [NIFI-2534] Fix JS error when stopping a process gro...
GitHub user scottyaslan opened a pull request: https://github.com/apache/nifi/pull/867 [NIFI-2534] Fix JS error when stopping a process group You can merge this pull request into a Git repository by running: $ git pull https://github.com/scottyaslan/nifi responsiveDevBranch Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/867.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #867 commit 318a60cb5a32529b4502a249c5d435b274e31178 Author: Scott Aslan Date: 2016-08-15T21:04:28Z [NIFI-2534] Fix JS error when stopping a process group --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-2551) Rare condition causes FileSystemRepository NPE
[
https://issues.apache.org/jira/browse/NIFI-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-2551:
-
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Rare condition causes FileSystemRepository NPE
> --
>
> Key: NIFI-2551
> URL: https://issues.apache.org/jira/browse/NIFI-2551
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Michael Moser
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 1.0.0, 0.8.0
>
>
> In rare unpredictable cases when NiFi is processing a heavy load, we see
> FileSystemRepository throw a NullPointerException
> {noformat}
> java.lang.NullPointerException
> at o.a.n.c.r.FileSystemRepository$2.write(FileSystemRepository.java:918)
> [nifi-framework-core-0.7.0.jar]
> at
> o.a.n.c.r.io.DisableOnCloseOutputStream.write(DisableOnCloseOutputStream.java:49)
>
> Suppressed: java.lang.NullPointerException
> at
> o.a.n.c.r.FileSystemRepository$2.flush(FileSystemRepository.java:934)
> [nifi-framework-core-0.7.0.jar]
> at
> o.a.n.c.r.io.DisableOnCloseOutputStream.close(DisableOnCloseOutputStream.java:68)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2551) Rare condition causes FileSystemRepository NPE
[
https://issues.apache.org/jira/browse/NIFI-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421667#comment-15421667
]
ASF GitHub Bot commented on NIFI-2551:
--
Github user markap14 closed the pull request at:
https://github.com/apache/nifi/pull/847
> Rare condition causes FileSystemRepository NPE
> --
>
> Key: NIFI-2551
> URL: https://issues.apache.org/jira/browse/NIFI-2551
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Michael Moser
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 1.0.0, 0.8.0
>
>
> In rare unpredictable cases when NiFi is processing a heavy load, we see
> FileSystemRepository throw a NullPointerException
> {noformat}
> java.lang.NullPointerException
> at o.a.n.c.r.FileSystemRepository$2.write(FileSystemRepository.java:918)
> [nifi-framework-core-0.7.0.jar]
> at
> o.a.n.c.r.io.DisableOnCloseOutputStream.write(DisableOnCloseOutputStream.java:49)
>
> Suppressed: java.lang.NullPointerException
> at
> o.a.n.c.r.FileSystemRepository$2.flush(FileSystemRepository.java:934)
> [nifi-framework-core-0.7.0.jar]
> at
> o.a.n.c.r.io.DisableOnCloseOutputStream.close(DisableOnCloseOutputStream.java:68)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2551) Rare condition causes FileSystemRepository NPE
[
https://issues.apache.org/jira/browse/NIFI-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421666#comment-15421666
]
ASF subversion and git services commented on NIFI-2551:
---
Commit 6af768d0a133c63239e50213f69ee7d36d91448e in nifi's branch
refs/heads/master from [~markap14]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=6af768d ]
NIFI-2551: Addressed a threading issue in the StandardResourceClaimManager and
performed some refactoring so that we can ensure thread-safety across different
components of the application, such as ProcessSession and WriteAheadFlowFile
Repository when interacting with the StandardResourceClaimManager. Update
DebugFlow to allow it to write to a FlowFile multiple times, which exposes the
concurrency bug. Also avoided calling ContentRepository.remove() from
ProcessSession whenever the content is no longer needed, as that can cause
problems now that the Resource Claim is backing the content claim.
> Rare condition causes FileSystemRepository NPE
> --
>
> Key: NIFI-2551
> URL: https://issues.apache.org/jira/browse/NIFI-2551
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Michael Moser
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 1.0.0, 0.8.0
>
>
> In rare unpredictable cases when NiFi is processing a heavy load, we see
> FileSystemRepository throw a NullPointerException
> {noformat}
> java.lang.NullPointerException
> at o.a.n.c.r.FileSystemRepository$2.write(FileSystemRepository.java:918)
> [nifi-framework-core-0.7.0.jar]
> at
> o.a.n.c.r.io.DisableOnCloseOutputStream.write(DisableOnCloseOutputStream.java:49)
>
> Suppressed: java.lang.NullPointerException
> at
> o.a.n.c.r.FileSystemRepository$2.flush(FileSystemRepository.java:934)
> [nifi-framework-core-0.7.0.jar]
> at
> o.a.n.c.r.io.DisableOnCloseOutputStream.close(DisableOnCloseOutputStream.java:68)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2551) Rare condition causes FileSystemRepository NPE
[
https://issues.apache.org/jira/browse/NIFI-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421668#comment-15421668
]
ASF GitHub Bot commented on NIFI-2551:
--
Github user markap14 closed the pull request at:
https://github.com/apache/nifi/pull/848
> Rare condition causes FileSystemRepository NPE
> --
>
> Key: NIFI-2551
> URL: https://issues.apache.org/jira/browse/NIFI-2551
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Michael Moser
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 1.0.0, 0.8.0
>
>
> In rare unpredictable cases when NiFi is processing a heavy load, we see
> FileSystemRepository throw a NullPointerException
> {noformat}
> java.lang.NullPointerException
> at o.a.n.c.r.FileSystemRepository$2.write(FileSystemRepository.java:918)
> [nifi-framework-core-0.7.0.jar]
> at
> o.a.n.c.r.io.DisableOnCloseOutputStream.write(DisableOnCloseOutputStream.java:49)
>
> Suppressed: java.lang.NullPointerException
> at
> o.a.n.c.r.FileSystemRepository$2.flush(FileSystemRepository.java:934)
> [nifi-framework-core-0.7.0.jar]
> at
> o.a.n.c.r.io.DisableOnCloseOutputStream.close(DisableOnCloseOutputStream.java:68)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] nifi pull request #847: NIFI-2551: Addressed a threading issue in the Standa...
Github user markap14 closed the pull request at: https://github.com/apache/nifi/pull/847 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #848: NIFI-2551: Addressed a threading issue in the Standa...
Github user markap14 closed the pull request at: https://github.com/apache/nifi/pull/848 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2551) Rare condition causes FileSystemRepository NPE
[
https://issues.apache.org/jira/browse/NIFI-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421662#comment-15421662
]
ASF subversion and git services commented on NIFI-2551:
---
Commit 25f816d32ab466d2ac1ad96cdb95e7359d085884 in nifi's branch refs/heads/0.x
from [~markap14]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=25f816d ]
NIFI-2551: Addressed a threading issue in the StandardResourceClaimManager and
performed some refactoring so that we can ensure thread-safety across different
components of the application, such as ProcessSession and WriteAheadFlowFile
Repository when interacting with the StandardResourceClaimManager. Update
DebugFlow to allow it to write to a FlowFile multiple times, which exposes the
concurrency bug. Also avoided calling ContentRepository.remove() from
ProcessSession whenever the content is no longer needed, as that can cause
problems now that the Resource Claim is backing the content claim.
> Rare condition causes FileSystemRepository NPE
> --
>
> Key: NIFI-2551
> URL: https://issues.apache.org/jira/browse/NIFI-2551
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Michael Moser
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 1.0.0, 0.8.0
>
>
> In rare unpredictable cases when NiFi is processing a heavy load, we see
> FileSystemRepository throw a NullPointerException
> {noformat}
> java.lang.NullPointerException
> at o.a.n.c.r.FileSystemRepository$2.write(FileSystemRepository.java:918)
> [nifi-framework-core-0.7.0.jar]
> at
> o.a.n.c.r.io.DisableOnCloseOutputStream.write(DisableOnCloseOutputStream.java:49)
>
> Suppressed: java.lang.NullPointerException
> at
> o.a.n.c.r.FileSystemRepository$2.flush(FileSystemRepository.java:934)
> [nifi-framework-core-0.7.0.jar]
> at
> o.a.n.c.r.io.DisableOnCloseOutputStream.close(DisableOnCloseOutputStream.java:68)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2551) Rare condition causes FileSystemRepository NPE
[
https://issues.apache.org/jira/browse/NIFI-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421661#comment-15421661
]
ASF GitHub Bot commented on NIFI-2551:
--
Github user markap14 commented on the issue:
https://github.com/apache/nifi/pull/847
Great, thanks, @mosermw ! Will remove those outdated comments and push.
> Rare condition causes FileSystemRepository NPE
> --
>
> Key: NIFI-2551
> URL: https://issues.apache.org/jira/browse/NIFI-2551
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Michael Moser
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 1.0.0, 0.8.0
>
>
> In rare unpredictable cases when NiFi is processing a heavy load, we see
> FileSystemRepository throw a NullPointerException
> {noformat}
> java.lang.NullPointerException
> at o.a.n.c.r.FileSystemRepository$2.write(FileSystemRepository.java:918)
> [nifi-framework-core-0.7.0.jar]
> at
> o.a.n.c.r.io.DisableOnCloseOutputStream.write(DisableOnCloseOutputStream.java:49)
>
> Suppressed: java.lang.NullPointerException
> at
> o.a.n.c.r.FileSystemRepository$2.flush(FileSystemRepository.java:934)
> [nifi-framework-core-0.7.0.jar]
> at
> o.a.n.c.r.io.DisableOnCloseOutputStream.close(DisableOnCloseOutputStream.java:68)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] nifi issue #847: NIFI-2551: Addressed a threading issue in the StandardResou...
Github user markap14 commented on the issue: https://github.com/apache/nifi/pull/847 Great, thanks, @mosermw ! Will remove those outdated comments and push. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Assigned] (NIFI-2551) Rare condition causes FileSystemRepository NPE
[
https://issues.apache.org/jira/browse/NIFI-2551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne reassigned NIFI-2551:
Assignee: Mark Payne (was: Michael Moser)
> Rare condition causes FileSystemRepository NPE
> --
>
> Key: NIFI-2551
> URL: https://issues.apache.org/jira/browse/NIFI-2551
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Michael Moser
>Assignee: Mark Payne
>Priority: Blocker
> Fix For: 1.0.0, 0.8.0
>
>
> In rare unpredictable cases when NiFi is processing a heavy load, we see
> FileSystemRepository throw a NullPointerException
> {noformat}
> java.lang.NullPointerException
> at o.a.n.c.r.FileSystemRepository$2.write(FileSystemRepository.java:918)
> [nifi-framework-core-0.7.0.jar]
> at
> o.a.n.c.r.io.DisableOnCloseOutputStream.write(DisableOnCloseOutputStream.java:49)
>
> Suppressed: java.lang.NullPointerException
> at
> o.a.n.c.r.FileSystemRepository$2.flush(FileSystemRepository.java:934)
> [nifi-framework-core-0.7.0.jar]
> at
> o.a.n.c.r.io.DisableOnCloseOutputStream.close(DisableOnCloseOutputStream.java:68)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-2552) HiveQL processors fail when Zookeeper quorum in URL
[ https://issues.apache.org/jira/browse/NIFI-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yolanda M. Davis updated NIFI-2552: --- Resolution: Resolved Status: Resolved (was: Patch Available) > HiveQL processors fail when Zookeeper quorum in URL > --- > > Key: NIFI-2552 > URL: https://issues.apache.org/jira/browse/NIFI-2552 > Project: Apache NiFi > Issue Type: Bug >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.0.0 > > > [~YolandaMDavis] discovered a bug in the HiveQL processors, when using a > Zookeeper quorum in the Hive JDBC URL: > java.lang.NoSuchMethodError: > org.apache.curator.utils.ZKPaths.fixForNamespace(Ljava/lang/String;Ljava/lang/String;Z)Ljava/lang/String; > at > org.apache.curator.framework.imps.NamespaceImpl.fixForNamespace(NamespaceImpl.java:104) > ~[curator-framework-2.11.0.jar:na] > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.fixForNamespace(CuratorFrameworkImpl.java:594) > ~[curator-framework-2.11.0.jar:na] > at > org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:198) > ~[curator-framework-2.11.0.jar:na] > at > org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:40) > ~[curator-framework-2.11.0.jar:na] > This method is in the Apache Curator 2.11.0 JAR (and has been since 2.8.0), > but the Hive NAR's parent is the Hadoop Libraries NAR, which transitively > brings in Curator 2.6.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2566) Clustered Nodes can become out of sync regarding which node is coordinator
[
https://issues.apache.org/jira/browse/NIFI-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421657#comment-15421657
]
ASF GitHub Bot commented on NIFI-2566:
--
Github user JPercivall commented on a diff in the pull request:
https://github.com/apache/nifi/pull/866#discussion_r74833401
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/cluster/ClusterProtocolHeartbeater.java
---
@@ -18,102 +18,80 @@
package org.apache.nifi.controller.cluster;
import java.io.IOException;
-import java.nio.charset.StandardCharsets;
-import java.util.Properties;
-
-import org.apache.curator.RetryPolicy;
-import org.apache.curator.framework.CuratorFramework;
-import org.apache.curator.framework.CuratorFrameworkFactory;
-import org.apache.curator.retry.RetryNTimes;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import org.apache.nifi.cluster.coordination.ClusterCoordinator;
+import org.apache.nifi.cluster.coordination.node.ClusterRoles;
+import org.apache.nifi.cluster.coordination.node.NodeConnectionStatus;
+import org.apache.nifi.cluster.protocol.HeartbeatPayload;
+import org.apache.nifi.cluster.protocol.NodeIdentifier;
import org.apache.nifi.cluster.protocol.NodeProtocolSender;
import org.apache.nifi.cluster.protocol.ProtocolException;
import org.apache.nifi.cluster.protocol.message.HeartbeatMessage;
-import org.apache.zookeeper.WatchedEvent;
-import org.apache.zookeeper.Watcher;
+import org.apache.nifi.cluster.protocol.message.HeartbeatResponseMessage;
+import org.apache.nifi.controller.leader.election.LeaderElectionManager;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
- * Uses ZooKeeper in order to determine which node is the elected Cluster
Coordinator and to indicate
- * that this node is part of the cluster. However, once the Cluster
Coordinator is known, heartbeats are
+ * Uses Leader Election Manager in order to determine which node is the
elected Cluster Coordinator and to indicate
+ * that this node is part of the cluster. Once the Cluster Coordinator is
known, heartbeats are
* sent directly to the Cluster Coordinator.
*/
public class ClusterProtocolHeartbeater implements Heartbeater {
private static final Logger logger =
LoggerFactory.getLogger(ClusterProtocolHeartbeater.class);
private final NodeProtocolSender protocolSender;
-private final CuratorFramework curatorClient;
-private final String nodesPathPrefix;
-
-private final String coordinatorPath;
-private volatile String coordinatorAddress;
+private final LeaderElectionManager electionManager;
+private final ClusterCoordinator clusterCoordinator;
-
-public ClusterProtocolHeartbeater(final NodeProtocolSender
protocolSender, final Properties properties) {
+public ClusterProtocolHeartbeater(final NodeProtocolSender
protocolSender, final ClusterCoordinator clusterCoordinator, final
LeaderElectionManager electionManager) {
this.protocolSender = protocolSender;
-
-final RetryPolicy retryPolicy = new RetryNTimes(10, 500);
-final ZooKeeperClientConfig zkConfig =
ZooKeeperClientConfig.createConfig(properties);
-
-curatorClient =
CuratorFrameworkFactory.newClient(zkConfig.getConnectString(),
-zkConfig.getSessionTimeoutMillis(),
zkConfig.getConnectionTimeoutMillis(), retryPolicy);
-
-curatorClient.start();
-nodesPathPrefix = zkConfig.resolvePath("cluster/nodes");
-coordinatorPath = nodesPathPrefix + "/coordinator";
+this.clusterCoordinator = clusterCoordinator;
+this.electionManager = electionManager;
}
@Override
public String getHeartbeatAddress() throws IOException {
-final String curAddress = coordinatorAddress;
-if (curAddress != null) {
-return curAddress;
+final String heartbeatAddress =
electionManager.getLeader(ClusterRoles.CLUSTER_COORDINATOR);
+if (heartbeatAddress == null) {
+throw new ProtocolException("Cannot send heartbeat because
there is no Cluster Coordinator currently elected");
}
-try {
-// Get coordinator address and add watcher to change who we
are heartbeating to if the value changes.
-final byte[] coordinatorAddressBytes =
curatorClient.getData().usingWatcher(new Watcher() {
-@Override
-public void process(final WatchedEvent event) {
-coordinatorAddress = null;
-
[GitHub] nifi pull request #866: NIFI-2566: Refactoring to improve robustness of clus...
Github user JPercivall commented on a diff in the pull request:
https://github.com/apache/nifi/pull/866#discussion_r74833401
--- Diff:
nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/cluster/ClusterProtocolHeartbeater.java
---
@@ -18,102 +18,80 @@
package org.apache.nifi.controller.cluster;
import java.io.IOException;
-import java.nio.charset.StandardCharsets;
-import java.util.Properties;
-
-import org.apache.curator.RetryPolicy;
-import org.apache.curator.framework.CuratorFramework;
-import org.apache.curator.framework.CuratorFrameworkFactory;
-import org.apache.curator.retry.RetryNTimes;
+import java.util.List;
+import java.util.Map;
+import java.util.stream.Collectors;
+
+import org.apache.nifi.cluster.coordination.ClusterCoordinator;
+import org.apache.nifi.cluster.coordination.node.ClusterRoles;
+import org.apache.nifi.cluster.coordination.node.NodeConnectionStatus;
+import org.apache.nifi.cluster.protocol.HeartbeatPayload;
+import org.apache.nifi.cluster.protocol.NodeIdentifier;
import org.apache.nifi.cluster.protocol.NodeProtocolSender;
import org.apache.nifi.cluster.protocol.ProtocolException;
import org.apache.nifi.cluster.protocol.message.HeartbeatMessage;
-import org.apache.zookeeper.WatchedEvent;
-import org.apache.zookeeper.Watcher;
+import org.apache.nifi.cluster.protocol.message.HeartbeatResponseMessage;
+import org.apache.nifi.controller.leader.election.LeaderElectionManager;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;
/**
- * Uses ZooKeeper in order to determine which node is the elected Cluster
Coordinator and to indicate
- * that this node is part of the cluster. However, once the Cluster
Coordinator is known, heartbeats are
+ * Uses Leader Election Manager in order to determine which node is the
elected Cluster Coordinator and to indicate
+ * that this node is part of the cluster. Once the Cluster Coordinator is
known, heartbeats are
* sent directly to the Cluster Coordinator.
*/
public class ClusterProtocolHeartbeater implements Heartbeater {
private static final Logger logger =
LoggerFactory.getLogger(ClusterProtocolHeartbeater.class);
private final NodeProtocolSender protocolSender;
-private final CuratorFramework curatorClient;
-private final String nodesPathPrefix;
-
-private final String coordinatorPath;
-private volatile String coordinatorAddress;
+private final LeaderElectionManager electionManager;
+private final ClusterCoordinator clusterCoordinator;
-
-public ClusterProtocolHeartbeater(final NodeProtocolSender
protocolSender, final Properties properties) {
+public ClusterProtocolHeartbeater(final NodeProtocolSender
protocolSender, final ClusterCoordinator clusterCoordinator, final
LeaderElectionManager electionManager) {
this.protocolSender = protocolSender;
-
-final RetryPolicy retryPolicy = new RetryNTimes(10, 500);
-final ZooKeeperClientConfig zkConfig =
ZooKeeperClientConfig.createConfig(properties);
-
-curatorClient =
CuratorFrameworkFactory.newClient(zkConfig.getConnectString(),
-zkConfig.getSessionTimeoutMillis(),
zkConfig.getConnectionTimeoutMillis(), retryPolicy);
-
-curatorClient.start();
-nodesPathPrefix = zkConfig.resolvePath("cluster/nodes");
-coordinatorPath = nodesPathPrefix + "/coordinator";
+this.clusterCoordinator = clusterCoordinator;
+this.electionManager = electionManager;
}
@Override
public String getHeartbeatAddress() throws IOException {
-final String curAddress = coordinatorAddress;
-if (curAddress != null) {
-return curAddress;
+final String heartbeatAddress =
electionManager.getLeader(ClusterRoles.CLUSTER_COORDINATOR);
+if (heartbeatAddress == null) {
+throw new ProtocolException("Cannot send heartbeat because
there is no Cluster Coordinator currently elected");
}
-try {
-// Get coordinator address and add watcher to change who we
are heartbeating to if the value changes.
-final byte[] coordinatorAddressBytes =
curatorClient.getData().usingWatcher(new Watcher() {
-@Override
-public void process(final WatchedEvent event) {
-coordinatorAddress = null;
-}
-}).forPath(coordinatorPath);
-final String address = coordinatorAddress = new
String(coordinatorAddressBytes, StandardCharsets.UTF_8);
-
-logger.info("Determined that Cluster Coordinat
[jira] [Commented] (NIFI-2552) HiveQL processors fail when Zookeeper quorum in URL
[ https://issues.apache.org/jira/browse/NIFI-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421652#comment-15421652 ] ASF GitHub Bot commented on NIFI-2552: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/865 > HiveQL processors fail when Zookeeper quorum in URL > --- > > Key: NIFI-2552 > URL: https://issues.apache.org/jira/browse/NIFI-2552 > Project: Apache NiFi > Issue Type: Bug >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.0.0 > > > [~YolandaMDavis] discovered a bug in the HiveQL processors, when using a > Zookeeper quorum in the Hive JDBC URL: > java.lang.NoSuchMethodError: > org.apache.curator.utils.ZKPaths.fixForNamespace(Ljava/lang/String;Ljava/lang/String;Z)Ljava/lang/String; > at > org.apache.curator.framework.imps.NamespaceImpl.fixForNamespace(NamespaceImpl.java:104) > ~[curator-framework-2.11.0.jar:na] > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.fixForNamespace(CuratorFrameworkImpl.java:594) > ~[curator-framework-2.11.0.jar:na] > at > org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:198) > ~[curator-framework-2.11.0.jar:na] > at > org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:40) > ~[curator-framework-2.11.0.jar:na] > This method is in the Apache Curator 2.11.0 JAR (and has been since 2.8.0), > but the Hive NAR's parent is the Hadoop Libraries NAR, which transitively > brings in Curator 2.6.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #865: NIFI-2552: Updated Curator for HiveQL processors usi...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/865 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2552) HiveQL processors fail when Zookeeper quorum in URL
[ https://issues.apache.org/jira/browse/NIFI-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421651#comment-15421651 ] ASF subversion and git services commented on NIFI-2552: --- Commit ed42f2e3e339b5db9113fbbc60db94c74c1f9d29 in nifi's branch refs/heads/master from [~mattyb149] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=ed42f2e ] NIFI-2552: Updated Curator for HiveQL processors using Zookeeper quorum in URL Signed-off-by: Yolanda M. Davis This closes #865 > HiveQL processors fail when Zookeeper quorum in URL > --- > > Key: NIFI-2552 > URL: https://issues.apache.org/jira/browse/NIFI-2552 > Project: Apache NiFi > Issue Type: Bug >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.0.0 > > > [~YolandaMDavis] discovered a bug in the HiveQL processors, when using a > Zookeeper quorum in the Hive JDBC URL: > java.lang.NoSuchMethodError: > org.apache.curator.utils.ZKPaths.fixForNamespace(Ljava/lang/String;Ljava/lang/String;Z)Ljava/lang/String; > at > org.apache.curator.framework.imps.NamespaceImpl.fixForNamespace(NamespaceImpl.java:104) > ~[curator-framework-2.11.0.jar:na] > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.fixForNamespace(CuratorFrameworkImpl.java:594) > ~[curator-framework-2.11.0.jar:na] > at > org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:198) > ~[curator-framework-2.11.0.jar:na] > at > org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:40) > ~[curator-framework-2.11.0.jar:na] > This method is in the Apache Curator 2.11.0 JAR (and has been since 2.8.0), > but the Hive NAR's parent is the Hadoop Libraries NAR, which transitively > brings in Curator 2.6.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2575) HiveQL Processors Fail due to invalid JDBC URI resolution when using Zookeeper URI
[
https://issues.apache.org/jira/browse/NIFI-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Burgess updated NIFI-2575:
---
Description:
When configuring a HiveQL processor using the Zookeeper URL (e.g.
jdbc:hive2://ydavis-hdp-nifi-test-3.openstacklocal:2181,ydavis-hdp-nifi-test-1.openstacklocal:2181,ydavis-hdp-nifi-test-2.openstacklocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2),
it appears that the JDBC driver does not properly build the the uri in the
expected format. This is because HS2 is storing JDBC parameters in ZK
(https://issues.apache.org/jira/browse/HIVE-11581) and it is expecting the
driver to be able to parse and use those values to configure the connection.
However it appears the driver is expecting zookeeper to simply return the
host:port and subsequently building an invalid URI.
This problem has result in two variation of errors. The following was
experienced by [~mattyb149]
{noformat}
2016-08-15 12:45:12,918 INFO [Timer-Driven Process Thread-2]
org.apache.hive.jdbc.Utils Resolved authority:
hive.server2.authentication=KERBEROS;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=hdp-cluster-2-2.novalocal;hive.server2.thrift.port=1;hive.server2.use.SSL=false;hive.server2.authentication.kerberos.principal=hive/_h...@hdf.com
2016-08-15 12:45:13,835 INFO [Timer-Driven Process Thread-2]
org.apache.hive.jdbc.HiveConnection Will try to open client transport with JDBC
Uri:
jdbc:hive2://hive.server2.authentication=KERBEROS;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=hdp-cluster-2-2.novalocal;hive.server2.thrift.port=1;hive.server2.use.SSL=false;hive.server2.authentication.kerberos.principal=hive/_h...@hdf.com/default;principal=hive/_h...@hdf.com;serviceDiscoveryMode=zookeeper;zooKeeperNamespace=hiveserver2
2016-08-15 12:45:13,835 INFO [Timer-Driven Process Thread-2]
org.apache.hive.jdbc.HiveConnection Could not open client transport with JDBC
Uri:
jdbc:hive2://hive.server2.authentication=KERBEROS;hive.server2.transport.mode=binary;hive.server2.thrift.sasl.qop=auth;hive.server2.thrift.bind.host=hdp-cluster-2-2.novalocal;hive.server2.thrift.port=1;hive.server2.use.SSL=false;hive.server2.authentication.kerberos.principal=hive/_h...@hdf.com/default;principal=hive/_h...@hdf.com;serviceDiscoveryMode=zookeeper;zooKeeperNamespace=hiveserver2
2016-08-15 12:45:13,836 INFO [Timer-Driven Process Thread-2]
o.a.c.f.imps.CuratorFrameworkImpl Starting
2016-08-15 12:45:14,064 INFO [Timer-Driven Process Thread-2-EventThread]
o.a.c.f.state.ConnectionStateManager State change: CONNECTED
2016-08-15 12:45:14,182 INFO [Curator-Framework-0]
o.a.c.f.imps.CuratorFrameworkImpl backgroundOperationsLoop exiting
2016-08-15 12:45:14,337 ERROR [Timer-Driven Process Thread-2]
o.a.nifi.processors.hive.SelectHiveQL
SelectHiveQL[id=7aaffd71-0156-1000-d962-8102c06b23df]
SelectHiveQL[id=7aaffd71-0156-1000-d962-8102c06b23df] failed to process due to
java.lang.reflect.UndeclaredThrowableException; rolling back session:
java.lang.reflect.UndeclaredThrowableException
2016-08-15 12:45:14,346 ERROR [Timer-Driven Process Thread-2]
o.a.nifi.processors.hive.SelectHiveQL
java.lang.reflect.UndeclaredThrowableException: null
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
~[na:na]
at
org.apache.nifi.dbcp.hive.HiveConnectionPool.getConnection(HiveConnectionPool.java:255)
~[na:na]
at sun.reflect.GeneratedMethodAccessor331.invoke(Unknown Source)
~[na:na]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.8.0_65]
at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_65]
at
org.apache.nifi.controller.service.StandardControllerServiceProvider$1.invoke(StandardControllerServiceProvider.java:174)
~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at com.sun.proxy.$Proxy81.getConnection(Unknown Source) ~[na:na]
at
org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:158)
~[na:na]
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
~[nifi-api-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1060)
[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127
[jira] [Commented] (NIFI-2554) Components with references to Controller Services should limit configuration options based on CS policies
[ https://issues.apache.org/jira/browse/NIFI-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421648#comment-15421648 ] ASF GitHub Bot commented on NIFI-2554: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/860 > Components with references to Controller Services should limit configuration > options based on CS policies > - > > Key: NIFI-2554 > URL: https://issues.apache.org/jira/browse/NIFI-2554 > Project: Apache NiFi > Issue Type: Bug >Reporter: Joseph Percivall >Assignee: Matt Gilman >Priority: Critical > Fix For: 1.0.0 > > > Take the situation below: > * InvokeHTTP processor I have view and modify permissions for > * There are multiple SSL contexts some of which I do not have view or modify > access to > * I am able to change the SSL Context Service property of the InvokeHTTP > processor to use a Controller service I do not have access to > This should not be allowed. The user should not be able to create references > to Controller Services they cannot view or modify. > That said, since the user has the explicit permission to modify the > processor, the user should be able to keep property referencing a CS they > can't view/modify if someone else configured it that way. > The UI will need to be explicit in conveying this to the user since it will > be a bit complicated (limiting a user's options when configuring a component > they have full access to) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2554) Components with references to Controller Services should limit configuration options based on CS policies
[ https://issues.apache.org/jira/browse/NIFI-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-2554: -- Resolution: Fixed Status: Resolved (was: Patch Available) > Components with references to Controller Services should limit configuration > options based on CS policies > - > > Key: NIFI-2554 > URL: https://issues.apache.org/jira/browse/NIFI-2554 > Project: Apache NiFi > Issue Type: Bug >Reporter: Joseph Percivall >Assignee: Matt Gilman >Priority: Critical > Fix For: 1.0.0 > > > Take the situation below: > * InvokeHTTP processor I have view and modify permissions for > * There are multiple SSL contexts some of which I do not have view or modify > access to > * I am able to change the SSL Context Service property of the InvokeHTTP > processor to use a Controller service I do not have access to > This should not be allowed. The user should not be able to create references > to Controller Services they cannot view or modify. > That said, since the user has the explicit permission to modify the > processor, the user should be able to keep property referencing a CS they > can't view/modify if someone else configured it that way. > The UI will need to be explicit in conveying this to the user since it will > be a bit complicated (limiting a user's options when configuring a component > they have full access to) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2554) Components with references to Controller Services should limit configuration options based on CS policies
[ https://issues.apache.org/jira/browse/NIFI-2554?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421647#comment-15421647 ] ASF subversion and git services commented on NIFI-2554: --- Commit 7d8dd27027b42134d4825f3d5a5da6aedb962b1a in nifi's branch refs/heads/master from [~mcgilman] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=7d8dd27 ] NIFI-2554: - Requiring READ permissions on the referenced controller service when creating/updating processors, controller services, and reporting tasks. - Preventing client side selection of unauthorized controller services unless they were the previously configured value. This closes #860. Signed-off-by: Bryan Bende > Components with references to Controller Services should limit configuration > options based on CS policies > - > > Key: NIFI-2554 > URL: https://issues.apache.org/jira/browse/NIFI-2554 > Project: Apache NiFi > Issue Type: Bug >Reporter: Joseph Percivall >Assignee: Matt Gilman >Priority: Critical > Fix For: 1.0.0 > > > Take the situation below: > * InvokeHTTP processor I have view and modify permissions for > * There are multiple SSL contexts some of which I do not have view or modify > access to > * I am able to change the SSL Context Service property of the InvokeHTTP > processor to use a Controller service I do not have access to > This should not be allowed. The user should not be able to create references > to Controller Services they cannot view or modify. > That said, since the user has the explicit permission to modify the > processor, the user should be able to keep property referencing a CS they > can't view/modify if someone else configured it that way. > The UI will need to be explicit in conveying this to the user since it will > be a bit complicated (limiting a user's options when configuring a component > they have full access to) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #860: Enforcing READ permissions when referencing Controll...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/860 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #860: Enforcing READ permissions when referencing ControllerServi...
Github user bbende commented on the issue: https://github.com/apache/nifi/pull/860 +1 full build passed with contrib-check and tested a lot of combinations of components referencing controller services to ensure i could only use controller services i had READ access to, will merge to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Comment Edited] (NIFI-2574) Update NiFiProperties usage to a context based accessor rather than static singleton
[
https://issues.apache.org/jira/browse/NIFI-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421635#comment-15421635
]
Andy LoPresto edited comment on NIFI-2574 at 8/15/16 8:40 PM:
--
We also would like to prevent changing the property values, so the object
should be immutable. I believe the plan is as follows:
{{NiFiProperties}} becomes an {{interface}}
* it has all the static strings for key access
* it has a {{#getProperty(String key)}} which returns a {{String}} value
(potentially {{Object}}?)
{{StandardNiFiProperties}} implements {{NiFiProperties}}
* it has an internal {{Properties}} instance
* it does not have a {{#setProperty()}} method
* it contains the _default_ values (moved from current {{NiFiProperties}})
{{NiFiProperties}} stays in the *nifi-properties* module
{{StandardNiFiProperties}} moves to *nifi-framework-core* module
{{NiFiPropertiesLoader}} is there as well. This class handles instantiation,
file-retrieval, unprotection/decryption, etc. and makes the instance available
to the various contexts.
was (Author: alopresto):
We also would like to prevent changing the property values, so the object
should be immutable. I believe the plan is as follows:
{{NiFiProperties}} becomes an {{interface}}
* it has all the static strings for key access
* it has a {{#getProperty(String key)}} which returns a {{String}} value
(potentially {{Object}}?)
{{StandardNiFiProperties}} implements {{NiFiProperties}}
* it has an internal {{Properties}} instance
* it does not have a {{#setProperty()}} method
{{NiFiProperties}} stays in the *nifi-properties* module
{{StandardNiFiProperties}} moves to *nifi-framework-core* module
{{NiFiPropertiesLoader}} is there as well. This class handles instantiation,
file-retrieval, unprotection/decryption, etc. and makes the instance available
to the various contexts.
> Update NiFiProperties usage to a context based accessor rather than static
> singleton
>
>
> Key: NIFI-2574
> URL: https://issues.apache.org/jira/browse/NIFI-2574
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Configuration, Core Framework
>Reporter: Joseph Witt
>Assignee: Joseph Witt
> Fix For: 1.0.0
>
>
> This is a key step to move our longstanding odd static singleton usage of
> NiFiProperties which enables a loader to exist which will encapsulate all
> logic for dealing with encrypted properties as is the goal of the parent task.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2574) Update NiFiProperties usage to a context based accessor rather than static singleton
[
https://issues.apache.org/jira/browse/NIFI-2574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421635#comment-15421635
]
Andy LoPresto commented on NIFI-2574:
-
We also would like to prevent changing the property values, so the object
should be immutable. I believe the plan is as follows:
{{NiFiProperties}} becomes an {{interface}}
* it has all the static strings for key access
* it has a {{#getProperty(String key)}} which returns a {{String}} value
(potentially {{Object}}?)
{{StandardNiFiProperties}} implements {{NiFiProperties}}
* it has an internal {{Properties}} instance
* it does not have a {{#setProperty()}} method
{{NiFiProperties}} stays in the *nifi-properties* module
{{StandardNiFiProperties}} moves to *nifi-framework-core* module
{{NiFiPropertiesLoader}} is there as well. This class handles instantiation,
file-retrieval, unprotection/decryption, etc. and makes the instance available
to the various contexts.
> Update NiFiProperties usage to a context based accessor rather than static
> singleton
>
>
> Key: NIFI-2574
> URL: https://issues.apache.org/jira/browse/NIFI-2574
> Project: Apache NiFi
> Issue Type: Sub-task
> Components: Configuration, Core Framework
>Reporter: Joseph Witt
>Assignee: Joseph Witt
> Fix For: 1.0.0
>
>
> This is a key step to move our longstanding odd static singleton usage of
> NiFiProperties which enables a loader to exist which will encapsulate all
> logic for dealing with encrypted properties as is the goal of the parent task.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2566) Clustered Nodes can become out of sync regarding which node is coordinator
[ https://issues.apache.org/jira/browse/NIFI-2566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421632#comment-15421632 ] ASF GitHub Bot commented on NIFI-2566: -- GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/866 NIFI-2566: Refactoring to improve robustness of cluster You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2566 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/866.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #866 commit 90dc74627157296b4084ed65c1b752bfe25db1e8 Author: Mark Payne Date: 2016-08-13T23:38:07Z NIFI-2566: Refactored to allow just the Leader Election Manager to be responsible for determining who is the Cluster Coordinator commit 3fd121b2ff57c5fc51171791351580c29c58e553 Author: Mark Payne Date: 2016-08-14T23:40:31Z NIFI-2566: Removed storage of cluster roles from heartbeats and NodeConnectionStatus; use LeaderElectionManager to determine roles instead commit 457a2df5c3296013ce896ca7900807d9bdb69a71 Author: Mark Payne Date: 2016-08-15T20:35:56Z NIFI-2566: Updated Heartbeats so that if a node is out-of-sync with cluster topology, cluster coordinator will provide updated information back to the nodes > Clustered Nodes can become out of sync regarding which node is coordinator > -- > > Key: NIFI-2566 > URL: https://issues.apache.org/jira/browse/NIFI-2566 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Reporter: Mark Payne >Assignee: Mark Payne > Fix For: 1.0.0 > > > Occasionally, I will see the UI telling me that no Cluster Coordinator has > been elected. However, I can see in the logs that the node is sending > heartbeats to the coordinator. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #866: NIFI-2566: Refactoring to improve robustness of clus...
GitHub user markap14 opened a pull request: https://github.com/apache/nifi/pull/866 NIFI-2566: Refactoring to improve robustness of cluster You can merge this pull request into a Git repository by running: $ git pull https://github.com/markap14/nifi NIFI-2566 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/866.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #866 commit 90dc74627157296b4084ed65c1b752bfe25db1e8 Author: Mark Payne Date: 2016-08-13T23:38:07Z NIFI-2566: Refactored to allow just the Leader Election Manager to be responsible for determining who is the Cluster Coordinator commit 3fd121b2ff57c5fc51171791351580c29c58e553 Author: Mark Payne Date: 2016-08-14T23:40:31Z NIFI-2566: Removed storage of cluster roles from heartbeats and NodeConnectionStatus; use LeaderElectionManager to determine roles instead commit 457a2df5c3296013ce896ca7900807d9bdb69a71 Author: Mark Payne Date: 2016-08-15T20:35:56Z NIFI-2566: Updated Heartbeats so that if a node is out-of-sync with cluster topology, cluster coordinator will provide updated information back to the nodes --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-2575) HiveQL Processors Fail due to invalid JDBC URI resolution when using Zookeeper URI
[
https://issues.apache.org/jira/browse/NIFI-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yolanda M. Davis updated NIFI-2575:
---
Summary: HiveQL Processors Fail due to invalid JDBC URI resolution when
using Zookeeper URI (was: HiveQL Processors Fails due to invalid JDBC URI
resolution when using Zookeeper URI)
> HiveQL Processors Fail due to invalid JDBC URI resolution when using
> Zookeeper URI
> --
>
> Key: NIFI-2575
> URL: https://issues.apache.org/jira/browse/NIFI-2575
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Yolanda M. Davis
> Fix For: 1.0.0
>
>
> When configuring a HiveQL processor using the Zookeeper URL (e.g.
> jdbc:hive2://ydavis-hdp-nifi-test-3.openstacklocal:2181,ydavis-hdp-nifi-test-1.openstacklocal:2181,ydavis-hdp-nifi-test-2.openstacklocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2),
> it appears that the JDBC driver does not properly build the the uri in the
> expected format. This is because HS2 is storing JDBC parameters in ZK
> (https://issues.apache.org/jira/browse/HIVE-11581) and it is expecting the
> driver to be able to parse and use those values to configure the connection.
> However it appears the driver is expecting zookeeper to simply return the
> host:port and subsequently building an invalid URI.
> This problem has result in two variation of errors. The following was
> experienced by [~mattyb149]
> {noformat}
> 2016-08-15 12:36:14,883 INFO [Timer-Driven Process Thread-5]
> org.apache.hive.jdbc.HiveConnection Could not open client transport with JDBC
> Uri:
> jdbc:hive2://hdp-cluster-2-2.novalocal:1/default;hive.server2.authentication=KERBEROS;hive.serve...;serviceDiscoveryMode=zookeeper;zooKeeperNamespace=hiveserver2
> 2016-08-15 12:36:16,019 INFO [Timer-Driven Process Thread-5]
> o.a.c.f.imps.CuratorFrameworkImpl Starting
> 2016-08-15 12:36:16,054 INFO [Write-Ahead Local State Provider Maintenance]
> org.wali.MinimalLockingWriteAheadLog
> org.wali.MinimalLockingWriteAheadLog@434c179e checkpointed with 2 Records and
> 0 Swap Files in 1170 milliseconds (Stop-the-world time = 1159 milliseconds,
> Clear Edit Logs time = 8 millis), max Transaction ID 5
> 2016-08-15 12:36:16,335 INFO [Timer-Driven Process Thread-5-EventThread]
> o.a.c.f.state.ConnectionStateManager State change: CONNECTED
> 2016-08-15 12:36:16,368 INFO [pool-16-thread-1]
> org.wali.MinimalLockingWriteAheadLog
> org.wali.MinimalLockingWriteAheadLog@1caf77f6 checkpointed with 0 Records and
> 0 Swap Files in 1484 milliseconds (Stop-the-world time = 1438 milliseconds,
> Clear Edit Logs time = 45 millis), max Transaction ID -1
> 2016-08-15 12:36:16,368 INFO [pool-16-thread-1]
> o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
> Repository with 0 records in 1484 milliseconds
> 2016-08-15 12:36:16,457 INFO [Curator-Framework-0]
> o.a.c.f.imps.CuratorFrameworkImpl backgroundOperationsLoop exiting
> 2016-08-15 12:36:16,588 ERROR [Timer-Driven Process Thread-5]
> o.a.nifi.processors.hive.SelectHiveQL
> SelectHiveQL[id=7aaffd71-0156-1000-d962-8102c06b23df]
> SelectHiveQL[id=7aaffd71-0156-1000-d962-8102c06b23df] failed to process due
> to java.lang.reflect.UndeclaredThrowableException; rolling back session:
> java.lang.reflect.UndeclaredThrowableException
> 2016-08-15 12:36:16,596 ERROR [Timer-Driven Process Thread-5]
> o.a.nifi.processors.hive.SelectHiveQL
> java.lang.reflect.UndeclaredThrowableException: null
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> ~[na:na]
> at
> org.apache.nifi.dbcp.hive.HiveConnectionPool.getConnection(HiveConnectionPool.java:255)
> ~[na:na]
> at sun.reflect.GeneratedMethodAccessor331.invoke(Unknown Source)
> ~[na:na]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_65]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_65]
> at
> org.apache.nifi.controller.service.StandardControllerServiceProvider$1.invoke(StandardControllerServiceProvider.java:174)
> ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy81.getConnection(Unknown Source) ~[na:na]
> at
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:158)
> ~[na:na]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> ~[nifi-api-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1060)
> [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProc
[jira] [Updated] (NIFI-2575) HiveQL Processors Fails due to invalid JDBC URI resolution when using Zookeeper URI
[
https://issues.apache.org/jira/browse/NIFI-2575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yolanda M. Davis updated NIFI-2575:
---
Summary: HiveQL Processors Fails due to invalid JDBC URI resolution when
using Zookeeper URI (was: HiveQL Processor Fails due to invalid JDBC URI
resolution when using Zookeeper URI)
> HiveQL Processors Fails due to invalid JDBC URI resolution when using
> Zookeeper URI
> ---
>
> Key: NIFI-2575
> URL: https://issues.apache.org/jira/browse/NIFI-2575
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Reporter: Yolanda M. Davis
> Fix For: 1.0.0
>
>
> When configuring a HiveQL processor using the Zookeeper URL (e.g.
> jdbc:hive2://ydavis-hdp-nifi-test-3.openstacklocal:2181,ydavis-hdp-nifi-test-1.openstacklocal:2181,ydavis-hdp-nifi-test-2.openstacklocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2),
> it appears that the JDBC driver does not properly build the the uri in the
> expected format. This is because HS2 is storing JDBC parameters in ZK
> (https://issues.apache.org/jira/browse/HIVE-11581) and it is expecting the
> driver to be able to parse and use those values to configure the connection.
> However it appears the driver is expecting zookeeper to simply return the
> host:port and subsequently building an invalid URI.
> This problem has result in two variation of errors. The following was
> experienced by [~mattyb149]
> {noformat}
> 2016-08-15 12:36:14,883 INFO [Timer-Driven Process Thread-5]
> org.apache.hive.jdbc.HiveConnection Could not open client transport with JDBC
> Uri:
> jdbc:hive2://hdp-cluster-2-2.novalocal:1/default;hive.server2.authentication=KERBEROS;hive.serve...;serviceDiscoveryMode=zookeeper;zooKeeperNamespace=hiveserver2
> 2016-08-15 12:36:16,019 INFO [Timer-Driven Process Thread-5]
> o.a.c.f.imps.CuratorFrameworkImpl Starting
> 2016-08-15 12:36:16,054 INFO [Write-Ahead Local State Provider Maintenance]
> org.wali.MinimalLockingWriteAheadLog
> org.wali.MinimalLockingWriteAheadLog@434c179e checkpointed with 2 Records and
> 0 Swap Files in 1170 milliseconds (Stop-the-world time = 1159 milliseconds,
> Clear Edit Logs time = 8 millis), max Transaction ID 5
> 2016-08-15 12:36:16,335 INFO [Timer-Driven Process Thread-5-EventThread]
> o.a.c.f.state.ConnectionStateManager State change: CONNECTED
> 2016-08-15 12:36:16,368 INFO [pool-16-thread-1]
> org.wali.MinimalLockingWriteAheadLog
> org.wali.MinimalLockingWriteAheadLog@1caf77f6 checkpointed with 0 Records and
> 0 Swap Files in 1484 milliseconds (Stop-the-world time = 1438 milliseconds,
> Clear Edit Logs time = 45 millis), max Transaction ID -1
> 2016-08-15 12:36:16,368 INFO [pool-16-thread-1]
> o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
> Repository with 0 records in 1484 milliseconds
> 2016-08-15 12:36:16,457 INFO [Curator-Framework-0]
> o.a.c.f.imps.CuratorFrameworkImpl backgroundOperationsLoop exiting
> 2016-08-15 12:36:16,588 ERROR [Timer-Driven Process Thread-5]
> o.a.nifi.processors.hive.SelectHiveQL
> SelectHiveQL[id=7aaffd71-0156-1000-d962-8102c06b23df]
> SelectHiveQL[id=7aaffd71-0156-1000-d962-8102c06b23df] failed to process due
> to java.lang.reflect.UndeclaredThrowableException; rolling back session:
> java.lang.reflect.UndeclaredThrowableException
> 2016-08-15 12:36:16,596 ERROR [Timer-Driven Process Thread-5]
> o.a.nifi.processors.hive.SelectHiveQL
> java.lang.reflect.UndeclaredThrowableException: null
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
> ~[na:na]
> at
> org.apache.nifi.dbcp.hive.HiveConnectionPool.getConnection(HiveConnectionPool.java:255)
> ~[na:na]
> at sun.reflect.GeneratedMethodAccessor331.invoke(Unknown Source)
> ~[na:na]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_65]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_65]
> at
> org.apache.nifi.controller.service.StandardControllerServiceProvider$1.invoke(StandardControllerServiceProvider.java:174)
> ~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at com.sun.proxy.$Proxy81.getConnection(Unknown Source) ~[na:na]
> at
> org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:158)
> ~[na:na]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> ~[nifi-api-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1060)
> [nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunPr
[jira] [Commented] (NIFI-2552) HiveQL processors fail when Zookeeper quorum in URL
[ https://issues.apache.org/jira/browse/NIFI-2552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421620#comment-15421620 ] ASF GitHub Bot commented on NIFI-2552: -- Github user YolandaMDavis commented on the issue: https://github.com/apache/nifi/pull/865 @mattyb149 was able to run tests without issue, deployed on three node cluster attempting PutHiveQL. The previous error: java.lang.NoSuchMethodError: org.apache.curator.utils.ZKPaths.fixForNamespace(Ljava/lang/String;Ljava/lang/String;Z) was not seen. A subsequent exception did occur, however it is unrelated to this fix and was logged separately (https://issues.apache.org/jira/browse/NIFI-2575) +1 Will work to get this merged in shortly. > HiveQL processors fail when Zookeeper quorum in URL > --- > > Key: NIFI-2552 > URL: https://issues.apache.org/jira/browse/NIFI-2552 > Project: Apache NiFi > Issue Type: Bug >Reporter: Matt Burgess >Assignee: Matt Burgess > Fix For: 1.0.0 > > > [~YolandaMDavis] discovered a bug in the HiveQL processors, when using a > Zookeeper quorum in the Hive JDBC URL: > java.lang.NoSuchMethodError: > org.apache.curator.utils.ZKPaths.fixForNamespace(Ljava/lang/String;Ljava/lang/String;Z)Ljava/lang/String; > at > org.apache.curator.framework.imps.NamespaceImpl.fixForNamespace(NamespaceImpl.java:104) > ~[curator-framework-2.11.0.jar:na] > at > org.apache.curator.framework.imps.CuratorFrameworkImpl.fixForNamespace(CuratorFrameworkImpl.java:594) > ~[curator-framework-2.11.0.jar:na] > at > org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:198) > ~[curator-framework-2.11.0.jar:na] > at > org.apache.curator.framework.imps.GetChildrenBuilderImpl.forPath(GetChildrenBuilderImpl.java:40) > ~[curator-framework-2.11.0.jar:na] > This method is in the Apache Curator 2.11.0 JAR (and has been since 2.8.0), > but the Hive NAR's parent is the Hadoop Libraries NAR, which transitively > brings in Curator 2.6.0. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #865: NIFI-2552: Updated Curator for HiveQL processors using Zook...
Github user YolandaMDavis commented on the issue: https://github.com/apache/nifi/pull/865 @mattyb149 was able to run tests without issue, deployed on three node cluster attempting PutHiveQL. The previous error: java.lang.NoSuchMethodError: org.apache.curator.utils.ZKPaths.fixForNamespace(Ljava/lang/String;Ljava/lang/String;Z) was not seen. A subsequent exception did occur, however it is unrelated to this fix and was logged separately (https://issues.apache.org/jira/browse/NIFI-2575) +1 Will work to get this merged in shortly. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (NIFI-2575) HiveQL Processor Fails due to invalid JDBC URI resolution when using Zookeeper URI
Yolanda M. Davis created NIFI-2575:
--
Summary: HiveQL Processor Fails due to invalid JDBC URI resolution
when using Zookeeper URI
Key: NIFI-2575
URL: https://issues.apache.org/jira/browse/NIFI-2575
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Reporter: Yolanda M. Davis
Fix For: 1.0.0
When configuring a HiveQL processor using the Zookeeper URL (e.g.
jdbc:hive2://ydavis-hdp-nifi-test-3.openstacklocal:2181,ydavis-hdp-nifi-test-1.openstacklocal:2181,ydavis-hdp-nifi-test-2.openstacklocal:2181/;serviceDiscoveryMode=zooKeeper;zooKeeperNamespace=hiveserver2),
it appears that the JDBC driver does not properly build the the uri in the
expected format. This is because HS2 is storing JDBC parameters in ZK
(https://issues.apache.org/jira/browse/HIVE-11581) and it is expecting the
driver to be able to parse and use those values to configure the connection.
However it appears the driver is expecting zookeeper to simply return the
host:port and subsequently building an invalid URI.
This problem has result in two variation of errors. The following was
experienced by [~mattyb149]
{noformat}
2016-08-15 12:36:14,883 INFO [Timer-Driven Process Thread-5]
org.apache.hive.jdbc.HiveConnection Could not open client transport with JDBC
Uri:
jdbc:hive2://hdp-cluster-2-2.novalocal:1/default;hive.server2.authentication=KERBEROS;hive.serve...;serviceDiscoveryMode=zookeeper;zooKeeperNamespace=hiveserver2
2016-08-15 12:36:16,019 INFO [Timer-Driven Process Thread-5]
o.a.c.f.imps.CuratorFrameworkImpl Starting
2016-08-15 12:36:16,054 INFO [Write-Ahead Local State Provider Maintenance]
org.wali.MinimalLockingWriteAheadLog
org.wali.MinimalLockingWriteAheadLog@434c179e checkpointed with 2 Records and 0
Swap Files in 1170 milliseconds (Stop-the-world time = 1159 milliseconds, Clear
Edit Logs time = 8 millis), max Transaction ID 5
2016-08-15 12:36:16,335 INFO [Timer-Driven Process Thread-5-EventThread]
o.a.c.f.state.ConnectionStateManager State change: CONNECTED
2016-08-15 12:36:16,368 INFO [pool-16-thread-1]
org.wali.MinimalLockingWriteAheadLog
org.wali.MinimalLockingWriteAheadLog@1caf77f6 checkpointed with 0 Records and 0
Swap Files in 1484 milliseconds (Stop-the-world time = 1438 milliseconds, Clear
Edit Logs time = 45 millis), max Transaction ID -1
2016-08-15 12:36:16,368 INFO [pool-16-thread-1]
o.a.n.c.r.WriteAheadFlowFileRepository Successfully checkpointed FlowFile
Repository with 0 records in 1484 milliseconds
2016-08-15 12:36:16,457 INFO [Curator-Framework-0]
o.a.c.f.imps.CuratorFrameworkImpl backgroundOperationsLoop exiting
2016-08-15 12:36:16,588 ERROR [Timer-Driven Process Thread-5]
o.a.nifi.processors.hive.SelectHiveQL
SelectHiveQL[id=7aaffd71-0156-1000-d962-8102c06b23df]
SelectHiveQL[id=7aaffd71-0156-1000-d962-8102c06b23df] failed to process due to
java.lang.reflect.UndeclaredThrowableException; rolling back session:
java.lang.reflect.UndeclaredThrowableException
2016-08-15 12:36:16,596 ERROR [Timer-Driven Process Thread-5]
o.a.nifi.processors.hive.SelectHiveQL
java.lang.reflect.UndeclaredThrowableException: null
at
org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1671)
~[na:na]
at
org.apache.nifi.dbcp.hive.HiveConnectionPool.getConnection(HiveConnectionPool.java:255)
~[na:na]
at sun.reflect.GeneratedMethodAccessor331.invoke(Unknown Source)
~[na:na]
at
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
~[na:1.8.0_65]
at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_65]
at
org.apache.nifi.controller.service.StandardControllerServiceProvider$1.invoke(StandardControllerServiceProvider.java:174)
~[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at com.sun.proxy.$Proxy81.getConnection(Unknown Source) ~[na:na]
at
org.apache.nifi.processors.hive.SelectHiveQL.onTrigger(SelectHiveQL.java:158)
~[na:na]
at
org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
~[nifi-api-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1060)
[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
[nifi-framework-core-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
at
java.util.concurrent.Executors$RunnableAdapter.call(Exec
[jira] [Updated] (NIFI-2559) UI - Controller Service referencing component run status missing
[ https://issues.apache.org/jira/browse/NIFI-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleg Zhurakousky updated NIFI-2559: --- Resolution: Fixed Status: Resolved (was: Patch Available) > UI - Controller Service referencing component run status missing > > > Key: NIFI-2559 > URL: https://issues.apache.org/jira/browse/NIFI-2559 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Scott Aslan > Fix For: 1.0.0 > > > In the Controller Service dialogs (configuration, enable, disable, etc) we > dynamically generate markup for components that reference that Controller > Service. Some of the information that we include is not rendering correctly > because they are reference old images/styles that are no longer applicable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2559) UI - Controller Service referencing component run status missing
[ https://issues.apache.org/jira/browse/NIFI-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421615#comment-15421615 ] ASF GitHub Bot commented on NIFI-2559: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/864 > UI - Controller Service referencing component run status missing > > > Key: NIFI-2559 > URL: https://issues.apache.org/jira/browse/NIFI-2559 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Scott Aslan > Fix For: 1.0.0 > > > In the Controller Service dialogs (configuration, enable, disable, etc) we > dynamically generate markup for components that reference that Controller > Service. Some of the information that we include is not rendering correctly > because they are reference old images/styles that are no longer applicable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2559) UI - Controller Service referencing component run status missing
[ https://issues.apache.org/jira/browse/NIFI-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421614#comment-15421614 ] ASF subversion and git services commented on NIFI-2559: --- Commit 2a92747046b39d7b3f7604258c1d3bb81a9447bd in nifi's branch refs/heads/master from [~scottyaslan] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2a92747 ] [NIFI-2559] UI - Controller Service update missing icons This closes #864 > UI - Controller Service referencing component run status missing > > > Key: NIFI-2559 > URL: https://issues.apache.org/jira/browse/NIFI-2559 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Scott Aslan > Fix For: 1.0.0 > > > In the Controller Service dialogs (configuration, enable, disable, etc) we > dynamically generate markup for components that reference that Controller > Service. Some of the information that we include is not rendering correctly > because they are reference old images/styles that are no longer applicable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #864: [NIFI-2559] UI - Controller Service update missing i...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/864 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2559) UI - Controller Service referencing component run status missing
[ https://issues.apache.org/jira/browse/NIFI-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421609#comment-15421609 ] ASF GitHub Bot commented on NIFI-2559: -- Github user olegz commented on the issue: https://github.com/apache/nifi/pull/864 +1 Verified font changes, icons, alignment. Merging > UI - Controller Service referencing component run status missing > > > Key: NIFI-2559 > URL: https://issues.apache.org/jira/browse/NIFI-2559 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Scott Aslan > Fix For: 1.0.0 > > > In the Controller Service dialogs (configuration, enable, disable, etc) we > dynamically generate markup for components that reference that Controller > Service. Some of the information that we include is not rendering correctly > because they are reference old images/styles that are no longer applicable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #864: [NIFI-2559] UI - Controller Service update missing icons
Github user olegz commented on the issue: https://github.com/apache/nifi/pull/864 +1 Verified font changes, icons, alignment. Merging --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-2571) Deprecate NiFiProperties.getInstance() in 0.x
[
https://issues.apache.org/jira/browse/NIFI-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Witt updated NIFI-2571:
--
Fix Version/s: (was: 1.0.0)
> Deprecate NiFiProperties.getInstance() in 0.x
> -
>
> Key: NIFI-2571
> URL: https://issues.apache.org/jira/browse/NIFI-2571
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 0.7.0
>Reporter: Andy LoPresto
> Labels: nifiproperties, singleton
>
> As per the discussion on PR 834 ([NIFI-1831]), the singleton pattern of
> {{NiFiProperties.getInstance()}} is being removed from the 1.0 release. It
> should be marked as deprecated in 0.x. A {{NiFiPropertiesLoader}} class will
> be provided and the contexts (processors, reporting tasks, controller
> services) will gain access to an instance of application properties.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-2571) Deprecate NiFiProperties.getInstance() in 0.x
[
https://issues.apache.org/jira/browse/NIFI-2571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Witt updated NIFI-2571:
--
Fix Version/s: 1.0.0
> Deprecate NiFiProperties.getInstance() in 0.x
> -
>
> Key: NIFI-2571
> URL: https://issues.apache.org/jira/browse/NIFI-2571
> Project: Apache NiFi
> Issue Type: Improvement
> Components: Core Framework
>Affects Versions: 0.7.0
>Reporter: Andy LoPresto
> Labels: nifiproperties, singleton
>
> As per the discussion on PR 834 ([NIFI-1831]), the singleton pattern of
> {{NiFiProperties.getInstance()}} is being removed from the 1.0 release. It
> should be marked as deprecated in 0.x. A {{NiFiPropertiesLoader}} class will
> be provided and the contexts (processors, reporting tasks, controller
> services) will gain access to an instance of application properties.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (NIFI-2574) Update NiFiProperties usage to a context based accessor rather than static singleton
Joseph Witt created NIFI-2574: - Summary: Update NiFiProperties usage to a context based accessor rather than static singleton Key: NIFI-2574 URL: https://issues.apache.org/jira/browse/NIFI-2574 Project: Apache NiFi Issue Type: Sub-task Reporter: Joseph Witt Assignee: Joseph Witt This is a key step to move our longstanding odd static singleton usage of NiFiProperties which enables a loader to exist which will encapsulate all logic for dealing with encrypted properties as is the goal of the parent task. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2501) UI - Process Group configuration requires write permissions
[ https://issues.apache.org/jira/browse/NIFI-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421554#comment-15421554 ] ASF GitHub Bot commented on NIFI-2501: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/835 > UI - Process Group configuration requires write permissions > --- > > Key: NIFI-2501 > URL: https://issues.apache.org/jira/browse/NIFI-2501 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > > Since Controller Services are accessed through the Process Group > configuration, we need to allow users without read/write permissions to > access that dialog in case they do have permissions to a controller service > in there. The Controller settings dialog is already updated to support this > case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2501) UI - Process Group configuration requires write permissions
[ https://issues.apache.org/jira/browse/NIFI-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Percivall updated NIFI-2501: --- Resolution: Fixed Status: Resolved (was: Patch Available) > UI - Process Group configuration requires write permissions > --- > > Key: NIFI-2501 > URL: https://issues.apache.org/jira/browse/NIFI-2501 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > > Since Controller Services are accessed through the Process Group > configuration, we need to allow users without read/write permissions to > access that dialog in case they do have permissions to a controller service > in there. The Controller settings dialog is already updated to support this > case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #835: Ensuring users can access the Controller Service and Report...
Github user JPercivall commented on the issue: https://github.com/apache/nifi/pull/835 +1 Visually verified code and did a contrib check build. In a 3 node secure cluster tested various flows of logic regarding mixed access polices in PG, CS, RT and Controller. All worked as expected. Thanks @mcgilman, I will merge it in. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2501) UI - Process Group configuration requires write permissions
[ https://issues.apache.org/jira/browse/NIFI-2501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421552#comment-15421552 ] ASF subversion and git services commented on NIFI-2501: --- Commit d45114e48dafe357742c8766250e096c690e499e in nifi's branch refs/heads/master from [~mcgilman] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=d45114e ] NIFI-2501: - Ensuring users can access the controller service list regardless of permissions on the corresponding process group or controller. - Better handling the case where the user had write permissions but no read permissions. - Returning to the appropriate controller service listing after going to the usage page. This closes #835 Signed-off-by: jpercivall > UI - Process Group configuration requires write permissions > --- > > Key: NIFI-2501 > URL: https://issues.apache.org/jira/browse/NIFI-2501 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.0.0 > > > Since Controller Services are accessed through the Process Group > configuration, we need to allow users without read/write permissions to > access that dialog in case they do have permissions to a controller service > in there. The Controller settings dialog is already updated to support this > case. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #835: Ensuring users can access the Controller Service and...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/835 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2559) UI - Controller Service referencing component run status missing
[ https://issues.apache.org/jira/browse/NIFI-2559?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421531#comment-15421531 ] ASF GitHub Bot commented on NIFI-2559: -- Github user olegz commented on the issue: https://github.com/apache/nifi/pull/864 Reviewing > UI - Controller Service referencing component run status missing > > > Key: NIFI-2559 > URL: https://issues.apache.org/jira/browse/NIFI-2559 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Reporter: Matt Gilman >Assignee: Scott Aslan > Fix For: 1.0.0 > > > In the Controller Service dialogs (configuration, enable, disable, etc) we > dynamically generate markup for components that reference that Controller > Service. Some of the information that we include is not rendering correctly > because they are reference old images/styles that are no longer applicable. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #864: [NIFI-2559] UI - Controller Service update missing icons
Github user olegz commented on the issue: https://github.com/apache/nifi/pull/864 Reviewing --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #860: Enforcing READ permissions when referencing ControllerServi...
Github user bbende commented on the issue: https://github.com/apache/nifi/pull/860 Reviewing... --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-2569) Allow multiple NiFi instances on same host, make range generation easier
[ https://issues.apache.org/jira/browse/NIFI-2569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-2569: -- Status: Patch Available (was: Open) > Allow multiple NiFi instances on same host, make range generation easier > > > Key: NIFI-2569 > URL: https://issues.apache.org/jira/browse/NIFI-2569 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Bryan Rosander >Assignee: Bryan Rosander > Fix For: 1.0.0 > > > In the standalone tool: > * In order to facilitate local secure clustering, we should be able to > generate multiple certificates for the same hostname. > * We should also be able to update ports in nifi.properties to increment in > that scenario. > * The keystore entries in nifi.properties should start with ./conf/ so that > the files in each host folder can just be copied to a NiFi instance. > * Specifying a range of hosts should be easier. > To that end, I've added range expansion. [] denote a range to include in the > hostname, () denote instance numbers for that hostname. > This can be used as follows: > nifi[01-3].domain expands to nifi01.domain, nifi02.domain, nifi03.domain > nifi(2) will create 2 instances of nifi with corresponding certificates and > nifi.properties with incremented port numbers. > These can be combined to something like: > nifi[1-10].subdomain[1-4].domain(2) for 2 instances of nifi running on each > host in the expanded set of ranges > If you need distinct ports for all nifi instances (not just those with the > same hostname), that can be achieved by specifying a global order with -G. > ex: bin/tls-toolkit.sh standalone -n nifi[3-5].domain -G nifi[1-100].domain > This would generate nifi3.domain, nifi4.domain, nifi5.domain and allot them > distinct ports derteministically mapped from the range of nifi[1-100].domain > so that later updates specifying the same global order won't have conflicting > port assignments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2569) Allow multiple NiFi instances on same host, make range generation easier
[ https://issues.apache.org/jira/browse/NIFI-2569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-2569: -- Fix Version/s: 1.0.0 > Allow multiple NiFi instances on same host, make range generation easier > > > Key: NIFI-2569 > URL: https://issues.apache.org/jira/browse/NIFI-2569 > Project: Apache NiFi > Issue Type: Sub-task >Reporter: Bryan Rosander >Assignee: Bryan Rosander > Fix For: 1.0.0 > > > In the standalone tool: > * In order to facilitate local secure clustering, we should be able to > generate multiple certificates for the same hostname. > * We should also be able to update ports in nifi.properties to increment in > that scenario. > * The keystore entries in nifi.properties should start with ./conf/ so that > the files in each host folder can just be copied to a NiFi instance. > * Specifying a range of hosts should be easier. > To that end, I've added range expansion. [] denote a range to include in the > hostname, () denote instance numbers for that hostname. > This can be used as follows: > nifi[01-3].domain expands to nifi01.domain, nifi02.domain, nifi03.domain > nifi(2) will create 2 instances of nifi with corresponding certificates and > nifi.properties with incremented port numbers. > These can be combined to something like: > nifi[1-10].subdomain[1-4].domain(2) for 2 instances of nifi running on each > host in the expanded set of ranges > If you need distinct ports for all nifi instances (not just those with the > same hostname), that can be achieved by specifying a global order with -G. > ex: bin/tls-toolkit.sh standalone -n nifi[3-5].domain -G nifi[1-100].domain > This would generate nifi3.domain, nifi4.domain, nifi5.domain and allot them > distinct ports derteministically mapped from the range of nifi[1-100].domain > so that later updates specifying the same global order won't have conflicting > port assignments. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2546) NPE while creating a template
[
https://issues.apache.org/jira/browse/NIFI-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Bende updated NIFI-2546:
--
Resolution: Fixed
Status: Resolved (was: Patch Available)
> NPE while creating a template
> -
>
> Key: NIFI-2546
> URL: https://issues.apache.org/jira/browse/NIFI-2546
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
>Priority: Critical
> Fix For: 1.0.0
>
>
> Attempted to create a template on latest master.
> {quote}
> 2016-08-10 18:10:16,596 ERROR [NiFi Web Server-150]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at java.util.UUID.fromString(UUID.java:192) ~[na:1.8.0_66]
> at
> org.apache.nifi.web.api.dto.FlowSnippetDTO.removeInstanceIdentifierIfNecessary(FlowSnippetDTO.java:233)
> ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.api.dto.FlowSnippetDTO.setProcessors(FlowSnippetDTO.java:157)
> ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.util.SnippetUtils.populateFlowSnippet(SnippetUtils.java:230)
> ~[classes/:na]
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.createTemplate(StandardNiFiServiceFacade.java:1532)
> ~[classes/:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> ~[classes/:1.0.0-SNAPSHOT]
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> ~[spring-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.apache.nifi.web.NiFiServiceFacadeLock.createLock(NiFiServiceFacadeLock.java:41)
> ~[classes/:1.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.8.0_66]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[na:1.8.0_66]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_66]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$2872927f.createTemplate()
> ~[classes/:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.api.ProcessGroupResource.createTemplate(ProcessGroupResource.java:1876)
> ~[classes/:na]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.8.0_66]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[na:1.8.0_66]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_66]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> ~[jersey-server-1.19.jar:1.19]
> at
> c
[jira] [Commented] (NIFI-2546) NPE while creating a template
[
https://issues.apache.org/jira/browse/NIFI-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421476#comment-15421476
]
ASF GitHub Bot commented on NIFI-2546:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/859
> NPE while creating a template
> -
>
> Key: NIFI-2546
> URL: https://issues.apache.org/jira/browse/NIFI-2546
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
>Priority: Critical
> Fix For: 1.0.0
>
>
> Attempted to create a template on latest master.
> {quote}
> 2016-08-10 18:10:16,596 ERROR [NiFi Web Server-150]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at java.util.UUID.fromString(UUID.java:192) ~[na:1.8.0_66]
> at
> org.apache.nifi.web.api.dto.FlowSnippetDTO.removeInstanceIdentifierIfNecessary(FlowSnippetDTO.java:233)
> ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.api.dto.FlowSnippetDTO.setProcessors(FlowSnippetDTO.java:157)
> ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.util.SnippetUtils.populateFlowSnippet(SnippetUtils.java:230)
> ~[classes/:na]
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.createTemplate(StandardNiFiServiceFacade.java:1532)
> ~[classes/:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> ~[classes/:1.0.0-SNAPSHOT]
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> ~[spring-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.apache.nifi.web.NiFiServiceFacadeLock.createLock(NiFiServiceFacadeLock.java:41)
> ~[classes/:1.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.8.0_66]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[na:1.8.0_66]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_66]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$2872927f.createTemplate()
> ~[classes/:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.api.ProcessGroupResource.createTemplate(ProcessGroupResource.java:1876)
> ~[classes/:na]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.8.0_66]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[na:1.8.0_66]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_66]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke
[jira] [Commented] (NIFI-2546) NPE while creating a template
[
https://issues.apache.org/jira/browse/NIFI-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421475#comment-15421475
]
ASF subversion and git services commented on NIFI-2546:
---
Commit 51c566ffc8cb89d28b2e5e4b40fe1fa60a2d477a in nifi's branch
refs/heads/master from [~ozhurakousky]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=51c566f ]
NIFI-2546 fixed NPE in FlowSnippetDTO fixed NPE caused by non-configured
ControllerService
This closes #859.
Signed-off-by: Bryan Bende
> NPE while creating a template
> -
>
> Key: NIFI-2546
> URL: https://issues.apache.org/jira/browse/NIFI-2546
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
>Priority: Critical
> Fix For: 1.0.0
>
>
> Attempted to create a template on latest master.
> {quote}
> 2016-08-10 18:10:16,596 ERROR [NiFi Web Server-150]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at java.util.UUID.fromString(UUID.java:192) ~[na:1.8.0_66]
> at
> org.apache.nifi.web.api.dto.FlowSnippetDTO.removeInstanceIdentifierIfNecessary(FlowSnippetDTO.java:233)
> ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.api.dto.FlowSnippetDTO.setProcessors(FlowSnippetDTO.java:157)
> ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.util.SnippetUtils.populateFlowSnippet(SnippetUtils.java:230)
> ~[classes/:na]
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.createTemplate(StandardNiFiServiceFacade.java:1532)
> ~[classes/:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> ~[classes/:1.0.0-SNAPSHOT]
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> ~[spring-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.apache.nifi.web.NiFiServiceFacadeLock.createLock(NiFiServiceFacadeLock.java:41)
> ~[classes/:1.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.8.0_66]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[na:1.8.0_66]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_66]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$2872927f.createTemplate()
> ~[classes/:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.api.ProcessGroupResource.createTemplate(ProcessGroupResource.java:1876)
> ~[classes/:na]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.8.0_66]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[na:1.8.0_66]
> at
> sun.re
[GitHub] nifi pull request #859: NIFI-2546 fixed NPE in FlowSnippetDTO
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/859 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #859: NIFI-2546 fixed NPE in FlowSnippetDTO
Github user bbende commented on the issue: https://github.com/apache/nifi/pull/859 Looks good, will merge to master --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2546) NPE while creating a template
[
https://issues.apache.org/jira/browse/NIFI-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421474#comment-15421474
]
ASF GitHub Bot commented on NIFI-2546:
--
Github user bbende commented on the issue:
https://github.com/apache/nifi/pull/859
Looks good, will merge to master
> NPE while creating a template
> -
>
> Key: NIFI-2546
> URL: https://issues.apache.org/jira/browse/NIFI-2546
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
>Priority: Critical
> Fix For: 1.0.0
>
>
> Attempted to create a template on latest master.
> {quote}
> 2016-08-10 18:10:16,596 ERROR [NiFi Web Server-150]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at java.util.UUID.fromString(UUID.java:192) ~[na:1.8.0_66]
> at
> org.apache.nifi.web.api.dto.FlowSnippetDTO.removeInstanceIdentifierIfNecessary(FlowSnippetDTO.java:233)
> ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.api.dto.FlowSnippetDTO.setProcessors(FlowSnippetDTO.java:157)
> ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.util.SnippetUtils.populateFlowSnippet(SnippetUtils.java:230)
> ~[classes/:na]
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.createTemplate(StandardNiFiServiceFacade.java:1532)
> ~[classes/:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> ~[classes/:1.0.0-SNAPSHOT]
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> ~[spring-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.apache.nifi.web.NiFiServiceFacadeLock.createLock(NiFiServiceFacadeLock.java:41)
> ~[classes/:1.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.8.0_66]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[na:1.8.0_66]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_66]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$2872927f.createTemplate()
> ~[classes/:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.api.ProcessGroupResource.createTemplate(ProcessGroupResource.java:1876)
> ~[classes/:na]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.8.0_66]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[na:1.8.0_66]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_66]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
> at
> com.sun.jersey.spi.contain
[jira] [Commented] (NIFI-2524) Unable to Merge Journal Files after OutOfMemory Error
[
https://issues.apache.org/jira/browse/NIFI-2524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421460#comment-15421460
]
ASF subversion and git services commented on NIFI-2524:
---
Commit fbd3201157883e2606746af448f5f3057d214d1e in nifi's branch
refs/heads/master from [~YolandaMDavis]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=fbd3201 ]
NIFI-2524 - Fixes to improve handling of missing journal files during
rollover/merge execution.Includes:
This closes #840
Removed partial file check (based on missing first file)
Added condition to merge if at least one journal files available on disk. If
all files are missing from disk that is considered an error.
Added retry logic to prevent endless thread execution when encountering errors
(such as missing files).
> Unable to Merge Journal Files after OutOfMemory Error
> -
>
> Key: NIFI-2524
> URL: https://issues.apache.org/jira/browse/NIFI-2524
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Yolanda M. Davis
>Assignee: Yolanda M. Davis
> Fix For: 1.0.0
>
> Attachments: nifi-app-1.log, nifi-app-2.log
>
>
> While running a flow that attempted to SplitText by eol an OutOfMemory Error
> occurred (which wasn't surprising since this was a standalone instance with
> no tuning for memory, backpressure, etc).
> {noformat}
> ERROR [Timer-Driven Process Thread-9] o.a.nifi.processors.standard.SplitText
> SplitText[id=6bb0fe36-0156-1000-795b-55cf4237a389]
> SplitText[id=6bb0fe36-0156-1000-795b-55cf4237a389] failed to process due to
> java.lang.OutOfMemoryError: Java heap space; rolling back session:
> java.lang.OutOfMemoryError: Java heap space
> 2016-08-08 20:12:36,068 ERROR
> [LeaseRenewer:root@ydavis-hdp-nifi-test-1.openstacklocal:8020]
> org.apache.nifi.NiFi An Unknown Error Occurred in Thread
> Thread[LeaseRenewer:root@ydavis-hdp-nifi-test-1.openstacklocal:8020,5,main]:
> java.lang.OutOfMemoryError: Java heap space
> 2016-08-08 20:12:36,072 ERROR [Timer-Driven Process Thread-8]
> o.a.n.p.PersistentProvenanceRepository Failed to persist Provenance Event due
> to java.io.IOException: Stream Closed.
> 2016-08-08 20:12:55,111 ERROR
> [LeaseRenewer:root@ydavis-hdp-nifi-test-1.openstacklocal:8020]
> org.apache.nifi.NiFi
> java.lang.OutOfMemoryError: Java heap space
> at java.util.ArrayList.iterator(ArrayList.java:834) ~[na:1.8.0_101]
> at
> org.apache.hadoop.hdfs.LeaseRenewer.clientsRunning(LeaseRenewer.java:241)
> ~[na:na]
> at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:491)
> ~[na:na]
> at org.apache.hadoop.hdfs.LeaseRenewer.access$700(LeaseRenewer.java:71)
> ~[na:na]
> at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:304)
> ~[na:na]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_101]
> 2016-08-08 20:12:58,022 ERROR [Timer-Driven Process Thread-9]
> o.a.nifi.processors.standard.SplitText
> java.lang.OutOfMemoryError: Java heap space
> at java.util.HashMap.newNode(HashMap.java:1742) ~[na:1.8.0_101]
> at java.util.HashMap.putVal(HashMap.java:630) ~[na:1.8.0_101]
> at java.util.HashMap.putMapEntries(HashMap.java:514) ~[na:1.8.0_101]
> at java.util.HashMap.putAll(HashMap.java:784) ~[na:1.8.0_101]
> at
> org.apache.nifi.controller.repository.StandardFlowFileRecord$Builder.fromFlowFile(StandardFlowFileRecord.java:307)
> ~[nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.putAttribute(StandardProcessSession.java:1462)
> ~[nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.processors.standard.SplitText$1.process(SplitText.java:499)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1880)
> ~[nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851)
> ~[nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.processors.standard.SplitText.onTrigger(SplitText.java:420)
> ~[na:na]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> ~[nifi-api-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1060)
> [nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
> [nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> [nifi-framew
[jira] [Commented] (NIFI-2524) Unable to Merge Journal Files after OutOfMemory Error
[
https://issues.apache.org/jira/browse/NIFI-2524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421462#comment-15421462
]
ASF GitHub Bot commented on NIFI-2524:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/840
> Unable to Merge Journal Files after OutOfMemory Error
> -
>
> Key: NIFI-2524
> URL: https://issues.apache.org/jira/browse/NIFI-2524
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Yolanda M. Davis
>Assignee: Yolanda M. Davis
> Fix For: 1.0.0
>
> Attachments: nifi-app-1.log, nifi-app-2.log
>
>
> While running a flow that attempted to SplitText by eol an OutOfMemory Error
> occurred (which wasn't surprising since this was a standalone instance with
> no tuning for memory, backpressure, etc).
> {noformat}
> ERROR [Timer-Driven Process Thread-9] o.a.nifi.processors.standard.SplitText
> SplitText[id=6bb0fe36-0156-1000-795b-55cf4237a389]
> SplitText[id=6bb0fe36-0156-1000-795b-55cf4237a389] failed to process due to
> java.lang.OutOfMemoryError: Java heap space; rolling back session:
> java.lang.OutOfMemoryError: Java heap space
> 2016-08-08 20:12:36,068 ERROR
> [LeaseRenewer:root@ydavis-hdp-nifi-test-1.openstacklocal:8020]
> org.apache.nifi.NiFi An Unknown Error Occurred in Thread
> Thread[LeaseRenewer:root@ydavis-hdp-nifi-test-1.openstacklocal:8020,5,main]:
> java.lang.OutOfMemoryError: Java heap space
> 2016-08-08 20:12:36,072 ERROR [Timer-Driven Process Thread-8]
> o.a.n.p.PersistentProvenanceRepository Failed to persist Provenance Event due
> to java.io.IOException: Stream Closed.
> 2016-08-08 20:12:55,111 ERROR
> [LeaseRenewer:root@ydavis-hdp-nifi-test-1.openstacklocal:8020]
> org.apache.nifi.NiFi
> java.lang.OutOfMemoryError: Java heap space
> at java.util.ArrayList.iterator(ArrayList.java:834) ~[na:1.8.0_101]
> at
> org.apache.hadoop.hdfs.LeaseRenewer.clientsRunning(LeaseRenewer.java:241)
> ~[na:na]
> at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:491)
> ~[na:na]
> at org.apache.hadoop.hdfs.LeaseRenewer.access$700(LeaseRenewer.java:71)
> ~[na:na]
> at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:304)
> ~[na:na]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_101]
> 2016-08-08 20:12:58,022 ERROR [Timer-Driven Process Thread-9]
> o.a.nifi.processors.standard.SplitText
> java.lang.OutOfMemoryError: Java heap space
> at java.util.HashMap.newNode(HashMap.java:1742) ~[na:1.8.0_101]
> at java.util.HashMap.putVal(HashMap.java:630) ~[na:1.8.0_101]
> at java.util.HashMap.putMapEntries(HashMap.java:514) ~[na:1.8.0_101]
> at java.util.HashMap.putAll(HashMap.java:784) ~[na:1.8.0_101]
> at
> org.apache.nifi.controller.repository.StandardFlowFileRecord$Builder.fromFlowFile(StandardFlowFileRecord.java:307)
> ~[nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.putAttribute(StandardProcessSession.java:1462)
> ~[nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.processors.standard.SplitText$1.process(SplitText.java:499)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1880)
> ~[nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851)
> ~[nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.processors.standard.SplitText.onTrigger(SplitText.java:420)
> ~[na:na]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> ~[nifi-api-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1060)
> [nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
> [nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> [nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
> [nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_101]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_101]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [n
[jira] [Updated] (NIFI-2524) Unable to Merge Journal Files after OutOfMemory Error
[
https://issues.apache.org/jira/browse/NIFI-2524?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Oleg Zhurakousky updated NIFI-2524:
---
Resolution: Fixed
Status: Resolved (was: Patch Available)
> Unable to Merge Journal Files after OutOfMemory Error
> -
>
> Key: NIFI-2524
> URL: https://issues.apache.org/jira/browse/NIFI-2524
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Yolanda M. Davis
>Assignee: Yolanda M. Davis
> Fix For: 1.0.0
>
> Attachments: nifi-app-1.log, nifi-app-2.log
>
>
> While running a flow that attempted to SplitText by eol an OutOfMemory Error
> occurred (which wasn't surprising since this was a standalone instance with
> no tuning for memory, backpressure, etc).
> {noformat}
> ERROR [Timer-Driven Process Thread-9] o.a.nifi.processors.standard.SplitText
> SplitText[id=6bb0fe36-0156-1000-795b-55cf4237a389]
> SplitText[id=6bb0fe36-0156-1000-795b-55cf4237a389] failed to process due to
> java.lang.OutOfMemoryError: Java heap space; rolling back session:
> java.lang.OutOfMemoryError: Java heap space
> 2016-08-08 20:12:36,068 ERROR
> [LeaseRenewer:root@ydavis-hdp-nifi-test-1.openstacklocal:8020]
> org.apache.nifi.NiFi An Unknown Error Occurred in Thread
> Thread[LeaseRenewer:root@ydavis-hdp-nifi-test-1.openstacklocal:8020,5,main]:
> java.lang.OutOfMemoryError: Java heap space
> 2016-08-08 20:12:36,072 ERROR [Timer-Driven Process Thread-8]
> o.a.n.p.PersistentProvenanceRepository Failed to persist Provenance Event due
> to java.io.IOException: Stream Closed.
> 2016-08-08 20:12:55,111 ERROR
> [LeaseRenewer:root@ydavis-hdp-nifi-test-1.openstacklocal:8020]
> org.apache.nifi.NiFi
> java.lang.OutOfMemoryError: Java heap space
> at java.util.ArrayList.iterator(ArrayList.java:834) ~[na:1.8.0_101]
> at
> org.apache.hadoop.hdfs.LeaseRenewer.clientsRunning(LeaseRenewer.java:241)
> ~[na:na]
> at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:491)
> ~[na:na]
> at org.apache.hadoop.hdfs.LeaseRenewer.access$700(LeaseRenewer.java:71)
> ~[na:na]
> at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:304)
> ~[na:na]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_101]
> 2016-08-08 20:12:58,022 ERROR [Timer-Driven Process Thread-9]
> o.a.nifi.processors.standard.SplitText
> java.lang.OutOfMemoryError: Java heap space
> at java.util.HashMap.newNode(HashMap.java:1742) ~[na:1.8.0_101]
> at java.util.HashMap.putVal(HashMap.java:630) ~[na:1.8.0_101]
> at java.util.HashMap.putMapEntries(HashMap.java:514) ~[na:1.8.0_101]
> at java.util.HashMap.putAll(HashMap.java:784) ~[na:1.8.0_101]
> at
> org.apache.nifi.controller.repository.StandardFlowFileRecord$Builder.fromFlowFile(StandardFlowFileRecord.java:307)
> ~[nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.putAttribute(StandardProcessSession.java:1462)
> ~[nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.processors.standard.SplitText$1.process(SplitText.java:499)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1880)
> ~[nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851)
> ~[nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.processors.standard.SplitText.onTrigger(SplitText.java:420)
> ~[na:na]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> ~[nifi-api-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1060)
> [nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
> [nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> [nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
> [nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_101]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_101]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
> [na:1.8.0_101]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$
[GitHub] nifi pull request #840: NIFI-2524 - Fixes to improve handling of missing jou...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/840 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-2573) After modifying a Controller Service or Reporting Task policies should return to originating screen
[ https://issues.apache.org/jira/browse/NIFI-2573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Percivall updated NIFI-2573: --- Fix Version/s: 1.1.0 > After modifying a Controller Service or Reporting Task policies should return > to originating screen > --- > > Key: NIFI-2573 > URL: https://issues.apache.org/jira/browse/NIFI-2573 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Joseph Percivall >Priority: Trivial > Fix For: 1.1.0 > > > Open a Process Group or the Controller configuration > click the "key" to modify a Controller Service or Reporting Task access > policies > see the policy window open > close the policy window > now at the canvas > After closing the policy window the User should be returned to where they > were before opening the window instead of returning to the canvas. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2573) After modifying a Controller Service or Reporting Task policies should return to originating screen
Joseph Percivall created NIFI-2573: -- Summary: After modifying a Controller Service or Reporting Task policies should return to originating screen Key: NIFI-2573 URL: https://issues.apache.org/jira/browse/NIFI-2573 Project: Apache NiFi Issue Type: Improvement Reporter: Joseph Percivall Priority: Trivial Open a Process Group or the Controller configuration click the "key" to modify a Controller Service or Reporting Task access policies see the policy window open close the policy window now at the canvas After closing the policy window the User should be returned to where they were before opening the window instead of returning to the canvas. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2524) Unable to Merge Journal Files after OutOfMemory Error
[
https://issues.apache.org/jira/browse/NIFI-2524?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421449#comment-15421449
]
ASF GitHub Bot commented on NIFI-2524:
--
Github user olegz commented on the issue:
https://github.com/apache/nifi/pull/840
+1. Merging
> Unable to Merge Journal Files after OutOfMemory Error
> -
>
> Key: NIFI-2524
> URL: https://issues.apache.org/jira/browse/NIFI-2524
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Yolanda M. Davis
>Assignee: Yolanda M. Davis
> Fix For: 1.0.0
>
> Attachments: nifi-app-1.log, nifi-app-2.log
>
>
> While running a flow that attempted to SplitText by eol an OutOfMemory Error
> occurred (which wasn't surprising since this was a standalone instance with
> no tuning for memory, backpressure, etc).
> {noformat}
> ERROR [Timer-Driven Process Thread-9] o.a.nifi.processors.standard.SplitText
> SplitText[id=6bb0fe36-0156-1000-795b-55cf4237a389]
> SplitText[id=6bb0fe36-0156-1000-795b-55cf4237a389] failed to process due to
> java.lang.OutOfMemoryError: Java heap space; rolling back session:
> java.lang.OutOfMemoryError: Java heap space
> 2016-08-08 20:12:36,068 ERROR
> [LeaseRenewer:root@ydavis-hdp-nifi-test-1.openstacklocal:8020]
> org.apache.nifi.NiFi An Unknown Error Occurred in Thread
> Thread[LeaseRenewer:root@ydavis-hdp-nifi-test-1.openstacklocal:8020,5,main]:
> java.lang.OutOfMemoryError: Java heap space
> 2016-08-08 20:12:36,072 ERROR [Timer-Driven Process Thread-8]
> o.a.n.p.PersistentProvenanceRepository Failed to persist Provenance Event due
> to java.io.IOException: Stream Closed.
> 2016-08-08 20:12:55,111 ERROR
> [LeaseRenewer:root@ydavis-hdp-nifi-test-1.openstacklocal:8020]
> org.apache.nifi.NiFi
> java.lang.OutOfMemoryError: Java heap space
> at java.util.ArrayList.iterator(ArrayList.java:834) ~[na:1.8.0_101]
> at
> org.apache.hadoop.hdfs.LeaseRenewer.clientsRunning(LeaseRenewer.java:241)
> ~[na:na]
> at org.apache.hadoop.hdfs.LeaseRenewer.run(LeaseRenewer.java:491)
> ~[na:na]
> at org.apache.hadoop.hdfs.LeaseRenewer.access$700(LeaseRenewer.java:71)
> ~[na:na]
> at org.apache.hadoop.hdfs.LeaseRenewer$1.run(LeaseRenewer.java:304)
> ~[na:na]
> at java.lang.Thread.run(Thread.java:745) ~[na:1.8.0_101]
> 2016-08-08 20:12:58,022 ERROR [Timer-Driven Process Thread-9]
> o.a.nifi.processors.standard.SplitText
> java.lang.OutOfMemoryError: Java heap space
> at java.util.HashMap.newNode(HashMap.java:1742) ~[na:1.8.0_101]
> at java.util.HashMap.putVal(HashMap.java:630) ~[na:1.8.0_101]
> at java.util.HashMap.putMapEntries(HashMap.java:514) ~[na:1.8.0_101]
> at java.util.HashMap.putAll(HashMap.java:784) ~[na:1.8.0_101]
> at
> org.apache.nifi.controller.repository.StandardFlowFileRecord$Builder.fromFlowFile(StandardFlowFileRecord.java:307)
> ~[nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.putAttribute(StandardProcessSession.java:1462)
> ~[nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.processors.standard.SplitText$1.process(SplitText.java:499)
> ~[na:na]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1880)
> ~[nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.repository.StandardProcessSession.read(StandardProcessSession.java:1851)
> ~[nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.processors.standard.SplitText.onTrigger(SplitText.java:420)
> ~[na:na]
> at
> org.apache.nifi.processor.AbstractProcessor.onTrigger(AbstractProcessor.java:27)
> ~[nifi-api-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.StandardProcessorNode.onTrigger(StandardProcessorNode.java:1060)
> [nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:136)
> [nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.tasks.ContinuallyRunProcessorTask.call(ContinuallyRunProcessorTask.java:47)
> [nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> org.apache.nifi.controller.scheduling.TimerDrivenSchedulingAgent$1.run(TimerDrivenSchedulingAgent.java:127)
> [nifi-framework-core-1.0.0-BETA.jar:1.0.0-BETA]
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
> [na:1.8.0_101]
> at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
> [na:1.8.0_101]
> at
> java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.j
[GitHub] nifi issue #840: NIFI-2524 - Fixes to improve handling of missing journal fi...
Github user olegz commented on the issue: https://github.com/apache/nifi/pull/840 +1. Merging --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Created] (NIFI-2572) Reporting tasks should be schedulable by users with "write" but not "view" permissions
Joseph Percivall created NIFI-2572: -- Summary: Reporting tasks should be schedulable by users with "write" but not "view" permissions Key: NIFI-2572 URL: https://issues.apache.org/jira/browse/NIFI-2572 Project: Apache NiFi Issue Type: Improvement Reporter: Joseph Percivall Priority: Minor Fix For: 1.1.0 Currently processors can be scheduled by users with "write" but not "view" permissions. While, Controller services and Reporting tasks require "view" and "write" in order to "enable"/"play" them. Controller services makes sense to require "view" and "write" to enable/disable because you need to know all the referencing components. Reporting tasks on the other hand are more analogous to Processors with respect to scheduling and for consistency should be able to be scheduled with just "write" permissions. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2546) NPE while creating a template
[
https://issues.apache.org/jira/browse/NIFI-2546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15421441#comment-15421441
]
ASF GitHub Bot commented on NIFI-2546:
--
Github user bbende commented on the issue:
https://github.com/apache/nifi/pull/859
Reviewing...
> NPE while creating a template
> -
>
> Key: NIFI-2546
> URL: https://issues.apache.org/jira/browse/NIFI-2546
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Reporter: Joseph Witt
>Assignee: Oleg Zhurakousky
>Priority: Critical
> Fix For: 1.0.0
>
>
> Attempted to create a template on latest master.
> {quote}
> 2016-08-10 18:10:16,596 ERROR [NiFi Web Server-150]
> o.a.nifi.web.api.config.ThrowableMapper An unexpected error has occurred:
> java.lang.NullPointerException. Returning Internal Server Error response.
> java.lang.NullPointerException: null
> at java.util.UUID.fromString(UUID.java:192) ~[na:1.8.0_66]
> at
> org.apache.nifi.web.api.dto.FlowSnippetDTO.removeInstanceIdentifierIfNecessary(FlowSnippetDTO.java:233)
> ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.api.dto.FlowSnippetDTO.setProcessors(FlowSnippetDTO.java:157)
> ~[nifi-client-dto-1.0.0-SNAPSHOT.jar:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.util.SnippetUtils.populateFlowSnippet(SnippetUtils.java:230)
> ~[classes/:na]
> at
> org.apache.nifi.web.StandardNiFiServiceFacade.createTemplate(StandardNiFiServiceFacade.java:1532)
> ~[classes/:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$FastClassBySpringCGLIB$$358780e0.invoke()
> ~[classes/:1.0.0-SNAPSHOT]
> at
> org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
> ~[spring-core-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:720)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.aspectj.MethodInvocationProceedingJoinPoint.proceed(MethodInvocationProceedingJoinPoint.java:85)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.apache.nifi.web.NiFiServiceFacadeLock.createLock(NiFiServiceFacadeLock.java:41)
> ~[classes/:1.0.0-SNAPSHOT]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.8.0_66]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[na:1.8.0_66]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_66]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethodWithGivenArgs(AbstractAspectJAdvice.java:621)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.aspectj.AbstractAspectJAdvice.invokeAdviceMethod(AbstractAspectJAdvice.java:610)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.aspectj.AspectJAroundAdvice.invoke(AspectJAroundAdvice.java:68)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.interceptor.ExposeInvocationInterceptor.invoke(ExposeInvocationInterceptor.java:92)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:655)
> ~[spring-aop-4.2.4.RELEASE.jar:4.2.4.RELEASE]
> at
> org.apache.nifi.web.StandardNiFiServiceFacade$$EnhancerBySpringCGLIB$$2872927f.createTemplate()
> ~[classes/:1.0.0-SNAPSHOT]
> at
> org.apache.nifi.web.api.ProcessGroupResource.createTemplate(ProcessGroupResource.java:1876)
> ~[classes/:na]
> at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
> ~[na:1.8.0_66]
> at
> sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
> ~[na:1.8.0_66]
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> ~[na:1.8.0_66]
> at java.lang.reflect.Method.invoke(Method.java:497) ~[na:1.8.0_66]
> at
> com.sun.jersey.spi.container.JavaMethodInvoker
