[jira] [Updated] (NIFI-604) ExecuteStreamCommand does not support arguments with semicolons
[
https://issues.apache.org/jira/browse/NIFI-604?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Ricky Saltzer updated NIFI-604:
---
Attachment: NIFI-604.2.patch
Submitting formatted patch
ExecuteStreamCommand does not support arguments with semicolons
Key: NIFI-604
URL: https://issues.apache.org/jira/browse/NIFI-604
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 0.1.0
Reporter: Ricky Saltzer
Assignee: Mark Payne
Fix For: 0.2.0
Attachments: NIFI-604.1.patch, NIFI-604.2.patch
The following code in ExecuteStreamCommand assumes you're not passing
semicolons within your argument. This is a problem for people who need to
pass semicolons to the executing program as part of the argument.
{code}
224for (String arg : commandArguments.split(;)) {
{code}
To allow for escaped semicolons, I propose we change this to the following
regex.
{code}
224for (String arg : commandArguments.split([^\\];)) {
{code}
*or*... could we just change the way arguments are passed to make it more
similar to how ExecuteCommand works? The whole semicolon per argument took
some getting used to, and doesn't seem very intuitive.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-379) MergeContent does not merge when Maximum number of Bins is set to 1
[ https://issues.apache.org/jira/browse/NIFI-379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-379: - Priority: Major (was: Trivial) MergeContent does not merge when Maximum number of Bins is set to 1 --- Key: NIFI-379 URL: https://issues.apache.org/jira/browse/NIFI-379 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 0.0.1 Reporter: Michael Moser Assignee: Joseph Witt Fix For: 0.2.0 Set the Maximum number of Bins property of the MergeContent processor to 1. The merged output always contains 1 file, no matter what the other settings are. Might be best to prevent the user setting Maximum number of Bins to 1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-379) MergeContent does not merge when Maximum number of Bins is set to 1
[ https://issues.apache.org/jira/browse/NIFI-379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-379: - Attachment: nifi-379.patch MergeContent does not merge when Maximum number of Bins is set to 1 --- Key: NIFI-379 URL: https://issues.apache.org/jira/browse/NIFI-379 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 0.0.1 Reporter: Michael Moser Assignee: Joseph Witt Fix For: 0.2.0 Attachments: nifi-379.patch Set the Maximum number of Bins property of the MergeContent processor to 1. The merged output always contains 1 file, no matter what the other settings are. Might be best to prevent the user setting Maximum number of Bins to 1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-601) Use dynamic properties as environment variables in ExecuteStreamCommand
[ https://issues.apache.org/jira/browse/NIFI-601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573926#comment-14573926 ] Aldrin Piri commented on NIFI-601: -- Looks good! Thanks for the contribution to provide a more consistent experience across the two execution processors. Merging. Use dynamic properties as environment variables in ExecuteStreamCommand --- Key: NIFI-601 URL: https://issues.apache.org/jira/browse/NIFI-601 Project: Apache NiFi Issue Type: New Feature Affects Versions: 0.1.0 Reporter: Ricky Saltzer Assignee: Ricky Saltzer Priority: Minor Fix For: 0.2.0 Attachments: NIFI-601.1.patch, NIFI-601.2.patch This is already being done with the ExecuteProcess processor, but not with the ExecuteStreamCommand. I'm working on adding this in now, and will submit patch here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
incubator-nifi git commit: NIFI-601: Add Dynamic Props - ExecuteStreamCommand
Repository: incubator-nifi
Updated Branches:
refs/heads/develop aeefe73f0 - b53948a6b
NIFI-601: Add Dynamic Props - ExecuteStreamCommand
- Added dynamic property support to ExecuteStreamCommand. All
properties added by user will be inserted into the command's
runtime environment.
Signed-off-by: Aldrin Piri ald...@apache.org
Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/b53948a6
Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/b53948a6
Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/b53948a6
Branch: refs/heads/develop
Commit: b53948a6ba1b3d1615d402eb49238dd5fd0f316b
Parents: aeefe73
Author: ricky ri...@cloudera.com
Authored: Fri May 8 13:37:37 2015 -0400
Committer: Aldrin Piri ald...@apache.org
Committed: Thu Jun 4 21:41:00 2015 -0400
--
.../nifi-standard-processors/pom.xml| 1 +
.../standard/ExecuteStreamCommand.java | 19
.../src/test/java/TestDynamicEnvironment.java | 29 +++
.../standard/TestExecuteStreamCommand.java | 26 +
.../ExecuteCommand/TestDynamicEnvironment.jar | Bin 0 - 1200 bytes
5 files changed, 75 insertions(+)
--
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/b53948a6/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml
--
diff --git
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml
index f8c272c..0d56bbb 100644
---
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml
+++
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/pom.xml
@@ -248,6 +248,7 @@
excludesrc/test/resources/CompressedData/SampleFileConcat.txt.bz2/exclude
excludesrc/test/resources/ExecuteCommand/TestIngestAndUpdate.jar/exclude
excludesrc/test/resources/ExecuteCommand/TestSuccess.jar/exclude
+
excludesrc/test/resources/ExecuteCommand/TestDynamicEnvironment.jar/exclude
excludesrc/test/resources/TestIdentifyMimeType/1.jar/exclude
excludesrc/test/resources/TestIdentifyMimeType/1.tar/exclude
excludesrc/test/resources/TestIdentifyMimeType/1.tar.gz/exclude
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/b53948a6/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteStreamCommand.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteStreamCommand.java
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteStreamCommand.java
index 76512dc..4c4288a 100644
---
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteStreamCommand.java
+++
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ExecuteStreamCommand.java
@@ -33,6 +33,7 @@ import java.util.Set;
import org.apache.commons.io.IOUtils;
import org.apache.commons.lang3.StringUtils;
+import org.apache.nifi.annotation.behavior.DynamicProperty;
import org.apache.nifi.annotation.behavior.EventDriven;
import org.apache.nifi.annotation.behavior.SupportsBatching;
import org.apache.nifi.annotation.documentation.CapabilityDescription;
@@ -113,6 +114,7 @@ import org.apache.nifi.stream.io.StreamUtils;
@SupportsBatching
@Tags({command execution, command, stream, execute})
@CapabilityDescription(Executes an external command on the contents of a flow
file, and creates a new flow file with the results of the command.)
+@DynamicProperty(name = An environment variable name, value = An
environment variable value, description = These environment variables are
passed to the process spawned by this Processor)
@WritesAttributes({
@WritesAttribute(attribute = execution.command, description = The name
of the command executed to create the new FlowFile),
@WritesAttribute(attribute = execution.command.args, description = The
semi-colon delimited list of arguments),
@@ -203,6 +205,16 @@ public class ExecuteStreamCommand extends
AbstractProcessor {
}
@Override
+protected PropertyDescriptor getSupportedDynamicPropertyDescriptor(final
String propertyDescriptorName) {
+return new PropertyDescriptor.Builder()
+
[jira] [Commented] (NIFI-601) Use dynamic properties as environment variables in ExecuteStreamCommand
[ https://issues.apache.org/jira/browse/NIFI-601?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573927#comment-14573927 ] ASF subversion and git services commented on NIFI-601: -- Commit b53948a6ba1b3d1615d402eb49238dd5fd0f316b in incubator-nifi's branch refs/heads/develop from [~rickysaltzer] [ https://git-wip-us.apache.org/repos/asf?p=incubator-nifi.git;h=b53948a ] NIFI-601: Add Dynamic Props - ExecuteStreamCommand - Added dynamic property support to ExecuteStreamCommand. All properties added by user will be inserted into the command's runtime environment. Signed-off-by: Aldrin Piri ald...@apache.org Use dynamic properties as environment variables in ExecuteStreamCommand --- Key: NIFI-601 URL: https://issues.apache.org/jira/browse/NIFI-601 Project: Apache NiFi Issue Type: New Feature Affects Versions: 0.1.0 Reporter: Ricky Saltzer Assignee: Ricky Saltzer Priority: Minor Fix For: 0.2.0 Attachments: NIFI-601.1.patch, NIFI-601.2.patch This is already being done with the ExecuteProcess processor, but not with the ExecuteStreamCommand. I'm working on adding this in now, and will submit patch here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-453) Update extensions that use AllowableValue to specify a description where appropriate
[ https://issues.apache.org/jira/browse/NIFI-453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-453: - Assignee: (was: Joseph Witt) Update extensions that use AllowableValue to specify a description where appropriate Key: NIFI-453 URL: https://issues.apache.org/jira/browse/NIFI-453 Project: Apache NiFi Issue Type: Improvement Components: Extensions Affects Versions: 0.1.0 Reporter: Dan Bress Priority: Minor AllowableValue allows you to specify a description of the AllowableValue. This can be very helpful for the conflict resolution strategies, as the difference between reject fail and ignore might not be clear to new users -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-453) Update extensions that use AllowableValue to specify a description where appropriate
[ https://issues.apache.org/jira/browse/NIFI-453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573898#comment-14573898 ] Joseph Witt commented on NIFI-453: -- Looked through a bunch of these. I would recommend we punt this ticket frankly and simply put in the developer guide that there is value in using this feature. As folks go back through these they can update them as needed but having a ticket out for it generically seems unlikely to get knocked out. Update extensions that use AllowableValue to specify a description where appropriate Key: NIFI-453 URL: https://issues.apache.org/jira/browse/NIFI-453 Project: Apache NiFi Issue Type: Improvement Components: Extensions Affects Versions: 0.1.0 Reporter: Dan Bress Assignee: Joseph Witt Priority: Minor AllowableValue allows you to specify a description of the AllowableValue. This can be very helpful for the conflict resolution strategies, as the difference between reject fail and ignore might not be clear to new users -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-453) Update extensions that use AllowableValue to specify a description where appropriate
[ https://issues.apache.org/jira/browse/NIFI-453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-453: - Fix Version/s: (was: 0.2.0) Update extensions that use AllowableValue to specify a description where appropriate Key: NIFI-453 URL: https://issues.apache.org/jira/browse/NIFI-453 Project: Apache NiFi Issue Type: Improvement Components: Extensions Affects Versions: 0.1.0 Reporter: Dan Bress Priority: Minor AllowableValue allows you to specify a description of the AllowableValue. This can be very helpful for the conflict resolution strategies, as the difference between reject fail and ignore might not be clear to new users -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-379) MergeContent does not merge when Maximum number of Bins is set to 1
[ https://issues.apache.org/jira/browse/NIFI-379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573969#comment-14573969 ] Joseph Witt commented on NIFI-379: -- The bug appears to have been just an improper handling for edge testing. Should have been '=' but was just . This bug was most 'obvious' in the case of a single bin but it would also impact the efficiency of packing bins whenever the max number of bins is reached for any reason. I am not entirely sure the logic for 'migrateBins' of BinFiles which kicks out the oldest bin whenever the max bins is reached and no bins are ready to be processed is correct. But can't quite reason over it enough to flag it so moving on. MergeContent does not merge when Maximum number of Bins is set to 1 --- Key: NIFI-379 URL: https://issues.apache.org/jira/browse/NIFI-379 Project: Apache NiFi Issue Type: Bug Components: Extensions Affects Versions: 0.0.1 Reporter: Michael Moser Assignee: Joseph Witt Fix For: 0.2.0 Set the Maximum number of Bins property of the MergeContent processor to 1. The merged output always contains 1 file, no matter what the other settings are. Might be best to prevent the user setting Maximum number of Bins to 1. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-453) Update extensions that use AllowableValue to specify a description where appropriate
[ https://issues.apache.org/jira/browse/NIFI-453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573891#comment-14573891 ] Joseph Witt commented on NIFI-453: -- Found use of AllowableValue here: ./nifi-nar-bundles/nifi-kafka-bundle/nifi-kafka-processors/src/main/java/org/apache/nifi/processors/kafka/PutKafka.java: ./nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/HandleHttpRequest.java ./nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/RouteOnAttribute.java ./nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/SplitContent.java ./nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/MergeContent.java ./nifi-nar-bundles/nifi-standard-services/nifi-dbcp-service-bundle/nifi-dbcp-service/src/main/java/org/apache/nifi/dbcp/DatabaseSystemDescriptor.java ./nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/sns/AbstractSNSProcessor.java ./nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/src/main/java/org/apache/nifi/processors/aws/AbstractAWSProcessor.java ./nifi-nar-bundles/nifi-social-media-bundle/nifi-twitter-processors/src/main/java/org/apache/nifi/processors/twitter/GetTwitter.java ./nifi-nar-bundles/nifi-solr-bundle/nifi-solr-processors/src/main/java/org/apache/nifi/processors/solr/SolrProcessor.java Update extensions that use AllowableValue to specify a description where appropriate Key: NIFI-453 URL: https://issues.apache.org/jira/browse/NIFI-453 Project: Apache NiFi Issue Type: Improvement Components: Extensions Affects Versions: 0.1.0 Reporter: Dan Bress Assignee: Joseph Witt Priority: Minor Fix For: 0.2.0 AllowableValue allows you to specify a description of the AllowableValue. This can be very helpful for the conflict resolution strategies, as the difference between reject fail and ignore might not be clear to new users -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[1/3] incubator-nifi git commit: NIFI-379 fixed what appears to be a faulty edge condition handling
Repository: incubator-nifi
Updated Branches:
refs/heads/develop b53948a6b - a49a03da4
NIFI-379 fixed what appears to be a faulty edge condition handling
Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/819b65f7
Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/819b65f7
Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/819b65f7
Branch: refs/heads/develop
Commit: 819b65f7e08ff5a836bd7c9fd29826539344abe9
Parents: b53948a
Author: joewitt joew...@apache.org
Authored: Thu Jun 4 22:48:17 2015 -0400
Committer: joewitt joew...@apache.org
Committed: Thu Jun 4 22:48:38 2015 -0400
--
.../nifi/processors/standard/BinFiles.java | 23 ++--
.../processors/standard/TestMergeContent.java | 20 +
2 files changed, 31 insertions(+), 12 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/819b65f7/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/BinFiles.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/BinFiles.java
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/BinFiles.java
index 399a12b..a7b4b28 100644
---
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/BinFiles.java
+++
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/BinFiles.java
@@ -171,17 +171,17 @@ public abstract class BinFiles extends
AbstractSessionFactoryProcessor {
@Override
public final void onTrigger(final ProcessContext context, final
ProcessSessionFactory sessionFactory) throws ProcessException {
-int binsAdded = binFlowFiles(context, sessionFactory);
-getLogger().debug(Binned {} FlowFiles, new Object[]{binsAdded});
+final int flowFilesBinned = binFlowFiles(context, sessionFactory);
+getLogger().debug(Binned {} FlowFiles, new
Object[]{flowFilesBinned});
if (!isScheduled()) {
return;
}
-binsAdded += migrateBins(context);
-
+final int binsMigrated = migrateBins(context);
final int binsProcessed = processBins(context, sessionFactory);
-if (binsProcessed == 0 binsAdded == 0) {
+//If we accomplished nothing then let's yield
+if (flowFilesBinned == 0 binsMigrated == 0 binsProcessed == 0) {
context.yield();
}
}
@@ -203,7 +203,6 @@ public abstract class BinFiles extends
AbstractSessionFactoryProcessor {
this.readyBins.add(bin);
}
}
-
return added;
}
@@ -251,16 +250,16 @@ public abstract class BinFiles extends
AbstractSessionFactoryProcessor {
}
private int binFlowFiles(final ProcessContext context, final
ProcessSessionFactory sessionFactory) {
-int binsAdded = 0;
-while (binManager.getBinCount()
context.getProperty(MAX_BIN_COUNT).asInteger().intValue()) {
+int flowFilesBinned = 0;
+while (binManager.getBinCount() =
context.getProperty(MAX_BIN_COUNT).asInteger().intValue()) {
if (!isScheduled()) {
-return binsAdded;
+break;
}
final ProcessSession session = sessionFactory.createSession();
FlowFile flowFile = session.get();
if (flowFile == null) {
-return binsAdded;
+break;
}
flowFile = this.preprocessFlowFile(context, session, flowFile);
@@ -276,10 +275,10 @@ public abstract class BinFiles extends
AbstractSessionFactoryProcessor {
this.readyBins.add(bin);
}
-binsAdded++;
+flowFilesBinned++;
}
-return binsAdded;
+return flowFilesBinned;
}
@OnScheduled
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/819b65f7/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestMergeContent.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestMergeContent.java
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestMergeContent.java
index aad2593..65925f7 100644
---
[3/3] incubator-nifi git commit: NIFI-613 and NIFI-379 checkstyle fix
NIFI-613 and NIFI-379 checkstyle fix
Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/a49a03da
Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/a49a03da
Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/a49a03da
Branch: refs/heads/develop
Commit: a49a03da43c52cd84a154f53aaa0ddbd366e6c1b
Parents: 1fb6aa4
Author: joewitt joew...@apache.org
Authored: Thu Jun 4 23:29:32 2015 -0400
Committer: joewitt joew...@apache.org
Committed: Thu Jun 4 23:29:32 2015 -0400
--
.../org/apache/nifi/processors/standard/TestMergeContent.java| 4 ++--
.../org/apache/nifi/processors/standard/TestScanContent.java | 1 -
2 files changed, 2 insertions(+), 3 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/a49a03da/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestMergeContent.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestMergeContent.java
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestMergeContent.java
index 65925f7..daad455 100644
---
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestMergeContent.java
+++
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestMergeContent.java
@@ -73,7 +73,7 @@ public class TestMergeContent {
bundle.assertContentEquals(Hello, World!.getBytes(UTF-8));
bundle.assertAttributeEquals(CoreAttributes.MIME_TYPE.key(),
application/plain-text);
}
-
+
@Test
public void testSimpleBinaryConcatSingleBin() throws IOException,
InterruptedException {
final TestRunner runner = TestRunners.newTestRunner(new
MergeContent());
@@ -92,7 +92,7 @@ public class TestMergeContent {
final MockFlowFile bundle =
runner.getFlowFilesForRelationship(MergeContent.REL_MERGED).get(0);
bundle.assertContentEquals(Hello, World!.getBytes(UTF-8));
bundle.assertAttributeEquals(CoreAttributes.MIME_TYPE.key(),
application/plain-text);
-}
+}
@Test
public void testSimpleBinaryConcatWithTextDelimiters() throws IOException,
InterruptedException {
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/a49a03da/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestScanContent.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestScanContent.java
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestScanContent.java
index c221965..4b87387 100644
---
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestScanContent.java
+++
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestScanContent.java
@@ -18,7 +18,6 @@ package org.apache.nifi.processors.standard;
import java.io.DataOutputStream;
import java.io.IOException;
-import java.io.StringWriter;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
[2/3] incubator-nifi git commit: NIFI-613 just needed to check for line of length 0. Added unit test to prove fail and fix
NIFI-613 just needed to check for line of length 0. Added unit test to prove
fail and fix
Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/1fb6aa49
Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/1fb6aa49
Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/1fb6aa49
Branch: refs/heads/develop
Commit: 1fb6aa49ba6a5e2fd45e8482deaf35401bcaace2
Parents: 819b65f
Author: joewitt joew...@apache.org
Authored: Thu Jun 4 23:22:23 2015 -0400
Committer: joewitt joew...@apache.org
Committed: Thu Jun 4 23:22:23 2015 -0400
--
.../nifi/processors/standard/ScanContent.java| 2 +-
.../processors/standard/TestScanContent.java | 19 +++
2 files changed, 20 insertions(+), 1 deletion(-)
--
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/1fb6aa49/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ScanContent.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ScanContent.java
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ScanContent.java
index ab5e8b5..445249b 100644
---
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ScanContent.java
+++
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/ScanContent.java
@@ -261,7 +261,7 @@ public class ScanContent extends AbstractProcessor {
@Override
public SearchTermbyte[] nextTerm() throws IOException {
final String nextLine = reader.readLine();
-if (nextLine == null) {
+if (nextLine == null || nextLine.isEmpty()) {
return null;
}
return new SearchTerm(nextLine.getBytes(UTF-8));
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/1fb6aa49/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestScanContent.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestScanContent.java
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestScanContent.java
index 8c36845..c221965 100644
---
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestScanContent.java
+++
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/test/java/org/apache/nifi/processors/standard/TestScanContent.java
@@ -18,6 +18,7 @@ package org.apache.nifi.processors.standard;
import java.io.DataOutputStream;
import java.io.IOException;
+import java.io.StringWriter;
import java.nio.file.Files;
import java.nio.file.Path;
import java.nio.file.Paths;
@@ -34,6 +35,24 @@ import org.junit.Test;
public class TestScanContent {
+@Test
+public void testBlankLineInDictionaryTextEncoding() throws IOException {
+final String dictionaryWithBlankLine = Line1\n\nLine3;
+final byte[] dictionaryBytes =
dictionaryWithBlankLine.getBytes(ScanContent.UTF8);
+final Path dictionaryPath = Paths.get(target/dictionary);
+Files.write(dictionaryPath, dictionaryBytes,
StandardOpenOption.CREATE, StandardOpenOption.WRITE);
+
+final TestRunner runner = TestRunners.newTestRunner(new ScanContent());
+runner.setThreadCount(1);
+runner.setProperty(ScanContent.DICTIONARY, dictionaryPath.toString());
+runner.setProperty(ScanContent.DICTIONARY_ENCODING,
ScanContent.TEXT_ENCODING);
+
+runner.enqueue(new byte[0]);
+runner.run();
+runner.assertTransferCount(ScanContent.REL_NO_MATCH, 1);
+
+}
+
@Ignore(This test has a race condition/ordering problem)
@Test
public void testBinaryScan() throws IOException {
[jira] [Updated] (NIFI-601) Use dynamic properties as environment variables in ExecuteStreamCommand
[ https://issues.apache.org/jira/browse/NIFI-601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ricky Saltzer updated NIFI-601: --- Attachment: NIFI-601.2.patch Attaching formatted patch Use dynamic properties as environment variables in ExecuteStreamCommand --- Key: NIFI-601 URL: https://issues.apache.org/jira/browse/NIFI-601 Project: Apache NiFi Issue Type: New Feature Affects Versions: 0.1.0 Reporter: Ricky Saltzer Assignee: Ricky Saltzer Priority: Minor Fix For: 0.2.0 Attachments: NIFI-601.1.patch, NIFI-601.2.patch This is already being done with the ExecuteProcess processor, but not with the ExecuteStreamCommand. I'm working on adding this in now, and will submit patch here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-632) Documentation Generation: Nar ClassLoader
[ https://issues.apache.org/jira/browse/NIFI-632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573597#comment-14573597 ] Matt Gilman commented on NIFI-632: -- [~danbress] While your working through the remainder of this ticket, I wanted to add that you should probably be invoking OnRemoved/OnShutdown on the components once your done with them (or whatever is appropriate for the component in question). We create and initialize the components which gives them the opportunity to some set up. We need to ensure these components also have the opportunity to clean up. Documentation Generation: Nar ClassLoader - Key: NIFI-632 URL: https://issues.apache.org/jira/browse/NIFI-632 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 0.1.0 Reporter: Matt Gilman Assignee: Dan Bress Priority: Minor Fix For: 0.2.0 When auto generating component documentation the NarThreadContextClassLoader needs to be set in case the component attempts to load anything off of its classpath. Without this, the documentation generation fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[2/3] incubator-nifi git commit: NIFI-657: Ignored Unit tests and removed maven plugin for embedding mongodb, as they fail in certain environments
NIFI-657: Ignored Unit tests and removed maven plugin for embedding mongodb, as
they fail in certain environments
Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/fcadaa89
Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/fcadaa89
Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/fcadaa89
Branch: refs/heads/develop
Commit: fcadaa894b39318d11fa6dffa7846d87c9a50d8c
Parents: 9fbd88f
Author: Mark Payne marka...@hotmail.com
Authored: Thu Jun 4 13:01:23 2015 -0400
Committer: Mark Payne marka...@hotmail.com
Committed: Thu Jun 4 13:01:23 2015 -0400
--
.../nifi-mongodb-processors/pom.xml | 31
.../nifi/processors/mongodb/GetMongoTest.java | 4 ++-
.../nifi/processors/mongodb/PutMongoTest.java | 8 +++--
3 files changed, 8 insertions(+), 35 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/fcadaa89/nifi/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/pom.xml
--
diff --git
a/nifi/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/pom.xml
b/nifi/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/pom.xml
index db1c498..020f4dc 100644
--- a/nifi/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/pom.xml
+++ b/nifi/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/pom.xml
@@ -64,35 +64,4 @@
scopetest/scope
/dependency
/dependencies
-
-build
-plugins
-plugin
-groupIdcom.github.joelittlejohn.embedmongo/groupId
-artifactIdembedmongo-maven-plugin/artifactId
-version0.1.12/version
-executions
-execution
-idstart/id
-goals
-goalstart/goal
-/goals
-phasetest-compile/phase
-configuration
-
databaseDirectory${project.build.directory}/embedmongo/db/databaseDirectory
-loggingfile/logging
-
logFile${project.build.directory}/embedmongo.log/logFile
-/configuration
-/execution
-execution
-idstop/id
-goals
-goalstop/goal
-/goals
-phaseprepare-package/phase
-/execution
-/executions
-/plugin
-/plugins
-/build
/project
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/fcadaa89/nifi/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/GetMongoTest.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/GetMongoTest.java
b/nifi/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/GetMongoTest.java
index 53f47f2..810fc4d 100644
---
a/nifi/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/GetMongoTest.java
+++
b/nifi/nifi-nar-bundles/nifi-mongodb-bundle/nifi-mongodb-processors/src/test/java/org/apache/nifi/processors/mongodb/GetMongoTest.java
@@ -33,6 +33,7 @@ import org.bson.Document;
import org.junit.After;
import org.junit.Assert;
import org.junit.Before;
+import org.junit.Ignore;
import org.junit.Test;
import com.google.common.collect.Lists;
@@ -40,6 +41,7 @@ import com.mongodb.MongoClient;
import com.mongodb.MongoClientURI;
import com.mongodb.client.MongoCollection;
+@Ignore(Integration tests that cause failures in some environments. Require
that they be run from Maven to run the embedded mongo maven plugin. Maven
Plugin also fails in my CentOS 7 environment.)
public class GetMongoTest {
private static final String MONGO_URI = mongodb://localhost;
private static final String DB_NAME =
GetMongoTest.class.getSimpleName().toLowerCase();
@@ -49,7 +51,7 @@ public class GetMongoTest {
new Document(_id, doc_1).append(a, 1).append(b, 2).append(c,
3),
new Document(_id, doc_2).append(a, 1).append(b, 2).append(c,
4),
new Document(_id, doc_3).append(a, 1).append(b, 3)
-);
+);
private TestRunner runner;
private MongoClient mongoClient;
[jira] [Commented] (NIFI-647) Duplicating provenance reporting via session creates unexpected behavior
[
https://issues.apache.org/jira/browse/NIFI-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573156#comment-14573156
]
ASF subversion and git services commented on NIFI-647:
--
Commit 9fbd88f034e6d5006b0f9cc7c4ffcd09c7c65a01 in incubator-nifi's branch
refs/heads/develop from [~markap14]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-nifi.git;h=9fbd88f ]
NIFI-647: When FORK Event emitted by processor, Framework generates CREATE
events for each child in addition to the FORK event; this commit fixes that bug
Duplicating provenance reporting via session creates unexpected behavior
Key: NIFI-647
URL: https://issues.apache.org/jira/browse/NIFI-647
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 0.0.2, 0.1.0
Reporter: Aldrin Piri
Assignee: Mark Payne
Fix For: 0.2.0
Attachments:
0001-NIFI-647-When-FORK-Event-emitted-by-processor-Framew.patch
This is related to NIFI-643. While that processor bug was resolved, there are
issues with how this is managed in the framework.
The unexpected behavior is created via the following:
https://github.com/apache/incubator-nifi/blob/d29a2d688e437bae42c12115768cdb038b7406c5/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/SplitJson.java#L196
Summarily, a series of files (splits) are created from the parent file which
generates the associated FORK provenance events. The separate call to the
provenance reporter at Line 196 also requests a FORK for each of the
generated files. This manifests itself as being CREATE events for each of
the FORKed files and causes an error:
{code}
Found cycle in graph. This indicates that multiple events were registered
claiming to have generated the same FlowFile (UUID =
21b663c2-2406-49cb-b740-76dd64f54be8)
{code}
Ideally, such behavior would also be detected by the test framework.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[3/3] incubator-nifi git commit: Merge branch 'develop' of http://git-wip-us.apache.org/repos/asf/incubator-nifi into develop
Merge branch 'develop' of http://git-wip-us.apache.org/repos/asf/incubator-nifi into develop Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/efc862ea Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/efc862ea Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/efc862ea Branch: refs/heads/develop Commit: efc862eac8f24058916b6c37c1674f2aa9577e01 Parents: fcadaa8 fb75a4b Author: Mark Payne marka...@hotmail.com Authored: Thu Jun 4 13:09:07 2015 -0400 Committer: Mark Payne marka...@hotmail.com Committed: Thu Jun 4 13:09:07 2015 -0400 -- .../nifi-hl7-query-language/pom.xml | 2 +- nifi/nifi-docs/pom.xml | 2 +- nifi/nifi-external/nifi-spark-receiver/pom.xml | 1 - .../nifi-aws-bundle/nifi-aws-processors/pom.xml | 2 +- .../nifi-framework-cluster/pom.xml | 2 +- .../nifi-framework/nifi-framework-core/pom.xml | 2 +- .../nifi-web/nifi-web-content-viewer/pom.xml| 2 +- .../nifi-framework/nifi-web/nifi-web-ui/pom.xml | 2 +- .../nifi-hl7-bundle/nifi-hl7-processors/pom.xml | 2 +- .../nifi-solr-processors/pom.xml| 2 +- .../nifi-standard-processors/pom.xml| 2 +- nifi/pom.xml| 45 +++- 12 files changed, 35 insertions(+), 31 deletions(-) --
[1/2] incubator-nifi git commit: NIFI-644: - Using mime.type attribute when known. If not specified or is octet-stream continue using content detection (tika).
Repository: incubator-nifi
Updated Branches:
refs/heads/develop efc862eac - e2227cdaf
NIFI-644:
- Using mime.type attribute when known. If not specified or is octet-stream
continue using content detection (tika).
Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/be50793d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/be50793d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/be50793d
Branch: refs/heads/develop
Commit: be50793decde1279f78e68548f5807a4cc646780
Parents: 07628f7
Author: Matt Gilman matt.c.gil...@gmail.com
Authored: Thu Jun 4 12:53:42 2015 -0400
Committer: Matt Gilman matt.c.gil...@gmail.com
Committed: Thu Jun 4 12:53:42 2015 -0400
--
.../apache/nifi/web/api/ProvenanceResource.java | 4 +-
.../nifi/web/ContentViewerController.java | 210 ++-
2 files changed, 111 insertions(+), 103 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/be50793d/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProvenanceResource.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProvenanceResource.java
b/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProvenanceResource.java
index afa404d..a2e7ef3 100644
---
a/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProvenanceResource.java
+++
b/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-api/src/main/java/org/apache/nifi/web/api/ProvenanceResource.java
@@ -276,7 +276,7 @@ public class ProvenanceResource extends ApplicationResource
{
*/
@GET
@Consumes(MediaType.WILDCARD)
-@Produces(MediaType.APPLICATION_OCTET_STREAM)
+@Produces(MediaType.WILDCARD)
@Path(/events/{id}/content/input)
@PreAuthorize(hasRole('ROLE_PROVENANCE'))
@ApiOperation(
@@ -375,7 +375,7 @@ public class ProvenanceResource extends ApplicationResource
{
*/
@GET
@Consumes(MediaType.WILDCARD)
-@Produces(MediaType.APPLICATION_OCTET_STREAM)
+@Produces(MediaType.WILDCARD)
@Path(/events/{id}/content/output)
@PreAuthorize(hasRole('ROLE_PROVENANCE'))
@ApiOperation(
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/be50793d/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-content-viewer/src/main/java/org/apache/nifi/web/ContentViewerController.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-content-viewer/src/main/java/org/apache/nifi/web/ContentViewerController.java
b/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-content-viewer/src/main/java/org/apache/nifi/web/ContentViewerController.java
index 304f85e..984b8a9 100644
---
a/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-content-viewer/src/main/java/org/apache/nifi/web/ContentViewerController.java
+++
b/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-content-viewer/src/main/java/org/apache/nifi/web/ContentViewerController.java
@@ -132,130 +132,138 @@ public class ContentViewerController extends
HttpServlet {
}
// buffer the content to support reseting in case we need to detect
the content type or char encoding
-final BufferedInputStream bis = new
BufferedInputStream(downloadableContent.getContent());
+try (final BufferedInputStream bis = new
BufferedInputStream(downloadableContent.getContent());) {
+final String mimeType;
-// detect the content type
-final DefaultDetector detector = new DefaultDetector();
+// when standalone and we don't know the type is null as we were
able to directly access the content bypassing the rest endpoint,
+// when clustered and we don't know the type set to octet stream
since the content was retrieved from the node's rest endpoint
+if (downloadableContent.getType() == null ||
downloadableContent.getType().equals(MediaType.OCTET_STREAM.toString())) {
+// attempt to detect the content stream if we don't know what
it is ()
+final DefaultDetector detector = new DefaultDetector();
-// create the stream for tika to process, buffered to support reseting
-final TikaInputStream tikaStream = TikaInputStream.get(bis);
+// create the stream for
[jira] [Commented] (NIFI-644) Data Viewer: Use mime.type when known
[ https://issues.apache.org/jira/browse/NIFI-644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573166#comment-14573166 ] ASF subversion and git services commented on NIFI-644: -- Commit be50793decde1279f78e68548f5807a4cc646780 in incubator-nifi's branch refs/heads/develop from [~mcgilman] [ https://git-wip-us.apache.org/repos/asf?p=incubator-nifi.git;h=be50793 ] NIFI-644: - Using mime.type attribute when known. If not specified or is octet-stream continue using content detection (tika). Data Viewer: Use mime.type when known - Key: NIFI-644 URL: https://issues.apache.org/jira/browse/NIFI-644 Project: Apache NiFi Issue Type: Improvement Components: Core UI Reporter: Matt Gilman Assignee: Matt Gilman Fix For: 0.2.0 Attachments: 0001-NIFI-644.patch If the mime.type is known, use it. If it is not known, then revert to the standard content detection process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[2/2] incubator-nifi git commit: Merge branch 'develop' of https://git-wip-us.apache.org/repos/asf/incubator-nifi into develop
Merge branch 'develop' of https://git-wip-us.apache.org/repos/asf/incubator-nifi into develop Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/e2227cda Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/e2227cda Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/e2227cda Branch: refs/heads/develop Commit: e2227cdafadabf9d8e57437a5b71bc01d21d3367 Parents: be50793 efc862e Author: Matt Gilman matt.c.gil...@gmail.com Authored: Thu Jun 4 13:10:27 2015 -0400 Committer: Matt Gilman matt.c.gil...@gmail.com Committed: Thu Jun 4 13:10:27 2015 -0400 -- .../nifi-hl7-query-language/pom.xml | 2 +- nifi/nifi-docs/pom.xml | 2 +- .../nifi-aws-bundle/nifi-aws-processors/pom.xml | 2 +- .../nifi-framework-cluster/pom.xml | 2 +- .../nifi-framework/nifi-framework-core/pom.xml | 2 +- .../repository/StandardProcessSession.java | 173 ++- .../nifi-web/nifi-web-content-viewer/pom.xml| 2 +- .../nifi-framework/nifi-web/nifi-web-ui/pom.xml | 2 +- .../nifi-hl7-bundle/nifi-hl7-processors/pom.xml | 2 +- .../nifi-mongodb-processors/pom.xml | 31 .../nifi/processors/mongodb/GetMongoTest.java | 4 +- .../nifi/processors/mongodb/PutMongoTest.java | 8 +- .../nifi-solr-processors/pom.xml| 2 +- .../nifi-standard-processors/pom.xml| 2 +- 14 files changed, 106 insertions(+), 130 deletions(-) --
[jira] [Commented] (NIFI-657) MongoDB unit tests fail
[ https://issues.apache.org/jira/browse/NIFI-657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573157#comment-14573157 ] ASF subversion and git services commented on NIFI-657: -- Commit fcadaa894b39318d11fa6dffa7846d87c9a50d8c in incubator-nifi's branch refs/heads/develop from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=incubator-nifi.git;h=fcadaa8 ] NIFI-657: Ignored Unit tests and removed maven plugin for embedding mongodb, as they fail in certain environments MongoDB unit tests fail --- Key: NIFI-657 URL: https://issues.apache.org/jira/browse/NIFI-657 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Mark Payne Fix For: 0.2.0 Attachments: 0001-NIFI-657-Ignored-Unit-tests-and-removed-maven-plugin.patch Trying to do a build of NiFi on a brand new VM running CentOS 7. Environment: [mark@nifi-build nifi]$ mvn -version Apache Maven 3.0.5 (Red Hat 3.0.5-16) Maven home: /usr/share/maven Java version: 1.7.0_79, vendor: Oracle Corporation Java home: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79-2.5.5.1.el7_1.x86_64/jre Default locale: en_US, platform encoding: UTF-8 OS name: linux, version: 3.10.0-229.el7.x86_64, arch: amd64, family: unix Output from mvn: --- T E S T S --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- embedmongo-maven-plugin:0.1.12:start (start) @ nifi-mongodb-processors --- Extract /home/mark/.embedmongo/linux/mongodb-linux-x86_64-2.2.1.tgz START [INFO] [INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ nifi-standard-processors --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 9 resources [INFO] Copying 3 resources Extract /home/mark/.embedmongo/linux/mongodb-linux-x86_64-2.2.1.tgz DONE [INFO] [INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ nifi-standard-processors --- Jun 04, 2015 10:29:37 AM de.flapdoodle.embed.process.runtime.AbstractProcess init SEVERE: failed to call onAfterProcessStart() java.io.IOException: Could not start process: EOF at de.flapdoodle.embed.mongo.AbstractMongoProcess.onAfterProcessStart(AbstractMongoProcess.java:75) at de.flapdoodle.embed.process.runtime.AbstractProcess.init(AbstractProcess.java:110) at de.flapdoodle.embed.mongo.AbstractMongoProcess.init(AbstractMongoProcess.java:52) at de.flapdoodle.embed.mongo.MongodProcess.init(MongodProcess.java:50) at de.flapdoodle.embed.mongo.MongodExecutable.start(MongodExecutable.java:44) at de.flapdoodle.embed.mongo.MongodExecutable.start(MongodExecutable.java:34) at de.flapdoodle.embed.process.runtime.Executable.start(Executable.java:99) at com.github.joelittlejohn.embedmongo.StartEmbeddedMongoMojo.execute(StartEmbeddedMongoMojo.java:271) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84) at org.apache.maven.lifecycle.internal.LifecycleThreadedBuilder$1.call(LifecycleThreadedBuilder.java:167) at org.apache.maven.lifecycle.internal.LifecycleThreadedBuilder$1.call(LifecycleThreadedBuilder.java:163) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Jun 04, 2015 10:29:37 AM MongodProcess ctor INFO: de.flapdoodle.embed.mongo.config.MongodConfigBuilder$ImmutableMongodConfig@701cce38 Jun 04, 2015 10:29:37 AM de.flapdoodle.embed.mongo.runtime.Mongod sendShutdown WARNING: sendShutdown nifi-build.beefy/127.0.0.1:27017 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at
[2/2] incubator-nifi git commit: Merge branch 'nifi-643' into develop
Merge branch 'nifi-643' into develop Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/8f55e35a Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/8f55e35a Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/8f55e35a Branch: refs/heads/develop Commit: 8f55e35a0a63e9d106c1270962e531ed87a1d987 Parents: e2227cd 88bfa66 Author: Aldrin Piri ald...@apache.org Authored: Thu Jun 4 13:19:31 2015 -0400 Committer: Aldrin Piri ald...@apache.org Committed: Thu Jun 4 13:19:31 2015 -0400 -- .../main/java/org/apache/nifi/processors/standard/SplitJson.java | 2 -- 1 file changed, 2 deletions(-) --
[jira] [Commented] (NIFI-632) Documentation Generation: Nar ClassLoader
[ https://issues.apache.org/jira/browse/NIFI-632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573301#comment-14573301 ] Dan Bress commented on NIFI-632: Got it. I'll check that out tonight. So you would advocate moving using of NarClosable/NarThreadContextClassLoader to [here|https://github.com/apache/incubator-nifi/blob/develop/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/init/ProcessorInitializer.java#L34], [here|https://github.com/apache/incubator-nifi/blob/develop/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/init/ControllerServiceInitializer.java#L36] and [here|https://github.com/apache/incubator-nifi/blob/develop/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-documentation/src/main/java/org/apache/nifi/documentation/init/ReportingTaskingInitializer.java#L35] instead of where I am doing it now? Is that what you mean by: bq. Additionally, you should only need to wrap the call into the extension Documentation Generation: Nar ClassLoader - Key: NIFI-632 URL: https://issues.apache.org/jira/browse/NIFI-632 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 0.1.0 Reporter: Matt Gilman Assignee: Dan Bress Priority: Minor Fix For: 0.2.0 When auto generating component documentation the NarThreadContextClassLoader needs to be set in case the component attempts to load anything off of its classpath. Without this, the documentation generation fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[1/2] incubator-nifi git commit: NIFI-645 adjusted to more reasonable defaults
Repository: incubator-nifi Updated Branches: refs/heads/develop 8f55e35a0 - 8201381c9 NIFI-645 adjusted to more reasonable defaults Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/0aa904c8 Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/0aa904c8 Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/0aa904c8 Branch: refs/heads/develop Commit: 0aa904c82a467fa25623d6439579533f6206aa99 Parents: e2227cd Author: joewitt joew...@apache.org Authored: Thu Jun 4 13:21:57 2015 -0400 Committer: joewitt joew...@apache.org Committed: Thu Jun 4 13:21:57 2015 -0400 -- nifi/nifi-assembly/pom.xml | 16 1 file changed, 8 insertions(+), 8 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/0aa904c8/nifi/nifi-assembly/pom.xml -- diff --git a/nifi/nifi-assembly/pom.xml b/nifi/nifi-assembly/pom.xml index 7d3a32b..02a16f9 100644 --- a/nifi/nifi-assembly/pom.xml +++ b/nifi/nifi-assembly/pom.xml @@ -243,11 +243,11 @@ language governing permissions and limitations under the License. -- nifi.content.claim.max.appendable.size10 MB/nifi.content.claim.max.appendable.size nifi.content.claim.max.flow.files100/nifi.content.claim.max.flow.files nifi.content.repository.directory.default./content_repository/nifi.content.repository.directory.default -nifi.content.repository.archive.max.retention.period / -nifi.content.repository.archive.max.usage.percentage / - nifi.content.repository.archive.enabledfalse/nifi.content.repository.archive.enabled +nifi.content.repository.archive.max.retention.period12 hours/nifi.content.repository.archive.max.retention.period + nifi.content.repository.archive.max.usage.percentage50%/nifi.content.repository.archive.max.usage.percentage + nifi.content.repository.archive.enabledtrue/nifi.content.repository.archive.enabled nifi.content.repository.always.syncfalse/nifi.content.repository.always.sync -nifi.content.viewer.url / + nifi.content.viewer.url/nifi-content-viewer//nifi.content.viewer.url nifi.restore.directory / nifi.ui.banner.text / @@ -267,11 +267,11 @@ language governing permissions and limitations under the License. -- nifi.provenance.repository.directory.default./provenance_repository/nifi.provenance.repository.directory.default nifi.provenance.repository.max.storage.time24 hours/nifi.provenance.repository.max.storage.time nifi.provenance.repository.max.storage.size1 GB/nifi.provenance.repository.max.storage.size -nifi.provenance.repository.rollover.time5 mins/nifi.provenance.repository.rollover.time +nifi.provenance.repository.rollover.time30 secs/nifi.provenance.repository.rollover.time nifi.provenance.repository.rollover.size100 MB/nifi.provenance.repository.rollover.size nifi.provenance.repository.query.threads2/nifi.provenance.repository.query.threads nifi.provenance.repository.compress.on.rollovertrue/nifi.provenance.repository.compress.on.rollover -nifi.provenance.repository.indexed.fieldsEventType, FlowFileUUID, Filename, ProcessorID/nifi.provenance.repository.indexed.fields +nifi.provenance.repository.indexed.fieldsEventType, FlowFileUUID, Filename, ProcessorID, Relationship/nifi.provenance.repository.indexed.fields nifi.provenance.repository.indexed.attributes / nifi.provenance.repository.index.shard.size500 MB/nifi.provenance.repository.index.shard.size nifi.provenance.repository.always.syncfalse/nifi.provenance.repository.always.sync @@ -282,8 +282,8 @@ language governing permissions and limitations under the License. -- !-- Component status repository properties -- nifi.components.status.repository.implementationorg.apache.nifi.controller.status.history.VolatileComponentStatusRepository/nifi.components.status.repository.implementation - nifi.components.status.repository.buffer.size288/nifi.components.status.repository.buffer.size -nifi.components.status.snapshot.frequency5 mins/nifi.components.status.snapshot.frequency + nifi.components.status.repository.buffer.size1440/nifi.components.status.repository.buffer.size +nifi.components.status.snapshot.frequency1 min/nifi.components.status.snapshot.frequency !-- nifi.properties: web properties -- nifi.web.war.directory./lib/nifi.web.war.directory
[2/2] incubator-nifi git commit: Merge branch 'develop' of https://git-wip-us.apache.org/repos/asf/incubator-nifi into develop
Merge branch 'develop' of https://git-wip-us.apache.org/repos/asf/incubator-nifi into develop Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/8201381c Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/8201381c Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/8201381c Branch: refs/heads/develop Commit: 8201381c9b89addf63c50c2e098cd97b51071c08 Parents: 0aa904c 8f55e35 Author: joewitt joew...@apache.org Authored: Thu Jun 4 13:49:38 2015 -0400 Committer: joewitt joew...@apache.org Committed: Thu Jun 4 13:49:38 2015 -0400 -- .../main/java/org/apache/nifi/processors/standard/SplitJson.java | 2 -- 1 file changed, 2 deletions(-) --
[jira] [Commented] (NIFI-645) Set content archive and content viewer on by default
[ https://issues.apache.org/jira/browse/NIFI-645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573236#comment-14573236 ] Matt Gilman commented on NIFI-645: -- New defaults are good. Lowering the minimum threshold to trigger the component status history and provenance is good. +1 Set content archive and content viewer on by default Key: NIFI-645 URL: https://issues.apache.org/jira/browse/NIFI-645 Project: Apache NiFi Issue Type: Improvement Components: Configuration Reporter: Joseph Witt Assignee: Joseph Witt Fix For: 0.2.0 Attachments: NIFI-645.patch The content viewer is now available in the build and it would be nice if it was preconfigured to be used by default. If the user doesn't want it available they can remove it (seems unlikely) or if they have an alternative they can point to it. The same is true of content archive. We should keep content around for a few hours by default (or some number of some time unit) and for X% of space. The core thing we're seeing is that users are wanting to use these features but not realizing they're not on by default. At this point in their maturity they should be on by default and only shut off when a user wants that. Also have the component status repository update be every 30 seconds by default and keep for 24 hours -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-602) Error on shutdown bin/nifi.sh stop
[ https://issues.apache.org/jira/browse/NIFI-602?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-602: Attachment: 0001-NIFI-602-Ensure-we-read-all-data-from-socket-after-s.patch Error on shutdown bin/nifi.sh stop -- Key: NIFI-602 URL: https://issues.apache.org/jira/browse/NIFI-602 Project: Apache NiFi Issue Type: Bug Components: Configuration Reporter: Joseph Witt Fix For: 0.2.0 Attachments: 0001-NIFI-602-Ensure-we-read-all-data-from-socket-after-s.patch Davis Joki: I downloaded the RC and built from source - and then ran a quick test in the UI. Everything worked great but saw this on shutdown: davis 11:09 PM ~/dev/nifi/nifi-0.1.0-incubating/nifi-assembly/target/nifi-0.1.0-incubating-bin/nifi-0.1.0-incubating$ ./bin/nifi.sh stop Java home: /usr/lib/jvm/java-7-openjdk-amd64 NiFi home: /home/davis/dev/nifi/nifi-0.1.0-incubating/nifi-assembly/target/nifi-0.1.0-incubating-bin/nifi-0.1.0-incubating Bootstrap Config File: /home/davis/dev/nifi/nifi-0.1.0-incubating/nifi-assembly/target/nifi-0.1.0-incubating-bin/nifi-0.1.0-incubating/conf/bootstrap.conf May 08, 2015 11:09:23 PM org.apache.nifi.bootstrap.RunNiFi stop SEVERE: Failed to send shutdown command to port 42,148 due to java.net.SocketException: Socket is closed Not sure if that's really an error, could be something with my environment. I restarted a couple of times and each time I get the SEVERE log line. The port appears to be open when it is running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-645) Set content archive and content viewer on by default
[ https://issues.apache.org/jira/browse/NIFI-645?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573239#comment-14573239 ] ASF subversion and git services commented on NIFI-645: -- Commit 0aa904c82a467fa25623d6439579533f6206aa99 in incubator-nifi's branch refs/heads/develop from [~joewitt] [ https://git-wip-us.apache.org/repos/asf?p=incubator-nifi.git;h=0aa904c ] NIFI-645 adjusted to more reasonable defaults Set content archive and content viewer on by default Key: NIFI-645 URL: https://issues.apache.org/jira/browse/NIFI-645 Project: Apache NiFi Issue Type: Improvement Components: Configuration Reporter: Joseph Witt Assignee: Joseph Witt Fix For: 0.2.0 Attachments: NIFI-645.patch The content viewer is now available in the build and it would be nice if it was preconfigured to be used by default. If the user doesn't want it available they can remove it (seems unlikely) or if they have an alternative they can point to it. The same is true of content archive. We should keep content around for a few hours by default (or some number of some time unit) and for X% of space. The core thing we're seeing is that users are wanting to use these features but not realizing they're not on by default. At this point in their maturity they should be on by default and only shut off when a user wants that. Also have the component status repository update be every 30 seconds by default and keep for 24 hours -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-632) Documentation Generation: Nar ClassLoader
[ https://issues.apache.org/jira/browse/NIFI-632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573358#comment-14573358 ] Matt Gilman commented on NIFI-632: -- That is what I was suggesting. It likely doesn't really matter here but I think its more clear going forward that the snippet is around the critical code. The critical code in this case meaning anything loaded from a sibling NAR. Setting the NarThreadContextClassLoader is necessary anytime that boundary is crossed. Documentation Generation: Nar ClassLoader - Key: NIFI-632 URL: https://issues.apache.org/jira/browse/NIFI-632 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 0.1.0 Reporter: Matt Gilman Assignee: Dan Bress Priority: Minor Fix For: 0.2.0 When auto generating component documentation the NarThreadContextClassLoader needs to be set in case the component attempts to load anything off of its classpath. Without this, the documentation generation fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-647) Duplicating provenance reporting via session creates unexpected behavior
[
https://issues.apache.org/jira/browse/NIFI-647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573149#comment-14573149
]
Aldrin Piri commented on NIFI-647:
--
Code looks good and confirmed expected behavior of how FORKs are handled. +1
Duplicating provenance reporting via session creates unexpected behavior
Key: NIFI-647
URL: https://issues.apache.org/jira/browse/NIFI-647
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 0.0.2, 0.1.0
Reporter: Aldrin Piri
Assignee: Mark Payne
Fix For: 0.2.0
Attachments:
0001-NIFI-647-When-FORK-Event-emitted-by-processor-Framew.patch
This is related to NIFI-643. While that processor bug was resolved, there are
issues with how this is managed in the framework.
The unexpected behavior is created via the following:
https://github.com/apache/incubator-nifi/blob/d29a2d688e437bae42c12115768cdb038b7406c5/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/SplitJson.java#L196
Summarily, a series of files (splits) are created from the parent file which
generates the associated FORK provenance events. The separate call to the
provenance reporter at Line 196 also requests a FORK for each of the
generated files. This manifests itself as being CREATE events for each of
the FORKed files and causes an error:
{code}
Found cycle in graph. This indicates that multiple events were registered
claiming to have generated the same FlowFile (UUID =
21b663c2-2406-49cb-b740-76dd64f54be8)
{code}
Ideally, such behavior would also be detected by the test framework.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-551) Improve error handling for ConvertJSONToAvro processor
[ https://issues.apache.org/jira/browse/NIFI-551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573302#comment-14573302 ] Joseph Witt commented on NIFI-551: -- Patch seems good. While attempting to apply it I noticed that it doesn't have any attribution info. Sent Ricky an email suggesting how he could fix it if he wants to (contributor guide explains it). Will wait a bit and see if he wants to correct it. Improve error handling for ConvertJSONToAvro processor -- Key: NIFI-551 URL: https://issues.apache.org/jira/browse/NIFI-551 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Ricky Saltzer Assignee: Joseph Witt Labels: patch, review Fix For: 0.2.0 Attachments: NIFI-551.1.patch, NIFI-551.2.patch Currently, if the ConvertJSONToAvro processor fails to process an individual record, a counter is incremented, but no alerts are produced. It would be better to notify the bulletin board that we've failed to process some records for a flowfile. Further, we should stream the records we fail to process down the failure relationship for further inspection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-551) Improve error handling for ConvertJSONToAvro processor
[ https://issues.apache.org/jira/browse/NIFI-551?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ricky Saltzer updated NIFI-551: --- Attachment: NIFI-551.3.patch Uploading formatted version of patch. Improve error handling for ConvertJSONToAvro processor -- Key: NIFI-551 URL: https://issues.apache.org/jira/browse/NIFI-551 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Ricky Saltzer Assignee: Joseph Witt Labels: patch, review Fix For: 0.2.0 Attachments: NIFI-551.1.patch, NIFI-551.2.patch, NIFI-551.3.patch Currently, if the ConvertJSONToAvro processor fails to process an individual record, a counter is incremented, but no alerts are produced. It would be better to notify the bulletin board that we've failed to process some records for a flowfile. Further, we should stream the records we fail to process down the failure relationship for further inspection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-632) Documentation Generation: Nar ClassLoader
[ https://issues.apache.org/jira/browse/NIFI-632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573253#comment-14573253 ] Matt Gilman commented on NIFI-632: -- [~danbress] Typically NarClosable is used. But it may not be accessible here without some refactoring. Additionally, you should only need to wrap the call into the extension. https://github.com/apache/incubator-nifi/blob/develop/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-core/src/main/java/org/apache/nifi/controller/FlowController.java#L824-L828 Documentation Generation: Nar ClassLoader - Key: NIFI-632 URL: https://issues.apache.org/jira/browse/NIFI-632 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 0.1.0 Reporter: Matt Gilman Assignee: Dan Bress Priority: Minor Fix For: 0.2.0 When auto generating component documentation the NarThreadContextClassLoader needs to be set in case the component attempts to load anything off of its classpath. Without this, the documentation generation fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-553) Remove experimental tag from some processors
[
https://issues.apache.org/jira/browse/NIFI-553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573484#comment-14573484
]
Joseph Witt commented on NIFI-553:
--
literally just removed tags. Pushing.
Remove experimental tag from some processors
Key: NIFI-553
URL: https://issues.apache.org/jira/browse/NIFI-553
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions
Affects Versions: 0.0.2
Reporter: Michael Moser
Assignee: Joseph Witt
Priority: Trivial
Fix For: 0.2.0
Several processors have an @Tag{experimental} and they do not seem to
actually be experimental. An experimental tag is odd to begin with.
Remove the experimental tag.
Here is a list of standard processors using the experimental tag.
Base64EncodeContent
DetectDuplicate
EncodeContent
EvaluateXQuery
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-553) Remove experimental tag from some processors
[
https://issues.apache.org/jira/browse/NIFI-553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573486#comment-14573486
]
ASF subversion and git services commented on NIFI-553:
--
Commit aeefe73f0b7bdbbe18ef38457fbc3521da3fba3d in incubator-nifi's branch
refs/heads/develop from [~joewitt]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-nifi.git;h=aeefe73 ]
NIFI-553
Remove experimental tag from some processors
Key: NIFI-553
URL: https://issues.apache.org/jira/browse/NIFI-553
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions
Affects Versions: 0.0.2
Reporter: Michael Moser
Assignee: Joseph Witt
Priority: Trivial
Fix For: 0.2.0
Several processors have an @Tag{experimental} and they do not seem to
actually be experimental. An experimental tag is odd to begin with.
Remove the experimental tag.
Here is a list of standard processors using the experimental tag.
Base64EncodeContent
DetectDuplicate
EncodeContent
EvaluateXQuery
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (NIFI-553) Remove experimental tag from some processors
[
https://issues.apache.org/jira/browse/NIFI-553?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Witt resolved NIFI-553.
--
Resolution: Fixed
Remove experimental tag from some processors
Key: NIFI-553
URL: https://issues.apache.org/jira/browse/NIFI-553
Project: Apache NiFi
Issue Type: Improvement
Components: Extensions
Affects Versions: 0.0.2
Reporter: Michael Moser
Assignee: Joseph Witt
Priority: Trivial
Fix For: 0.2.0
Several processors have an @Tag{experimental} and they do not seem to
actually be experimental. An experimental tag is odd to begin with.
Remove the experimental tag.
Here is a list of standard processors using the experimental tag.
Base64EncodeContent
DetectDuplicate
EncodeContent
EvaluateXQuery
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-650) TestExecuteStreamCommand tests failing on Ubuntu 12.04.5 LTS
[
https://issues.apache.org/jira/browse/NIFI-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573090#comment-14573090
]
Aldrin Piri commented on NIFI-650:
--
Ubuntu LTS has very low open file/proc limits by default. Adjusting these to
be inline with those in the Administrator guide allowed me to build
successfully. Provided this guidance to the associated dev list thread.
{quote}
All,
Looks like the reported failures were a bit of a red herring, but provides some
interesting points of consideration for our Quickstart guide.
Pat,
After some inspection, it seems that the default settings for both open files
and processes in 12.04 LTS is quite low which lead to the test failures. It
was only while watching over the tests running did I see the notification about
too many open files. I would speculate this is the problem you are
experiencing.
I was able to resolve the issue by applying the best practices laid out in the
administration guide [1] to my VM running the version of Ubuntu in question.
Please give those settings a go when you have the opportunity and let us know
if that resolves your build issues.
[1]
http://nifi.incubator.apache.org/docs/nifi-docs/html/administration-guide.html#configuration-best-practices
{quote}
TestExecuteStreamCommand tests failing on Ubuntu 12.04.5 LTS
-
Key: NIFI-650
URL: https://issues.apache.org/jira/browse/NIFI-650
Project: Apache NiFi
Issue Type: Bug
Components: Extensions
Affects Versions: 0.1.0
Reporter: Aldrin Piri
Assignee: Aldrin Piri
Reported errors on the mailing list:
{code}
...
Running org.apache.nifi.processors.standard.TestExecuteStreamCommand
Tests run: 5, Failures: 4, Errors: 0, Skipped: 1, Time elapsed: 1.144 sec
FAILURE! - in org.apache.nifi.processors.standard.TestE
xecuteStreamCommand
testExecuteIngestAndUpdateWithWorkingDir(org.apache.nifi.processors.standard.TestExecuteStreamCommand)
Time elapsed: 0.112 sec
FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.nifi.processors.standard.TestExecuteStreamCommand.testExecuteIngestAndUpdateWithWorkingDir(TestExecuteStreamCommand.java:156)
testExecuteJarWithBadPath(org.apache.nifi.processors.standard.TestExecuteStreamCommand)
Time elapsed: 0.006 sec FAILURE!
org.junit.ComparisonFailure: expected:[Error: Unable to access jarfile]
but was:[Unable to access jarfile /home/]
at org.junit.Assert.assertEquals(Assert.java:115)
at org.junit.Assert.assertEquals(Assert.java:144)
at
org.apache.nifi.processors.standard.TestExecuteStreamCommand.testExecuteJarWithBadPath(TestExecuteStreamCommand.java:103)
testExecuteJar(org.apache.nifi.processors.standard.TestExecuteStreamCommand)
Time elapsed: 0.1 sec FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.nifi.processors.standard.TestExecuteStreamCommand.testExecuteJar(TestExecuteStreamCommand.java:73)
testExecuteIngestAndUpdate(org.apache.nifi.processors.standard.TestExecuteStreamCommand)
Time elapsed: 0.925 sec FAILURE!
java.lang.AssertionError: null
at org.junit.Assert.fail(Assert.java:86)
at org.junit.Assert.assertTrue(Assert.java:41)
at org.junit.Assert.assertTrue(Assert.java:52)
at
org.apache.nifi.processors.standard.TestExecuteStreamCommand.testExecuteIngestAndUpdate(TestExecuteStreamCommand.java:134)
...
[INFO] BUILD FAILURE
[INFO]
[INFO] Total time: 7:46.419s
[INFO] Finished at: Wed Jun 03 14:02:07 EDT 2015
[INFO] Final Memory: 141M/1477M
[INFO]
[ERROR] Failed to execute goal
org.apache.maven.plugins:maven-surefire-plugin:2.18:test (default-test) on
project nifi-standard-processors: There are test failures.
{code}
Full thread is available at:
http://apache-nifi-incubating-developer-list.39713.n7.nabble.com/mvn-fail-td1716.html
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
incubator-nifi git commit: NIFI-659 updated all instances of child overrides to combine their excludes and parents
Repository: incubator-nifi Updated Branches: refs/heads/develop 07628f76f - fb75a4ba6 NIFI-659 updated all instances of child overrides to combine their excludes and parents Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/fb75a4ba Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/fb75a4ba Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/fb75a4ba Branch: refs/heads/develop Commit: fb75a4ba692ec834e470fc89de24bdd483439796 Parents: 07628f7 Author: joewitt joew...@apache.org Authored: Thu Jun 4 12:11:35 2015 -0400 Committer: joewitt joew...@apache.org Committed: Thu Jun 4 12:11:35 2015 -0400 -- nifi/nifi-commons/nifi-hl7-query-language/pom.xml | 2 +- nifi/nifi-docs/pom.xml | 2 +- nifi/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/pom.xml | 2 +- .../nifi-framework/nifi-framework-cluster/pom.xml | 2 +- .../nifi-framework/nifi-framework-core/pom.xml | 2 +- .../nifi-framework/nifi-web/nifi-web-content-viewer/pom.xml| 2 +- .../nifi-framework/nifi-web/nifi-web-ui/pom.xml| 2 +- nifi/nifi-nar-bundles/nifi-hl7-bundle/nifi-hl7-processors/pom.xml | 2 +- .../nifi-nar-bundles/nifi-solr-bundle/nifi-solr-processors/pom.xml | 2 +- .../nifi-standard-bundle/nifi-standard-processors/pom.xml | 2 +- 10 files changed, 10 insertions(+), 10 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/fb75a4ba/nifi/nifi-commons/nifi-hl7-query-language/pom.xml -- diff --git a/nifi/nifi-commons/nifi-hl7-query-language/pom.xml b/nifi/nifi-commons/nifi-hl7-query-language/pom.xml index eb46841..b6ead09 100644 --- a/nifi/nifi-commons/nifi-hl7-query-language/pom.xml +++ b/nifi/nifi-commons/nifi-hl7-query-language/pom.xml @@ -48,7 +48,7 @@ groupIdorg.apache.rat/groupId artifactIdapache-rat-plugin/artifactId configuration -excludes +excludes combine.children=append excludesrc/test/resources/hypoglycemia/exclude excludesrc/test/resources/hyperglycemia/exclude /excludes http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/fb75a4ba/nifi/nifi-docs/pom.xml -- diff --git a/nifi/nifi-docs/pom.xml b/nifi/nifi-docs/pom.xml index 1494745..c3d6735 100644 --- a/nifi/nifi-docs/pom.xml +++ b/nifi/nifi-docs/pom.xml @@ -107,7 +107,7 @@ groupIdorg.apache.rat/groupId artifactIdapache-rat-plugin/artifactId configuration -excludes +excludes combine.children=append excludesrc/main/asciidoc/asciidoc-mod.css/exclude !-- MIT license confirmed. Excluding due to parse error-- /excludes /configuration http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/fb75a4ba/nifi/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/pom.xml -- diff --git a/nifi/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/pom.xml b/nifi/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/pom.xml index cada04e..c69c3c8 100644 --- a/nifi/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/pom.xml +++ b/nifi/nifi-nar-bundles/nifi-aws-bundle/nifi-aws-processors/pom.xml @@ -61,7 +61,7 @@ groupIdorg.apache.rat/groupId artifactIdapache-rat-plugin/artifactId configuration -excludes +excludes combine.children=append excludesrc/test/resources/hello.txt/exclude /excludes /configuration http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/fb75a4ba/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/pom.xml -- diff --git a/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/pom.xml b/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/pom.xml index 4cb2196..82712e2 100644 --- a/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/pom.xml +++ b/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-framework-cluster/pom.xml @@ -137,7 +137,7 @@ groupIdorg.apache.rat/groupId artifactIdapache-rat-plugin/artifactId configuration -
[jira] [Commented] (NIFI-657) MongoDB unit tests fail
[ https://issues.apache.org/jira/browse/NIFI-657?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573145#comment-14573145 ] Joseph Witt commented on NIFI-657: -- +1 Very welcome adjustment. That plugin caused a lot of strange behavior locally and this is not the best place for integration level tests anyway. MongoDB unit tests fail --- Key: NIFI-657 URL: https://issues.apache.org/jira/browse/NIFI-657 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Mark Payne Fix For: 0.2.0 Attachments: 0001-NIFI-657-Ignored-Unit-tests-and-removed-maven-plugin.patch Trying to do a build of NiFi on a brand new VM running CentOS 7. Environment: [mark@nifi-build nifi]$ mvn -version Apache Maven 3.0.5 (Red Hat 3.0.5-16) Maven home: /usr/share/maven Java version: 1.7.0_79, vendor: Oracle Corporation Java home: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79-2.5.5.1.el7_1.x86_64/jre Default locale: en_US, platform encoding: UTF-8 OS name: linux, version: 3.10.0-229.el7.x86_64, arch: amd64, family: unix Output from mvn: --- T E S T S --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- embedmongo-maven-plugin:0.1.12:start (start) @ nifi-mongodb-processors --- Extract /home/mark/.embedmongo/linux/mongodb-linux-x86_64-2.2.1.tgz START [INFO] [INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ nifi-standard-processors --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 9 resources [INFO] Copying 3 resources Extract /home/mark/.embedmongo/linux/mongodb-linux-x86_64-2.2.1.tgz DONE [INFO] [INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ nifi-standard-processors --- Jun 04, 2015 10:29:37 AM de.flapdoodle.embed.process.runtime.AbstractProcess init SEVERE: failed to call onAfterProcessStart() java.io.IOException: Could not start process: EOF at de.flapdoodle.embed.mongo.AbstractMongoProcess.onAfterProcessStart(AbstractMongoProcess.java:75) at de.flapdoodle.embed.process.runtime.AbstractProcess.init(AbstractProcess.java:110) at de.flapdoodle.embed.mongo.AbstractMongoProcess.init(AbstractMongoProcess.java:52) at de.flapdoodle.embed.mongo.MongodProcess.init(MongodProcess.java:50) at de.flapdoodle.embed.mongo.MongodExecutable.start(MongodExecutable.java:44) at de.flapdoodle.embed.mongo.MongodExecutable.start(MongodExecutable.java:34) at de.flapdoodle.embed.process.runtime.Executable.start(Executable.java:99) at com.github.joelittlejohn.embedmongo.StartEmbeddedMongoMojo.execute(StartEmbeddedMongoMojo.java:271) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84) at org.apache.maven.lifecycle.internal.LifecycleThreadedBuilder$1.call(LifecycleThreadedBuilder.java:167) at org.apache.maven.lifecycle.internal.LifecycleThreadedBuilder$1.call(LifecycleThreadedBuilder.java:163) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Jun 04, 2015 10:29:37 AM MongodProcess ctor INFO: de.flapdoodle.embed.mongo.config.MongodConfigBuilder$ImmutableMongodConfig@701cce38 Jun 04, 2015 10:29:37 AM de.flapdoodle.embed.mongo.runtime.Mongod sendShutdown WARNING: sendShutdown nifi-build.beefy/127.0.0.1:27017 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at de.flapdoodle.embed.mongo.runtime.Mongod.sendShutdown(Mongod.java:83) at
[jira] [Created] (NIFI-658) Create a Getting Started/Try it now documentation path
Aldrin Piri created NIFI-658: Summary: Create a Getting Started/Try it now documentation path Key: NIFI-658 URL: https://issues.apache.org/jira/browse/NIFI-658 Project: Apache NiFi Issue Type: Improvement Components: Documentation Website Reporter: Aldrin Piri I think we need to call out how to get NiFI running. Right now, the only path that is immediately obvious is to do the development path which is out of the realm of many people who may be nothing more than just users and/or want to sample the application before delving in further. Otherwise, your path is to go through nifi docs and find your way to the admin guide just to run the latest release. We are making a more arduous path than necessary and subjecting users to a path that may have no relevance to their intended usage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-659) rat-plugin needs to be configured to exclude child and parent items
[
https://issues.apache.org/jira/browse/NIFI-659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Witt updated NIFI-659:
-
Fix Version/s: 0.2.0
rat-plugin needs to be configured to exclude child and parent items
---
Key: NIFI-659
URL: https://issues.apache.org/jira/browse/NIFI-659
Project: Apache NiFi
Issue Type: Task
Components: Tools and Build
Reporter: Joseph Witt
Assignee: Joseph Witt
Fix For: 0.2.0
Attachments: NIFI-659.patch
Presently when doing the proper -Pcontrib-check on patches and such we can
run into issues with modules that implement their own excludes above and
beyond what the nifi-parent excludes. This is due to how the configuration
override/merge/append mechanism works in Maven 3. What we really want it to
do is 'append' to the child's list whatever was in the parents list. So for
example:
{code:xml}
plugin
groupIdorg.apache.rat/groupId
artifactIdapache-rat-plugin/artifactId
configuration
excludes
excludesrc/test/resources/conf/0bytes.xml/exclude
excludesrc/test/resources/conf/termination-only.xml/exclude
excludesrc/test/resources/hello.txt/exclude
excludesrc/test/resources/old-swap-file.swap/exclude
/excludes
/configuration
/plugin
{code}
would work as we really want it to if it were
{code:xml}
plugin
groupIdorg.apache.rat/groupId
artifactIdapache-rat-plugin/artifactId
configuration
excludes combine.children=append
excludesrc/test/resources/conf/0bytes.xml/exclude
excludesrc/test/resources/conf/termination-only.xml/exclude
excludesrc/test/resources/hello.txt/exclude
excludesrc/test/resources/old-swap-file.swap/exclude
/excludes
/configuration
/plugin
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-659) rat-plugin needs to be configured to exclude child and parent items
[
https://issues.apache.org/jira/browse/NIFI-659?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Witt updated NIFI-659:
-
Attachment: NIFI-659.patch
rat-plugin needs to be configured to exclude child and parent items
---
Key: NIFI-659
URL: https://issues.apache.org/jira/browse/NIFI-659
Project: Apache NiFi
Issue Type: Task
Components: Tools and Build
Reporter: Joseph Witt
Assignee: Joseph Witt
Fix For: 0.2.0
Attachments: NIFI-659.patch
Presently when doing the proper -Pcontrib-check on patches and such we can
run into issues with modules that implement their own excludes above and
beyond what the nifi-parent excludes. This is due to how the configuration
override/merge/append mechanism works in Maven 3. What we really want it to
do is 'append' to the child's list whatever was in the parents list. So for
example:
{code:xml}
plugin
groupIdorg.apache.rat/groupId
artifactIdapache-rat-plugin/artifactId
configuration
excludes
excludesrc/test/resources/conf/0bytes.xml/exclude
excludesrc/test/resources/conf/termination-only.xml/exclude
excludesrc/test/resources/hello.txt/exclude
excludesrc/test/resources/old-swap-file.swap/exclude
/excludes
/configuration
/plugin
{code}
would work as we really want it to if it were
{code:xml}
plugin
groupIdorg.apache.rat/groupId
artifactIdapache-rat-plugin/artifactId
configuration
excludes combine.children=append
excludesrc/test/resources/conf/0bytes.xml/exclude
excludesrc/test/resources/conf/termination-only.xml/exclude
excludesrc/test/resources/hello.txt/exclude
excludesrc/test/resources/old-swap-file.swap/exclude
/excludes
/configuration
/plugin
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-659) rat-plugin needs to be configured to exclude child and parent items
[
https://issues.apache.org/jira/browse/NIFI-659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573088#comment-14573088
]
ASF subversion and git services commented on NIFI-659:
--
Commit fb75a4ba692ec834e470fc89de24bdd483439796 in incubator-nifi's branch
refs/heads/develop from [~joewitt]
[ https://git-wip-us.apache.org/repos/asf?p=incubator-nifi.git;h=fb75a4b ]
NIFI-659 updated all instances of child overrides to combine their excludes and
parents
rat-plugin needs to be configured to exclude child and parent items
---
Key: NIFI-659
URL: https://issues.apache.org/jira/browse/NIFI-659
Project: Apache NiFi
Issue Type: Task
Components: Tools and Build
Reporter: Joseph Witt
Assignee: Joseph Witt
Fix For: 0.2.0
Attachments: NIFI-659.patch
Presently when doing the proper -Pcontrib-check on patches and such we can
run into issues with modules that implement their own excludes above and
beyond what the nifi-parent excludes. This is due to how the configuration
override/merge/append mechanism works in Maven 3. What we really want it to
do is 'append' to the child's list whatever was in the parents list. So for
example:
{code:xml}
plugin
groupIdorg.apache.rat/groupId
artifactIdapache-rat-plugin/artifactId
configuration
excludes
excludesrc/test/resources/conf/0bytes.xml/exclude
excludesrc/test/resources/conf/termination-only.xml/exclude
excludesrc/test/resources/hello.txt/exclude
excludesrc/test/resources/old-swap-file.swap/exclude
/excludes
/configuration
/plugin
{code}
would work as we really want it to if it were
{code:xml}
plugin
groupIdorg.apache.rat/groupId
artifactIdapache-rat-plugin/artifactId
configuration
excludes combine.children=append
excludesrc/test/resources/conf/0bytes.xml/exclude
excludesrc/test/resources/conf/termination-only.xml/exclude
excludesrc/test/resources/hello.txt/exclude
excludesrc/test/resources/old-swap-file.swap/exclude
/excludes
/configuration
/plugin
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-644) Data Viewer: Use mime.type when known
[ https://issues.apache.org/jira/browse/NIFI-644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-644: - Attachment: 0001-NIFI-644.patch Data Viewer: Use mime.type when known - Key: NIFI-644 URL: https://issues.apache.org/jira/browse/NIFI-644 Project: Apache NiFi Issue Type: Improvement Components: Core UI Reporter: Matt Gilman Assignee: Matt Gilman Fix For: 0.2.0 Attachments: 0001-NIFI-644.patch If the mime.type is known, use it. If it is not known, then revert to the standard content detection process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-643) SplitJson creates a cycle of provenance events
[
https://issues.apache.org/jira/browse/NIFI-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573130#comment-14573130
]
Mark Payne commented on NIFI-643:
-
So to be clear, what I am suggesting is really a +0 I guess on the idea. I
can go either way.
But if you'd prefer to have the patch in, +1 on the code. All looks good to me.
SplitJson creates a cycle of provenance events
---
Key: NIFI-643
URL: https://issues.apache.org/jira/browse/NIFI-643
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 0.0.2, 0.1.0
Reporter: Aldrin Piri
Assignee: Aldrin Piri
Fix For: 0.2.0
Attachments:
0001-Removing-duplicate-provenance-FORK-in-SplitJson.patch
Duplicate reporting of the fork event in SplitJson causes an error due to a
cycle being created.
{code}
Found cycle in graph. This indicates that multiple events were registered
claiming to have generated the same FlowFile (UUID =
21b663c2-2406-49cb-b740-76dd64f54be8)
{code}
This additionally treats each generated split as a CREATE event.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-658) Create a Getting Started/Try it now documentation path
[ https://issues.apache.org/jira/browse/NIFI-658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572997#comment-14572997 ] Aldrin Piri commented on NIFI-658: -- Additionally, this guide should then provide a choose your own adventure of links and explanations on how people can get additional information pointing to the existing dev/admin guides. Create a Getting Started/Try it now documentation path -- Key: NIFI-658 URL: https://issues.apache.org/jira/browse/NIFI-658 Project: Apache NiFi Issue Type: Improvement Components: Documentation Website Reporter: Aldrin Piri I think we need to call out how to get NiFI running. Right now, the only path that is immediately obvious is to do the development path which is out of the realm of many people who may be nothing more than just users and/or want to sample the application before delving in further. Otherwise, your path is to go through nifi docs and find your way to the admin guide just to run the latest release. We are making a more arduous path than necessary and subjecting users to a path that may have no relevance to their intended usage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-659) rat-plugin needs to be configured to exclude child and parent items
[
https://issues.apache.org/jira/browse/NIFI-659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573053#comment-14573053
]
Joseph Witt commented on NIFI-659:
--
We could push this to nifi-parent later if it becomes worthwhile. Doing so
would force all configuration instances of excludes within this plugin to be
'combine.childen=append' which is likely what we always want but doing so
would prevent children from doing otherwise. So it's a little evil to do. For
now I think this is the right answer (doing it on the children themselves so
they can specify the desired behavior).
rat-plugin needs to be configured to exclude child and parent items
---
Key: NIFI-659
URL: https://issues.apache.org/jira/browse/NIFI-659
Project: Apache NiFi
Issue Type: Task
Components: Tools and Build
Reporter: Joseph Witt
Assignee: Joseph Witt
Presently when doing the proper -Pcontrib-check on patches and such we can
run into issues with modules that implement their own excludes above and
beyond what the nifi-parent excludes. This is due to how the configuration
override/merge/append mechanism works in Maven 3. What we really want it to
do is 'append' to the child's list whatever was in the parents list. So for
example:
{code:xml}
plugin
groupIdorg.apache.rat/groupId
artifactIdapache-rat-plugin/artifactId
configuration
excludes
excludesrc/test/resources/conf/0bytes.xml/exclude
excludesrc/test/resources/conf/termination-only.xml/exclude
excludesrc/test/resources/hello.txt/exclude
excludesrc/test/resources/old-swap-file.swap/exclude
/excludes
/configuration
/plugin
{code}
would work as we really want it to if it were
{code:xml}
plugin
groupIdorg.apache.rat/groupId
artifactIdapache-rat-plugin/artifactId
configuration
excludes combine.children=append
excludesrc/test/resources/conf/0bytes.xml/exclude
excludesrc/test/resources/conf/termination-only.xml/exclude
excludesrc/test/resources/hello.txt/exclude
excludesrc/test/resources/old-swap-file.swap/exclude
/excludes
/configuration
/plugin
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-657) MongoDB unit tests fail
[ https://issues.apache.org/jira/browse/NIFI-657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Mark Payne updated NIFI-657: Attachment: 0001-NIFI-657-Ignored-Unit-tests-and-removed-maven-plugin.patch MongoDB unit tests fail --- Key: NIFI-657 URL: https://issues.apache.org/jira/browse/NIFI-657 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Mark Payne Fix For: 0.2.0 Attachments: 0001-NIFI-657-Ignored-Unit-tests-and-removed-maven-plugin.patch Trying to do a build of NiFi on a brand new VM running CentOS 7. Environment: [mark@nifi-build nifi]$ mvn -version Apache Maven 3.0.5 (Red Hat 3.0.5-16) Maven home: /usr/share/maven Java version: 1.7.0_79, vendor: Oracle Corporation Java home: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79-2.5.5.1.el7_1.x86_64/jre Default locale: en_US, platform encoding: UTF-8 OS name: linux, version: 3.10.0-229.el7.x86_64, arch: amd64, family: unix Output from mvn: --- T E S T S --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- embedmongo-maven-plugin:0.1.12:start (start) @ nifi-mongodb-processors --- Extract /home/mark/.embedmongo/linux/mongodb-linux-x86_64-2.2.1.tgz START [INFO] [INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ nifi-standard-processors --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 9 resources [INFO] Copying 3 resources Extract /home/mark/.embedmongo/linux/mongodb-linux-x86_64-2.2.1.tgz DONE [INFO] [INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ nifi-standard-processors --- Jun 04, 2015 10:29:37 AM de.flapdoodle.embed.process.runtime.AbstractProcess init SEVERE: failed to call onAfterProcessStart() java.io.IOException: Could not start process: EOF at de.flapdoodle.embed.mongo.AbstractMongoProcess.onAfterProcessStart(AbstractMongoProcess.java:75) at de.flapdoodle.embed.process.runtime.AbstractProcess.init(AbstractProcess.java:110) at de.flapdoodle.embed.mongo.AbstractMongoProcess.init(AbstractMongoProcess.java:52) at de.flapdoodle.embed.mongo.MongodProcess.init(MongodProcess.java:50) at de.flapdoodle.embed.mongo.MongodExecutable.start(MongodExecutable.java:44) at de.flapdoodle.embed.mongo.MongodExecutable.start(MongodExecutable.java:34) at de.flapdoodle.embed.process.runtime.Executable.start(Executable.java:99) at com.github.joelittlejohn.embedmongo.StartEmbeddedMongoMojo.execute(StartEmbeddedMongoMojo.java:271) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84) at org.apache.maven.lifecycle.internal.LifecycleThreadedBuilder$1.call(LifecycleThreadedBuilder.java:167) at org.apache.maven.lifecycle.internal.LifecycleThreadedBuilder$1.call(LifecycleThreadedBuilder.java:163) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Jun 04, 2015 10:29:37 AM MongodProcess ctor INFO: de.flapdoodle.embed.mongo.config.MongodConfigBuilder$ImmutableMongodConfig@701cce38 Jun 04, 2015 10:29:37 AM de.flapdoodle.embed.mongo.runtime.Mongod sendShutdown WARNING: sendShutdown nifi-build.beefy/127.0.0.1:27017 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at de.flapdoodle.embed.mongo.runtime.Mongod.sendShutdown(Mongod.java:83) at de.flapdoodle.embed.mongo.AbstractMongoProcess.sendStopToMongoInstance(AbstractMongoProcess.java:125) at
[jira] [Commented] (NIFI-659) rat-plugin needs to be configured to exclude child and parent items
[
https://issues.apache.org/jira/browse/NIFI-659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573087#comment-14573087
]
Matt Gilman commented on NIFI-659:
--
+1 parent configuration successfully inherited
rat-plugin needs to be configured to exclude child and parent items
---
Key: NIFI-659
URL: https://issues.apache.org/jira/browse/NIFI-659
Project: Apache NiFi
Issue Type: Task
Components: Tools and Build
Reporter: Joseph Witt
Assignee: Joseph Witt
Fix For: 0.2.0
Attachments: NIFI-659.patch
Presently when doing the proper -Pcontrib-check on patches and such we can
run into issues with modules that implement their own excludes above and
beyond what the nifi-parent excludes. This is due to how the configuration
override/merge/append mechanism works in Maven 3. What we really want it to
do is 'append' to the child's list whatever was in the parents list. So for
example:
{code:xml}
plugin
groupIdorg.apache.rat/groupId
artifactIdapache-rat-plugin/artifactId
configuration
excludes
excludesrc/test/resources/conf/0bytes.xml/exclude
excludesrc/test/resources/conf/termination-only.xml/exclude
excludesrc/test/resources/hello.txt/exclude
excludesrc/test/resources/old-swap-file.swap/exclude
/excludes
/configuration
/plugin
{code}
would work as we really want it to if it were
{code:xml}
plugin
groupIdorg.apache.rat/groupId
artifactIdapache-rat-plugin/artifactId
configuration
excludes combine.children=append
excludesrc/test/resources/conf/0bytes.xml/exclude
excludesrc/test/resources/conf/termination-only.xml/exclude
excludesrc/test/resources/hello.txt/exclude
excludesrc/test/resources/old-swap-file.swap/exclude
/excludes
/configuration
/plugin
{code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-643) SplitJson creates a cycle of provenance events
[
https://issues.apache.org/jira/browse/NIFI-643?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573104#comment-14573104
]
Mark Payne commented on NIFI-643:
-
[~aldrin] I would actually opt to leave the code as-is. This really is a bug in
the framework (NIFI-647, which you linked to). The call to the Provenance
Reporter is not necessary but should be okay, and I like the explicit behavior
of it.
SplitJson creates a cycle of provenance events
---
Key: NIFI-643
URL: https://issues.apache.org/jira/browse/NIFI-643
Project: Apache NiFi
Issue Type: Bug
Affects Versions: 0.0.2, 0.1.0
Reporter: Aldrin Piri
Assignee: Aldrin Piri
Fix For: 0.2.0
Attachments:
0001-Removing-duplicate-provenance-FORK-in-SplitJson.patch
Duplicate reporting of the fork event in SplitJson causes an error due to a
cycle being created.
{code}
Found cycle in graph. This indicates that multiple events were registered
claiming to have generated the same FlowFile (UUID =
21b663c2-2406-49cb-b740-76dd64f54be8)
{code}
This additionally treats each generated split as a CREATE event.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-644) Data Viewer: Use mime.type when known
[ https://issues.apache.org/jira/browse/NIFI-644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573142#comment-14573142 ] Joseph Witt commented on NIFI-644: -- +1. And tested on a live flow where filename was changed to have a nonsense suffix and it still rendered the proper type since the mime.type attribute was present. Data Viewer: Use mime.type when known - Key: NIFI-644 URL: https://issues.apache.org/jira/browse/NIFI-644 Project: Apache NiFi Issue Type: Improvement Components: Core UI Reporter: Matt Gilman Assignee: Matt Gilman Fix For: 0.2.0 Attachments: 0001-NIFI-644.patch If the mime.type is known, use it. If it is not known, then revert to the standard content detection process. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-658) Create a Getting Started/Try it now documentation path
[ https://issues.apache.org/jira/browse/NIFI-658?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573002#comment-14573002 ] Joseph Witt commented on NIFI-658: -- 100% on board Aldrin! Our getting started page should morph to support two personas 1) I am someone who wants to pull down a convenience binary, unpack this thing, fire it up, and see what it can do. Help me quickly! 2) I am a developer. I am really curious about the code. I want to build it. Be able to tweak it. Show me what i need to do to prep my environment and do a full build. Right now we are only addressing the needs of the second persona. Thanks Joe Create a Getting Started/Try it now documentation path -- Key: NIFI-658 URL: https://issues.apache.org/jira/browse/NIFI-658 Project: Apache NiFi Issue Type: Improvement Components: Documentation Website Reporter: Aldrin Piri I think we need to call out how to get NiFI running. Right now, the only path that is immediately obvious is to do the development path which is out of the realm of many people who may be nothing more than just users and/or want to sample the application before delving in further. Otherwise, your path is to go through nifi docs and find your way to the admin guide just to run the latest release. We are making a more arduous path than necessary and subjecting users to a path that may have no relevance to their intended usage. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-654) Update dependency versions
[ https://issues.apache.org/jira/browse/NIFI-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573004#comment-14573004 ] Matt Gilman commented on NIFI-654: -- +1 build successful and all appears good Update dependency versions --- Key: NIFI-654 URL: https://issues.apache.org/jira/browse/NIFI-654 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: Joseph Witt Assignee: Joseph Witt Fix For: 0.2.0 Attachments: NIFI-654.patch The following output comes from running mvn versions:update-dependency-versions [INFO] --- versions-maven-plugin:2.2:display-dependency-updates (default-cli) @ nifi-spark-receiver --- [INFO] The following dependencies in Dependency Management have newer versions: [INFO] ch.qos.logback:logback-classic 1.1.2 - 1.1.3 [INFO] com.jcraft:jsch . 0.1.51 - 0.1.52 [INFO] javax.jms:javax.jms-api . 2.0 - 2.0.1 [INFO] joda-time:joda-time ... 2.7 - 2.8 [INFO] net.sf.saxon:Saxon-HE . 9.6.0-4 - 9.6.0-5 [INFO] org.apache.activemq:activemq-client . 5.10.0 - 5.11.1 [INFO] org.apache.avro:avro .. 1.7.6 - 1.7.7 [INFO] org.apache.commons:commons-lang3 3.3.2 - 3.4 [INFO] org.apache.commons:commons-pool2 2.3 - 2.4.1 [INFO] org.apache.hadoop:hadoop-annotations .. 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-client ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-common ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-hdfs . 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-mapreduce-client-core 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-yarn-api . 2.6.0 - 2.7.0 [INFO] org.apache.httpcomponents:httpclient .. 4.3.6 - 4.4.1 [INFO] org.apache.lucene:lucene-analyzers-common 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-core 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-queryparser . 4.10.3 - 5.1.0 [INFO] org.aspectj:aspectjweaver . 1.8.4 - 1.8.5 [INFO] org.slf4j:jcl-over-slf4j 1.7.10 - 1.7.12 [INFO] org.slf4j:jul-to-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:log4j-over-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-api . 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-simple .. 1.7.10 - 1.7.12 [INFO] org.springframework:spring-aop 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-beans .. 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context-support ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-core ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-expression ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-jdbc ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-tx . 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-web 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework.security:spring-security-acl ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-config ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-core ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-web ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] [INFO] artifact org.apache.spark:spark-streaming_2.10: checking for updates from central [INFO] The following dependencies in Dependencies have newer versions: [INFO] org.apache.spark:spark-streaming_2.10 . 1.2.0 - 1.3.1 [INFO] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-654) Update dependency versions
[ https://issues.apache.org/jira/browse/NIFI-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573030#comment-14573030 ] ASF subversion and git services commented on NIFI-654: -- Commit 07628f76fedfe8596c62fbfb2edd159ddf051665 in incubator-nifi's branch refs/heads/develop from [~joewitt] [ https://git-wip-us.apache.org/repos/asf?p=incubator-nifi.git;h=07628f7 ] NIFI-654 updated all non controversial deps. Tested locally and on a remote system with full 2-way SSL Update dependency versions --- Key: NIFI-654 URL: https://issues.apache.org/jira/browse/NIFI-654 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: Joseph Witt Assignee: Joseph Witt Fix For: 0.2.0 Attachments: NIFI-654.patch The following output comes from running mvn versions:update-dependency-versions [INFO] --- versions-maven-plugin:2.2:display-dependency-updates (default-cli) @ nifi-spark-receiver --- [INFO] The following dependencies in Dependency Management have newer versions: [INFO] ch.qos.logback:logback-classic 1.1.2 - 1.1.3 [INFO] com.jcraft:jsch . 0.1.51 - 0.1.52 [INFO] javax.jms:javax.jms-api . 2.0 - 2.0.1 [INFO] joda-time:joda-time ... 2.7 - 2.8 [INFO] net.sf.saxon:Saxon-HE . 9.6.0-4 - 9.6.0-5 [INFO] org.apache.activemq:activemq-client . 5.10.0 - 5.11.1 [INFO] org.apache.avro:avro .. 1.7.6 - 1.7.7 [INFO] org.apache.commons:commons-lang3 3.3.2 - 3.4 [INFO] org.apache.commons:commons-pool2 2.3 - 2.4.1 [INFO] org.apache.hadoop:hadoop-annotations .. 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-client ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-common ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-hdfs . 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-mapreduce-client-core 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-yarn-api . 2.6.0 - 2.7.0 [INFO] org.apache.httpcomponents:httpclient .. 4.3.6 - 4.4.1 [INFO] org.apache.lucene:lucene-analyzers-common 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-core 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-queryparser . 4.10.3 - 5.1.0 [INFO] org.aspectj:aspectjweaver . 1.8.4 - 1.8.5 [INFO] org.slf4j:jcl-over-slf4j 1.7.10 - 1.7.12 [INFO] org.slf4j:jul-to-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:log4j-over-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-api . 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-simple .. 1.7.10 - 1.7.12 [INFO] org.springframework:spring-aop 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-beans .. 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context-support ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-core ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-expression ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-jdbc ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-tx . 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-web 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework.security:spring-security-acl ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-config ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-core ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-web ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] [INFO] artifact org.apache.spark:spark-streaming_2.10: checking for updates from central [INFO] The following dependencies in Dependencies have newer versions: [INFO] org.apache.spark:spark-streaming_2.10 . 1.2.0 - 1.3.1
incubator-nifi git commit: NIFI-654 updated all non controversial deps. Tested locally and on a remote system with full 2-way SSL
Repository: incubator-nifi Updated Branches: refs/heads/develop 310ae3ebc - 07628f76f NIFI-654 updated all non controversial deps. Tested locally and on a remote system with full 2-way SSL Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/07628f76 Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/07628f76 Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/07628f76 Branch: refs/heads/develop Commit: 07628f76fedfe8596c62fbfb2edd159ddf051665 Parents: 310ae3e Author: joewitt joew...@apache.org Authored: Thu Jun 4 11:01:39 2015 -0400 Committer: joewitt joew...@apache.org Committed: Thu Jun 4 11:01:39 2015 -0400 -- nifi/nifi-external/nifi-spark-receiver/pom.xml | 1 - nifi/pom.xml | 45 - 2 files changed, 25 insertions(+), 21 deletions(-) -- http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/07628f76/nifi/nifi-external/nifi-spark-receiver/pom.xml -- diff --git a/nifi/nifi-external/nifi-spark-receiver/pom.xml b/nifi/nifi-external/nifi-spark-receiver/pom.xml index cd03dd4..af436c7 100644 --- a/nifi/nifi-external/nifi-spark-receiver/pom.xml +++ b/nifi/nifi-external/nifi-spark-receiver/pom.xml @@ -27,7 +27,6 @@ dependency groupIdorg.apache.spark/groupId artifactIdspark-streaming_2.10/artifactId -version1.2.0/version /dependency dependency groupIdorg.apache.nifi/groupId http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/07628f76/nifi/pom.xml -- diff --git a/nifi/pom.xml b/nifi/pom.xml index 58b4e88..7ff110b 100644 --- a/nifi/pom.xml +++ b/nifi/pom.xml @@ -37,13 +37,13 @@ modulenifi-external/module /modules properties -org.slf4j.version1.7.10/org.slf4j.version -jetty.version9.2.10.v20150310/jetty.version -lucene.version4.10.3/lucene.version -spring.version4.1.4.RELEASE/spring.version -spring.security.version3.2.5.RELEASE/spring.security.version +org.slf4j.version1.7.12/org.slf4j.version +jetty.version9.2.11.v20150529/jetty.version +lucene.version4.10.4/lucene.version +spring.version4.1.6.RELEASE/spring.version +spring.security.version3.2.7.RELEASE/spring.security.version jersey.version1.19/jersey.version -hadoop.version2.6.0/hadoop.version +hadoop.version2.7.0/hadoop.version inceptionYear2014/inceptionYear /properties dependencyManagement @@ -51,12 +51,12 @@ dependency groupIdch.qos.logback/groupId artifactIdlogback-classic/artifactId -version1.1.2/version +version1.1.3/version /dependency dependency groupIdch.qos.logback/groupId artifactIdjcl-over-slf4j/artifactId -version1.1.2/version +version1.1.3/version scopeprovided/scope exclusions exclusion @@ -97,12 +97,12 @@ dependency groupIdorg.mockito/groupId artifactIdmockito-core/artifactId -version1.10.16/version +version1.10.19/version /dependency dependency groupIdorg.mockito/groupId artifactIdmockito-all/artifactId -version1.10.8/version +version1.10.19/version scopetest/scope /dependency dependency @@ -118,7 +118,7 @@ dependency groupIdorg.apache.commons/groupId artifactIdcommons-lang3/artifactId -version3.3.2/version +version3.4/version /dependency dependency groupIdorg.antlr/groupId @@ -158,12 +158,12 @@ dependency groupIdcom.jcraft/groupId artifactIdjsch/artifactId -version0.1.51/version +version0.1.52/version /dependency dependency groupIdorg.apache.httpcomponents/groupId artifactIdhttpclient/artifactId -version4.3.6/version +version4.4.1/version /dependency dependency groupIdjavax.mail/groupId @@ -183,7 +183,7 @@ dependency groupIdnet.sf.saxon/groupId artifactIdSaxon-HE/artifactId -version9.6.0-4/version +version9.6.0-5/version
[jira] [Updated] (NIFI-647) Duplicating provenance reporting via session creates unexpected behavior
[
https://issues.apache.org/jira/browse/NIFI-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Mark Payne updated NIFI-647:
Attachment: 0001-NIFI-647-When-FORK-Event-emitted-by-processor-Framew.patch
Duplicating provenance reporting via session creates unexpected behavior
Key: NIFI-647
URL: https://issues.apache.org/jira/browse/NIFI-647
Project: Apache NiFi
Issue Type: Bug
Components: Core Framework
Affects Versions: 0.0.2, 0.1.0
Reporter: Aldrin Piri
Assignee: Mark Payne
Fix For: 0.2.0
Attachments:
0001-NIFI-647-When-FORK-Event-emitted-by-processor-Framew.patch
This is related to NIFI-643. While that processor bug was resolved, there are
issues with how this is managed in the framework.
The unexpected behavior is created via the following:
https://github.com/apache/incubator-nifi/blob/d29a2d688e437bae42c12115768cdb038b7406c5/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/SplitJson.java#L196
Summarily, a series of files (splits) are created from the parent file which
generates the associated FORK provenance events. The separate call to the
provenance reporter at Line 196 also requests a FORK for each of the
generated files. This manifests itself as being CREATE events for each of
the FORKed files and causes an error:
{code}
Found cycle in graph. This indicates that multiple events were registered
claiming to have generated the same FlowFile (UUID =
21b663c2-2406-49cb-b740-76dd64f54be8)
{code}
Ideally, such behavior would also be detected by the test framework.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-313) When template is deleted, UI still shows Displaying X of Y instead of Displaying X of (Y-1)
[ https://issues.apache.org/jira/browse/NIFI-313?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572817#comment-14572817 ] Matt Gilman commented on NIFI-313: -- For some reason the commit doesn't show up in the ticket activity... https://github.com/apache/incubator-nifi/commit/89801675b4bf66ee106bb452ba89996e4f0be56e When template is deleted, UI still shows Displaying X of Y instead of Displaying X of (Y-1) --- Key: NIFI-313 URL: https://issues.apache.org/jira/browse/NIFI-313 Project: Apache NiFi Issue Type: Bug Affects Versions: 0.0.2 Reporter: Mark Payne Assignee: Matt Gilman Priority: Trivial Fix For: 0.2.0 Attachments: 0001-NIFI-313.patch -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-657) MongoDB unit tests fail
Mark Payne created NIFI-657: --- Summary: MongoDB unit tests fail Key: NIFI-657 URL: https://issues.apache.org/jira/browse/NIFI-657 Project: Apache NiFi Issue Type: Bug Components: Extensions Reporter: Mark Payne Fix For: 0.2.0 Trying to do a build of NiFi on a brand new VM running CentOS 7. Environment: [mark@nifi-build nifi]$ mvn -version Apache Maven 3.0.5 (Red Hat 3.0.5-16) Maven home: /usr/share/maven Java version: 1.7.0_79, vendor: Oracle Corporation Java home: /usr/lib/jvm/java-1.7.0-openjdk-1.7.0.79-2.5.5.1.el7_1.x86_64/jre Default locale: en_US, platform encoding: UTF-8 OS name: linux, version: 3.10.0-229.el7.x86_64, arch: amd64, family: unix Output from mvn: --- T E S T S --- [INFO] Nothing to compile - all classes are up to date [INFO] [INFO] --- embedmongo-maven-plugin:0.1.12:start (start) @ nifi-mongodb-processors --- Extract /home/mark/.embedmongo/linux/mongodb-linux-x86_64-2.2.1.tgz START [INFO] [INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ nifi-standard-processors --- [INFO] Using 'UTF-8' encoding to copy filtered resources. [INFO] Copying 9 resources [INFO] Copying 3 resources Extract /home/mark/.embedmongo/linux/mongodb-linux-x86_64-2.2.1.tgz DONE [INFO] [INFO] --- maven-compiler-plugin:3.2:compile (default-compile) @ nifi-standard-processors --- Jun 04, 2015 10:29:37 AM de.flapdoodle.embed.process.runtime.AbstractProcess init SEVERE: failed to call onAfterProcessStart() java.io.IOException: Could not start process: EOF at de.flapdoodle.embed.mongo.AbstractMongoProcess.onAfterProcessStart(AbstractMongoProcess.java:75) at de.flapdoodle.embed.process.runtime.AbstractProcess.init(AbstractProcess.java:110) at de.flapdoodle.embed.mongo.AbstractMongoProcess.init(AbstractMongoProcess.java:52) at de.flapdoodle.embed.mongo.MongodProcess.init(MongodProcess.java:50) at de.flapdoodle.embed.mongo.MongodExecutable.start(MongodExecutable.java:44) at de.flapdoodle.embed.mongo.MongodExecutable.start(MongodExecutable.java:34) at de.flapdoodle.embed.process.runtime.Executable.start(Executable.java:99) at com.github.joelittlejohn.embedmongo.StartEmbeddedMongoMojo.execute(StartEmbeddedMongoMojo.java:271) at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo(DefaultBuildPluginManager.java:101) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:209) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:153) at org.apache.maven.lifecycle.internal.MojoExecutor.execute(MojoExecutor.java:145) at org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject(LifecycleModuleBuilder.java:84) at org.apache.maven.lifecycle.internal.LifecycleThreadedBuilder$1.call(LifecycleThreadedBuilder.java:167) at org.apache.maven.lifecycle.internal.LifecycleThreadedBuilder$1.call(LifecycleThreadedBuilder.java:163) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) at java.util.concurrent.FutureTask.run(FutureTask.java:262) at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) at java.lang.Thread.run(Thread.java:745) Jun 04, 2015 10:29:37 AM MongodProcess ctor INFO: de.flapdoodle.embed.mongo.config.MongodConfigBuilder$ImmutableMongodConfig@701cce38 Jun 04, 2015 10:29:37 AM de.flapdoodle.embed.mongo.runtime.Mongod sendShutdown WARNING: sendShutdown nifi-build.beefy/127.0.0.1:27017 java.net.ConnectException: Connection refused at java.net.PlainSocketImpl.socketConnect(Native Method) at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:339) at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:200) at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:182) at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392) at java.net.Socket.connect(Socket.java:579) at de.flapdoodle.embed.mongo.runtime.Mongod.sendShutdown(Mongod.java:83) at de.flapdoodle.embed.mongo.AbstractMongoProcess.sendStopToMongoInstance(AbstractMongoProcess.java:125) at de.flapdoodle.embed.mongo.AbstractMongoProcess.stopInternal(AbstractMongoProcess.java:99) at de.flapdoodle.embed.process.runtime.AbstractProcess.stop(AbstractProcess.java:165) at de.flapdoodle.embed.process.runtime.AbstractProcess.init(AbstractProcess.java:115) at
[jira] [Commented] (NIFI-654) Update dependency versions
[ https://issues.apache.org/jira/browse/NIFI-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572945#comment-14572945 ] Joseph Witt commented on NIFI-654: -- Updated numerous dependencies. Am comfortable with what remains. Though bumping across some of these major versions will require coding (lucene, jetty, jersey, spring-security, etc..). So those need review in due time. For now moved to latest minor or incremental releases available. This is what the results are now of dependency check: [INFO] com.h2database:h2 . 1.3.176 - 1.4.187 [INFO] com.yammer.metrics:metrics-ganglia .. 2.2.0 - 3.0.0-BETA1 [INFO] javax.el:javax.el-api . 3.0.0 - 3.0.1-b04 [INFO] javax.mail:mail ... 1.4.7 - 1.5.0-b01 [INFO] javax.servlet.jsp:javax.servlet.jsp-api ... 2.3.1 - 2.3.2-b01 [INFO] org.apache.httpcomponents:httpclient 4.4.1 - 4.5 [INFO] org.apache.lucene:lucene-analyzers-common 4.10.4 - 5.1.0 [INFO] org.apache.lucene:lucene-core 4.10.4 - 5.1.0 [INFO] org.apache.lucene:lucene-queryparser . 4.10.4 - 5.1.0 [INFO] org.eclipse.jetty:jetty-jsp . 9.2.11.v20150529 - 9.3.0.M1 [INFO] org.eclipse.jetty:jetty-server . 9.2.11.v20150529 - 9.3.0.RC1 [INFO] org.eclipse.jetty:jetty-servlet 9.2.11.v20150529 - 9.3.0.RC1 [INFO] org.eclipse.jetty:jetty-servlets ... 9.2.11.v20150529 - 9.3.0.RC1 [INFO] org.eclipse.jetty:jetty-webapp . 9.2.11.v20150529 - 9.3.0.RC1 [INFO] org.mockito:mockito-all 1.10.19 - 2.0.2-beta [INFO] org.mockito:mockito-core .. 1.10.19 - 2.0.11-beta [INFO] org.springframework.security:spring-security-acl ... [INFO] 3.2.7.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-config ... [INFO] 3.2.7.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-core ... [INFO] 3.2.7.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-web ... [INFO] 3.2.7.RELEASE - 4.0.1.RELEASE Update dependency versions --- Key: NIFI-654 URL: https://issues.apache.org/jira/browse/NIFI-654 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: Joseph Witt Assignee: Joseph Witt Fix For: 0.2.0 Attachments: NIFI-654.patch The following output comes from running mvn versions:update-dependency-versions [INFO] --- versions-maven-plugin:2.2:display-dependency-updates (default-cli) @ nifi-spark-receiver --- [INFO] The following dependencies in Dependency Management have newer versions: [INFO] ch.qos.logback:logback-classic 1.1.2 - 1.1.3 [INFO] com.jcraft:jsch . 0.1.51 - 0.1.52 [INFO] javax.jms:javax.jms-api . 2.0 - 2.0.1 [INFO] joda-time:joda-time ... 2.7 - 2.8 [INFO] net.sf.saxon:Saxon-HE . 9.6.0-4 - 9.6.0-5 [INFO] org.apache.activemq:activemq-client . 5.10.0 - 5.11.1 [INFO] org.apache.avro:avro .. 1.7.6 - 1.7.7 [INFO] org.apache.commons:commons-lang3 3.3.2 - 3.4 [INFO] org.apache.commons:commons-pool2 2.3 - 2.4.1 [INFO] org.apache.hadoop:hadoop-annotations .. 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-client ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-common ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-hdfs . 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-mapreduce-client-core 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-yarn-api . 2.6.0 - 2.7.0 [INFO] org.apache.httpcomponents:httpclient .. 4.3.6 - 4.4.1 [INFO] org.apache.lucene:lucene-analyzers-common 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-core 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-queryparser . 4.10.3 - 5.1.0 [INFO] org.aspectj:aspectjweaver . 1.8.4 - 1.8.5 [INFO] org.slf4j:jcl-over-slf4j 1.7.10 - 1.7.12 [INFO] org.slf4j:jul-to-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:log4j-over-slf4j .. 1.7.10 - 1.7.12
[jira] [Resolved] (NIFI-321) New Report Generator Processor
[ https://issues.apache.org/jira/browse/NIFI-321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toivo Adams resolved NIFI-321. -- New Report Generator Processor -- Key: NIFI-321 URL: https://issues.apache.org/jira/browse/NIFI-321 Project: Apache NiFi Issue Type: New Feature Components: Extensions Reporter: Toivo Adams Assignee: Toivo Adams Priority: Minor Labels: processor It would be very useful for generating different business reports. Underlying report generator is Birt engine. http://eclipse.org/birt/ Input: data in XML Parameter 1: report design file name Parameter 2: desired output format (PDF, Word doc, Excel. Html, etc) Output: report in desired format TransformXml can be used as starting point. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-654) Update dependency versions
[ https://issues.apache.org/jira/browse/NIFI-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572788#comment-14572788 ] Joseph Witt commented on NIFI-654: -- bumped lucene to 4.10.4 rather than 5.1.0 due to compile errors. That will require more work. Update dependency versions --- Key: NIFI-654 URL: https://issues.apache.org/jira/browse/NIFI-654 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: Joseph Witt Assignee: Joseph Witt Fix For: 0.2.0 The following output comes from running mvn versions:update-dependency-versions [INFO] --- versions-maven-plugin:2.2:display-dependency-updates (default-cli) @ nifi-spark-receiver --- [INFO] The following dependencies in Dependency Management have newer versions: [INFO] ch.qos.logback:logback-classic 1.1.2 - 1.1.3 [INFO] com.jcraft:jsch . 0.1.51 - 0.1.52 [INFO] javax.jms:javax.jms-api . 2.0 - 2.0.1 [INFO] joda-time:joda-time ... 2.7 - 2.8 [INFO] net.sf.saxon:Saxon-HE . 9.6.0-4 - 9.6.0-5 [INFO] org.apache.activemq:activemq-client . 5.10.0 - 5.11.1 [INFO] org.apache.avro:avro .. 1.7.6 - 1.7.7 [INFO] org.apache.commons:commons-lang3 3.3.2 - 3.4 [INFO] org.apache.commons:commons-pool2 2.3 - 2.4.1 [INFO] org.apache.hadoop:hadoop-annotations .. 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-client ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-common ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-hdfs . 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-mapreduce-client-core 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-yarn-api . 2.6.0 - 2.7.0 [INFO] org.apache.httpcomponents:httpclient .. 4.3.6 - 4.4.1 [INFO] org.apache.lucene:lucene-analyzers-common 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-core 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-queryparser . 4.10.3 - 5.1.0 [INFO] org.aspectj:aspectjweaver . 1.8.4 - 1.8.5 [INFO] org.slf4j:jcl-over-slf4j 1.7.10 - 1.7.12 [INFO] org.slf4j:jul-to-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:log4j-over-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-api . 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-simple .. 1.7.10 - 1.7.12 [INFO] org.springframework:spring-aop 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-beans .. 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context-support ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-core ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-expression ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-jdbc ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-tx . 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-web 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework.security:spring-security-acl ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-config ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-core ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-web ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] [INFO] artifact org.apache.spark:spark-streaming_2.10: checking for updates from central [INFO] The following dependencies in Dependencies have newer versions: [INFO] org.apache.spark:spark-streaming_2.10 . 1.2.0 - 1.3.1 [INFO] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-321) New Report Generator Processor
[ https://issues.apache.org/jira/browse/NIFI-321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toivo Adams reassigned NIFI-321: Assignee: Toivo Adams New Report Generator Processor -- Key: NIFI-321 URL: https://issues.apache.org/jira/browse/NIFI-321 Project: Apache NiFi Issue Type: New Feature Components: Extensions Reporter: Toivo Adams Assignee: Toivo Adams Priority: Minor Labels: processor It would be very useful for generating different business reports. Underlying report generator is Birt engine. http://eclipse.org/birt/ Input: data in XML Parameter 1: report design file name Parameter 2: desired output format (PDF, Word doc, Excel. Html, etc) Output: report in desired format TransformXml can be used as starting point. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-654) Update dependency versions
[ https://issues.apache.org/jira/browse/NIFI-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572925#comment-14572925 ] Joseph Witt commented on NIFI-654: -- disregard that last point: Bad logging config. Didn't have !-- Suppress non-error messages due to known warning about redundant path annotation (NIFI-574) -- logger name=com.sun.jersey.spi.inject.Errors level=ERROR/ Update dependency versions --- Key: NIFI-654 URL: https://issues.apache.org/jira/browse/NIFI-654 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: Joseph Witt Assignee: Joseph Witt Fix For: 0.2.0 The following output comes from running mvn versions:update-dependency-versions [INFO] --- versions-maven-plugin:2.2:display-dependency-updates (default-cli) @ nifi-spark-receiver --- [INFO] The following dependencies in Dependency Management have newer versions: [INFO] ch.qos.logback:logback-classic 1.1.2 - 1.1.3 [INFO] com.jcraft:jsch . 0.1.51 - 0.1.52 [INFO] javax.jms:javax.jms-api . 2.0 - 2.0.1 [INFO] joda-time:joda-time ... 2.7 - 2.8 [INFO] net.sf.saxon:Saxon-HE . 9.6.0-4 - 9.6.0-5 [INFO] org.apache.activemq:activemq-client . 5.10.0 - 5.11.1 [INFO] org.apache.avro:avro .. 1.7.6 - 1.7.7 [INFO] org.apache.commons:commons-lang3 3.3.2 - 3.4 [INFO] org.apache.commons:commons-pool2 2.3 - 2.4.1 [INFO] org.apache.hadoop:hadoop-annotations .. 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-client ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-common ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-hdfs . 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-mapreduce-client-core 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-yarn-api . 2.6.0 - 2.7.0 [INFO] org.apache.httpcomponents:httpclient .. 4.3.6 - 4.4.1 [INFO] org.apache.lucene:lucene-analyzers-common 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-core 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-queryparser . 4.10.3 - 5.1.0 [INFO] org.aspectj:aspectjweaver . 1.8.4 - 1.8.5 [INFO] org.slf4j:jcl-over-slf4j 1.7.10 - 1.7.12 [INFO] org.slf4j:jul-to-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:log4j-over-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-api . 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-simple .. 1.7.10 - 1.7.12 [INFO] org.springframework:spring-aop 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-beans .. 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context-support ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-core ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-expression ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-jdbc ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-tx . 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-web 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework.security:spring-security-acl ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-config ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-core ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-web ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] [INFO] artifact org.apache.spark:spark-streaming_2.10: checking for updates from central [INFO] The following dependencies in Dependencies have newer versions: [INFO] org.apache.spark:spark-streaming_2.10 . 1.2.0 - 1.3.1 [INFO] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-654) Update dependency versions
[ https://issues.apache.org/jira/browse/NIFI-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572792#comment-14572792 ] Joseph Witt commented on NIFI-654: -- Moving spring security to 3.2.7-RELEASE as 4.0.1 causes compile errors [ERROR] /home/joe/apache-nifi/nifi/nifi/nifi-nar-bundles/nifi-framework-bundle/nifi-framework/nifi-web/nifi-web-security/src/main/java/org/apache/nifi/web/security/anonymous/NiFiAnonymousUserFilter.java:[82,4] error: method does not override or implement a method from a supertype Update dependency versions --- Key: NIFI-654 URL: https://issues.apache.org/jira/browse/NIFI-654 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: Joseph Witt Assignee: Joseph Witt Fix For: 0.2.0 The following output comes from running mvn versions:update-dependency-versions [INFO] --- versions-maven-plugin:2.2:display-dependency-updates (default-cli) @ nifi-spark-receiver --- [INFO] The following dependencies in Dependency Management have newer versions: [INFO] ch.qos.logback:logback-classic 1.1.2 - 1.1.3 [INFO] com.jcraft:jsch . 0.1.51 - 0.1.52 [INFO] javax.jms:javax.jms-api . 2.0 - 2.0.1 [INFO] joda-time:joda-time ... 2.7 - 2.8 [INFO] net.sf.saxon:Saxon-HE . 9.6.0-4 - 9.6.0-5 [INFO] org.apache.activemq:activemq-client . 5.10.0 - 5.11.1 [INFO] org.apache.avro:avro .. 1.7.6 - 1.7.7 [INFO] org.apache.commons:commons-lang3 3.3.2 - 3.4 [INFO] org.apache.commons:commons-pool2 2.3 - 2.4.1 [INFO] org.apache.hadoop:hadoop-annotations .. 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-client ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-common ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-hdfs . 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-mapreduce-client-core 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-yarn-api . 2.6.0 - 2.7.0 [INFO] org.apache.httpcomponents:httpclient .. 4.3.6 - 4.4.1 [INFO] org.apache.lucene:lucene-analyzers-common 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-core 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-queryparser . 4.10.3 - 5.1.0 [INFO] org.aspectj:aspectjweaver . 1.8.4 - 1.8.5 [INFO] org.slf4j:jcl-over-slf4j 1.7.10 - 1.7.12 [INFO] org.slf4j:jul-to-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:log4j-over-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-api . 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-simple .. 1.7.10 - 1.7.12 [INFO] org.springframework:spring-aop 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-beans .. 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context-support ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-core ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-expression ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-jdbc ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-tx . 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-web 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework.security:spring-security-acl ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-config ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-core ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-web ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] [INFO] artifact org.apache.spark:spark-streaming_2.10: checking for updates from central [INFO] The following dependencies in Dependencies have newer versions: [INFO] org.apache.spark:spark-streaming_2.10 . 1.2.0 - 1.3.1 [INFO] --
[jira] [Commented] (NIFI-654) Update dependency versions
[ https://issues.apache.org/jira/browse/NIFI-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572908#comment-14572908 ] Joseph Witt commented on NIFI-654: -- After updating jetty and running everything seems fine and works. However, I do see this in the logs (this is on a version setup for 2-way ssl and so on) 2015-06-04 14:31:37,700 WARN [NiFi Web Server-134] com.sun.jersey.spi.inject.Errors The following warnings have been detected with resource and/or provider classes: WARNING: A sub-resource method, public javax.ws.rs.core.Response org.apache.nifi.web.api.ProcessGroupResource.updateProcessGroup(javax.servlet.http.HttpServletRequest,org.apache.nifi.web.api.request.LongParameter,org.apache.nifi.web.api.request.ClientIdParameter,java.lang.Boolean), with URI template, , is treated as a resource method WARNING: A sub-resource method, public javax.ws.rs.core.Response org.apache.nifi.web.api.ProcessGroupResource.updateProcessGroup(javax.servlet.http.HttpServletRequest,org.apache.nifi.web.api.entity.ProcessGroupEntity), with URI template, , is treated as a resource method WARNING: A sub-resource method, public javax.ws.rs.core.Response org.apache.nifi.web.api.ProcessGroupResource.getProcessGroup(org.apache.nifi.web.api.request.ClientIdParameter,java.lang.Boolean,java.lang.Boolean), with URI template, , is treated as a resource method 2015-06-04 14:32:10,664 WARN [NiFi Web Server-132] com.sun.jersey.spi.inject.Errors The following warnings have been detected with resource and/or provider classes: WARNING: A sub-resource method, public javax.ws.rs.core.Response org.apache.nifi.web.api.ProvenanceResource.submitProvenanceRequest(javax.servlet.http.HttpServletRequest,org.apache.nifi.web.api.request.ClientIdParameter,org.apache.nifi.web.api.request.DateTimeParameter,org.apache.nifi.web.api.request.DateTimeParameter,java.lang.String,java.lang.String,org.apache.nifi.web.api.request.IntegerParameter,java.lang.String,javax.ws.rs.core.MultivaluedMap), with URI template, , is treated as a resource method WARNING: A sub-resource method, public javax.ws.rs.core.Response org.apache.nifi.web.api.ProvenanceResource.submitProvenanceRequest(javax.servlet.http.HttpServletRequest,org.apache.nifi.web.api.entity.ProvenanceEntity), with URI template, , is treated as a resource method Update dependency versions --- Key: NIFI-654 URL: https://issues.apache.org/jira/browse/NIFI-654 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: Joseph Witt Assignee: Joseph Witt Fix For: 0.2.0 The following output comes from running mvn versions:update-dependency-versions [INFO] --- versions-maven-plugin:2.2:display-dependency-updates (default-cli) @ nifi-spark-receiver --- [INFO] The following dependencies in Dependency Management have newer versions: [INFO] ch.qos.logback:logback-classic 1.1.2 - 1.1.3 [INFO] com.jcraft:jsch . 0.1.51 - 0.1.52 [INFO] javax.jms:javax.jms-api . 2.0 - 2.0.1 [INFO] joda-time:joda-time ... 2.7 - 2.8 [INFO] net.sf.saxon:Saxon-HE . 9.6.0-4 - 9.6.0-5 [INFO] org.apache.activemq:activemq-client . 5.10.0 - 5.11.1 [INFO] org.apache.avro:avro .. 1.7.6 - 1.7.7 [INFO] org.apache.commons:commons-lang3 3.3.2 - 3.4 [INFO] org.apache.commons:commons-pool2 2.3 - 2.4.1 [INFO] org.apache.hadoop:hadoop-annotations .. 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-client ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-common ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-hdfs . 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-mapreduce-client-core 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-yarn-api . 2.6.0 - 2.7.0 [INFO] org.apache.httpcomponents:httpclient .. 4.3.6 - 4.4.1 [INFO] org.apache.lucene:lucene-analyzers-common 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-core 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-queryparser . 4.10.3 - 5.1.0 [INFO] org.aspectj:aspectjweaver . 1.8.4 - 1.8.5 [INFO] org.slf4j:jcl-over-slf4j 1.7.10 - 1.7.12 [INFO] org.slf4j:jul-to-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:log4j-over-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-api
[jira] [Updated] (NIFI-645) Set content archive and content viewer on by default
[ https://issues.apache.org/jira/browse/NIFI-645?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-645: - Fix Version/s: (was: 0.3.0) 0.2.0 Set content archive and content viewer on by default Key: NIFI-645 URL: https://issues.apache.org/jira/browse/NIFI-645 Project: Apache NiFi Issue Type: Improvement Components: Configuration Reporter: Joseph Witt Assignee: Joseph Witt Fix For: 0.2.0 The content viewer is now available in the build and it would be nice if it was preconfigured to be used by default. If the user doesn't want it available they can remove it (seems unlikely) or if they have an alternative they can point to it. The same is true of content archive. We should keep content around for a few hours by default (or some number of some time unit) and for X% of space. The core thing we're seeing is that users are wanting to use these features but not realizing they're not on by default. At this point in their maturity they should be on by default and only shut off when a user wants that. Also have the component status repository update be every 30 seconds by default and keep for 24 hours -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-321) New Report Generator Processor
[ https://issues.apache.org/jira/browse/NIFI-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572805#comment-14572805 ] Toivo Adams commented on NIFI-321: -- Right In the coming weeks I don’t have plans to do further Birt processor development anyway. I closed the ticket. Thanks Toivo New Report Generator Processor -- Key: NIFI-321 URL: https://issues.apache.org/jira/browse/NIFI-321 Project: Apache NiFi Issue Type: New Feature Components: Extensions Reporter: Toivo Adams Assignee: Toivo Adams Priority: Minor Labels: processor It would be very useful for generating different business reports. Underlying report generator is Birt engine. http://eclipse.org/birt/ Input: data in XML Parameter 1: report design file name Parameter 2: desired output format (PDF, Word doc, Excel. Html, etc) Output: report in desired format TransformXml can be used as starting point. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-654) Update dependency versions
[ https://issues.apache.org/jira/browse/NIFI-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572807#comment-14572807 ] Joseph Witt commented on NIFI-654: -- moving jetty to 9.2.11.v20150529 Update dependency versions --- Key: NIFI-654 URL: https://issues.apache.org/jira/browse/NIFI-654 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: Joseph Witt Assignee: Joseph Witt Fix For: 0.2.0 The following output comes from running mvn versions:update-dependency-versions [INFO] --- versions-maven-plugin:2.2:display-dependency-updates (default-cli) @ nifi-spark-receiver --- [INFO] The following dependencies in Dependency Management have newer versions: [INFO] ch.qos.logback:logback-classic 1.1.2 - 1.1.3 [INFO] com.jcraft:jsch . 0.1.51 - 0.1.52 [INFO] javax.jms:javax.jms-api . 2.0 - 2.0.1 [INFO] joda-time:joda-time ... 2.7 - 2.8 [INFO] net.sf.saxon:Saxon-HE . 9.6.0-4 - 9.6.0-5 [INFO] org.apache.activemq:activemq-client . 5.10.0 - 5.11.1 [INFO] org.apache.avro:avro .. 1.7.6 - 1.7.7 [INFO] org.apache.commons:commons-lang3 3.3.2 - 3.4 [INFO] org.apache.commons:commons-pool2 2.3 - 2.4.1 [INFO] org.apache.hadoop:hadoop-annotations .. 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-client ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-common ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-hdfs . 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-mapreduce-client-core 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-yarn-api . 2.6.0 - 2.7.0 [INFO] org.apache.httpcomponents:httpclient .. 4.3.6 - 4.4.1 [INFO] org.apache.lucene:lucene-analyzers-common 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-core 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-queryparser . 4.10.3 - 5.1.0 [INFO] org.aspectj:aspectjweaver . 1.8.4 - 1.8.5 [INFO] org.slf4j:jcl-over-slf4j 1.7.10 - 1.7.12 [INFO] org.slf4j:jul-to-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:log4j-over-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-api . 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-simple .. 1.7.10 - 1.7.12 [INFO] org.springframework:spring-aop 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-beans .. 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context-support ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-core ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-expression ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-jdbc ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-tx . 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-web 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework.security:spring-security-acl ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-config ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-core ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-web ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] [INFO] artifact org.apache.spark:spark-streaming_2.10: checking for updates from central [INFO] The following dependencies in Dependencies have newer versions: [INFO] org.apache.spark:spark-streaming_2.10 . 1.2.0 - 1.3.1 [INFO] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-653) Rename Provenance Modules
[ https://issues.apache.org/jira/browse/NIFI-653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572910#comment-14572910 ] Mark Payne commented on NIFI-653: - I have no objection to renaming nifi-data-provenance-utils to nifi-provenance-utils. However, I would not rename nifi-volatile-provenance-repository to nifi-provenance-repository-volatile or rename the nifi-persistent-provenance-repository. I can appreciate wanting to have them together in the IDE. But I think nifi-volatile-provenance-repository is a more intuitive name, for instance. Instead, I recommend creating a Working Set in Eclipse and adding the provenance stuff to the Working Set. Any reason that this doesn't give you what you need? Rename Provenance Modules - Key: NIFI-653 URL: https://issues.apache.org/jira/browse/NIFI-653 Project: Apache NiFi Issue Type: Improvement Reporter: BELUGA BEHR Priority: Minor It would be nice, if in my IDE, all provenance-related modules were co-located by sorting the module names alphabetically. This change is a little beyond my scope. nifi-provenance-repository-bundle nifi-provenance-repository-nar nifi-volatile-provenance-repository - nifi-provenance-repository-volatile nifi-persistent-provenance-repository - nifi-provenance-repository-persistent nifi-data-provenance-utils - nifi-provenance-utils -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-654) Update dependency versions
[ https://issues.apache.org/jira/browse/NIFI-654?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-654: - Attachment: NIFI-654.patch Update dependency versions --- Key: NIFI-654 URL: https://issues.apache.org/jira/browse/NIFI-654 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: Joseph Witt Assignee: Joseph Witt Fix For: 0.2.0 Attachments: NIFI-654.patch The following output comes from running mvn versions:update-dependency-versions [INFO] --- versions-maven-plugin:2.2:display-dependency-updates (default-cli) @ nifi-spark-receiver --- [INFO] The following dependencies in Dependency Management have newer versions: [INFO] ch.qos.logback:logback-classic 1.1.2 - 1.1.3 [INFO] com.jcraft:jsch . 0.1.51 - 0.1.52 [INFO] javax.jms:javax.jms-api . 2.0 - 2.0.1 [INFO] joda-time:joda-time ... 2.7 - 2.8 [INFO] net.sf.saxon:Saxon-HE . 9.6.0-4 - 9.6.0-5 [INFO] org.apache.activemq:activemq-client . 5.10.0 - 5.11.1 [INFO] org.apache.avro:avro .. 1.7.6 - 1.7.7 [INFO] org.apache.commons:commons-lang3 3.3.2 - 3.4 [INFO] org.apache.commons:commons-pool2 2.3 - 2.4.1 [INFO] org.apache.hadoop:hadoop-annotations .. 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-client ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-common ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-hdfs . 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-mapreduce-client-core 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-yarn-api . 2.6.0 - 2.7.0 [INFO] org.apache.httpcomponents:httpclient .. 4.3.6 - 4.4.1 [INFO] org.apache.lucene:lucene-analyzers-common 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-core 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-queryparser . 4.10.3 - 5.1.0 [INFO] org.aspectj:aspectjweaver . 1.8.4 - 1.8.5 [INFO] org.slf4j:jcl-over-slf4j 1.7.10 - 1.7.12 [INFO] org.slf4j:jul-to-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:log4j-over-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-api . 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-simple .. 1.7.10 - 1.7.12 [INFO] org.springframework:spring-aop 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-beans .. 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context-support ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-core ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-expression ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-jdbc ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-tx . 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-web 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework.security:spring-security-acl ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-config ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-core ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-web ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] [INFO] artifact org.apache.spark:spark-streaming_2.10: checking for updates from central [INFO] The following dependencies in Dependencies have newer versions: [INFO] org.apache.spark:spark-streaming_2.10 . 1.2.0 - 1.3.1 [INFO] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-551) Improve error handling for ConvertJSONToAvro processor
[ https://issues.apache.org/jira/browse/NIFI-551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573444#comment-14573444 ] ASF subversion and git services commented on NIFI-551: -- Commit 5eed335804ccbbb15316e80fb5fb09b36897c69f in incubator-nifi's branch refs/heads/develop from [~joewitt] [ https://git-wip-us.apache.org/repos/asf?p=incubator-nifi.git;h=5eed335 ] NIFI-551 checkstyle mods to provided patch. Main issue was extraneous imports. No biggie. Improve error handling for ConvertJSONToAvro processor -- Key: NIFI-551 URL: https://issues.apache.org/jira/browse/NIFI-551 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Ricky Saltzer Assignee: Joseph Witt Labels: patch, review Fix For: 0.2.0 Attachments: NIFI-551.1.patch, NIFI-551.2.patch, NIFI-551.3.patch Currently, if the ConvertJSONToAvro processor fails to process an individual record, a counter is incremented, but no alerts are produced. It would be better to notify the bulletin board that we've failed to process some records for a flowfile. Further, we should stream the records we fail to process down the failure relationship for further inspection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-551) Improve error handling for ConvertJSONToAvro processor
[ https://issues.apache.org/jira/browse/NIFI-551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573443#comment-14573443 ] ASF subversion and git services commented on NIFI-551: -- Commit 6f32e6e9776548b746f6388d25067cd17aa1898c in incubator-nifi's branch refs/heads/develop from [~rickysaltzer] [ https://git-wip-us.apache.org/repos/asf?p=incubator-nifi.git;h=6f32e6e ] NIFI-551 - ConvertJSONToAvro improve error message - Report failure counts as an log error message - Send record parsing errors to a separate flowfile which is transfered down the failure relationship Signed-off-by: joewitt joew...@apache.org Improve error handling for ConvertJSONToAvro processor -- Key: NIFI-551 URL: https://issues.apache.org/jira/browse/NIFI-551 Project: Apache NiFi Issue Type: Improvement Components: Extensions Reporter: Ricky Saltzer Assignee: Joseph Witt Labels: patch, review Fix For: 0.2.0 Attachments: NIFI-551.1.patch, NIFI-551.2.patch, NIFI-551.3.patch Currently, if the ConvertJSONToAvro processor fails to process an individual record, a counter is incremented, but no alerts are produced. It would be better to notify the bulletin board that we've failed to process some records for a flowfile. Further, we should stream the records we fail to process down the failure relationship for further inspection. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[2/2] incubator-nifi git commit: NIFI-551 checkstyle mods to provided patch. Main issue was extraneous imports. No biggie.
NIFI-551 checkstyle mods to provided patch. Main issue was extraneous imports.
No biggie.
Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/5eed3358
Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/5eed3358
Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/5eed3358
Branch: refs/heads/develop
Commit: 5eed335804ccbbb15316e80fb5fb09b36897c69f
Parents: 6f32e6e
Author: joewitt joew...@apache.org
Authored: Thu Jun 4 15:34:58 2015 -0400
Committer: joewitt joew...@apache.org
Committed: Thu Jun 4 15:34:58 2015 -0400
--
.../nifi/processors/kite/AbstractKiteProcessor.java | 3 +--
.../apache/nifi/processors/kite/ConvertJSONToAvro.java | 8
.../nifi/processors/kite/TestJSONToAvroProcessor.java | 12 +++-
3 files changed, 8 insertions(+), 15 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/5eed3358/nifi/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/main/java/org/apache/nifi/processors/kite/AbstractKiteProcessor.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/main/java/org/apache/nifi/processors/kite/AbstractKiteProcessor.java
b/nifi/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/main/java/org/apache/nifi/processors/kite/AbstractKiteProcessor.java
index fec8239..da1c046 100644
---
a/nifi/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/main/java/org/apache/nifi/processors/kite/AbstractKiteProcessor.java
+++
b/nifi/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/main/java/org/apache/nifi/processors/kite/AbstractKiteProcessor.java
@@ -106,8 +106,7 @@ abstract class AbstractKiteProcessor extends
AbstractProcessor {
};
/**
- * Resolves a {@link Schema} for the given string, either a URI or a JSON
- * literal.
+ * Resolves a {@link Schema} for the given string, either a URI or a JSON
literal.
*/
protected static Schema getSchema(String uriOrLiteral, Configuration conf)
{
URI uri;
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/5eed3358/nifi/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/main/java/org/apache/nifi/processors/kite/ConvertJSONToAvro.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/main/java/org/apache/nifi/processors/kite/ConvertJSONToAvro.java
b/nifi/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/main/java/org/apache/nifi/processors/kite/ConvertJSONToAvro.java
index d4cc760..7969a8f 100644
---
a/nifi/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/main/java/org/apache/nifi/processors/kite/ConvertJSONToAvro.java
+++
b/nifi/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/main/java/org/apache/nifi/processors/kite/ConvertJSONToAvro.java
@@ -122,7 +122,7 @@ public class ConvertJSONToAvro extends
AbstractKiteProcessor {
writer.setCodec(CodecFactory.snappyCodec());
try {
- successfulRecords = session.write(successfulRecords, new
StreamCallback() {
+successfulRecords = session.write(successfulRecords, new
StreamCallback() {
@Override
public void process(InputStream in, OutputStream out) throws
IOException {
FlowFile failedRecords = session.create();
@@ -143,8 +143,8 @@ public class ConvertJSONToAvro extends
AbstractKiteProcessor {
failedRecords =
session.append(failedRecords, new OutputStreamCallback() {
@Override
public void process(OutputStream out)
throws IOException {
-out.write((e.getMessage() + [ +
- e.getCause().getMessage() +
]\n).getBytes());
+out.write((e.getMessage() + [
++
e.getCause().getMessage() + ]\n).getBytes());
}
});
errors += 1;
@@ -154,7 +154,7 @@ public class ConvertJSONToAvro extends
AbstractKiteProcessor {
session.adjustCounter(Converted records, written,
false /* update only if file transfer is
successful */);
session.adjustCounter(Conversion errors, errors,
- false /* update only if file transfer is successful
*/);
+false /* update only if file transfer is
[1/2] incubator-nifi git commit: NIFI-551 - ConvertJSONToAvro improve error message
Repository: incubator-nifi
Updated Branches:
refs/heads/develop 8201381c9 - 5eed33580
NIFI-551 - ConvertJSONToAvro improve error message
- Report failure counts as an log error message
- Send record parsing errors to a separate flowfile which is transfered down the
failure relationship
Signed-off-by: joewitt joew...@apache.org
Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/6f32e6e9
Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/6f32e6e9
Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/6f32e6e9
Branch: refs/heads/develop
Commit: 6f32e6e9776548b746f6388d25067cd17aa1898c
Parents: 8201381
Author: ricky ri...@cloudera.com
Authored: Tue Apr 28 15:58:09 2015 -0400
Committer: joewitt joew...@apache.org
Committed: Thu Jun 4 14:52:16 2015 -0400
--
.../nifi/processors/kite/ConvertJSONToAvro.java | 38 ++--
.../kite/TestJSONToAvroProcessor.java | 23 ++--
2 files changed, 47 insertions(+), 14 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/6f32e6e9/nifi/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/main/java/org/apache/nifi/processors/kite/ConvertJSONToAvro.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/main/java/org/apache/nifi/processors/kite/ConvertJSONToAvro.java
b/nifi/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/main/java/org/apache/nifi/processors/kite/ConvertJSONToAvro.java
index 78f80b9..d4cc760 100644
---
a/nifi/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/main/java/org/apache/nifi/processors/kite/ConvertJSONToAvro.java
+++
b/nifi/nifi-nar-bundles/nifi-kite-bundle/nifi-kite-processors/src/main/java/org/apache/nifi/processors/kite/ConvertJSONToAvro.java
@@ -38,6 +38,7 @@ import org.apache.nifi.processor.ProcessContext;
import org.apache.nifi.processor.ProcessSession;
import org.apache.nifi.processor.Relationship;
import org.apache.nifi.processor.exception.ProcessException;
+import org.apache.nifi.processor.io.OutputStreamCallback;
import org.apache.nifi.processor.io.StreamCallback;
import org.kitesdk.data.DatasetException;
import org.kitesdk.data.DatasetIOException;
@@ -97,22 +98,22 @@ public class ConvertJSONToAvro extends
AbstractKiteProcessor {
}
@Override
-public void onTrigger(ProcessContext context, final ProcessSession session)
+public void onTrigger(final ProcessContext context, final ProcessSession
session)
throws ProcessException {
-FlowFile flowFile = session.get();
-if (flowFile == null) {
+FlowFile successfulRecords = session.get();
+if (successfulRecords == null) {
return;
}
String schemaProperty = context.getProperty(SCHEMA)
-.evaluateAttributeExpressions(flowFile)
+.evaluateAttributeExpressions(successfulRecords)
.getValue();
final Schema schema;
try {
schema = getSchema(schemaProperty, DefaultConfiguration.get());
} catch (SchemaNotFoundException e) {
getLogger().error(Cannot find schema: + schemaProperty);
-session.transfer(flowFile, FAILURE);
+session.transfer(successfulRecords, FAILURE);
return;
}
@@ -121,21 +122,31 @@ public class ConvertJSONToAvro extends
AbstractKiteProcessor {
writer.setCodec(CodecFactory.snappyCodec());
try {
-flowFile = session.write(flowFile, new StreamCallback() {
+ successfulRecords = session.write(successfulRecords, new
StreamCallback() {
@Override
public void process(InputStream in, OutputStream out) throws
IOException {
+FlowFile failedRecords = session.create();
long written = 0L;
long errors = 0L;
+long total = 0L;
try (JSONFileReaderRecord reader = new JSONFileReader(
in, schema, Record.class)) {
reader.initialize();
try (DataFileWriterRecord w = writer.create(schema,
out)) {
while (reader.hasNext()) {
+total += 1;
try {
Record record = reader.next();
w.append(record);
written += 1;
-} catch (DatasetRecordException e) {
+} catch (final DatasetRecordException e) {
+
[jira] [Commented] (NIFI-602) Error on shutdown bin/nifi.sh stop
[ https://issues.apache.org/jira/browse/NIFI-602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573460#comment-14573460 ] Joseph Witt commented on NIFI-602: -- +1 WOOOT! Tested it in several fun ways. All smooth as heck. Great step forward for that thing. Error on shutdown bin/nifi.sh stop -- Key: NIFI-602 URL: https://issues.apache.org/jira/browse/NIFI-602 Project: Apache NiFi Issue Type: Bug Components: Configuration Reporter: Joseph Witt Fix For: 0.2.0 Attachments: 0001-NIFI-602-Ensure-we-read-all-data-from-socket-after-s.patch Davis Joki: I downloaded the RC and built from source - and then ran a quick test in the UI. Everything worked great but saw this on shutdown: davis 11:09 PM ~/dev/nifi/nifi-0.1.0-incubating/nifi-assembly/target/nifi-0.1.0-incubating-bin/nifi-0.1.0-incubating$ ./bin/nifi.sh stop Java home: /usr/lib/jvm/java-7-openjdk-amd64 NiFi home: /home/davis/dev/nifi/nifi-0.1.0-incubating/nifi-assembly/target/nifi-0.1.0-incubating-bin/nifi-0.1.0-incubating Bootstrap Config File: /home/davis/dev/nifi/nifi-0.1.0-incubating/nifi-assembly/target/nifi-0.1.0-incubating-bin/nifi-0.1.0-incubating/conf/bootstrap.conf May 08, 2015 11:09:23 PM org.apache.nifi.bootstrap.RunNiFi stop SEVERE: Failed to send shutdown command to port 42,148 due to java.net.SocketException: Socket is closed Not sure if that's really an error, could be something with my environment. I restarted a couple of times and each time I get the SEVERE log line. The port appears to be open when it is running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-628) Document steps in migrating from 0.0.x to 0.1.x
[ https://issues.apache.org/jira/browse/NIFI-628?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573467#comment-14573467 ] Joseph Witt commented on NIFI-628: -- SHould start a confluence/wiki page for Migration guides. It should explain 0.0.x to 0.1.x, 0.1.x to 0.2.x, etc... It is where we can keep this information going forward. Most often it should be very lightweight. But from 0.0.x to 0.1.x there are some important changes. Document steps in migrating from 0.0.x to 0.1.x --- Key: NIFI-628 URL: https://issues.apache.org/jira/browse/NIFI-628 Project: Apache NiFi Issue Type: Task Components: Documentation Website Reporter: Joseph Witt Assignee: Joseph Witt Priority: Critical Andrew Hulbert on the user mailing list rightly points out that some assistance is needed to for users going from 0.0.x to 0.1.x. That version bump did involve important changes to how controller services and reporting tasks are handled among a few other important options like the ability to view content and so on. Once the confluence wiki space for nifi is established https://issues.apache.org/jira/browse/INFRA-9695 this should be documented there. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
incubator-nifi git commit: NIFI-602: Ensure we read all data from socket after sending 'SHUTDOWN' command
Repository: incubator-nifi
Updated Branches:
refs/heads/develop 5eed33580 - c7c4427cb
NIFI-602: Ensure we read all data from socket after sending 'SHUTDOWN' command
Signed-off-by: joewitt joew...@apache.org
Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/c7c4427c
Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/c7c4427c
Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/c7c4427c
Branch: refs/heads/develop
Commit: c7c4427cbf964e932be32b2000df26958255b043
Parents: 5eed335
Author: Mark Payne marka...@hotmail.com
Authored: Thu Jun 4 14:10:43 2015 -0400
Committer: joewitt joew...@apache.org
Committed: Thu Jun 4 15:37:32 2015 -0400
--
.../java/org/apache/nifi/bootstrap/RunNiFi.java | 34
1 file changed, 20 insertions(+), 14 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/c7c4427c/nifi/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java
--
diff --git
a/nifi/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java
b/nifi/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java
index bb83e3d..e6b1bc5 100644
--- a/nifi/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java
+++ b/nifi/nifi-bootstrap/src/main/java/org/apache/nifi/bootstrap/RunNiFi.java
@@ -229,7 +229,10 @@ public class RunNiFi {
props.load(fis);
}
-logger.log(Level.FINE, Properties: {0}, props);
+final MapObject, Object modified = new HashMap(props);
+modified.remove(secret.key);
+logger.log(Level.FINE, Properties: {0}, modified);
+
return props;
}
@@ -250,8 +253,8 @@ public class RunNiFi {
Files.setPosixFilePermissions(statusFile.toPath(), perms);
} catch (final Exception e) {
logger.log(Level.WARNING, Failed to set permissions so that only
the owner can read status file {0};
-+ this may allows others to have access to the key needed
to communicate with NiFi.
-+ Permissions should be changed so that only the owner
can read this file, statusFile);
++ this may allows others to have access to the key needed to
communicate with NiFi.
++ Permissions should be changed so that only the owner can
read this file, statusFile);
}
try (final FileOutputStream fos = new FileOutputStream(statusFile)) {
@@ -328,8 +331,8 @@ public class RunNiFi {
boolean running = false;
String line;
try (final InputStream in = proc.getInputStream();
-final Reader streamReader = new InputStreamReader(in);
-final BufferedReader reader = new
BufferedReader(streamReader)) {
+final Reader streamReader = new InputStreamReader(in);
+final BufferedReader reader = new
BufferedReader(streamReader)) {
while ((line = reader.readLine()) != null) {
if (line.trim().startsWith(pid)) {
@@ -395,7 +398,7 @@ public class RunNiFi {
final Status status = getStatus();
if (status.isRespondingToPing()) {
logger.log(Level.INFO, Apache NiFi is currently running,
listening to Bootstrap on port {0}, PID={1},
-new Object[]{status.getPort(), status.getPid() == null ?
unknkown : status.getPid()});
+new Object[]{status.getPort(), status.getPid() == null ?
unknkown : status.getPid()});
return;
}
@@ -488,12 +491,15 @@ public class RunNiFi {
final OutputStream out = socket.getOutputStream();
out.write((SHUTDOWN_CMD + + secretKey +
\n).getBytes(StandardCharsets.UTF_8));
out.flush();
-out.close();
+socket.shutdownOutput();
final InputStream in = socket.getInputStream();
-final BufferedReader reader = new BufferedReader(new
InputStreamReader(in));
-final String response = reader.readLine();
-reader.close();
+int lastChar;
+final StringBuilder sb = new StringBuilder();
+while ((lastChar = in.read()) -1) {
+sb.append((char) lastChar);
+}
+final String response = sb.toString().trim();
logger.log(Level.FINE, Received response to SHUTDOWN command:
{0}, response);
@@ -557,7 +563,7 @@ public class RunNiFi {
final Process proc = Runtime.getRuntime().exec(new String[]{ps,
-o, pid, --no-headers, --ppid, ppid});
final ListString childPids = new ArrayList();
try (final InputStream in = proc.getInputStream();
-
[jira] [Commented] (NIFI-602) Error on shutdown bin/nifi.sh stop
[ https://issues.apache.org/jira/browse/NIFI-602?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573470#comment-14573470 ] ASF subversion and git services commented on NIFI-602: -- Commit c7c4427cbf964e932be32b2000df26958255b043 in incubator-nifi's branch refs/heads/develop from [~markap14] [ https://git-wip-us.apache.org/repos/asf?p=incubator-nifi.git;h=c7c4427 ] NIFI-602: Ensure we read all data from socket after sending 'SHUTDOWN' command Signed-off-by: joewitt joew...@apache.org Error on shutdown bin/nifi.sh stop -- Key: NIFI-602 URL: https://issues.apache.org/jira/browse/NIFI-602 Project: Apache NiFi Issue Type: Bug Components: Configuration Reporter: Joseph Witt Fix For: 0.2.0 Attachments: 0001-NIFI-602-Ensure-we-read-all-data-from-socket-after-s.patch Davis Joki: I downloaded the RC and built from source - and then ran a quick test in the UI. Everything worked great but saw this on shutdown: davis 11:09 PM ~/dev/nifi/nifi-0.1.0-incubating/nifi-assembly/target/nifi-0.1.0-incubating-bin/nifi-0.1.0-incubating$ ./bin/nifi.sh stop Java home: /usr/lib/jvm/java-7-openjdk-amd64 NiFi home: /home/davis/dev/nifi/nifi-0.1.0-incubating/nifi-assembly/target/nifi-0.1.0-incubating-bin/nifi-0.1.0-incubating Bootstrap Config File: /home/davis/dev/nifi/nifi-0.1.0-incubating/nifi-assembly/target/nifi-0.1.0-incubating-bin/nifi-0.1.0-incubating/conf/bootstrap.conf May 08, 2015 11:09:23 PM org.apache.nifi.bootstrap.RunNiFi stop SEVERE: Failed to send shutdown command to port 42,148 due to java.net.SocketException: Socket is closed Not sure if that's really an error, could be something with my environment. I restarted a couple of times and each time I get the SEVERE log line. The port appears to be open when it is running. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-633) Documentation Generation: Logger
[ https://issues.apache.org/jira/browse/NIFI-633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14573482#comment-14573482 ] Matt Gilman commented on NIFI-633: -- [~danbress] Looks good. Verified the use of the logger does not trigger a NPE. +1 Documentation Generation: Logger Key: NIFI-633 URL: https://issues.apache.org/jira/browse/NIFI-633 Project: Apache NiFi Issue Type: Bug Components: Core Framework Affects Versions: 0.1.0 Reporter: Matt Gilman Assignee: Dan Bress Priority: Minor Fix For: 0.2.0 Attachments: 0001-NIFI-633-SQUASHING.patch During the documentation generation, the processor is created and initialized. During initialization the processor is given a mock ProcessorInitializationContext which returns null when calling getLogger(). If a processor attempts to log anything without verifying the logger is not null, a NPE is thrown and documentation generation fails. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
incubator-nifi git commit: NIFI-553
Repository: incubator-nifi
Updated Branches:
refs/heads/develop c7c4427cb - aeefe73f0
NIFI-553
Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo
Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/aeefe73f
Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/aeefe73f
Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/aeefe73f
Branch: refs/heads/develop
Commit: aeefe73f0b7bdbbe18ef38457fbc3521da3fba3d
Parents: c7c4427
Author: joewitt joew...@apache.org
Authored: Thu Jun 4 15:59:06 2015 -0400
Committer: joewitt joew...@apache.org
Committed: Thu Jun 4 15:59:06 2015 -0400
--
.../org/apache/nifi/processors/standard/Base64EncodeContent.java | 2 +-
.../java/org/apache/nifi/processors/standard/DetectDuplicate.java | 2 +-
.../java/org/apache/nifi/processors/standard/EncodeContent.java| 2 +-
.../java/org/apache/nifi/processors/standard/EvaluateXQuery.java | 2 +-
4 files changed, 4 insertions(+), 4 deletions(-)
--
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/aeefe73f/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/Base64EncodeContent.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/Base64EncodeContent.java
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/Base64EncodeContent.java
index d0b78a5..9887e38 100644
---
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/Base64EncodeContent.java
+++
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/Base64EncodeContent.java
@@ -49,7 +49,7 @@ import org.apache.nifi.util.StopWatch;
@EventDriven
@SideEffectFree
@SupportsBatching
-@Tags({experimental, encode, base64})
+@Tags({encode, base64})
@CapabilityDescription(Encodes or decodes content to and from base64)
public class Base64EncodeContent extends AbstractProcessor {
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/aeefe73f/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DetectDuplicate.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DetectDuplicate.java
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DetectDuplicate.java
index 5e1fffa..b825972 100644
---
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DetectDuplicate.java
+++
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/DetectDuplicate.java
@@ -51,7 +51,7 @@ import org.apache.nifi.processor.util.StandardValidators;
@EventDriven
@SupportsBatching
-@Tags({experimental, hash, dupe, duplicate, dedupe})
+@Tags({hash, dupe, duplicate, dedupe})
@CapabilityDescription(Caches a value, computed from FlowFile attributes, for
each incoming FlowFile and determines if the cached value has already been
seen.
+ If so, routes the FlowFile to 'duplicate' with an attribute named
'original.identifier' that specifies the original FlowFile's
+ \description\, which is specified in the FlowFile Description
property. If the FlowFile is not determined to be a duplicate, the Processor
http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/aeefe73f/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java
--
diff --git
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java
index 3e26857..67c2214 100644
---
a/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java
+++
b/nifi/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/EncodeContent.java
@@ -55,7 +55,7 @@ import org.apache.nifi.util.StopWatch;
@EventDriven
@SideEffectFree
@SupportsBatching
-@Tags({experimental, encode, decode, base64, hex})
+@Tags({encode, decode, base64,
[jira] [Commented] (NIFI-656) Release Manager: Release 0.2.0
[ https://issues.apache.org/jira/browse/NIFI-656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572727#comment-14572727 ] ASF subversion and git services commented on NIFI-656: -- Commit 310ae3ebcd7dfa62a14275a0aa7b67159ab957c0 in incubator-nifi's branch refs/heads/develop from [~joewitt] [ https://git-wip-us.apache.org/repos/asf?p=incubator-nifi.git;h=310ae3e ] NIFI-656 moving from 0.1.1 to 0.2.0 to reflect new feature merged to develop Release Manager: Release 0.2.0 -- Key: NIFI-656 URL: https://issues.apache.org/jira/browse/NIFI-656 Project: Apache NiFi Issue Type: Task Components: Tools and Build Reporter: Joseph Witt Assignee: Joseph Witt Fix For: 0.2.0 This is a ticket to track general commits needed to conduct the release management activities themselves. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-655) Provide a pluggable mechanism for authentication
Mark Payne created NIFI-655: --- Summary: Provide a pluggable mechanism for authentication Key: NIFI-655 URL: https://issues.apache.org/jira/browse/NIFI-655 Project: Apache NiFi Issue Type: New Feature Components: Configuration, Core Framework, Core UI, Documentation Website Reporter: Mark Payne Fix For: 0.3.0 NiFi provides a pluggable authorization mechanism but authentication is done only via browser certificates. We should allow authentication to be pluggable as we do authorization. This could be done using JAAS, for instance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-654) Update dependency versions
[ https://issues.apache.org/jira/browse/NIFI-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572693#comment-14572693 ] Aldrin Piri commented on NIFI-654: -- Any reason why this is just the nifi-spark-receiver? Update dependency versions --- Key: NIFI-654 URL: https://issues.apache.org/jira/browse/NIFI-654 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: Joseph Witt Assignee: Joseph Witt Fix For: 0.2.0 The following output comes from running mvn versions:update-dependency-versions [INFO] --- versions-maven-plugin:2.2:display-dependency-updates (default-cli) @ nifi-spark-receiver --- [INFO] The following dependencies in Dependency Management have newer versions: [INFO] ch.qos.logback:logback-classic 1.1.2 - 1.1.3 [INFO] com.jcraft:jsch . 0.1.51 - 0.1.52 [INFO] javax.jms:javax.jms-api . 2.0 - 2.0.1 [INFO] joda-time:joda-time ... 2.7 - 2.8 [INFO] net.sf.saxon:Saxon-HE . 9.6.0-4 - 9.6.0-5 [INFO] org.apache.activemq:activemq-client . 5.10.0 - 5.11.1 [INFO] org.apache.avro:avro .. 1.7.6 - 1.7.7 [INFO] org.apache.commons:commons-lang3 3.3.2 - 3.4 [INFO] org.apache.commons:commons-pool2 2.3 - 2.4.1 [INFO] org.apache.hadoop:hadoop-annotations .. 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-client ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-common ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-hdfs . 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-mapreduce-client-core 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-yarn-api . 2.6.0 - 2.7.0 [INFO] org.apache.httpcomponents:httpclient .. 4.3.6 - 4.4.1 [INFO] org.apache.lucene:lucene-analyzers-common 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-core 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-queryparser . 4.10.3 - 5.1.0 [INFO] org.aspectj:aspectjweaver . 1.8.4 - 1.8.5 [INFO] org.slf4j:jcl-over-slf4j 1.7.10 - 1.7.12 [INFO] org.slf4j:jul-to-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:log4j-over-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-api . 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-simple .. 1.7.10 - 1.7.12 [INFO] org.springframework:spring-aop 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-beans .. 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context-support ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-core ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-expression ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-jdbc ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-tx . 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-web 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework.security:spring-security-acl ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-config ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-core ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-web ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] [INFO] artifact org.apache.spark:spark-streaming_2.10: checking for updates from central [INFO] The following dependencies in Dependencies have newer versions: [INFO] org.apache.spark:spark-streaming_2.10 . 1.2.0 - 1.3.1 [INFO] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-656) Release Manager: Release 0.2.0
Joseph Witt created NIFI-656: Summary: Release Manager: Release 0.2.0 Key: NIFI-656 URL: https://issues.apache.org/jira/browse/NIFI-656 Project: Apache NiFi Issue Type: Task Components: Tools and Build Reporter: Joseph Witt Assignee: Joseph Witt Fix For: 0.2.0 This is a ticket to track general commits needed to conduct the release management activities themselves. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-654) Update dependency versions
[ https://issues.apache.org/jira/browse/NIFI-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572720#comment-14572720 ] Joseph Witt commented on NIFI-654: -- the versions plugin used to find these seems to just list them in a growing list as it rolls through a multi module. So this just reflects that last one on the pile that I grabbed. Update dependency versions --- Key: NIFI-654 URL: https://issues.apache.org/jira/browse/NIFI-654 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: Joseph Witt Assignee: Joseph Witt Fix For: 0.2.0 The following output comes from running mvn versions:update-dependency-versions [INFO] --- versions-maven-plugin:2.2:display-dependency-updates (default-cli) @ nifi-spark-receiver --- [INFO] The following dependencies in Dependency Management have newer versions: [INFO] ch.qos.logback:logback-classic 1.1.2 - 1.1.3 [INFO] com.jcraft:jsch . 0.1.51 - 0.1.52 [INFO] javax.jms:javax.jms-api . 2.0 - 2.0.1 [INFO] joda-time:joda-time ... 2.7 - 2.8 [INFO] net.sf.saxon:Saxon-HE . 9.6.0-4 - 9.6.0-5 [INFO] org.apache.activemq:activemq-client . 5.10.0 - 5.11.1 [INFO] org.apache.avro:avro .. 1.7.6 - 1.7.7 [INFO] org.apache.commons:commons-lang3 3.3.2 - 3.4 [INFO] org.apache.commons:commons-pool2 2.3 - 2.4.1 [INFO] org.apache.hadoop:hadoop-annotations .. 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-client ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-common ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-hdfs . 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-mapreduce-client-core 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-yarn-api . 2.6.0 - 2.7.0 [INFO] org.apache.httpcomponents:httpclient .. 4.3.6 - 4.4.1 [INFO] org.apache.lucene:lucene-analyzers-common 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-core 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-queryparser . 4.10.3 - 5.1.0 [INFO] org.aspectj:aspectjweaver . 1.8.4 - 1.8.5 [INFO] org.slf4j:jcl-over-slf4j 1.7.10 - 1.7.12 [INFO] org.slf4j:jul-to-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:log4j-over-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-api . 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-simple .. 1.7.10 - 1.7.12 [INFO] org.springframework:spring-aop 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-beans .. 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context-support ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-core ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-expression ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-jdbc ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-tx . 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-web 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework.security:spring-security-acl ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-config ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-core ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-web ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] [INFO] artifact org.apache.spark:spark-streaming_2.10: checking for updates from central [INFO] The following dependencies in Dependencies have newer versions: [INFO] org.apache.spark:spark-streaming_2.10 . 1.2.0 - 1.3.1 [INFO] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[2/2] incubator-nifi git commit: NIFI-656 moving from 0.1.1 to 0.2.0 to reflect new feature merged to develop
NIFI-656 moving from 0.1.1 to 0.2.0 to reflect new feature merged to develop Project: http://git-wip-us.apache.org/repos/asf/incubator-nifi/repo Commit: http://git-wip-us.apache.org/repos/asf/incubator-nifi/commit/310ae3eb Tree: http://git-wip-us.apache.org/repos/asf/incubator-nifi/tree/310ae3eb Diff: http://git-wip-us.apache.org/repos/asf/incubator-nifi/diff/310ae3eb Branch: refs/heads/develop Commit: 310ae3ebcd7dfa62a14275a0aa7b67159ab957c0 Parents: 8980167 Author: joewitt joew...@apache.org Authored: Thu Jun 4 09:09:30 2015 -0400 Committer: joewitt joew...@apache.org Committed: Thu Jun 4 09:09:30 2015 -0400 -- nifi/nifi-api/pom.xml | 2 +- nifi/nifi-assembly/pom.xml | 12 +-- nifi/nifi-bootstrap/pom.xml | 2 +- .../nifi-data-provenance-utils/pom.xml | 2 +- .../nifi-expression-language/pom.xml| 2 +- .../nifi-commons/nifi-flowfile-packager/pom.xml | 2 +- .../nifi-hl7-query-language/pom.xml | 2 +- nifi/nifi-commons/nifi-logging-utils/pom.xml| 2 +- .../nifi-processor-utilities/pom.xml| 2 +- nifi/nifi-commons/nifi-properties/pom.xml | 2 +- nifi/nifi-commons/nifi-security-utils/pom.xml | 2 +- .../nifi-site-to-site-client/pom.xml| 4 +- nifi/nifi-commons/nifi-socket-utils/pom.xml | 2 +- nifi/nifi-commons/nifi-utils/pom.xml| 4 +- nifi/nifi-commons/nifi-web-utils/pom.xml| 2 +- nifi/nifi-commons/nifi-write-ahead-log/pom.xml | 2 +- nifi/nifi-commons/pom.xml | 2 +- nifi/nifi-docs/pom.xml | 2 +- nifi/nifi-external/nifi-spark-receiver/pom.xml | 2 +- nifi/nifi-external/pom.xml | 2 +- .../nifi-processor-bundle-archetype/pom.xml | 2 +- nifi/nifi-maven-archetypes/pom.xml | 2 +- nifi/nifi-mock/pom.xml | 2 +- .../nifi-aws-bundle/nifi-aws-nar/pom.xml| 4 +- .../nifi-aws-bundle/nifi-aws-processors/pom.xml | 2 +- nifi/nifi-nar-bundles/nifi-aws-bundle/pom.xml | 2 +- .../nifi-framework-nar/pom.xml | 2 +- .../nifi-framework/nifi-administration/pom.xml | 2 +- .../nifi-framework/nifi-client-dto/pom.xml | 2 +- .../nifi-cluster-authorization-provider/pom.xml | 2 +- .../nifi-framework/nifi-documentation/pom.xml | 2 +- .../nifi-file-authorization-provider/pom.xml| 2 +- .../nifi-framework-cluster-protocol/pom.xml | 2 +- .../nifi-framework-cluster-web/pom.xml | 2 +- .../nifi-framework-cluster/pom.xml | 2 +- .../nifi-framework-core-api/pom.xml | 2 +- .../nifi-framework/nifi-framework-core/pom.xml | 2 +- .../nifi-framework/nifi-nar-utils/pom.xml | 2 +- .../nifi-framework/nifi-resources/pom.xml | 2 +- .../nifi-framework/nifi-runtime/pom.xml | 2 +- .../nifi-framework/nifi-security/pom.xml| 2 +- .../nifi-framework/nifi-site-to-site/pom.xml| 2 +- .../nifi-framework/nifi-user-actions/pom.xml| 2 +- .../nifi-web/nifi-custom-ui-utilities/pom.xml | 2 +- .../nifi-framework/nifi-web/nifi-jetty/pom.xml | 2 +- .../nifi-web/nifi-ui-extension/pom.xml | 2 +- .../nifi-web/nifi-web-api/pom.xml | 2 +- .../nifi-web/nifi-web-content-access/pom.xml| 2 +- .../nifi-web/nifi-web-content-viewer/pom.xml| 2 +- .../nifi-web/nifi-web-docs/pom.xml | 2 +- .../nifi-web/nifi-web-error/pom.xml | 2 +- .../nifi-web-optimistic-locking/pom.xml | 2 +- .../nifi-web/nifi-web-security/pom.xml | 2 +- .../nifi-framework/nifi-web/nifi-web-ui/pom.xml | 2 +- .../nifi-framework/nifi-web/pom.xml | 12 +-- .../nifi-framework/pom.xml | 2 +- .../nifi-framework-bundle/pom.xml | 38 - .../nifi-geo-bundle/nifi-geo-nar/pom.xml| 2 +- .../nifi-geo-bundle/nifi-geo-processors/pom.xml | 2 +- nifi/nifi-nar-bundles/nifi-geo-bundle/pom.xml | 4 +- .../nifi-hadoop-bundle/nifi-hadoop-nar/pom.xml | 2 +- .../nifi-hdfs-processors/pom.xml| 2 +- .../nifi-nar-bundles/nifi-hadoop-bundle/pom.xml | 4 +- .../nifi-hadoop-libraries-nar/pom.xml | 2 +- .../nifi-hadoop-libraries-bundle/pom.xml| 2 +- .../nifi-hl7-bundle/nifi-hl7-nar/pom.xml| 4 +- .../nifi-hl7-bundle/nifi-hl7-processors/pom.xml | 4 +- nifi/nifi-nar-bundles/nifi-hl7-bundle/pom.xml | 2 +- nifi/nifi-nar-bundles/nifi-jetty-bundle/pom.xml | 2 +- .../nifi-kafka-bundle/nifi-kafka-nar/pom.xml| 2 +- .../nifi-kafka-processors/pom.xml | 2 +- nifi/nifi-nar-bundles/nifi-kafka-bundle/pom.xml | 4 +- .../nifi-kite-bundle/nifi-kite-nar/pom.xml | 2 +- .../nifi-kite-processors/pom.xml| 2 +- nifi/nifi-nar-bundles/nifi-kite-bundle/pom.xml | 4 +-
[1/2] incubator-nifi git commit: NIFI-656 moving from 0.1.1 to 0.2.0 to reflect new feature merged to develop
Repository: incubator-nifi Updated Branches: refs/heads/develop 89801675b - 310ae3ebc http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/310ae3eb/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/pom.xml -- diff --git a/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/pom.xml b/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/pom.xml index 1545245..14459fc 100644 --- a/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/pom.xml +++ b/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-persistent-provenance-repository/pom.xml @@ -17,7 +17,7 @@ parent groupIdorg.apache.nifi/groupId artifactIdnifi-provenance-repository-bundle/artifactId -version0.1.1-incubating-SNAPSHOT/version +version0.2.0-incubating-SNAPSHOT/version /parent artifactIdnifi-persistent-provenance-repository/artifactId packagingjar/packaging http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/310ae3eb/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-provenance-repository-nar/pom.xml -- diff --git a/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-provenance-repository-nar/pom.xml b/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-provenance-repository-nar/pom.xml index 87aedcc..b44b462 100644 --- a/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-provenance-repository-nar/pom.xml +++ b/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-provenance-repository-nar/pom.xml @@ -17,7 +17,7 @@ parent groupIdorg.apache.nifi/groupId artifactIdnifi-provenance-repository-bundle/artifactId -version0.1.1-incubating-SNAPSHOT/version +version0.2.0-incubating-SNAPSHOT/version /parent artifactIdnifi-provenance-repository-nar/artifactId packagingnar/packaging http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/310ae3eb/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-volatile-provenance-repository/pom.xml -- diff --git a/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-volatile-provenance-repository/pom.xml b/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-volatile-provenance-repository/pom.xml index e1f1e62..e955498 100644 --- a/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-volatile-provenance-repository/pom.xml +++ b/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/nifi-volatile-provenance-repository/pom.xml @@ -17,7 +17,7 @@ parent groupIdorg.apache.nifi/groupId artifactIdnifi-provenance-repository-bundle/artifactId -version0.1.1-incubating-SNAPSHOT/version +version0.2.0-incubating-SNAPSHOT/version /parent artifactIdnifi-volatile-provenance-repository/artifactId packagingjar/packaging http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/310ae3eb/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/pom.xml -- diff --git a/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/pom.xml b/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/pom.xml index 574beb6..be19f92 100644 --- a/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/pom.xml +++ b/nifi/nifi-nar-bundles/nifi-provenance-repository-bundle/pom.xml @@ -17,7 +17,7 @@ parent groupIdorg.apache.nifi/groupId artifactIdnifi-nar-bundles/artifactId -version0.1.1-incubating-SNAPSHOT/version +version0.2.0-incubating-SNAPSHOT/version /parent artifactIdnifi-provenance-repository-bundle/artifactId packagingpom/packaging @@ -31,12 +31,12 @@ dependency groupIdorg.apache.nifi/groupId artifactIdnifi-persistent-provenance-repository/artifactId -version0.1.1-incubating-SNAPSHOT/version +version0.2.0-incubating-SNAPSHOT/version /dependency dependency groupIdorg.apache.nifi/groupId artifactIdnifi-volatile-provenance-repository/artifactId -version0.1.1-incubating-SNAPSHOT/version +version0.2.0-incubating-SNAPSHOT/version /dependency /dependencies /dependencyManagement http://git-wip-us.apache.org/repos/asf/incubator-nifi/blob/310ae3eb/nifi/nifi-nar-bundles/nifi-social-media-bundle/nifi-social-media-nar/pom.xml -- diff --git a/nifi/nifi-nar-bundles/nifi-social-media-bundle/nifi-social-media-nar/pom.xml
[jira] [Commented] (NIFI-654) Update dependency versions
[ https://issues.apache.org/jira/browse/NIFI-654?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572725#comment-14572725 ] Aldrin Piri commented on NIFI-654: -- Yeah, got a good feel of that when I scoped out an execution of it myself against the source. It was a doozy of a listing. Update dependency versions --- Key: NIFI-654 URL: https://issues.apache.org/jira/browse/NIFI-654 Project: Apache NiFi Issue Type: Improvement Components: Core Framework, Extensions Reporter: Joseph Witt Assignee: Joseph Witt Fix For: 0.2.0 The following output comes from running mvn versions:update-dependency-versions [INFO] --- versions-maven-plugin:2.2:display-dependency-updates (default-cli) @ nifi-spark-receiver --- [INFO] The following dependencies in Dependency Management have newer versions: [INFO] ch.qos.logback:logback-classic 1.1.2 - 1.1.3 [INFO] com.jcraft:jsch . 0.1.51 - 0.1.52 [INFO] javax.jms:javax.jms-api . 2.0 - 2.0.1 [INFO] joda-time:joda-time ... 2.7 - 2.8 [INFO] net.sf.saxon:Saxon-HE . 9.6.0-4 - 9.6.0-5 [INFO] org.apache.activemq:activemq-client . 5.10.0 - 5.11.1 [INFO] org.apache.avro:avro .. 1.7.6 - 1.7.7 [INFO] org.apache.commons:commons-lang3 3.3.2 - 3.4 [INFO] org.apache.commons:commons-pool2 2.3 - 2.4.1 [INFO] org.apache.hadoop:hadoop-annotations .. 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-client ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-common ... 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-hdfs . 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-mapreduce-client-core 2.6.0 - 2.7.0 [INFO] org.apache.hadoop:hadoop-yarn-api . 2.6.0 - 2.7.0 [INFO] org.apache.httpcomponents:httpclient .. 4.3.6 - 4.4.1 [INFO] org.apache.lucene:lucene-analyzers-common 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-core 4.10.3 - 5.1.0 [INFO] org.apache.lucene:lucene-queryparser . 4.10.3 - 5.1.0 [INFO] org.aspectj:aspectjweaver . 1.8.4 - 1.8.5 [INFO] org.slf4j:jcl-over-slf4j 1.7.10 - 1.7.12 [INFO] org.slf4j:jul-to-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:log4j-over-slf4j .. 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-api . 1.7.10 - 1.7.12 [INFO] org.slf4j:slf4j-simple .. 1.7.10 - 1.7.12 [INFO] org.springframework:spring-aop 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-beans .. 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-context-support ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-core ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-expression ... [INFO] 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-jdbc ... 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-tx . 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework:spring-web 4.1.4.RELEASE - 4.1.6.RELEASE [INFO] org.springframework.security:spring-security-acl ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-config ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-core ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] org.springframework.security:spring-security-web ... [INFO] 3.2.5.RELEASE - 4.0.1.RELEASE [INFO] [INFO] artifact org.apache.spark:spark-streaming_2.10: checking for updates from central [INFO] The following dependencies in Dependencies have newer versions: [INFO] org.apache.spark:spark-streaming_2.10 . 1.2.0 - 1.3.1 [INFO] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-642) Eliminate hardcoded HDFS compression codec classnames.
[ https://issues.apache.org/jira/browse/NIFI-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-642: - Fix Version/s: (was: 0.3.0) 0.2.0 Eliminate hardcoded HDFS compression codec classnames. -- Key: NIFI-642 URL: https://issues.apache.org/jira/browse/NIFI-642 Project: Apache NiFi Issue Type: Improvement Reporter: Tim Reardon Priority: Minor Fix For: 0.2.0 Available HDFS compression codec classes were hardcoded as part of NIFI-600. This ticket will allow PutHDFS to discover the available codecs via CompressionCodecFactory, and allow GetHDFS to choose the codec to use based on file extension. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-601) Use dynamic properties as environment variables in ExecuteStreamCommand
[ https://issues.apache.org/jira/browse/NIFI-601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-601: - Fix Version/s: (was: 0.3.0) 0.2.0 Use dynamic properties as environment variables in ExecuteStreamCommand --- Key: NIFI-601 URL: https://issues.apache.org/jira/browse/NIFI-601 Project: Apache NiFi Issue Type: New Feature Affects Versions: 0.1.0 Reporter: Ricky Saltzer Assignee: Ricky Saltzer Priority: Minor Fix For: 0.2.0 Attachments: NIFI-601.1.patch This is already being done with the ExecuteProcess processor, but not with the ExecuteStreamCommand. I'm working on adding this in now, and will submit patch here. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-626) ExecuteSQL processor for executing arbitrary SQL queries
[
https://issues.apache.org/jira/browse/NIFI-626?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Joseph Witt updated NIFI-626:
-
Fix Version/s: (was: 0.3.0)
0.2.0
ExecuteSQL processor for executing arbitrary SQL queries
Key: NIFI-626
URL: https://issues.apache.org/jira/browse/NIFI-626
Project: Apache NiFi
Issue Type: Sub-task
Affects Versions: 0.2.0
Reporter: Toivo Adams
Assignee: Toivo Adams
Labels: patch
Fix For: 0.2.0
Attachments: NIFI-626_25may2015.patch, NIFI-626_26may2015.patch,
NIFI-626_31may2015.patch
For example query can be:
SELECT * FROM orders WHERE orderId = '${order.id}'
where ${order.id} is FlowFile attribute.
Result will serialized using Avro.
Avro gives as possibility to have query Resultset metadata (column name and
types) included in FlowFile.
Also Avro should allow streaming.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-583) Allow ExecuteProcess to consume an incoming flowfile
[ https://issues.apache.org/jira/browse/NIFI-583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-583: - Fix Version/s: (was: 0.3.0) 0.2.0 Allow ExecuteProcess to consume an incoming flowfile Key: NIFI-583 URL: https://issues.apache.org/jira/browse/NIFI-583 Project: Apache NiFi Issue Type: Improvement Affects Versions: 0.1.0 Reporter: Ricky Saltzer Assignee: Ricky Saltzer Fix For: 0.2.0 Attachments: NIFI-583.1.patch In some cases it would be really nice to allow a FlowFile to trigger an OS action. For instance, after a daily dump of data is written to an Impala table in HDFS, I would like to execute a refresh on the table via the shell. As it stands, the ExecuteProcess processor will allow a FlowFile in a connection to trigger execution, but unless your connection has an expiration set, the FlowFile will stay there indefinitely. The main issue here is that it will continue to re-execute your ExecuteProcess processor over and over. As far as I know, there's only two clear ways around this. (1) - you can use the ExecuteStreamCommand, instead, but *only* if that command can properly handle STDIN. (2) - you can set your ExecuteProcess processor to execute on a schedule (e.g. 1 per minute) and expire the FlowFile before it can re-execute (e.g. 10 seconds). It would be useful if the ExecuteProcess processor consumed the FlowFile, and passed it through a passthrough relationship of some kind. A second option would be to make it configurable (false by default) to drop the FlowFile, or to pass it through a second relationship, that way it doesn't break anyone's current pipelines. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-413) KafkaPut should support compression option
[ https://issues.apache.org/jira/browse/NIFI-413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Witt updated NIFI-413: - Fix Version/s: (was: 0.3.0) 0.2.0 KafkaPut should support compression option -- Key: NIFI-413 URL: https://issues.apache.org/jira/browse/NIFI-413 Project: Apache NiFi Issue Type: Improvement Reporter: Michael Braun Priority: Minor Labels: beginner, newbie Fix For: 0.2.0 Kafka supports compression out of the box (GZIP, Snappy with another dependency) but the KafkaPut operator does not expose the option. It would be nice if this was exposed in processor settings so processing could benefit from compression. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-376) New NiFi NAR plugin feature
[ https://issues.apache.org/jira/browse/NIFI-376?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572749#comment-14572749 ] Joseph Witt commented on NIFI-376: -- we should do these together as they will mean putting out a new nifi-parent and nifi-nar-maven-plugin release. New NiFi NAR plugin feature --- Key: NIFI-376 URL: https://issues.apache.org/jira/browse/NIFI-376 Project: Apache NiFi Issue Type: Improvement Components: Tools and Build Reporter: Matt Gilman Assignee: Matt Gilman Priority: Minor Fix For: 0.3.0 Attachments: 0001-NIFI-376.patch Provide a goal for determining the dependencies provided by a nar dependency. As developers begin to write their own nars they will need a way of knowing what dependencies will be included in their classpath automatically based on the configured nar hierarchy. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-609) update to apache parent version 17
[ https://issues.apache.org/jira/browse/NIFI-609?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572752#comment-14572752 ] Joseph Witt commented on NIFI-609: -- we should do these together as they will mean putting out a new nifi-parent and nifi-nar-maven-plugin release. update to apache parent version 17 -- Key: NIFI-609 URL: https://issues.apache.org/jira/browse/NIFI-609 Project: Apache NiFi Issue Type: Task Components: Tools and Build Affects Versions: nifi-parent-1.0.0 Reporter: Sean Busbey Assignee: Sean Busbey Fix For: 0.3.0 Attachments: NIFI-609.1.patch we should update to version 17 of the apache parent pom so we can get MPOM-47 and do .tar.gz source release artifacts. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-321) New Report Generator Processor
[ https://issues.apache.org/jira/browse/NIFI-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572766#comment-14572766 ] Toivo Adams commented on NIFI-321: -- I’d like to have open source community version (official release). More users mean more features, better implementation, more tested, less bugs, etc. But it seems there is little interest. NiFi current footprint is already fairly large and Birt processor I also very big. So I am not sure is it wise to include Birt processor in NiFi standard distribution. Maybe NiFi can be splitted to Core NiFi which includes only most common processors and services. And all other is distributed separately using separate packages? Maybe processors with similar use cases can be grouped together? Thanks Toivo New Report Generator Processor -- Key: NIFI-321 URL: https://issues.apache.org/jira/browse/NIFI-321 Project: Apache NiFi Issue Type: New Feature Components: Extensions Reporter: Toivo Adams Priority: Minor Labels: processor It would be very useful for generating different business reports. Underlying report generator is Birt engine. http://eclipse.org/birt/ Input: data in XML Parameter 1: report design file name Parameter 2: desired output format (PDF, Word doc, Excel. Html, etc) Output: report in desired format TransformXml can be used as starting point. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-321) New Report Generator Processor
[ https://issues.apache.org/jira/browse/NIFI-321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14572770#comment-14572770 ] Joseph Witt commented on NIFI-321: -- Toivo Yeah i really agree with you there about the idea of splitting core nifi and extension bundles up. Not entirely clear how best to pursue this in the apache context to me yet. But i do think it is the right path. Was thinking that if we go to TLP status that we could then more readily have child projects and we could structure these extensions in that manner (one or more child projects). We can probably figure out a way to establish a registry or some mechanism similar to 'spark packages' concept. Thanks Joe New Report Generator Processor -- Key: NIFI-321 URL: https://issues.apache.org/jira/browse/NIFI-321 Project: Apache NiFi Issue Type: New Feature Components: Extensions Reporter: Toivo Adams Priority: Minor Labels: processor It would be very useful for generating different business reports. Underlying report generator is Birt engine. http://eclipse.org/birt/ Input: data in XML Parameter 1: report design file name Parameter 2: desired output format (PDF, Word doc, Excel. Html, etc) Output: report in desired format TransformXml can be used as starting point. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
