[jira] [Created] (NIFI-2851) Improve performance of SplitText
Mark Payne created NIFI-2851: Summary: Improve performance of SplitText Key: NIFI-2851 URL: https://issues.apache.org/jira/browse/NIFI-2851 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Mark Payne Assignee: Mark Payne Fix For: 1.1.0 SplitText is fairly CPU-intensive and quite slow. A simple flow that splits a 1.4 million line text file into 5k line chunks and then splits those 5k line chunks into 1 line chunks is only capable of pushing through about 10k lines per second. This equates to about 10 MB/sec. JVisualVM shows that the majority of the time is spent in the locateSplitPoint() method. Isolating this code and inspecting how it works, and using some micro-benchmarking, it appears that if we refactor the calls to InputStream.read() to instead read into a byte array, we can improve performance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2850) Provide ability for a FlowFile to be migrated from one Process Session to another
Mark Payne created NIFI-2850: Summary: Provide ability for a FlowFile to be migrated from one Process Session to another Key: NIFI-2850 URL: https://issues.apache.org/jira/browse/NIFI-2850 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Reporter: Mark Payne Assignee: Mark Payne Fix For: 1.1.0 Currently, the MergeContent processor creates a separate ProcessSession for each FlowFile that it pulls. This is done so that we can ensure that we can commit all Process Sessions when a bin is full. Unfortunately, this means that MergeContent is required to call ProcessSession.get() many times, which adds a lot of contention on the FlowFile Queue. If we allow FlowFiles to be migrated from 1 session to another, we can have a session per bin, and then use ProcessSession.get(100) to greatly reduce lock contention. This will likely have benefits in other processors as well. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2341) Create a processor to parse logs formated using CEF
[ https://issues.apache.org/jira/browse/NIFI-2341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15537363#comment-15537363 ] ASF GitHub Bot commented on NIFI-2341: -- Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/785 Sounds good thanks, will take a look soon > Create a processor to parse logs formated using CEF > --- > > Key: NIFI-2341 > URL: https://issues.apache.org/jira/browse/NIFI-2341 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Andre >Assignee: Andre > Fix For: 1.1.0 > > > As NiFi continue to increase its abilities to complement SIEM, Splunk and ELK > deployments, a number of users will be looking to parse CEF formatted > logs[1][2]. > CEF is a format specified by Arcsight (now part of HPE) and is described in > detail in here: > https://www.protect724.hpe.com/docs/DOC-1072 > [1] > http://apache-nifi.1125220.n5.nabble.com/Suggestion-of-processors-td9795.html > [2] > https://community.hortonworks.com/questions/43185/which-processor-is-used-to-parse-cef-format-logs.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2341) Create a processor to parse logs formated using CEF
[ https://issues.apache.org/jira/browse/NIFI-2341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15537361#comment-15537361 ] ASF GitHub Bot commented on NIFI-2341: -- Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/785 @mattyb149 all feedback addressed. this should be ready for review > Create a processor to parse logs formated using CEF > --- > > Key: NIFI-2341 > URL: https://issues.apache.org/jira/browse/NIFI-2341 > Project: Apache NiFi > Issue Type: Improvement >Reporter: Andre >Assignee: Andre > Fix For: 1.1.0 > > > As NiFi continue to increase its abilities to complement SIEM, Splunk and ELK > deployments, a number of users will be looking to parse CEF formatted > logs[1][2]. > CEF is a format specified by Arcsight (now part of HPE) and is described in > detail in here: > https://www.protect724.hpe.com/docs/DOC-1072 > [1] > http://apache-nifi.1125220.n5.nabble.com/Suggestion-of-processors-td9795.html > [2] > https://community.hortonworks.com/questions/43185/which-processor-is-used-to-parse-cef-format-logs.html -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #785: NIFI-2341 - Introduce ParseCEF processor
Github user mattyb149 commented on the issue: https://github.com/apache/nifi/pull/785 Sounds good thanks, will take a look soon --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #785: NIFI-2341 - Introduce ParseCEF processor
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/785 @mattyb149 all feedback addressed. this should be ready for review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15537340#comment-15537340 ] ASF GitHub Bot commented on NIFI-2709: -- Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1083 @bbende code should pass contrib-check and be ready for review. > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Assignee: Andre >Priority: Minor > Attachments: 1083.patch, screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #1083: NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to genera...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1083 @bbende code should pass contrib-check and be ready for review. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2848) Queues aren't fairly drained when leading to a single component
[ https://issues.apache.org/jira/browse/NIFI-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15537055#comment-15537055 ] Joseph Gresock commented on NIFI-2848: -- You're right, it appears to be the same issue. > Queues aren't fairly drained when leading to a single component > --- > > Key: NIFI-2848 > URL: https://issues.apache.org/jira/browse/NIFI-2848 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.0.0, 0.7.0 >Reporter: Joseph Gresock > Attachments: Backpressure_prioritization_test.xml, dominate_1.png, > dominate_2.png, queue_drain.png > > > Consider the scenario where multiple queues lead to a single component and > all of them are full due to back pressure. With the attached template, it is > easily observable that once a single queue starts to drain due to relieved > back pressure, it will continue to drain as long as it has incoming flow > files. This means that if there's a constant flow of incoming flow files to > this queue, the other queues will never be drained (at least, that's my > theory based on several hours of observation). > To reproduce this: > # Load the template into NiFi 1.0.0 > # Play all three GenerateFlowFile processors, but not the UpdateAttribute > processor (this simulates backpressure). Wait until each queue has 1,000 > flow files (max backpressure) > # Stop the GenerateFlowFile processors, and play the UpdateAttribute > processor (this relieves the backpressure) > # Observe which queue has started to drain, and start its GenerateFlowFile > processor > # Observe that the other two queues remain full indefinitely, while the > draining queue continues to replenish and be drained indefinitely -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2774) ConsumeJMS processor losses messages on NiFi restart
[
https://issues.apache.org/jira/browse/NIFI-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536978#comment-15536978
]
ASF GitHub Bot commented on NIFI-2774:
--
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1036#discussion_r81410549
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/ConsumeJMS.java
---
@@ -54,8 +61,31 @@
@SeeAlso(value = { PublishJMS.class, JMSConnectionFactoryProvider.class })
public class ConsumeJMS extends AbstractJMSProcessor {
+static final AllowableValue AUTO_ACK = new
AllowableValue(String.valueOf(Session.AUTO_ACKNOWLEDGE),
+"AUTO_ACKNOWLEDGE",
+"Automatically acknowledges a client's receipt of a message,
regardless if NiFi session has been commited. "
++ "Can result in data loss in the event where NiFi
abruptly stopped before session was commited.");
+
+static final AllowableValue CLIENT_ACK = new
AllowableValue(String.valueOf(Session.CLIENT_ACKNOWLEDGE),
+"CLIENT_ACKNOWLEDGE",
+"(DEFAULT) Manually acknowledges a client's receipt of a
message after NiFi Session was commited, thus ensuring no data loss");
+
+static final AllowableValue DUPS_OK = new
AllowableValue(String.valueOf(Session.DUPS_OK_ACKNOWLEDGE),
+"DUPS_OK_ACKNOWLEDGE",
+"This acknowledgment mode instructs the session to lazily
acknowledge the delivery of messages. May result uin both data "
--- End diff --
Spelling "uin"
> ConsumeJMS processor losses messages on NiFi restart
>
>
> Key: NIFI-2774
> URL: https://issues.apache.org/jira/browse/NIFI-2774
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Christopher McDermott
>Assignee: Oleg Zhurakousky
>Priority: Critical
> Fix For: 1.1.0, 0.8.0
>
> Attachments: 2774.patch
>
>
> ConsumeJMS processor uses auto-acknowledge mode. Unlike the deprecated
> GetJMSQueue processor it does not provide a way to specify a different ACK
> mode (i.e. client-acknowledge.) Using auto-acknowledge, acknowledges message
> receipt from JMS *before* the messages are actually added to the flow. This
> leads to data-loss on NiFi stop (or crash.)
> I believe the fix for this is to allow the user to specify the ACK mode in
> the processor configuration like is allowed by the GetJMSQueue processor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2774) ConsumeJMS processor losses messages on NiFi restart
[
https://issues.apache.org/jira/browse/NIFI-2774?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536979#comment-15536979
]
ASF GitHub Bot commented on NIFI-2774:
--
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1036#discussion_r81411506
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/ConsumeJMS.java
---
@@ -116,18 +154,26 @@ protected JMSConsumer
finishBuildingTargetResource(JmsTemplate jmsTemplate) {
}
/**
+ *
+ */
+@Override
+protected List getSupportedPropertyDescriptors() {
+List pd = new
ArrayList<>(super.getSupportedPropertyDescriptors());
+pd.add(ACKNOWLEDGEMENT_MODE);
+return pd;
--- End diff --
Not an incredibly big deal, but getSupportedPropertyDescriptors() is called
very often (most often in validation during a UI Refresh) which causes lots of
ArrayLists to be created. I prefer the way DistributeLoad handles this, by
creating one ArrayList and reusing it, while also using an AtomicBoolean to
know when it should be recreated.
> ConsumeJMS processor losses messages on NiFi restart
>
>
> Key: NIFI-2774
> URL: https://issues.apache.org/jira/browse/NIFI-2774
> Project: Apache NiFi
> Issue Type: Bug
> Components: Core Framework
>Affects Versions: 1.0.0, 0.7.0
>Reporter: Christopher McDermott
>Assignee: Oleg Zhurakousky
>Priority: Critical
> Fix For: 1.1.0, 0.8.0
>
> Attachments: 2774.patch
>
>
> ConsumeJMS processor uses auto-acknowledge mode. Unlike the deprecated
> GetJMSQueue processor it does not provide a way to specify a different ACK
> mode (i.e. client-acknowledge.) Using auto-acknowledge, acknowledges message
> receipt from JMS *before* the messages are actually added to the flow. This
> leads to data-loss on NiFi stop (or crash.)
> I believe the fix for this is to allow the user to specify the ACK mode in
> the processor configuration like is allowed by the GetJMSQueue processor.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] nifi pull request #1036: NIFI-2774 added configurable QoS options
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1036#discussion_r81410549
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/ConsumeJMS.java
---
@@ -54,8 +61,31 @@
@SeeAlso(value = { PublishJMS.class, JMSConnectionFactoryProvider.class })
public class ConsumeJMS extends AbstractJMSProcessor {
+static final AllowableValue AUTO_ACK = new
AllowableValue(String.valueOf(Session.AUTO_ACKNOWLEDGE),
+"AUTO_ACKNOWLEDGE",
+"Automatically acknowledges a client's receipt of a message,
regardless if NiFi session has been commited. "
++ "Can result in data loss in the event where NiFi
abruptly stopped before session was commited.");
+
+static final AllowableValue CLIENT_ACK = new
AllowableValue(String.valueOf(Session.CLIENT_ACKNOWLEDGE),
+"CLIENT_ACKNOWLEDGE",
+"(DEFAULT) Manually acknowledges a client's receipt of a
message after NiFi Session was commited, thus ensuring no data loss");
+
+static final AllowableValue DUPS_OK = new
AllowableValue(String.valueOf(Session.DUPS_OK_ACKNOWLEDGE),
+"DUPS_OK_ACKNOWLEDGE",
+"This acknowledgment mode instructs the session to lazily
acknowledge the delivery of messages. May result uin both data "
--- End diff --
Spelling "uin"
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[GitHub] nifi pull request #1036: NIFI-2774 added configurable QoS options
Github user mosermw commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1036#discussion_r81411506
--- Diff:
nifi-nar-bundles/nifi-jms-bundle/nifi-jms-processors/src/main/java/org/apache/nifi/jms/processors/ConsumeJMS.java
---
@@ -116,18 +154,26 @@ protected JMSConsumer
finishBuildingTargetResource(JmsTemplate jmsTemplate) {
}
/**
+ *
+ */
+@Override
+protected List getSupportedPropertyDescriptors() {
+List pd = new
ArrayList<>(super.getSupportedPropertyDescriptors());
+pd.add(ACKNOWLEDGEMENT_MODE);
+return pd;
--- End diff --
Not an incredibly big deal, but getSupportedPropertyDescriptors() is called
very often (most often in validation during a UI Refresh) which causes lots of
ArrayLists to be created. I prefer the way DistributeLoad handles this, by
creating one ArrayList and reusing it, while also using an AtomicBoolean to
know when it should be recreated.
---
If your project is set up for it, you can reply to this email and have your
reply appear on GitHub as well. If your project does not have this feature
enabled and wishes so, or if the feature is enabled but not working, please
contact infrastructure at infrastruct...@apache.org or file a JIRA ticket
with INFRA.
---
[jira] [Commented] (NIFI-2603) Bringing Some UI Color Back
[ https://issues.apache.org/jira/browse/NIFI-2603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536855#comment-15536855 ] Peter Wicks commented on NIFI-2603: --- Sounds good. We've been running a modified 1.0 with my UI Color branch merged into it. I've found that in many cases I agree with some of the guidance, like getting rid of color on mouseovers, they feel more distracting then anything now. > Bringing Some UI Color Back > --- > > Key: NIFI-2603 > URL: https://issues.apache.org/jira/browse/NIFI-2603 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Peter Wicks >Priority: Minor > > In the new 1.0 UI all of the colors associated with status (except the orange > triangle) are gone; replaced with a dark gray color. > I propose bringing back color. The screenshots are in the format of before > on the top and after on the bottom, except were labeled in the picture itself: > - Top Status Menu: https://goo.gl/photos/se1JnvhRwU7N4Fap7 > - Process Group: https://goo.gl/photos/dqjG4KvC6xqxQfgT7 > - Processes (Running/Stopped/Invalid): > https://goo.gl/photos/dSS8vgE2RkrXtc77A > - Operate Play/Stop buttons (only on mouse hover): > https://goo.gl/photos/Am5SUEEn7G9RjmMe6 > - Processor/Processor Group Context Menu: > https://goo.gl/photos/Jq3qFg4ezaN91qms5 > This is not a "100% done, I've covered everything" before/after list. I know > I need to do the NiFi summary page also at the minimum. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi-minifi pull request #40: MINIFI-46 - Supporting multiple relationships ...
GitHub user brosander opened a pull request: https://github.com/apache/nifi-minifi/pull/40 MINIFI-46 - Supporting multiple relationships for MiNiFi connections You can merge this pull request into a Git repository by running: $ git pull https://github.com/brosander/nifi-minifi MINIFI-46 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi-minifi/pull/40.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #40 commit 7e423ea3eff428a706dbddfe96ed6c4ad3ba9217 Author: Bryan Rosander Date: 2016-09-30T13:32:40Z MINIFI-46 - Supporting multiple relationships for MiNiFi connections --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-766) UI should indicate when backpressure is configured for a Connection
[ https://issues.apache.org/jira/browse/NIFI-766?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536797#comment-15536797 ] ASF GitHub Bot commented on NIFI-766: - Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/1080 @pvillard31, Following some of the discussion on the mailing list recently and the original commentary on the JIRA [1], I'd like to propose shifting from an icon based approach when back pressure is engaged to a progress bar style approach where the user can quickly get a sense of how 'full' the queue is currently.  The mockup shows various configuration scenarios. - Expiration (clock icon) - Object and Data size backpressure (left and right bars respectively) - Empty bar - backpressure not configured - Partial bar - backpressure configured, partial represent precentage full - Full solid bar - backpressure engaged If you're up for some additional cycles on this PR I'd be happy to work through the changes with you. If not, I'm happy to take what you started and continue toward the mockup above. Thanks! [1] https://issues.apache.org/jira/browse/NIFI-766 > UI should indicate when backpressure is configured for a Connection > --- > > Key: NIFI-766 > URL: https://issues.apache.org/jira/browse/NIFI-766 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework, Core UI >Reporter: Mark Payne >Assignee: Pierre Villard > Fix For: 1.1.0 > > Attachments: backpressure.png, backpressure_and_expiration.png, > normal.png > > > It is sometimes unclear why a Processor is not running, if it is due to > backpressure. Recommend we add an icon to the Connection label to indicate > that backpressure is configured. If backpressure is "applied" (i.e., the > backpressure threshold has been reached), that icon should be highlighted > somehow. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #1080: NIFI-766 Added icon on connection when backpressure is ena...
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/1080 @pvillard31, Following some of the discussion on the mailing list recently and the original commentary on the JIRA [1], I'd like to propose shifting from an icon based approach when back pressure is engaged to a progress bar style approach where the user can quickly get a sense of how 'full' the queue is currently.  The mockup shows various configuration scenarios. - Expiration (clock icon) - Object and Data size backpressure (left and right bars respectively) - Empty bar - backpressure not configured - Partial bar - backpressure configured, partial represent precentage full - Full solid bar - backpressure engaged If you're up for some additional cycles on this PR I'd be happy to work through the changes with you. If not, I'm happy to take what you started and continue toward the mockup above. Thanks! [1] https://issues.apache.org/jira/browse/NIFI-766 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2603) Bringing Some UI Color Back
[ https://issues.apache.org/jira/browse/NIFI-2603?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536759#comment-15536759 ] Rob Moran commented on NIFI-2603: - Sorry just getting back on this. I am working on some ideas and will attach here when complete. > Bringing Some UI Color Back > --- > > Key: NIFI-2603 > URL: https://issues.apache.org/jira/browse/NIFI-2603 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Peter Wicks >Priority: Minor > > In the new 1.0 UI all of the colors associated with status (except the orange > triangle) are gone; replaced with a dark gray color. > I propose bringing back color. The screenshots are in the format of before > on the top and after on the bottom, except were labeled in the picture itself: > - Top Status Menu: https://goo.gl/photos/se1JnvhRwU7N4Fap7 > - Process Group: https://goo.gl/photos/dqjG4KvC6xqxQfgT7 > - Processes (Running/Stopped/Invalid): > https://goo.gl/photos/dSS8vgE2RkrXtc77A > - Operate Play/Stop buttons (only on mouse hover): > https://goo.gl/photos/Am5SUEEn7G9RjmMe6 > - Processor/Processor Group Context Menu: > https://goo.gl/photos/Jq3qFg4ezaN91qms5 > This is not a "100% done, I've covered everything" before/after list. I know > I need to do the NiFi summary page also at the minimum. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2848) Queues aren't fairly drained when leading to a single component
[ https://issues.apache.org/jira/browse/NIFI-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536700#comment-15536700 ] Pierre Villard commented on NIFI-2848: -- After looking into it, I believe this could be related to NIFI-2751. Indeed in processors such as UpdateAttribute or LogAttribute we are getting batch of flow files. Will look further into it. > Queues aren't fairly drained when leading to a single component > --- > > Key: NIFI-2848 > URL: https://issues.apache.org/jira/browse/NIFI-2848 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.0.0, 0.7.0 >Reporter: Joseph Gresock > Attachments: Backpressure_prioritization_test.xml, dominate_1.png, > dominate_2.png, queue_drain.png > > > Consider the scenario where multiple queues lead to a single component and > all of them are full due to back pressure. With the attached template, it is > easily observable that once a single queue starts to drain due to relieved > back pressure, it will continue to drain as long as it has incoming flow > files. This means that if there's a constant flow of incoming flow files to > this queue, the other queues will never be drained (at least, that's my > theory based on several hours of observation). > To reproduce this: > # Load the template into NiFi 1.0.0 > # Play all three GenerateFlowFile processors, but not the UpdateAttribute > processor (this simulates backpressure). Wait until each queue has 1,000 > flow files (max backpressure) > # Stop the GenerateFlowFile processors, and play the UpdateAttribute > processor (this relieves the backpressure) > # Observe which queue has started to drain, and start its GenerateFlowFile > processor > # Observe that the other two queues remain full indefinitely, while the > draining queue continues to replenish and be drained indefinitely -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2756) nifi-processor-bundle-archetype lacks displayName property
[
https://issues.apache.org/jira/browse/NIFI-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536647#comment-15536647
]
ASF GitHub Bot commented on NIFI-2756:
--
Github user asfgit closed the pull request at:
https://github.com/apache/nifi/pull/1004
> nifi-processor-bundle-archetype lacks displayName property
> --
>
> Key: NIFI-2756
> URL: https://issues.apache.org/jira/browse/NIFI-2756
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Andre
>Assignee: Andre
> Fix For: 1.1.0
>
>
> When using {{nifi-processor-bundle-archetype}} to create a new bundle, the
> resulting code lacks displayName leading to a number of PR and contributions
> to arrive to peer review without displayName configured
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (NIFI-2756) nifi-processor-bundle-archetype lacks displayName property
[
https://issues.apache.org/jira/browse/NIFI-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Bende updated NIFI-2756:
--
Fix Version/s: 1.1.0
> nifi-processor-bundle-archetype lacks displayName property
> --
>
> Key: NIFI-2756
> URL: https://issues.apache.org/jira/browse/NIFI-2756
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Andre
>Assignee: Andre
> Fix For: 1.1.0
>
>
> When using {{nifi-processor-bundle-archetype}} to create a new bundle, the
> resulting code lacks displayName leading to a number of PR and contributions
> to arrive to peer review without displayName configured
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (NIFI-2756) nifi-processor-bundle-archetype lacks displayName property
[
https://issues.apache.org/jira/browse/NIFI-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bryan Bende resolved NIFI-2756.
---
Resolution: Fixed
> nifi-processor-bundle-archetype lacks displayName property
> --
>
> Key: NIFI-2756
> URL: https://issues.apache.org/jira/browse/NIFI-2756
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Andre
>Assignee: Andre
> Fix For: 1.1.0
>
>
> When using {{nifi-processor-bundle-archetype}} to create a new bundle, the
> resulting code lacks displayName leading to a number of PR and contributions
> to arrive to peer review without displayName configured
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] nifi pull request #1004: NIFI-2756 - Add displayName to maven archetypes
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1004 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2756) nifi-processor-bundle-archetype lacks displayName property
[
https://issues.apache.org/jira/browse/NIFI-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536645#comment-15536645
]
ASF subversion and git services commented on NIFI-2756:
---
Commit 2054888fb1cbb442a70e3677139dc7a26348671d in nifi's branch
refs/heads/master from Andre F de Miranda
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=2054888 ]
NIFI-2756 - Add displayName to maven archetypes and remove unused import
- Code generated generated by the NiFi archetypes should pass contrib-check by
default
This closes #1004.
Signed-off-by: Bryan Bende
> nifi-processor-bundle-archetype lacks displayName property
> --
>
> Key: NIFI-2756
> URL: https://issues.apache.org/jira/browse/NIFI-2756
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Andre
>Assignee: Andre
>
> When using {{nifi-processor-bundle-archetype}} to create a new bundle, the
> resulting code lacks displayName leading to a number of PR and contributions
> to arrive to peer review without displayName configured
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2756) nifi-processor-bundle-archetype lacks displayName property
[
https://issues.apache.org/jira/browse/NIFI-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536456#comment-15536456
]
ASF GitHub Bot commented on NIFI-2756:
--
Github user trixpan commented on the issue:
https://github.com/apache/nifi/pull/1004
@bbende hopefully the second commit should address your feedback.
May I ask you to test and if ok, merge into master?
Kind regards
> nifi-processor-bundle-archetype lacks displayName property
> --
>
> Key: NIFI-2756
> URL: https://issues.apache.org/jira/browse/NIFI-2756
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Andre
>Assignee: Andre
>
> When using {{nifi-processor-bundle-archetype}} to create a new bundle, the
> resulting code lacks displayName leading to a number of PR and contributions
> to arrive to peer review without displayName configured
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] nifi issue #1004: NIFI-2756 - Add displayName to maven archetypes
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1004 @bbende hopefully the second commit should address your feedback. May I ask you to test and if ok, merge into master? Kind regards --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1024: Adding EL support to TailFile processor
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1024 @apsaltis is the PR still required? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2685) Builds fails due to "java.awt.AWTError: Can't connect to X11 window server "
[
https://issues.apache.org/jira/browse/NIFI-2685?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536366#comment-15536366
]
Andre commented on NIFI-2685:
-
[~tkurc]
may I merge?
Cheers
> Builds fails due to "java.awt.AWTError: Can't connect to X11 window server "
>
>
> Key: NIFI-2685
> URL: https://issues.apache.org/jira/browse/NIFI-2685
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Andre
>Assignee: Andre
>Priority: Trivial
>
> When building NiFi on a machine with ssh X11 forwarding enabled, maven may
> fail to build due to the following error:
> {code}
> testResize(org.apache.nifi.processors.image.TestResizeImage) Time elapsed:
> 2.22 sec <<< FAILURE!
> java.lang.AssertionError: java.awt.AWTError: Can't connect to X11 window
> server using 'localhost:12.0' as the value of the DISPLAY variable.
> at
> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:192)
> at
> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:151)
> at
> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:146)
> at
> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:141)
> at
> org.apache.nifi.util.StandardProcessorTestRunner.run(StandardProcessorTestRunner.java:136)
> at
> org.apache.nifi.processors.image.TestResizeImage.testResize(TestResizeImage.java:44)
> {code}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] nifi issue #1083: NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to genera...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1083 User reveals patch works as expected: https://community.hortonworks.com/comments/59274/view.html --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536336#comment-15536336 ] ASF GitHub Bot commented on NIFI-2709: -- Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1083 User reveals patch works as expected: https://community.hortonworks.com/comments/59274/view.html > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Assignee: Andre >Priority: Minor > Attachments: 1083.patch, screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2825) Remote Process Group should use num of queued flow files in remote nodes to distribute load
[ https://issues.apache.org/jira/browse/NIFI-2825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-2825: Labels: Cluster (was: ) > Remote Process Group should use num of queued flow files in remote nodes to > distribute load > --- > > Key: NIFI-2825 > URL: https://issues.apache.org/jira/browse/NIFI-2825 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 > Environment: Clustered >Reporter: Koji Kawamura >Assignee: Koji Kawamura > Labels: Cluster > Fix For: 1.1.0 > > > A RPG (Site-to-Site client) should take into account of the number of queued > flow files in each remote cluster nodes, to calculate the load distribution > balance among target nodes. So that it can send more data to nodes those have > less queued files, or receive more data from nodes those have more queued > files. > However, after the clustering mechanism has changed since 1.0, this load > distribution logic is not handled appropriately. Site-to-Site remote > component always returns flow file count as 0. So RPG distribute load evenly, > even if a remote cluster has unbalanced load among nodes. > We need to fix the remote node API to expose queued flow file count correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2825) Remote Process Group should use num of queued flow files in remote nodes to distribute load
[ https://issues.apache.org/jira/browse/NIFI-2825?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Koji Kawamura updated NIFI-2825: Status: Patch Available (was: Open) > Remote Process Group should use num of queued flow files in remote nodes to > distribute load > --- > > Key: NIFI-2825 > URL: https://issues.apache.org/jira/browse/NIFI-2825 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 > Environment: Clustered >Reporter: Koji Kawamura >Assignee: Koji Kawamura > Fix For: 1.1.0 > > > A RPG (Site-to-Site client) should take into account of the number of queued > flow files in each remote cluster nodes, to calculate the load distribution > balance among target nodes. So that it can send more data to nodes those have > less queued files, or receive more data from nodes those have more queued > files. > However, after the clustering mechanism has changed since 1.0, this load > distribution logic is not handled appropriately. Site-to-Site remote > component always returns flow file count as 0. So RPG distribute load evenly, > even if a remote cluster has unbalanced load among nodes. > We need to fix the remote node API to expose queued flow file count correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2825) Remote Process Group should use num of queued flow files in remote nodes to distribute load
[ https://issues.apache.org/jira/browse/NIFI-2825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536241#comment-15536241 ] ASF GitHub Bot commented on NIFI-2825: -- GitHub user ijokarumawak opened a pull request: https://github.com/apache/nifi/pull/1084 NIFI-2825: Fix S2S getPeers flow file count - Added ClusterWorkload message to retrieve workload information from a cluster coordinator - Use cluster workload to return queued flow file count to site-to-site client so that it can calculate distribution of data transfer Added related unit tests, as well as test with running NiFi secure/unsecure cluster, and standalone. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijokarumawak/nifi nifi-2825 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1084.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1084 commit 2fde9b9d478f3a421bc883ea0d23a21bfa79d5b2 Author: Koji Kawamura Date: 2016-09-28T02:07:22Z NIFI-2825: Fix S2S getPeers flow file count - Added ClusterWorkload message to retrieve workload information from a cluster coordinator - Use cluster workload to return queued flow file count to site-to-site client so that it can calculate distribution of data transfer > Remote Process Group should use num of queued flow files in remote nodes to > distribute load > --- > > Key: NIFI-2825 > URL: https://issues.apache.org/jira/browse/NIFI-2825 > Project: Apache NiFi > Issue Type: Bug > Components: Core Framework >Affects Versions: 1.0.0 > Environment: Clustered >Reporter: Koji Kawamura >Assignee: Koji Kawamura > Fix For: 1.1.0 > > > A RPG (Site-to-Site client) should take into account of the number of queued > flow files in each remote cluster nodes, to calculate the load distribution > balance among target nodes. So that it can send more data to nodes those have > less queued files, or receive more data from nodes those have more queued > files. > However, after the clustering mechanism has changed since 1.0, this load > distribution logic is not handled appropriately. Site-to-Site remote > component always returns flow file count as 0. So RPG distribute load evenly, > even if a remote cluster has unbalanced load among nodes. > We need to fix the remote node API to expose queued flow file count correctly. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1084: NIFI-2825: Fix S2S getPeers flow file count
GitHub user ijokarumawak opened a pull request: https://github.com/apache/nifi/pull/1084 NIFI-2825: Fix S2S getPeers flow file count - Added ClusterWorkload message to retrieve workload information from a cluster coordinator - Use cluster workload to return queued flow file count to site-to-site client so that it can calculate distribution of data transfer Added related unit tests, as well as test with running NiFi secure/unsecure cluster, and standalone. You can merge this pull request into a Git repository by running: $ git pull https://github.com/ijokarumawak/nifi nifi-2825 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1084.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1084 commit 2fde9b9d478f3a421bc883ea0d23a21bfa79d5b2 Author: Koji Kawamura Date: 2016-09-28T02:07:22Z NIFI-2825: Fix S2S getPeers flow file count - Added ClusterWorkload message to retrieve workload information from a cluster coordinator - Use cluster workload to return queued flow file count to site-to-site client so that it can calculate distribution of data transfer --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2848) Queues aren't fairly drained when leading to a single component
[ https://issues.apache.org/jira/browse/NIFI-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536240#comment-15536240 ] Toivo Adams commented on NIFI-2848: --- I removed Funnel and tested again. One queue tends to dominate. This time third GenerateFlowFile queue. After stopping third GenerateFlowFile, second started dominate. Once queue gets opportunity it uses forever and don’t give others change to send anything. See dominate_1.png and dominate_2.png > Queues aren't fairly drained when leading to a single component > --- > > Key: NIFI-2848 > URL: https://issues.apache.org/jira/browse/NIFI-2848 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.0.0, 0.7.0 >Reporter: Joseph Gresock > Attachments: Backpressure_prioritization_test.xml, dominate_1.png, > dominate_2.png, queue_drain.png > > > Consider the scenario where multiple queues lead to a single component and > all of them are full due to back pressure. With the attached template, it is > easily observable that once a single queue starts to drain due to relieved > back pressure, it will continue to drain as long as it has incoming flow > files. This means that if there's a constant flow of incoming flow files to > this queue, the other queues will never be drained (at least, that's my > theory based on several hours of observation). > To reproduce this: > # Load the template into NiFi 1.0.0 > # Play all three GenerateFlowFile processors, but not the UpdateAttribute > processor (this simulates backpressure). Wait until each queue has 1,000 > flow files (max backpressure) > # Stop the GenerateFlowFile processors, and play the UpdateAttribute > processor (this relieves the backpressure) > # Observe which queue has started to drain, and start its GenerateFlowFile > processor > # Observe that the other two queues remain full indefinitely, while the > draining queue continues to replenish and be drained indefinitely -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2848) Queues aren't fairly drained when leading to a single component
[ https://issues.apache.org/jira/browse/NIFI-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toivo Adams updated NIFI-2848: -- Attachment: dominate_2.png dominate_1.png > Queues aren't fairly drained when leading to a single component > --- > > Key: NIFI-2848 > URL: https://issues.apache.org/jira/browse/NIFI-2848 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.0.0, 0.7.0 >Reporter: Joseph Gresock > Attachments: Backpressure_prioritization_test.xml, dominate_1.png, > dominate_2.png, queue_drain.png > > > Consider the scenario where multiple queues lead to a single component and > all of them are full due to back pressure. With the attached template, it is > easily observable that once a single queue starts to drain due to relieved > back pressure, it will continue to drain as long as it has incoming flow > files. This means that if there's a constant flow of incoming flow files to > this queue, the other queues will never be drained (at least, that's my > theory based on several hours of observation). > To reproduce this: > # Load the template into NiFi 1.0.0 > # Play all three GenerateFlowFile processors, but not the UpdateAttribute > processor (this simulates backpressure). Wait until each queue has 1,000 > flow files (max backpressure) > # Stop the GenerateFlowFile processors, and play the UpdateAttribute > processor (this relieves the backpressure) > # Observe which queue has started to drain, and start its GenerateFlowFile > processor > # Observe that the other two queues remain full indefinitely, while the > draining queue continues to replenish and be drained indefinitely -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #1054: Dummy commit focused in closing stalled PR requests
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1054 @apiri done --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2477) Document tls-toolkit
[ https://issues.apache.org/jira/browse/NIFI-2477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536232#comment-15536232 ] ASF GitHub Bot commented on NIFI-2477: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/849 > Document tls-toolkit > > > Key: NIFI-2477 > URL: https://issues.apache.org/jira/browse/NIFI-2477 > Project: Apache NiFi > Issue Type: Task >Reporter: Bryan Rosander >Assignee: Andrew Lim > Fix For: 1.0.0 > > > tls-toolkit created as part of NIFI-2193 needs to be documented in order to > be a useful utility to ease the burden of configuring tls for one or more > NiFi instances. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1054: Dummy commit focused in closing stalled PR requests
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1054 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #769: how to kafka data to hive
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/769 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #849: NIFI-2477 Document TLS generation tool in Admin and ...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/849 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #335: Adjust batch.size property description and validatio...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/335 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi pull request #595: pull new code
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/595 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-1794) For rules that have very long names, the name extends beyond the size of the delete confirmation window
[ https://issues.apache.org/jira/browse/NIFI-1794?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536194#comment-15536194 ] Andrew Lim commented on NIFI-1794: -- This does not occur in 1.1.0. However, instead of wrapping in the window, the user has to scroll to see the whole name of the rule. > For rules that have very long names, the name extends beyond the size of the > delete confirmation window > --- > > Key: NIFI-1794 > URL: https://issues.apache.org/jira/browse/NIFI-1794 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 0.6.0 >Reporter: Andrew Lim >Priority: Minor > Attachments: NIFI-1794_deleteLongRule.png > > > 1. Create a rule with a very long name. > 2. Select "x" next to the rule to delete it. > 3. In the Delete Confirmation pop-up window, the name of the rule extends > beyond the window (and likely beyond the rules window itself). > Screenshot attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2849) UI - Policy Management Enhancements
[ https://issues.apache.org/jira/browse/NIFI-2849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-2849: -- Description: 1 - Show component name whenever possible (specifically in the message regarding the inherited policy). 2 - Do not allow editing of inherited policies. Require explicit navigation to the component in question. Support linking to the component in the inherited policy message to make this easier. 3 - When overriding a policy, allow the user to optionally copy the inherited policy rules. was: 1 - Show component name whenever possible (specifically in the message regarding the inherited policy). 2 - Do not allow editing of inherited policies. Require explicit navigation to the component in question. Supporting linking to the component in the inherited policy message to make this easier. 3 - When overriding a policy, allow the user to optionally copy the inherited policy rules. > UI - Policy Management Enhancements > --- > > Key: NIFI-2849 > URL: https://issues.apache.org/jira/browse/NIFI-2849 > Project: Apache NiFi > Issue Type: Improvement > Components: Core UI >Reporter: Matt Gilman >Assignee: Matt Gilman > Fix For: 1.1.0 > > > 1 - Show component name whenever possible (specifically in the message > regarding the inherited policy). > 2 - Do not allow editing of inherited policies. Require explicit navigation > to the component in question. Support linking to the component in the > inherited policy message to make this easier. > 3 - When overriding a policy, allow the user to optionally copy the inherited > policy rules. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2849) UI - Policy Management Enhancements
Matt Gilman created NIFI-2849: - Summary: UI - Policy Management Enhancements Key: NIFI-2849 URL: https://issues.apache.org/jira/browse/NIFI-2849 Project: Apache NiFi Issue Type: Improvement Components: Core UI Reporter: Matt Gilman Assignee: Matt Gilman Fix For: 1.1.0 1 - Show component name whenever possible (specifically in the message regarding the inherited policy). 2 - Do not allow editing of inherited policies. Require explicit navigation to the component in question. Supporting linking to the component in the inherited policy message to make this easier. 3 - When overriding a policy, allow the user to optionally copy the inherited policy rules. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #1074: NIFI-2832 State management should be part of the documenta...
Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/1074 Thanks @pvillard31! This has been merged to master. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2832) State management should be part of the documentation/usage
[ https://issues.apache.org/jira/browse/NIFI-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536082#comment-15536082 ] ASF GitHub Bot commented on NIFI-2832: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1074 > State management should be part of the documentation/usage > -- > > Key: NIFI-2832 > URL: https://issues.apache.org/jira/browse/NIFI-2832 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Minor > Labels: documentation > Fix For: 1.1.0 > > > At the moment, ``@Stateful`` annotation if only used in the 'View State' > panel. This may not be intuitive for users (NIFI-2705) and this information > should be part of the documentation (in the 'Usage' panel and the web > documentation as well). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2848) Queues aren't fairly drained when leading to a single component
[ https://issues.apache.org/jira/browse/NIFI-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Toivo Adams updated NIFI-2848: -- Attachment: queue_drain.png > Queues aren't fairly drained when leading to a single component > --- > > Key: NIFI-2848 > URL: https://issues.apache.org/jira/browse/NIFI-2848 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.0.0, 0.7.0 >Reporter: Joseph Gresock > Attachments: Backpressure_prioritization_test.xml, queue_drain.png > > > Consider the scenario where multiple queues lead to a single component and > all of them are full due to back pressure. With the attached template, it is > easily observable that once a single queue starts to drain due to relieved > back pressure, it will continue to drain as long as it has incoming flow > files. This means that if there's a constant flow of incoming flow files to > this queue, the other queues will never be drained (at least, that's my > theory based on several hours of observation). > To reproduce this: > # Load the template into NiFi 1.0.0 > # Play all three GenerateFlowFile processors, but not the UpdateAttribute > processor (this simulates backpressure). Wait until each queue has 1,000 > flow files (max backpressure) > # Stop the GenerateFlowFile processors, and play the UpdateAttribute > processor (this relieves the backpressure) > # Observe which queue has started to drain, and start its GenerateFlowFile > processor > # Observe that the other two queues remain full indefinitely, while the > draining queue continues to replenish and be drained indefinitely -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2832) State management should be part of the documentation/usage
[ https://issues.apache.org/jira/browse/NIFI-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Matt Gilman updated NIFI-2832: -- Resolution: Fixed Status: Resolved (was: Patch Available) > State management should be part of the documentation/usage > -- > > Key: NIFI-2832 > URL: https://issues.apache.org/jira/browse/NIFI-2832 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Minor > Labels: documentation > Fix For: 1.1.0 > > > At the moment, ``@Stateful`` annotation if only used in the 'View State' > panel. This may not be intuitive for users (NIFI-2705) and this information > should be part of the documentation (in the 'Usage' panel and the web > documentation as well). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2832) State management should be part of the documentation/usage
[ https://issues.apache.org/jira/browse/NIFI-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536084#comment-15536084 ] ASF GitHub Bot commented on NIFI-2832: -- Github user mcgilman commented on the issue: https://github.com/apache/nifi/pull/1074 Thanks @pvillard31! This has been merged to master. > State management should be part of the documentation/usage > -- > > Key: NIFI-2832 > URL: https://issues.apache.org/jira/browse/NIFI-2832 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Minor > Labels: documentation > Fix For: 1.1.0 > > > At the moment, ``@Stateful`` annotation if only used in the 'View State' > panel. This may not be intuitive for users (NIFI-2705) and this information > should be part of the documentation (in the 'Usage' panel and the web > documentation as well). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2832) State management should be part of the documentation/usage
[ https://issues.apache.org/jira/browse/NIFI-2832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536079#comment-15536079 ] ASF subversion and git services commented on NIFI-2832: --- Commit d6867283b55a8b40a4467197ed611c5c8669dff8 in nifi's branch refs/heads/master from [~pvillard] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=d686728 ] NIFI-2832 State management should be part of the documentation/usage. This closes #1074 > State management should be part of the documentation/usage > -- > > Key: NIFI-2832 > URL: https://issues.apache.org/jira/browse/NIFI-2832 > Project: Apache NiFi > Issue Type: Improvement > Components: Documentation & Website >Reporter: Pierre Villard >Assignee: Pierre Villard >Priority: Minor > Labels: documentation > Fix For: 1.1.0 > > > At the moment, ``@Stateful`` annotation if only used in the 'View State' > panel. This may not be intuitive for users (NIFI-2705) and this information > should be part of the documentation (in the 'Usage' panel and the web > documentation as well). -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1074: NIFI-2832 State management should be part of the do...
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1074 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2848) Queues aren't fairly drained when leading to a single component
[ https://issues.apache.org/jira/browse/NIFI-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536071#comment-15536071 ] Toivo Adams commented on NIFI-2848: --- Hi Joseph Gresock I observed similar behaviour. First GenerateFlowFile (from left to right) got opportunity and continued to push new files. Second did not have any chance. Third had opportunity (only short period) before I started first again. Thanks Toivo > Queues aren't fairly drained when leading to a single component > --- > > Key: NIFI-2848 > URL: https://issues.apache.org/jira/browse/NIFI-2848 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.0.0, 0.7.0 >Reporter: Joseph Gresock > Attachments: Backpressure_prioritization_test.xml > > > Consider the scenario where multiple queues lead to a single component and > all of them are full due to back pressure. With the attached template, it is > easily observable that once a single queue starts to drain due to relieved > back pressure, it will continue to drain as long as it has incoming flow > files. This means that if there's a constant flow of incoming flow files to > this queue, the other queues will never be drained (at least, that's my > theory based on several hours of observation). > To reproduce this: > # Load the template into NiFi 1.0.0 > # Play all three GenerateFlowFile processors, but not the UpdateAttribute > processor (this simulates backpressure). Wait until each queue has 1,000 > flow files (max backpressure) > # Stop the GenerateFlowFile processors, and play the UpdateAttribute > processor (this relieves the backpressure) > # Observe which queue has started to drain, and start its GenerateFlowFile > processor > # Observe that the other two queues remain full indefinitely, while the > draining queue continues to replenish and be drained indefinitely -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2847) In NiFi Cluster window, filtering "by status" or "by address" do not work as expected
[ https://issues.apache.org/jira/browse/NIFI-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535977#comment-15535977 ] Andrew Lim commented on NIFI-2847: -- Thanks [~jameswing]! I was testing in a secured 1.0.0 cluster, so before Matt's commit. I'll re-test in 1.1.0. > In NiFi Cluster window, filtering "by status" or "by address" do not work as > expected > - > > Key: NIFI-2847 > URL: https://issues.apache.org/jira/browse/NIFI-2847 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Andrew Lim > Attachments: NIFI-2847_NiFiClusterFilter.png > > > Attaching a screenshot of what this window displays when no filters are > applied to the displayed results for a 3 node cluster. > When filtering "by status": > -If I enter "CONNECTED", all 3 results are shown. > -But if I enter "CONNECTED," or "COORDINATOR", no results are shown. I > expected this to filter out all nodes except the cluster coordinator. > When filtering "by address": > -if I enter "localhost", all 3 results are shown. > -But if I enter "localhost:" or any of the ports (9443, for example), no > results are shown. I think it is a very common use case to filter by port. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (NIFI-2847) In NiFi Cluster window, filtering "by status" or "by address" do not work as expected
[ https://issues.apache.org/jira/browse/NIFI-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Lim reassigned NIFI-2847: Assignee: Andrew Lim > In NiFi Cluster window, filtering "by status" or "by address" do not work as > expected > - > > Key: NIFI-2847 > URL: https://issues.apache.org/jira/browse/NIFI-2847 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Andrew Lim >Assignee: Andrew Lim > Attachments: NIFI-2847_NiFiClusterFilter.png > > > Attaching a screenshot of what this window displays when no filters are > applied to the displayed results for a 3 node cluster. > When filtering "by status": > -If I enter "CONNECTED", all 3 results are shown. > -But if I enter "CONNECTED," or "COORDINATOR", no results are shown. I > expected this to filter out all nodes except the cluster coordinator. > When filtering "by address": > -if I enter "localhost", all 3 results are shown. > -But if I enter "localhost:" or any of the ports (9443, for example), no > results are shown. I think it is a very common use case to filter by port. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (NIFI-2783) Command line site-to-site client in toolkit would be useful
[ https://issues.apache.org/jira/browse/NIFI-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende resolved NIFI-2783. --- Resolution: Fixed > Command line site-to-site client in toolkit would be useful > --- > > Key: NIFI-2783 > URL: https://issues.apache.org/jira/browse/NIFI-2783 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Bryan Rosander >Assignee: Bryan Rosander > Fix For: 1.1.0 > > > A command line site-to-site client in the toolkit would be beneficial in > several scenarios: > 1. Easily pipe data into, out of NiFi flows using command line utilities or > other (non-java) programs makes integrating other tools with NiFi trivial > 2. Easier to debug site-to-site issues when only one NiFi instance is > involved in the troubleshooting process > 3. Dev testing site-to-site is easier command line reproduction of > issues/validation of functionality -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2783) Command line site-to-site client in toolkit would be useful
[ https://issues.apache.org/jira/browse/NIFI-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-2783: -- Assignee: Bryan Rosander > Command line site-to-site client in toolkit would be useful > --- > > Key: NIFI-2783 > URL: https://issues.apache.org/jira/browse/NIFI-2783 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Bryan Rosander >Assignee: Bryan Rosander > Fix For: 1.1.0 > > > A command line site-to-site client in the toolkit would be beneficial in > several scenarios: > 1. Easily pipe data into, out of NiFi flows using command line utilities or > other (non-java) programs makes integrating other tools with NiFi trivial > 2. Easier to debug site-to-site issues when only one NiFi instance is > involved in the troubleshooting process > 3. Dev testing site-to-site is easier command line reproduction of > issues/validation of functionality -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2783) Command line site-to-site client in toolkit would be useful
[ https://issues.apache.org/jira/browse/NIFI-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Bryan Bende updated NIFI-2783: -- Fix Version/s: 1.1.0 > Command line site-to-site client in toolkit would be useful > --- > > Key: NIFI-2783 > URL: https://issues.apache.org/jira/browse/NIFI-2783 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Bryan Rosander > Fix For: 1.1.0 > > > A command line site-to-site client in the toolkit would be beneficial in > several scenarios: > 1. Easily pipe data into, out of NiFi flows using command line utilities or > other (non-java) programs makes integrating other tools with NiFi trivial > 2. Easier to debug site-to-site issues when only one NiFi instance is > involved in the troubleshooting process > 3. Dev testing site-to-site is easier command line reproduction of > issues/validation of functionality -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2783) Command line site-to-site client in toolkit would be useful
[ https://issues.apache.org/jira/browse/NIFI-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535958#comment-15535958 ] ASF subversion and git services commented on NIFI-2783: --- Commit 4e13fef7243935962848ae83b3e31a5283c71c66 in nifi's branch refs/heads/master from [~bryanrosan...@gmail.com] [ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=4e13fef ] NIFI-2783 - Site-to-site command line client - s2s.sh shell script warnings -> stderr, usage improvement with examples - Increasing heap settings for s2s cli This closes #1056. Signed-off-by: Bryan Bende > Command line site-to-site client in toolkit would be useful > --- > > Key: NIFI-2783 > URL: https://issues.apache.org/jira/browse/NIFI-2783 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Bryan Rosander > Fix For: 1.1.0 > > > A command line site-to-site client in the toolkit would be beneficial in > several scenarios: > 1. Easily pipe data into, out of NiFi flows using command line utilities or > other (non-java) programs makes integrating other tools with NiFi trivial > 2. Easier to debug site-to-site issues when only one NiFi instance is > involved in the troubleshooting process > 3. Dev testing site-to-site is easier command line reproduction of > issues/validation of functionality -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2783) Command line site-to-site client in toolkit would be useful
[ https://issues.apache.org/jira/browse/NIFI-2783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535961#comment-15535961 ] ASF GitHub Bot commented on NIFI-2783: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1056 > Command line site-to-site client in toolkit would be useful > --- > > Key: NIFI-2783 > URL: https://issues.apache.org/jira/browse/NIFI-2783 > Project: Apache NiFi > Issue Type: New Feature >Reporter: Bryan Rosander > Fix For: 1.1.0 > > > A command line site-to-site client in the toolkit would be beneficial in > several scenarios: > 1. Easily pipe data into, out of NiFi flows using command line utilities or > other (non-java) programs makes integrating other tools with NiFi trivial > 2. Easier to debug site-to-site issues when only one NiFi instance is > involved in the troubleshooting process > 3. Dev testing site-to-site is easier command line reproduction of > issues/validation of functionality -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1056: NIFI-2783 - Site-to-site command line client
Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1056 --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andre updated NIFI-2709: Attachment: 1083.patch > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Assignee: Andre >Priority: Minor > Attachments: 1083.patch, screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andre updated NIFI-2709: Status: Patch Available (was: Open) > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Assignee: Andre >Priority: Minor > Attachments: 1083.patch, screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535943#comment-15535943 ] ASF GitHub Bot commented on NIFI-2709: -- Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1083 content no long loaded into memory. except for the issue around jUnits (2 & 3), it should be ready to review > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Assignee: Andre >Priority: Minor > Attachments: screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #1083: NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to genera...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1083 content no long loaded into memory. except for the issue around jUnits (2 & 3), it should be ready to review --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1054: Dummy commit focused in closing stalled PR requests
Github user apiri commented on the issue: https://github.com/apache/nifi/pull/1054 @trixpan yeah, feel free to merge. I think there has been sufficient time for folks to chime in. Thanks! --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1083: NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to genera...
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1083 @bbende this should solve the IMAP issue. Notes: 1. This PR reads the content of the message into memory, in contrast with the original design that relied on InputStream to write the content into the FlowFile. This is due to the fact that JavaMail doesn't seem to have an inputstream method capable of providing an RFC2822 message. In case anyone has an idea on how to improve that, I am keen to incorporate into this PR. 2. The PR includes a GreenMail based junit and required dependency to ease debugging. Since I am not sure if we want to refactor the jUnits already in place (they seem to work well) with GreenMail based jUnits I aded very little coverage. Let me know your preference and I will either remove the new jUnit or to refactor @olegz original jUnit to use GreenMail based unit testing. 3. This will fail contrib-check. To be addressed once we decide what to do with the jUnit tests mentioned above. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535864#comment-15535864 ] ASF GitHub Bot commented on NIFI-2709: -- Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1083 @bbende this should solve the IMAP issue. Notes: 1. This PR reads the content of the message into memory, in contrast with the original design that relied on InputStream to write the content into the FlowFile. This is due to the fact that JavaMail doesn't seem to have an inputstream method capable of providing an RFC2822 message. In case anyone has an idea on how to improve that, I am keen to incorporate into this PR. 2. The PR includes a GreenMail based junit and required dependency to ease debugging. Since I am not sure if we want to refactor the jUnits already in place (they seem to work well) with GreenMail based jUnits I aded very little coverage. Let me know your preference and I will either remove the new jUnit or to refactor @olegz original jUnit to use GreenMail based unit testing. 3. This will fail contrib-check. To be addressed once we decide what to do with the jUnit tests mentioned above. > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Assignee: Andre >Priority: Minor > Attachments: screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2756) nifi-processor-bundle-archetype lacks displayName property
[
https://issues.apache.org/jira/browse/NIFI-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535852#comment-15535852
]
ASF GitHub Bot commented on NIFI-2756:
--
Github user bbende commented on the issue:
https://github.com/apache/nifi/pull/1004
@trixpan sure...
1) mvn clean install on this branch
2) In some other directory:
`mvn archetype:generate -DarchetypeGroupId=org.apache.nifi
-DarchetypeArtifactId=nifi-processor-bundle-archetype
-DarchetypeVersion=1.1.0-SNAPSHOT -DnifiVersion=1.1.0-SNAPSHOT
-DarchetypeCatalog=local`
3) Fill in something like groupId org.apache.nifi, artifactId
displaytest-bundle, basename displaytest
4) Try to build the generated project using mvn clean install
-Pcontrib-check
5) Repeat with service archetype:
`mvn archetype:generate -DarchetypeGroupId=org.apache.nifi
-DarchetypeArtifactId=nifi-service-bundle-archetype
-DarchetypeVersion=1.1.0-SNAPSHOT -DnifiVersion=1.1.0-SNAPSHOT
-DarchetypeCatalog=local`
For additional reference:
https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions
> nifi-processor-bundle-archetype lacks displayName property
> --
>
> Key: NIFI-2756
> URL: https://issues.apache.org/jira/browse/NIFI-2756
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Andre
>Assignee: Andre
>
> When using {{nifi-processor-bundle-archetype}} to create a new bundle, the
> resulting code lacks displayName leading to a number of PR and contributions
> to arrive to peer review without displayName configured
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] nifi issue #1004: NIFI-2756 - Add displayName to maven archetypes
Github user bbende commented on the issue: https://github.com/apache/nifi/pull/1004 @trixpan sure... 1) mvn clean install on this branch 2) In some other directory: `mvn archetype:generate -DarchetypeGroupId=org.apache.nifi -DarchetypeArtifactId=nifi-processor-bundle-archetype -DarchetypeVersion=1.1.0-SNAPSHOT -DnifiVersion=1.1.0-SNAPSHOT -DarchetypeCatalog=local` 3) Fill in something like groupId org.apache.nifi, artifactId displaytest-bundle, basename displaytest 4) Try to build the generated project using mvn clean install -Pcontrib-check 5) Repeat with service archetype: `mvn archetype:generate -DarchetypeGroupId=org.apache.nifi -DarchetypeArtifactId=nifi-service-bundle-archetype -DarchetypeVersion=1.1.0-SNAPSHOT -DnifiVersion=1.1.0-SNAPSHOT -DarchetypeCatalog=local` For additional reference: https://cwiki.apache.org/confluence/display/NIFI/Maven+Projects+for+Extensions --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535828#comment-15535828 ] ASF GitHub Bot commented on NIFI-2709: -- GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/1083 NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to generate RFC2822 compliant flowfiles You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-2709 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1083.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1083 commit 4fc0779960480b6d8e81372506201e2a9a9bae29 Author: Andre F de Miranda Date: 2016-09-30T09:00:59Z NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to generate RFC2822 compliant flowfiles > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Assignee: Andre >Priority: Minor > Attachments: screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi pull request #1083: NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to...
GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/1083 NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to generate RFC2822 compliant flowfiles You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-2709 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1083.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1083 commit 4fc0779960480b6d8e81372506201e2a9a9bae29 Author: Andre F de Miranda Date: 2016-09-30T09:00:59Z NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to generate RFC2822 compliant flowfiles --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Updated] (NIFI-2848) Queues aren't fairly drained when leading to a single component
[ https://issues.apache.org/jira/browse/NIFI-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Gresock updated NIFI-2848: - Description: Consider the scenario where multiple queues lead to a single component and all of them are full due to back pressure. With the attached template, it is easily observable that once a single queue starts to drain due to relieved back pressure, it will continue to drain as long as it has incoming flow files. This means that if there's a constant flow of incoming flow files to this queue, the other queues will never be drained (at least, that's my theory based on an hour of observation). To reproduce this: # Load the template into NiFi 1.0.0 # Play all three GenerateFlowFile processors, but not the UpdateAttribute processor (this simulates backpressure). Wait until each queue has 1,000 flow files (max backpressure) # Stop the GenerateFlowFile processors, and play the UpdateAttribute processor (this relieves the backpressure) # Observe which queue has started to drain, and start its GenerateFlowFile processor # Observe that the other two queues remain full indefinitely, while the draining queue continues to replenish and be drained indefinitely was: Consider the scenario where multiple queues lead to a single component and all of them are full due to back pressure. With the attached template, it is easily observable that once a single queue starts to drain due to relieved back pressure, it will continue to drain as long as it has incoming flow files. This means that if there's a constant flow of incoming flow files to this queue, the other two queues will never be drained (at least, that's my theory based on an hour of observation). To reproduce this: # Load the template into NiFi 1.0.0 # Play all three GenerateFlowFile processors, but not the UpdateAttribute processor (this simulates backpressure). Wait until each queue has 1,000 flow files (max backpressure) # Stop the GenerateFlowFile processors, and play the UpdateAttribute processor (this relieves the backpressure) # Observe which queue has started to drain, and start its GenerateFlowFile processor # Observe that the other two queues remain full indefinitely, while the draining queue continues to replenish and be drained indefinitely > Queues aren't fairly drained when leading to a single component > --- > > Key: NIFI-2848 > URL: https://issues.apache.org/jira/browse/NIFI-2848 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.0.0, 0.7.0 >Reporter: Joseph Gresock > Attachments: Backpressure_prioritization_test.xml > > > Consider the scenario where multiple queues lead to a single component and > all of them are full due to back pressure. With the attached template, it is > easily observable that once a single queue starts to drain due to relieved > back pressure, it will continue to drain as long as it has incoming flow > files. This means that if there's a constant flow of incoming flow files to > this queue, the other queues will never be drained (at least, that's my > theory based on an hour of observation). > To reproduce this: > # Load the template into NiFi 1.0.0 > # Play all three GenerateFlowFile processors, but not the UpdateAttribute > processor (this simulates backpressure). Wait until each queue has 1,000 > flow files (max backpressure) > # Stop the GenerateFlowFile processors, and play the UpdateAttribute > processor (this relieves the backpressure) > # Observe which queue has started to drain, and start its GenerateFlowFile > processor > # Observe that the other two queues remain full indefinitely, while the > draining queue continues to replenish and be drained indefinitely -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2848) Queues aren't fairly drained when leading to a single component
[ https://issues.apache.org/jira/browse/NIFI-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Gresock updated NIFI-2848: - Description: Consider the scenario where multiple queues lead to a single component and all of them are full due to back pressure. With the attached template, it is easily observable that once a single queue starts to drain due to relieved back pressure, it will continue to drain as long as it has incoming flow files. This means that if there's a constant flow of incoming flow files to this queue, the other queues will never be drained (at least, that's my theory based on several hours of observation). To reproduce this: # Load the template into NiFi 1.0.0 # Play all three GenerateFlowFile processors, but not the UpdateAttribute processor (this simulates backpressure). Wait until each queue has 1,000 flow files (max backpressure) # Stop the GenerateFlowFile processors, and play the UpdateAttribute processor (this relieves the backpressure) # Observe which queue has started to drain, and start its GenerateFlowFile processor # Observe that the other two queues remain full indefinitely, while the draining queue continues to replenish and be drained indefinitely was: Consider the scenario where multiple queues lead to a single component and all of them are full due to back pressure. With the attached template, it is easily observable that once a single queue starts to drain due to relieved back pressure, it will continue to drain as long as it has incoming flow files. This means that if there's a constant flow of incoming flow files to this queue, the other queues will never be drained (at least, that's my theory based on an hour of observation). To reproduce this: # Load the template into NiFi 1.0.0 # Play all three GenerateFlowFile processors, but not the UpdateAttribute processor (this simulates backpressure). Wait until each queue has 1,000 flow files (max backpressure) # Stop the GenerateFlowFile processors, and play the UpdateAttribute processor (this relieves the backpressure) # Observe which queue has started to drain, and start its GenerateFlowFile processor # Observe that the other two queues remain full indefinitely, while the draining queue continues to replenish and be drained indefinitely > Queues aren't fairly drained when leading to a single component > --- > > Key: NIFI-2848 > URL: https://issues.apache.org/jira/browse/NIFI-2848 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.0.0, 0.7.0 >Reporter: Joseph Gresock > Attachments: Backpressure_prioritization_test.xml > > > Consider the scenario where multiple queues lead to a single component and > all of them are full due to back pressure. With the attached template, it is > easily observable that once a single queue starts to drain due to relieved > back pressure, it will continue to drain as long as it has incoming flow > files. This means that if there's a constant flow of incoming flow files to > this queue, the other queues will never be drained (at least, that's my > theory based on several hours of observation). > To reproduce this: > # Load the template into NiFi 1.0.0 > # Play all three GenerateFlowFile processors, but not the UpdateAttribute > processor (this simulates backpressure). Wait until each queue has 1,000 > flow files (max backpressure) > # Stop the GenerateFlowFile processors, and play the UpdateAttribute > processor (this relieves the backpressure) > # Observe which queue has started to drain, and start its GenerateFlowFile > processor > # Observe that the other two queues remain full indefinitely, while the > draining queue continues to replenish and be drained indefinitely -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535638#comment-15535638
]
Andre commented on NIFI-2709:
-
[~bbende] seems like there is a disconnect between what ConsumeImap produces by
default and what the extraction processors expect, hence the "validation
errors".
The following code confirm this. Note how the FlowFile content equals to the
body of the generated message instead of the whole email message...
{code}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.processors.email;
import com.icegreen.greenmail.user.GreenMailUser;
import com.icegreen.greenmail.util.GreenMail;
import com.icegreen.greenmail.util.ServerSetupTest;
import org.apache.nifi.util.MockFlowFile;
import org.apache.nifi.util.TestRunner;
import org.apache.nifi.util.TestRunners;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import javax.mail.Message;
import javax.mail.MessagingException;
import javax.mail.Session;
import javax.mail.internet.AddressException;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;
import java.util.List;
import java.util.Properties;
public class TestConsumeIMAP {
private GreenMail mockImapServer;
private GreenMailUser user;
// Setup mock imap server
@Before
public void setUp() {
mockImapServer = new GreenMail(ServerSetupTest.IMAP);
mockImapServer.start();
user = mockImapServer.setUser("t...@nifi.org", "nifiUser",
"nifiPassword");
}
@After
public void cleanUp() {
mockImapServer.stop();
}
public void addMessage(String testName) throws MessagingException {
Properties prop = new Properties();
Session session = Session.getDefaultInstance(prop);
MimeMessage message = new MimeMessage(session);
message.setFrom(new InternetAddress("al...@nifi.org"));
message.addRecipient(Message.RecipientType.TO, new
InternetAddress("t...@nifi.org"));
message.setSubject("Test email" + testName);
message.setText("test test test chocolate");
user.deliver(message);
}
// Start the testing units
@Test
public void testConsumeImap() throws Exception {
final TestRunner runner = TestRunners.newTestRunner(new ConsumeIMAP());
runner.setProperty(ConsumeIMAP.HOST,
ServerSetupTest.IMAP.getBindAddress());
runner.setProperty(ConsumeIMAP.PORT,
String.valueOf(ServerSetupTest.IMAP.getPort()));
runner.setProperty(ConsumeIMAP.USER, "nifiUser");
runner.setProperty(ConsumeIMAP.PASSWORD, "nifiPassword");
runner.setProperty(ConsumeIMAP.FOLDER, "INBOX");
runner.setProperty(ConsumeIMAP.USE_SSL, "false");
addMessage("testConsumeImap");
runner.run();
runner.assertTransferCount(ConsumeIMAP.REL_SUCCESS, 1);
final List messages =
runner.getFlowFilesForRelationship(ConsumeIMAP.REL_SUCCESS);
MockFlowFile x = messages.get(0);
x.assertContentEquals("test test test chocolate");
}
}
{code}
> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Attachments: screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML
> messages with no RFC2822 headers. I have also tried setting Exchange to force
> emails to be text only, sadly only the body with some Content-Type: fields
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify R
[jira] [Assigned] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andre reassigned NIFI-2709: --- Assignee: Andre > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Assignee: Andre >Priority: Minor > Attachments: screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (NIFI-2848) Queues aren't fairly drained when leading to a single component
[ https://issues.apache.org/jira/browse/NIFI-2848?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph Gresock updated NIFI-2848: - Attachment: Backpressure_prioritization_test.xml > Queues aren't fairly drained when leading to a single component > --- > > Key: NIFI-2848 > URL: https://issues.apache.org/jira/browse/NIFI-2848 > Project: Apache NiFi > Issue Type: Improvement > Components: Core Framework >Affects Versions: 1.0.0, 0.7.0 >Reporter: Joseph Gresock > Attachments: Backpressure_prioritization_test.xml > > > Consider the scenario where multiple queues lead to a single component and > all of them are full due to back pressure. With the attached template, it is > easily observable that once a single queue starts to drain due to relieved > back pressure, it will continue to drain as long as it has incoming flow > files. This means that if there's a constant flow of incoming flow files to > this queue, the other two queues will never be drained (at least, that's my > theory based on an hour of observation). > To reproduce this: > # Load the template into NiFi 1.0.0 > # Play all three GenerateFlowFile processors, but not the UpdateAttribute > processor (this simulates backpressure). Wait until each queue has 1,000 > flow files (max backpressure) > # Stop the GenerateFlowFile processors, and play the UpdateAttribute > processor (this relieves the backpressure) > # Observe which queue has started to drain, and start its GenerateFlowFile > processor > # Observe that the other two queues remain full indefinitely, while the > draining queue continues to replenish and be drained indefinitely -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (NIFI-2848) Queues aren't fairly drained when leading to a single component
Joseph Gresock created NIFI-2848: Summary: Queues aren't fairly drained when leading to a single component Key: NIFI-2848 URL: https://issues.apache.org/jira/browse/NIFI-2848 Project: Apache NiFi Issue Type: Improvement Components: Core Framework Affects Versions: 0.7.0, 1.0.0 Reporter: Joseph Gresock Consider the scenario where multiple queues lead to a single component and all of them are full due to back pressure. With the attached template, it is easily observable that once a single queue starts to drain due to relieved back pressure, it will continue to drain as long as it has incoming flow files. This means that if there's a constant flow of incoming flow files to this queue, the other two queues will never be drained (at least, that's my theory based on an hour of observation). To reproduce this: # Load the template into NiFi 1.0.0 # Play all three GenerateFlowFile processors, but not the UpdateAttribute processor (this simulates backpressure). Wait until each queue has 1,000 flow files (max backpressure) # Stop the GenerateFlowFile processors, and play the UpdateAttribute processor (this relieves the backpressure) # Observe which queue has started to drain, and start its GenerateFlowFile processor # Observe that the other two queues remain full indefinitely, while the draining queue continues to replenish and be drained indefinitely -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2756) nifi-processor-bundle-archetype lacks displayName property
[
https://issues.apache.org/jira/browse/NIFI-2756?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535553#comment-15535553
]
ASF GitHub Bot commented on NIFI-2756:
--
Github user trixpan commented on the issue:
https://github.com/apache/nifi/pull/1004
@bbende mind sharing the steps you took? Keen to try to reproduce and fix.
> nifi-processor-bundle-archetype lacks displayName property
> --
>
> Key: NIFI-2756
> URL: https://issues.apache.org/jira/browse/NIFI-2756
> Project: Apache NiFi
> Issue Type: Bug
>Reporter: Andre
>Assignee: Andre
>
> When using {{nifi-processor-bundle-archetype}} to create a new bundle, the
> resulting code lacks displayName leading to a number of PR and contributions
> to arrive to peer review without displayName configured
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[GitHub] nifi issue #1004: NIFI-2756 - Add displayName to maven archetypes
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1004 @bbende mind sharing the steps you took? Keen to try to reproduce and fix. --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2380) ExtractEmailAttachments processor should support TNEF files (aka winmail.dat)
[ https://issues.apache.org/jira/browse/NIFI-2380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535529#comment-15535529 ] ASF GitHub Bot commented on NIFI-2380: -- Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/817 @olegz I guess you have been busy but I was wondering if you are still planning to review this? Cheers > ExtractEmailAttachments processor should support TNEF files (aka winmail.dat) > - > > Key: NIFI-2380 > URL: https://issues.apache.org/jira/browse/NIFI-2380 > Project: Apache NiFi > Issue Type: Improvement >Affects Versions: 1.0.0 >Reporter: Andre >Assignee: Andre > Fix For: 1.1.0 > > > during the review of NIFI-1899 Dan Marshall highlighted some use cases for > email processing that have not been addressed as part of the initial > development cycle. > One of these use cases was the decoding of Microsoft Transport Neutral > Encoding Files (TNEF). > This type of attachments is popularly know as winmail.dat and uses a non RFC > compliant structure to transfer attachments across different Microsoft > Outlook clients. > Given the prevalence of outlook and the issues with winmail.dat files, it > would be nice to be able to decode TNEF as we currently do with MIME > attachments. > Permalink to Dan's comments > http://mail-archives.apache.org/mod_mbox/nifi-dev/201607.mbox/%3C1468716836729-12827.post%40n7.nabble.com%3E -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[GitHub] nifi issue #817: NIFI-2380 - Introduce ExtractTNEFAttachments
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/817 @olegz I guess you have been busy but I was wondering if you are still planning to review this? Cheers --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[GitHub] nifi issue #1054: Dummy commit focused in closing stalled PR requests
Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1054 @apiri should we go ahead and commit the PR? --- If your project is set up for it, you can reply to this email and have your reply appear on GitHub as well. If your project does not have this feature enabled and wishes so, or if the feature is enabled but not working, please contact infrastructure at infrastruct...@apache.org or file a JIRA ticket with INFRA. ---
[jira] [Commented] (NIFI-2847) In NiFi Cluster window, filtering "by status" or "by address" do not work as expected
[ https://issues.apache.org/jira/browse/NIFI-2847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535265#comment-15535265 ] James Wing commented on NIFI-2847: -- How recent of a build did you test this in? I believe this was true in the 1.0.0 release, but [~mcgilman] addressed both of these issues in commit [508b218|https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=508b218] as part of an update to the cluster UI for NIFI-2795. I just want to double check if the version you are working with is before or after that recent change. > In NiFi Cluster window, filtering "by status" or "by address" do not work as > expected > - > > Key: NIFI-2847 > URL: https://issues.apache.org/jira/browse/NIFI-2847 > Project: Apache NiFi > Issue Type: Bug > Components: Core UI >Affects Versions: 1.0.0 >Reporter: Andrew Lim > Attachments: NIFI-2847_NiFiClusterFilter.png > > > Attaching a screenshot of what this window displays when no filters are > applied to the displayed results for a 3 node cluster. > When filtering "by status": > -If I enter "CONNECTED", all 3 results are shown. > -But if I enter "CONNECTED," or "COORDINATOR", no results are shown. I > expected this to filter out all nodes except the cluster coordinator. > When filtering "by address": > -if I enter "localhost", all 3 results are shown. > -But if I enter "localhost:" or any of the ports (9443, for example), no > results are shown. I think it is a very common use case to filter by port. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
