[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15515502#comment-15515502 ] Andre commented on NIFI-2709: - [~lightraven24] Thanks for raising this issue. Can you confirm if setting up the message retrieval format as documented in [here|https://technet.microsoft.com/en-us/library/aa997869(v=exchg.150).aspx] works? > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Priority: Minor > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15515789#comment-15515789 ] Andre commented on NIFI-2709: - [~ozhurakousky] I am not sure what the Spring approach but commons mail seems to address this compatibility issue: https://commons.apache.org/proper/commons-net/apidocs/org/apache/commons/net/imap/IMAPClient.FETCH_ITEM_NAMES.html > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Priority: Minor > Attachments: screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15532729#comment-15532729 ] Andre commented on NIFI-2709: - [~bende] I think this is the same issue the folk at the community experienced. > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Priority: Minor > Attachments: screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535638#comment-15535638
]
Andre commented on NIFI-2709:
-
[~bbende] seems like there is a disconnect between what ConsumeImap produces by
default and what the extraction processors expect, hence the "validation
errors".
The following code confirm this. Note how the FlowFile content equals to the
body of the generated message instead of the whole email message...
{code}
/*
* Licensed to the Apache Software Foundation (ASF) under one or more
* contributor license agreements. See the NOTICE file distributed with
* this work for additional information regarding copyright ownership.
* The ASF licenses this file to You under the Apache License, Version 2.0
* (the "License"); you may not use this file except in compliance with
* the License. You may obtain a copy of the License at
*
* http://www.apache.org/licenses/LICENSE-2.0
*
* Unless required by applicable law or agreed to in writing, software
* distributed under the License is distributed on an "AS IS" BASIS,
* WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
* See the License for the specific language governing permissions and
* limitations under the License.
*/
package org.apache.nifi.processors.email;
import com.icegreen.greenmail.user.GreenMailUser;
import com.icegreen.greenmail.util.GreenMail;
import com.icegreen.greenmail.util.ServerSetupTest;
import org.apache.nifi.util.MockFlowFile;
import org.apache.nifi.util.TestRunner;
import org.apache.nifi.util.TestRunners;
import org.junit.After;
import org.junit.Before;
import org.junit.Test;
import javax.mail.Message;
import javax.mail.MessagingException;
import javax.mail.Session;
import javax.mail.internet.AddressException;
import javax.mail.internet.InternetAddress;
import javax.mail.internet.MimeMessage;
import java.util.List;
import java.util.Properties;
public class TestConsumeIMAP {
private GreenMail mockImapServer;
private GreenMailUser user;
// Setup mock imap server
@Before
public void setUp() {
mockImapServer = new GreenMail(ServerSetupTest.IMAP);
mockImapServer.start();
user = mockImapServer.setUser("t...@nifi.org", "nifiUser",
"nifiPassword");
}
@After
public void cleanUp() {
mockImapServer.stop();
}
public void addMessage(String testName) throws MessagingException {
Properties prop = new Properties();
Session session = Session.getDefaultInstance(prop);
MimeMessage message = new MimeMessage(session);
message.setFrom(new InternetAddress("al...@nifi.org"));
message.addRecipient(Message.RecipientType.TO, new
InternetAddress("t...@nifi.org"));
message.setSubject("Test email" + testName);
message.setText("test test test chocolate");
user.deliver(message);
}
// Start the testing units
@Test
public void testConsumeImap() throws Exception {
final TestRunner runner = TestRunners.newTestRunner(new ConsumeIMAP());
runner.setProperty(ConsumeIMAP.HOST,
ServerSetupTest.IMAP.getBindAddress());
runner.setProperty(ConsumeIMAP.PORT,
String.valueOf(ServerSetupTest.IMAP.getPort()));
runner.setProperty(ConsumeIMAP.USER, "nifiUser");
runner.setProperty(ConsumeIMAP.PASSWORD, "nifiPassword");
runner.setProperty(ConsumeIMAP.FOLDER, "INBOX");
runner.setProperty(ConsumeIMAP.USE_SSL, "false");
addMessage("testConsumeImap");
runner.run();
runner.assertTransferCount(ConsumeIMAP.REL_SUCCESS, 1);
final List messages =
runner.getFlowFilesForRelationship(ConsumeIMAP.REL_SUCCESS);
MockFlowFile x = messages.get(0);
x.assertContentEquals("test test test chocolate");
}
}
{code}
> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Attachments: screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML
> messages with no RFC2822 headers. I have also tried setting Exchange to force
> emails to be text only, sadly only the body with some Content-Type: fields
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify R
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535828#comment-15535828 ] ASF GitHub Bot commented on NIFI-2709: -- GitHub user trixpan opened a pull request: https://github.com/apache/nifi/pull/1083 NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to generate RFC2822 compliant flowfiles You can merge this pull request into a Git repository by running: $ git pull https://github.com/trixpan/nifi NIFI-2709 Alternatively you can review and apply these changes as the patch at: https://github.com/apache/nifi/pull/1083.patch To close this pull request, make a commit to your master/trunk branch with (at least) the following in the commit message: This closes #1083 commit 4fc0779960480b6d8e81372506201e2a9a9bae29 Author: Andre F de Miranda Date: 2016-09-30T09:00:59Z NIFI-2709 - Refactor ConsumeIMAP and ConsumePOP3 to generate RFC2822 compliant flowfiles > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Assignee: Andre >Priority: Minor > Attachments: screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535864#comment-15535864 ] ASF GitHub Bot commented on NIFI-2709: -- Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1083 @bbende this should solve the IMAP issue. Notes: 1. This PR reads the content of the message into memory, in contrast with the original design that relied on InputStream to write the content into the FlowFile. This is due to the fact that JavaMail doesn't seem to have an inputstream method capable of providing an RFC2822 message. In case anyone has an idea on how to improve that, I am keen to incorporate into this PR. 2. The PR includes a GreenMail based junit and required dependency to ease debugging. Since I am not sure if we want to refactor the jUnits already in place (they seem to work well) with GreenMail based jUnits I aded very little coverage. Let me know your preference and I will either remove the new jUnit or to refactor @olegz original jUnit to use GreenMail based unit testing. 3. This will fail contrib-check. To be addressed once we decide what to do with the jUnit tests mentioned above. > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Assignee: Andre >Priority: Minor > Attachments: screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15535943#comment-15535943 ] ASF GitHub Bot commented on NIFI-2709: -- Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1083 content no long loaded into memory. except for the issue around jUnits (2 & 3), it should be ready to review > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Assignee: Andre >Priority: Minor > Attachments: screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15536336#comment-15536336 ] ASF GitHub Bot commented on NIFI-2709: -- Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1083 User reveals patch works as expected: https://community.hortonworks.com/comments/59274/view.html > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Assignee: Andre >Priority: Minor > Attachments: 1083.patch, screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15537340#comment-15537340 ] ASF GitHub Bot commented on NIFI-2709: -- Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1083 @bbende code should pass contrib-check and be ready for review. > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Assignee: Andre >Priority: Minor > Attachments: 1083.patch, screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542167#comment-15542167
]
ASF GitHub Bot commented on NIFI-2709:
--
Github user olegz commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1083#discussion_r81530413
--- Diff:
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestConsumeEmail.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.email;
+
+import com.icegreen.greenmail.user.GreenMailUser;
+import com.icegreen.greenmail.util.GreenMail;
+import com.icegreen.greenmail.util.ServerSetupTest;
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.Assert;
+import org.springframework.integration.mail.AbstractMailReceiver;
+
+import javax.mail.Message;
+import javax.mail.MessagingException;
+import javax.mail.Session;
+import javax.mail.internet.InternetAddress;
+import javax.mail.internet.MimeMessage;
+import java.lang.reflect.Field;
+import java.util.List;
+import java.util.Properties;
+
+import static org.junit.Assert.assertEquals;
+
+
+public class TestConsumeEmail {
--- End diff --
Why was the name of the class changed?
> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML
> messages with no RFC2822 headers. I have also tried setting Exchange to force
> emails to be text only, sadly only the body with some Content-Type: fields
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542168#comment-15542168
]
ASF GitHub Bot commented on NIFI-2709:
--
Github user olegz commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1083#discussion_r81530213
--- Diff:
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java
---
@@ -348,14 +345,11 @@ private void transfer(Message emailMessage,
ProcessContext context, ProcessSessi
long start = System.nanoTime();
FlowFile flowFile = processSession.create();
-flowFile = processSession.append(flowFile, new
OutputStreamCallback() {
-@Override
-public void process(final OutputStream out) throws IOException
{
-try {
-StreamUtils.copy(emailMessage.getInputStream(), out);
-} catch (MessagingException e) {
-throw new IOException(e);
-}
+flowFile = processSession.append(flowFile, out -> {
--- End diff --
This makes it incompatible with Java 7
> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML
> messages with no RFC2822 headers. I have also tried setting Exchange to force
> emails to be text only, sadly only the body with some Content-Type: fields
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542171#comment-15542171
]
ASF GitHub Bot commented on NIFI-2709:
--
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1083#discussion_r81530739
--- Diff:
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java
---
@@ -348,14 +345,11 @@ private void transfer(Message emailMessage,
ProcessContext context, ProcessSessi
long start = System.nanoTime();
FlowFile flowFile = processSession.create();
-flowFile = processSession.append(flowFile, new
OutputStreamCallback() {
-@Override
-public void process(final OutputStream out) throws IOException
{
-try {
-StreamUtils.copy(emailMessage.getInputStream(), out);
-} catch (MessagingException e) {
-throw new IOException(e);
-}
+flowFile = processSession.append(flowFile, out -> {
--- End diff --
NiFi 1.0 requires Java 8 or above?
> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML
> messages with no RFC2822 headers. I have also tried setting Exchange to force
> emails to be text only, sadly only the body with some Content-Type: fields
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542180#comment-15542180
]
ASF GitHub Bot commented on NIFI-2709:
--
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1083#discussion_r81531262
--- Diff:
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestConsumeEmail.java
---
@@ -0,0 +1,180 @@
+/*
+ * Licensed to the Apache Software Foundation (ASF) under one or more
+ * contributor license agreements. See the NOTICE file distributed with
+ * this work for additional information regarding copyright ownership.
+ * The ASF licenses this file to You under the Apache License, Version 2.0
+ * (the "License"); you may not use this file except in compliance with
+ * the License. You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package org.apache.nifi.processors.email;
+
+import com.icegreen.greenmail.user.GreenMailUser;
+import com.icegreen.greenmail.util.GreenMail;
+import com.icegreen.greenmail.util.ServerSetupTest;
+import org.apache.nifi.util.MockFlowFile;
+import org.apache.nifi.util.TestRunner;
+import org.apache.nifi.util.TestRunners;
+import org.junit.After;
+import org.junit.Before;
+import org.junit.Test;
+import org.junit.Assert;
+import org.springframework.integration.mail.AbstractMailReceiver;
+
+import javax.mail.Message;
+import javax.mail.MessagingException;
+import javax.mail.Session;
+import javax.mail.internet.InternetAddress;
+import javax.mail.internet.MimeMessage;
+import java.lang.reflect.Field;
+import java.util.List;
+import java.util.Properties;
+
+import static org.junit.Assert.assertEquals;
+
+
+public class TestConsumeEmail {
--- End diff --
The initial commit introduced a temporary test unit based on GreenMail, to
avoid overlap, I gave it a different name. After user confirmed patch worked, I
migrated some of the test you had created into the new jUnit.
This allowed me to run debug the two test units while I was developing the
final iteration of the test unit.
I guess Test*.java was used was this naming convention used in other
processors of the package, happy to rename back to the original if necessary.
> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML
> messages with no RFC2822 headers. I have also tried setting Exchange to force
> emails to be text only, sadly only the body with some Content-Type: fields
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542182#comment-15542182
]
ASF GitHub Bot commented on NIFI-2709:
--
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1083#discussion_r81531370
--- Diff:
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java
---
@@ -348,14 +345,11 @@ private void transfer(Message emailMessage,
ProcessContext context, ProcessSessi
long start = System.nanoTime();
FlowFile flowFile = processSession.create();
-flowFile = processSession.append(flowFile, new
OutputStreamCallback() {
-@Override
-public void process(final OutputStream out) throws IOException
{
-try {
-StreamUtils.copy(emailMessage.getInputStream(), out);
-} catch (MessagingException e) {
-throw new IOException(e);
-}
+flowFile = processSession.append(flowFile, out -> {
--- End diff --
having said that, if necessary, happy to try to avoid using the lambda
> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML
> messages with no RFC2822 headers. I have also tried setting Exchange to force
> emails to be text only, sadly only the body with some Content-Type: fields
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542193#comment-15542193
]
ASF GitHub Bot commented on NIFI-2709:
--
Github user olegz commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1083#discussion_r81532172
--- Diff:
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java
---
@@ -348,14 +345,11 @@ private void transfer(Message emailMessage,
ProcessContext context, ProcessSessi
long start = System.nanoTime();
FlowFile flowFile = processSession.create();
-flowFile = processSession.append(flowFile, new
OutputStreamCallback() {
-@Override
-public void process(final OutputStream out) throws IOException
{
-try {
-StreamUtils.copy(emailMessage.getInputStream(), out);
-} catch (MessagingException e) {
-throw new IOException(e);
-}
+flowFile = processSession.append(flowFile, out -> {
--- End diff --
The email support was introduced prior to 1.0, so if this patch only meant
for 1.0, then it's fine, but at the moment the JIRA is not versioned and I
assume the fix should be applied on both branches 0.* and master
> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML
> messages with no RFC2822 headers. I have also tried setting Exchange to force
> emails to be text only, sadly only the body with some Content-Type: fields
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542219#comment-15542219
]
ASF GitHub Bot commented on NIFI-2709:
--
Github user trixpan commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1083#discussion_r81533345
--- Diff:
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java
---
@@ -348,14 +345,11 @@ private void transfer(Message emailMessage,
ProcessContext context, ProcessSessi
long start = System.nanoTime();
FlowFile flowFile = processSession.create();
-flowFile = processSession.append(flowFile, new
OutputStreamCallback() {
-@Override
-public void process(final OutputStream out) throws IOException
{
-try {
-StreamUtils.copy(emailMessage.getInputStream(), out);
-} catch (MessagingException e) {
-throw new IOException(e);
-}
+flowFile = processSession.append(flowFile, out -> {
--- End diff --
@olegz happy to follow you lead but are you sure it isn't 1.0 only?
If I recall correctly the email processors were being ironed out few days
prior to 1.0 getting released.
> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML
> messages with no RFC2822 headers. I have also tried setting Exchange to force
> emails to be text only, sadly only the body with some Content-Type: fields
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[
https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542224#comment-15542224
]
ASF GitHub Bot commented on NIFI-2709:
--
Github user olegz commented on a diff in the pull request:
https://github.com/apache/nifi/pull/1083#discussion_r81533566
--- Diff:
nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/main/java/org/apache/nifi/processors/email/AbstractEmailProcessor.java
---
@@ -348,14 +345,11 @@ private void transfer(Message emailMessage,
ProcessContext context, ProcessSessi
long start = System.nanoTime();
FlowFile flowFile = processSession.create();
-flowFile = processSession.append(flowFile, new
OutputStreamCallback() {
-@Override
-public void process(final OutputStream out) throws IOException
{
-try {
-StreamUtils.copy(emailMessage.getInputStream(), out);
-} catch (MessagingException e) {
-throw new IOException(e);
-}
+flowFile = processSession.append(flowFile, out -> {
--- End diff --
actually my bad, i've mistaken it with JMS. Indeed they were introduced for
1.0 only, so all is good.
> Email processors with Exchange don't output to RFC2822 format
> -
>
> Key: NIFI-2709
> URL: https://issues.apache.org/jira/browse/NIFI-2709
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.0.0
> Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless
>Reporter: Emil Frank
>Assignee: Andre
>Priority: Minor
> Attachments: 1083.patch, screenshot-1.png
>
>
> When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft
> Exchange 2013 IMAP server the flowfiles which are produced are simple HTML
> messages with no RFC2822 headers. I have also tried setting Exchange to force
> emails to be text only, sadly only the body with some Content-Type: fields
> are outputed.
> This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used
> directly with these processors.
> In Python, I can force Exchange to output the headers by specify RFC822 in
> the connection settings:
> - https://docs.python.org/3/library/imaplib.html#imap4-example
> Is a similar option available for the spring mail framework?
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542533#comment-15542533 ] ASF GitHub Bot commented on NIFI-2709: -- Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1083 @olegz any feedback or :+1: ? > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Assignee: Andre >Priority: Minor > Fix For: 1.1.0 > > Attachments: 1083.patch, screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542536#comment-15542536 ] ASF GitHub Bot commented on NIFI-2709: -- Github user olegz commented on the issue: https://github.com/apache/nifi/pull/1083 @trixpan I thought I replied (3 hrs ago). Anyway it may not have been clear. I am ok, all is good. > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Assignee: Andre >Priority: Minor > Fix For: 1.1.0 > > Attachments: 1083.patch, screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542544#comment-15542544 ] ASF GitHub Bot commented on NIFI-2709: -- Github user trixpan commented on the issue: https://github.com/apache/nifi/pull/1083 @olegz thanks. I wasn't sure if the comment was referring to the Java 8 or the whole thing. Thanks for clarifying and thank you for reviewing. Cheers > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Assignee: Andre >Priority: Minor > Fix For: 1.1.0 > > Attachments: 1083.patch, screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (NIFI-2709) Email processors with Exchange don't output to RFC2822 format
[ https://issues.apache.org/jira/browse/NIFI-2709?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15542570#comment-15542570 ] ASF GitHub Bot commented on NIFI-2709: -- Github user asfgit closed the pull request at: https://github.com/apache/nifi/pull/1083 > Email processors with Exchange don't output to RFC2822 format > - > > Key: NIFI-2709 > URL: https://issues.apache.org/jira/browse/NIFI-2709 > Project: Apache NiFi > Issue Type: Bug > Components: Extensions >Affects Versions: 1.0.0 > Environment: Ubuntu 16.04 LTS with openjdk-8-jre-headless >Reporter: Emil Frank >Assignee: Andre >Priority: Minor > Fix For: 1.1.0 > > Attachments: 1083.patch, screenshot-1.png > > > When using the new ConsumeIMAP and ConsumePOP3 processors with a Microsoft > Exchange 2013 IMAP server the flowfiles which are produced are simple HTML > messages with no RFC2822 headers. I have also tried setting Exchange to force > emails to be text only, sadly only the body with some Content-Type: fields > are outputed. > This mean that ExtractEmailHeaders and ExtractEmailAttachments cannot be used > directly with these processors. > In Python, I can force Exchange to output the headers by specify RFC822 in > the connection settings: > - https://docs.python.org/3/library/imaplib.html#imap4-example > Is a similar option available for the spring mail framework? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
