[
https://issues.apache.org/jira/browse/NIFI-2519?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15420227#comment-15420227
]
Joseph Witt commented on NIFI-2519:
-----------------------------------
Created a new PR based on Oleg's previous PR. After tweaking and removing
issues blocking proper processor stopping there were data corruption issues
still present during testing. Further evaluation revealed it was due to
improper use of the subethasmtp API specifically as it relates to the
MessageHandler which is something they expect to create for each message but
that our implementation was creating as a single instance for the processor.
Also noticed that the recipient method was just overwriting previous recipient
so changed that to accept multiple calls and write out all specified recipients.
So, taking a step back and in looking through this discussion thread and
thinking about it the requirements are:
SubethaSMTP has its own threading and lifecycle model. It has a message
handler with start/stop methods we can connect to.
NiFi has its own threading and lifecycle model.
We want to ensure that unless we capture and commit data we return exceptions
to the user. This ensures no data loss but could mean duplication. This is ok.
We want to ensure data is streamed from the client directly to disk.
We want to ensure the client cannot send arbitrarily large data.
We want to ensure Subethasmtp doesn't open too many connections.
We want to ensure when the processor is stopped it will kill all the
subethasmtp threads.
We want to capture all the flow file metadata we can about the client ip, host,
certificates, etc..
We should be able to give the message hanlder the process session factory and
let it handle the lifecycle of nifi directly.
We don't need to make NiFi threads in addition to their threads. Let's just
use their model.
So implemented PR 856.
Unit tests cover normal non secure and TLS based transfers and excessive
message size.
Tested excessive sized message. Works.
Tested manual mailing with intentional timeouts during DATA and other commands.
Works.
Tested killing sender and killing ListenSMTP. Behaved as expected and well.
Tested with more senders than ListenSMTP could handle. It rejected excess
requests.
Tested at full rate. Seems reasonable.
Tested and can no longer reproduce threading issues.
> TestListenSMTP ValidEmail fails during parallel build
> -----------------------------------------------------
>
> Key: NIFI-2519
> URL: https://issues.apache.org/jira/browse/NIFI-2519
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
> Reporter: Joseph Witt
> Assignee: Oleg Zhurakousky
> Fix For: 1.0.0
>
> Attachments:
> 0001-NIFI-2519-fixed-max-connections-and-max-size-enforce.patch
>
>
> While running a full NiFi parallel build received the following. So there is
> some test issue at least that is impacting build stability.
> [INFO] --- maven-compiler-plugin:3.2:testCompile (default-testCompile) @
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Compiling 4 source files to
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/test-classes
> [WARNING]
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[122,24]
> [deprecation] stop() in Thread has been deprecated
> [WARNING]
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[186,24]
> [deprecation] stop() in Thread has been deprecated
> [WARNING]
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/src/test/java/org/apache/nifi/processors/email/TestListenSMTP.java:[307,24]
> [deprecation] stop() in Thread has been deprecated
> [INFO]
> [INFO] --- maven-compiler-plugin:3.2:testCompile (groovy-tests) @
> nifi-email-processors ---
> [INFO] Changes detected - recompiling the module!
> [INFO] Nothing to compile - all classes are up to date
> [INFO]
> [INFO] --- maven-surefire-plugin:2.18:test (default-test) @
> nifi-email-processors ---
> [INFO] Surefire report directory:
> /home/travis/build/apache/nifi/nifi-nar-bundles/nifi-email-bundle/nifi-email-processors/target/surefire-reports
> [INFO] Using configured provider
> org.apache.maven.surefire.junit4.JUnit4Provider
> -------------------------------------------------------
> T E S T S
> -------------------------------------------------------
> Running org.apache.nifi.processors.email.TestListenSMTP
> Tests run: 4, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 1.473 sec <<<
> FAILURE! - in org.apache.nifi.processors.email.TestListenSMTP
> ValidEmail(org.apache.nifi.processors.email.TestListenSMTP) Time elapsed:
> 0.038 sec <<< FAILURE!
> java.lang.AssertionError: Sending email failed
> at org.junit.Assert.fail(Assert.java:88)
> at org.junit.Assert.assertTrue(Assert.java:41)
> at org.junit.Assert.assertFalse(Assert.java:64)
> at
> org.apache.nifi.processors.email.TestListenSMTP.ValidEmail(TestListenSMTP.java:188)
> Running org.apache.nifi.processors.email.TestExtractEmailAttachments
> Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.1 sec - in
> org.apache.nifi.processors.email.TestExtractEmailAttachments
> Running org.apache.nifi.processors.email.TestExtractEmailHeaders
> Tests run: 3, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.029 sec -
> in org.apache.nifi.processors.email.TestExtractEmailHeaders
> Results :
> Failed tests:
> TestListenSMTP.ValidEmail:188 Sending email failed
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
