[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486503#comment-16486503
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user asfgit closed the pull request at:

https://github.com/apache/nifi/pull/2657


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF subversion and git services (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486502#comment-16486502
 ] 

ASF subversion and git services commented on NIFI-5109:
---

Commit dd8b25ab48314dde8cad351a9b564196d807d90e in nifi's branch 
refs/heads/master from [~dalong]
[ https://git-wip-us.apache.org/repos/asf?p=nifi.git;h=dd8b25a ]

NIFI-5109 Reset justElectedPrimaryNode flag right after reelection happen

Read full cluster state and return if no update required

This closes #2657.

Signed-off-by: Koji Kawamura 


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16486500#comment-16486500
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user ijokarumawak commented on the issue:

https://github.com/apache/nifi/pull/2657
  
@viazovskyi LGTM, +1. Thanks for finding this, providing the detailed steps 
to produce the issue and fixing it! The newly added unit test clarifies how it 
should behave. Merging in.


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483943#comment-16483943
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2657
  
@ijokarumawak  can you take a look and merge if seems ok?


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483941#comment-16483941
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2657
  
@mgaido91 I dont have any other comments for now - i'm looking into some 
other things so if ya'll are good then I think we're set.


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483652#comment-16483652
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user mgaido91 commented on the issue:

https://github.com/apache/nifi/pull/2657
  
LGTM, thanks @viazovskyi.

@ijokarumawak @joewitt any other comments?


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16483489#comment-16483489
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user viazovskyi commented on the issue:

https://github.com/apache/nifi/pull/2657
  
I think now PR looks better


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482601#comment-16482601
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user viazovskyi commented on the issue:

https://github.com/apache/nifi/pull/2657
  
Thanks Marco, I'll try it a bil later


On Mon, May 21, 2018 at 5:03 PM Marco Gaido 
wrote:

> @viazovskyi  I suggest you something like:
>
> git checkout nifi/master
> git checkout -b nifi-5109_2
> // do your changes on this branch and commit them
> git branch -D nifi-5109
> git branch -m nifi-5109
> git push -f -u origin nifi-5109
>
> If you want details/explanation on each command let me know. Or let me
> know if I can help you somehow. Thanks.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> , or mute
> the thread
> 

> .
>



> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482528#comment-16482528
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user mgaido91 commented on the issue:

https://github.com/apache/nifi/pull/2657
  
@viazovskyi I suggest you something like:
```
git checkout nifi/master
git checkout -b nifi-5109_2
// do your changes on this branch and commit them
git branch -D nifi-5109
git branch -m nifi-5109
git push -f -u origin nifi-5109
```
If you want details/explanation on each command let me know.  Or let me 
know if I can help you somehow. Thanks.


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482406#comment-16482406
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user mgaido91 commented on the issue:

https://github.com/apache/nifi/pull/2657
  
@viazovskyi I think that the easiest way is probably creating a new branch 
locally from the master, reapply your changes on it and force push to this 
branch.


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482401#comment-16482401
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user viazovskyi commented on the issue:

https://github.com/apache/nifi/pull/2657
  
yep, i see, i'm new for git, could you please guide how can i rollback this
merge?

On Mon, May 21, 2018 at 1:37 PM Marco Gaido 
wrote:

> @viazovskyi  your last merge screwed a bit
> this PR. May you please restore it? Thanks.
>
> —
> You are receiving this because you were mentioned.
> Reply to this email directly, view it on GitHub
> , or mute
> the thread
> 

> .
>



> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482386#comment-16482386
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user mgaido91 commented on the issue:

https://github.com/apache/nifi/pull/2657
  
@viazovskyi your last merge screwed a bit this PR. May you please restore 
it? Thanks.


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16482032#comment-16482032
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user viazovskyi commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2657#discussion_r189470396
  
--- Diff: 
nifi-nar-bundles/nifi-extension-utils/nifi-processor-utils/src/main/java/org/apache/nifi/processor/util/list/AbstractListProcessor.java
 ---
@@ -375,7 +376,7 @@ public void onTrigger(final ProcessContext context, 
final ProcessSession session
 // If our determined timestamp is the same as that 
of our last listing, skip this execution as there are no updates
 if 
(minTimestampToListMillis.equals(this.lastListedLatestEntryTimestampMillis)) {
 context.yield();
--- End diff --

ok, never mind. I'll give variable new name and move context.yield()


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481994#comment-16481994
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user viazovskyi commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2657#discussion_r189464759
  
--- Diff: 
nifi-nar-bundles/nifi-extension-utils/nifi-processor-utils/src/main/java/org/apache/nifi/processor/util/list/AbstractListProcessor.java
 ---
@@ -375,7 +376,7 @@ public void onTrigger(final ProcessContext context, 
final ProcessSession session
 // If our determined timestamp is the same as that 
of our last listing, skip this execution as there are no updates
 if 
(minTimestampToListMillis.equals(this.lastListedLatestEntryTimestampMillis)) {
 context.yield();
--- End diff --

I'd like to move the original 'if' with yield out of the loop closer catch, 
then I don't need this new 'if' and variable. What do you think?


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481985#comment-16481985
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2657#discussion_r189462936
  
--- Diff: 
nifi-nar-bundles/nifi-extension-utils/nifi-processor-utils/src/main/java/org/apache/nifi/processor/util/list/AbstractListProcessor.java
 ---
@@ -375,7 +376,7 @@ public void onTrigger(final ProcessContext context, 
final ProcessSession session
 // If our determined timestamp is the same as that 
of our last listing, skip this execution as there are no updates
 if 
(minTimestampToListMillis.equals(this.lastListedLatestEntryTimestampMillis)) {
 context.yield();
--- End diff --

no I just meant moving this line `context.yield();` in the `if` you 
introduced, before the return statement.


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481968#comment-16481968
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user viazovskyi commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2657#discussion_r189461079
  
--- Diff: 
nifi-nar-bundles/nifi-extension-utils/nifi-processor-utils/src/main/java/org/apache/nifi/processor/util/list/AbstractListProcessor.java
 ---
@@ -375,7 +376,7 @@ public void onTrigger(final ProcessContext context, 
final ProcessSession session
 // If our determined timestamp is the same as that 
of our last listing, skip this execution as there are no updates
 if 
(minTimestampToListMillis.equals(this.lastListedLatestEntryTimestampMillis)) {
 context.yield();
--- End diff --

Do you mean move the whole if and do not use yielded variable?


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481887#comment-16481887
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2657#discussion_r189455715
  
--- Diff: 
nifi-nar-bundles/nifi-extension-utils/nifi-processor-utils/src/main/java/org/apache/nifi/processor/util/list/AbstractListProcessor.java
 ---
@@ -375,7 +376,7 @@ public void onTrigger(final ProcessContext context, 
final ProcessSession session
 // If our determined timestamp is the same as that 
of our last listing, skip this execution as there are no updates
 if 
(minTimestampToListMillis.equals(this.lastListedLatestEntryTimestampMillis)) {
 context.yield();
--- End diff --

nit: I'd prefer if we move this before the return statement


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481888#comment-16481888
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2657#discussion_r189455719
  
--- Diff: 
nifi-nar-bundles/nifi-extension-utils/nifi-processor-utils/src/main/java/org/apache/nifi/processor/util/list/AbstractListProcessor.java
 ---
@@ -356,6 +356,7 @@ private EntityListing deserialize(final String 
serializedState) throws JsonParse
 @Override
 public void onTrigger(final ProcessContext context, final 
ProcessSession session) throws ProcessException {
 Long minTimestampToListMillis = 
lastListedLatestEntryTimestampMillis;
+boolean yielded = false;
--- End diff --

nit: I'd propose a more meaningful name, like `isUpToDate` or something 
similar.


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16481777#comment-16481777
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user viazovskyi commented on the issue:

https://github.com/apache/nifi/pull/2657
  
@mgaido91 Adding UT I realized that just reset the flag and then return is 
not enough, because cluster state is not fully read (e.g. 
latestIdentifiersProcessed was not getting initialized) if i do return after 
yield. Please review the new commit after all checks completed. The idea is 
give the loop a chance to iterate over all read cluster' state and if processor 
should yield then exit the method after resetting the flag.


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16480336#comment-16480336
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user ijokarumawak commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2657#discussion_r189195817
  
--- Diff: 
nifi-nar-bundles/nifi-extension-utils/nifi-processor-utils/src/main/java/org/apache/nifi/processor/util/list/AbstractListProcessor.java
 ---
@@ -375,7 +375,7 @@ public void onTrigger(final ProcessContext context, 
final ProcessSession session
 // If our determined timestamp is the same as that 
of our last listing, skip this execution as there are no updates
 if 
(minTimestampToListMillis.equals(this.lastListedLatestEntryTimestampMillis)) {
 context.yield();
-return;
+break;
--- End diff --

@mgaido91 Good catch, I was only looking at the method partially.. Thanks, 
that makes sense to me.


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478690#comment-16478690
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user mgaido91 commented on the issue:

https://github.com/apache/nifi/pull/2657
  
@viazovskyi do you think it is possible to add a UT for this?


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16478687#comment-16478687
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user mgaido91 commented on a diff in the pull request:

https://github.com/apache/nifi/pull/2657#discussion_r188866746
  
--- Diff: 
nifi-nar-bundles/nifi-extension-utils/nifi-processor-utils/src/main/java/org/apache/nifi/processor/util/list/AbstractListProcessor.java
 ---
@@ -375,7 +375,7 @@ public void onTrigger(final ProcessContext context, 
final ProcessSession session
 // If our determined timestamp is the same as that 
of our last listing, skip this execution as there are no updates
 if 
(minTimestampToListMillis.equals(this.lastListedLatestEntryTimestampMillis)) {
 context.yield();
-return;
+break;
--- End diff --

sorry, I just realize that now, with this fix we are performing an extra 
listing which is not needed. Maybe it is better to just have return as before, 
but prepend a `justElectedPrimaryNode = false;`. So I am thinking of 
substituting this line with:

```
justElectedPrimaryNode = false;
return;
```

what do you think @ijokarumawak @joewitt  @viazovskyi ?
Thanks


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477909#comment-16477909
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user viazovskyi commented on the issue:

https://github.com/apache/nifi/pull/2657
  
done


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16477507#comment-16477507
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user mgaido91 commented on the issue:

https://github.com/apache/nifi/pull/2657
  
I also agree on @ijokarumawak's suggestion, may you please update the PR 
accordingly @viazovskyi ?


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16471041#comment-16471041
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user viazovskyi commented on the issue:

https://github.com/apache/nifi/pull/2657
  
hi @ijokarumawak, it looks good to me, @joewitt what do you think?


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16468163#comment-16468163
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user ijokarumawak commented on the issue:

https://github.com/apache/nifi/pull/2657
  
Maybe I'm thinking the issue oversimplified, but changing the line 378 
`return` to `break` would address the issue while ensuring that we don't reset 
the flag until we know we've pulled the latest state, wouldn't it?

```
 if (LATEST_LISTED_ENTRY_TIMESTAMP_KEY.equals(k)) {
 minTimestampToListMillis = Long.parseLong(v);
 // If our determined timestamp is the same as that 
of our last listing, skip this execution as there are no updates
 if 
(minTimestampToListMillis.equals(this.lastListedLatestEntryTimestampMillis)) {
 context.yield();
 return; // How about changing this to 'break'?
 } else {
 this.lastListedLatestEntryTimestampMillis = 
minTimestampToListMillis;
 }
 } else if 
(LAST_PROCESSED_LATEST_ENTRY_TIMESTAMP_KEY.equals(k)) {
 this.lastProcessedLatestEntryTimestampMillis = 
Long.parseLong(v);
 } else if (k.startsWith(IDENTIFIER_PREFIX)) {
 latestIdentifiersProcessed.add(v);
 }
```


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16467617#comment-16467617
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2657
  
@viazovskyi i think it is important to address that issue (ensuring we dont 
reset the state check until we know we've pulled latest state if available) 
before we move forward.


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16467539#comment-16467539
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user viazovskyi commented on the issue:

https://github.com/apache/nifi/pull/2657
  
@joewitt Hi Joe, is there any update?


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465986#comment-16465986
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user viazovskyi commented on the issue:

https://github.com/apache/nifi/pull/2657
  
I agree, it's possible. Then probably place flag reset after the getting 
the state, but before checking what we have read?


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465971#comment-16465971
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2657
  
With the location of the reset now being where it is placed what happens in 
the event that the pulling of state from the state manager results in an 
exception?  My read is that it would mean we'd not try again to pull the state 
later because we cleared the flag but also we'd not have the latest state.

Do you not believe that is possible?


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465957#comment-16465957
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user viazovskyi commented on the issue:

https://github.com/apache/nifi/pull/2657
  
@joewitt  not sure what is another problem you're talking about, the 
original [issue](https://issues.apache.org/jira/browse/NIFI-5109) is about 
onTrigger method never goes further line 377 and stops calling performListing 
after reelection of primary happened and same node became the primary again, 
that's why I reset the flag and it makes sense to me, because the election 
finished and further logic should only depend on another 2 fields 
lastListedLatestEntryTimestampMillis, lastProcessedLatestEntryTimestampMillis 
which are initialized from the state manager. But if performListing is not 
calling anymore then no any updates for the state and these 2 fields keep the 
same values. huh, hope it shows the issue in details :) @joewitt what is 
another issue?


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465933#comment-16465933
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2657
  
@viazovskyi Ok fair enough.  So between that and the other problem which is 
you could fail to properly handle the case when it does become primary i think 
we still have some logic work to do.  


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465930#comment-16465930
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user viazovskyi commented on the issue:

https://github.com/apache/nifi/pull/2657
  
@joewitt if i put it closer to the catch i'm afraid the flag will not be 
reset due to [line 
377](https://github.com/apache/nifi/blob/rel/nifi-1.6.0/nifi-nar-bundles/nifi-extension-utils/nifi-processor-utils/src/main/java/org/apache/nifi/processor/util/list/AbstractListProcessor.java#L377)
 where onTrigger method hits the context.yield() and returns if nothing was 
changed from the previous run. That was the initial idea to reset it as soon as 
possible.


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465898#comment-16465898
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2657
  
@viazovskyi I just noticed this processor is trigger serially which means 
it isn't designed for concurrent use and in nifi will only ever have one 
thread.  So we can relax on the check/modify piece.  I think you want the flag 
reset lower in the block (end of try inside try/catch closer to where it was).


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465202#comment-16465202
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2657
  
fields being volatile can be fine.  volatile makes a more explicit 
assertion about how java will guarantee the value to be read across threads.  
The issue, at least from a quick look, is the potential race condition between 
when it is checked versus when it is set.  I belive it looked like more than 
ine thread could get through the check.

It needs to be looked into more and if necessary just protected for the 
whole check/set operation to be guarded by a lock.


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16465195#comment-16465195
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user viazovskyi commented on the issue:

https://github.com/apache/nifi/pull/2657
  
Thanks Joe. There are more volatile fields 
(lastListedLatestEntryTimestampMillis, lastProcessedLatestEntryTimestampMillis, 
etc) altered after the check. So this should be redesigned what do you think?


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463851#comment-16463851
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user joewitt commented on the issue:

https://github.com/apache/nifi/pull/2657
  
@viazovskyi I would need to evaluate more thoroughly but I believe this 
change has a check and reset concurrent issue.  While you're altering a 
volatile field (so the change will be consistent across threads in reading) 
what is not guaranteed is the timing of when the check happens versus the 
reset.  This might need an alternative approach or a lock.

cc @markap14 


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[Joseph Witt (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16463062#comment-16463062
 ] 

Joseph Witt commented on NIFI-5109:
---

Please dont set the fix version until we can get review traction.  There are a 
lot of PRs to go through.  Definitely appreciate the contribution

> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Commented] (NIFI-5109) AbstractListProcessor stop calling performListing after primary node reelection

[ASF GitHub Bot (JIRA)]

[ 
https://issues.apache.org/jira/browse/NIFI-5109?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16460148#comment-16460148
 ] 

ASF GitHub Bot commented on NIFI-5109:
--

Github user viazovskyi commented on the issue:

https://github.com/apache/nifi/pull/2657
  
@bbende @ijokarumawak @mcgilman Guys, would you like to perform code review?


> AbstractListProcessor stop calling performListing after primary node 
> reelection
> ---
>
> Key: NIFI-5109
> URL: https://issues.apache.org/jira/browse/NIFI-5109
> Project: Apache NiFi
>  Issue Type: Bug
>  Components: Core Framework
>Affects Versions: 1.6.0
>Reporter: Maksym Viazovskyi
>Priority: Major
> Attachments: 
> 0001-NIFI-2109-Reset-justElectedPrimaryNode-flag-right-af.patch, 
> PutAndListSFTP.xml, docker-compose.yml
>
>   Original Estimate: 72h
>  Remaining Estimate: 72h
>
> AbstractListProcessor stops performing listing of entities when after 
> reelection the same node becomes Primary.
> STEPS TO REPRODUCE:
> 0. Make sure that you have docker 18.03.0-ce
> 1. Download attached [^docker-compose.yml] and [^PutAndListSFTP.xml]
> 2. Run *_docker-compose up -d_* in the folder where you downloaded 
> docker-compose.yml. This command will run 3 containers - Zookeeper, NiFi 
> 1.6.0 in Cluster Mode with 1 node, SFTPs server 
> 3. After NiFi is up and running (check http://localhost:8080/nifi) upload 
> PutAndListSFTP.xml template and place it on NiFi canvas. It has simple logic 
> - generate flow file each 10 seconds and put it on SFTP, in parallel 
> ListSFTP processor "listens" for the new files and pushes data about them to 
> NiFi log with LogAttribute processor 
> 4. Update PutSFTP and ListSFTP processors with password admin (to check SFTP 
> container use following credentials SFTP://admin:admin@localhost:) 
> 5. Run the flow 
> 6. Check the NiFi log (*_docker logs -f --tail 100 nifi_*) - you should see 
> attributes for the uploaded files on SFTP 
> 7. Check that generated each 10 seconds files appear on SFTP /upload folder 
> 8. Run following command to restart Zookeeper to put NiFi in reelection 
> state: *_docker restart zookeeper_*
> 9. After reelection phase finished check that new files appears on SFTP, but 
> NO new files info appears in NiFi log
>  



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)