Re: Nifi 1.1.0 cluster on Docker Swarm

2017-03-23 Thread Bryan Bende
I created a JIRA for a possible improvement we can make to help with this scenario: https://issues.apache.org/jira/browse/NIFI-3642 On Wed, Mar 22, 2017 at 3:53 PM, Bryan Bende wrote: > I was looking into this a little bit and although I don't have a > definitive answer, here is what I determine

Re: Nifi 1.1.0 cluster on Docker Swarm

2017-03-22 Thread Bryan Bende
I was looking into this a little bit and although I don't have a definitive answer, here is what I determined... There are basically two pairs of values being passed around... 1) The cluster socket address and cluster socket port (nifi.cluster.node.address and nifi.cluster.node.protocol.port) whi

Re: Nifi 1.1.0 cluster on Docker Swarm

2017-03-17 Thread ddewaele
Hi Jeremy, The issue we are facing is that we need to keep the nifi.web.http.host blank in order to have a working swarm setup, but this conflicts with the way nifi does cluster communication. Let me try to explain: I have 2 nifi instances (cluster nodes) in a docker swarm connected to zookeeper

Re: Nifi 1.1.0 cluster on Docker Swarm

2017-03-16 Thread Raf Huys
Still no sigar... I'm just posting some more output in case somebody is in a helpful state of mind. So logs say we're running smoothly, web.port is on 8085, web.host is the FQDN on the overlay network. Inside the container, this looks as expected. Something is listening on port 8085, and it looks

Re: Nifi 1.1.0 cluster on Docker Swarm

2017-03-15 Thread Raf Huys
Great. I have an overlay network listed in docker-compose, sure. I want to narrow down the issue, so if the Nifi configuration is working, that's already a big step. Thank you for your time. On Wed, Mar 15, 2017 at 3:28 PM, Jeremy Dyer wrote: > Raf - Ok so good news and bad news. Good news its w

Re: Nifi 1.1.0 cluster on Docker Swarm

2017-03-15 Thread Jeremy Dyer
Raf - Ok so good news and bad news. Good news its working for me. Bad news its working for me =) Here is the complete list of things that I changed. Hopefully this can at least really help narrow down what is causing the issue. - I ran on a single machine. All that was available to me while at the

Re: Nifi 1.1.0 cluster on Docker Swarm

2017-03-15 Thread Jeremy Dyer
Thanks for sharing! Give me a little bit to test out and I will get back to you. I obviously will not have your custom processors so will have to make that small adjustment in the Dockerfile but it is highly unlikely that is the actual issue anyway. On Wed, Mar 15, 2017 at 9:04 AM, Raf Huys wrote

Re: Nifi 1.1.0 cluster on Docker Swarm

2017-03-15 Thread Raf Huys
Awesome, we wrap https://github.com/mkobit/docker-nifi to add some custom processors: Tx FROM mkobit/nifi:1.1.0-openjdk-8COPY . /opt/nifi/WORKDIR /opt/nifi/RUN cp dependencies/*.nar $NIFI_HOME/libEXPOSE 8080CMD tail -F ${NIFI_HOME}/logs/nifi-app.log & ${NIFI_HOME}/bin/nifi.sh run On Wed,

Re: Nifi 1.1.0 cluster on Docker Swarm

2017-03-15 Thread Jeremy Dyer
Raf - Glad to help you troubleshoot this. Are you using the official NiFi docker image? If not could you please share your Dockerfile so I can try out your configuration locally On Wed, Mar 15, 2017 at 6:48 AM, Raf Huys wrote: > I'm having difficulty getting my 2-node Nifi cluster up and running

Nifi 1.1.0 cluster on Docker Swarm

2017-03-15 Thread Raf Huys
I'm having difficulty getting my 2-node Nifi cluster up and running with a separate 3-node Zookeeper ensemble. Part of the issue might be that it's being ran as a Docker Swarm cluster (with an overlay network). In particular, I cannot connect to the UI. Configuration and logs can be found here: h