Re: Nifi 1.1.0 cluster on Docker Swarm

2017-03-16 Thread Raf Huys
0.0.9.20 port 8085: Connection refused * Closing connection 0 curl: (7) Failed to connect to 10.0.9.20 port 8085: Connection refused Making logs more verbose, did also not help (in case the `Connection refused` was returned from the Nifi application). nifi-user.log is empty. On Wed, Mar 15, 2017 at

Re: Nifi 1.1.0 cluster on Docker Swarm

2017-03-15 Thread Raf Huys
e to >> make that small adjustment in the Dockerfile but it is highly unlikely that >> is the actual issue anyway. >> >> On Wed, Mar 15, 2017 at 9:04 AM, Raf Huys wrote: >> >>> Awesome, we wrap https://github.com/mkobit/docker-nifi to add some custom

Re: Nifi 1.1.0 cluster on Docker Swarm

2017-03-15 Thread Raf Huys
Wed, Mar 15, 2017 at 1:59 PM, Jeremy Dyer wrote: > Raf - Glad to help you troubleshoot this. Are you using the official NiFi > docker image? If not could you please share your Dockerfile so I can try > out your configuration locally > > On Wed, Mar 15, 2017 at 6:48 AM, Raf Huys wro

Nifi 1.1.0 cluster on Docker Swarm

2017-03-15 Thread Raf Huys
tion issue. However: I cannot curl Nifi on 8080 on the container itself. So I'm stuck. What would help me out, is a doublecheck on the `nifi.properties` file for obvious errors. The nifi-properties file on the second node is similar (except for hostnames) Tx Raf Huys

Re: Kafka offset different from Consume_Kafka_0_10 flowfile count

2017-01-16 Thread Raf Huys
ocess published 807 messages, then > ConsumeKafka can only consume 700 messages those are sent after > ConsumeKafka is connected to the topic. > > Otherwise, if it's reproducible, please let us know of your > environment detail, such as NiFi clustered/standalone, num

Kafka offset different from Consume_Kafka_0_10 flowfile count

2017-01-11 Thread Raf Huys
s, so every incoming message should result in 1 flowFile to my understanding. Not really sure what to think of this. Are there any other places i can look for the total amount of flowfiles (in/out) during a certain amount of time? -- Mvg, Raf Huys

Re: merge flowfiles

2016-12-19 Thread Raf Huys
l-case aggregation processor [1]. >> >> With some more details around your scenario we might be able to figure >> out how to get it to work for you with the standard processors. >> >> [1] https://issues.apache.org/jira/browse/NIFI-1926 >> >> On Mon, Dec 19,

merge flowfiles

2016-12-19 Thread Raf Huys
I want to batch incoming flowfiles based on an attribute. As soon as this attributes' value changes, the current batch should be transferred downstream and be reset. So basically I'm looking for a tumbling window. Can this be done with the MergeContent processor (which strategy?) or should I write

Re: puttcp

2016-11-16 Thread Raf Huys
"some time later it becomes worse... no flowfiles are >> generated", are you saying GenerateFlowFile is no longer generating flow >> files? when this happens do you have a lot of flow files in queues, and do >> you have back-pressure configured? >> >> Thanks,

puttcp

2016-11-16 Thread Raf Huys
ove pipeline actually stops doing anything...no flowfiles are generated, no errors are thrown... Thanks, Raf Huys

Re: stream one large file, only once

2016-11-14 Thread Raf Huys
il based on the JVM heap > allocated > > and file size. A recommended best practice in this case is to introduce a > > series of 2 SplitText processors. 1st pass would split into e.g. 10 000 > row > > chunks, 2nd - into individual. Adjust for your expected file sizes and > >

stream one large file, only once

2016-11-14 Thread Raf Huys
hat's the typical Nifi for this? Tx Raf Huys