RE: Question about [MergeContent] processor

do: quinta-feira, 5 de novembro de 2020 23:16 Para: users@nifi.apache.org Assunto: Question about [MergeContent] processor Hi all, My have a job consist of following steps: first consuming data from kafka, and then packing data every 5 minutes into one file, finally put the packed file into hdfs. I

Question about [MergeContent] processor

Hi all, My have a job consist of following steps: first consuming data from kafka, and then packing data every 5 minutes into one file, finally put the packed file into hdfs. I use the [MergeContent] processor to accomplish the “packing” step. The properties of MergeContent I configured is list

Re: MergeContent processor

in directory, I want to merge >> them and the output I want is 1 flowfile for further processing ( I want to >> extract folder name and path of that 8 files). >> >> I tried with MergeContent processor and this is my setup. >> >> >> >> But every time i run the flow I get multiple flowfiles in the output >> queue. Sometimes 2, sometimes 3 or even 4. What am I doing wrong? >> Or is there some other way to do this in Nifi? >> >> Thanks in advance, >> BR. >> Tom >> > >

Re: MergeContent processor

in one directory. So if I have e.g. 8 files in directory, I want to merge them and the output I want is 1 flowfile for further processing ( I want to extract folder name and path of that 8 files). I tried with MergeContent processor and this is my setup. But every time i run the flow I

Re: MergeContent processor

lder name and path of that 8 files). > > I tried with MergeContent processor and this is my setup. > > [image: image.png] > > But every time i run the flow I get multiple flowfiles in the output > queue. Sometimes 2, sometimes 3 or even 4. What am I doing wrong? > Or

MergeContent processor

with MergeContent processor and this is my setup. [image: image.png] But every time i run the flow I get multiple flowfiles in the output queue. Sometimes 2, sometimes 3 or even 4. What am I doing wrong? Or is there some other way to do this in Nifi? Thanks in advance, BR. Tom

Re: A question about [MergeContent] processor

Jianan Zhang wrote: > Hello Mike, > This link helps a lot, and I found another cause for the problems I met. > I am running my job on a cluster of three nodes, and the MergeContent > processor's "Scheduling--Execution" field is set to "all nodes", that ma

Re: A question about [MergeContent] processor

Hello Mike, This link helps a lot, and I found another cause for the problems I met. I am running my job on a cluster of three nodes, and the MergeContent processor's "Scheduling--Execution" field is set to "all nodes", that makes the MergeContent processor will randomly

Re: A question about [MergeContent] processor

hat? > > Cheers Josef > > > > > > *From: *Jianan Zhang > *Reply-To: *"users@nifi.apache.org" > *Date: *Friday, 4 January 2019 at 12:46 > *To: *"users@nifi.apache.org" > *Subject: *Re: A question about [MergeContent] processor > &

Re: A question about [MergeContent] processor

Hi Josef, Thanks for reply. In my opinion the “Minimum Number of Entries” is should not and can not stronger than the “Max Bin Age”. Suppose I have only ONE flowfile from datasource put into MergeContent processor, and I set "Minimum Number of Entries" = 2, then this ONE flowfile

Re: A question about [MergeContent] processor

-To: "users@nifi.apache.org" Date: Friday, 4 January 2019 at 11:16 To: "users@nifi.apache.org" Subject: A question about [MergeContent] processor Hi all, I have a job consist of following steps: first consuming data from kafka, and then packing data every 5 minutes into o

A question about [MergeContent] processor

Hi all, I have a job consist of following steps: first consuming data from kafka, and then packing data every 5 minutes into one file, finally put the packed file into hdfs. I use the [MergeContent] processor to accomplish the “packing” step. The properties of MergeContent I configured is list

Re: NPE MergeContent processor

with the reaction to issues/ bugs etc. in this community. Regards Conrad From: Mark Payne <marka...@hotmail.com> Reply-To: "users@nifi.apache.org" <users@nifi.apache.org> Date: Tuesday, 15 November 2016 at 18:09 To: "users@nifi.apache.org" <users@nifi.apache.org> S

Re: NPE MergeContent processor

>> Does this help at all? >> If you need anything else just shout. >> Regards >> Conrad >> >> >> >> >> >> additional out of shot >> · compression level : 1 >> · Keep Path : false >> >> >> From:

Re: NPE MergeContent processor

oes this help at all? >> If you need anything else just shout. >> Regards >> Conrad >> >> >> >> >> >> additional out of shot >> · compression level : 1 >> · Keep Path : false >> >> >> From: Ole

Re: NPE MergeContent processor

; Conrad > > > > > > additional out of shot > · compression level : 1 > · Keep Path : false > > > From: Oleg Zhurakousky <ozhurakou...@hortonworks.com> > Reply-To: "users@nifi.apache.org" <users@nifi.apache.org> > Date: Th

Re: NPE MergeContent processor

quot; <users@nifi.apache.org<mailto:users@nifi.apache.org>> Subject: Re: NPE MergeContent processor Conrad Any chance you an provide a bit more info about your flow? I was able to find a condition when something like this can happen, but it would have to be with some legacy NiFi dis

Re: NPE MergeContent processor

@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.org>> Date: Thursday, 10 November 2016 at 14:59 To: "users@nifi.apache.org<mailto:users@nifi.apache.org>" <users@nifi.apache.org<mailto:users@nifi.apache.or

Re: NPE MergeContent processor

. Regards Conrad From: Bryan Bende <bbe...@gmail.com> Reply-To: "users@nifi.apache.org" <users@nifi.apache.org> Date: Thursday, 10 November 2016 at 14:59 To: "users@nifi.apache.org" <users@nifi.apache.org> Subject: Re: NPE MergeContent processor Conrad

Re: NPE MergeContent processor

Conrad, Thanks for reporting this. I wonder if this is also related to: https://issues.apache.org/jira/browse/NIFI-2015 Seems like there is some case where the UUID is ending up as null. -Bryan On Wed, Nov 9, 2016 at 11:57 AM, Conrad Crampton < conrad.cramp...@secdata.com> wrote: > Hi, > >

NPE MergeContent processor

Hi, I saw this error after I upgraded to 1.0.0 but thought it was maybe due to the issues I had with that upgrade (entirely my fault it turns out!), but I have seen it a number of times since so I turned debugging on to get a better stacktrace. Relevant log section as below. Nothing out of the

Re: Build a CSV file using MergeContent processor

Both options worked. Thanks guys. On Thu, May 12, 2016 at 3:57 PM, Bryan Bende wrote: > When using the "text" strategy you may have to do shift+enter in the > demarcator field to create a new line. > > On Thu, May 12, 2016 at 3:52 PM, Joe Witt wrote: > >>

Re: Build a CSV file using MergeContent processor

When using the "text" strategy you may have to do shift+enter in the demarcator field to create a new line. On Thu, May 12, 2016 at 3:52 PM, Joe Witt wrote: > Igor, > > I believe it will encode whatever you give it in UTF-8 and place those > bytes in. For absolute control

Re: Build a CSV file using MergeContent processor

Joe, If I put \n or '\n' the processor adds at as a string. How do i add it as ASCII? On Thu, May 12, 2016 at 2:43 PM, Joe Witt wrote: > Igor, > > MergeContent [1] has a property for this purpose called "Demarcator" > and you can set the "Delimiter Strategy" to "text" and

Re: MergeContent Processor

size or number of entries is greater than or equal to their respective, > optionally, specified maximum > > Let us know if you have any other questions! > > On Thu, Nov 19, 2015 at 8:09 AM, Elli Schwarz <eliezer_schw...@yahoo.com> > wrote: > > Hello, > > I'm a bit conf

Re: MergeContent Processor

rz <eliezer_schw...@yahoo.com> wrote: Hello, I'm a bit confused about the relationship of certain properties of the MergeContent processor. Specifically, how do the properties min entries, max entries, max bin age, max number of bins interact? If the MergeContent processor receives the min n

MergeContent Processor

Hello, I'm a bit confused about the relationship of certain properties of the MergeContent processor. Specifically, how do the properties min entries, max entries, max bin age, max number of bins interact? If the MergeContent processor receives the min number of entries, does it merge without

Re: MergeContent Processor

;eliezer_schw...@yahoo.com> wrote: > Hello, > > I'm a bit confused about the relationship of certain properties of the > MergeContent processor. Specifically, how do the properties min entries, > max entries, max bin age, max number of bins interact? If the MergeContent > processor

Referencing Source File Attributes with MergeContent Processor

I have use case where I am trying to merge content using the NIFI MergeContent processor. Scenario is that I am retrieving image files from a server, base64 encoding them and then batching them together into a JSON format using MergeContent. I am using the Header, Footer and Demarcator fields

Re: Referencing Source File Attributes with MergeContent Processor

t; I have use case where I am trying to merge content using the NIFI > MergeContent processor. > > Scenario is that I am retrieving image files from a server, base64 > encoding them and then batching them together into a JSON format using > MergeContent. I am using the Header, Foot