Thank you Matt. Understood. Thanks again for taking the time to reply to my
questions. -Jim
On Mon, Aug 20, 2018 at 9:13 AM, Matt Burgess wrote:
> Jim,
>
> If you know all the possible fields that can occur, you can create a
> schema that contains the three mandatory fields and include all the
>
Jim,
If you know all the possible fields that can occur, you can create a
schema that contains the three mandatory fields and include all the
others as "optional", this is done by setting the type of the field to
["null", ]. This can even be done for the lookup field so
you can inherit the record
I do have a follow-up question. In my example I have oversimplified the
structure. In my production space I have two complicating factors: the
number of fields can vary, and only three fields are mandatory and so must
be there. And the fields order can vary: the messages posted to the queue
that we
This sounds like just what I need. Thank you very much Matt. I'll dig in
and give this a try. Thanks again to each of you guys who responded.
Cheers, Jim
On Fri, Aug 17, 2018 at 4:24 PM, Matt Burgess wrote:
> Jim,
>
> You can use UpdateRecord for this, your input schema would have "last"
> and "
Jim,
You can use UpdateRecord for this, your input schema would have "last"
and "first" in it (and I think you can have an optional "myKey" field
so you can use the same schema for the writer), and the output schema
would have all three fields in it. Then you'd set the Replacement
Value Strategy t
I do appreciate your point, Tim and Lee. What if I do this instead: append
select attributes to my data payload. Would that minimize the impact on
RAM? Can I do that?
More specifically, my data payload is a string representation of a JSON
object, like so:
{"last":"manson","first":"marilyn"}
and I
Jim,
I think the ExtractText processor with a large enough MaxCaptureGroup length
(default :1024) will do that. Though, I share Tim’s concerns when you
scale up
Thanks,
Lee
> On Aug 17, 2018, at 11:52 AM, Timothy Tschampel
> wrote:
>
>
> This may not be applicable to your use case depe
This may not be applicable to your use case depending on message volume / # of
attributes; but I would avoid putting payloads into attributes for scalability
reasons (especially RAM usage).
> On Aug 17, 2018, at 10:47 AM, James McMahon wrote:
>
> I have flowfiles with data payloads that rep
I have flowfiles with data payloads that represent small strings of text
(messages consumed from AMQP queues). I want to create an attribute that
holds the entire payload for downstream use. How can I capture the entire
data payload of a flowfile in a new attribute on the flowfile? Thank you in
adv
