If you don't do record splitting, how are you getting SQL to send to
PutSQL? Can you describe your flow (processors, e.g.)?
Thanks,
Matt
On Wed, Feb 6, 2019 at 3:41 PM l vic <lvic4...@gmail.com> wrote:
>
> Hi Matt,
> No, I don't do record splitting, data looks like { "attr1":"val1",...[{....}]}
> where "parent" data is saved into 1 record in "parent" table and array data
> is saved into multiple records in "child" table...
> What's "lineage duration"?
> Event Duration
> < 1ms
> Lineage Duration
> 00:00:00.070
>
> On Wed, Feb 6, 2019 at 2:59 PM Matt Burgess <mattyb...@apache.org> wrote:
>>
>> In your flow, what does the data look like? Are you splitting it into
>> individual records, then converting to SQL (probably via JSON) and
>> calling PutSQL? If so, that's not going to be very performant; the
>> PutDatabaseRecord processor combines all that together so you can
>> leave your data in its original state (i.e. many records in one flow
>> file). For benchmarking PutDatabaseRecord (PDR), you could provide
>> sample data via GenerateFlowFile, run a few through PDR, and check the
>> provenance events for fields such as durationMillis or calculations
>> like (timestampMills - lineageStart).
>>
>> Regards,
>> Matt
>>
>> On Wed, Feb 6, 2019 at 2:07 PM l vic <lvic4...@gmail.com> wrote:
>> >
>> > I have performance issues with PutSQL i my flow... Is there some way to
>> > benchmark time required to write certain number of records to table from
>> > GenerateFlowFile?
>> > Thank you,
