[
https://issues.apache.org/jira/browse/NIFI-7711?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Daniel Lorych updated NIFI-7711:
Description:
h6. Description
GenerateTableFetch generates "select all" where clause (1=1) for an empty
table, which leads to inconsistent state (and duplicate flowfiles).
Generated SQL statement (1=1) can return values, which were created after
collecting max values during statement generation. On a subsequent run, with
existing data, the statement will contain a where clause with maxValue limited.
h6. Environment:
* Table is empty
* 'Maximum-value Columns' property is set to PK
* 'Partition Size' property set to 0
h6. Expected behaviour:
No or empty flowfile should be generated.
h6. Root Cause:
`numberOfFetches` is calculated incorrectly for `partitionSize == 0`.
Calculation should take into account returned `rowCount`.
[GenerateTableFetch.java#L462|https://github.com/apache/nifi/blob/4d940bb151eb8d250b0319318b96d23c4a9819ae/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java#L462]
h6. Possible fix:
{code:java}
numberOfFetches = (partitionSize == 0) ? (rowCount == 0 ? 0 : 1) : (rowCount /
partitionSize) + (rowCount % partitionSize == 0 ? 0 : 1);
{code}
was:
h6. Description
GenerateTableFetch generates "select all" where clause (1=1) for an empty
table, which leads to inconsistent state (and duplicate flowfiles).
Generated SQL statement (1=1) can return values, which were created after
collecting max values during statement generation. On a subsequent run, with
existing data, the statement will contain a where clause with maxValue limited.
h6. Environment:
* Table is empty
* 'Maximum-value Columns' property is set to PK
* 'Partition Size' property set to 0
h6. Expected behaviour:
Fetch statement should not be generated.
h6. Root Cause:
`numberOfFetches` is calculated incorrectly for `partitionSize == 0`.
Calculation should take into account returned `rowCount`.
[GenerateTableFetch.java#L462|https://github.com/apache/nifi/blob/4d940bb151eb8d250b0319318b96d23c4a9819ae/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java#L462]
h6. Possible fix:
{code:java}
numberOfFetches = (partitionSize == 0) ? (rowCount == 0 ? 0 : 1) : (rowCount /
partitionSize) + (rowCount % partitionSize == 0 ? 0 : 1);
{code}
> GenerateTableFetch generates fetch on empty table
> -
>
> Key: NIFI-7711
> URL: https://issues.apache.org/jira/browse/NIFI-7711
> Project: Apache NiFi
> Issue Type: Bug
> Components: Extensions
>Affects Versions: 1.11.4
>Reporter: Daniel Lorych
>Priority: Major
>
> h6. Description
> GenerateTableFetch generates "select all" where clause (1=1) for an empty
> table, which leads to inconsistent state (and duplicate flowfiles).
> Generated SQL statement (1=1) can return values, which were created after
> collecting max values during statement generation. On a subsequent run, with
> existing data, the statement will contain a where clause with maxValue
> limited.
> h6. Environment:
> * Table is empty
> * 'Maximum-value Columns' property is set to PK
> * 'Partition Size' property set to 0
>
> h6. Expected behaviour:
> No or empty flowfile should be generated.
> h6. Root Cause:
> `numberOfFetches` is calculated incorrectly for `partitionSize == 0`.
> Calculation should take into account returned `rowCount`.
> [GenerateTableFetch.java#L462|https://github.com/apache/nifi/blob/4d940bb151eb8d250b0319318b96d23c4a9819ae/nifi-nar-bundles/nifi-standard-bundle/nifi-standard-processors/src/main/java/org/apache/nifi/processors/standard/GenerateTableFetch.java#L462]
> h6. Possible fix:
> {code:java}
> numberOfFetches = (partitionSize == 0) ? (rowCount == 0 ? 0 : 1) : (rowCount
> / partitionSize) + (rowCount % partitionSize == 0 ? 0 : 1);
> {code}
>
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
