[ https://issues.apache.org/jira/browse/NIFI-4696?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Pierre Villard updated NIFI-4696: --------------------------------- Resolution: Fixed Fix Version/s: 1.5.0 Status: Resolved (was: Patch Available) > Support concurrent tasks in PutHiveStreaming > -------------------------------------------- > > Key: NIFI-4696 > URL: https://issues.apache.org/jira/browse/NIFI-4696 > Project: Apache NiFi > Issue Type: Improvement > Reporter: Matt Burgess > Assignee: Matt Burgess > Fix For: 1.5.0 > > > Currently PutHiveStreaming (PHS) can only support a single task at a time. > Before NIFI-4342, that meant each target table would need its own PHS > instance, which can be cumbersome with large numbers of tables. After > NIFI-4342, Expression Language could be used for SDLC purposes > (database/table changes between development and production, e.g.). > However it would be nice to be able to support at least database/table names > using flow file attributes, and also to support multiple tasks to handle them > concurrently. Due to the nature of PHS and the Streaming Ingest APIs (and > implementation), it is likely not prudent to allow two tasks to write to the > same table and partition at the same time. > I propose adding flow file attribute EL evaluation where prudent, and > allowing per-table concurrency in PHS. A thread will attempt to get a lock on > a table, and if it cannot, will rollback and return. -- This message was sent by Atlassian JIRA (v6.4.14#64029)