Mark Payne created NIFI-16131:
---------------------------------
Summary: Use Event ID to improve performance of Provenance Lineage
queries
Key: NIFI-16131
URL: https://issues.apache.org/jira/browse/NIFI-16131
Project: Apache NiFi
Issue Type: Improvement
Components: Core Framework, Core UI
Reporter: Mark Payne
Currently the {{LineageRequestDTO}} documents the Event ID as being ignored if
the Lineage Request Type is FLOWFILE and the UUID is provided:
{code:java}
@Schema(description = """
The event id that was used to generate this lineage, if
applicable.
The event id is allowed for any type of lineageRequestType.
If the lineageRequestType is FLOWFILE and the flowfile uuid is
also included in the request, the event id will be ignored.
"""
)
public Long getEventId() {
return eventId;
}{code}
However, this is rather inefficient. While we don't really need the Event ID in
order to compute the lineage, we could use it to lookup the Provenance Event
with that ID (which is very fast and efficient) and then from that determine
the Lineage Start Date. This would allow us to then prune the lucene indices
that we need to query for the lineage data. This can improve performance and
reduce computation significantly in some cases.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
