turcsanyip commented on a change in pull request #3979: NIFI-7009: Atlas
reporting task retrieves only the active flow compon…
URL: https://github.com/apache/nifi/pull/3979#discussion_r367328847
##########
File path:
nifi-nar-bundles/nifi-atlas-bundle/nifi-atlas-reporting-task/src/main/java/org/apache/nifi/atlas/NiFiAtlasClient.java
##########
@@ -230,6 +231,35 @@ public NiFiFlow fetchNiFiFlow(String rootProcessGroupId,
String clusterName) thr
return nifiFlow;
}
+ /**
+ * Retrieves the flow components of type {@code componentType} from Atlas
server.
+ * Deleted components will be filtered out before calling Atlas.
+ * Atlas object ids will be initialized with all the attributes (guid,
type, unique attributes) in order to be able
+ * to match ids retrieved from Atlas (having guid) and ids created by the
reporting task (not having guid yet).
+ *
+ * @param componentType Atlas type of the flow component (nifi_flow_path,
nifi_queue, nifi_input_port, nifi_output_port)
+ * @param referredEntities referred entities of the flow entity (returned
when the flow fetched) containing the basic data (id, status) of the flow
components
+ * @return flow component entities mapped to their object ids
+ */
+ private Map<AtlasObjectId, AtlasEntity> fetchFlowComponents(String
componentType, Map<String, AtlasEntity> referredEntities) {
+ return referredEntities.values().stream()
+ .filter(referredEntity ->
referredEntity.getTypeName().equals(componentType))
+ .filter(referredEntity -> referredEntity.getStatus() ==
AtlasEntity.Status.ACTIVE)
+ .map(referredEntity -> {
+ final Map<String, Object> uniqueAttributes =
Collections.singletonMap(ATTR_QUALIFIED_NAME,
referredEntity.getAttribute(ATTR_QUALIFIED_NAME));
+ final AtlasObjectId id = new
AtlasObjectId(referredEntity.getGuid(), componentType, uniqueAttributes);
+ try {
+ final AtlasEntity.AtlasEntityWithExtInfo
fetchedEntityExt = searchEntityDef(id);
+ return new Tuple<>(id, fetchedEntityExt.getEntity());
+ } catch (AtlasServiceException e) {
+ logger.warn("Failed to search entity by id {}, due to
{}", id, e);
+ return null;
Review comment:
It comes from the previous code. I saw it could be a problem, but I did want
to change the existing behaviour in this ticket, only optimize it.
I'll create a jira for investigating it and finding a proper error handling
for this case, if it is fine with you.
----------------------------------------------------------------
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
For queries about this service, please contact Infrastructure at:
[email protected]
With regards,
Apache Git Services
