[
https://issues.apache.org/jira/browse/ATLAS-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17954886#comment-17954886
]
ASF subversion and git services commented on ATLAS-4984:
--------------------------------------------------------
Commit 09d02b99f88292ac3fd2e8edada1d5ae85e7bb5a in atlas's branch
refs/heads/master from chaitalicod
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=09d02b99f ]
ATLAS-4984: Option to ignore spark_process attributes details and
sparkPlanDescription (#331)
Co-authored-by: chaitalithombare <[email protected]>
> Option to ignore spark_process attributes details and sparkPlanDescription
> ---------------------------------------------------------------------------
>
> Key: ATLAS-4984
> URL: https://issues.apache.org/jira/browse/ATLAS-4984
> Project: Atlas
> Issue Type: Improvement
> Components: spark-integration
> Affects Versions: 3.0.0
> Reporter: Madhan Neethiraj
> Assignee: chaitali borole
> Priority: Major
> Time Spent: 40m
> Remaining Estimate: 0h
>
> {{spark_process}} entity attributes {{details}} and {{sparkPlanDescription}}
> are populated with query plan details, which can contain a large amount of
> text, often in megabytes. In environments that don't require these
> attributes, it will help to ignore these attributes while processing
> create/update of {{spark_process}} entities. I suggest Atlas server
> notification processor to support following configurations:
> -
> {{atlas.notification.consumer.preprocess.spark_process.ignore-attribute.sparkPlanDescription}}
> -
> {{atlas.notification.consumer.preprocess.spark_process.ignore-attribute.details}}
> A deployment can choose to ignore these attributes using above configurations
> in Atlas server.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
