[
https://issues.apache.org/jira/browse/ATLAS-4984?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17965523#comment-17965523
]
ASF subversion and git services commented on ATLAS-4984:
--------------------------------------------------------
Commit d400fed61141f0dffdf83398932357197e418b54 in atlas's branch
refs/heads/master from chaitalicod
[ https://gitbox.apache.org/repos/asf?p=atlas.git;h=d400fed61 ]
ATLAS-4984: Option to ignore spark_process attributes details and
sparkPlanDescription (#376)
Co-authored-by: chaitalithombare <chaitalithomb...@apache.org>
> Option to ignore spark_process attributes details and sparkPlanDescription
> ---------------------------------------------------------------------------
>
> Key: ATLAS-4984
> URL: https://issues.apache.org/jira/browse/ATLAS-4984
> Project: Atlas
> Issue Type: Improvement
> Components: spark-integration
> Affects Versions: 3.0.0
> Reporter: Madhan Neethiraj
> Assignee: chaitali borole
> Priority: Major
> Fix For: 3.0.0
>
> Time Spent: 1h 10m
> Remaining Estimate: 0h
>
> {{spark_process}} entity attributes {{details}} and {{sparkPlanDescription}}
> are populated with query plan details, which can contain a large amount of
> text, often in megabytes. In environments that don't require these
> attributes, it will help to ignore these attributes while processing
> create/update of {{spark_process}} entities. I suggest Atlas server
> notification processor to support following configurations:
> -
> {{atlas.notification.consumer.preprocess.spark_process.ignore-attribute.sparkPlanDescription}}
> -
> {{atlas.notification.consumer.preprocess.spark_process.ignore-attribute.details}}
> A deployment can choose to ignore these attributes using above configurations
> in Atlas server.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
