vladhlinsky opened a new pull request #94: ATLAS-3665 Add 'recentQueries' attribute to the 'spark_process' type URL: https://github.com/apache/atlas/pull/94 ## What changes were proposed in this pull request? Add `recentQueries` attribute to the `spark_process` type in order to make `spark_process` more readable by the user. The `recentQueries` attribute stores exact SQL quries that are executed within Spark session. ## How was this patch tested? Manually using modified version of Spark Atlas Connector: - Install and start Atlas. - Stop Atlas. - `1100-spark_model.json` is updated with proposed changes. - Start Atlas. - Executed the next statements using spark-shell: ``` spark.sql("create table source_2_77_10(id int, new_id int, name string)"); spark.sql("create table ctas_2_77_10 as select id as new_id from source_2_77_10"); ``` - Verified that corresponding 'spark_process' is created and has `recentQueries`: ``` create table source_2_77_10(id int, new_id int, name string create table ctas_2_77_10 as select id as new_id from source_2_77_10 ```
---------------------------------------------------------------- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. For queries about this service, please contact Infrastructure at: us...@infra.apache.org With regards, Apache Git Services
