Re: Review Request 70600: Adding the model files for Spark

2019-05-16 Thread Arun Mahadevan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70600/#review215306
---


Ship it!




Ship It!

- Arun Mahadevan


On May 16, 2019, 6:24 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70600/
> ---
> 
> (Updated May 16, 2019, 6:24 p.m.)
> 
> 
> Review request for atlas, Ankur Gupta, Arun Mahadevan, Ashutosh Mestry, 
> Jungtaek Lim, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This is an initial mode file for the SPARK models. Please refer to the file 
> added here. This file is a GET call on typeDefs on a cluster which had SAC 
> enabled and SAC has created the models. The SAC version used was 
> https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/1100-spark_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/70600/diff/2/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> SAC Models
>   
> https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>

Re: Review Request 70600: Adding the model files for Spark

2019-05-16 Thread Aadarsh Jajodia 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70600/
---

(Updated May 16, 2019, 6:24 p.m.)


Review request for atlas, Ankur Gupta, Arun Mahadevan, Ashutosh Mestry, 
Jungtaek Lim, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian.


Changes
---

Renaming the file to 1100-spark model since we already have a 1090 version for 
Imapala. Also added the attribtues suggested by Jungtaek. Have added 
relationship label as suggested by Madhan, when both ends of a relationship 
have isLegacyAttribtue set to true. Have also changed the type of executionTime 
field to date


Repository: atlas


Description
---

This is an initial mode file for the SPARK models. Please refer to the file 
added here. This file is a GET call on typeDefs on a cluster which had SAC 
enabled and SAC has created the models. The SAC version used was 
https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev


Diffs (updated)
-

  addons/models/1000-Hadoop/1100-spark_model.json PRE-CREATION 


Diff: https://reviews.apache.org/r/70600/diff/2/

Changes: https://reviews.apache.org/r/70600/diff/1-2/


Testing
---


File Attachments


SAC Models
  
https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt


Thanks,

Aadarsh Jajodia

Re: Review Request 70600: Adding the model files for Spark

2019-05-15 Thread Arun Mahadevan 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70600/#review215292
---




addons/models/1000-Hadoop/1090-spark_model.json
Lines 48 (patched)


Ensure the fields added recently are included.

```json
{
  "enumDefs": [],
  "structDefs": [],
  "classificationDefs": [],
  "entityDefs": [
{
  "category": "ENTITY",
  "guid": "a17bded7-bd62-431c-841d-ae34be4f5e10",
  "createdBy": "admin",
  "updatedBy": "admin",
  "createTime": 1557859427155,
  "updateTime": 1557859427155,
  "version": 1,
  "name": "spark_table",
  "description": "spark_table",
  "typeVersion": "1.0",
  "attributeDefs": [
{
  "name": "qualifiedName",
  "typeName": "string",
  "isOptional": false,
  "cardinality": "SINGLE",
  "valuesMinCount": 1,
  "valuesMaxCount": 1,
  "isUnique": true,
  "isIndexable": true,
  "includeInNotification": false
},
{
  "name": "tableType",
  "typeName": "string",
  "isOptional": true,
  "cardinality": "SINGLE",
  "valuesMinCount": 0,
  "valuesMaxCount": 1,
  "isUnique": false,
  "isIndexable": false,
  "includeInNotification": false
},
{
  "name": "schemaDesc",
  "typeName": "string",
  "isOptional": true,
  "cardinality": "SINGLE",
  "valuesMinCount": 0,
  "valuesMaxCount": 1,
  "isUnique": false,
  "isIndexable": false,
  "includeInNotification": false
},
{
  "name": "provider",
  "typeName": "string",
  "isOptional": true,
  "cardinality": "SINGLE",
  "valuesMinCount": 0,
  "valuesMaxCount": 1,
  "isUnique": false,
  "isIndexable": false,
  "includeInNotification": false
},
{
  "name": "partitionProvider",
  "typeName": "string",
  "isOptional": true,
  "cardinality": "SINGLE",
  "valuesMinCount": 0,
  "valuesMaxCount": 1,
  "isUnique": false,
  "isIndexable": false,
  "includeInNotification": false
},
{
  "name": "partitionColumnNames",
  "typeName": "array",
  "isOptional": true,
  "cardinality": "SINGLE",
  "valuesMinCount": 0,
  "valuesMaxCount": 1,
  "isUnique": false,
  "isIndexable": false,
  "includeInNotification": false
},
{
  "name": "bucketSpec",
  "typeName": "map",
  "isOptional": true,
  "cardinality": "SINGLE",
  "valuesMinCount": 0,
  "valuesMaxCount": 1,
  "isUnique": false,
  "isIndexable": false,
  "includeInNotification": false
},
{
  "name": "owner",
  "typeName": "string",
  "isOptional": true,
  "cardinality": "SINGLE",
  "valuesMinCount": 0,
  "valuesMaxCount": 1,
  "isUnique": false,
  "isIndexable": false,
  "includeInNotification": false
},
{
  "name": "ownerType",
  "typeName": "string",
  "isOptional": true,
  "cardinality": "SINGLE",
  "valuesMinCount": 0,
  "valuesMaxCount": 1,
  "isUnique": false,
  "isIndexable": false,
  "includeInNotification": false
},
{
  "name": "createTime",
  "typeName": "date",
  "isOptional": true,
  "cardinality": "SINGLE",
  "valuesMinCount": 0,
  "valuesMaxCount": 1,
  "isUnique": false,
  "isIndexable": false,
  "includeInNotification": false
},
{
  "name": "parameters",
  "typeName": "map",
  "isOptional": true,
  "cardinality": "SINGLE",
  "valuesMinCount": 0,
  "valuesMaxCount": 1,
  "isUnique": false,
  "isIndexable": false,
  "includeInNotification": false
},
{
  "name": "comment",
  "typeName": "string",
  "isOptional": true,

Re: Review Request 70600: Adding the model files for Spark

2019-05-15 Thread Jungtaek Lim 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70600/#review215289
---



Sorry we are adding new attributes for Spark model to resolve our own issue 
quickly.

https://github.com/hortonworks-spark/spark-atlas-connector/commit/0843cf9b74ed1d58d99100a049a5cc56aa4d7a1e

Below is the change for `spark_table` model:

```
AtlasTypeUtil.createOptionalAttrDef("schemaDesc", new AtlasStringType)
AtlasTypeUtil.createOptionalAttrDef("partitionProvider", new AtlasStringType)
```

Could we reflect the change to here? Thanks in advance.

- Jungtaek Lim


On 5 15, 2019, 7:39 오후, Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70600/
> ---
> 
> (Updated 5 15, 2019, 7:39 오후)
> 
> 
> Review request for atlas, Ankur Gupta, Arun Mahadevan, Ashutosh Mestry, 
> Jungtaek Lim, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This is an initial mode file for the SPARK models. Please refer to the file 
> added here. This file is a GET call on typeDefs on a cluster which had SAC 
> enabled and SAC has created the models. The SAC version used was 
> https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/1090-spark_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/70600/diff/1/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> SAC Models
>   
> https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>

Re: Review Request 70600: Adding the model files for Spark

2019-05-15 Thread Sridhar K 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70600/#review215286
---




addons/models/1000-Hadoop/1090-spark_model.json
Lines 266 (patched)


Can we get a better descriprtion for it. In general, same comment for all 
similar items.



addons/models/1000-Hadoop/1090-spark_model.json
Lines 312 (patched)


date types prefered.



addons/models/1000-Hadoop/1090-spark_model.json
Lines 359 (patched)


I am assuming that there will be relationships expressing lineage from hive 
to spark tables and vice versa?


- Sridhar K


On May 15, 2019, 7:39 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70600/
> ---
> 
> (Updated May 15, 2019, 7:39 p.m.)
> 
> 
> Review request for atlas, Ankur Gupta, Arun Mahadevan, Ashutosh Mestry, 
> Jungtaek Lim, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This is an initial mode file for the SPARK models. Please refer to the file 
> added here. This file is a GET call on typeDefs on a cluster which had SAC 
> enabled and SAC has created the models. The SAC version used was 
> https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/1090-spark_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/70600/diff/1/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> SAC Models
>   
> https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>

Re: Review Request 70600: Adding the model files for Spark

2019-05-15 Thread Sarath Subramanian 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70600/#review215285
---




addons/models/1000-Hadoop/1090-spark_model.json
Lines 8 (patched)


give a meaningful description value - "Spark Database"; Please review other 
description as well.



addons/models/1000-Hadoop/1090-spark_model.json
Lines 9 (patched)


add "serviceType": "spark" for each spark type; please review for other 
types in spark as well.



addons/models/1000-Hadoop/1090-spark_model.json
Lines 10 (patched)


consider including "searchWeight" for relavent fields which might be used 
for search.



addons/models/1000-Hadoop/1090-spark_model.json
Lines 44 (patched)


move "superTypes" above next to - name, description, superTypes; to be 
consistent with other model types


- Sarath Subramanian


On May 15, 2019, 12:39 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70600/
> ---
> 
> (Updated May 15, 2019, 12:39 p.m.)
> 
> 
> Review request for atlas, Ankur Gupta, Arun Mahadevan, Ashutosh Mestry, 
> Jungtaek Lim, Sridhar K, Le Ma, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This is an initial mode file for the SPARK models. Please refer to the file 
> added here. This file is a GET call on typeDefs on a cluster which had SAC 
> enabled and SAC has created the models. The SAC version used was 
> https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/1090-spark_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/70600/diff/1/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> SAC Models
>   
> https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>

Re: Review Request 70600: Adding the model files for Spark

2019-05-14 Thread Aadarsh Jajodia 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70600/#review215266
---




addons/models/1000-Hadoop/1090-spark_model.json
Lines 22 (patched)


This should not be optional. If you refer to the corresponding hive model, 
it is not optional



addons/models/1000-Hadoop/1090-spark_model.json
Lines 49 (patched)


Jungtaek, if you look at the spark table and see the differences between 
spark and hive table and let us know if these differences are expected and fine.



addons/models/1000-Hadoop/1090-spark_model.json
Lines 139 (patched)


In the corresponding hive_column models i see there is something called a 
position attribute. Are we expecting to not have that attribute for Spark Column



addons/models/1000-Hadoop/1090-spark_model.json
Lines 207 (patched)


I just wanted to highlight that i see some differences between spark 
storage desc and hive storage desc in the attributes



addons/models/1000-Hadoop/1090-spark_model.json
Lines 249 (patched)


Why is this set to true here? Any particular reason?



addons/models/1000-Hadoop/1090-spark_model.json
Lines 313 (patched)


This should be changed to date type. I will update in the next version


- Aadarsh Jajodia


On May 10, 2019, 9:18 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70600/
> ---
> 
> (Updated May 10, 2019, 9:18 p.m.)
> 
> 
> Review request for atlas, Ankur Gupta, Ashutosh Mestry, Jungtaek Lim, Sridhar 
> K, Le Ma, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This is an initial mode file for the SPARK models. Please refer to the file 
> added here. This file is a GET call on typeDefs on a cluster which had SAC 
> enabled and SAC has created the models. The SAC version used was 
> https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/1090-spark_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/70600/diff/1/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> SAC Models
>   
> https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>

Re: Review Request 70600: Adding the model files for Spark

2019-05-12 Thread Aadarsh Jajodia 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70600/#review215207
---



Hi Jungtaek, I spoke to Ankur on Friday and he mentions all spark 
tables/db/cols in CDP will always be created in HMS. If that is the case we 
should remove spark tables/db/cols models. In case spark will use Hbase to 
create thsee then we should map them to hbase models that we have in Atlas. 
Ankur could you please comment on the same. Thanks

- Aadarsh Jajodia


On May 10, 2019, 9:18 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70600/
> ---
> 
> (Updated May 10, 2019, 9:18 p.m.)
> 
> 
> Review request for atlas, Ankur Gupta, Ashutosh Mestry, Jungtaek Lim, Sridhar 
> K, Le Ma, Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This is an initial mode file for the SPARK models. Please refer to the file 
> added here. This file is a GET call on typeDefs on a cluster which had SAC 
> enabled and SAC has created the models. The SAC version used was 
> https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/1090-spark_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/70600/diff/1/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> SAC Models
>   
> https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>

Re: Review Request 70600: Adding the model files for Spark

2019-05-09 Thread Jungtaek Lim 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70600/#review215177
---


Ship it!




I think it's better to point to public repository when referring some other 
project: Apache Atlas is one of the ASF projects.
So we can replace the link to 
https://github.com/hortonworks-spark/spark-atlas-connector/commits/master, more 
clearly 
https://github.com/hortonworks-spark/spark-atlas-connector/commit/ec30195c045424e8543f646c479cbc99395c7261

Btw, I have tested the json models via adding file to vanilla Atlas 1.1 and run 
Atlas via standalone mode, and ran basic queries with latest SAC. Looks good.

- Jungtaek Lim


On 5 7, 2019, 8:01 오후, Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70600/
> ---
> 
> (Updated 5 7, 2019, 8:01 오후)
> 
> 
> Review request for atlas, Ashutosh Mestry, Jungtaek Lim, Sridhar K, Le Ma, 
> Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This is an initial mode file for the SPARK models. Please refer to the file 
> added here. This file is a GET call on typeDefs on a cluster which had SAC 
> enabled and SAC has created the models. The SAC version used was 
> https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/1090-spark_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/70600/diff/1/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> SAC Models
>   
> https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>

Re: Review Request 70600: Adding the model files for Spark

2019-05-09 Thread Madhan Neethiraj 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70600/#review215080
---




addons/models/1000-Hadoop/1090-spark_model.json
Lines 395 (patched)


When both ends have isLegacyAttribute=true, consider adding 
'relationshipLabel' filed to the relationship-def. This will eliminate the need 
to create 2 edges (as in legacy case) - end1->end2 & end2->end1.

Please review and update following relationship-defs:
 - spark_table_columns
 - spark_table_storagedesc


- Madhan Neethiraj


On May 7, 2019, 8:01 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70600/
> ---
> 
> (Updated May 7, 2019, 8:01 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Jungtaek Lim, Sridhar K, Le Ma, 
> Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This is an initial mode file for the SPARK models. Please refer to the file 
> added here. This file is a GET call on typeDefs on a cluster which had SAC 
> enabled and SAC has created the models. The SAC version used was 
> https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/1090-spark_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/70600/diff/1/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> SAC Models
>   
> https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>

Re: Review Request 70600: Adding the model files for Spark

2019-05-09 Thread Jungtaek Lim 


> On 5 7, 2019, 7:05 오후, Asif Hoda wrote:
> > addons/models/1000-Hadoop/1090-spark_model.json
> > Lines 1 (patched)
> > 
> >
> > I believe we should have the following value for most of the entitiies:
> > 
> > 1. "isIndexable": true
> > 2. isOptional": false
> > 3. "isUnique": true
> > 
> > Do we need to discuss these further ?

I guess all of models are inherited from `Referenceable` which should have 
`qualifiedName` attribute. Isn't it enough?


- Jungtaek


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70600/#review215097
---


On 5 7, 2019, 8:01 오후, Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70600/
> ---
> 
> (Updated 5 7, 2019, 8:01 오후)
> 
> 
> Review request for atlas, Ashutosh Mestry, Jungtaek Lim, Sridhar K, Le Ma, 
> Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This is an initial mode file for the SPARK models. Please refer to the file 
> added here. This file is a GET call on typeDefs on a cluster which had SAC 
> enabled and SAC has created the models. The SAC version used was 
> https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/1090-spark_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/70600/diff/1/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> SAC Models
>   
> https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>

Re: Review Request 70600: Adding the model files for Spark

2019-05-07 Thread Sridhar K 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70600/#review215100
---




addons/models/1000-Hadoop/1090-spark_model.json
Lines 7 (patched)


I am assuming that we are going to reconsider these models. Especially, 
there is a possibility of these objects are represented as hive or habase 
objects in Atlas already.


- Sridhar K


On May 7, 2019, 8:01 p.m., Aadarsh Jajodia wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/70600/
> ---
> 
> (Updated May 7, 2019, 8:01 p.m.)
> 
> 
> Review request for atlas, Ashutosh Mestry, Jungtaek Lim, Sridhar K, Le Ma, 
> Madhan Neethiraj, and Sarath Subramanian.
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This is an initial mode file for the SPARK models. Please refer to the file 
> added here. This file is a GET call on typeDefs on a cluster which had SAC 
> enabled and SAC has created the models. The SAC version used was 
> https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev
> 
> 
> Diffs
> -
> 
>   addons/models/1000-Hadoop/1090-spark_model.json PRE-CREATION 
> 
> 
> Diff: https://reviews.apache.org/r/70600/diff/1/
> 
> 
> Testing
> ---
> 
> 
> File Attachments
> 
> 
> SAC Models
>   
> https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt
> 
> 
> Thanks,
> 
> Aadarsh Jajodia
> 
>

Re: Review Request 70600: Adding the model files for Spark

2019-05-06 Thread Aadarsh Jajodia 

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/70600/
---

(Updated May 7, 2019, 1:32 a.m.)


Review request for atlas, Ashutosh Mestry, Sridhar K, Le Ma, Madhan Neethiraj, 
and Sarath Subramanian.


Repository: atlas


Description (updated)
---

This is an initial mode file for the SPARK models. Please refer to the file 
added here. This file is a GET call on typeDefs on a cluster which had SAC 
enabled and SAC has created the models. The SAC version used was 
https://github.com/hortonworks/spark-atlas-connector/tree/SAC-CDP-1.0-dev


Diffs
-

  addons/models/1000-Hadoop/1090-spark_model.json PRE-CREATION 


Diff: https://reviews.apache.org/r/70600/diff/1/


Testing
---


File Attachments


SAC Models
  
https://reviews.apache.org/media/uploaded/files/2019/05/07/e31827b2-04c5-4fab-b5b7-4271398831e4__spark_sac_api_model_definition.txt


Thanks,

Aadarsh Jajodia