Build failed in Jenkins: apache-atlas-nightly #578

[Apache Jenkins Server]

See 

Changes:

[madhan] ATLAS-1502: added configuration to restrict entity-types editable via 
UI

--
[...truncated 5230 lines...]
127.0.0.1 - - [31/Jan/2017:23:42:23 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:23 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:23 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:23 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:23 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:24 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:24 +] "GET 
/api/atlas/discovery/search?query=from+DB&limit=10&offset=0 HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:26 +] "GET 
/api/atlas/discovery/search?query=DB&limit=10&offset=0 HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:27 +] "GET 
/api/atlas/discovery/search?query=DB+where+name%3D%22Reporting%22&limit=10&offset=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:28 +] "GET 
/api/atlas/discovery/search?query=DB+where+DB.name%3D%22Reporting%22&limit=10&offset=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:28 +] "GET 
/api/atlas/discovery/search?query=DB+name+%3D+%22Reporting%22&limit=10&offset=0 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:29 +] "GET 
/api/atlas/discovery/search?query=DB+DB.name+%3D+%22Reporting%22&limit=10&offset=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:31 +] "GET 
/api/atlas/discovery/search?query=DB+where+name%3D%22Reporting%22+select+name,+owner&limit=10&offset=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:33 +] "GET 
/api/atlas/discovery/search?query=DB+where+DB.name%3D%22Reporting%22+select+name,+owner&limit=10&offset=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:34 +] "GET 
/api/atlas/discovery/search?query=DB+has+name&limit=10&offset=0 HTTP/1.1" 200 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:35 +] "GET 
/api/atlas/discovery/search?query=DB+where+DB+has+name&limit=10&offset=0 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:36 +] "GET 
/api/atlas/discovery/search?query=DB,+Table&limit=10&offset=0 HTTP/1.1" 200 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:36 +] "GET 
/api/atlas/discovery/search?query=DB+is+JdbcAccess&limit=10&offset=0 HTTP/1.1" 
200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:37 +] "GET 
/api/atlas/discovery/search?query=from+Table&limit=10&offset=0 HTTP/1.1" 200 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:38 +] "GET 
/api/atlas/discovery/search?query=Table&limit=10&offset=0 HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:39 +] "GET 
/api/atlas/discovery/search?query=Table+is+Dimension&limit=10&offset=0 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:40 +] "GET 
/api/atlas/discovery/search?query=Column+where+Column+isa+PII&limit=10&offset=0 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:41 +] "GET 
/api/atlas/discovery/search?query=View+is+Dimension&limit=10&offset=0 HTTP/1.1" 
200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:41 +] "GET 
/api/atlas/discovery/search?query=Column+select+Column.name&limit=10&offset=0 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:42 +] "GET 
/api/atlas/discovery/search?query=Column+select+name&limit=10&offset=0 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:43 +] "GET 
/api/atlas/discovery/search?query=Column+where+Column.name%3D%22customer_id%22&limit=10&offset=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:44 +] "GET 
/api/atlas/discovery/search?query=from+Table+select+Table.name&limit=10&offset=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:45 +] "GET 
/api/atlas/discovery/search?query=DB+where+(name+%3D+%22Reporting%22)&limit=10&offset=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:46 +] "GET 
/api/atlas/discovery/search?query=DB+where+(name+%3D+%22Reporting%22)+select+name+as+_col_0,+owner+as+_col_1&limit=10&offset=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:47 +] "GET 
/api/atlas/discovery/search?query=DB+where+DB+is+JdbcAccess&limit=10&offset=0 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [31/Jan/2017:23:42:48 +] "GET 
/api/atlas/discovery/search?query=DB+where+DB+has+name&limit=10&offset=0 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [3

Re: Review Request 56113: Fixed issues with deletion during updates

[Vimal Sharma]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56113/#review163797
---




repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 (line 242)


Didn't understand why the code has been commented. Won't this result into 
stray edges of reverse attribute references


- Vimal Sharma


On Feb. 1, 2017, midnight, Suma Shivaprasad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56113/
> ---
> 
> (Updated Feb. 1, 2017, midnight)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-1509
> https://issues.apache.org/jira/browse/ATLAS-1509
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Added tests for deletion during updates and fixed issues found
> 
> 
> Diffs
> -
> 
>   intg/src/test/java/org/apache/atlas/TestUtilsV2.java 4f2916c 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
>  48bb77e 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  34e02c6 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  c98d340 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java
>  22851ca 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java
>  25bae44 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
>  c902f81 
>   repository/src/test/java/org/apache/atlas/services/MetricsServiceTest.java 
> bb8c223 
>   server-api/src/main/java/org/apache/atlas/RequestContextV1.java 1fc11a2 
>   
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasEntityFormatConverter.java
>  75a7183 
>   
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasStructFormatConverter.java
>  8777510 
>   webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 
> 47e1fa3 
> 
> Diff: https://reviews.apache.org/r/56113/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Suma Shivaprasad
> 
>

Apache Atlas Bridge for Flink

[Stefan Papp]

Hi,


Is an Apache Atlas Bridge for Flink available?

Has this feature already been requested.

Stefan

Stefan Papp
Data Architect

The unbelievable Machine Company GmbH
Museumsplatz 1/10/13
1070 Wien

Tel: +43 - 1 - 361 99 77 - 215
Mobil: +43664 2614367

https://www.unbelievable-machine.com 

*Blog  | *Karriere 
 | *Facebook 
 | *Twitter 


Geschäftsführer: Ravin Mehta, Benno Wiedfeld
Sitz der Gesellschaft: Berlin, Deutschland
Handelsregister: Amtsgericht Charlottenburg HRB 115071B
Umsatzsteuer-Identifikationsnummer: DE261090159

Ausgezeichnet: *um ist zweifacher „Big Data Leader 2017“ 

Re: Review Request 56102: ATLAS-1388 : Cache entities that are created/updated

[Vimal Sharma]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56102/#review163800
---



The cache size can grow large if the number of registered entities is high. It 
would be good to have a cache eviction policy just like it was done for DSL 
Query caching in ATLAS-1387

- Vimal Sharma


On Jan. 30, 2017, 10:42 p.m., Jeff Hagelberg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56102/
> ---
> 
> (Updated Jan. 30, 2017, 10:42 p.m.)
> 
> 
> Review request for atlas and David Kantor.
> 
> 
> Bugs: ATLAS-1388
> https://issues.apache.org/jira/browse/ATLAS-1388
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Cache created entities in RequestContext when they are created.
> Update/refactor DefaultMetadataService.loadEntities() to use the cached 
> created entities.
> Use the cache in the following places:
> FullTextMapper
> DefaultMetadataService.onEntitiesAdded - check cache before calling 
> DefaultMetadataService.loadEntities
> DefaultMetadataService.onEntitiesUpdated - check cache before calling 
> DefaultMetadataService.loadEntities
> EntityResource.getEntityDefinition - check cache before calling 
> DefaultMetadataService.getEntityDefinition
> EntityResource.getResponse - check cache before calling 
> DefaultMetadataService.getEntityDefinition
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/graph/DefaultGraphPersistenceStrategy.java
>  f4d8f00da863deeb98209aec2dfc4ccb22734cfa 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapper.java
>  5be8d0bc349294dc4340ffe5831bfb1265825ed8 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
>  be02891cad3020e50d5244fd2aaf3c4af3a5954f 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java
>  7b2b753b930467e015d364819c5baa983fce0cf8 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  bae8b2ac0cae6bf6392f91f24030d81089ae660b 
>   repository/src/test/java/org/apache/atlas/TestUtils.java 
> cda9eac38861fd55c1494d6d90b5e750e6133545 
>   
> repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java
>  f2ca6a8fb1573755d16497236358358a07881e9c 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  9e850a9dc0ac122cf3fcdbf7df72a93643abc98d 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java
>  7444bf38834f59454d67cf763c932d665d7ef31c 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
>  c902f8126f6b516e84ab9d89b0ec80ac8ed819fd 
>   
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
>  aa1cdaa7848aeb767725b3161ccd904c8750536f 
>   
> repository/src/test/java/org/apache/atlas/util/CompiledQueryCacheKeyTest.java 
> c926f4d7d793ef095b20853277f3ba979483e039 
>   server-api/src/main/java/org/apache/atlas/RequestContext.java 
> 651a71dc238a3adaac9504a77ba439785fa07ca8 
> 
> Diff: https://reviews.apache.org/r/56102/diff/
> 
> 
> Testing
> ---
> 
> Ran full build, no regresssions found.
> 
> 
> Thanks,
> 
> Jeff Hagelberg
> 
>

Review Request 56161: ATLAS-1513: updated AtlasEntityType with methods to get foreignKey references to a type; added helper methods in AtlasAttribute

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56161/
---

Review request for atlas.


Bugs: ATLAS-1513
https://issues.apache.org/jira/browse/ATLAS-1513


Repository: atlas


Description
---

- updated AtlasEntityType to get foreignKey-references to the type
- moved mappedFromRef from AtlasStructType to AtlasEntityType (as it is not 
applicable to structs)
- simplified use of AtlasAttribute, by adding few helper methods and removing a 
constructor parameter
- fixed incorrect persistence of mappedFromRef/foreignKey constraints to store; 
this was exposed in a recent change to hive-model.json


Diffs
-

  intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 2c00f54 
  intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java d06364c 
  intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java b83a241 
  intg/src/main/java/org/apache/atlas/type/AtlasStructType.java bac5800 
  intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java e4f1eea 
  intg/src/test/java/org/apache/atlas/TestUtilsV2.java 4f2916c 
  intg/src/test/java/org/apache/atlas/model/ModelTestUtil.java 5c72470 
  intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java 4e15edd 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java
 e63ea6e 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java
 4f5b4f4 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java
 163ddef 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 34e02c6 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
 c98d340 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/GraphMutationContext.java
 89419d9 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/MapVertexMapper.java
 efafcc9 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java
 22851ca 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java
 2698504 
  webapp/src/main/java/org/apache/atlas/util/RestUtils.java 5e69262 
  
webapp/src/main/java/org/apache/atlas/web/adapters/AtlasStructFormatConverter.java
 8777510 
  webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java ee1174a 
  
webapp/src/test/java/org/apache/atlas/web/resources/TypedefsJerseyResourceIT.java
 8eacc73 

Diff: https://reviews.apache.org/r/56161/diff/


Testing
---

- added unit tests
- performed sanity tests with Atlas web application, like:
  - hive notification processing
  - search
  - lineage
  - delete entities


Thanks,

Madhan Neethiraj

[jira] [Updated] (ATLAS-1513) Update AtlasEntityType with a method to get foreign-key references

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-1513:

Attachment: ATLAS-1513.patch

> Update AtlasEntityType with a method to get foreign-key references
> --
>
> Key: ATLAS-1513
> URL: https://issues.apache.org/jira/browse/ATLAS-1513
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Madhan Neethiraj
> Attachments: ATLAS-1513.patch
>
>
> To enable entity instance update/delete implementation to handle foreign-key 
> references, it should be possible to get all foreign-key references via 
> AtlasEntityType instance.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ATLAS-1513) Update AtlasEntityType with a method to get foreign-key references

[Madhan Neethiraj (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15848223#comment-15848223
 ] 

Madhan Neethiraj commented on ATLAS-1513:
-

Patch is available in the review board at: https://reviews.apache.org/r/56161/

> Update AtlasEntityType with a method to get foreign-key references
> --
>
> Key: ATLAS-1513
> URL: https://issues.apache.org/jira/browse/ATLAS-1513
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Madhan Neethiraj
> Attachments: ATLAS-1513.patch
>
>
> To enable entity instance update/delete implementation to handle foreign-key 
> references, it should be possible to get all foreign-key references via 
> AtlasEntityType instance.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Apache Atlas Bridge for Flink

[Vimal Sharma]

Hi Stefan,
Apache Atlas does not have a bridge for Flink currently.

I don’t think there has been any request for this feature in the past.

Hope this helps. Thanks
Vimal





On 2/1/17, 4:05 PM, "Stefan Papp"  wrote:

>Hi,
>
>
>Is an Apache Atlas Bridge for Flink available?
>
>Has this feature already been requested.
>
>Stefan
>
>Stefan Papp
>Data Architect
>
>The unbelievable Machine Company GmbH
>Museumsplatz 1/10/13
>1070 Wien
>
>Tel: +43 - 1 - 361 99 77 - 215
>Mobil: +43664 2614367
>
>https://www.unbelievable-machine.com 
>
>*Blog  | *Karriere 
> | *Facebook 
> | *Twitter 
>
>
>Geschäftsführer: Ravin Mehta, Benno Wiedfeld
>Sitz der Gesellschaft: Berlin, Deutschland
>Handelsregister: Amtsgericht Charlottenburg HRB 115071B
>Umsatzsteuer-Identifikationsnummer: DE261090159
>
>Ausgezeichnet: *um ist zweifacher „Big Data Leader 2017“ 
>

Re: Review Request 56161: ATLAS-1513: updated AtlasEntityType with methods to get foreignKey references to a type; added helper methods in AtlasAttribute

[keval bhatt]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56161/#review163804
---


Ship it!




Ship It!

- keval bhatt


On Feb. 1, 2017, 10:47 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56161/
> ---
> 
> (Updated Feb. 1, 2017, 10:47 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-1513
> https://issues.apache.org/jira/browse/ATLAS-1513
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> - updated AtlasEntityType to get foreignKey-references to the type
> - moved mappedFromRef from AtlasStructType to AtlasEntityType (as it is not 
> applicable to structs)
> - simplified use of AtlasAttribute, by adding few helper methods and removing 
> a constructor parameter
> - fixed incorrect persistence of mappedFromRef/foreignKey constraints to 
> store; this was exposed in a recent change to hive-model.json
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> 2c00f54 
>   intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 
> d06364c 
>   intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java b83a241 
>   intg/src/main/java/org/apache/atlas/type/AtlasStructType.java bac5800 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java e4f1eea 
>   intg/src/test/java/org/apache/atlas/TestUtilsV2.java 4f2916c 
>   intg/src/test/java/org/apache/atlas/model/ModelTestUtil.java 5c72470 
>   intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java 4e15edd 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java
>  e63ea6e 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java
>  4f5b4f4 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java
>  163ddef 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  34e02c6 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  c98d340 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/GraphMutationContext.java
>  89419d9 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/MapVertexMapper.java
>  efafcc9 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java
>  22851ca 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java
>  2698504 
>   webapp/src/main/java/org/apache/atlas/util/RestUtils.java 5e69262 
>   
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasStructFormatConverter.java
>  8777510 
>   webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java ee1174a 
>   
> webapp/src/test/java/org/apache/atlas/web/resources/TypedefsJerseyResourceIT.java
>  8eacc73 
> 
> Diff: https://reviews.apache.org/r/56161/diff/
> 
> 
> Testing
> ---
> 
> - added unit tests
> - performed sanity tests with Atlas web application, like:
>   - hive notification processing
>   - search
>   - lineage
>   - delete entities
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

Re: Review Request 56161: ATLAS-1513: updated AtlasEntityType with methods to get foreignKey references to a type; added helper methods in AtlasAttribute

[keval bhatt]

> On Feb. 1, 2017, 12:36 p.m., keval bhatt wrote:
> > Ship It!

Tried Following usecase.

* Created entity
* search that created entity
* lineage of that created entity
* delete entity
* created hive_table entity using UI
* created hive_column entity using UI


- keval


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56161/#review163804
---


On Feb. 1, 2017, 10:47 a.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56161/
> ---
> 
> (Updated Feb. 1, 2017, 10:47 a.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-1513
> https://issues.apache.org/jira/browse/ATLAS-1513
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> - updated AtlasEntityType to get foreignKey-references to the type
> - moved mappedFromRef from AtlasStructType to AtlasEntityType (as it is not 
> applicable to structs)
> - simplified use of AtlasAttribute, by adding few helper methods and removing 
> a constructor parameter
> - fixed incorrect persistence of mappedFromRef/foreignKey constraints to 
> store; this was exposed in a recent change to hive-model.json
> 
> 
> Diffs
> -
> 
>   intg/src/main/java/org/apache/atlas/model/typedef/AtlasStructDef.java 
> 2c00f54 
>   intg/src/main/java/org/apache/atlas/type/AtlasClassificationType.java 
> d06364c 
>   intg/src/main/java/org/apache/atlas/type/AtlasEntityType.java b83a241 
>   intg/src/main/java/org/apache/atlas/type/AtlasStructType.java bac5800 
>   intg/src/main/java/org/apache/atlas/type/AtlasTypeUtil.java e4f1eea 
>   intg/src/test/java/org/apache/atlas/TestUtilsV2.java 4f2916c 
>   intg/src/test/java/org/apache/atlas/model/ModelTestUtil.java 5c72470 
>   intg/src/test/java/org/apache/atlas/type/TestAtlasEntityType.java 4e15edd 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/ArrayVertexMapper.java
>  e63ea6e 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java
>  4f5b4f4 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasStructDefStoreV1.java
>  163ddef 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  34e02c6 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  c98d340 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/GraphMutationContext.java
>  89419d9 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/MapVertexMapper.java
>  efafcc9 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java
>  22851ca 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/UniqAttrBasedEntityResolver.java
>  2698504 
>   webapp/src/main/java/org/apache/atlas/util/RestUtils.java 5e69262 
>   
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasStructFormatConverter.java
>  8777510 
>   webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java ee1174a 
>   
> webapp/src/test/java/org/apache/atlas/web/resources/TypedefsJerseyResourceIT.java
>  8eacc73 
> 
> Diff: https://reviews.apache.org/r/56161/diff/
> 
> 
> Testing
> ---
> 
> - added unit tests
> - performed sanity tests with Atlas web application, like:
>   - hive notification processing
>   - search
>   - lineage
>   - delete entities
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

[jira] [Commented] (ATLAS-1513) Update AtlasEntityType with a method to get foreign-key references

[Keval Bhatt (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1513?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15848353#comment-15848353
 ] 

Keval Bhatt commented on ATLAS-1513:


+1
Committed to master 
(http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/5d8fae55)

> Update AtlasEntityType with a method to get foreign-key references
> --
>
> Key: ATLAS-1513
> URL: https://issues.apache.org/jira/browse/ATLAS-1513
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Madhan Neethiraj
> Attachments: ATLAS-1513.patch
>
>
> To enable entity instance update/delete implementation to handle foreign-key 
> references, it should be possible to get all foreign-key references via 
> AtlasEntityType instance.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (ATLAS-1514) Duplicates in class array attribute are not removed during delete

[David Kantor (JIRA)]

David Kantor created ATLAS-1514:
---

 Summary: Duplicates in class array attribute are not removed 
during delete
 Key: ATLAS-1514
 URL: https://issues.apache.org/jira/browse/ATLAS-1514
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Reporter: David Kantor


Duplicates entries in a class array attribute are not removed when the target 
entity is deleted, leaving a dangling reference.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ATLAS-1512) Hive Hook fails due to - Table not found exception

[Russell Anderson (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1512?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15848409#comment-15848409
 ] 

Russell Anderson commented on ATLAS-1512:
-

ATLAS-1274 - table not found exception with temporary tables reported and not 
resolved may also be related.

> Hive Hook fails due to - Table not found exception
> --
>
> Key: ATLAS-1512
> URL: https://issues.apache.org/jira/browse/ATLAS-1512
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core, atlas-intg
> Environment: newly built, and configured Apache Atlas .7.1rc3
> BigInsights 4.2.0.0 -
> The example worked fine using Apache Atlas .7.0rc2
>Reporter: Russell Anderson
>Priority: Critical
>
> After configuring Hive Hook: Ran the following Hive SQL command:
> Create table sysibm.sparktest as select * from sysibm.sparktable;
> -the table sysibm.sparktable exists
> -the table system.sparktest is getting created as a result of this command.
> -the table gets successfully created within the schema sysibm but the Hive 
> Hook does is not able deal with this correctly.
> 2017-01-31 07:49:56,979 INFO  metastore.HiveMetaStore 
> (HiveMetaStore.java:logIn\
> fo(746)) - 6: get_table : db=sysibm tbl=sparktest
> 2017-01-31 07:49:56,980 INFO  HiveMetaStore.audit 
> (HiveMetaStore.java:logAuditE\
> vent(371)) - ugi=hiveip=unknown-ip-addr  cmd=get_table : db=sysibm 
> tbl=\
> sparktest
> 2017-01-31 07:49:56,984 ERROR metadata.Hive (Hive.java:getTable(1119)) - 
> Table \
> sparktest not found: sysibm.sparktest table not found
> 2017-01-31 07:49:56,984 ERROR hook.HiveHook (HiveHook.java:run(207)) - Atlas 
> ho\
> ok failed due to error
> java.lang.reflect.UndeclaredThrowableException
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInform\
> ation.java:1672)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:197)
> at 
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:5\
> 11)
> at java.util.concurrent.FutureTask.run(FutureTask.java:266)
> at 
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor\
> .java:1142)
> at 
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecuto\
> r.java:617)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: org.apache.hadoop.hive.ql.metadata.InvalidTableException: Table 
> not \
> found sparktest
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1120)
> at org.apache.hadoop.hive.ql.metadata.Hive.getTable(Hive.java:1090)
> at 
> org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.\
> java:559)
> at 
> org.apache.atlas.hive.hook.HiveHook.createOrUpdateEntities(HiveHook.\
> java:581)
> at 
> org.apache.atlas.hive.hook.HiveHook.processHiveEntity(HiveHook.java:\
> 669)
> at 
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:64\
> 9)
> at org.apache.atlas.hive.hook.HiveHook.collect(HiveHook.java:270)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:85)
> at org.apache.atlas.hive.hook.HiveHook$2$1.run(HiveHook.java:200)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:422)
> at 
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInform\
> ation.java:1657)
> ... 6 more
> 2017-01-31 07:49:57,029 INFO  log.PerfLogger 
> (PerfLogger.java:PerfLogEnd(148)) \



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Build failed in Jenkins: apache-atlas-nightly #579

[Apache Jenkins Server]

See 

Changes:

[kbhatt] ATLAS-1513: updated AtlasEntityType with methods to get foreign-key

--
[...truncated 5635 lines...]
127.0.0.1 - - [01/Feb/2017:14:30:13 +] "GET /api/atlas/types/classification 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:13 +] "GET /api/atlas/types/hive_db_v1 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:14 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:15 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:16 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:16 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:16 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:16 +] "POST 
/api/atlas/entities/fa4b1bb1-5917-40f3-a77d-93d292bed4b2/traits HTTP/1.1" 201 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:17 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:17 +] "POST 
/api/atlas/entities/fa4b1bb1-5917-40f3-a77d-93d292bed4b2/traits HTTP/1.1" 201 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:17 +] "DELETE 
/api/atlas/entities/fa4b1bb1-5917-40f3-a77d-93d292bed4b2/traits/TraitJhVOd9MINO 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:17 +] "POST 
/api/atlas/entities/fa4b1bb1-5917-40f3-a77d-93d292bed4b2?property=description 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:17 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:17 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:17 +] "DELETE 
/api/atlas/entities?type=hive_table&property=qualifiedName&value=table-RNt8OKLtNX
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
Tests run: 5, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.917 sec - in 
org.apache.atlas.notification.EntityNotificationIT
Running org.apache.atlas.notification.NotificationHookConsumerIT
127.0.0.1 - - [01/Feb/2017:14:30:21 +] "GET /api/atlas/types/tableType 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:21 +] "GET /api/atlas/types/serdeType 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:21 +] "GET /api/atlas/types/classification 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:21 +] "GET /api/atlas/types/hive_db_v1 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:22 +] "GET 
/api/atlas/discovery/search/dsl?query=hive_db+where+qualifiedName%3D'dbultdfcjQes'&limit=10&offset=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:23 +] "GET 
/api/atlas/entities?type=hive_db&property=qualifiedName&value=dbultdfcjQes 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:23 +] "GET 
/api/atlas/entities/34887e8e-96ed-4f34-a2cd-d30c4179ed6e/audit?count=1 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:23 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:23 +] "GET 
/api/atlas/entities/5f28c885-f2c5-497c-a143-72fdeced1b52 HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:23 +] "GET 
/api/atlas/entities/5f28c885-f2c5-497c-a143-72fdeced1b52 HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:23 +] "GET 
/api/atlas/discovery/search/dsl?query=hive_db+where+name%3D'dbWzpPbdA8UK'&limit=10&offset=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:24 +] "GET 
/api/atlas/discovery/search/dsl?query=hive_db+where+name%3D'dbWzpPbdA8UK'&limit=10&offset=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:25 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:25 +] "GET 
/api/atlas/discovery/search/dsl?query=hive_db+where+qualifiedName%3D'dbY4im8atYSe'&limit=10&offset=0
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:26 +] "GET 
/api/atlas/entities?type=hive_db&property=qualifiedName&value=dbY4im8atYSe 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:26 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:26 +] "GET 
/api/atlas/entities?type=hive_db&property=qualifiedName&value=dbWL32lIZOPf 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:26 +] "GET 
/api/atlas/entities?type=hive_db&property=qualifiedName&value=dbWL32lIZOPf 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:14:30:26 +] "GET 
/api/atlas/entities?type=hive_db&property=qualifiedName&value=

[jira] [Commented] (ATLAS-1514) Duplicates in class array attribute are not removed during delete

[David Kantor (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15848466#comment-15848466
 ] 

David Kantor commented on ATLAS-1514:
-

In progress

> Duplicates in class array attribute are not removed during delete
> -
>
> Key: ATLAS-1514
> URL: https://issues.apache.org/jira/browse/ATLAS-1514
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Reporter: David Kantor
>
> Duplicates entries in a class array attribute are not removed when the target 
> entity is deleted, leaving a dangling reference.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 56102: ATLAS-1388 : Cache entities that are created/updated

[Jeff Hagelberg]

> On Feb. 1, 2017, 10:42 a.m., Vimal Sharma wrote:
> > The cache size can grow large if the number of registered entities is high. 
> > It would be good to have a cache eviction policy just like it was done for 
> > DSL Query caching in ATLAS-1387

We could do that.  It seems like overkill though.  The big difference is that 
RequestContext is thread local, and a new RequestContext is created every time 
a http request comes in.  Once the http request finishes, 
RequestContext.clear() is called, and the cache goes out of scope and will be 
garbage collected.  (See AuditFilter for details).  The cache is limited to the 
instances used for processsing the request.  In the case of the compiled query 
cache, there is one effectively static instance of the cache that is used for 
all requests.  Without the eviction policy, that would continue growing until 
the JVM runs out of memory.


- Jeff


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56102/#review163800
---


On Jan. 30, 2017, 10:42 p.m., Jeff Hagelberg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56102/
> ---
> 
> (Updated Jan. 30, 2017, 10:42 p.m.)
> 
> 
> Review request for atlas and David Kantor.
> 
> 
> Bugs: ATLAS-1388
> https://issues.apache.org/jira/browse/ATLAS-1388
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Cache created entities in RequestContext when they are created.
> Update/refactor DefaultMetadataService.loadEntities() to use the cached 
> created entities.
> Use the cache in the following places:
> FullTextMapper
> DefaultMetadataService.onEntitiesAdded - check cache before calling 
> DefaultMetadataService.loadEntities
> DefaultMetadataService.onEntitiesUpdated - check cache before calling 
> DefaultMetadataService.loadEntities
> EntityResource.getEntityDefinition - check cache before calling 
> DefaultMetadataService.getEntityDefinition
> EntityResource.getResponse - check cache before calling 
> DefaultMetadataService.getEntityDefinition
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/graph/DefaultGraphPersistenceStrategy.java
>  f4d8f00da863deeb98209aec2dfc4ccb22734cfa 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapper.java
>  5be8d0bc349294dc4340ffe5831bfb1265825ed8 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
>  be02891cad3020e50d5244fd2aaf3c4af3a5954f 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java
>  7b2b753b930467e015d364819c5baa983fce0cf8 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  bae8b2ac0cae6bf6392f91f24030d81089ae660b 
>   repository/src/test/java/org/apache/atlas/TestUtils.java 
> cda9eac38861fd55c1494d6d90b5e750e6133545 
>   
> repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java
>  f2ca6a8fb1573755d16497236358358a07881e9c 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  9e850a9dc0ac122cf3fcdbf7df72a93643abc98d 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java
>  7444bf38834f59454d67cf763c932d665d7ef31c 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
>  c902f8126f6b516e84ab9d89b0ec80ac8ed819fd 
>   
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
>  aa1cdaa7848aeb767725b3161ccd904c8750536f 
>   
> repository/src/test/java/org/apache/atlas/util/CompiledQueryCacheKeyTest.java 
> c926f4d7d793ef095b20853277f3ba979483e039 
>   server-api/src/main/java/org/apache/atlas/RequestContext.java 
> 651a71dc238a3adaac9504a77ba439785fa07ca8 
> 
> Diff: https://reviews.apache.org/r/56102/diff/
> 
> 
> Testing
> ---
> 
> Ran full build, no regresssions found.
> 
> 
> Thanks,
> 
> Jeff Hagelberg
> 
>

Re: Review Request 56102: ATLAS-1388 : Cache entities that are created/updated

[Jeff Hagelberg]

> On Feb. 1, 2017, 10:42 a.m., Vimal Sharma wrote:
> > The cache size can grow large if the number of registered entities is high. 
> > It would be good to have a cache eviction policy just like it was done for 
> > DSL Query caching in ATLAS-1387
> 
> Jeff Hagelberg wrote:
> We could do that.  It seems like overkill though.  The big difference is 
> that RequestContext is thread local, and a new RequestContext is created 
> every time a http request comes in.  Once the http request finishes, 
> RequestContext.clear() is called, and the cache goes out of scope and will be 
> garbage collected.  (See AuditFilter for details).  The cache is limited to 
> the instances used for processsing the request.  In the case of the compiled 
> query cache, there is one effectively static instance of the cache that is 
> used for all requests.  Without the eviction policy, that would continue 
> growing until the JVM runs out of memory.

Is there a specific use case you are concerned about?  For example, if someone 
runs a DSL query that needs to return 100,000 entities, those entities would be 
cached.  However, even without the cache we would still need to have the 
100,000 entities in memory so they can be serialized to json and sent back to 
the client.  I'm not sure that the cache increases the amount of memory 
required by much.  What it does do, though, is keep the instances that are 
created in the java heap for a longer period of time.


- Jeff


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56102/#review163800
---


On Jan. 30, 2017, 10:42 p.m., Jeff Hagelberg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56102/
> ---
> 
> (Updated Jan. 30, 2017, 10:42 p.m.)
> 
> 
> Review request for atlas and David Kantor.
> 
> 
> Bugs: ATLAS-1388
> https://issues.apache.org/jira/browse/ATLAS-1388
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Cache created entities in RequestContext when they are created.
> Update/refactor DefaultMetadataService.loadEntities() to use the cached 
> created entities.
> Use the cache in the following places:
> FullTextMapper
> DefaultMetadataService.onEntitiesAdded - check cache before calling 
> DefaultMetadataService.loadEntities
> DefaultMetadataService.onEntitiesUpdated - check cache before calling 
> DefaultMetadataService.loadEntities
> EntityResource.getEntityDefinition - check cache before calling 
> DefaultMetadataService.getEntityDefinition
> EntityResource.getResponse - check cache before calling 
> DefaultMetadataService.getEntityDefinition
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/graph/DefaultGraphPersistenceStrategy.java
>  f4d8f00da863deeb98209aec2dfc4ccb22734cfa 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapper.java
>  5be8d0bc349294dc4340ffe5831bfb1265825ed8 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
>  be02891cad3020e50d5244fd2aaf3c4af3a5954f 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java
>  7b2b753b930467e015d364819c5baa983fce0cf8 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  bae8b2ac0cae6bf6392f91f24030d81089ae660b 
>   repository/src/test/java/org/apache/atlas/TestUtils.java 
> cda9eac38861fd55c1494d6d90b5e750e6133545 
>   
> repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java
>  f2ca6a8fb1573755d16497236358358a07881e9c 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  9e850a9dc0ac122cf3fcdbf7df72a93643abc98d 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java
>  7444bf38834f59454d67cf763c932d665d7ef31c 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
>  c902f8126f6b516e84ab9d89b0ec80ac8ed819fd 
>   
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
>  aa1cdaa7848aeb767725b3161ccd904c8750536f 
>   
> repository/src/test/java/org/apache/atlas/util/CompiledQueryCacheKeyTest.java 
> c926f4d7d793ef095b20853277f3ba979483e039 
>   server-api/src/main/java/org/apache/atlas/RequestContext.java 
> 651a71dc238a3adaac9504a77ba439785fa07ca8 
> 
> Diff: https://reviews.apache.org/r/56102/diff/
> 
> 
> Testing
> ---
> 
> Ran full build, no regresssions found.
> 
> 
> Thanks,
> 
> Jeff Hagelberg
> 
>

Re: Review Request 54680: ATLAS-1369 - Optimize gremlin queries generated by DSL translator

[Jeff Hagelberg]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/54680/#review163818
---



Any other comments on this?  If not, I will commit it this afternoon.

- Jeff Hagelberg


On Jan. 31, 2017, 10:54 p.m., Jeff Hagelberg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/54680/
> ---
> 
> (Updated Jan. 31, 2017, 10:54 p.m.)
> 
> 
> Review request for atlas and David Kantor.
> 
> 
> Bugs: ATLAS-1369
> https://issues.apache.org/jira/browse/ATLAS-1369
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> The way the queries are generated now, in many cases the indices are not used 
> when executing queries. The main problem is that gremlin cannot currently use 
> "has" expressions within an "and" or an "or". This JIRA introduces a 
> GremlinQueryOptimizer that attempts to extract "has" expressions out of "and" 
> expressions and turn "or" expressions into unions.
> 
> 
> Major changes:
> 
> * Updated version of Guava used by Atlas, so that could use 
> Lists.cartesianProduct
> * New package - org.apache.atlas.gremlin.optimizer - contains 
> optimization framework
> * Refactoring of Groovy Expression classes
> * new - AbstractFunctionExpression - super class of anything with a 
> caller
> * added 4 new methods to GroovyExpression
> * now, each GroovyExpression is a TraversalStepType associated with it
> These are used by the optimization framework
> 
> * Added feature toggle : atlas.query.gremlinOptimizerEnabled. The feature 
> turns optimization on/off.  It is provided as a fallback in the event that 
> issues are found with the optimizer.
> 
> * Most of the new logic is in GremlinQueryOptimizer, 
> ExpandAndsOptimization, and ExpandOrsOptimization
> * 
> Note - with these changes we saw a very significant improvement when using 
> IBM Graph as the Atlas Backend (which uses Titan 1 under the covers).
> 
> Please review these changes.  Silence implies consent.
> 
> 
> Diffs
> -
> 
>   
> common/src/main/java/org/apache/atlas/groovy/AbstractFunctionExpression.java 
> PRE-CREATION 
>   common/src/main/java/org/apache/atlas/groovy/AbstractGroovyExpression.java 
> 49eaae87e593d4e350ecb9a51a83e967d95b514b 
>   common/src/main/java/org/apache/atlas/groovy/ArithmeticExpression.java 
> 0aec5d03ea6e2a108129b20003d390e826f76bb5 
>   common/src/main/java/org/apache/atlas/groovy/BinaryExpression.java 
> ccc9204cdc9613efeb2b6548d8d426bee7cfec06 
>   common/src/main/java/org/apache/atlas/groovy/CastExpression.java 
> 963724c7cefcd871918d340c25c197143d155d56 
>   common/src/main/java/org/apache/atlas/groovy/ClosureExpression.java 
> 2d702093937b69a7d82e8d5f58fb528144377de5 
>   common/src/main/java/org/apache/atlas/groovy/CodeBlockExpression.java 
> 9a726f2f82764b0a93c523467b3fa316e2dbf4e0 
>   common/src/main/java/org/apache/atlas/groovy/ComparisonExpression.java 
> 345f838635f62457b0fd42efed83fd5b96b5c16c 
>   
> common/src/main/java/org/apache/atlas/groovy/ComparisonOperatorExpression.java
>  63638b73b048ed301f5a172bf7cec0e2f618d019 
>   common/src/main/java/org/apache/atlas/groovy/FieldExpression.java 
> f6d06bd194e91d332fd7418da2a9ff24342b4647 
>   common/src/main/java/org/apache/atlas/groovy/FunctionCallExpression.java 
> dd9b1d58e944a4c172a571f4dcacc6601bce22be 
>   common/src/main/java/org/apache/atlas/groovy/GroovyExpression.java 
> 493bd3df1274fe67f47abe6606a2f447af8fa470 
>   common/src/main/java/org/apache/atlas/groovy/IdentifierExpression.java 
> 6abdbf02ed54389f6bd7d9516fa14dad5d791b8d 
>   common/src/main/java/org/apache/atlas/groovy/ListExpression.java 
> f7acaace9c4e5c9f87c596f7c11c4c41c1b13db5 
>   common/src/main/java/org/apache/atlas/groovy/LiteralExpression.java 
> 008c885686985ff38e0d1ad28a5e0f736e700a39 
>   common/src/main/java/org/apache/atlas/groovy/LogicalExpression.java 
> ee5829b39e8c4bd6c80942328100259d2762c506 
>   common/src/main/java/org/apache/atlas/groovy/RangeExpression.java 
> 7322f697474f05630398c6379ec0aa4c3ba178d6 
>   common/src/main/java/org/apache/atlas/groovy/StatementListExpression.java 
> PRE-CREATION 
>   common/src/main/java/org/apache/atlas/groovy/TernaryOperatorExpression.java 
> 75a2f8682f62d1d7bf173a6d704c7accb0c27808 
>   common/src/main/java/org/apache/atlas/groovy/TraversalStepType.java 
> PRE-CREATION 
>   common/src/main/java/org/apache/atlas/groovy/TypeCoersionExpression.java 
> 4a6105234fe9ca33830516558a40f459ad7082f9 
>   
> common/src/main/java/org/apache/atlas/groovy/VariableAssignmentExpression.java
>  7e018f1dcffed654b958c59d70378d6178db4367 
>   distro/src/conf/atlas-application.properties 
> d9e2f6efae611b1360d1a14ecabc981247f2a5b8 
>   graphdb/titan0/pom.xml 58a5cb852ad9d1e78fc8

[jira] [Created] (ATLAS-1515) Build (tests) fail with python3, only works with python2

[JIRA]

Péter Gergő Barna created ATLAS-1515:


 Summary: Build (tests) fail with python3, only works with python2
 Key: ATLAS-1515
 URL: https://issues.apache.org/jira/browse/ATLAS-1515
 Project: Atlas
  Issue Type: Bug
Affects Versions: 0.6-incubating, 0.7-incubating
Reporter: Péter Gergő Barna
Priority: Trivial


Build (tests) fail with python3, only works with python2
Only atlas-distro module is affected.

{noformat}
[INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ atlas-distro ---
Traceback (most recent call last):
  File "unitTests.py", line 118, in 
main()
  File "unitTests.py", line 87, in main
suite = all_tests_suite()
  File "unitTests.py", line 78, in all_tests_suite
suite = unittest.TestLoader().loadTestsFromNames(tests_list)
  File "/Users/pbarna/anaconda3/lib/python3.5/unittest/loader.py", line 219, in 
loadTestsFromNames
suites = [self.loadTestsFromName(name, module) for name in names]
  File "/Users/pbarna/anaconda3/lib/python3.5/unittest/loader.py", line 219, in 

suites = [self.loadTestsFromName(name, module) for name in names]
  File "/Users/pbarna/anaconda3/lib/python3.5/unittest/loader.py", line 153, in 
loadTestsFromName
module = __import__(module_name)
  File 
"/Users/pbarna/Desktop/apache-source/incubator-atlas/distro/src/test/python/scripts/TestMetadata.py",
 line 26, in 
import atlas_config as mc
  File 
"/Users/pbarna/Desktop/apache-source/incubator-atlas/distro/src/bin/atlas_config.py",
 line 135
except OSError, e:
  ^
SyntaxError: invalid syntax
[INFO] 
[INFO] BUILD FAILURE
[INFO] 
{noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ATLAS-1515) Build (tests) fail with python3, only works with python2

[JIRA]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1515?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Péter Gergő Barna updated ATLAS-1515:
-
Attachment: ATLAS-1515.patch

> Build (tests) fail with python3, only works with python2
> 
>
> Key: ATLAS-1515
> URL: https://issues.apache.org/jira/browse/ATLAS-1515
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.6-incubating, 0.7-incubating
>Reporter: Péter Gergő Barna
>Priority: Trivial
> Attachments: ATLAS-1515.patch
>
>   Original Estimate: 1h
>  Remaining Estimate: 1h
>
> Build (tests) fail with python3, only works with python2
> Only atlas-distro module is affected.
> {noformat}
> [INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ atlas-distro ---
> Traceback (most recent call last):
>   File "unitTests.py", line 118, in 
> main()
>   File "unitTests.py", line 87, in main
> suite = all_tests_suite()
>   File "unitTests.py", line 78, in all_tests_suite
> suite = unittest.TestLoader().loadTestsFromNames(tests_list)
>   File "/Users/pbarna/anaconda3/lib/python3.5/unittest/loader.py", line 219, 
> in loadTestsFromNames
> suites = [self.loadTestsFromName(name, module) for name in names]
>   File "/Users/pbarna/anaconda3/lib/python3.5/unittest/loader.py", line 219, 
> in 
> suites = [self.loadTestsFromName(name, module) for name in names]
>   File "/Users/pbarna/anaconda3/lib/python3.5/unittest/loader.py", line 153, 
> in loadTestsFromName
> module = __import__(module_name)
>   File 
> "/Users/pbarna/Desktop/apache-source/incubator-atlas/distro/src/test/python/scripts/TestMetadata.py",
>  line 26, in 
> import atlas_config as mc
>   File 
> "/Users/pbarna/Desktop/apache-source/incubator-atlas/distro/src/bin/atlas_config.py",
>  line 135
> except OSError, e:
>   ^
> SyntaxError: invalid syntax
> [INFO] 
> 
> [INFO] BUILD FAILURE
> [INFO] 
> 
> {noformat}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 56102: ATLAS-1388 : Cache entities that are created/updated

[David Kantor]

> On Feb. 1, 2017, 10:42 a.m., Vimal Sharma wrote:
> > The cache size can grow large if the number of registered entities is high. 
> > It would be good to have a cache eviction policy just like it was done for 
> > DSL Query caching in ATLAS-1387
> 
> Jeff Hagelberg wrote:
> We could do that.  It seems like overkill though.  The big difference is 
> that RequestContext is thread local, and a new RequestContext is created 
> every time a http request comes in.  Once the http request finishes, 
> RequestContext.clear() is called, and the cache goes out of scope and will be 
> garbage collected.  (See AuditFilter for details).  The cache is limited to 
> the instances used for processsing the request.  In the case of the compiled 
> query cache, there is one effectively static instance of the cache that is 
> used for all requests.  Without the eviction policy, that would continue 
> growing until the JVM runs out of memory.
> 
> Jeff Hagelberg wrote:
> Is there a specific use case you are concerned about?  For example, if 
> someone runs a DSL query that needs to return 100,000 entities, those 
> entities would be cached.  However, even without the cache we would still 
> need to have the 100,000 entities in memory so they can be serialized to json 
> and sent back to the client.  I'm not sure that the cache increases the 
> amount of memory required by much.  What it does do, though, is keep the 
> instances that are created in the java heap for a longer period of time.

There no need for a cache eviction policy as this cache is only active for the 
duration of the request.   Perhaps to address the comment, the cache could be 
cleared upon completion of the request to make the entries available for 
garbage collection sooner.


- David


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56102/#review163800
---


On Jan. 30, 2017, 10:42 p.m., Jeff Hagelberg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56102/
> ---
> 
> (Updated Jan. 30, 2017, 10:42 p.m.)
> 
> 
> Review request for atlas and David Kantor.
> 
> 
> Bugs: ATLAS-1388
> https://issues.apache.org/jira/browse/ATLAS-1388
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Cache created entities in RequestContext when they are created.
> Update/refactor DefaultMetadataService.loadEntities() to use the cached 
> created entities.
> Use the cache in the following places:
> FullTextMapper
> DefaultMetadataService.onEntitiesAdded - check cache before calling 
> DefaultMetadataService.loadEntities
> DefaultMetadataService.onEntitiesUpdated - check cache before calling 
> DefaultMetadataService.loadEntities
> EntityResource.getEntityDefinition - check cache before calling 
> DefaultMetadataService.getEntityDefinition
> EntityResource.getResponse - check cache before calling 
> DefaultMetadataService.getEntityDefinition
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/graph/DefaultGraphPersistenceStrategy.java
>  f4d8f00da863deeb98209aec2dfc4ccb22734cfa 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapper.java
>  5be8d0bc349294dc4340ffe5831bfb1265825ed8 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
>  be02891cad3020e50d5244fd2aaf3c4af3a5954f 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java
>  7b2b753b930467e015d364819c5baa983fce0cf8 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  bae8b2ac0cae6bf6392f91f24030d81089ae660b 
>   repository/src/test/java/org/apache/atlas/TestUtils.java 
> cda9eac38861fd55c1494d6d90b5e750e6133545 
>   
> repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java
>  f2ca6a8fb1573755d16497236358358a07881e9c 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  9e850a9dc0ac122cf3fcdbf7df72a93643abc98d 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java
>  7444bf38834f59454d67cf763c932d665d7ef31c 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
>  c902f8126f6b516e84ab9d89b0ec80ac8ed819fd 
>   
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
>  aa1cdaa7848aeb767725b3161ccd904c8750536f 
>   
> repository/src/test/java/org/apache/atlas/util/CompiledQueryCacheKeyTest.java 
> c926f4d7d793ef095b20853277f3ba979483e039 
>   server-api/src/main/java/org/apache/atlas/RequestContext.java 
> 651a

[jira] [Created] (ATLAS-1516) fix unit test failure in DefaultMetadataServiceTest.testSpecialCharacters

[Madhan Neethiraj (JIRA)]

Madhan Neethiraj created ATLAS-1516:
---

 Summary: fix unit test failure in 
DefaultMetadataServiceTest.testSpecialCharacters
 Key: ATLAS-1516
 URL: https://issues.apache.org/jira/browse/ATLAS-1516
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 0.8-incubating
Reporter: Madhan Neethiraj


DefaultMetadataServiceTest.testSpecialCharacters() fails frequently - due to 
use of json-ified strings to compare equality of two Referencable objects. This 
comparison fails when value-map entries are serialized in different orders in 
the compared objects. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Changes to JIRA ServiceDesk

[Chris Lambertus]

> On Jan 30, 2017, at 2:54 PM, Pono Takamori  wrote:
> 
> Greetings from Infra,
> We recently began our upgrade path for modernizing JIRA and in the
> process ServiceDesk was also upgraded.  Before this upgrade the use of
> ServiceDesk was very straightforward for admins, but now we've
> discovered that it has irrevocably changed the permission scheme for
> ticket assignment.


All, thanks for your patience with this problem. We have identified a potential 
solution for the problem of project admins not being able to add agents. We are 
discussing and testing internally at the moment and expect to have another 
update in a few hours.

-Chris
ASF Infra



signature.asc
Description: Message signed with OpenPGP

Review Request 56182: ATLAS-1516: fix unit test failure in DefaultMetadataServiceTest.testSpecialCharacters()

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56182/
---

Review request for atlas.


Bugs: ATLAS-1516
https://issues.apache.org/jira/browse/ATLAS-1516


Repository: atlas


Description
---

There were couple of issues:
- the test used Json-ified strings to compare 2 Referenceable objects - which 
failed sometime due to Json strings having different order of map entries of 
Referenceable values. Test was updated to use Referenceable.equals() to compare 
the objects
- array type attribute in one Referenceable was String[], while the other had 
List. This resulted in comparision failure. Test was updated to replace 
use of String[] with List


Diffs
-

  
repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
 aa1cdaa 
  typesystem/src/main/java/org/apache/atlas/typesystem/Referenceable.java 
401df5f 

Diff: https://reviews.apache.org/r/56182/diff/


Testing
---

- ran the updated unit test a number of times and verified that it passes


Thanks,

Madhan Neethiraj

[jira] [Updated] (ATLAS-1516) fix unit test failure in DefaultMetadataServiceTest.testSpecialCharacters

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-1516:

Attachment: ATLAS-1516.patch

Patch is available at the review board here: https://reviews.apache.org/r/56182/

> fix unit test failure in DefaultMetadataServiceTest.testSpecialCharacters
> -
>
> Key: ATLAS-1516
> URL: https://issues.apache.org/jira/browse/ATLAS-1516
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Madhan Neethiraj
> Attachments: ATLAS-1516.patch
>
>
> DefaultMetadataServiceTest.testSpecialCharacters() fails frequently - due to 
> use of json-ified strings to compare equality of two Referencable objects. 
> This comparison fails when value-map entries are serialized in different 
> orders in the compared objects. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 56182: ATLAS-1516: fix unit test failure in DefaultMetadataServiceTest.testSpecialCharacters()

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56182/#review163845
---


Ship it!




Ship It!

- Apoorv Naik


On Feb. 1, 2017, 6:09 p.m., Madhan Neethiraj wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56182/
> ---
> 
> (Updated Feb. 1, 2017, 6:09 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-1516
> https://issues.apache.org/jira/browse/ATLAS-1516
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> There were couple of issues:
> - the test used Json-ified strings to compare 2 Referenceable objects - which 
> failed sometime due to Json strings having different order of map entries of 
> Referenceable values. Test was updated to use Referenceable.equals() to 
> compare the objects
> - array type attribute in one Referenceable was String[], while the other had 
> List. This resulted in comparision failure. Test was updated to 
> replace use of String[] with List
> 
> 
> Diffs
> -
> 
>   
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
>  aa1cdaa 
>   typesystem/src/main/java/org/apache/atlas/typesystem/Referenceable.java 
> 401df5f 
> 
> Diff: https://reviews.apache.org/r/56182/diff/
> 
> 
> Testing
> ---
> 
> - ran the updated unit test a number of times and verified that it passes
> 
> 
> Thanks,
> 
> Madhan Neethiraj
> 
>

[jira] [Commented] (ATLAS-1516) fix unit test failure in DefaultMetadataServiceTest.testSpecialCharacters

[Madhan Neethiraj (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15848829#comment-15848829
 ] 

Madhan Neethiraj commented on ATLAS-1516:
-

Committed to master: 
http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/f640da72

> fix unit test failure in DefaultMetadataServiceTest.testSpecialCharacters
> -
>
> Key: ATLAS-1516
> URL: https://issues.apache.org/jira/browse/ATLAS-1516
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Madhan Neethiraj
> Attachments: ATLAS-1516.patch
>
>
> DefaultMetadataServiceTest.testSpecialCharacters() fails frequently - due to 
> use of json-ified strings to compare equality of two Referencable objects. 
> This comparison fails when value-map entries are serialized in different 
> orders in the compared objects. 



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Build failed in Jenkins: apache-atlas-nightly #580

[Apache Jenkins Server]

See 

Changes:

[madhan] ATLAS-1516: fix unit test failure in

--
[...truncated 5542 lines...]
127.0.0.1 - - [01/Feb/2017:21:09:24 +] "GET 
/api/atlas/lineage/hive/table/sales_fact_monthly_mvZ8JgkKNdXD/inputs/graph 
HTTP/1.1" 500 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:24 +] "GET 
/api/atlas/entities?type=hive_table_v1&property=qualifiedName&value=sales_fact_monthly_mvZ8JgkKNdXD
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:24 +] "GET 
/api/atlas/lineage/fa46ec6b-5f8b-4909-a867-79c0bb3803ab/inputs/graph HTTP/1.1" 
200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:25 +] "GET 
/api/atlas/lineage/hive/table/sales_factrCbXr5k5sA/outputs/graph HTTP/1.1" 500 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:25 +] "GET 
/api/atlas/entities?type=hive_table_v1&property=qualifiedName&value=sales_factrCbXr5k5sA
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:25 +] "GET 
/api/atlas/lineage/8de3e264-98f9-4496-9fe7-928ba7ceba71/outputs/graph HTTP/1.1" 
200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:27 +] "GET 
/api/atlas/lineage/hive/table/sales_factrCbXr5k5sA/schema HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:27 +] "GET 
/api/atlas/lineage/hive/table/SalesaKuHwrZmpT/schema HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:27 +] "GET 
/api/atlas/entities?type=hive_table_v1&property=qualifiedName&value=sales_factrCbXr5k5sA
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:27 +] "GET 
/api/atlas/lineage/8de3e264-98f9-4496-9fe7-928ba7ceba71/schema HTTP/1.1" 200 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:28 +] "GET 
/api/atlas/lineage/hive/table/blah/schema HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:28 +] "GET 
/api/atlas/entities?type=hive_table_v1&property=qualifiedName&value=sales_fact_monthly_mvj3A4ZtWjxK
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:28 +] "GET 
/api/atlas/v2/lineage/25c8db55-02e8-463a-9929-93b84c811f70?direction=INPUT&depth=5
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:29 +] "GET 
/api/atlas/entities?type=hive_table_v1&property=qualifiedName&value=sales_fact_monthly_mvj3A4ZtWjxK
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:29 +] "GET 
/api/atlas/v2/lineage/25c8db55-02e8-463a-9929-93b84c811f70?direction=BOTH&depth=5
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:31 +] "GET 
/api/atlas/entities?type=hive_table_v1&property=qualifiedName&value=sales_factWTICokaear
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:31 +] "GET 
/api/atlas/v2/lineage/74fe5833-239e-4f88-830f-c76b89524918?direction=OUTPUT&depth=5
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
Tests run: 11, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 13.802 sec <<< 
FAILURE! - in org.apache.atlas.web.resources.EntityLineageJerseyResourceIT
testInputsGraph(org.apache.atlas.web.resources.EntityLineageJerseyResourceIT)  
Time elapsed: 0.025 sec  <<< FAILURE!
org.apache.atlas.AtlasServiceException: Metadata service API 
org.apache.atlas.AtlasBaseClient$APIInfo@2b780d7a failed with status 500 
(Internal Server Error) Response Body ({"error":"Failed with 
java.lang.NullPointerException"})

testOutputsGraph(org.apache.atlas.web.resources.EntityLineageJerseyResourceIT)  
Time elapsed: 0.013 sec  <<< FAILURE!
org.apache.atlas.AtlasServiceException: Metadata service API 
org.apache.atlas.AtlasBaseClient$APIInfo@27916d4b failed with status 500 
(Internal Server Error) Response Body ({"error":"Failed with 
java.lang.NullPointerException"})

Running org.apache.atlas.CredentialProviderUtilityIT
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.564 sec - in 
org.apache.atlas.CredentialProviderUtilityIT
Running org.apache.atlas.notification.EntityNotificationIT
127.0.0.1 - - [01/Feb/2017:21:09:38 +] "GET /api/atlas/types/tableType 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:39 +] "GET /api/atlas/types/serdeType 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:39 +] "GET /api/atlas/types/classification 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:39 +] "GET /api/atlas/types/hive_db_v1 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:39 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:40 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:41 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:41 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [01/Feb/2017:21:09:41 +] "POST /api/atlas/

Re: Review Request 56113: Fixed issues with deletion during updates

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56113/#review163881
---


Ship it!




Suma - I merged this patch with other recent changes. I will commit this patch 
shortly. Thanks!

- Madhan Neethiraj


On Feb. 1, 2017, midnight, Suma Shivaprasad wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56113/
> ---
> 
> (Updated Feb. 1, 2017, midnight)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-1509
> https://issues.apache.org/jira/browse/ATLAS-1509
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Added tests for deletion during updates and fixed issues found
> 
> 
> Diffs
> -
> 
>   intg/src/test/java/org/apache/atlas/TestUtilsV2.java 4f2916c 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
>  48bb77e 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
>  34e02c6 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/EntityGraphMapper.java
>  c98d340 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/StructVertexMapper.java
>  22851ca 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/AtlasTypeDefGraphStoreTest.java
>  25bae44 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
>  c902f81 
>   repository/src/test/java/org/apache/atlas/services/MetricsServiceTest.java 
> bb8c223 
>   server-api/src/main/java/org/apache/atlas/RequestContextV1.java 1fc11a2 
>   
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasEntityFormatConverter.java
>  75a7183 
>   
> webapp/src/main/java/org/apache/atlas/web/adapters/AtlasStructFormatConverter.java
>  8777510 
>   webapp/src/test/java/org/apache/atlas/web/adapters/TestEntitiesREST.java 
> 47e1fa3 
> 
> Diff: https://reviews.apache.org/r/56113/diff/
> 
> 
> Testing
> ---
> 
> 
> Thanks,
> 
> Suma Shivaprasad
> 
>

Re: Review Request 56102: ATLAS-1388 : Cache entities that are created/updated

[Jeff Hagelberg]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56102/
---

(Updated Feb. 1, 2017, 9:41 p.m.)


Review request for atlas and David Kantor.


Changes
---

Rebased changes, addressed code review comments -- now we're clearing the cache 
explicitly at the end of each request.


Bugs: ATLAS-1388
https://issues.apache.org/jira/browse/ATLAS-1388


Repository: atlas


Description
---

Cache created entities in RequestContext when they are created.
Update/refactor DefaultMetadataService.loadEntities() to use the cached 
created entities.
Use the cache in the following places:
FullTextMapper
DefaultMetadataService.onEntitiesAdded - check cache before calling 
DefaultMetadataService.loadEntities
DefaultMetadataService.onEntitiesUpdated - check cache before calling 
DefaultMetadataService.loadEntities
EntityResource.getEntityDefinition - check cache before calling 
DefaultMetadataService.getEntityDefinition
EntityResource.getResponse - check cache before calling 
DefaultMetadataService.getEntityDefinition


Diffs (updated)
-

  
repository/src/main/java/org/apache/atlas/discovery/graph/DefaultGraphPersistenceStrategy.java
 f4d8f00da863deeb98209aec2dfc4ccb22734cfa 
  
repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapper.java 
5be8d0bc349294dc4340ffe5831bfb1265825ed8 
  
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
 6608551be44033f07ddbd2ac3f3764d1ac3b0f22 
  
repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java
 38a553abf0151221c5d802e9ce1977a693d1a698 
  
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
 bae8b2ac0cae6bf6392f91f24030d81089ae660b 
  repository/src/test/java/org/apache/atlas/TestUtils.java 
cda9eac38861fd55c1494d6d90b5e750e6133545 
  
repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java
 d447c2d90fec6c03c7fecb8d8ef57a6ea072d35a 
  
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
 9e850a9dc0ac122cf3fcdbf7df72a93643abc98d 
  
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java
 7444bf38834f59454d67cf763c932d665d7ef31c 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
 c902f8126f6b516e84ab9d89b0ec80ac8ed819fd 
  
repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
 03ef4fe02f9b4eba273297ea954f00c9181f1f02 
  server-api/src/main/java/org/apache/atlas/RequestContext.java 
651a71dc238a3adaac9504a77ba439785fa07ca8 

Diff: https://reviews.apache.org/r/56102/diff/


Testing
---

Ran full build, no regresssions found.


Thanks,

Jeff Hagelberg

[jira] [Commented] (ATLAS-1509) Fix issues with deletion of edges, owned vertices during update

[Madhan Neethiraj (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15848974#comment-15848974
 ] 

Madhan Neethiraj commented on ATLAS-1509:
-

Patch is available at the review board here: https://reviews.apache.org/r/56113/


> Fix issues with deletion of edges, owned vertices during update
> ---
>
> Key: ATLAS-1509
> URL: https://issues.apache.org/jira/browse/ATLAS-1509
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Suma Shivaprasad
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ATLAS-1509) Fix issues with deletion of edges, owned vertices during update

[Madhan Neethiraj (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1509?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15848975#comment-15848975
 ] 

Madhan Neethiraj commented on ATLAS-1509:
-

Patch committed to master: 
http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/4367c491

> Fix issues with deletion of edges, owned vertices during update
> ---
>
> Key: ATLAS-1509
> URL: https://issues.apache.org/jira/browse/ATLAS-1509
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Suma Shivaprasad
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 56102: ATLAS-1388 : Cache entities that are created/updated

[Jeff Hagelberg]

> On Feb. 1, 2017, 10:42 a.m., Vimal Sharma wrote:
> > The cache size can grow large if the number of registered entities is high. 
> > It would be good to have a cache eviction policy just like it was done for 
> > DSL Query caching in ATLAS-1387
> 
> Jeff Hagelberg wrote:
> We could do that.  It seems like overkill though.  The big difference is 
> that RequestContext is thread local, and a new RequestContext is created 
> every time a http request comes in.  Once the http request finishes, 
> RequestContext.clear() is called, and the cache goes out of scope and will be 
> garbage collected.  (See AuditFilter for details).  The cache is limited to 
> the instances used for processsing the request.  In the case of the compiled 
> query cache, there is one effectively static instance of the cache that is 
> used for all requests.  Without the eviction policy, that would continue 
> growing until the JVM runs out of memory.
> 
> Jeff Hagelberg wrote:
> Is there a specific use case you are concerned about?  For example, if 
> someone runs a DSL query that needs to return 100,000 entities, those 
> entities would be cached.  However, even without the cache we would still 
> need to have the 100,000 entities in memory so they can be serialized to json 
> and sent back to the client.  I'm not sure that the cache increases the 
> amount of memory required by much.  What it does do, though, is keep the 
> instances that are created in the java heap for a longer period of time.
> 
> David Kantor wrote:
> There no need for a cache eviction policy as this cache is only active 
> for the duration of the request.   Perhaps to address the comment, the cache 
> could be cleared upon completion of the request to make the entries available 
> for garbage collection sooner.

Sure, I've changed it so that we explicitly clear the cache.


- Jeff


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56102/#review163800
---


On Feb. 1, 2017, 9:41 p.m., Jeff Hagelberg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56102/
> ---
> 
> (Updated Feb. 1, 2017, 9:41 p.m.)
> 
> 
> Review request for atlas and David Kantor.
> 
> 
> Bugs: ATLAS-1388
> https://issues.apache.org/jira/browse/ATLAS-1388
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Cache created entities in RequestContext when they are created.
> Update/refactor DefaultMetadataService.loadEntities() to use the cached 
> created entities.
> Use the cache in the following places:
> FullTextMapper
> DefaultMetadataService.onEntitiesAdded - check cache before calling 
> DefaultMetadataService.loadEntities
> DefaultMetadataService.onEntitiesUpdated - check cache before calling 
> DefaultMetadataService.loadEntities
> EntityResource.getEntityDefinition - check cache before calling 
> DefaultMetadataService.getEntityDefinition
> EntityResource.getResponse - check cache before calling 
> DefaultMetadataService.getEntityDefinition
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/graph/DefaultGraphPersistenceStrategy.java
>  f4d8f00da863deeb98209aec2dfc4ccb22734cfa 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapper.java
>  5be8d0bc349294dc4340ffe5831bfb1265825ed8 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
>  6608551be44033f07ddbd2ac3f3764d1ac3b0f22 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java
>  38a553abf0151221c5d802e9ce1977a693d1a698 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  bae8b2ac0cae6bf6392f91f24030d81089ae660b 
>   repository/src/test/java/org/apache/atlas/TestUtils.java 
> cda9eac38861fd55c1494d6d90b5e750e6133545 
>   
> repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java
>  d447c2d90fec6c03c7fecb8d8ef57a6ea072d35a 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  9e850a9dc0ac122cf3fcdbf7df72a93643abc98d 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java
>  7444bf38834f59454d67cf763c932d665d7ef31c 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
>  c902f8126f6b516e84ab9d89b0ec80ac8ed819fd 
>   
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
>  03ef4fe02f9b4eba273297ea954f00c9181f1f02 
>   server-api/src/main/java/org/apache/atlas/RequestContext.java 
> 651a71dc238a3adaac9504a77ba439785fa07ca8 

Review Request 56184: ATLAS-1499: Notification processing using V2 Store

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56184/
---

Review request for atlas, Madhan Neethiraj, Sarath Subramanian, Suma 
Shivaprasad, and Vimal Sharma.


Bugs: ATLAS-1499
https://issues.apache.org/jira/browse/ATLAS-1499


Repository: atlas


Description
---

ATLAS-1499: Notification processing using V2 Store


Diffs
-

  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
 1ef803c0 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasAbstractFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasArrayFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasClassificationFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasEntityFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasEnumFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasFormatConverters.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasInstanceConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasMapFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasObjectIdFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasPrimitiveFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasStructFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java
 c42f95ff 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java
 2848a20e 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
 1590aeea 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 08361eac 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
 31a619e5 
  server-api/src/main/java/org/apache/atlas/RequestContextV1.java 23eb4ce0 
  
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
 f2416810 
  webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java 51077672 
  webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java aa209f98 
  webapp/src/test/java/org/apache/atlas/LocalAtlasClientTest.java c5616dfe 
  
webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerKafkaTest.java
 873e5625 
  
webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerTest.java
 f06f7912 

Diff: https://reviews.apache.org/r/56184/diff/


Testing
---

mvn clean install -DskipITs executed successfully.

UI/Manual testing details

1. Manually created table in hive, showed up in graph and as an entity from the 
search UI
2. Created a view from the above table and the lineage showed up in atlas 
dashboard
3. added a column to the above table (changes could be seen on the UI)
4. altered table columns (changes could be seen on the UI)


Thanks,

Apoorv Naik

Re: [5/5] incubator-atlas git commit: ATLAS-1369 - Optimize gremlin queries generated by DSL translator

[Madhan Neethiraj]

Jeff,

Does this patch require JDK8? Compiling with JDK7 fails with the failures like:
[ERROR] 
/Users/mneethiraj/Apache/git/incubator-atlas/repository/src/main/java/org/apache/atlas/gremlin/optimizer/ExpressionFinder.java:20:
 error: package java.util.function does not exist
[ERROR] import java.util.function.Function;
[ERROR]  ^
[ERROR] 
/Users/mneethiraj/Apache/git/incubator-atlas/repository/src/main/java/org/apache/atlas/gremlin/optimizer/ExpressionFinder.java:32:
 error: cannot find symbol
[ERROR] private final Function predicate;
[ERROR]   ^


Can you please review and update to make it compatible with JDK7? 

Thanks,
Madhan




On 2/1/17, 1:05 PM, "jnhagelb...@apache.org"  wrote:

ATLAS-1369 - Optimize gremlin queries generated by DSL translator


Project: http://git-wip-us.apache.org/repos/asf/incubator-atlas/repo
Commit: 
http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/aa74c73d
Tree: http://git-wip-us.apache.org/repos/asf/incubator-atlas/tree/aa74c73d
Diff: http://git-wip-us.apache.org/repos/asf/incubator-atlas/diff/aa74c73d

Branch: refs/heads/master
Commit: aa74c73d0e9fbfd9c51de9f6cdcdea6141575e8e
Parents: f640da7
Author: Jeff Hagelberg 
Authored: Wed Feb 1 16:03:34 2017 -0500
Committer: Jeff Hagelberg 
Committed: Wed Feb 1 16:04:39 2017 -0500

--
 .../groovy/AbstractFunctionExpression.java  |  57 ++
 .../atlas/groovy/AbstractGroovyExpression.java  |   9 +
 .../atlas/groovy/ArithmeticExpression.java  |  12 +
 .../apache/atlas/groovy/BinaryExpression.java   |   9 +-
 .../org/apache/atlas/groovy/CastExpression.java |  14 +-
 .../apache/atlas/groovy/ClosureExpression.java  |  90 ++-
 .../atlas/groovy/CodeBlockExpression.java   |  61 --
 .../atlas/groovy/ComparisonExpression.java  |  12 +
 .../groovy/ComparisonOperatorExpression.java|   8 +
 .../apache/atlas/groovy/FieldExpression.java|  21 +-
 .../atlas/groovy/FunctionCallExpression.java|  88 ++-
 .../apache/atlas/groovy/GroovyExpression.java   |  42 +-
 .../atlas/groovy/IdentifierExpression.java  |  31 +
 .../apache/atlas/groovy/LabeledExpression.java  |  54 ++
 .../org/apache/atlas/groovy/ListExpression.java |  12 +
 .../apache/atlas/groovy/LiteralExpression.java  |  25 +-
 .../apache/atlas/groovy/LogicalExpression.java  |  12 +
 .../apache/atlas/groovy/RangeExpression.java|  62 +-
 .../atlas/groovy/StatementListExpression.java   |  98 +++
 .../atlas/groovy/TernaryOperatorExpression.java |  25 +-
 .../apache/atlas/groovy/TraversalStepType.java  | 121 
 .../atlas/groovy/TypeCoersionExpression.java|  19 +-
 .../groovy/VariableAssignmentExpression.java|  16 +-
 distro/src/conf/atlas-application.properties|   7 +
 graphdb/titan0/pom.xml  |   8 +
 intg/pom.xml|   1 -
 pom.xml |   8 +-
 release-log.txt |   5 +-
 .../atlas/discovery/DataSetLineageService.java  |   7 +-
 .../gremlin/Gremlin2ExpressionFactory.java  | 139 +++-
 .../gremlin/Gremlin3ExpressionFactory.java  | 184 +++--
 .../atlas/gremlin/GremlinExpressionFactory.java | 274 +--
 .../atlas/gremlin/optimizer/AliasFinder.java| 103 +++
 .../gremlin/optimizer/CallHierarchyVisitor.java |  62 ++
 .../optimizer/ExpandAndsOptimization.java   | 130 
 .../optimizer/ExpandOrsOptimization.java| 584 +++
 .../gremlin/optimizer/ExpressionFinder.java |  69 ++
 .../gremlin/optimizer/FunctionGenerator.java| 326 +
 .../gremlin/optimizer/GremlinOptimization.java  |  48 ++
 .../optimizer/GremlinQueryOptimizer.java| 262 +++
 .../gremlin/optimizer/HasForbiddenType.java |  52 ++
 .../apache/atlas/gremlin/optimizer/IsOr.java|  48 ++
 .../atlas/gremlin/optimizer/IsOrParent.java |  60 ++
 .../gremlin/optimizer/OptimizationContext.java  | 116 +++
 .../atlas/gremlin/optimizer/OrderFinder.java|  68 ++
 .../gremlin/optimizer/PathExpressionFinder.java |  61 ++
 .../atlas/gremlin/optimizer/RangeFinder.java|  68 ++
 .../gremlin/optimizer/SplitPointFinder.java | 161 +
 .../gremlin/optimizer/UpdatedExpressions.java   |  45 ++
 .../graph/GraphBackedMetadataRepository.java|  23 +-
 .../graph/GraphToTypedInstanceMapper.java   |  13 +-
 .../util/AtlasRepositoryConfiguration.java  |  19 +-
 .../org/apache/atlas/query/GremlinQuery.scala   | 103 ++-
 .../GraphBackedDiscoveryServiceTest.java|   3 +
 .../AbstractGremlinQueryOptimizerTest.java  | 705 +++
 .../graph/Gremlin2QueryOptimizerTe

[jira] [Commented] (ATLAS-1514) Duplicates in class array attribute are not removed during delete

[David Kantor (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1514?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15849081#comment-15849081
 ] 

David Kantor commented on ATLAS-1514:
-

Committed to master - 
https://github.com/apache/incubator-atlas/commit/a810f15648e764d1e15f1e810718da8eba7758f6

> Duplicates in class array attribute are not removed during delete
> -
>
> Key: ATLAS-1514
> URL: https://issues.apache.org/jira/browse/ATLAS-1514
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Reporter: David Kantor
>
> Duplicates entries in a class array attribute are not removed when the target 
> entity is deleted, leaving a dangling reference.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

RE: [5/5] incubator-atlas git commit: ATLAS-1369 - Optimize gremlin queries generated by DSL translator

[Jeffrey N Hagelberg]

FYI - This is fixed now.

-Original Message-
From: Madhan Neethiraj [mailto:mad...@apache.org]
Sent: Wednesday, February 01, 2017 5:21 PM
To: jnhagelb...@apache.org
Cc: dev@atlas.incubator.apache.org
Subject: Re: [5/5] incubator-atlas git commit: ATLAS-1369 - Optimize
gremlin queries generated by DSL translator

Jeff,

Does this patch require JDK8? Compiling with JDK7 fails with the failures
like:

[ERROR] 
/Users/mneethiraj/Apache/git/incubator-atlas/repository/src/main/java/org/apache/atlas/gremlin/optimizer/ExpressionFinder.java:20:
 error: package java.util.function does not exist
[ERROR] import java.util.function.Function;
[ERROR]  ^

[ERROR] 
/Users/mneethiraj/Apache/git/incubator-atlas/repository/src/main/java/org/apache/atlas/gremlin/optimizer/ExpressionFinder.java:32:
 error: cannot find symbol
[ERROR] private final Function
predicate;
[ERROR]   ^


Can you please review and update to make it compatible with JDK7?

Thanks,
Madhan




On 2/1/17, 1:05 PM, "jnhagelb...@apache.org" 
wrote:

ATLAS-1369 - Optimize gremlin queries generated by DSL translator


Project: http://git-wip-us.apache.org/repos/asf/incubator-atlas/repo
Commit:
http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/aa74c73d
Tree:
http://git-wip-us.apache.org/repos/asf/incubator-atlas/tree/aa74c73d
Diff:
http://git-wip-us.apache.org/repos/asf/incubator-atlas/diff/aa74c73d

Branch: refs/heads/master
Commit: aa74c73d0e9fbfd9c51de9f6cdcdea6141575e8e
Parents: f640da7
Author: Jeff Hagelberg 
Authored: Wed Feb 1 16:03:34 2017 -0500
Committer: Jeff Hagelberg 
Committed: Wed Feb 1 16:04:39 2017 -0500

--
 .../groovy/AbstractFunctionExpression.java  |  57 ++
 .../atlas/groovy/AbstractGroovyExpression.java  |   9 +
 .../atlas/groovy/ArithmeticExpression.java  |  12 +
 .../apache/atlas/groovy/BinaryExpression.java   |   9 +-
 .../org/apache/atlas/groovy/CastExpression.java |  14 +-
 .../apache/atlas/groovy/ClosureExpression.java  |  90 ++-
 .../atlas/groovy/CodeBlockExpression.java   |  61 --
 .../atlas/groovy/ComparisonExpression.java  |  12 +
 .../groovy/ComparisonOperatorExpression.java|   8 +
 .../apache/atlas/groovy/FieldExpression.java|  21 +-
 .../atlas/groovy/FunctionCallExpression.java|  88 ++-
 .../apache/atlas/groovy/GroovyExpression.java   |  42 +-
 .../atlas/groovy/IdentifierExpression.java  |  31 +
 .../apache/atlas/groovy/LabeledExpression.java  |  54 ++
 .../org/apache/atlas/groovy/ListExpression.java |  12 +
 .../apache/atlas/groovy/LiteralExpression.java  |  25 +-
 .../apache/atlas/groovy/LogicalExpression.java  |  12 +
 .../apache/atlas/groovy/RangeExpression.java|  62 +-
 .../atlas/groovy/StatementListExpression.java   |  98 +++
 .../atlas/groovy/TernaryOperatorExpression.java |  25 +-
 .../apache/atlas/groovy/TraversalStepType.java  | 121 
 .../atlas/groovy/TypeCoersionExpression.java|  19 +-
 .../groovy/VariableAssignmentExpression.java|  16 +-
 distro/src/conf/atlas-application.properties|   7 +
 graphdb/titan0/pom.xml  |   8 +
 intg/pom.xml|   1 -
 pom.xml |   8 +-
 release-log.txt |   5 +-
 .../atlas/discovery/DataSetLineageService.java  |   7 +-
 .../gremlin/Gremlin2ExpressionFactory.java  | 139 +++-
 .../gremlin/Gremlin3ExpressionFactory.java  | 184 +++--
 .../atlas/gremlin/GremlinExpressionFactory.java | 274 +--
 .../atlas/gremlin/optimizer/AliasFinder.java| 103 +++
 .../gremlin/optimizer/CallHierarchyVisitor.java |  62 ++
 .../optimizer/ExpandAndsOptimization.java   | 130 
 .../optimizer/ExpandOrsOptimization.java| 584 +++
 .../gremlin/optimizer/ExpressionFinder.java |  69 ++
 .../gremlin/optimizer/FunctionGenerator.java| 326 +
 .../gremlin/optimizer/GremlinOptimization.java  |  48 ++
 .../optimizer/GremlinQueryOptimizer.java| 262 +++
 .../gremlin/optimizer/HasForbiddenType.java |  52 ++
 .../apache/atlas/gremlin/optimizer/IsOr.java|  48 ++
 .../atlas/gremlin/optimizer/IsOrParent.java |  60 ++
 .../gremlin/optimizer/OptimizationContext.java  | 116 +++
 .../atlas/gremlin/optimizer/OrderFinder.java|  68 ++
 .../gremlin/optimizer/PathExpressionFinder.java |  61 ++
 .../atlas/gremlin/optimizer/RangeFinder.java|  68 ++
 .../gremlin/optimizer/SplitPointFinder.java | 161 +
 .../gremlin/optimizer/UpdatedExpressions.java   |  45 ++
 .../graph/GraphBackedMetadataRepository.java|  23 +-
 .../graph/GraphToTypedInstanceMapper.java   |  13 +-

Build failed in Jenkins: apache-atlas-nightly #581

[Apache Jenkins Server]

See 

Changes:

[jnhagelberg] ATLAS-1369 - Optimize gremlin queries generated by DSL translator

[madhan] ATLAS-1509: fixed issues with deletion during updates

[dkantor] ATLAS-1514 Remove duplicates from class array attribute when target is

[jnhagelberg] ATLAS-1369 - Java 7 Support

--
[...truncated 3289 lines...]
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:673)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:842)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1166)
at 
org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
at org.testng.TestRunner.runWorkers(TestRunner.java:1178)
at org.testng.TestRunner.privateRun(TestRunner.java:757)
at org.testng.TestRunner.run(TestRunner.java:608)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1158)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1083)
at org.testng.TestNG.run(TestNG.java:999)
at 
org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:115)
at 
org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.executeSingleClass(TestNGDirectoryTestSuite.java:129)
at 
org.apache.maven.surefire.testng.TestNGDirectoryTestSuite.execute(TestNGDirectoryTestSuite.java:113)
at 
org.apache.maven.surefire.testng.TestNGProvider.invoke(TestNGProvider.java:111)
at 
org.apache.maven.surefire.booter.ForkedBooter.invokeProviderInSameClassLoader(ForkedBooter.java:203)
at 
org.apache.maven.surefire.booter.ForkedBooter.runSuitesInProcess(ForkedBooter.java:155)
at 
org.apache.maven.surefire.booter.ForkedBooter.main(ForkedBooter.java:103)

testListPagination(org.apache.atlas.repository.audit.HBaseBasedAuditRepositoryTest)
  Time elapsed: 0.002 sec  <<< FAILURE!
java.lang.NullPointerException: null
at 
org.apache.atlas.repository.audit.HBaseBasedAuditRepository.putEvents(HBaseBasedAuditRepository.java:130)
at 
org.apache.atlas.repository.audit.HBaseBasedAuditRepository.putEvents(HBaseBasedAuditRepository.java:114)
at 
org.apache.atlas.repository.audit.AuditRepositoryTestBase.testListPagination(AuditRepositoryTestBase.java:64)
at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57)
at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
at java.lang.reflect.Method.invoke(Method.java:606)
at 
org.testng.internal.MethodInvocationHelper.invokeMethod(MethodInvocationHelper.java:80)
at org.testng.internal.Invoker.invokeMethod(Invoker.java:673)
at org.testng.internal.Invoker.invokeTestMethod(Invoker.java:842)
at org.testng.internal.Invoker.invokeTestMethods(Invoker.java:1166)
at 
org.testng.internal.TestMethodWorker.invokeTestMethods(TestMethodWorker.java:125)
at org.testng.internal.TestMethodWorker.run(TestMethodWorker.java:109)
at org.testng.TestRunner.runWorkers(TestRunner.java:1178)
at org.testng.TestRunner.privateRun(TestRunner.java:757)
at org.testng.TestRunner.run(TestRunner.java:608)
at org.testng.SuiteRunner.runTest(SuiteRunner.java:334)
at org.testng.SuiteRunner.runSequentially(SuiteRunner.java:329)
at org.testng.SuiteRunner.privateRun(SuiteRunner.java:291)
at org.testng.SuiteRunner.run(SuiteRunner.java:240)
at org.testng.SuiteRunnerWorker.runSuite(SuiteRunnerWorker.java:52)
at org.testng.SuiteRunnerWorker.run(SuiteRunnerWorker.java:86)
at org.testng.TestNG.runSuitesSequentially(TestNG.java:1158)
at org.testng.TestNG.runSuitesLocally(TestNG.java:1083)
at org.testng.TestNG.run(TestNG.java:999)
at 
org.apache.maven.surefire.testng.TestNGExecutor.run(TestNGExecutor.java:115)
at 
org.apache.maven.surefire.testng.TestNGDirectoryTes

UT failure: caused by guava version update in ATLAS-1369?

[Madhan Neethiraj]

Jeff,

I notice failure in following unit tests, which are most likely caused by 
change to guava version (from 18.0 => 19.0) in ATLAS-1369. Can you please look 
into this and address this failure?

Thanks,
Madhan

Failed tests:
  HBaseBasedAuditRepositoryTest.setup:40 » Runtime Master not initialized after 
...
  HBaseBasedAuditRepositoryTest>AuditRepositoryTestBase.testAddEvents:44 » 
NullPointer
  HBaseBasedAuditRepositoryTest>AuditRepositoryTestBase.testInvalidEntityId:87 
» NullPointer
  HBaseBasedAuditRepositoryTest>AuditRepositoryTestBase.testListPagination:64 » 
NullPointer

More error details from test output/logs:

2017-02-01 16:07:02,439 ERROR - [10.22.16.32:57237.activeMasterManager:] ~ 
Unhandled exception. Starting shutdown. (HMaster:2098)
java.lang.IllegalAccessError: tried to access method 
atlas.shaded.hbase.guava.common.base.Stopwatch.()V from class 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator
at 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.blockUntilAvailable(MetaTableLocator.java:596)
at 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.waitMetaRegionLocation(MetaTableLocator.java:217)
at 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.getMetaServerConnection(MetaTableLocator.java:363)
at 
org.apache.hadoop.hbase.zookeeper.MetaTableLocator.verifyMetaRegionLocation(MetaTableLocator.java:283)
at org.apache.hadoop.hbase.master.HMaster.assignMeta(HMaster.java:906)
at 
org.apache.hadoop.hbase.master.HMaster.finishActiveMasterInitialization(HMaster.java:742)

Re: Review Request 56184: ATLAS-1499: Notification processing using V2 Store

[Apoorv Naik]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56184/
---

(Updated Feb. 2, 2017, 12:57 a.m.)


Review request for atlas, Madhan Neethiraj, Sarath Subramanian, Suma 
Shivaprasad, and Vimal Sharma.


Changes
---

Updated test results.


Bugs: ATLAS-1499
https://issues.apache.org/jira/browse/ATLAS-1499


Repository: atlas


Description
---

ATLAS-1499: Notification processing using V2 Store


Diffs
-

  
repository/src/main/java/org/apache/atlas/repository/audit/EntityAuditListener.java
 1ef803c0 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasAbstractFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasArrayFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasClassificationFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasEntityFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasEnumFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasFormatConverters.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasInstanceConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasMapFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasObjectIdFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasPrimitiveFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/converters/AtlasStructFormatConverter.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/AtlasEntityStore.java
 c42f95ff 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityGraphDiscoveryV1.java
 2848a20e 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1.java
 1590aeea 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/DeleteHandlerV1.java
 08361eac 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
 31a619e5 
  server-api/src/main/java/org/apache/atlas/RequestContextV1.java 23eb4ce0 
  
webapp/src/main/java/org/apache/atlas/notification/NotificationHookConsumer.java
 f2416810 
  webapp/src/main/java/org/apache/atlas/web/rest/EntitiesREST.java 51077672 
  webapp/src/main/java/org/apache/atlas/web/rest/EntityREST.java aa209f98 
  webapp/src/test/java/org/apache/atlas/LocalAtlasClientTest.java c5616dfe 
  
webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerKafkaTest.java
 873e5625 
  
webapp/src/test/java/org/apache/atlas/notification/NotificationHookConsumerTest.java
 f06f7912 

Diff: https://reviews.apache.org/r/56184/diff/


Testing (updated)
---

mvn clean install -DskipITs executed successfully.

UI/Manual testing details

1. Manually created table in hive, showed up in graph and as an entity from the 
search UI
2. Created a view from the above table and the lineage showed up in atlas 
dashboard
3. added a column to the above table (changes could be seen on the UI)
4. altered table columns (changes couldn't be seen on the UI) **Error**


Thanks,

Apoorv Naik

[jira] [Created] (ATLAS-1517) Update hive_model to include schema related attributes

[Sarath Subramanian (JIRA)]

Sarath Subramanian created ATLAS-1517:
-

 Summary: Update hive_model to include schema related attributes
 Key: ATLAS-1517
 URL: https://issues.apache.org/jira/browse/ATLAS-1517
 Project: Atlas
  Issue Type: Improvement
  Components:  atlas-core
Affects Versions: 0.8-incubating
Reporter: Sarath Subramanian


Add schema attributes to hive_model for UI to use during schema rendering

hive_table :  options { schemaElementsAttr : "columns"}
hive_column :  options { schemaAttributes : "name, desc, version, type"}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ATLAS-1517) Update hive_model to include schema related attributes

[Sarath Subramanian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-1517:
--
Attachment: ATLAS-1507.patch

> Update hive_model to include schema related attributes
> --
>
> Key: ATLAS-1517
> URL: https://issues.apache.org/jira/browse/ATLAS-1517
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Sarath Subramanian
> Attachments: ATLAS-1507.patch
>
>
> Add schema attributes to hive_model for UI to use during schema rendering
> hive_table :  options { schemaElementsAttr : "columns"}
> hive_column :  options { schemaAttributes : "name, desc, version, type"}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ATLAS-1517) Update hive_model to include schema related attributes

[Sarath Subramanian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-1517:
--
Attachment: ATLAS-1517.1.patch

> Update hive_model to include schema related attributes
> --
>
> Key: ATLAS-1517
> URL: https://issues.apache.org/jira/browse/ATLAS-1517
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Sarath Subramanian
> Attachments: ATLAS-1517.1.patch
>
>
> Add schema attributes to hive_model for UI to use during schema rendering
> hive_table :  options { schemaElementsAttr : "columns"}
> hive_column :  options { schemaAttributes : "name, desc, version, type"}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ATLAS-1517) Update hive_model to include schema related attributes

[Sarath Subramanian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-1517:
--
Attachment: (was: ATLAS-1507.patch)

> Update hive_model to include schema related attributes
> --
>
> Key: ATLAS-1517
> URL: https://issues.apache.org/jira/browse/ATLAS-1517
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Sarath Subramanian
> Attachments: ATLAS-1517.1.patch
>
>
> Add schema attributes to hive_model for UI to use during schema rendering
> hive_table :  options { schemaElementsAttr : "columns"}
> hive_column :  options { schemaAttributes : "name, desc, version, type"}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ATLAS-1517) Update hive_model to include schema related attributes

[Sarath Subramanian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-1517:
--
Attachment: (was: ATLAS-1517.1.patch)

> Update hive_model to include schema related attributes
> --
>
> Key: ATLAS-1517
> URL: https://issues.apache.org/jira/browse/ATLAS-1517
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Sarath Subramanian
> Attachments: ATLAS-1517.3.patch
>
>
> Add schema attributes to hive_model for UI to use during schema rendering
> hive_table :  options { schemaElementsAttr : "columns"}
> hive_column :  options { schemaAttributes : "name, desc, version, type"}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ATLAS-1517) Update hive_model to include schema related attributes

[Sarath Subramanian (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Sarath Subramanian updated ATLAS-1517:
--
Attachment: ATLAS-1517.3.patch

> Update hive_model to include schema related attributes
> --
>
> Key: ATLAS-1517
> URL: https://issues.apache.org/jira/browse/ATLAS-1517
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Sarath Subramanian
> Attachments: ATLAS-1517.3.patch
>
>
> Add schema attributes to hive_model for UI to use during schema rendering
> hive_table :  options { schemaElementsAttr : "columns"}
> hive_column :  options { schemaAttributes : "name, desc, version, type"}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ATLAS-1517) Update hive_model to include schema related attributes

[Madhan Neethiraj (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1517?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15849347#comment-15849347
 ] 

Madhan Neethiraj commented on ATLAS-1517:
-

+1 for the patch

> Update hive_model to include schema related attributes
> --
>
> Key: ATLAS-1517
> URL: https://issues.apache.org/jira/browse/ATLAS-1517
> Project: Atlas
>  Issue Type: Improvement
>  Components:  atlas-core
>Affects Versions: 0.8-incubating
>Reporter: Sarath Subramanian
> Attachments: ATLAS-1517.3.patch
>
>
> Add schema attributes to hive_model for UI to use during schema rendering
> hive_table :  options { schemaElementsAttr : "columns"}
> hive_column :  options { schemaAttributes : "name, desc, version, type"}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Apache Atlas 0.8 (incubating) release preparations

[Madhan Neethiraj]

Atlas team,

 

It has been about 7 months since we released Apache Atlas 0.7 (incubating) 
version.

We added several features/fixes in master branch during this time, including:

 

1. a new REST API with structured/well-defined input and output data 
structures. This would make it easier for applications to integrate with Apache 
Atlas

2. abstraction layer to access graph database, enabling Atlas to work with 
different database versions

3. responsive UI with less chatty calls with the server side

4. UI to create HBase/Kafka/HDFS entity instances, which enables tag-based 
authorization for these components

5. updated type-model with constructs like foreignKey, mappedFromRef, 
onDelete-cascade/update, etc.

6. simplified approach to updates types – using patch files

7. centralized error code catalog

8. performance improvements in entity create/update

9. updated QuickStart to demonstrate use of new REST APIs

 

I propose we make the next release of Apache Atlas – 0.8 (incubating). How 
about we plan to release by 2/24? This should give us enough time to close on 
various dev items currently in progress and go through the voting process.

 

Please share your comments and suggestions.

 

Thanks,

Madhan

 

 

 

[jira] [Created] (ATLAS-1518) Bugzilla

[Christopher Blackwell (JIRA)]

Christopher Blackwell created ATLAS-1518:


 Summary: Bugzilla
 Key: ATLAS-1518
 URL: https://issues.apache.org/jira/browse/ATLAS-1518
 Project: Atlas
  Issue Type: Improvement
Reporter: Christopher Blackwell
Priority: Trivial
 Attachments: 500 -.txt





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (ATLAS-1519) Bugzilla

[Christopher Blackwell (JIRA)]

Christopher Blackwell created ATLAS-1519:


 Summary: Bugzilla
 Key: ATLAS-1519
 URL: https://issues.apache.org/jira/browse/ATLAS-1519
 Project: Atlas
  Issue Type: Improvement
Reporter: Christopher Blackwell
Priority: Trivial
 Attachments: 500 -.txt





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (ATLAS-1520) Bugzilla

[Christopher Blackwell (JIRA)]

Christopher Blackwell created ATLAS-1520:


 Summary: Bugzilla
 Key: ATLAS-1520
 URL: https://issues.apache.org/jira/browse/ATLAS-1520
 Project: Atlas
  Issue Type: Improvement
Reporter: Christopher Blackwell
Priority: Trivial
 Attachments: 500 -.txt





--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Build failed in Jenkins: apache-atlas-nightly #582

[Apache Jenkins Server]

See 

Changes:

[madhan] Revert "ATLAS-1369 - Java 7 Support"

[madhan] Revert "ATLAS-1369 - Optimize gremlin queries generated by DSL

[madhan] ATLAS-1517: updated hive_model to include schema related attributes

--
[...truncated 5620 lines...]
127.0.0.1 - - [02/Feb/2017:05:25:19 +] "GET 
/api/atlas/lineage/hive/table/sales_fact_monthly_mvCJi99VcWyE/inputs/graph 
HTTP/1.1" 500 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:19 +] "GET 
/api/atlas/entities?type=hive_table_v1&property=qualifiedName&value=sales_fact_monthly_mvCJi99VcWyE
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:19 +] "GET 
/api/atlas/lineage/6d265e24-d93d-41f3-b5a7-28f7e396ae63/inputs/graph HTTP/1.1" 
200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:20 +] "GET 
/api/atlas/lineage/hive/table/sales_factZ7GVo34O5S/outputs/graph HTTP/1.1" 500 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:20 +] "GET 
/api/atlas/entities?type=hive_table_v1&property=qualifiedName&value=sales_factZ7GVo34O5S
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:20 +] "GET 
/api/atlas/lineage/fa44e749-5c31-4c9c-82a3-5d9b74fb0c98/outputs/graph HTTP/1.1" 
200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:21 +] "GET 
/api/atlas/lineage/hive/table/sales_factZ7GVo34O5S/schema HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:22 +] "GET 
/api/atlas/lineage/hive/table/SalesMCwguzWLaZ/schema HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:22 +] "GET 
/api/atlas/entities?type=hive_table_v1&property=qualifiedName&value=sales_factZ7GVo34O5S
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:22 +] "GET 
/api/atlas/lineage/fa44e749-5c31-4c9c-82a3-5d9b74fb0c98/schema HTTP/1.1" 200 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:23 +] "GET 
/api/atlas/lineage/hive/table/blah/schema HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:23 +] "GET 
/api/atlas/entities?type=hive_table_v1&property=qualifiedName&value=sales_fact_monthly_mvbZWwbMmt5p
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:23 +] "GET 
/api/atlas/v2/lineage/3fac9104-0124-4aa7-aa3b-2857d4e85ba5?direction=INPUT&depth=5
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:24 +] "GET 
/api/atlas/entities?type=hive_table_v1&property=qualifiedName&value=sales_fact_monthly_mvbZWwbMmt5p
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:24 +] "GET 
/api/atlas/v2/lineage/3fac9104-0124-4aa7-aa3b-2857d4e85ba5?direction=BOTH&depth=5
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:26 +] "GET 
/api/atlas/entities?type=hive_table_v1&property=qualifiedName&value=sales_factVC8BR9ugrH
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:26 +] "GET 
/api/atlas/v2/lineage/48e01f2f-4909-4557-a015-0d65cff1a549?direction=OUTPUT&depth=5
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
Tests run: 11, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 14.046 sec <<< 
FAILURE! - in org.apache.atlas.web.resources.EntityLineageJerseyResourceIT
testInputsGraph(org.apache.atlas.web.resources.EntityLineageJerseyResourceIT)  
Time elapsed: 0.031 sec  <<< FAILURE!
org.apache.atlas.AtlasServiceException: Metadata service API 
org.apache.atlas.AtlasBaseClient$APIInfo@63a5022f failed with status 500 
(Internal Server Error) Response Body ({"error":"Failed with 
java.lang.NullPointerException"})

testOutputsGraph(org.apache.atlas.web.resources.EntityLineageJerseyResourceIT)  
Time elapsed: 0.014 sec  <<< FAILURE!
org.apache.atlas.AtlasServiceException: Metadata service API 
org.apache.atlas.AtlasBaseClient$APIInfo@73a75a3d failed with status 500 
(Internal Server Error) Response Body ({"error":"Failed with 
java.lang.NullPointerException"})

Running org.apache.atlas.CredentialProviderUtilityIT
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.806 sec - in 
org.apache.atlas.CredentialProviderUtilityIT
Running org.apache.atlas.notification.EntityNotificationIT
127.0.0.1 - - [02/Feb/2017:05:25:34 +] "GET /api/atlas/types/tableType 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:34 +] "GET /api/atlas/types/serdeType 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:34 +] "GET /api/atlas/types/classification 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:34 +] "GET /api/atlas/types/hive_db_v1 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:35 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:36 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [02/Feb/2017:05:25:37 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - 

Re: Review Request 56102: ATLAS-1388 : Cache entities that are created/updated

[Vimal Sharma]

> On Feb. 1, 2017, 10:42 a.m., Vimal Sharma wrote:
> > The cache size can grow large if the number of registered entities is high. 
> > It would be good to have a cache eviction policy just like it was done for 
> > DSL Query caching in ATLAS-1387
> 
> Jeff Hagelberg wrote:
> We could do that.  It seems like overkill though.  The big difference is 
> that RequestContext is thread local, and a new RequestContext is created 
> every time a http request comes in.  Once the http request finishes, 
> RequestContext.clear() is called, and the cache goes out of scope and will be 
> garbage collected.  (See AuditFilter for details).  The cache is limited to 
> the instances used for processsing the request.  In the case of the compiled 
> query cache, there is one effectively static instance of the cache that is 
> used for all requests.  Without the eviction policy, that would continue 
> growing until the JVM runs out of memory.
> 
> Jeff Hagelberg wrote:
> Is there a specific use case you are concerned about?  For example, if 
> someone runs a DSL query that needs to return 100,000 entities, those 
> entities would be cached.  However, even without the cache we would still 
> need to have the 100,000 entities in memory so they can be serialized to json 
> and sent back to the client.  I'm not sure that the cache increases the 
> amount of memory required by much.  What it does do, though, is keep the 
> instances that are created in the java heap for a longer period of time.
> 
> David Kantor wrote:
> There no need for a cache eviction policy as this cache is only active 
> for the duration of the request.   Perhaps to address the comment, the cache 
> could be cleared upon completion of the request to make the entries available 
> for garbage collection sooner.
> 
> Jeff Hagelberg wrote:
> Sure, I've changed it so that we explicitly clear the cache.

Got it. I didn't observe that the scope of cache is limited to a single request 
context. The current implementation looks good.


- Vimal


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56102/#review163800
---


On Feb. 1, 2017, 9:41 p.m., Jeff Hagelberg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56102/
> ---
> 
> (Updated Feb. 1, 2017, 9:41 p.m.)
> 
> 
> Review request for atlas and David Kantor.
> 
> 
> Bugs: ATLAS-1388
> https://issues.apache.org/jira/browse/ATLAS-1388
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Cache created entities in RequestContext when they are created.
> Update/refactor DefaultMetadataService.loadEntities() to use the cached 
> created entities.
> Use the cache in the following places:
> FullTextMapper
> DefaultMetadataService.onEntitiesAdded - check cache before calling 
> DefaultMetadataService.loadEntities
> DefaultMetadataService.onEntitiesUpdated - check cache before calling 
> DefaultMetadataService.loadEntities
> EntityResource.getEntityDefinition - check cache before calling 
> DefaultMetadataService.getEntityDefinition
> EntityResource.getResponse - check cache before calling 
> DefaultMetadataService.getEntityDefinition
> 
> 
> Diffs
> -
> 
>   
> repository/src/main/java/org/apache/atlas/discovery/graph/DefaultGraphPersistenceStrategy.java
>  f4d8f00da863deeb98209aec2dfc4ccb22734cfa 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapper.java
>  5be8d0bc349294dc4340ffe5831bfb1265825ed8 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
>  6608551be44033f07ddbd2ac3f3764d1ac3b0f22 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java
>  38a553abf0151221c5d802e9ce1977a693d1a698 
>   
> repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
>  bae8b2ac0cae6bf6392f91f24030d81089ae660b 
>   repository/src/test/java/org/apache/atlas/TestUtils.java 
> cda9eac38861fd55c1494d6d90b5e750e6133545 
>   
> repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java
>  d447c2d90fec6c03c7fecb8d8ef57a6ea072d35a 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
>  9e850a9dc0ac122cf3fcdbf7df72a93643abc98d 
>   
> repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java
>  7444bf38834f59454d67cf763c932d665d7ef31c 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityStoreV1Test.java
>  c902f8126f6b516e84ab9d89b0ec80ac8ed819fd 
>   
> repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTe

[jira] [Created] (ATLAS-1521) Update UI to eliminate REST calls to obtain schema data

[Madhan Neethiraj (JIRA)]

Madhan Neethiraj created ATLAS-1521:
---

 Summary: Update UI to eliminate REST calls to obtain schema data
 Key: ATLAS-1521
 URL: https://issues.apache.org/jira/browse/ATLAS-1521
 Project: Atlas
  Issue Type: Improvement
  Components: atlas-webui
Affects Versions: 0.8-incubating
Reporter: Madhan Neethiraj


When rendering entity details page, UI makes a REST API call to /schema/{guid} 
to obtain the schema data. For entity types that don't support the notion of 
schema, this REST API call returns an error. When such error is returned, UI 
does not add 'Schema' tab to the details page. When the call returns data, UI 
renders the data in 'Schema' tab.

Given that only few entity-types (like hive_table) support schema, most calls 
to retrieve schema return an error. Having additional schema related 
information in the type can eliminate the need for unnecessary REST calls. 
ATLAS-1517 introduces additional data in 'typeDefOptions' attribute of 
entity-type definition. UI can use the following logic to render the schema tab 
contents:

# Does an entity-type support schema?
  Is {{entityDef.typeDefOptions\[schemaElementsAttribute]}} set and its value 
non empty? If yes, schema is supported else not-supported. For hive_table type, 
this will be set to 'columns' - as shown below:
{code}
  "typeDefOptions": {
"schemaElementsAttribute": "columns"
  }
{code}

# How does UI get the data to display in schema tab?
  Value of {{entityDef.typeDefOptions\[schemaElementsAttribute]}} will be the 
name of the attribute in the entity being rendered. This attribute should be of 
type array. The value of this attribute will have the data needed to render the 
schema tab

# What are the columns to display in schema tab?
  Each element of the array attribute value will hold an entity/struct 
instance.For example, In case of hive_table, columns attribute will have an 
array of hive_column objects. From each such element, UI should get value of 
attributes listed in {{elementDef.typeDefOptions\[schemaAttributes]}} to render 
the schema table. For example, hive_column will have the following:

{code}
  "typeDefOptions": {
"schemaAttributes": "[\"name\", \"description\", \"owner\", \"type\", 
\"comment\", \"position\"]"
  }
{code}

The value will be a json-ified string array.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)