[jira] [Updated] (ATLAS-1008) Change log message in HiveMetaStoreBridge

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-1008:

Fix Version/s: 0.8-incubating

> Change log message in HiveMetaStoreBridge
> -
>
> Key: ATLAS-1008
> URL: https://issues.apache.org/jira/browse/ATLAS-1008
> Project: Atlas
>  Issue Type: Bug
>Reporter: Vimal Sharma
>Assignee: Vimal Sharma
>Priority: Trivial
> Fix For: 0.8-incubating
>
>
> LOG.error("Unable to import {} tables out of {} tables from {}", 
> tablesImported, hiveTables.size(), databaseName);
> Unable to import 18 tables out of 19 tables from default 
> (HiveMetaStoreBridge:318)
> No of tables successfully imported are print instead of the number of tables 
> for which import failed



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66064: ATLAS-2478 Elasticsearch support broken with JanusGraph 0.2.0

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66064/#review203349
---


Ship it!




Ship It!

- Madhan Neethiraj


On May 17, 2018, 2:39 p.m., Pierre Padovani wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66064/
> ---
> 
> (Updated May 17, 2018, 2:39 p.m.)
> 
> 
> Review request for atlas, David Radley, Madhan Neethiraj, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-2478
> https://issues.apache.org/jira/browse/ATLAS-2478
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch fixes the Elasticsearch support for JanusGraph 0.2.0 and updates 
> documentation.
> 
> Included with this patch is an update to the berkley-elasticsearch profile to 
> automatically download and include elasticsearch as a side application much 
> like solr is. Updates to the start/stop/conf scripts are included as well.
> 
> NOTE: This patch includes a **BACKWARDS INCOMPATIBLE** change to 
> /atlas/common/src/main/java/org/apache/atlas/repository/Constants.java. There 
> are six constants that are incorrectly named with a '.' (dot). This is not 
> supported in Elasticsearch 5 and beyond when defining a mapping **UNLESS** 
> the field names can be collectively thought of as an object. In the case of 
> the fields defined in the Constants.java file, 'type' is defined as a string 
> field, and 'type.name' is also defined as a string field. Elasticsearch sees 
> this as an error, since it cannot convert type to an object. The fix included 
> simply changes the field names from using a '.' (dot) to an '_' (underscore). 
> This should NOT affect compatibility with hbase/solr for new installs. For 
> existing installations, a reindex will be required as the field names will 
> have changed.
> 
> **Query**: There is a way we can simplify integration/unit tests for the 
> in-memory graph store by using a maven plugin that will download and run an 
> elasticsearch node. This is nothing more than a maven change, and change to 
> the atlas-application.properties to switch to elasticsearch from solr. I did 
> not implement this, but am curious if this change would be desired. If so, 
> this can be done with a separate ticket.
> 
> 
> Diffs
> -
> 
>   common/src/main/java/org/apache/atlas/repository/Constants.java 3732556ae 
>   distro/pom.xml 379ff0d39 
>   distro/src/bin/atlas_config.py 9062da649 
>   distro/src/bin/atlas_start.py 61d69eb21 
>   distro/src/bin/atlas_stop.py 94c3d6d46 
>   distro/src/conf/atlas-env.sh 053cbd500 
>   distro/src/main/assemblies/standalone-package.xml dc88add56 
>   docs/src/site/twiki/Configuration.twiki 63c3fce96 
>   docs/src/site/twiki/HighAvailability.twiki 4270d0974 
>   docs/src/site/twiki/InstallationSteps.twiki dca0618e3 
>   graphdb/janus/pom.xml 016a09c33 
>   pom.xml 3469a393a 
>   webapp/pom.xml 03b84087f 
> 
> 
> Diff: https://reviews.apache.org/r/66064/diff/6/
> 
> 
> Testing
> ---
> 
> We currently use this fix with our Atlas setup in a fork of the Atlas code, 
> and have found no issues with it. Additionally, with the update to the 
> berkley-elasticsearch profile, I have extensively tested that profile to make 
> sure that management of Elasticsearch functioned correctly.
> 
> 
> Thanks,
> 
> Pierre Padovani
> 
>

[jira] [Updated] (ATLAS-2478) Elasticsearch support (Tech Preview) is broken for JanusGraph

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-2478:

Summary: Elasticsearch support (Tech Preview) is broken for JanusGraph  
(was: Elasticsearch support is broken for JanusGraph)

> Elasticsearch support (Tech Preview) is broken for JanusGraph
> -
>
> Key: ATLAS-2478
> URL: https://issues.apache.org/jira/browse/ATLAS-2478
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.0.0-alpha
>Reporter: Pierre Padovani
>Assignee: Pierre Padovani
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: ATLAS-2478.patch
>
>
> With JanusGraph the Elasticsearch support moved to 5.x+. This introduced a 
> change where fields that contained '.' (dots) were not allowed unless either 
> a specific cluster wide setting was enabled AND the mapping was formatted 
> such that each of the fields that contained a '.' could be considered part of 
> an object.
> Example:
> {code:java}
> foo.x
> foo.y
> foo.z{code}
>  Elasticsearch looks at these fields as if they are truly:
> {code:java}
> foo : {
>   x,
>   y,
>   z
> }{code}
> In the file:
> /atlas/common/src/main/java/org/apache/atlas/repository/Constants.java
> {code:java}
> /**
>  * Properties for type store graph.
>  */
> public static final String TYPE_CATEGORY_PROPERTY_KEY = 
> INTERNAL_PROPERTY_KEY_PREFIX + "type.category";
> public static final String VERTEX_TYPE_PROPERTY_KEY = 
> INTERNAL_PROPERTY_KEY_PREFIX + "type";
> public static final String TYPENAME_PROPERTY_KEY = 
> INTERNAL_PROPERTY_KEY_PREFIX + "type.name";
> public static final String TYPEDESCRIPTION_PROPERTY_KEY = 
> INTERNAL_PROPERTY_KEY_PREFIX + "type.description";
> public static final String TYPEVERSION_PROPERTY_KEY = 
> INTERNAL_PROPERTY_KEY_PREFIX + "type.version";
> public static final String TYPEOPTIONS_PROPERTY_KEY = 
> INTERNAL_PROPERTY_KEY_PREFIX + "type.options";
> {code}
> These are the only fields that cause Elasticsearch issue. As you can see a 
> field called 'type' is created, then additional fields type.name, 
> type.description etc. This will cause a mapping conflict exception in 
> Elasticsearch and it will refuse to create the mapping.
>  
> The easy fix is to simply replace the '.' with an '_' (underscore) but this 
> will be a backwards incompatible change for existing customers. 



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 66928: Data Migration: Import: Infer Types that Store Edge Ids

[Sarath Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66928/#review203360
---




repository/src/main/java/org/apache/atlas/repository/migration/TypesWithCollectionsFinder.java
Lines 66 (patched)


getProperties() method in 66 can be replaced with one in line 82. 
AtlasStructDef is supertype of AtlasEntityDef.



repository/src/main/java/org/apache/atlas/repository/migration/TypesWithCollectionsFinder.java
Lines 109 (patched)


rename 109 and 113 with getArrayElementType and getMapValueType for better 
readability.



repository/src/main/java/org/apache/atlas/repository/migration/TypesWithCollectionsFinder.java
Lines 117 (patched)


update() and updatePrimitive() looks duplicate of each other with only 
"_PRIMTITIVE" string difference, can be refactored into a single method.


- Sarath Subramanian


On May 15, 2018, 10:58 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66928/
> ---
> 
> (Updated May 15, 2018, 10:58 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Ruchi Solani, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-2637
> https://issues.apache.org/jira/browse/ATLAS-2637
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Approach**
> New class: _TypesStoringEdgeIds_: Navigates through all the types in the 
> _typeRegistry_, returns map of entity type and properties that store edge ids.
> Modified: _DataMigrationService_: Uses output from class above and passes it 
> down to migration.
> Modified: _PostProcessManager_: Uses the map generated above and uses it for 
> post processing.
> Modified: _GraphSONUtility_: Improvement to check for vertex of type. This 
> avoids potential exeception when a non-existent property is checked for 
> presence.
> 
> Added PostProcess framework.
> Added logic for handling new Array and Map representation.
> 
> 
> Diffs
> -
> 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraph.java 
> 607baf664 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java
>  c0b9c1741 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphDatabase.java
>  16aecd5e2 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/AtlasGraphSONReader.java
>  ae119b0bc 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/GraphSONUtility.java
>  ec320b03e 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/PostProcessManager.java
>  d0a65f7b1 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/RelationshipTypeCache.java
>  e4e82649b 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/postProcess/PostProcessListProperty.java
>  PRE-CREATION 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/postProcess/PostProcessor.java
>  PRE-CREATION 
>   
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/BaseUtils.java
>  e863d9fae 
>   
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/GraphSONUtilityPostProcessTest.java
>  4d73c78ef 
>   
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/GraphSONUtilityTest.java
>  794b5471e 
>   
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/MappedElementCacheTest.java
>  cac09d229 
>   
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/PostProcessListPropertyTest.java
>  PRE-CREATION 
>   graphdb/janus/src/test/resources/col-2-legacy.json PRE-CREATION 
>   graphdb/janus/src/test/resources/edge-legacy-col.json PRE-CREATION 
>   graphdb/janus/src/test/resources/edge-legacy-col2.json PRE-CREATION 
>   graphdb/janus/src/test/resources/edge-legacy-col3.json PRE-CREATION 
>   graphdb/janus/src/test/resources/edge-legacy-col4.json PRE-CREATION 
>   graphdb/janus/src/test/resources/table-v-147504.json 898dce5df 
>   intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java b05754f4b 
>   intg/src/test/java/org/apache/atlas/TestUtilsV2.java 886ce77f5 
>   
> repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationService.java
>  22cd55217 
>   
> repository/src/main/java/org/apache/atlas/repository/migration/TypesWithCollectionsFinder.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasTypeDefGraphStoreV1.j

[jira] [Resolved] (ATLAS-2517) denied audit log is missing for read entity operation

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2517?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-2517.
-
   Resolution: Cannot Reproduce
Fix Version/s: (was: 1.0.0)

> denied audit log is missing for read entity operation
> -
>
> Key: ATLAS-2517
> URL: https://issues.apache.org/jira/browse/ATLAS-2517
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 1.0.0
>Reporter: Deepak Sharma
>Priority: Major
>
> Scenario:
> 1) remove read entity permission for an user.
> 2) through that user try to read the entity
> Result:
> read operation fails due to "admin is not authorized to perform read entity"
> but denied audit log is missing for this operation



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Re: Review Request 67085: ATLAS-2673: Decode query string for DSL search

[Madhan Neethiraj]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/67085/#review203367
---


Ship it!




Ship It!

- Madhan Neethiraj


On May 16, 2018, 10:04 a.m., Nixon Rodrigues wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/67085/
> ---
> 
> (Updated May 16, 2018, 10:04 a.m.)
> 
> 
> Review request for atlas, Apoorv Naik, Ashutosh Mestry, Madhan Neethiraj, and 
> Sarath Subramanian.
> 
> 
> Bugs: ATLAS-2673
> https://issues.apache.org/jira/browse/ATLAS-2673
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This patch handles decoding of encoded query string for DSL search.
> 
> Use case:-
> 
> DSL query :
> 
> /api/atlas/v2/search/dsl?typeName=hive_table&query=name="weather2"
> fired on PASSIVE server redirects to ACTIVE server as
> 
> /api/atlas/v2/search/dsl?limit=25&offset=0&query=name=%2522weather2%2522&typeName=hive_table
> fails with
> 
> {
> errorCode: "ATLAS-400-00-059",
> errorMessage: "Invalid DSL query: `hive_table` name=%22weather2%22 | Reason: 
> DSL Semantic Error - weather2 type not found, DSL Semantic Error - weather2 
> type not found, DSL Semantic Error - 22 type not found, DSL Semantic Error - 
> 22 type not found. Please refer to Atlas DSL grammar for more information"
> }
> 
> 
> Diffs
> -
> 
>   webapp/src/main/java/org/apache/atlas/examples/QuickStartV2.java 0ca0ba748 
>   webapp/src/main/java/org/apache/atlas/web/rest/DiscoveryREST.java ad3785935 
>   webapp/src/main/java/org/apache/atlas/web/util/Servlets.java ec340cfee 
> 
> 
> Diff: https://reviews.apache.org/r/67085/diff/2/
> 
> 
> Testing
> ---
> 
> Tested by accessing URL 
> /api/atlas/v2/search/dsl?typeName=Table&query=name="sales_fact" on passive 
> which redirects on active node now working properly.
> 
> And also on Active node.
> 
> Added few encode queryparam in ITs
> 
> Integration testcase working
> https://builds.apache.org/job/PreCommit-ATLAS-Build-Test/388/console
> 
> 2018-05-11 10:39:16,828  
> 2018-05-11T10:39:16.566Z|admin|127.0.0.1|GET|http://localhost:31000/api/atlas/v2/search/dsl?offset=0&query=DB+where+name%3D%22Reporting%22&limit=10|200|262
> 2018-05-11 10:39:17,089  
> 2018-05-11T10:39:16.832Z|admin|127.0.0.1|GET|http://localhost:31000/api/atlas/v2/search/dsl?offset=0&query=DB+where+name%3D%22encode_db_name%22&limit=10|200|257
> 2018-05-11 10:39:17,351  
> 2018-05-11T10:39:17.093Z|admin|127.0.0.1|GET|http://localhost:31000/api/atlas/v2/search/dsl?offset=0&query=Table+where+name%3D%2522sales_fact%2522&limit=10|200|258
> 
> 
> Thanks,
> 
> Nixon Rodrigues
> 
>

Re: Review Request 66928: Data Migration: Import: Infer Types that Store Edge Ids

[Ashutosh Mestry]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66928/
---

(Updated May 17, 2018, 8:48 p.m.)


Review request for atlas, Madhan Neethiraj, Ruchi Solani, and Sarath 
Subramanian.


Changes
---

Updates include: Addressed review comments.


Bugs: ATLAS-2637
https://issues.apache.org/jira/browse/ATLAS-2637


Repository: atlas


Description
---

**Approach**
New class: _TypesStoringEdgeIds_: Navigates through all the types in the 
_typeRegistry_, returns map of entity type and properties that store edge ids.
Modified: _DataMigrationService_: Uses output from class above and passes it 
down to migration.
Modified: _PostProcessManager_: Uses the map generated above and uses it for 
post processing.
Modified: _GraphSONUtility_: Improvement to check for vertex of type. This 
avoids potential exeception when a non-existent property is checked for 
presence.

Added PostProcess framework.
Added logic for handling new Array and Map representation.


Diffs (updated)
-

  graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraph.java 
607baf664 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java
 c0b9c1741 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphDatabase.java
 16aecd5e2 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/AtlasGraphSONReader.java
 ae119b0bc 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/GraphSONUtility.java
 ec320b03e 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/PostProcessManager.java
 d0a65f7b1 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/RelationshipTypeCache.java
 e4e82649b 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/postProcess/PostProcessListProperty.java
 PRE-CREATION 
  
graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/postProcess/PostProcessor.java
 PRE-CREATION 
  
graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/BaseUtils.java
 e863d9fae 
  
graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/GraphSONUtilityPostProcessTest.java
 4d73c78ef 
  
graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/GraphSONUtilityTest.java
 794b5471e 
  
graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/MappedElementCacheTest.java
 cac09d229 
  
graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/PostProcessListPropertyTest.java
 PRE-CREATION 
  graphdb/janus/src/test/resources/col-2-legacy.json PRE-CREATION 
  graphdb/janus/src/test/resources/edge-legacy-col.json PRE-CREATION 
  graphdb/janus/src/test/resources/edge-legacy-col2.json PRE-CREATION 
  graphdb/janus/src/test/resources/edge-legacy-col3.json PRE-CREATION 
  graphdb/janus/src/test/resources/edge-legacy-col4.json PRE-CREATION 
  graphdb/janus/src/test/resources/table-v-147504.json 898dce5df 
  intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java b05754f4b 
  intg/src/test/java/org/apache/atlas/TestUtilsV2.java 886ce77f5 
  
repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationService.java
 22cd55217 
  
repository/src/main/java/org/apache/atlas/repository/migration/TypesWithCollectionsFinder.java
 PRE-CREATION 
  
repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasTypeDefGraphStoreV1.java
 3c84e3c22 
  
repository/src/test/java/org/apache/atlas/repository/migration/ComplexAttributesTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/repository/migration/HiveParititionTest.java
 ac0b79d38 
  
repository/src/test/java/org/apache/atlas/repository/migration/HiveStocksTest.java
 ffbf3200b 
  
repository/src/test/java/org/apache/atlas/repository/migration/MigrationBaseAsserts.java
 ec6e64a25 
  
repository/src/test/java/org/apache/atlas/repository/migration/TypesWithCollectionsFinderTest.java
 PRE-CREATION 
  
repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityTestBase.java
 d810a1166 
  repository/src/test/resources/complex-attr_db/atlas-migration-data.json 
PRE-CREATION 
  repository/src/test/resources/complex-attr_db/atlas-migration-typesdef.json 
PRE-CREATION 
  repository/src/test/resources/parts_db/atlas-migration-data.json 1414ea160 


Diff: https://reviews.apache.org/r/66928/diff/15/

Changes: https://reviews.apache.org/r/66928/diff/14-15/


Testing
---

**Unit tests**

Additional tests added.

**Functional tests**

Regular flow verified.

**[Pre-commit 
build](https://builds.apache.org/view/A/view/Atlas/job/PreCommit-ATLAS-Build-Test/416/)**


Thanks,

Ashutosh Mestry

[jira] [Comment Edited] (ATLAS-2139) Map IGC meta model and Atlas OMRS open metadata types

[V Seth (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-2139?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16466907#comment-16466907
 ] 

V Seth edited comment on ATLAS-2139 at 5/17/18 9:54 PM:


Thanks [~davidrad] [~grahamwallis] for the response.

[~mandy_chessell], here are the mappings from IGC to OMRS per your response:

1 IGC database maps to DeployedDatabaseSchema entity.  ServerDeployment 
relationship ties the DeployedDatabaseSchema entity with Host entity.

2a IGC schema is a DataSet entity in OMRS.  Database thru DataContentForDataSet 
relationship contains the schema.
 2b For IGC schema a second entity and relationship will have to be created.  
RelationalDBSchemaType entity and it links to DataSet(schema) via 
AssetSchemaType relationship.  This second entity identifies
    the DataSet as a schema.

3a IGC table is a DataSet entity in OMRS.  Dataset(Schema) thru 
DataContentForDataSet relationship contains the table.
 3b For IGC table a second entity and relationship will have to be created - 
RelationalTableType entity and it links to DataSet(table) via AssetSchemaType 
relationship.  This second entity identifies the DataSet as a table.
 3c For IGC table a third entity and relationship will have to be created - 
RelationalTable entity and it links to RelationalTableType entity via 
AttributeForSchema relationship

4a IGC column is an Asset entity in OMRS. DataSet(Table) thru 
DataContentForDataSet relationship contains the column.
 4b For IGC column a second entity and relationship will have to be created - 
RelationalColumnType entity and it links to Asset(column) via AssetSchemaType 
relationship.  This second entity identified the Asset as a column.
 4c For IGC column a third entity and relationship will have to be created - 
RelationalColumn entity and it links to RelationalColumnType entity via 
SchemaAttributeType relationship.

Here are few questions I have:
 1 Would Asset qualifiedName be the right place to put schema name, table name, 
column name.  i guess i can duplicate that name in SchemaType and 
SchemaAttribute because there is nothing in IGC which differentiates.

2 Does the above mapping look correct to you?

3 I am concerned about the number of messages that will be generated to create 
an entity.  For a single schema entity, two entities and two relationships.  
For a single table or column three entities and three relationships.  Any 
thoughts on this?

Thanks in advance.


was (Author: sethv):
Thanks [~davidrad] [~grahamwallis] for the response.

[~mandy_chessell], here are the mappings from IGC to OMRS per your response:

1 IGC database maps to DeployedDatabaseSchema entity.  ServerDeployment 
relationship ties the DeployedDatabaseSchema entity with Host entity.

2a IGC schema is a DataSet entity in OMRS.  Database thru DataContentForDataSet 
relationship contains the schema.
2b For IGC schema a second entity and relationship will have to be created.  
RelationalDBSchemaType entity and it links to DataSet(schema) via 
AssetSchemaType relationship.  This second entity identifies
   the DataSet as a schema.

3a IGC table is a DataSet entity in OMRS.  Dataset(Schema) thru 
DataContentForDataSet relationship contains the table.
3b For IGC table a second entity and relationship will have to be created - 
RelationalTableType entity and it links to DataSet(table) via AssetSchemaType 
relationship.  This second entity identifies the DataSet as a table.
3c For IGC table a third entity and relationship will have to be created - 
RelationalTable entity and it links to RelationalTableType entity via 
AttributeForSchema relationship

4a IGC column is a DataSet entity in OMRS. DataSet(Table) thru 
DataContentForDataSet relationship contains the column.
4b For IGC column a second entity and relationship will have to be created - 
RelationalColumnType entity and it links to DataSet(column) via AssetSchemaType 
relationship.  This second entity identified the DataSet as a column.
4c For IGC column a third entity and relationship will have to be created - 
RelationalColumn entity and it links to RelationalColumnType entity via 
SchemaAttributeType relationship.


Here are few questions I have:
1 Would Asset qualifiedName be the right place to put schema name, table name, 
column name.  i guess i can duplicate that name in SchemaType and 
SchemaAttribute because there is nothing in IGC which differentiates.


2 Does the above mapping look correct to you?


3 I am concerned about the number of messages that will be generated to create 
an entity.  For a single schema entity, two entities and two relationships.  
For a single table or column three entities and three relationships.  Any 
thoughts on this?

Thanks in advance.

> Map IGC meta model and Atlas OMRS open metadata types
> -
>
> Key: ATLAS-2139
> URL: https:

[jira] [Commented] (ATLAS-1968) Export/Import - Exception thrown when Import fired with zip file found on server

[Ashutosh Mestry (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16479841#comment-16479841
 ] 

Ashutosh Mestry commented on ATLAS-1968:


Commit: 3d3be4084

> Export/Import -  Exception thrown when Import fired with zip file found on 
> server
> -
>
> Key: ATLAS-1968
> URL: https://issues.apache.org/jira/browse/ATLAS-1968
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8.1, 1.0.0
>Reporter: Sharmadha Sainath
>Assignee: Ashutosh Mestry
>Priority: Blocker
> Attachments: ATLAS-1968-Importfile-option.patch, 
> ImportExceptionWhenZipFileOnServer.txt
>
>
> 1. Exported an entity on the cluster1 ,created zip file and placed it in /tmp 
> location of cluster2 .
> 2. Fired the following import command against cluster2 :
> {code}
> curl -v -X POST -u admin:admin  
> "http://cluster2:21000/api/atlas/admin/importFile?FILENAME=/tmp/entity.zip"; 
> {code}
> The import command failed with 500 internal server error.
> Attached the exception stack trace found in cluster2's application logs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-746) After updating a set of entities, response contains only the first entity definition

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-746?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-746:
---
Fix Version/s: 0.8-incubating
  Component/s:  atlas-core

Committed to master: 
http://git-wip-us.apache.org/repos/asf/incubator-atlas/commit/5f508c97

> After updating a set of entities, response contains only the first entity 
> definition
> 
>
> Key: ATLAS-746
> URL: https://issues.apache.org/jira/browse/ATLAS-746
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.7-incubating
>Reporter: Vishwanath Savakar
>Assignee: Jeffrey Hagelberg
>Priority: Major
> Fix For: 0.8-incubating
>
> Attachments: rb56283.patch
>
>
> Create two entities (POST /entities) and note guid1 and guid2 that get 
> generated.
> Update these entities in the same PUT /entities and response contains 
> (truncated for convenience):
> {
> "requestId": "...",
> "GUID": [
> "guid1",
> "guid2"
> ],
> "definition": "
> {..*\"id\":\"guid1\"*,}
> }
> Issue: definition does not contain the data for second entity.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-780) UI: Atlas UI shows an API response page after login sometimes

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-780?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-780:
---
Fix Version/s: 0.7-incubating

> UI: Atlas UI shows an API response page after login sometimes
> -
>
> Key: ATLAS-780
> URL: https://issues.apache.org/jira/browse/ATLAS-780
> Project: Atlas
>  Issue Type: Bug
>Reporter: Hemanth Yamijala
>Priority: Minor
> Fix For: 0.7-incubating
>
>
> I left the Atlas UI up for a while. After maybe an hour I came back and 
> performed some action on it. Possibly because of session expiry it took me 
> back to a login page and I logged in. Instead of taking me to the UI, this 
> showed the JSON response of an API. The URL displayed after login was 
> {{http://localhost:21000/api/atlas/types?type=TRAIT}}. This would probably be 
> a little confusing to the end user.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-876) No lineage for a re-created ctas table

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-876:
---
Fix Version/s: 0.8-incubating

> No lineage for a re-created ctas table
> --
>
> Key: ATLAS-876
> URL: https://issues.apache.org/jira/browse/ATLAS-876
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Pathan
>Assignee: Shwetha G S
>Priority: Critical
> Fix For: 0.8-incubating
>
>
> No lineage for a re-created ctas table
> Steps to reproduce:
> create table account (name String, balance Int);
> create table account_ctas as select * from account;
> drop table account_ctas;
> create table account_ctas as select * from account;
> Empty responses from server for recreated ctas
> {noformat}
> curl 
> 'http://localhost:21000/api/atlas/lineage/ebd6a5a9-8453-4dbd-b8d9-1575a464ebc3/inputs/graph'
>  -H 'Cookie: JSESSIONID=12rn8aybkawa5ewiqdb2hv663' -H 'Accept-Encoding: gzip, 
> deflate, sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: 
> Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, 
> like Gecko) Chrome/50.0.2661.102 Safari/537.36' -H 'Accept: application/json, 
> text/javascript, */*; q=0.01' -H 'Referer: http://localhost:21000/index.html' 
> -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' 
> --compressed | python -m json.tool
>   % Total% Received % Xferd  Average Speed   TimeTime Time  
> Current
>  Dload  Upload   Total   SpentLeft  Speed
> 100   2370   2370 0281  0 --:--:-- --:--:-- --:--:--   281
> {
> "requestId": "qtp1211076369-876 - ad29b29b-7faf-44f2-a2bd-8bbb00f07120",
> "results": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct135",
> "values": {
> "edges": {},
> "vertices": {}
> }
> }
> }
> curl 
> 'http://localhost:21000/api/atlas/lineage/ebd6a5a9-8453-4dbd-b8d9-1575a464ebc3/outputs/graph'
>  -H 'Cookie: JSESSIONID=12rn8aybkawa5ewiqdb2hv663' -H 'Accept-Encoding: gzip, 
> deflate, sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: 
> Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, 
> like Gecko) Chrome/50.0.2661.102 Safari/537.36' -H 'Accept: application/json, 
> text/javascript, */*; q=0.01' -H 'Referer: http://localhost:21000/index.html' 
> -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' 
> --compressed | python -m json.tool
>   % Total% Received % Xferd  Average Speed   TimeTime Time  
> Current
>  Dload  Upload   Total   SpentLeft  Speed
> 100   2380   2380 0271  0 --:--:-- --:--:-- --:--:--   271
> {
> "requestId": "qtp1211076369-1104 - ee65b8d3-0cf9-4477-a644-d93aab6b625b",
> "results": {
> "jsonClass": 
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct141",
> "values": {
> "edges": {},
> "vertices": {}
> }
> }
> }
> {noformat}
> UI snapshot: https://monosnap.com/file/3XcCcffPjrayjLaIByWcjBHU4k2pgV



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-918) UI : Deleted entities show status as DELETED and Datasets should be differentiable in lineage

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-918:
---
Fix Version/s: 0.7-incubating

> UI : Deleted entities show status as DELETED and Datasets should be 
> differentiable in lineage
> -
>
> Key: ATLAS-918
> URL: https://issues.apache.org/jira/browse/ATLAS-918
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
>Assignee: Keval Bhatt
>Priority: Critical
> Fix For: 0.7-incubating
>
> Attachments: Screen Shot 2016-06-20 at 2.20.32 PM.png
>
>
> Keval,
> This jira is to track showing deleted entities with a diff color in lineage. 
> Is there anything missing in the lineage API to implement this? We need to 
> take care of this since no one looks at lineage via a process and will be 
> mostly looking at lineage from a table or  a dataset. Also a table can be 
> dropped and recreated in hive, leading to linage showing as attached in 
> screen shot
> Also can we show the property (status)__status for deleted entities  as 
> "DELETED" . We could display that in "properties" section.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-937) Hive Classpath is printed instead of Hive configuration in import_hive.sh

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-937:
---
Fix Version/s: 0.8-incubating
   0.7.1-incubating

> Hive Classpath is printed instead of Hive configuration in import_hive.sh
> -
>
> Key: ATLAS-937
> URL: https://issues.apache.org/jira/browse/ATLAS-937
> Project: Atlas
>  Issue Type: Bug
>Reporter: Vimal Sharma
>Assignee: Vimal Sharma
>Priority: Trivial
> Fix For: 0.8-incubating, 0.7.1-incubating
>
>




--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Resolved] (ATLAS-944) Build failure: GraphBackedMetadataRepositoryTest.testFullTextSearch:528 expected:

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj resolved ATLAS-944.

Resolution: Cannot Reproduce  (was: Fixed)

> Build failure: GraphBackedMetadataRepositoryTest.testFullTextSearch:528 
> expected:
> -
>
> Key: ATLAS-944
> URL: https://issues.apache.org/jira/browse/ATLAS-944
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: trunk
> Environment: CentOS 7.2.1511
> OpenJDK 1.7.0_101
> Maven 3.3.9
> *fresh git clone*
>Reporter: Nigel Jones
>Priority: Major
>
> Build of Atlas fails
> [INFO] Apache Atlas Repository  FAILURE [13:40 
> min]
> Cause:
> Results :
> Failed tests:
>   GraphBackedMetadataRepositoryTest.testFullTextSearch:528 expected:<1> but 
> was:<0>
> Root error:
> Running org.apache.atlas.repository.graph.GraphBackedMetadataRepositoryTest
> Tests run: 26, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 31.471 sec 
> <<< FAILURE! - in 
> org.apache.atlas.repository.graph.GraphBackedMetadataRepositoryTest
> testFullTextSearch(org.apache.atlas.repository.graph.GraphBackedMetadataRepositoryTest)
>   Time elapsed: 1.237 sec  <<< FAILURE!
> java.lang.AssertionError: expected:<1> but was:<0>
> at org.testng.Assert.fail(Assert.java:89)
> at org.testng.Assert.failNotEquals(Assert.java:489)
> at org.testng.Assert.assertEquals(Assert.java:118)
> at org.testng.Assert.assertEquals(Assert.java:365)
> at org.testng.Assert.assertEquals(Assert.java:375)
> at 
> org.apache.atlas.repository.graph.GraphBackedMetadataRepositoryTest.testFullTextSearch(GraphBackedMetadataRepositoryTest.java:528)
> Looking at the code in
> ./repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java
> @Test(dependsOnMethods = "testSubmitEntity")
> public void testFullTextSearch() throws Exception {
> //todo fix this
> //Weird: with lucene, the test passes without sleep
> //but with elasticsearch, doesn't work without sleep. why??
> long sleepInterval = 1000;
> The 1000ms looks arbitary... I'm building on a 4 CPU vm which should be ok, 
> but to a shared drive, perhaps I/O limited. increasing the timeout



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-1196) Support circular lineage with Gremlin 3

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-1196:

Fix Version/s: 0.8-incubating

> Support circular lineage with Gremlin 3
> ---
>
> Key: ATLAS-1196
> URL: https://issues.apache.org/jira/browse/ATLAS-1196
> Project: Atlas
>  Issue Type: Sub-task
>Reporter: Jeffrey Hagelberg
>Priority: Major
> Fix For: 0.8-incubating
>
>
> Circular lineage support is something that has been added to Atlas since the 
> gremlin 3 translation logic was put in.  We need to update the DSL translator 
> to support this with Gremlin 3 as well.  See the logic for translating 
> LoopExpression in GreminQuery.scala.  In Gremlin 2, the logic was changed so 
> that if "times" is not specified, the loop exits if the path already contains 
> the object being processed.  Something similar needs to be added to the 
> Gremlin 3 logic.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-1968) Export/Import - Exception thrown when Import fired with zip file found on server

[Madhan Neethiraj (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1968?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Madhan Neethiraj updated ATLAS-1968:

Fix Version/s: 0.8.1
   1.0.0

> Export/Import -  Exception thrown when Import fired with zip file found on 
> server
> -
>
> Key: ATLAS-1968
> URL: https://issues.apache.org/jira/browse/ATLAS-1968
> Project: Atlas
>  Issue Type: Bug
>  Components:  atlas-core
>Affects Versions: 0.8.1, 1.0.0
>Reporter: Sharmadha Sainath
>Assignee: Ashutosh Mestry
>Priority: Blocker
> Fix For: 0.8.1, 1.0.0
>
> Attachments: ATLAS-1968-Importfile-option.patch, 
> ImportExceptionWhenZipFileOnServer.txt
>
>
> 1. Exported an entity on the cluster1 ,created zip file and placed it in /tmp 
> location of cluster2 .
> 2. Fired the following import command against cluster2 :
> {code}
> curl -v -X POST -u admin:admin  
> "http://cluster2:21000/api/atlas/admin/importFile?FILENAME=/tmp/entity.zip"; 
> {code}
> The import command failed with 500 internal server error.
> Attached the exception stack trace found in cluster2's application logs.



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Created] (ATLAS-2698) [Glossary] Add docs related to Glossary for Apache site

[Apoorv Naik (JIRA)]

Apoorv Naik created ATLAS-2698:
--

 Summary: [Glossary] Add docs related to Glossary for Apache site
 Key: ATLAS-2698
 URL: https://issues.apache.org/jira/browse/ATLAS-2698
 Project: Atlas
  Issue Type: Improvement
Affects Versions: 1.0.0-alpha
Reporter: Apoorv Naik
Assignee: Apoorv Naik
 Fix For: 1.0.0


Updating the docs module to include documentation around Glossary



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2698) [Glossary] Add docs related to Glossary for Apache site

[Apoorv Naik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apoorv Naik updated ATLAS-2698:
---
Attachment: 0001-ATLAS-2698-Glossary-related-documentation.patch

> [Glossary] Add docs related to Glossary for Apache site
> ---
>
> Key: ATLAS-2698
> URL: https://issues.apache.org/jira/browse/ATLAS-2698
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 1.0.0-alpha
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: 0001-ATLAS-2698-Glossary-related-documentation.patch
>
>
> Updating the docs module to include documentation around Glossary



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2698) [Glossary] Add docs related to Glossary for Apache site

[Apoorv Naik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apoorv Naik updated ATLAS-2698:
---
Attachment: 0001-ATLAS-2698-Glossary-related-documentation.patch

> [Glossary] Add docs related to Glossary for Apache site
> ---
>
> Key: ATLAS-2698
> URL: https://issues.apache.org/jira/browse/ATLAS-2698
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 1.0.0-alpha
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: 0001-ATLAS-2698-Glossary-related-documentation.patch
>
>
> Updating the docs module to include documentation around Glossary



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

[jira] [Updated] (ATLAS-2698) [Glossary] Add docs related to Glossary for Apache site

[Apoorv Naik (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-2698?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Apoorv Naik updated ATLAS-2698:
---
Attachment: (was: 0001-ATLAS-2698-Glossary-related-documentation.patch)

> [Glossary] Add docs related to Glossary for Apache site
> ---
>
> Key: ATLAS-2698
> URL: https://issues.apache.org/jira/browse/ATLAS-2698
> Project: Atlas
>  Issue Type: Improvement
>Affects Versions: 1.0.0-alpha
>Reporter: Apoorv Naik
>Assignee: Apoorv Naik
>Priority: Major
> Fix For: 1.0.0
>
> Attachments: 0001-ATLAS-2698-Glossary-related-documentation.patch
>
>
> Updating the docs module to include documentation around Glossary



--
This message was sent by Atlassian JIRA
(v7.6.3#76005)

Build failed in Jenkins: Atlas-1.0-AllTests #6

[Apache Jenkins Server]

See 


Changes:

[madhan] ATLAS-2697: audit log shows 'Propagated classification added' for

--
[...truncated 299.52 KB...]
[INFO] --- apache-rat-plugin:0.12:check (rat-check) @ atlas-repository ---
[INFO] Enabled default license matchers.
[INFO] Will parse SCM ignores for exclusions...
[INFO] Finished adding exclusions from SCM ignore files.
[INFO] 61 implicit excludes (use -debug for more details).
[INFO] Exclude: **/antlr4/**
[INFO] Exclude: **/dependency-reduced-pom.xml
[INFO] Exclude: **/javax.script.ScriptEngineFactory
[INFO] Exclude: .reviewboardrc
[INFO] Exclude: 3party-licenses/**
[INFO] Exclude: **/.cache
[INFO] Exclude: **/.cache-main
[INFO] Exclude: **/.cache-tests
[INFO] Exclude: **/.checkstyle
[INFO] Exclude: **/*.txt
[INFO] Exclude: **/*.json
[INFO] Exclude: .pc/**
[INFO] Exclude: debian/**
[INFO] Exclude: .svn/**
[INFO] Exclude: .git/**
[INFO] Exclude: .gitignore
[INFO] Exclude: **/.idea/**
[INFO] Exclude: **/*.twiki
[INFO] Exclude: **/*.iml
[INFO] Exclude: **/*.json
[INFO] Exclude: **/*.log
[INFO] Exclude: **/target/**
[INFO] Exclude: **/target*/**
[INFO] Exclude: **/build/**
[INFO] Exclude: **/*.patch
[INFO] Exclude: derby.log
[INFO] Exclude: **/logs/**
[INFO] Exclude: **/.classpath
[INFO] Exclude: **/.project
[INFO] Exclude: **/.settings/**
[INFO] Exclude: **/test-output/**
[INFO] Exclude: **/mock/**
[INFO] Exclude: **/data/**
[INFO] Exclude: **/maven-eclipse.xml
[INFO] Exclude: **/.externalToolBuilders/**
[INFO] Exclude: **/build.log
[INFO] Exclude: **/.bowerrc
[INFO] Exclude: *.json
[INFO] Exclude: **/overlays/**
[INFO] Exclude: dev-support/**
[INFO] Exclude: **/users-credentials.properties
[INFO] Exclude: **/public/css/animate.min.css
[INFO] Exclude: **/public/css/bootstrap-sidebar.css
[INFO] Exclude: **/public/js/external_lib/**
[INFO] Exclude: **/node_modules/**
[INFO] Exclude: **/public/js/libs/**
[INFO] Exclude: **/atlas.data/**
[INFO] Exclude: **/${sys:atlas.data}/**
[INFO] Exclude: **/policy-store.txt
[INFO] Exclude: **/*rebel*.xml
[INFO] Exclude: **/*rebel*.xml.bak
[INFO] Exclude: **/test/resources/**
[INFO] 174 resources included (use -debug for more details)
[INFO] Rat check: Summary over all files. Unapproved: 0, unknown: 0, generated: 
0, approved: 174 licenses.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:copy-resources (copy-resources-solr) @ 
atlas-repository ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 9 resources
[INFO] 
[INFO] --- maven-remote-resources-plugin:1.5:process (default) @ 
atlas-repository ---
[INFO] 
[INFO] --- maven-resources-plugin:2.7:resources (default-resources) @ 
atlas-repository ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] skip non existing resourceDirectory 

[INFO] Copying 2 resources to META-INF
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:compile (default-compile) @ 
atlas-repository ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 132 source files to 

[INFO] 
:
 Some input files use or override a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for details.
[INFO] 
:
 Some input files use unchecked or unsafe operations.
[INFO] 
:
 Recompile with -Xlint:unchecked for details.
[INFO] 
[INFO] --- maven-resources-plugin:2.7:testResources (default-testResources) @ 
atlas-repository ---
[INFO] Using 'UTF-8' encoding to copy filtered resources.
[INFO] Copying 35 resources
[INFO] Copying 3 resources
[INFO] 
[INFO] --- maven-compiler-plugin:3.7.0:testCompile (default-testCompile) @ 
atlas-repository ---
[INFO] Changes detected - recompiling the module!
[INFO] Compiling 45 source files to 

[INFO] 
:
 

 uses or overrides a deprecated API.
[INFO] 
:
 Recompile with -Xlint:deprecation for

Re: Review Request 66928: Data Migration: Import: Infer Types that Store Edge Ids

[Sarath Subramanian]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/66928/#review203388
---


Ship it!




Ship It!

- Sarath Subramanian


On May 17, 2018, 1:48 p.m., Ashutosh Mestry wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/66928/
> ---
> 
> (Updated May 17, 2018, 1:48 p.m.)
> 
> 
> Review request for atlas, Madhan Neethiraj, Ruchi Solani, and Sarath 
> Subramanian.
> 
> 
> Bugs: ATLAS-2637
> https://issues.apache.org/jira/browse/ATLAS-2637
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> **Approach**
> New class: _TypesStoringEdgeIds_: Navigates through all the types in the 
> _typeRegistry_, returns map of entity type and properties that store edge ids.
> Modified: _DataMigrationService_: Uses output from class above and passes it 
> down to migration.
> Modified: _PostProcessManager_: Uses the map generated above and uses it for 
> post processing.
> Modified: _GraphSONUtility_: Improvement to check for vertex of type. This 
> avoids potential exeception when a non-existent property is checked for 
> presence.
> 
> Added PostProcess framework.
> Added logic for handling new Array and Map representation.
> 
> 
> Diffs
> -
> 
>   
> graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraph.java 
> 607baf664 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraph.java
>  c0b9c1741 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/AtlasJanusGraphDatabase.java
>  16aecd5e2 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/AtlasGraphSONReader.java
>  ae119b0bc 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/GraphSONUtility.java
>  ec320b03e 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/PostProcessManager.java
>  d0a65f7b1 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/RelationshipTypeCache.java
>  e4e82649b 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/postProcess/PostProcessListProperty.java
>  PRE-CREATION 
>   
> graphdb/janus/src/main/java/org/apache/atlas/repository/graphdb/janus/migration/postProcess/PostProcessor.java
>  PRE-CREATION 
>   
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/BaseUtils.java
>  e863d9fae 
>   
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/GraphSONUtilityPostProcessTest.java
>  4d73c78ef 
>   
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/GraphSONUtilityTest.java
>  794b5471e 
>   
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/MappedElementCacheTest.java
>  cac09d229 
>   
> graphdb/janus/src/test/java/org/apache/atlas/repository/graphdb/janus/migration/PostProcessListPropertyTest.java
>  PRE-CREATION 
>   graphdb/janus/src/test/resources/col-2-legacy.json PRE-CREATION 
>   graphdb/janus/src/test/resources/edge-legacy-col.json PRE-CREATION 
>   graphdb/janus/src/test/resources/edge-legacy-col2.json PRE-CREATION 
>   graphdb/janus/src/test/resources/edge-legacy-col3.json PRE-CREATION 
>   graphdb/janus/src/test/resources/edge-legacy-col4.json PRE-CREATION 
>   graphdb/janus/src/test/resources/table-v-147504.json 898dce5df 
>   intg/src/main/java/org/apache/atlas/store/AtlasTypeDefStore.java b05754f4b 
>   intg/src/test/java/org/apache/atlas/TestUtilsV2.java 886ce77f5 
>   
> repository/src/main/java/org/apache/atlas/repository/migration/DataMigrationService.java
>  22cd55217 
>   
> repository/src/main/java/org/apache/atlas/repository/migration/TypesWithCollectionsFinder.java
>  PRE-CREATION 
>   
> repository/src/main/java/org/apache/atlas/repository/store/graph/v1/AtlasTypeDefGraphStoreV1.java
>  3c84e3c22 
>   
> repository/src/test/java/org/apache/atlas/repository/migration/ComplexAttributesTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/migration/HiveParititionTest.java
>  ac0b79d38 
>   
> repository/src/test/java/org/apache/atlas/repository/migration/HiveStocksTest.java
>  ffbf3200b 
>   
> repository/src/test/java/org/apache/atlas/repository/migration/MigrationBaseAsserts.java
>  ec6e64a25 
>   
> repository/src/test/java/org/apache/atlas/repository/migration/TypesWithCollectionsFinderTest.java
>  PRE-CREATION 
>   
> repository/src/test/java/org/apache/atlas/repository/store/graph/v1/AtlasEntityTestBase.java
>  d810a1166 
>   repository/src/test/resources/complex-attr_db/atlas-migration-data.json 
> PRE-CREATION 
>   repository/src/test/resources/complex-attr_db/atlas-migration-ty

Jenkins build is back to normal : Atlas-1.0-AllTests #7

[Apache Jenkins Server]

See