-----------------------------------------------------------
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/47810/
-----------------------------------------------------------
(Updated May 27, 2016, 4:04 p.m.)
Review request for atlas, David Kantor and Neeru Gupta.
Bugs: ATLAS-694
https://issues.apache.org/jira/browse/ATLAS-694
Repository: atlas
Description (updated)
-------
ATLAS-694: Update Atlas to use abstraction layer. All of the Atlas code (with
the exception of the catalog, which was only updated minimally) has been
updated to use the graph database abstraction layer. In addition, there is now
a required Atlas configuration property that specifies the class of the
abstraction layer database to use. I basically put all of the changes in here
with the exception of the actual Titan 1 implementation of code. This includes
the changes to support Tinkerpop 3 syntax. This is mostly to expedite getting
the changes into Atlas. Originally the TP3 changes were going to be brought in
as part of the Titan 1 implementation task.
Change Summary:
- change Atlas classes to use AtlasGraph,AtlasVertex,AtlasEdge, etc instead
of TitanGraph/Vertex/Edge, etc
- compile time dependency on titan 0.5.4/TP 2 removed (except in Catalog,
which was only changed to use AtlasGraphProvider/AtlasGraph)
- updated DSL translation to generate Gremlin that is compliant with TP3
when TP3 is being used
- TitanGraphProvider replaced by AtlasGraphProvider. Graph database
implementation is determined from a new required configuration property
- search indexer enhanced to wait for all indices to be enabled
- HiveTitanSample is no longer used by tests. It has been replaced by
hive-instances.json (which uses normal Atlas json syntax). The data is saved
with a new JSONImporter class. This was needed because the graphson syntax
used by HiveTitanSample is not compatible with TP3.
- RepositoryMetaModule - GraphBackedSearchIndexer changed to an eager
singleton to force index creation to happen before we execute any gremlin
queries (which happens during the type system restoration triggered by the
DefaultMetadataService constructor)
- atlas_config.py - minor change to allow Atlas to use a non-default JVM.
It now supports having the java home in ATLAS_JAVA_HOME. If that is not set,
we fall back to the previous behavior looking for JAVA_HOME and then checking
the path.
Note - the diff here is cumulative and includes the changes from ATLAS-693. I
could not figure out how to remove those.
Diffs
-----
addons/falcon-bridge/pom.xml 14c60901a6f194497ca0fb7929a032d2261f85fa
addons/hdfs-model/pom.xml f4033bce0fe3ea5a33cfd32ebccd47c01207dc35
addons/hdfs-model/src/test/java/org/apache/atlas/fs/model/HDFSModelTest.java
ac60294e328835ba0340e150799ddfb348ccdb52
addons/hive-bridge/pom.xml 47e72e814a421f03b00a9b7c6a13163e1b67ca14
addons/sqoop-bridge/pom.xml 343bb4eeae93031ab24cd48ed7b535769150d6c7
addons/storm-bridge/pom.xml b1a7a9b1a88442d9e65db4a01b8388c8d3522bfe
authorization/pom.xml 74c433bf740a346d1d907a868605e9ab457151bb
catalog/pom.xml 8a49d3d673a5936613d45e53479aa99a5a14517d
catalog/src/main/java/org/apache/atlas/catalog/query/BaseQuery.java
c2eabf70de57f2893a4c30a045fa8f10aaf0b837
catalog/src/test/java/org/apache/atlas/catalog/query/AtlasEntityQueryTest.java
838d4c16579efcc2519d668149610edfe78f83c5
distro/pom.xml 22f70ae970d3ee4ce3d56d15f0e9df5a3fa21b94
distro/src/bin/atlas_config.py fab4046345e05e0b7749ca75834c8bdcb0d01752
distro/src/conf/atlas-application.properties
bb12538197ea47aad0cad013529c6821e1b1a766
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasEdge.java
71b577b5c2e8aee8fec816b9224672ebc2cad5b3
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasEdgeDirection.java
e7da1c9a5075004973ba68b8a36a2c95756e0e62
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasElement.java
3c41693e531eb523f04a5c5781f1f123ab38ef07
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraph.java
f312117af9cb6ae8de21d565bfcf3b10d9bdfcc6
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphIndex.java
PRE-CREATION
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphManagement.java
2776b2242c4469dcae344616ccfe100633a79432
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasGraphQuery.java
f0dca9a28eebd8996d55d71dcc3254cce9cd4098
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasIndexQuery.java
7ee6b2831a7610e7d5b1d36ae69282a9315dc06b
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasPropertyKey.java
PRE-CREATION
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasSchemaViolationException.java
fda83b807f3640c04a660001941376f41be6d9be
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertex.java
e027b6962cf27a8c2dfbbf6fcc9683fa19e00ae5
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/AtlasVertexQuery.java
fd20a6524c4795b4212599c2f4ced69c2bbf14c0
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/GraphDatabase.java
a608eb27921d4be44e639f773ace818115f39ff2
graphdb/api/src/main/java/org/apache/atlas/repository/graphdb/GremlinVersion.java
e817cba200c6d3c4c593b242cb2317e141bab570
graphdb/pom.xml 666fe5a01e55d7b761d391e3ef6172f769488254
graphdb/titan0/pom.xml PRE-CREATION
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/GraphDbObjectFactory.java
PRE-CREATION
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0DatabaseManager.java
PRE-CREATION
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Edge.java
PRE-CREATION
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Element.java
PRE-CREATION
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Graph.java
PRE-CREATION
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0GraphIndex.java
PRE-CREATION
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0GraphQuery.java
PRE-CREATION
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0IndexQuery.java
PRE-CREATION
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0PropertyKey.java
PRE-CREATION
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0Vertex.java
PRE-CREATION
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/Titan0VertexQuery.java
PRE-CREATION
graphdb/titan0/src/main/java/org/apache/atlas/repository/graphdb/titan0/TitanObjectFactory.java
PRE-CREATION
graphdb/titan0/src/main/java/org/apache/atlas/utils/EdgeToAtlasEdgeFunction.java
PRE-CREATION
graphdb/titan0/src/main/java/org/apache/atlas/utils/VertexToAtlasVertexFunction.java
PRE-CREATION
graphdb/titan0/src/test/java/org/apache/atlas/repository/graphdb/titan0/Titan0DatabaseTest.java
PRE-CREATION
graphdb/titan0/src/test/resources/atlas-application.properties PRE-CREATION
pom.xml 6013cdc64a4c04c97d9c6b3e6b4fc10b9debd0ff
repository/pom.xml 533e48b8dbf8f1d80903b804ddc7da9818fe6702
repository/src/main/java/org/apache/atlas/GraphTransactionInterceptor.java
b9689f4eb408819e91e8184d40534e983bf5d048
repository/src/main/java/org/apache/atlas/RepositoryMetadataModule.java
15201dd45913539b9b3bd430b46ce6cf30c327ac
repository/src/main/java/org/apache/atlas/discovery/DataSetLineageService.java
39dde2af0aa86d7c9a3dc5f006c8e1660cd29d5c
repository/src/main/java/org/apache/atlas/discovery/graph/DefaultGraphPersistenceStrategy.java
90718edfd261c9f30ecc6b8f2424c232fbc9a66b
repository/src/main/java/org/apache/atlas/discovery/graph/GraphBackedDiscoveryService.java
f97b83d1b302dedc12c566a8bb4729d4cd68de75
repository/src/main/java/org/apache/atlas/repository/Constants.java
893f1b64fa5d38c8025e6faf86335864f77d1439
repository/src/main/java/org/apache/atlas/repository/MetadataRepository.java
43e9f85163e8bb453601ca502d9e7a32c757b037
repository/src/main/java/org/apache/atlas/repository/graph/AtlasEdgeLabel.java
d905c012a58ae4f53012f217253bf2495550da33
repository/src/main/java/org/apache/atlas/repository/graph/AtlasGraphProvider.java
PRE-CREATION
repository/src/main/java/org/apache/atlas/repository/graph/DeleteHandler.java
91f9bd008a6939dfe78656f5c1845637a30ee9eb
repository/src/main/java/org/apache/atlas/repository/graph/FullTextMapper.java
37cacb05e65d6b1f537bd208208c8c4b9b501645
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java
0d82d900bd750b1ca91c24623384fb2547790efe
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexer.java
c385df35c0ed37490033601487bf59eaf7dc677f
repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java
4f6d0118072380342f5ea302d0aaf7902784849a
repository/src/main/java/org/apache/atlas/repository/graph/GraphProvider.java
f89bdf5c7e8596b11dafced6700d0a4245fd32cf
repository/src/main/java/org/apache/atlas/repository/graph/GraphSchemaInitializer.java
6141927eb92bc7e681b43118fbaa399ada6c81f8
repository/src/main/java/org/apache/atlas/repository/graph/GraphToTypedInstanceMapper.java
e240fb68ed9547ef6ef4cb773a12b2594dcffa09
repository/src/main/java/org/apache/atlas/repository/graph/HardDeleteHandler.java
36367913252b59f7adf7f42924a886365e60819f
repository/src/main/java/org/apache/atlas/repository/graph/SoftDeleteHandler.java
25aa7c5844cd3fc39bd7cc9ee23a6c336ca1bfac
repository/src/main/java/org/apache/atlas/repository/graph/TitanGraphProvider.java
2cc1a50e2ea139c3a462c918b94945fff602cf17
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java
4c1f5591f4adead41c3336e64b0bc956836f7edb
repository/src/main/java/org/apache/atlas/repository/typestore/GraphBackedTypeStore.java
5ed9e02eb85ec25b63540cab40d148914d410786
repository/src/main/java/org/apache/atlas/services/DefaultMetadataService.java
c1af0f691e9d26a3102fece20acc904bb85702d6
repository/src/main/scala/org/apache/atlas/query/ClosureQuery.scala
c4621cd509ae049b30a08a9e5b3ace03f888a10c
repository/src/main/scala/org/apache/atlas/query/Expressions.scala
240575087468632c52512aea94759f8ace3943a0
repository/src/main/scala/org/apache/atlas/query/GraphPersistenceStrategies.scala
34d101a2704e28a46426fdf6b67e7c36fde0507a
repository/src/main/scala/org/apache/atlas/query/GremlinEvaluator.scala
7de03c32edf16c1676db94179266e02928e01f35
repository/src/main/scala/org/apache/atlas/query/GremlinQuery.scala
73981c04d410a7292de357d9f63b6f3b3d241ffa
repository/src/main/scala/org/apache/atlas/query/QueryParser.scala
60b57d9a4d0181cd1e3c45a4e6a928d15ed499db
repository/src/main/scala/org/apache/atlas/query/QueryProcessor.scala
0d2a908201f526aa487dd4cad926d308f53a04ba
repository/src/main/scala/org/apache/atlas/query/TypeUtils.scala
5a64c53bdb94174bf5699d9e034960f42450f696
repository/src/test/java/org/apache/atlas/BaseRepositoryTest.java
d1f9430124ab4071971950bf4acd2f09dd000da0
repository/src/test/java/org/apache/atlas/RepositoryServiceLoadingTest.java
4195955051d842af973d3ad29602fb97c1e54ecd
repository/src/test/java/org/apache/atlas/TestUtils.java
345e874942cf047ae8b6467f2981c7cf2317707c
repository/src/test/java/org/apache/atlas/discovery/GraphBackedDiscoveryServiceTest.java
5e7de88559dbbf2460c819ba7470a2cc287ca8dc
repository/src/test/java/org/apache/atlas/repository/audit/AuditRepositoryTestBase.java
f6994048b7f92ba086a1b0949030897d1cacff41
repository/src/test/java/org/apache/atlas/repository/audit/HBaseBasedAuditRepositoryTest.java
677eb392c4aa7674e27c48f86a44e953c80144c7
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryDeleteTestBase.java
449e066a167ba4546b118a77c8e3de5fd99f077b
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java
2d1c33a9bef5dc9179225fa7ce7228eebc5fdf2b
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedRepositoryHardDeleteTest.java
cc6026461bcd38b83dde0bcec2e835f5824e894f
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedRepositorySoftDeleteTest.java
90bb635126a02cd38c045bd5332499c4481d3751
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexerMockTest.java
94dd29862cd37fd784a699fb45640e36992cff88
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedSearchIndexerTest.java
3d9d45a1f6695a0ee1af8c42fb24dcfa41dfba79
repository/src/test/java/org/apache/atlas/repository/graph/GraphRepoMapperScaleTest.java
2f02ae28b2947ce7f7955c00bf9ca88ab3ca607a
repository/src/test/java/org/apache/atlas/repository/graph/TitanGraphProviderTest.java
6fc700823800eaa943a34afecb2c2e6fba50637a
repository/src/test/java/org/apache/atlas/repository/typestore/GraphBackedTypeStoreTest.java
961442bbcaff86b472a876e6c348aa9e65c1efca
repository/src/test/java/org/apache/atlas/service/DefaultMetadataServiceTest.java
e6dd23081715458471d50f44f45ccd8374cbffda
repository/src/test/java/org/apache/atlas/services/DependencyTreeNode.java
PRE-CREATION
repository/src/test/java/org/apache/atlas/services/JSONImporter.java
PRE-CREATION
repository/src/test/resources/hive-instances.json PRE-CREATION
repository/src/test/scala/org/apache/atlas/query/GremlinTest.scala
b23c0f60ace59fa15886c94f35e35d3113872783
repository/src/test/scala/org/apache/atlas/query/GremlinTest2.scala
f65cedbf44f5fa9e674833a20aae8de6d2537834
repository/src/test/scala/org/apache/atlas/query/HiveTitanSample.scala
67ce12bf31dd6decf1a1323924303a066f90a4c1
repository/src/test/scala/org/apache/atlas/query/LineageQueryTest.scala
0e0ac8633a256dc7c8966c2b8d52eb942b52e1c6
repository/src/test/scala/org/apache/atlas/query/QueryTestsUtils.scala
b5faaf33cd5fd1a63bac4fd41d67588ef519dc11
repository/src/test/titan0-resources/atlas-application.properties
PRE-CREATION
repository/src/test/titan1-resources/atlas-application.properties
PRE-CREATION
titan/pom.xml 09006d9dd5e21aa24f53524aa42a3fcd6f3d9fe6
titan/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/AdminMask.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/ConnectionMask.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/HBaseAdmin0_98.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/HBaseAdmin1_0.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/HBaseCompat.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/HBaseCompat0_98.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/HBaseCompat1_0.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/HBaseCompat1_1.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/HBaseCompatLoader.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/HBaseKeyColumnValueStore.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/HBaseStoreManager.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/HBaseTransaction.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/HConnection0_98.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/HConnection1_0.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/HTable0_98.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/HTable1_0.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/hbase/TableMask.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/locking/LocalLockMediator.java
titan/src/main/java/com/thinkaurelius/titan/diskstorage/solr/Solr5Index.java
titan/src/test/java/com/thinkaurelius/titan/diskstorage/locking/LocalLockMediatorTest.java
typesystem/src/main/java/org/apache/atlas/typesystem/persistence/StructInstance.java
6fb2087a7cbeb6eefb0dd15b9a3c0c049b6b857b
typesystem/src/main/java/org/apache/atlas/typesystem/types/AbstractDataType.java
dc9cdf2567b45b35b324ecfbb5afcc7921409b8f
typesystem/src/main/java/org/apache/atlas/typesystem/types/HierarchicalTypeDependencySorter.java
aaec05c8c3913b6974332668ebe5af31332cd121
typesystem/src/main/java/org/apache/atlas/typesystem/types/Multiplicity.java
06da32e807aa4b8e4d1f49b3929c9f3c9a8d4846
typesystem/src/main/resources/atlas-application.properties
3318ff306a8b8ab5e320129a8450d144215f009f
typesystem/src/main/scala/org/apache/atlas/typesystem/json/TypesSerialization.scala
5618938e4bd9f9ef934e89bf2bf3199073294706
typesystem/src/test/java/org/apache/atlas/typesystem/types/ClassTest.java
daecdd78c952747964885d4cb8cdfb59727ffee5
typesystem/src/test/java/org/apache/atlas/typesystem/types/HierarchicalTypeDependencySorterTest.java
19bdccfc6baf503f3144fce8fe0b85914693d054
typesystem/src/test/java/org/apache/atlas/typesystem/types/TypeSystemTest.java
574e0f9811d195cd34e12a9f23b999568d54a1c7
typesystem/src/test/scala/org/apache/atlas/typesystem/json/TypesSerializationTest.scala
cfd4bdb26482c187c4e80f110b7349df8b524e6e
webapp/pom.xml 5b023ebfbd4f77f80b0ef676e0a160dda6576f5e
webapp/src/main/java/org/apache/atlas/web/listeners/GuiceServletConfig.java
010fa2aa979e12063a1d78c8db4ad7519c3ca973
webapp/src/main/java/org/apache/atlas/web/resources/RexsterGraphResource.java
150f913d881138c7afeb87881197ee0f1e7d3194
webapp/src/test/java/org/apache/atlas/web/listeners/TestGuiceServletConfig.java
08bb125241012b6a1c1852efc6443cc7a4ebecc3
Diff: https://reviews.apache.org/r/47810/diff/
Testing
-------
Built entire Atlas project, ran all unit/integration tests. No issues found.
Thanks,
Jeff Hagelberg
