[jira] [Assigned] (ATLAS-175) DSL query fails when where is no alias
[ https://issues.apache.org/jira/browse/ATLAS-175?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad reassigned ATLAS-175: -- Assignee: Suma Shivaprasad > DSL query fails when where is no alias > --- > > Key: ATLAS-175 > URL: https://issues.apache.org/jira/browse/ATLAS-175 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad > Assignee: Suma Shivaprasad > > Check GraphBackedDiscoveryServiceTest.testSearchByDSLQueries > "hive_column where hive_column isa PII select hive_column.name", 6}, -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-100) Auto create solr collections
[ https://issues.apache.org/jira/browse/ATLAS-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906158#comment-14906158 ] Suma Shivaprasad commented on ATLAS-100: [~jspeidel] This is needed as part of Ambari Integration for Solr > Auto create solr collections > > > Key: ATLAS-100 > URL: https://issues.apache.org/jira/browse/ATLAS-100 > Project: Atlas > Issue Type: Sub-task >Reporter: Shwetha G S >Assignee: John Speidel > > Currently, creating collections corresponding to indexes are manual steps. > Enable programmatic creation of collections to avoid the manual steps -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-100) Auto create solr collections
[ https://issues.apache.org/jira/browse/ATLAS-100?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14906158#comment-14906158 ] Suma Shivaprasad edited comment on ATLAS-100 at 9/24/15 10:28 AM: -- [~jspeidel] This is needed as part of Ambari Integration for Solr. Need to create 3 collections vertex_index, edge_index, fulltext_index as part of Solr bootstrap in cloud mode which has been documented in http://atlas.incubator.apache.org/InstallationSteps.html was (Author: suma.shivaprasad): [~jspeidel] This is needed as part of Ambari Integration for Solr > Auto create solr collections > > > Key: ATLAS-100 > URL: https://issues.apache.org/jira/browse/ATLAS-100 > Project: Atlas > Issue Type: Sub-task >Reporter: Shwetha G S >Assignee: John Speidel > > Currently, creating collections corresponding to indexes are manual steps. > Enable programmatic creation of collections to avoid the manual steps -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-187) DSL - Fix issues while loading arrays in a select expression
[ https://issues.apache.org/jira/browse/ATLAS-187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-187: --- Summary: DSL - Fix issues while loading arrays in a select expression (was: Fix issues while loading arrays in a select expression) > DSL - Fix issues while loading arrays in a select expression > - > > Key: ATLAS-187 > URL: https://issues.apache.org/jira/browse/ATLAS-187 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad > > Currently the loading of lists is not supported and throws an Exception while > trying to do do. eg query : StorageDesc cols in > GraphBackedDiscoveryServiceTest -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-187) Fix issues while loading arrays in a select expression
Suma Shivaprasad created ATLAS-187: -- Summary: Fix issues while loading arrays in a select expression Key: ATLAS-187 URL: https://issues.apache.org/jira/browse/ATLAS-187 Project: Atlas Issue Type: Bug Reporter: Suma Shivaprasad Currently the loading of lists is not supported and throws an Exception while trying to do do. eg query : StorageDesc cols in GraphBackedDiscoveryServiceTest -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-128) DSL - Add support for comparisions on list type
[ https://issues.apache.org/jira/browse/ATLAS-128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-128: --- Summary: DSL - Add support for comparisions on list type (was: Add support for list type) > DSL - Add support for comparisions on list type > --- > > Key: ATLAS-128 > URL: https://issues.apache.org/jira/browse/ATLAS-128 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.6-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-128.patch > > > Add support for equality matches on lists -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-128) Add support for list type
[ https://issues.apache.org/jira/browse/ATLAS-128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-128: --- Description: Add support for equality matches on lists > Add support for list type > - > > Key: ATLAS-128 > URL: https://issues.apache.org/jira/browse/ATLAS-128 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.6-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-128.patch > > > Add support for equality matches on lists -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-128) Add support for list type
[ https://issues.apache.org/jira/browse/ATLAS-128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903097#comment-14903097 ] Suma Shivaprasad commented on ATLAS-128: Updated reviewboard with final patch [~shwethags] [~s_venkatesh] Can you pls review > Add support for list type > - > > Key: ATLAS-128 > URL: https://issues.apache.org/jira/browse/ATLAS-128 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.6-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-128.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (ATLAS-102) Issue with SolrIndex
[ https://issues.apache.org/jira/browse/ATLAS-102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad reassigned ATLAS-102: -- Assignee: Suma Shivaprasad > Issue with SolrIndex > > > Key: ATLAS-102 > URL: https://issues.apache.org/jira/browse/ATLAS-102 > Project: Atlas > Issue Type: Sub-task >Reporter: Shwetha G S > Assignee: Suma Shivaprasad > > Currently, atlas packages titan-solr-0.5.4 and solr 5.2.1. > com.thinkaurelius.titan.diskstorage.solr.SolrIndex in titan-solr doesn't work > with solr 5.2.1. SolrIndex has PreInitializeConfigOptions annotation and > titan tries to load this class which fails. This comes as a nasty error in > application log, but titan comes up successfully. The only way that we can > avoid is renaming Solr5Index in atlas to SolrIndex so that SolrIndex in > titan-solr is not loaded by classloader -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-30) GraphBackedDiscoveryServiceTest.testSearchByDSLQueries
[ https://issues.apache.org/jira/browse/ATLAS-30?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14903103#comment-14903103 ] Suma Shivaprasad commented on ATLAS-30: --- Fixed as part of ATLAS-128 - https://reviews.apache.org/r/38179 > GraphBackedDiscoveryServiceTest.testSearchByDSLQueries > -- > > Key: ATLAS-30 > URL: https://issues.apache.org/jira/browse/ATLAS-30 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.5-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > > Currently , the tests are accepting only the input query in the dataprovider, > they should accept both input query and expected number of rows. > Also all the queries running are returning query 0 rows. This needs to be > investigated and fixed. > query [from DB] returned [0] rows > query [DB] returned [0] rows > query [DB where DB.name="Reporting"] returned [0] rows > query [DB DB.name = "Reporting"] returned [0] rows > query [DB where DB.name="Reporting" select name, owner] returned [0] rows > query [DB has name] returned [0] rows > query [DB, Table] returned [0] rows > query [DB is JdbcAccess] returned [0] rows > query [from Table] returned [0] rows > query [Table] returned [0] rows > query [Table is Dimension] returned [0] rows > query [Column where Column isa PII] returned [0] rows > query [View is Dimension] returned [0] rows > query [Column select Column.name] returned [0] rows > query [Column select name] returned [0] rows > query [Column where Column.name="customer_id"] returned [0] rows > query [from Table select Table.name] returned [0] rows > query [DB where (name = "Reporting")] returned [0] rows > query [DB where (name = "Reporting") select name as _col_0, owner as _col_1] > returned [0] rows > query [DB where DB is JdbcAccess] returned [0] rows > query [DB where DB has name] returned [0] rows > query [DB Table] returned [0] rows > query [DB where DB has name] returned [0] rows > query [DB as db1 Table where (db1.name = "Reporting")] returned [0] rows > query [DB where (name = "Reporting") select name as _col_0, (createTime + 1) > as _col_1 ] returned [0] rows > query [Table where (name = "sales_fact" and created > > "2014-01-01T00:00:00.000Z" ) select name as _col_0, created as _col_1 ] > returned [0] rows > query [Table where (name = "sales_fact" and created > > "2014-12-11T02:35:58.440Z" ) select name as _col_0, created as _col_1 ] > returned [0] rows > query [Dimension] returned [0] rows > query [JdbcAccess] returned [0] rows > query [ETL] returned [0] rows > query [Metric] returned [0] rows > query [PII] returned [0] rows > query [Table LoadProcess outputTable] returned [0] rows > query [Table loop (LoadProcess outputTable)] returned [0] rows > query [Table as _loop0 loop (LoadProcess outputTable) withPath] returned [0] > rows > query [Table as src loop (LoadProcess outputTable) as dest select src.name as > srcTable, dest.name as destTable withPath] returned [0] rows > query [Table as t, sd, Column as c where t.name="sales_fact" select c.name as > colName, c.dataType as colType] returned [0] rows > query [Table where name='sales_fact', db where name='Reporting'] returned [0] > rows -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-134) Some defects found when reviewing the source code.
[
https://issues.apache.org/jira/browse/ATLAS-134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14902275#comment-14902275
]
Suma Shivaprasad commented on ATLAS-134:
[~ltfxyz] I meant the check for windows which you had mentioned above is not in
the patch attached. Can you pls check?
{noformat}
if IS_WINDOWS:
java_mock.assert_called_with(
'org.apache.atlas.Main',
['-app', 'metadata_home/server/webapp/atlas'],
'metadata_home/conf:metadata_home/server/webapp/atlas/WEB-INF/classes:metadata_home/server/webapp/atlas/WEB-INF/lib\\*:metadata_home/libext
*',
['-Datlas.log.dir=metadata_home/logs', '-Datlas.log.file=application.log',
'-Datlas.home=metadata_home', '-Datlas.conf=metadata_home/conf', '-Xmx1024m',
'-Dlog4j.configuration=atlas-log4j.xml'], 'metadata_home/logs')
else:
{noformat}
> Some defects found when reviewing the source code.
> --
>
> Key: ATLAS-134
> URL: https://issues.apache.org/jira/browse/ATLAS-134
> Project: Atlas
> Issue Type: Bug
>Affects Versions: trunk
>Reporter: liutongfeng
>Priority: Minor
> Labels: patch
> Fix For: trunk
>
> Attachments: ATLAS-134.patch
>
>
> 1.The second getKeyType() should be getValueType().
> file : ObjectGraphTraversal.java
> line : 56
> IDataType keyType = ((DataTypes.MapType) dT).getKeyType();
> IDataType valueType = ((DataTypes.MapType) dT).getKeyType();
> processMap(keyType, valueType, val);
> 2.The second getKeyType() should be getValueType().
> file : ObjectGraphWalker.java
> line : 94
> IDataType keyType = ((DataTypes.MapType) dT).getKeyType();
> IDataType valueType = ((DataTypes.MapType) dT).getKeyType();
> visitMap(keyType, valueType, val);
> 3.When OS is WINDOWS, '/' should be '\\', otherwise the test will fail.
> file : TestMetadata.py
> line : 52
> if IS_WINDOWS:
> java_mock.assert_called_with(
> 'org.apache.atlas.Main',
> ['-app', 'metadata_home/server/webapp/atlas'],
>
> 'metadata_home/conf:metadata_home/server/webapp/atlas/WEB-INF/classes:metadata_home/server/webapp/atlas/WEB-INF/lib\\*:metadata_home/libext\\*',
> ['-Datlas.log.dir=metadata_home/logs',
> '-Datlas.log.file=application.log', '-Datlas.home=metadata_home',
> '-Datlas.conf=metadata_home/conf', '-Xmx1024m',
> '-Dlog4j.configuration=atlas-log4j.xml'], 'metadata_home/logs')
> else:
> 4.Should use return.
> file :
> incubator-atlas\repository\src\main\java\org\apache\atlas\repository\memory\AttributeStores.java
> line : 58
> static IAttributeStore createStore(AttributeInfo i) throws
> RepositoryException {
> switch (i.dataType().getTypeCategory()) {
> case PRIMITIVE:
> if (i.dataType() == DataTypes.BOOLEAN_TYPE) {
> return new BooleanAttributeStore(i);
> } else if (i.dataType() == DataTypes.BYTE_TYPE) {
> return new ByteAttributeStore(i);
> } else if (i.dataType() == DataTypes.SHORT_TYPE) {
> new ShortAttributeStore(i); // should be return
> } else if (i.dataType() == DataTypes.INT_TYPE) {
> return new IntAttributeStore(i);
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-147) Fix a code issue when defineTypes
[
https://issues.apache.org/jira/browse/ATLAS-147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14900841#comment-14900841
]
Suma Shivaprasad edited comment on ATLAS-147 at 9/21/15 3:41 PM:
-
+1. Thanks [~ltfxyz] . Will be committed tmrw
was (Author: suma.shivaprasad):
+1.
> Fix a code issue when defineTypes
> -
>
> Key: ATLAS-147
> URL: https://issues.apache.org/jira/browse/ATLAS-147
> Project: Atlas
> Issue Type: Bug
>Affects Versions: trunk
>Reporter: liutongfeng
>Priority: Minor
> Labels: patch
> Fix For: trunk
>
> Attachments: ATLAS-147.patch
>
>
> file : org/apache/atlas/typesystem/types/TypeSystem.java
> for (StructTypeDefinition structDef : structDefs) {
> constructStructureType(structDef);
> typeCategoriesToTypeNamesMap.put(DataTypes.TypeCategory.CLASS,
> structDef.typeName);
> }
> The above TypeCategory.CLASS should be TypeCategory.STRUCT.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (ATLAS-178) Fix loop expressions to support array types
Suma Shivaprasad created ATLAS-178: -- Summary: Fix loop expressions to support array types Key: ATLAS-178 URL: https://issues.apache.org/jira/browse/ATLAS-178 Project: Atlas Issue Type: Bug Reporter: Suma Shivaprasad In GraphBackedDiscoveryTest, query tests with DataSet loop (Process outputs) since DataSet is of ClassType where as outputs is of array type. Needs to be fixed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-102) Issue with SolrIndex
[ https://issues.apache.org/jira/browse/ATLAS-102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-102: --- Attachment: ATLAS-102.patch > Issue with SolrIndex > > > Key: ATLAS-102 > URL: https://issues.apache.org/jira/browse/ATLAS-102 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.6-incubating >Reporter: Shwetha G S >Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-102.patch > > > Currently, atlas packages titan-solr-0.5.4 and solr 5.2.1. > com.thinkaurelius.titan.diskstorage.solr.SolrIndex in titan-solr doesn't work > with solr 5.2.1. SolrIndex has PreInitializeConfigOptions annotation and > titan tries to load this class which fails. This comes as a nasty error in > application log, but titan comes up successfully. The only way that we can > avoid is renaming Solr5Index in atlas to SolrIndex so that SolrIndex in > titan-solr is not loaded by classloader -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (ATLAS-369) Publish 0.6-incubating artifacts
[ https://issues.apache.org/jira/browse/ATLAS-369?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad reassigned ATLAS-369: -- Assignee: Suma Shivaprasad > Publish 0.6-incubating artifacts > > > Key: ATLAS-369 > URL: https://issues.apache.org/jira/browse/ATLAS-369 > Project: Atlas > Issue Type: Sub-task >Reporter: Shwetha G S > Assignee: Suma Shivaprasad > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-47) Entity mutations for complex types
[ https://issues.apache.org/jira/browse/ATLAS-47?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15046428#comment-15046428 ] Suma Shivaprasad commented on ATLAS-47: --- +1 > Entity mutations for complex types > -- > > Key: ATLAS-47 > URL: https://issues.apache.org/jira/browse/ATLAS-47 > Project: Atlas > Issue Type: New Feature > Reporter: Suma Shivaprasad > Assignee: Suma Shivaprasad > Fix For: trunk > > Attachments: ATLAS-47-v3.patch, ATLAS-47-v4.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-373) Renew TGT from KeyTab for ATLAS service principal
Suma Shivaprasad created ATLAS-373: -- Summary: Renew TGT from KeyTab for ATLAS service principal Key: ATLAS-373 URL: https://issues.apache.org/jira/browse/ATLAS-373 Project: Atlas Issue Type: Bug Affects Versions: 0.6, 0.6-incubating Reporter: Suma Shivaprasad Assignee: Suma Shivaprasad Fix For: 0.6, 0.6-incubating ATLAS server doesnt renew the service principal upon expiry -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-352) Fix performance issues with type and entity creation with Hbase as storage backend
[ https://issues.apache.org/jira/browse/ATLAS-352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-352: --- Attachment: ATLAS-352.1.patch Fixed review comments and test failures during deployment > Fix performance issues with type and entity creation with Hbase as storage > backend > --- > > Key: ATLAS-352 > URL: https://issues.apache.org/jira/browse/ATLAS-352 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.6-incubating >Reporter: Suma Shivaprasad > Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-352.1.patch, ATLAS-352.patch > > > Current perf microbenchmark measurements > DataSet quickstart model and entity definitions > No of class types 6 > traits6 with no attributes > server start up time with existing schema 17 secs > type batch creation 27 secs > entity creation - DB 12 secs > entity creation - Table 11 secs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-361) Add validation when index backends are switched in ATLAS configuration
[ https://issues.apache.org/jira/browse/ATLAS-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-361: --- Attachment: ATLAS-361.patch > Add validation when index backends are switched in ATLAS configuration > -- > > Key: ATLAS-361 > URL: https://issues.apache.org/jira/browse/ATLAS-361 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-361.patch > > > Currently, the index backend configs can be changed y user but it ignored by > Titan since it can have bad repercussions i.e index data loss. > Hence we need to validate that the configuration has been switched and throw > and error during start up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-360) Secure cluster Atlas-solr integration instructions
[ https://issues.apache.org/jira/browse/ATLAS-360?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044567#comment-15044567 ] Suma Shivaprasad commented on ATLAS-360: Pls changes the lines to removed the hardcoded hostname . This be the solr server host name? + kadmin.local: addprinc -randkey solr/c6401.ambari.apache@example.com + kadmin.local: xst -k solr.keytab solr/c6401.ambari.apache@example.com > Secure cluster Atlas-solr integration instructions > -- > > Key: ATLAS-360 > URL: https://issues.apache.org/jira/browse/ATLAS-360 > Project: Atlas > Issue Type: Bug >Reporter: Tom Beerbower >Assignee: Tom Beerbower > Attachments: ATLAS-360.patch > > > Kerberized secure cluster solr configuration instructions for Atlas. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-242) The qualified name for hive entities should be backward compatible
[
https://issues.apache.org/jira/browse/ATLAS-242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15044564#comment-15044564
]
Suma Shivaprasad commented on ATLAS-242:
One minor comment. Pls change to camel case for "colid" to "colId". +1.
> The qualified name for hive entities should be backward compatible
> --
>
> Key: ATLAS-242
> URL: https://issues.apache.org/jira/browse/ATLAS-242
> Project: Atlas
> Issue Type: Bug
>Reporter: David Kaspar
>Assignee: Shwetha G S
> Fix For: trunk
>
> Attachments: ATLAS-242.patch
>
>
> This issue is filed for "RangerAtlasDemo_v4.ova" VM-image release.
> When I call:
> http://127.0.0.1:21000/api/atlas/lineage/hive/table/{tableName}/schema
> REST API method, then it always returns 404 Not Found and returns an error
> which claims that given {tableName} does not exist.
> Since REST API documentation (application.wadl) does not state details, I
> have tried to use existing Hive table names, existing GUIDs of Hive tables
> but none of them works.
> Similarly for /api/atlas/lineage/hive/table/{tableName}/outputs/graph and
> /api/atlas/lineage/hive/table/{tableName}/inputs/graph.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-380) Fix ATLAS source artifact generation
[ https://issues.apache.org/jira/browse/ATLAS-380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15048298#comment-15048298 ] Suma Shivaprasad edited comment on ATLAS-380 at 12/9/15 8:37 AM: - Committed to master and branch-0.6-incubating. Thanks for the review [~shwethags] was (Author: suma.shivaprasad): Committed to trunk. Thanks for the review [~shwethags] > Fix ATLAS source artifact generation > > > Key: ATLAS-380 > URL: https://issues.apache.org/jira/browse/ATLAS-380 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-380.1.patch, ATLAS-380.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-380) Fix ATLAS source artifact generation
[ https://issues.apache.org/jira/browse/ATLAS-380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-380: --- Attachment: ATLAS-380.1.patch > Fix ATLAS source artifact generation > > > Key: ATLAS-380 > URL: https://issues.apache.org/jira/browse/ATLAS-380 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-380.1.patch, ATLAS-380.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-380) Fix ATLAS source artifact generation
[ https://issues.apache.org/jira/browse/ATLAS-380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-380: --- Attachment: ATLAS-380.patch > Fix ATLAS source artifact generation > > > Key: ATLAS-380 > URL: https://issues.apache.org/jira/browse/ATLAS-380 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-380.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-380) Fix ATLAS source artifact generation
[ https://issues.apache.org/jira/browse/ATLAS-380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-380: --- Affects Version/s: 0.6-incubating > Fix ATLAS source artifact generation > > > Key: ATLAS-380 > URL: https://issues.apache.org/jira/browse/ATLAS-380 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-380) Fix ATLAS source artifact generation
Suma Shivaprasad created ATLAS-380: -- Summary: Fix ATLAS source artifact generation Key: ATLAS-380 URL: https://issues.apache.org/jira/browse/ATLAS-380 Project: Atlas Issue Type: Bug Reporter: Suma Shivaprasad Assignee: Suma Shivaprasad -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-380) Fix ATLAS source artifact generation
[ https://issues.apache.org/jira/browse/ATLAS-380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15048136#comment-15048136 ] Suma Shivaprasad commented on ATLAS-380: The distro module currently generates source articfact which includes the current modules src files and should generate an artifact having the entire ATLAS source code > Fix ATLAS source artifact generation > > > Key: ATLAS-380 > URL: https://issues.apache.org/jira/browse/ATLAS-380 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-20) Rename env variables from METADATA to ATLAS
[ https://issues.apache.org/jira/browse/ATLAS-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15048667#comment-15048667 ] Suma Shivaprasad edited comment on ATLAS-20 at 12/9/15 1:29 PM: [~rishabhbhardwaj] The patch LGTM. Can you run some sanity checks ? For this pls make deploy a stand alone ATLAS build by running mvn -Pdist clean installl and then follow the instructions in http://atlas.incubator.apache.org/InstallationSteps.html and http://atlas.incubator.apache.org/QuickStart.html and update the sanity check results here. was (Author: suma.shivaprasad): The patch LGTM. Can you run some sanity checks ? For this pls make deploy a stand alone ATLAS build by running mvn -Pdist clean installl and then follow the instructions in http://atlas.incubator.apache.org/InstallationSteps.html and http://atlas.incubator.apache.org/QuickStart.html and update the sanity check results here. > Rename env variables from METADATA to ATLAS > --- > > Key: ATLAS-20 > URL: https://issues.apache.org/jira/browse/ATLAS-20 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.5-incubating >Reporter: Venkatesh Seetharam >Assignee: Rishabh Bhardwaj > Attachments: ATLAS-20.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-263) Searching for a multi word trait always returns empty result
[
https://issues.apache.org/jira/browse/ATLAS-263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15048524#comment-15048524
]
Suma Shivaprasad commented on ATLAS-263:
[~girishrp] I tested the above steps with the latest ATLAS build and the
discovery query works with the backtick ` like you mentioned.
curl -v
http://localhost:21000/api/atlas/discovery/search?query=`multi%20word%20trait`
{"requestId":"qtp1311760211-13 -
e8ca55f3-0a63-4f01-973f-e6e7ed5b8069","query":"`multi word
trait`","queryType":"dsl","count":1,"results":[{"$typeName$":"__tempQueryResultStruct41","instanceInfo":{"$typeName$":"__IdType","guid":"f0c989c4-7e3c-43be-8015-11997a4b6dc8","typeName":"Table"},"traitDetails":null}],"dataType":{"typeName":"__tempQueryResultStruct41","attributeDefinitions":[{"name":"traitDetails","dataTypeName":"multi
word
trait","multiplicity":{"lower":0,"upper":1,"isUnique":false},"isComposite":false,"isUnique":false,"isIndexable":true,"reverseAttributeName":null},{"name":"instanceInfo","dataTypeName":"__IdType","multiplicity":{"lower":0,"upper":1,"isUnique":false},"isComposite":false,"isUnique":false,"isIndexable":true,"reverseAttributeName":null}]}}
Can you please instead submit a patch for documentation example in Search.twiki
and QuickStart giving an example of this?
> Searching for a multi word trait always returns empty result
>
>
> Key: ATLAS-263
> URL: https://issues.apache.org/jira/browse/ATLAS-263
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.6-incubating
>Reporter: Ayub Khan
>Assignee: Girish Ramachandran
>Priority: Blocker
> Fix For: trunk
>
> Attachments: types_query_fix.patch
>
>
> Searching for a multi word trait always returns empty result..
> 1. Create a trait name with multiple words..
> Example:
> {noformat}
> curl 'http://localhost:21000/api/atlas/types' -H 'Origin:
> http://localhost:21000' -H 'Content-Type: application/json;charset=UTF-8'
> --data-binary
> '{"enumTypes":[],"traitTypes":[{"superTypes":[],"typeName":"multi word
> trait","attributeDefinitions":[],"hierarchicalMetaTypeName":"org.apache.atlas.typesystem.types.TraitType"}],"structTypes":[],"classTypes":[]}'
> --compressed
> {noformat}
> 2. Attach the tag with an entity
> Example:
> {noformat}
> curl -v
> 'http://localhost:21000/api/atlas/entity/803f5bdd-28f9-49e1-8c21-3dc0f9fd5c61/traits'
> -H 'Content-Type: application/json;charset=UTF-8' --data-binary
> '{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct","typeName":"multi
> word trait","values":{}}' --compressed
> {noformat}
> 3. Now try to get all the entities associated with the tag
> Example:
> {noformat}
> curl
> 'http://localhost:21000/api/atlas/discovery/search?query=multi+word+trait'
> {noformat}
>
> Result of the above is always empty because the above query is
> triggered as "full text" and it always returns empty result even if you have
> associated entites to the tag.
>
> This is a big issue from an end user's perspective. marking this as a blocker
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ATLAS-20) Rename env variables from METADATA to ATLAS
[ https://issues.apache.org/jira/browse/ATLAS-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15048667#comment-15048667 ] Suma Shivaprasad commented on ATLAS-20: --- The patch LGTM. Can you run some sanity checks ? For this pls make deploy a stand alone ATLAS build by running mvn -Pdist clean installl and then follow the instructions in http://atlas.incubator.apache.org/InstallationSteps.html and http://atlas.incubator.apache.org/QuickStart.html and update the sanity check results here. > Rename env variables from METADATA to ATLAS > --- > > Key: ATLAS-20 > URL: https://issues.apache.org/jira/browse/ATLAS-20 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.5-incubating >Reporter: Venkatesh Seetharam >Assignee: Rishabh Bhardwaj > Attachments: ATLAS-20.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-179) Atlas hook causes mem leak and hive server 2 crashes
[ https://issues.apache.org/jira/browse/ATLAS-179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-179: --- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: (was: 0.5-incubating) 0.6-incubating > Atlas hook causes mem leak and hive server 2 crashes > > > Key: ATLAS-179 > URL: https://issues.apache.org/jira/browse/ATLAS-179 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Critical > Fix For: 0.6-incubating > > Attachments: ATLAS-179-v2.patch, ATLAS-179.patch > > > Two issues: > 1. While queueing for ThreadPoolExecutor, we add hive query plan. Looks like > query plan contains the fetch tasks and their job confs. Job confs contain > java.util.Properties and can be huge > 2. We have unbounded queue in ThreadPoolExecutor -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-36) Need separate persisted properties for HTTP and HTTPS ports
[ https://issues.apache.org/jira/browse/ATLAS-36?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-36: -- Fix Version/s: (was: 0.5.1-incubating) 0.6-incubating Affects Version/s: (was: 0.5-incubating) 0.6-incubating > Need separate persisted properties for HTTP and HTTPS ports > --- > > Key: ATLAS-36 > URL: https://issues.apache.org/jira/browse/ATLAS-36 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: Jonathan Maron >Assignee: Jonathan Maron > Fix For: 0.6-incubating > > Original Estimate: 48h > Remaining Estimate: 48h > > There is currently only a command line option to specify a port, and default > values for both the HTTP and HTTPS ports. This may confuse users and makes > management via tools such as Ambari confusing and error prone. The solution > is, like many other hadoop services, to clearly specify the values to be used > for HTTP and HTTPS as distinct application properties. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-345) UI: Should allow tag addition on any search result that returns a reference-able entity
[ https://issues.apache.org/jira/browse/ATLAS-345?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-345: --- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: 0.6-incubating > UI: Should allow tag addition on any search result that returns a > reference-able entity > --- > > Key: ATLAS-345 > URL: https://issues.apache.org/jira/browse/ATLAS-345 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.6-incubating >Reporter: Hemanth Yamijala >Assignee: Darshan Kumar > Labels: patch-available > Fix For: 0.6-incubating > > Attachments: ATLAS-345_V1.patch, ATLAS-345_V2.patch, > ATLAS-345_v0.patch, hive_storagedesc_results.png, > hive_storagedesc_results_json.txt > > > We have a use case to associate tags to entities such as columns in hive > tables and thereby drive integration with other components that can provide > tag based policy enforcement, such as Apache Ranger. > In order to enable a user to associate tags to columns, the UI should allow > an "add tag" action on the search results page against entities that are > provided in the result list. > For e.g. if we search for hive_column where name=, we > should allow add tags against each column that comes up in the result. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-296) IllegalArgumentException during hive HiveHookIT integration tests.
[
https://issues.apache.org/jira/browse/ATLAS-296?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-296:
---
Fix Version/s: (was: trunk)
0.6-incubating
Affects Version/s: 0.6-incubating
> IllegalArgumentException during hive HiveHookIT integration tests.
> --
>
> Key: ATLAS-296
> URL: https://issues.apache.org/jira/browse/ATLAS-296
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.6-incubating
>Reporter: Tom Beerbower
>Assignee: Tom Beerbower
> Fix For: 0.6-incubating
>
> Attachments: ATLAS-296.patch
>
>
> In the hive-bridge tests (HiveHookIT), the following stacktrace is printed.
> The tests succeed though.
> Introduced with patch for entity notification. ATLAS-158.
> {code}
> 127.0.0.1 - - [10/Nov/2015:04:31:24 +] "OPTIONS
> /api/atlas/entities?user.name=sshivalingamurthy HTTP/1.1" 200 3317 "-"
> "Java/1.7.0_79"
> 2015-11-10 10:01:24.622:WARN:oejs.ServletHandler:qtp276244868-27:
> /api/atlas/entities
> java.lang.IllegalArgumentException: Input String cannot be null cannot be null
> at org.apache.atlas.ParamChecker.notNull(ParamChecker.java:34)
> at org.apache.atlas.web.util.Servlets.escapeJsonString(Servlets.java:146)
> at org.apache.atlas.web.util.Servlets.getErrorResponse(Servlets.java:125)
> at org.apache.atlas.web.util.Servlets.getErrorResponse(Servlets.java:107)
> at
> org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:136)
> at sun.reflect.GeneratedMethodAccessor119.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:606)
> at
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)
> at
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)
> at
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:790)
> at
> com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:287)
> at
> com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:277)
> at
> com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:182)
> at
> com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
> at org.apache.atlas.web.filters.AuditFilter.doFilter(AuditFilter.java:67)
> at
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:119)
> at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:133)
> at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:130)
> at
> com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:203)
> at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:130)
> at
> org.eclipse.jetty.servlet.ServletHandler$CachedChain.doFilter(ServletHandler.java:1652)
> at
> org.eclipse.jetty.servlet.ServletHandler.doHandle(ServletHandle
[jira] [Updated] (ATLAS-208) Remove "\n" characters in the REST API json response
[
https://issues.apache.org/jira/browse/ATLAS-208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-208:
---
Fix Version/s: (was: trunk)
0.6-incubating
> Remove "\n" characters in the REST API json response
>
>
> Key: ATLAS-208
> URL: https://issues.apache.org/jira/browse/ATLAS-208
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.6-incubating
>Reporter: Ayub Khan
>Assignee: Satya Narayan
>Priority: Minor
> Fix For: 0.6-incubating
>
> Attachments: ATLAS-208.txt
>
>
> "\n" characters are seen in REST API response, which is not needed and can be
> removed..
> curl -k -s --negotiate -u :
> http://localhost:21000/api/atlas/types/traitWithSuperTypefxgy2bhnev | python
> -m json.tool
> {
> "definition": "{\n \"enumTypes\":[\n\n ],\n \"structTypes\":[\n
> \n ],\n \"traitTypes\":[\n{\n \"superTypes\":[\n
> \"traitSuperType1fxgy2bhnev\",\n\"traitSuperType2fxgy2bhnev\"\n
> ],\n
> \"hierarchicalMetaTypeName\":\"org.apache.atlas.typesystem.types.TraitType\",\n
> \"typeName\":\"traitWithSuperTypefxgy2bhnev\",\n
> \"attributeDefinitions\":[\n{\n \"name\":\"string\",\n
>\"dataTypeName\":\"string\",\n \"multiplicity\":\"optional\",\n
>\"isComposite\":false,\n \"isUnique\":false,\n
> \"isIndexable\":true,\n \"reverseAttributeName\":null\n}\n
>]\n}\n ],\n \"classTypes\":[\n\n ]\n}",
> "requestId": "1146437560@qtp-1778535015-267 -
> d982526c-f6b6-4fe5-a4ed-722cbc29eb38",
> "typeName": "traitWithSuperTypefxgy2bhnev"
> }
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-128) DSL - Add support for comparisions on list type
[ https://issues.apache.org/jira/browse/ATLAS-128?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-128: --- Fix Version/s: (was: trunk) 0.6-incubating > DSL - Add support for comparisions on list type > --- > > Key: ATLAS-128 > URL: https://issues.apache.org/jira/browse/ATLAS-128 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.6-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-128.1.patch, ATLAS-128.patch > > > Add support for equality matches on lists -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-16) jersey jaxb exception
[
https://issues.apache.org/jira/browse/ATLAS-16?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-16:
--
Fix Version/s: (was: trunk)
0.6-incubating
Affects Version/s: (was: trunk)
0.6-incubating
> jersey jaxb exception
> -
>
> Key: ATLAS-16
> URL: https://issues.apache.org/jira/browse/ATLAS-16
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.6-incubating
>Reporter: Shwetha G S
>Assignee: Shwetha G S
> Fix For: 0.6-incubating
>
> Attachments: ATLAS-16-v2.patch, ATLAS-16-v3.patch, ATLAS-16.patch
>
>
> {noformat}
> Jun 16, 2015 1:37:42 PM
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator
> attachTypes
> INFO: Couldn't find JAX-B element for class javax.ws.rs.core.Response
> Jun 16, 2015 1:37:42 PM
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8
> resolve
> SEVERE: null
> java.lang.IllegalAccessException: Class
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8 can
> not access a member of class javax.ws.rs.core.Response with modifiers
> "protected"
> at sun.reflect.Reflection.ensureMemberAccess(Reflection.java:109)
> at java.lang.Class.newInstance(Class.java:373)
> at
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator$8.resolve(WadlGeneratorJAXBGrammarGenerator.java:467)
> at
> com.sun.jersey.server.wadl.WadlGenerator$ExternalGrammarDefinition.resolve(WadlGenerator.java:181)
> at
> com.sun.jersey.server.wadl.ApplicationDescription.resolve(ApplicationDescription.java:81)
> at
> com.sun.jersey.server.wadl.generators.WadlGeneratorJAXBGrammarGenerator.attachTypes(WadlGeneratorJAXBGrammarGenerator.java:518)
> at com.sun.jersey.server.wadl.WadlBuilder.generate(WadlBuilder.java:151)
> at
> com.sun.jersey.server.impl.wadl.WadlApplicationContextImpl.getApplication(WadlApplicationContextImpl.java:125)
> at
> com.sun.jersey.server.impl.wadl.WadlMethodFactory$WadlOptionsMethodDispatcher.dispatch(WadlMethodFactory.java:98)
> at
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)
> at
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)
> at
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)
> at
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)
> at
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)
> at
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:416)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:537)
> at
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:699)
> at javax.servlet.http.HttpServlet.service(HttpServlet.java:820)
> at
> com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:287)
> at
> com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:277)
> at
> com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:182)
> at
> com.google.inject.servlet.ManagedServletPipeline.service(ManagedServletPipeline.java:91)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:85)
> at
> org.apache.atlas.web.filters.AuditFilter.doFilter(AuditFilter.java:67)
> at
> com.google.inject.servlet.FilterChainInvocation.doFilter(FilterChainInvocation.java:82)
> at
> com.google.inject.servlet.ManagedFilterPipeline.dispatch(ManagedFilterPipeline.java:119)
> at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:133)
> at com.google.inject.servlet.GuiceFilter$1.call(GuiceFilter.java:130)
> at
> com.google.inject.servlet.GuiceFilter$Context.call(GuiceFilter.java:203)
> at com.google.inject.servlet.GuiceFilter.doFilter(GuiceFilter.java:130)
>
[jira] [Updated] (ATLAS-29) client authentication fails in secure environment from server processes
[ https://issues.apache.org/jira/browse/ATLAS-29?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-29: -- Fix Version/s: (was: 0.5.1-incubating) 0.6-incubating Affects Version/s: (was: 0.5-incubating) 0.6-incubating > client authentication fails in secure environment from server processes > --- > > Key: ATLAS-29 > URL: https://issues.apache.org/jira/browse/ATLAS-29 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: Jonathan Maron >Assignee: Jonathan Maron > Fix For: 0.6-incubating > > Original Estimate: 24h > Remaining Estimate: 24h > > During client invocations in secure environments the invocation may fail with > exceptions such as: > Caused by: org.apache.hadoop.security.AccessControlException: SIMPLE > authentication is not enabled. Available:[TOKEN, KERBEROS] -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-32) Atlas hook fails to fire in certain secure deployments
[ https://issues.apache.org/jira/browse/ATLAS-32?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-32: -- Fix Version/s: (was: 0.5.1-incubating) 0.6-incubating Affects Version/s: (was: 0.5-incubating) 0.6-incubating > Atlas hook fails to fire in certain secure deployments > -- > > Key: ATLAS-32 > URL: https://issues.apache.org/jira/browse/ATLAS-32 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: Jonathan Maron >Assignee: Jonathan Maron >Priority: Critical > Fix For: 0.6-incubating > > Original Estimate: 72h > Remaining Estimate: 72h > > In certain secure deployments, the interactions from hive to atlas trigger > security exceptions like the following: > 2015-06-20 07:29:00,506 INFO [Atlas Logger 0]: hook.HiveHook > (HiveHook.java:run(154)) - Atlas hook failed > com.sun.jersey.api.client.ClientHandlerException: java.io.IOException: > org.apache.hadoop.security.authentication.client.AuthenticationException: > GSSException: No valid credentials provided (Mechanism level: Failed to find > any Kerberos tgt) > this is more than likely due the need to perform the interactions as an > identity that has an associated kerberos ticket. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-21) quick_start script run throws an exception
[ https://issues.apache.org/jira/browse/ATLAS-21?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-21: -- Fix Version/s: (was: 0.5.1-incubating) 0.6-incubating Affects Version/s: (was: 0.5-incubating) 0.6-incubating > quick_start script run throws an exception > -- > > Key: ATLAS-21 > URL: https://issues.apache.org/jira/browse/ATLAS-21 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: Venkatesh Seetharam >Assignee: Jonathan Maron > Fix For: 0.6-incubating > > > related to log dir - can we source atlas-env.sh? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-367) Create branch-0.6-incubating
[ https://issues.apache.org/jira/browse/ATLAS-367?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15048733#comment-15048733 ] Suma Shivaprasad commented on ATLAS-367: Have updated project version > Create branch-0.6-incubating > > > Key: ATLAS-367 > URL: https://issues.apache.org/jira/browse/ATLAS-367 > Project: Atlas > Issue Type: Sub-task >Reporter: Shwetha G S >Assignee: Shwetha G S > > Create branch-0.6-incubating and change project version to 0.6-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-79) Unique constraint is not honoured
[ https://issues.apache.org/jira/browse/ATLAS-79?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-79: -- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: 0.6-incubating > Unique constraint is not honoured > - > > Key: ATLAS-79 > URL: https://issues.apache.org/jira/browse/ATLAS-79 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.6-incubating > > Attachments: ATLAS-79-v2.patch, ATLAS-79-v3.patch, ATLAS-79-v4.patch, > ATLAS-79.patch > > > Atlas supports attributes where they are marked as unique. But this > constraint is not enforced and atlas allows 2 entities with same value for > this unique attribute -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-102) Issue with SolrIndex
[ https://issues.apache.org/jira/browse/ATLAS-102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-102: --- Fix Version/s: (was: trunk) 0.6-incubating > Issue with SolrIndex > > > Key: ATLAS-102 > URL: https://issues.apache.org/jira/browse/ATLAS-102 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.6-incubating >Reporter: Shwetha G S >Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-102.patch > > > Currently, atlas packages titan-solr-0.5.4 and solr 5.2.1. > com.thinkaurelius.titan.diskstorage.solr.SolrIndex in titan-solr doesn't work > with solr 5.2.1. SolrIndex has PreInitializeConfigOptions annotation and > titan tries to load this class which fails. This comes as a nasty error in > application log, but titan comes up successfully. The only way that we can > avoid is renaming Solr5Index in atlas to SolrIndex so that SolrIndex in > titan-solr is not loaded by classloader -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-33) Atlas restart fails
[
https://issues.apache.org/jira/browse/ATLAS-33?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-33:
--
Fix Version/s: (was: trunk)
0.6-incubating
Affects Version/s: 0.6-incubating
> Atlas restart fails
> ---
>
> Key: ATLAS-33
> URL: https://issues.apache.org/jira/browse/ATLAS-33
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.6-incubating
>Reporter: Shwetha G S
>Assignee: Shwetha G S
> Fix For: 0.6-incubating
>
> Attachments: ATLAS-33.patch
>
>
> {code}
> 2015-06-20 07:33:19,494 ERROR - [main:] ~ Failed startup of context
> org.mortbay.jetty.webapp.WebAppContext@5b7cb3e0{/,/var/lib/atlas/server/webapp/atlas}
> (log:87)
> com.google.inject.CreationException: Unable to create injector, see the
> following errors:
> 1) Error injecting constructor, java.lang.IllegalArgumentException:
> Comparison method violates its general contract!
> at
> org.apache.atlas.services.DefaultMetadataService.(DefaultMetadataService.java:84)
> at
> org.apache.atlas.services.DefaultMetadataService.class(DefaultMetadataService.java:73)
> while locating org.apache.atlas.services.DefaultMetadataService
> at
> org.apache.atlas.RepositoryMetadataModule.configure(RepositoryMetadataModule.java:65)
> while locating org.apache.atlas.services.MetadataService
> 1 error
> at
> com.google.inject.internal.Errors.throwCreationExceptionIfErrorsExist(Errors.java:466)
> at
> com.google.inject.internal.InternalInjectorCreator.injectDynamically(InternalInjectorCreator.java:184)
> at
> com.google.inject.internal.InternalInjectorCreator.build(InternalInjectorCreator.java:110)
> at com.google.inject.Guice.createInjector(Guice.java:96)
> at com.google.inject.Guice.createInjector(Guice.java:73)
> at com.google.inject.Guice.createInjector(Guice.java:62)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.getInjector(GuiceServletConfig.java:60)
> at
> com.google.inject.servlet.GuiceServletContextListener.contextInitialized(GuiceServletContextListener.java:47)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.contextInitialized(GuiceServletConfig.java:99)
> at
> org.mortbay.jetty.handler.ContextHandler.startContext(ContextHandler.java:549)
> at org.mortbay.jetty.servlet.Context.startContext(Context.java:136)
> at
> org.mortbay.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1282)
> at
> org.mortbay.jetty.handler.ContextHandler.doStart(ContextHandler.java:518)
> at
> org.mortbay.jetty.webapp.WebAppContext.doStart(WebAppContext.java:499)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.mortbay.jetty.handler.HandlerWrapper.doStart(HandlerWrapper.java:130)
> at org.mortbay.jetty.Server.doStart(Server.java:224)
> at
> org.mortbay.component.AbstractLifeCycle.start(AbstractLifeCycle.java:50)
> at
> org.apache.atlas.web.service.EmbeddedServer.start(EmbeddedServer.java:79)
> at org.apache.atlas.Main.main(Main.java:83)
> Caused by: java.lang.IllegalArgumentException: Comparison method violates its
> general contract!
> at java.util.ComparableTimSort.mergeLo(ComparableTimSort.java:714)
> at java.util.ComparableTimSort.mergeAt(ComparableTimSort.java:451)
> at
> java.util.ComparableTimSort.mergeForceCollapse(ComparableTimSort.java:392)
> at java.util.ComparableTimSort.sort(ComparableTimSort.java:191)
> at java.util.ComparableTimSort.sort(ComparableTimSort.java:146)
> at java.util.Arrays.sort(Arrays.java:472)
> at java.util.Collections.sort(Collections.java:155)
> at
> org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.step3(TypeSystem.java:507)
> at
> org.apache.atlas.typesystem.types.TypeSystem$TransientTypeSystem.defineTypes(TypeSystem.java:547)
> at
> org.apache.atlas.typesystem.types.TypeSystem.defineTypes(TypeSystem.java:249)
> at
> org.apache.atlas.typesystem.types.TypeSystem.defineTypes(TypeSystem.java:241)
> at
> org.apache.atlas.services.DefaultMetadataService.restoreTypeSystem(DefaultMetadataService.java:97)
> at
> org.apache.atlas.services.DefaultMetadataService.(DefaultMetadataService.java:90)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.refle
[jira] [Updated] (ATLAS-155) Images do not show up on the dashboard
[ https://issues.apache.org/jira/browse/ATLAS-155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-155: --- Fix Version/s: (was: trunk) 0.6-incubating > Images do not show up on the dashboard > -- > > Key: ATLAS-155 > URL: https://issues.apache.org/jira/browse/ATLAS-155 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: Venkatesh Seetharam >Assignee: Darshan Kumar >Priority: Critical > Labels: patch-available > Fix For: 0.6-incubating > > Attachments: ATLAS-155-v0.patch > > > Post ATLAS-90 and ATLAS-109, the images do not show up, was not moved from v2 > folder. I forced deleted v2 folder. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-354) Kerberized cluster: quick_start.py fails to add sample data
[
https://issues.apache.org/jira/browse/ATLAS-354?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-354:
---
Fix Version/s: (was: trunk)
0.6-incubating
Affects Version/s: (was: 0.5-incubating)
0.6-incubating
> Kerberized cluster: quick_start.py fails to add sample data
> ---
>
> Key: ATLAS-354
> URL: https://issues.apache.org/jira/browse/ATLAS-354
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.6-incubating
>Reporter: Ayub Khan
>Assignee: Shwetha G S
>Priority: Blocker
> Fix For: 0.6-incubating
>
> Attachments: ATLAS-354-v2.patch, ATLAS-354-v3.patch, ATLAS-354.patch
>
>
> Check the ticket cache available
> {noformat}
> [atlas@os-r7-apathan-hbase-1 ~]$ klist
> Ticket cache: FILE:/tmp/krb5cc_1010
> Default principal: atlas/os-r7-apathan-hbase-1.novalo...@example.com
> Valid starting Expires Service principal
> 12/01/2015 17:57:14 12/02/2015 17:57:14 krbtgt/example@example.com
> [atlas@os-r7-apathan-hbase-1 ~]$
> {noformat}
> Below is the client.properties from kerberized cluster
> {noformat}
> # Security Properties #
> # SSL config
> atlas.enableTLS=false
> truststore.file=/path/to/truststore.jks
> cert.stores.credential.provider.path=jceks://file/path/to/credentialstore.jceks
> # following only required for 2-way SSL
> keystore.file=/path/to/keystore.jks
> # Authentication config
> # enabled: true or false
> atlas.http.authentication.enabled=false
> # type: simple or kerberos
> atlas.http.authentication.type=simple
> # Security Properties #
> {noformat}
> Now try running quick_start.py, it throws below exception
> {noformat}
> Exception in thread "main" com.sun.jersey.api.client.ClientHandlerException:
> java.io.IOException:
> org.apache.hadoop.security.authentication.client.AuthenticationException:
> Authentication failed, status: 401, message: Authentication required
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:149)
> at com.sun.jersey.api.client.Client.handle(Client.java:648)
> at com.sun.jersey.api.client.WebResource.handle(WebResource.java:670)
> at com.sun.jersey.api.client.WebResource.access$200(WebResource.java:74)
> at
> com.sun.jersey.api.client.WebResource$Builder.method(WebResource.java:623)
> at
> org.apache.atlas.AtlasClient.callAPIWithResource(AtlasClient.java:351)
> at org.apache.atlas.AtlasClient.callAPI(AtlasClient.java:370)
> at org.apache.atlas.AtlasClient.createType(AtlasClient.java:170)
> at org.apache.atlas.examples.QuickStart.createTypes(QuickStart.java:97)
> at org.apache.atlas.examples.QuickStart.main(QuickStart.java:57)
> Caused by: java.io.IOException:
> org.apache.hadoop.security.authentication.client.AuthenticationException:
> Authentication failed, status: 401, message: Authentication required
> at
> org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:107)
> at
> org.apache.atlas.security.SecureClientUtils$1$1.run(SecureClientUtils.java:99)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.atlas.security.SecureClientUtils$1.getHttpURLConnection(SecureClientUtils.java:99)
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler._invoke(URLConnectionClientHandler.java:159)
> at
> com.sun.jersey.client.urlconnection.URLConnectionClientHandler.handle(URLConnectionClientHandler.java:147)
> ... 9 more
> Caused by:
> org.apache.hadoop.security.authentication.client.AuthenticationException:
> Authentication failed, status: 401, message: Authentication required
> at
> org.apache.hadoop.security.authentication.client.AuthenticatedURL.extractToken(AuthenticatedURL.java:274)
> at
> org.apache.hadoop.security.authentication.client.PseudoAuthenticator.authenticate(PseudoAuthenticator.java:77)
> at
> org.apache.hadoop.security.token.delegation.web.DelegationTokenAuthenticator.authenticate(DelegationTokenAuthenticator.java:128)
> at
> org.apache.hadoop.security.authentication.client.AuthenticatedURL.openConnection(AuthenticatedURL.java:215)
> at
> org.apache.hadoop.security.token.del
[jira] [Updated] (ATLAS-56) atlas_config.py should give an informative error if jar or java binaries can't be found
[ https://issues.apache.org/jira/browse/ATLAS-56?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-56: -- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: (was: trunk) 0.6-incubating > atlas_config.py should give an informative error if jar or java binaries > can't be found > --- > > Key: ATLAS-56 > URL: https://issues.apache.org/jira/browse/ATLAS-56 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.6-incubating >Reporter: Aaron Dossett >Assignee: Aaron Dossett >Priority: Minor > Fix For: 0.6-incubating > > Attachments: ATLAS-56-v3.patch, ATLAS-56.patch, ATLAS-56_2.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-80) Support for variables in application properties
[ https://issues.apache.org/jira/browse/ATLAS-80?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-80: -- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: 0.6-incubating > Support for variables in application properties > --- > > Key: ATLAS-80 > URL: https://issues.apache.org/jira/browse/ATLAS-80 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Critical > Fix For: 0.6-incubating > > Attachments: ATLAS-80-v2.patch, ATLAS-80.patch > > > We use commons-configuration for application properties and 1.x that we use > doesn't support variables in properties. 2.x has support for variables, but > is in beta and is not production ready. > Meanwhile, we can support it using EL expression and substitute using system > properties. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-195) Document Hbase configs
[ https://issues.apache.org/jira/browse/ATLAS-195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-195: --- Fix Version/s: (was: trunk) 0.6-incubating > Document Hbase configs > -- > > Key: ATLAS-195 > URL: https://issues.apache.org/jira/browse/ATLAS-195 > Project: Atlas > Issue Type: Task >Affects Versions: 0.6-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-195.patch > > > Graph Persistence backend as hbase needs to be documented -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-25) Fix Atlas on Java 8
[ https://issues.apache.org/jira/browse/ATLAS-25?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-25: -- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: 0.6-incubating > Fix Atlas on Java 8 > --- > > Key: ATLAS-25 > URL: https://issues.apache.org/jira/browse/ATLAS-25 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: sandeep samudrala >Assignee: sandeep samudrala >Priority: Minor > Fix For: 0.6-incubating > > Attachments: ATLAS-25-V5.patch, ATLAS-25-v3.patch, ATLAS-25-v4.patch, > ATLAS-25-v6.patch, ATLAS-25-v7.patch, ATLAS-25.patch > > > 1) Hashing /indexing in Map is changed in java 8 and so unit tests doing > map.tostring are failing. > 2) a) With java 8 kerberos security for keytab has been moved to subject > (HADOOP-10786) and there by UserGroupInformation.isLoginKeytabBased() fails. > b) SecureEmbeddedServerIT fails for authentication which is fixed in > HADOOP-10786 client. > 3) Enunciate plugin is not supported with jdk8 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-350) Document jaas config details for atlas
[ https://issues.apache.org/jira/browse/ATLAS-350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-350: --- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: (was: 0.5-incubating) 0.6-incubating > Document jaas config details for atlas > -- > > Key: ATLAS-350 > URL: https://issues.apache.org/jira/browse/ATLAS-350 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: Ayub Khan >Assignee: Tom Beerbower > Fix For: 0.6-incubating > > Attachments: ATLAS-350.patch > > > When Atlas is configured with HBase as the storage backend in a secure > cluster, a JAAS configuration file should be created and specified so that > the HBase client can attempt to SASL-authenticate. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-92) import-hive.sh failed to find HiveMetaStoreBridge
[ https://issues.apache.org/jira/browse/ATLAS-92?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-92: -- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: (was: 0.5-incubating) 0.6-incubating > import-hive.sh failed to find HiveMetaStoreBridge > - > > Key: ATLAS-92 > URL: https://issues.apache.org/jira/browse/ATLAS-92 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: Chen He >Assignee: Chen He > Fix For: 0.6-incubating > > Attachments: ATLAS-92.000.patch > > > \>./bin/import-hive.sh > Error: Could not find or load main class > org.apache.atlas.hive.bridge.HiveMetaStoreBridge > Failed to import Hive Data Model!!! > Actually, HiveMetaStoreBridge class is in the hive-bridge-0.6.x.jar which is > under the ./hook/hive/ directory. To fix this is simple. We either change the > building process and put the HiveMetaStoreBridge into jars under > /bridge/hive , or import jars under ./hook/hive/ in the import-hive.sh -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-242) The qualified name for hive entities should be backward compatible
[
https://issues.apache.org/jira/browse/ATLAS-242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-242:
---
Fix Version/s: (was: trunk)
0.6-incubating
Affects Version/s: 0.6-incubating
> The qualified name for hive entities should be backward compatible
> --
>
> Key: ATLAS-242
> URL: https://issues.apache.org/jira/browse/ATLAS-242
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.6-incubating
>Reporter: David Kaspar
>Assignee: Shwetha G S
> Fix For: 0.6-incubating
>
> Attachments: ATLAS-242-v2.patch, ATLAS-242.patch
>
>
> This issue is filed for "RangerAtlasDemo_v4.ova" VM-image release.
> When I call:
> http://127.0.0.1:21000/api/atlas/lineage/hive/table/{tableName}/schema
> REST API method, then it always returns 404 Not Found and returns an error
> which claims that given {tableName} does not exist.
> Since REST API documentation (application.wadl) does not state details, I
> have tried to use existing Hive table names, existing GUIDs of Hive tables
> but none of them works.
> Similarly for /api/atlas/lineage/hive/table/{tableName}/outputs/graph and
> /api/atlas/lineage/hive/table/{tableName}/inputs/graph.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-335) Kerberized cluster: Atlas fails to come up with hbase as backend
[
https://issues.apache.org/jira/browse/ATLAS-335?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-335:
---
Fix Version/s: (was: trunk)
0.6-incubating
> Kerberized cluster: Atlas fails to come up with hbase as backend
>
>
> Key: ATLAS-335
> URL: https://issues.apache.org/jira/browse/ATLAS-335
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.6-incubating
>Reporter: Ayub Khan
>Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.6-incubating
>
> Attachments: ATLAS-335.patch
>
>
> With the secure cluster deployed using ambari, I tried following the steps
> mentioned in the below doc(setting hbase as storage backend) and looks like
> atlas is failing to come up with GSSException.
> From the below logs looks like "kinit"(authentication) is not done by ambari.
> Isn't this supposed to be done by ambari?
> {noformat}
> 2015-11-23 11:22:14,100 WARN - [hconnection-0x1b969687-shared--pool1-t1:] ~
> Exception encountered while connecting to the server :
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)] (AbstractRpcClient:699)
> 2015-11-23 11:22:14,100 FATAL - [hconnection-0x1b969687-shared--pool1-t1:] ~
> SASL authentication failed. The most likely cause is missing or invalid
> credentials. Consider 'kinit'. (AbstractRpcClient:709)
> javax.security.sasl.SaslException: GSS initiate failed [Caused by
> GSSException: No valid credentials provided (Mechanism level: Failed to find
> any Kerberos tgt)]
> at
> com.sun.security.sasl.gsskerb.GssKrb5Client.evaluateChallenge(GssKrb5Client.java:212)
> at
> org.apache.hadoop.hbase.security.HBaseSaslRpcClient.saslConnect(HBaseSaslRpcClient.java:179)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupSaslConnection(RpcClientImpl.java:642)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.access$600(RpcClientImpl.java:166)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:769)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection$2.run(RpcClientImpl.java:766)
> at java.security.AccessController.doPrivileged(Native Method)
> at javax.security.auth.Subject.doAs(Subject.java:415)
> at
> org.apache.hadoop.security.UserGroupInformation.doAs(UserGroupInformation.java:1657)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.setupIOstreams(RpcClientImpl.java:766)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.writeRequest(RpcClientImpl.java:920)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl$Connection.tracedWriteRequest(RpcClientImpl.java:889)
> at
> org.apache.hadoop.hbase.ipc.RpcClientImpl.call(RpcClientImpl.java:1222)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient.callBlockingMethod(AbstractRpcClient.java:213)
> at
> org.apache.hadoop.hbase.ipc.AbstractRpcClient$BlockingRpcChannelImplementation.callBlockingMethod(AbstractRpcClient.java:287)
> at
> org.apache.hadoop.hbase.protobuf.generated.ClientProtos$ClientService$BlockingStub.scan(ClientProtos.java:32651)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.openScanner(ScannerCallable.java:372)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:199)
> at
> org.apache.hadoop.hbase.client.ScannerCallable.call(ScannerCallable.java:62)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithoutRetries(RpcRetryingCaller.java:200)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:346)
> at
> org.apache.hadoop.hbase.client.ScannerCallableWithReplicas$RetryingRPC.call(ScannerCallableWithReplicas.java:320)
> at
> org.apache.hadoop.hbase.client.RpcRetryingCaller.callWithRetries(RpcRetryingCaller.java:126)
> at
> org.apache.hadoop.hbase.client.ResultBoundedCompletionService$QueueingFuture.run(ResultBoundedCompletionService.java:64)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> Caused by: GSSException: No valid credentia
[jira] [Updated] (ATLAS-367) Create branch-0.6-incubating
[ https://issues.apache.org/jira/browse/ATLAS-367?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-367: --- Assignee: Shwetha G S (was: Suma Shivaprasad) > Create branch-0.6-incubating > > > Key: ATLAS-367 > URL: https://issues.apache.org/jira/browse/ATLAS-367 > Project: Atlas > Issue Type: Sub-task >Reporter: Shwetha G S >Assignee: Shwetha G S > > Create branch-0.6-incubating and change project version to 0.6-incubating -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-37) atlas repository, webapp, hive-bridge tests fails with Hbase and Solr as Titan storage backend
[
https://issues.apache.org/jira/browse/ATLAS-37?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-37:
--
Fix Version/s: (was: 0.5.1-incubating)
0.6-incubating
Affects Version/s: (was: 0.5.1-incubating)
0.6-incubating
> atlas repository, webapp, hive-bridge tests fails with Hbase and Solr as
> Titan storage backend
> --
>
> Key: ATLAS-37
> URL: https://issues.apache.org/jira/browse/ATLAS-37
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.6-incubating
>Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
> Fix For: 0.6-incubating
>
> Attachments: ATLAS-37.1.patch, ATLAS-37.patch
>
>
> {noformat}
> Running org.apache.atlas.discovery.GraphBackedDiscoveryServiceTest
> Tests run: 45, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 170.609 sec
> - in org.apache.atlas.discovery.GraphBackedDiscoveryServiceTest
> Running org.apache.atlas.discovery.HiveLineageServiceTest
> Tests run: 45, Failures: 1, Errors: 0, Skipped: 44, Time elapsed: 533.15 sec
> <<< FAILURE! - in org.apache.atlas.discovery.HiveLineageServiceTest
> setUp(org.apache.atlas.discovery.HiveLineageServiceTest) Time elapsed:
> 533.067 sec <<< FAILURE!
> java.lang.IllegalArgumentException: The type of given name is not a key:
> View.inputTables
> at
> com.thinkaurelius.titan.graphdb.transaction.StandardTitanTx.getOrCreatePropertyKey(StandardTitanTx.java:906)
> at
> com.thinkaurelius.titan.graphdb.vertices.AbstractVertex.setProperty(AbstractVertex.java:239)
> at
> org.apache.atlas.repository.graph.GraphBackedMetadataRepository.addProperty(GraphBackedMetadataRepository.java:870)
> at
> org.apache.atlas.repository.graph.GraphBackedMetadataRepository.access$900(GraphBackedMetadataRepository.java:74)
> at
> org.apache.atlas.repository.graph.GraphBackedMetadataRepository$TypedInstanceToGraphMapper.mapArrayCollectionToVertex(GraphBackedMetadataRepository.java:710)
> at
> org.apache.atlas.repository.graph.GraphBackedMetadataRepository$TypedInstanceToGraphMapper.mapAttributesToVertex(GraphBackedMetadataRepository.java:660)
> at
> org.apache.atlas.repository.graph.GraphBackedMetadataRepository$TypedInstanceToGraphMapper.mapInstanceToVertex(GraphBackedMetadataRepository.java:633)
> at
> org.apache.atlas.repository.graph.GraphBackedMetadataRepository$TypedInstanceToGraphMapper.addDiscoveredInstances(GraphBackedMetadataRepository.java:607)
> at
> org.apache.atlas.repository.graph.GraphBackedMetadataRepository$TypedInstanceToGraphMapper.mapTypedInstanceToGraph(GraphBackedMetadataRepository.java:446)
> at
> org.apache.atlas.repository.graph.GraphBackedMetadataRepository$TypedInstanceToGraphMapper.access$100(GraphBackedMetadataRepository.java:432)
> at
> org.apache.atlas.repository.graph.GraphBackedMetadataRepository.createEntity(GraphBackedMetadataRepository.java:148)
> at
> org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:41)
> at
> org.apache.atlas.services.DefaultMetadataService.createEntity(DefaultMetadataService.java:231)
> at
> org.apache.atlas.discovery.HiveLineageServiceTest.createInstance(HiveLineageServiceTest.java:518)
> at
> org.apache.atlas.discovery.HiveLineageServiceTest.view(HiveLineageServiceTest.java:509)
> at
> org.apache.atlas.discovery.HiveLineageServiceTest.setupInstances(HiveLineageServiceTest.java:409)
> at
> org.apache.atlas.discovery.HiveLineageServiceTest.setUp(HiveLineageServiceTest.java:75)
> Running org.apache.atlas.query.BaseGremlinTest
> Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.201 sec -
> in org.apache.atlas.query.BaseGremlinTest
> Running org.apache.atlas.query.ExpressionTest
> Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.21 sec - in
> org.apache.atlas.query.ExpressionTest
> Running org.apache.atlas.query.GremlinTest
> Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.235 sec -
> in org.apache.atlas.query.GremlinTest
> Running org.apache.atlas.query.LexerTest
> Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.189 sec -
> in org.apache.atlas.query.LexerTest
> Running org.apache.atlas.query.LineageQueryTest
> Tests run: 0, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 0.216 sec -
> in org.apache.atlas.query.LineageQueryTest
> Running org.apache.atlas.query.ParserTest
> Tests run: 0, Failu
[jira] [Updated] (ATLAS-54) Rename configs in hive hook
[ https://issues.apache.org/jira/browse/ATLAS-54?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-54: -- Fix Version/s: (was: 0.5.1-incubating) 0.6-incubating > Rename configs in hive hook > --- > > Key: ATLAS-54 > URL: https://issues.apache.org/jira/browse/ATLAS-54 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.6-incubating > > Attachments: ATLAS-54.patch > > > The configs for hive hook are added in hive-site.xml and hive complains that > they are not recognised -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-31) ATLAS build fails with clean repo
[
https://issues.apache.org/jira/browse/ATLAS-31?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-31:
--
Fix Version/s: (was: trunk)
0.6-incubating
Affects Version/s: (was: 0.5-incubating)
0.6-incubating
> ATLAS build fails with clean repo
> -
>
> Key: ATLAS-31
> URL: https://issues.apache.org/jira/browse/ATLAS-31
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.6-incubating
> Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
> Fix For: 0.6-incubating
>
> Attachments: ATLAS-31.patch
>
>
> {noformat}
> [ERROR] Failed to execute goal
> org.apache.maven.plugins:maven-site-plugin:3.3:site (default) on project
> atlas-docs: SiteToolException: The site descriptor cannot be resolved from
> the repository: ArtifactResolutionException: Unable to locate site
> descriptor: Could not transfer artifact org.apache:apache:xml:site_en:13
> from/to Codehaus repository (http://repository.codehaus.org/): Failed to
> transfer file:
> http://repository.codehaus.org/org/apache/apache/13/apache-13-site_en.xml.
> Return code is: 410 , ReasonPhrase:Gone.
> [ERROR] org.apache:apache:xml:13
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-297) KafkaNotificationTest.testSendReceiveMessage fails when atlas-server is running on the same machine
[ https://issues.apache.org/jira/browse/ATLAS-297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-297: --- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: 0.6-incubating > KafkaNotificationTest.testSendReceiveMessage fails when atlas-server is > running on the same machine > > > Key: ATLAS-297 > URL: https://issues.apache.org/jira/browse/ATLAS-297 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating > Environment: Dev environment >Reporter: Hemanth Yamijala >Assignee: Hemanth Yamijala >Priority: Minor > Fix For: 0.6-incubating > > Attachments: ATLAS-297-1.patch, ATLAS-297-2.patch, ATLAS-297.patch > > > Start the atlas-server on local machine. Then execute mvn > -Dtest=KafkaNotificationTest -DfailIfNoTests=false test > Get the following stack trace and test failure: > Running org.apache.atlas.kafka.KafkaNotificationTest > Tests run: 3, Failures: 1, Errors: 0, Skipped: 2, Time elapsed: 1.262 sec <<< > FAILURE! - in org.apache.atlas.kafka.KafkaNotificationTest > setUp(org.apache.atlas.kafka.KafkaNotificationTest) Time elapsed: 0.88 sec > <<< FAILURE! > org.apache.atlas.AtlasException: Failed to start embedded kafka > at sun.nio.ch.Net.bind0(Native Method) > at sun.nio.ch.Net.bind(Net.java:463) > at sun.nio.ch.Net.bind(Net.java:455) > at > sun.nio.ch.ServerSocketChannelImpl.bind(ServerSocketChannelImpl.java:223) > at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:74) > at sun.nio.ch.ServerSocketAdaptor.bind(ServerSocketAdaptor.java:67) > at > org.apache.zookeeper.server.NIOServerCnxnFactory.configure(NIOServerCnxnFactory.java:95) > at > org.apache.zookeeper.server.ServerCnxnFactory.createFactory(ServerCnxnFactory.java:126) > at > org.apache.atlas.kafka.KafkaNotification.startZk(KafkaNotification.java:140) > at > org.apache.atlas.kafka.KafkaNotification.start(KafkaNotification.java:170) > at > org.apache.atlas.kafka.KafkaNotificationTest.setUp(KafkaNotificationTest.java:40) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-74) Create notification framework
[ https://issues.apache.org/jira/browse/ATLAS-74?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-74: -- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: 0.6-incubating > Create notification framework > - > > Key: ATLAS-74 > URL: https://issues.apache.org/jira/browse/ATLAS-74 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.6-incubating >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.6-incubating > > Attachments: ATLAS-74-v2.patch, ATLAS-74-v3.patch, ATLAS-74.patch > > > Add notification framework with kafka as default implementation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-344) Document HBase permissions for secure cluster
[ https://issues.apache.org/jira/browse/ATLAS-344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-344: --- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: 0.6-incubating > Document HBase permissions for secure cluster > - > > Key: ATLAS-344 > URL: https://issues.apache.org/jira/browse/ATLAS-344 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: Tom Beerbower >Assignee: Tom Beerbower > Fix For: 0.6-incubating > > Attachments: ATLAS-344.patch > > > In a secure cluster with HBase configured for the storage backend, titan > needs permissions set to create / access the hbase table 'titan'. > ​ > With Ranger: Policy has to be configured for 'titan'. > Without Ranger: ACL has to be configured for the user, in this case atlas. > Without the permissions set, an "Insufficient user permission" exception is > generated. > Document this and the procedure to set the permissions in the Atlas docs. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-196) Fix solr documentation
[ https://issues.apache.org/jira/browse/ATLAS-196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-196: --- Fix Version/s: (was: trunk) 0.6-incubating > Fix solr documentation > -- > > Key: ATLAS-196 > URL: https://issues.apache.org/jira/browse/ATLAS-196 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-196.1.patch, ATLAS-196.patch > > > Configuration section doesnt have solr configs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-194) Thread pool in hive hook should be static
[ https://issues.apache.org/jira/browse/ATLAS-194?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-194: --- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: 0.6-incubating > Thread pool in hive hook should be static > - > > Key: ATLAS-194 > URL: https://issues.apache.org/jira/browse/ATLAS-194 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Blocker > Fix For: 0.6-incubating > > Attachments: ATLAS-194.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-95) import-hive.sh reports illegal java parameters
[ https://issues.apache.org/jira/browse/ATLAS-95?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-95: -- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: (was: 0.5-incubating) 0.6-incubating > import-hive.sh reports illegal java parameters > -- > > Key: ATLAS-95 > URL: https://issues.apache.org/jira/browse/ATLAS-95 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: Chen He >Assignee: Chen He > Fix For: 0.6-incubating > > Attachments: ATLAS-95.000.patch > > > [root@hostname ~]# > /scratch/atlas/apache-atlas-0.6-incubating-SNAPSHOT/bin/import-hive.sh > Using Hive configuration directory [/etc/hive/conf] > Logs for import are in > /scratch/atlas/apache-atlas-0.6-incubating-SNAPSHOT/logs/import-hive.log > Unrecognized option: > -atlas.log.dir=/scratch/atlas/apache-atlas-0.6-incubating-SNAPSHOT/logs > Error: Could not create the Java Virtual Machine. > Error: A fatal exception has occurred. Program will exit. > Failed to import Hive Data Model!!! > There should be a "D" before "atlas.log.dir". Patch will be attached. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-134) Some defects found when reviewing the source code.
[
https://issues.apache.org/jira/browse/ATLAS-134?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-134:
---
Fix Version/s: (was: trunk)
0.6-incubating
Affects Version/s: (was: trunk)
0.6-incubating
> Some defects found when reviewing the source code.
> --
>
> Key: ATLAS-134
> URL: https://issues.apache.org/jira/browse/ATLAS-134
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.6-incubating
>Reporter: liutongfeng
>Assignee: liutongfeng
>Priority: Minor
> Labels: patch
> Fix For: 0.6-incubating
>
> Attachments: ATLAS-134.patch
>
>
> 1.The second getKeyType() should be getValueType().
> file : ObjectGraphTraversal.java
> line : 56
> IDataType keyType = ((DataTypes.MapType) dT).getKeyType();
> IDataType valueType = ((DataTypes.MapType) dT).getKeyType();
> processMap(keyType, valueType, val);
> 2.The second getKeyType() should be getValueType().
> file : ObjectGraphWalker.java
> line : 94
> IDataType keyType = ((DataTypes.MapType) dT).getKeyType();
> IDataType valueType = ((DataTypes.MapType) dT).getKeyType();
> visitMap(keyType, valueType, val);
> 3.When OS is WINDOWS, '/' should be '\\', otherwise the test will fail.
> file : TestMetadata.py
> line : 52
> if IS_WINDOWS:
> java_mock.assert_called_with(
> 'org.apache.atlas.Main',
> ['-app', 'metadata_home/server/webapp/atlas'],
>
> 'metadata_home/conf:metadata_home/server/webapp/atlas/WEB-INF/classes:metadata_home/server/webapp/atlas/WEB-INF/lib\\*:metadata_home/libext\\*',
> ['-Datlas.log.dir=metadata_home/logs',
> '-Datlas.log.file=application.log', '-Datlas.home=metadata_home',
> '-Datlas.conf=metadata_home/conf', '-Xmx1024m',
> '-Dlog4j.configuration=atlas-log4j.xml'], 'metadata_home/logs')
> else:
> 4.Should use return.
> file :
> incubator-atlas\repository\src\main\java\org\apache\atlas\repository\memory\AttributeStores.java
> line : 58
> static IAttributeStore createStore(AttributeInfo i) throws
> RepositoryException {
> switch (i.dataType().getTypeCategory()) {
> case PRIMITIVE:
> if (i.dataType() == DataTypes.BOOLEAN_TYPE) {
> return new BooleanAttributeStore(i);
> } else if (i.dataType() == DataTypes.BYTE_TYPE) {
> return new ByteAttributeStore(i);
> } else if (i.dataType() == DataTypes.SHORT_TYPE) {
> new ShortAttributeStore(i); // should be return
> } else if (i.dataType() == DataTypes.INT_TYPE) {
> return new IntAttributeStore(i);
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-201) Rename org.apache.atlas.Main to org.apache.atlas.Atlas
[ https://issues.apache.org/jira/browse/ATLAS-201?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-201: --- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: 0.6-incubating > Rename org.apache.atlas.Main to org.apache.atlas.Atlas > -- > > Key: ATLAS-201 > URL: https://issues.apache.org/jira/browse/ATLAS-201 > Project: Atlas > Issue Type: Task >Affects Versions: 0.6-incubating >Reporter: Shwetha G S >Assignee: Rishabh Bhardwaj >Priority: Minor > Fix For: 0.6-incubating > > Attachments: ATLAS-201-v2.patch, ATLAS-201.patch > > > Rename org.apache.atlas.Main to org.apache.atlas.Atlas so that jps command > shows process running as Atlas -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-198) Atlas UI Requires Internet Access
[ https://issues.apache.org/jira/browse/ATLAS-198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-198: --- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: (was: 0.5-incubating) > Atlas UI Requires Internet Access > - > > Key: ATLAS-198 > URL: https://issues.apache.org/jira/browse/ATLAS-198 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: Erik Bergenholtz >Assignee: Sanjay Patel >Priority: Blocker > Labels: ATLAS-UI-BUGS > Fix For: 0.6-incubating > > Attachments: 0.5_ATLAS-198_v1.patch, 0.6_ATLAS-198_v1.patch, > 0.6_ATLAS-198_v2.patch > > > PROBLEM: Web resources are being sourced from the internet for the Atlas web > application. Installations do not necessarily have internet access. This > results in a timeout for the requested resources. These resources need to be > moved into atlas.war so the resources can be served up by the Atlas web > application. > The list needs to be verified, but includes at least: > http://maxcdn.bootstrapcdn.com/bootstrap/3.3.4/css/bootstrap.min.css > http://cdnjs.cloudflare.com/ajax/libs/d3/3.5.5/d3.min.js > The fix needs to bundle resources such that Atlas web-app can be utilized > without internet connectivity. > The issue needs to be fixed in master and requires patch for 0.5 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-232) Fix the API incompatibility introduced in ATLAS-58
[ https://issues.apache.org/jira/browse/ATLAS-232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-232: --- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: 0.6-incubating > Fix the API incompatibility introduced in ATLAS-58 > -- > > Key: ATLAS-232 > URL: https://issues.apache.org/jira/browse/ATLAS-232 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.6-incubating > > Attachments: ATLAS-232-amend.patch, ATLAS-232.patch > > > 1. Entity create should handle with single entity json or jsonarray > 2. Move the APIs from entities/entity -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-168) Atlas UI - Max column in hive 4 ?
[ https://issues.apache.org/jira/browse/ATLAS-168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-168: --- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: 0.6-incubating > Atlas UI - Max column in hive 4 ? > - > > Key: ATLAS-168 > URL: https://issues.apache.org/jira/browse/ATLAS-168 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: Darshan Kumar >Assignee: Darshan Kumar > Labels: patch-available > Fix For: 0.6-incubating > > Attachments: ATLAS-168-v1.patch > > > what version are they using? And what do we mean by show? In the UI or the > rest api’s do not work? > We have a test where the table has a lot more than 4 columns but we only test > api’s. > https://github.com/hortonworks/certification/blob/nightly-branch/HDPTests/tests/atlas/regression/src/test/java/org/apache/atlas/regression/tests/HiveIntegrationCreateTableTest.java#L93 > Meeting with waterline – they say that they cannot show schema with more than > 4 column. Actually this seems to be a real bug. Can someone from eng > confirm this was not a sandbox artifact ? -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-180) Cleanup atlas doc packaging
[ https://issues.apache.org/jira/browse/ATLAS-180?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-180: --- Fix Version/s: (was: trunk) 0.6-incubating Affects Version/s: 0.6-incubating > Cleanup atlas doc packaging > --- > > Key: ATLAS-180 > URL: https://issues.apache.org/jira/browse/ATLAS-180 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.6-incubating > > Attachments: ATLAS-180-v2.patch, ATLAS-180.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-368) Change trunk version to 0.7-incubating
[ https://issues.apache.org/jira/browse/ATLAS-368?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-368: --- Attachment: ATLAS-368.patch > Change trunk version to 0.7-incubating > -- > > Key: ATLAS-368 > URL: https://issues.apache.org/jira/browse/ATLAS-368 > Project: Atlas > Issue Type: Sub-task >Affects Versions: trunk >Reporter: Shwetha G S >Assignee: Suma Shivaprasad > Fix For: trunk > > Attachments: ATLAS-368.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-361) Add validation when index backends are switched in ATLAS configuration
[ https://issues.apache.org/jira/browse/ATLAS-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-361: --- Attachment: ATLAS-361.3.patch Fixed review comments > Add validation when index backends are switched in ATLAS configuration > -- > > Key: ATLAS-361 > URL: https://issues.apache.org/jira/browse/ATLAS-361 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-361.1.patch, ATLAS-361.2.patch, ATLAS-361.3.patch, > ATLAS-361.patch > > > Currently, the index backend configs can be changed y user but it ignored by > Titan since it can have bad repercussions i.e index data loss. > Hence we need to validate that the configuration has been switched and throw > and error during start up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-359) Unable to store date arrays
[
https://issues.apache.org/jira/browse/ATLAS-359?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15038005#comment-15038005
]
Suma Shivaprasad commented on ATLAS-359:
[~dkantor] Is this an issue with the Date array or the creation and
modification timestamps?
> Unable to store date arrays
> ---
>
> Key: ATLAS-359
> URL: https://issues.apache.org/jira/browse/ATLAS-359
> Project: Atlas
> Issue Type: Bug
>Reporter: David Kantor
>Priority: Minor
>
> POST http://localhost:21000/api/atlas/entities
> json:
> [{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"-787535393397482","version":0,"typeName":"TypesPackage_Types"},"typeName":"TypesPackage_Types","values":{"intArrayAttr":[-2147483648,0,2147483647],"omCreationTimestamp":"2015-12-02T21:45:38.675Z","shortArrayAttr":[-32768,0,32767],"omOptimisticLockId":0,"booleanArrayAttr":[true,false,true],"omModificationTimestamp":"2015-12-02T21:45:38.674Z","byteArrayAttr":[-128,0,127],"bigDecimalArrayAttr":[1,0,10],"omModifiedByUser":"oms","stringArrayAttr":["item
> 1","item 2","item
> 2"],"dateArrayAttr":["2015-12-02T21:45:33.245Z","2015-12-02T21:45:33.245Z","2015-12-02T21:45:33.245Z"],"bigIntegerArrayAttr":[1,10,10],"omCreatedByUser":"oms","longArrayAttr":[-9223372036854775808,0,9223372036854775807],"floatArrayAttr":[1.4E-45,0,127],"doubleArrayAttr":[4.9E-324,0,1.7976931348623157E308]},"traitNames":[],"traits":{}}]
> This results in the following message appearing in the Atlas application.log.
> org.apache.atlas.repository.RepositoryException:
> org.apache.atlas.typesystem.types.ValueConversionException: Cannot convert
> value 'Wed Dec 02 15:45:33 CST 2015' to datatype date
> at
> org.apache.atlas.repository.graph.GraphBackedMetadataRepository.createEntities(GraphBackedMetadataRepository.java:158)
> at
> org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:42)
> at
> org.apache.atlas.services.DefaultMetadataService.createEntities(DefaultMetadataService.java:251)
> at
> org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:113)
> at sun.reflect.GeneratedMethodAccessor37.invoke(Unknown Source)
> at
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke(Method.java:602)
> ...
> Caused by: org.apache.atlas.typesystem.types.ValueConversionException: Cannot
> convert value 'Wed Dec 02 15:45:33 CST 2015' to datatype date
> at
> org.apache.atlas.typesystem.types.DataTypes$DateType.convert(DataTypes.java:430)
> at
> org.apache.atlas.typesystem.types.DataTypes$DateType.convert(DataTypes.java:409)
> at
> org.apache.atlas.typesystem.types.DataTypes$ArrayType.convert(DataTypes.java:514)
> at
> org.apache.atlas.typesystem.types.DataTypes$ArrayType.convert(DataTypes.java:477)
> at
> org.apache.atlas.typesystem.persistence.StructInstance.set(StructInstance.java:119)
> at
> org.apache.atlas.repository.graph.GraphBackedMetadataRepository$GraphToTypedInstanceMapper.mapVertexToArrayInstance(GraphBackedMetadataRepository.java:1068)
> at
> org.apache.atlas.repository.graph.GraphBackedMetadataRepository$GraphToTypedInstanceMapper.mapVertexToAttribute(GraphBackedMetadataRepository.java:998)
> at
> org.apache.atlas.repository.graph.GraphBackedMetadataRepository$GraphToTypedInstanceMapper.mapVertexToInstance(GraphBackedMetadataRepository.java:971)
> at
> org.apache.atlas.repository.graph.GraphBackedMetadataRepository$GraphToTypedInstanceMapper.mapGraphToTypedInstance(GraphBackedMetadataRepository.java:950)
> at
> org.apache.atlas.repository.graph.GraphBackedMetadataRepository$TypedInstanceToGraphMapper.getFullTextForVertex(GraphBackedMetadataRepository.java:522)
> at
> org.apache.atlas.repository.graph.GraphBackedMetadataRepository$TypedInstanceToGraphMapper.addFullTextProperty(GraphBackedMetadataRepository.java:515)
> at
> org.apache.atlas.repository.graph.GraphBackedMetadataRepository$TypedInstanceToGraphMapper.mapTypedInstanceToGraph(GraphBackedMetadataRepository.java:499)
>
[jira] [Updated] (ATLAS-361) Add validation when index backends are switched in ATLAS configuration
[ https://issues.apache.org/jira/browse/ATLAS-361?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-361: --- Attachment: ATLAS-361.2.patch Fixed docs > Add validation when index backends are switched in ATLAS configuration > -- > > Key: ATLAS-361 > URL: https://issues.apache.org/jira/browse/ATLAS-361 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-361.1.patch, ATLAS-361.2.patch, ATLAS-361.patch > > > Currently, the index backend configs can be changed y user but it ignored by > Titan since it can have bad repercussions i.e index data loss. > Hence we need to validate that the configuration has been switched and throw > and error during start up. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-352) Fix performance issues with type and entity creation with Hbase as storage backend
[ https://issues.apache.org/jira/browse/ATLAS-352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-352: --- Description: Current perf microbenchmark measurements DataSet quickstart model and entity definitions No of class types 6 traits 6 with no attributes server start up time with existing schema 17 secs type batch creation 27 secs entity creation - DB12 secs entity creation - Table 11 secs was: DataSet quickstart model and entity definitions No of class types 6 traits 6 with no attributes server start up time with existing schema 17 secs type batch creation 27 secs entity creation - DB12 secs entity creation - Table 11 secs > Fix performance issues with type and entity creation with Hbase as storage > backend > --- > > Key: ATLAS-352 > URL: https://issues.apache.org/jira/browse/ATLAS-352 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.6-incubating >Reporter: Suma Shivaprasad > Assignee: Suma Shivaprasad > > Current perf microbenchmark measurements > DataSet quickstart model and entity definitions > No of class types 6 > traits6 with no attributes > server start up time with existing schema 17 secs > type batch creation 27 secs > entity creation - DB 12 secs > entity creation - Table 11 secs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-352) Fix performance issues with type and entity creation with Hbase as storage backend
[ https://issues.apache.org/jira/browse/ATLAS-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15034074#comment-15034074 ] Suma Shivaprasad commented on ATLAS-352: On investigating, it was found that Titan holds locks at 2 levels - one at a local level among threads and another at a global level as a distributed lock among titan instances. The disrtibuted lock is maintained for every KeyColumn that its trying to mutate in Hbase and it ends up doing a lot of writes for locking and lot of reads to check if the current local original values before the mutation began for the keyColumn are matching with the ones during transaction commit. If it doesnt match, the transaction is failed. As for Atlas , since it operates only a single instance of Titan currently, the locks can be maintained only at a local level among different threads in the same JVM(Atlas) > Fix performance issues with type and entity creation with Hbase as storage > backend > --- > > Key: ATLAS-352 > URL: https://issues.apache.org/jira/browse/ATLAS-352 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.6-incubating >Reporter: Suma Shivaprasad > Assignee: Suma Shivaprasad > > Current perf microbenchmark measurements > DataSet quickstart model and entity definitions > No of class types 6 > traits6 with no attributes > server start up time with existing schema 17 secs > type batch creation 27 secs > entity creation - DB 12 secs > entity creation - Table 11 secs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-352) Fix performance issues with type and entity creation with Hbase as storage backend
[ https://issues.apache.org/jira/browse/ATLAS-352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-352: --- Affects Version/s: 0.6-incubating > Fix performance issues with type and entity creation with Hbase as storage > backend > --- > > Key: ATLAS-352 > URL: https://issues.apache.org/jira/browse/ATLAS-352 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.6-incubating >Reporter: Suma Shivaprasad > Assignee: Suma Shivaprasad > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-352) Fix performance issues with type and entity creation with Hbase as storage backend
[ https://issues.apache.org/jira/browse/ATLAS-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15034134#comment-15034134 ] Suma Shivaprasad commented on ATLAS-352: perf numbers after applying the patch type batch creation 5 secs entity creation - DB1.2 sec entity creation - Table 2.2 secs There is almost a 10x reduction in latency for a single request > Fix performance issues with type and entity creation with Hbase as storage > backend > --- > > Key: ATLAS-352 > URL: https://issues.apache.org/jira/browse/ATLAS-352 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.6-incubating >Reporter: Suma Shivaprasad > Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-352.patch > > > Current perf microbenchmark measurements > DataSet quickstart model and entity definitions > No of class types 6 > traits6 with no attributes > server start up time with existing schema 17 secs > type batch creation 27 secs > entity creation - DB 12 secs > entity creation - Table 11 secs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-352) Fix performance issues with type and entity creation with Hbase as storage backend
Suma Shivaprasad created ATLAS-352: -- Summary: Fix performance issues with type and entity creation with Hbase as storage backend Key: ATLAS-352 URL: https://issues.apache.org/jira/browse/ATLAS-352 Project: Atlas Issue Type: Improvement Reporter: Suma Shivaprasad Assignee: Suma Shivaprasad -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-352) Fix performance issues with type and entity creation with Hbase as storage backend
[ https://issues.apache.org/jira/browse/ATLAS-352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-352: --- Attachment: ATLAS-352.patch > Fix performance issues with type and entity creation with Hbase as storage > backend > --- > > Key: ATLAS-352 > URL: https://issues.apache.org/jira/browse/ATLAS-352 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.6-incubating >Reporter: Suma Shivaprasad > Assignee: Suma Shivaprasad > Attachments: ATLAS-352.patch > > > Current perf microbenchmark measurements > DataSet quickstart model and entity definitions > No of class types 6 > traits6 with no attributes > server start up time with existing schema 17 secs > type batch creation 27 secs > entity creation - DB 12 secs > entity creation - Table 11 secs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-352) Fix performance issues with type and entity creation with Hbase as storage backend
[ https://issues.apache.org/jira/browse/ATLAS-352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15034167#comment-15034167 ] Suma Shivaprasad commented on ATLAS-352: Also added ENCODING on all column families which is set to FAST_DIFF - Refer http://hbase.apache.org/book.html#compression > Fix performance issues with type and entity creation with Hbase as storage > backend > --- > > Key: ATLAS-352 > URL: https://issues.apache.org/jira/browse/ATLAS-352 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.6-incubating >Reporter: Suma Shivaprasad > Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-352.patch > > > Current perf microbenchmark measurements > DataSet quickstart model and entity definitions > No of class types 6 > traits6 with no attributes > server start up time with existing schema 17 secs > type batch creation 27 secs > entity creation - DB 12 secs > entity creation - Table 11 secs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-352) Fix performance issues with type and entity creation with Hbase as storage backend
[ https://issues.apache.org/jira/browse/ATLAS-352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-352: --- Fix Version/s: 0.6-incubating > Fix performance issues with type and entity creation with Hbase as storage > backend > --- > > Key: ATLAS-352 > URL: https://issues.apache.org/jira/browse/ATLAS-352 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.6-incubating >Reporter: Suma Shivaprasad > Assignee: Suma Shivaprasad > Fix For: 0.6-incubating > > Attachments: ATLAS-352.patch > > > Current perf microbenchmark measurements > DataSet quickstart model and entity definitions > No of class types 6 > traits6 with no attributes > server start up time with existing schema 17 secs > type batch creation 27 secs > entity creation - DB 12 secs > entity creation - Table 11 secs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-244) UI: Add Tag Tab
[ https://issues.apache.org/jira/browse/ATLAS-244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad resolved ATLAS-244. Resolution: Fixed Fix Version/s: trunk > UI: Add Tag Tab > --- > > Key: ATLAS-244 > URL: https://issues.apache.org/jira/browse/ATLAS-244 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.6-incubating >Reporter: Erik Bergenholtz >Assignee: Darshan Kumar > Fix For: 0.6-incubating, trunk > > Attachments: ATLAS-244-v1.patch > > > The detailed entity screen needs to add a Tag tab. See: > https://monosnap.com/file/en3YL2ARN111d9UDlCozZpBnUhkPwv for additional > details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-244) UI: Add Tag Tab
[ https://issues.apache.org/jira/browse/ATLAS-244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15052656#comment-15052656 ] Suma Shivaprasad commented on ATLAS-244: Committed to trunk . Thanks [~darshankumar1989] and [~yhemanth] > UI: Add Tag Tab > --- > > Key: ATLAS-244 > URL: https://issues.apache.org/jira/browse/ATLAS-244 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.6-incubating >Reporter: Erik Bergenholtz >Assignee: Darshan Kumar > Fix For: 0.6-incubating, trunk > > Attachments: ATLAS-244-v1.patch > > > The detailed entity screen needs to add a Tag tab. See: > https://monosnap.com/file/en3YL2ARN111d9UDlCozZpBnUhkPwv for additional > details. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-382) Fixed Hive Bridge doc for ATLAS cluster name
Suma Shivaprasad created ATLAS-382: -- Summary: Fixed Hive Bridge doc for ATLAS cluster name Key: ATLAS-382 URL: https://issues.apache.org/jira/browse/ATLAS-382 Project: Atlas Issue Type: Bug Reporter: Suma Shivaprasad ---++ Importing Hive Metadata org.apache.atlas.hive.bridge.HiveMetaStoreBridge imports the hive metadata into Atlas using the model defined in org.apache.atlas.hive.model.HiveDataModelGenerator. import-hive.sh command can be used to facilitate this. Set the following configuration in /client.properties and set environment variable HIVE_CONFIG to the hive conf directory: atlas.cluster.name primary THis should instead mention $HIVE_CONFIG/hive-site.xml instead of client.properties -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-20) Rename env variables from METADATA to ATLAS
[ https://issues.apache.org/jira/browse/ATLAS-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15050130#comment-15050130 ] Suma Shivaprasad commented on ATLAS-20: --- +1 > Rename env variables from METADATA to ATLAS > --- > > Key: ATLAS-20 > URL: https://issues.apache.org/jira/browse/ATLAS-20 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.5-incubating >Reporter: Venkatesh Seetharam >Assignee: Rishabh Bhardwaj > Attachments: ATLAS-20.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-382) Fixed Hive Bridge doc for ATLAS cluster name
[ https://issues.apache.org/jira/browse/ATLAS-382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-382: --- Attachment: ATLAS-382.patch > Fixed Hive Bridge doc for ATLAS cluster name > > > Key: ATLAS-382 > URL: https://issues.apache.org/jira/browse/ATLAS-382 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating, trunk > Reporter: Suma Shivaprasad > Fix For: 0.6-incubating, trunk > > Attachments: ATLAS-382.patch > > > ---++ Importing Hive Metadata > org.apache.atlas.hive.bridge.HiveMetaStoreBridge imports the hive metadata > into Atlas using the model defined in > org.apache.atlas.hive.model.HiveDataModelGenerator. import-hive.sh command > can be used to facilitate this. > Set the following configuration in /client.properties and set > environment variable HIVE_CONFIG to the hive conf directory: > > > atlas.cluster.name > primary > > > THis should instead mention $HIVE_CONFIG/hive-site.xml instead of > client.properties -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-182) Add data model for Storm topology elements
[ https://issues.apache.org/jira/browse/ATLAS-182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055902#comment-15055902 ] Suma Shivaprasad edited comment on ATLAS-182 at 12/14/15 12:45 PM: --- Initial review comments. Thanks to [~yhemanth] for the review 1. pom.xml - The dependencies could be removed in storm hook pom since they are being added by parent pom already + +org.slf4j +slf4j-api + + + +org.slf4j +slf4j-log4j12 + 2. pom.xml - httpConnector port needs to be changed to 31000 and stop port to 310001 - Pls refer https://github.com/apache/incubator-atlas/blob/master/webapp/pom.xml 3.Whats the use of "endTime" attribute in "Topology" . Should we removed "endTime" - Didnt see it geting used anywhere? 4.Topology "id" is used to indicate a run id or instance id ? Didnt understand why we need to capture lineage between two "DataSet"s across different runs of the same Topology? We could just capture it a Topology level and leave out the "instance" id? 5. Is there anything in the Topology conf that is of interest/searchable since "conf" ~ (map(string, string), optional) - could be huge for a Storm topology? 6. "name" attribute could be removed in KAFKA, HBase, HDFS and JMS Data Set since its already part of "DataSet" 7. Need to document that Hive Data Model needs to be created before the Storm Data Model 8. JMS_TOPIC can be removed since its not getting used in the Hook ? 9. HBASE_TABLE and HDFS_DATA_SET could be renamed to STORM_SINK_HBASE_TABLE and STORM_SINK_HDFS_PATH since we are planning to have a generic model for these anyways and will conflict with the names and maybe the model also then? This will also need a migration story when we have the generic models. 10. We should also add "clusterName" to kafka topic, hdfs and hbase path was (Author: suma.shivaprasad): Initial review comments 1. pom.xml - The dependencies could be removed in storm hook pom since they are being added by parent pom already + +org.slf4j +slf4j-api + + + +org.slf4j +slf4j-log4j12 + 2. pom.xml - httpConnector port needs to be changed to 31000 and stop port to 310001 - Pls refer https://github.com/apache/incubator-atlas/blob/master/webapp/pom.xml 3.Whats the use of "endTime" attribute in "Topology" . Should we removed "endTime" - Didnt see it geting used anywhere? 4.Topology "id" is used to indicate a run id or instance id ? Didnt understand why we need to capture lineage between two "DataSet"s across different runs of the same Topology? We could just capture it a Topology level and leave out the "instance" id? 5. Is there anything in the Topology conf that is of interest/searchable since "conf" ~ (map(string, string), optional) - could be huge for a Storm topology? 6. "name" attribute could be removed in KAFKA, HBase, HDFS and JMS Data Set since its already part of "DataSet" 7. Need to document that Hive Data Model needs to be created before the Storm Data Model 8. JMS_TOPIC can be removed since its not getting used in the Hook ? 9. HBASE_TABLE and HDFS_DATA_SET could be renamed to STORM_SINK_HBASE_TABLE and STORM_SINK_HDFS_PATH since we are planning to have a generic model for these anyways and will conflict with the names and maybe the model also then? This will also need a migration story when we have the generic models. 10. We should also add "clusterName" to kafka topic, hdfs and hbase path > Add data model for Storm topology elements > -- > > Key: ATLAS-182 > URL: https://issues.apache.org/jira/browse/ATLAS-182 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.6-incubating >Reporter: Venkatesh Seetharam >Assignee: Venkatesh Seetharam > Fix For: 0.6-incubating > > Attachments: ATLAS-182-v1.patch, ATLAS-182.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-386) Handle hive rename commands
[ https://issues.apache.org/jira/browse/ATLAS-386?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055873#comment-15055873 ] Suma Shivaprasad commented on ATLAS-386: +1 > Handle hive rename commands > --- > > Key: ATLAS-386 > URL: https://issues.apache.org/jira/browse/ATLAS-386 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: trunk > > Attachments: ATLAS-386-v2.patch, ATLAS-386.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-20) Rename env variables from METADATA to ATLAS
[ https://issues.apache.org/jira/browse/ATLAS-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055953#comment-15055953 ] Suma Shivaprasad edited comment on ATLAS-20 at 12/14/15 1:06 PM: - +1. [~rishabhbhardwaj], have you tested the new patch as requested above? was (Author: suma.shivaprasad): +1. [~rishabhagindore], have you tested the new patch as requested above? > Rename env variables from METADATA to ATLAS > --- > > Key: ATLAS-20 > URL: https://issues.apache.org/jira/browse/ATLAS-20 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.5-incubating >Reporter: Venkatesh Seetharam >Assignee: Rishabh Bhardwaj > Attachments: ATLAS-20.patch, ATLAS-20.v2.patch, ATLAS-20.v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-20) Rename env variables from METADATA to ATLAS
[ https://issues.apache.org/jira/browse/ATLAS-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055953#comment-15055953 ] Suma Shivaprasad commented on ATLAS-20: --- +1. @Rishabh, have you tested the new patch as requested above? > Rename env variables from METADATA to ATLAS > --- > > Key: ATLAS-20 > URL: https://issues.apache.org/jira/browse/ATLAS-20 > Project: Atlas > Issue Type: Improvement >Affects Versions: 0.5-incubating >Reporter: Venkatesh Seetharam >Assignee: Rishabh Bhardwaj > Attachments: ATLAS-20.patch, ATLAS-20.v2.patch, ATLAS-20.v3.patch > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-183) Add a Hook in Storm to post the topology metadata
[
https://issues.apache.org/jira/browse/ATLAS-183?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15055945#comment-15055945
]
Suma Shivaprasad commented on ATLAS-183:
Initial review comments. Thanks [~yhemanth] for reviewing
1. dataSetReferenceable.set("owner", "storm.kerberos.principal"); - needs to be
changed to read value from conf
2. What does HdfsBolt.rotationActions indicate ?
3. hive db, column, table qualifiedName have been changed recently to have
@clusterName - Pls refer HiveMetaStoreBridge.
4. KafkaSpout - "name" should be unique since it will be used to dedup the
entity for updates. Currently its set to "KafkaSpout" . Similarly for
"HBaseBolt" and "HDFSBolt" etc
5. What if users havent their own spout/bolt implementations for Kafka spout,
HDFS bolt etc? What "name" will be provided by Storm API then?
> Add a Hook in Storm to post the topology metadata
> -
>
> Key: ATLAS-183
> URL: https://issues.apache.org/jira/browse/ATLAS-183
> Project: Atlas
> Issue Type: Sub-task
>Affects Versions: 0.6-incubating
>Reporter: Venkatesh Seetharam
> Fix For: 0.6-incubating
>
> Attachments: ATLAS-183.patch
>
>
> Apache Storm Integration with Apache Atlas (incubating)
> Introduction
> Apache Storm is a distributed real-time computation system. Storm makes it
> easy to reliably process unbounded streams of data, doing for real-time
> processing what Hadoop did for batch processing. The process is essentially
> a DAG of nodes, which is called topology.
> Apache Atlas is a metadata repository that enables end-to-end data lineage,
> search and associate business classification.
> Overview
> The goal of this integration is to at minimum push the operational topology
> metadata along with the underlying data source(s), target(s), derivation
> processes and any available business context so Atlas can capture the lineage
> for this topology.
> It would also help to support custom user annotations per node in the
> topology.
> There are 2 parts in this process detailed below:
> Data model to represent the concepts in Storm
> Storm Bridge to update metadata in Atlas
> Data Model
> A data model is represented as a Type in Atlas. It contains the descriptions
> of various nodes in the DAG, such as spouts and bolts and the corresponding
> source and target types. These need to be expressed as Types in Atlas type
> system. At the least, we need to create types for:
> Storm topology containing spouts, bolts, etc. with associations between them
> Source (typically Kafka, etc.)
> Target (typically Hive, HBase, HDFS, etc.)
> You can take a look at the data model code for Hive. Storm should only be
> simpler than Hive from a data modeling perspective.
> Pushing Metadata into Atlas
> There are 2 parts to the bridge:
> Storm Bridge
> This is a one-time import for Storm to list all the active topologies and
> push the metadata into Atlas to address cases where Storm deployments exist
> before Atlas.
> You can refer to the bridge code for Hive.
> Post-execution Hook
> Atlas needs to be notified when a new topology is registered successfully in
> Storm or when someone changes the definition of an existing topology.
> You can refer to the hook code for Hive.
>
> Example use case:
> Custom annotations associated with each node in the topology.
> For example: Data Quality Rules, Error Handling, etc. A set of annotations
> that enumerates rules handling nulls– all nulls for a column get filtered,
> etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ATLAS-106) Store createTimestamp and modified timestamp separately for an entity
[
https://issues.apache.org/jira/browse/ATLAS-106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057890#comment-15057890
]
Suma Shivaprasad commented on ATLAS-106:
[~dkantor] Can you pls merge and upload the patch . Unable to apply it
currently.
{noformat}
git apply -v ATLAS-106.patch
ATLAS-106.patch:27: trailing whitespace.
ATLAS-106.patch:79: trailing whitespace.
ATLAS-106.patch:91: trailing whitespace.
ATLAS-106.patch:116: trailing whitespace.
ATLAS-106.patch:136: trailing whitespace.
Checking patch
repository/src/main/java/org/apache/atlas/repository/Constants.java...
Checking patch
repository/src/main/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepository.java...
Hunk #2 succeeded at 44 (offset 1 line).
Hunk #3 succeeded at 221 (offset 3 lines).
Hunk #4 succeeded at 281 (offset 3 lines).
Checking patch
repository/src/main/java/org/apache/atlas/repository/graph/GraphHelper.java...
Checking patch
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java...
error: while searching for:
//TODO - Handle Trait updates
addTraits(typedInstance, instanceVertex, classType);
}
}
private void mapInstanceToVertex(ITypedInstance typedInstance, Vertex
instanceVertex,
error: patch failed:
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java:158
error:
repository/src/main/java/org/apache/atlas/repository/graph/TypedInstanceToGraphMapper.java:
patch does not apply
Checking patch
repository/src/test/java/org/apache/atlas/repository/graph/GraphBackedMetadataRepositoryTest.java...
Hunk #1 succeeded at 215 (offset -2 lines).
Hunk #2 succeeded at 252 (offset -2 lines).
Hunk #3 succeeded at 272 (offset -2 lines).
Hunk #4 succeeded at 326 (offset -2 lines).
Hunk #5 succeeded at 343 (offset -2 lines).
Hunk #6 succeeded at 519 (offset -2 lines).
Hunk #7 succeeded at 536 (offset -2 lines).
Hunk #8 succeeded at 553 (offset -2 lines).
{noformat}
> Store createTimestamp and modified timestamp separately for an entity
> -
>
> Key: ATLAS-106
> URL: https://issues.apache.org/jira/browse/ATLAS-106
> Project: Atlas
> Issue Type: Improvement
> Reporter: Suma Shivaprasad
>Assignee: David Kantor
> Attachments: ATLAS-106-v1.patch
>
>
> Currently we store only the Create timestamp in atlas. Would be better to
> separate to track create and modified time separately , in cases where we
> want to support search queries give all entities which have been modified in
> the past 1 day etc.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ATLAS-122) Support for Deletion of Entities
[ https://issues.apache.org/jira/browse/ATLAS-122?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15057900#comment-15057900 ] Suma Shivaprasad commented on ATLAS-122: [~dkantor] Can you pls publish the design/changes that you propose before starting work on this jira. There are a couple of issues that need to be addressed wrt cascading deletes in the current hive model. For eg: table -> partitions is a composite relationship but is currently not modelled that way. We will need to address that as well as part of the work. > Support for Deletion of Entities > > > Key: ATLAS-122 > URL: https://issues.apache.org/jira/browse/ATLAS-122 > Project: Atlas > Issue Type: New Feature > Reporter: Suma Shivaprasad >Assignee: David Kantor > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-346) Atlas server loses messages sent from Hive hook if restarted after unclean shutdown
[
https://issues.apache.org/jira/browse/ATLAS-346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054093#comment-15054093
]
Suma Shivaprasad commented on ATLAS-346:
LGTM.
Not sure if there could be incompatibilities with jersey 1.19 client and jersey
1.10 server . Should we also upgrade the server then to jersey-1.19 from 1.10
? I dont see any issues mentioned in the migration guide -
https://jersey.java.net/documentation/latest/migration.html.
> Atlas server loses messages sent from Hive hook if restarted after unclean
> shutdown
> ---
>
> Key: ATLAS-346
> URL: https://issues.apache.org/jira/browse/ATLAS-346
> Project: Atlas
> Issue Type: Bug
>Reporter: Hemanth Yamijala
>Assignee: Hemanth Yamijala
>Priority: Critical
> Attachments: ATLAS-346.patch, hive_server2_import_bug_snip.txt
>
>
> * Start Atlas server pointed to an external Kafka instance
> * Configure HiveServer2 with Atlas hook pointing to the same Kafka instance.
> * Run a hive DDL script like
> {code}
> for i in `seq 1 25`; do ./bin/beeline -u jdbc:hive2://localhost:1 -n
> ${user} -p ${pass} -e "create table tbl${i} (col${i}1 int, col${i}2
> string);"; done
> {code}
> * While the script is executing, kill -9 the Atlas server
> * Let the script complete.
> * Verify that all the events are added to the ATLAS_HOOK topic.
> * Verify if the tables are added to Atlas.
> The observation is that the Kafka topic has all relevant messages, but the
> tables aren't added to Atlas.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ATLAS-20) Rename env variables from METADATA to ATLAS
[
https://issues.apache.org/jira/browse/ATLAS-20?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054107#comment-15054107
]
Suma Shivaprasad commented on ATLAS-20:
---
[~rishabhbhardwaj] Can you pls merge and reattach the patch
{noformat}
12:59:17-sshivaprasad~/workspace2/master-commit (master)$ curl -k -o
ATLAS-20.patch
https://issues.apache.org/jira/secure/attachment/12775494/ATLAS-20.patch
% Total% Received % Xferd Average Speed TimeTime Time Current
Dload Upload Total SpentLeft Speed
100 26146 100 261460 0 6590 0 0:00:03 0:00:03 --:--:-- 6589
12:59:30-sshivaprasad~/workspace2/master-commit (master)$ git apply -v
ATLAS-20.patch
Checking patch addons/hive-bridge/src/bin/import-hive.sh...
Checking patch distro/src/bin/atlas_config.py...
Checking patch distro/src/bin/atlas_start.py...
error: while searching for:
import atlas_config as mc
METADATA_LOG_OPTS="-Datlas.log.dir=%s -Datlas.log.file=application.log"
METADATA_COMMAND_OPTS="-Datlas.home=%s"
METADATA_CONFIG_OPTS="-Datlas.conf=%s"
DEFAULT_JVM_OPTS="-Xmx1024m -XX:MaxPermSize=512m
-Dlog4j.configuration=atlas-log4j.xml"
CONF_FILE="application.properties"
HBASE_STORAGE_CONF_ENTRY="atlas.graph.storage.backend\s*=\s*hbase"
def main():
metadata_home = mc.metadataDir()
confdir = mc.dirMustExist(mc.confDir(metadata_home))
mc.executeEnvSh(confdir)
logdir = mc.dirMustExist(mc.logDir(metadata_home))
#create sys property for conf dirs
jvm_opts_list = (METADATA_LOG_OPTS % logdir).split()
cmd_opts = (METADATA_COMMAND_OPTS % metadata_home)
jvm_opts_list.extend(cmd_opts.split())
config_opts = (METADATA_CONFIG_OPTS % confdir)
jvm_opts_list.extend(config_opts.split())
default_jvm_opts = DEFAULT_JVM_OPTS
metadata_jvm_opts = os.environ.get(mc.METADATA_OPTS, default_jvm_opts)
jvm_opts_list.extend(metadata_jvm_opts.split())
#expand web app dir
web_app_dir = mc.webAppDir(metadata_home)
mc.expandWebApp(metadata_home)
#add hbase-site.xml to classpath
hbase_conf_dir = mc.hbaseConfDir(confdir)
p = os.pathsep
metadata_classpath = confdir + p \
+ os.path.join(web_app_dir, "atlas", "WEB-INF",
"classes" ) + p \
+ os.path.join(web_app_dir, "atlas", "WEB-INF", "lib",
"*" ) + p \
+ os.path.join(metadata_home, "libext", "*")
if os.path.exists(hbase_conf_dir):
metadata_classpath = metadata_classpath + p \
+ hbase_conf_dir
else:
storage_backend = mc.grep(os.path.join(confdir, CONF_FILE),
HBASE_STORAGE_CONF_ENTRY)
if storage_backend != None:
raise Exception("Could not find hbase-site.xml in %s. Please set env
var HBASE_CONF_DIR to the hbase client conf dir", hbase_conf_dir)
metadata_pid_file = mc.pidFile(metadata_home)
if os.path.isfile(metadata_pid_file):
#Check if process listed in atlas.pid file is still running
pf = file(metadata_pid_file, 'r')
pid = pf.read().strip()
pf.close()
error: patch failed: distro/src/bin/atlas_start.py:21
error: distro/src/bin/atlas_start.py: patch does not apply
Checking patch distro/src/bin/atlas_stop.py...
Checking patch distro/src/bin/cputil.py...
Checking patch distro/src/bin/quick_start.py...
Checking patch distro/src/conf/atlas-env.sh...
Checking patch distro/src/test/python/scripts/TestMetadata.py...
error: while searching for:
@patch.object(mc,"unix_exist_pid")
@patch.object(mc,"writePid")
@patch.object(mc, "executeEnvSh")
@patch.object(mc,"metadataDir")
@patch.object(mc, "expandWebApp")
@patch("os.path.exists")
@patch.object(mc, "java")
def test_main(self, java_mock, exists_mock, expandWebApp_mock,
metadataDir_mock, executeEnvSh_mock, writePid_mock, unix_exist_pid_mock,
win_exist_pid_mock):
sys.argv = []
exists_mock.return_value = True
expandWebApp_mock.return_value = "webapp"
metadataDir_mock.return_value = "metadata_home"
win_exist_pid_mock("789")
win_exist_pid_mock.assert_called_with((str)(789))
unix_exist_pid_mock(789)
unix_exist_pid_mock.assert_called_with(789)
metadata.main()
self.assertTrue(java_mock.called)
if IS_WINDOWS:
java_mock.assert_called_with(
'org.apache.atlas.Atlas',
['-app', 'metadata_home\\server\\webapp\\atlas'],
'metadata_home\\conf;metadata_home\\server\\webapp\\atlas\\WEB-INF\\classes;metadata_home\\server\\webapp\\atlas\\WEB-INF\\lib\\*;metadata_home\\libext\\*;metadata_home\\hbase\\conf',
['-Datlas.log.dir=metadata_home\\logs',
'
[jira] [Commented] (ATLAS-346) Atlas server loses messages sent from Hive hook if restarted after unclean shutdown
[
https://issues.apache.org/jira/browse/ATLAS-346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054196#comment-15054196
]
Suma Shivaprasad commented on ATLAS-346:
Build succeeds for the patch
[INFO]
[INFO] Reactor Summary:
[INFO]
[INFO] apache-atlas ... SUCCESS [ 5.722 s]
[INFO] Apache Atlas Common SUCCESS [ 9.189 s]
[INFO] Apache Atlas Typesystem SUCCESS [ 46.908 s]
[INFO] Apache Atlas Server API SUCCESS [ 5.558 s]
[INFO] Apache Atlas Client SUCCESS [ 15.424 s]
[INFO] Apache Atlas Notification .. SUCCESS [ 14.212 s]
[INFO] Apache Atlas Titan . SUCCESS [ 18.916 s]
[INFO] Apache Atlas Repository SUCCESS [03:56 min]
[INFO] Apache Atlas UI SUCCESS [01:30 min]
[INFO] Apache Atlas Web Application ... SUCCESS [02:29 min]
[INFO] Apache Atlas Documentation . SUCCESS [ 3.723 s]
[INFO] Apache Atlas Hive Bridge ... SUCCESS [01:12 min]
[INFO] Apache Atlas Distribution .. SUCCESS [ 32.675 s]
[INFO]
[INFO] BUILD SUCCESS
[INFO]
[INFO] Total time: 11:42 min
[INFO] Finished at: 2015-12-12T15:32:03+05:30
[INFO] Final Memory: 183M/715M
[INFO]
> Atlas server loses messages sent from Hive hook if restarted after unclean
> shutdown
> ---
>
> Key: ATLAS-346
> URL: https://issues.apache.org/jira/browse/ATLAS-346
> Project: Atlas
> Issue Type: Bug
>Reporter: Hemanth Yamijala
>Assignee: Hemanth Yamijala
>Priority: Critical
> Attachments: ATLAS-346.patch, ATLAS-346.patch,
> hive_server2_import_bug_snip.txt
>
>
> * Start Atlas server pointed to an external Kafka instance
> * Configure HiveServer2 with Atlas hook pointing to the same Kafka instance.
> * Run a hive DDL script like
> {code}
> for i in `seq 1 25`; do ./bin/beeline -u jdbc:hive2://localhost:1 -n
> ${user} -p ${pass} -e "create table tbl${i} (col${i}1 int, col${i}2
> string);"; done
> {code}
> * While the script is executing, kill -9 the Atlas server
> * Let the script complete.
> * Verify that all the events are added to the ATLAS_HOOK topic.
> * Verify if the tables are added to Atlas.
> The observation is that the Kafka topic has all relevant messages, but the
> tables aren't added to Atlas.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ATLAS-346) Atlas server loses messages sent from Hive hook if restarted after unclean shutdown
[
https://issues.apache.org/jira/browse/ATLAS-346?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054179#comment-15054179
]
Suma Shivaprasad commented on ATLAS-346:
Will run the tests with the patch once and check.
Two minor comments
1. Pls remove testng version and scope since this is mentioned in parent pom
2. typo - > \"Version\":\"version-rrelease\"
> Atlas server loses messages sent from Hive hook if restarted after unclean
> shutdown
> ---
>
> Key: ATLAS-346
> URL: https://issues.apache.org/jira/browse/ATLAS-346
> Project: Atlas
> Issue Type: Bug
>Reporter: Hemanth Yamijala
>Assignee: Hemanth Yamijala
>Priority: Critical
> Attachments: ATLAS-346.patch, ATLAS-346.patch,
> hive_server2_import_bug_snip.txt
>
>
> * Start Atlas server pointed to an external Kafka instance
> * Configure HiveServer2 with Atlas hook pointing to the same Kafka instance.
> * Run a hive DDL script like
> {code}
> for i in `seq 1 25`; do ./bin/beeline -u jdbc:hive2://localhost:1 -n
> ${user} -p ${pass} -e "create table tbl${i} (col${i}1 int, col${i}2
> string);"; done
> {code}
> * While the script is executing, kill -9 the Atlas server
> * Let the script complete.
> * Verify that all the events are added to the ATLAS_HOOK topic.
> * Verify if the tables are added to Atlas.
> The observation is that the Kafka topic has all relevant messages, but the
> tables aren't added to Atlas.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ATLAS-374) Doc: Create a wiki for documenting fault tolerance and HA options for Atlas data
[ https://issues.apache.org/jira/browse/ATLAS-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054120#comment-15054120 ] Suma Shivaprasad commented on ATLAS-374: Few minor comments 1. The Architecture notes are nicely documented and could be moved to "Architecture.twiki" and referred here 2. In solr startup docs, pls remove "-s $SOLR_HOME/data/node2/solr" since this is optional and is needed on\y when 2 instances of solr are needed on the same host. By default it starts from SOLR_HOME/server/solr > Doc: Create a wiki for documenting fault tolerance and HA options for Atlas > data > > > Key: ATLAS-374 > URL: https://issues.apache.org/jira/browse/ATLAS-374 > Project: Atlas > Issue Type: Improvement >Reporter: Hemanth Yamijala >Assignee: Hemanth Yamijala > Attachments: ATLAS-374.patch, ATLAS-374.patch > > > With switching to HBase and Solr as backends for Atlas's datastore, it is now > possible to have a high level of fault tolerance and availability guarantees > for Atlas data. This JIRA is to enhance the documentation to have a guide for > setting up Atlas in a fault tolerant and available mode. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-374) Doc: Create a wiki for documenting fault tolerance and HA options for Atlas data
[ https://issues.apache.org/jira/browse/ATLAS-374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15054121#comment-15054121 ] Suma Shivaprasad commented on ATLAS-374: 3. Also should we just point to the InstallationSteps twiki for Solr startup and collection creation ? This would avoid changes in two places, if any are needed. Similarly for Hbase configuration? > Doc: Create a wiki for documenting fault tolerance and HA options for Atlas > data > > > Key: ATLAS-374 > URL: https://issues.apache.org/jira/browse/ATLAS-374 > Project: Atlas > Issue Type: Improvement >Reporter: Hemanth Yamijala >Assignee: Hemanth Yamijala > Attachments: ATLAS-374.patch, ATLAS-374.patch > > > With switching to HBase and Solr as backends for Atlas's datastore, it is now > possible to have a high level of fault tolerance and availability guarantees > for Atlas data. This JIRA is to enhance the documentation to have a guide for > setting up Atlas in a fault tolerant and available mode. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
