Some code questions

2015-09-01 Thread ltf 
Hello,
I'm very new to Atlas, and I still in the status of learning.
When I browsed the source code of Atlas, I have the following 4 questions, 
maybe or maybe not issues, thanks.


1.
file : 
incubator-atlas\typesystem\src\main\java\org\apache\atlas\typesystem\types\ObjectGraphTraversal.java
line : 56
void processValue(IDataType dT, Object val) throws AtlasException {
if (val != null) {
if (dT.getTypeCategory() == DataTypes.TypeCategory.ARRAY) {
IDataType elemType = ((DataTypes.ArrayType) dT).getElemType();
processCollection(elemType, val);
} else if (dT.getTypeCategory() == DataTypes.TypeCategory.MAP) {
IDataType keyType = ((DataTypes.MapType) dT).getKeyType();
IDataType valueType = ((DataTypes.MapType) dT).getKeyType();
//   should be getValueType(); ???
processMap(keyType, valueType, val);
} else if (dT.getTypeCategory() == DataTypes.TypeCategory.STRUCT
|| dT.getTypeCategory() == DataTypes.TypeCategory.TRAIT) {
processStruct(val);
} else if (dT.getTypeCategory() == DataTypes.TypeCategory.CLASS) {
processReferenceableInstance(val);
}
}
}


2. 
file : 
incubator-atlas\typesystem\src\main\java\org\apache\atlas\typesystem\types\ObjectGraphWalker.java
line : 94
void traverseValue(IDataType dT, Object val) throws AtlasException {
if (val != null) {
if (dT.getTypeCategory() == DataTypes.TypeCategory.ARRAY) {
IDataType elemType = ((DataTypes.ArrayType) dT).getElemType();
visitCollection(elemType, val);
} else if (dT.getTypeCategory() == DataTypes.TypeCategory.MAP) {
IDataType keyType = ((DataTypes.MapType) dT).getKeyType();
IDataType valueType = ((DataTypes.MapType) dT).getKeyType();
//   should be getValueType(); ???
visitMap(keyType, valueType, val);
} else if (dT.getTypeCategory() == DataTypes.TypeCategory.STRUCT
|| dT.getTypeCategory() == DataTypes.TypeCategory.TRAIT) {
visitStruct(val);
} else if (dT.getTypeCategory() == DataTypes.TypeCategory.CLASS) {
visitReferenceableInstance(val);
}
}
}


3.
file : 
incubator-atlas\repository\src\main\java\org\apache\atlas\repository\memory\AttributeStores.java
line : 58
static IAttributeStore createStore(AttributeInfo i) throws 
RepositoryException {
switch (i.dataType().getTypeCategory()) {
case PRIMITIVE:
if (i.dataType() == DataTypes.BOOLEAN_TYPE) {
return new BooleanAttributeStore(i);
} else if (i.dataType() == DataTypes.BYTE_TYPE) {
return new ByteAttributeStore(i);
} else if (i.dataType() == DataTypes.SHORT_TYPE) {
new ShortAttributeStore(i); 
 //   should be return ???
} else if (i.dataType() == DataTypes.INT_TYPE) {
return new IntAttributeStore(i);


4.
file : incubator-atlas\src\test\python\scripts\TestMetadata.py
line : 52
self.assertTrue(java_mock.called)
if IS_WINDOWS:
  java_mock.assert_called_with(
'org.apache.atlas.Main',
['-app', 'metadata_home/server/webapp/atlas'],  
 //  when os is WINDOWS, '/' should be '\\', otherwise the test fails, 
build will stop!!!

'metadata_home/conf:metadata_home/server/webapp/atlas/WEB-INF/classes:metadata_home/server/webapp/atlas/WEB-INF/lib\\*:metadata_home/libext\\*',
['-Datlas.log.dir=metadata_home/logs', 
'-Datlas.log.file=application.log', '-Datlas.home=metadata_home', 
'-Datlas.conf=metadata_home/conf', '-Xmx1024m', 
'-Dlog4j.configuration=atlas-log4j.xml'], 'metadata_home/logs')
else:


Best, liutongfeng

Re: Some code questions

2015-09-01 Thread Shwetha Shivalingamurthy 
1, 2 and 4 look like valid issues. Can you file a bug and submit the
patches. Thanks

AttributeStores is used in some tests. Memory based repository is not
used. But can be fixed



Regards,
Shwetha






On 01/09/15 11:43 am, "ltf"  wrote:

>Hello,
>I'm very new to Atlas, and I still in the status of learning.
>When I browsed the source code of Atlas, I have the following 4
>questions, maybe or maybe not issues, thanks.
>
>
>1.
>file : 
>incubator-atlas\typesystem\src\main\java\org\apache\atlas\typesystem\types
>\ObjectGraphTraversal.java
>line : 56
>void processValue(IDataType dT, Object val) throws AtlasException {
>if (val != null) {
>if (dT.getTypeCategory() == DataTypes.TypeCategory.ARRAY) {
>IDataType elemType = ((DataTypes.ArrayType)
>dT).getElemType();
>processCollection(elemType, val);
>} else if (dT.getTypeCategory() ==
>DataTypes.TypeCategory.MAP) {
>IDataType keyType = ((DataTypes.MapType) dT).getKeyType();
>IDataType valueType = ((DataTypes.MapType)
>dT).getKeyType();//   should be getValueType(); ???
>processMap(keyType, valueType, val);
>} else if (dT.getTypeCategory() ==
>DataTypes.TypeCategory.STRUCT
>|| dT.getTypeCategory() ==
>DataTypes.TypeCategory.TRAIT) {
>processStruct(val);
>} else if (dT.getTypeCategory() ==
>DataTypes.TypeCategory.CLASS) {
>processReferenceableInstance(val);
>}
>}
>}
>
>
>2. 
>file : 
>incubator-atlas\typesystem\src\main\java\org\apache\atlas\typesystem\types
>\ObjectGraphWalker.java
>line : 94
>void traverseValue(IDataType dT, Object val) throws AtlasException {
>if (val != null) {
>if (dT.getTypeCategory() == DataTypes.TypeCategory.ARRAY) {
>IDataType elemType = ((DataTypes.ArrayType)
>dT).getElemType();
>visitCollection(elemType, val);
>} else if (dT.getTypeCategory() ==
>DataTypes.TypeCategory.MAP) {
>IDataType keyType = ((DataTypes.MapType) dT).getKeyType();
>IDataType valueType = ((DataTypes.MapType)
>dT).getKeyType();//   should be getValueType(); ???
>visitMap(keyType, valueType, val);
>} else if (dT.getTypeCategory() ==
>DataTypes.TypeCategory.STRUCT
>|| dT.getTypeCategory() ==
>DataTypes.TypeCategory.TRAIT) {
>visitStruct(val);
>} else if (dT.getTypeCategory() ==
>DataTypes.TypeCategory.CLASS) {
>visitReferenceableInstance(val);
>}
>}
>}
>
>
>3.
>file : 
>incubator-atlas\repository\src\main\java\org\apache\atlas\repository\memor
>y\AttributeStores.java
>line : 58
>static IAttributeStore createStore(AttributeInfo i) throws
>RepositoryException {
>switch (i.dataType().getTypeCategory()) {
>case PRIMITIVE:
>if (i.dataType() == DataTypes.BOOLEAN_TYPE) {
>return new BooleanAttributeStore(i);
>} else if (i.dataType() == DataTypes.BYTE_TYPE) {
>return new ByteAttributeStore(i);
>} else if (i.dataType() == DataTypes.SHORT_TYPE) {
>new ShortAttributeStore(i);
>   //   should be return ???
>} else if (i.dataType() == DataTypes.INT_TYPE) {
>return new IntAttributeStore(i);
>
>
>4.
>file : incubator-atlas\src\test\python\scripts\TestMetadata.py
>line : 52
>self.assertTrue(java_mock.called)
>if IS_WINDOWS:
>  java_mock.assert_called_with(
>'org.apache.atlas.Main',
>['-app', 'metadata_home/server/webapp/atlas'],
>   //  when os is WINDOWS, '/' should be '\\', otherwise the test
>fails, build will stop!!!
>
>'metadata_home/conf:metadata_home/server/webapp/atlas/WEB-INF/classes:meta
>data_home/server/webapp/atlas/WEB-INF/lib\\*:metadata_home/libext\\*',
>['-Datlas.log.dir=metadata_home/logs',
>'-Datlas.log.file=application.log', '-Datlas.home=metadata_home',
>'-Datlas.conf=metadata_home/conf', '-Xmx1024m',
>'-Dlog4j.configuration=atlas-log4j.xml'], 'metadata_home/logs')
>else:
>
>
>Best, liutongfeng

[jira] [Commented] (ATLAS-127) Handle complex types in DSL queries

2015-09-01 Thread Suma Shivaprasad (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-127?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14725085#comment-14725085
 ] 

Suma Shivaprasad commented on ATLAS-127:


Map type is not supported by Titan internally and also indexes will not work on 
list, map types. So we will have to unwrap it while storing like we are doing 
currently.

> Handle complex types in DSL queries
> ---
>
> Key: ATLAS-127
> URL: https://issues.apache.org/jira/browse/ATLAS-127
> Project: Atlas
>  Issue Type: New Feature
>Reporter: Suma Shivaprasad
>




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-110) UI: Lineage should be clickable

2015-09-01 Thread Erik Bergenholtz (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-110?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14725639#comment-14725639
 ] 

Erik Bergenholtz commented on ATLAS-110:


+1 LGTM [~svenkat] ?

> UI: Lineage should be clickable
> ---
>
> Key: ATLAS-110
> URL: https://issues.apache.org/jira/browse/ATLAS-110
> Project: Atlas
>  Issue Type: Bug
>Affects Versions: 0.6-incubating
>Reporter: Erik Bergenholtz
>Assignee: Vishal Kadam
> Fix For: 0.6-incubating
>
> Attachments: ATLAS-110.patch
>
>
> It should be possible to drill down on lineage.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ATLAS-134) Some defects found when reviewing the source code.

2015-09-01 Thread liutongfeng (JIRA) 
liutongfeng created ATLAS-134:
-

 Summary: Some defects found when reviewing the source code.
 Key: ATLAS-134
 URL: https://issues.apache.org/jira/browse/ATLAS-134
 Project: Atlas
  Issue Type: Bug
Affects Versions: trunk
Reporter: liutongfeng
Priority: Minor


1.The second getKeyType() should be getValueType().
file : ObjectGraphTraversal.java
line : 56

IDataType keyType = ((DataTypes.MapType) dT).getKeyType();
IDataType valueType = ((DataTypes.MapType) dT).getKeyType();
processMap(keyType, valueType, val);

2.The second getKeyType() should be getValueType().
file : ObjectGraphWalker.java
line : 94

IDataType keyType = ((DataTypes.MapType) dT).getKeyType();
IDataType valueType = ((DataTypes.MapType) dT).getKeyType(); 
visitMap(keyType, valueType, val);

3.When OS is WINDOWS, '/' should be '\\', otherwise the test will fail.
file : TestMetadata.py
line : 52

if IS_WINDOWS:
  java_mock.assert_called_with(
'org.apache.atlas.Main',
['-app', 'metadata_home/server/webapp/atlas'],

'metadata_home/conf:metadata_home/server/webapp/atlas/WEB-INF/classes:metadata_home/server/webapp/atlas/WEB-INF/lib\\*:metadata_home/libext\\*',
['-Datlas.log.dir=metadata_home/logs', '-Datlas.log.file=application.log', 
'-Datlas.home=metadata_home', '-Datlas.conf=metadata_home/conf', '-Xmx1024m', 
'-Dlog4j.configuration=atlas-log4j.xml'], 'metadata_home/logs')
else:

4.Should use return.
file : 
incubator-atlas\repository\src\main\java\org\apache\atlas\repository\memory\AttributeStores.java
line : 58

static IAttributeStore createStore(AttributeInfo i) throws RepositoryException {
switch (i.dataType().getTypeCategory()) {
case PRIMITIVE:
if (i.dataType() == DataTypes.BOOLEAN_TYPE) {
return new BooleanAttributeStore(i);
} else if (i.dataType() == DataTypes.BYTE_TYPE) {
return new ByteAttributeStore(i);
} else if (i.dataType() == DataTypes.SHORT_TYPE) {
new ShortAttributeStore(i); //   should be return
} else if (i.dataType() == DataTypes.INT_TYPE) {
return new IntAttributeStore(i);



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Commented] (ATLAS-117) Build fails on the latest commit 48343db999...

2015-09-01 Thread Aaron Dossett (JIRA) 

[ 
https://issues.apache.org/jira/browse/ATLAS-117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14725373#comment-14725373
 ] 

Aaron Dossett commented on ATLAS-117:
-

[~svenkat] Per the maven assembly documentation at 
http://maven.apache.org/plugins/maven-assembly-plugin/

-
The main goal in the assembly plugin is the single goal. It is used to create 
all assemblies. All other goals are deprecated and will be removed in a future 
release.

Usage of the assembly:assembly, assembly:attached, assembly:directory, and 
assembly:directory-inline are deprecated, since they wreak havoc with normal 
build processes and promote non-standard build practices.
--

I really like the approach that Apache Knox takes with its release module, 
which is to include shell scripts, log4j settings, other configuration files, 
and similar items.  I've taken that approach with some internal projects and 
found that it works well.

Apache Knox release module: 
https://github.com/apache/knox/tree/master/gateway-release/home

> Build fails on the latest commit 48343db999...
> --
>
> Key: ATLAS-117
> URL: https://issues.apache.org/jira/browse/ATLAS-117
> Project: Atlas
>  Issue Type: Bug
>Reporter: David Kaspar
>Assignee: Tom Beerbower
> Attachments: ATLAS-117-2.patch, ATLAS-117.patch
>
>
> Build fails when building the latest commit 
> 48343db999b495458409644c8b9d2fd0bd9fa99d:
> {code}
> [INFO] Scanning for projects...
> [WARNING] 
> [WARNING] Some problems were encountered while building the effective model 
> for org.apache.atlas:atlas-webapp:war:0.6-incubating-SNAPSHOT
> [WARNING] 'build.plugins.plugin.version' for 
> org.codehaus.mojo:keytool-maven-plugin is missing. @ 
> org.apache.atlas:atlas-webapp:[unknown-version], 
> /Users/MYSELF/Projects/External/incubator-atlas/webapp/pom.xml, line 309, 
> column 21
> [WARNING] 
> [WARNING] It is highly recommended to fix these problems because they 
> threaten the stability of your build.
> [WARNING] 
> [WARNING] For this reason, future Maven versions might no longer support 
> building such malformed projects.
> [WARNING] 
> [INFO] 
> 
> [INFO] Reactor Build Order:
> [INFO] 
> [INFO] apache-atlas
> [INFO] Apache Atlas Typesystem
> [INFO] Apache Atlas Client
> [INFO] Apache Atlas Notification
> [INFO] Apache Atlas Repository
> [INFO] Apache Atlas Web Application
> [INFO] Apache Atlas Documentation
> [INFO] Apache Atlas Hive Bridge
> [INFO]
>  
> [INFO] 
> 
> [INFO] Building apache-atlas 0.6-incubating-SNAPSHOT
> [INFO] 
> 
> [INFO] 
> [INFO] --- maven-clean-plugin:2.5:clean (default-clean) @ apache-atlas ---
> [INFO] Deleting /Users/MYSELF/Projects/External/incubator-atlas/target
> [INFO] Deleting /Users/MYSELF/Projects/External/incubator-atlas (includes = 
> [**/*.pyc], excludes = [])
> [INFO] 
> [INFO] --- buildnumber-maven-plugin:1.0:create (default) @ apache-atlas ---
> [INFO] Checking for local modifications: skipped.
> [INFO] Updating project files from SCM: skipped.
> [WARNING] Cannot get the revision information from the scm repository, 
> proceeding with revision of release : 
> No such provider: 'https'.
> [INFO] Storing buildNumber: release at timestamp: 1440071695692
> [WARNING] Cannot get the branch information from the scm repository, 
> proceeding with UNKNOWN_BRANCH : 
> No such provider: 'https'.
> [INFO] Storing buildScmBranch: UNKNOWN_BRANCH
> [INFO] 
> [INFO] --- maven-remote-resources-plugin:1.4:process (default) @ apache-atlas 
> ---
> [INFO] 
> [INFO] --- scala-maven-plugin:3.2.0:compile (scala-compile-first) @ 
> apache-atlas ---
> [INFO] No sources to compile
> [INFO] 
> [INFO] --- scala-maven-plugin:3.2.0:testCompile (scala-test-compile-first) @ 
> apache-atlas ---
> [INFO] No sources to compile
> [INFO] 
> [INFO] --- exec-maven-plugin:1.2.1:exec (python-test) @ apache-atlas ---
> test_jar_java_lookups_fail (TestMetadata.TestMetadata) ... ok
> test_jar_java_lookups_succeed_from_path (TestMetadata.TestMetadata) ... ok
> test_main (TestMetadata.TestMetadata) ... Apache Atlas Server started!!!
> ok
> --
> Ran 3 tests in 0.003s
> OK
> [INFO] 
> [INFO] --- maven-site-plugin:3.2:attach-descriptor (attach-descriptor) @ 
> apache-atlas ---
> [INFO] 
> [INFO] --- maven-assembly-plugin:2.2.1:single (default) @ apache-atlas ---
> [INFO] Reading assembly descriptor: src/main/assemblies/standalone-package.xml
> [INFO] Reading assembly descriptor: src/main/assemblies/src-package.xml
> [INFO] 
>

[jira] [Created] (ATLAS-132) Search Controller Optimization

2015-09-01 Thread Erik Bergenholtz (JIRA) 
Erik Bergenholtz created ATLAS-132:
--

 Summary: Search Controller Optimization
 Key: ATLAS-132
 URL: https://issues.apache.org/jira/browse/ATLAS-132
 Project: Atlas
  Issue Type: Improvement
Affects Versions: 0.5.1-incubating
Reporter: Erik Bergenholtz
Assignee: Rohit
 Fix For: 0.6-incubating


Search Controller currently makes multiple calls. This can and should be 
optimized as a single API call.



--
This message was sent by Atlassian JIRA
(v6.3.4#6332)

[jira] [Created] (ATLAS-133) Use TypeResource instead of NavigationResource

2015-09-01 Thread Erik Bergenholtz (JIRA) 
Erik Bergenholtz created ATLAS-133:
--

 Summary: Use TypeResource instead of NavigationResource 
 Key: ATLAS-133
 URL: https://issues.apache.org/jira/browse/ATLAS-133
 Project: Atlas
  Issue Type: Bug
Affects Versions: 0.5.1-incubating
Reporter: Erik Bergenholtz
Assignee: Rohit
 Fix For: 0.6-incubating


Use TypeResource instead of Navigation Resource and refactor types controller 
code




--
This message was sent by Atlassian JIRA
(v6.3.4#6332)