[jira] [Commented] (ATLAS-819) All user defined types should have a set of common attributes
[ https://issues.apache.org/jira/browse/ATLAS-819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15339363#comment-15339363 ] Shwetha G S commented on ATLAS-819: --- Yes, it makes sense to set column owner to table owner, will make the change > All user defined types should have a set of common attributes > - > > Key: ATLAS-819 > URL: https://issues.apache.org/jira/browse/ATLAS-819 > Project: Atlas > Issue Type: Bug >Reporter: Hemanth Yamijala >Assignee: Shwetha G S > Fix For: 0.7-incubating > > Attachments: ATLAS-819.patch > > > It would be very convenient if all user defined types have a conventional set > of common attributes including: > * name > * description > * owner > * created at > * modified at -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-915) Fix docs for import-hive changes
[ https://issues.apache.org/jira/browse/ATLAS-915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15339359#comment-15339359 ] Shwetha G S commented on ATLAS-915: --- +1 > Fix docs for import-hive changes > > > Key: ATLAS-915 > URL: https://issues.apache.org/jira/browse/ATLAS-915 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Vimal Sharma > Fix For: 0.7-incubating > > Attachments: ATLAS-915.patch, ATLAS-915.patch > > > Pls fix According to changes in ATLAS-688 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-653) insert into local dir, load from local dir does not show up in lineage
[ https://issues.apache.org/jira/browse/ATLAS-653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-653: -- Issue Type: Sub-task (was: Bug) Parent: ATLAS-492 > insert into local dir, load from local dir does not show up in lineage > -- > > Key: ATLAS-653 > URL: https://issues.apache.org/jira/browse/ATLAS-653 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > > All the above commands do not show up in lineage since they dont have any > input entities registered. > Local FS Paths can be modelled without any unique paths/attributes and can be > displayed in lineage -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-702) "ALTER TABLE .. NOT STORED AS DIRECTORIES" metadata update is not sent to Atlas.
[
https://issues.apache.org/jira/browse/ATLAS-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-702:
--
Issue Type: Sub-task (was: Bug)
Parent: ATLAS-492
> "ALTER TABLE .. NOT STORED AS DIRECTORIES" metadata update is not sent to
> Atlas.
>
>
> Key: ATLAS-702
> URL: https://issues.apache.org/jira/browse/ATLAS-702
> Project: Atlas
> Issue Type: Sub-task
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Suma Shivaprasad
>
> 1. CREATE TABLE list_bucket_single (key STRING, value STRING) SKEWED BY (key)
> ON (1,5,6) STORED AS DIRECTORIES;
> {noformat}
> 0: jdbc:hive2://localhost:1/default> describe formatted
> list_bucket_single2;
> +---++---+--+
> | col_name| data_type
>|comment|
> +---++---+--+
> | # col_name| data_type
>| comment |
> | | NULL
>| NULL |
> | key | string
>| |
> | value | string
>| |
> | | NULL
>| NULL |
> | # Detailed Table Information | NULL
>| NULL |
> | Database: | default
>| NULL |
> | Owner:| apathan
>| NULL |
> | CreateTime: | Mon Apr 25 16:27:21 IST 2016
>| NULL |
> | LastAccessTime: | UNKNOWN
>| NULL |
> | Protect Mode: | None
>| NULL |
> | Retention:| 0
>| NULL |
> | Location: |
> hdfs://localhost:9000/user/hive/warehouse/list_bucket_single2 | NULL
> |
> | Table Type: | MANAGED_TABLE
>| NULL |
> | Table Parameters: | NULL
>| NULL |
> | | transient_lastDdlTime
>| 1461581841|
> | | NULL
>| NULL |
> | # Storage Information | NULL
>| NULL |
> | SerDe Library:|
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL
> |
> | InputFormat: | org.apache.hadoop.mapred.TextInputFormat
>| NULL |
> | OutputFormat: |
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL
> |
> | Compressed: | No
>| NULL |
> | Num Buckets: | -1
>| NULL |
> | Bucket Columns: | []
>| NULL |
> | Sort Columns: | []
>| NULL |
> | Stored As SubDirectories: | Yes
>| NULL |
> | Skewed Columns: | [key]
>| NULL |
> | Skewed Values:| [[1], [5], [6]]
>| NULL |
> | Storage Desc Params: | NULL
>|
[jira] [Updated] (ATLAS-778) ALTER TABLE TOUCH is not modelled
[
https://issues.apache.org/jira/browse/ATLAS-778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-778:
--
Issue Type: Sub-task (was: Bug)
Parent: ATLAS-492
> ALTER TABLE TOUCH is not modelled
> -
>
> Key: ATLAS-778
> URL: https://issues.apache.org/jira/browse/ATLAS-778
> Project: Atlas
> Issue Type: Sub-task
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>
> ALTER TABLE TOUCH is not modelled.
> ALTER TABLE TOUCH changes metadata of hive table(like create time,
> transient_lastDdlTime). No entity update message is sent to atlas when table
> touch command is fired.
> {noformat}
> 0: jdbc:hive2://localhost:1/default> describe formatted tg2;
> +---+-+---+--+
> | col_name|data_type
> |comment|
> +---+-+---+--+
> | # col_name| data_type
> | comment |
> | | NULL
> | NULL |
> | abc | string
> | |
> | | NULL
> | NULL |
> | # Detailed Table Information | NULL
> | NULL |
> | Database: | db2sdbxnpqone
> | NULL |
> | Owner:| apathan
> | NULL |
> | CreateTime: | Wed May 11 23:19:55 IST 2016
> | NULL |
> | LastAccessTime: | UNKNOWN
> | NULL |
> | Protect Mode: | None
> | NULL |
> | Retention:| 0
> | NULL |
> | Location: |
> hdfs://localhost:9000/user/hive/warehouse/db2sdbxnpqone.db/tg2 | NULL
> |
> | Table Type: | MANAGED_TABLE
> | NULL |
> | Table Parameters: | NULL
> | NULL |
> | | transient_lastDdlTime
> | 1462988995|
> | | NULL
> | NULL |
> | # Storage Information | NULL
> | NULL |
> | SerDe Library:|
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL
> |
> | InputFormat: | org.apache.hadoop.mapred.TextInputFormat
> | NULL |
> | OutputFormat: |
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL
> |
> | Compressed: | No
> | NULL |
> | Num Buckets: | -1
> | NULL |
> | Bucket Columns: | []
> | NULL |
> | Sort Columns: | []
> | NULL |
> | Storage Desc Params: | NULL
> | NULL |
> | | serialization.format
> | 1 |
> +---+-+---+--+
> 26 rows selected (0.113 seconds)
> 0: jdbc:hive2://localhost:1/default> alter table tg2 touch;
> No rows affected (0.173 seconds)
> 0: jdbc:hive2://localhost:1/default> describe formatted tg2;
> +---+
[jira] [Updated] (ATLAS-538) Capture update delta details
[ https://issues.apache.org/jira/browse/ATLAS-538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-538: -- Fix Version/s: (was: 0.7-incubating) > Capture update delta details > > > Key: ATLAS-538 > URL: https://issues.apache.org/jira/browse/ATLAS-538 > Project: Atlas > Issue Type: Sub-task >Reporter: Shwetha G S >Assignee: Shwetha G S > > Atlas supports partial/full update which can take the entire entity. We need > to figure out the exact attributes updated for entity history -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-566) Remove BDB References
[ https://issues.apache.org/jira/browse/ATLAS-566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-566. --- Resolution: Duplicate > Remove BDB References > - > > Key: ATLAS-566 > URL: https://issues.apache.org/jira/browse/ATLAS-566 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Erik Bergenholtz >Assignee: Tom Beerbower > Fix For: 0.7-incubating > > > Berkley DB should be removed from Apache Atlas and should not be referenced > from any POMs or pulled in to any DOs of Atlas. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-581) Atlas UI should show state of Entity
[ https://issues.apache.org/jira/browse/ATLAS-581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-581. --- Resolution: Duplicate > Atlas UI should show state of Entity > > > Key: ATLAS-581 > URL: https://issues.apache.org/jira/browse/ATLAS-581 > Project: Atlas > Issue Type: Improvement >Reporter: Erik Bergenholtz >Assignee: Keval Bhatt > Fix For: 0.7-incubating > > > Atlas will be introducing an attribute indicating whether or not an entity is > deleted. In the search results page the UI should differentiate between a > deleted entity and a non-deleted entity. The best way to do this is up for > discussion, several options include: > - Introduce a separate 'deleted' column with a check mark for deleted Entities > - Show deleted Entities in muted gray text > - Show deleted Entities in -strikethrough- text -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-630) Network partition of active instance before first time initialization completes could cause a split-brain scenario
[ https://issues.apache.org/jira/browse/ATLAS-630?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-630: -- Fix Version/s: (was: 0.7-incubating) > Network partition of active instance before first time initialization > completes could cause a split-brain scenario > -- > > Key: ATLAS-630 > URL: https://issues.apache.org/jira/browse/ATLAS-630 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Hemanth Yamijala >Assignee: Hemanth Yamijala > > Start HA in active mode for the first time - when all dependent Graph stores > are being initialized the first time. > Before the initialization completes, create a network partition such that the > active instance loses connection to Zookeeper. This could cause a split brain > scenario where the active instance does not stop the initialization and > another instance becomes active and starts initialization in parallel. > This is only an issue for the very first initialization and the probability > of it happening is possibly very low. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-590) HBaseBasedAuditRepository.start() fails
[
https://issues.apache.org/jira/browse/ATLAS-590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-590:
--
Fix Version/s: (was: 0.7-incubating)
> HBaseBasedAuditRepository.start() fails
> ---
>
> Key: ATLAS-590
> URL: https://issues.apache.org/jira/browse/ATLAS-590
> Project: Atlas
> Issue Type: Sub-task
>Reporter: Shwetha G S
>Assignee: Shwetha G S
>
> The application doesn't fail, but prints this warning. Need to investigate
> {noformat}
> 2016-03-23 11:01:45,496 WARN - [main:] ~ Failed to identify the fs of dir
> hdfs://localhost.localdomain:8020/apps/hbase/data/lib, ignored
> (DynamicClassLoader:106)
> java.io.IOException: No FileSystem for scheme: hdfs
> at
> org.apache.hadoop.fs.FileSystem.getFileSystemClass(FileSystem.java:2644)
> at
> org.apache.hadoop.fs.FileSystem.createFileSystem(FileSystem.java:2651)
> at org.apache.hadoop.fs.FileSystem.access$200(FileSystem.java:92)
> at
> org.apache.hadoop.fs.FileSystem$Cache.getInternal(FileSystem.java:2687)
> at org.apache.hadoop.fs.FileSystem$Cache.get(FileSystem.java:2669)
> at org.apache.hadoop.fs.FileSystem.get(FileSystem.java:371)
> at org.apache.hadoop.fs.Path.getFileSystem(Path.java:295)
> at
> org.apache.hadoop.hbase.util.DynamicClassLoader.(DynamicClassLoader.java:104)
> at
> org.apache.hadoop.hbase.protobuf.ProtobufUtil.(ProtobufUtil.java:241)
> at org.apache.hadoop.hbase.ClusterId.parseFrom(ClusterId.java:64)
> at
> org.apache.hadoop.hbase.zookeeper.ZKClusterId.readClusterIdZNode(ZKClusterId.java:75)
> at
> org.apache.hadoop.hbase.client.ZooKeeperRegistry.getClusterId(ZooKeeperRegistry.java:105)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.retrieveClusterId(ConnectionManager.java:879)
> at
> org.apache.hadoop.hbase.client.ConnectionManager$HConnectionImplementation.(ConnectionManager.java:635)
> at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native
> Method)
> at
> sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
> at
> sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
> at java.lang.reflect.Constructor.newInstance(Constructor.java:422)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:238)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:218)
> at
> org.apache.hadoop.hbase.client.ConnectionFactory.createConnection(ConnectionFactory.java:119)
> at
> org.apache.atlas.repository.audit.HBaseBasedAuditRepository.start(HBaseBasedAuditRepository.java:257)
> at org.apache.atlas.service.Services.start(Services.java:45)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.startServices(GuiceServletConfig.java:130)
> at
> org.apache.atlas.web.listeners.GuiceServletConfig.contextInitialized(GuiceServletConfig.java:124)
> at
> org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)
> at
> org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)
> at
> org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)
> at
> org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
> at
> org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
> at
> org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
> at
> org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.start(ContainerLifeCycle.java:132)
> at org.eclipse.jetty.server.Server.start(Server.java:387)
> at
> org.eclipse.jetty.util.component.ContainerLifeCycle.doStart(ContainerLifeCycle.java:114)
> at
> org.eclipse.jetty.server.handler.AbstractHandler.doStart(AbstractHandler.java:61)
> at org.eclipse.jetty.server.Server.doStart(Server.java:354)
> at
> org.eclipse.jetty.util.component.AbstractLifeCycle.start(AbstractLifeCycle.java:68)
> at
> org.apache.atlas.web.service.EmbeddedServer.start(EmbeddedServer.java:93)
> at org.apache.atlas.Atlas.main(Atlas.java:107)
> 2016-03-23 11:01:45,555 INFO - [main:] ~ Checking if table
> ATLAS_ENTITY_AUDIT_EVENTS exists (HBaseBasedAuditRep
[jira] [Updated] (ATLAS-595) 'Description/Comment' attribute needs to be normalized hive_db, hive_table
[ https://issues.apache.org/jira/browse/ATLAS-595?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-595: -- Fix Version/s: (was: 0.7-incubating) > 'Description/Comment' attribute needs to be normalized hive_db, hive_table > -- > > Key: ATLAS-595 > URL: https://issues.apache.org/jira/browse/ATLAS-595 > Project: Atlas > Issue Type: Sub-task >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad > > Database comment is not captured in the model -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-609) Secure calls to HBase
[ https://issues.apache.org/jira/browse/ATLAS-609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-609: -- Fix Version/s: (was: 0.7-incubating) > Secure calls to HBase > - > > Key: ATLAS-609 > URL: https://issues.apache.org/jira/browse/ATLAS-609 > Project: Atlas > Issue Type: Sub-task >Reporter: Shwetha G S >Assignee: Shwetha G S > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-671) Include binary dist in release
[
https://issues.apache.org/jira/browse/ATLAS-671?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-671:
--
Fix Version/s: (was: 0.7-incubating)
> Include binary dist in release
> --
>
> Key: ATLAS-671
> URL: https://issues.apache.org/jira/browse/ATLAS-671
> Project: Atlas
> Issue Type: Task
>Reporter: Jean-Baptiste Onofré
>Assignee: Jean-Baptiste Onofré
>
> The 0.6-incubating release is not available on Maven Central (whereas it is
> on {{dist.apache.org}}).
> In addition of the source distribution, it would be helpful for users to be
> able to find binary distribution on Central.
> I propose to enable dist profile on the maven-release-plugin.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-653) insert into local dir, load from local dir does not show up in lineage
[ https://issues.apache.org/jira/browse/ATLAS-653?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-653: -- Fix Version/s: (was: 0.7-incubating) > insert into local dir, load from local dir does not show up in lineage > -- > > Key: ATLAS-653 > URL: https://issues.apache.org/jira/browse/ATLAS-653 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > > All the above commands do not show up in lineage since they dont have any > input entities registered. > Local FS Paths can be modelled without any unique paths/attributes and can be > displayed in lineage -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-704) With Authentication and HA enabled, UI redirection asks for password from already authenticated user
[ https://issues.apache.org/jira/browse/ATLAS-704?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-704: -- Fix Version/s: (was: 0.7-incubating) > With Authentication and HA enabled, UI redirection asks for password from > already authenticated user > > > Key: ATLAS-704 > URL: https://issues.apache.org/jira/browse/ATLAS-704 > Project: Atlas > Issue Type: Bug >Reporter: Hemanth Yamijala >Assignee: Hemanth Yamijala > > Steps to reproduce: > * Enabled both HA and authentication settings (file based) in > atlas-application.properties. > * Bring up two instances. > * Set up a proxy in between as documented in HA documentation. > * Run quick_start on active instance. > * Connect Atlas UI to proxy host and port. It first asks for authentication. > Authenticate with a valid user / password. > * Bring down the active instance. > * Click on any of the links in search page. > This prompts for user login again, although the user is already > authenticated. After logging in, the user context is also lost and work needs > to be repeated. This is an inconvenient user experience. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-702) "ALTER TABLE .. NOT STORED AS DIRECTORIES" metadata update is not sent to Atlas.
[
https://issues.apache.org/jira/browse/ATLAS-702?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-702:
--
Fix Version/s: (was: 0.7-incubating)
> "ALTER TABLE .. NOT STORED AS DIRECTORIES" metadata update is not sent to
> Atlas.
>
>
> Key: ATLAS-702
> URL: https://issues.apache.org/jira/browse/ATLAS-702
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Suma Shivaprasad
>
> 1. CREATE TABLE list_bucket_single (key STRING, value STRING) SKEWED BY (key)
> ON (1,5,6) STORED AS DIRECTORIES;
> {noformat}
> 0: jdbc:hive2://localhost:1/default> describe formatted
> list_bucket_single2;
> +---++---+--+
> | col_name| data_type
>|comment|
> +---++---+--+
> | # col_name| data_type
>| comment |
> | | NULL
>| NULL |
> | key | string
>| |
> | value | string
>| |
> | | NULL
>| NULL |
> | # Detailed Table Information | NULL
>| NULL |
> | Database: | default
>| NULL |
> | Owner:| apathan
>| NULL |
> | CreateTime: | Mon Apr 25 16:27:21 IST 2016
>| NULL |
> | LastAccessTime: | UNKNOWN
>| NULL |
> | Protect Mode: | None
>| NULL |
> | Retention:| 0
>| NULL |
> | Location: |
> hdfs://localhost:9000/user/hive/warehouse/list_bucket_single2 | NULL
> |
> | Table Type: | MANAGED_TABLE
>| NULL |
> | Table Parameters: | NULL
>| NULL |
> | | transient_lastDdlTime
>| 1461581841|
> | | NULL
>| NULL |
> | # Storage Information | NULL
>| NULL |
> | SerDe Library:|
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL
> |
> | InputFormat: | org.apache.hadoop.mapred.TextInputFormat
>| NULL |
> | OutputFormat: |
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL
> |
> | Compressed: | No
>| NULL |
> | Num Buckets: | -1
>| NULL |
> | Bucket Columns: | []
>| NULL |
> | Sort Columns: | []
>| NULL |
> | Stored As SubDirectories: | Yes
>| NULL |
> | Skewed Columns: | [key]
>| NULL |
> | Skewed Values:| [[1], [5], [6]]
>| NULL |
> | Storage Desc Params: | NULL
>| NULL |
[jira] [Updated] (ATLAS-726) UI : dataset lineage process should be sorted according to process start time
[ https://issues.apache.org/jira/browse/ATLAS-726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-726: -- Fix Version/s: (was: 0.7-incubating) > UI : dataset lineage process should be sorted according to process start time > - > > Key: ATLAS-726 > URL: https://issues.apache.org/jira/browse/ATLAS-726 > Project: Atlas > Issue Type: Bug >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt > > Currently transformation processes between datasets(all entities that inherit > from Process) are not displayed in any sort order. They need to be sorted by > start_time when displayed as the transformation between any 2 datasets. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-772) Ordering of columns is not maintained in schema query response, where as hive table entity response maintains the ordering
[
https://issues.apache.org/jira/browse/ATLAS-772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-772:
--
Fix Version/s: (was: 0.7-incubating)
> Ordering of columns is not maintained in schema query response, where as hive
> table entity response maintains the ordering
> --
>
> Key: ATLAS-772
> URL: https://issues.apache.org/jira/browse/ATLAS-772
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>
> Ordering of columns is not maintained in schema query response, where as hive
> table entity response maintains the ordering
> Table schema
> {noformat}
> 0: jdbc:hive2://localhost:1/default> describe formatted table_pbrscdldkm;
> +---+--++--+
> | col_name| data_type
> |comment |
> +---+--++--+
> | # col_name| data_type
> | comment|
> | | NULL
> | NULL |
> | viewtime | int
> ||
> | userid| bigint
> ||
> | page_url | string
> ||
> | referrer_url | string
> ||
> | ip| string
> ||
> | | NULL
> | NULL |
> | # Partition Information | NULL
> | NULL |
> | # col_name| data_type
> | comment|
> | | NULL
> | NULL |
> | dt| string
> ||
> | country | string
> | partitioned columns comments. |
> | | NULL
> | NULL |
> | # Detailed Table Information | NULL
> | NULL |
> | Database: | db2pbrscdldkm
> | NULL |
> | Owner:| apathan
> | NULL |
> | CreateTime: | Tue May 10 16:36:56 IST 2016
> | NULL |
> | LastAccessTime: | UNKNOWN
> | NULL |
> | Protect Mode: | None
> | NULL |
> | Retention:| 0
> | NULL |
> | Location: |
> hdfs://localhost:9000/user/hive/warehouse/db2pbrscdldkm.db/table_pbrscdldkm
> | NULL |
> | Table Type: | MANAGED_TABLE
> | NULL |
> | Table Parameters: | NULL
>
[jira] [Resolved] (ATLAS-775) Rename write action to create action in Atlas Authorization
[
https://issues.apache.org/jira/browse/ATLAS-775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S resolved ATLAS-775.
---
Resolution: Fixed
Looks like its already done:
{code}
public enum AtlasActionTypes {
READ, CREATE, UPDATE, DELETE;
}
{code}
please re-open if its still an issue
> Rename write action to create action in Atlas Authorization
> ---
>
> Key: ATLAS-775
> URL: https://issues.apache.org/jira/browse/ATLAS-775
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Nixon Rodrigues
>Assignee: Saqeeb Shaikh
> Fix For: 0.7-incubating
>
>
> Rename write action to create action in Atlas Authorization.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-778) ALTER TABLE TOUCH is not modelled
[
https://issues.apache.org/jira/browse/ATLAS-778?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-778:
--
Fix Version/s: (was: 0.7-incubating)
> ALTER TABLE TOUCH is not modelled
> -
>
> Key: ATLAS-778
> URL: https://issues.apache.org/jira/browse/ATLAS-778
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>
> ALTER TABLE TOUCH is not modelled.
> ALTER TABLE TOUCH changes metadata of hive table(like create time,
> transient_lastDdlTime). No entity update message is sent to atlas when table
> touch command is fired.
> {noformat}
> 0: jdbc:hive2://localhost:1/default> describe formatted tg2;
> +---+-+---+--+
> | col_name|data_type
> |comment|
> +---+-+---+--+
> | # col_name| data_type
> | comment |
> | | NULL
> | NULL |
> | abc | string
> | |
> | | NULL
> | NULL |
> | # Detailed Table Information | NULL
> | NULL |
> | Database: | db2sdbxnpqone
> | NULL |
> | Owner:| apathan
> | NULL |
> | CreateTime: | Wed May 11 23:19:55 IST 2016
> | NULL |
> | LastAccessTime: | UNKNOWN
> | NULL |
> | Protect Mode: | None
> | NULL |
> | Retention:| 0
> | NULL |
> | Location: |
> hdfs://localhost:9000/user/hive/warehouse/db2sdbxnpqone.db/tg2 | NULL
> |
> | Table Type: | MANAGED_TABLE
> | NULL |
> | Table Parameters: | NULL
> | NULL |
> | | transient_lastDdlTime
> | 1462988995|
> | | NULL
> | NULL |
> | # Storage Information | NULL
> | NULL |
> | SerDe Library:|
> org.apache.hadoop.hive.serde2.lazy.LazySimpleSerDe | NULL
> |
> | InputFormat: | org.apache.hadoop.mapred.TextInputFormat
> | NULL |
> | OutputFormat: |
> org.apache.hadoop.hive.ql.io.HiveIgnoreKeyTextOutputFormat | NULL
> |
> | Compressed: | No
> | NULL |
> | Num Buckets: | -1
> | NULL |
> | Bucket Columns: | []
> | NULL |
> | Sort Columns: | []
> | NULL |
> | Storage Desc Params: | NULL
> | NULL |
> | | serialization.format
> | 1 |
> +---+-+---+--+
> 26 rows selected (0.113 seconds)
> 0: jdbc:hive2://localhost:1/default> alter table tg2 touch;
> No rows affected (0.173 seconds)
> 0: jdbc:hive2://localhost:1/default> describe formatted tg2;
> +---+---
[jira] [Updated] (ATLAS-806) Create default taxonomy at server startup
[ https://issues.apache.org/jira/browse/ATLAS-806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-806: -- Fix Version/s: (was: 0.7-incubating) > Create default taxonomy at server startup > - > > Key: ATLAS-806 > URL: https://issues.apache.org/jira/browse/ATLAS-806 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Hemanth Yamijala >Assignee: John Speidel > Attachments: ATLAS-806.patch > > > Currently, the UI is creating a default taxonomy object as the first call in > its flow. This would not work if used from an API. Hence, creating this > automatically at server startup time will help. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-767) Atlas utilities can improve error reporting on authentication / authorization errors
[ https://issues.apache.org/jira/browse/ATLAS-767?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-767: -- Fix Version/s: (was: 0.7-incubating) > Atlas utilities can improve error reporting on authentication / authorization > errors > > > Key: ATLAS-767 > URL: https://issues.apache.org/jira/browse/ATLAS-767 > Project: Atlas > Issue Type: Bug >Reporter: Hemanth Yamijala > > Atlas utilities print HTML code as error in case of authentication / > authorization exceptions. This is hard to read for end users. It would be > good to provide a better error message. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-814) Too many logs from Atlas authorization
[
https://issues.apache.org/jira/browse/ATLAS-814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-814:
--
Fix Version/s: (was: 0.7-incubating)
> Too many logs from Atlas authorization
> --
>
> Key: ATLAS-814
> URL: https://issues.apache.org/jira/browse/ATLAS-814
> Project: Atlas
> Issue Type: Bug
>Reporter: Shwetha G S
>Assignee: Nixon Rodrigues
>
> For every request, the following debug logs are printed. I don't think some
> of them have any useful info. Please cleanup
> {noformat}
> 2016-05-20 14:37:24,963 DEBUG - [qtp1539768275-18:] ~ ==>
> AuthorizationFilter.doFilter (AtlasAuthorizationFilter:81)
> 2016-05-20 14:37:24,963 DEBUG - [qtp1539768275-18:] ~
> /api/atlas/lineage/a8fcde45-7619-482a-832e-84cb84c71a2a/schema is a valid
> REST API request!!! (AtlasAuthorizationFilter:88)
> 2016-05-20 14:37:24,963 DEBUG - [qtp1539768275-18:] ~ ==>
> AtlasAuthorizationFilter getAtlasAction HTTP Method GET mapped to AtlasAction
> : READ (AtlasAuthorizationUtils:89)
> 2016-05-20 14:37:24,963 DEBUG - [qtp1539768275-18:] ~
>
> UserName ==>> admin
> Groups ==>> [ADMIN]
> URL ::
> http://localhost:21000/api/atlas/lineage/a8fcde45-7619-482a-832e-84cb84c71a2a/schema
> Action :: READ
> request.getServletPath() ::
> /api/atlas/lineage/a8fcde45-7619-482a-832e-84cb84c71a2a/schema
>
> (AtlasAuthorizationFilter:118)
> 2016-05-20 14:37:24,963 DEBUG - [qtp1539768275-18:] ~ getAtlasResourceType
> <=== for /api/atlas/lineage/a8fcde45-7619-482a-832e-84cb84c71a2a/schema
> (AtlasAuthorizationUtils:98)
> 2016-05-20 14:37:24,963 DEBUG - [qtp1539768275-18:] ~ getApi <=== from
> /api/atlas/lineage/a8fcde45-7619-482a-832e-84cb84c71a2a/schema
> (AtlasAuthorizationUtils:52)
> 2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ Extracted
> lineage/a8fcde45-7619-482a-832e-84cb84c71a2a/schema from path :
> /api/atlas/lineage/a8fcde45-7619-482a-832e-84cb84c71a2a/schema
> (AtlasAuthorizationUtils:45)
> 2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ Returning
> AtlasResources [ENTITY] for api lineage (AtlasAuthorizationUtils:133)
> 2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ getAtlasResource <===
> This implementation will be changed for Resource level Authorization.
> (AtlasAuthorizationUtils:143)
> 2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ <==
> AtlasAccessRequestImpl-- Initializing AtlasAccessRequest
> (AtlasAccessRequest:41)
> 2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ <==
> SimpleAtlasAuthorizer isAccessAllowed (SimpleAtlasAuthorizer:116)
> 2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~
> isAccessAllowd(AtlasAccessRequest [resourceType=[ENTITY], resource=*,
> action=READ, user=admin, userGroups=[ADMIN], accessTime=null,
> clientIPAddress=null]) (SimpleAtlasAuthorizer:117)
> 2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ Checking for ::
> User :: admin
> Groups :: [ADMIN]
> Action :: READ
> Resource :: * (SimpleAtlasAuthorizer:125)
> 2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ checkAccess for
> Operation :: READ on Resource [ENTITY]:* (SimpleAtlasAuthorizer:139)
> 2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ <==
> SimpleAtlasAuthorizer checkAccess (SimpleAtlasAuthorizer:185)
> 2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ Now checking access for
> accessor : admin
> Resource Types : [ENTITY]
> Resource : *
> Map : {saqeeb={ENTITY=[*], TERM=[*], TYPE=[*], TAXONOMY=[*]},
> admin={OPERATION=[*], ENTITY=[*], TERM=[*], TYPE=[*], TAXONOMY=[*]},
> nixon={ENTITY=[*], TERM=[*], TYPE=[*], TAXONOMY=[*]}}
> (SimpleAtlasAuthorizer:186)
> 2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~
> Checking for resource : * in list : [*]
> (SimpleAtlasAuthorizer:195)
> 2016-05-20 14:37:24,964 DEBUG - [qtp1539768275-18:] ~ <==
> SimpleAtlasAuthorizer isMatch (SimpleAtlasAuthorizer:272)
> 2016-05-20 14:37:24,965 DEBUG - [qtp1539768275-18:] ~ <==
> SimpleAtlasAuthorizer resourceMatchHelper (SimpleAtlasAuthorizer:239)
> 2016-05-20 14:37:24,965 DEBUG - [qtp1539768275-18:] ~ ==>
> SimpleAtlasAuthorizer resourceMatchHelper (SimpleAtlasAuthorizer:265)
> 2016-05-20 14:37:24,965 DEBUG - [qtp1539768275-18:] ~ ==>
> SimpleAtlasAuthorizer isMatch(*): true (SimpleAtlasAuthorizer:315)
> 2016-05-20 14:37:24,965 DEBUG - [qtp1539768275-18:] ~ Check for admin :: true
> (SimpleAtlasAuthorizer:210)
> 2016-05-20 14:37:24,965 DEBUG - [qtp1539768275-18:] ~ ==>
> SimpleAtlasAuthorizer checkAccess (SimpleAtlasAuthorizer:211)
> 2016-05-20 14:37:24,965 DEBUG - [qtp1539768275-18:] ~ ==>
> +SimpleAtlasAuthorizer isAccessAllowed = true (SimpleAtlasAuthorizer:176)
> 2016-05-20 14:37:24,965 DEBUG - [qtp1539768275-18:] ~ Authorizer result ::
> true (AtlasAuthorizationFilter:134)
> 2016-05-20 14:37:24,965 DEBUG - [q
[jira] [Updated] (ATLAS-815) Authentication error message
[
https://issues.apache.org/jira/browse/ATLAS-815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-815:
--
Fix Version/s: (was: 0.7-incubating)
> Authentication error message
>
>
> Key: ATLAS-815
> URL: https://issues.apache.org/jira/browse/ATLAS-815
> Project: Atlas
> Issue Type: Bug
>Reporter: Shwetha G S
>
> Currently, UI maps username/password errors to invalid credentials. But even
> REST APIs should return 'invalid credentials'
> {noformat}
> apache-atlas-0.7-incubating-SNAPSHOT$ curl -u admin2:admin
> http://localhost:21000/api/atlas/admin/version
>
>
>
> Error 401 Username not found.admin2
>
> HTTP ERROR 401
> Problem accessing /api/atlas/admin/version. Reason:
> Username not found.admin2Powered by
> Jetty://
>
>
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-827) Add audit implementation for non-hbase environment
[ https://issues.apache.org/jira/browse/ATLAS-827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-827: -- Fix Version/s: (was: 0.7-incubating) > Add audit implementation for non-hbase environment > -- > > Key: ATLAS-827 > URL: https://issues.apache.org/jira/browse/ATLAS-827 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S > > Currently there is HBaseBasedAuditRepository which is hbase based > implementation for audit store. In the case that hbase is not used for titan, > there is no alternative implementation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-836) Add timeout for kafka notifications
[ https://issues.apache.org/jira/browse/ATLAS-836?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-836: -- Fix Version/s: (was: 0.7-incubating) > Add timeout for kafka notifications > --- > > Key: ATLAS-836 > URL: https://issues.apache.org/jira/browse/ATLAS-836 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Vimal Sharma > > In atlas conf, we don't have timeout for calls to kafka and have seen cases > where the kafka calls get stuck for a long time. We should have reasonable > timeouts -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-832) atlas_start should start embedded hbase and solr with -Pembedded_services
[ https://issues.apache.org/jira/browse/ATLAS-832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-832: -- Fix Version/s: (was: 0.7-incubating) > atlas_start should start embedded hbase and solr with -Pembedded_services > - > > Key: ATLAS-832 > URL: https://issues.apache.org/jira/browse/ATLAS-832 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Vimal Sharma > > As part of ATLAS-823, embedded hbase and solr was moved to profile > 'embedded_services'. After the package is built using '-Pembedded_services', > the package configuration points to local hbase and solr, but atlas_start > doesn't start embedded hbase and solr -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-855) Atlas logs contain stale transaction eviction message
[
https://issues.apache.org/jira/browse/ATLAS-855?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-855:
--
Fix Version/s: (was: 0.7-incubating)
> Atlas logs contain stale transaction eviction message
> -
>
> Key: ATLAS-855
> URL: https://issues.apache.org/jira/browse/ATLAS-855
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Hemanth Yamijala
>Priority: Critical
> Attachments: application.log.tar.gz
>
>
> In situations that are hard to replicate, we are getting a message in the
> Atlas server logs as follows:
> {code}
> Evicted [16@c0a801069313-hemanthijalambp-home1] from cache but waiting too
> long for transactions to close. Stale transaction alert on:
> [standardtitantx[null]] (ManagementLogger:189)
> {code}
> I started seeing this a lot more often with the business catalog delete
> functionality (ATLAS-793), although it is not consistent with this.
> This jira is to track down the cause and fix it.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-881) UI: not displaying multiple taxonomies
[
https://issues.apache.org/jira/browse/ATLAS-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-881:
--
Fix Version/s: (was: 0.7-incubating)
> UI: not displaying multiple taxonomies
> --
>
> Key: ATLAS-881
> URL: https://issues.apache.org/jira/browse/ATLAS-881
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Keval Bhatt
>Priority: Critical
>
> UI: not displaying multiple taxonomies
> API response returns multiple taxonomies
> {noformat}
> curl 'http://localhost:21000/api/atlas/v1/taxonomies/' -H 'Cookie:
> JSESSIONID=19eplhd4dz8mueh7466qcbjwf' -H 'Accept-Encoding: gzip, deflate,
> sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0
> (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko)
> Chrome/50.0.2661.102 Safari/537.36' -H 'Accept: application/json,
> text/javascript, */*; q=0.01' -H 'Referer: http://localhost:21000/index.html'
> -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive'
> --compressed | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 3310 3310 0 4096 0 --:--:-- --:--:-- --:--:-- 4137
> [
> {
> "href": "http://localhost:21000/api/atlas/v1/taxonomies/empty";,
> "name": "empty"
> },
> {
> "href": "http://localhost:21000/api/atlas/v1/taxonomies/tax1";,
> "name": "tax1"
> },
> {
> "href": "http://localhost:21000/api/atlas/v1/taxonomies/default";,
> "name": "default"
> }
> ]
> {noformat}
> UI snapshot: https://monosnap.com/file/1M6dOB8D08IPMqkqj3LsCr8VYvXWJG
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-879) Business catalog: Terms are audited as tags.
[ https://issues.apache.org/jira/browse/ATLAS-879?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-879: -- Fix Version/s: (was: 0.7-incubating) > Business catalog: Terms are audited as tags. > > > Key: ATLAS-879 > URL: https://issues.apache.org/jira/browse/ATLAS-879 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Ayub Khan > > Is there a need to differentiate terms from tags from an audit perspective? > From a implementation point of view, terms are modelled as tags. But from an > end user perspective, terms are different from tags. In auditing, terms audit > logs are displayed as tag audit. > UI Snapshot: https://monosnap.com/file/560YS5CRHRioAoRbdDpRSP7Alm8o2B -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-881) UI: not displaying multiple taxonomies
[
https://issues.apache.org/jira/browse/ATLAS-881?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15339209#comment-15339209
]
Shwetha G S commented on ATLAS-881:
---
0.7 release will not support multiple taxonomies
> UI: not displaying multiple taxonomies
> --
>
> Key: ATLAS-881
> URL: https://issues.apache.org/jira/browse/ATLAS-881
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Keval Bhatt
>Priority: Critical
>
> UI: not displaying multiple taxonomies
> API response returns multiple taxonomies
> {noformat}
> curl 'http://localhost:21000/api/atlas/v1/taxonomies/' -H 'Cookie:
> JSESSIONID=19eplhd4dz8mueh7466qcbjwf' -H 'Accept-Encoding: gzip, deflate,
> sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0
> (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko)
> Chrome/50.0.2661.102 Safari/537.36' -H 'Accept: application/json,
> text/javascript, */*; q=0.01' -H 'Referer: http://localhost:21000/index.html'
> -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive'
> --compressed | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 3310 3310 0 4096 0 --:--:-- --:--:-- --:--:-- 4137
> [
> {
> "href": "http://localhost:21000/api/atlas/v1/taxonomies/empty";,
> "name": "empty"
> },
> {
> "href": "http://localhost:21000/api/atlas/v1/taxonomies/tax1";,
> "name": "tax1"
> },
> {
> "href": "http://localhost:21000/api/atlas/v1/taxonomies/default";,
> "name": "default"
> }
> ]
> {noformat}
> UI snapshot: https://monosnap.com/file/1M6dOB8D08IPMqkqj3LsCr8VYvXWJG
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-882) with multiple taxonomies, UI does not allow associating terms with entities. "Add Terms" drop down shows all the taxonomies
[ https://issues.apache.org/jira/browse/ATLAS-882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-882: -- Fix Version/s: (was: 0.7-incubating) > with multiple taxonomies, UI does not allow associating terms with entities. > "Add Terms" drop down shows all the taxonomies > --- > > Key: ATLAS-882 > URL: https://issues.apache.org/jira/browse/ATLAS-882 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Ayub Khan >Assignee: Keval Bhatt > Attachments: Screen Shot 2016-06-08 at 4.20.45 PM.png > > > with multiple taxonomies, UI does not allow associating terms with entities. > "Add Terms" drop down shows all the taxonomies > Attaching UI snapshot. > Create multiple taxonomies with terms and try associating terms to entities, > UI does not allow it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (ATLAS-647) entityText property should be prefixed with __
[ https://issues.apache.org/jira/browse/ATLAS-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S reassigned ATLAS-647: - Assignee: Shwetha G S > entityText property should be prefixed with __ > -- > > Key: ATLAS-647 > URL: https://issues.apache.org/jira/browse/ATLAS-647 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.7-incubating > > > All internal properties for the vertex in the repository are prefixed with __ > to avoid conflicts with user defined attribute names. Looks like entityText > is missed out -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-916) Return system attributes in get entity definition
Shwetha G S created ATLAS-916: - Summary: Return system attributes in get entity definition Key: ATLAS-916 URL: https://issues.apache.org/jira/browse/ATLAS-916 Project: Atlas Issue Type: Improvement Reporter: Shwetha G S Atlas should maintain system attributes created time, last modified time, created by user and last modified by user for every entity. This information should be returned in get entity definition -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-819) All user defined types should have a set of common attributes
[ https://issues.apache.org/jira/browse/ATLAS-819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-819: -- Attachment: ATLAS-819.patch Added base type for attributes name, description and owner. Will create another jira for returning system attributes > All user defined types should have a set of common attributes > - > > Key: ATLAS-819 > URL: https://issues.apache.org/jira/browse/ATLAS-819 > Project: Atlas > Issue Type: Bug >Reporter: Hemanth Yamijala >Assignee: Shwetha G S > Fix For: 0.7-incubating > > Attachments: ATLAS-819.patch > > > It would be very convenient if all user defined types have a conventional set > of common attributes including: > * name > * description > * owner > * created at > * modified at -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-819) All user defined types should have a set of common attributes
[ https://issues.apache.org/jira/browse/ATLAS-819?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15339054#comment-15339054 ] Shwetha G S commented on ATLAS-819: --- I will add a base type with the attributes name, owner and description that any type can extend and the users can set these attributes. However, the attributes like createdby, modifiedby, createtime and modifiedtime are system attributes that are set by atlas and are returned along with id and state in get entity details. > All user defined types should have a set of common attributes > - > > Key: ATLAS-819 > URL: https://issues.apache.org/jira/browse/ATLAS-819 > Project: Atlas > Issue Type: Bug >Reporter: Hemanth Yamijala >Assignee: Shwetha G S > Fix For: 0.7-incubating > > > It would be very convenient if all user defined types have a conventional set > of common attributes including: > * name > * description > * owner > * created at > * modified at -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-819) All user defined types should have a set of common attributes
[ https://issues.apache.org/jira/browse/ATLAS-819?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-819: -- Fix Version/s: 0.7-incubating > All user defined types should have a set of common attributes > - > > Key: ATLAS-819 > URL: https://issues.apache.org/jira/browse/ATLAS-819 > Project: Atlas > Issue Type: Bug >Reporter: Hemanth Yamijala >Assignee: Shwetha G S > Fix For: 0.7-incubating > > > It would be very convenient if all user defined types have a conventional set > of common attributes including: > * name > * description > * owner > * created at > * modified at -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-810) CTAS tables creation throws exceptions in Atlas ,but the tables are imported and lineage graphs are also rendered.
[ https://issues.apache.org/jira/browse/ATLAS-810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-810. --- Resolution: Duplicate > CTAS tables creation throws exceptions in Atlas ,but the tables are imported > and lineage graphs are also rendered. > -- > > Key: ATLAS-810 > URL: https://issues.apache.org/jira/browse/ATLAS-810 > Project: Atlas > Issue Type: Bug > Environment: Atlas (commit id : > 98769871e56d9a97792e2dba52345e876908ac63 ) > HBase , External Kafka, Solr >Reporter: Sharmadha Sainath >Assignee: Shwetha G S >Priority: Minor > > Atlas is populated with "n" tables and then CTAS tables are created from > those"n" tables. > Atlas throws the following exceptions : > 2016-05-19 02:59:39,362 ERROR - [NotificationHookConsumer thread-0:] ~ Unable > to persist entity instance (EntityResource:148) > java.lang.NullPointerException > at > org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:124) > at org.apache.atlas.LocalAtlasClient$1.invoke(LocalAtlasClient.java:82) > at > org.apache.atlas.LocalAtlasClient$EntityOperation.run(LocalAtlasClient.java:118) > at > org.apache.atlas.LocalAtlasClient.createEntity(LocalAtlasClient.java:85) > at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:580) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:203) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:187) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2016-05-19 02:59:39,379 WARN - [NotificationHookConsumer thread-0:] ~ Error > handling message > org.apache.atlas.notification.hook.HookNotification$EntityCreateRequest@24fc6736 > (NotificationHookConsumer:233) > org.apache.atlas.AtlasServiceException: Metadata service API CREATE_ENTITY > failed with status 500(Internal Server Error) Response Body > (java.lang.NullPointerException > at > org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:124) > at org.apache.atlas.LocalAtlasClient$1.invoke(LocalAtlasClient.java:82) > at > org.apache.atlas.LocalAtlasClient$EntityOperation.run(LocalAtlasClient.java:118) > at > org.apache.atlas.LocalAtlasClient.createEntity(LocalAtlasClient.java:85) > at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:580) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:203) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:187) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > ) > at > org.apache.atlas.LocalAtlasClient$EntityOperation.run(LocalAtlasClient.java:122) > at > org.apache.atlas.LocalAtlasClient.createEntity(LocalAtlasClient.java:85) > at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:580) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:203) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:187) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2016-05-19 02:59:39,380 DEBUG - [NotificationHookConsumer thread-0:] ~ > Committed offset: 2 (KafkaConsumer:96) > No exceptions in Hive logs. > But all CTAS tables are imported and lineage graphs are properly rendered on > Atlas UI. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-499) Automatically update inverse references to prevent repository corruption
[ https://issues.apache.org/jira/browse/ATLAS-499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15336638#comment-15336638 ] Shwetha G S commented on ATLAS-499: --- Since you updated the patch in reviewboard, thought it must be complete. Sure, thanks > Automatically update inverse references to prevent repository corruption > > > Key: ATLAS-499 > URL: https://issues.apache.org/jira/browse/ATLAS-499 > Project: Atlas > Issue Type: Improvement >Reporter: David Kantor >Assignee: David Kantor > Fix For: 0.7-incubating > > > For references that are defined in the type system as bi-directional, Atlas > does not enforce that clients update both sides of the reference, and relies > on well behaved client applications for repository consistency. This allows > the repository to be easily corrupted with unbalanced references. Atlas > should automatically update the inverse reference to avoid this corruption. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-835) Falcon Integration with Atlas
[
https://issues.apache.org/jira/browse/ATLAS-835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-835:
--
Attachment: ATLAS-835-v3.patch
Fixed storm error
> Falcon Integration with Atlas
> -
>
> Key: ATLAS-835
> URL: https://issues.apache.org/jira/browse/ATLAS-835
> Project: Atlas
> Issue Type: New Feature
>Reporter: Sowmya Ramesh
>Assignee: Sowmya Ramesh
> Fix For: trunk
>
> Attachments: ATLAS-835-v1.patch, ATLAS-835-v2.patch,
> ATLAS-835-v3.patch, ATLAS-835.patch
>
>
> Currently only Falcon process registration is supported in Atlas. Extend this
> to support Falcon feed, replication feed, cluster entity addition and updates.
> Here are the types to be defined:
> {noformat}
> falcon_cluster - models falcon cluster
> falcon_feed - extends Dataset - models falcon feed
> falcon_process - extends Process - models falcon process with inputs and
> outputs as falcon_feed
> falcon_feed_creation - extends Process - input as hive_table or hdfs_path,
> output as falcon_feed
> falcon_replication - extends Process - input as falcon_feed in source
> cluster, output as falcon_feed in target cluster
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ATLAS-499) Automatically update inverse references to prevent repository corruption
[ https://issues.apache.org/jira/browse/ATLAS-499?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15336591#comment-15336591 ] Shwetha G S commented on ATLAS-499: --- [~dkantor], is this patch available for review, is it complete? If yes, can you make it patch available? Thanks > Automatically update inverse references to prevent repository corruption > > > Key: ATLAS-499 > URL: https://issues.apache.org/jira/browse/ATLAS-499 > Project: Atlas > Issue Type: Improvement >Reporter: David Kantor >Assignee: David Kantor > Fix For: 0.7-incubating > > > For references that are defined in the type system as bi-directional, Atlas > does not enforce that clients update both sides of the reference, and relies > on well behaved client applications for repository consistency. This allows > the repository to be easily corrupted with unbalanced references. Atlas > should automatically update the inverse reference to avoid this corruption. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-877) CreateTime of an entity(for example hive table) changes for every update
[
https://issues.apache.org/jira/browse/ATLAS-877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15336555#comment-15336555
]
Shwetha G S commented on ATLAS-877:
---
+1
> CreateTime of an entity(for example hive table) changes for every update
>
>
> Key: ATLAS-877
> URL: https://issues.apache.org/jira/browse/ATLAS-877
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Suma Shivaprasad
>Priority: Critical
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-877.patch
>
>
> Each time a hive table is updated(example column renamed), create time of
> table is also updated.
> UI snapshot: https://monosnap.com/file/6NoT86CNpHgQmlyfBQTSUa1y3Ncc2Y
> hive table metadata snapshot from hive cli
> {noformat}
> 0: jdbc:hive2://localhost:1/default> describe formatted
> column_changepgwcbgq1bd;
> OK
> +---+-+---+--+
> | col_name|
> data_type |comment|
> +---+-+---+--+
> | # col_name| data_type
> | comment |
> | | NULL
> | NULL |
> | a3| int
> | |
> | b | int
> | |
> | c | int
> | |
> | d | int
> | |
> | e | int
> | |
> | f | int
> | |
> | | NULL
> | NULL |
> | # Detailed Table Information | NULL
> | NULL |
> | Database: | db2pgwcbgq1bd
> | NULL |
> | Owner:| apathan
> | NULL |
> | CreateTime: | Wed Jun 08 12:41:28 IST 2016
> | NULL |
> | LastAccessTime: | UNKNOWN
> | NULL |
> | Protect Mode: | None
> | NULL |
> | Retention:| 0
> | NULL |
> | Location: |
> hdfs://localhost:9000/user/hive/warehouse/db2pgwcbgq1bd.db/column_changepgwcbgq1bd
> | NULL |
> | Table Type: | MANAGED_TABLE
> | NULL |
> | Table Parameters: | NULL
> | NULL |
> | | COLUMN_STATS_ACCURATE
> | false |
> | | last_modified_by
> | apathan |
> | | last_modified_time
> | 1465370834|
> | | numFiles
> | 0 |
> |
[jira] [Updated] (ATLAS-835) Falcon Integration with Atlas
[
https://issues.apache.org/jira/browse/ATLAS-835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-835:
--
Attachment: ATLAS-835-v2.patch
> Falcon Integration with Atlas
> -
>
> Key: ATLAS-835
> URL: https://issues.apache.org/jira/browse/ATLAS-835
> Project: Atlas
> Issue Type: New Feature
>Reporter: Sowmya Ramesh
>Assignee: Sowmya Ramesh
> Fix For: trunk
>
> Attachments: ATLAS-835-v1.patch, ATLAS-835-v2.patch, ATLAS-835.patch
>
>
> Currently only Falcon process registration is supported in Atlas. Extend this
> to support Falcon feed, replication feed, cluster entity addition and updates.
> Here are the types to be defined:
> {noformat}
> falcon_cluster - models falcon cluster
> falcon_feed - extends Dataset - models falcon feed
> falcon_process - extends Process - models falcon process with inputs and
> outputs as falcon_feed
> falcon_feed_creation - extends Process - input as hive_table or hdfs_path,
> output as falcon_feed
> falcon_replication - extends Process - input as falcon_feed in source
> cluster, output as falcon_feed in target cluster
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-835) Falcon Integration with Atlas
[
https://issues.apache.org/jira/browse/ATLAS-835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-835:
--
Attachment: ATLAS-835-v1.patch
re-based patch
> Falcon Integration with Atlas
> -
>
> Key: ATLAS-835
> URL: https://issues.apache.org/jira/browse/ATLAS-835
> Project: Atlas
> Issue Type: New Feature
>Reporter: Sowmya Ramesh
>Assignee: Sowmya Ramesh
> Fix For: trunk
>
> Attachments: ATLAS-835-v1.patch, ATLAS-835.patch
>
>
> Currently only Falcon process registration is supported in Atlas. Extend this
> to support Falcon feed, replication feed, cluster entity addition and updates.
> Here are the types to be defined:
> {noformat}
> falcon_cluster - models falcon cluster
> falcon_feed - extends Dataset - models falcon feed
> falcon_process - extends Process - models falcon process with inputs and
> outputs as falcon_feed
> falcon_feed_creation - extends Process - input as hive_table or hdfs_path,
> output as falcon_feed
> falcon_replication - extends Process - input as falcon_feed in source
> cluster, output as falcon_feed in target cluster
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-835) Falcon Integration with Atlas
[
https://issues.apache.org/jira/browse/ATLAS-835?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-835:
--
Attachment: ATLAS-835.patch
Attached patch with test fixes, tests added and other fixes:
1. In hooks, messages was a global variable. Moved to local variable per
invocation
2. Added falcon tests
> Falcon Integration with Atlas
> -
>
> Key: ATLAS-835
> URL: https://issues.apache.org/jira/browse/ATLAS-835
> Project: Atlas
> Issue Type: New Feature
>Reporter: Sowmya Ramesh
>Assignee: Sowmya Ramesh
> Fix For: trunk
>
> Attachments: ATLAS-835.patch
>
>
> Currently only Falcon process registration is supported in Atlas. Extend this
> to support Falcon feed, replication feed, cluster entity addition and updates.
> Here are the types to be defined:
> {noformat}
> falcon_cluster - models falcon cluster
> falcon_feed - extends Dataset - models falcon feed
> falcon_process - extends Process - models falcon process with inputs and
> outputs as falcon_feed
> falcon_feed_creation - extends Process - input as hive_table or hdfs_path,
> output as falcon_feed
> falcon_replication - extends Process - input as falcon_feed in source
> cluster, output as falcon_feed in target cluster
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ATLAS-911) Get entity by unique attribute doesn't enforce type
[ https://issues.apache.org/jira/browse/ATLAS-911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15335457#comment-15335457 ] Shwetha G S commented on ATLAS-911: --- Have verified the build succeeds. Committed the patch. Thanks Hemanth for reviewing > Get entity by unique attribute doesn't enforce type > --- > > Key: ATLAS-911 > URL: https://issues.apache.org/jira/browse/ATLAS-911 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-911.patch > > > Currently, get entity by unique attribute just filters on the attribute > property. Since the attribute name contains the typename as well, type filter > was not added. But if multiple types have the same super type, and the super > type has the unique attribute, the attribute name will be same for multiple > types. With this, the type returned will not be same as the type requested. > We should add filter on the type as well -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-911) Get entity by unique attribute doesn't enforce type
[ https://issues.apache.org/jira/browse/ATLAS-911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15335452#comment-15335452 ] Shwetha G S commented on ATLAS-911: --- The attached patch fixes the tests as well > Get entity by unique attribute doesn't enforce type > --- > > Key: ATLAS-911 > URL: https://issues.apache.org/jira/browse/ATLAS-911 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-911.patch > > > Currently, get entity by unique attribute just filters on the attribute > property. Since the attribute name contains the typename as well, type filter > was not added. But if multiple types have the same super type, and the super > type has the unique attribute, the attribute name will be same for multiple > types. With this, the type returned will not be same as the type requested. > We should add filter on the type as well -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-911) Get entity by unique attribute doesn't enforce type
[ https://issues.apache.org/jira/browse/ATLAS-911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-911: -- Attachment: ATLAS-911.patch > Get entity by unique attribute doesn't enforce type > --- > > Key: ATLAS-911 > URL: https://issues.apache.org/jira/browse/ATLAS-911 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-911.patch > > > Currently, get entity by unique attribute just filters on the attribute > property. Since the attribute name contains the typename as well, type filter > was not added. But if multiple types have the same super type, and the super > type has the unique attribute, the attribute name will be same for multiple > types. With this, the type returned will not be same as the type requested. > We should add filter on the type as well -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-911) Get entity by unique attribute doesn't enforce type
[ https://issues.apache.org/jira/browse/ATLAS-911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-911: -- Attachment: (was: ATLAS-911.patch) > Get entity by unique attribute doesn't enforce type > --- > > Key: ATLAS-911 > URL: https://issues.apache.org/jira/browse/ATLAS-911 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Critical > > Currently, get entity by unique attribute just filters on the attribute > property. Since the attribute name contains the typename as well, type filter > was not added. But if multiple types have the same super type, and the super > type has the unique attribute, the attribute name will be same for multiple > types. With this, the type returned will not be same as the type requested. > We should add filter on the type as well -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-911) Get entity by unique attribute doesn't enforce type
[ https://issues.apache.org/jira/browse/ATLAS-911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-911: -- Attachment: ATLAS-911.patch Some test are failing with this patch. Need to investigate. Attaching if someone is blocked on this > Get entity by unique attribute doesn't enforce type > --- > > Key: ATLAS-911 > URL: https://issues.apache.org/jira/browse/ATLAS-911 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S >Priority: Critical > Attachments: ATLAS-911.patch > > > Currently, get entity by unique attribute just filters on the attribute > property. Since the attribute name contains the typename as well, type filter > was not added. But if multiple types have the same super type, and the super > type has the unique attribute, the attribute name will be same for multiple > types. With this, the type returned will not be same as the type requested. > We should add filter on the type as well -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-911) Get entity by unique attribute doesn't enforce type
Shwetha G S created ATLAS-911: - Summary: Get entity by unique attribute doesn't enforce type Key: ATLAS-911 URL: https://issues.apache.org/jira/browse/ATLAS-911 Project: Atlas Issue Type: Bug Reporter: Shwetha G S Assignee: Shwetha G S Priority: Critical Currently, get entity by unique attribute just filters on the attribute property. Since the attribute name contains the typename as well, type filter was not added. But if multiple types have the same super type, and the super type has the unique attribute, the attribute name will be same for multiple types. With this, the type returned will not be same as the type requested. We should add filter on the type as well -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-910) Avoid un-necessary configs to kafka
Shwetha G S created ATLAS-910:
-
Summary: Avoid un-necessary configs to kafka
Key: ATLAS-910
URL: https://issues.apache.org/jira/browse/ATLAS-910
Project: Atlas
Issue Type: Bug
Reporter: Shwetha G S
Priority: Minor
Remove un-necessary configs for kafka:
{noformat}
2016-06-16 14:11:17,277 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration port = 19027 was supplied but isn't a known config.
(AbstractConfig:173)
2016-06-16 14:11:17,277 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration log.flush.interval.messages = 1 was supplied but isn't a known
config. (AbstractConfig:173)
2016-06-16 14:11:17,277 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration consumer.timeout.ms = 100 was supplied but isn't a known config.
(AbstractConfig:173)
2016-06-16 14:11:17,277 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration hook.group.id = atlas was supplied but isn't a known config.
(AbstractConfig:173)
2016-06-16 14:11:17,277 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration zookeeper.sync.time.ms = 20 was supplied but isn't a known
config. (AbstractConfig:173)
2016-06-16 14:11:17,277 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration data =
/Users/sshivalingamurthy/git/atlas-clone/addons/falcon-bridge/target/data/kafka
was supplied but isn't a known config. (AbstractConfig:173)
2016-06-16 14:11:17,277 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration entities.group.id = atlas_entities was supplied but isn't a known
config. (AbstractConfig:173)
2016-06-16 14:11:17,277 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration zookeeper.session.timeout.ms = 4000 was supplied but isn't a
known config. (AbstractConfig:173)
2016-06-16 14:11:17,277 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration auto.commit.interval.ms = 100 was supplied but isn't a known
config. (AbstractConfig:173)
2016-06-16 14:11:17,277 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration log.dirs =
/Users/sshivalingamurthy/git/atlas-clone/addons/falcon-bridge/target/data/kafka/kafka
was supplied but isn't a known config. (AbstractConfig:173)
2016-06-16 14:11:17,277 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration zookeeper.connect = localhost:19026 was supplied but isn't a
known config. (AbstractConfig:173)
2016-06-16 14:11:17,277 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration host.name = localhost was supplied but isn't a known config.
(AbstractConfig:173)
2016-06-16 14:11:17,277 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration key.deserializer =
org.apache.kafka.common.serialization.StringDeserializer was supplied but isn't
a known config. (AbstractConfig:173)
2016-06-16 14:11:17,277 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration auto.offset.reset = smallest was supplied but isn't a known
config. (AbstractConfig:173)
2016-06-16 14:11:17,278 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration broker.id = 1 was supplied but isn't a known config.
(AbstractConfig:173)
2016-06-16 14:11:17,278 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration value.deserializer =
org.apache.kafka.common.serialization.StringDeserializer was supplied but isn't
a known config. (AbstractConfig:173)
2016-06-16 14:11:17,278 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration auto.commit.enable = false was supplied but isn't a known config.
(AbstractConfig:173)
2016-06-16 14:11:17,278 WARN - [NotificationHookConsumer thread-0:] ~ The
configuration partition.assignment.strategy = roundrobin was supplied but isn't
a known config. (AbstractConfig:173)
2016-06-16 14:12:04,763 WARN - [main:] ~ Property bootstrap.servers is not
valid (Logging$class:83)
2016-06-16 14:12:04,764 WARN - [main:] ~ Property broker.id is not valid
(Logging$class:83)
2016-06-16 14:12:04,764 WARN - [main:] ~ Property data is not valid
(Logging$class:83)
2016-06-16 14:12:04,764 WARN - [main:] ~ Property entities.group.id is not
valid (Logging$class:83)
2016-06-16 14:12:04,764 WARN - [main:] ~ Property hook.group.id is not valid
(Logging$class:83)
2016-06-16 14:12:04,764 WARN - [main:] ~ Property host.name is not valid
(Logging$class:83)
2016-06-16 14:12:04,764 WARN - [main:] ~ Property key.deserializer is not
valid (Logging$class:83)
2016-06-16 14:12:04,765 WARN - [main:] ~ Property key.serializer is not valid
(Logging$class:83)
2016-06-16 14:12:04,765 WARN - [main:] ~ Property log.dirs is not valid
(Logging$class:83)
2016-06-16 14:12:04,765 WARN - [main:] ~ Property log.flush.interval.messages
is not valid (Logging$class:83)
2016-06-16 14:12:04,765 WARN - [main:] ~ Property port is not valid
(Logging$class:83)
2016-06-16 14:12:04,765 WARN - [main:] ~ Property value.deserializer is not
[jira] [Updated] (ATLAS-762) Assertion in NegativeSSLAndKerberosTest.testUnsecuredClient needs to be hardened
[
https://issues.apache.org/jira/browse/ATLAS-762?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-762:
--
Assignee: (was: Vimal Sharma)
> Assertion in NegativeSSLAndKerberosTest.testUnsecuredClient needs to be
> hardened
>
>
> Key: ATLAS-762
> URL: https://issues.apache.org/jira/browse/ATLAS-762
> Project: Atlas
> Issue Type: Bug
>Reporter: Hemanth Yamijala
>Priority: Critical
> Fix For: 0.7-incubating
>
>
> With changes being done to incorporate a new authentication framework in
> Atlas (ATLAS-661), the assertion in
> {{NegativeSSLAndKerberosTest.testUnsecuredClient}} was made weaker for
> Kerberos based environment tests. This bug is to track the change done there
> and harden the assertion back as is required by the new framework.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-902) Atlas throws exception due to null definition in Hive create table statement
[
https://issues.apache.org/jira/browse/ATLAS-902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-902:
--
Assignee: Vimal Sharma
> Atlas throws exception due to null definition in Hive create table statement
>
>
> Key: ATLAS-902
> URL: https://issues.apache.org/jira/browse/ATLAS-902
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.5-incubating
> Environment: HDP 2.4 Sandbox
>Reporter: Ryan Templeton
>Assignee: Vimal Sharma
>Priority: Minor
>
> I was using the following SQL statement to create a Hive table, but Atlas was
> throwing the following exception.
> create external table got_deaths(
> name varchar(64),
> allegiances varchar(32),
> death_year smallint,
> book_of_death tinyint,
> death_chapter tinyint,
> book_intro_chapter tinyint,
> gender tinyint,
> nobility tinyint,
> got tinyint,
> cok tinyint,
> sos tinyint,
> ffc tinyint,
> dwd tinyint
> )
> row format delimited fields terminated by ',' lines terminated by '\n' null
> defined as ''
> stored as TEXTFILE
> location '/user/root/simpleHive/'
> tblproperties ("skip.header.line.count"="1")
> After dropping the "null defined as '' " portion of the create statement,
> Atlas stopped throwing exceptions. I will point out that although Atlas was
> throwing an exception, Hive was able to create the table without issue. So I
> don't think this is semantically incorrect.
> FAILED: Hive Internal Error: org.apache.atlas.AtlasServiceException(Metadata
> service API CREATE_ENTITY failed with status 400(Bad Request) Response Body
> ({"error":"Cannot convert value 'org.apache.atlas.typesystem.Struct@621c7b32'
> to datatype
> hive_serde","stackTrace":"org.apache.atlas.typesystem.types.ValueConversionException:
> Cannot convert value 'org.apache.atlas.typesystem.Struct@621c7b32' to
> datatype hive_serde\n\tat
> org.apache.atlas.typesystem.types.TypedStructHandler.convert(TypedStructHandler.java:67)\n\tat
>
> org.apache.atlas.typesystem.types.StructType.convert(StructType.java:168)\n\tat
>
> org.apache.atlas.typesystem.types.StructType.convert(StructType.java:31)\n\tat
>
> org.apache.atlas.typesystem.persistence.StructInstance.set(StructInstance.java:118)\n\tat
>
> org.apache.atlas.typesystem.types.ClassType.convert(ClassType.java:141)\n\tat
> org.apache.atlas.services.DefaultMetadataService.deserializeClassInstance(DefaultMetadataService.java:252)\n\tat
>
> org.apache.atlas.services.DefaultMetadataService.createEntity(DefaultMetadataService.java:230)\n\tat
>
> org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:96)\n\tat
> sun.reflect.GeneratedMethodAccessor32.invoke(Unknown Source)\n\tat
> sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)\n\tat
> java.lang.reflect.Method.invoke(Method.java:606)\n\tat
> com.sun.jersey.spi.container.JavaMethodInvokerFactory$1.invoke(JavaMethodInvokerFactory.java:60)\n\tat
>
> com.sun.jersey.server.impl.model.method.dispatch.AbstractResourceMethodDispatchProvider$ResponseOutInvoker._dispatch(AbstractResourceMethodDispatchProvider.java:205)\n\tat
>
> com.sun.jersey.server.impl.model.method.dispatch.ResourceJavaMethodDispatcher.dispatch(ResourceJavaMethodDispatcher.java:75)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.HttpMethodRule.accept(HttpMethodRule.java:288)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.ResourceClassRule.accept(ResourceClassRule.java:108)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.RightHandPathRule.accept(RightHandPathRule.java:147)\n\tat
>
> com.sun.jersey.server.impl.uri.rules.RootResourceClassesRule.accept(RootResourceClassesRule.java:84)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1469)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl._handleRequest(WebApplicationImpl.java:1400)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1349)\n\tat
>
> com.sun.jersey.server.impl.application.WebApplicationImpl.handleRequest(WebApplicationImpl.java:1339)\n\tat
>
> com.sun.jersey.spi.container.servlet.WebComponent.service(WebComponent.java:409)\n\tat
>
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:558)\n\tat
>
> com.sun.jersey.spi.container.servlet.ServletContainer.service(ServletContainer.java:733)\n\tat
> javax.servlet.http.HttpServlet.service(HttpServlet.java:790)\n\tat
> com.google.inject.servlet.ServletDefinition.doServiceImpl(ServletDefinition.java:287)\n\tat
>
> com.google.inject.servlet.ServletDefinition.doService(ServletDefinition.java:277)\n\tat
>
> com.google.inject.servlet.ServletDefinition.service(ServletDefinition.java:182)\n\tat
>
> com.goo
[jira] [Updated] (ATLAS-903) EntityNotFoundException exception is observed when an entity is deleted.
[
https://issues.apache.org/jira/browse/ATLAS-903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-903:
--
Fix Version/s: (was: 0.7-incubating)
> EntityNotFoundException exception is observed when an entity is deleted.
>
>
> Key: ATLAS-903
> URL: https://issues.apache.org/jira/browse/ATLAS-903
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Shwetha G S
>Priority: Blocker
>
> Create hive table and drop hive table.
> {noformat}
> 2016-05-05 19:21:04,095 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
> ATLAS_ENTITIES:
> {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"2fdeeeb5-ad02-4746-afd4-8c478e9825c6","version":0,"typeName":"hive_table","state":"ACTIVE"},"typeName":"hive_table","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
> (KafkaNotification:192)
> 2016-05-05 19:21:04,096 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
> ATLAS_ENTITIES:
> {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"efe89140-3635-4779-a495-1d8371f8f939","version":0,"typeName":"hive_storagedesc","state":"ACTIVE"},"typeName":"hive_storagedesc","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
> (KafkaNotification:192)
> 2016-05-05 19:21:04,096 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
> ATLAS_ENTITIES:
> {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"57fd8ebc-e954-4669-bc47-5f617f8bc2fb","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
> (KafkaNotification:192)
> 2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
> ATLAS_ENTITIES, partition - 0, offset - 92 (KafkaNotification:199)
> 2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
> ATLAS_ENTITIES, partition - 0, offset - 93 (KafkaNotification:199)
> 2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
> ATLAS_ENTITIES, partition - 0, offset - 94 (KafkaNotification:199)
> 2016-05-05 19:21:04,099 DEBUG - [pool-9-thread-1:] ~ API
> http://localhost:21000/api/atlas/entities?type=hive_table&property=name&value=default.t3@cl1
> returned status 200 (AtlasClient:976)
> 2016-05-05 19:21:04,100 DEBUG - [pool-9-thread-1:] ~ Read message: conumerId:
> 0, topic - ATLAS_HOOK, partition - 0, offset - 33, message -
> {"typeName":"hive_table","attribute":"name","attributeValue":"default.t3@cl1","type":"ENTITY_DELETE"}
> (KafkaConsumer:68)
> 2016-05-05 19:21:04,108 INFO - [pool-9-thread-1:] ~ Real User: UNKNOWN
> (auth:SIMPLE), is from ticket cache? false (SecureClientUtils:90)
> 2016-05-05 19:21:04,108 INFO - [pool-9-thread-1:] ~ doAsUser: UNKNOWN
> (SecureClientUtils:93)
> 2016-05-05 19:21:04,142 INFO - [pool-9-thread-1:] ~ Client has only one
> service URL, will use that for all actions: http://localhost:21000
> (AtlasClient:204)
> 2016-05-05 19:21:04,143 DEBUG - [qtp1007251739-17 -
> 4b58a3a8-b1e2-4c7d-a830-1ba4251ca1f3:] ~ Audit: UNKNOWN/127.0.0.1 performed
> request OPTIONS
> http://localhost:21000/api/atlas/entities?type=hive_table&property=name&value=default.t3@cl1&doAs=UNKNOWN&user.name=UNKNOWN
> (127.0.0.1) at time 2016-05-05T19:21Z (AuditFilter:91)
> 2016-05-05 19:21:04,146 DEBUG - [qtp1007251739-1307 -
> bd408baa-baad-4582-8d37-53b3169b0bf6:] ~ Audit: UNKNOWN/127.0.0.1 performed
> request DELETE
> http://localhost:21000/api/atlas/entities?type=hive_table&property=name&value=default.t3@cl1&doAs=UNKNOWN
> (127.0.0.1) at time 2016-05-05T19:21Z (AuditFilter:91)
> 2016-05-05 19:21:04,146 INFO - [qtp1007251739-1307 -
> bd408baa-baad-4582-8d37-53b3169b0bf6:] ~ Retrieving entity with
> type=hive_table and name=default.t3@cl1 (GraphBackedMetadataRepository:154)
> 2016-05-05 19:21:04,147 DEBUG - [qtp1007251739-1307 -
> bd408baa-baad-4582-8d37-53b3169b0bf6:] ~ Finding vertex with DataSet.name =
> default.t3@cl1, __state = ACTIVE, (GraphHelper:151)
> 2016-05-05 19:21:04,160 DEBUG - [qtp1007251739-1307 -
> bd408baa-baad-4582-8d37-53b31
[jira] [Updated] (ATLAS-876) No lineage for a re-created ctas table
[
https://issues.apache.org/jira/browse/ATLAS-876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-876:
--
Fix Version/s: (was: 0.7-incubating)
> No lineage for a re-created ctas table
> --
>
> Key: ATLAS-876
> URL: https://issues.apache.org/jira/browse/ATLAS-876
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Shwetha G S
>Priority: Critical
>
> No lineage for a re-created ctas table
> Steps to reproduce:
> create table account (name String, balance Int);
> create table account_ctas as select * from account;
> drop table account_ctas;
> create table account_ctas as select * from account;
> Empty responses from server for recreated ctas
> {noformat}
> curl
> 'http://localhost:21000/api/atlas/lineage/ebd6a5a9-8453-4dbd-b8d9-1575a464ebc3/inputs/graph'
> -H 'Cookie: JSESSIONID=12rn8aybkawa5ewiqdb2hv663' -H 'Accept-Encoding: gzip,
> deflate, sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent:
> Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML,
> like Gecko) Chrome/50.0.2661.102 Safari/537.36' -H 'Accept: application/json,
> text/javascript, */*; q=0.01' -H 'Referer: http://localhost:21000/index.html'
> -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive'
> --compressed | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 2370 2370 0281 0 --:--:-- --:--:-- --:--:-- 281
> {
> "requestId": "qtp1211076369-876 - ad29b29b-7faf-44f2-a2bd-8bbb00f07120",
> "results": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct135",
> "values": {
> "edges": {},
> "vertices": {}
> }
> }
> }
> curl
> 'http://localhost:21000/api/atlas/lineage/ebd6a5a9-8453-4dbd-b8d9-1575a464ebc3/outputs/graph'
> -H 'Cookie: JSESSIONID=12rn8aybkawa5ewiqdb2hv663' -H 'Accept-Encoding: gzip,
> deflate, sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent:
> Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML,
> like Gecko) Chrome/50.0.2661.102 Safari/537.36' -H 'Accept: application/json,
> text/javascript, */*; q=0.01' -H 'Referer: http://localhost:21000/index.html'
> -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive'
> --compressed | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 2380 2380 0271 0 --:--:-- --:--:-- --:--:-- 271
> {
> "requestId": "qtp1211076369-1104 - ee65b8d3-0cf9-4477-a644-d93aab6b625b",
> "results": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct141",
> "values": {
> "edges": {},
> "vertices": {}
> }
> }
> }
> {noformat}
> UI snapshot: https://monosnap.com/file/3XcCcffPjrayjLaIByWcjBHU4k2pgV
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-273) Search query for a trait results in GUID and type. The query response can also include entity name.
[
https://issues.apache.org/jira/browse/ATLAS-273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-273:
--
Assignee: (was: Shwetha G S)
> Search query for a trait results in GUID and type. The query response can
> also include entity name.
> ---
>
> Key: ATLAS-273
> URL: https://issues.apache.org/jira/browse/ATLAS-273
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.6-incubating
>Reporter: Ayub Khan
>Priority: Critical
> Attachments: Apache-Atlas-GUID-instead-of-name.png
>
>
> Search query for a trait results in GUID and type. The query response can
> also include entity name.
> 1. Currently an API request querying for trait results in GUID and typename.
> For example:
> {noformat}
> curl 'http://52.27.169.250:3172/api/atlas/discovery/search?query=Dimension'
> -H 'Accept: application/json, text/plain, */*' -H 'Connection: keep-alive' -H
> 'Accept-Encoding: gzip, deflate, sdch' -H 'Referer:
> http://52.27.169.250:3172/' -H 'Accept-Language: en-US,en;q=0.8' -H
> 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80 Safari/537.36'
> --compressed | python -m json.tool
> {
> "count": 8,
> "dataType": {
> "attributeDefinitions": [
> {
> "dataTypeName": "Dimension",
> "isComposite": false,
> "isIndexable": true,
> "isUnique": false,
> "multiplicity": {
> "isUnique": false,
> "lower": 0,
> "upper": 1
> },
> "name": "traitDetails",
> "reverseAttributeName": null
> },
> {
> "dataTypeName": "__IdType",
> "isComposite": false,
> "isIndexable": true,
> "isUnique": false,
> "multiplicity": {
> "isUnique": false,
> "lower": 0,
> "upper": 1
> },
> "name": "instanceInfo",
> "reverseAttributeName": null
> }
> ],
> "typeName": "__tempQueryResultStruct5049"
> },
> "query": "Dimension",
> "queryType": "dsl",
> "requestId": "qtp1863266395-352 - f8082ab5-5fc5-4f64-8aa8-898144bebc24",
> "results": [
> {
> "$typeName$": "__tempQueryResultStruct5049",
> "instanceInfo": {
> "$typeName$": "__IdType",
> "guid": "c0276563-a802-4cc3-8ff6-fa336a7971cf",
> "typeName": "Table"
> },
> "traitDetails": null
> },
> {
> "$typeName$": "__tempQueryResultStruct5049",
> "instanceInfo": {
> "$typeName$": "__IdType",
> "guid": "40017f65-d1fe-49a4-a0a8-a3649fa19232",
> "typeName": "Table"
> },
> "traitDetails": null
> },
> {
> "$typeName$": "__tempQueryResultStruct5049",
> "instanceInfo": {
> "$typeName$": "__IdType",
> "guid": "d4ecce17-4f5b-4b95-874f-5970eda77b32",
> "typeName": "Table"
> },
> "traitDetails": null
> },
> {
> "$typeName$": "__tempQueryResultStruct5049",
> "instanceInfo": {
> "$typeName$": "__IdType",
> "guid": "0af41c8b-8574-4870-b3cb-21981c4bee57",
> "typeName": "View"
> },
> "traitDetails": null
> },
> {
> "$typeName$": "__tempQueryResultStruct5049",
> "instanceInfo": {
> "$typeName$": "__IdType",
> "guid": "d8b804c4-bda1-4a46-badd-8e7bd361836c",
> "typeName": "View"
> },
> "traitDetails": null
> },
> {
> "$typeName$": "__tempQueryResultStruct5049",
> "instanceInfo": {
> "$typeName$": "__IdType",
> "guid": "3d04e90d-9879-407c-b1ec-ccc0121cc045",
> "typeName": "DB"
> },
> "traitDetails": null
> },
> {
> "$typeName$": "__tempQueryResultStruct5049",
> "instanceInfo": {
> "$typeName$": "__IdType",
> "guid": "869107f0-5f3b-4964-8478-fa87ae2d2866",
> "typeName": "DB"
> },
> "traitDetails": null
> },
> {
> "$typeName$": "__tempQueryResultStruct5049",
> "instanceInfo": {
> "$typeName$": "__IdType",
>
[jira] [Commented] (ATLAS-273) Search query for a trait results in GUID and type. The query response can also include entity name.
[
https://issues.apache.org/jira/browse/ATLAS-273?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331338#comment-15331338
]
Shwetha G S commented on ATLAS-273:
---
I think the issue is still valid
> Search query for a trait results in GUID and type. The query response can
> also include entity name.
> ---
>
> Key: ATLAS-273
> URL: https://issues.apache.org/jira/browse/ATLAS-273
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.6-incubating
>Reporter: Ayub Khan
>Priority: Critical
> Attachments: Apache-Atlas-GUID-instead-of-name.png
>
>
> Search query for a trait results in GUID and type. The query response can
> also include entity name.
> 1. Currently an API request querying for trait results in GUID and typename.
> For example:
> {noformat}
> curl 'http://52.27.169.250:3172/api/atlas/discovery/search?query=Dimension'
> -H 'Accept: application/json, text/plain, */*' -H 'Connection: keep-alive' -H
> 'Accept-Encoding: gzip, deflate, sdch' -H 'Referer:
> http://52.27.169.250:3172/' -H 'Accept-Language: en-US,en;q=0.8' -H
> 'User-Agent: Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5)
> AppleWebKit/537.36 (KHTML, like Gecko) Chrome/46.0.2490.80 Safari/537.36'
> --compressed | python -m json.tool
> {
> "count": 8,
> "dataType": {
> "attributeDefinitions": [
> {
> "dataTypeName": "Dimension",
> "isComposite": false,
> "isIndexable": true,
> "isUnique": false,
> "multiplicity": {
> "isUnique": false,
> "lower": 0,
> "upper": 1
> },
> "name": "traitDetails",
> "reverseAttributeName": null
> },
> {
> "dataTypeName": "__IdType",
> "isComposite": false,
> "isIndexable": true,
> "isUnique": false,
> "multiplicity": {
> "isUnique": false,
> "lower": 0,
> "upper": 1
> },
> "name": "instanceInfo",
> "reverseAttributeName": null
> }
> ],
> "typeName": "__tempQueryResultStruct5049"
> },
> "query": "Dimension",
> "queryType": "dsl",
> "requestId": "qtp1863266395-352 - f8082ab5-5fc5-4f64-8aa8-898144bebc24",
> "results": [
> {
> "$typeName$": "__tempQueryResultStruct5049",
> "instanceInfo": {
> "$typeName$": "__IdType",
> "guid": "c0276563-a802-4cc3-8ff6-fa336a7971cf",
> "typeName": "Table"
> },
> "traitDetails": null
> },
> {
> "$typeName$": "__tempQueryResultStruct5049",
> "instanceInfo": {
> "$typeName$": "__IdType",
> "guid": "40017f65-d1fe-49a4-a0a8-a3649fa19232",
> "typeName": "Table"
> },
> "traitDetails": null
> },
> {
> "$typeName$": "__tempQueryResultStruct5049",
> "instanceInfo": {
> "$typeName$": "__IdType",
> "guid": "d4ecce17-4f5b-4b95-874f-5970eda77b32",
> "typeName": "Table"
> },
> "traitDetails": null
> },
> {
> "$typeName$": "__tempQueryResultStruct5049",
> "instanceInfo": {
> "$typeName$": "__IdType",
> "guid": "0af41c8b-8574-4870-b3cb-21981c4bee57",
> "typeName": "View"
> },
> "traitDetails": null
> },
> {
> "$typeName$": "__tempQueryResultStruct5049",
> "instanceInfo": {
> "$typeName$": "__IdType",
> "guid": "d8b804c4-bda1-4a46-badd-8e7bd361836c",
> "typeName": "View"
> },
> "traitDetails": null
> },
> {
> "$typeName$": "__tempQueryResultStruct5049",
> "instanceInfo": {
> "$typeName$": "__IdType",
> "guid": "3d04e90d-9879-407c-b1ec-ccc0121cc045",
> "typeName": "DB"
> },
> "traitDetails": null
> },
> {
> "$typeName$": "__tempQueryResultStruct5049",
> "instanceInfo": {
> "$typeName$": "__IdType",
> "guid": "869107f0-5f3b-4964-8478-fa87ae2d2866",
> "typeName": "DB"
> },
> "traitDetails": null
> },
> {
> "$typeName$": "__tempQueryResultStruct5049",
> "instanceInfo
[jira] [Commented] (ATLAS-96) import-hive.sh failed
[ https://issues.apache.org/jira/browse/ATLAS-96?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331336#comment-15331336 ] Shwetha G S commented on ATLAS-96: -- Check if the atlas endpoint is set correctly in conf(atlas.rest.address property) and the right conf is loaded(you should see a log - Loading atlas-application.properties from some path). If these are right, check if atlas server is up > import-hive.sh failed > - > > Key: ATLAS-96 > URL: https://issues.apache.org/jira/browse/ATLAS-96 > Project: Atlas > Issue Type: Bug >Reporter: Jason Xing > > Exception in thread "main" org.apache.hadoop.hive.ql.metadata.HiveException: > java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient > at > org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1236) > at > org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:94) > at > org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:90) > at > org.apache.atlas.hive.bridge.HiveMetaStoreBridge.main(HiveMetaStoreBridge.java:507) > Caused by: java.lang.RuntimeException: Unable to instantiate > org.apache.hadoop.hive.ql.metadata.SessionHiveMetaStoreClient > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1523) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.(RetryingMetaStoreClient.java:86) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:132) > at > org.apache.hadoop.hive.metastore.RetryingMetaStoreClient.getProxy(RetryingMetaStoreClient.java:104) > at > org.apache.hadoop.hive.ql.metadata.Hive.createMetaStoreClient(Hive.java:3000) > at org.apache.hadoop.hive.ql.metadata.Hive.getMSC(Hive.java:3019) > at > org.apache.hadoop.hive.ql.metadata.Hive.getAllDatabases(Hive.java:1234) > ... 3 more > Caused by: java.lang.reflect.InvocationTargetException > at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method) > at > sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57) > at > sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45) > at java.lang.reflect.Constructor.newInstance(Constructor.java:526) > at > org.apache.hadoop.hive.metastore.MetaStoreUtils.newInstance(MetaStoreUtils.java:1521) > ... 9 more > Caused by: javax.jdo.JDOFatalInternalException: Error creating transactional > connection factory > NestedThrowables: > java.lang.reflect.InvocationTargetException > at > org.datanucleus.api.jdo.NucleusJDOHelper.getJDOExceptionForNucleusException(NucleusJDOHelper.java:587) > at > org.datanucleus.api.jdo.JDOPersistenceManagerFactory.freezeConfiguration(JDOPersistenceManagerFactory.java:788) > at > org.datanucleus.api.jdo.JDOPersistenceManagerFactory.createPersistenceManagerFactory(JDOPersistenceManagerFactory.java:333) > at > org.datanucleus.api.jdo.JDOPersistenceManagerFactory.getPersistenceManagerFactory(JDOPersistenceManagerFactory.java:202) > at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method) > at > sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:57) > at > sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43) > at java.lang.reflect.Method.invoke(Method.java:606) > at javax.jdo.JDOHelper$16.run(JDOHelper.java:1965) > at java.security.AccessController.doPrivileged(Native Method) > at javax.jdo.JDOHelper.invoke(JDOHelper.java:1960) > at > javax.jdo.JDOHelper.invokeGetPersistenceManagerFactoryOnImplementation(JDOHelper.java:1166) > at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:808) > at javax.jdo.JDOHelper.getPersistenceManagerFactory(JDOHelper.java:701) > at > org.apache.hadoop.hive.metastore.ObjectStore.getPMF(ObjectStore.java:365) > at > org.apache.hadoop.hive.metastore.ObjectStore.getPersistenceManager(ObjectStore.java:394) > at > org.apache.hadoop.hive.metastore.ObjectStore.initialize(ObjectStore.java:291) > at > org.apache.hadoop.hive.metastore.ObjectStore.setConf(ObjectStore.java:258) > at > org.apache.hadoop.util.ReflectionUtils.setConf(ReflectionUtils.java:76) > at > org.apache.hadoop.util.ReflectionUtils.newInstance(ReflectionUtils.java:136) > at > org.apache.hadoop.hive.metastore.RawStoreProxy.(RawStoreProxy.java:57) > at > org.apache.hadoop.hive.metastore.RawStoreProxy.getProxy(RawStoreProxy.java:66) > at > org.apache.hadoop.hive.metastore.HiveMetaStore$HMSHandler.newRawStore(HiveMetaStore
[jira] [Commented] (ATLAS-888) NPE in NotificationHookConsumer
[ https://issues.apache.org/jira/browse/ATLAS-888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15331199#comment-15331199 ] Shwetha G S commented on ATLAS-888: --- +1 > NPE in NotificationHookConsumer > --- > > Key: ATLAS-888 > URL: https://issues.apache.org/jira/browse/ATLAS-888 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Blocker > Fix For: 0.7-incubating > > Attachments: ATLAS-888.1.patch, ATLAS-888.patch > > > 2016-06-08 15:54:22,375 ERROR - [NotificationHookConsumer thread-0:] ~ Unable > to persist entity instance (EntityResource:137) > java.lang.NullPointerException > at > org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:122) > at > org.apache.atlas.LocalAtlasClient$1.invoke(LocalAtlasClient.java:82) > at > org.apache.atlas.LocalAtlasClient$EntityOperation.run(LocalAtlasClient.java:118) > at > org.apache.atlas.LocalAtlasClient.createEntity(LocalAtlasClient.java:85) > at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:652) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:203) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:187) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2016-06-08 15:54:22,376 WARN - [NotificationHookConsumer thread-0:] ~ Error > handling message > org.apache.atlas.notification.hook.HookNotification$EntityCreateRequest@6b8492f1 > (NotificationHookConsumer:233) > org.apache.atlas.AtlasServiceException: Metadata service API CREATE_ENTITY > failed with status 500(Internal Server Error) Response Body > (java.lang.NullPointerException > at > org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:122) > at > org.apache.atlas.LocalAtlasClient$1.invoke(LocalAtlasClient.java:82) > at > org.apache.atlas.LocalAtlasClient$EntityOperation.run(LocalAtlasClient.java:118) > at > org.apache.atlas.LocalAtlasClient.createEntity(LocalAtlasClient.java:85) > at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:652) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:203) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:187) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > ) > at > org.apache.atlas.LocalAtlasClient$EntityOperation.run(LocalAtlasClient.java:122) > at > org.apache.atlas.LocalAtlasClient.createEntity(LocalAtlasClient.java:85) > at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:652) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:203) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:187) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-592) Queue for failed hook messages
[ https://issues.apache.org/jira/browse/ATLAS-592?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15329093#comment-15329093 ] Shwetha G S commented on ATLAS-592: --- Yes, separate logger for now sounds fine > Queue for failed hook messages > -- > > Key: ATLAS-592 > URL: https://issues.apache.org/jira/browse/ATLAS-592 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S > > If NotificationHookConsumer fails to process a message, its just logged and > the message is lost. Instead, it should be added to failed queue, possibly > for manual intervention -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-890) Log received messages in case of error
[ https://issues.apache.org/jira/browse/ATLAS-890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15329084#comment-15329084 ] Shwetha G S commented on ATLAS-890: --- The patch contains rollback of some previous commits, can you rebase? > Log received messages in case of error > -- > > Key: ATLAS-890 > URL: https://issues.apache.org/jira/browse/ATLAS-890 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.7-incubating > > Attachments: ATLAS-890.patch > > > This will be useful for further debugging whenever an error occurs on ATLAS > writes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-884) Process registration should call Entity update instead of create
[ https://issues.apache.org/jira/browse/ATLAS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328972#comment-15328972 ] Shwetha G S commented on ATLAS-884: --- In testUpdateProcess(), can you use new query for the update say, "insert overwrite DIRECTORY '" + pFile1 + "' select id from " + tableName and verify that query string is updated? Looks good otherwise. You can fix and commit. Thanks > Process registration should call Entity update instead of create > > > Key: ATLAS-884 > URL: https://issues.apache.org/jira/browse/ATLAS-884 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating, 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.7-incubating > > Attachments: ATLAS-884.1.patch, ATLAS-884.patch > > > Process registration in hive hook currently calls Entity create . But should > call update entity instead -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-888) NPE in NotificationHookConsumer
[
https://issues.apache.org/jira/browse/ATLAS-888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328963#comment-15328963
]
Shwetha G S commented on ATLAS-888:
---
{code}
UriBuilder builder = UriBuilder
156 .fromPath(AtlasConstants.DEFAULT_ATLAS_REST_ADDRESS)
157 .path(uriPath).path(guids.get(0));
158
{code}
This will fail if guids is empty
What are the changes in log statements?
> NPE in NotificationHookConsumer
> ---
>
> Key: ATLAS-888
> URL: https://issues.apache.org/jira/browse/ATLAS-888
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-888.patch
>
>
> 2016-06-08 15:54:22,375 ERROR - [NotificationHookConsumer thread-0:] ~ Unable
> to persist entity instance (EntityResource:137)
> java.lang.NullPointerException
> at
> org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:122)
> at
> org.apache.atlas.LocalAtlasClient$1.invoke(LocalAtlasClient.java:82)
> at
> org.apache.atlas.LocalAtlasClient$EntityOperation.run(LocalAtlasClient.java:118)
> at
> org.apache.atlas.LocalAtlasClient.createEntity(LocalAtlasClient.java:85)
> at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:652)
> at
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:203)
> at
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:187)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-08 15:54:22,376 WARN - [NotificationHookConsumer thread-0:] ~ Error
> handling message
> org.apache.atlas.notification.hook.HookNotification$EntityCreateRequest@6b8492f1
> (NotificationHookConsumer:233)
> org.apache.atlas.AtlasServiceException: Metadata service API CREATE_ENTITY
> failed with status 500(Internal Server Error) Response Body
> (java.lang.NullPointerException
> at
> org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:122)
> at
> org.apache.atlas.LocalAtlasClient$1.invoke(LocalAtlasClient.java:82)
> at
> org.apache.atlas.LocalAtlasClient$EntityOperation.run(LocalAtlasClient.java:118)
> at
> org.apache.atlas.LocalAtlasClient.createEntity(LocalAtlasClient.java:85)
> at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:652)
> at
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:203)
> at
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:187)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> )
> at
> org.apache.atlas.LocalAtlasClient$EntityOperation.run(LocalAtlasClient.java:122)
> at
> org.apache.atlas.LocalAtlasClient.createEntity(LocalAtlasClient.java:85)
> at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:652)
> at
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:203)
> at
> org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:187)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ATLAS-505) Create Entity response from API should match the data in ATLAS_ENTITIES kafka topic when some attributes are set to null
[ https://issues.apache.org/jira/browse/ATLAS-505?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15328952#comment-15328952 ] Shwetha G S commented on ATLAS-505: --- If we release this out now, we will have to support it for a long time. Hence I wanted to take a look at it now to figure out the issue and scope of fix. Looking at input_output.txt, the entity definition json is same, but the topic message has more fields like message type, which I think is valid. So, will resolve this as not a problem > Create Entity response from API should match the data in ATLAS_ENTITIES kafka > topic when some attributes are set to null > > > Key: ATLAS-505 > URL: https://issues.apache.org/jira/browse/ATLAS-505 > Project: Atlas > Issue Type: Bug >Affects Versions: trunk >Reporter: Chethana >Priority: Critical > Fix For: 0.7-incubating > > Attachments: Type.txt, input_output.txt > > > create a type in atlas > create an entity of type > Make sure this entity is pushed to atlas topic > The response from create entity API and data pushed to kafka queue should > match > This could be possibly caused by different JSON libraries used between > Notification and API classes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-505) Create Entity response from API should match the data in ATLAS_ENTITIES kafka topic when some attributes are set to null
[ https://issues.apache.org/jira/browse/ATLAS-505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-505. --- Resolution: Not A Problem > Create Entity response from API should match the data in ATLAS_ENTITIES kafka > topic when some attributes are set to null > > > Key: ATLAS-505 > URL: https://issues.apache.org/jira/browse/ATLAS-505 > Project: Atlas > Issue Type: Bug >Affects Versions: trunk >Reporter: Chethana >Priority: Critical > Fix For: 0.7-incubating > > Attachments: Type.txt, input_output.txt > > > create a type in atlas > create an entity of type > Make sure this entity is pushed to atlas topic > The response from create entity API and data pushed to kafka queue should > match > This could be possibly caused by different JSON libraries used between > Notification and API classes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-831) Search issues with business terms
[ https://issues.apache.org/jira/browse/ATLAS-831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-831: -- Assignee: (was: Shwetha G S) > Search issues with business terms > - > > Key: ATLAS-831 > URL: https://issues.apache.org/jira/browse/ATLAS-831 > Project: Atlas > Issue Type: Bug >Reporter: Hemanth Yamijala >Priority: Critical > > With ATLAS-821 in, there are a couple of issues that we need to think from > the business terms search perspective. This might involve changes between > core, taxonomy API and UI. Hence, not filing against any one area. > * Searching for assets associated with a business term returns the taxonomy > object it is assigned to. This is an artifact of our associating the 'master > copy' of the business term against the taxonomy object - to root it for > lookups. > * More importantly, searching for assets associated to a parent path in the > business term does not return assets associated to the children. For e.g. if > a term 'a.b.c' is associated with an asset A, then searching for 'a.b' does > not return A. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-896) SqoopHookIT as integration test
Shwetha G S created ATLAS-896: - Summary: SqoopHookIT as integration test Key: ATLAS-896 URL: https://issues.apache.org/jira/browse/ATLAS-896 Project: Atlas Issue Type: Bug Reporter: Shwetha G S SqoopHookIT can use derby as external db and should use sqoop commands to test the end to end flow. It currently just prepares the data and call sqoop hook. Instead, it should use sqoop command execution to prepare the data -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-837) Enhance Sqoop addon to handle export operation
[ https://issues.apache.org/jira/browse/ATLAS-837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15327222#comment-15327222 ] Shwetha G S commented on ATLAS-837: --- Verified end to end flow. Export will work only with SQOOP-2936, however the patch compiles without SQOOP-2936. So, committing this > Enhance Sqoop addon to handle export operation > -- > > Key: ATLAS-837 > URL: https://issues.apache.org/jira/browse/ATLAS-837 > Project: Atlas > Issue Type: Bug >Reporter: Venkat Ranganathan >Assignee: Venkat Ranganathan > Attachments: ATLAS-837.patch > > > Sqoop is adding change to publish hcatalog export data in SQOOP-2936. > Enhancing the Sqoop Hook to support it. No change in Sqoop proces model but > based on operation the inputs/outputs will be swapped -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-884) Process registration should call Entity update instead of create
[ https://issues.apache.org/jira/browse/ATLAS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15326847#comment-15326847 ] Shwetha G S commented on ATLAS-884: --- +1, code changes look good. Can you add a test and then commit? Thanks > Process registration should call Entity update instead of create > > > Key: ATLAS-884 > URL: https://issues.apache.org/jira/browse/ATLAS-884 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating, 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.7-incubating > > Attachments: ATLAS-884.patch > > > Process registration in hive hook currently calls Entity create . But should > call update entity instead -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-763) UI: Tag Association does not prompt for attribute values
[ https://issues.apache.org/jira/browse/ATLAS-763?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-763. --- Resolution: Not A Problem Works on trunk > UI: Tag Association does not prompt for attribute values > > > Key: ATLAS-763 > URL: https://issues.apache.org/jira/browse/ATLAS-763 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Erik Bergenholtz >Assignee: Nixon Rodrigues >Priority: Critical > Fix For: 0.7-incubating > > > In the previous version of the UI when a tag was associated with an Entity, > the user was prompted for any attributes that were a part of the the tag > definition; this is not longer the case in dashboardv2 and needs to be > created. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-756) UI: Hive lineage is not properly rendered when source table has multiple edges
[
https://issues.apache.org/jira/browse/ATLAS-756?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S resolved ATLAS-756.
---
Resolution: Not A Problem
Verified that this works fine on trunk
> UI: Hive lineage is not properly rendered when source table has multiple edges
> --
>
> Key: ATLAS-756
> URL: https://issues.apache.org/jira/browse/ATLAS-756
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Erik Bergenholtz
>Priority: Critical
> Fix For: 0.7-incubating
>
>
> Follow below steps:
> create table source (name string);
> create table source_as_select as select * from source;
> create view source_view as select * from source;
> Take a look at lineage of source table.
> lineage snapshot: https://monosnap.com/file/mBcAUjdrt0dHJRaZpTGyyiVEtmMx7P
> Lineage API response seems to be fine.
> {noformat}
> curl
> 'http://os-r6-apathan-atlas-erie-nosec-3.openstacklocal:21000/api/atlas/lineage/hive/table/default.source@cl1/outputs/graph'
> -H 'Cookie: JSESSIONID=1k06n72ew9ltc17tp62zjrv7eq' -H 'Accept-Encoding:
> gzip, deflate, sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent:
> Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML,
> like Gecko) Chrome/50.0.2661.94 Safari/537.36' -H 'Accept: application/json,
> text/javascript, */*; q=0.01' -H 'Referer:
> http://os-r6-apathan-atlas-erie-nosec-3.openstacklocal:21000/index.html' -H
> 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive' --compressed |
> python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 17340 17340 0972 0 --:--:-- 0:00:01 --:--:-- 971
> {
> "requestId": "qtp1007251739-64812 - 8747b792-e181-47d4-8057-29240d2f5ed4",
> "results": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct541",
> "values": {
> "edges": {
> "388d67da-0211-4033-a57e-3fadd89b0df6": [
> "9ec42ecd-5749-41b6-950b-ca5626195767"
> ],
> "4eb1e18e-b9e5-4fe6-89af-a3bfc0a7e5a1": [
> "388d67da-0211-4033-a57e-3fadd89b0df6",
> "d48abe3f-da89-4e80-b281-b579dc3c16e9"
> ],
> "d48abe3f-da89-4e80-b281-b579dc3c16e9": [
> "da268dfc-6cf9-4307-be7e-a7874ae4ad0a"
> ]
> },
> "vertices": {
> "4eb1e18e-b9e5-4fe6-89af-a3bfc0a7e5a1": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct540",
> "values": {
> "name": "default.source@cl1",
> "vertexId": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__IdType",
> "values": {
> "guid":
> "4eb1e18e-b9e5-4fe6-89af-a3bfc0a7e5a1",
> "typeName": "hive_table"
> }
> }
> }
> },
> "9ec42ecd-5749-41b6-950b-ca5626195767": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct540",
> "values": {
> "name": "default.source_view@cl1",
> "vertexId": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__IdType",
> "values": {
> "guid":
> "9ec42ecd-5749-41b6-950b-ca5626195767",
> "typeName": "hive_table"
> }
> }
> }
> },
> "da268dfc-6cf9-4307-be7e-a7874ae4ad0a": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct540",
> "values": {
> "name": "default.source_as_select@cl1",
> "vertexId": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
>
[jira] [Resolved] (ATLAS-797) Tags are not disassociating correctly from entities
[ https://issues.apache.org/jira/browse/ATLAS-797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-797. --- Resolution: Not A Problem Verified that its not an issue on trunk > Tags are not disassociating correctly from entities > --- > > Key: ATLAS-797 > URL: https://issues.apache.org/jira/browse/ATLAS-797 > Project: Atlas > Issue Type: Bug >Reporter: Hemanth Yamijala >Priority: Blocker > Attachments: json-response.txt > > > * Create a tag > * Attach it to an entity > * Delete the tag from an entity page. The JSON response seems to be OK and > the entry gets removed. > * Go to the assets page and do an ISA query again for the tag > Expected result: Should get not results back. > Actual result: See that the entity is being returned back in the result set. > Further, adding the tag again to the same entity and doing a DSL query is > showing it up twice. Will attach JSON response returned. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-469) Create a new release version for the next release
[ https://issues.apache.org/jira/browse/ATLAS-469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-469. --- Resolution: Fixed > Create a new release version for the next release > - > > Key: ATLAS-469 > URL: https://issues.apache.org/jira/browse/ATLAS-469 > Project: Atlas > Issue Type: Task >Reporter: Selvamohan Neethiraj >Priority: Critical > > Currently, all changes are being tracked as version - 'trunk' ... Instead, I > suggest that we create the next possible release '0.7.0-incubating' and > create a roadmap for all features and bugfixes for the specific release > version. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (ATLAS-125) Inputs not being set for a hive_process on a plain select query
[ https://issues.apache.org/jira/browse/ATLAS-125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S reopened ATLAS-125: --- Assignee: (was: Rohit) > Inputs not being set for a hive_process on a plain select query > --- > > Key: ATLAS-125 > URL: https://issues.apache.org/jira/browse/ATLAS-125 > Project: Atlas > Issue Type: Bug >Reporter: Suma Shivaprasad >Priority: Critical > Labels: ATLAS-UI-BUGS > Attachments: screenshot-1.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-125) Inputs not being set for a hive_process on a plain select query
[ https://issues.apache.org/jira/browse/ATLAS-125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-125. --- Resolution: Not A Problem > Inputs not being set for a hive_process on a plain select query > --- > > Key: ATLAS-125 > URL: https://issues.apache.org/jira/browse/ATLAS-125 > Project: Atlas > Issue Type: Bug >Reporter: Suma Shivaprasad >Priority: Critical > Labels: ATLAS-UI-BUGS > Attachments: screenshot-1.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-125) Inputs not being set for a hive_process on a plain select query
[ https://issues.apache.org/jira/browse/ATLAS-125?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-125. --- Resolution: Fixed Plain select queries are not supported for lineage > Inputs not being set for a hive_process on a plain select query > --- > > Key: ATLAS-125 > URL: https://issues.apache.org/jira/browse/ATLAS-125 > Project: Atlas > Issue Type: Bug >Reporter: Suma Shivaprasad >Assignee: Rohit >Priority: Critical > Labels: ATLAS-UI-BUGS > Attachments: screenshot-1.png > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-216) UI: showing invalid date and time values for hive table
[
https://issues.apache.org/jira/browse/ATLAS-216?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S updated ATLAS-216:
--
Fix Version/s: 0.7-incubating
> UI: showing invalid date and time values for hive table
> ---
>
> Key: ATLAS-216
> URL: https://issues.apache.org/jira/browse/ATLAS-216
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.6-incubating
>Reporter: Ayub Khan
> Labels: ATLAS-UI-BUGS
> Fix For: 0.7-incubating
>
> Attachments: Invalid_Timestamps_for_hivetables.png
>
>
> While viewing the hive tables metadata in Atlas UI, I see invalid time values
> for createTime and transient_lastDdlTime entries. Please check the attached
> UI snapshot.
> The same query through API results in correct timestamps. Below is the API
> response.
> {code}
> curl
> http://localhost:21000/api/atlas/entity/f286e5d3-0514-4f28-9cef-373473cd
> {"requestId":"qtp186909238-384 -
> 71055c83-f290-4370-a279-24412918fb58","GUID":"f286e5d3-0514-4f28-9cef-373473cd","definition":"{\n
>
> \"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference\",\n
> \"id\":{\n
> \"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Id\",\n
> \"id\":\"f286e5d3-0514-4f28-9cef-373473cd\",\n\"version\":0,\n
> \"typeName\":\"hive_table\"\n },\n \"typeName\":\"hive_table\",\n
> \"values\":{\n\"tableType\":\"MANAGED_TABLE\",\n
> \"name\":\"primary.default.t\",\n\"viewExpandedText\":null,\n
> \"createTime\":1444395991,\n\"description\":null,\n
> \"temporary\":false,\n\"db\":{\n
> \"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Id\",\n
> \"id\":\"cca28963-f7b1-44fd-a631-f4b6f3b09612\",\n
> \"version\":0,\n \"typeName\":\"hive_db\"\n},\n
> \"viewOriginalText\":null,\n\"retention\":0,\n\"tableName\":\"t\",\n
> \"columns\":[\n {\n
> \"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference\",\n
> \"id\":{\n
> \"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Id\",\n
> \"id\":\"5f89e208-a9ff-40eb-912b-b4394f8d8c0e\",\n
> \"version\":0,\n \"typeName\":\"hive_column\"\n},\n
> \"typeName\":\"hive_column\",\n\"values\":{\n
> \"comment\":null,\n
> \"qualifiedName\":\"primary.default.t.salary\",\n
> \"type\":\"string\",\n \"name\":\"salary\"\n},\n
> \"traitNames\":[\n \n],\n\"traits\":{\n \n
> }\n },\n {\n
> \"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference\",\n
> \"id\":{\n
> \"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Id\",\n
> \"id\":\"14870ed2-f596-46a3-9500-c720e559703e\",\n
> \"version\":0,\n \"typeName\":\"hive_column\"\n},\n
> \"typeName\":\"hive_column\",\n\"values\":{\n
> \"comment\":null,\n
> \"qualifiedName\":\"primary.default.t.integer\",\n
> \"type\":\"int\",\n \"name\":\"integer\"\n},\n
> \"traitNames\":[\n \n],\n\"traits\":{\n \n
> }\n },\n {\n
> \"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference\",\n
> \"id\":{\n
> \"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Id\",\n
> \"id\":\"535e002f-f7ed-4e6e-a363-4cc38ebc3b46\",\n
> \"version\":0,\n \"typeName\":\"hive_column\"\n},\n
> \"typeName\":\"hive_column\",\n\"values\":{\n
> \"comment\":null,\n
> \"qualifiedName\":\"primary.default.t.number\",\n
> \"type\":\"int\",\n \"name\":\"number\"\n},\n
> \"traitNames\":[\n \n],\n\"traits\":{\n \n
> }\n }\n],\n\"comment\":null,\n\"lastAccessTime\":0,\n
> \"owner\":\"apathan\",\n\"sd\":{\n
> \"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference\",\n
> \"id\":{\n
> \"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Id\",\n
> \"id\":\"168ba76f-c5db-4398-bded-ed25bb626785\",\n
> \"version\":0,\n\"typeName\":\"hive_storagedesc\"\n },\n
> \"typeName\":\"hive_storagedesc\",\n \"values\":{\n
> \"location\":\"hdfs:\/\/localhost:9000\/user\/hive\/warehouse\/t\",\n
> \"serdeInfo\":{\n
> \"jsonClass\":\"org.apache.atlas.typesystem.json.InstanceSerialization$_Struct\",\n
> \"typeName\":\"hive_serde\",\n
[jira] [Resolved] (ATLAS-579) DSL queries take long time to process
[ https://issues.apache.org/jira/browse/ATLAS-579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-579. --- Resolution: Duplicate > DSL queries take long time to process > - > > Key: ATLAS-579 > URL: https://issues.apache.org/jira/browse/ATLAS-579 > Project: Atlas > Issue Type: Bug > Environment: Atlas - 0.7 External Kafka / HBase / Solr >Reporter: Sharmadha Sainath >Priority: Critical > > On Atlas populated with 50,000 tables, querying by full text query "table_1" > takes less than a second to be processed, whereas DSL query "hive_table where > name=dbname.table_1@clustername" takes 40 seconds to be processed. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-505) Create Entity response from API should match the data in ATLAS_ENTITIES kafka topic when some attributes are set to null
[ https://issues.apache.org/jira/browse/ATLAS-505?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-505: -- Fix Version/s: 0.7-incubating > Create Entity response from API should match the data in ATLAS_ENTITIES kafka > topic when some attributes are set to null > > > Key: ATLAS-505 > URL: https://issues.apache.org/jira/browse/ATLAS-505 > Project: Atlas > Issue Type: Bug >Affects Versions: trunk >Reporter: Chethana >Priority: Critical > Fix For: 0.7-incubating > > Attachments: Type.txt, input_output.txt > > > create a type in atlas > create an entity of type > Make sure this entity is pushed to atlas topic > The response from create entity API and data pushed to kafka queue should > match > This could be possibly caused by different JSON libraries used between > Notification and API classes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-837) Enhance Sqoop addon to handle export operation
[ https://issues.apache.org/jira/browse/ATLAS-837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15322360#comment-15322360 ] Shwetha G S commented on ATLAS-837: --- +1, will test and commit > Enhance Sqoop addon to handle export operation > -- > > Key: ATLAS-837 > URL: https://issues.apache.org/jira/browse/ATLAS-837 > Project: Atlas > Issue Type: Bug >Reporter: Venkat Ranganathan >Assignee: Venkat Ranganathan > Attachments: ATLAS-837.patch > > > Sqoop is adding change to publish hcatalog export data in SQOOP-2936. > Enhancing the Sqoop Hook to support it. No change in Sqoop proces model but > based on operation the inputs/outputs will be swapped -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-884) Process registration should call Entity update instead of create
[ https://issues.apache.org/jira/browse/ATLAS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15322065#comment-15322065 ] Shwetha G S commented on ATLAS-884: --- With soft deletes, updating inputs/outputs results in all references being present. So, in the final lineage, both old and new inputs and outputs will be present, which will be more confusing, no? > Process registration should call Entity update instead of create > > > Key: ATLAS-884 > URL: https://issues.apache.org/jira/browse/ATLAS-884 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating, 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.7-incubating > > Attachments: ATLAS-884.patch > > > Process registration in hive hook currently calls Entity create . But should > call update entity instead -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Reopened] (ATLAS-871) Make audit repository implementation configurable
[ https://issues.apache.org/jira/browse/ATLAS-871?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S reopened ATLAS-871: --- > Make audit repository implementation configurable > - > > Key: ATLAS-871 > URL: https://issues.apache.org/jira/browse/ATLAS-871 > Project: Atlas > Issue Type: Task >Reporter: Jeffrey Hagelberg >Assignee: Jeffrey Hagelberg > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-880) Audit log: Tag association to hive column is not logged
[ https://issues.apache.org/jira/browse/ATLAS-880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15322003#comment-15322003 ] Shwetha G S commented on ATLAS-880: --- Since column is an entity, audit is logged under column entity. Table is referenced entity in this case, and audit events of column can't go under table entity > Audit log: Tag association to hive column is not logged > --- > > Key: ATLAS-880 > URL: https://issues.apache.org/jira/browse/ATLAS-880 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Ayub Khan >Assignee: Shwetha G S > Fix For: 0.7-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-880) Audit log: Tag association to hive column is not logged
[ https://issues.apache.org/jira/browse/ATLAS-880?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S resolved ATLAS-880. --- Resolution: Not A Problem > Audit log: Tag association to hive column is not logged > --- > > Key: ATLAS-880 > URL: https://issues.apache.org/jira/browse/ATLAS-880 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Ayub Khan >Assignee: Shwetha G S > Fix For: 0.7-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-749) Make EntityAuditRepository implementation configurable
[ https://issues.apache.org/jira/browse/ATLAS-749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-749: -- Attachment: ATLAS-749.patch > Make EntityAuditRepository implementation configurable > -- > > Key: ATLAS-749 > URL: https://issues.apache.org/jira/browse/ATLAS-749 > Project: Atlas > Issue Type: Improvement >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.7-incubating > > Attachments: ATLAS-749.patch > > > EntityAuditRepository is bound to HBaseBasedAuditRepository in > RepositoryMetadataModule, which essentially forces hbase setup. Make it > configurable using atlas-application.properties on the lines of DeleteHandler > configuration -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-880) Audit log: Tag association to hive column is not logged
[ https://issues.apache.org/jira/browse/ATLAS-880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320396#comment-15320396 ] Shwetha G S commented on ATLAS-880: --- [~ayubkhan], associating tag to column is logged under the column entity. I verified that it works. Did you mean that this association is not logged under table entity? > Audit log: Tag association to hive column is not logged > --- > > Key: ATLAS-880 > URL: https://issues.apache.org/jira/browse/ATLAS-880 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Ayub Khan >Assignee: Shwetha G S > Fix For: 0.7-incubating > > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (ATLAS-876) No lineage for a re-created ctas table
[
https://issues.apache.org/jira/browse/ATLAS-876?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shwetha G S reassigned ATLAS-876:
-
Assignee: Shwetha G S
> No lineage for a re-created ctas table
> --
>
> Key: ATLAS-876
> URL: https://issues.apache.org/jira/browse/ATLAS-876
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Shwetha G S
>Priority: Critical
> Fix For: 0.7-incubating
>
>
> No lineage for a re-created ctas table
> Steps to reproduce:
> create table account (name String, balance Int);
> create table account_ctas as select * from account;
> drop table account_ctas;
> create table account_ctas as select * from account;
> Empty responses from server for recreated ctas
> {noformat}
> curl
> 'http://localhost:21000/api/atlas/lineage/ebd6a5a9-8453-4dbd-b8d9-1575a464ebc3/inputs/graph'
> -H 'Cookie: JSESSIONID=12rn8aybkawa5ewiqdb2hv663' -H 'Accept-Encoding: gzip,
> deflate, sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent:
> Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML,
> like Gecko) Chrome/50.0.2661.102 Safari/537.36' -H 'Accept: application/json,
> text/javascript, */*; q=0.01' -H 'Referer: http://localhost:21000/index.html'
> -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive'
> --compressed | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 2370 2370 0281 0 --:--:-- --:--:-- --:--:-- 281
> {
> "requestId": "qtp1211076369-876 - ad29b29b-7faf-44f2-a2bd-8bbb00f07120",
> "results": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct135",
> "values": {
> "edges": {},
> "vertices": {}
> }
> }
> }
> curl
> 'http://localhost:21000/api/atlas/lineage/ebd6a5a9-8453-4dbd-b8d9-1575a464ebc3/outputs/graph'
> -H 'Cookie: JSESSIONID=12rn8aybkawa5ewiqdb2hv663' -H 'Accept-Encoding: gzip,
> deflate, sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent:
> Mozilla/5.0 (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML,
> like Gecko) Chrome/50.0.2661.102 Safari/537.36' -H 'Accept: application/json,
> text/javascript, */*; q=0.01' -H 'Referer: http://localhost:21000/index.html'
> -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive'
> --compressed | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 2380 2380 0271 0 --:--:-- --:--:-- --:--:-- 271
> {
> "requestId": "qtp1211076369-1104 - ee65b8d3-0cf9-4477-a644-d93aab6b625b",
> "results": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Struct",
> "typeName": "__tempQueryResultStruct141",
> "values": {
> "edges": {},
> "vertices": {}
> }
> }
> }
> {noformat}
> UI snapshot: https://monosnap.com/file/3XcCcffPjrayjLaIByWcjBHU4k2pgV
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ATLAS-871) Make audit repository implementation configurable
[ https://issues.apache.org/jira/browse/ATLAS-871?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15320337#comment-15320337 ] Shwetha G S commented on ATLAS-871: --- [~jnhagelb], I am working on this in ATLAS-749 > Make audit repository implementation configurable > - > > Key: ATLAS-871 > URL: https://issues.apache.org/jira/browse/ATLAS-871 > Project: Atlas > Issue Type: Task >Reporter: Jeffrey Hagelberg >Assignee: Jeffrey Hagelberg > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (ATLAS-749) Make EntityAuditRepository implementation configurable
[ https://issues.apache.org/jira/browse/ATLAS-749?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S reassigned ATLAS-749: - Assignee: Shwetha G S > Make EntityAuditRepository implementation configurable > -- > > Key: ATLAS-749 > URL: https://issues.apache.org/jira/browse/ATLAS-749 > Project: Atlas > Issue Type: Improvement >Reporter: Shwetha G S >Assignee: Shwetha G S > Fix For: 0.7-incubating > > > EntityAuditRepository is bound to HBaseBasedAuditRepository in > RepositoryMetadataModule, which essentially forces hbase setup. Make it > configurable using atlas-application.properties on the lines of DeleteHandler > configuration -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-832) atlas_start should start embedded hbase and solr with -Pembedded_services
[ https://issues.apache.org/jira/browse/ATLAS-832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-832: -- Assignee: Vimal Sharma (was: Tom Beerbower) > atlas_start should start embedded hbase and solr with -Pembedded_services > - > > Key: ATLAS-832 > URL: https://issues.apache.org/jira/browse/ATLAS-832 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Vimal Sharma > Fix For: 0.7-incubating > > > As part of ATLAS-823, embedded hbase and solr was moved to profile > 'embedded_services'. After the package is built using '-Pembedded_services', > the package configuration points to local hbase and solr, but atlas_start > doesn't start embedded hbase and solr -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-832) atlas_start should start embedded hbase and solr with -Pembedded_services
[ https://issues.apache.org/jira/browse/ATLAS-832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15318196#comment-15318196 ] Shwetha G S commented on ATLAS-832: --- [~tbeerbower], are you working on this? If you haven't started looking at this, [~svimal2106] can pick it up. Thanks > atlas_start should start embedded hbase and solr with -Pembedded_services > - > > Key: ATLAS-832 > URL: https://issues.apache.org/jira/browse/ATLAS-832 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Tom Beerbower > Fix For: 0.7-incubating > > > As part of ATLAS-823, embedded hbase and solr was moved to profile > 'embedded_services'. After the package is built using '-Pembedded_services', > the package configuration points to local hbase and solr, but atlas_start > doesn't start embedded hbase and solr -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-727) Enhance Atlas client to add support for offset while searching.
[ https://issues.apache.org/jira/browse/ATLAS-727?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15317966#comment-15317966 ] Shwetha G S commented on ATLAS-727: --- [~guptaneeru], are you working on this? If you haven't started working on this, I can take it up. Let me know. Thanks > Enhance Atlas client to add support for offset while searching. > --- > > Key: ATLAS-727 > URL: https://issues.apache.org/jira/browse/ATLAS-727 > Project: Atlas > Issue Type: Improvement >Reporter: Neeru Gupta >Assignee: Neeru Gupta > -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-836) Add timeout for kafka notifications
[ https://issues.apache.org/jira/browse/ATLAS-836?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316237#comment-15316237 ] Shwetha G S commented on ATLAS-836: --- 30 secs is too much, we should set a lower value? > Add timeout for kafka notifications > --- > > Key: ATLAS-836 > URL: https://issues.apache.org/jira/browse/ATLAS-836 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S > Fix For: 0.7-incubating > > > In atlas conf, we don't have timeout for calls to kafka and have seen cases > where the kafka calls get stuck for a long time. We should have reasonable > timeouts -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-801) Atlas hooks would lose messages if Kafka is down for extended period of time
[ https://issues.apache.org/jira/browse/ATLAS-801?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15316226#comment-15316226 ] Shwetha G S commented on ATLAS-801: --- Agree with the short term solution for now. 1. Ensure multiple replicas for Kafka - the number of consumer threads will still be 1, right? Else, we might end up with some message handling failures 2. Add retries when sending messages to Kafka - Doesn't all hooks have configurable retries already, with default 3 3. Build some way of alerting admins of errors in communicating to Kafka - we probably need to hook into alerting mechanism of each component? Additionally, we can also write failed messages to separate file using log4j and use log4j to roll the file based on size > Atlas hooks would lose messages if Kafka is down for extended period of time > > > Key: ATLAS-801 > URL: https://issues.apache.org/jira/browse/ATLAS-801 > Project: Atlas > Issue Type: Improvement >Reporter: Hemanth Yamijala >Assignee: Hemanth Yamijala > > All integration hooks in Atlas write messages to Kafka which are picked up by > the Atlas server. If communication to Kafka breaks, then this results in loss > of metadata messages. This can be mitigated to some extent using multiple > replicas for Kafka topics (see ATLAS-515). This JIRA is to see if we can make > this even more robust and have some form of store and forward mechanism for > increased fault tolerance. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-834) Handle exceptions from HiveHook executor.submit()
[ https://issues.apache.org/jira/browse/ATLAS-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15305126#comment-15305126 ] Shwetha G S commented on ATLAS-834: --- +1 > Handle exceptions from HiveHook executor.submit() > - > > Key: ATLAS-834 > URL: https://issues.apache.org/jira/browse/ATLAS-834 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating, 0.7-incubating >Reporter: Shwetha G S >Assignee: Suma Shivaprasad >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-834.patch > > > Atlas hive hook uses threadpool with fixed queue size(to restrict memory). In > cases of high load, all threads may be busy and queue may be full in which > case, threadpool.submit() fails and hive command execution fails. We should > ignore this failure(will result in loss of data for atlas, but we don't have > any store forward solution) and let the hive command execution complete. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-834) Handle exceptions from HiveHook executor.submit()
[ https://issues.apache.org/jira/browse/ATLAS-834?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15305126#comment-15305126 ] Shwetha G S edited comment on ATLAS-834 at 5/28/16 2:56 AM: +1. Thanks Suma was (Author: shwethags): +1 > Handle exceptions from HiveHook executor.submit() > - > > Key: ATLAS-834 > URL: https://issues.apache.org/jira/browse/ATLAS-834 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating, 0.7-incubating >Reporter: Shwetha G S >Assignee: Suma Shivaprasad >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-834.patch > > > Atlas hive hook uses threadpool with fixed queue size(to restrict memory). In > cases of high load, all threads may be busy and queue may be full in which > case, threadpool.submit() fails and hive command execution fails. We should > ignore this failure(will result in loss of data for atlas, but we don't have > any store forward solution) and let the hive command execution complete. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-836) Add timeout for kafka notifications
Shwetha G S created ATLAS-836: - Summary: Add timeout for kafka notifications Key: ATLAS-836 URL: https://issues.apache.org/jira/browse/ATLAS-836 Project: Atlas Issue Type: Bug Reporter: Shwetha G S Fix For: 0.7-incubating In atlas conf, we don't have timeout for calls to kafka and have seen cases where the kafka calls get stuck for a long time. We should have reasonable timeouts -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-834) Handle exceptions from HiveHook executor.submit()
Shwetha G S created ATLAS-834: - Summary: Handle exceptions from HiveHook executor.submit() Key: ATLAS-834 URL: https://issues.apache.org/jira/browse/ATLAS-834 Project: Atlas Issue Type: Bug Reporter: Shwetha G S Priority: Critical Atlas hive hook uses threadpool with fixed queue size(to restrict memory). In cases of high load, all threads may be busy and queue may be full in which case, threadpool.submit() fails and hive command execution fails. We should ignore this failure(will result in loss of data for atlas, but we don't have any store forward solution) and let the hive command execution complete. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-834) Handle exceptions from HiveHook executor.submit()
[ https://issues.apache.org/jira/browse/ATLAS-834?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shwetha G S updated ATLAS-834: -- Assignee: Suma Shivaprasad > Handle exceptions from HiveHook executor.submit() > - > > Key: ATLAS-834 > URL: https://issues.apache.org/jira/browse/ATLAS-834 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Suma Shivaprasad >Priority: Critical > Fix For: 0.7-incubating > > > Atlas hive hook uses threadpool with fixed queue size(to restrict memory). In > cases of high load, all threads may be busy and queue may be full in which > case, threadpool.submit() fails and hive command execution fails. We should > ignore this failure(will result in loss of data for atlas, but we don't have > any store forward solution) and let the hive command execution complete. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
