[jira] [Updated] (ATLAS-884) Process registration should call Entity update instead of create
[ https://issues.apache.org/jira/browse/ATLAS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-884: --- Attachment: ATLAS-884.1.patch Thanks [~shwethags] . Added 2 tests - one to update with the same query, input and output tables and another one to CTAS, drop output table, rerun CTAS. the second test case's asserts might need to be fixed in ATLAS-876. > Process registration should call Entity update instead of create > > > Key: ATLAS-884 > URL: https://issues.apache.org/jira/browse/ATLAS-884 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating, 0.7-incubating > Reporter: Suma Shivaprasad > Assignee: Suma Shivaprasad > Fix For: 0.7-incubating > > Attachments: ATLAS-884.1.patch, ATLAS-884.patch > > > Process registration in hive hook currently calls Entity create . But should > call update entity instead -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-899) Fix Hive Hook documentation
Suma Shivaprasad created ATLAS-899: -- Summary: Fix Hive Hook documentation Key: ATLAS-899 URL: https://issues.apache.org/jira/browse/ATLAS-899 Project: Atlas Issue Type: Bug Affects Versions: 0.7-incubating Reporter: Suma Shivaprasad Assignee: Suma Shivaprasad Fix For: 0.7-incubating Hive Hook docs need to be fixed since there were model changes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-890) Log received messages in case of error
[ https://issues.apache.org/jira/browse/ATLAS-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-890: --- Attachment: ATLAS-890.patch > Log received messages in case of error > -- > > Key: ATLAS-890 > URL: https://issues.apache.org/jira/browse/ATLAS-890 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.7-incubating > > Attachments: ATLAS-890.patch > > > This will be useful for further debugging whenever an error occurs on ATLAS > writes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Assigned] (ATLAS-877) CreateTime of an entity(for example hive table) changes for every update
[
https://issues.apache.org/jira/browse/ATLAS-877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad reassigned ATLAS-877:
--
Assignee: Suma Shivaprasad (was: Shwetha G S)
> CreateTime of an entity(for example hive table) changes for every update
>
>
> Key: ATLAS-877
> URL: https://issues.apache.org/jira/browse/ATLAS-877
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Suma Shivaprasad
>Priority: Critical
> Fix For: 0.7-incubating
>
>
> Each time a hive table is updated(example column renamed), create time of
> table is also updated.
> UI snapshot: https://monosnap.com/file/6NoT86CNpHgQmlyfBQTSUa1y3Ncc2Y
> hive table metadata snapshot from hive cli
> {noformat}
> 0: jdbc:hive2://localhost:1/default> describe formatted
> column_changepgwcbgq1bd;
> OK
> +---+-+---+--+
> | col_name|
> data_type |comment|
> +---+-+---+--+
> | # col_name| data_type
> | comment |
> | | NULL
> | NULL |
> | a3| int
> | |
> | b | int
> | |
> | c | int
> | |
> | d | int
> | |
> | e | int
> | |
> | f | int
> | |
> | | NULL
> | NULL |
> | # Detailed Table Information | NULL
> | NULL |
> | Database: | db2pgwcbgq1bd
> | NULL |
> | Owner:| apathan
> | NULL |
> | CreateTime: | Wed Jun 08 12:41:28 IST 2016
> | NULL |
> | LastAccessTime: | UNKNOWN
> | NULL |
> | Protect Mode: | None
> | NULL |
> | Retention:| 0
> | NULL |
> | Location: |
> hdfs://localhost:9000/user/hive/warehouse/db2pgwcbgq1bd.db/column_changepgwcbgq1bd
> | NULL |
> | Table Type: | MANAGED_TABLE
> | NULL |
> | Table Parameters: | NULL
> | NULL |
> | | COLUMN_STATS_ACCURATE
> | false |
> | | last_modified_by
> | apathan |
> | | last_modified_time
[jira] [Updated] (ATLAS-882) with multiple taxonomies, UI does not allow associating terms with entities. "Add Terms" drop down shows all the taxonomies
[ https://issues.apache.org/jira/browse/ATLAS-882?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-882: --- Assignee: Keval Bhatt > with multiple taxonomies, UI does not allow associating terms with entities. > "Add Terms" drop down shows all the taxonomies > --- > > Key: ATLAS-882 > URL: https://issues.apache.org/jira/browse/ATLAS-882 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Ayub Khan >Assignee: Keval Bhatt >Priority: Blocker > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-06-08 at 4.20.45 PM.png > > > with multiple taxonomies, UI does not allow associating terms with entities. > "Add Terms" drop down shows all the taxonomies > Attaching UI snapshot. > Create multiple taxonomies with terms and try associating terms to entities, > UI does not allow it. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-881) UI: not displaying multiple taxonomies
[
https://issues.apache.org/jira/browse/ATLAS-881?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-881:
---
Assignee: Keval Bhatt
> UI: not displaying multiple taxonomies
> --
>
> Key: ATLAS-881
> URL: https://issues.apache.org/jira/browse/ATLAS-881
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Keval Bhatt
>Priority: Critical
> Fix For: 0.7-incubating
>
>
> UI: not displaying multiple taxonomies
> API response returns multiple taxonomies
> {noformat}
> curl 'http://localhost:21000/api/atlas/v1/taxonomies/' -H 'Cookie:
> JSESSIONID=19eplhd4dz8mueh7466qcbjwf' -H 'Accept-Encoding: gzip, deflate,
> sdch' -H 'Accept-Language: en-US,en;q=0.8' -H 'User-Agent: Mozilla/5.0
> (Macintosh; Intel Mac OS X 10_10_5) AppleWebKit/537.36 (KHTML, like Gecko)
> Chrome/50.0.2661.102 Safari/537.36' -H 'Accept: application/json,
> text/javascript, */*; q=0.01' -H 'Referer: http://localhost:21000/index.html'
> -H 'X-Requested-With: XMLHttpRequest' -H 'Connection: keep-alive'
> --compressed | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 3310 3310 0 4096 0 --:--:-- --:--:-- --:--:-- 4137
> [
> {
> "href": "http://localhost:21000/api/atlas/v1/taxonomies/empty";,
> "name": "empty"
> },
> {
> "href": "http://localhost:21000/api/atlas/v1/taxonomies/tax1";,
> "name": "tax1"
> },
> {
> "href": "http://localhost:21000/api/atlas/v1/taxonomies/default";,
> "name": "default"
> }
> ]
> {noformat}
> UI snapshot: https://monosnap.com/file/1M6dOB8D08IPMqkqj3LsCr8VYvXWJG
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-734) DSL queries are currently case sensitive, from customer's perspective it is better to make them case insensitive
[ https://issues.apache.org/jira/browse/ATLAS-734?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-734: --- Fix Version/s: (was: 0.7-incubating) > DSL queries are currently case sensitive, from customer's perspective it is > better to make them case insensitive > > > Key: ATLAS-734 > URL: https://issues.apache.org/jira/browse/ATLAS-734 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Ayub Khan > > DSL queries are currently case sensitive, from customer's perspective it is > better to make them case insensitive. > For example: hive_table DSL query would succeed but hive_Table DSL query will > fail. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-725) UI : Filter out or highlight deleted entities in search result outputs(dsl, text) , schema view, and lineage graph
[ https://issues.apache.org/jira/browse/ATLAS-725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-725: --- Assignee: Keval Bhatt Affects Version/s: 0.7-incubating Priority: Blocker (was: Major) > UI : Filter out or highlight deleted entities in search result outputs(dsl, > text) , schema view, and lineage graph > -- > > Key: ATLAS-725 > URL: https://issues.apache.org/jira/browse/ATLAS-725 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Blocker > Fix For: 0.7-incubating > > > As part of soft deletes, the entities deleted are also being returned in the > search results, entity definition GET json, schema outputs and lineage graph > results. These need to be highlighted in a different notation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-685) UI : HDFS datasets are not displayed correctly
[ https://issues.apache.org/jira/browse/ATLAS-685?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad resolved ATLAS-685. Resolution: Invalid Issue no loner exists in new ATLAS UI > UI : HDFS datasets are not displayed correctly > -- > > Key: ATLAS-685 > URL: https://issues.apache.org/jira/browse/ATLAS-685 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-04-08 at 2.40.24 PM.png > > > While testing HDFS datasets, found that the table alignment etc is not done > correctly and also need to decide on what attributes need to be displayed here -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-688) import-hive should depend on Hive CLASSPATH jars instead of packaging everything
[ https://issues.apache.org/jira/browse/ATLAS-688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-688: --- Priority: Critical (was: Major) > import-hive should depend on Hive CLASSPATH jars instead of packaging > everything > > > Key: ATLAS-688 > URL: https://issues.apache.org/jira/browse/ATLAS-688 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad >Assignee: Vimal Sharma >Priority: Critical > Attachments: ATLAS-688.patch > > > Currently, import-hive.sh packages all hive, hadoop jars etc. Instead it > should depend on HIVE_CLASSPATH from which it can get the jars and should be > paakged along with hook on hive server m/cs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-642) import-hive should create the lineage for external tables
[ https://issues.apache.org/jira/browse/ATLAS-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15329983#comment-15329983 ] Suma Shivaprasad commented on ATLAS-642: [~svimal2106] Pls rebase the patch - Unable to apply it 10:17:22-sshivaprasad~/workspace-latest/atlas-commit2 (master)$ git apply -v ATLAS-642.patch Checking patch addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java... error: while searching for: import org.apache.commons.configuration.Configuration; import org.apache.commons.lang.RandomStringUtils; import org.apache.hadoop.hive.conf.HiveConf; import org.apache.hadoop.hive.metastore.api.Database; import org.apache.hadoop.hive.metastore.api.FieldSchema; import org.apache.hadoop.hive.metastore.api.Order; error: patch failed: addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java:36 error: addons/hive-bridge/src/main/java/org/apache/atlas/hive/bridge/HiveMetaStoreBridge.java: patch does not apply Checking patch addons/hive-bridge/src/main/java/org/apache/atlas/hive/hook/HiveHook.java... Hunk #1 succeeded at 659 (offset 1 line). > import-hive should create the lineage for external tables > - > > Key: ATLAS-642 > URL: https://issues.apache.org/jira/browse/ATLAS-642 > Project: Atlas > Issue Type: Sub-task > Reporter: Suma Shivaprasad >Assignee: Vimal Sharma > Fix For: 0.7-incubating > > Attachments: ATLAS-642.patch > > > import-hive can also start showing lineage between HDFS paths and tables. > However the create table process should also register the exact query that > was used to create the table. Show create table does that by reconstructing > the query. Need to figure out if we can potentially use that to create the > query -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-688) import-hive should depend on Hive CLASSPATH jars instead of packaging everything
[ https://issues.apache.org/jira/browse/ATLAS-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15329997#comment-15329997 ] Suma Shivaprasad commented on ATLAS-688: HADOOP_CLASSPATH is expected to be set . Pls remove HADOOP_CP > import-hive should depend on Hive CLASSPATH jars instead of packaging > everything > > > Key: ATLAS-688 > URL: https://issues.apache.org/jira/browse/ATLAS-688 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad >Assignee: Vimal Sharma >Priority: Critical > Attachments: ATLAS-688.patch > > > Currently, import-hive.sh packages all hive, hadoop jars etc. Instead it > should depend on HIVE_CLASSPATH from which it can get the jars and should be > paakged along with hook on hive server m/cs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-890) Log received messages in case of error
[ https://issues.apache.org/jira/browse/ATLAS-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-890: --- Attachment: (was: ATLAS-890.patch) > Log received messages in case of error > -- > > Key: ATLAS-890 > URL: https://issues.apache.org/jira/browse/ATLAS-890 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.7-incubating > > > This will be useful for further debugging whenever an error occurs on ATLAS > writes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-890) Log received messages in case of error
[ https://issues.apache.org/jira/browse/ATLAS-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-890: --- Attachment: ATLAS-890.patch Attaching rebased patch > Log received messages in case of error > -- > > Key: ATLAS-890 > URL: https://issues.apache.org/jira/browse/ATLAS-890 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.7-incubating > > Attachments: ATLAS-890.patch > > > This will be useful for further debugging whenever an error occurs on ATLAS > writes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-688) import-hive should depend on Hive CLASSPATH jars instead of packaging everything
[ https://issues.apache.org/jira/browse/ATLAS-688?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-688: --- Fix Version/s: 0.7-incubating > import-hive should depend on Hive CLASSPATH jars instead of packaging > everything > > > Key: ATLAS-688 > URL: https://issues.apache.org/jira/browse/ATLAS-688 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad >Assignee: Vimal Sharma >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-688.patch > > > Currently, import-hive.sh packages all hive, hadoop jars etc. Instead it > should depend on HIVE_CLASSPATH from which it can get the jars and should be > paakged along with hook on hive server m/cs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-726) UI : dataset lineage process should be sorted according to process start time
[ https://issues.apache.org/jira/browse/ATLAS-726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-726: --- Priority: Critical (was: Major) > UI : dataset lineage process should be sorted according to process start time > - > > Key: ATLAS-726 > URL: https://issues.apache.org/jira/browse/ATLAS-726 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad >Priority: Critical > Fix For: 0.7-incubating > > > Currently transformation processes between datasets(all entities that inherit > from Process) are not displayed in any sort order. They need to be sorted by > start_time when displayed as the transformation between any 2 datasets. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-726) UI : dataset lineage process should be sorted according to process start time
[ https://issues.apache.org/jira/browse/ATLAS-726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-726: --- Assignee: Keval Bhatt > UI : dataset lineage process should be sorted according to process start time > - > > Key: ATLAS-726 > URL: https://issues.apache.org/jira/browse/ATLAS-726 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Fix For: 0.7-incubating > > > Currently transformation processes between datasets(all entities that inherit > from Process) are not displayed in any sort order. They need to be sorted by > start_time when displayed as the transformation between any 2 datasets. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-726) UI : dataset lineage process should be sorted according to process start time
[ https://issues.apache.org/jira/browse/ATLAS-726?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15330814#comment-15330814 ] Suma Shivaprasad commented on ATLAS-726: This is hard to consolidate across hooks currently since a base entity type with common attributes is not defined for process or any other entity. Hence downgrading to major > UI : dataset lineage process should be sorted according to process start time > - > > Key: ATLAS-726 > URL: https://issues.apache.org/jira/browse/ATLAS-726 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Fix For: 0.7-incubating > > > Currently transformation processes between datasets(all entities that inherit > from Process) are not displayed in any sort order. They need to be sorted by > start_time when displayed as the transformation between any 2 datasets. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-726) UI : dataset lineage process should be sorted according to process start time
[ https://issues.apache.org/jira/browse/ATLAS-726?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-726: --- Priority: Major (was: Critical) > UI : dataset lineage process should be sorted according to process start time > - > > Key: ATLAS-726 > URL: https://issues.apache.org/jira/browse/ATLAS-726 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt > Fix For: 0.7-incubating > > > Currently transformation processes between datasets(all entities that inherit > from Process) are not displayed in any sort order. They need to be sorted by > start_time when displayed as the transformation between any 2 datasets. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-884) Process registration should call Entity update instead of create
[ https://issues.apache.org/jira/browse/ATLAS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15330855#comment-15330855 ] Suma Shivaprasad commented on ATLAS-884: Fixed review comments > Process registration should call Entity update instead of create > > > Key: ATLAS-884 > URL: https://issues.apache.org/jira/browse/ATLAS-884 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating, 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.7-incubating > > Attachments: ATLAS-884.1.patch, ATLAS-884.2.patch, ATLAS-884.patch > > > Process registration in hive hook currently calls Entity create . But should > call update entity instead -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-884) Process registration should call Entity update instead of create
[ https://issues.apache.org/jira/browse/ATLAS-884?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-884: --- Attachment: ATLAS-884.2.patch > Process registration should call Entity update instead of create > > > Key: ATLAS-884 > URL: https://issues.apache.org/jira/browse/ATLAS-884 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.6-incubating, 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad > Fix For: 0.7-incubating > > Attachments: ATLAS-884.1.patch, ATLAS-884.2.patch, ATLAS-884.patch > > > Process registration in hive hook currently calls Entity create . But should > call update entity instead -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-888) NPE in NotificationHookConsumer
[ https://issues.apache.org/jira/browse/ATLAS-888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-888: --- Attachment: ATLAS-888.1.patch Fixed review comments > NPE in NotificationHookConsumer > --- > > Key: ATLAS-888 > URL: https://issues.apache.org/jira/browse/ATLAS-888 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Blocker > Fix For: 0.7-incubating > > Attachments: ATLAS-888.1.patch, ATLAS-888.patch > > > 2016-06-08 15:54:22,375 ERROR - [NotificationHookConsumer thread-0:] ~ Unable > to persist entity instance (EntityResource:137) > java.lang.NullPointerException > at > org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:122) > at > org.apache.atlas.LocalAtlasClient$1.invoke(LocalAtlasClient.java:82) > at > org.apache.atlas.LocalAtlasClient$EntityOperation.run(LocalAtlasClient.java:118) > at > org.apache.atlas.LocalAtlasClient.createEntity(LocalAtlasClient.java:85) > at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:652) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:203) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:187) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > 2016-06-08 15:54:22,376 WARN - [NotificationHookConsumer thread-0:] ~ Error > handling message > org.apache.atlas.notification.hook.HookNotification$EntityCreateRequest@6b8492f1 > (NotificationHookConsumer:233) > org.apache.atlas.AtlasServiceException: Metadata service API CREATE_ENTITY > failed with status 500(Internal Server Error) Response Body > (java.lang.NullPointerException > at > org.apache.atlas.web.resources.EntityResource.submit(EntityResource.java:122) > at > org.apache.atlas.LocalAtlasClient$1.invoke(LocalAtlasClient.java:82) > at > org.apache.atlas.LocalAtlasClient$EntityOperation.run(LocalAtlasClient.java:118) > at > org.apache.atlas.LocalAtlasClient.createEntity(LocalAtlasClient.java:85) > at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:652) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:203) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:187) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) > at > java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145) > at > java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615) > at java.lang.Thread.run(Thread.java:745) > ) > at > org.apache.atlas.LocalAtlasClient$EntityOperation.run(LocalAtlasClient.java:122) > at > org.apache.atlas.LocalAtlasClient.createEntity(LocalAtlasClient.java:85) > at org.apache.atlas.AtlasClient.createEntity(AtlasClient.java:652) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.handleMessage(NotificationHookConsumer.java:203) > at > org.apache.atlas.notification.NotificationHookConsumer$HookConsumer.run(NotificationHookConsumer.java:187) > at > java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471) > at java.util.concurrent.FutureTask.run(FutureTask.java:262) -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-899) Fix Hive Hook documentation
[ https://issues.apache.org/jira/browse/ATLAS-899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-899: --- Priority: Critical (was: Major) > Fix Hive Hook documentation > --- > > Key: ATLAS-899 > URL: https://issues.apache.org/jira/browse/ATLAS-899 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Critical > Fix For: 0.7-incubating > > > Hive Hook docs need to be fixed since there were model changes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-744) UI: HDFS_PATH DSL query results are showing href links for qualifiedName and Path
[ https://issues.apache.org/jira/browse/ATLAS-744?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-744: --- Priority: Critical (was: Major) > UI: HDFS_PATH DSL query results are showing href links for qualifiedName and > Path > - > > Key: ATLAS-744 > URL: https://issues.apache.org/jira/browse/ATLAS-744 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Ayub Khan >Assignee: Keval Bhatt >Priority: Critical > Fix For: 0.7-incubating > > > snapshot link: https://monosnap.com/file/TkmOEu6m77pWiia55wKXr9t5hcDfkR -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-761) UI: is ignoring table parameters
[
https://issues.apache.org/jira/browse/ATLAS-761?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-761:
---
Priority: Blocker (was: Major)
> UI: is ignoring table parameters
>
>
> Key: ATLAS-761
> URL: https://issues.apache.org/jira/browse/ATLAS-761
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Keval Bhatt
>Priority: Blocker
> Fix For: 0.7-incubating
>
>
> Entity api response showing table parameters which are ignored by UI.
> {noformat}
> curl
> http://localhost:21000/api/atlas/entities/b17b2c93-dbf7-4592-b59e-4614c6e3d37e
> | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 65080 65080 0 4152 0 --:--:-- 0:00:01 --:--:-- 4150
> {
> "GUID": "b17b2c93-dbf7-4592-b59e-4614c6e3d37e",
> "definition": {
> "id": {
> "id": "b17b2c93-dbf7-4592-b59e-4614c6e3d37e",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_table",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_table",
> "values": {
> "columns": [
> {
> "id": {
> "id": "f118ccf4-bd8b-4a0e-af35-17a5c5018e06",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "viewtime",
> "qualifiedName":
> "default.table3_s1vxgkt0bh.viewtime@primary",
> "table": {
> "id": "b17b2c93-dbf7-4592-b59e-4614c6e3d37e",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_table",
> "version": 0
> },
> "type": "int"
> }
> },
> {
> "id": {
> "id": "4b0d47e1-16ba-4cdd-85a5-6f35a968c0a3",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "userid",
> "qualifiedName":
> "default.table3_s1vxgkt0bh.userid@primary",
> "table": {
> "id&quo
[jira] [Commented] (ATLAS-761) UI: is ignoring table parameters
[
https://issues.apache.org/jira/browse/ATLAS-761?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15330934#comment-15330934
]
Suma Shivaprasad commented on ATLAS-761:
Table parameters are used to store user defined attributes
> UI: is ignoring table parameters
>
>
> Key: ATLAS-761
> URL: https://issues.apache.org/jira/browse/ATLAS-761
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Keval Bhatt
>Priority: Blocker
> Fix For: 0.7-incubating
>
>
> Entity api response showing table parameters which are ignored by UI.
> {noformat}
> curl
> http://localhost:21000/api/atlas/entities/b17b2c93-dbf7-4592-b59e-4614c6e3d37e
> | python -m json.tool
> % Total% Received % Xferd Average Speed TimeTime Time
> Current
> Dload Upload Total SpentLeft Speed
> 100 65080 65080 0 4152 0 --:--:-- 0:00:01 --:--:-- 4150
> {
> "GUID": "b17b2c93-dbf7-4592-b59e-4614c6e3d37e",
> "definition": {
> "id": {
> "id": "b17b2c93-dbf7-4592-b59e-4614c6e3d37e",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_table",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_table",
> "values": {
> "columns": [
> {
> "id": {
> "id": "f118ccf4-bd8b-4a0e-af35-17a5c5018e06",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "viewtime",
> "qualifiedName":
> "default.table3_s1vxgkt0bh.viewtime@primary",
> "table": {
> "id": "b17b2c93-dbf7-4592-b59e-4614c6e3d37e",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_table",
> "version": 0
> },
> "type": "int"
> }
> },
> {
> "id": {
> "id": "4b0d47e1-16ba-4cdd-85a5-6f35a968c0a3",
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "state": "ACTIVE",
> "typeName": "hive_column",
> "version": 0
> },
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "traitNames": [],
> "traits": {},
> "typeName": "hive_column",
> "values": {
> "comment": null,
> "name": "userid",
> "qualifiedName":
> "default.table3_s1vxgkt0bh.userid@primary",
>
[jira] [Commented] (ATLAS-755) UI: User defined tableproperties are ignored/not displayed.
[
https://issues.apache.org/jira/browse/ATLAS-755?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15330935#comment-15330935
]
Suma Shivaprasad commented on ATLAS-755:
[~ayubkhan] Is this a duplicate of ATLAS-761? If so , can you pls mark it
Resolved.
> UI: User defined tableproperties are ignored/not displayed.
> ---
>
> Key: ATLAS-755
> URL: https://issues.apache.org/jira/browse/ATLAS-755
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Keval Bhatt
> Fix For: 0.7-incubating
>
>
> Add hive table props: ALTER TABLE test_table set TBLPROPERTIES
> ('key1'='value1','comment'='test comment')
> Hive entity udpate message reflecting the table property changes..
> {noformat}
> {
> "entities": [
> {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "id": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "id": "-85198870717493",
> "version": 0,
> "typeName": "hive_db",
> "state": "ACTIVE"
> },
> "typeName": "hive_db",
> "values": {
> "name": "default",
> "description": "Default Hive database",
> "ownerType": 2,
> "qualifiedName": "default@cl1",
> "locationUri":
> "hdfs://os-r6-apathan-atlas-erie-nosec-2.openstacklocal:8020/apps/hive/warehouse",
> "owner": "public",
> "clusterName": "cl1",
> "parameters": {}
> },
> "traitNames": [],
> "traits": {}
> },
> {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "id": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "id": "-85198897495080",
> "version": 0,
> "typeName": "hive_table",
> "state": "ACTIVE"
> },
> "typeName": "hive_table",
> "values": {
> "tableType": "MANAGED_TABLE",
> "name": "default.test_table@cl1",
> "createTime": "2016-05-06T10:00:52.000Z",
> "temporary": false,
> "db": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Reference",
> "id": {
> "jsonClass":
> "org.apache.atlas.typesystem.json.InstanceSerialization$_Id",
> "id": "-85198870717493",
> "version": 0,
> "typeName": "hive_db",
> "state": "ACTIVE"
> },
> "typeName": "hive_db",
> "values": {
> "name": "default",
> "description": "Default Hive database",
> "ownerType": 2,
> "qualifiedName": "default@cl1",
> "locationUri":
> "hdfs://os-r6-apathan-atlas-erie-nosec-2.openstacklocal:8020/apps/hive/warehouse",
> "owner": "public",
> "clusterName": "cl1",
> "parameters": {}
> },
> "traitNames": [],
> "traits": {}
> },
> "
[jira] [Updated] (ATLAS-642) import-hive should create the lineage for external tables
[ https://issues.apache.org/jira/browse/ATLAS-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-642: --- Priority: Critical (was: Major) > import-hive should create the lineage for external tables > - > > Key: ATLAS-642 > URL: https://issues.apache.org/jira/browse/ATLAS-642 > Project: Atlas > Issue Type: Sub-task > Reporter: Suma Shivaprasad >Assignee: Vimal Sharma >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-642.patch > > > import-hive can also start showing lineage between HDFS paths and tables. > However the create table process should also register the exact query that > was used to create the table. Show create table does that by reconstructing > the query. Need to figure out if we can potentially use that to create the > query -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-890) Log received messages in case of error
[ https://issues.apache.org/jira/browse/ATLAS-890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-890: --- Priority: Critical (was: Major) > Log received messages in case of error > -- > > Key: ATLAS-890 > URL: https://issues.apache.org/jira/browse/ATLAS-890 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-890.patch > > > This will be useful for further debugging whenever an error occurs on ATLAS > writes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-903) EntityNotFoundException exception is observed when an entity is deleted.
Suma Shivaprasad created ATLAS-903:
--
Summary: EntityNotFoundException exception is observed when an
entity is deleted.
Key: ATLAS-903
URL: https://issues.apache.org/jira/browse/ATLAS-903
Project: Atlas
Issue Type: Bug
Affects Versions: 0.7-incubating
Reporter: Ayub Khan
Fix For: 0.7-incubating
Create hive table and drop hive table.
{noformat}
2016-05-05 19:21:04,095 DEBUG - [qtp1007251739-16 -
bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
ATLAS_ENTITIES:
{"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"2fdeeeb5-ad02-4746-afd4-8c478e9825c6","version":0,"typeName":"hive_table","state":"ACTIVE"},"typeName":"hive_table","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
(KafkaNotification:192)
2016-05-05 19:21:04,096 DEBUG - [qtp1007251739-16 -
bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
ATLAS_ENTITIES:
{"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"efe89140-3635-4779-a495-1d8371f8f939","version":0,"typeName":"hive_storagedesc","state":"ACTIVE"},"typeName":"hive_storagedesc","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
(KafkaNotification:192)
2016-05-05 19:21:04,096 DEBUG - [qtp1007251739-16 -
bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
ATLAS_ENTITIES:
{"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"57fd8ebc-e954-4669-bc47-5f617f8bc2fb","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
(KafkaNotification:192)
2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
ATLAS_ENTITIES, partition - 0, offset - 92 (KafkaNotification:199)
2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
ATLAS_ENTITIES, partition - 0, offset - 93 (KafkaNotification:199)
2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
ATLAS_ENTITIES, partition - 0, offset - 94 (KafkaNotification:199)
2016-05-05 19:21:04,099 DEBUG - [pool-9-thread-1:] ~ API
http://localhost:21000/api/atlas/entities?type=hive_table&property=name&value=default.t3@cl1
returned status 200 (AtlasClient:976)
2016-05-05 19:21:04,100 DEBUG - [pool-9-thread-1:] ~ Read message: conumerId:
0, topic - ATLAS_HOOK, partition - 0, offset - 33, message -
{"typeName":"hive_table","attribute":"name","attributeValue":"default.t3@cl1","type":"ENTITY_DELETE"}
(KafkaConsumer:68)
2016-05-05 19:21:04,108 INFO - [pool-9-thread-1:] ~ Real User: UNKNOWN
(auth:SIMPLE), is from ticket cache? false (SecureClientUtils:90)
2016-05-05 19:21:04,108 INFO - [pool-9-thread-1:] ~ doAsUser: UNKNOWN
(SecureClientUtils:93)
2016-05-05 19:21:04,142 INFO - [pool-9-thread-1:] ~ Client has only one
service URL, will use that for all actions: http://localhost:21000
(AtlasClient:204)
2016-05-05 19:21:04,143 DEBUG - [qtp1007251739-17 -
4b58a3a8-b1e2-4c7d-a830-1ba4251ca1f3:] ~ Audit: UNKNOWN/127.0.0.1 performed
request OPTIONS
http://localhost:21000/api/atlas/entities?type=hive_table&property=name&value=default.t3@cl1&doAs=UNKNOWN&user.name=UNKNOWN
(127.0.0.1) at time 2016-05-05T19:21Z (AuditFilter:91)
2016-05-05 19:21:04,146 DEBUG - [qtp1007251739-1307 -
bd408baa-baad-4582-8d37-53b3169b0bf6:] ~ Audit: UNKNOWN/127.0.0.1 performed
request DELETE
http://localhost:21000/api/atlas/entities?type=hive_table&property=name&value=default.t3@cl1&doAs=UNKNOWN
(127.0.0.1) at time 2016-05-05T19:21Z (AuditFilter:91)
2016-05-05 19:21:04,146 INFO - [qtp1007251739-1307 -
bd408baa-baad-4582-8d37-53b3
[jira] [Commented] (ATLAS-688) import-hive should depend on Hive CLASSPATH jars instead of packaging everything
[ https://issues.apache.org/jira/browse/ATLAS-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15330953#comment-15330953 ] Suma Shivaprasad commented on ATLAS-688: 1] If commons-httpclient is used, move to HttpComponents.HttpClient 4.3.6 or later 2] If using HttpComponents.HttpClient, make sure it is 4.3.6 or later See Umbrella JIRA BUG-57167 - [Umbrella] Update HttpClient Dependencies OPEN for details. > import-hive should depend on Hive CLASSPATH jars instead of packaging > everything > > > Key: ATLAS-688 > URL: https://issues.apache.org/jira/browse/ATLAS-688 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad >Assignee: Vimal Sharma >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-688.patch > > > Currently, import-hive.sh packages all hive, hadoop jars etc. Instead it > should depend on HIVE_CLASSPATH from which it can get the jars and should be > paakged along with hook on hive server m/cs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-688) import-hive should depend on Hive CLASSPATH jars instead of packaging everything
[ https://issues.apache.org/jira/browse/ATLAS-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15330953#comment-15330953 ] Suma Shivaprasad edited comment on ATLAS-688 at 6/15/16 12:49 AM: -- 1] If commons-httpclient is used, move to HttpComponents.HttpClient 4.3.6 or later 2] If using HttpComponents.HttpClient, make sure it is 4.3.6 or later was (Author: suma.shivaprasad): 1] If commons-httpclient is used, move to HttpComponents.HttpClient 4.3.6 or later 2] If using HttpComponents.HttpClient, make sure it is 4.3.6 or later See Umbrella JIRA BUG-57167 - [Umbrella] Update HttpClient Dependencies OPEN for details. > import-hive should depend on Hive CLASSPATH jars instead of packaging > everything > > > Key: ATLAS-688 > URL: https://issues.apache.org/jira/browse/ATLAS-688 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad >Assignee: Vimal Sharma >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-688.patch > > > Currently, import-hive.sh packages all hive, hadoop jars etc. Instead it > should depend on HIVE_CLASSPATH from which it can get the jars and should be > paakged along with hook on hive server m/cs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-688) import-hive should depend on Hive CLASSPATH jars instead of packaging everything
[ https://issues.apache.org/jira/browse/ATLAS-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15330953#comment-15330953 ] Suma Shivaprasad edited comment on ATLAS-688 at 6/15/16 12:52 AM: -- Update httpclient deps 1] If commons-httpclient is used, move to HttpComponents.HttpClient 4.3.6 or later 2] If using HttpComponents.HttpClient, make sure it is 4.3.6 or later was (Author: suma.shivaprasad): 1] If commons-httpclient is used, move to HttpComponents.HttpClient 4.3.6 or later 2] If using HttpComponents.HttpClient, make sure it is 4.3.6 or later > import-hive should depend on Hive CLASSPATH jars instead of packaging > everything > > > Key: ATLAS-688 > URL: https://issues.apache.org/jira/browse/ATLAS-688 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad >Assignee: Vimal Sharma >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-688.patch > > > Currently, import-hive.sh packages all hive, hadoop jars etc. Instead it > should depend on HIVE_CLASSPATH from which it can get the jars and should be > paakged along with hook on hive server m/cs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-903) EntityNotFoundException exception is observed when an entity is deleted.
[
https://issues.apache.org/jira/browse/ATLAS-903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-903:
---
Priority: Blocker (was: Major)
> EntityNotFoundException exception is observed when an entity is deleted.
>
>
> Key: ATLAS-903
> URL: https://issues.apache.org/jira/browse/ATLAS-903
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Priority: Blocker
> Fix For: 0.7-incubating
>
>
> Create hive table and drop hive table.
> {noformat}
> 2016-05-05 19:21:04,095 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
> ATLAS_ENTITIES:
> {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"2fdeeeb5-ad02-4746-afd4-8c478e9825c6","version":0,"typeName":"hive_table","state":"ACTIVE"},"typeName":"hive_table","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
> (KafkaNotification:192)
> 2016-05-05 19:21:04,096 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
> ATLAS_ENTITIES:
> {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"efe89140-3635-4779-a495-1d8371f8f939","version":0,"typeName":"hive_storagedesc","state":"ACTIVE"},"typeName":"hive_storagedesc","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
> (KafkaNotification:192)
> 2016-05-05 19:21:04,096 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
> ATLAS_ENTITIES:
> {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"57fd8ebc-e954-4669-bc47-5f617f8bc2fb","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
> (KafkaNotification:192)
> 2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
> ATLAS_ENTITIES, partition - 0, offset - 92 (KafkaNotification:199)
> 2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
> ATLAS_ENTITIES, partition - 0, offset - 93 (KafkaNotification:199)
> 2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
> ATLAS_ENTITIES, partition - 0, offset - 94 (KafkaNotification:199)
> 2016-05-05 19:21:04,099 DEBUG - [pool-9-thread-1:] ~ API
> http://localhost:21000/api/atlas/entities?type=hive_table&property=name&value=default.t3@cl1
> returned status 200 (AtlasClient:976)
> 2016-05-05 19:21:04,100 DEBUG - [pool-9-thread-1:] ~ Read message: conumerId:
> 0, topic - ATLAS_HOOK, partition - 0, offset - 33, message -
> {"typeName":"hive_table","attribute":"name","attributeValue":"default.t3@cl1","type":"ENTITY_DELETE"}
> (KafkaConsumer:68)
> 2016-05-05 19:21:04,108 INFO - [pool-9-thread-1:] ~ Real User: UNKNOWN
> (auth:SIMPLE), is from ticket cache? false (SecureClientUtils:90)
> 2016-05-05 19:21:04,108 INFO - [pool-9-thread-1:] ~ doAsUser: UNKNOWN
> (SecureClientUtils:93)
> 2016-05-05 19:21:04,142 INFO - [pool-9-thread-1:] ~ Client has only one
> service URL, will use that for all actions: http://localhost:21000
> (AtlasClient:204)
> 2016-05-05 19:21:04,143 DEBUG - [qtp1007251739-17 -
> 4b58a3a8-b1e2-4c7d-a830-1ba4251ca1f3:] ~ Audit: UNKNOWN/127.0.0.1 performed
> request OPTIONS
> http://localhost:21000/api/atlas/entities?type=hive_table&property=name&val
[jira] [Commented] (ATLAS-903) EntityNotFoundException exception is observed when an entity is deleted.
[
https://issues.apache.org/jira/browse/ATLAS-903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332221#comment-15332221
]
Suma Shivaprasad commented on ATLAS-903:
[~ayubkhan] Tried this with the latest apache build and unable to reproduce
this issue. Can you pls confirm is this issue still exists in the latest apache
build and steps to reproduce?
Seeing only this in logs
2016-06-15 11:01:48,823 INFO - [NotificationHookConsumer thread-0:] ~
Retrieving entity with type=hive_table and qualifiedName=default.test@primary
(GraphBackedMetadataRepository:158)
2016-06-15 11:01:48,937 INFO - [NotificationHookConsumer thread-0:] ~ Putting
0 events (HBaseBasedAuditRepository:103)
2016-06-15 11:01:48,938 INFO - [NotificationHookConsumer thread-0:] ~ Putting
0 events (HBaseBasedAuditRepository:103)
2016-06-15 11:01:48,938 INFO - [NotificationHookConsumer thread-0:] ~ Putting
3 events (HBaseBasedAuditRepository:103)
And the table is also dropped . I created and dropped a hive table as you
suggested.
> EntityNotFoundException exception is observed when an entity is deleted.
>
>
> Key: ATLAS-903
> URL: https://issues.apache.org/jira/browse/ATLAS-903
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Shwetha G S
>Priority: Blocker
> Fix For: 0.7-incubating
>
>
> Create hive table and drop hive table.
> {noformat}
> 2016-05-05 19:21:04,095 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
> ATLAS_ENTITIES:
> {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"2fdeeeb5-ad02-4746-afd4-8c478e9825c6","version":0,"typeName":"hive_table","state":"ACTIVE"},"typeName":"hive_table","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
> (KafkaNotification:192)
> 2016-05-05 19:21:04,096 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
> ATLAS_ENTITIES:
> {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"efe89140-3635-4779-a495-1d8371f8f939","version":0,"typeName":"hive_storagedesc","state":"ACTIVE"},"typeName":"hive_storagedesc","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
> (KafkaNotification:192)
> 2016-05-05 19:21:04,096 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
> ATLAS_ENTITIES:
> {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"57fd8ebc-e954-4669-bc47-5f617f8bc2fb","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
> (KafkaNotification:192)
> 2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
> ATLAS_ENTITIES, partition - 0, offset - 92 (KafkaNotification:199)
> 2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
> ATLAS_ENTITIES, partition - 0, offset - 93 (KafkaNotification:199)
> 2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
> ATLAS_ENTITIES, partition - 0, offset - 94 (KafkaNotification:199)
> 2016-05-05 19:21:04,099 DEBUG - [pool-9-thread-1:] ~ API
> http://localhost:21000/api/atlas/entities?type=hive_table&property=name&value=default.t3@cl1
> returned status 200 (AtlasClient:976)
> 2016-05-05 19:21:04,100 DEBUG - [pool-9-thread-1:] ~ Read message: conumerId:
> 0, topic - ATLAS_HOOK, partition - 0, of
[jira] [Commented] (ATLAS-903) EntityNotFoundException exception is observed when an entity is deleted.
[
https://issues.apache.org/jira/browse/ATLAS-903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332228#comment-15332228
]
Suma Shivaprasad commented on ATLAS-903:
This could occur in the case where the hive table didnt exist in ATLAS at all (
import was not run) and then the table is deleted. It is a known issue in that
case.
> EntityNotFoundException exception is observed when an entity is deleted.
>
>
> Key: ATLAS-903
> URL: https://issues.apache.org/jira/browse/ATLAS-903
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Shwetha G S
>Priority: Blocker
> Fix For: 0.7-incubating
>
>
> Create hive table and drop hive table.
> {noformat}
> 2016-05-05 19:21:04,095 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
> ATLAS_ENTITIES:
> {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"2fdeeeb5-ad02-4746-afd4-8c478e9825c6","version":0,"typeName":"hive_table","state":"ACTIVE"},"typeName":"hive_table","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
> (KafkaNotification:192)
> 2016-05-05 19:21:04,096 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
> ATLAS_ENTITIES:
> {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"efe89140-3635-4779-a495-1d8371f8f939","version":0,"typeName":"hive_storagedesc","state":"ACTIVE"},"typeName":"hive_storagedesc","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
> (KafkaNotification:192)
> 2016-05-05 19:21:04,096 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
> ATLAS_ENTITIES:
> {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"57fd8ebc-e954-4669-bc47-5f617f8bc2fb","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
> (KafkaNotification:192)
> 2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
> ATLAS_ENTITIES, partition - 0, offset - 92 (KafkaNotification:199)
> 2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
> ATLAS_ENTITIES, partition - 0, offset - 93 (KafkaNotification:199)
> 2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
> ATLAS_ENTITIES, partition - 0, offset - 94 (KafkaNotification:199)
> 2016-05-05 19:21:04,099 DEBUG - [pool-9-thread-1:] ~ API
> http://localhost:21000/api/atlas/entities?type=hive_table&property=name&value=default.t3@cl1
> returned status 200 (AtlasClient:976)
> 2016-05-05 19:21:04,100 DEBUG - [pool-9-thread-1:] ~ Read message: conumerId:
> 0, topic - ATLAS_HOOK, partition - 0, offset - 33, message -
> {"typeName":"hive_table","attribute":"name","attributeValue":"default.t3@cl1","type":"ENTITY_DELETE"}
> (KafkaConsumer:68)
> 2016-05-05 19:21:04,108 INFO - [pool-9-thread-1:] ~ Real User: UNKNOWN
> (auth:SIMPLE), is from ticket cache? false (SecureClientUtils:90)
> 2016-05-05 19:21:04,108 INFO - [pool-9-thread-1:] ~ doAsUser: UNKNOWN
> (SecureClientUtils:93)
> 2016-05-05 19:21:04,142 INFO - [pool-9-thread-1:] ~ Client has only one
> service URL, will use that for all actions: http://localhost:21000
> (AtlasClient:204)
> 2
[jira] [Updated] (ATLAS-725) UI : Filter out or highlight deleted entities in search result outputs(dsl, text) , schema view, and lineage graph
[ https://issues.apache.org/jira/browse/ATLAS-725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-725: --- Attachment: c7d7bd29-837c-497d-8b39-766dd4123615.json Attached json response for a dropped entity ( In this case hive_table). > UI : Filter out or highlight deleted entities in search result outputs(dsl, > text) , schema view, and lineage graph > -- > > Key: ATLAS-725 > URL: https://issues.apache.org/jira/browse/ATLAS-725 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Andrew Ahn >Priority: Blocker > Fix For: 0.7-incubating > > Attachments: c7d7bd29-837c-497d-8b39-766dd4123615.json > > > As part of soft deletes, the entities deleted are also being returned in the > search results, entity definition GET json, schema outputs and lineage graph > results. These need to be highlighted in a different notation -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-725) UI : Filter out or highlight deleted entities in search result outputs(dsl, text) , schema view, and lineage graph
[
https://issues.apache.org/jira/browse/ATLAS-725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332234#comment-15332234
]
Suma Shivaprasad commented on ATLAS-725:
As can be observed in the response,
Each entity has an Id which has the attribute state as below
{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"c7d7bd29-837c-497d-8b39-766dd4123615","version":0,"typeName":"hive_table","state":"DELETED"}
state can currently be ACTIVE, DELETED.
> UI : Filter out or highlight deleted entities in search result outputs(dsl,
> text) , schema view, and lineage graph
> --
>
> Key: ATLAS-725
> URL: https://issues.apache.org/jira/browse/ATLAS-725
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
>Assignee: Andrew Ahn
>Priority: Blocker
> Fix For: 0.7-incubating
>
> Attachments: c7d7bd29-837c-497d-8b39-766dd4123615.json
>
>
> As part of soft deletes, the entities deleted are also being returned in the
> search results, entity definition GET json, schema outputs and lineage graph
> results. These need to be highlighted in a different notation
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ATLAS-904) Hive hook fails due to rewrite query error in some cases like export to hdfs path
[
https://issues.apache.org/jira/browse/ATLAS-904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332287#comment-15332287
]
Suma Shivaprasad commented on ATLAS-904:
Hook should proceed with original query in this case.
> Hive hook fails due to rewrite query error in some cases like export to hdfs
> path
> -
>
> Key: ATLAS-904
> URL: https://issues.apache.org/jira/browse/ATLAS-904
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
>
> {noformat}
> 2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:normalize(557)) - Could not rewrite query due to error.
> Proceeding with original query EXPORT TABLE test_export_table to
> 'hdfs://localhost:9000/hive_tables/test_path1'
> java.lang.NullPointerException: Conf non-local session path expected to be
> non-null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
> at org.apache.hadoop.hive.ql.Context.(Context.java:133)
> at org.apache.hadoop.hive.ql.Context.(Context.java:120)
> at
> org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
> at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-15 11:34:30,423 ERROR [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:run(184)) - Atlas hook failed due to error
> java.lang.NullPointerException
> at java.lang.StringBuilder.(StringBuilder.java:109)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessQualifiedName(HiveHook.java:738)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:703)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-904) Hive hook fails due to rewrite query error in some cases like export to hdfs path
[
https://issues.apache.org/jira/browse/ATLAS-904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332287#comment-15332287
]
Suma Shivaprasad edited comment on ATLAS-904 at 6/15/16 6:40 PM:
-
Hook should proceed with original query in this case. However it is being
returned as null causing it to fail.
was (Author: suma.shivaprasad):
Hook should proceed with original query in this case.
> Hive hook fails due to rewrite query error in some cases like export to hdfs
> path
> -
>
> Key: ATLAS-904
> URL: https://issues.apache.org/jira/browse/ATLAS-904
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
>
> {noformat}
> 2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:normalize(557)) - Could not rewrite query due to error.
> Proceeding with original query EXPORT TABLE test_export_table to
> 'hdfs://localhost:9000/hive_tables/test_path1'
> java.lang.NullPointerException: Conf non-local session path expected to be
> non-null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
> at org.apache.hadoop.hive.ql.Context.(Context.java:133)
> at org.apache.hadoop.hive.ql.Context.(Context.java:120)
> at
> org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
> at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-15 11:34:30,423 ERROR [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:run(184)) - Atlas hook failed due to error
> java.lang.NullPointerException
> at java.lang.StringBuilder.(StringBuilder.java:109)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessQualifiedName(HiveHook.java:738)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:703)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (ATLAS-904) Hive hook fails due to rewrite query error in some cases like export to hdfs path
Suma Shivaprasad created ATLAS-904:
--
Summary: Hive hook fails due to rewrite query error in some cases
like export to hdfs path
Key: ATLAS-904
URL: https://issues.apache.org/jira/browse/ATLAS-904
Project: Atlas
Issue Type: Bug
Affects Versions: 0.7-incubating
Reporter: Suma Shivaprasad
Assignee: Suma Shivaprasad
Priority: Blocker
Fix For: 0.7-incubating
{noformat}
2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
(HiveHook.java:normalize(557)) - Could not rewrite query due to error.
Proceeding with original query EXPORT TABLE test_export_table to
'hdfs://localhost:9000/hive_tables/test_path1'
java.lang.NullPointerException: Conf non-local session path expected to be
non-null
at
com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
at
org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
at org.apache.hadoop.hive.ql.Context.(Context.java:133)
at org.apache.hadoop.hive.ql.Context.(Context.java:120)
at
org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
at
org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
at
org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
2016-06-15 11:34:30,423 ERROR [Atlas Logger 0]: hook.HiveHook
(HiveHook.java:run(184)) - Atlas hook failed due to error
java.lang.NullPointerException
at java.lang.StringBuilder.(StringBuilder.java:109)
at
org.apache.atlas.hive.hook.HiveHook.getProcessQualifiedName(HiveHook.java:738)
at
org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:703)
at
org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
at
java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
{noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-904) Hive hook fails due to rewrite query error in some cases like export to hdfs path
[
https://issues.apache.org/jira/browse/ATLAS-904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332287#comment-15332287
]
Suma Shivaprasad edited comment on ATLAS-904 at 6/15/16 9:10 PM:
-
Hook should proceed with original query in this case. However it is being
returned as null causing it to fail. This is a workaround fix until we figure
out how to set SessionState in hive before parsing optimally (since its a heavy
call)
was (Author: suma.shivaprasad):
Hook should proceed with original query in this case. However it is being
returned as null causing it to fail.
> Hive hook fails due to rewrite query error in some cases like export to hdfs
> path
> -
>
> Key: ATLAS-904
> URL: https://issues.apache.org/jira/browse/ATLAS-904
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
>
> {noformat}
> 2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:normalize(557)) - Could not rewrite query due to error.
> Proceeding with original query EXPORT TABLE test_export_table to
> 'hdfs://localhost:9000/hive_tables/test_path1'
> java.lang.NullPointerException: Conf non-local session path expected to be
> non-null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
> at org.apache.hadoop.hive.ql.Context.(Context.java:133)
> at org.apache.hadoop.hive.ql.Context.(Context.java:120)
> at
> org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
> at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-15 11:34:30,423 ERROR [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:run(184)) - Atlas hook failed due to error
> java.lang.NullPointerException
> at java.lang.StringBuilder.(StringBuilder.java:109)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessQualifiedName(HiveHook.java:738)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:703)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-907) UI: Hive process - Recent queries is not being displayed for hive_process
[ https://issues.apache.org/jira/browse/ATLAS-907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-907: --- Attachment: 620c821b-743a-4ef7-9219-a32b6f19f872.json sample json response > UI: Hive process - Recent queries is not being displayed for hive_process > - > > Key: ATLAS-907 > URL: https://issues.apache.org/jira/browse/ATLAS-907 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Priority: Critical > Fix For: 0.7-incubating > > Attachments: 620c821b-743a-4ef7-9219-a32b6f19f872.json > > > hive_process.recentQueries is showing up as Blank even though it is being > returned in the entity response. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-907) UI: Hive process - Recent queries is not being displayed for hive_process
[ https://issues.apache.org/jira/browse/ATLAS-907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-907: --- Assignee: Keval Bhatt > UI: Hive process - Recent queries is not being displayed for hive_process > - > > Key: ATLAS-907 > URL: https://issues.apache.org/jira/browse/ATLAS-907 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Fix For: 0.7-incubating > > Attachments: 620c821b-743a-4ef7-9219-a32b6f19f872.json, Screen Shot > 2016-06-15 at 2.48.02 PM.png > > > hive_process.recentQueries is showing up as Blank even though it is being > returned in the entity response. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-907) UI: Hive process - Recent queries is not being displayed for hive_process
[ https://issues.apache.org/jira/browse/ATLAS-907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332664#comment-15332664 ] Suma Shivaprasad commented on ATLAS-907: cc [~madhan.neethiraj] > UI: Hive process - Recent queries is not being displayed for hive_process > - > > Key: ATLAS-907 > URL: https://issues.apache.org/jira/browse/ATLAS-907 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Fix For: 0.7-incubating > > Attachments: 620c821b-743a-4ef7-9219-a32b6f19f872.json, Screen Shot > 2016-06-15 at 2.48.02 PM.png > > > hive_process.recentQueries is showing up as Blank even though it is being > returned in the entity response. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-907) UI: Hive process - Recent queries is not being displayed for hive_process
Suma Shivaprasad created ATLAS-907: -- Summary: UI: Hive process - Recent queries is not being displayed for hive_process Key: ATLAS-907 URL: https://issues.apache.org/jira/browse/ATLAS-907 Project: Atlas Issue Type: Bug Affects Versions: 0.7-incubating Reporter: Suma Shivaprasad Priority: Critical Fix For: 0.7-incubating hive_process.recentQueries is showing up as Blank even though it is being returned in the entity response. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-907) UI: Hive process - Recent queries is not being displayed for hive_process
[ https://issues.apache.org/jira/browse/ATLAS-907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-907: --- Attachment: Screen Shot 2016-06-15 at 2.48.02 PM.png > UI: Hive process - Recent queries is not being displayed for hive_process > - > > Key: ATLAS-907 > URL: https://issues.apache.org/jira/browse/ATLAS-907 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Priority: Critical > Fix For: 0.7-incubating > > Attachments: 620c821b-743a-4ef7-9219-a32b6f19f872.json, Screen Shot > 2016-06-15 at 2.48.02 PM.png > > > hive_process.recentQueries is showing up as Blank even though it is being > returned in the entity response. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-904) Hive hook fails due to rewrite query error in some cases like export to hdfs path
[
https://issues.apache.org/jira/browse/ATLAS-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-904:
---
Attachment: ATLAS-904.patch
Attached patch after testing manually. Writing a test for this would require
HIve Hook IT to test the same with HDFS paths. Will add that as part of another
jira
> Hive hook fails due to rewrite query error in some cases like export to hdfs
> path
> -
>
> Key: ATLAS-904
> URL: https://issues.apache.org/jira/browse/ATLAS-904
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-904.patch
>
>
> {noformat}
> 2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:normalize(557)) - Could not rewrite query due to error.
> Proceeding with original query EXPORT TABLE test_export_table to
> 'hdfs://localhost:9000/hive_tables/test_path1'
> java.lang.NullPointerException: Conf non-local session path expected to be
> non-null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
> at org.apache.hadoop.hive.ql.Context.(Context.java:133)
> at org.apache.hadoop.hive.ql.Context.(Context.java:120)
> at
> org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
> at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-15 11:34:30,423 ERROR [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:run(184)) - Atlas hook failed due to error
> java.lang.NullPointerException
> at java.lang.StringBuilder.(StringBuilder.java:109)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessQualifiedName(HiveHook.java:738)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:703)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (ATLAS-724) UI: Lineage view should display datasets as the source and target view ( more intuitive )
[ https://issues.apache.org/jira/browse/ATLAS-724?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad resolved ATLAS-724. Resolution: Fixed Did not observe this issue in the new UI . Hence closing. > UI: Lineage view should display datasets as the source and target view ( more > intuitive ) > - > > Key: ATLAS-724 > URL: https://issues.apache.org/jira/browse/ATLAS-724 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-04-28 at 12.43.33 PM.png > > > Attaching a snapshot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-904) Hive hook fails due to rewrite query error in some cases like export to hdfs path
[
https://issues.apache.org/jira/browse/ATLAS-904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332891#comment-15332891
]
Suma Shivaprasad commented on ATLAS-904:
This also includes fix for ATLAS-877
> Hive hook fails due to rewrite query error in some cases like export to hdfs
> path
> -
>
> Key: ATLAS-904
> URL: https://issues.apache.org/jira/browse/ATLAS-904
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-904.patch
>
>
> {noformat}
> 2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:normalize(557)) - Could not rewrite query due to error.
> Proceeding with original query EXPORT TABLE test_export_table to
> 'hdfs://localhost:9000/hive_tables/test_path1'
> java.lang.NullPointerException: Conf non-local session path expected to be
> non-null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
> at org.apache.hadoop.hive.ql.Context.(Context.java:133)
> at org.apache.hadoop.hive.ql.Context.(Context.java:120)
> at
> org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
> at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-15 11:34:30,423 ERROR [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:run(184)) - Atlas hook failed due to error
> java.lang.NullPointerException
> at java.lang.StringBuilder.(StringBuilder.java:109)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessQualifiedName(HiveHook.java:738)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:703)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-904) Hive hook fails due to rewrite query error in some cases like export to hdfs path
[
https://issues.apache.org/jira/browse/ATLAS-904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332891#comment-15332891
]
Suma Shivaprasad edited comment on ATLAS-904 at 6/16/16 2:02 AM:
-
This also includes fix for ATLAS-877 since the fix is minor
was (Author: suma.shivaprasad):
This also includes fix for ATLAS-877
> Hive hook fails due to rewrite query error in some cases like export to hdfs
> path
> -
>
> Key: ATLAS-904
> URL: https://issues.apache.org/jira/browse/ATLAS-904
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-904.patch
>
>
> {noformat}
> 2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:normalize(557)) - Could not rewrite query due to error.
> Proceeding with original query EXPORT TABLE test_export_table to
> 'hdfs://localhost:9000/hive_tables/test_path1'
> java.lang.NullPointerException: Conf non-local session path expected to be
> non-null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
> at org.apache.hadoop.hive.ql.Context.(Context.java:133)
> at org.apache.hadoop.hive.ql.Context.(Context.java:120)
> at
> org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
> at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-15 11:34:30,423 ERROR [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:run(184)) - Atlas hook failed due to error
> java.lang.NullPointerException
> at java.lang.StringBuilder.(StringBuilder.java:109)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessQualifiedName(HiveHook.java:738)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:703)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ATLAS-904) Hive hook fails due to rewrite query error in some cases like export to hdfs path
[
https://issues.apache.org/jira/browse/ATLAS-904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332889#comment-15332889
]
Suma Shivaprasad commented on ATLAS-904:
Reverted changes with ATLAS-619 since doing a SessionState.start in post
execution hook and hive parising is considered risky and resource intensive
after discussions with Hive team
> Hive hook fails due to rewrite query error in some cases like export to hdfs
> path
> -
>
> Key: ATLAS-904
> URL: https://issues.apache.org/jira/browse/ATLAS-904
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-904.patch
>
>
> {noformat}
> 2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:normalize(557)) - Could not rewrite query due to error.
> Proceeding with original query EXPORT TABLE test_export_table to
> 'hdfs://localhost:9000/hive_tables/test_path1'
> java.lang.NullPointerException: Conf non-local session path expected to be
> non-null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
> at org.apache.hadoop.hive.ql.Context.(Context.java:133)
> at org.apache.hadoop.hive.ql.Context.(Context.java:120)
> at
> org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
> at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-15 11:34:30,423 ERROR [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:run(184)) - Atlas hook failed due to error
> java.lang.NullPointerException
> at java.lang.StringBuilder.(StringBuilder.java:109)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessQualifiedName(HiveHook.java:738)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:703)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-904) Hive hook fails due to rewrite query error in some cases like export to hdfs path
[
https://issues.apache.org/jira/browse/ATLAS-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-904:
---
Attachment: ATLAS-904.1.patch
> Hive hook fails due to rewrite query error in some cases like export to hdfs
> path
> -
>
> Key: ATLAS-904
> URL: https://issues.apache.org/jira/browse/ATLAS-904
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Suma Shivaprasad
> Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-904.1.patch, ATLAS-904.patch
>
>
> {noformat}
> 2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:normalize(557)) - Could not rewrite query due to error.
> Proceeding with original query EXPORT TABLE test_export_table to
> 'hdfs://localhost:9000/hive_tables/test_path1'
> java.lang.NullPointerException: Conf non-local session path expected to be
> non-null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
> at org.apache.hadoop.hive.ql.Context.(Context.java:133)
> at org.apache.hadoop.hive.ql.Context.(Context.java:120)
> at
> org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
> at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-15 11:34:30,423 ERROR [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:run(184)) - Atlas hook failed due to error
> java.lang.NullPointerException
> at java.lang.StringBuilder.(StringBuilder.java:109)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessQualifiedName(HiveHook.java:738)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:703)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Resolved] (ATLAS-907) UI: Hive process - Recent queries is not being displayed for hive_process
[ https://issues.apache.org/jira/browse/ATLAS-907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad resolved ATLAS-907. Resolution: Won't Fix This may not be needed after ATLAS-904. Hence closing. > UI: Hive process - Recent queries is not being displayed for hive_process > - > > Key: ATLAS-907 > URL: https://issues.apache.org/jira/browse/ATLAS-907 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > Fix For: 0.7-incubating > > Attachments: 620c821b-743a-4ef7-9219-a32b6f19f872.json, Screen Shot > 2016-06-15 at 2.48.02 PM.png > > > hive_process.recentQueries is showing up as Blank even though it is being > returned in the entity response. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-904) Hive hook fails due to session state not being set
[
https://issues.apache.org/jira/browse/ATLAS-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-904:
---
Summary: Hive hook fails due to session state not being set (was: Hive
hook fails due to rewrite query error in some cases like export to hdfs path)
> Hive hook fails due to session state not being set
> --
>
> Key: ATLAS-904
> URL: https://issues.apache.org/jira/browse/ATLAS-904
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
> Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-904.1.patch, ATLAS-904.patch
>
>
> {noformat}
> 2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:normalize(557)) - Could not rewrite query due to error.
> Proceeding with original query EXPORT TABLE test_export_table to
> 'hdfs://localhost:9000/hive_tables/test_path1'
> java.lang.NullPointerException: Conf non-local session path expected to be
> non-null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
> at org.apache.hadoop.hive.ql.Context.(Context.java:133)
> at org.apache.hadoop.hive.ql.Context.(Context.java:120)
> at
> org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
> at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-15 11:34:30,423 ERROR [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:run(184)) - Atlas hook failed due to error
> java.lang.NullPointerException
> at java.lang.StringBuilder.(StringBuilder.java:109)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessQualifiedName(HiveHook.java:738)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:703)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-909) Lineage view is broken when there are lots of queries /processes between two datasets
[ https://issues.apache.org/jira/browse/ATLAS-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-909: --- Attachment: Screen Shot 2016-06-15 at 6.03.01 PM.png > Lineage view is broken when there are lots of queries /processes between two > datasets > - > > Key: ATLAS-909 > URL: https://issues.apache.org/jira/browse/ATLAS-909 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-06-15 at 6.03.01 PM.png > > > LIneage view does not have a scroll option and its height/width seem to be > limited which was not the case with the older dashboard. This is seen easily > when there are around more than 15-20 queries between two tables. Uploading > screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-909) Lineage view is broken when there are lots of queries /processes between two datasets
Suma Shivaprasad created ATLAS-909: -- Summary: Lineage view is broken when there are lots of queries /processes between two datasets Key: ATLAS-909 URL: https://issues.apache.org/jira/browse/ATLAS-909 Project: Atlas Issue Type: Bug Affects Versions: 0.7-incubating Reporter: Suma Shivaprasad Assignee: Keval Bhatt Fix For: 0.7-incubating Attachments: Screen Shot 2016-06-15 at 6.03.01 PM.png LIneage view does not have a scroll option and its height/width seem to be limited which was not the case with the older dashboard. This is seen easily when there are around more than 15-20 queries between two tables. Uploading screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-909) Lineage view is broken when there are lots of queries /processes between two datasets
[ https://issues.apache.org/jira/browse/ATLAS-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332917#comment-15332917 ] Suma Shivaprasad commented on ATLAS-909: cc [~madhan.neethiraj] > Lineage view is broken when there are lots of queries /processes between two > datasets > - > > Key: ATLAS-909 > URL: https://issues.apache.org/jira/browse/ATLAS-909 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-06-15 at 6.03.01 PM.png > > > LIneage view does not have a scroll option and its height/width seem to be > limited which was not the case with the older dashboard. This is seen easily > when there are around more than 15-20 queries between two tables. Uploading > screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-909) Lineage view is broken when there are lots of queries /processes between two datasets
[ https://issues.apache.org/jira/browse/ATLAS-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-909: --- Priority: Blocker (was: Major) > Lineage view is broken when there are lots of queries /processes between two > datasets > - > > Key: ATLAS-909 > URL: https://issues.apache.org/jira/browse/ATLAS-909 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Blocker > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-06-15 at 6.03.01 PM.png > > > LIneage view does not have a scroll option and its height/width seem to be > limited which was not the case with the older dashboard. This is seen easily > when there are around more than 15-20 queries between two tables. Uploading > screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-899) Fix Hive Hook documentation
[ https://issues.apache.org/jira/browse/ATLAS-899?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-899: --- Attachment: ATLAS-899.patch > Fix Hive Hook documentation > --- > > Key: ATLAS-899 > URL: https://issues.apache.org/jira/browse/ATLAS-899 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating > Reporter: Suma Shivaprasad >Assignee: Suma Shivaprasad >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-899.patch > > > Hive Hook docs need to be fixed since there were model changes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-909) Lineage view is broken when there are lots of queries /processes between two datasets
[ https://issues.apache.org/jira/browse/ATLAS-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332917#comment-15332917 ] Suma Shivaprasad edited comment on ATLAS-909 at 6/16/16 2:39 AM: - cc [~madhan.neethiraj] [~yhemanth] was (Author: suma.shivaprasad): cc [~madhan.neethiraj] > Lineage view is broken when there are lots of queries /processes between two > datasets > - > > Key: ATLAS-909 > URL: https://issues.apache.org/jira/browse/ATLAS-909 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Blocker > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-06-15 at 6.03.01 PM.png > > > LIneage view does not have a scroll option and its height/width seem to be > limited which was not the case with the older dashboard. This is seen easily > when there are around more than 15-20 queries between two tables. Uploading > screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-642) import-hive should create the lineage for external tables
[ https://issues.apache.org/jira/browse/ATLAS-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332967#comment-15332967 ] Suma Shivaprasad commented on ATLAS-642: [~svimal2106] there were some conflicts again. Rebased patch which I am uploading. > import-hive should create the lineage for external tables > - > > Key: ATLAS-642 > URL: https://issues.apache.org/jira/browse/ATLAS-642 > Project: Atlas > Issue Type: Sub-task > Reporter: Suma Shivaprasad >Assignee: Vimal Sharma >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-642.patch, ATLAS-642.patch > > > import-hive can also start showing lineage between HDFS paths and tables. > However the create table process should also register the exact query that > was used to create the table. Show create table does that by reconstructing > the query. Need to figure out if we can potentially use that to create the > query -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-642) import-hive should create the lineage for external tables
[ https://issues.apache.org/jira/browse/ATLAS-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332967#comment-15332967 ] Suma Shivaprasad edited comment on ATLAS-642 at 6/16/16 3:05 AM: - [~svimal2106] there were some conflicts again. Rebased patch which I am uploading. But there are some tests failing after rebasing. Can you pls check? was (Author: suma.shivaprasad): [~svimal2106] there were some conflicts again. Rebased patch which I am uploading. > import-hive should create the lineage for external tables > - > > Key: ATLAS-642 > URL: https://issues.apache.org/jira/browse/ATLAS-642 > Project: Atlas > Issue Type: Sub-task > Reporter: Suma Shivaprasad >Assignee: Vimal Sharma >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-642.patch, ATLAS-642.patch, ATLAS-642.patch > > > import-hive can also start showing lineage between HDFS paths and tables. > However the create table process should also register the exact query that > was used to create the table. Show create table does that by reconstructing > the query. Need to figure out if we can potentially use that to create the > query -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-642) import-hive should create the lineage for external tables
[ https://issues.apache.org/jira/browse/ATLAS-642?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-642: --- Attachment: ATLAS-642.patch > import-hive should create the lineage for external tables > - > > Key: ATLAS-642 > URL: https://issues.apache.org/jira/browse/ATLAS-642 > Project: Atlas > Issue Type: Sub-task > Reporter: Suma Shivaprasad >Assignee: Vimal Sharma >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-642.patch, ATLAS-642.patch, ATLAS-642.patch > > > import-hive can also start showing lineage between HDFS paths and tables. > However the create table process should also register the exact query that > was used to create the table. Show create table does that by reconstructing > the query. Need to figure out if we can potentially use that to create the > query -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-642) import-hive should create the lineage for external tables
[
https://issues.apache.org/jira/browse/ATLAS-642?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332967#comment-15332967
]
Suma Shivaprasad edited comment on ATLAS-642 at 6/16/16 3:06 AM:
-
[~svimal2106] there were some conflicts again. Rebased patch which I am
uploading. But there are some tests failing after rebasing. Can you pls check?
{noformat}
Running org.apache.atlas.hive.bridge.HiveLiteralRewriterTest
Tests run: 1, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 6.226 sec - in
org.apache.atlas.hive.bridge.HiveLiteralRewriterTest
Running org.apache.atlas.hive.bridge.HiveMetaStoreBridgeTest
Tests run: 3, Failures: 2, Errors: 0, Skipped: 0, Time elapsed: 5.466 sec <<<
FAILURE! - in org.apache.atlas.hive.bridge.HiveMetaStoreBridgeTest
testImportThatUpdatesRegisteredTable(org.apache.atlas.hive.bridge.HiveMetaStoreBridgeTest)
Time elapsed: 0.174 sec <<< FAILURE!
java.lang.NullPointerException: null
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getEntityReferenceFromDSL(HiveMetaStoreBridge.java:220)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getProcessReference(HiveMetaStoreBridge.java:325)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importTables(HiveMetaStoreBridge.java:270)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:125)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:117)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridgeTest.testImportThatUpdatesRegisteredTable(HiveMetaStoreBridgeTest.java:104)
testImportWhenPartitionKeysAreNull(org.apache.atlas.hive.bridge.HiveMetaStoreBridgeTest)
Time elapsed: 0.026 sec <<< FAILURE!
java.lang.AssertionError: Partition with null key caused import to fail with
exception
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getEntityReferenceFromDSL(HiveMetaStoreBridge.java:220)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.getProcessReference(HiveMetaStoreBridge.java:325)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importTables(HiveMetaStoreBridge.java:270)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importDatabases(HiveMetaStoreBridge.java:125)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridge.importHiveMetadata(HiveMetaStoreBridge.java:117)
at
org.apache.atlas.hive.bridge.HiveMetaStoreBridgeTest.testImportWhenPartitionKeysAreNull(HiveMetaStoreBridgeTest.java:154)
{noformat}
was (Author: suma.shivaprasad):
[~svimal2106] there were some conflicts again. Rebased patch which I am
uploading. But there are some tests failing after rebasing. Can you pls check?
> import-hive should create the lineage for external tables
> -
>
> Key: ATLAS-642
> URL: https://issues.apache.org/jira/browse/ATLAS-642
> Project: Atlas
> Issue Type: Sub-task
>Reporter: Suma Shivaprasad
>Assignee: Vimal Sharma
>Priority: Critical
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-642.patch, ATLAS-642.patch, ATLAS-642.patch
>
>
> import-hive can also start showing lineage between HDFS paths and tables.
> However the create table process should also register the exact query that
> was used to create the table. Show create table does that by reconstructing
> the query. Need to figure out if we can potentially use that to create the
> query
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ATLAS-688) import-hive should depend on Hive CLASSPATH jars instead of packaging everything
[ https://issues.apache.org/jira/browse/ATLAS-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332974#comment-15332974 ] Suma Shivaprasad commented on ATLAS-688: Some minor comments. 1. revert change for exclusion of http client -commons-httpclient -* - > import-hive should depend on Hive CLASSPATH jars instead of packaging > everything > > > Key: ATLAS-688 > URL: https://issues.apache.org/jira/browse/ATLAS-688 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad >Assignee: Vimal Sharma >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-688-v2.patch, ATLAS-688.patch > > > Currently, import-hive.sh packages all hive, hadoop jars etc. Instead it > should depend on HIVE_CLASSPATH from which it can get the jars and should be > paakged along with hook on hive server m/cs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-688) import-hive should depend on Hive CLASSPATH jars instead of packaging everything
[ https://issues.apache.org/jira/browse/ATLAS-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332974#comment-15332974 ] Suma Shivaprasad edited comment on ATLAS-688 at 6/16/16 3:09 AM: - Some minor comments. 1. revert change for exclusion of http client -commons-httpclient -* - 2. Revert BASEDIR changes also. dirName and referring to parent directory should work in this case as well, rt? was (Author: suma.shivaprasad): Some minor comments. 1. revert change for exclusion of http client -commons-httpclient -* - > import-hive should depend on Hive CLASSPATH jars instead of packaging > everything > > > Key: ATLAS-688 > URL: https://issues.apache.org/jira/browse/ATLAS-688 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad >Assignee: Vimal Sharma >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-688-v2.patch, ATLAS-688.patch > > > Currently, import-hive.sh packages all hive, hadoop jars etc. Instead it > should depend on HIVE_CLASSPATH from which it can get the jars and should be > paakged along with hook on hive server m/cs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-688) import-hive should depend on Hive CLASSPATH jars instead of packaging everything
[ https://issues.apache.org/jira/browse/ATLAS-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332974#comment-15332974 ] Suma Shivaprasad edited comment on ATLAS-688 at 6/16/16 3:13 AM: - Some minor comments. 1. revert change for exclusion of http client -commons-httpclient -* - 2. Revert BASEDIR changes also. dirName and referring to parent directory should work in this case as well, rt? 3. change error from Could not find a valid HADOOP configuration to "env HADOOP_CLASSPATH or HADOOP_HOME need to be set" was (Author: suma.shivaprasad): Some minor comments. 1. revert change for exclusion of http client -commons-httpclient -* - 2. Revert BASEDIR changes also. dirName and referring to parent directory should work in this case as well, rt? > import-hive should depend on Hive CLASSPATH jars instead of packaging > everything > > > Key: ATLAS-688 > URL: https://issues.apache.org/jira/browse/ATLAS-688 > Project: Atlas > Issue Type: Bug >Reporter: Suma Shivaprasad >Assignee: Vimal Sharma >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-688-v2.patch, ATLAS-688.patch > > > Currently, import-hive.sh packages all hive, hadoop jars etc. Instead it > should depend on HIVE_CLASSPATH from which it can get the jars and should be > paakged along with hook on hive server m/cs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-688) import-hive should depend on Hive CLASSPATH jars instead of packaging everything
[ https://issues.apache.org/jira/browse/ATLAS-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15332974#comment-15332974 ] Suma Shivaprasad edited comment on ATLAS-688 at 6/16/16 3:17 AM: - Some minor comments. 1. revert change for exclusion of http client -commons-httpclient -* - 2. Revert BASEDIR changes also. dirName and referring to parent directory should work in this case as well, rt? 3. change error from Could not find a valid HADOOP configuration to "env HADOOP_CLASSPATH or HADOOP_HOME need to be set" 4. Better to have CP as ATLAS_CP:HIVE_CP:HADOOP_CP - would allow us to override any jars if required. was (Author: suma.shivaprasad): Some minor comments. 1. revert change for exclusion of http client -commons-httpclient -* - 2. Revert BASEDIR changes also. dirName and referring to parent directory should work in this case as well, rt? 3. change error from Could not find a valid HADOOP configuration to "env HADOOP_CLASSPATH or HADOOP_HOME need to be set" > import-hive should depend on Hive CLASSPATH jars instead of packaging > everything > > > Key: ATLAS-688 > URL: https://issues.apache.org/jira/browse/ATLAS-688 > Project: Atlas > Issue Type: Bug >Reporter: Suma Shivaprasad >Assignee: Vimal Sharma >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-688-v2.patch, ATLAS-688.patch > > > Currently, import-hive.sh packages all hive, hadoop jars etc. Instead it > should depend on HIVE_CLASSPATH from which it can get the jars and should be > paakged along with hook on hive server m/cs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Resolved] (ATLAS-903) EntityNotFoundException exception is observed when an entity is deleted.
[
https://issues.apache.org/jira/browse/ATLAS-903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad resolved ATLAS-903.
Resolution: Won't Fix
> EntityNotFoundException exception is observed when an entity is deleted.
>
>
> Key: ATLAS-903
> URL: https://issues.apache.org/jira/browse/ATLAS-903
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Shwetha G S
>Priority: Blocker
>
> Create hive table and drop hive table.
> {noformat}
> 2016-05-05 19:21:04,095 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
> ATLAS_ENTITIES:
> {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"2fdeeeb5-ad02-4746-afd4-8c478e9825c6","version":0,"typeName":"hive_table","state":"ACTIVE"},"typeName":"hive_table","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
> (KafkaNotification:192)
> 2016-05-05 19:21:04,096 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
> ATLAS_ENTITIES:
> {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"efe89140-3635-4779-a495-1d8371f8f939","version":0,"typeName":"hive_storagedesc","state":"ACTIVE"},"typeName":"hive_storagedesc","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
> (KafkaNotification:192)
> 2016-05-05 19:21:04,096 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sending message for topic
> ATLAS_ENTITIES:
> {"entity":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Reference","id":{"jsonClass":"org.apache.atlas.typesystem.json.InstanceSerialization$_Id","id":"57fd8ebc-e954-4669-bc47-5f617f8bc2fb","version":0,"typeName":"hive_column","state":"ACTIVE"},"typeName":"hive_column","values":{},"traitNames":[],"traits":{}},"operationType":"ENTITY_DELETE","traits":[]}
> (KafkaNotification:192)
> 2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
> ATLAS_ENTITIES, partition - 0, offset - 92 (KafkaNotification:199)
> 2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
> ATLAS_ENTITIES, partition - 0, offset - 93 (KafkaNotification:199)
> 2016-05-05 19:21:04,098 DEBUG - [qtp1007251739-16 -
> bef53ab8-0078-425c-a2e2-48baa08dae2f:] ~ Sent message for topic -
> ATLAS_ENTITIES, partition - 0, offset - 94 (KafkaNotification:199)
> 2016-05-05 19:21:04,099 DEBUG - [pool-9-thread-1:] ~ API
> http://localhost:21000/api/atlas/entities?type=hive_table&property=name&value=default.t3@cl1
> returned status 200 (AtlasClient:976)
> 2016-05-05 19:21:04,100 DEBUG - [pool-9-thread-1:] ~ Read message: conumerId:
> 0, topic - ATLAS_HOOK, partition - 0, offset - 33, message -
> {"typeName":"hive_table","attribute":"name","attributeValue":"default.t3@cl1","type":"ENTITY_DELETE"}
> (KafkaConsumer:68)
> 2016-05-05 19:21:04,108 INFO - [pool-9-thread-1:] ~ Real User: UNKNOWN
> (auth:SIMPLE), is from ticket cache? false (SecureClientUtils:90)
> 2016-05-05 19:21:04,108 INFO - [pool-9-thread-1:] ~ doAsUser: UNKNOWN
> (SecureClientUtils:93)
> 2016-05-05 19:21:04,142 INFO - [pool-9-thread-1:] ~ Client has only one
> service URL, will use that for all actions: http://localhost:21000
> (AtlasClient:204)
> 2016-05-05 19:21:04,143 DEBUG - [qtp1007251739-17 -
> 4b58a3a8-b1e2-4c7d-a830-1ba4251ca1f3:] ~ Audit: UNKNOWN/127.0.0.1 performed
> request OPTIONS
> http://localhost:21000/api/atlas/entities?type=hive_table&property=name&value=
[jira] [Commented] (ATLAS-912) Update to use Kafka 0.10.0.0 (from 0.9.0.0)
[ https://issues.apache.org/jira/browse/ATLAS-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15334840#comment-15334840 ] Suma Shivaprasad commented on ATLAS-912: +1 > Update to use Kafka 0.10.0.0 (from 0.9.0.0) > --- > > Key: ATLAS-912 > URL: https://issues.apache.org/jira/browse/ATLAS-912 > Project: Atlas > Issue Type: Task >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj > Attachments: ATLAS-912.patch > > > Atlas currently uses Kafka version 0.9.0.0. A newer version of Kafka, > 0.10.0.0, is available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-912) Update to use Kafka 0.10.0.0 (from 0.9.0.0)
[
https://issues.apache.org/jira/browse/ATLAS-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15334906#comment-15334906
]
Suma Shivaprasad commented on ATLAS-912:
Seeing an OOM error in StormHook while running tests
[WARNING] FAILED
org.eclipse.jetty.server.handler.ContextHandlerCollection@14c8d0b0[o.e.j.m.p.JettyWebAppContext@32bf5cde{/,file:/Users/sshivaprasad/workspace-latest/atlas-commit2/webapp/target/atlas-webapp-0.7-incubating-SNAPSHOT/,STARTING}{/Users/sshivaprasad/workspace-latest/atlas-commit2/webapp/target/atlas-webapp-0.7-incubating-SNAPSHOT.war}]:
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
at kafka.log.SkimpyOffsetMap.(OffsetMap.scala:43)
at kafka.log.LogCleaner$CleanerThread.(LogCleaner.scala:193)
at kafka.log.LogCleaner$$anonfun$1.apply(LogCleaner.scala:83)
at kafka.log.LogCleaner$$anonfun$1.apply(LogCleaner.scala:83)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.Range.foreach(Range.scala:141)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at kafka.log.LogCleaner.(LogCleaner.scala:83)
at kafka.log.LogManager.(LogManager.scala:66)
at kafka.server.KafkaServer.createLogManager(KafkaServer.scala:609)
at kafka.server.KafkaServer.startup(KafkaServer.scala:183)
at
org.apache.atlas.kafka.KafkaNotification.startKafka(KafkaNotification.java:335)
at
org.apache.atlas.kafka.KafkaNotification.start(KafkaNotification.java:140)
at org.apache.atlas.service.Services.start(Services.java:45)
at
org.apache.atlas.web.listeners.GuiceServletConfig.startServices(GuiceServletConfig.java:142)
at
org.apache.atlas.web.listeners.TestGuiceServletConfig.startServices(TestGuiceServletConfig.java:74)
at
org.apache.atlas.web.listeners.GuiceServletConfig.contextInitialized(GuiceServletConfig.java:136)
at
org.apache.atlas.web.listeners.TestGuiceServletConfig.contextInitialized(TestGuiceServletConfig.java:42)
at
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)
at
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)
at
org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)
at
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)
at
org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
at
org.eclipse.jetty.maven.plugin.JettyWebAppContext.startWebapp(JettyWebAppContext.java:296)
at
org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
at
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
at
org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
at
org.eclipse.jetty.maven.plugin.JettyWebAppContext.doStart(JettyWebAppContext.java:365)
> Update to use Kafka 0.10.0.0 (from 0.9.0.0)
> ---
>
> Key: ATLAS-912
> URL: https://issues.apache.org/jira/browse/ATLAS-912
> Project: Atlas
> Issue Type: Task
>Reporter: Madhan Neethiraj
>Assignee: Madhan Neethiraj
> Attachments: ATLAS-912.patch
>
>
> Atlas currently uses Kafka version 0.9.0.0. A newer version of Kafka,
> 0.10.0.0, is available.
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-912) Update to use Kafka 0.10.0.0 (from 0.9.0.0)
[
https://issues.apache.org/jira/browse/ATLAS-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15334906#comment-15334906
]
Suma Shivaprasad edited comment on ATLAS-912 at 6/16/16 11:11 PM:
--
Seeing an OOM error in StormHook while running tests with 1G heap size with the
new Kafka version
[WARNING] FAILED
org.eclipse.jetty.server.handler.ContextHandlerCollection@14c8d0b0[o.e.j.m.p.JettyWebAppContext@32bf5cde{/,file:/Users/sshivaprasad/workspace-latest/atlas-commit2/webapp/target/atlas-webapp-0.7-incubating-SNAPSHOT/,STARTING}{/Users/sshivaprasad/workspace-latest/atlas-commit2/webapp/target/atlas-webapp-0.7-incubating-SNAPSHOT.war}]:
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
at kafka.log.SkimpyOffsetMap.(OffsetMap.scala:43)
at kafka.log.LogCleaner$CleanerThread.(LogCleaner.scala:193)
at kafka.log.LogCleaner$$anonfun$1.apply(LogCleaner.scala:83)
at kafka.log.LogCleaner$$anonfun$1.apply(LogCleaner.scala:83)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.Range.foreach(Range.scala:141)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at kafka.log.LogCleaner.(LogCleaner.scala:83)
at kafka.log.LogManager.(LogManager.scala:66)
at kafka.server.KafkaServer.createLogManager(KafkaServer.scala:609)
at kafka.server.KafkaServer.startup(KafkaServer.scala:183)
at
org.apache.atlas.kafka.KafkaNotification.startKafka(KafkaNotification.java:335)
at
org.apache.atlas.kafka.KafkaNotification.start(KafkaNotification.java:140)
at org.apache.atlas.service.Services.start(Services.java:45)
at
org.apache.atlas.web.listeners.GuiceServletConfig.startServices(GuiceServletConfig.java:142)
at
org.apache.atlas.web.listeners.TestGuiceServletConfig.startServices(TestGuiceServletConfig.java:74)
at
org.apache.atlas.web.listeners.GuiceServletConfig.contextInitialized(GuiceServletConfig.java:136)
at
org.apache.atlas.web.listeners.TestGuiceServletConfig.contextInitialized(TestGuiceServletConfig.java:42)
at
org.eclipse.jetty.server.handler.ContextHandler.callContextInitialized(ContextHandler.java:800)
at
org.eclipse.jetty.servlet.ServletContextHandler.callContextInitialized(ServletContextHandler.java:444)
at
org.eclipse.jetty.server.handler.ContextHandler.startContext(ContextHandler.java:791)
at
org.eclipse.jetty.servlet.ServletContextHandler.startContext(ServletContextHandler.java:294)
at
org.eclipse.jetty.webapp.WebAppContext.startWebapp(WebAppContext.java:1349)
at
org.eclipse.jetty.maven.plugin.JettyWebAppContext.startWebapp(JettyWebAppContext.java:296)
at
org.eclipse.jetty.webapp.WebAppContext.startContext(WebAppContext.java:1342)
at
org.eclipse.jetty.server.handler.ContextHandler.doStart(ContextHandler.java:741)
at
org.eclipse.jetty.webapp.WebAppContext.doStart(WebAppContext.java:505)
at
org.eclipse.jetty.maven.plugin.JettyWebAppContext.doStart(JettyWebAppContext.java:365)
was (Author: suma.shivaprasad):
Seeing an OOM error in StormHook while running tests
[WARNING] FAILED
org.eclipse.jetty.server.handler.ContextHandlerCollection@14c8d0b0[o.e.j.m.p.JettyWebAppContext@32bf5cde{/,file:/Users/sshivaprasad/workspace-latest/atlas-commit2/webapp/target/atlas-webapp-0.7-incubating-SNAPSHOT/,STARTING}{/Users/sshivaprasad/workspace-latest/atlas-commit2/webapp/target/atlas-webapp-0.7-incubating-SNAPSHOT.war}]:
java.lang.OutOfMemoryError: Java heap space
java.lang.OutOfMemoryError: Java heap space
at java.nio.HeapByteBuffer.(HeapByteBuffer.java:57)
at java.nio.ByteBuffer.allocate(ByteBuffer.java:331)
at kafka.log.SkimpyOffsetMap.(OffsetMap.scala:43)
at kafka.log.LogCleaner$CleanerThread.(LogCleaner.scala:193)
at kafka.log.LogCleaner$$anonfun$1.apply(LogCleaner.scala:83)
at kafka.log.LogCleaner$$anonfun$1.apply(LogCleaner.scala:83)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at
scala.collection.TraversableLike$$anonfun$map$1.apply(TraversableLike.scala:244)
at scala.collection.immutable.Range.foreach(Range.scala:141)
at scala.collection.TraversableLike$class.map(TraversableLike.scala:244)
at scala.collection.AbstractTraversable.map(Traversable.scala:105)
at kafka.log.LogCleaner.(LogCleaner.scala:83)
at kafka.log.LogMan
[jira] [Commented] (ATLAS-912) Update to use Kafka 0.10.0.0 (from 0.9.0.0)
[ https://issues.apache.org/jira/browse/ATLAS-912?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15335071#comment-15335071 ] Suma Shivaprasad commented on ATLAS-912: -Xmx=1536 works > Update to use Kafka 0.10.0.0 (from 0.9.0.0) > --- > > Key: ATLAS-912 > URL: https://issues.apache.org/jira/browse/ATLAS-912 > Project: Atlas > Issue Type: Task >Reporter: Madhan Neethiraj >Assignee: Madhan Neethiraj > Attachments: ATLAS-912.patch > > > Atlas currently uses Kafka version 0.9.0.0. A newer version of Kafka, > 0.10.0.0, is available. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-877) CreateTime of an entity(for example hive table) changes for every update
[
https://issues.apache.org/jira/browse/ATLAS-877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-877:
---
Attachment: ATLAS-877.patch
> CreateTime of an entity(for example hive table) changes for every update
>
>
> Key: ATLAS-877
> URL: https://issues.apache.org/jira/browse/ATLAS-877
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
>Reporter: Ayub Khan
>Assignee: Suma Shivaprasad
>Priority: Critical
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-877.patch
>
>
> Each time a hive table is updated(example column renamed), create time of
> table is also updated.
> UI snapshot: https://monosnap.com/file/6NoT86CNpHgQmlyfBQTSUa1y3Ncc2Y
> hive table metadata snapshot from hive cli
> {noformat}
> 0: jdbc:hive2://localhost:1/default> describe formatted
> column_changepgwcbgq1bd;
> OK
> +---+-+---+--+
> | col_name|
> data_type |comment|
> +---+-+---+--+
> | # col_name| data_type
> | comment |
> | | NULL
> | NULL |
> | a3| int
> | |
> | b | int
> | |
> | c | int
> | |
> | d | int
> | |
> | e | int
> | |
> | f | int
> | |
> | | NULL
> | NULL |
> | # Detailed Table Information | NULL
> | NULL |
> | Database: | db2pgwcbgq1bd
> | NULL |
> | Owner:| apathan
> | NULL |
> | CreateTime: | Wed Jun 08 12:41:28 IST 2016
> | NULL |
> | LastAccessTime: | UNKNOWN
> | NULL |
> | Protect Mode: | None
> | NULL |
> | Retention:| 0
> | NULL |
> | Location: |
> hdfs://localhost:9000/user/hive/warehouse/db2pgwcbgq1bd.db/column_changepgwcbgq1bd
> | NULL |
> | Table Type: | MANAGED_TABLE
> | NULL |
> | Table Parameters: | NULL
> | NULL |
> | | COLUMN_STATS_ACCURATE
> | false |
> | | last_modified_by
> | apathan |
> | | last_modified_time
[jira] [Commented] (ATLAS-688) import-hive should depend on Hive CLASSPATH jars instead of packaging everything
[ https://issues.apache.org/jira/browse/ATLAS-688?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15336604#comment-15336604 ] Suma Shivaprasad commented on ATLAS-688: [~svimal2106] The patch LGTM. Will commit. > import-hive should depend on Hive CLASSPATH jars instead of packaging > everything > > > Key: ATLAS-688 > URL: https://issues.apache.org/jira/browse/ATLAS-688 > Project: Atlas > Issue Type: Bug > Reporter: Suma Shivaprasad >Assignee: Vimal Sharma >Priority: Critical > Fix For: 0.7-incubating > > Attachments: ATLAS-688-v2.patch, ATLAS-688-v3.patch, ATLAS-688.patch > > > Currently, import-hive.sh packages all hive, hadoop jars etc. Instead it > should depend on HIVE_CLASSPATH from which it can get the jars and should be > paakged along with hook on hive server m/cs -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-915) Fix docs for import-hive changes
Suma Shivaprasad created ATLAS-915: -- Summary: Fix docs for import-hive changes Key: ATLAS-915 URL: https://issues.apache.org/jira/browse/ATLAS-915 Project: Atlas Issue Type: Bug Affects Versions: 0.7-incubating Reporter: Suma Shivaprasad Assignee: Vimal Sharma Fix For: 0.7-incubating Pls fix According to changes in ATLAS-688 -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-904) Hive hook fails due to session state not being set
[
https://issues.apache.org/jira/browse/ATLAS-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-904:
---
Attachment: ATLAS-904.2.patch
Changes to address [~yhemanth] review comments.
1. Process qualified name = HiveOperation.name + sorted inputs + sorted outputs
2. HiveOperation.name doesnt provide identifiers for identiifying INSERT,
INSERT_OVERWRITE, UPDATE, DELETE etc separately . Hence adding
WriteEntity.WriteType as well which exhibits the following behaviour
a. If there are multiple outputs, for each output, adds the query
type(WriteType)
b. if query being run if is type INSERT [into/overwrite] TABLE [PARTITION],
WriteType is INSERT/INSERT_OVERWRITE
b. If query is of type INSERT OVERWRITE hdfs_path, adds WriteType as PATH_WRITE
c. If query is of type UPDATE/DELETE, adds type as UPDATE/DELETE [ Note -
linage is not available for this since this is single table operation]
> Hive hook fails due to session state not being set
> --
>
> Key: ATLAS-904
> URL: https://issues.apache.org/jira/browse/ATLAS-904
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
> Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-904.1.patch, ATLAS-904.2.patch, ATLAS-904.patch
>
>
> {noformat}
> 2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:normalize(557)) - Could not rewrite query due to error.
> Proceeding with original query EXPORT TABLE test_export_table to
> 'hdfs://localhost:9000/hive_tables/test_path1'
> java.lang.NullPointerException: Conf non-local session path expected to be
> non-null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
> at org.apache.hadoop.hive.ql.Context.(Context.java:133)
> at org.apache.hadoop.hive.ql.Context.(Context.java:120)
> at
> org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
> at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-15 11:34:30,423 ERROR [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:run(184)) - Atlas hook failed due to error
> java.lang.NullPointerException
> at java.lang.StringBuilder.(StringBuilder.java:109)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessQualifiedName(HiveHook.java:738)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:703)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-904) Hive hook fails due to session state not being set
[
https://issues.apache.org/jira/browse/ATLAS-904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15338949#comment-15338949
]
Suma Shivaprasad edited comment on ATLAS-904 at 6/20/16 3:49 AM:
-
Changes to address [~yhemanth] review comments.
1. Process qualified name = HiveOperation.name + sorted inputs + sorted outputs
2. HiveOperation.name doesnt provide identifiers for identiifying INSERT,
INSERT_OVERWRITE, UPDATE, DELETE etc separately . Hence adding
WriteEntity.WriteType as well which exhibits the following behaviour
a. If there are multiple outputs, for each output, adds the query
type(WriteType)
b. if query being run if is type INSERT [into/overwrite] TABLE [PARTITION],
WriteType is INSERT/INSERT_OVERWRITE
b. If query is of type INSERT OVERWRITE hdfs_path, adds WriteType as PATH_WRITE
c. If query is of type UPDATE/DELETE, adds type as UPDATE/DELETE [ Note -
linage is not available for this since this is single table operation]
3.When input is of type local dir or hdfs path currently, it doesnt add it to
qualified name. The reason is that partition based paths cause a lot of
processes to be created in this case instead of updating the same process.
Pending:
Address [~shwethags] suggestion to add hdfs paths to process qualified name
only in case of non-partition based queries. This needs to be done per
HiveOperation type
1. if HiveOperation = LOAD, IMPORT, EXPORT - detect if the current query
context is dealing with partitions and do not add if it is partition based.
2. If HiveOperation = INSERT OVERWRITE DFS_PATH/LOCAL_PATH , then detect if the
query context is dealing with a partitioned table in inputs and decide if we
need to add or not.
was (Author: suma.shivaprasad):
Changes to address [~yhemanth] review comments.
1. Process qualified name = HiveOperation.name + sorted inputs + sorted outputs
2. HiveOperation.name doesnt provide identifiers for identiifying INSERT,
INSERT_OVERWRITE, UPDATE, DELETE etc separately . Hence adding
WriteEntity.WriteType as well which exhibits the following behaviour
a. If there are multiple outputs, for each output, adds the query
type(WriteType)
b. if query being run if is type INSERT [into/overwrite] TABLE [PARTITION],
WriteType is INSERT/INSERT_OVERWRITE
b. If query is of type INSERT OVERWRITE hdfs_path, adds WriteType as PATH_WRITE
c. If query is of type UPDATE/DELETE, adds type as UPDATE/DELETE [ Note -
linage is not available for this since this is single table operation]
> Hive hook fails due to session state not being set
> --
>
> Key: ATLAS-904
> URL: https://issues.apache.org/jira/browse/ATLAS-904
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
> Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-904.1.patch, ATLAS-904.2.patch, ATLAS-904.patch
>
>
> {noformat}
> 2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:normalize(557)) - Could not rewrite query due to error.
> Proceeding with original query EXPORT TABLE test_export_table to
> 'hdfs://localhost:9000/hive_tables/test_path1'
> java.lang.NullPointerException: Conf non-local session path expected to be
> non-null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
> at org.apache.hadoop.hive.ql.Context.(Context.java:133)
> at org.apache.hadoop.hive.ql.Context.(Context.java:120)
> at
> org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
> at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-15 11:34:30,423 ERROR [A
[jira] [Commented] (ATLAS-904) Hive hook fails due to session state not being set
[
https://issues.apache.org/jira/browse/ATLAS-904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15338961#comment-15338961
]
Suma Shivaprasad commented on ATLAS-904:
https://reviews.apache.org/r/48939
> Hive hook fails due to session state not being set
> --
>
> Key: ATLAS-904
> URL: https://issues.apache.org/jira/browse/ATLAS-904
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
> Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-904.1.patch, ATLAS-904.2.patch, ATLAS-904.patch
>
>
> {noformat}
> 2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:normalize(557)) - Could not rewrite query due to error.
> Proceeding with original query EXPORT TABLE test_export_table to
> 'hdfs://localhost:9000/hive_tables/test_path1'
> java.lang.NullPointerException: Conf non-local session path expected to be
> non-null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
> at org.apache.hadoop.hive.ql.Context.(Context.java:133)
> at org.apache.hadoop.hive.ql.Context.(Context.java:120)
> at
> org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
> at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-15 11:34:30,423 ERROR [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:run(184)) - Atlas hook failed due to error
> java.lang.NullPointerException
> at java.lang.StringBuilder.(StringBuilder.java:109)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessQualifiedName(HiveHook.java:738)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:703)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-904) Hive hook fails due to session state not being set
[
https://issues.apache.org/jira/browse/ATLAS-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-904:
---
Attachment: (was: ATLAS-904.2.patch)
> Hive hook fails due to session state not being set
> --
>
> Key: ATLAS-904
> URL: https://issues.apache.org/jira/browse/ATLAS-904
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
> Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-904.1.patch, ATLAS-904.patch
>
>
> {noformat}
> 2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:normalize(557)) - Could not rewrite query due to error.
> Proceeding with original query EXPORT TABLE test_export_table to
> 'hdfs://localhost:9000/hive_tables/test_path1'
> java.lang.NullPointerException: Conf non-local session path expected to be
> non-null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
> at org.apache.hadoop.hive.ql.Context.(Context.java:133)
> at org.apache.hadoop.hive.ql.Context.(Context.java:120)
> at
> org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
> at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-15 11:34:30,423 ERROR [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:run(184)) - Atlas hook failed due to error
> java.lang.NullPointerException
> at java.lang.StringBuilder.(StringBuilder.java:109)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessQualifiedName(HiveHook.java:738)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:703)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-904) Hive hook fails due to session state not being set
[
https://issues.apache.org/jira/browse/ATLAS-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-904:
---
Attachment: ATLAS-904.2.patch
> Hive hook fails due to session state not being set
> --
>
> Key: ATLAS-904
> URL: https://issues.apache.org/jira/browse/ATLAS-904
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
> Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-904.1.patch, ATLAS-904.2.patch, ATLAS-904.patch
>
>
> {noformat}
> 2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:normalize(557)) - Could not rewrite query due to error.
> Proceeding with original query EXPORT TABLE test_export_table to
> 'hdfs://localhost:9000/hive_tables/test_path1'
> java.lang.NullPointerException: Conf non-local session path expected to be
> non-null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
> at org.apache.hadoop.hive.ql.Context.(Context.java:133)
> at org.apache.hadoop.hive.ql.Context.(Context.java:120)
> at
> org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
> at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-15 11:34:30,423 ERROR [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:run(184)) - Atlas hook failed due to error
> java.lang.NullPointerException
> at java.lang.StringBuilder.(StringBuilder.java:109)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessQualifiedName(HiveHook.java:738)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:703)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Updated] (ATLAS-492) Hive Hook Improvements
[ https://issues.apache.org/jira/browse/ATLAS-492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-492: --- Fix Version/s: (was: 0.7-incubating) > Hive Hook Improvements > -- > > Key: ATLAS-492 > URL: https://issues.apache.org/jira/browse/ATLAS-492 > Project: Atlas > Issue Type: New Feature >Affects Versions: 0.7-incubating >Reporter: Erik Bergenholtz >Assignee: Suma Shivaprasad > > h2. Requirements: > * Support ALTER statements (add/replace/rename cols) > * Support DROP statements ( support drop database, table, view) > * Support LOAD statements > * Remodelling changes -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-625) GraphBackedDiscoveryServiceTest throws an exception in shutdown.
[ https://issues.apache.org/jira/browse/ATLAS-625?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-625: --- Fix Version/s: (was: 0.7-incubating) > GraphBackedDiscoveryServiceTest throws an exception in shutdown. > > > Key: ATLAS-625 > URL: https://issues.apache.org/jira/browse/ATLAS-625 > Project: Atlas > Issue Type: Bug >Reporter: Hemanth Yamijala > > The tests don't fail because this is guarded by a try-catch block, but > ideally we should check why we are getting this error and make sure the right > fix is done. This was pointed out by [~nixonrodrigues] initially. > Filing this here just so we don't lose track. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-647) entityText property should be prefixed with __
[ https://issues.apache.org/jira/browse/ATLAS-647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-647: --- Fix Version/s: (was: 0.7-incubating) > entityText property should be prefixed with __ > -- > > Key: ATLAS-647 > URL: https://issues.apache.org/jira/browse/ATLAS-647 > Project: Atlas > Issue Type: Bug >Reporter: Shwetha G S >Assignee: Shwetha G S > > All internal properties for the vertex in the repository are prefixed with __ > to avoid conflicts with user defined attribute names. Looks like entityText > is missed out -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-904) Hive hook fails due to session state not being set
[
https://issues.apache.org/jira/browse/ATLAS-904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Suma Shivaprasad updated ATLAS-904:
---
Attachment: ATLAS-904.3.patch
Addressed review comments and rebased
> Hive hook fails due to session state not being set
> --
>
> Key: ATLAS-904
> URL: https://issues.apache.org/jira/browse/ATLAS-904
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
> Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-904.1.patch, ATLAS-904.2.patch, ATLAS-904.3.patch,
> ATLAS-904.patch
>
>
> {noformat}
> 2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:normalize(557)) - Could not rewrite query due to error.
> Proceeding with original query EXPORT TABLE test_export_table to
> 'hdfs://localhost:9000/hive_tables/test_path1'
> java.lang.NullPointerException: Conf non-local session path expected to be
> non-null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
> at org.apache.hadoop.hive.ql.Context.(Context.java:133)
> at org.apache.hadoop.hive.ql.Context.(Context.java:120)
> at
> org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
> at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-15 11:34:30,423 ERROR [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:run(184)) - Atlas hook failed due to error
> java.lang.NullPointerException
> at java.lang.StringBuilder.(StringBuilder.java:109)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessQualifiedName(HiveHook.java:738)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:703)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-904) Hive hook fails due to session state not being set
[
https://issues.apache.org/jira/browse/ATLAS-904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340107#comment-15340107
]
Suma Shivaprasad edited comment on ATLAS-904 at 6/20/16 6:23 PM:
-
Addressed review comments and rebased. Will address the partition based checks
and handling in a separate jira
was (Author: suma.shivaprasad):
Addressed review comments and rebased
> Hive hook fails due to session state not being set
> --
>
> Key: ATLAS-904
> URL: https://issues.apache.org/jira/browse/ATLAS-904
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
> Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-904.1.patch, ATLAS-904.2.patch, ATLAS-904.3.patch,
> ATLAS-904.patch
>
>
> {noformat}
> 2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:normalize(557)) - Could not rewrite query due to error.
> Proceeding with original query EXPORT TABLE test_export_table to
> 'hdfs://localhost:9000/hive_tables/test_path1'
> java.lang.NullPointerException: Conf non-local session path expected to be
> non-null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
> at org.apache.hadoop.hive.ql.Context.(Context.java:133)
> at org.apache.hadoop.hive.ql.Context.(Context.java:120)
> at
> org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
> at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-15 11:34:30,423 ERROR [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:run(184)) - Atlas hook failed due to error
> java.lang.NullPointerException
> at java.lang.StringBuilder.(StringBuilder.java:109)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessQualifiedName(HiveHook.java:738)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:703)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Created] (ATLAS-917) Add hdfs paths to process qualified name for non-partition based queries
Suma Shivaprasad created ATLAS-917: -- Summary: Add hdfs paths to process qualified name for non-partition based queries Key: ATLAS-917 URL: https://issues.apache.org/jira/browse/ATLAS-917 Project: Atlas Issue Type: Bug Affects Versions: 0.7-incubating Reporter: Suma Shivaprasad Assignee: Suma Shivaprasad Add hdfs paths to process qualified name only in case of non-partition based queries. This needs to be done per HiveOperation type 1. if HiveOperation = LOAD, IMPORT, EXPORT - detect if the current query context is dealing with partitions and do not add if it is partition based. 2. If HiveOperation = INSERT OVERWRITE DFS_PATH/LOCAL_PATH , then detect if the query context is dealing with a partitioned table in inputs and decide if we need to add or not. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-904) Hive hook fails due to session state not being set
[
https://issues.apache.org/jira/browse/ATLAS-904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340107#comment-15340107
]
Suma Shivaprasad edited comment on ATLAS-904 at 6/20/16 6:25 PM:
-
Addressed review comments and rebased. Will address the partition based checks
and handling in a separate jira - ATLAS-917
was (Author: suma.shivaprasad):
Addressed review comments and rebased. Will address the partition based checks
and handling in a separate jira
> Hive hook fails due to session state not being set
> --
>
> Key: ATLAS-904
> URL: https://issues.apache.org/jira/browse/ATLAS-904
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 0.7-incubating
> Reporter: Suma Shivaprasad
>Assignee: Suma Shivaprasad
>Priority: Blocker
> Fix For: 0.7-incubating
>
> Attachments: ATLAS-904.1.patch, ATLAS-904.2.patch, ATLAS-904.3.patch,
> ATLAS-904.patch
>
>
> {noformat}
> 2016-06-15 11:34:30,423 WARN [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:normalize(557)) - Could not rewrite query due to error.
> Proceeding with original query EXPORT TABLE test_export_table to
> 'hdfs://localhost:9000/hive_tables/test_path1'
> java.lang.NullPointerException: Conf non-local session path expected to be
> non-null
> at
> com.google.common.base.Preconditions.checkNotNull(Preconditions.java:204)
> at
> org.apache.hadoop.hive.ql.session.SessionState.getHDFSSessionPath(SessionState.java:641)
> at org.apache.hadoop.hive.ql.Context.(Context.java:133)
> at org.apache.hadoop.hive.ql.Context.(Context.java:120)
> at
> org.apache.atlas.hive.rewrite.HiveASTRewriter.(HiveASTRewriter.java:44)
> at org.apache.atlas.hive.hook.HiveHook.normalize(HiveHook.java:554)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:702)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> 2016-06-15 11:34:30,423 ERROR [Atlas Logger 0]: hook.HiveHook
> (HiveHook.java:run(184)) - Atlas hook failed due to error
> java.lang.NullPointerException
> at java.lang.StringBuilder.(StringBuilder.java:109)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessQualifiedName(HiveHook.java:738)
> at
> org.apache.atlas.hive.hook.HiveHook.getProcessReferenceable(HiveHook.java:703)
> at
> org.apache.atlas.hive.hook.HiveHook.registerProcess(HiveHook.java:596)
> at org.apache.atlas.hive.hook.HiveHook.fireAndForget(HiveHook.java:222)
> at org.apache.atlas.hive.hook.HiveHook.access$200(HiveHook.java:77)
> at org.apache.atlas.hive.hook.HiveHook$2.run(HiveHook.java:182)
> at
> java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:471)
> at java.util.concurrent.FutureTask.run(FutureTask.java:262)
> at
> java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
> at
> java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
> at java.lang.Thread.run(Thread.java:745)
> {noformat}
--
This message was sent by Atlassian JIRA
(v6.3.4#6332)
[jira] [Commented] (ATLAS-698) Remove Rexster Graph and Gremlin REST API
[ https://issues.apache.org/jira/browse/ATLAS-698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340229#comment-15340229 ] Suma Shivaprasad commented on ATLAS-698: [~shwethags] For gremlin queries, was thinking it would be useful for querying in debugging if there are issues/bugs with DSL queries . Could we mark it InterfaceAudience.Private instead of removing? > Remove Rexster Graph and Gremlin REST API > - > > Key: ATLAS-698 > URL: https://issues.apache.org/jira/browse/ATLAS-698 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Erik Bergenholtz >Assignee: Vimal Sharma > Fix For: 0.7-incubating > > Attachments: ATLAS-698.patch > > > Since these expose the internals of Atlas and are not intended for public > consumption, these should be removed: > api/atlas/graph > api/atlas/discovery/search/gremlin -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-698) Remove Rexster Graph and Gremlin REST API
[ https://issues.apache.org/jira/browse/ATLAS-698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340229#comment-15340229 ] Suma Shivaprasad edited comment on ATLAS-698 at 6/20/16 7:21 PM: - [~shwethags] For gremlin queries, was thinking it would be useful for querying in debugging if there are issues/bugs with DSL queries . Could we mark it InterfaceAudience.Private instead of removing and just remove them from docs? +1 for Rexster API change was (Author: suma.shivaprasad): [~shwethags] For gremlin queries, was thinking it would be useful for querying in debugging if there are issues/bugs with DSL queries . Could we mark it InterfaceAudience.Private instead of removing? > Remove Rexster Graph and Gremlin REST API > - > > Key: ATLAS-698 > URL: https://issues.apache.org/jira/browse/ATLAS-698 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Erik Bergenholtz >Assignee: Vimal Sharma > Fix For: 0.7-incubating > > Attachments: ATLAS-698.patch > > > Since these expose the internals of Atlas and are not intended for public > consumption, these should be removed: > api/atlas/graph > api/atlas/discovery/search/gremlin -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-698) Remove Rexster Graph and Gremlin REST API
[ https://issues.apache.org/jira/browse/ATLAS-698?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340229#comment-15340229 ] Suma Shivaprasad edited comment on ATLAS-698 at 6/20/16 7:22 PM: - [~shwethags] For gremlin queries, was thinking it would be useful for querying / debugging if there are issues/bugs with DSL queries . Could we mark it InterfaceAudience.Private instead of removing and just remove them from docs? +1 for Rexster API change was (Author: suma.shivaprasad): [~shwethags] For gremlin queries, was thinking it would be useful for querying in debugging if there are issues/bugs with DSL queries . Could we mark it InterfaceAudience.Private instead of removing and just remove them from docs? +1 for Rexster API change > Remove Rexster Graph and Gremlin REST API > - > > Key: ATLAS-698 > URL: https://issues.apache.org/jira/browse/ATLAS-698 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Erik Bergenholtz >Assignee: Vimal Sharma > Fix For: 0.7-incubating > > Attachments: ATLAS-698.patch > > > Since these expose the internals of Atlas and are not intended for public > consumption, these should be removed: > api/atlas/graph > api/atlas/discovery/search/gremlin -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-844) Remove titan berkeley and elastic search jars if hbase/solr based profiles are chosen
[ https://issues.apache.org/jira/browse/ATLAS-844?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340273#comment-15340273 ] Suma Shivaprasad commented on ATLAS-844: +1 > Remove titan berkeley and elastic search jars if hbase/solr based profiles > are chosen > - > > Key: ATLAS-844 > URL: https://issues.apache.org/jira/browse/ATLAS-844 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Hemanth Yamijala >Assignee: Hemanth Yamijala > Fix For: 0.7-incubating > > Attachments: ATLAS-844.patch > > > With ATLAS-833, users of Atlas now have the option of using an external > HBase/Solr installation, an self-contained HBase/Solr installation (embedded > mode) or BerkeleyDB/Elastic Search installation. > When choosing either of the first two modes, we can potentially remove the > Titan berkeley DB or elastic search jars. This helps distributions which have > restrictions on using these jars from a contractual perspective. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-909) Lineage view is broken when there are lots of queries /processes between two datasets
[ https://issues.apache.org/jira/browse/ATLAS-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-909: --- Assignee: Andrew Ahn (was: Keval Bhatt) > Lineage view is broken when there are lots of queries /processes between two > datasets > - > > Key: ATLAS-909 > URL: https://issues.apache.org/jira/browse/ATLAS-909 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Andrew Ahn >Priority: Minor > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-06-15 at 6.03.01 PM.png > > > LIneage view does not have a scroll option and its height/width seem to be > limited which was not the case with the older dashboard. This is seen easily > when there are around more than 15-20 queries between two tables. Uploading > screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Commented] (ATLAS-909) Lineage view is broken when there are lots of queries /processes between two datasets
[ https://issues.apache.org/jira/browse/ATLAS-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340308#comment-15340308 ] Suma Shivaprasad commented on ATLAS-909: [~kevalbhatt18] I tested this again and I observed that I am able to move up/down in the window and resize if required to get to the required table. Hence downgrading to Minor. Assigning to [~aahn] to decide on UX changes needed if any > Lineage view is broken when there are lots of queries /processes between two > datasets > - > > Key: ATLAS-909 > URL: https://issues.apache.org/jira/browse/ATLAS-909 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Blocker > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-06-15 at 6.03.01 PM.png > > > LIneage view does not have a scroll option and its height/width seem to be > limited which was not the case with the older dashboard. This is seen easily > when there are around more than 15-20 queries between two tables. Uploading > screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-909) Lineage view is broken when there are lots of queries /processes between two datasets
[ https://issues.apache.org/jira/browse/ATLAS-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-909: --- Priority: Minor (was: Blocker) > Lineage view is broken when there are lots of queries /processes between two > datasets > - > > Key: ATLAS-909 > URL: https://issues.apache.org/jira/browse/ATLAS-909 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Minor > Fix For: 0.7-incubating > > Attachments: Screen Shot 2016-06-15 at 6.03.01 PM.png > > > LIneage view does not have a scroll option and its height/width seem to be > limited which was not the case with the older dashboard. This is seen easily > when there are around more than 15-20 queries between two tables. Uploading > screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-909) Lineage view is broken when there are lots of queries /processes between two datasets
[ https://issues.apache.org/jira/browse/ATLAS-909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-909: --- Fix Version/s: (was: 0.7-incubating) > Lineage view is broken when there are lots of queries /processes between two > datasets > - > > Key: ATLAS-909 > URL: https://issues.apache.org/jira/browse/ATLAS-909 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Andrew Ahn >Priority: Minor > Attachments: Screen Shot 2016-06-15 at 6.03.01 PM.png > > > LIneage view does not have a scroll option and its height/width seem to be > limited which was not the case with the older dashboard. This is seen easily > when there are around more than 15-20 queries between two tables. Uploading > screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Comment Edited] (ATLAS-909) Lineage view is broken when there are lots of queries /processes between two datasets
[ https://issues.apache.org/jira/browse/ATLAS-909?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15340308#comment-15340308 ] Suma Shivaprasad edited comment on ATLAS-909 at 6/20/16 7:59 PM: - [~kevalbhatt18] I tested this again and I observed that I am able to move up/down in the window and resize if required to get to the required table(Was not aware of this earlier). Hence downgrading to Minor. Assigning to [~aahn] to decide on UX changes needed if any was (Author: suma.shivaprasad): [~kevalbhatt18] I tested this again and I observed that I am able to move up/down in the window and resize if required to get to the required table. Hence downgrading to Minor. Assigning to [~aahn] to decide on UX changes needed if any > Lineage view is broken when there are lots of queries /processes between two > datasets > - > > Key: ATLAS-909 > URL: https://issues.apache.org/jira/browse/ATLAS-909 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Andrew Ahn >Priority: Minor > Attachments: Screen Shot 2016-06-15 at 6.03.01 PM.png > > > LIneage view does not have a scroll option and its height/width seem to be > limited which was not the case with the older dashboard. This is seen easily > when there are around more than 15-20 queries between two tables. Uploading > screen shot -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Created] (ATLAS-918) UI : Deleted entities show status as DELETED and should be differentiable in lineage
Suma Shivaprasad created ATLAS-918: -- Summary: UI : Deleted entities show status as DELETED and should be differentiable in lineage Key: ATLAS-918 URL: https://issues.apache.org/jira/browse/ATLAS-918 Project: Atlas Issue Type: Bug Affects Versions: 0.7-incubating Reporter: Suma Shivaprasad Assignee: Keval Bhatt Keval, This jira is to track showing deleted entities with a diff color in lineage. Is there anything missing in the lineage API to implement this? We need to take care of this since no none looks at lineage via a process and will be mostly looking at lineage from a table Also can we show the property (status)__status for deleted entities as "DELETED" . We could display that in "properties" section. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
[jira] [Updated] (ATLAS-918) UI : Deleted entities show status as DELETED and should be differentiable in lineage
[ https://issues.apache.org/jira/browse/ATLAS-918?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Suma Shivaprasad updated ATLAS-918: --- Priority: Critical (was: Major) > UI : Deleted entities show status as DELETED and should be differentiable in > lineage > > > Key: ATLAS-918 > URL: https://issues.apache.org/jira/browse/ATLAS-918 > Project: Atlas > Issue Type: Bug >Affects Versions: 0.7-incubating >Reporter: Suma Shivaprasad >Assignee: Keval Bhatt >Priority: Critical > > Keval, > This jira is to track showing deleted entities with a diff color in lineage. > Is there anything missing in the lineage API to implement this? We need to > take care of this since no none looks at lineage via a process and will be > mostly looking at lineage from a table > Also can we show the property (status)__status for deleted entities as > "DELETED" . We could display that in "properties" section. -- This message was sent by Atlassian JIRA (v6.3.4#6332)
