[jira] [Created] (ATLAS-1574) Regression : "user" attribute is not found in Audit response for entity not created through UI.

[Sharmadha Sainath (JIRA)]

Sharmadha Sainath created ATLAS-1574:


 Summary: Regression : "user" attribute is not found in Audit 
response for entity not created through UI.
 Key: ATLAS-1574
 URL: https://issues.apache.org/jira/browse/ATLAS-1574
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 0.8-incubating
Reporter: Sharmadha Sainath
 Fix For: 0.8-incubating
 Attachments: AuditResposne.png

Created hive_table through hook. GET Audit response for the hive_table
didn't have user attribute for "Entity Created" audit,whereas for "Tag added" 
audit , user attribute is present with value "admin".
Attached the Screenshot of the Audit response for the above scenario.

Entities created through UI have "Entity Created" audit with user attribute.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ATLAS-1502) UI to create/update entities - configuration to restrict entity-types editable via UI

[Sharmadha Sainath (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1502?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15875373#comment-15875373
 ] 

Sharmadha Sainath commented on ATLAS-1502:
--

Set the following in atlas-application.properties. 

{code}
atlas.ui.editable.entity.types=*
{code}

and restarted Atlas. Not able to see all the entity types in the drop down. 
What am I missing here ? Opened Atlas in Chrome incognito window hence no 
browser cache. 


> UI to create/update entities - configuration to restrict entity-types 
> editable via UI
> -
>
> Key: ATLAS-1502
> URL: https://issues.apache.org/jira/browse/ATLAS-1502
> Project: Atlas
>  Issue Type: Bug
>  Components: atlas-webui
>Affects Versions: 0.8-incubating
>Reporter: Madhan Neethiraj
>Assignee: Kalyani Kashikar
> Attachments: ATLAS-1502.patch
>
>
> ATLAS-1193 introduced UI to create and update entities. Currently the UI 
> allows an user to create entity of any entity-type - including the superTypes 
> like Referenceable, Asset, Process, etc. It may not be useful to support 
> entity creation of many such entity-types via UI. Restricting the UI to 
> support only a subset of entities would make the UI more usable and avoid 
> potential confusion.
> It should be possible to configure the list of entity-types supported via a 
> configuration, to enable deployments to satisfy specific needs. I would 
> suggest to use a configuration like:
> {code}
> atlas.ui.editable.entity.types="type-1,type-2,type3"
> {code}
> - if the configuration is set to an empty value (or not set): UI should not 
> support create/edit of any entity
> - if the configuration is set to is set to "*", then UI should allow 
> create/edit of any type of entity



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Re: Review Request 56825: Taxonomy test failure fixes

[Jeff Hagelberg]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56825/#review166106
---


Fix it, then Ship it!





webapp/src/test/java/org/apache/atlas/web/resources/TaxonomyServiceTest.java 


Minor comment - now the transactionInitialized variable is unused and can 
be removed.


- Jeff Hagelberg


On Feb. 19, 2017, 7:26 p.m., Apoorv Naik wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56825/
> ---
> 
> (Updated Feb. 19, 2017, 7:26 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-1570
> https://issues.apache.org/jira/browse/ATLAS-1570
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> Introduction of the StateTransactionCleanupFilter caused a side-effect that 
> led to test failures
> 
> 
> Diffs
> -
> 
>   
> webapp/src/test/java/org/apache/atlas/web/resources/TaxonomyServiceTest.java 
> 45df9c7c 
> 
> Diff: https://reviews.apache.org/r/56825/diff/
> 
> 
> Testing
> ---
> 
> mvn clean install executes successfully with some regular IT failures
> 
> 
> Thanks,
> 
> Apoorv Naik
> 
>

Re: Review Request 56856: ATLAS-1535 - Resolve failures in TaxonomyServiceTest

[David Kantor]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56856/#review166105
---


Ship it!




Ship It!

- David Kantor


On Feb. 20, 2017, 7:03 p.m., Jeff Hagelberg wrote:
> 
> ---
> This is an automatically generated e-mail. To reply, visit:
> https://reviews.apache.org/r/56856/
> ---
> 
> (Updated Feb. 20, 2017, 7:03 p.m.)
> 
> 
> Review request for atlas.
> 
> 
> Bugs: ATLAS-1535
> https://issues.apache.org/jira/browse/ATLAS-1535
> 
> 
> Repository: atlas
> 
> 
> Description
> ---
> 
> This resolves the test failures in TaxonomyServiceTest.  The service no 
> longer needs to explicitly manage its transactions.  The test has been 
> updated to reflect this.
> 
> 
> Diffs
> -
> 
>   
> webapp/src/test/java/org/apache/atlas/web/resources/TaxonomyServiceTest.java 
> 45df9c7c6d55826c551e8b54e52b22be3a63bac0 
> 
> Diff: https://reviews.apache.org/r/56856/diff/
> 
> 
> Testing
> ---
> 
> TaxonomyServiceTest now passes.
> 
> 
> Thanks,
> 
> Jeff Hagelberg
> 
>

Re: Review Request 56856: ATLAS-1535 - Resolve failures in TaxonomyServiceTest

[Jeff Hagelberg]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56856/
---

(Updated Feb. 20, 2017, 7:03 p.m.)


Review request for atlas.


Bugs: ATLAS-1535
https://issues.apache.org/jira/browse/ATLAS-1535


Repository: atlas


Description (updated)
---

This resolves the test failures in TaxonomyServiceTest.  The service no longer 
needs to explicitly manage its transactions.  The test has been updated to 
reflect this.


Diffs
-

  webapp/src/test/java/org/apache/atlas/web/resources/TaxonomyServiceTest.java 
45df9c7c6d55826c551e8b54e52b22be3a63bac0 

Diff: https://reviews.apache.org/r/56856/diff/


Testing
---

TaxonomyServiceTest now passes.


Thanks,

Jeff Hagelberg

[jira] [Created] (ATLAS-1573) Regression : Full text search returns empty results for any query

[Sharmadha Sainath (JIRA)]

Sharmadha Sainath created ATLAS-1573:


 Summary: Regression : Full text search returns empty results for 
any query
 Key: ATLAS-1573
 URL: https://issues.apache.org/jira/browse/ATLAS-1573
 Project: Atlas
  Issue Type: Bug
  Components:  atlas-core
Affects Versions: 0.8-incubating
Reporter: Sharmadha Sainath
Priority: Blocker
 Fix For: 0.8-incubating


DSL search works fine.Any Full text search is not yielding any result. GET 
response of the Full text search is empty.

Both v2 entity API response and old API response are empty
{code}
/api/atlas/v2/search/fulltext?limit=25&query=topic1
/api/atlas/discovery/search/fulltext?query=topic1
{code}



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ATLAS-1535) Some webapp tests are failing due to a stale Titan transaction

[Jeffrey Hagelberg (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1535?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Jeffrey Hagelberg updated ATLAS-1535:
-
Attachment: rb56856.patch

Attaching patch to fix TaxonomyServiceTest

> Some webapp tests are failing due to a stale Titan transaction
> --
>
> Key: ATLAS-1535
> URL: https://issues.apache.org/jira/browse/ATLAS-1535
> Project: Atlas
>  Issue Type: Bug
>Reporter: Jeffrey Hagelberg
>Assignee: Jeffrey Hagelberg
> Attachments: rb56417.patch, rb56856.patch
>
>
> I've been debugging some of the recent test failures we're seeing in the 
> webapp project.  It looks like at least some of them are occurring when the 
> test executes a thread that runs in the context of a stale Titan transaction. 
>  There is logic in BaseService to rollback the transaction associated with a 
> thread whenever a new request comes in.  However, this logic is not used in 
> all places.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

Review Request 56856: ATLAS-1535 - Resolve failures in TaxonomyServiceTest

[Jeff Hagelberg]

---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56856/
---

Review request for atlas.


Bugs: ATLAS-1535
https://issues.apache.org/jira/browse/ATLAS-1535


Repository: atlas


Description
---

This resolves the test failures in TaxonomyServiceTest.  The service no longer 
needs to explicitly manage its transactions.  The test has been update to 
reflect this.


Diffs
-

  webapp/src/test/java/org/apache/atlas/web/resources/TaxonomyServiceTest.java 
45df9c7c6d55826c551e8b54e52b22be3a63bac0 

Diff: https://reviews.apache.org/r/56856/diff/


Testing
---

TaxonomyServiceTest now passes.


Thanks,

Jeff Hagelberg

RE: Review Request 56417: ATLAS-1535: Some webapp tests are failing due to a stale Titan transaction

[Jeffrey N Hagelberg]

Sure, I'll investigate.

-Original Message-
From: Madhan Neethiraj [mailto:mad...@apache.org]
Sent: Saturday, February 18, 2017 9:36 PM
To: dev@atlas.incubator.apache.org; Jeffrey N Hagelberg
; Dave Kantor 
Subject: Re: Review Request 56417: ATLAS-1535: Some webapp tests are
failing due to a stale Titan transaction

Jeff,

Following test failures seem to be caused by removal of
TaxonomyServiceTest.initializeGraphTransaction() method in this patch. Can
you please review?

  TaxonomyServiceTest.testCreateSubTerm:456 expected: but was:
  TaxonomyServiceTest.testCreateTaxonomy:158 expected: but
was:
  TaxonomyServiceTest.testCreateTerm:416 expected: but was:
  TaxonomyServiceTest.testDeleteSubTerm:529 expected: but was:
  TaxonomyServiceTest.testDeleteTaxonomy:194 expected: but
was:
  TaxonomyServiceTest.testDeleteTerm:493 expected: but was:
  TaxonomyServiceTest.testGetSubTerms_collection:378 expected: but
was:
  TaxonomyServiceTest.testGetSubTerms_instance:326 expected: but
was:
  TaxonomyServiceTest.testGetTaxonomies:125 expected: but was:
  TaxonomyServiceTest.testGetTaxonomy:84 expected: but was:
  TaxonomyServiceTest.testGetTaxonomyTerm:235 expected: but
was:
  TaxonomyServiceTest.testGetTaxonomyTerms:277 expected: but
was:

Thanks,
Madhan

On 2/8/17, 2:29 PM, "Jeff Hagelberg"  wrote:


---
This is an automatically generated e-mail. To reply, visit:
https://reviews.apache.org/r/56417/
---

(Updated Feb. 8, 2017, 10:29 p.m.)


Review request for atlas and David Kantor.


Bugs: ATLAS-1535
https://issues.apache.org/jira/browse/ATLAS-1535


Repository: atlas


Description (updated)
---

When debugging some of the test failures in webapp, I found that many
are occurring when a http request runs in the context of a stale Titan
transaction. There is logic in BaseService to rollback the transaction
associated with a thread whenever a new request comes in. However, this
logic is not used in all places.

This changes add logic to fix this.  I've added a new filter to
consistently clean up stale transactions when processing http requests.  I
removed the old logic, which was only used by two service classes.  The new
filter is applied for every http request that comes in, so the transaction
used during request processing will never be stale now.


Diffs
-


webapp/src/main/java/org/apache/atlas/web/filters/StaleTransactionCleanupFilter.java
 PRE-CREATION

webapp/src/main/java/org/apache/atlas/web/listeners/GuiceServletConfig.java
d0437fc54087e6803ead83af59b85e9f6df333ad
  webapp/src/main/java/org/apache/atlas/web/resources/BaseService.java
fb77b11e04c875a3d443dfb49be50cb818bed441

webapp/src/test/java/org/apache/atlas/web/resources/TaxonomyServiceTest.java
 e1734e465ad553d95735b466f4dfe0fa8ad061b8

Diff: https://reviews.apache.org/r/56417/diff/


Testing
---

Ran webapp tests.  Now, we're only left with the following failures:

QuickStartIT.runQuickStart:44 » AtlasService Metadata service API
org.apache.a...
QuickStartV2IT.runQuickStart:47 » AtlasService Metadata service API
org.apache...
NotificationHookConsumerIT.testUpdatePartialUpdatingQualifiedName:170
expected:<0> but was:<1>

Previously, there were a number of other tests that were failing
sporatically with a NullPointerException coming from Titan.  These changes
fix that.


Thanks,

Jeff Hagelberg

Build failed in Jenkins: apache-atlas-nightly #632

[Apache Jenkins Server]

See 


Changes:

[kbhatt] ATLAS-1546: Changed to LOG.warn instead error in Hive hook should 
choose

--
[...truncated 485.52 KB...]
127.0.0.1 - - [20/Feb/2017:13:30:06 +] "GET 
/api/atlas/lineage/bb537e3c-795d-4d33-9bcc-9b08b0abc1c9/inputs/graph HTTP/1.1" 
200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:07 +] "GET 
/api/atlas/lineage/hive/table/sales_factrUBnHOhhGY/outputs/graph HTTP/1.1" 200 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:10 +] "GET 
/api/atlas/entities?type=hive_table_v1&property=qualifiedName&value=sales_factrUBnHOhhGY
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:10 +] "GET 
/api/atlas/lineage/1da3967f-d947-4d97-8255-4476ef8dfea9/outputs/graph HTTP/1.1" 
200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:11 +] "GET 
/api/atlas/lineage/hive/table/sales_factrUBnHOhhGY/schema HTTP/1.1" 200 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:12 +] "GET 
/api/atlas/lineage/hive/table/Salesms21Ft8wbx/schema HTTP/1.1" 404 - "-" 
"Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:12 +] "GET 
/api/atlas/entities?type=hive_table_v1&property=qualifiedName&value=sales_factrUBnHOhhGY
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:12 +] "GET 
/api/atlas/lineage/1da3967f-d947-4d97-8255-4476ef8dfea9/schema HTTP/1.1" 200 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:13 +] "GET 
/api/atlas/lineage/hive/table/blah/schema HTTP/1.1" 404 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:13 +] "GET 
/api/atlas/entities?type=hive_table_v1&property=qualifiedName&value=sales_fact_monthly_mvIPOET02UtU
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:13 +] "GET 
/api/atlas/v2/lineage/04ad6d92-440b-480d-ab05-665fab033d60?direction=INPUT&depth=5
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:14 +] "GET 
/api/atlas/entities?type=hive_table_v1&property=qualifiedName&value=sales_fact_monthly_mvIPOET02UtU
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:14 +] "GET 
/api/atlas/v2/lineage/04ad6d92-440b-480d-ab05-665fab033d60?direction=BOTH&depth=5
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:16 +] "GET 
/api/atlas/entities?type=hive_table_v1&property=qualifiedName&value=sales_factlSWu65jzik
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:16 +] "GET 
/api/atlas/v2/lineage/a752a26a-2a6d-4628-a0e2-d4e15818dc61?direction=OUTPUT&depth=5
 HTTP/1.1" 200 - "-" "Java/1.7.0_80"
Tests run: 11, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 18.589 sec - 
in org.apache.atlas.web.resources.EntityLineageJerseyResourceIT
Running org.apache.atlas.CredentialProviderUtilityIT
Tests run: 4, Failures: 0, Errors: 0, Skipped: 0, Time elapsed: 1.706 sec - in 
org.apache.atlas.CredentialProviderUtilityIT
Running org.apache.atlas.notification.EntityNotificationIT
127.0.0.1 - - [20/Feb/2017:13:30:23 +] "GET /api/atlas/types/tableType 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:23 +] "GET /api/atlas/types/serdeType 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:24 +] "GET /api/atlas/types/classification 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:24 +] "GET /api/atlas/types/hive_db_v1 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:24 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:25 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:26 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:26 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:26 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:27 +] "POST 
/api/atlas/entities/d44679d0-2a04-4ead-8007-4d9c45bfaf78/traits HTTP/1.1" 201 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:27 +] "POST /api/atlas/types HTTP/1.1" 201 
- "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:27 +] "POST 
/api/atlas/entities/d44679d0-2a04-4ead-8007-4d9c45bfaf78/traits HTTP/1.1" 201 - 
"-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:27 +] "DELETE 
/api/atlas/entities/d44679d0-2a04-4ead-8007-4d9c45bfaf78/traits/TraitZT45ReoqYU 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:27 +] "POST 
/api/atlas/entities/d44679d0-2a04-4ead-8007-4d9c45bfaf78?property=description 
HTTP/1.1" 200 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:27 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 - - [20/Feb/2017:13:30:27 +] "POST /api/atlas/entities HTTP/1.1" 
201 - "-" "Java/1.7.0_80"
127.0.0.1 

[jira] [Comment Edited] (ATLAS-1546) Hive hook should choose appropriate JAAS config if host uses kerberos ticket-cache

[Keval Bhatt (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15874451#comment-15874451
 ] 

Keval Bhatt edited comment on ATLAS-1546 at 2/20/17 12:30 PM:
--

+1 for the ATLAS-1546.4.patch


was (Author: kevalbhatt18):
+1 for the patch ATLAS-1546.4.patch

> Hive hook should choose appropriate JAAS config if host uses kerberos 
> ticket-cache
> --
>
> Key: ATLAS-1546
> URL: https://issues.apache.org/jira/browse/ATLAS-1546
> Project: Atlas
>  Issue Type: Improvement
>  Components: atlas-intg
>Affects Versions: 0.7-incubating, 0.8-incubating
>Reporter: Madhan Neethiraj
>Assignee: Nixon Rodrigues
> Fix For: 0.8-incubating
>
> Attachments: ATLAS-1546.1.patch, ATLAS-1546.2.patch, 
> ATLAS-1546.3.patch, ATLAS-1546.4.patch, ATLAS-1546.patch, hiveenviro, 
> hiveserver2_log.txt, hiveserver2-site.xml, hive-site.xml, hs2.log.gz
>
>
> In a kerberized environment, Atlas hook uses JAAS configuration section named 
> "KakfaClient" to authenticate with Kafka broker. In a typical Hive deployment 
> this configuration section is set to use the keytab and principal of 
> HiveServer2 process. The hook running in HiveCLI might fail to authenticate 
> with Kafka if the user can't read the configured keytab.
> Given that HiveCLI users would have performed kinit, the hook in HiveCLI 
> should use the ticket-cache generated by kinit. When ticket cache is not 
> available (for example in HiveServer2), the hook should use the configuration 
> provided in KafkaClient JAAS section.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Commented] (ATLAS-1546) Hive hook should choose appropriate JAAS config if host uses kerberos ticket-cache

[Keval Bhatt (JIRA)]

[ 
https://issues.apache.org/jira/browse/ATLAS-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15874451#comment-15874451
 ] 

Keval Bhatt commented on ATLAS-1546:


+1 for the patch ATLAS-1546.4.patch

> Hive hook should choose appropriate JAAS config if host uses kerberos 
> ticket-cache
> --
>
> Key: ATLAS-1546
> URL: https://issues.apache.org/jira/browse/ATLAS-1546
> Project: Atlas
>  Issue Type: Improvement
>  Components: atlas-intg
>Affects Versions: 0.7-incubating, 0.8-incubating
>Reporter: Madhan Neethiraj
>Assignee: Nixon Rodrigues
> Fix For: 0.8-incubating
>
> Attachments: ATLAS-1546.1.patch, ATLAS-1546.2.patch, 
> ATLAS-1546.3.patch, ATLAS-1546.4.patch, ATLAS-1546.patch, hiveenviro, 
> hiveserver2_log.txt, hiveserver2-site.xml, hive-site.xml, hs2.log.gz
>
>
> In a kerberized environment, Atlas hook uses JAAS configuration section named 
> "KakfaClient" to authenticate with Kafka broker. In a typical Hive deployment 
> this configuration section is set to use the keytab and principal of 
> HiveServer2 process. The hook running in HiveCLI might fail to authenticate 
> with Kafka if the user can't read the configured keytab.
> Given that HiveCLI users would have performed kinit, the hook in HiveCLI 
> should use the ticket-cache generated by kinit. When ticket cache is not 
> available (for example in HiveServer2), the hook should use the configuration 
> provided in KafkaClient JAAS section.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Updated] (ATLAS-1546) Hive hook should choose appropriate JAAS config if host uses kerberos ticket-cache

[Nixon Rodrigues (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

Nixon Rodrigues updated ATLAS-1546:
---
Attachment: ATLAS-1546.4.patch

Changed to LOG.warn instead LOG.error in isLoginKeytabBased & 
isLoginTicketBased method in ATLAS-1546.4.patch

> Hive hook should choose appropriate JAAS config if host uses kerberos 
> ticket-cache
> --
>
> Key: ATLAS-1546
> URL: https://issues.apache.org/jira/browse/ATLAS-1546
> Project: Atlas
>  Issue Type: Improvement
>  Components: atlas-intg
>Affects Versions: 0.7-incubating, 0.8-incubating
>Reporter: Madhan Neethiraj
>Assignee: Nixon Rodrigues
> Fix For: 0.8-incubating
>
> Attachments: ATLAS-1546.1.patch, ATLAS-1546.2.patch, 
> ATLAS-1546.3.patch, ATLAS-1546.4.patch, ATLAS-1546.patch, hiveenviro, 
> hiveserver2_log.txt, hiveserver2-site.xml, hive-site.xml, hs2.log.gz
>
>
> In a kerberized environment, Atlas hook uses JAAS configuration section named 
> "KakfaClient" to authenticate with Kafka broker. In a typical Hive deployment 
> this configuration section is set to use the keytab and principal of 
> HiveServer2 process. The hook running in HiveCLI might fail to authenticate 
> with Kafka if the user can't read the configured keytab.
> Given that HiveCLI users would have performed kinit, the hook in HiveCLI 
> should use the ticket-cache generated by kinit. When ticket cache is not 
> available (for example in HiveServer2), the hook should use the configuration 
> provided in KafkaClient JAAS section.



--
This message was sent by Atlassian JIRA
(v6.3.15#6346)

[jira] [Created] (ATLAS-1572) sqoop metadata ingest fails with "org.apache.atlas.exception.AtlasBaseException: expected type List or Set; found org.apache.atlas.typesystem.Referenceable"

[Ayub Khan (JIRA)]

Ayub Khan created ATLAS-1572:


 Summary: sqoop metadata ingest fails with 
"org.apache.atlas.exception.AtlasBaseException: expected type List or Set; 
found org.apache.atlas.typesystem.Referenceable"
 Key: ATLAS-1572
 URL: https://issues.apache.org/jira/browse/ATLAS-1572
 Project: Atlas
  Issue Type: Bug
  Components: atlas-intg
Affects Versions: trunk, 0.8-incubating
Reporter: Ayub Khan
Priority: Blocker
 Fix For: trunk, 0.8-incubating


Sqoop import succeeds but the metadata ingest fails with below error

{noformat}
2017-02-20 11:20:43,179 WARN  - [NotificationHookConsumer thread-0:] ~ Max 
retries exceeded for message [{Id='(type: sqoop_dbdatastore, id: 
)', traits=[], values={owner=hrt_qa, 
storeUri=jdbc:mysql://ctr-e129-1487033772569-11862-01-02.hwx.site/sqoopTests?zeroDateTimeBehavior=convertToNull,
 dbStoreType=mysql, qualifiedName=mysql --url 
jdbc:mysql://ctr-e129-1487033772569-11862-01-02.hwx.site/sqoopTests?zeroDateTimeBehavior=convertToNull
 --table createSqoopTablejmnwwf8rji, name=mysql --url 
jdbc:mysql://ctr-e129-1487033772569-11862-01-02.hwx.site/sqoopTests?zeroDateTimeBehavior=convertToNull
 --table createSqoopTablejmnwwf8rji, description=, 
source=createSqoopTablejmnwwf8rji, storeUse=TABLE}}, {Id='(type: hive_db, id: 
)', traits=[], values={name=default, qualifiedName=default@cl1, 
clusterName=cl1}}, {Id='(type: hive_table, id: )', traits=[], 
values={name=createsqooptablejmnwwf8rji, 
qualifiedName=default.createsqooptablejmnwwf8rji@cl1, db={Id='(type: hive_db, 
id: )', traits=[], values={name=default, qualifiedName=default@cl1, 
clusterName=cl1, {Id='(type: sqoop_process, id: )', traits=[], 
values={outputs={Id='(type: hive_table, id: )', traits=[], 
values={name=createsqooptablejmnwwf8rji, 
qualifiedName=default.createsqooptablejmnwwf8rji@cl1, db={Id='(type: hive_db, 
id: )', traits=[], values={name=default, qualifiedName=default@cl1, 
clusterName=cl1, commandlineOpts={db.clear.staging.table=false, 
hive.import=true, db.require.password=true, 
codegen.output.delimiters.enclose=0, codegen.input.delimiters.field=0, 
customtool.options.jsonmap={}, hive.compute.stats.table=false, 
db.connect.string=jdbc:mysql://ctr-e129-1487033772569-11862-01-02.hwx.site/sqoopTests?zeroDateTimeBehavior=convertToNull,
 incremental.mode=None, db.table=createSqoopTablejmnwwf8rji, verbose=false, 
codegen.output.delimiters.enclose.required=false, mapreduce.num.mappers=4, 
hdfs.append.dir=false, direct.import=false, hive.drop.delims=false, 
hive.overwrite.table=false, hbase.bulk.load.enabled=false, 
hive.fail.table.exists=false, relaxed.isolation=false, 
hdfs.delete-target.dir=false, split.limit=null, db.username=sqoop, 
codegen.input.delimiters.enclose.required=false, codegen.output.dir=., 
import.direct.split.size=0, reset.onemapper=false, 
codegen.output.delimiters.record=10, temporary.dirRoot=_sqoop, 
hcatalog.create.table=false, db.batch=false, import.fetch.size=-2147483648, 
accumulo.max.latency=5000, hdfs.file.format=TextFile, 
codegen.output.delimiters.field=1, mainframe.input.dataset.type=p, 
codegen.output.delimiters.escape=0, hcatalog.drop.and.create.table=false, 
import.max.inline.lob.size=16777216, hbase.create.table=false, 
codegen.auto.compile.dir=true, 
codegen.compile.dir=/tmp/sqoop-hrt_qa/compile/f7c5181710812973a20a3e4776c5a6ef, 
codegen.input.delimiters.enclose=0, export.new.update=UpdateOnly, 
enable.compression=false, skip.dist.cache=false, accumulo.batch.size=1024, 
hdfs.warehouse.dir=/user/hrt_qa, codegen.input.delimiters.record=0, 
codegen.input.delimiters.escape=0, accumulo.create.table=false}, 
inputs={Id='(type: sqoop_dbdatastore, id: )', traits=[], 
values={owner=hrt_qa, 
storeUri=jdbc:mysql://ctr-e129-1487033772569-11862-01-02.hwx.site/sqoopTests?zeroDateTimeBehavior=convertToNull,
 dbStoreType=mysql, qualifiedName=mysql --url 
jdbc:mysql://ctr-e129-1487033772569-11862-01-02.hwx.site/sqoopTests?zeroDateTimeBehavior=convertToNull
 --table createSqoopTablejmnwwf8rji, name=mysql --url 
jdbc:mysql://ctr-e129-1487033772569-11862-01-02.hwx.site/sqoopTests?zeroDateTimeBehavior=convertToNull
 --table createSqoopTablejmnwwf8rji, description=, 
source=createSqoopTablejmnwwf8rji, storeUse=TABLE}}, qualifiedName=sqoop import 
--connect 
jdbc:mysql://ctr-e129-1487033772569-11862-01-02.hwx.site/sqoopTests?zeroDateTimeBehavior=convertToNull
 --table createSqoopTablejmnwwf8rji --hive-import --hive-database default 
--hive-table createsqooptablejmnwwf8rji --hive-cluster cl1, name=sqoop import 
--connect 
jdbc:mysql://ctr-e129-1487033772569-11862-01-02.hwx.site/sqoopTests?zeroDateTimeBehavior=convertToNull
 --table createSqoopTablejmnwwf8rji --hive-import --hive-database default 
--hive-table createsqooptablejmnwwf8rji --hive-cluster cl1, 
startTime=2017-02-20T11:20:04.901Z, endTime=2017-02-20T11:20:40.055Z, 
us

[jira] [Updated] (ATLAS-1401) Document in detail how to set up Eclipse for Atlas dev environment.

[carmelo zhang (JIRA)]

 [ 
https://issues.apache.org/jira/browse/ATLAS-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

carmelo zhang updated ATLAS-1401:
-

Hi David:
   I saw you had closed this issue, So I can`t write comments on it, I have 
a question, Did you use hortonworks`  HDP environment with apache atlas ,or you 
just use apache atlas alone? 
   If you use atlas alone , how do you config to make Atlas import hive 
metadata automatically ? Your reply will be very important for me , thank you !












On 2/10/17, 02:03, "David Radley (JIRA)"  wrote:


 [ 
https://issues.apache.org/jira/browse/ATLAS-1401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
 ]

David Radley updated ATLAS-1401:

Attachment: rb54899(3).patch

> Document in detail how to set up Eclipse for Atlas dev environment.
> ---
>
> Key: ATLAS-1401
> URL: https://issues.apache.org/jira/browse/ATLAS-1401
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Attachments: rb54899(2).patch, rb54899(3).patch, rb54899.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)




> Document in detail how to set up Eclipse for Atlas dev environment.
> ---
>
> Key: ATLAS-1401
> URL: https://issues.apache.org/jira/browse/ATLAS-1401
> Project: Atlas
>  Issue Type: Improvement
>Reporter: David Radley
>Assignee: David Radley
> Fix For: 0.8-incubating
>
> Attachments: rb54899(2).patch, rb54899(3).patch, rb54899.patch
>
>




--
This message was sent by Atlassian JIRA
(v6.3.15#6346)