Re: Review Request 74706: Adds a Couchbase hook for Apache Atlas
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74706/ --- (Updated Nov. 1, 2023, 3:24 p.m.) Review request for atlas. Changes --- Moves type creation to typedefs Repository: atlas Description (updated) --- We at couchbase developed a bridge for Atlas that connectes to our database using DCP, analyzes metadata for documents that are being stored/updated on the cluster and then sends the extracted metadata onto an Atlas instance. Additional documentation on the bridge is provided in `docs/src/documents/Hook/HookCouchbase.md` Thank you for reviewing! Diffs (updated) - addons/couchbase-bridge/pom.xml PRE-CREATION addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/AtlasConfig.java PRE-CREATION addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/CBConfig.java PRE-CREATION addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/CouchbaseHook.java PRE-CREATION addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseAtlasEntity.java PRE-CREATION addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseBucket.java PRE-CREATION addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseCluster.java PRE-CREATION addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseCollection.java PRE-CREATION addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseField.java PRE-CREATION addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseFieldType.java PRE-CREATION addons/couchbase-bridge/src/main/java/com/couchbase/atlas/connector/entities/CouchbaseScope.java PRE-CREATION addons/couchbase-bridge/src/main/resources/atlas-application.properties PRE-CREATION addons/couchbase-bridge/src/test/java/com/couchbase/atlas/connector/CouchbaseHookTest.java PRE-CREATION addons/couchbase-bridge/src/test/java/com/couchbase/atlas/connector/entities/CouchbaseAtlasEntityTest.java PRE-CREATION addons/couchbase-bridge/src/test/resources/log4j.xml PRE-CREATION addons/models/5000-Couchbase/5020-couchbase_model.json PRE-CREATION distro/pom.xml ed477dfbb distro/src/main/assemblies/atlas-couchbase-hook-package.xml PRE-CREATION distro/src/main/assemblies/standalone-package.xml 3e2ca1c39 docs/src/documents/Hook/HookCouchbase.md PRE-CREATION pom.xml f76d0ea04 Diff: https://reviews.apache.org/r/74706/diff/2/ Changes: https://reviews.apache.org/r/74706/diff/1-2/ Testing --- Testing done manually and there's also a couple of integration tests. Thanks, Dmitrii Chechetkin
Request to join Atlas project
Hi team, I would like to contribute to the Atlas project, Can you please add me as contributor. My Apache id is : priyanshi.shah Regards, Priyanshi Shah
[jira] [Commented] (ATLAS-4807) Fix IT test related to Labels
[ https://issues.apache.org/jira/browse/ATLAS-4807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781688#comment-17781688 ] ASF subversion and git services commented on ATLAS-4807: Commit 123f7619aabd630464e17a3e4457fcd77bedc74d in atlas's branch refs/heads/branch-2.0 from Pinal Shah [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=123f7619a ] ATLAS-4807: Fix IT test related to Labels Signed-off-by: Pinal Shah > Fix IT test related to Labels > - > > Key: ATLAS-4807 > URL: https://issues.apache.org/jira/browse/ATLAS-4807 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Pinal Shah >Assignee: Pinal Shah >Priority: Major > > testSetLabelsByTypeName(org.apache.atlas.web.integration.EntityV2JerseyResourceIT) > test is failing -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (ATLAS-4807) Fix IT test related to Labels
[ https://issues.apache.org/jira/browse/ATLAS-4807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781687#comment-17781687 ] ASF subversion and git services commented on ATLAS-4807: Commit 55d79065b352fc623bd5bad14324ffc043dd5fcd in atlas's branch refs/heads/master from Pinal Shah [ https://gitbox.apache.org/repos/asf?p=atlas.git;h=55d79065b ] ATLAS-4807: Fix IT test related to Labels Signed-off-by: Pinal Shah > Fix IT test related to Labels > - > > Key: ATLAS-4807 > URL: https://issues.apache.org/jira/browse/ATLAS-4807 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Pinal Shah >Assignee: Pinal Shah >Priority: Major > > testSetLabelsByTypeName(org.apache.atlas.web.integration.EntityV2JerseyResourceIT) > test is failing -- This message was sent by Atlassian Jira (v8.20.10#820010)
Re: Review Request 74712: ATLAS-4807: Fix IT test related to Labels
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74712/#review225916 --- Ship it! Ship It! - Jayendra Parab On Nov. 1, 2023, 9:59 a.m., Pinal Shah wrote: > > --- > This is an automatically generated e-mail. To reply, visit: > https://reviews.apache.org/r/74712/ > --- > > (Updated Nov. 1, 2023, 9:59 a.m.) > > > Review request for atlas and Jayendra Parab. > > > Bugs: ATLAS-4807 > https://issues.apache.org/jira/browse/ATLAS-4807 > > > Repository: atlas > > > Description > --- > > testSetLabelsByTypeName(org.apache.atlas.web.integration.EntityV2JerseyResourceIT) > test is failing because of ATLAS-4785 > > > Diffs > - > > > webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java > 00f0aab1a > > > Diff: https://reviews.apache.org/r/74712/diff/1/ > > > Testing > --- > > PC run: > https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1496/ > > > Thanks, > > Pinal Shah > >
Re: Review Request 74712: ATLAS-4807: Fix IT test related to Labels
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74712/ --- (Updated Nov. 1, 2023, 9:59 a.m.) Review request for atlas and Jayendra Parab. Bugs: ATLAS-4807 https://issues.apache.org/jira/browse/ATLAS-4807 Repository: atlas Description --- testSetLabelsByTypeName(org.apache.atlas.web.integration.EntityV2JerseyResourceIT) test is failing because of ATLAS-4785 Diffs - webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java 00f0aab1a Diff: https://reviews.apache.org/r/74712/diff/1/ Testing (updated) --- PC run: https://ci-builds.apache.org/job/Atlas/job/PreCommit-ATLAS-Build-Test/1496/ Thanks, Pinal Shah
[jira] [Updated] (ATLAS-4804) Migrated Data: Process Entity Name not set to QualifiedName for impala_process and impala_process_execution
[
https://issues.apache.org/jira/browse/ATLAS-4804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paresh Devalia updated ATLAS-4804:
--
Attachment: ATLAS-4804-Migrated-Data-Process-Entity-Name-not-set.patch
> Migrated Data: Process Entity Name not set to QualifiedName for
> impala_process and impala_process_execution
> ---
>
> Key: ATLAS-4804
> URL: https://issues.apache.org/jira/browse/ATLAS-4804
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Paresh Devalia
>Assignee: Paresh Devalia
>Priority: Major
> Attachments:
> ATLAS-4804-Migrated-Data-Process-Entity-Name-not-set.patch
>
>
> *Background*
> In Atlas process names ({_}impala_process.name{_}) was using _queryText_ as
> the name. Latest version onwards where _name_ and _qualifiedName_ are same.
> *Solution*
> Add Java patch that updates the name property.
> *Impact of Not Doing this Update*
> The _queryText_ in _impala_process_ and impala_process_execution entities
> tends to be large. The name field is part of {_}AtlasEntityHeader{_}. When
> fetching search results, lineage display are some of the flows that have
> these entities.
> *Issue* *occurred*
> Caused by: org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException:
> Error from server at
> [http://cdhworker11.cinconet.local:8993/solr/vertex_index_shard3_replica_n14:]
> Exception writing document id muydq8 to the index; possible analysis error:
> Document contains at least one immense term in field="cgsl_s" (whose UTF8
> encoding is longer than the max length 32766), all of which were skipped.
> Please correct the analyzer to not produce such terms. The prefix of the
> first immense term is: '[105, 110, 115, 101, 114, 116, 32, 105, 110, 116,
> 111, 32, 103, 105, 103, 121, 97, 46, 115, 116, 103, 95, 108, 97, 98, 95, 103,
> 105, 103, 121]...', original message: bytes can be at most 32766 in length;
> got 41069. Perhaps the document has an indexed string field (solr.StrField)
> which is too large at
> org.apache.solr.client.solrj.impl.CloudSolrClient.getRouteException(CloudSolrClient.java:125)
> at
> org.apache.solr.client.solrj.impl.CloudSolrClient.getRouteException(CloudSolrClient.java:46)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.directUpdate(BaseCloudSolrClient.java:557)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1046)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:906)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:838)
> at
> org.janusgraph.diskstorage.solr.Solr6Index.commitChanges(Solr6Index.java:633)
> at org.janusgraph.diskstorage.solr.Solr6Index.restore(Solr6Index.java:593)
> ... 9 more
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ATLAS-4804) Migrated Data: Process Entity Name not set to QualifiedName for impala_process and impala_process_execution
[
https://issues.apache.org/jira/browse/ATLAS-4804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paresh Devalia updated ATLAS-4804:
--
External issue URL: (was: https://reviews.apache.org/r/74708/)
> Migrated Data: Process Entity Name not set to QualifiedName for
> impala_process and impala_process_execution
> ---
>
> Key: ATLAS-4804
> URL: https://issues.apache.org/jira/browse/ATLAS-4804
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Paresh Devalia
>Assignee: Paresh Devalia
>Priority: Major
>
> *Background*
> In Atlas process names ({_}impala_process.name{_}) was using _queryText_ as
> the name. Latest version onwards where _name_ and _qualifiedName_ are same.
> *Solution*
> Add Java patch that updates the name property.
> *Impact of Not Doing this Update*
> The _queryText_ in _impala_process_ and impala_process_execution entities
> tends to be large. The name field is part of {_}AtlasEntityHeader{_}. When
> fetching search results, lineage display are some of the flows that have
> these entities.
> *Issue* *occurred*
> Caused by: org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException:
> Error from server at
> [http://cdhworker11.cinconet.local:8993/solr/vertex_index_shard3_replica_n14:]
> Exception writing document id muydq8 to the index; possible analysis error:
> Document contains at least one immense term in field="cgsl_s" (whose UTF8
> encoding is longer than the max length 32766), all of which were skipped.
> Please correct the analyzer to not produce such terms. The prefix of the
> first immense term is: '[105, 110, 115, 101, 114, 116, 32, 105, 110, 116,
> 111, 32, 103, 105, 103, 121, 97, 46, 115, 116, 103, 95, 108, 97, 98, 95, 103,
> 105, 103, 121]...', original message: bytes can be at most 32766 in length;
> got 41069. Perhaps the document has an indexed string field (solr.StrField)
> which is too large at
> org.apache.solr.client.solrj.impl.CloudSolrClient.getRouteException(CloudSolrClient.java:125)
> at
> org.apache.solr.client.solrj.impl.CloudSolrClient.getRouteException(CloudSolrClient.java:46)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.directUpdate(BaseCloudSolrClient.java:557)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1046)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:906)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:838)
> at
> org.janusgraph.diskstorage.solr.Solr6Index.commitChanges(Solr6Index.java:633)
> at org.janusgraph.diskstorage.solr.Solr6Index.restore(Solr6Index.java:593)
> ... 9 more
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Commented] (ATLAS-4804) Migrated Data: Process Entity Name not set to QualifiedName for impala_process and impala_process_execution
[
https://issues.apache.org/jira/browse/ATLAS-4804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17781636#comment-17781636
]
Paresh Devalia commented on ATLAS-4804:
---
Code review board :- https://reviews.apache.org/r/74708/
> Migrated Data: Process Entity Name not set to QualifiedName for
> impala_process and impala_process_execution
> ---
>
> Key: ATLAS-4804
> URL: https://issues.apache.org/jira/browse/ATLAS-4804
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Paresh Devalia
>Assignee: Paresh Devalia
>Priority: Major
>
> *Background*
> In Atlas process names ({_}impala_process.name{_}) was using _queryText_ as
> the name. Latest version onwards where _name_ and _qualifiedName_ are same.
> *Solution*
> Add Java patch that updates the name property.
> *Impact of Not Doing this Update*
> The _queryText_ in _impala_process_ and impala_process_execution entities
> tends to be large. The name field is part of {_}AtlasEntityHeader{_}. When
> fetching search results, lineage display are some of the flows that have
> these entities.
> *Issue* *occurred*
> Caused by: org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException:
> Error from server at
> [http://cdhworker11.cinconet.local:8993/solr/vertex_index_shard3_replica_n14:]
> Exception writing document id muydq8 to the index; possible analysis error:
> Document contains at least one immense term in field="cgsl_s" (whose UTF8
> encoding is longer than the max length 32766), all of which were skipped.
> Please correct the analyzer to not produce such terms. The prefix of the
> first immense term is: '[105, 110, 115, 101, 114, 116, 32, 105, 110, 116,
> 111, 32, 103, 105, 103, 121, 97, 46, 115, 116, 103, 95, 108, 97, 98, 95, 103,
> 105, 103, 121]...', original message: bytes can be at most 32766 in length;
> got 41069. Perhaps the document has an indexed string field (solr.StrField)
> which is too large at
> org.apache.solr.client.solrj.impl.CloudSolrClient.getRouteException(CloudSolrClient.java:125)
> at
> org.apache.solr.client.solrj.impl.CloudSolrClient.getRouteException(CloudSolrClient.java:46)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.directUpdate(BaseCloudSolrClient.java:557)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1046)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:906)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:838)
> at
> org.janusgraph.diskstorage.solr.Solr6Index.commitChanges(Solr6Index.java:633)
> at org.janusgraph.diskstorage.solr.Solr6Index.restore(Solr6Index.java:593)
> ... 9 more
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Updated] (ATLAS-4804) Migrated Data: Process Entity Name not set to QualifiedName for impala_process and impala_process_execution
[
https://issues.apache.org/jira/browse/ATLAS-4804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Paresh Devalia updated ATLAS-4804:
--
External issue URL: https://reviews.apache.org/r/74708/
> Migrated Data: Process Entity Name not set to QualifiedName for
> impala_process and impala_process_execution
> ---
>
> Key: ATLAS-4804
> URL: https://issues.apache.org/jira/browse/ATLAS-4804
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Paresh Devalia
>Assignee: Paresh Devalia
>Priority: Major
>
> *Background*
> In Atlas process names ({_}impala_process.name{_}) was using _queryText_ as
> the name. Latest version onwards where _name_ and _qualifiedName_ are same.
> *Solution*
> Add Java patch that updates the name property.
> *Impact of Not Doing this Update*
> The _queryText_ in _impala_process_ and impala_process_execution entities
> tends to be large. The name field is part of {_}AtlasEntityHeader{_}. When
> fetching search results, lineage display are some of the flows that have
> these entities.
> *Issue* *occurred*
> Caused by: org.apache.solr.client.solrj.impl.CloudSolrClient$RouteException:
> Error from server at
> [http://cdhworker11.cinconet.local:8993/solr/vertex_index_shard3_replica_n14:]
> Exception writing document id muydq8 to the index; possible analysis error:
> Document contains at least one immense term in field="cgsl_s" (whose UTF8
> encoding is longer than the max length 32766), all of which were skipped.
> Please correct the analyzer to not produce such terms. The prefix of the
> first immense term is: '[105, 110, 115, 101, 114, 116, 32, 105, 110, 116,
> 111, 32, 103, 105, 103, 121, 97, 46, 115, 116, 103, 95, 108, 97, 98, 95, 103,
> 105, 103, 121]...', original message: bytes can be at most 32766 in length;
> got 41069. Perhaps the document has an indexed string field (solr.StrField)
> which is too large at
> org.apache.solr.client.solrj.impl.CloudSolrClient.getRouteException(CloudSolrClient.java:125)
> at
> org.apache.solr.client.solrj.impl.CloudSolrClient.getRouteException(CloudSolrClient.java:46)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.directUpdate(BaseCloudSolrClient.java:557)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.sendRequest(BaseCloudSolrClient.java:1046)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.requestWithRetryOnStaleState(BaseCloudSolrClient.java:906)
> at
> org.apache.solr.client.solrj.impl.BaseCloudSolrClient.request(BaseCloudSolrClient.java:838)
> at
> org.janusgraph.diskstorage.solr.Solr6Index.commitChanges(Solr6Index.java:633)
> at org.janusgraph.diskstorage.solr.Solr6Index.restore(Solr6Index.java:593)
> ... 9 more
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
Review Request 74712: ATLAS-4807: Fix IT test related to Labels
--- This is an automatically generated e-mail. To reply, visit: https://reviews.apache.org/r/74712/ --- Review request for atlas and Jayendra Parab. Bugs: ATLAS-4807 https://issues.apache.org/jira/browse/ATLAS-4807 Repository: atlas Description --- testSetLabelsByTypeName(org.apache.atlas.web.integration.EntityV2JerseyResourceIT) test is failing because of ATLAS-4785 Diffs - webapp/src/test/java/org/apache/atlas/web/integration/EntityV2JerseyResourceIT.java 00f0aab1a Diff: https://reviews.apache.org/r/74712/diff/1/ Testing --- PC run: Thanks, Pinal Shah
[jira] [Created] (ATLAS-4807) Fix IT test related to Labels
Pinal Shah created ATLAS-4807: - Summary: Fix IT test related to Labels Key: ATLAS-4807 URL: https://issues.apache.org/jira/browse/ATLAS-4807 Project: Atlas Issue Type: Bug Components: atlas-core Reporter: Pinal Shah Assignee: Pinal Shah testSetLabelsByTypeName(org.apache.atlas.web.integration.EntityV2JerseyResourceIT) test is failing -- This message was sent by Atlassian Jira (v8.20.10#820010)
