[jira] [Assigned] (ATLAS-4838) Export/Import : changeMarker is not set to entity's lastupdatetime or its closer timestamp value
[
https://issues.apache.org/jira/browse/ATLAS-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayendra Parab reassigned ATLAS-4838:
-
Assignee: Jayendra Parab
> Export/Import : changeMarker is not set to entity's lastupdatetime or its
> closer timestamp value
>
>
> Key: ATLAS-4838
> URL: https://issues.apache.org/jira/browse/ATLAS-4838
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Sharmadha S
>Assignee: Jayendra Parab
>Priority: Major
>
> When an hive table entity is exported using fetchType incremental with
> changeMarker 0 , after exporting , the changeMarker in the export response is
> not set to recent timestamp.
> For example , on exporting iceberg table with fetchType incremental and
> changeMarker as 0 :
> {code:java}
> 2024-02-08 13:28:43,838 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~
> export(item=AtlasObjectId{guid='null', typeName='iceberg_table',
> uniqueAttributes={qualifiedName:default.iceberg_table2@cm}}; matchType=null,
> fetchType=incremental): found 1 entities: options:
> {"itemsToExport":[{"typeName":"iceberg_table","uniqueAttributes":{"qualifiedName":"default.iceberg_table2@cm"}}],"options":{"fetchType":"incremental","changeMarker":0},"fetchTypeOptionValue":"incremental","skipLineageOptionValue":false,"changeTokenFromOptions":0}
> (StartEntityFetchByExportRequest:123)
> 2024-02-08 13:28:48,697 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ Assigned guid =
> f5e6342a-dfdf-4d54-a1b3-b24f987c4ecb to newly created entity (DataAccess:85)
> 2024-02-08 13:28:48,697 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ addAuditEntry: user: admin, source:
> cm, target: , operation: EXPORT (ExportImportAuditService:168)
> 2024-02-08 13:28:48,699 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ <== export(user=admin,
> from=10.19.28.124): status SUCCESS: changeMarker: 1707225183928
> (ExportService:107) {code}
> changeMarker from response is 1707225183928 => Tuesday, February 6, 2024
> 6:43:03.928 PM
>
> But lastUpdatetime of iceberg table is
> {code:java}
> "createTime": 1707398866970 => Thursday, February 8, 2024 6:57:46.970 PM
> "updateTime": 1707398867375. => Thursday, February 8, 2024 6:57:47.375 PM
> {code}
> Value is set to 2 days back. This is not then an incremental export , as it
> will fetch all updates from Tuesday, February 6, 2024 6:43:03.928 PM in next
> export
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
[jira] [Created] (ATLAS-4692) Upgrade Apache Commons Text to 1.10.0
Jayendra Parab created ATLAS-4692: - Summary: Upgrade Apache Commons Text to 1.10.0 Key: ATLAS-4692 URL: https://issues.apache.org/jira/browse/ATLAS-4692 Project: Atlas Issue Type: Task Components: atlas-core Affects Versions: 2.2.0 Reporter: Jayendra Parab Assignee: Jayendra Parab Upgrade Apache Commons Text to 1.10.0 -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4691) Discrepancy in the atlas debug metrics between the active and the not active servers
[ https://issues.apache.org/jira/browse/ATLAS-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab updated ATLAS-4691: -- Description: There is a discrepancy in the atlas debug metrics between the active and the not active servers. Steps to reproduce: - Setup Atlas in HA mode with two instances of Atlas - add "atlas.debug.metrics.enabled=true" to the Atlas configuration (conf/atlas-application.properties) - restart Atlas - open Atlas and create a new entity (e.g. hdfs_path) - check the debug metrics on both atlas instances using API call: https://:/api/atlas/admin/debug/metrics - The active instance returns correct metrics, whereas passive instance returns empty metrics The reason for this is that the debug metrics are maintained in-memory in the active server. When "admin/debug/metrics" API call is accessed on the passive server, the request isn't redirected to the active server like it does for the other APIs, hence the passive node returns empty metrics. was: There is a discrepancy in the atlas debug metrics between the active and the not active servers. Steps to reproduce: - Setup Atlas in HA mode with two instances of Atlas - add "atlas.debug.metrics.enabled=true" to the Atlas configuration (conf/atlas-application.properties_role_safety_valve) - restart Atlas - open Atlas and create a new entity (e.g. hdfs_path) - check the debug metrics on both atlas instances using API call: https://:/api/atlas/admin/debug/metrics - The active instance returns correct metrics, whereas passive instance returns empty metrics The reason for this is that the debug metrics are maintained in-memory in the active server. When "admin/debug/metrics" API call is accessed on the passive server, the request isn't redirected to the active server like it does for the other APIs, hence the passive node returns empty metrics. > Discrepancy in the atlas debug metrics between the active and the not active > servers > > > Key: ATLAS-4691 > URL: https://issues.apache.org/jira/browse/ATLAS-4691 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.2.0 >Reporter: Jayendra Parab >Assignee: Jayendra Parab >Priority: Major > > There is a discrepancy in the atlas debug metrics between the active and the > not active servers. > Steps to reproduce: > - Setup Atlas in HA mode with two instances of Atlas > - add "atlas.debug.metrics.enabled=true" to the Atlas configuration > (conf/atlas-application.properties) > - restart Atlas > - open Atlas and create a new entity (e.g. hdfs_path) > - check the debug metrics on both atlas instances using API call: > https://:/api/atlas/admin/debug/metrics > - The active instance returns correct metrics, whereas passive instance > returns empty metrics > The reason for this is that the debug metrics are maintained in-memory in the > active server. When "admin/debug/metrics" API call is accessed on the passive > server, the request isn't redirected to the active server like it does for > the other APIs, hence the passive node returns empty metrics. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Assigned] (ATLAS-4691) Discrepancy in the atlas debug metrics between the active and the not active servers
[ https://issues.apache.org/jira/browse/ATLAS-4691?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab reassigned ATLAS-4691: - Assignee: Jayendra Parab > Discrepancy in the atlas debug metrics between the active and the not active > servers > > > Key: ATLAS-4691 > URL: https://issues.apache.org/jira/browse/ATLAS-4691 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Affects Versions: 2.2.0 >Reporter: Jayendra Parab >Assignee: Jayendra Parab >Priority: Major > > There is a discrepancy in the atlas debug metrics between the active and the > not active servers. > Steps to reproduce: > - Setup Atlas in HA mode with two instances of Atlas > - add "atlas.debug.metrics.enabled=true" to the Atlas configuration > (conf/atlas-application.properties_role_safety_valve) > - restart Atlas > - open Atlas and create a new entity (e.g. hdfs_path) > - check the debug metrics on both atlas instances using API call: > https://:/api/atlas/admin/debug/metrics > - The active instance returns correct metrics, whereas passive instance > returns empty metrics > The reason for this is that the debug metrics are maintained in-memory in the > active server. When "admin/debug/metrics" API call is accessed on the passive > server, the request isn't redirected to the active server like it does for > the other APIs, hence the passive node returns empty metrics. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Created] (ATLAS-4691) Discrepancy in the atlas debug metrics between the active and the not active servers
Jayendra Parab created ATLAS-4691: - Summary: Discrepancy in the atlas debug metrics between the active and the not active servers Key: ATLAS-4691 URL: https://issues.apache.org/jira/browse/ATLAS-4691 Project: Atlas Issue Type: Bug Components: atlas-core Affects Versions: 2.2.0 Reporter: Jayendra Parab There is a discrepancy in the atlas debug metrics between the active and the not active servers. Steps to reproduce: - Setup Atlas in HA mode with two instances of Atlas - add "atlas.debug.metrics.enabled=true" to the Atlas configuration (conf/atlas-application.properties_role_safety_valve) - restart Atlas - open Atlas and create a new entity (e.g. hdfs_path) - check the debug metrics on both atlas instances using API call: https://:/api/atlas/admin/debug/metrics - The active instance returns correct metrics, whereas passive instance returns empty metrics The reason for this is that the debug metrics are maintained in-memory in the active server. When "admin/debug/metrics" API call is accessed on the passive server, the request isn't redirected to the active server like it does for the other APIs, hence the passive node returns empty metrics. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Commented] (ATLAS-4459) Fix solr indexes for testcases failures
[ https://issues.apache.org/jira/browse/ATLAS-4459?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17432254#comment-17432254 ] Jayendra Parab commented on ATLAS-4459: --- +1 > Fix solr indexes for testcases failures > --- > > Key: ATLAS-4459 > URL: https://issues.apache.org/jira/browse/ATLAS-4459 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Pinal Shah >Assignee: Pinal Shah >Priority: Major > Labels: Testing > Attachments: ATLAS-4459.patch > > > Fixing solr indexes which are used for the testcases in solrConfig.xml > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4458) Commons-Logging Exclusion Causes Startup Problems
[ https://issues.apache.org/jira/browse/ATLAS-4458?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17431443#comment-17431443 ] Jayendra Parab commented on ATLAS-4458: --- +1 > Commons-Logging Exclusion Causes Startup Problems > - > > Key: ATLAS-4458 > URL: https://issues.apache.org/jira/browse/ATLAS-4458 > Project: Atlas > Issue Type: Bug >Reporter: Ashutosh Mestry >Assignee: Ashutosh Mestry >Priority: Major > Attachments: ATLAS-4458-Commons-logging-reference-fix.patch > > > Earlier commit of ATLAS-4351, caused _commons-logging*.jar_ to be excluded. > This causes startup problem due to missing dependency. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4278) Deferred Actions : Deleting a tag after disassociating it from a table which propagated fails
[
https://issues.apache.org/jira/browse/ATLAS-4278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17342286#comment-17342286
]
Jayendra Parab commented on ATLAS-4278:
---
The exception seen in the logs at the time of deletion of classification is :
{code:java}
2021-05-09 01:24:23,250 ERROR - [etp1992550266-179 -
55d97d73-3c4f-4985-81e2-696b41a3df1e:] ~ graph rollback due to exception
(GraphTransactionInterceptor:200)
org.apache.atlas.exception.AtlasBaseException: Given type tag1 has references
at
org.apache.atlas.repository.store.graph.v2.AtlasClassificationDefStoreV2.preDeleteByGuid(AtlasClassificationDefStoreV2.java:313)
at
org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.deleteTypesDef(AtlasTypeDefGraphStore.java:587)
at
org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore.deleteTypeByName(AtlasTypeDefGraphStore.java:713)
at
org.apache.atlas.repository.store.graph.AtlasTypeDefGraphStore$$FastClassBySpringCGLIB$$5226c80b.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
at
org.springframework.aop.framework.CglibAopProxy$CglibMethodInvocation.invokeJoinpoint(CglibAopProxy.java:737)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:157)
at
org.apache.atlas.GraphTransactionInterceptor.invoke(GraphTransactionInterceptor.java:111)
at
org.springframework.aop.framework.ReflectiveMethodInvocation.proceed(ReflectiveMethodInvocation.java:179)
at
org.springframework.aop.framework.CglibAopProxy$DynamicAdvisedInterceptor.intercept(CglibAopProxy.java:672)
at
org.apache.atlas.repository.store.graph.v2.AtlasTypeDefGraphStoreV2$$EnhancerBySpringCGLIB$$cc451202.deleteTypeByName()
at
org.apache.atlas.web.rest.TypesREST.deleteAtlasTypeByName(TypesREST.java:478)
at
org.apache.atlas.web.rest.TypesREST$$FastClassBySpringCGLIB$$1c2e37c6.invoke()
at org.springframework.cglib.proxy.MethodProxy.invoke(MethodProxy.java:204)
{code}
> Deferred Actions : Deleting a tag after disassociating it from a table which
> propagated fails
> -
>
> Key: ATLAS-4278
> URL: https://issues.apache.org/jira/browse/ATLAS-4278
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Jayendra Parab
>Assignee: Jayendra Parab
>Priority: Major
>
> # Create a lineage : table1 ---> process1 > table2
> # Add tag tag1 to table1.
> # Wait for task to complete, it propagates to table2
> # Disassociate tag from table1
> # Wait for task to complete , it is removed from all tables and process.
> # Attempt to delete the tag.
> # Tag deletion fails with `tag1` has references , though the tag is not
> associated to any entity.
> If the tag was added to entity without propagate flag set to False , and then
> disassociated and deleted , it is deleted successfully.
> This issue is seen only with deferred actions enabled.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-4279) Precommit is failing for DebugMetricsIT testcase
Jayendra Parab created ATLAS-4279: - Summary: Precommit is failing for DebugMetricsIT testcase Key: ATLAS-4279 URL: https://issues.apache.org/jira/browse/ATLAS-4279 Project: Atlas Issue Type: Bug Reporter: Jayendra Parab Assignee: Jayendra Parab DebugMetricsIT test case in introduced as part of commit for https://issues.apache.org/jira/browse/ATLAS-4106, is failing because the debug metrics are disabled by default and the test case expects the metrics to be enabled -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-4278) Deferred Actions : Deleting a tag after disassociating it from a table which propagated fails
Jayendra Parab created ATLAS-4278: - Summary: Deferred Actions : Deleting a tag after disassociating it from a table which propagated fails Key: ATLAS-4278 URL: https://issues.apache.org/jira/browse/ATLAS-4278 Project: Atlas Issue Type: Bug Affects Versions: 2.1.0 Reporter: Jayendra Parab Assignee: Jayendra Parab # Create a lineage : table1 ---> process1 > table2 # Add tag tag1 to table1. # Wait for task to complete, it propagates to table2 # Disassociate tag from table1 # Wait for task to complete , it is removed from all tables and process. # Attempt to delete the tag. # Tag deletion fails with `tag1` has references , though the tag is not associated to any entity. If the tag was added to entity without propagate flag set to False , and then disassociated and deleted , it is deleted successfully. This issue is seen only with deferred actions enabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4263) KafkaUtils sets invalid dynamic JAAS config
[
https://issues.apache.org/jira/browse/ATLAS-4263?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17338882#comment-17338882
]
Jayendra Parab commented on ATLAS-4263:
---
+1 for the patch
> KafkaUtils sets invalid dynamic JAAS config
> ---
>
> Key: ATLAS-4263
> URL: https://issues.apache.org/jira/browse/ATLAS-4263
> Project: Atlas
> Issue Type: Task
> Components: atlas-core
>Affects Versions: 2.1.0, 3.0.0
>Reporter: Vladislav Glinskiy
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
> Time Spent: 40m
> Remaining Estimate: 0h
>
> [KafkaUtils|https://github.com/apache/atlas/blob/8d3c4ab0e8844f04e29f66acb3577e9d40de9a16/common/src/main/java/org/apache/atlas/utils/KafkaUtils.java#L195]
> doesn't always
> [enclose|https://github.com/apache/atlas/blob/8d3c4ab0e8844f04e29f66acb3577e9d40de9a16/common/src/main/java/org/apache/atlas/utils/KafkaUtils.java#L316]
> property values in double-quotes, thus, setting invalid dynamic JAAS config
> for token auth in some cases.
> Faced this issue with Spark Atlas Connector trying to configure Atlas client
> to use delegation tokens. The following configuration is not handled properly:
> {code:java}
> atlas.jaas.KafkaClient.option.username=30CQ4q1hQMy0dB6X0eXfxQ
> atlas.jaas.KafkaClient.option.password=KdaUQ4FlKWlDxwQrAeFGUVbb6sR0P+zoqOZDZjtIRP1wseXbSbhiTjz3QI9Ur9o4LTYZSv8TE1QqUC4FSwnoTA==
> {code}
> and results in the following error:
> {code:java}
> java.lang.IllegalArgumentException: Value not specified for key 'null' in
> JAAS config
> at
> org.apache.kafka.common.security.JaasConfig.parseAppConfigurationEntry(JaasConfig.java:116)
> at
> org.apache.kafka.common.security.JaasConfig.(JaasConfig.java:63)
> at
> org.apache.kafka.common.security.JaasContext.load(JaasContext.java:90)
> at
> org.apache.kafka.common.security.JaasContext.loadClientContext(JaasContext.java:84)
> {code}
> [KafkaUtils|https://github.com/apache/atlas/blob/8d3c4ab0e8844f04e29f66acb3577e9d40de9a16/common/src/main/java/org/apache/atlas/utils/KafkaUtils.java#L195]
> should always enclose property values in double-quotes, since unenclosed
> digits and '+' sign can not be parsed by Kafka
> [JaasConfig|https://github.com/apache/kafka/blob/2.0.0/clients/src/main/java/org/apache/kafka/common/security/JaasConfig.java#L116].
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4271) When unauthorised user is tries to create a relationship, message thrown is incorrect
[
https://issues.apache.org/jira/browse/ATLAS-4271?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayendra Parab updated ATLAS-4271:
--
Description:
When an unauthorized user (say user1) tries to create a relationship between
two entities (say type1 and type2), the error message thrown is
{code:java}
user1 is not authorized to perform create relationship-def{code}
The expected message should be
{code:java}
user1 is not authorized to perform read type-def of category ENTITY type1{code}
The current message should be thrown when an user with read-only access tries
to create a relationship.
was:
When an unauthorized user (say user1) tries to create a relationship between
two entities (say type1 and type2), the error message thrown is
user1 is not authorized to perform create relationship-def
The expected message should be
{code:java}
user1 is not authorized to perform read type-def of category ENTITY type1{code}
The current message should be thrown when an user with read-only access tries
to create a relationship.
> When unauthorised user is tries to create a relationship, message thrown is
> incorrect
> -
>
> Key: ATLAS-4271
> URL: https://issues.apache.org/jira/browse/ATLAS-4271
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Jayendra Parab
>Assignee: Jayendra Parab
>Priority: Major
>
> When an unauthorized user (say user1) tries to create a relationship between
> two entities (say type1 and type2), the error message thrown is
> {code:java}
> user1 is not authorized to perform create relationship-def{code}
> The expected message should be
> {code:java}
> user1 is not authorized to perform read type-def of category ENTITY
> type1{code}
> The current message should be thrown when an user with read-only access tries
> to create a relationship.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-4271) When unauthorised user is tries to create a relationship, message thrown is incorrect
Jayendra Parab created ATLAS-4271:
-
Summary: When unauthorised user is tries to create a relationship,
message thrown is incorrect
Key: ATLAS-4271
URL: https://issues.apache.org/jira/browse/ATLAS-4271
Project: Atlas
Issue Type: Bug
Affects Versions: 2.1.0
Reporter: Jayendra Parab
Assignee: Jayendra Parab
When an unauthorized user (say user1) tries to create a relationship between
two entities (say type1 and type2), the error message thrown is
user1 is not authorized to perform create relationship-def
The expected message should be
{code:java}
user1 is not authorized to perform read type-def of category ENTITY type1{code}
The current message should be thrown when an user with read-only access tries
to create a relationship.
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Assigned] (ATLAS-4206) Swagger: File upload APIs does not work
[ https://issues.apache.org/jira/browse/ATLAS-4206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab reassigned ATLAS-4206: - Assignee: Nikhil P Bonte > Swagger: File upload APIs does not work > --- > > Key: ATLAS-4206 > URL: https://issues.apache.org/jira/browse/ATLAS-4206 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Affects Versions: 2.1.0 >Reporter: Nikhil Bonte >Assignee: Nikhil P Bonte >Priority: Major > Labels: REST_API, swagger > > APIs affected: > POST /v2/glossary/import > POST /v2/entity/businessmetadata/import > > # Swagger UI for the above APIs shows two params _fileDetail_ & > _inputStream_ both of type _file_, the expectation is to have the only > _inputStream_ to upload a file. > # Param _inputStream_ have name as "_inputStream_", expected name is > "_file_". -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ATLAS-4259) Swagger: Improve Header validation
[ https://issues.apache.org/jira/browse/ATLAS-4259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab reassigned ATLAS-4259: - Assignee: Nikhil P Bonte > Swagger: Improve Header validation > --- > > Key: ATLAS-4259 > URL: https://issues.apache.org/jira/browse/ATLAS-4259 > Project: Atlas > Issue Type: Improvement > Components: atlas-core >Reporter: Nikhil Bonte >Assignee: Nikhil P Bonte >Priority: Major > Labels: REST_API, swagger > Attachments: ATLAS-4259-Swagger-Improve-Header-validation-v0.patch > > > This will add support for ATLAS-4064 in swagger. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ATLAS-4260) Swagger: click to expand API does not work properly in some cases
[ https://issues.apache.org/jira/browse/ATLAS-4260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab reassigned ATLAS-4260: - Assignee: Nikhil P Bonte > Swagger: click to expand API does not work properly in some cases > - > > Key: ATLAS-4260 > URL: https://issues.apache.org/jira/browse/ATLAS-4260 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Nikhil Bonte >Assignee: Nikhil P Bonte >Priority: Major > Labels: rest_api, swagger > > Click on API POST /v2/entity also expands API POST /v2/entity/bulk & vice > versa > > Root Cause: > This is due to the same id of div element of both APIs. > The swagger spec has "operationId" same for both API. > > Need to check if any other API has the same issue. > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (ATLAS-4250) Support HA for admin/task API
[ https://issues.apache.org/jira/browse/ATLAS-4250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17326467#comment-17326467 ] Jayendra Parab commented on ATLAS-4250: --- +1 for the patch > Support HA for admin/task API > - > > Key: ATLAS-4250 > URL: https://issues.apache.org/jira/browse/ATLAS-4250 > Project: Atlas > Issue Type: Improvement > Components: atlas-webui >Reporter: Nixon Rodrigues >Assignee: Nixon Rodrigues >Priority: Major > Attachments: ATLAS-4250.patch > > > Support HA for newly added *admin/task* API -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4055) Basic search doesn't work if the search text has special characters
[ https://issues.apache.org/jira/browse/ATLAS-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab updated ATLAS-4055: -- Fix Version/s: 2.2.0 3.0.0 > Basic search doesn't work if the search text has special characters > --- > > Key: ATLAS-4055 > URL: https://issues.apache.org/jira/browse/ATLAS-4055 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Jayendra Parab >Assignee: Jayendra Parab >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > If the user tries to perform a search with text which has special characters > mentioned in [Lucene > documentation|https://lucene.apache.org/core/6_5_1/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Escaping_Special_Characters], > then the results don't include entities matching the search text, even > though the document exists in Solr. > The special characters are escaped as part of fix provided in ATLAS-3983, but > it also needs to be enclosed the search text in double quotes -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4055) Basic search doesn't work if the search text has special characters
[ https://issues.apache.org/jira/browse/ATLAS-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab resolved ATLAS-4055. --- Resolution: Fixed > Basic search doesn't work if the search text has special characters > --- > > Key: ATLAS-4055 > URL: https://issues.apache.org/jira/browse/ATLAS-4055 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Jayendra Parab >Assignee: Jayendra Parab >Priority: Major > > If the user tries to perform a search with text which has special characters > mentioned in [Lucene > documentation|https://lucene.apache.org/core/6_5_1/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Escaping_Special_Characters], > then the results don't include entities matching the search text, even > though the document exists in Solr. > The special characters are escaped as part of fix provided in ATLAS-3983, but > it also needs to be enclosed the search text in double quotes -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-4055) Basic search doesn't work if the search text has special characters
[ https://issues.apache.org/jira/browse/ATLAS-4055?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab updated ATLAS-4055: -- Summary: Basic search doesn't work if the search text has special characters (was: Basic search doesn't work if the basic search with attributes text has special characters) > Basic search doesn't work if the search text has special characters > --- > > Key: ATLAS-4055 > URL: https://issues.apache.org/jira/browse/ATLAS-4055 > Project: Atlas > Issue Type: Bug >Affects Versions: 2.1.0 >Reporter: Jayendra Parab >Assignee: Jayendra Parab >Priority: Major > > If the user tries to perform a search with text which has special characters > mentioned in [Lucene > documentation|https://lucene.apache.org/core/6_5_1/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Escaping_Special_Characters], > then the results don't include entities matching the search text, even > though the document exists in Solr. > The special characters are escaped as part of fix provided in ATLAS-3983, but > it also needs to be enclosed the search text in double quotes -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-4055) Basic search doesn't work if the basic search with attributes text has special characters
Jayendra Parab created ATLAS-4055: - Summary: Basic search doesn't work if the basic search with attributes text has special characters Key: ATLAS-4055 URL: https://issues.apache.org/jira/browse/ATLAS-4055 Project: Atlas Issue Type: Bug Affects Versions: 2.1.0 Reporter: Jayendra Parab Assignee: Jayendra Parab If the user tries to perform a search with text which has special characters mentioned in [Lucene documentation|https://lucene.apache.org/core/6_5_1/queryparser/org/apache/lucene/queryparser/classic/package-summary.html#Escaping_Special_Characters], then the results don't include entities matching the search text, even though the document exists in Solr. The special characters are escaped as part of fix provided in ATLAS-3983, but it also needs to be enclosed the search text in double quotes -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Assigned] (ATLAS-4011) Docker: While running Atlas Docker container, data should be stored on host
[ https://issues.apache.org/jira/browse/ATLAS-4011?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab reassigned ATLAS-4011: - Assignee: Jayendra Parab > Docker: While running Atlas Docker container, data should be stored on host > --- > > Key: ATLAS-4011 > URL: https://issues.apache.org/jira/browse/ATLAS-4011 > Project: Atlas > Issue Type: Improvement >Reporter: Jayendra Parab >Assignee: Jayendra Parab >Priority: Major > > Currently, if Atlas is started with Embedded Solr and HBase in Docker > container, the data is persisted inside the container. So if the container is > destroyed and a new container is started, then its old data is lost. > The data directory should be mounted as a volume, so that data can be > restored across container destroy and restore. > To achieve this, Solr data directory should be configurable and by default it > should be $ATLAS_HOME/data/solr. HBase and Kafka data directory is already > configured to $ATLAS_HOME/data -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Created] (ATLAS-4011) Docker: While running Atlas Docker container, data should be stored on host
Jayendra Parab created ATLAS-4011: - Summary: Docker: While running Atlas Docker container, data should be stored on host Key: ATLAS-4011 URL: https://issues.apache.org/jira/browse/ATLAS-4011 Project: Atlas Issue Type: Improvement Reporter: Jayendra Parab Currently, if Atlas is started with Embedded Solr and HBase in Docker container, the data is persisted inside the container. So if the container is destroyed and a new container is started, then its old data is lost. The data directory should be mounted as a volume, so that data can be restored across container destroy and restore. To achieve this, Solr data directory should be configurable and by default it should be $ATLAS_HOME/data/solr. HBase and Kafka data directory is already configured to $ATLAS_HOME/data -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3864) Atlas should use AdminClient where possible
[ https://issues.apache.org/jira/browse/ATLAS-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab updated ATLAS-3864: -- Affects Version/s: 2.1.0 > Atlas should use AdminClient where possible > --- > > Key: ATLAS-3864 > URL: https://issues.apache.org/jira/browse/ATLAS-3864 > Project: Atlas > Issue Type: Improvement >Affects Versions: 2.1.0 >Reporter: Viktor Somogyi-Vass >Assignee: Jayendra Parab >Priority: Major > > Zookeeper access should be restricted wherever possible as going forward > Kafka won't use Zookeeper (see > [KIP-500|https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum]). > Also the Zookeeper classes in Kafka aren't public apis so usage should be > avoided wherever possible. As a tactical fix (see ATLAS-3910) though it is > possible to continue using KafkaZkClient as that class likely won't change > much until the completion of KIP-500. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Updated] (ATLAS-3864) Atlas should use AdminClient where possible
[ https://issues.apache.org/jira/browse/ATLAS-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab updated ATLAS-3864: -- Fix Version/s: 2.2.0 3.0.0 > Atlas should use AdminClient where possible > --- > > Key: ATLAS-3864 > URL: https://issues.apache.org/jira/browse/ATLAS-3864 > Project: Atlas > Issue Type: Improvement >Affects Versions: 2.1.0 >Reporter: Viktor Somogyi-Vass >Assignee: Jayendra Parab >Priority: Major > Fix For: 3.0.0, 2.2.0 > > > Zookeeper access should be restricted wherever possible as going forward > Kafka won't use Zookeeper (see > [KIP-500|https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum]). > Also the Zookeeper classes in Kafka aren't public apis so usage should be > avoided wherever possible. As a tactical fix (see ATLAS-3910) though it is > possible to continue using KafkaZkClient as that class likely won't change > much until the completion of KIP-500. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-3864) Atlas should use AdminClient where possible
[ https://issues.apache.org/jira/browse/ATLAS-3864?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jayendra Parab resolved ATLAS-3864. --- Assignee: Jayendra Parab Resolution: Fixed Fixed as part of https://reviews.apache.org/r/72671/ > Atlas should use AdminClient where possible > --- > > Key: ATLAS-3864 > URL: https://issues.apache.org/jira/browse/ATLAS-3864 > Project: Atlas > Issue Type: Improvement >Reporter: Viktor Somogyi-Vass >Assignee: Jayendra Parab >Priority: Major > > Zookeeper access should be restricted wherever possible as going forward > Kafka won't use Zookeeper (see > [KIP-500|https://cwiki.apache.org/confluence/display/KAFKA/KIP-500%3A+Replace+ZooKeeper+with+a+Self-Managed+Metadata+Quorum]). > Also the Zookeeper classes in Kafka aren't public apis so usage should be > avoided wherever possible. As a tactical fix (see ATLAS-3910) though it is > possible to continue using KafkaZkClient as that class likely won't change > much until the completion of KIP-500. -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Resolved] (ATLAS-4005) DSL search gives error if select clause contains attributes with null values
[
https://issues.apache.org/jira/browse/ATLAS-4005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayendra Parab resolved ATLAS-4005.
---
Fix Version/s: 2.2.0
3.0.0
Assignee: Jayendra Parab
Resolution: Fixed
Submitted the fix in this review request
https://reviews.apache.org/r/72987/
> DSL search gives error if select clause contains attributes with null values
>
>
> Key: ATLAS-4005
> URL: https://issues.apache.org/jira/browse/ATLAS-4005
> Project: Atlas
> Issue Type: Bug
>Reporter: Jayendra Parab
>Assignee: Jayendra Parab
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
>
> When we do DSL search in the atlas, if one or more attributes in select
> clause are not set for a vertex then the below given exception is thrown:
>
> {code:java}
> java.lang.IllegalStateException: The property does not exist as the key has
> no associated value for the provided element
> {code}
> Example, in the query below, if owner isn't set for any of the entities, but
> name and description are present then the above exception is thrown
> {code:java}
> from hdfs_path where name="testPath" select name, owner, description{code}
> Expected Behaviour: Query should return name and description of the filtered
> entities with owner as blank
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Updated] (ATLAS-4005) DSL search gives error if select clause contains attributes with null values
[
https://issues.apache.org/jira/browse/ATLAS-4005?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Jayendra Parab updated ATLAS-4005:
--
Affects Version/s: 2.1.0
> DSL search gives error if select clause contains attributes with null values
>
>
> Key: ATLAS-4005
> URL: https://issues.apache.org/jira/browse/ATLAS-4005
> Project: Atlas
> Issue Type: Bug
>Affects Versions: 2.1.0
>Reporter: Jayendra Parab
>Assignee: Jayendra Parab
>Priority: Major
> Fix For: 3.0.0, 2.2.0
>
>
> When we do DSL search in the atlas, if one or more attributes in select
> clause are not set for a vertex then the below given exception is thrown:
>
> {code:java}
> java.lang.IllegalStateException: The property does not exist as the key has
> no associated value for the provided element
> {code}
> Example, in the query below, if owner isn't set for any of the entities, but
> name and description are present then the above exception is thrown
> {code:java}
> from hdfs_path where name="testPath" select name, owner, description{code}
> Expected Behaviour: Query should return name and description of the filtered
> entities with owner as blank
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-4005) DSL search gives error if select clause contains attributes with null values
Jayendra Parab created ATLAS-4005:
-
Summary: DSL search gives error if select clause contains
attributes with null values
Key: ATLAS-4005
URL: https://issues.apache.org/jira/browse/ATLAS-4005
Project: Atlas
Issue Type: Bug
Reporter: Jayendra Parab
When we do DSL search in the atlas, if one or more attributes in select clause
are not set for a vertex then the below given exception is thrown:
{code:java}
java.lang.IllegalStateException: The property does not exist as the key has no
associated value for the provided element
{code}
Example, in the query below, if owner isn't set for any of the entities, but
name and description are present then the above exception is thrown
{code:java}
from hdfs_path where name="testPath" select name, owner, description{code}
Expected Behaviour: Query should return name and description of the filtered
entities with owner as blank
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
[jira] [Created] (ATLAS-3919) Handling classification propagation as deferred-action
Jayendra Parab created ATLAS-3919: - Summary: Handling classification propagation as deferred-action Key: ATLAS-3919 URL: https://issues.apache.org/jira/browse/ATLAS-3919 Project: Atlas Issue Type: Improvement Components: atlas-core Reporter: Jayendra Parab Assignee: Jayendra Parab Currently, whenever a user assigns a tag or updates a tag on an entity, it gets propagated to all the entities derived from the tagged entity. This operation takes quite a lot of time to complete (sometimes into minutes) and causes usability issues on the UI and other clients invoking the REST API To resolve this issue, tag-propagation needs to be handled as deferred-action, so that time consuming like propagation, audits & notifications can be processed in background threads -- This message was sent by Atlassian Jira (v8.3.4#803005)
