[jira] [Updated] (ATLAS-4847) Export/Import : Atlas export fails and throws NullPointerException.
[ https://issues.apache.org/jira/browse/ATLAS-4847?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4847: -- Fix Version/s: 3.0.0 2.3.0 > Export/Import : Atlas export fails and throws NullPointerException. > --- > > Key: ATLAS-4847 > URL: https://issues.apache.org/jira/browse/ATLAS-4847 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: ATLAS-4847.patch, NullPointerException_stackTrace.txt > > > Steps to reproduce: > 1. Create a hive_table entity and assign a term to it. > 2. Perform 1st incremental export of that entity without using changeMarker. > 3. Without any modification on that entity perform 2nd consecutive > incremental export with changeMarker from previous export. > Problem is it throws NullPointerException and export is failed. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4842) Export/Import: fetchType as "incremental" does full export instead of connected
[ https://issues.apache.org/jira/browse/ATLAS-4842?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4842: -- Fix Version/s: 3.0.0 2.3.0 > Export/Import: fetchType as "incremental" does full export instead of > connected > --- > > Key: ATLAS-4842 > URL: https://issues.apache.org/jira/browse/ATLAS-4842 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Sheetal Shah >Assignee: Priyanshi Shah >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: ATLAS-4842.patch > > > Create two hive_tables in same database: > # create table hive1(id int); > # create table hive1_ctas as select * from hive1; > # create table hive2(name string); > Perform incremental export of hive1 with changeMarker as 0 and skipLineage > value as false. > Import the zip. > Expectation is only the requested hive_table hive1 and its related entities > should be exported, but it does full export and hive2 is also imported. > -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4845) Atlas Import is failing with fetchType: Incremental if there are no changes between two consecutive runs
[ https://issues.apache.org/jira/browse/ATLAS-4845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Priyanshi Shah updated ATLAS-4845: -- Fix Version/s: 3.0.0 2.3.0 > Atlas Import is failing with fetchType: Incremental if there are no changes > between two consecutive runs > > > Key: ATLAS-4845 > URL: https://issues.apache.org/jira/browse/ATLAS-4845 > Project: Atlas > Issue Type: Bug > Components: atlas-core >Reporter: Priyanshi Shah >Assignee: Priyanshi Shah >Priority: Major > Fix For: 3.0.0, 2.3.0 > > Attachments: ATLAS-4845.patch > > > Repro steps: > # First export + import with changeMarker “0” - worked as expected. > # I tried to run 2nd export + import without any changes in atlas, 2nd > export also worked and gave different changeMarker in output as expected. > *But 2nd import failed* > In other fetch Types this will work. So we also require similar behaviour in > case of Incremental type also because this is very normal scenario and > customer will hit this very frequently. -- This message was sent by Atlassian Jira (v8.20.10#820010)
[jira] [Updated] (ATLAS-4838) Export/Import : changeMarker is not set to entity's lastupdatetime or its closer timestamp value
[
https://issues.apache.org/jira/browse/ATLAS-4838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Priyanshi Shah updated ATLAS-4838:
--
Fix Version/s: 3.0.0
2.3.0
> Export/Import : changeMarker is not set to entity's lastupdatetime or its
> closer timestamp value
>
>
> Key: ATLAS-4838
> URL: https://issues.apache.org/jira/browse/ATLAS-4838
> Project: Atlas
> Issue Type: Bug
> Components: atlas-core
>Reporter: Sharmadha S
>Assignee: Priyanshi Shah
>Priority: Major
> Fix For: 3.0.0, 2.3.0
>
> Attachments: ATLAS-4838-V2.patch
>
>
> When an hive table entity is exported using fetchType incremental with
> changeMarker 0 , after exporting , the changeMarker in the export response is
> not set to recent timestamp.
> For example , on exporting iceberg table with fetchType incremental and
> changeMarker as 0 :
> {code:java}
> 2024-02-08 13:28:43,838 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~
> export(item=AtlasObjectId{guid='null', typeName='iceberg_table',
> uniqueAttributes={qualifiedName:default.iceberg_table2@cm}}; matchType=null,
> fetchType=incremental): found 1 entities: options:
> {"itemsToExport":[{"typeName":"iceberg_table","uniqueAttributes":{"qualifiedName":"default.iceberg_table2@cm"}}],"options":{"fetchType":"incremental","changeMarker":0},"fetchTypeOptionValue":"incremental","skipLineageOptionValue":false,"changeTokenFromOptions":0}
> (StartEntityFetchByExportRequest:123)
> 2024-02-08 13:28:48,697 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ Assigned guid =
> f5e6342a-dfdf-4d54-a1b3-b24f987c4ecb to newly created entity (DataAccess:85)
> 2024-02-08 13:28:48,697 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ addAuditEntry: user: admin, source:
> cm, target: , operation: EXPORT (ExportImportAuditService:168)
> 2024-02-08 13:28:48,699 INFO - [etp891461509-20 -
> db3f1141-531c-46c0-909a-5e443d4353a2:] ~ <== export(user=admin,
> from=10.19.28.124): status SUCCESS: changeMarker: 1707225183928
> (ExportService:107) {code}
> changeMarker from response is 1707225183928 => Tuesday, February 6, 2024
> 6:43:03.928 PM
>
> But lastUpdatetime of iceberg table is
> {code:java}
> "createTime": 1707398866970 => Thursday, February 8, 2024 6:57:46.970 PM
> "updateTime": 1707398867375. => Thursday, February 8, 2024 6:57:47.375 PM
> {code}
> Value is set to 2 days back. This is not then an incremental export , as it
> will fetch all updates from Tuesday, February 6, 2024 6:43:03.928 PM in next
> export
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
