2MD opened a new issue, #12187:
URL: https://github.com/apache/iceberg/issues/12187
### Query engine
Iceberg version 1.7.1
Spark version 3.3.2
### Question
We have spark streaming application **A1** which in every microbatch does:
```
s"""
|MERGE INTO table as t
|USING (select * from $tempViewName) as s
|ON $joinCondition
|WHEN MATCHED AND s.$versionColumnName > t.$versionColumnName THEN
UPDATE SET *
|WHEN NOT MATCHED THEN INSERT *
|"""
```
**We can update any data file.**
**Table "table" without partition.**
And we have some spark batch application **A2** which does:
remove old snapshot, rewrite manifest , compaction (binpack and sometimes
z-order), rewrite position delete files, delete orphan files.
For this table.
How we can avoid concurrent troubles between two application?
(We are still thinking about launching **A2** inside mircobatch **A1**...
but is not the best solution)
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
