morningman commented on code in PR #69224:
URL: https://github.com/apache/doris/pull/69224#discussion_r4308089782
##########
docker/thirdparties/docker-compose/iceberg/iceberg.yaml.tpl:
##########
@@ -35,8 +35,8 @@ services:
- ./spark-defaults.conf:/opt/spark/conf/spark-defaults.conf
-
./data/input/jars/iceberg-aws-bundle-1.10.1.jar:/opt/spark/jars/iceberg-aws-bundle-1.10.1.jar
-
./data/input/jars/iceberg-spark-runtime-4.0_2.13-1.10.1.jar:/opt/spark/jars/iceberg-spark-runtime-4.0_2.13-1.10.1.jar
- -
./data/input/jars/paimon-s3-1.3.1.jar:/opt/spark/jars/paimon-s3-1.3.1.jar
- -
./data/input/jars/paimon-spark-4.0-1.3.1.jar:/opt/spark/jars/paimon-spark-4.0-1.3.1.jar
+ -
./data/input/jars/paimon-s3-1.4.2.jar:/opt/spark/jars/paimon-s3-1.4.2.jar
+ -
./data/input/jars/paimon-spark-4.0_2.13-1.4.2.jar:/opt/spark/jars/paimon-spark-4.0_2.13-1.4.2.jar
Review Comment:
Fixed in `10327556968`. Agreed on the mechanism: `start_iceberg` downloads
the archive only when `data/` is absent, so an environment extracted from the
previous archive kept only the Paimon 1.3.1 jars, and Compose created empty
directories for the new 1.4.2 mount sources. CI starts from a clean environment
on every run, which is why the External pipeline passed; local environments
stayed broken until `data/` was removed by hand. branch-4.x and the earlier
archive switch in #61149 have the same gap.
`start_iceberg` now checks every jar that `iceberg.yaml` mounts from
`data/input/jars` before it decides whether to download. If one is missing, or
is a directory Compose created, the stale `data/` is removed and the current
archive is extracted again, the same as a fresh install. A later jar change in
`iceberg.yaml` is picked up the same way.
Checked on four layouts: all mounted jars present (kept), a `data/` from the
old archive (re-extracted), a jar path that Compose had created as a directory
(re-extracted), and no `data/` (downloaded as before).
`run-thirdparties-docker.sh -c iceberg` against an up-to-date `data/` keeps it
and starts the stack as before.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
