ravjot28 opened a new pull request, #20595:
URL: https://github.com/apache/hudi/pull/20595
### Describe the issue this Pull Request addresses
Closes #20585
`ArchiveExecutorUtils.archive` sets
`withEmbeddedTimelineServerEnabled(false)` and then applies the caller's
`options`
last, so `CALL archive_commits(..., options =>
'hoodie.embed.timeline.server=true')` turns the server back on. With the
default marker type `TIMELINE_SERVER_BASED` the archiver then deletes the
leftover markers of each archived instant
through a timeline server that was never started, and the call fails with
`HoodieRemoteException: Failed to delete
marker directory ... Connection refused`.
### Summary and Changelog
The archive procedure no longer lets its options turn the embedded timeline
server on. With the default local file
system view, timeline-server-based markers fall back to direct markers and
the leftover marker directories of
archived instants are removed. Explicit remote-view settings keep their
existing behaviour.
- `ArchiveExecutorUtils.archive`: `withEmbeddedTimelineServerEnabled(false)`
moves after `withProps(options)`, with a
one-line comment; the earlier call, which the options overrode, is
removed. Every other option keeps its precedence.
- `TestArchiveCommitsProcedure`: archives a table at table version 6 and at
the current table version with
`hoodie.embed.timeline.server=true` and
`hoodie.write.markers.type=TIMELINE_SERVER_BASED` in `options`, and checks
that a leftover marker directory of an archived instant is deleted.
### Impact
No API or config change. `hoodie.embed.timeline.server` in the archive
procedure's options no longer has an effect;
the procedure never starts a timeline server.
**Backport:** please include this fix in the next 1.2.x patch release. The
commit cherry-picks cleanly onto
`release-1.2.1`.
### Risk Level
low. Only the one key is forced, and it could never work in this code path.
Tests, run on master with JDK 11, checkstyle (main and test sources) and
scalastyle on, with
`mvn test -pl hudi-spark-datasource/hudi-spark,hudi-cli -am
-Dtest=TestArchiveCommand -DwildcardSuites=...TestArchiveCommitsProcedure`:
- Without the main code change: the two new procedure cases ("with the
embedded timeline server enabled in options",
table versions 6 and 10) fail with `Connection refused` when the archiver
deletes a marker directory.
- With the change: `TestArchiveCommand` 1/1 and ScalaTest
`TestArchiveCommitsProcedure` 6/6 pass.
### Documentation Update
none
### Contributor's checklist
- [x] Read through [contributor's
guide](https://hudi.apache.org/contribute/how-to-contribute)
- [x] Enough context is provided in the sections above
- [x] Adequate tests were added if applicable
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]