Ryan19929 opened a new pull request, #63831:
URL: https://github.com/apache/doris/pull/63831
Prevent optional dynamic partition storage properties from creating
incomplete metadata on non-dynamic tables, and tolerate historical incomplete
dynamic partition properties during table metadata copy.
### What problem does this PR solve?
Issue Number: close #xxx
Related PR: #xxx
Problem Summary:
Some historical table metadata may contain incomplete dynamic partition
properties, for example only `dynamic_partition.storage_medium` or
`dynamic_partition.storage_policy`, while the table is not actually a dynamic
partition table.
When backup, CCR backup snapshot, or other metadata copy paths call
`OlapTable.selectiveCopy()`, FE rebuilds `TableProperty` through Gson deep
copy. `TableProperty.executeBuildDynamicProperty()` then constructs
`DynamicPartitionProperty` from these incomplete raw properties. Because
required fields such as `dynamic_partition.end`, `dynamic_partition.prefix`,
and `dynamic_partition.buckets` are missing, `DynamicPartitionProperty` may
parse a null value and throw `NumberFormatException: Cannot parse null string`.
This causes backup/CCR snapshot to fail with errors like `failed to copy
table`.
This PR fixes the problem in two ways:
1. Prevent new incomplete dynamic partition metadata from being written by
making `dynamic_partition.storage_medium` and
`dynamic_partition.storage_policy` participate in dynamic partition property
validation.
2. Tolerate historical incomplete dynamic partition metadata during
`TableProperty` rebuild by ignoring incomplete dynamic partition properties
instead of constructing an invalid `DynamicPartitionProperty`.
Note: This fix does not actively delete incomplete dynamic partition
properties from the underlying properties map; it only ignores them during
`DynamicPartitionProperty` construction to avoid exceptions.
### Release note
None
### Check List (For Author)
- Test <!-- At least one of them must be included. -->
- [ ] Regression test
- [ ] Unit Test
- [ ] Manual test (add detailed scripts or steps below)
- [ ] No need to test or manual test. Explain why:
- [ ] This is a refactor/code format and no logic has been changed.
- [ ] Previous test can cover this change.
- [ ] No code files have been changed.
- [ ] Other reason <!-- Add your reason? -->
- Behavior changed:
- [ ] No.
- [ ] Yes. <!-- Explain the behavior change -->
- Does this need documentation?
- [ ] No.
- [ ] Yes. <!-- Add document PR link here. eg:
https://github.com/apache/doris-website/pull/1214 -->
### Check List (For Reviewer who merge this PR)
- [ ] Confirm the release note
- [ ] Confirm test cases
- [ ] Confirm document
- [ ] Add branch pick label <!-- Add branch pick label that this PR should
merge into -->
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
---------------------------------------------------------------------
To unsubscribe, e-mail: [email protected]
For additional commands, e-mail: [email protected]
