[
https://issues.apache.org/jira/browse/HUDI-2176?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17383991#comment-17383991
]
ASF GitHub Bot commented on HUDI-2176:
--------------------------------------
nsivabalan opened a new pull request #3306:
URL: https://github.com/apache/hudi/pull/3306
## What is the purpose of the pull request
- Adding virtual keys support to COW table for all table operations (insert,
upsert, delete, etc)
- Fixed clustering
- Ensured metadata table supports virtual keys
Changes not covered in this PR:
- Flink and java in not in the scope of this PR.
- Orc format is not covered.
- cli commands.
To discuss:
With virtual keys, we are imposing a constraint that keyGen for a given
table cannot change from the time of its inception. So, given this constraint,
should we add some validation in HoodieSparkSqlWriter or WriteClient so that
keyGen does not change overtime for a given table?
## Brief change log
- Made changes to all write handles to leverage keyGenerator instead of meta
fields to compute record keys and partition path.
- Added support to SimpleIndex to leverage keyGen instead of meta fields to
fetch record key and partition path.
- Moved BaseKeyGenerator and other supported interfaces (KeyGenerator,
KeyGeneratorInterface) to hudi-common as to use it across all modules and
classes.
- Fixed most tests in TestHoodieClientCopyOnWriteStorage and
TestHoodieBackedMetadata
## Verify this pull request
This change added tests and can be verified as follows:
- Fixed TestHoodieClientCopyOnWriteStorage and TestHoodieBackedMetadata to
test tables w/ virtual keys enabled
## Committer checklist
- [ ] Has a corresponding JIRA in PR title & commit
- [ ] Commit message is descriptive of the change
- [ ] CI is green
- [ ] Necessary doc changes done or have another open PR
- [ ] For large changes, please consider breaking it into sub-tasks under
an umbrella JIRA.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: commits-unsubscr...@hudi.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
> Virutal keys support for COW all operations
> -------------------------------------------
>
> Key: HUDI-2176
> URL: https://issues.apache.org/jira/browse/HUDI-2176
> Project: Apache Hudi
> Issue Type: Improvement
> Components: Writer Core
> Reporter: sivabalan narayanan
> Priority: Major
> Labels: pull-request-available
> Fix For: 0.9.0
>
>
> Virutal keys support for COW all operations
> (merge handle)
--
This message was sent by Atlassian Jira
(v8.3.4#803005)
