This is an automated email from the ASF dual-hosted git repository.
danny0405 pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/hudi.git
The following commit(s) were added to refs/heads/master by this push:
new b0d16f94b3b5 docs(rfc): close RFC-99 with a reduced scope (#20488)
b0d16f94b3b5 is described below
commit b0d16f94b3b5f4ce790a9a09e41ebf891b35a53e
Author: voonhous <[email protected]>
AuthorDate: Mon Nov 2 11:34:02 2026 +0800
docs(rfc): close RFC-99 with a reduced scope (#20488)
RFC-99 delivered HoodieSchema as Hudi's schema API, the phase 1
consolidation onto it, and the VARIANT, VECTOR and BLOB types. The
rest of its original plan is either owned elsewhere or not planned.
Add a Scope outcome section listing what was delivered and where
each remaining item moved: the outstanding logical types,
translation layers and Avro-free internals to RFC-88 (restructure
proposed), type ids under NBCC to RFC-104, and the Arrow in-memory
representation, metadata-table schema tracking and SQL type DDL to
future work. Keep the type and interoperability tables as the
reference specification, and mark RFC-99 COMPLETED in the index.
---
rfc/README.md | 2 +-
rfc/rfc-99/rfc-99.md | 37 +++++++++++++++++++++++++++++++------
2 files changed, 32 insertions(+), 7 deletions(-)
diff --git a/rfc/README.md b/rfc/README.md
index 4535e18bb400..9de43d567e15 100644
--- a/rfc/README.md
+++ b/rfc/README.md
@@ -134,7 +134,7 @@ The list of all RFCs can be found here.
| 96 | Introduce Unified Bucket Index
| :eyes:
`UNDER REVIEW` |
| 97 | Deprecate Hudi Payload Class Usage
| :eyes:
`UNDER REVIEW` |
| 98 | [Spark Datasource V2 Read](./rfc-98/rfc-98.md)
| :eyes:
`UNDER REVIEW` |
-| 99 | [Hudi Type System Redesign](./rfc-99/rfc-99.md)
| :eyes:
`UNDER REVIEW` |
+| 99 | [Hudi Type System Redesign](./rfc-99/rfc-99.md)
|
:white_check_mark: `COMPLETED` |
| 100 | [Unstructured Data Storage in Hudi](./rfc-100/rfc-100.md)
| :eyes:
`UNDER REVIEW` |
| 101 | [Updates to the HoodieRecordMerger API](./rfc-101/rfc-101.md)
|
:hammer_and_wrench: `IN PROGRESS` |
| 102 | [Spark Batch Vector Search in Apache
Hudi](./rfc-102/rfc-102/md)
| :white_check_mark: `COMPLETED` |
diff --git a/rfc/rfc-99/rfc-99.md b/rfc/rfc-99/rfc-99.md
index 281753fa1884..42f55e289607 100644
--- a/rfc/rfc-99/rfc-99.md
+++ b/rfc/rfc-99/rfc-99.md
@@ -30,8 +30,35 @@
## Status
+Completed with a reduced scope; see [Scope outcome](#scope-outcome).
+
Umbrella ticket: [HUDI-9730](https://issues.apache.org/jira/browse/HUDI-9730)
+## Scope outcome
+
+### Delivered
+
+- `HoodieSchema`, `HoodieSchemaField` and `HoodieSchemaType` in `hudi-common`
as Hudi's schema API
+ ([#14265](https://github.com/apache/hudi/pull/14265)), and phase 1 schema
consolidation of Avro `Schema` call sites
+ onto it ([#14263](https://github.com/apache/hudi/issues/14263)).
+- `VARIANT`, with unshredded and shredded storage; see [Variant Type
Implementation](#variant-type-implementation).
+- `VECTOR` and `BLOB` as `HoodieSchemaType` logical types. The VECTOR layout
is in
+ [vector-appendix.md](vector-appendix.md); BLOB semantics are owned by
[RFC-100](../rfc-100/rfc-100.md).
+- The [Type Specification](#type-specification) and [Interoperability
Mapping](#interoperability-mapping) tables
+ stay as the reference specification for each type and its mapping to other
systems.
+
+### Moved out of this RFC
+
+| Item | New home |
+|------|----------|
+| Logical types not yet in `HoodieSchemaType`: TINYINT, SMALLINT, unsigned
integers, FLOAT16, LARGE_STRING, LARGE_BINARY, DATE64, DURATION, DICTIONARY |
[RFC-88](https://github.com/apache/hudi/pull/12795) (restructure proposed): a
type lands together with its mappings to every engine, format and catalog, and
their tests |
+| Translation layers to and from Avro, Parquet, Spark, Flink and other systems
| RFC-88 |
+| Removing Avro from `HoodieSchema` internals | RFC-88 |
+| Type ids for schema evolution and their interplay with NBCC |
[RFC-104](https://github.com/apache/hudi/pull/18661) |
+| Arrow-based in-memory representation | Future work; not planned in any open
RFC |
+| Table schema tracked in the metadata table | Future work; not planned in any
open RFC |
+| SQL extensions for Hudi-native type DDL | Future work; not planned in any
open RFC |
+
## Abstract
The main goal is to propose a native Hudi type system as the authoritative
representation for Hudi data types, making the system more extensible and the
semantics of data types clear and unified. While Hudi currently uses Avro for
schema representation, introducing a more comprehensive, Arrow-based type
system will make it easier to provide consistent handling and implementation of
data types across different engines and improve support for modern data
paradigms like multi-modal and semi [...]
@@ -188,12 +215,10 @@ The following table defines the canonical mapping from
the proposed logical type
## Implementation
-A specific hudi core module "hudi-core-type" will define the above types. The
translation layer to and from other type-systems such as Avro, Spark, Flink,
Parquet,.. will reside in their own separate modules to keep the dependency
clean.
-
-The table schema itself will need to be tracked in metadata table.
-SQL Extensions needs to be added to define the table in a hudi type native
way.
-
-TODO: There is an open question regarding the need to maintain type ids to
track schema evolution and how it would interplay with NBCC.
+The types are defined by `HoodieSchemaType` in `hudi-common` rather than a
separate `hudi-core-type` module, and
+`HoodieSchema` currently wraps an Avro `Schema`. The rest of the original plan
(translation layers, Avro-free internals,
+table schema in the metadata table, SQL extensions and type ids under NBCC)
moved as listed in
+[Scope outcome](#scope-outcome).
---
