GitHub user vinothchandar edited a discussion: Splitting hudi-common classes
into `common` and `core`
All , the hudi-common package has grown in scope. and I have spent sometime
with #19193 introducing a new `core` subpackage here to house all the core
Hudi timeline, filesystem view and the consistency/transactional stuff, as well
as key I/O. I would like `common` to be mostly holding common models, common
base infrastructure (functions, expressions, schema) used by everyother module.
Eventually, I want to split off `hudi-common` and `hudi-core` as separate
modules where something like `hudi-io` can depend on `hudi-common`. Today due
to this mixing of classes with different scope, the dependency is other way
around and we are keeping strictly I/O related classes in hudi-common without a
choice.
End state looks like this, with proper homing of classes.
```
hudi-common. // all base models and classes and utils , e.g schema,
enginecontext, expression, filegroup, fileslice. ...
|============> hudi-io // all core I/O classes, lower level (ideally
including reading/writing files, but can't have hadoop deps)
|================> hudi-core // everything around txns, timeline,
metadata, filesystemview, hudi specific reader/writers (single read/write
handle level like filegroupreader)
|============> hudi-hadoop-common // how it is today
```
Thoughts/suggestions welcome!
GitHub link: https://github.com/apache/hudi/discussions/19194
----
This is an automatically sent email for [email protected].
To unsubscribe, please send an email to: [email protected]
