This is an automated email from the ASF dual-hosted git repository.
yihua pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/hudi-rs.git
The following commit(s) were added to refs/heads/main by this push:
new c3704673 docs: add a context7.json for coding-agent doc indexes (#941)
c3704673 is described below
commit c37046736a9f372a9854944295546ec8bfe73377
Author: Y Ethan Guo <[email protected]>
AuthorDate: Fri Oct 30 13:57:32 2026 -0700
docs: add a context7.json for coding-agent doc indexes (#941)
---
.licenserc.yaml | 1 +
AGENTS.md | 20 ++++++++++++++++++--
context7.json | 31 +++++++++++++++++++++++++++++++
crates/AGENTS.md | 11 ++++++-----
python/AGENTS.md | 10 +++++-----
5 files changed, 61 insertions(+), 12 deletions(-)
diff --git a/.licenserc.yaml b/.licenserc.yaml
index fb1c3bda..3fc784ef 100644
--- a/.licenserc.yaml
+++ b/.licenserc.yaml
@@ -23,6 +23,7 @@ header:
paths-ignore:
- 'LICENSE'
- 'NOTICE'
+ - 'context7.json'
- 'AGENTS.md'
- 'CLAUDE.md'
- '**/data/**'
diff --git a/AGENTS.md b/AGENTS.md
index 6fd5b0a5..4722e15e 100644
--- a/AGENTS.md
+++ b/AGENTS.md
@@ -9,8 +9,8 @@ Arrow-native, `object_store` for all I/O, timeline-based MVCC.
```
crates/
- core/ hudi-core — avro_to_arrow, config, expr, file_group, hfile,
keygen, merge, metadata,
- record, schema, statistics, storage, table, timeline, util
+ core/ hudi-core — avro_to_arrow, bloom, config, expr, file_group,
hfile, index, keygen,
+ merge, metadata, record, schema, statistics, storage, table,
timeline, util, write
datafusion/ hudi-datafusion — DataFusion TableProvider (feature:
datafusion)
ffi/ hudi-ffi — plain C ABI for JVM and C++ engine callers via the
Arrow C Data Interface
hudi/ public umbrella crate; re-exports core + (optional) datafusion
@@ -22,6 +22,22 @@ benchmark/filegroup/ file-group read benchmark (fg-bench /
fg-gen)
benchmark/upsert/ upsert/delete benchmark for hudi-rs writes (Python, run.sh)
```
+### User docs and resources
+
+Point users and coding agents at these; the Java Hudi docs cover Spark and
Flink, not this API.
+
+- Quick start: [Python/Rust quick start
guide](https://hudi.apache.org/docs/python-rust-quick-start-guide)
+ on the Hudi website, and the shorter [README quick
start](./README.md#quick-start).
+- Rust API reference: [docs.rs/hudi](https://docs.rs/hudi),
[docs.rs/hudi-core](https://docs.rs/hudi-core),
+ [docs.rs/hudi-datafusion](https://docs.rs/hudi-datafusion).
+- Python API reference: the docstrings in
[`python/hudi/_internal.pyi`](./python/hudi/_internal.pyi).
+- Runnable Rust example:
[`crates/core/examples/table_writes.rs`](./crates/core/examples/table_writes.rs).
+- Reader semantics: [`docs/reader-spec.md`](./docs/reader-spec.md). Writer
design:
+ [`docs/writer-design.md`](./docs/writer-design.md).
+- Packages: [PyPI](https://pypi.org/project/hudi/),
[crates.io](https://crates.io/crates/hudi).
+- Hudi concepts:
[hudi.apache.org/docs/overview](https://hudi.apache.org/docs/overview).
+- Agent doc index config: [`context7.json`](./context7.json).
+
## Commands
The [`Makefile`](./Makefile) is the canonical command surface — **prefer `make
<target>`** so agents
diff --git a/context7.json b/context7.json
new file mode 100644
index 00000000..fb23e593
--- /dev/null
+++ b/context7.json
@@ -0,0 +1,31 @@
+{
+ "$schema": "https://context7.com/schema/context7.json";,
+ "projectTitle": "Apache Hudi Rust and Python (hudi-rs)",
+ "description": "Native Rust implementation of Apache Hudi with Python and
C++ bindings: create, write (append, upsert, update, delete, overwrite), and
read Hudi tables as Arrow without a JVM.",
+ "excludeFolders": [
+ ".github",
+ "benchmark",
+ "crates/test",
+ "demo/infra",
+ "python/tests",
+ "release",
+ "scripts"
+ ],
+ "excludeFiles": [
+ "AGENTS.md",
+ "CHANGELOG.md",
+ "CONTRIBUTING.md"
+ ],
+ "rules": [
+ "Docs and resources: quick start guide
https://hudi.apache.org/docs/python-rust-quick-start-guide and the shorter
README quick start https://github.com/apache/hudi-rs#quick-start; Rust API
reference https://docs.rs/hudi (re-exports https://docs.rs/hudi-core and, with
the `datafusion` feature, https://docs.rs/hudi-datafusion); Python API
reference in the docstrings of the bundled type stubs
https://github.com/apache/hudi-rs/blob/main/python/hudi/_internal.pyi; runnable
Rust example ht [...]
+ "Versions: the write APIs (create, append, upsert, update, delete,
overwrite, compact) ship in hudi-rs 0.6; 0.5 and earlier only read. Python:
install with `pip install hudi` and import from `hudi` (`HudiTable`,
`HudiTableBuilder`, `HudiReadOptions`, `HudiQueryType`). Reads return a list of
`pyarrow.RecordBatch`; build a table with `pyarrow.Table.from_batches`.",
+ "Rust: depend on the `hudi` crate; its APIs are async (tokio). Entry
points are `hudi::table::Table` (`Table::create` to create,
`append`/`upsert`/`update`/`delete`/`delete_keys`/`overwrite` to write (each
returns a `WriteResult`, `HudiWriteResult` in Python, with `requested_time`,
`completion_time` and row counts; both times are `None` when the write made no
commit), `compact`/`await_compaction` to compact a merge-on-read table,
`read`/`read_stream` to read) and `hudi::table::builde [...]
+ "Write inputs: in Rust, `append`/`upsert`/`overwrite` take `RecordBatch`
iterators, `update` takes a filter plus a one-row `RecordBatch`, `delete` takes
a filter string, and `delete_keys` takes `HoodieKey`s (record key plus
partition path); in Python, `append`/`upsert`/`overwrite` take a PyArrow table,
record batch, reader, list of batches, or any object with `__arrow_c_stream__`
(polars, pandas >= 2.2), `update` takes a predicate plus a dict or one-row
batch, `delete` takes a predic [...]
+ "For an incremental read after a write, pass the write result's
`completion_time` (not `requested_time`) as the start timestamp: on table
version 8 and later the incremental window bounds commit completion time. The
start is exclusive and the end inclusive.",
+ "Compaction (experimental, merge-on-read only): upserts, updates and
deletes on a MOR table write log files; `Table::compact` (Rust) /
`HudiTable.compact()` (Python) folds them into new base files under one
compaction commit, and returns a result whose `requested_time` is `None` when
there is nothing to compact. To compact automatically once
`hoodie.compact.inline.max.delta.commits` (default 5) delta commits accumulate,
set `hoodie.compact.inline=true` (in the write call) or `hoodie. [...]
+ "Reads support table versions 6, 8 and 9; writes support 8 and 9 (plus a
narrow append path on 6). Both Copy-on-Write and Merge-on-Read tables are
supported.",
+ "To query with SQL, use DataFusion: the Rust `datafusion` feature provides
`hudi::HudiDataSource`; in Python, `HudiDataFusionDataSource` is always
available and the `hudi[datafusion]` extra installs the `datafusion` package.
Register it on a `SessionContext`.",
+ "Storage is chosen by the table URI's scheme (e.g. `s3://`, `az://`,
`gs://`). Credentials and storage settings come from `AWS_*`, `AZURE_*`,
`GOOGLE_*` environment variables or from table options such as `aws_region`.",
+ "Hudi configs are `hoodie.*` string keys; in Rust and Python prefer the
typed enums `HudiReadConfig`, `HudiTableConfig` and `HudiPlanConfig` over raw
strings."
+ ]
+}
diff --git a/crates/AGENTS.md b/crates/AGENTS.md
index eb853704..73482a3c 100644
--- a/crates/AGENTS.md
+++ b/crates/AGENTS.md
@@ -4,16 +4,17 @@
- **No `.unwrap()` / `.expect()` / `panic!()`** in non-test code (Critical).
`unreachable!()`
only with a comment justifying the invariant.
-- **Errors carry context** — typed `thiserror` variants or `anyhow::Context`;
messages name the
- offending value. Avoid bare `.map_err(Into::into)`.
+- **Errors carry context** — typed `thiserror` variants (`CoreError` in
`hudi-core`) whose
+ messages name the offending value. Avoid bare `.map_err(Into::into)`.
```rust
// GOOD — with context
-let file = File::open(&path)
- .with_context(|| format!("Failed to open table metadata at {}",
path.display()))?;
+let version: isize = value
+ .parse()
+ .map_err(|e| CoreError::InvalidValue(format!("table version '{value}' is
not a number: {e}")))?;
// BAD — no context
-let file = File::open(&path).map_err(|e| HudiError::Io(e))?;
+let version: isize = value.parse().map_err(|e|
CoreError::InvalidValue(e.to_string()))?;
```
## Async
diff --git a/python/AGENTS.md b/python/AGENTS.md
index 158bd622..5b77d7d3 100644
--- a/python/AGENTS.md
+++ b/python/AGENTS.md
@@ -18,7 +18,7 @@ Convert Rust errors to specific Python exceptions; never let
a panic surface to
#[pyfunction]
fn read_table(path: &str) -> PyResult<PyObject> {
let result = hudi_core::read_table(path)
- .map_err(|e| PyRuntimeError::new_err(format!("Failed to read table:
{e}")))?;
+ .map_err(convert_to_py_err)?; // raises hudi.HudiCoreError, a subclass
of Exception
// ...
}
@@ -32,11 +32,11 @@ fn read_table(path: &str) -> PyObject {
### GIL Management
-Release the GIL with `py.allow_threads(...)` for blocking I/O.
+Release the GIL with `py.detach(...)` (PyO3's former `allow_threads`) for
blocking I/O.
```rust
fn read_files(&self, py: Python<'_>, paths: Vec<String>) ->
PyResult<Vec<PyObject>> {
- let batches = py.allow_threads(|| {
+ let batches = py.detach(|| {
self.inner.read_files_blocking(&paths)
})?;
// ...
@@ -53,7 +53,7 @@ Tests in `python/tests/`.
```python
import pytest
-from hudi import HudiReadOptions, HudiTableBuilder
+from hudi import HudiCoreError, HudiReadOptions, HudiTableBuilder
def test_read_snapshot_with_filters():
@@ -63,6 +63,6 @@ def test_read_snapshot_with_filters():
def test_invalid_path_raises():
- with pytest.raises(RuntimeError, match="Failed to"):
+ with pytest.raises(HudiCoreError):
HudiTableBuilder.from_base_uri("/nonexistent").build()
```
