Hi everyone, I’ve updated the “Python Client API” section and also the accompanying example to match it. The connection config is now a Map, giving users a more flexible, dynamic setup. Thanks for all the great feedback—if there are no further comments, I’ll open the vote shortly. Best regards, Jim
> 2025年8月12日 21:49，Jim Hu <[email protected]> 写道： > > Hi, Jark. > > Thanks for your detailed review. I'll address all of your concerns. > > 1. The FIP’s specification intentionally outlines the complete > interface—including Arrow, Pandas, and DuckDB helpers—to ensure forward > compatibility as the client evolves. The initial release, however, will be > limited to bounded batch operations; it will expose only the scan method with > an explicit end_timestamp and the corresponding to_arrow and to_pandas > conversions. > > 2. AppendWriter.write_arrow is asynchronous. It returns a coroutine that > completes only after the Arrow table is durably appended, similar to a Java > CompletableFuture. > > 3. scan returns a lazy iterator that never materialises the entire log. > to_arrow() collects the iterator into a single PyArrow Table—use it only when > the data is small enough to fit in memory. > to_arrow_batch() yields a RecordBatchReader that streams batches > incrementally, keeping memory pressure low and matching the underlying Rust > scanner’s streaming semantics. > > 4. The second parametre was intended to be a timeout in ms. However, after > reviewing, we only have these configurations: bootstrap_server, > request_max_size, writer_acks, writer_retries, and writer_batch_size. I have > revised the Python demo to only contain the bootstrap_server. > > Best, > Jim > > Jark Wu <[email protected] <mailto:[email protected]>> 于2025年8月12日周二 11:33写道： >> Thanks Jim for proposing the python and rust client. >> >> The overall design looks good to me. +1 for Rust bindings (via PyO3 or >> similar) over solutions like Py4J or a separate Python-native >> implementation. >> >> This aligns with the current industry standard for high-performance Python >> integrations, like Lance, Iceberg, and DeltaLake. Leveraging Rust bindings >> will not only deliver better performance but also allow us to maximize code >> reuse from the existing Rust client, reducing maintenance overhead and >> ensuring consistency across language interfaces. >> >> Regarding the FIP, I have some following comments: >> >> 1. Could you clarify the intended scope of this FIP? Specifically, does it >> include integration with PyArrow, Pandas, and DuckDB? The example shows >> DuckDB integration, but it’s also listed under Future Work. >> >> 2. What's the semantics of AppendWriter#write_arrow? Is it a blocking API >> or a non-blocking API? If it is a non-blocking API, can we have something >> like Java CompletableFuture as the return type to allow users when the call >> is successful? >> >> 3. What's the behavior of LogScanner#scan_earliest? Does it load all data >> into client memory upfront and return a fully materialized ScanResult, or >> does it support streaming semantics, fetching data incrementally to reduce >> memory pressure? >> >> 4. What's the interface of `fluss.Config`? The config = fluss.Config(" >> 127.0.0.1:9123 <http://127.0.0.1:9123/>", 30) in the example confuses me >> about what the parameters >> mean? >> >> >> Best, >> Jark >> >> >> >> On Fri, 8 Aug 2025 at 20:03, Jim Hu <[email protected] >> <mailto:[email protected]>> wrote: >> >> > Hi Fluss community, >> > I’d like to kick off the public discussion on FIP-11¹, which proposes a >> > thin PyO3 wrapper around the existing Rust client so that Python users can >> > read/write Fluss without spinning up a JVM. >> > >> > - Goal: Java-SDK parity (streaming & batch reads/writes, DDL, point >> > queries) but 100 % Pythonic. >> > - Implementation: PyO3 bridge compiled to a native .so/.pyd; zero-copy, >> > lock-free, no sockets, no JVM start-up cost. >> > - Repository: fluss-rust/bindings/python. >> > >> > I’ve prototyped a minimal PoC at naivedogger/Fluss-Python-Client at >> > python-client >> > <https://github.com/naivedogger/Fluss-Python-Client/tree/python-client> >> > Looking forward to your thoughts. >> > Best regards, >> > Jim >> > [1] >> > >> > https://cwiki.apache.org/confluence/display/FLUSS/FIP-11%3A+Fluss+Python+binding+on+Fluss+Rust >> >
