JingsongLi commented on code in PR #10734:
URL: https://github.com/apache/paimon/pull/10734#discussion_r4343565962
##########
paimon-python/pyproject.toml:
##########
@@ -102,7 +102,8 @@ lerobot = [
"datasets>=4,<5; python_version >= '3.10'",
"pandas>=2.2.2,<3; python_version >= '3.10'",
"torch>=2.3; python_version >= '3.10'",
- "lerobot>=0.4.4,<0.5; python_version >= '3.10'",
+ "lerobot>=0.4.4,<0.5; python_version >= '3.10' and python_version <
'3.12'",
+ "lerobot[dataset]>=0.6,<0.7; python_version >= '3.12'",
Review Comment:
[P2] Preserve boolean writer statistics when selecting LeRobot 0.6
This dependency change also upgrades `PaimonLeRobotWriter` on Python 3.12+.
For an episode containing at least two frames of a bool feature (for example,
shape `(2,)` with `[True, False]` and `[False, True]`), LeRobot 0.6's
`RunningQuantileStats.update()` promotes the input to float32, so the episode's
`min`/`max` become floating-point values. Paimon's `_episode_schema()` still
declares those statistics as bool, and `flush()`/`finalize()` then fails at
`pa.Table.from_pylist(..., schema=self._episodes_schema)` with `ArrowInvalid:
Could not convert 0.0 with type float: tried to convert to boolean`, preventing
the episode from being committed.
I reproduced this with the same Paimon source, NumPy 2.2.6 and PyArrow
22.0.0: LeRobot 0.4.4 commits and reads back both frames; 0.6.1 fails. Please
preserve/recompute boolean extrema with the expected dtype before writing the
episode metadata and add a multi-frame bool regression test. The existing
`test_writes_multidimensional_and_boolean_numpy_features` has only one frame,
which bypasses the changed running-statistics path.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
