pitrou commented on PR #51038: URL: https://github.com/apache/arrow/pull/51038#issuecomment-5870262585
> Within Arrow, I narrowed it down to `InferringChunkedArrayBuilder.MultipleChunkIntegerParallel`. I tried different iteration counts: 16 iterations passes, while 32 and higher trigger the failure in the subsequent empty parse. I also changed the test to use `TaskGroup::MakeSerial()` instead of the threaded task group, and the failure still occurs, so it doesn't appear to depend on the task group threading. I am confused on how to proceed now? This is what the docstring says, which I think explains the fluctuating behaviour you're seeing: > this value is only meaningful under the conditions below. It is computed from stage-1 bookkeeping, and outside these conditions it is not merely imprecise, it is arbitrary – it can exceed [size_in_bytes()](https://simdjson.github.io/simdjson/classsimdjson_1_1dom_1_1document__stream.html#a2a08525ae815167b73b6393a8999e46b) or wrap around to a huge value > I was going to open an issue on simdjson, but their issue template asks for the problem to be reproducible with a standalone program using the vendored simdjson source. That's if you open an issue in the "bug" category, but I think you can use the more general "standard issue" template and let them decide how they feel about it. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
