quinnj opened a new pull request, #611: URL: https://github.com/apache/arrow-julia/pull/611
Addresses #610. A dictionary replacement could overtake queued record-batch tasks, causing earlier rows to use later dictionary values. Wait for outstanding batch construction before each dictionary update, outside the dictionary lock, and clear completed tasks. Use the same chained dictionary storage as Arrow.Stream so later deltas keep column types stable. Dictionary updates become synchronization points; consecutive record batches with unchanged dictionaries still run in parallel. Dictionary payloads are not copied. This fixes the current 2.8 reader; the separate 3.0 rewrite in #609 already handles replacement through its own decoder. Validation: the new regression failed six checks on unchanged main and passes all 12 after the fix on one and four threads. It covers replacement followed by delta, compressed input, and multiple input buffers. Frozen PyArrow V4/V5 fixtures also return correct values. A channel-gated probe verifies four unchanged-dictionary batch tasks start before any completes. Full suites passed 66,207 assertions each on Julia 1.9.4 and 1.12.6 using CI's local ArrowTypes setup. Formatting checks passed. Co-authored by Codex -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
