Rafferty97 commented on PR #9494: URL: https://github.com/apache/arrow-rs/pull/9494#issuecomment-4101558479
> Thanks again for this @Rafferty97 and for your patience > > I took a look at the PR. my major comments are: > > 1. Can you please document the design / rationale (and why are there LazyLocks being used for what seem to be very small enums) > 2. Can you ensure the behavior is the same as the existing code? > > If we want to change the inference behavior I recommend proposing those changes in a separate PR so that we can evaluate the potential impact. Hi @alamb, thank you for taking a look over the PR and for the detailed feedback. The `LazyLock`s are an optimisation to avoid allocating a bunch of identical `Arc`s for the primitive types. You're right that this warrants some explanatory comments. The behaviour intentionally diverges from the existing code, because the existing code would perform coercions that the actual JSON reader itself doesn't do. So, when such a JSON file is encountered, the previous code would infer successfully but the actual reading into record batches would fail. This new code would return an error at inference time, which I think is more useful and less surprising to the end user. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
