liukun4515 commented on issue #2387: URL: https://github.com/apache/arrow-rs/issues/2387#issuecomment-1224088465
> > Can you tell me where we handle the overflow? I can't get your thoughts > > In all the places where we are currently doing validation? > > > But as far as Decimal data type, If I cast the Decimal(8,0) array) to Decimal(12,0) array`, each element can't be overflow or out of the range for the new precision, we don't need to do validation and improve the performance. > > Only if you can assert that the data in the `Decimal(8,0)` was in bounds of the precision of 8 😁 That's what is so wildly inconsistent at the moment, if you don't consistently validate, you can't elide as you don't know if validation has actually occurred > > > do we need to do validation and check the overflow for the casting > > No, which is why I am trying to argue for not bothering to do the same for decimals... But currently we sort of do and sort of don't and I don't really understand what is going on > > Edit: apparently we are doing checked casting for numerics, so I don't know anymore... other critical path for decimal, do we need to do validation when reading the parquet data to decimal? @tustvold -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
