Re: [ARROW-3329] Re: Decimal casting or scaling

2020-02-12 Thread Wes McKinney
On Wed, Feb 12, 2020 at 2:37 PM Jacek Pliszka wrote: > > Actually these options still make some sense - but not as much as before. > > The use case: unit conversion > > Data about prices exported from sql in Decimal(38,10) which uses 128 > bit but the numbers are actually prices which expressed in

Re: [ARROW-3329] Re: Decimal casting or scaling

2020-02-12 Thread Jacek Pliszka
Actually these options still make some sense - but not as much as before. The use case: unit conversion Data about prices exported from sql in Decimal(38,10) which uses 128 bit but the numbers are actually prices which expressed in cents fit perfectly in uint32 Having scaling would reduce bandwi

Re: [ARROW-3329] Re: Decimal casting or scaling

2020-02-12 Thread Jacek Pliszka
OK, then what I proposed does not make sense and I can just copy the solution you pointed out. Thank you, Jacek śr., 12 lut 2020 o 19:27 Wes McKinney napisał(a): > > On Wed, Feb 12, 2020 at 12:09 PM Jacek Pliszka > wrote: > > > > Hi! > > > > ARROW-3329 - we can discuss there. > > > > > It see

Re: [ARROW-3329] Re: Decimal casting or scaling

2020-02-12 Thread Wes McKinney
On Wed, Feb 12, 2020 at 12:09 PM Jacek Pliszka wrote: > > Hi! > > ARROW-3329 - we can discuss there. > > > It seems like it makes sense to implement both lossless safe casts > > (when all zeros after the decimal point) and lossy casts (fractional > > part discarded) from decimal to integer, do I h

[ARROW-3329] Re: Decimal casting or scaling

2020-02-12 Thread Jacek Pliszka
Hi! ARROW-3329 - we can discuss there. > It seems like it makes sense to implement both lossless safe casts > (when all zeros after the decimal point) and lossy casts (fractional > part discarded) from decimal to integer, do I have that right? Yes, though if I understood your examples are the sa

Re: Decimal casting or scaling

2020-02-12 Thread Wes McKinney
hi Jacek, What is the JIRA issue for this change? In the interest of organizing the discussion (may make sense to move some of this to that issue) There are no casts implemented DecimalType at all in [1], either to decimal or from decimal to anything else. It seems like it makes sense to impleme

Decimal casting or scaling

2020-02-12 Thread Jacek Pliszka
Hi! I am interested in having cast from Decimal to Int in pyarrow. I have couple ideas but I am a newbie so I might be wrong: Do I understand correctly that the problem lies in the fact that CastFunctor knows nothing about decimal scale? Were there any ideas how to handle this properly? My ide