Re: guidance on extension types

Yup we’ve run into this as well. Though I think you could control this by implementing a pandas extension dtype to go with the arrow extension type? On Wed, Sep 21, 2022 at 9:17 PM Micah Kornfield wrote: > Also, note I've raised a similar issue ( > https://issues.apache.org/jira/browse/ARROW-17

Re: guidance on extension types

Also, note I've raised a similar issue ( https://issues.apache.org/jira/browse/ARROW-17535) for to_pandas calls. One thing that I think would be nice is to be able to hook into the python conversion when necessary translate to Python objects when necessary. On Wed, Sep 21, 2022 at 8:49 PM Chang

Re: guidance on extension types

Thanks Wes. => Array.to_numpy : I opened ARROW-17813 as you suggested and added some details / repro code. There's also a follow-up thing about the other direction, converting from a pandas DataFrame column to an Arrow list. => You're right, I w

?????? [c++][compute]Is there any other way to use Join besides Acero??

Thank you very much, your reply is very helpful. I have one more question to ask. Since our data is actually stored in S3, I would like to ask if we can set project during SCAN. My understanding is that we will only get the columns we need from S3 instead of scanning the entire file. This will g