Re: [I] [Python] Import of pyarrow.parquet and loading of non-existing file threw exception with incompatible pandas [arrow]
jorisvandenbossche commented on issue #46151: URL: https://github.com/apache/arrow/issues/46151#issuecomment-2814968413 @vadimkantorov I think the problem is an issue with your installation. The pandas package seems to be broken. We do a try/except around the pandas import in pyarrow: -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Python] Import of pyarrow.parquet and loading of non-existing file threw exception with incompatible pandas [arrow]
vadimkantorov commented on issue #46151: URL: https://github.com/apache/arrow/issues/46151#issuecomment-2815198601 My suggestion would be to print a warning for broken pandas imports (even for not ImportErrors), so it can still be visible and can be suppressed by standard python means if visible -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Python] Import of pyarrow.parquet and loading of non-existing file threw exception with incompatible pandas [arrow]
jorisvandenbossche commented on issue #46151: URL: https://github.com/apache/arrow/issues/46151#issuecomment-2814996885 @vadimkantorov as you suggested, the problem is indeed issue with your installation. The pandas package seems to be broken (because of the numpy and pandas versions being imcompatible. Import pandas directly will give a similar ValueError). We do a try/except around the pandas import in pyarrow: https://github.com/apache/arrow/blob/c2ec1cda88918a3920b7b29eca8d2e3ca7a49fb2/python/pyarrow/pandas-shim.pxi#L48-L57 But we are only catching ImportErrors, which generally is sufficient for a package not being installed. But so in practice we are not catching a ValueError which you got in this case. As you mentioned, "I hoped that pyarrow does not depend on pandas / pandas installed correctly", and so pyarrow does not depend on pandas, but we actually do depend somewhat on the fact that _if_ pandas is installed, it is installed correctly .. We could broaden that try/except, but the problem with that is that it can also then hide an actual problem (if you were assuming pandas was installed, and pyarrow would have its pandas support enabled). -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Python] Import of pyarrow.parquet and loading of non-existing file threw exception with incompatible pandas [arrow]
vadimkantorov commented on issue #46151: URL: https://github.com/apache/arrow/issues/46151#issuecomment-2808757135 I suggest `import pyarrow.parquet` does not throw if pandas can't be imported correctly - maybe better to try-catch protect the pandas import (if it's indeed optional)... -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Python] Import of pyarrow.parquet and loading of non-existing file threw exception with incompatible pandas [arrow]
AlenkaF commented on issue #46151: URL: https://github.com/apache/arrow/issues/46151#issuecomment-2808741941 > I think the actual error in my case was incompat of old pandas and new numpy, which got triggered at pandas import inside pyarrow Ah, yeah, that makes sense. But I do not think this is something we need to check on our side. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Python] Import of pyarrow.parquet and loading of non-existing file threw exception with incompatible pandas [arrow]
vadimkantorov commented on issue #46151: URL: https://github.com/apache/arrow/issues/46151#issuecomment-2808721015 Yeah, if pandas dependency is optional, I think the imports of pandas should be try-catch protected. PyArrow might print a warning of bad pandas version and even not attempt to import it, or just catch errors at pandas import, but not fail completely. -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [I] [Python] Import of pyarrow.parquet and loading of non-existing file threw exception with incompatible pandas [arrow]
vadimkantorov commented on issue #46151: URL: https://github.com/apache/arrow/issues/46151#issuecomment-2808602726 Oh, somehow the error message did not get pasted here fully. It was quite strange on size of some pyobject structure not matching the correct one -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@arrow.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
