jrbourbeau opened a new issue, #15079:
URL: https://github.com/apache/arrow/issues/15079

   ### Describe the bug, including details regarding any error messages, 
version, and platform.
   
   While adding support in Dask for the upcoming `pandas=2.0` release, some of 
our unit tests began failing due to `pyarrow` no longer round-tripping 
`datetime64` data in the same way `pandas` does (or will in the `pandas=2.0` 
release).
   
   Here's a minimal reproducer:
   
   ```python
   import pandas as pd
   import pyarrow as pa
   df = pd.DataFrame({"x": [3, 2, 1]}).astype("M8[us]")
   print(f"{df.x.dtypes = }")
   # Roundtrip through pyarrow.Table
   t = pa.Table.from_pandas(df)
   df2 = t.to_pandas()
   print(f"{df2.x.dtypes = }")
   pd.testing.assert_frame_equal(df, df2)
   ```
   
   which output
   
   ```
   df.x.dtypes = dtype('<M8[us]')
   df2.x.dtypes = dtype('<M8[ns]')
   Traceback (most recent call last):
     File "/Users/james/projects/dask/dask/test-roundtrip.py", line 11, in 
<module>
       pd.testing.assert_frame_equal(df, df2)
     File 
"/Users/james/mambaforge/envs/dask/lib/python3.10/site-packages/pandas/_testing/asserters.py",
 line 1187, in assert_frame_equal
       assert_series_equal(
     File 
"/Users/james/mambaforge/envs/dask/lib/python3.10/site-packages/pandas/_testing/asserters.py",
 line 898, in assert_series_equal
       assert_attr_equal("dtype", left, right, obj=f"Attributes of {obj}")
     File 
"/Users/james/mambaforge/envs/dask/lib/python3.10/site-packages/pandas/_testing/asserters.py",
 line 408, in assert_attr_equal
       raise_assert_detail(obj, msg, left_attr, right_attr)
     File 
"/Users/james/mambaforge/envs/dask/lib/python3.10/site-packages/pandas/_testing/asserters.py",
 line 592, in raise_assert_detail
       raise AssertionError(msg)
   AssertionError: Attributes of DataFrame.iloc[:, 0] (column name="x") are 
different
   Attribute "dtype" are different
   [left]:  datetime64[us]
   [right]: datetime64[ns]
   ```
   
   when using the nightly version of `pandas` installed with
   
   ```
   python -m pip install --no-deps --pre -i 
https://pypi.anaconda.org/scipy-wheels-nightly/simple pandas
   ```
   
   My guess is this is related to this upcoming change in `pandas` 
https://pandas.pydata.org/docs/dev/whatsnew/v2.0.0.html#construction-with-datetime64-or-timedelta64-dtype-with-unsupported-resolution
 (though that's just a guess).
   
   ### Component(s)
   
   Python


-- 
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.

To unsubscribe, e-mail: issues-unsubscr...@arrow.apache.org.apache.org

For queries about this service, please contact Infrastructure at:
us...@infra.apache.org

Reply via email to