This is an automated email from the ASF dual-hosted git repository.
wesm pushed a commit to branch master
in repository https://gitbox.apache.org/repos/asf/arrow.git
The following commit(s) were added to refs/heads/master by this push:
new 53cc09e ARROW-2626: [Python] Add column name to exception message
when writing pandas df fails
53cc09e is described below
commit 53cc09e535154dd1477c9e89197592b64c695866
Author: Louis Potok <[email protected]>
AuthorDate: Tue Jun 12 16:28:20 2018 -0400
ARROW-2626: [Python] Add column name to exception message when writing
pandas df fails
Goal is to make debugging easier - it's now possible to see which column
failed.
One quick question, I'm not sure if we need to be catching both
ArrowInvalid and ArrowTypeError - I think this used to throw ArrowInvalid but
now only throws ArrowTypeError. Thoughts?
Author: Louis Potok <[email protected]>
Closes #2075 from louispotok/master and squashes the following commits:
cf23778b <Louis Potok> pylint fixup
bc1a2b54 <Louis Potok> Add column name to exception message when writing
pandas df fails
---
python/pyarrow/pandas_compat.py | 6 +++++-
python/pyarrow/tests/test_convert_pandas.py | 5 +++++
2 files changed, 10 insertions(+), 1 deletion(-)
diff --git a/python/pyarrow/pandas_compat.py b/python/pyarrow/pandas_compat.py
index f3ae4fb..aabe2bb 100644
--- a/python/pyarrow/pandas_compat.py
+++ b/python/pyarrow/pandas_compat.py
@@ -365,7 +365,11 @@ def dataframe_to_arrays(df, schema, preserve_index,
nthreads=1, columns=None):
nthreads = 1
def convert_column(col, ty):
- return pa.array(col, from_pandas=True, type=ty)
+ try:
+ return pa.array(col, from_pandas=True, type=ty)
+ except (pa.ArrowInvalid, pa.ArrowTypeError) as e:
+ e.args += ("Conversion failed for column %s" % col.name,)
+ raise e
if nthreads == 1:
arrays = [convert_column(c, t)
diff --git a/python/pyarrow/tests/test_convert_pandas.py
b/python/pyarrow/tests/test_convert_pandas.py
index f71369f..2a70463 100644
--- a/python/pyarrow/tests/test_convert_pandas.py
+++ b/python/pyarrow/tests/test_convert_pandas.py
@@ -1895,6 +1895,11 @@ class TestConvertMisc(object):
with pytest.raises(pa.ArrowTypeError):
pa.Table.from_pandas(data)
+ data = pd.DataFrame({'a': ['a', 1, 2.0]})
+ expected_msg = 'Conversion failed for column a'
+ with pytest.raises(pa.ArrowTypeError, match=expected_msg):
+ pa.Table.from_pandas(data)
+
def test_strided_data_import(self):
cases = []
--
To stop receiving notification emails like this one, please contact
[email protected].
