Diego Argueta created ARROW-4885:
------------------------------------
Summary: [Python] read_csv() can't handle decimal128() columns
Key: ARROW-4885
URL: https://issues.apache.org/jira/browse/ARROW-4885
Project: Apache Arrow
Issue Type: Bug
Components: Python
Affects Versions: 0.12.1
Environment: Python: 3.7.2, 2.7.15
PyArrow: 0.12.1
OS: MacOS 10.13.4 (High Sierra)
Reporter: Diego Argueta
h1. Summary
CSV cannot use {{Decimal128Type}}. The cause is that there's no converter
listed
[here|https://github.com/apache/arrow/blob/master/cpp/src/arrow/csv/converter.cc#L301-L315].
I haven't tested it yet but I suspect adding the following line _might_ fix it:
{code:c++}
CONVERTER_CASE(Type::DECIMAL, NumericConverter<Decimal128Type)
{code}
h1. To Reproduce
1) First, create a CSV file like so and save it somewhere:
{code}
Header
123.45
{code}
2) Run the following code on Python 2 or 3:
{code:python}
import pyarrow.csv as pa_csv
import pyarrow as pa
import io
types = {'Header': pa.decimal128(11, 2)}
convert_options = pa_csv.ConvertOptions(column_types=types)
pa_csv.read_csv('/Users/diegoargueta/Desktop/test.csv',
convert_options=convert_options)
{code}
read_csv() crashes with the following exception:
{code}
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
File "pyarrow/_csv.pyx", line 397, in pyarrow._csv.read_csv
File "pyarrow/error.pxi", line 89, in pyarrow.lib.check_status
pyarrow.lib.ArrowNotImplementedError: CSV conversion to decimal(11, 2) is not
supported
CSV conversion to decimal(11, 2) is not supported
{code}
--
This message was sent by Atlassian JIRA
(v7.6.3#76005)
