[ https://issues.apache.org/jira/browse/ARROW-13028?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17360291#comment-17360291 ]
Antoine Pitrou commented on ARROW-13028: ---------------------------------------- I'm unsure how much flexibility we want to add to CSV type inference. You can of course pass column types explicitly if you want to optimize memory footprint. [~npr] [~jonkeane] What do you think? > [C++] CSV add convert option to attempt 32bit number inferences > --------------------------------------------------------------- > > Key: ARROW-13028 > URL: https://issues.apache.org/jira/browse/ARROW-13028 > Project: Apache Arrow > Issue Type: Improvement > Components: C++ > Reporter: Nate Clark > Assignee: Nate Clark > Priority: Major > > When types are being inferred by CSV the numbers are always 64 bit. For large > data sets it could be better to use 32 bit types to save over all memory. To > do this it would be useful to add an option to ConvertOptions to try 32 bit > numbers before 64 bit. By default this option would be disabled. -- This message was sent by Atlassian Jira (v8.3.4#803005)