[ https://issues.apache.org/jira/browse/ARROW-10344?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Gert Hulselmans updated ARROW-10344: ------------------------------------ Description: Is there a way to get all column names (or schema) from a Feather file before loading the full Feather file? My Feather files are big (like 100GB) and the names of the columns are different per analysis and can't be hard coded. {code:python} import pyarrow.feather as feather # Code here to check which columns are in the feather file. ... my_columns = ... # Result is pandas.DataFrame read_df = feather.read_feather('/path/to/file', columns=my_columns) # Result is pyarrow.Table read_arrow = feather.read_table('/path/to/file', columns=my_columns) {code} was: Is there a way to get all column names (and e.g. number of columns and number of rows) from a Feather file before loading the full Feather file? My Feather files are big (like 100GB) and the names of the columns are different per analysis and can't be hard coded. {code:python} import pyarrow.feather as feather # Code here to check which columns are in the feather file. ... my_columns = ... # Result is pandas.DataFrame read_df = feather.read_feather('/path/to/file', columns=my_columns) # Result is pyarrow.Table read_arrow = feather.read_table('/path/to/file', columns=my_columns) {code} Summary: [Python] Get all columns names (or schema) from Feather file, before loading whole Feather file (was: [Python] Get all columns names from Feather file, before loading whole Feather file) > [Python] Get all columns names (or schema) from Feather file, before loading > whole Feather file > ------------------------------------------------------------------------------------------------ > > Key: ARROW-10344 > URL: https://issues.apache.org/jira/browse/ARROW-10344 > Project: Apache Arrow > Issue Type: New Feature > Components: Python > Affects Versions: 1.0.1 > Reporter: Gert Hulselmans > Priority: Major > > Is there a way to get all column names (or schema) from a Feather file before > loading the full Feather file? > My Feather files are big (like 100GB) and the names of the columns are > different per analysis and can't be hard coded. > {code:python} > import pyarrow.feather as feather > # Code here to check which columns are in the feather file. > ... > my_columns = ... > # Result is pandas.DataFrame > read_df = feather.read_feather('/path/to/file', columns=my_columns) > # Result is pyarrow.Table > read_arrow = feather.read_table('/path/to/file', columns=my_columns) > {code} -- This message was sent by Atlassian Jira (v8.3.4#803005)