This is an automated email from the ASF dual-hosted git repository.
AlenkaF pushed a commit to branch main
in repository https://gitbox.apache.org/repos/asf/arrow.git
The following commit(s) were added to refs/heads/main by this push:
new 65244af1ff GH-49232: [Python] deprecate feather python (#49590)
65244af1ff is described below
commit 65244af1ffb1b7b9b8ae3d46f4768736d9ef9ece
Author: piyushka-ally <[email protected]>
AuthorDate: Wed Jul 1 13:07:16 2026 +0530
GH-49232: [Python] deprecate feather python (#49590)
### Rationale for this change
Arrow C++ is deprecating the Feather reader/writer (#49231) so we should
update the Python functions too. Feather V2 is exactly the Arrow IPC file
format, making the separate `pyarrow.feather` module redundant. Users should
migrate to `pyarrow.ipc`.
### What changes are included in this PR?
- Added `FutureWarning` deprecation warnings to all four public APIs in
`pyarrow.feather`: `write_feather()`, `read_feather()`, `read_table()`, and
`FeatherDataset`
- Extracted `_read_table_internal()` so that `read_feather()` calling
`read_table()` internally does not produce double warnings
- Updated `FeatherDataset.read_table()` to use the internal function for
the same reason
- Added `.. deprecated:: 24.0.0` directives to all four public API
docstrings
- Added module-level `pytestmark` filter in `test_feather.py` to suppress
warnings in existing tests
- Added 5 new tests verifying deprecation warnings are emitted correctly
(including a no-double-warning test)
- Added `@ pytest.mark.filterwarnings` to 5 test functions in
`test_dataset.py` that use feather as a utility
- Updated `feather.rst` with a deprecation notice and migration guide
(including a note that `pyarrow.ipc` does not compress by default unlike
`feather.write_feather`)
- Updated `formats.rst` to mark the Feather API section as deprecated
### Are these changes tested?
Yes. Five new tests verify that each deprecated function emits exactly one
`FutureWarning`, and existing tests continue to pass with module/function-level
warning filters.
### Are there any user-facing changes?
Yes — calling `pyarrow.feather.write_feather()`, `read_feather()`,
`read_table()`, or `FeatherDataset()` now emits a `FutureWarning` directing
users to `pyarrow.ipc` equivalents. Existing functionality is unchanged.
* GitHub Issue: #49232
Lead-authored-by: Piyush Kanti Chanda <[email protected]>
Co-authored-by: Isaac
Signed-off-by: AlenkaF <[email protected]>
---
docs/source/python/api/formats.rst | 7 ++-
docs/source/python/feather.rst | 36 +++++++++++++
python/pyarrow/feather.py | 101 +++++++++++++++++++++++++++--------
python/pyarrow/tests/test_dataset.py | 5 ++
python/pyarrow/tests/test_feather.py | 48 +++++++++++++++++
5 files changed, 173 insertions(+), 24 deletions(-)
diff --git a/docs/source/python/api/formats.rst
b/docs/source/python/api/formats.rst
index 57a5e824fa..0d2cd8975c 100644
--- a/docs/source/python/api/formats.rst
+++ b/docs/source/python/api/formats.rst
@@ -42,11 +42,14 @@ CSV Files
.. _api.feather:
-Feather Files
--------------
+Feather Files (Deprecated)
+--------------------------
.. currentmodule:: pyarrow.feather
+.. deprecated:: 24.0.0
+ The Feather API is deprecated. Use the :ref:`IPC <ipc>` API instead.
+
.. autosummary::
:toctree: ../generated/
diff --git a/docs/source/python/feather.rst b/docs/source/python/feather.rst
index 026ea987a2..76520e912b 100644
--- a/docs/source/python/feather.rst
+++ b/docs/source/python/feather.rst
@@ -22,6 +22,10 @@
Feather File Format
===================
+.. deprecated:: 24.0.0
+ The ``pyarrow.feather`` module is deprecated. Feather V2 is the Arrow IPC
+ file format. Use :mod:`pyarrow.ipc` instead. See :ref:`ipc` for details.
+
Feather is a portable file format for storing Arrow tables or data frames (from
languages like Python or R) that utilizes the :ref:`Arrow IPC format <ipc>`
internally. Feather was created early in the Arrow project as a proof of
@@ -107,3 +111,35 @@ Writing Version 1 (V1) Files
For compatibility with libraries without support for Version 2 files, you can
write the version 1 format by passing ``version=1`` to ``write_feather``. We
intend to maintain read support for V1 for the foreseeable future.
+
+Migration to IPC
+----------------
+
+Since Feather V2 is the Arrow IPC file format, you can use the
+:mod:`pyarrow.ipc` module as a direct replacement:
+
+.. code-block:: python
+
+ import pyarrow as pa
+ import pyarrow.ipc
+
+ table = pa.table({"col1": [1, 2, 3], "col2": ["a", "b", "c"]})
+
+ # Writing (replaces feather.write_feather)
+ options = pa.ipc.IpcWriteOptions(compression='lz4')
+ with pa.ipc.new_file("data.arrow", table.schema, options=options) as writer:
+ writer.write_table(table)
+
+ # Reading (replaces feather.read_table)
+ with pa.ipc.open_file("data.arrow") as reader:
+ result = reader.read_all()
+
+.. note::
+
+ ``feather.write_feather`` defaults to LZ4 compression, while
+ ``ipc.new_file`` does not compress by default. To preserve the same
+ behavior, pass ``compression='lz4'`` via
+ :class:`~pyarrow.ipc.IpcWriteOptions` as shown above.
+
+For reading multiple files, use the :mod:`pyarrow.dataset` module with
+``format='ipc'`` instead of :class:`~pyarrow.feather.FeatherDataset`.
diff --git a/python/pyarrow/feather.py b/python/pyarrow/feather.py
index 241c27706a..68f708c914 100644
--- a/python/pyarrow/feather.py
+++ b/python/pyarrow/feather.py
@@ -18,6 +18,7 @@
from collections.abc import Sequence
import os
+import warnings
from pyarrow.pandas_compat import _pandas_api # noqa
from pyarrow.lib import (Codec, Table, # noqa
@@ -31,6 +32,9 @@ class FeatherDataset:
"""
Encapsulates details of reading a list of Feather files.
+ .. deprecated:: 24.0.0
+ Use :func:`pyarrow.dataset.dataset` with ``format='ipc'`` instead.
+
Parameters
----------
path_or_paths : List[str]
@@ -40,6 +44,12 @@ class FeatherDataset:
"""
def __init__(self, path_or_paths, validate_schema=True):
+ warnings.warn(
+ "pyarrow.feather.FeatherDataset is deprecated as of 24.0.0. "
+ "Use pyarrow.dataset.dataset() with format='ipc' instead.",
+ FutureWarning,
+ stacklevel=2
+ )
self.paths = path_or_paths
self.validate_schema = validate_schema
@@ -57,12 +67,12 @@ class FeatherDataset:
pyarrow.Table
Content of the file as a table (of columns)
"""
- _fil = read_table(self.paths[0], columns=columns)
+ _fil = _read_table_internal(self.paths[0], columns=columns)
self._tables = [_fil]
self.schema = _fil.schema
for path in self.paths[1:]:
- table = read_table(path, columns=columns)
+ table = _read_table_internal(path, columns=columns)
if self.validate_schema:
self.validate_schemas(path, table)
self._tables.append(table)
@@ -117,6 +127,11 @@ def write_feather(df, dest, compression=None,
compression_level=None,
"""
Write a pandas.DataFrame to Feather format.
+ .. deprecated:: 24.0.0
+ Use :func:`pyarrow.ipc.new_file` /
+ :class:`pyarrow.ipc.RecordBatchFileWriter` instead.
+ Feather V2 is the Arrow IPC file format.
+
Parameters
----------
df : pandas.DataFrame or pyarrow.Table
@@ -137,6 +152,13 @@ def write_feather(df, dest, compression=None,
compression_level=None,
Feather file version. Version 2 is the current. Version 1 is the more
limited legacy format
"""
+ warnings.warn(
+ "pyarrow.feather.write_feather is deprecated as of 24.0.0. "
+ "Use pyarrow.ipc.new_file() / RecordBatchFileWriter instead. "
+ "Feather V2 is the Arrow IPC file format.",
+ FutureWarning,
+ stacklevel=2
+ )
if _pandas_api.have_pandas:
if (_pandas_api.has_sparse and
isinstance(df, _pandas_api.pd.SparseDataFrame)):
@@ -201,6 +223,11 @@ def read_feather(source, columns=None, use_threads=True,
Read a pandas.DataFrame from Feather format. To read as pyarrow.Table use
feather.read_table.
+ .. deprecated:: 24.0.0
+ Use :func:`pyarrow.ipc.open_file` /
+ :class:`pyarrow.ipc.RecordBatchFileReader` instead.
+ Feather V2 is the Arrow IPC file format.
+
Parameters
----------
source : str file path, or file-like object
@@ -222,31 +249,23 @@ def read_feather(source, columns=None, use_threads=True,
df : pandas.DataFrame
The contents of the Feather file as a pandas.DataFrame
"""
- return (read_table(
+ warnings.warn(
+ "pyarrow.feather.read_feather is deprecated as of 24.0.0. "
+ "Use pyarrow.ipc.open_file() / RecordBatchFileReader instead. "
+ "Feather V2 is the Arrow IPC file format.",
+ FutureWarning,
+ stacklevel=2
+ )
+ return (_read_table_internal(
source, columns=columns, memory_map=memory_map,
use_threads=use_threads).to_pandas(use_threads=use_threads, **kwargs))
-def read_table(source, columns=None, memory_map=False, use_threads=True):
+def _read_table_internal(source, columns=None, memory_map=False,
+ use_threads=True):
"""
- Read a pyarrow.Table from Feather format
-
- Parameters
- ----------
- source : str file path, or file-like object
- You can use MemoryMappedFile as source, for explicitly use memory map.
- columns : sequence, optional
- Only read a specific set of columns. If not provided, all columns are
- read.
- memory_map : boolean, default False
- Use memory mapping when opening file on disk, when source is a str
- use_threads : bool, default True
- Whether to parallelize reading using multiple threads.
-
- Returns
- -------
- table : pyarrow.Table
- The contents of the Feather file as a pyarrow.Table
+ Internal implementation for reading a Feather file as a pyarrow.Table.
+ Does not emit deprecation warnings.
"""
reader = _feather.FeatherReader(
source, use_memory_map=memory_map, use_threads=use_threads)
@@ -277,3 +296,41 @@ def read_table(source, columns=None, memory_map=False,
use_threads=True):
else:
# follow exact order / selection of names
return table.select(columns)
+
+
+def read_table(source, columns=None, memory_map=False, use_threads=True):
+ """
+ Read a pyarrow.Table from Feather format
+
+ .. deprecated:: 24.0.0
+ Use :func:`pyarrow.ipc.open_file` /
+ :class:`pyarrow.ipc.RecordBatchFileReader` instead.
+ Feather V2 is the Arrow IPC file format.
+
+ Parameters
+ ----------
+ source : str file path, or file-like object
+ You can use MemoryMappedFile as source, for explicitly use memory map.
+ columns : sequence, optional
+ Only read a specific set of columns. If not provided, all columns are
+ read.
+ memory_map : boolean, default False
+ Use memory mapping when opening file on disk, when source is a str
+ use_threads : bool, default True
+ Whether to parallelize reading using multiple threads.
+
+ Returns
+ -------
+ table : pyarrow.Table
+ The contents of the Feather file as a pyarrow.Table
+ """
+ warnings.warn(
+ "pyarrow.feather.read_table is deprecated as of 24.0.0. "
+ "Use pyarrow.ipc.open_file() / RecordBatchFileReader instead. "
+ "Feather V2 is the Arrow IPC file format.",
+ FutureWarning,
+ stacklevel=2
+ )
+ return _read_table_internal(source, columns=columns,
+ memory_map=memory_map,
+ use_threads=use_threads)
diff --git a/python/pyarrow/tests/test_dataset.py
b/python/pyarrow/tests/test_dataset.py
index 0b339f18fd..63c537eae5 100644
--- a/python/pyarrow/tests/test_dataset.py
+++ b/python/pyarrow/tests/test_dataset.py
@@ -1927,6 +1927,7 @@ def
test_fragments_parquet_subset_with_nested_fields(tempdir):
@pytest.mark.pandas
@pytest.mark.parquet
[email protected]("ignore:pyarrow.feather:FutureWarning")
def test_fragments_repr(tempdir, dataset):
# partitioned parquet dataset
fragment = list(dataset.get_fragments())[0]
@@ -3699,6 +3700,7 @@ def test_column_names_encoding(tempdir, dataset_reader):
assert dataset_transcoded.to_table().equals(expected_table)
[email protected]("ignore:pyarrow.feather:FutureWarning")
def test_feather_format(tempdir, dataset_reader):
from pyarrow.feather import write_feather
@@ -4080,6 +4082,7 @@ def test_dataset_project_null_column(tempdir,
dataset_reader):
assert dataset_reader.to_table(dataset).equals(expected)
[email protected]("ignore:pyarrow.feather:FutureWarning")
def test_dataset_project_columns(tempdir, dataset_reader):
# basic column re-projection with expressions
from pyarrow import feather
@@ -4431,6 +4434,7 @@ def test_write_dataset_with_dataset(tempdir):
@pytest.mark.pandas
[email protected]("ignore:pyarrow.feather:FutureWarning")
def test_write_dataset_existing_data(tempdir):
directory = tempdir / 'ds'
table = pa.table({'b': ['x', 'y', 'z'], 'c': [1, 2, 3]})
@@ -5054,6 +5058,7 @@ def test_write_dataset_arrow_schema_metadata(tempdir):
assert result["a"].type.tz == "Europe/Brussels"
[email protected]("ignore:pyarrow.feather:FutureWarning")
def test_write_dataset_schema_metadata(tempdir):
# ensure that schema metadata gets written
from pyarrow import feather
diff --git a/python/pyarrow/tests/test_feather.py
b/python/pyarrow/tests/test_feather.py
index 054bf920b2..c04cef534e 100644
--- a/python/pyarrow/tests/test_feather.py
+++ b/python/pyarrow/tests/test_feather.py
@@ -19,6 +19,7 @@ import io
import os
import sys
import tempfile
+import warnings
import pytest
import hypothesis as h
import hypothesis.strategies as st
@@ -40,6 +41,12 @@ try:
except ImportError:
pass
+# Suppress deprecation warnings for existing tests since pyarrow.feather
+# is deprecated as of 24.0.0
+pytestmark = pytest.mark.filterwarnings(
+ "ignore:pyarrow.feather:FutureWarning"
+)
+
@pytest.fixture(scope='module')
def datadir(base_datadir):
@@ -882,3 +889,44 @@ def
test_feather_datetime_resolution_arrow_to_pandas(tempdir):
assert expected_0 == result['date'][0]
assert expected_1 == result['date'][1]
+
+
+# --- Deprecation warning tests ---
+
[email protected]
[email protected]("default:pyarrow.feather:FutureWarning")
+def test_feather_deprecation_warnings(tempdir):
+ table = pa.table({"a": [1, 2, 3]})
+ path = str(tempdir / "test.feather")
+
+ with pytest.warns(FutureWarning, match="write_feather is deprecated"):
+ write_feather(table, path)
+
+ with pytest.warns(FutureWarning, match="read_table is deprecated"):
+ read_table(path)
+
+ with pytest.warns(FutureWarning, match="read_feather is deprecated"):
+ read_feather(path)
+
+
[email protected]("default:pyarrow.feather:FutureWarning")
+def test_feather_dataset_deprecated():
+ with pytest.warns(FutureWarning, match="FeatherDataset is deprecated"):
+ FeatherDataset([])
+
+
[email protected]
[email protected]("default:pyarrow.feather:FutureWarning")
+def test_read_feather_no_double_warning(tempdir):
+ """Verify read_feather emits exactly one FutureWarning, not two."""
+ table = pa.table({"a": [1, 2, 3]})
+ path = str(tempdir / "test.feather")
+ with warnings.catch_warnings():
+ warnings.simplefilter("ignore", FutureWarning)
+ write_feather(table, path)
+ with warnings.catch_warnings(record=True) as w:
+ warnings.simplefilter("always")
+ read_feather(path)
+ future_warnings = [x for x in w if issubclass(x.category,
+ FutureWarning)]
+ assert len(future_warnings) == 1
