anishmehta24 opened a new issue, #40572:
URL: https://github.com/apache/beam/issues/40572
### What happened?
`DeferredSeries.agg` with a list loses the Series name when the list
includes one of the hand-implemented aggregations (`mean`, `std`, `var`,
`skew`, `kurt`, `nunique`, ...). Those results are wrapped with `pd.Series(x,
index=[base_func])`, which has no name, so `agg(['mean'])` comes back unnamed
and `agg(['sum', 'mean'])` loses the name when the rows are concatenated.
```python
import apache_beam as beam
import pandas as pd
from apache_beam.dataframe.convert import to_dataframe, to_pcollection
values = [3.0, 1.0, 2.0, 5.0, 4.0]
print(pd.Series(values, name='x').agg(['sum', 'mean']))
with beam.Pipeline() as p:
df = to_dataframe(p | beam.Create([beam.Row(x=v) for v in values]))
_ = (
to_pcollection(df.x.agg(['sum', 'mean']), yield_elements='pandas')
| beam.Map(print))
```
pandas:
```
sum 15.0
mean 3.0
Name: x, dtype: float64
```
Beam:
```
sum 15.0
mean 3.0
dtype: float64
```
`agg(['sum'])` and `agg(['sum', 'max'])` keep the name. The existing
`test_series_agg_modes` uses an unnamed Series, so it doesn't catch this.
Seen on Beam 2.77.0 and on master (a9f808e4c45), pandas 2.3.3, Python 3.12.
### Issue Priority
Priority: 2 (default / most bugs should be filed as P2)
### Issue Components
- [x] Component: Python SDK
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
