[ https://issues.apache.org/jira/browse/SPARK-40063?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ]
Marcelo Rossini Castro updated SPARK-40063: ------------------------------------------- Description: When using the apply function to apply a function to a DataFrame column, it ends up mixing the column's rows ordering. A command like this: {code:java} def example_func(df_col): return df_col ** 2 df['row_to_apply_function'] = df.apply(lambda row: example_func(row['row_to_apply_function']), axis=1) {code} A workaround is to assign the results to a new column instead of the same one, but if the old column is dropped, the same error is produced. was: When using the apply function to apply a function to a DataFrame column, it ends up mixing the column's rows ordering. A command like this: {code:java} def example_func(df_col): return df_col ** 2 df['row_to_apply_function'] = df.apply(lambda row: example_func(row['row_to_apply_function']), axis=1) {code} A workaround is to assign the results to a new column instead of the same one, but if the old column is dropped, the same error is produced. > pyspark.pandas .apply() changing rows ordering > ---------------------------------------------- > > Key: SPARK-40063 > URL: https://issues.apache.org/jira/browse/SPARK-40063 > Project: Spark > Issue Type: Bug > Components: Pandas API on Spark > Affects Versions: 3.3.0 > Reporter: Marcelo Rossini Castro > Priority: Minor > > When using the apply function to apply a function to a DataFrame column, it > ends up mixing the column's rows ordering. > A command like this: > {code:java} > def example_func(df_col): > return df_col ** 2 > df['row_to_apply_function'] = df.apply(lambda row: > example_func(row['row_to_apply_function']), axis=1) {code} > A workaround is to assign the results to a new column instead of the same > one, but if the old column is dropped, the same error is produced. -- This message was sent by Atlassian Jira (v8.20.10#820010) --------------------------------------------------------------------- To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org