[jira] [Updated] (SPARK-45155) Add API Docs
[ https://issues.apache.org/jira/browse/SPARK-45155?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Pengfei Xu updated SPARK-45155: --- Fix Version/s: 4.0.0 > Add API Docs > > > Key: SPARK-45155 > URL: https://issues.apache.org/jira/browse/SPARK-45155 > Project: Spark > Issue Type: Improvement > Components: Connect >Affects Versions: 3.5.0 >Reporter: Venkata Sai Akhil Gudesa >Priority: Major > Labels: pull-request-available > Fix For: 4.0.0 > > > Similar to the pages listed here for regular Spark - > [https://spark.apache.org/docs/latest/api/scala/org/apache/spark/index.html] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-48714) Implement df.mergeInto in PySpark
Pengfei Xu created SPARK-48714: -- Summary: Implement df.mergeInto in PySpark Key: SPARK-48714 URL: https://issues.apache.org/jira/browse/SPARK-48714 Project: Spark Issue Type: New Feature Components: PySpark Affects Versions: 4.0.0 Reporter: Pengfei Xu -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-48683) Schema evolution with `df.mergeInto` losing `when` clauses
[
https://issues.apache.org/jira/browse/SPARK-48683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengfei Xu updated SPARK-48683:
---
Component/s: SQL
(was: Spark Core)
> Schema evolution with `df.mergeInto` losing `when` clauses
> --
>
> Key: SPARK-48683
> URL: https://issues.apache.org/jira/browse/SPARK-48683
> Project: Spark
> Issue Type: New Feature
> Components: SQL
>Affects Versions: 4.0.0
>Reporter: Pengfei Xu
>Priority: Major
>
> When calling {{df.mergeInto(...).when...(...).withSchemaEvolution()}} all
> {{when}} clauses will be lost.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-48683) Schema evolution with `df.mergeInto` losing `when` clauses
Pengfei Xu created SPARK-48683:
--
Summary: Schema evolution with `df.mergeInto` losing `when` clauses
Key: SPARK-48683
URL: https://issues.apache.org/jira/browse/SPARK-48683
Project: Spark
Issue Type: New Feature
Components: Spark Core
Affects Versions: 4.0.0
Reporter: Pengfei Xu
When calling {{df.mergeInto(...).when...(...).withSchemaEvolution()}} all
{{when}} clauses will be lost.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-48510) Support UDAF.toColumn API in Spark Connect
Pengfei Xu created SPARK-48510: -- Summary: Support UDAF.toColumn API in Spark Connect Key: SPARK-48510 URL: https://issues.apache.org/jira/browse/SPARK-48510 Project: Spark Issue Type: New Feature Components: Connect Affects Versions: 4.0.0 Reporter: Pengfei Xu Following up on https://issues.apache.org/jira/browse/SPARK-48008, we still have to support udaf.toColumn API in Spark Connect. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Updated] (SPARK-48008) Support UDAF in Spark Connect
[
https://issues.apache.org/jira/browse/SPARK-48008?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Pengfei Xu updated SPARK-48008:
---
Description:
Currently Spark Connect supports only UDFs. We need to add support for UDAFs,
specifically `Aggregator[INT, BUF, OUT]`.
The user-facing API should not change, which includes Aggregator methods and
the `spark.udf.register("agg", udaf(agg))` API.
was:
Currently Spark Connect supports only UDFs. We need to add support for UDAFs,
specifically `Aggregator[INT, BUF, OUT]`.
The user-facing API should not change, which includes Aggregator methods and
the `
spark.udf.register("agg", udaf(agg))` API.
> Support UDAF in Spark Connect
> -
>
> Key: SPARK-48008
> URL: https://issues.apache.org/jira/browse/SPARK-48008
> Project: Spark
> Issue Type: New Feature
> Components: Connect
>Affects Versions: 4.0.0
>Reporter: Pengfei Xu
>Priority: Major
>
> Currently Spark Connect supports only UDFs. We need to add support for UDAFs,
> specifically `Aggregator[INT, BUF, OUT]`.
> The user-facing API should not change, which includes Aggregator methods and
> the `spark.udf.register("agg", udaf(agg))` API.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Created] (SPARK-47545) [Connect] DF observe support for the scala client
Pengfei Xu created SPARK-47545: -- Summary: [Connect] DF observe support for the scala client Key: SPARK-47545 URL: https://issues.apache.org/jira/browse/SPARK-47545 Project: Spark Issue Type: New Feature Components: Connect Affects Versions: 4.0.0 Reporter: Pengfei Xu -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
