[jira] [Assigned] (SPARK-42495) Scala Client: Add 2nd batch of functions
[ https://issues.apache.org/jira/browse/SPARK-42495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42495: Assignee: Herman van Hövell (was: Apache Spark) > Scala Client: Add 2nd batch of functions > > > Key: SPARK-42495 > URL: https://issues.apache.org/jira/browse/SPARK-42495 > Project: Spark > Issue Type: Task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Herman van Hövell >Assignee: Herman van Hövell >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42495) Scala Client: Add 2nd batch of functions
[ https://issues.apache.org/jira/browse/SPARK-42495?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42495: Assignee: Apache Spark (was: Herman van Hövell) > Scala Client: Add 2nd batch of functions > > > Key: SPARK-42495 > URL: https://issues.apache.org/jira/browse/SPARK-42495 > Project: Spark > Issue Type: Task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Herman van Hövell >Assignee: Apache Spark >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42495) Scala Client: Add 2nd batch of functions
[ https://issues.apache.org/jira/browse/SPARK-42495?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17691002#comment-17691002 ] Apache Spark commented on SPARK-42495: -- User 'hvanhovell' has created a pull request for this issue: https://github.com/apache/spark/pull/40089 > Scala Client: Add 2nd batch of functions > > > Key: SPARK-42495 > URL: https://issues.apache.org/jira/browse/SPARK-42495 > Project: Spark > Issue Type: Task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Herman van Hövell >Assignee: Herman van Hövell >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42427) Conv should return an error if the internal conversion overflows
[ https://issues.apache.org/jira/browse/SPARK-42427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690970#comment-17690970 ] Apache Spark commented on SPARK-42427: -- User 'HyukjinKwon' has created a pull request for this issue: https://github.com/apache/spark/pull/40088 > Conv should return an error if the internal conversion overflows > > > Key: SPARK-42427 > URL: https://issues.apache.org/jira/browse/SPARK-42427 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.4.0 >Reporter: Gengliang Wang >Assignee: Gengliang Wang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42493) Spark SQL, DataFrames and Datasets Guide - make Python the first example tab
[ https://issues.apache.org/jira/browse/SPARK-42493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42493: Assignee: (was: Apache Spark) > Spark SQL, DataFrames and Datasets Guide - make Python the first example tab > > > Key: SPARK-42493 > URL: https://issues.apache.org/jira/browse/SPARK-42493 > Project: Spark > Issue Type: Documentation > Components: Spark Core >Affects Versions: 3.4.0 >Reporter: Allan Folting >Priority: Major > > Python is the easiest approachable and most popular language so it should be > the primary language in examples etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42493) Spark SQL, DataFrames and Datasets Guide - make Python the first example tab
[ https://issues.apache.org/jira/browse/SPARK-42493?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690958#comment-17690958 ] Apache Spark commented on SPARK-42493: -- User 'allanf-db' has created a pull request for this issue: https://github.com/apache/spark/pull/40087 > Spark SQL, DataFrames and Datasets Guide - make Python the first example tab > > > Key: SPARK-42493 > URL: https://issues.apache.org/jira/browse/SPARK-42493 > Project: Spark > Issue Type: Documentation > Components: Spark Core >Affects Versions: 3.4.0 >Reporter: Allan Folting >Priority: Major > > Python is the easiest approachable and most popular language so it should be > the primary language in examples etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42493) Spark SQL, DataFrames and Datasets Guide - make Python the first example tab
[ https://issues.apache.org/jira/browse/SPARK-42493?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42493: Assignee: Apache Spark > Spark SQL, DataFrames and Datasets Guide - make Python the first example tab > > > Key: SPARK-42493 > URL: https://issues.apache.org/jira/browse/SPARK-42493 > Project: Spark > Issue Type: Documentation > Components: Spark Core >Affects Versions: 3.4.0 >Reporter: Allan Folting >Assignee: Apache Spark >Priority: Major > > Python is the easiest approachable and most popular language so it should be > the primary language in examples etc. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42492) Add new function filter_value
[
https://issues.apache.org/jira/browse/SPARK-42492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42492:
Assignee: Apache Spark
> Add new function filter_value
> -
>
> Key: SPARK-42492
> URL: https://issues.apache.org/jira/browse/SPARK-42492
> Project: Spark
> Issue Type: New Feature
> Components: SQL
>Affects Versions: 3.3.2
>Reporter: Adam Binford
>Assignee: Apache Spark
>Priority: Major
>
> Doing data validation in Spark can lead to a lot of extra evaluations of
> expressions. This is because conditionally evaluated expressions aren't
> candidates for subexpression elimination. For example a simple expression
> such as
> {{when(validate(col), col)}}
> to only keep col if it matches some condition, will lead to col being
> evaluated twice. And if call itself is made up of a series of expensive
> expressions itself, like regular expression checks, this can lead to a lot of
> wasted computation time.
> The initial attempt to resolve this was
> https://issues.apache.org/jira/browse/SPARK-35564, adding support for
> subexpression elimination to conditionally evaluated expressions. However I
> have not been able to get that merged, so this is an alternative (though I
> believe that is still useful on top of this).
> We can add a new higher order function "filter_value" that takes the column
> you want to validate as an argument, and then a function that runs a lambda
> expression returning a boolean on whether to keep that column or not. It
> would have the same semantics as the above when expression, except it would
> guarantee to only evaluate the initial column once.
> An alternative would be to implement a real definition for the NullIf
> expression, but that would only support exact equals checks and not any
> generic condition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42492) Add new function filter_value
[
https://issues.apache.org/jira/browse/SPARK-42492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690894#comment-17690894
]
Apache Spark commented on SPARK-42492:
--
User 'Kimahriman' has created a pull request for this issue:
https://github.com/apache/spark/pull/40085
> Add new function filter_value
> -
>
> Key: SPARK-42492
> URL: https://issues.apache.org/jira/browse/SPARK-42492
> Project: Spark
> Issue Type: New Feature
> Components: SQL
>Affects Versions: 3.3.2
>Reporter: Adam Binford
>Priority: Major
>
> Doing data validation in Spark can lead to a lot of extra evaluations of
> expressions. This is because conditionally evaluated expressions aren't
> candidates for subexpression elimination. For example a simple expression
> such as
> {{when(validate(col), col)}}
> to only keep col if it matches some condition, will lead to col being
> evaluated twice. And if call itself is made up of a series of expensive
> expressions itself, like regular expression checks, this can lead to a lot of
> wasted computation time.
> The initial attempt to resolve this was
> https://issues.apache.org/jira/browse/SPARK-35564, adding support for
> subexpression elimination to conditionally evaluated expressions. However I
> have not been able to get that merged, so this is an alternative (though I
> believe that is still useful on top of this).
> We can add a new higher order function "filter_value" that takes the column
> you want to validate as an argument, and then a function that runs a lambda
> expression returning a boolean on whether to keep that column or not. It
> would have the same semantics as the above when expression, except it would
> guarantee to only evaluate the initial column once.
> An alternative would be to implement a real definition for the NullIf
> expression, but that would only support exact equals checks and not any
> generic condition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42492) Add new function filter_value
[
https://issues.apache.org/jira/browse/SPARK-42492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42492:
Assignee: (was: Apache Spark)
> Add new function filter_value
> -
>
> Key: SPARK-42492
> URL: https://issues.apache.org/jira/browse/SPARK-42492
> Project: Spark
> Issue Type: New Feature
> Components: SQL
>Affects Versions: 3.3.2
>Reporter: Adam Binford
>Priority: Major
>
> Doing data validation in Spark can lead to a lot of extra evaluations of
> expressions. This is because conditionally evaluated expressions aren't
> candidates for subexpression elimination. For example a simple expression
> such as
> {{when(validate(col), col)}}
> to only keep col if it matches some condition, will lead to col being
> evaluated twice. And if call itself is made up of a series of expensive
> expressions itself, like regular expression checks, this can lead to a lot of
> wasted computation time.
> The initial attempt to resolve this was
> https://issues.apache.org/jira/browse/SPARK-35564, adding support for
> subexpression elimination to conditionally evaluated expressions. However I
> have not been able to get that merged, so this is an alternative (though I
> believe that is still useful on top of this).
> We can add a new higher order function "filter_value" that takes the column
> you want to validate as an argument, and then a function that runs a lambda
> expression returning a boolean on whether to keep that column or not. It
> would have the same semantics as the above when expression, except it would
> guarantee to only evaluate the initial column once.
> An alternative would be to implement a real definition for the NullIf
> expression, but that would only support exact equals checks and not any
> generic condition.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42490) Upgrade protobuf-java to 3.22.0
[ https://issues.apache.org/jira/browse/SPARK-42490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690879#comment-17690879 ] Apache Spark commented on SPARK-42490: -- User 'LuciferYang' has created a pull request for this issue: https://github.com/apache/spark/pull/40084 > Upgrade protobuf-java to 3.22.0 > --- > > Key: SPARK-42490 > URL: https://issues.apache.org/jira/browse/SPARK-42490 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 3.5.0 >Reporter: Yang Jie >Priority: Major > > https://github.com/protocolbuffers/protobuf/releases/tag/v22.0 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42487) Upgrade Netty to 4.1.89
[ https://issues.apache.org/jira/browse/SPARK-42487?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690878#comment-17690878 ] Apache Spark commented on SPARK-42487: -- User 'LuciferYang' has created a pull request for this issue: https://github.com/apache/spark/pull/40081 > Upgrade Netty to 4.1.89 > --- > > Key: SPARK-42487 > URL: https://issues.apache.org/jira/browse/SPARK-42487 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 3.5.0 >Reporter: Yang Jie >Priority: Major > > This release contains a fix for two regressions that were introduced by > 4.1.88.Final: > * Don't fail on HttpObjectDecoder's maxHeaderSize greater then > (Integer.MAX_VALUE - 2) ([#13216|https://github.com/netty/netty/pull/13216]) > * dyld: Symbol not found: _netty_jni_util_JNI_OnLoad when upgrading from > 4.1.87.Final to 4.1.88.Final > ([#13214|https://github.com/netty/netty/pull/13214]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42490) Upgrade protobuf-java to 3.22.0
[ https://issues.apache.org/jira/browse/SPARK-42490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42490: Assignee: Apache Spark > Upgrade protobuf-java to 3.22.0 > --- > > Key: SPARK-42490 > URL: https://issues.apache.org/jira/browse/SPARK-42490 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 3.5.0 >Reporter: Yang Jie >Assignee: Apache Spark >Priority: Major > > https://github.com/protocolbuffers/protobuf/releases/tag/v22.0 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42490) Upgrade protobuf-java to 3.22.0
[ https://issues.apache.org/jira/browse/SPARK-42490?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42490: Assignee: (was: Apache Spark) > Upgrade protobuf-java to 3.22.0 > --- > > Key: SPARK-42490 > URL: https://issues.apache.org/jira/browse/SPARK-42490 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 3.5.0 >Reporter: Yang Jie >Priority: Major > > https://github.com/protocolbuffers/protobuf/releases/tag/v22.0 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42487) Upgrade Netty to 4.1.89
[ https://issues.apache.org/jira/browse/SPARK-42487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42487: Assignee: (was: Apache Spark) > Upgrade Netty to 4.1.89 > --- > > Key: SPARK-42487 > URL: https://issues.apache.org/jira/browse/SPARK-42487 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 3.5.0 >Reporter: Yang Jie >Priority: Major > > This release contains a fix for two regressions that were introduced by > 4.1.88.Final: > * Don't fail on HttpObjectDecoder's maxHeaderSize greater then > (Integer.MAX_VALUE - 2) ([#13216|https://github.com/netty/netty/pull/13216]) > * dyld: Symbol not found: _netty_jni_util_JNI_OnLoad when upgrading from > 4.1.87.Final to 4.1.88.Final > ([#13214|https://github.com/netty/netty/pull/13214]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42487) Upgrade Netty to 4.1.89
[ https://issues.apache.org/jira/browse/SPARK-42487?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42487: Assignee: Apache Spark > Upgrade Netty to 4.1.89 > --- > > Key: SPARK-42487 > URL: https://issues.apache.org/jira/browse/SPARK-42487 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 3.5.0 >Reporter: Yang Jie >Assignee: Apache Spark >Priority: Major > > This release contains a fix for two regressions that were introduced by > 4.1.88.Final: > * Don't fail on HttpObjectDecoder's maxHeaderSize greater then > (Integer.MAX_VALUE - 2) ([#13216|https://github.com/netty/netty/pull/13216]) > * dyld: Symbol not found: _netty_jni_util_JNI_OnLoad when upgrading from > 4.1.87.Final to 4.1.88.Final > ([#13214|https://github.com/netty/netty/pull/13214]) -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42490) Upgrade protobuf-java to 3.22.0
[ https://issues.apache.org/jira/browse/SPARK-42490?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690876#comment-17690876 ] Apache Spark commented on SPARK-42490: -- User 'LuciferYang' has created a pull request for this issue: https://github.com/apache/spark/pull/40084 > Upgrade protobuf-java to 3.22.0 > --- > > Key: SPARK-42490 > URL: https://issues.apache.org/jira/browse/SPARK-42490 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 3.5.0 >Reporter: Yang Jie >Priority: Major > > https://github.com/protocolbuffers/protobuf/releases/tag/v22.0 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42489) Upgrdae scala-parser-combinators from 2.1.1 to 2.2.0
[ https://issues.apache.org/jira/browse/SPARK-42489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42489: Assignee: Apache Spark > Upgrdae scala-parser-combinators from 2.1.1 to 2.2.0 > > > Key: SPARK-42489 > URL: https://issues.apache.org/jira/browse/SPARK-42489 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 3.5.0 >Reporter: Yang Jie >Assignee: Apache Spark >Priority: Minor > > https://github.com/scala/scala-parser-combinators/releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42489) Upgrdae scala-parser-combinators from 2.1.1 to 2.2.0
[ https://issues.apache.org/jira/browse/SPARK-42489?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42489: Assignee: (was: Apache Spark) > Upgrdae scala-parser-combinators from 2.1.1 to 2.2.0 > > > Key: SPARK-42489 > URL: https://issues.apache.org/jira/browse/SPARK-42489 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 3.5.0 >Reporter: Yang Jie >Priority: Minor > > https://github.com/scala/scala-parser-combinators/releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42489) Upgrdae scala-parser-combinators from 2.1.1 to 2.2.0
[ https://issues.apache.org/jira/browse/SPARK-42489?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690875#comment-17690875 ] Apache Spark commented on SPARK-42489: -- User 'LuciferYang' has created a pull request for this issue: https://github.com/apache/spark/pull/40083 > Upgrdae scala-parser-combinators from 2.1.1 to 2.2.0 > > > Key: SPARK-42489 > URL: https://issues.apache.org/jira/browse/SPARK-42489 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 3.5.0 >Reporter: Yang Jie >Priority: Minor > > https://github.com/scala/scala-parser-combinators/releases -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42488) Upgrade commons-crypto from 1.1.0 to 1.2.0
[ https://issues.apache.org/jira/browse/SPARK-42488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42488: Assignee: (was: Apache Spark) > Upgrade commons-crypto from 1.1.0 to 1.2.0 > -- > > Key: SPARK-42488 > URL: https://issues.apache.org/jira/browse/SPARK-42488 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 3.5.0 >Reporter: Yang Jie >Priority: Minor > > https://github.com/apache/commons-crypto/compare/rel/commons-crypto-1.1.0...rel/commons-crypto-1.2.0 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42488) Upgrade commons-crypto from 1.1.0 to 1.2.0
[ https://issues.apache.org/jira/browse/SPARK-42488?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42488: Assignee: Apache Spark > Upgrade commons-crypto from 1.1.0 to 1.2.0 > -- > > Key: SPARK-42488 > URL: https://issues.apache.org/jira/browse/SPARK-42488 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 3.5.0 >Reporter: Yang Jie >Assignee: Apache Spark >Priority: Minor > > https://github.com/apache/commons-crypto/compare/rel/commons-crypto-1.1.0...rel/commons-crypto-1.2.0 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42488) Upgrade commons-crypto from 1.1.0 to 1.2.0
[ https://issues.apache.org/jira/browse/SPARK-42488?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690874#comment-17690874 ] Apache Spark commented on SPARK-42488: -- User 'LuciferYang' has created a pull request for this issue: https://github.com/apache/spark/pull/40082 > Upgrade commons-crypto from 1.1.0 to 1.2.0 > -- > > Key: SPARK-42488 > URL: https://issues.apache.org/jira/browse/SPARK-42488 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 3.5.0 >Reporter: Yang Jie >Priority: Minor > > https://github.com/apache/commons-crypto/compare/rel/commons-crypto-1.1.0...rel/commons-crypto-1.2.0 -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42406) [PROTOBUF] Recursive field handling is incompatible with delta
[
https://issues.apache.org/jira/browse/SPARK-42406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690838#comment-17690838
]
Apache Spark commented on SPARK-42406:
--
User 'rangadi' has created a pull request for this issue:
https://github.com/apache/spark/pull/40080
> [PROTOBUF] Recursive field handling is incompatible with delta
> --
>
> Key: SPARK-42406
> URL: https://issues.apache.org/jira/browse/SPARK-42406
> Project: Spark
> Issue Type: Bug
> Components: Protobuf
>Affects Versions: 3.4.0
>Reporter: Raghu Angadi
>Assignee: Raghu Angadi
>Priority: Major
> Fix For: 3.4.0
>
>
> Protobuf deserializer (`from_protobuf()` function()) optionally supports
> recursive fields by limiting the depth to certain level. See example below.
> It assigns a 'NullType' for such a field when allowed depth is reached.
> It causes a few issues. E.g. a repeated field as in the following example
> results in a Array field with 'NullType'. Delta does not support null type in
> a complex type.
> Actually `Array[NullType]` is not really useful anyway.
> How about this fix: Drop the recursive field when the limit reached rather
> than using a NullType.
> The example below makes it clear:
> Consider a recursive Protobuf:
>
> {code:python}
> message TreeNode {
> string value = 1;
> repeated TreeNode children = 2;
> }
> {code}
> Allow depth of 2:
>
> {code:python}
> df.select(
> 'proto',
> messageName = 'TreeNode',
> options = { ... "recursive.fields.max.depth" : "2" }
> ).printSchema()
> {code}
> Schema looks like this:
> {noformat}
> root
> |– from_protobuf(proto): struct (nullable = true)|
> | |– value: string (nullable = true)|
> | |– children: array (nullable = false)|
> | | |– element: struct (containsNull = false)|
> | | | |– value: string (nullable = true)|
> | | | |– children: array (nullable = false)|
> | | | | |– element: struct (containsNull = false)|
> | | | | | |– value: string (nullable = true)|
> | | | | | |– children: array (nullable = false). [ === Proposed fix: Drop
> this field === ]|
> | | | | | | |– element: void (containsNull = false) [ === NOTICE 'void' HERE
> === ]
> {noformat}
> When we try to write this to a delta table, we get an error:
> {noformat}
> AnalysisException: Found nested NullType in column
> from_protobuf(proto).children which is of ArrayType. Delta doesn't support
> writing NullType in complex types.
> {noformat}
>
> We could just drop the field 'element' when recursion depth is reached. It is
> simpler and does not need to deal with NullType. We are ignoring the value
> anyway. There is no use in keeping the field.
> Another issue is setting for 'recursive.fields.max.depth': It is not enforced
> correctly. '0' does not make sense.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42406) [PROTOBUF] Recursive field handling is incompatible with delta
[
https://issues.apache.org/jira/browse/SPARK-42406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42406:
Assignee: Apache Spark (was: Raghu Angadi)
> [PROTOBUF] Recursive field handling is incompatible with delta
> --
>
> Key: SPARK-42406
> URL: https://issues.apache.org/jira/browse/SPARK-42406
> Project: Spark
> Issue Type: Bug
> Components: Protobuf
>Affects Versions: 3.4.0
>Reporter: Raghu Angadi
>Assignee: Apache Spark
>Priority: Major
> Fix For: 3.4.0
>
>
> Protobuf deserializer (`from_protobuf()` function()) optionally supports
> recursive fields by limiting the depth to certain level. See example below.
> It assigns a 'NullType' for such a field when allowed depth is reached.
> It causes a few issues. E.g. a repeated field as in the following example
> results in a Array field with 'NullType'. Delta does not support null type in
> a complex type.
> Actually `Array[NullType]` is not really useful anyway.
> How about this fix: Drop the recursive field when the limit reached rather
> than using a NullType.
> The example below makes it clear:
> Consider a recursive Protobuf:
>
> {code:python}
> message TreeNode {
> string value = 1;
> repeated TreeNode children = 2;
> }
> {code}
> Allow depth of 2:
>
> {code:python}
> df.select(
> 'proto',
> messageName = 'TreeNode',
> options = { ... "recursive.fields.max.depth" : "2" }
> ).printSchema()
> {code}
> Schema looks like this:
> {noformat}
> root
> |– from_protobuf(proto): struct (nullable = true)|
> | |– value: string (nullable = true)|
> | |– children: array (nullable = false)|
> | | |– element: struct (containsNull = false)|
> | | | |– value: string (nullable = true)|
> | | | |– children: array (nullable = false)|
> | | | | |– element: struct (containsNull = false)|
> | | | | | |– value: string (nullable = true)|
> | | | | | |– children: array (nullable = false). [ === Proposed fix: Drop
> this field === ]|
> | | | | | | |– element: void (containsNull = false) [ === NOTICE 'void' HERE
> === ]
> {noformat}
> When we try to write this to a delta table, we get an error:
> {noformat}
> AnalysisException: Found nested NullType in column
> from_protobuf(proto).children which is of ArrayType. Delta doesn't support
> writing NullType in complex types.
> {noformat}
>
> We could just drop the field 'element' when recursion depth is reached. It is
> simpler and does not need to deal with NullType. We are ignoring the value
> anyway. There is no use in keeping the field.
> Another issue is setting for 'recursive.fields.max.depth': It is not enforced
> correctly. '0' does not make sense.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42406) [PROTOBUF] Recursive field handling is incompatible with delta
[
https://issues.apache.org/jira/browse/SPARK-42406?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690837#comment-17690837
]
Apache Spark commented on SPARK-42406:
--
User 'rangadi' has created a pull request for this issue:
https://github.com/apache/spark/pull/40080
> [PROTOBUF] Recursive field handling is incompatible with delta
> --
>
> Key: SPARK-42406
> URL: https://issues.apache.org/jira/browse/SPARK-42406
> Project: Spark
> Issue Type: Bug
> Components: Protobuf
>Affects Versions: 3.4.0
>Reporter: Raghu Angadi
>Assignee: Raghu Angadi
>Priority: Major
> Fix For: 3.4.0
>
>
> Protobuf deserializer (`from_protobuf()` function()) optionally supports
> recursive fields by limiting the depth to certain level. See example below.
> It assigns a 'NullType' for such a field when allowed depth is reached.
> It causes a few issues. E.g. a repeated field as in the following example
> results in a Array field with 'NullType'. Delta does not support null type in
> a complex type.
> Actually `Array[NullType]` is not really useful anyway.
> How about this fix: Drop the recursive field when the limit reached rather
> than using a NullType.
> The example below makes it clear:
> Consider a recursive Protobuf:
>
> {code:python}
> message TreeNode {
> string value = 1;
> repeated TreeNode children = 2;
> }
> {code}
> Allow depth of 2:
>
> {code:python}
> df.select(
> 'proto',
> messageName = 'TreeNode',
> options = { ... "recursive.fields.max.depth" : "2" }
> ).printSchema()
> {code}
> Schema looks like this:
> {noformat}
> root
> |– from_protobuf(proto): struct (nullable = true)|
> | |– value: string (nullable = true)|
> | |– children: array (nullable = false)|
> | | |– element: struct (containsNull = false)|
> | | | |– value: string (nullable = true)|
> | | | |– children: array (nullable = false)|
> | | | | |– element: struct (containsNull = false)|
> | | | | | |– value: string (nullable = true)|
> | | | | | |– children: array (nullable = false). [ === Proposed fix: Drop
> this field === ]|
> | | | | | | |– element: void (containsNull = false) [ === NOTICE 'void' HERE
> === ]
> {noformat}
> When we try to write this to a delta table, we get an error:
> {noformat}
> AnalysisException: Found nested NullType in column
> from_protobuf(proto).children which is of ArrayType. Delta doesn't support
> writing NullType in complex types.
> {noformat}
>
> We could just drop the field 'element' when recursion depth is reached. It is
> simpler and does not need to deal with NullType. We are ignoring the value
> anyway. There is no use in keeping the field.
> Another issue is setting for 'recursive.fields.max.depth': It is not enforced
> correctly. '0' does not make sense.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42406) [PROTOBUF] Recursive field handling is incompatible with delta
[
https://issues.apache.org/jira/browse/SPARK-42406?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42406:
Assignee: Raghu Angadi (was: Apache Spark)
> [PROTOBUF] Recursive field handling is incompatible with delta
> --
>
> Key: SPARK-42406
> URL: https://issues.apache.org/jira/browse/SPARK-42406
> Project: Spark
> Issue Type: Bug
> Components: Protobuf
>Affects Versions: 3.4.0
>Reporter: Raghu Angadi
>Assignee: Raghu Angadi
>Priority: Major
> Fix For: 3.4.0
>
>
> Protobuf deserializer (`from_protobuf()` function()) optionally supports
> recursive fields by limiting the depth to certain level. See example below.
> It assigns a 'NullType' for such a field when allowed depth is reached.
> It causes a few issues. E.g. a repeated field as in the following example
> results in a Array field with 'NullType'. Delta does not support null type in
> a complex type.
> Actually `Array[NullType]` is not really useful anyway.
> How about this fix: Drop the recursive field when the limit reached rather
> than using a NullType.
> The example below makes it clear:
> Consider a recursive Protobuf:
>
> {code:python}
> message TreeNode {
> string value = 1;
> repeated TreeNode children = 2;
> }
> {code}
> Allow depth of 2:
>
> {code:python}
> df.select(
> 'proto',
> messageName = 'TreeNode',
> options = { ... "recursive.fields.max.depth" : "2" }
> ).printSchema()
> {code}
> Schema looks like this:
> {noformat}
> root
> |– from_protobuf(proto): struct (nullable = true)|
> | |– value: string (nullable = true)|
> | |– children: array (nullable = false)|
> | | |– element: struct (containsNull = false)|
> | | | |– value: string (nullable = true)|
> | | | |– children: array (nullable = false)|
> | | | | |– element: struct (containsNull = false)|
> | | | | | |– value: string (nullable = true)|
> | | | | | |– children: array (nullable = false). [ === Proposed fix: Drop
> this field === ]|
> | | | | | | |– element: void (containsNull = false) [ === NOTICE 'void' HERE
> === ]
> {noformat}
> When we try to write this to a delta table, we get an error:
> {noformat}
> AnalysisException: Found nested NullType in column
> from_protobuf(proto).children which is of ArrayType. Delta doesn't support
> writing NullType in complex types.
> {noformat}
>
> We could just drop the field 'element' when recursion depth is reached. It is
> simpler and does not need to deal with NullType. We are ignoring the value
> anyway. There is no use in keeping the field.
> Another issue is setting for 'recursive.fields.max.depth': It is not enforced
> correctly. '0' does not make sense.
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42486) Upgrade ZooKeeper from 3.6.3 to 3.6.4
[ https://issues.apache.org/jira/browse/SPARK-42486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690801#comment-17690801 ] Apache Spark commented on SPARK-42486: -- User 'bjornjorgensen' has created a pull request for this issue: https://github.com/apache/spark/pull/40079 > Upgrade ZooKeeper from 3.6.3 to 3.6.4 > - > > Key: SPARK-42486 > URL: https://issues.apache.org/jira/browse/SPARK-42486 > Project: Spark > Issue Type: Dependency upgrade > Components: Build >Affects Versions: 3.5.0 >Reporter: Bjørn Jørgensen >Priority: Major > > [ZooKeeper 3.6.3 is EoL since 30th December, > 2022|https://zookeeper.apache.org/releases.html] > [Release notes|https://zookeeper.apache.org/doc/r3.6.4/releasenotes.html] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42486) Upgrade ZooKeeper from 3.6.3 to 3.6.4
[ https://issues.apache.org/jira/browse/SPARK-42486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42486: Assignee: Apache Spark > Upgrade ZooKeeper from 3.6.3 to 3.6.4 > - > > Key: SPARK-42486 > URL: https://issues.apache.org/jira/browse/SPARK-42486 > Project: Spark > Issue Type: Dependency upgrade > Components: Build >Affects Versions: 3.5.0 >Reporter: Bjørn Jørgensen >Assignee: Apache Spark >Priority: Major > > [ZooKeeper 3.6.3 is EoL since 30th December, > 2022|https://zookeeper.apache.org/releases.html] > [Release notes|https://zookeeper.apache.org/doc/r3.6.4/releasenotes.html] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42486) Upgrade ZooKeeper from 3.6.3 to 3.6.4
[ https://issues.apache.org/jira/browse/SPARK-42486?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42486: Assignee: (was: Apache Spark) > Upgrade ZooKeeper from 3.6.3 to 3.6.4 > - > > Key: SPARK-42486 > URL: https://issues.apache.org/jira/browse/SPARK-42486 > Project: Spark > Issue Type: Dependency upgrade > Components: Build >Affects Versions: 3.5.0 >Reporter: Bjørn Jørgensen >Priority: Major > > [ZooKeeper 3.6.3 is EoL since 30th December, > 2022|https://zookeeper.apache.org/releases.html] > [Release notes|https://zookeeper.apache.org/doc/r3.6.4/releasenotes.html] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42486) Upgrade ZooKeeper from 3.6.3 to 3.6.4
[ https://issues.apache.org/jira/browse/SPARK-42486?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690800#comment-17690800 ] Apache Spark commented on SPARK-42486: -- User 'bjornjorgensen' has created a pull request for this issue: https://github.com/apache/spark/pull/40079 > Upgrade ZooKeeper from 3.6.3 to 3.6.4 > - > > Key: SPARK-42486 > URL: https://issues.apache.org/jira/browse/SPARK-42486 > Project: Spark > Issue Type: Dependency upgrade > Components: Build >Affects Versions: 3.5.0 >Reporter: Bjørn Jørgensen >Priority: Major > > [ZooKeeper 3.6.3 is EoL since 30th December, > 2022|https://zookeeper.apache.org/releases.html] > [Release notes|https://zookeeper.apache.org/doc/r3.6.4/releasenotes.html] -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42430) Add documentation for TimestampNTZ type
[ https://issues.apache.org/jira/browse/SPARK-42430?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690668#comment-17690668 ] Apache Spark commented on SPARK-42430: -- User 'gengliangwang' has created a pull request for this issue: https://github.com/apache/spark/pull/40074 > Add documentation for TimestampNTZ type > --- > > Key: SPARK-42430 > URL: https://issues.apache.org/jira/browse/SPARK-42430 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.4.0 >Reporter: Gengliang Wang >Assignee: Gengliang Wang >Priority: Major > Fix For: 3.4.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42048) Different column name of lit(np.int8)
[
https://issues.apache.org/jira/browse/SPARK-42048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42048:
Assignee: Apache Spark
> Different column name of lit(np.int8)
> -
>
> Key: SPARK-42048
> URL: https://issues.apache.org/jira/browse/SPARK-42048
> Project: Spark
> Issue Type: Sub-task
> Components: Connect, PySpark
>Affects Versions: 3.4.0
>Reporter: Ruifeng Zheng
>Assignee: Apache Spark
>Priority: Minor
>
> {code:java}
> ('1', 'tinyint')
> ('CAST(1 AS TINYINT)', 'tinyint')
> - [('1', 'tinyint')]
> + [('CAST(1 AS TINYINT)', 'tinyint')]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42048) Different column name of lit(np.int8)
[
https://issues.apache.org/jira/browse/SPARK-42048?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690667#comment-17690667
]
Apache Spark commented on SPARK-42048:
--
User 'ueshin' has created a pull request for this issue:
https://github.com/apache/spark/pull/40076
> Different column name of lit(np.int8)
> -
>
> Key: SPARK-42048
> URL: https://issues.apache.org/jira/browse/SPARK-42048
> Project: Spark
> Issue Type: Sub-task
> Components: Connect, PySpark
>Affects Versions: 3.4.0
>Reporter: Ruifeng Zheng
>Priority: Minor
>
> {code:java}
> ('1', 'tinyint')
> ('CAST(1 AS TINYINT)', 'tinyint')
> - [('1', 'tinyint')]
> + [('CAST(1 AS TINYINT)', 'tinyint')]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42048) Different column name of lit(np.int8)
[
https://issues.apache.org/jira/browse/SPARK-42048?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42048:
Assignee: (was: Apache Spark)
> Different column name of lit(np.int8)
> -
>
> Key: SPARK-42048
> URL: https://issues.apache.org/jira/browse/SPARK-42048
> Project: Spark
> Issue Type: Sub-task
> Components: Connect, PySpark
>Affects Versions: 3.4.0
>Reporter: Ruifeng Zheng
>Priority: Minor
>
> {code:java}
> ('1', 'tinyint')
> ('CAST(1 AS TINYINT)', 'tinyint')
> - [('1', 'tinyint')]
> + [('CAST(1 AS TINYINT)', 'tinyint')]
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42484) Better logging for UnsafeRowUtils
[ https://issues.apache.org/jira/browse/SPARK-42484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42484: Assignee: Apache Spark > Better logging for UnsafeRowUtils > - > > Key: SPARK-42484 > URL: https://issues.apache.org/jira/browse/SPARK-42484 > Project: Spark > Issue Type: Improvement > Components: Structured Streaming >Affects Versions: 3.3.3 >Reporter: Wei Liu >Assignee: Apache Spark >Priority: Major > > Current `UnsafeRowUtils.validateStructuralIntegrity` only returns a boolean, > making it hard to track exactly where the problem is. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42484) Better logging for UnsafeRowUtils
[ https://issues.apache.org/jira/browse/SPARK-42484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690662#comment-17690662 ] Apache Spark commented on SPARK-42484: -- User 'WweiL' has created a pull request for this issue: https://github.com/apache/spark/pull/40073 > Better logging for UnsafeRowUtils > - > > Key: SPARK-42484 > URL: https://issues.apache.org/jira/browse/SPARK-42484 > Project: Spark > Issue Type: Improvement > Components: Structured Streaming >Affects Versions: 3.3.3 >Reporter: Wei Liu >Priority: Major > > Current `UnsafeRowUtils.validateStructuralIntegrity` only returns a boolean, > making it hard to track exactly where the problem is. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42484) Better logging for UnsafeRowUtils
[ https://issues.apache.org/jira/browse/SPARK-42484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690663#comment-17690663 ] Apache Spark commented on SPARK-42484: -- User 'WweiL' has created a pull request for this issue: https://github.com/apache/spark/pull/40073 > Better logging for UnsafeRowUtils > - > > Key: SPARK-42484 > URL: https://issues.apache.org/jira/browse/SPARK-42484 > Project: Spark > Issue Type: Improvement > Components: Structured Streaming >Affects Versions: 3.3.3 >Reporter: Wei Liu >Priority: Major > > Current `UnsafeRowUtils.validateStructuralIntegrity` only returns a boolean, > making it hard to track exactly where the problem is. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42484) Better logging for UnsafeRowUtils
[ https://issues.apache.org/jira/browse/SPARK-42484?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42484: Assignee: (was: Apache Spark) > Better logging for UnsafeRowUtils > - > > Key: SPARK-42484 > URL: https://issues.apache.org/jira/browse/SPARK-42484 > Project: Spark > Issue Type: Improvement > Components: Structured Streaming >Affects Versions: 3.3.3 >Reporter: Wei Liu >Priority: Major > > Current `UnsafeRowUtils.validateStructuralIntegrity` only returns a boolean, > making it hard to track exactly where the problem is. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42483) Regenerate benchmark results
[ https://issues.apache.org/jira/browse/SPARK-42483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690650#comment-17690650 ] Apache Spark commented on SPARK-42483: -- User 'dongjoon-hyun' has created a pull request for this issue: https://github.com/apache/spark/pull/40072 > Regenerate benchmark results > > > Key: SPARK-42483 > URL: https://issues.apache.org/jira/browse/SPARK-42483 > Project: Spark > Issue Type: Test > Components: Tests >Affects Versions: 3.5.0 >Reporter: Dongjoon Hyun >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42483) Regenerate benchmark results
[ https://issues.apache.org/jira/browse/SPARK-42483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42483: Assignee: Apache Spark > Regenerate benchmark results > > > Key: SPARK-42483 > URL: https://issues.apache.org/jira/browse/SPARK-42483 > Project: Spark > Issue Type: Test > Components: Tests >Affects Versions: 3.5.0 >Reporter: Dongjoon Hyun >Assignee: Apache Spark >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42483) Regenerate benchmark results
[ https://issues.apache.org/jira/browse/SPARK-42483?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690649#comment-17690649 ] Apache Spark commented on SPARK-42483: -- User 'dongjoon-hyun' has created a pull request for this issue: https://github.com/apache/spark/pull/40072 > Regenerate benchmark results > > > Key: SPARK-42483 > URL: https://issues.apache.org/jira/browse/SPARK-42483 > Project: Spark > Issue Type: Test > Components: Tests >Affects Versions: 3.5.0 >Reporter: Dongjoon Hyun >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42483) Regenerate benchmark results
[ https://issues.apache.org/jira/browse/SPARK-42483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42483: Assignee: (was: Apache Spark) > Regenerate benchmark results > > > Key: SPARK-42483 > URL: https://issues.apache.org/jira/browse/SPARK-42483 > Project: Spark > Issue Type: Test > Components: Tests >Affects Versions: 3.5.0 >Reporter: Dongjoon Hyun >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42482) Scala client Write API V1
[ https://issues.apache.org/jira/browse/SPARK-42482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690600#comment-17690600 ] Apache Spark commented on SPARK-42482: -- User 'zhenlineo' has created a pull request for this issue: https://github.com/apache/spark/pull/40061 > Scala client Write API V1 > - > > Key: SPARK-42482 > URL: https://issues.apache.org/jira/browse/SPARK-42482 > Project: Spark > Issue Type: Improvement > Components: Connect >Affects Versions: 3.4.0 >Reporter: Zhen Li >Priority: Major > > Add basic Dataset#write API for Scala client. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42482) Scala client Write API V1
[ https://issues.apache.org/jira/browse/SPARK-42482?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690599#comment-17690599 ] Apache Spark commented on SPARK-42482: -- User 'zhenlineo' has created a pull request for this issue: https://github.com/apache/spark/pull/40061 > Scala client Write API V1 > - > > Key: SPARK-42482 > URL: https://issues.apache.org/jira/browse/SPARK-42482 > Project: Spark > Issue Type: Improvement > Components: Connect >Affects Versions: 3.4.0 >Reporter: Zhen Li >Priority: Major > > Add basic Dataset#write API for Scala client. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42482) Scala client Write API V1
[ https://issues.apache.org/jira/browse/SPARK-42482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42482: Assignee: (was: Apache Spark) > Scala client Write API V1 > - > > Key: SPARK-42482 > URL: https://issues.apache.org/jira/browse/SPARK-42482 > Project: Spark > Issue Type: Improvement > Components: Connect >Affects Versions: 3.4.0 >Reporter: Zhen Li >Priority: Major > > Add basic Dataset#write API for Scala client. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42482) Scala client Write API V1
[ https://issues.apache.org/jira/browse/SPARK-42482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42482: Assignee: Apache Spark > Scala client Write API V1 > - > > Key: SPARK-42482 > URL: https://issues.apache.org/jira/browse/SPARK-42482 > Project: Spark > Issue Type: Improvement > Components: Connect >Affects Versions: 3.4.0 >Reporter: Zhen Li >Assignee: Apache Spark >Priority: Major > > Add basic Dataset#write API for Scala client. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-41818) Support DataFrameWriter.saveAsTable
[
https://issues.apache.org/jira/browse/SPARK-41818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690590#comment-17690590
]
Apache Spark commented on SPARK-41818:
--
User 'ueshin' has created a pull request for this issue:
https://github.com/apache/spark/pull/40071
> Support DataFrameWriter.saveAsTable
> ---
>
> Key: SPARK-41818
> URL: https://issues.apache.org/jira/browse/SPARK-41818
> Project: Spark
> Issue Type: Sub-task
> Components: Connect
>Affects Versions: 3.4.0
>Reporter: Sandeep Singh
>Assignee: Takuya Ueshin
>Priority: Major
> Fix For: 3.4.0
>
>
> {code:java}
> File
> "/Users/s.singh/personal/spark-oss/python/pyspark/sql/connect/readwriter.py",
> line 369, in pyspark.sql.connect.readwriter.DataFrameWriter.insertInto
> Failed example:
> df.write.saveAsTable("tblA")
> Exception raised:
> Traceback (most recent call last):
> File
> "/usr/local/Cellar/python@3.10/3.10.8/Frameworks/Python.framework/Versions/3.10/lib/python3.10/doctest.py",
> line 1350, in __run
> exec(compile(example.source, filename, "single",
> File " pyspark.sql.connect.readwriter.DataFrameWriter.insertInto[2]>", line 1, in
>
> df.write.saveAsTable("tblA")
> File
> "/Users/s.singh/personal/spark-oss/python/pyspark/sql/connect/readwriter.py",
> line 350, in saveAsTable
>
> self._spark.client.execute_command(self._write.command(self._spark.client))
> File
> "/Users/s.singh/personal/spark-oss/python/pyspark/sql/connect/client.py",
> line 459, in execute_command
> self._execute(req)
> File
> "/Users/s.singh/personal/spark-oss/python/pyspark/sql/connect/client.py",
> line 547, in _execute
> self._handle_error(rpc_error)
> File
> "/Users/s.singh/personal/spark-oss/python/pyspark/sql/connect/client.py",
> line 623, in _handle_error
> raise SparkConnectException(status.message, info.reason) from None
> pyspark.sql.connect.client.SparkConnectException:
> (java.lang.ClassNotFoundException) .DefaultSource{code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42481) Implement agg.{max,min,mean,count,avg,sum}
[
https://issues.apache.org/jira/browse/SPARK-42481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42481:
Assignee: Apache Spark (was: Rui Wang)
> Implement agg.{max,min,mean,count,avg,sum}
> --
>
> Key: SPARK-42481
> URL: https://issues.apache.org/jira/browse/SPARK-42481
> Project: Spark
> Issue Type: Sub-task
> Components: Connect
>Affects Versions: 3.4.0
>Reporter: Rui Wang
>Assignee: Apache Spark
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42481) Implement agg.{max,min,mean,count,avg,sum}
[
https://issues.apache.org/jira/browse/SPARK-42481?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690588#comment-17690588
]
Apache Spark commented on SPARK-42481:
--
User 'amaliujia' has created a pull request for this issue:
https://github.com/apache/spark/pull/40070
> Implement agg.{max,min,mean,count,avg,sum}
> --
>
> Key: SPARK-42481
> URL: https://issues.apache.org/jira/browse/SPARK-42481
> Project: Spark
> Issue Type: Sub-task
> Components: Connect
>Affects Versions: 3.4.0
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42481) Implement agg.{max,min,mean,count,avg,sum}
[
https://issues.apache.org/jira/browse/SPARK-42481?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42481:
Assignee: Rui Wang (was: Apache Spark)
> Implement agg.{max,min,mean,count,avg,sum}
> --
>
> Key: SPARK-42481
> URL: https://issues.apache.org/jira/browse/SPARK-42481
> Project: Spark
> Issue Type: Sub-task
> Components: Connect
>Affects Versions: 3.4.0
>Reporter: Rui Wang
>Assignee: Rui Wang
>Priority: Major
>
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42480) Improve the performance of drop partitions
[ https://issues.apache.org/jira/browse/SPARK-42480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42480: Assignee: (was: Apache Spark) > Improve the performance of drop partitions > -- > > Key: SPARK-42480 > URL: https://issues.apache.org/jira/browse/SPARK-42480 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.3.2 >Reporter: Wechar >Priority: Major > > Currently to drop the matching partitions, Spark will first get all matching > Partition objects from Hive metastore, and just use the partition values of > these Partition objects. > We can get the matching partition names instead of the partition objects for > the following reasons: > 1. we can also get partition values through a partition name (like a=1/b=2) > 2. the byte size of partition name is much smaller than partition object, > which will help improve the performance of drop partitions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42480) Improve the performance of drop partitions
[ https://issues.apache.org/jira/browse/SPARK-42480?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690534#comment-17690534 ] Apache Spark commented on SPARK-42480: -- User 'wecharyu' has created a pull request for this issue: https://github.com/apache/spark/pull/40069 > Improve the performance of drop partitions > -- > > Key: SPARK-42480 > URL: https://issues.apache.org/jira/browse/SPARK-42480 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.3.2 >Reporter: Wechar >Priority: Major > > Currently to drop the matching partitions, Spark will first get all matching > Partition objects from Hive metastore, and just use the partition values of > these Partition objects. > We can get the matching partition names instead of the partition objects for > the following reasons: > 1. we can also get partition values through a partition name (like a=1/b=2) > 2. the byte size of partition name is much smaller than partition object, > which will help improve the performance of drop partitions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42480) Improve the performance of drop partitions
[ https://issues.apache.org/jira/browse/SPARK-42480?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42480: Assignee: Apache Spark > Improve the performance of drop partitions > -- > > Key: SPARK-42480 > URL: https://issues.apache.org/jira/browse/SPARK-42480 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.3.2 >Reporter: Wechar >Assignee: Apache Spark >Priority: Major > > Currently to drop the matching partitions, Spark will first get all matching > Partition objects from Hive metastore, and just use the partition values of > these Partition objects. > We can get the matching partition names instead of the partition objects for > the following reasons: > 1. we can also get partition values through a partition name (like a=1/b=2) > 2. the byte size of partition name is much smaller than partition object, > which will help improve the performance of drop partitions. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42380) Upgrade maven to 3.9.0
[
https://issues.apache.org/jira/browse/SPARK-42380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42380:
Assignee: (was: Apache Spark)
> Upgrade maven to 3.9.0
> --
>
> Key: SPARK-42380
> URL: https://issues.apache.org/jira/browse/SPARK-42380
> Project: Spark
> Issue Type: Improvement
> Components: Build
>Affects Versions: 3.5.0
>Reporter: Yang Jie
>Priority: Minor
>
> {code:java}
> [ERROR] An error occurred attempting to read POM
> org.codehaus.plexus.util.xml.pull.XmlPullParserException: UTF-8 BOM plus xml
> decl of ISO-8859-1 is incompatible (position: START_DOCUMENT seen version="1.0" encoding="ISO-8859-1"... @1:42)
> at org.codehaus.plexus.util.xml.pull.MXParser.parseXmlDeclWithVersion
> (MXParser.java:3423)
> at org.codehaus.plexus.util.xml.pull.MXParser.parseXmlDecl
> (MXParser.java:3345)
> at org.codehaus.plexus.util.xml.pull.MXParser.parsePI (MXParser.java:3197)
> at org.codehaus.plexus.util.xml.pull.MXParser.parseProlog
> (MXParser.java:1828)
> at org.codehaus.plexus.util.xml.pull.MXParser.nextImpl
> (MXParser.java:1757)
> at org.codehaus.plexus.util.xml.pull.MXParser.next (MXParser.java:1375)
> at org.apache.maven.model.io.xpp3.MavenXpp3Reader.read
> (MavenXpp3Reader.java:3940)
> at org.apache.maven.model.io.xpp3.MavenXpp3Reader.read
> (MavenXpp3Reader.java:612)
> at org.apache.maven.model.io.xpp3.MavenXpp3Reader.read
> (MavenXpp3Reader.java:627)
> at org.cyclonedx.maven.BaseCycloneDxMojo.readPom
> (BaseCycloneDxMojo.java:759)
> at org.cyclonedx.maven.BaseCycloneDxMojo.readPom
> (BaseCycloneDxMojo.java:746)
> at org.cyclonedx.maven.BaseCycloneDxMojo.retrieveParentProject
> (BaseCycloneDxMojo.java:694)
> at org.cyclonedx.maven.BaseCycloneDxMojo.getClosestMetadata
> (BaseCycloneDxMojo.java:524)
> at org.cyclonedx.maven.BaseCycloneDxMojo.convert
> (BaseCycloneDxMojo.java:481)
> at org.cyclonedx.maven.CycloneDxMojo.execute (CycloneDxMojo.java:70)
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo
> (DefaultBuildPluginManager.java:126)
> at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2
> (MojoExecutor.java:342)
> at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute
> (MojoExecutor.java:330)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:213)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:175)
> at org.apache.maven.lifecycle.internal.MojoExecutor.access$000
> (MojoExecutor.java:76)
> at org.apache.maven.lifecycle.internal.MojoExecutor$1.run
> (MojoExecutor.java:163)
> at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute
> (DefaultMojosExecutionStrategy.java:39)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:160)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:105)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:73)
> at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
> (SingleThreadedBuilder.java:53)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
> (LifecycleStarter.java:118)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:260)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:172)
> at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:100)
> at org.apache.maven.cli.MavenCli.execute (MavenCli.java:821)
> at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:270)
> at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
> at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke (Method.java:498)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
> (Launcher.java:282)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launch
> (Launcher.java:225)
> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
> (Launcher.java:406)
> at org.codehaus.plexus.classworlds.launcher.Launcher.main
> (Launcher.java:347)
> {code}
> A existing problem
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42380) Upgrade maven to 3.9.0
[
https://issues.apache.org/jira/browse/SPARK-42380?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690516#comment-17690516
]
Apache Spark commented on SPARK-42380:
--
User 'LuciferYang' has created a pull request for this issue:
https://github.com/apache/spark/pull/40068
> Upgrade maven to 3.9.0
> --
>
> Key: SPARK-42380
> URL: https://issues.apache.org/jira/browse/SPARK-42380
> Project: Spark
> Issue Type: Improvement
> Components: Build
>Affects Versions: 3.5.0
>Reporter: Yang Jie
>Priority: Minor
>
> {code:java}
> [ERROR] An error occurred attempting to read POM
> org.codehaus.plexus.util.xml.pull.XmlPullParserException: UTF-8 BOM plus xml
> decl of ISO-8859-1 is incompatible (position: START_DOCUMENT seen version="1.0" encoding="ISO-8859-1"... @1:42)
> at org.codehaus.plexus.util.xml.pull.MXParser.parseXmlDeclWithVersion
> (MXParser.java:3423)
> at org.codehaus.plexus.util.xml.pull.MXParser.parseXmlDecl
> (MXParser.java:3345)
> at org.codehaus.plexus.util.xml.pull.MXParser.parsePI (MXParser.java:3197)
> at org.codehaus.plexus.util.xml.pull.MXParser.parseProlog
> (MXParser.java:1828)
> at org.codehaus.plexus.util.xml.pull.MXParser.nextImpl
> (MXParser.java:1757)
> at org.codehaus.plexus.util.xml.pull.MXParser.next (MXParser.java:1375)
> at org.apache.maven.model.io.xpp3.MavenXpp3Reader.read
> (MavenXpp3Reader.java:3940)
> at org.apache.maven.model.io.xpp3.MavenXpp3Reader.read
> (MavenXpp3Reader.java:612)
> at org.apache.maven.model.io.xpp3.MavenXpp3Reader.read
> (MavenXpp3Reader.java:627)
> at org.cyclonedx.maven.BaseCycloneDxMojo.readPom
> (BaseCycloneDxMojo.java:759)
> at org.cyclonedx.maven.BaseCycloneDxMojo.readPom
> (BaseCycloneDxMojo.java:746)
> at org.cyclonedx.maven.BaseCycloneDxMojo.retrieveParentProject
> (BaseCycloneDxMojo.java:694)
> at org.cyclonedx.maven.BaseCycloneDxMojo.getClosestMetadata
> (BaseCycloneDxMojo.java:524)
> at org.cyclonedx.maven.BaseCycloneDxMojo.convert
> (BaseCycloneDxMojo.java:481)
> at org.cyclonedx.maven.CycloneDxMojo.execute (CycloneDxMojo.java:70)
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo
> (DefaultBuildPluginManager.java:126)
> at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2
> (MojoExecutor.java:342)
> at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute
> (MojoExecutor.java:330)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:213)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:175)
> at org.apache.maven.lifecycle.internal.MojoExecutor.access$000
> (MojoExecutor.java:76)
> at org.apache.maven.lifecycle.internal.MojoExecutor$1.run
> (MojoExecutor.java:163)
> at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute
> (DefaultMojosExecutionStrategy.java:39)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:160)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:105)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:73)
> at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
> (SingleThreadedBuilder.java:53)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
> (LifecycleStarter.java:118)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:260)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:172)
> at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:100)
> at org.apache.maven.cli.MavenCli.execute (MavenCli.java:821)
> at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:270)
> at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
> at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke (Method.java:498)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
> (Launcher.java:282)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launch
> (Launcher.java:225)
> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
> (Launcher.java:406)
> at org.codehaus.plexus.classworlds.launcher.Launcher.main
> (Launcher.java:347)
> {code}
> A existing problem
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands,
[jira] [Assigned] (SPARK-42380) Upgrade maven to 3.9.0
[
https://issues.apache.org/jira/browse/SPARK-42380?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42380:
Assignee: Apache Spark
> Upgrade maven to 3.9.0
> --
>
> Key: SPARK-42380
> URL: https://issues.apache.org/jira/browse/SPARK-42380
> Project: Spark
> Issue Type: Improvement
> Components: Build
>Affects Versions: 3.5.0
>Reporter: Yang Jie
>Assignee: Apache Spark
>Priority: Minor
>
> {code:java}
> [ERROR] An error occurred attempting to read POM
> org.codehaus.plexus.util.xml.pull.XmlPullParserException: UTF-8 BOM plus xml
> decl of ISO-8859-1 is incompatible (position: START_DOCUMENT seen version="1.0" encoding="ISO-8859-1"... @1:42)
> at org.codehaus.plexus.util.xml.pull.MXParser.parseXmlDeclWithVersion
> (MXParser.java:3423)
> at org.codehaus.plexus.util.xml.pull.MXParser.parseXmlDecl
> (MXParser.java:3345)
> at org.codehaus.plexus.util.xml.pull.MXParser.parsePI (MXParser.java:3197)
> at org.codehaus.plexus.util.xml.pull.MXParser.parseProlog
> (MXParser.java:1828)
> at org.codehaus.plexus.util.xml.pull.MXParser.nextImpl
> (MXParser.java:1757)
> at org.codehaus.plexus.util.xml.pull.MXParser.next (MXParser.java:1375)
> at org.apache.maven.model.io.xpp3.MavenXpp3Reader.read
> (MavenXpp3Reader.java:3940)
> at org.apache.maven.model.io.xpp3.MavenXpp3Reader.read
> (MavenXpp3Reader.java:612)
> at org.apache.maven.model.io.xpp3.MavenXpp3Reader.read
> (MavenXpp3Reader.java:627)
> at org.cyclonedx.maven.BaseCycloneDxMojo.readPom
> (BaseCycloneDxMojo.java:759)
> at org.cyclonedx.maven.BaseCycloneDxMojo.readPom
> (BaseCycloneDxMojo.java:746)
> at org.cyclonedx.maven.BaseCycloneDxMojo.retrieveParentProject
> (BaseCycloneDxMojo.java:694)
> at org.cyclonedx.maven.BaseCycloneDxMojo.getClosestMetadata
> (BaseCycloneDxMojo.java:524)
> at org.cyclonedx.maven.BaseCycloneDxMojo.convert
> (BaseCycloneDxMojo.java:481)
> at org.cyclonedx.maven.CycloneDxMojo.execute (CycloneDxMojo.java:70)
> at org.apache.maven.plugin.DefaultBuildPluginManager.executeMojo
> (DefaultBuildPluginManager.java:126)
> at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute2
> (MojoExecutor.java:342)
> at org.apache.maven.lifecycle.internal.MojoExecutor.doExecute
> (MojoExecutor.java:330)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:213)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:175)
> at org.apache.maven.lifecycle.internal.MojoExecutor.access$000
> (MojoExecutor.java:76)
> at org.apache.maven.lifecycle.internal.MojoExecutor$1.run
> (MojoExecutor.java:163)
> at org.apache.maven.plugin.DefaultMojosExecutionStrategy.execute
> (DefaultMojosExecutionStrategy.java:39)
> at org.apache.maven.lifecycle.internal.MojoExecutor.execute
> (MojoExecutor.java:160)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:105)
> at
> org.apache.maven.lifecycle.internal.LifecycleModuleBuilder.buildProject
> (LifecycleModuleBuilder.java:73)
> at
> org.apache.maven.lifecycle.internal.builder.singlethreaded.SingleThreadedBuilder.build
> (SingleThreadedBuilder.java:53)
> at org.apache.maven.lifecycle.internal.LifecycleStarter.execute
> (LifecycleStarter.java:118)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:260)
> at org.apache.maven.DefaultMaven.doExecute (DefaultMaven.java:172)
> at org.apache.maven.DefaultMaven.execute (DefaultMaven.java:100)
> at org.apache.maven.cli.MavenCli.execute (MavenCli.java:821)
> at org.apache.maven.cli.MavenCli.doMain (MavenCli.java:270)
> at org.apache.maven.cli.MavenCli.main (MavenCli.java:192)
> at sun.reflect.NativeMethodAccessorImpl.invoke0 (Native Method)
> at sun.reflect.NativeMethodAccessorImpl.invoke
> (NativeMethodAccessorImpl.java:62)
> at sun.reflect.DelegatingMethodAccessorImpl.invoke
> (DelegatingMethodAccessorImpl.java:43)
> at java.lang.reflect.Method.invoke (Method.java:498)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launchEnhanced
> (Launcher.java:282)
> at org.codehaus.plexus.classworlds.launcher.Launcher.launch
> (Launcher.java:225)
> at org.codehaus.plexus.classworlds.launcher.Launcher.mainWithExitCode
> (Launcher.java:406)
> at org.codehaus.plexus.classworlds.launcher.Launcher.main
> (Launcher.java:347)
> {code}
> A existing problem
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42476) Spark Connect API reference.
[ https://issues.apache.org/jira/browse/SPARK-42476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42476: Assignee: Apache Spark > Spark Connect API reference. > > > Key: SPARK-42476 > URL: https://issues.apache.org/jira/browse/SPARK-42476 > Project: Spark > Issue Type: Sub-task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Haejoon Lee >Assignee: Apache Spark >Priority: Major > > We need an API documents for Spark Connect such as other components. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42476) Spark Connect API reference.
[ https://issues.apache.org/jira/browse/SPARK-42476?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690493#comment-17690493 ] Apache Spark commented on SPARK-42476: -- User 'itholic' has created a pull request for this issue: https://github.com/apache/spark/pull/40067 > Spark Connect API reference. > > > Key: SPARK-42476 > URL: https://issues.apache.org/jira/browse/SPARK-42476 > Project: Spark > Issue Type: Sub-task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Haejoon Lee >Priority: Major > > We need an API documents for Spark Connect such as other components. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42476) Spark Connect API reference.
[ https://issues.apache.org/jira/browse/SPARK-42476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42476: Assignee: (was: Apache Spark) > Spark Connect API reference. > > > Key: SPARK-42476 > URL: https://issues.apache.org/jira/browse/SPARK-42476 > Project: Spark > Issue Type: Sub-task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Haejoon Lee >Priority: Major > > We need an API documents for Spark Connect such as other components. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42382) Upgrade `cyclonedx-maven-plugin` to 2.7.5
[ https://issues.apache.org/jira/browse/SPARK-42382?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690445#comment-17690445 ] Apache Spark commented on SPARK-42382: -- User 'LuciferYang' has created a pull request for this issue: https://github.com/apache/spark/pull/40065 > Upgrade `cyclonedx-maven-plugin` to 2.7.5 > - > > Key: SPARK-42382 > URL: https://issues.apache.org/jira/browse/SPARK-42382 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 3.5.0 >Reporter: Yang Jie >Priority: Minor > > [https://github.com/CycloneDX/cyclonedx-maven-plugin/releases/tag/cyclonedx-maven-plugin-2.7.4] > [https://github.com/CycloneDX/cyclonedx-maven-plugin/releases/tag/cyclonedx-maven-plugin-2.7.5] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42382) Upgrade `cyclonedx-maven-plugin` to 2.7.5
[ https://issues.apache.org/jira/browse/SPARK-42382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42382: Assignee: Apache Spark > Upgrade `cyclonedx-maven-plugin` to 2.7.5 > - > > Key: SPARK-42382 > URL: https://issues.apache.org/jira/browse/SPARK-42382 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 3.5.0 >Reporter: Yang Jie >Assignee: Apache Spark >Priority: Minor > > [https://github.com/CycloneDX/cyclonedx-maven-plugin/releases/tag/cyclonedx-maven-plugin-2.7.4] > [https://github.com/CycloneDX/cyclonedx-maven-plugin/releases/tag/cyclonedx-maven-plugin-2.7.5] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42382) Upgrade `cyclonedx-maven-plugin` to 2.7.5
[ https://issues.apache.org/jira/browse/SPARK-42382?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42382: Assignee: (was: Apache Spark) > Upgrade `cyclonedx-maven-plugin` to 2.7.5 > - > > Key: SPARK-42382 > URL: https://issues.apache.org/jira/browse/SPARK-42382 > Project: Spark > Issue Type: Improvement > Components: Build >Affects Versions: 3.5.0 >Reporter: Yang Jie >Priority: Minor > > [https://github.com/CycloneDX/cyclonedx-maven-plugin/releases/tag/cyclonedx-maven-plugin-2.7.4] > [https://github.com/CycloneDX/cyclonedx-maven-plugin/releases/tag/cyclonedx-maven-plugin-2.7.5] > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42478) Make a serializable jobTrackerId instead of a non-serializable JobID in FileWriterFactory
[ https://issues.apache.org/jira/browse/SPARK-42478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42478: Assignee: (was: Apache Spark) > Make a serializable jobTrackerId instead of a non-serializable JobID in > FileWriterFactory > - > > Key: SPARK-42478 > URL: https://issues.apache.org/jira/browse/SPARK-42478 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.3.2 >Reporter: Yi kaifei >Priority: Major > > https://issues.apache.org/jira/browse/SPARK-41448 make consistent MR job IDs > in FileBatchWriter and FileFormatWriter, but it breaks a serializable issue, > JobId is non-serializable -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42478) Make a serializable jobTrackerId instead of a non-serializable JobID in FileWriterFactory
[ https://issues.apache.org/jira/browse/SPARK-42478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42478: Assignee: Apache Spark > Make a serializable jobTrackerId instead of a non-serializable JobID in > FileWriterFactory > - > > Key: SPARK-42478 > URL: https://issues.apache.org/jira/browse/SPARK-42478 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.3.2 >Reporter: Yi kaifei >Assignee: Apache Spark >Priority: Major > > https://issues.apache.org/jira/browse/SPARK-41448 make consistent MR job IDs > in FileBatchWriter and FileFormatWriter, but it breaks a serializable issue, > JobId is non-serializable -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42478) Make a serializable jobTrackerId instead of a non-serializable JobID in FileWriterFactory
[ https://issues.apache.org/jira/browse/SPARK-42478?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690371#comment-17690371 ] Apache Spark commented on SPARK-42478: -- User 'Yikf' has created a pull request for this issue: https://github.com/apache/spark/pull/40064 > Make a serializable jobTrackerId instead of a non-serializable JobID in > FileWriterFactory > - > > Key: SPARK-42478 > URL: https://issues.apache.org/jira/browse/SPARK-42478 > Project: Spark > Issue Type: Bug > Components: SQL >Affects Versions: 3.3.2 >Reporter: Yi kaifei >Priority: Major > > https://issues.apache.org/jira/browse/SPARK-41448 make consistent MR job IDs > in FileBatchWriter and FileFormatWriter, but it breaks a serializable issue, > JobId is non-serializable -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42477) accept user_agent in spark connect's connection string
[
https://issues.apache.org/jira/browse/SPARK-42477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690367#comment-17690367
]
Apache Spark commented on SPARK-42477:
--
User 'nija-at' has created a pull request for this issue:
https://github.com/apache/spark/pull/40054
> accept user_agent in spark connect's connection string
> ---
>
> Key: SPARK-42477
> URL: https://issues.apache.org/jira/browse/SPARK-42477
> Project: Spark
> Issue Type: New Feature
> Components: Connect
>Affects Versions: 3.3.2
>Reporter: Niranjan Jayakar
>Priority: Major
>
> Currently, the Spark Connect service's {{client_type}} attribute (which is
> really user agent) is set to {{_SPARK_CONNECT_PYTHON}} to signify PySpark.
> Accept an optional {{user_agent}} parameter in the connection string and
> plumb this down to the Spark Connect service.
> This enables partners using Spark Connect to set their application as the
> user agent,
> which then allows visibility and measurement of integrations and usages of
> spark
> connect.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42477) accept user_agent in spark connect's connection string
[
https://issues.apache.org/jira/browse/SPARK-42477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42477:
Assignee: (was: Apache Spark)
> accept user_agent in spark connect's connection string
> ---
>
> Key: SPARK-42477
> URL: https://issues.apache.org/jira/browse/SPARK-42477
> Project: Spark
> Issue Type: New Feature
> Components: Connect
>Affects Versions: 3.3.2
>Reporter: Niranjan Jayakar
>Priority: Major
>
> Currently, the Spark Connect service's {{client_type}} attribute (which is
> really user agent) is set to {{_SPARK_CONNECT_PYTHON}} to signify PySpark.
> Accept an optional {{user_agent}} parameter in the connection string and
> plumb this down to the Spark Connect service.
> This enables partners using Spark Connect to set their application as the
> user agent,
> which then allows visibility and measurement of integrations and usages of
> spark
> connect.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42477) accept user_agent in spark connect's connection string
[
https://issues.apache.org/jira/browse/SPARK-42477?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690366#comment-17690366
]
Apache Spark commented on SPARK-42477:
--
User 'nija-at' has created a pull request for this issue:
https://github.com/apache/spark/pull/40054
> accept user_agent in spark connect's connection string
> ---
>
> Key: SPARK-42477
> URL: https://issues.apache.org/jira/browse/SPARK-42477
> Project: Spark
> Issue Type: New Feature
> Components: Connect
>Affects Versions: 3.3.2
>Reporter: Niranjan Jayakar
>Priority: Major
>
> Currently, the Spark Connect service's {{client_type}} attribute (which is
> really user agent) is set to {{_SPARK_CONNECT_PYTHON}} to signify PySpark.
> Accept an optional {{user_agent}} parameter in the connection string and
> plumb this down to the Spark Connect service.
> This enables partners using Spark Connect to set their application as the
> user agent,
> which then allows visibility and measurement of integrations and usages of
> spark
> connect.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42477) accept user_agent in spark connect's connection string
[
https://issues.apache.org/jira/browse/SPARK-42477?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-42477:
Assignee: Apache Spark
> accept user_agent in spark connect's connection string
> ---
>
> Key: SPARK-42477
> URL: https://issues.apache.org/jira/browse/SPARK-42477
> Project: Spark
> Issue Type: New Feature
> Components: Connect
>Affects Versions: 3.3.2
>Reporter: Niranjan Jayakar
>Assignee: Apache Spark
>Priority: Major
>
> Currently, the Spark Connect service's {{client_type}} attribute (which is
> really user agent) is set to {{_SPARK_CONNECT_PYTHON}} to signify PySpark.
> Accept an optional {{user_agent}} parameter in the connection string and
> plumb this down to the Spark Connect service.
> This enables partners using Spark Connect to set their application as the
> user agent,
> which then allows visibility and measurement of integrations and usages of
> spark
> connect.
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42474) Add extraJVMOptions JVM GC option K8s test cases
[ https://issues.apache.org/jira/browse/SPARK-42474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42474: Assignee: Apache Spark > Add extraJVMOptions JVM GC option K8s test cases > > > Key: SPARK-42474 > URL: https://issues.apache.org/jira/browse/SPARK-42474 > Project: Spark > Issue Type: Test > Components: Kubernetes, Spark Core >Affects Versions: 3.4.0 >Reporter: Dongjoon Hyun >Assignee: Apache Spark >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42474) Add extraJVMOptions JVM GC option K8s test cases
[ https://issues.apache.org/jira/browse/SPARK-42474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690240#comment-17690240 ] Apache Spark commented on SPARK-42474: -- User 'dongjoon-hyun' has created a pull request for this issue: https://github.com/apache/spark/pull/40062 > Add extraJVMOptions JVM GC option K8s test cases > > > Key: SPARK-42474 > URL: https://issues.apache.org/jira/browse/SPARK-42474 > Project: Spark > Issue Type: Test > Components: Kubernetes, Spark Core >Affects Versions: 3.4.0 >Reporter: Dongjoon Hyun >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42474) Add extraJVMOptions JVM GC option K8s test cases
[ https://issues.apache.org/jira/browse/SPARK-42474?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690239#comment-17690239 ] Apache Spark commented on SPARK-42474: -- User 'dongjoon-hyun' has created a pull request for this issue: https://github.com/apache/spark/pull/40062 > Add extraJVMOptions JVM GC option K8s test cases > > > Key: SPARK-42474 > URL: https://issues.apache.org/jira/browse/SPARK-42474 > Project: Spark > Issue Type: Test > Components: Kubernetes, Spark Core >Affects Versions: 3.4.0 >Reporter: Dongjoon Hyun >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42474) Add extraJVMOptions JVM GC option K8s test cases
[ https://issues.apache.org/jira/browse/SPARK-42474?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42474: Assignee: (was: Apache Spark) > Add extraJVMOptions JVM GC option K8s test cases > > > Key: SPARK-42474 > URL: https://issues.apache.org/jira/browse/SPARK-42474 > Project: Spark > Issue Type: Test > Components: Kubernetes, Spark Core >Affects Versions: 3.4.0 >Reporter: Dongjoon Hyun >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42470) Remove unused declarations from Hive module
[ https://issues.apache.org/jira/browse/SPARK-42470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42470: Assignee: (was: Apache Spark) > Remove unused declarations from Hive module > --- > > Key: SPARK-42470 > URL: https://issues.apache.org/jira/browse/SPARK-42470 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.5.0 >Reporter: Yang Jie >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42470) Remove unused declarations from Hive module
[ https://issues.apache.org/jira/browse/SPARK-42470?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690094#comment-17690094 ] Apache Spark commented on SPARK-42470: -- User 'LuciferYang' has created a pull request for this issue: https://github.com/apache/spark/pull/40053 > Remove unused declarations from Hive module > --- > > Key: SPARK-42470 > URL: https://issues.apache.org/jira/browse/SPARK-42470 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.5.0 >Reporter: Yang Jie >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42470) Remove unused declarations from Hive module
[ https://issues.apache.org/jira/browse/SPARK-42470?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42470: Assignee: Apache Spark > Remove unused declarations from Hive module > --- > > Key: SPARK-42470 > URL: https://issues.apache.org/jira/browse/SPARK-42470 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.5.0 >Reporter: Yang Jie >Assignee: Apache Spark >Priority: Minor > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42002) Implement DataFrameWriterV2 (ReadwriterV2Tests)
[
https://issues.apache.org/jira/browse/SPARK-42002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690052#comment-17690052
]
Apache Spark commented on SPARK-42002:
--
User 'ueshin' has created a pull request for this issue:
https://github.com/apache/spark/pull/40060
> Implement DataFrameWriterV2 (ReadwriterV2Tests)
> ---
>
> Key: SPARK-42002
> URL: https://issues.apache.org/jira/browse/SPARK-42002
> Project: Spark
> Issue Type: Sub-task
> Components: Connect
>Affects Versions: 3.4.0
>Reporter: Hyukjin Kwon
>Assignee: Sandeep Singh
>Priority: Major
> Fix For: 3.4.0
>
>
> {code}
> pyspark/sql/tests/test_readwriter.py:182 (ReadwriterV2ParityTests.test_api)
> self =
> testMethod=test_api>
> def test_api(self):
> df = self.df
> > writer = df.writeTo("testcat.t")
> ../test_readwriter.py:185:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> self = DataFrame[key: bigint, value: string], args = ('testcat.t',), kwargs =
> {}
> def writeTo(self, *args: Any, **kwargs: Any) -> None:
> > raise NotImplementedError("writeTo() is not implemented.")
> E NotImplementedError: writeTo() is not implemented.
> ../../connect/dataframe.py:1529: NotImplementedError
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42002) Implement DataFrameWriterV2 (ReadwriterV2Tests)
[
https://issues.apache.org/jira/browse/SPARK-42002?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690051#comment-17690051
]
Apache Spark commented on SPARK-42002:
--
User 'ueshin' has created a pull request for this issue:
https://github.com/apache/spark/pull/40060
> Implement DataFrameWriterV2 (ReadwriterV2Tests)
> ---
>
> Key: SPARK-42002
> URL: https://issues.apache.org/jira/browse/SPARK-42002
> Project: Spark
> Issue Type: Sub-task
> Components: Connect
>Affects Versions: 3.4.0
>Reporter: Hyukjin Kwon
>Assignee: Sandeep Singh
>Priority: Major
> Fix For: 3.4.0
>
>
> {code}
> pyspark/sql/tests/test_readwriter.py:182 (ReadwriterV2ParityTests.test_api)
> self =
> testMethod=test_api>
> def test_api(self):
> df = self.df
> > writer = df.writeTo("testcat.t")
> ../test_readwriter.py:185:
> _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _ _
> _
> self = DataFrame[key: bigint, value: string], args = ('testcat.t',), kwargs =
> {}
> def writeTo(self, *args: Any, **kwargs: Any) -> None:
> > raise NotImplementedError("writeTo() is not implemented.")
> E NotImplementedError: writeTo() is not implemented.
> ../../connect/dataframe.py:1529: NotImplementedError
> {code}
--
This message was sent by Atlassian Jira
(v8.20.10#820010)
-
To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org
For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42469) Update MSSQL Dialect to use parentheses for TOP and add tests for Limit clause
[ https://issues.apache.org/jira/browse/SPARK-42469?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690040#comment-17690040 ] Apache Spark commented on SPARK-42469: -- User 'sadikovi' has created a pull request for this issue: https://github.com/apache/spark/pull/40059 > Update MSSQL Dialect to use parentheses for TOP and add tests for Limit clause > -- > > Key: SPARK-42469 > URL: https://issues.apache.org/jira/browse/SPARK-42469 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.4.0 >Reporter: Ivan Sadikov >Priority: Major > > Follow up for SPARK-42131. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42469) Update MSSQL Dialect to use parentheses for TOP and add tests for Limit clause
[ https://issues.apache.org/jira/browse/SPARK-42469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42469: Assignee: (was: Apache Spark) > Update MSSQL Dialect to use parentheses for TOP and add tests for Limit clause > -- > > Key: SPARK-42469 > URL: https://issues.apache.org/jira/browse/SPARK-42469 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.4.0 >Reporter: Ivan Sadikov >Priority: Major > > Follow up for SPARK-42131. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42469) Update MSSQL Dialect to use parentheses for TOP and add tests for Limit clause
[ https://issues.apache.org/jira/browse/SPARK-42469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42469: Assignee: Apache Spark > Update MSSQL Dialect to use parentheses for TOP and add tests for Limit clause > -- > > Key: SPARK-42469 > URL: https://issues.apache.org/jira/browse/SPARK-42469 > Project: Spark > Issue Type: Improvement > Components: SQL >Affects Versions: 3.4.0 >Reporter: Ivan Sadikov >Assignee: Apache Spark >Priority: Major > > Follow up for SPARK-42131. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-39859) Support v2 `DESCRIBE TABLE EXTENDED` for columns
[ https://issues.apache.org/jira/browse/SPARK-39859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39859: Assignee: Apache Spark > Support v2 `DESCRIBE TABLE EXTENDED` for columns > > > Key: SPARK-39859 > URL: https://issues.apache.org/jira/browse/SPARK-39859 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.4.0 >Reporter: Max Gekk >Assignee: Apache Spark >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-39859) Support v2 `DESCRIBE TABLE EXTENDED` for columns
[ https://issues.apache.org/jira/browse/SPARK-39859?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-39859: Assignee: (was: Apache Spark) > Support v2 `DESCRIBE TABLE EXTENDED` for columns > > > Key: SPARK-39859 > URL: https://issues.apache.org/jira/browse/SPARK-39859 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.4.0 >Reporter: Max Gekk >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-39859) Support v2 `DESCRIBE TABLE EXTENDED` for columns
[ https://issues.apache.org/jira/browse/SPARK-39859?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690023#comment-17690023 ] Apache Spark commented on SPARK-39859: -- User 'huaxingao' has created a pull request for this issue: https://github.com/apache/spark/pull/40058 > Support v2 `DESCRIBE TABLE EXTENDED` for columns > > > Key: SPARK-39859 > URL: https://issues.apache.org/jira/browse/SPARK-39859 > Project: Spark > Issue Type: Sub-task > Components: SQL >Affects Versions: 3.4.0 >Reporter: Max Gekk >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42468) Implement agg by (String, String)*
[ https://issues.apache.org/jira/browse/SPARK-42468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42468: Assignee: Rui Wang (was: Apache Spark) > Implement agg by (String, String)* > -- > > Key: SPARK-42468 > URL: https://issues.apache.org/jira/browse/SPARK-42468 > Project: Spark > Issue Type: Sub-task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42468) Implement agg by (String, String)*
[ https://issues.apache.org/jira/browse/SPARK-42468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42468: Assignee: Apache Spark (was: Rui Wang) > Implement agg by (String, String)* > -- > > Key: SPARK-42468 > URL: https://issues.apache.org/jira/browse/SPARK-42468 > Project: Spark > Issue Type: Sub-task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Rui Wang >Assignee: Apache Spark >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42468) Implement agg by (String, String)*
[ https://issues.apache.org/jira/browse/SPARK-42468?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17690017#comment-17690017 ] Apache Spark commented on SPARK-42468: -- User 'amaliujia' has created a pull request for this issue: https://github.com/apache/spark/pull/40057 > Implement agg by (String, String)* > -- > > Key: SPARK-42468 > URL: https://issues.apache.org/jira/browse/SPARK-42468 > Project: Spark > Issue Type: Sub-task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Rui Wang >Assignee: Rui Wang >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42465) ProtoToPlanTestSuite should analyze its input plans
[ https://issues.apache.org/jira/browse/SPARK-42465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42465: Assignee: Apache Spark > ProtoToPlanTestSuite should analyze its input plans > --- > > Key: SPARK-42465 > URL: https://issues.apache.org/jira/browse/SPARK-42465 > Project: Spark > Issue Type: Task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Herman van Hövell >Assignee: Apache Spark >Priority: Major > > ProtoToPlanTestSuite should analyze its input plans. We are moving to testing > functions, and most of these are left as unresolved function by the spark > planner. This is not enough to ensure that we are in fact binding to the > correct function. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42465) ProtoToPlanTestSuite should analyze its input plans
[ https://issues.apache.org/jira/browse/SPARK-42465?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42465: Assignee: (was: Apache Spark) > ProtoToPlanTestSuite should analyze its input plans > --- > > Key: SPARK-42465 > URL: https://issues.apache.org/jira/browse/SPARK-42465 > Project: Spark > Issue Type: Task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Herman van Hövell >Priority: Major > > ProtoToPlanTestSuite should analyze its input plans. We are moving to testing > functions, and most of these are left as unresolved function by the spark > planner. This is not enough to ensure that we are in fact binding to the > correct function. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42465) ProtoToPlanTestSuite should analyze its input plans
[ https://issues.apache.org/jira/browse/SPARK-42465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689861#comment-17689861 ] Apache Spark commented on SPARK-42465: -- User 'hvanhovell' has created a pull request for this issue: https://github.com/apache/spark/pull/40056 > ProtoToPlanTestSuite should analyze its input plans > --- > > Key: SPARK-42465 > URL: https://issues.apache.org/jira/browse/SPARK-42465 > Project: Spark > Issue Type: Task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Herman van Hövell >Priority: Major > > ProtoToPlanTestSuite should analyze its input plans. We are moving to testing > functions, and most of these are left as unresolved function by the spark > planner. This is not enough to ensure that we are in fact binding to the > correct function. -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42464) Fix 2.13 build errors caused by explain output changes and udfs.
[ https://issues.apache.org/jira/browse/SPARK-42464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689789#comment-17689789 ] Apache Spark commented on SPARK-42464: -- User 'hvanhovell' has created a pull request for this issue: https://github.com/apache/spark/pull/40055 > Fix 2.13 build errors caused by explain output changes and udfs. > > > Key: SPARK-42464 > URL: https://issues.apache.org/jira/browse/SPARK-42464 > Project: Spark > Issue Type: Task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Herman van Hövell >Assignee: Herman van Hövell >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42464) Fix 2.13 build errors caused by explain output changes and udfs.
[ https://issues.apache.org/jira/browse/SPARK-42464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42464: Assignee: Herman van Hövell (was: Apache Spark) > Fix 2.13 build errors caused by explain output changes and udfs. > > > Key: SPARK-42464 > URL: https://issues.apache.org/jira/browse/SPARK-42464 > Project: Spark > Issue Type: Task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Herman van Hövell >Assignee: Herman van Hövell >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42464) Fix 2.13 build errors caused by explain output changes and udfs.
[ https://issues.apache.org/jira/browse/SPARK-42464?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689788#comment-17689788 ] Apache Spark commented on SPARK-42464: -- User 'hvanhovell' has created a pull request for this issue: https://github.com/apache/spark/pull/40055 > Fix 2.13 build errors caused by explain output changes and udfs. > > > Key: SPARK-42464 > URL: https://issues.apache.org/jira/browse/SPARK-42464 > Project: Spark > Issue Type: Task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Herman van Hövell >Assignee: Herman van Hövell >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42464) Fix 2.13 build errors caused by explain output changes and udfs.
[ https://issues.apache.org/jira/browse/SPARK-42464?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42464: Assignee: Apache Spark (was: Herman van Hövell) > Fix 2.13 build errors caused by explain output changes and udfs. > > > Key: SPARK-42464 > URL: https://issues.apache.org/jira/browse/SPARK-42464 > Project: Spark > Issue Type: Task > Components: Connect >Affects Versions: 3.4.0 >Reporter: Herman van Hövell >Assignee: Apache Spark >Priority: Major > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42463) Clean up the third-party Java source code introduced by SPARK-27180
[ https://issues.apache.org/jira/browse/SPARK-42463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689571#comment-17689571 ] Apache Spark commented on SPARK-42463: -- User 'LuciferYang' has created a pull request for this issue: https://github.com/apache/spark/pull/40052 > Clean up the third-party Java source code introduced by SPARK-27180 > --- > > Key: SPARK-42463 > URL: https://issues.apache.org/jira/browse/SPARK-42463 > Project: Spark > Issue Type: Improvement > Components: Tests, YARN >Affects Versions: 3.5.0 >Reporter: Yang Jie >Priority: Minor > > * > resource-managers/yarn/src/test/java/org/apache/hadoop/net/ServerSocketUtil.java > * > resource-managers/yarn/src/test/java/org/eclipse/jetty/server/SessionManager.java > * > resource-managers/yarn/src/test/java/org/eclipse/jetty/server/session/SessionHandler.java -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42463) Clean up the third-party Java source code introduced by SPARK-27180
[ https://issues.apache.org/jira/browse/SPARK-42463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42463: Assignee: Apache Spark > Clean up the third-party Java source code introduced by SPARK-27180 > --- > > Key: SPARK-42463 > URL: https://issues.apache.org/jira/browse/SPARK-42463 > Project: Spark > Issue Type: Improvement > Components: Tests, YARN >Affects Versions: 3.5.0 >Reporter: Yang Jie >Assignee: Apache Spark >Priority: Minor > > * > resource-managers/yarn/src/test/java/org/apache/hadoop/net/ServerSocketUtil.java > * > resource-managers/yarn/src/test/java/org/eclipse/jetty/server/SessionManager.java > * > resource-managers/yarn/src/test/java/org/eclipse/jetty/server/session/SessionHandler.java -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Assigned] (SPARK-42463) Clean up the third-party Java source code introduced by SPARK-27180
[ https://issues.apache.org/jira/browse/SPARK-42463?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-42463: Assignee: (was: Apache Spark) > Clean up the third-party Java source code introduced by SPARK-27180 > --- > > Key: SPARK-42463 > URL: https://issues.apache.org/jira/browse/SPARK-42463 > Project: Spark > Issue Type: Improvement > Components: Tests, YARN >Affects Versions: 3.5.0 >Reporter: Yang Jie >Priority: Minor > > * > resource-managers/yarn/src/test/java/org/apache/hadoop/net/ServerSocketUtil.java > * > resource-managers/yarn/src/test/java/org/eclipse/jetty/server/SessionManager.java > * > resource-managers/yarn/src/test/java/org/eclipse/jetty/server/session/SessionHandler.java -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-27180) Fix testing issues with yarn module in Hadoop-3
[ https://issues.apache.org/jira/browse/SPARK-27180?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689569#comment-17689569 ] Apache Spark commented on SPARK-27180: -- User 'LuciferYang' has created a pull request for this issue: https://github.com/apache/spark/pull/40052 > Fix testing issues with yarn module in Hadoop-3 > --- > > Key: SPARK-27180 > URL: https://issues.apache.org/jira/browse/SPARK-27180 > Project: Spark > Issue Type: Sub-task > Components: Build, Spark Core, YARN >Affects Versions: 3.0.0 >Reporter: Yuming Wang >Assignee: Yuming Wang >Priority: Major > Fix For: 3.0.0 > > -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
[jira] [Commented] (SPARK-42463) Clean up the third-party Java source code introduced by SPARK-27180
[ https://issues.apache.org/jira/browse/SPARK-42463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17689568#comment-17689568 ] Apache Spark commented on SPARK-42463: -- User 'LuciferYang' has created a pull request for this issue: https://github.com/apache/spark/pull/40052 > Clean up the third-party Java source code introduced by SPARK-27180 > --- > > Key: SPARK-42463 > URL: https://issues.apache.org/jira/browse/SPARK-42463 > Project: Spark > Issue Type: Improvement > Components: Tests, YARN >Affects Versions: 3.5.0 >Reporter: Yang Jie >Priority: Minor > > * > resource-managers/yarn/src/test/java/org/apache/hadoop/net/ServerSocketUtil.java > * > resource-managers/yarn/src/test/java/org/eclipse/jetty/server/SessionManager.java > * > resource-managers/yarn/src/test/java/org/eclipse/jetty/server/session/SessionHandler.java -- This message was sent by Atlassian Jira (v8.20.10#820010) - To unsubscribe, e-mail: issues-unsubscr...@spark.apache.org For additional commands, e-mail: issues-h...@spark.apache.org
