[
https://issues.apache.org/jira/browse/SPARK-38073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38073:
Assignee: Apache Spark
> NameError: name 'sc' is not defined when running driver with IPy
[
https://issues.apache.org/jira/browse/SPARK-38073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38073:
Assignee: (was: Apache Spark)
> NameError: name 'sc' is not defined when running driv
[
https://issues.apache.org/jira/browse/SPARK-38073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487055#comment-17487055
]
Apache Spark commented on SPARK-38073:
--
User 'zero323' has created a pull request f
Keunhyun Oh created SPARK-38102:
---
Summary: Supporting custom commitProtocolClass and committer class
when using saveAsNewAPIHadoopDataset
Key: SPARK-38102
URL: https://issues.apache.org/jira/browse/SPARK-38102
[
https://issues.apache.org/jira/browse/SPARK-38102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Keunhyun Oh updated SPARK-38102:
Summary: Supporting custom commitProtocolClass when using
saveAsNewAPIHadoopDataset (was: Support
[
https://issues.apache.org/jira/browse/SPARK-38102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Keunhyun Oh updated SPARK-38102:
Description:
It is needed to support custom commitProtocolClass class when using
saveAsNewAPIHado
[
https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487134#comment-17487134
]
Rob D commented on SPARK-6305:
--
Thank you for this update. Has this change been included in
Max Gekk created SPARK-38103:
Summary: Use error classes in the parsing errors of transform
Key: SPARK-38103
URL: https://issues.apache.org/jira/browse/SPARK-38103
Project: Spark
Issue Type: Sub-
Max Gekk created SPARK-38104:
Summary: Use error classes in the parsing errors of windows
Key: SPARK-38104
URL: https://issues.apache.org/jira/browse/SPARK-38104
Project: Spark
Issue Type: Sub-ta
Max Gekk created SPARK-38105:
Summary: Use error classes in the parsing errors of joins
Key: SPARK-38105
URL: https://issues.apache.org/jira/browse/SPARK-38105
Project: Spark
Issue Type: Sub-task
Max Gekk created SPARK-38106:
Summary: Use error classes in the parsing errors of functions
Key: SPARK-38106
URL: https://issues.apache.org/jira/browse/SPARK-38106
Project: Spark
Issue Type: Sub-
[
https://issues.apache.org/jira/browse/SPARK-38102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Keunhyun Oh updated SPARK-38102:
Description:
There is no way to apply spark-hadoop-cloud's commitProtocolClass when using
saveAsN
[
https://issues.apache.org/jira/browse/SPARK-38102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487153#comment-17487153
]
Apache Spark commented on SPARK-38102:
--
User 'ocworld' has created a pull request f
[
https://issues.apache.org/jira/browse/SPARK-38102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38102:
Assignee: (was: Apache Spark)
> Supporting custom commitProtocolClass when using save
[
https://issues.apache.org/jira/browse/SPARK-38102?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38102:
Assignee: Apache Spark
> Supporting custom commitProtocolClass when using saveAsNewAPIHad
[
https://issues.apache.org/jira/browse/SPARK-38102?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487154#comment-17487154
]
Apache Spark commented on SPARK-38102:
--
User 'ocworld' has created a pull request f
Max Gekk created SPARK-38107:
Summary: Use error classes in the compilation errors of pandas UDFs
Key: SPARK-38107
URL: https://issues.apache.org/jira/browse/SPARK-38107
Project: Spark
Issue Type
Max Gekk created SPARK-38108:
Summary: Use error classes in the compilation errors of UDF/UDAF
Key: SPARK-38108
URL: https://issues.apache.org/jira/browse/SPARK-38108
Project: Spark
Issue Type: S
[
https://issues.apache.org/jira/browse/SPARK-38107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-38107:
-
Description:
Migrate the following errors in QueryCompilationErrors:
* pandasUDFAggregateNotSupportedInP
[
https://issues.apache.org/jira/browse/SPARK-38107?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-38107:
-
Summary: Use error classes in the compilation errors of python/pandas UDFs
(was: Use error classes in t
[
https://issues.apache.org/jira/browse/SPARK-38107?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487170#comment-17487170
]
Max Gekk commented on SPARK-38107:
--
[~hyukjin.kwon] Do you know someone who could be in
ss created SPARK-38109:
--
Summary: pyspark DataFrame.replace() is sensitive to column name
case in pyspark 3.2 but not in 3.1
Key: SPARK-38109
URL: https://issues.apache.org/jira/browse/SPARK-38109
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-38109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ss updated SPARK-38109:
---
Description:
The `subset` argument for `DataFrame.replace()` accepts one or more column
names. In pyspark 3.2 the c
[
https://issues.apache.org/jira/browse/SPARK-38109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ss updated SPARK-38109:
---
Description:
The `subset` argument for `DataFrame.replace()` accepts one or more column
names. In pyspark 3.2 the c
[
https://issues.apache.org/jira/browse/SPARK-38109?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
ss updated SPARK-38109:
---
Description:
The `subset` argument for `DataFrame.replace()` accepts one or more column
names. In pyspark 3.2 the c
Max Gekk created SPARK-38110:
Summary: Use error classes in the compilation errors of windows
Key: SPARK-38110
URL: https://issues.apache.org/jira/browse/SPARK-38110
Project: Spark
Issue Type: Su
[
https://issues.apache.org/jira/browse/SPARK-38110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Max Gekk updated SPARK-38110:
-
Description:
Migrate the following errors in QueryCompilationErrors:
* windowSpecificationNotDefinedErro
[
https://issues.apache.org/jira/browse/SPARK-38082?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487180#comment-17487180
]
Apache Spark commented on SPARK-38082:
--
User 'zero323' has created a pull request f
[
https://issues.apache.org/jira/browse/SPARK-38082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38082:
Assignee: Apache Spark
> Update minimum numpy version
>
>
>
[
https://issues.apache.org/jira/browse/SPARK-38082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-38082:
Assignee: (was: Apache Spark)
> Update minimum numpy version
> --
Fabien created SPARK-38111:
--
Summary: Retrieve a Spark dataframe as Arrow batches
Key: SPARK-38111
URL: https://issues.apache.org/jira/browse/SPARK-38111
Project: Spark
Issue Type: Question
[
https://issues.apache.org/jira/browse/SPARK-38111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fabien updated SPARK-38111:
---
Description:
Using the Java API, is there a way to efficiently retrieve a dataframe as Arrow
batches ?
I h
[
https://issues.apache.org/jira/browse/SPARK-38111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fabien updated SPARK-38111:
---
Labels: arrow (was: )
> Retrieve a Spark dataframe as Arrow batches
> -
[
https://issues.apache.org/jira/browse/SPARK-36665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487200#comment-17487200
]
Anton Okolnychyi commented on SPARK-36665:
--
[~kazuyukitanimura] {{RewritePredic
[
https://issues.apache.org/jira/browse/SPARK-36665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487200#comment-17487200
]
Anton Okolnychyi edited comment on SPARK-36665 at 2/4/22, 5:47 PM:
---
[
https://issues.apache.org/jira/browse/SPARK-36665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487222#comment-17487222
]
Kazuyuki Tanimura commented on SPARK-36665:
---
Understood, thank you [~aokolnych
Max Gekk created SPARK-38112:
Summary: Use error classes in the execution errors of
date/timestamp handling
Key: SPARK-38112
URL: https://issues.apache.org/jira/browse/SPARK-38112
Project: Spark
Max Gekk created SPARK-38113:
Summary: Use error classes in the execution errors of pivoting
Key: SPARK-38113
URL: https://issues.apache.org/jira/browse/SPARK-38113
Project: Spark
Issue Type: Sub
[
https://issues.apache.org/jira/browse/SPARK-36665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487235#comment-17487235
]
Anton Okolnychyi commented on SPARK-36665:
--
Thanks for the prompt reply, [~kazu
[
https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487244#comment-17487244
]
Dongjoon Hyun commented on SPARK-6305:
--
This is a part of Apache Spark 3.3 and we wi
[
https://issues.apache.org/jira/browse/SPARK-38101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487246#comment-17487246
]
Dongjoon Hyun commented on SPARK-38101:
---
Thank you for filing a JIRA, [~eejbyfeldt
[
https://issues.apache.org/jira/browse/SPARK-38101?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-38101:
--
Affects Version/s: 3.3.0
(was: 3.3)
> MetadataFetchFailedException
[
https://issues.apache.org/jira/browse/SPARK-38101?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487258#comment-17487258
]
L. C. Hsieh commented on SPARK-38101:
-
Thanks for reporting this issue, [~eejbyfeldt
SOUVIK PAUL created SPARK-38114:
---
Summary: Spark build fails in Windows
Key: SPARK-38114
URL: https://issues.apache.org/jira/browse/SPARK-38114
Project: Spark
Issue Type: Bug
Componen
[
https://issues.apache.org/jira/browse/SPARK-37630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487271#comment-17487271
]
James Inlow commented on SPARK-37630:
-
How can we know about if Spark is impacted by
[
https://issues.apache.org/jira/browse/SPARK-37630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487275#comment-17487275
]
PJ Fanning commented on SPARK-37630:
[~jinlow] there is little point commenting on t
[
https://issues.apache.org/jira/browse/SPARK-38098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luca Canali updated SPARK-38098:
Summary: Add support for ArrayType of nested StructType to arrow-based
conversion (was: Support A
[
https://issues.apache.org/jira/browse/SPARK-38098?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Luca Canali updated SPARK-38098:
Description:
This proposes to add support for ArrayType of nested StructType to arrow-based
conve
[
https://issues.apache.org/jira/browse/SPARK-37630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487279#comment-17487279
]
James Inlow commented on SPARK-37630:
-
[~pj.fanning] Thanks, I have seen that Spark
[
https://issues.apache.org/jira/browse/SPARK-37814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487285#comment-17487285
]
Stephen L. De Rudder commented on SPARK-37814:
--
With log4j 1.x line having
[
https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487286#comment-17487286
]
James Inlow commented on SPARK-6305:
As we wait for spark to be released with log4j v
[
https://issues.apache.org/jira/browse/SPARK-37814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487287#comment-17487287
]
John Crowe commented on SPARK-37814:
Did you use some sort of Title in your message
[
https://issues.apache.org/jira/browse/SPARK-38114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SOUVIK PAUL updated SPARK-38114:
Description:
java.lang.NoSuchMethodError:
org.fusesource.jansi.AnsiConsole.wrapOutputStream(Ljava
[
https://issues.apache.org/jira/browse/SPARK-38114?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
SOUVIK PAUL updated SPARK-38114:
Description:
java.lang.NoSuchMethodError:
org.fusesource.jansi.AnsiConsole.wrapOutputStream(Ljava
Karthik created SPARK-38115:
---
Summary: No spark conf to control the path of _temporary when
writing to target filesystem
Key: SPARK-38115
URL: https://issues.apache.org/jira/browse/SPARK-38115
Project: Spar
[
https://issues.apache.org/jira/browse/SPARK-38115?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
kk updated SPARK-38115:
---
Description: No default spark conf or param to control the '_temporary'
path when writing to filesystem. (was: Ther
[
https://issues.apache.org/jira/browse/SPARK-36986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rodrigo Boavida updated SPARK-36986:
Priority: Major (was: Minor)
> Improving external schema management flexibility
> ---
[
https://issues.apache.org/jira/browse/SPARK-36986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rodrigo Boavida updated SPARK-36986:
Docs Text:
Schema management improvements
1 - Retrieving a field name and type from a sch
[
https://issues.apache.org/jira/browse/SPARK-36986?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rodrigo Boavida updated SPARK-36986:
Description:
Our spark usage, requires us to build an external schema and pass it on while
[
https://issues.apache.org/jira/browse/SPARK-37416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-37416:
Assignee: (was: Apache Spark)
> Inline type hints for python/pyspark/ml/wrapper.py
>
[
https://issues.apache.org/jira/browse/SPARK-37416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487358#comment-17487358
]
Apache Spark commented on SPARK-37416:
--
User 'zero323' has created a pull request f
[
https://issues.apache.org/jira/browse/SPARK-37416?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-37416:
Assignee: Apache Spark
> Inline type hints for python/pyspark/ml/wrapper.py
> ---
[
https://issues.apache.org/jira/browse/SPARK-37416?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487359#comment-17487359
]
Apache Spark commented on SPARK-37416:
--
User 'zero323' has created a pull request f
[
https://issues.apache.org/jira/browse/SPARK-6305?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487376#comment-17487376
]
Sean R. Owen commented on SPARK-6305:
-
Spark doesn't use JDBCAppender, and doesn't us
[
https://issues.apache.org/jira/browse/SPARK-37408?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Maciej Szymkiewicz reassigned SPARK-37408:
--
Assignee: Maciej Szymkiewicz (was: Apache Spark)
> Inline type hints for pyt
[
https://issues.apache.org/jira/browse/SPARK-36837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-36837.
---
Fix Version/s: 3.3.0
Resolution: Fixed
This is resolved via https://github.com/apache
[
https://issues.apache.org/jira/browse/SPARK-38082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-38082.
---
Fix Version/s: 3.3.0
Resolution: Fixed
Issue resolved by pull request 35398
[https://
[
https://issues.apache.org/jira/browse/SPARK-38082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-38082:
-
Assignee: Maciej Szymkiewicz
> Update minimum numpy version
> -
[
https://issues.apache.org/jira/browse/SPARK-38082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-38082:
--
Summary: Update minimum numpy version to 1.15 (was: Update minimum numpy
version)
> Update m
[
https://issues.apache.org/jira/browse/SPARK-38073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun updated SPARK-38073:
--
Issue Type: Bug (was: Improvement)
> NameError: name 'sc' is not defined when running driver
[
https://issues.apache.org/jira/browse/SPARK-38073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun reassigned SPARK-38073:
-
Assignee: Maciej Szymkiewicz
> NameError: name 'sc' is not defined when running driver
[
https://issues.apache.org/jira/browse/SPARK-36665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487413#comment-17487413
]
Apache Spark commented on SPARK-36665:
--
User 'kazuyukitanimura' has created a pull
[
https://issues.apache.org/jira/browse/SPARK-36665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17487415#comment-17487415
]
Apache Spark commented on SPARK-36665:
--
User 'kazuyukitanimura' has created a pull
[
https://issues.apache.org/jira/browse/SPARK-38073?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dongjoon Hyun resolved SPARK-38073.
---
Fix Version/s: 3.3.0
3.2.2
Resolution: Fixed
Issue resolved by pul
74 matches
Mail list logo