Shane Knapp created SPARK-37571:
---
Summary: decouple jenkins from spark builds and tests
Key: SPARK-37571
URL: https://issues.apache.org/jira/browse/SPARK-37571
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-37445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun reassigned SPARK-37445:
Assignee: angerszhu
> Update hadoop-profile
> -
>
> Key:
[
https://issues.apache.org/jira/browse/SPARK-37571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shane Knapp updated SPARK-37571:
Attachment: audit.txt
> decouple amplab jenkins from spark website, builds and tests
>
angerszhu created SPARK-37573:
-
Summary: IsolatedClient fallbackVersion should be build in
version, not always 2.7.4
Key: SPARK-37573
URL: https://issues.apache.org/jira/browse/SPARK-37573
Project:
Shardul Mahadik created SPARK-37569:
---
Summary: View Analysis incorrectly marks nested fields as nullable
Key: SPARK-37569
URL: https://issues.apache.org/jira/browse/SPARK-37569
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-37571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454925#comment-17454925
]
Shane Knapp commented on SPARK-37571:
-
this is gonna take a while... nearly a decade later,
[
https://issues.apache.org/jira/browse/SPARK-37572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dagang Wei updated SPARK-37572:
---
Description:
Currently Spark launches executor processes by constructing and running a
command
Dagang Wei created SPARK-37572:
--
Summary: Flexible ways of launching executors
Key: SPARK-37572
URL: https://issues.apache.org/jira/browse/SPARK-37572
Project: Spark
Issue Type: New Feature
[
https://issues.apache.org/jira/browse/SPARK-37445?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Chao Sun resolved SPARK-37445.
--
Fix Version/s: 3.3.0
Resolution: Fixed
Issue resolved by pull request 34715
[
https://issues.apache.org/jira/browse/SPARK-37573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-37573:
Assignee: Apache Spark
> IsolatedClient fallbackVersion should be build in version, not
[
https://issues.apache.org/jira/browse/SPARK-37573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454945#comment-17454945
]
Apache Spark commented on SPARK-37573:
--
User 'AngersZh' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-37573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-37573:
Assignee: (was: Apache Spark)
> IsolatedClient fallbackVersion should be build in
Rafal Wojdyla created SPARK-37570:
-
Summary: mypy breaks on pyspark.pandas.plot.core.Bucketizer
Key: SPARK-37570
URL: https://issues.apache.org/jira/browse/SPARK-37570
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-37571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shane Knapp updated SPARK-37571:
Attachment: spark-repo-to-be-audited.txt
> decouple amplab jenkins from spark website, builds and
[
https://issues.apache.org/jira/browse/SPARK-37571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shane Knapp updated SPARK-37571:
Attachment: spark-repo-to-be-audited.txt
> decouple amplab jenkins from spark website, builds and
[
https://issues.apache.org/jira/browse/SPARK-37572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dagang Wei updated SPARK-37572:
---
Description:
Currently Spark launches executor processes by constructing and running
commands [1],
[
https://issues.apache.org/jira/browse/SPARK-37571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shane Knapp updated SPARK-37571:
Attachment: (was: spark-repo-to-be-audited.txt)
> decouple amplab jenkins from spark website,
[
https://issues.apache.org/jira/browse/SPARK-37572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dagang Wei updated SPARK-37572:
---
Description:
Currently Spark launches executor processes by constructing and running
commands [1],
[
https://issues.apache.org/jira/browse/SPARK-37572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dagang Wei updated SPARK-37572:
---
Description:
Currently Spark launches executor processes by constructing and running a
command
[
https://issues.apache.org/jira/browse/SPARK-37572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dagang Wei updated SPARK-37572:
---
Description:
Currently Spark launches executor processes by constructing and running a
command
[
https://issues.apache.org/jira/browse/SPARK-37572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dagang Wei updated SPARK-37572:
---
Description:
Currently Spark launches executor processes by constructing and running
commands [1],
[
https://issues.apache.org/jira/browse/SPARK-37571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shane Knapp updated SPARK-37571:
Description:
we will be turning off jenkins on dec 23rd, and we need to decouple the build
infra
[
https://issues.apache.org/jira/browse/SPARK-37571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shane Knapp updated SPARK-37571:
Summary: decouple amplab jenkins from spark website, builds and tests
(was: decouple jenkins
[
https://issues.apache.org/jira/browse/SPARK-37572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dagang Wei updated SPARK-37572:
---
Description:
Currently Spark launches executor processes by constructing and running
commands [1],
[
https://issues.apache.org/jira/browse/SPARK-37572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dagang Wei updated SPARK-37572:
---
Description:
Currently Spark launches executor processes by constructing and running
commands [1],
[
https://issues.apache.org/jira/browse/SPARK-37568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454947#comment-17454947
]
Kousuke Saruta commented on SPARK-37568:
cc: [~yoda-mon] [~YActs] Do you want to work on this?
[
https://issues.apache.org/jira/browse/SPARK-37573?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454946#comment-17454946
]
Apache Spark commented on SPARK-37573:
--
User 'AngersZh' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-37570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Rafal Wojdyla updated SPARK-37570:
--
Description:
Mypy breaks on a project with pyspark 3.2.0 dependency (worked fine for 3.1.2),
[
https://issues.apache.org/jira/browse/SPARK-37392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-37392.
-
Fix Version/s: 3.3.0
3.2.1
3.1.3
Resolution: Fixed
[
https://issues.apache.org/jira/browse/SPARK-37392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-37392:
---
Assignee: Wenchen Fan
> Catalyst optimizer very time-consuming and memory-intensive with
[
https://issues.apache.org/jira/browse/SPARK-37568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454960#comment-17454960
]
Kousuke Saruta commented on SPARK-37568:
[~yoda-mon] OK, please go ahead.
> Support 2-arguments
Dongjoon Hyun created SPARK-37576:
-
Summary: Support built-in K8s executor roll plugin
Key: SPARK-37576
URL: https://issues.apache.org/jira/browse/SPARK-37576
Project: Spark
Issue Type: New
[
https://issues.apache.org/jira/browse/SPARK-37572?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-37572:
-
Priority: Major (was: Critical)
> Flexible ways of launching executors
>
[
https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454974#comment-17454974
]
Guo Wei commented on SPARK-37575:
-
As default writerSettings in CSVOptions, nullValue is "",
[
https://issues.apache.org/jira/browse/SPARK-37516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-37516.
--
Fix Version/s: 3.3.0
Resolution: Fixed
Fixed in
[
https://issues.apache.org/jira/browse/SPARK-37516?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-37516:
Assignee: Hyukjin Kwon
> Uses Python's standard string formatter for SQL API in PySpark
[
https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454968#comment-17454968
]
Hyukjin Kwon commented on SPARK-37575:
--
can you set nullValue and emptyValue options?
> Empty
[
https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454976#comment-17454976
]
Hyukjin Kwon commented on SPARK-37575:
--
Spark 2.4.X is EOL so it won't likely be fixed. Does it
[
https://issues.apache.org/jira/browse/SPARK-37568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454958#comment-17454958
]
Leona Yoda commented on SPARK-37568:
I would like to work on this.
> Support 2-arguments by the
[
https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454989#comment-17454989
]
Guo Wei commented on SPARK-37575:
-
Spark 3.2.0 has the same behavior.
> Empty strings and null values
Cheng Pan created SPARK-37574:
-
Summary: Simplify fetchBlocks w/o retry
Key: SPARK-37574
URL: https://issues.apache.org/jira/browse/SPARK-37574
Project: Spark
Issue Type: Improvement
[
https://issues.apache.org/jira/browse/SPARK-37574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454950#comment-17454950
]
Apache Spark commented on SPARK-37574:
--
User 'pan3793' has created a pull request for this issue:
Guo Wei created SPARK-37575:
---
Summary: Empty strings and null values are both saved as quoted
empty Strings "" rather than "" (for empty strings) and nothing(for null values)
Key: SPARK-37575
URL:
[
https://issues.apache.org/jira/browse/SPARK-37571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Gengliang Wang updated SPARK-37571:
---
Affects Version/s: 3.3.0
(was: 3.2.0)
> decouple amplab jenkins
[
https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454966#comment-17454966
]
Guo Wei commented on SPARK-37575:
-
related issues:
https://issues.apache.org/jira/browse/SPARK-17916
[
https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-37575:
-
Affects Version/s: 3.2.0
> Empty strings and null values are both saved as quoted empty Strings
[
https://issues.apache.org/jira/browse/SPARK-37576?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454969#comment-17454969
]
Apache Spark commented on SPARK-37576:
--
User 'dongjoon-hyun' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-37576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-37576:
Assignee: (was: Apache Spark)
> Support built-in K8s executor roll plugin
>
[
https://issues.apache.org/jira/browse/SPARK-37570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454970#comment-17454970
]
Hyukjin Kwon commented on SPARK-37570:
--
cc [~itholic] [~XinrongM] [~zero323] FYI
> mypy breaks on
[
https://issues.apache.org/jira/browse/SPARK-37576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-37576:
Assignee: Apache Spark
> Support built-in K8s executor roll plugin
>
[
https://issues.apache.org/jira/browse/SPARK-37551?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454975#comment-17454975
]
Hyukjin Kwon commented on SPARK-37551:
--
cc [~XinrongM] too FYI
> Argument 1 to "rename" of
[
https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454974#comment-17454974
]
Guo Wei edited comment on SPARK-37575 at 12/8/21, 6:02 AM:
---
As default
[
https://issues.apache.org/jira/browse/SPARK-37575?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454974#comment-17454974
]
Guo Wei edited comment on SPARK-37575 at 12/8/21, 6:02 AM:
---
As default
[
https://issues.apache.org/jira/browse/SPARK-37574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-37574:
Assignee: Apache Spark
> Simplify fetchBlocks w/o retry
> --
[
https://issues.apache.org/jira/browse/SPARK-37574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-37574:
Assignee: (was: Apache Spark)
> Simplify fetchBlocks w/o retry
>
[
https://issues.apache.org/jira/browse/SPARK-37574?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454949#comment-17454949
]
Apache Spark commented on SPARK-37574:
--
User 'pan3793' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-37515?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454691#comment-17454691
]
Sungpeo Kook commented on SPARK-37515:
--
[~apachespark] Nobody check this issue?
>
[
https://issues.apache.org/jira/browse/SPARK-23607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454799#comment-17454799
]
Apache Spark commented on SPARK-23607:
--
User 'thejdeep' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-23607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-23607:
Assignee: Apache Spark
> Use HDFS extended attributes to store application summary to
[
https://issues.apache.org/jira/browse/SPARK-23607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-23607:
Assignee: (was: Apache Spark)
> Use HDFS extended attributes to store application
[
https://issues.apache.org/jira/browse/SPARK-37556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean R. Owen resolved SPARK-37556.
--
Fix Version/s: 3.3.0
3.0.4
3.2.1
[
https://issues.apache.org/jira/browse/SPARK-37556?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Sean R. Owen reassigned SPARK-37556:
Assignee: Daniel Dai
> Deser void class fail with Java serialization
>
[
https://issues.apache.org/jira/browse/SPARK-23607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454797#comment-17454797
]
Thejdeep Gudivada commented on SPARK-23607:
---
Posted a preview PR for this, will be adding
[
https://issues.apache.org/jira/browse/SPARK-23607?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thejdeep Gudivada reopened SPARK-23607:
---
> Use HDFS extended attributes to store application summary to improve the
> Spark
[
https://issues.apache.org/jira/browse/SPARK-23607?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454800#comment-17454800
]
Apache Spark commented on SPARK-23607:
--
User 'thejdeep' has created a pull request for this issue:
Fu Chen created SPARK-37566:
---
Summary: The sampling job will lead to the wrong statistics
Key: SPARK-37566
URL: https://issues.apache.org/jira/browse/SPARK-37566
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-37567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
junbiao chen updated SPARK-37567:
-
Attachment: execution stage(1)-query2.png
> reuse Exchange failed
> --
>
>
[
https://issues.apache.org/jira/browse/SPARK-37567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
junbiao chen updated SPARK-37567:
-
Attachment: physical plan-query2.png
> reuse Exchange failed
> --
>
>
junbiao chen created SPARK-37567:
Summary: reuse Exchange failed
Key: SPARK-37567
URL: https://issues.apache.org/jira/browse/SPARK-37567
Project: Spark
Issue Type: Bug
Components:
[
https://issues.apache.org/jira/browse/SPARK-37567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
junbiao chen updated SPARK-37567:
-
Attachment: execution stage-query2.png
> reuse Exchange failed
> --
>
>
[
https://issues.apache.org/jira/browse/SPARK-37567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
junbiao chen updated SPARK-37567:
-
Description:
use case:query2 in TPC-DS.There are three exchange subquery will scan the same
[
https://issues.apache.org/jira/browse/SPARK-37566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fu Chen updated SPARK-37566:
Attachment: 截屏2021-12-07 下午5.17.12.png
> The sampling job will lead to the wrong statistics
>
[
https://issues.apache.org/jira/browse/SPARK-37566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Fu Chen updated SPARK-37566:
Description:
code for reproduce
{code:java}
spark.range(0, 10)
.repartitionByRange(10,
[
https://issues.apache.org/jira/browse/SPARK-37566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454497#comment-17454497
]
Fu Chen commented on SPARK-37566:
-
The expected value of `number of output rows` is 10
> The sampling
[
https://issues.apache.org/jira/browse/SPARK-37567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
junbiao chen updated SPARK-37567:
-
Description:
use case:query2 in TPC-DS.There are three exchange subquery will scan the same
[
https://issues.apache.org/jira/browse/SPARK-37567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454524#comment-17454524
]
junbiao chen commented on SPARK-37567:
--
Hi,[~davies], Is this a reuse bug?
> reuse Exchange failed
[
https://issues.apache.org/jira/browse/SPARK-37566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-37566:
Assignee: Apache Spark
> The sampling job will lead to the wrong statistics
>
[
https://issues.apache.org/jira/browse/SPARK-37566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454534#comment-17454534
]
Apache Spark commented on SPARK-37566:
--
User 'cfmcgrady' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-37566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-37566:
Assignee: Apache Spark
> The sampling job will lead to the wrong statistics
>
[
https://issues.apache.org/jira/browse/SPARK-37566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-37566:
Assignee: (was: Apache Spark)
> The sampling job will lead to the wrong statistics
>
[
https://issues.apache.org/jira/browse/SPARK-32225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Stijn De Haes updated SPARK-32225:
--
Attachment: image-2021-12-07-13-37-12-197.png
> Parquet footer information is read twice
>
[
https://issues.apache.org/jira/browse/SPARK-32225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454622#comment-17454622
]
Stijn De Haes commented on SPARK-32225:
---
Could this be the reason that when you read a Parquet
[
https://issues.apache.org/jira/browse/SPARK-37478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-37478:
---
Assignee: dch nguyen
> Unify v1 and v2 DROP NAMESPACE tests
>
Max Gekk created SPARK-37568:
Summary: Support 2-arguments by the convert_timezone() function
Key: SPARK-37568
URL: https://issues.apache.org/jira/browse/SPARK-37568
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-37478?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-37478.
-
Fix Version/s: 3.3.0
Resolution: Fixed
Issue resolved by pull request 34819
[
https://issues.apache.org/jira/browse/SPARK-37568?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17454634#comment-17454634
]
Max Gekk commented on SPARK-37568:
--
[~beliefer] [~sarutak] [~angerszhuuu] [~xiaopenglei] Would you like
86 matches
Mail list logo