[
https://issues.apache.org/jira/browse/SPARK-24588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li updated SPARK-24588:
Description:
In https://github.com/apache/spark/pull/19080, we simplified the
distribution/partitioning
[
https://issues.apache.org/jira/browse/SPARK-24588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li updated SPARK-24588:
Description:
In https://github.com/apache/spark/pull/19080, we simplified the
distribution/partitioning
[
https://issues.apache.org/jira/browse/SPARK-24588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li updated SPARK-24588:
Target Version/s: 2.3.2, 2.4.0
> StreamingSymmetricHashJoinExec should require HashClusteredPartitioning
[
https://issues.apache.org/jira/browse/SPARK-24542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-24542.
-
Resolution: Fixed
Fix Version/s: 2.3.2
2.4.0
Issue resolved by pull
[
https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516621#comment-16516621
]
Wenbo Zhao commented on SPARK-24578:
Hi [~irashid], many thanks for clarifying my questions. I tried
[
https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516611#comment-16516611
]
Imran Rashid commented on SPARK-24578:
--
btw to answer your initial questions:
{quote}
1. what is
[
https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516609#comment-16516609
]
Wenbo Zhao commented on SPARK-24578:
For now, we could reproduce this issue in completely different
[
https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenbo Zhao updated SPARK-24578:
---
Description:
After Spark 2.3, we observed lots of errors like the following in some of our
[
https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516602#comment-16516602
]
Wenbo Zhao commented on SPARK-24578:
[~irashid] We didn't touch
[
https://issues.apache.org/jira/browse/SPARK-24591?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516595#comment-16516595
]
Apache Spark commented on SPARK-24591:
--
User 'MaxGekk' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-24591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24591:
Assignee: (was: Apache Spark)
> Number of cores and executors in the cluster
>
[
https://issues.apache.org/jira/browse/SPARK-24591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24591:
Assignee: Apache Spark
> Number of cores and executors in the cluster
>
Maxim Gekk created SPARK-24591:
--
Summary: Number of cores and executors in the cluster
Key: SPARK-24591
URL: https://issues.apache.org/jira/browse/SPARK-24591
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-24590?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516586#comment-16516586
]
Apache Spark commented on SPARK-24590:
--
User 'HyukjinKwon' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-24590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24590:
Assignee: Apache Spark
> Make Jenkins tests passed with hadoop 3 profile
>
[
https://issues.apache.org/jira/browse/SPARK-24590?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24590:
Assignee: (was: Apache Spark)
> Make Jenkins tests passed with hadoop 3 profile
>
Hyukjin Kwon created SPARK-24590:
Summary: Make Jenkins tests passed with hadoop 3 profile
Key: SPARK-24590
URL: https://issues.apache.org/jira/browse/SPARK-24590
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516527#comment-16516527
]
Imran Rashid commented on SPARK-24578:
--
[~wbzhao] [~icexelloss] you're saying this is *without*
[
https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcelo Vanzin updated SPARK-24552:
---
Target Version/s: 2.1.3, 2.2.2, 2.3.2
Added some target versions. We should take the chance
[
https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Marcelo Vanzin updated SPARK-24552:
---
Comment: was deleted
(was: User 'vanzin' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-24589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24589:
Assignee: (was: Apache Spark)
> OutputCommitCoordinator may allow duplicate commits
[
https://issues.apache.org/jira/browse/SPARK-24589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516524#comment-16516524
]
Apache Spark commented on SPARK-24589:
--
User 'vanzin' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-24589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24589:
Assignee: Apache Spark
> OutputCommitCoordinator may allow duplicate commits
>
[
https://issues.apache.org/jira/browse/SPARK-24589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516523#comment-16516523
]
Marcelo Vanzin commented on SPARK-24589:
[~tgraves] fyi
> OutputCommitCoordinator may allow
[
https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516522#comment-16516522
]
Marcelo Vanzin commented on SPARK-24552:
I forked the output commiter issue into SPARK-24589 so
Marcelo Vanzin created SPARK-24589:
--
Summary: OutputCommitCoordinator may allow duplicate commits
Key: SPARK-24589
URL: https://issues.apache.org/jira/browse/SPARK-24589
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-24588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan updated SPARK-24588:
Affects Version/s: (was: 2.4.0)
2.3.0
2.3.1
>
[
https://issues.apache.org/jira/browse/SPARK-24588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan updated SPARK-24588:
Labels: correctness (was: )
> StreamingSymmetricHashJoinExec should require
[
https://issues.apache.org/jira/browse/SPARK-24588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan updated SPARK-24588:
Priority: Blocker (was: Major)
> StreamingSymmetricHashJoinExec should require
[
https://issues.apache.org/jira/browse/SPARK-24588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24588:
Assignee: Wenchen Fan (was: Apache Spark)
> StreamingSymmetricHashJoinExec should
[
https://issues.apache.org/jira/browse/SPARK-24588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516504#comment-16516504
]
Apache Spark commented on SPARK-24588:
--
User 'cloud-fan' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-24588?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24588:
Assignee: Apache Spark (was: Wenchen Fan)
> StreamingSymmetricHashJoinExec should
Wenchen Fan created SPARK-24588:
---
Summary: StreamingSymmetricHashJoinExec should require
HashClusteredPartitioning from children
Key: SPARK-24588
URL: https://issues.apache.org/jira/browse/SPARK-24588
[
https://issues.apache.org/jira/browse/SPARK-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516482#comment-16516482
]
Marcelo Vanzin commented on SPARK-19084:
(Please ignore my PR above - it should have tagged
[
https://issues.apache.org/jira/browse/SPARK-24586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24586:
Assignee: Wenchen Fan (was: Apache Spark)
> Upcast should not allow casting from string
[
https://issues.apache.org/jira/browse/SPARK-24586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516457#comment-16516457
]
Apache Spark commented on SPARK-24586:
--
User 'cloud-fan' has created a pull request for this issue:
[
https://issues.apache.org/jira/browse/SPARK-24586?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24586:
Assignee: Apache Spark (was: Wenchen Fan)
> Upcast should not allow casting from string
Ryan Deak created SPARK-24587:
-
Summary: RDD.takeOrdered uses reduce, pulling all partition data
to the driver
Key: SPARK-24587
URL: https://issues.apache.org/jira/browse/SPARK-24587
Project: Spark
Wenchen Fan created SPARK-24586:
---
Summary: Upcast should not allow casting from string to other types
Key: SPARK-24586
URL: https://issues.apache.org/jira/browse/SPARK-24586
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-24375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516411#comment-16516411
]
Mridul Muralidharan edited comment on SPARK-24375 at 6/18/18 10:17 PM:
[
https://issues.apache.org/jira/browse/SPARK-24375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516411#comment-16516411
]
Mridul Muralidharan commented on SPARK-24375:
-
[~jiangxb1987] A couple of comments based on
[
https://issues.apache.org/jira/browse/SPARK-24375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516411#comment-16516411
]
Mridul Muralidharan edited comment on SPARK-24375 at 6/18/18 10:15 PM:
David Lewis created SPARK-24585:
---
Summary: Adding ability to audit file system before and after test
to ensure all files are cleaned up.
Key: SPARK-24585
URL: https://issues.apache.org/jira/browse/SPARK-24585
[
https://issues.apache.org/jira/browse/SPARK-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516371#comment-16516371
]
Stavros Kontopoulos commented on SPARK-14540:
-
[~srowen] We will prepare a design doc for
[
https://issues.apache.org/jira/browse/SPARK-24248?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516368#comment-16516368
]
Matt Cheah commented on SPARK-24248:
I've summarized what we ended up going with after some
[
https://issues.apache.org/jira/browse/SPARK-24583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516332#comment-16516332
]
Apache Spark commented on SPARK-24583:
--
User 'maryannxue' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-24583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24583:
Assignee: Apache Spark
> Wrong schema type in InsertIntoDataSourceCommand
>
[
https://issues.apache.org/jira/browse/SPARK-24583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24583:
Assignee: (was: Apache Spark)
> Wrong schema type in InsertIntoDataSourceCommand
>
[
https://issues.apache.org/jira/browse/SPARK-24432?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516305#comment-16516305
]
Henry Robinson commented on SPARK-24432:
I'm really interested in this feature. What's the
[
https://issues.apache.org/jira/browse/SPARK-24584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516300#comment-16516300
]
Matt Cheah commented on SPARK-24584:
Related to https://issues.apache.org/jira/browse/SPARK-24248
>
Matt Cheah created SPARK-24584:
--
Summary: [K8s] More efficient storage of executor pod state
Key: SPARK-24584
URL: https://issues.apache.org/jira/browse/SPARK-24584
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-24548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan reassigned SPARK-24548:
---
Assignee: Liang-Chi Hsieh
> JavaPairRDD to Dataset in SPARK generates ambiguous results
>
[
https://issues.apache.org/jira/browse/SPARK-24548?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenchen Fan resolved SPARK-24548.
-
Resolution: Fixed
Fix Version/s: 2.4.0
Issue resolved by pull request 21576
[
https://issues.apache.org/jira/browse/SPARK-24423?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16516071#comment-16516071
]
Dilip Biswal commented on SPARK-24423:
--
[~maropu] Hello, yes. I will open a PR today/tomorrow.
>
Maryann Xue created SPARK-24583:
---
Summary: Wrong schema type in InsertIntoDataSourceCommand
Key: SPARK-24583
URL: https://issues.apache.org/jira/browse/SPARK-24583
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-24526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-24526:
Assignee: Trystan Leftwich
> Spaces in the build dir causes failures in the build/mvn
[
https://issues.apache.org/jira/browse/SPARK-24526?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-24526.
--
Resolution: Fixed
Fix Version/s: 2.4.0
Issue resolved by pull request 21534
[
https://issues.apache.org/jira/browse/SPARK-23772?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-23772.
--
Resolution: Fixed
Fix Version/s: 2.4.0
Issue resolved by pull request 20929
[
https://issues.apache.org/jira/browse/SPARK-24433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24433:
Assignee: (was: Apache Spark)
> Add Spark R support
> ---
>
>
[
https://issues.apache.org/jira/browse/SPARK-24433?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515971#comment-16515971
]
Apache Spark commented on SPARK-24433:
--
User 'ifilonenko' has created a pull request for this
[
https://issues.apache.org/jira/browse/SPARK-24433?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24433:
Assignee: Apache Spark
> Add Spark R support
> ---
>
>
[
https://issues.apache.org/jira/browse/SPARK-24375?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng resolved SPARK-24375.
---
Resolution: Done
> Design sketch: support barrier scheduling in Apache Spark
>
[
https://issues.apache.org/jira/browse/SPARK-24375?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515913#comment-16515913
]
Xiangrui Meng commented on SPARK-24375:
---
I'm closing this Jira in favor of formal design
[
https://issues.apache.org/jira/browse/SPARK-24580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng reassigned SPARK-24580:
-
Assignee: Jiang Xingbo (was: Xiangrui Meng)
> List scenarios to be handled by barrier
[
https://issues.apache.org/jira/browse/SPARK-24580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-24580:
--
Description:
List scenarios to be handled by barrier execution mode to help the design. We
Xiangrui Meng created SPARK-24582:
-
Summary: Design: Barrier execution mode
Key: SPARK-24582
URL: https://issues.apache.org/jira/browse/SPARK-24582
Project: Spark
Issue Type: Story
Xiangrui Meng created SPARK-24581:
-
Summary: Design: BarrierTaskContext.barrier()
Key: SPARK-24581
URL: https://issues.apache.org/jira/browse/SPARK-24581
Project: Spark
Issue Type: Story
Xiangrui Meng created SPARK-24580:
-
Summary: List scenarios to be handled by barrier execution mode
properly
Key: SPARK-24580
URL: https://issues.apache.org/jira/browse/SPARK-24580
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-24579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-24579:
--
Labels: Hydrogen (was: )
> SPIP: Standardize Optimized Data Exchange between Spark and DL/AI
[
https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515879#comment-16515879
]
Li Jin edited comment on SPARK-24578 at 6/18/18 3:24 PM:
-
cc @gatorsmile
[
https://issues.apache.org/jira/browse/SPARK-24579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-24579:
--
Description:
(see attached SPIP pdf for more details)
At the crossroads of big data and AI,
[
https://issues.apache.org/jira/browse/SPARK-24579?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiangrui Meng updated SPARK-24579:
--
Attachment: [SPARK-24579] SPIP_ Standardize Optimized Data Exchange between
Apache Spark and
[
https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515879#comment-16515879
]
Li Jin commented on SPARK-24578:
cc @gatorsmile
We found this when switching from 2.2.1 to 2.3.0 in one
Xiangrui Meng created SPARK-24579:
-
Summary: SPIP: Standardize Optimized Data Exchange between Spark
and DL/AI frameworks
Key: SPARK-24579
URL: https://issues.apache.org/jira/browse/SPARK-24579
[
https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenbo Zhao updated SPARK-24578:
---
Description:
After Spark 2.3, we observed lots of errors like the following in some of our
[
https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515858#comment-16515858
]
Wenbo Zhao commented on SPARK-24578:
An easier reproduciable cluster setting is 10 executors each
[
https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Li Jin updated SPARK-24578:
---
Component/s: (was: Input/Output)
Spark Core
> Reading remote cache block behavior
[
https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenbo Zhao updated SPARK-24578:
---
Description:
After Spark 2.3, we observed lots of errors like the following in some of our
[
https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenbo Zhao updated SPARK-24578:
---
Description:
After Spark 2.3, we observed lots of errors like the following
{code:java}
18/06/15
[
https://issues.apache.org/jira/browse/SPARK-24578?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Wenbo Zhao updated SPARK-24578:
---
Description:
After Spark 2.3, we observed lots of errors like the following
{code:java}
18/06/15
Wenbo Zhao created SPARK-24578:
--
Summary: Reading remote cache block behavior changes and causes
timeout issue
Key: SPARK-24578
URL: https://issues.apache.org/jira/browse/SPARK-24578
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-23858?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515809#comment-16515809
]
SemanticBeeng commented on SPARK-23858:
---
Do you have failing tests as specs, Bryan, please?
>
[
https://issues.apache.org/jira/browse/SPARK-24577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kuku1 updated SPARK-24577:
--
Description:
The Spark-submit example in the [K8s
[
https://issues.apache.org/jira/browse/SPARK-24577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kuku1 updated SPARK-24577:
--
Description:
The Spark-submit example in the [K8s
[
https://issues.apache.org/jira/browse/SPARK-24577?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Kuku1 updated SPARK-24577:
--
Description:
The Spark-submit example in the [K8s
Kuku1 created SPARK-24577:
-
Summary: Spark submit fails with documentation example spark-pi
Key: SPARK-24577
URL: https://issues.apache.org/jira/browse/SPARK-24577
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-14540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16515648#comment-16515648
]
Lukas Rytz commented on SPARK-14540:
[~skonto] and me (both from Lightbend) are working on this
[
https://issues.apache.org/jira/browse/SPARK-24573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon resolved SPARK-24573.
--
Resolution: Fixed
Fix Version/s: 2.4.0
Issue resolved by pull request 21579
[
https://issues.apache.org/jira/browse/SPARK-24573?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-24573:
Assignee: Hyukjin Kwon
> SBT Java checkstyle affecting the build
>
89 matches
Mail list logo
