[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349444#comment-16349444
]
Thomas Graves commented on SPARK-23304:
---
[~smilegator] just to make sure you saw my comment above,
[
https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349431#comment-16349431
]
Thomas Graves commented on SPARK-23309:
---
I'm curious if anyone else is seeing the same behavior?
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349430#comment-16349430
]
Thomas Graves commented on SPARK-23304:
---
I filed Jira
Thomas Graves created SPARK-23309:
-
Summary: Spark 2.3 cached query performance 20-30% worse then
spark 2.2
Key: SPARK-23309
URL: https://issues.apache.org/jira/browse/SPARK-23309
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349369#comment-16349369
]
Thomas Graves commented on SPARK-23304:
---
Note I've removed some of the columns from the output, if
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-23304:
--
Attachment: spark23_oldorc_explain.txt
spark22_oldorc_explain.txt
> Spark SQL
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349367#comment-16349367
]
Thomas Graves commented on SPARK-23304:
---
ok I've attached 2 files one with spark 2.3 and one with
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349270#comment-16349270
]
Thomas Graves commented on SPARK-23304:
---
so with the new ORC code is there anyway to control the #
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-23304:
--
Description:
The query below seems to ignore the coalesce. This is running spark 2.2 or
spark
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-23304:
--
Priority: Major (was: Blocker)
> Spark SQL coalesce() against hive not working
>
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349227#comment-16349227
]
Thomas Graves commented on SPARK-23304:
---
Ok, I just realized what you are getting at, I tried on
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349220#comment-16349220
]
Thomas Graves commented on SPARK-23304:
---
If it helps , spark 2.3 # partitions is 317531 and spark
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16349212#comment-16349212
]
Thomas Graves commented on SPARK-23304:
---
yes there are difference in the # of partitions between
Thomas Graves created SPARK-23304:
-
Summary: Spark SQL coalesce() against hive not working
Key: SPARK-23304
URL: https://issues.apache.org/jira/browse/SPARK-23304
Project: Spark
Issue Type:
[
https://issues.apache.org/jira/browse/SPARK-23189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16338005#comment-16338005
]
Thomas Graves commented on SPARK-23189:
---
for large jobs the specific stage page is a pain to
[
https://issues.apache.org/jira/browse/SPARK-23189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16336332#comment-16336332
]
Thomas Graves commented on SPARK-23189:
---
Personally I use the executors page a lot so that is why I
[
https://issues.apache.org/jira/browse/SPARK-23189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16335808#comment-16335808
]
Thomas Graves commented on SPARK-23189:
---
[~attilapiros] what do you mean by the last 3? Are you
[
https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16331045#comment-16331045
]
Thomas Graves commented on SPARK-20928:
---
what is status of this, it looks like subtasks are
[
https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16309910#comment-16309910
]
Thomas Graves commented on SPARK-22683:
---
[~jcuquemelle] . just to confirm your applications
[
https://issues.apache.org/jira/browse/SPARK-22577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16298874#comment-16298874
]
Thomas Graves commented on SPARK-22577:
---
correct (unfortunately don't time), anyone can take it
[
https://issues.apache.org/jira/browse/SPARK-22577?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16298854#comment-16298854
]
Thomas Graves commented on SPARK-22577:
---
Yes I was thinking the stage UI (and perhaps executor UI).
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16298589#comment-16298589
]
Thomas Graves commented on SPARK-22765:
---
ok so at least based on this one experiment up front
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297549#comment-16297549
]
Thomas Graves commented on SPARK-22765:
---
with SPARK-21656 does upfront allocation vs exponential
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297335#comment-16297335
]
Thomas Graves commented on SPARK-22765:
---
SPARK-21656 and the dynamic allocation should handle that,
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16297307#comment-16297307
]
Thomas Graves commented on SPARK-22765:
---
yes between stages becomes a problem with lower timeout,
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16296811#comment-16296811
]
Thomas Graves commented on SPARK-22765:
---
[~CodingCat] with SPARK-21656 executors shouldn't timeout
[
https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292855#comment-16292855
]
Thomas Graves commented on SPARK-22683:
---
Thanks for the clarification, a few of those I misread and
[
https://issues.apache.org/jira/browse/SPARK-22465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292608#comment-16292608
]
Thomas Graves commented on SPARK-22465:
---
Yes I think that makes sense.
> Cogroup of two
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292594#comment-16292594
]
Thomas Graves commented on SPARK-22765:
---
I'm not sure how mr style and 4-core executor go together.
[
https://issues.apache.org/jira/browse/SPARK-22465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16292569#comment-16292569
]
Thomas Graves commented on SPARK-22465:
---
I don't have time at the moment to work on this so if you
[
https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16291037#comment-16291037
]
Thomas Graves commented on SPARK-22683:
---
Another way to approach this is to have a pluggable policy
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289793#comment-16289793
]
Thomas Graves commented on SPARK-22765:
---
ok so before you do anything else I would suggest trying
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289700#comment-16289700
]
Thomas Graves commented on SPARK-22765:
---
ok so its basically they idle timeout during DAG computing
[
https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289656#comment-16289656
]
Thomas Graves commented on SPARK-22683:
---
I am also curious, when you are comparing spark to MR
[
https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289614#comment-16289614
]
Thomas Graves commented on SPARK-22683:
---
So the issue brought up here seems to be resource waste vs
[
https://issues.apache.org/jira/browse/SPARK-22765?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16289613#comment-16289613
]
Thomas Graves commented on SPARK-22765:
---
why doesn't idle timeout very small < 5 seconds work? If
[
https://issues.apache.org/jira/browse/SPARK-22653?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16270977#comment-16270977
]
Thomas Graves commented on SPARK-22653:
---
will have patch up shortly
> executorAddress registered
Thomas Graves created SPARK-22653:
-
Summary: executorAddress registered in
CoarseGrainedSchedulerBackend.executorDataMap is null
Key: SPARK-22653
URL: https://issues.apache.org/jira/browse/SPARK-22653
[
https://issues.apache.org/jira/browse/SPARK-22620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16269517#comment-16269517
]
Thomas Graves commented on SPARK-22620:
---
Looking at this some more it seems that normally when the
[
https://issues.apache.org/jira/browse/SPARK-22620?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16268792#comment-16268792
]
Thomas Graves commented on SPARK-22620:
---
Note this was running spark on yarn in cluster mode, this
Thomas Graves created SPARK-22620:
-
Summary: Deadlock in blacklisting when executor is removed due to
a heartbeat timeout
Key: SPARK-22620
URL: https://issues.apache.org/jira/browse/SPARK-22620
Thomas Graves created SPARK-22577:
-
Summary: executor page blacklist status should update with TaskSet
level blacklisting
Key: SPARK-22577
URL: https://issues.apache.org/jira/browse/SPARK-22577
[
https://issues.apache.org/jira/browse/SPARK-22465?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16246595#comment-16246595
]
Thomas Graves commented on SPARK-22465:
---
Its not strictly the 2G limit. He did hit that but he hit
Thomas Graves created HDFS-12670:
Summary: can't renew HDFS tokens with only the hdfs client jar
Key: HDFS-12670
URL: https://issues.apache.org/jira/browse/HDFS-12670
Project: Hadoop HDFS
Thomas Graves created HDFS-12670:
Summary: can't renew HDFS tokens with only the hdfs client jar
Key: HDFS-12670
URL: https://issues.apache.org/jira/browse/HDFS-12670
Project: Hadoop HDFS
[
https://issues.apache.org/jira/browse/SPARK-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16198665#comment-16198665
]
Thomas Graves commented on SPARK-21737:
---
I haven't had time to think about it much since the pr
[
https://issues.apache.org/jira/browse/SPARK-22218?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-22218:
--
Description:
Running on yarn, If you have any application re-attempts using the spark 2.2
[
https://issues.apache.org/jira/browse/SPARK-22218?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16194803#comment-16194803
]
Thomas Graves commented on SPARK-22218:
---
Working on a patch for this.
> spark shuffle services
Thomas Graves created SPARK-22218:
-
Summary: spark shuffle services fails to update secret on
application re-attempts
Key: SPARK-22218
URL: https://issues.apache.org/jira/browse/SPARK-22218
Project:
[
https://issues.apache.org/jira/browse/SPARK-21737?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16184120#comment-16184120
]
Thomas Graves commented on SPARK-21737:
---
We definitely still want to do this. I don't think
[
https://issues.apache.org/jira/browse/SPARK-22151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-22151:
--
Description:
Running in yarn cluster mode and trying to set pythonpath via
Thomas Graves created SPARK-22151:
-
Summary: PYTHONPATH not picked up from the spark.yarn.appMasterEnv
properly
Key: SPARK-22151
URL: https://issues.apache.org/jira/browse/SPARK-22151
Project: Spark
[
https://issues.apache.org/jira/browse/HADOOP-14907?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16181120#comment-16181120
]
Thomas Graves commented on HADOOP-14907:
Can you give more details on where the heap dump is
[
https://issues.apache.org/jira/browse/SPARK-21157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16179496#comment-16179496
]
Thomas Graves commented on SPARK-21157:
---
Just to point out that yarn/mapreduce/tez already have
[
https://issues.apache.org/jira/browse/YARN-7204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated YARN-7204:
Description:
If a user sends an archive without any files in it (only directories), yarn
fails to
[
https://issues.apache.org/jira/browse/YARN-7204?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated YARN-7204:
Description:
If a user sends an archive without any files in it (only directories), yarn
fails to
Thomas Graves created YARN-7204:
---
Summary: Localizer errors on archive without any files
Key: YARN-7204
URL: https://issues.apache.org/jira/browse/YARN-7204
Project: Hadoop YARN
Issue Type:
Thomas Graves created YARN-7204:
---
Summary: Localizer errors on archive without any files
Key: YARN-7204
URL: https://issues.apache.org/jira/browse/YARN-7204
Project: Hadoop YARN
Issue Type:
Thomas Graves created SPARK-21945:
-
Summary: pyspark --py-files doesn't work in yarn client mode
Key: SPARK-21945
URL: https://issues.apache.org/jira/browse/SPARK-21945
Project: Spark
Issue
[
https://issues.apache.org/jira/browse/SPARK-21890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves resolved SPARK-21890.
---
Resolution: Fixed
Assignee: Sanket Reddy
Fix Version/s: 2.3.0
>
[
https://issues.apache.org/jira/browse/SPARK-21937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves closed SPARK-21937.
-
Resolution: Duplicate
> Spark SQL DDL/DML docs non-existent
>
[
https://issues.apache.org/jira/browse/SPARK-21937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-21937:
--
Issue Type: Improvement (was: Bug)
> Spark SQL DDL/DML docs non-existent
>
Thomas Graves created SPARK-21937:
-
Summary: Spark SQL DDL/DML docs non-existent
Key: SPARK-21937
URL: https://issues.apache.org/jira/browse/SPARK-21937
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16154006#comment-16154006
]
Thomas Graves commented on SPARK-21888:
---
the client needs to get the hbase credentials for secure
[
https://issues.apache.org/jira/browse/SPARK-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153744#comment-16153744
]
Thomas Graves commented on SPARK-21888:
---
also note that you can do this in client mode by using the
[
https://issues.apache.org/jira/browse/SPARK-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16153739#comment-16153739
]
Thomas Graves commented on SPARK-21888:
---
[~mgaido] I don't think that is true unless something
[
https://issues.apache.org/jira/browse/SPARK-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-21888:
--
Description: While running Spark on Yarn in cluster mode, currently there
is no way to add any
[
https://issues.apache.org/jira/browse/SPARK-21890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-21890:
--
Description:
I observed this while running a oozie job trying to connect to hbase via spark.
[
https://issues.apache.org/jira/browse/SPARK-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150530#comment-16150530
]
Thomas Graves edited comment on SPARK-21888 at 9/1/17 1:37 PM:
---
Putting
[
https://issues.apache.org/jira/browse/SPARK-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16150530#comment-16150530
]
Thomas Graves commented on SPARK-21888:
---
Putting things into SPARK_CONF_DIR will work, the question
[
https://issues.apache.org/jira/browse/SPARK-21888?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-21888:
--
Issue Type: Improvement (was: Bug)
> Cannot add stuff to Client Classpath for Yarn Cluster
[
https://issues.apache.org/jira/browse/SPARK-21841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145413#comment-16145413
]
Thomas Graves commented on SPARK-21841:
---
sorry for so many comments, I guess this goes back to
[
https://issues.apache.org/jira/browse/SPARK-21841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145399#comment-16145399
]
Thomas Graves commented on SPARK-21841:
---
Oh no I guess that is explicit check in spark 2.1:
if
[
https://issues.apache.org/jira/browse/SPARK-21841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145396#comment-16145396
]
Thomas Graves commented on SPARK-21841:
---
Unfortunately that isn't working for me, but might be
[
https://issues.apache.org/jira/browse/SPARK-21841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145392#comment-16145392
]
Thomas Graves edited comment on SPARK-21841 at 8/29/17 2:39 PM:
I'll try
[
https://issues.apache.org/jira/browse/SPARK-21841?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145392#comment-16145392
]
Thomas Graves commented on SPARK-21841:
---
I'll try that out.
Honestly I didn't even know you could
[
https://issues.apache.org/jira/browse/SPARK-15689?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16145304#comment-16145304
]
Thomas Graves commented on SPARK-15689:
---
I haven't had a chance to look at this, someone just
[
https://issues.apache.org/jira/browse/SPARK-21798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves resolved SPARK-21798.
---
Resolution: Fixed
Assignee: Parth Gandhi
Fix Version/s: 2.3.0
[
https://issues.apache.org/jira/browse/SPARK-21798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-21798:
--
Issue Type: Bug (was: Improvement)
> No config to replace deprecated SPARK_CLASSPATH config
Thomas Graves created SPARK-21841:
-
Summary: Spark SQL doesn't pick up column added in hive when table
created with saveAsTable
Key: SPARK-21841
URL: https://issues.apache.org/jira/browse/SPARK-21841
[
https://issues.apache.org/jira/browse/SPARK-21501?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves resolved SPARK-21501.
---
Resolution: Fixed
Assignee: Sanket Reddy
Fix Version/s: 2.3.0
> Spark
[
https://issues.apache.org/jira/browse/SPARK-17321?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16138347#comment-16138347
]
Thomas Graves commented on SPARK-17321:
---
Yes that sounds good. It wouldn't hurt to verify the
[
https://issues.apache.org/jira/browse/SPARK-21798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16137310#comment-16137310
]
Thomas Graves commented on SPARK-21798:
---
You could need to add things to the classpath for custom
[
https://issues.apache.org/jira/browse/SPARK-21753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129326#comment-16129326
]
Thomas Graves commented on SPARK-21753:
---
I should also state that many other things work fine with
[
https://issues.apache.org/jira/browse/SPARK-21753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16129320#comment-16129320
]
Thomas Graves commented on SPARK-21753:
---
[~holden.ka...@gmail.com] would be curious if you have run
Thomas Graves created SPARK-21753:
-
Summary: running pi example with pypy on spark fails to serialize
Key: SPARK-21753
URL: https://issues.apache.org/jira/browse/SPARK-21753
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves reassigned SPARK-21656:
-
Assignee: Jong Yoon Lee
> spark dynamic allocation should not idle timeout executors
[
https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves resolved SPARK-21656.
---
Resolution: Fixed
Fix Version/s: 2.3.0
2.2.1
> spark dynamic
[
https://issues.apache.org/jira/browse/SPARK-20589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127942#comment-16127942
]
Thomas Graves commented on SPARK-20589:
---
Note that this type of option is also already supported in
[
https://issues.apache.org/jira/browse/SPARK-21714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16127204#comment-16127204
]
Thomas Graves commented on SPARK-21714:
---
I haven't had time to get to it, so it would be great if
Thomas Graves created SPARK-21714:
-
Summary: SparkSubmit in Yarn Client mode downloads remote files
and then reuploads them again
Key: SPARK-21714
URL: https://issues.apache.org/jira/browse/SPARK-21714
[
https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123439#comment-16123439
]
Thomas Graves edited comment on SPARK-21656 at 8/11/17 2:43 PM:
Note,
[
https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123439#comment-16123439
]
Thomas Graves commented on SPARK-21656:
---
Note, I've never said there is no counter part
[
https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123435#comment-16123435
]
Thomas Graves commented on SPARK-21656:
---
Yes there is a trade off here, use some more resource or
[
https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123393#comment-16123393
]
Thomas Graves commented on SPARK-21656:
---
I don't know what you mean by busy driver. The example of
[
https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16123358#comment-16123358
]
Thomas Graves commented on SPARK-21656:
---
example of test results with this.
We have production job
Thomas Graves created SPARK-21695:
-
Summary: Spark scheduler locality algorithm can take longer then
expected
Key: SPARK-21695
URL: https://issues.apache.org/jira/browse/SPARK-21695
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-21656:
--
Description:
Right now with dynamic allocation spark starts by getting the number of
[
https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-21656:
--
Description:
Right now spark lets go of executors when they are idle for the 60s (or
[
https://issues.apache.org/jira/browse/SPARK-21656?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-21656:
--
Summary: spark dynamic allocation should not idle timeout executors when
there are enough
801 - 900 of 3203 matches
Mail list logo
