[
https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16512504#comment-16512504
]
Thomas Graves commented on SPARK-24552:
---
Note if this is a correctness bug and can
[
https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-24552:
--
Priority: Blocker (was: Major)
> Task attempt numbers are reused when stages are retried
> --
[
https://issues.apache.org/jira/browse/SPARK-24552?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16512500#comment-16512500
]
Thomas Graves commented on SPARK-24552:
---
I agree, I don't think changing the attem
[
https://issues.apache.org/jira/browse/SPARK-24415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-24415:
--
Priority: Critical (was: Major)
> Stage page aggregated executor metrics wrong when failures
[
https://issues.apache.org/jira/browse/SPARK-24415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16495224#comment-16495224
]
Thomas Graves commented on SPARK-24415:
---
ok so the issue here is in the AppStatusL
[
https://issues.apache.org/jira/browse/SPARK-24415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16495204#comment-16495204
]
Thomas Graves commented on SPARK-24415:
---
It also looks like in the history server
[
https://issues.apache.org/jira/browse/SPARK-24415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16495200#comment-16495200
]
Thomas Graves commented on SPARK-24415:
---
this might actually be an order of events
[
https://issues.apache.org/jira/browse/SPARK-24415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-24415:
--
Description:
Running with spark 2.3 on yarn and having task failures and blacklisting, the
ag
[
https://issues.apache.org/jira/browse/SPARK-24415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-24415:
--
Description:
Running with spark 2.3 on yarn and having task failures and blacklisting, the
ag
[
https://issues.apache.org/jira/browse/SPARK-24415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-24415:
--
Description:
Running with spark 2.3 on yarn and having task failures and blacklisting, the
ag
[
https://issues.apache.org/jira/browse/SPARK-24413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves resolved SPARK-24413.
---
Resolution: Duplicate
> Executor Blacklisting shouldn't immediately fail the application if
[
https://issues.apache.org/jira/browse/SPARK-24413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16494279#comment-16494279
]
Thomas Graves commented on SPARK-24413:
---
thanks for linking those we can just dup
[
https://issues.apache.org/jira/browse/SPARK-24414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16494265#comment-16494265
]
Thomas Graves commented on SPARK-24414:
---
also just an fyi I also filed SPARK-24415
[
https://issues.apache.org/jira/browse/SPARK-24414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16494237#comment-16494237
]
Thomas Graves commented on SPARK-24414:
---
I am looking to see if we can just return
[
https://issues.apache.org/jira/browse/SPARK-24414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16494199#comment-16494199
]
Thomas Graves commented on SPARK-24414:
---
looks like this was broken by SPARK-23147
[
https://issues.apache.org/jira/browse/SPARK-24415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-24415:
--
Description:
Running with spark 2.3 on yarn and having task failures and blacklisting, the
ag
Thomas Graves created SPARK-24415:
-
Summary: Stage page aggregated executor metrics wrong when
failures
Key: SPARK-24415
URL: https://issues.apache.org/jira/browse/SPARK-24415
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-24415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-24415:
--
Attachment: Screen Shot 2018-05-29 at 2.15.38 PM.png
> Stage page aggregated executor metrics
[
https://issues.apache.org/jira/browse/SPARK-24414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16494083#comment-16494083
]
Thomas Graves commented on SPARK-24414:
---
to reproduce this simply start a shell:
Thomas Graves created SPARK-24414:
-
Summary: Stages page doesn't show all task attempts when failures
Key: SPARK-24414
URL: https://issues.apache.org/jira/browse/SPARK-24414
Project: Spark
Is
[
https://issues.apache.org/jira/browse/SPARK-24413?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16493978#comment-16493978
]
Thomas Graves commented on SPARK-24413:
---
[~imranr] thoughts on this?
> Executor
[
https://issues.apache.org/jira/browse/SPARK-24413?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-24413:
--
Summary: Executor Blacklisting shouldn't immediately fail the application
if dynamic allocatio
Thomas Graves created SPARK-24413:
-
Summary: Executor Blacklisting shouldn't immediately fail the
application if dynamic allocation is enabled and it doesn't have any other
active executors
Key: SPARK-24413
URL:
[
https://issues.apache.org/jira/browse/SPARK-6235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16482746#comment-16482746
]
Thomas Graves commented on SPARK-6235:
--
>> Still unsupported:
* large task results
[
https://issues.apache.org/jira/browse/SPARK-21033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16467803#comment-16467803
]
Thomas Graves commented on SPARK-21033:
---
[~cloud_fan] the followup PR [https://gith
[
https://issues.apache.org/jira/browse/SPARK-24124?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16458835#comment-16458835
]
Thomas Graves commented on SPARK-24124:
---
[~vanzin] any objections to this?
> Spar
Thomas Graves created SPARK-24124:
-
Summary: Spark history server should create
spark.history.store.path and set permissions properly
Key: SPARK-24124
URL: https://issues.apache.org/jira/browse/SPARK-24124
[
https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves reassigned SPARK-22683:
-
Assignee: Julien Cuquemelle
> DynamicAllocation wastes resources by allocating container
[
https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16450131#comment-16450131
]
Thomas Graves commented on SPARK-22683:
---
Note this added a new config spark.dynamic
[
https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves resolved SPARK-22683.
---
Resolution: Fixed
Fix Version/s: 2.4.0
> DynamicAllocation wastes resources by allocat
[
https://issues.apache.org/jira/browse/SPARK-23850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16449054#comment-16449054
]
Thomas Graves commented on SPARK-23850:
---
the url seems somewhat silly to me to, loo
[
https://issues.apache.org/jira/browse/SPARK-23964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444043#comment-16444043
]
Thomas Graves commented on SPARK-23964:
---
so far in my testing I haven't seen any pe
[
https://issues.apache.org/jira/browse/SPARK-15703?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16440855#comment-16440855
]
Thomas Graves commented on SPARK-15703:
---
this Jira is purely making the size of the
[
https://issues.apache.org/jira/browse/SPARK-23964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16434479#comment-16434479
]
Thomas Graves commented on SPARK-23964:
---
I'm not sure, I'm trying to figure out if
[
https://issues.apache.org/jira/browse/SPARK-23964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-23964:
--
Description:
The spillable class has a check in maybeSpill as to when it tries to acquire
more
[
https://issues.apache.org/jira/browse/SPARK-23964?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16434454#comment-16434454
]
Thomas Graves commented on SPARK-23964:
---
[~andrewor14] [~matei] [~r...@databricks.
[
https://issues.apache.org/jira/browse/SPARK-23964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-23964:
--
Description:
The spillable class has a check:
if (elementsRead % 32 == 0 && currentMemory >= m
[
https://issues.apache.org/jira/browse/SPARK-23964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-23964:
--
Environment: (was: The spillable class has a check:
if (elementsRead % 32 == 0 && currentMe
Thomas Graves created SPARK-23964:
-
Summary: why does Spillable wait for 32 elements?
Key: SPARK-23964
URL: https://issues.apache.org/jira/browse/SPARK-23964
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432467#comment-16432467
]
Thomas Graves commented on SPARK-16630:
---
sorry I don't follow, the list we get from
[
https://issues.apache.org/jira/browse/SPARK-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16432244#comment-16432244
]
Thomas Graves commented on SPARK-16630:
---
yes I think it would make sense as the uni
[
https://issues.apache.org/jira/browse/SPARK-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16427377#comment-16427377
]
Thomas Graves commented on SPARK-16630:
---
the problem is that spark.executor.instanc
[
https://issues.apache.org/jira/browse/SPARK-23567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves resolved SPARK-23567.
---
Resolution: Duplicate
> spark.redaction.regex should not include user by default, docs not up
[
https://issues.apache.org/jira/browse/SPARK-23039?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16423996#comment-16423996
]
Thomas Graves commented on SPARK-23039:
---
seems to be a dup of
# SPARK-23057
> F
[
https://issues.apache.org/jira/browse/SPARK-23850?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422965#comment-16422965
]
Thomas Graves commented on SPARK-23850:
---
ping [~ash...@gmail.com] [~onursatici] [~L
Thomas Graves created SPARK-23850:
-
Summary: We should not redact username|user|url from UI by default
Key: SPARK-23850
URL: https://issues.apache.org/jira/browse/SPARK-23850
Project: Spark
I
[
https://issues.apache.org/jira/browse/SPARK-23806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-23806:
--
Description:
Very similar to https://issues.apache.org/jira/browse/SPARK-22618 . But this
coul
Thomas Graves created SPARK-23806:
-
Summary: Broadcast. unpersist can cause fatal exception when used
with dynamic allocation
Key: SPARK-23806
URL: https://issues.apache.org/jira/browse/SPARK-23806
Pr
[
https://issues.apache.org/jira/browse/SPARK-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16417466#comment-16417466
]
Thomas Graves commented on SPARK-22618:
---
I'll file a separate Jira for it and put u
[
https://issues.apache.org/jira/browse/SPARK-22618?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16416309#comment-16416309
]
Thomas Graves commented on SPARK-22618:
---
thanks for fixing this, hitting it now in
[
https://issues.apache.org/jira/browse/SPARK-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16390232#comment-16390232
]
Thomas Graves commented on SPARK-16630:
---
yes yarn tells you the # of nodemanagers.
[
https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387963#comment-16387963
]
Thomas Graves commented on SPARK-22683:
---
I left comments on the open PR already, le
[
https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387938#comment-16387938
]
Thomas Graves commented on SPARK-22683:
---
[~jcuquemelle] do you have time to update
[
https://issues.apache.org/jira/browse/SPARK-16630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387846#comment-16387846
]
Thomas Graves commented on SPARK-16630:
---
yes something along these lines is what I
[
https://issues.apache.org/jira/browse/SPARK-23567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383723#comment-16383723
]
Thomas Graves edited comment on SPARK-23567 at 3/2/18 3:57 PM:
[
https://issues.apache.org/jira/browse/SPARK-22479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383733#comment-16383733
]
Thomas Graves commented on SPARK-22479:
---
Also the example above shows the password,
[
https://issues.apache.org/jira/browse/SPARK-23567?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383723#comment-16383723
]
Thomas Graves commented on SPARK-23567:
---
I also question whether the url should be
Thomas Graves created SPARK-23567:
-
Summary: spark.redaction.regex should not include user by default,
docs not updated
Key: SPARK-23567
URL: https://issues.apache.org/jira/browse/SPARK-23567
Project:
[
https://issues.apache.org/jira/browse/SPARK-22479?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16383709#comment-16383709
]
Thomas Graves commented on SPARK-22479:
---
[~aash] [~onursatici] this seems to have r
[
https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16357346#comment-16357346
]
Thomas Graves commented on SPARK-23309:
---
sorry I haven't had time to make a query/d
[
https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16356168#comment-16356168
]
Thomas Graves commented on SPARK-22683:
---
I agree, I think default behavior stays 1.
[
https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16356168#comment-16356168
]
Thomas Graves edited comment on SPARK-22683 at 2/7/18 10:24 PM:
---
[
https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16355920#comment-16355920
]
Thomas Graves commented on SPARK-22683:
---
If the config is set to 1 which keeps the
[
https://issues.apache.org/jira/browse/SPARK-22683?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1637#comment-1637
]
Thomas Graves commented on SPARK-22683:
---
ok thanks, I would like to try this out m
[
https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16354336#comment-16354336
]
Thomas Graves commented on SPARK-23309:
---
I pulled in that patch ([https://github.co
[
https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350901#comment-16350901
]
Thomas Graves edited comment on SPARK-23309 at 2/2/18 8:29 PM:
[
https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350901#comment-16350901
]
Thomas Graves edited comment on SPARK-23309 at 2/2/18 8:29 PM:
[
https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350813#comment-16350813
]
Thomas Graves edited comment on SPARK-23309 at 2/2/18 8:15 PM:
[
https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350901#comment-16350901
]
Thomas Graves commented on SPARK-23309:
---
I should ask is there a log statement or q
[
https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350900#comment-16350900
]
Thomas Graves commented on SPARK-23309:
---
So the last test I did was spark 2.3 with
[
https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350813#comment-16350813
]
Thomas Graves edited comment on SPARK-23309 at 2/2/18 7:04 PM:
[
https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350813#comment-16350813
]
Thomas Graves commented on SPARK-23309:
---
I'm still seeing spark 2.3 slower by about
[
https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350533#comment-16350533
]
Thomas Graves commented on SPARK-23309:
---
Note the schema of "something" here is a "
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves resolved SPARK-23304.
---
Resolution: Invalid
> Spark SQL coalesce() against hive not working
> ---
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350440#comment-16350440
]
Thomas Graves commented on SPARK-23304:
---
ok so I guess by that logic then the coale
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350423#comment-16350423
]
Thomas Graves commented on SPARK-23304:
---
it doesn't look like sql("xyz").rdd.partit
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16350428#comment-16350428
]
Thomas Graves commented on SPARK-23304:
---
well I guess that give you end # of partit
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349555#comment-16349555
]
Thomas Graves commented on SPARK-23304:
---
I don't have any hive tables backed by par
[
https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349553#comment-16349553
]
Thomas Graves commented on SPARK-23309:
---
[~dongjoon] is there any native way with t
[
https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349551#comment-16349551
]
Thomas Graves commented on SPARK-23309:
---
seeing the same time difference after addi
[
https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349483#comment-16349483
]
Thomas Graves commented on SPARK-23309:
---
sure, I can also run with the --conf spar
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349480#comment-16349480
]
Thomas Graves commented on SPARK-23304:
---
I just ran the query (show()) and saw the
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-23304:
--
Attachment: spark23_oldorc_explain_convermetastoreorcfalse.txt
> Spark SQL coalesce() against h
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349446#comment-16349446
]
Thomas Graves commented on SPARK-23304:
---
It still seems like a bug to me since the
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349444#comment-16349444
]
Thomas Graves commented on SPARK-23304:
---
[~smilegator] just to make sure you saw my
[
https://issues.apache.org/jira/browse/SPARK-23309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349431#comment-16349431
]
Thomas Graves commented on SPARK-23309:
---
I'm curious if anyone else is seeing the s
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349430#comment-16349430
]
Thomas Graves commented on SPARK-23304:
---
I filed Jira https://issues.apache.org/
Thomas Graves created SPARK-23309:
-
Summary: Spark 2.3 cached query performance 20-30% worse then
spark 2.2
Key: SPARK-23309
URL: https://issues.apache.org/jira/browse/SPARK-23309
Project: Spark
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349369#comment-16349369
]
Thomas Graves commented on SPARK-23304:
---
Note I've removed some of the columns from
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-23304:
--
Attachment: spark23_oldorc_explain.txt
spark22_oldorc_explain.txt
> Spark SQL c
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349367#comment-16349367
]
Thomas Graves commented on SPARK-23304:
---
ok I've attached 2 files one with spark 2.
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349270#comment-16349270
]
Thomas Graves commented on SPARK-23304:
---
so with the new ORC code is there anyway t
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-23304:
--
Description:
The query below seems to ignore the coalesce. This is running spark 2.2 or
spark
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Thomas Graves updated SPARK-23304:
--
Priority: Major (was: Blocker)
> Spark SQL coalesce() against hive not working
> -
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349227#comment-16349227
]
Thomas Graves commented on SPARK-23304:
---
Ok, I just realized what you are getting a
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349220#comment-16349220
]
Thomas Graves commented on SPARK-23304:
---
If it helps , spark 2.3 # partitions is 31
[
https://issues.apache.org/jira/browse/SPARK-23304?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16349212#comment-16349212
]
Thomas Graves commented on SPARK-23304:
---
yes there are difference in the # of parti
Thomas Graves created SPARK-23304:
-
Summary: Spark SQL coalesce() against hive not working
Key: SPARK-23304
URL: https://issues.apache.org/jira/browse/SPARK-23304
Project: Spark
Issue Type: B
[
https://issues.apache.org/jira/browse/SPARK-23189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16338005#comment-16338005
]
Thomas Graves commented on SPARK-23189:
---
for large jobs the specific stage page is
[
https://issues.apache.org/jira/browse/SPARK-23189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336332#comment-16336332
]
Thomas Graves commented on SPARK-23189:
---
Personally I use the executors page a lot
701 - 800 of 1865 matches
Mail list logo
