[
https://issues.apache.org/jira/browse/SPARK-24927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556975#comment-16556975
]
Xiao Li commented on SPARK-24927:
-
cc [~jerryshao]
> The hadoop-provided profile doesn
Cheng Lian created SPARK-24927:
--
Summary: The hadoop-provided profile doesn't play well with
Snappy-compressed Parquet files
Key: SPARK-24927
URL: https://issues.apache.org/jira/browse/SPARK-24927
Projec
[
https://issues.apache.org/jira/browse/SPARK-24926?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556965#comment-16556965
]
Imran Rashid commented on SPARK-24926:
--
I was talking to [~nsheth] about this, he's
Imran Rashid created SPARK-24926:
Summary: Ensure numCores is used consistently in all netty
configuration (driver and executors)
Key: SPARK-24926
URL: https://issues.apache.org/jira/browse/SPARK-24926
[
https://issues.apache.org/jira/browse/SPARK-24918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556964#comment-16556964
]
Imran Rashid commented on SPARK-24918:
--
I have some changes with an initial draft o
[
https://issues.apache.org/jira/browse/SPARK-23128?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556961#comment-16556961
]
Carson Wang commented on SPARK-23128:
-
Thanks [~tgraves] very much. I'll follow this
[
https://issues.apache.org/jira/browse/SPARK-24882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556896#comment-16556896
]
Ryan Blue commented on SPARK-24882:
---
Sounds fine, but it's getting close and I wouldn'
[
https://issues.apache.org/jira/browse/SPARK-24882?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556885#comment-16556885
]
Wenchen Fan commented on SPARK-24882:
-
We don't need to rush for 2.4, but would be g
[
https://issues.apache.org/jira/browse/SPARK-24374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li updated SPARK-24374:
Affects Version/s: (was: 3.0.0)
2.4.0
> SPIP: Support Barrier Execution Mode in
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556837#comment-16556837
]
Genmao Yu edited comment on SPARK-24630 at 7/26/18 3:24 AM:
[
https://issues.apache.org/jira/browse/SPARK-24630?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556837#comment-16556837
]
Genmao Yu commented on SPARK-24630:
---
[~zsxwing] Is there plan to better support SQL on
[
https://issues.apache.org/jira/browse/SPARK-24921?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556826#comment-16556826
]
Hyukjin Kwon commented on SPARK-24921:
--
[~tommyshiou], is this rather a question? I
[
https://issues.apache.org/jira/browse/SPARK-24914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556824#comment-16556824
]
Hyukjin Kwon commented on SPARK-24914:
--
cc [~ZenWzh]
> totalSize is not a good est
[
https://issues.apache.org/jira/browse/SPARK-24925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556820#comment-16556820
]
yucai commented on SPARK-24925:
---
[~cloud_fan], [~xiaoli] , [~kiszk] , any comments?
> inp
[
https://issues.apache.org/jira/browse/SPARK-24925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24925:
Assignee: Apache Spark
> input bytesRead metrics fluctuate from time to time
> --
[
https://issues.apache.org/jira/browse/SPARK-24925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556819#comment-16556819
]
Apache Spark commented on SPARK-24925:
--
User 'yucai' has created a pull request for
[
https://issues.apache.org/jira/browse/SPARK-24925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24925:
Assignee: (was: Apache Spark)
> input bytesRead metrics fluctuate from time to time
>
[
https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556801#comment-16556801
]
Hyukjin Kwon edited comment on SPARK-24288 at 7/26/18 2:56 AM:
---
[
https://issues.apache.org/jira/browse/SPARK-24925?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556818#comment-16556818
]
yucai commented on SPARK-24925:
---
I think there could be two issues.
In FileScanRDD
1. Col
[
https://issues.apache.org/jira/browse/SPARK-24905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon updated SPARK-24905:
-
Priority: Major (was: Critical)
> Spark 2.3 Internal URL env variable
> ---
[
https://issues.apache.org/jira/browse/SPARK-24905?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556817#comment-16556817
]
Hyukjin Kwon commented on SPARK-24905:
--
(please avoid to set Critical+ which is usu
[
https://issues.apache.org/jira/browse/SPARK-24897?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556806#comment-16556806
]
Hyukjin Kwon commented on SPARK-24897:
--
I couldn't follow it too.
> DAGScheduler s
[
https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556801#comment-16556801
]
Hyukjin Kwon commented on SPARK-24288:
--
[~smilegator] should we resolve this {{Won'
[
https://issues.apache.org/jira/browse/SPARK-24925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-24925:
--
Attachment: bytesRead.gif
> input bytesRead metrics fluctuate from time to time
>
[
https://issues.apache.org/jira/browse/SPARK-24925?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-24925:
--
Description:
input bytesRead metrics fluctuate from time to time, it is worse when pushdown
enabled.
Query
{
yucai created SPARK-24925:
-
Summary: input bytesRead metrics fluctuate from time to time
Key: SPARK-24925
URL: https://issues.apache.org/jira/browse/SPARK-24925
Project: Spark
Issue Type: Bug
[
https://issues.apache.org/jira/browse/SPARK-24832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-24832:
--
Summary: Improve inputMetrics's bytesRead update for ColumnarBatch (was:
When pushdown enabled, input bytesRe
[
https://issues.apache.org/jira/browse/SPARK-24832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
yucai updated SPARK-24832:
--
Summary: When pushdown enabled, input bytesRead metrics is easy to
fluctuate from time to time (was: Improve
[
https://issues.apache.org/jira/browse/SPARK-24867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556760#comment-16556760
]
Saisai Shao commented on SPARK-24867:
-
I see, thanks! Please let me know when the JI
[
https://issues.apache.org/jira/browse/SPARK-12911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556470#comment-16556470
]
David Vogelbacher commented on SPARK-12911:
---
Hey [~hyukjin.kwon] [~sdicocco][~
[
https://issues.apache.org/jira/browse/SPARK-24916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Yuming Wang resolved SPARK-24916.
-
Resolution: Duplicate
> Fix type coercion for IN expression with subquery
>
[
https://issues.apache.org/jira/browse/SPARK-24867?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556447#comment-16556447
]
Xiao Li commented on SPARK-24867:
-
[~jerryshao] This ticket was just resolved. [~lian ch
[
https://issues.apache.org/jira/browse/SPARK-24867?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li resolved SPARK-24867.
-
Resolution: Fixed
Fix Version/s: 2.3.2
> Add AnalysisBarrier to DataFrameWriter
> --
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24924:
Assignee: Apache Spark
> Add mapping for built-in Avro data source
>
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556427#comment-16556427
]
Apache Spark commented on SPARK-24924:
--
User 'dongjoon-hyun' has created a pull req
[
https://issues.apache.org/jira/browse/SPARK-24924?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24924:
Assignee: (was: Apache Spark)
> Add mapping for built-in Avro data source
> -
Dongjoon Hyun created SPARK-24924:
-
Summary: Add mapping for built-in Avro data source
Key: SPARK-24924
URL: https://issues.apache.org/jira/browse/SPARK-24924
Project: Spark
Issue Type: Sub-t
[
https://issues.apache.org/jira/browse/SPARK-24906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556413#comment-16556413
]
Jason Guo commented on SPARK-24906:
---
[~maropu] [~viirya] What do you think about thi
[
https://issues.apache.org/jira/browse/SPARK-24923?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556383#comment-16556383
]
Apache Spark commented on SPARK-24923:
--
User 'rdblue' has created a pull request fo
[
https://issues.apache.org/jira/browse/SPARK-24923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24923:
Assignee: (was: Apache Spark)
> DataSourceV2: Add CTAS and RTAS logical operations
>
[
https://issues.apache.org/jira/browse/SPARK-24923?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24923:
Assignee: Apache Spark
> DataSourceV2: Add CTAS and RTAS logical operations
> ---
[
https://issues.apache.org/jira/browse/SPARK-24921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Tommy S updated SPARK-24921:
Component/s: Web UI
> SparkStreaming steadily increasing job generation delay due to apparent
> URLClassL
Ryan Blue created SPARK-24923:
-
Summary: DataSourceV2: Add CTAS and RTAS logical operations
Key: SPARK-24923
URL: https://issues.apache.org/jira/browse/SPARK-24923
Project: Spark
Issue Type: Sub-
[
https://issues.apache.org/jira/browse/SPARK-24802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556310#comment-16556310
]
Apache Spark commented on SPARK-24802:
--
User 'maryannxue' has created a pull reques
[
https://issues.apache.org/jira/browse/SPARK-1137?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556251#comment-16556251
]
Apache Spark commented on SPARK-1137:
-
User 'aarondav' has created a pull request for
[
https://issues.apache.org/jira/browse/SPARK-24874?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556241#comment-16556241
]
Reynold Xin commented on SPARK-24874:
-
Do we really need this? Seems like an uncommo
[
https://issues.apache.org/jira/browse/SPARK-24860?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li resolved SPARK-24860.
-
Resolution: Fixed
Assignee: Koert Kuipers
Fix Version/s: 2.4.0
> Expose dynamic partitio
[
https://issues.apache.org/jira/browse/SPARK-23146?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Matt Cheah resolved SPARK-23146.
Resolution: Fixed
Fix Version/s: 2.4.0
> Support client mode for Kubernetes cluster backend
[
https://issues.apache.org/jira/browse/SPARK-24915?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556126#comment-16556126
]
Bryan Cutler commented on SPARK-24915:
--
Hi [~stspencer], I've been trying fix simil
[
https://issues.apache.org/jira/browse/SPARK-24288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556119#comment-16556119
]
Apache Spark commented on SPARK-24288:
--
User 'maryannxue' has created a pull reques
[
https://issues.apache.org/jira/browse/SPARK-23146?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16556098#comment-16556098
]
Apache Spark commented on SPARK-23146:
--
User 'mccheah' has created a pull request f
[
https://issues.apache.org/jira/browse/SPARK-24849?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li resolved SPARK-24849.
-
Resolution: Fixed
Assignee: Maxim Gekk
Fix Version/s: 2.4.0
> Convert StructType to DDL
[
https://issues.apache.org/jira/browse/SPARK-24911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li resolved SPARK-24911.
-
Resolution: Fixed
Fix Version/s: 2.4.0
> SHOW CREATE TABLE drops escaping of nested column names
[
https://issues.apache.org/jira/browse/SPARK-24911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li reassigned SPARK-24911:
---
Assignee: Maxim Gekk
> SHOW CREATE TABLE drops escaping of nested column names
> --
[
https://issues.apache.org/jira/browse/SPARK-24922?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Dinesh Dharme updated SPARK-24922:
--
Description:
I am trying to do few (union + reduceByKey) operations on a hiearchical dataset
Dinesh Dharme created SPARK-24922:
-
Summary: Iterative rdd union + reduceByKey operations on small
dataset leads to "No space left on device" error on account of lot of shuffle
spill.
Key: SPARK-24922
URL: https:
Tommy S created SPARK-24921:
---
Summary: SparkStreaming steadily increasing job generation delay
due to apparent URLClassLoader contention
Key: SPARK-24921
URL: https://issues.apache.org/jira/browse/SPARK-24921
[
https://issues.apache.org/jira/browse/SPARK-24914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555998#comment-16555998
]
Bruce Robbins commented on SPARK-24914:
---
[~irashid]
{quote}
given HIVE-20079, can
[
https://issues.apache.org/jira/browse/SPARK-24914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Bruce Robbins updated SPARK-24914:
--
Description:
When determining whether to do a broadcast join, Spark estimates the size of
the
Imran Rashid created SPARK-24920:
Summary: Spark should share netty's memory pools across all uses
Key: SPARK-24920
URL: https://issues.apache.org/jira/browse/SPARK-24920
Project: Spark
Issue
Gengliang Wang created SPARK-24919:
--
Summary: Scala linter rule for sparkContext.hadoopConfiguration
Key: SPARK-24919
URL: https://issues.apache.org/jira/browse/SPARK-24919
Project: Spark
Is
[
https://issues.apache.org/jira/browse/SPARK-24919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24919:
Assignee: (was: Apache Spark)
> Scala linter rule for sparkContext.hadoopConfiguratio
[
https://issues.apache.org/jira/browse/SPARK-24918?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555939#comment-16555939
]
Imran Rashid commented on SPARK-24918:
--
[~jerryshao] [~tgraves] you might be intere
Imran Rashid created SPARK-24918:
Summary: Executor Plugin API
Key: SPARK-24918
URL: https://issues.apache.org/jira/browse/SPARK-24918
Project: Spark
Issue Type: New Feature
Compone
[
https://issues.apache.org/jira/browse/SPARK-24914?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555877#comment-16555877
]
Imran Rashid commented on SPARK-24914:
--
given HIVE-20079, can we also have a conf t
[
https://issues.apache.org/jira/browse/SPARK-24920?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Imran Rashid updated SPARK-24920:
-
Summary: Spark should allow sharing netty's memory pools across all uses
(was: Spark should sha
[
https://issues.apache.org/jira/browse/SPARK-24919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24919:
Assignee: Apache Spark
> Scala linter rule for sparkContext.hadoopConfiguration
> ---
[
https://issues.apache.org/jira/browse/SPARK-24919?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555972#comment-16555972
]
Apache Spark commented on SPARK-24919:
--
User 'gengliangwang' has created a pull req
[
https://issues.apache.org/jira/browse/SPARK-24904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555652#comment-16555652
]
Marco Gaido edited comment on SPARK-24904 at 7/25/18 1:28 PM:
[
https://issues.apache.org/jira/browse/SPARK-24904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1678#comment-1678
]
Shay Elbaz commented on SPARK-24904:
[~mgaido] Technically you *can* that, you just
[
https://issues.apache.org/jira/browse/SPARK-24904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shay Elbaz updated SPARK-24904:
---
Issue Type: Improvement (was: Question)
> Join with broadcasted dataframe causes shuffle of redunda
[
https://issues.apache.org/jira/browse/SPARK-19018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Xiao Li updated SPARK-19018:
Issue Type: Improvement (was: Bug)
> spark csv writer charset support
>
[
https://issues.apache.org/jira/browse/SPARK-24904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555842#comment-16555842
]
Marco Gaido commented on SPARK-24904:
-
[~shay_elbaz] In the case I mentioned before
[
https://issues.apache.org/jira/browse/SPARK-24917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vincent updated SPARK-24917:
Description:
Hello
while investigating some OOM errors in Spark 2.2 [(here's my call
stack)|https://imag
[
https://issues.apache.org/jira/browse/SPARK-24917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Vincent updated SPARK-24917:
Description:
Hello
while investigating some OOM errors in Spark 2.2 [(here's my call
stack)|https://imag
Vincent created SPARK-24917:
---
Summary: Sending a partition over netty results in 2x memory usage
Key: SPARK-24917
URL: https://issues.apache.org/jira/browse/SPARK-24917
Project: Spark
Issue Type: I
[
https://issues.apache.org/jira/browse/SPARK-24904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555769#comment-16555769
]
Shay Elbaz commented on SPARK-24904:
[~mgaido] indeed this assumption is not always
[
https://issues.apache.org/jira/browse/SPARK-24904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555652#comment-16555652
]
Marco Gaido commented on SPARK-24904:
-
I see now what you mean, but yes, It think th
[
https://issues.apache.org/jira/browse/SPARK-24904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Shay Elbaz updated SPARK-24904:
---
Description:
When joining a "large" dataframe with broadcasted small one, and join-type is
on the s
[
https://issues.apache.org/jira/browse/SPARK-24916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555435#comment-16555435
]
Apache Spark commented on SPARK-24916:
--
User 'wangyum' has created a pull request f
[
https://issues.apache.org/jira/browse/SPARK-24916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24916:
Assignee: (was: Apache Spark)
> Fix type coercion for IN expression with subquery
> -
[
https://issues.apache.org/jira/browse/SPARK-24916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Apache Spark reassigned SPARK-24916:
Assignee: Apache Spark
> Fix type coercion for IN expression with subquery
>
Yuming Wang created SPARK-24916:
---
Summary: Fix type coercion for IN expression with subquery
Key: SPARK-24916
URL: https://issues.apache.org/jira/browse/SPARK-24916
Project: Spark
Issue Type: B
[
https://issues.apache.org/jira/browse/SPARK-21063?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555403#comment-16555403
]
nick commented on SPARK-21063:
--
[~paulstaab]
It does work when both registering the dialec
[
https://issues.apache.org/jira/browse/SPARK-24904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16555477#comment-16555477
]
Marco Gaido commented on SPARK-24904:
-
You cannot do a broadcast join when it is on
[
https://issues.apache.org/jira/browse/SPARK-19018?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel
]
Hyukjin Kwon reassigned SPARK-19018:
Assignee: Carlos Peña
> spark csv writer charset support
> --
Stephen Spencer created SPARK-24915:
---
Summary: Calling SparkSession.createDataFrame with schema can
throw exception
Key: SPARK-24915
URL: https://issues.apache.org/jira/browse/SPARK-24915
Project: S
87 matches
Mail list logo
