[jira] [Commented] (SPARK-30108) Add robust accumulator for observable metrics

[ https://issues.apache.org/jira/browse/SPARK-30108?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001619#comment-17001619 ] Ankit Raj Boudh commented on SPARK-30108: - [~hvanhovell], Thank you, during deve

[jira] [Commented] (SPARK-11516) Spark application cannot be found from JSON API even though it exists

[ https://issues.apache.org/jira/browse/SPARK-11516?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001618#comment-17001618 ] sdhalex commented on SPARK-11516: - Excuse me, has this bug been fixed in the newest ver

[jira] [Commented] (SPARK-22711) _pickle.PicklingError: args[0] from __newobj__ args has the wrong class from cloudpickle.py

[ https://issues.apache.org/jira/browse/SPARK-22711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001603#comment-17001603 ] sdhalex commented on SPARK-22711: - Excuse me, has this bug been fixed in the newest ve

[jira] [Commented] (SPARK-19335) Spark should support doing an efficient DataFrame Upsert via JDBC

[ https://issues.apache.org/jira/browse/SPARK-19335?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17001220#comment-17001220 ] Cory Lassila commented on SPARK-19335: -- +1 I believe this would be useful, my scena

[jira] [Resolved] (SPARK-26418) Only OpenBlocks without any ChunkFetch for one stream will cause memory leak in ExternalShuffleService

[ https://issues.apache.org/jira/browse/SPARK-26418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-26418. Resolution: Duplicate > Only OpenBlocks without any ChunkFetch for one str

[jira] [Created] (SPARK-30324) Simplify API for JSON access in DataFrames/SQL

Burak Yavuz created SPARK-30324: --- Summary: Simplify API for JSON access in DataFrames/SQL Key: SPARK-30324 URL: https://issues.apache.org/jira/browse/SPARK-30324 Project: Spark Issue Type: New

[jira] [Created] (SPARK-30323) Support filters pushdown in CSV datasource

Maxim Gekk created SPARK-30323: -- Summary: Support filters pushdown in CSV datasource Key: SPARK-30323 URL: https://issues.apache.org/jira/browse/SPARK-30323 Project: Spark Issue Type: Improvemen

[jira] [Resolved] (SPARK-17398) Failed to query on external JSon Partitioned table

[ https://issues.apache.org/jira/browse/SPARK-17398?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Masiero Vanzin resolved SPARK-17398. Fix Version/s: (was: 2.0.1) 3.0.0

[jira] [Created] (SPARK-30322) Add stage level scheduling docs

Thomas Graves created SPARK-30322: - Summary: Add stage level scheduling docs Key: SPARK-30322 URL: https://issues.apache.org/jira/browse/SPARK-30322 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-30321) log weightSum in Algo that has weights support

Huaxin Gao created SPARK-30321: -- Summary: log weightSum in Algo that has weights support Key: SPARK-30321 URL: https://issues.apache.org/jira/browse/SPARK-30321 Project: Spark Issue Type: Improv

[jira] [Created] (SPARK-30320) Insert overwrite to DataSource table with dynamic partition error when running multiple task attempts

Du Ripeng created SPARK-30320: - Summary: Insert overwrite to DataSource table with dynamic partition error when running multiple task attempts Key: SPARK-30320 URL: https://issues.apache.org/jira/browse/SPARK-30320

[jira] [Created] (SPARK-30319) Adds a stricter version of as[T]

Enrico Minack created SPARK-30319: - Summary: Adds a stricter version of as[T] Key: SPARK-30319 URL: https://issues.apache.org/jira/browse/SPARK-30319 Project: Spark Issue Type: New Feature

[jira] [Resolved] (SPARK-30272) Remove usage of Guava that breaks in Guava 27

[ https://issues.apache.org/jira/browse/SPARK-30272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30272. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26911 [https://gi

[jira] [Assigned] (SPARK-29938) Add batching in alter table add partition flow

[ https://issues.apache.org/jira/browse/SPARK-29938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen reassigned SPARK-29938: Assignee: Prakhar Jain > Add batching in alter table add partition flow > ---

[jira] [Resolved] (SPARK-29938) Add batching in alter table add partition flow

[ https://issues.apache.org/jira/browse/SPARK-29938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-29938. -- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26569 [https://gi

[jira] [Resolved] (SPARK-30317) Spark streaming programming document updation

[ https://issues.apache.org/jira/browse/SPARK-30317?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30317. -- Resolution: Not A Problem > Spark streaming programming document updation > --

[jira] [Created] (SPARK-30318) Bump jetty to 9.3.27.v20190418

Sandeep Katta created SPARK-30318: - Summary: Bump jetty to 9.3.27.v20190418 Key: SPARK-30318 URL: https://issues.apache.org/jira/browse/SPARK-30318 Project: Spark Issue Type: Bug Co

[jira] [Resolved] (SPARK-30300) Update correct string in UI for metrics when driver updates same metrics id as tasks.

[ https://issues.apache.org/jira/browse/SPARK-30300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves resolved SPARK-30300. --- Fix Version/s: 3.0.0 Assignee: Niranjan Artal Resolution: Fixed > Update cor

[jira] [Assigned] (SPARK-29768) nondeterministic expression fails column pruning

[ https://issues.apache.org/jira/browse/SPARK-29768?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29768: --- Assignee: wuyi (was: Wenchen Fan) > nondeterministic expression fails column pruning > ---

[jira] [Resolved] (SPARK-30308) Update Netty and Netty-all to address CVE-2019-16869

[ https://issues.apache.org/jira/browse/SPARK-30308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean R. Owen resolved SPARK-30308. -- Fix Version/s: (was: 2.4.4) Resolution: Not A Problem > Update Netty and Netty-all

[jira] [Created] (SPARK-30317) Spark streaming programming document updation

jobit mathew created SPARK-30317: Summary: Spark streaming programming document updation Key: SPARK-30317 URL: https://issues.apache.org/jira/browse/SPARK-30317 Project: Spark Issue Type: Imp

[jira] [Resolved] (SPARK-30301) Datetimes as fields of complex types to hive string results wrong

[ https://issues.apache.org/jira/browse/SPARK-30301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-30301. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26942 [https://gith

[jira] [Assigned] (SPARK-30301) Datetimes as fields of complex types to hive string results wrong

[ https://issues.apache.org/jira/browse/SPARK-30301?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-30301: --- Assignee: Kent Yao > Datetimes as fields of complex types to hive string results wrong > --

[jira] [Commented] (SPARK-27021) Leaking Netty event loop group for shuffle chunk fetch requests

[ https://issues.apache.org/jira/browse/SPARK-27021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17000782#comment-17000782 ] Sandeep Katta commented on SPARK-27021: --- [~hyukjin.kwon] [~dongjoon] this issue is

[jira] [Comment Edited] (SPARK-25185) CBO rowcount statistics doesn't work for partitioned parquet external table

[ https://issues.apache.org/jira/browse/SPARK-25185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17000775#comment-17000775 ] Zhenhua Wang edited comment on SPARK-25185 at 12/20/19 9:58 AM: --

[jira] [Commented] (SPARK-25185) CBO rowcount statistics doesn't work for partitioned parquet external table

[ https://issues.apache.org/jira/browse/SPARK-25185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17000776#comment-17000776 ] Zhenhua Wang commented on SPARK-25185: -- [~lishuming] yes, you can analyze on extern

[jira] [Commented] (SPARK-25185) CBO rowcount statistics doesn't work for partitioned parquet external table

[ https://issues.apache.org/jira/browse/SPARK-25185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17000775#comment-17000775 ] Zhenhua Wang commented on SPARK-25185: -- Hi, [~imamitsehgal] and [~raoyvn], could yo

[jira] [Updated] (SPARK-30269) Should use old partition stats to decide whether to update stats when analyzing partition

[ https://issues.apache.org/jira/browse/SPARK-30269?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhenhua Wang updated SPARK-30269: - Parent: SPARK-16026 Issue Type: Sub-task (was: Bug) > Should use old partition stats to

[jira] [Commented] (SPARK-30246) Spark on Yarn External Shuffle Service Memory Leak

[ https://issues.apache.org/jira/browse/SPARK-30246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17000763#comment-17000763 ] Jan Filipiak commented on SPARK-30246: -- Hi, Feel free to send the PR along later i

[jira] [Comment Edited] (SPARK-30246) Spark on Yarn External Shuffle Service Memory Leak

[ https://issues.apache.org/jira/browse/SPARK-30246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17000763#comment-17000763 ] Jan Filipiak edited comment on SPARK-30246 at 12/20/19 9:39 AM: --

[jira] [Issue Comment Deleted] (SPARK-30308) Update Netty and Netty-all to address CVE-2019-16869

[ https://issues.apache.org/jira/browse/SPARK-30308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Raj Boudh updated SPARK-30308: Comment: was deleted (was: [~vishwaskumar], 4.1.43.Final version we need to update ?) > U

[jira] [Comment Edited] (SPARK-30308) Update Netty and Netty-all to address CVE-2019-16869

[ https://issues.apache.org/jira/browse/SPARK-30308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17000760#comment-17000760 ] Ankit Raj Boudh edited comment on SPARK-30308 at 12/20/19 9:35 AM: ---

[jira] [Commented] (SPARK-30308) Update Netty and Netty-all to address CVE-2019-16869

[ https://issues.apache.org/jira/browse/SPARK-30308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17000760#comment-17000760 ] Ankit Raj Boudh commented on SPARK-30308: - @srowen , please help me to close thi

[jira] [Issue Comment Deleted] (SPARK-30308) Update Netty and Netty-all to address CVE-2019-16869

[ https://issues.apache.org/jira/browse/SPARK-30308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Raj Boudh updated SPARK-30308: Comment: was deleted (was: need to mention as not a problem) > Update Netty and Netty-all

[jira] [Reopened] (SPARK-30308) Update Netty and Netty-all to address CVE-2019-16869

[ https://issues.apache.org/jira/browse/SPARK-30308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Raj Boudh reopened SPARK-30308: - need to mention as not a problem > Update Netty and Netty-all to address CVE-2019-16869 > -

[jira] [Resolved] (SPARK-30308) Update Netty and Netty-all to address CVE-2019-16869

[ https://issues.apache.org/jira/browse/SPARK-30308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ankit Raj Boudh resolved SPARK-30308. - Fix Version/s: 2.4.4 Resolution: Resolved > Update Netty and Netty-all to address

[jira] [Commented] (SPARK-30308) Update Netty and Netty-all to address CVE-2019-16869

[ https://issues.apache.org/jira/browse/SPARK-30308?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=17000756#comment-17000756 ] Ankit Raj Boudh commented on SPARK-30308: - It's already updated to 4.1.42.Final

[jira] [Created] (SPARK-30316) data size boom after shuffle writing dataframe save as parquet

Cesc created SPARK-30316: - Summary: data size boom after shuffle writing dataframe save as parquet Key: SPARK-30316 URL: https://issues.apache.org/jira/browse/SPARK-30316 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-30291) Catch the exception when do materialize in AQE

[ https://issues.apache.org/jira/browse/SPARK-30291?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-30291. - Fix Version/s: 3.0.0 Assignee: Ke Jia Resolution: Fixed > Catch the exception when do ma