[jira] [Resolved] (SPARK-29546) Recover jersey-guava test dependency in docker-integration-tests

2019-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29546?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29546. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26203 [https://

[jira] [Resolved] (SPARK-29324) saveAsTable with overwrite mode results in metadata loss

2019-10-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29324?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29324. -- Resolution: Not A Problem > saveAsTable with overwrite mode results in metadata loss > ---

[jira] [Commented] (SPARK-24666) Word2Vec generate infinity vectors when numIterations are large

2019-10-23 Thread carlos yan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-24666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957623#comment-16957623 ] carlos yan commented on SPARK-24666: I also get this question, and my spark version

[jira] [Updated] (SPARK-21492) Memory leak in SortMergeJoin

2019-10-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21492?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-21492: Fix Version/s: 2.4.5 > Memory leak in SortMergeJoin > > >

[jira] [Updated] (SPARK-29564) Cluster deploy mode should support Spark Thrift server

2019-10-23 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-29564: --- Summary: Cluster deploy mode should support Spark Thrift server (was: Cluster deploy mode should su

[jira] [Created] (SPARK-29564) Cluster deploy mode should support to launch Spark Thrift server

2019-10-23 Thread Lantao Jin (Jira)
Lantao Jin created SPARK-29564: -- Summary: Cluster deploy mode should support to launch Spark Thrift server Key: SPARK-29564 URL: https://issues.apache.org/jira/browse/SPARK-29564 Project: Spark

[jira] [Updated] (SPARK-29564) Cluster deploy mode should support Spark Thrift server

2019-10-23 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-29564: --- Description: Cluster deploy mode is not applicable to Spark Thrift server from SPARK-21403. This re

[jira] [Created] (SPARK-29565) OneHotEncoder should support single-column input/ouput

2019-10-23 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-29565: Summary: OneHotEncoder should support single-column input/ouput Key: SPARK-29565 URL: https://issues.apache.org/jira/browse/SPARK-29565 Project: Spark Issue

[jira] [Created] (SPARK-29566) Imputer should support single-column input/ouput

2019-10-23 Thread zhengruifeng (Jira)
zhengruifeng created SPARK-29566: Summary: Imputer should support single-column input/ouput Key: SPARK-29566 URL: https://issues.apache.org/jira/browse/SPARK-29566 Project: Spark Issue Type:

[jira] [Updated] (SPARK-29566) Imputer should support single-column input/ouput

2019-10-23 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29566?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-29566: - Description: Imputer should support single-column input/ouput refer to https://issues.apache.or

[jira] [Created] (SPARK-29567) Upgrade JDBC Integration Test Docker Images

2019-10-23 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-29567: - Summary: Upgrade JDBC Integration Test Docker Images Key: SPARK-29567 URL: https://issues.apache.org/jira/browse/SPARK-29567 Project: Spark Issue Type: Imp

[jira] [Commented] (SPARK-29565) OneHotEncoder should support single-column input/ouput

2019-10-23 Thread zhengruifeng (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957664#comment-16957664 ] zhengruifeng commented on SPARK-29565: -- [~huaxingao]  In  [https://github.com/apach

[jira] [Updated] (SPARK-29567) Update JDBC Integration Test Docker Images

2019-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29567: -- Summary: Update JDBC Integration Test Docker Images (was: Upgrade JDBC Integration Test Docke

[jira] [Updated] (SPARK-29564) Cluster deploy mode should support Spark Thrift server

2019-10-23 Thread Lantao Jin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Lantao Jin updated SPARK-29564: --- Description: Cluster deploy mode is not applicable to Spark Thrift server now. This restriction is

[jira] [Resolved] (SPARK-29352) Move active streaming query state to the SharedState

2019-10-23 Thread Burak Yavuz (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29352?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Burak Yavuz resolved SPARK-29352. - Fix Version/s: 3.0.0 Resolution: Fixed Resolved by [https://github.com/apache/spark/pull/

[jira] [Commented] (SPARK-15348) Hive ACID

2019-10-23 Thread Zhaoyang Qin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957679#comment-16957679 ] Zhaoyang Qin commented on SPARK-15348: -- [~asomani] when i use the following codes:`

[jira] [Comment Edited] (SPARK-15348) Hive ACID

2019-10-23 Thread Zhaoyang Qin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957679#comment-16957679 ] Zhaoyang Qin edited comment on SPARK-15348 at 10/23/19 9:15 AM: --

[jira] [Comment Edited] (SPARK-15348) Hive ACID

2019-10-23 Thread Zhaoyang Qin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957679#comment-16957679 ] Zhaoyang Qin edited comment on SPARK-15348 at 10/23/19 9:15 AM: --

[jira] [Created] (SPARK-29568) Add flag to stop existing stream when new copy starts

2019-10-23 Thread Burak Yavuz (Jira)
Burak Yavuz created SPARK-29568: --- Summary: Add flag to stop existing stream when new copy starts Key: SPARK-29568 URL: https://issues.apache.org/jira/browse/SPARK-29568 Project: Spark Issue Typ

[jira] [Updated] (SPARK-29569) doc build fails with `/api/scala/lib/jquery.js` doesn't exist

2019-10-23 Thread Xingbo Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang updated SPARK-29569: - Summary: doc build fails with `/api/scala/lib/jquery.js` doesn't exist (was: doc build fails be

[jira] [Created] (SPARK-29569) doc build fails because `/api/scala/lib/jquery.js` doesn't exist

2019-10-23 Thread Xingbo Jiang (Jira)
Xingbo Jiang created SPARK-29569: Summary: doc build fails because `/api/scala/lib/jquery.js` doesn't exist Key: SPARK-29569 URL: https://issues.apache.org/jira/browse/SPARK-29569 Project: Spark

[jira] [Commented] (SPARK-29569) doc build fails with `/api/scala/lib/jquery.js` doesn't exist

2019-10-23 Thread Xingbo Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957731#comment-16957731 ] Xingbo Jiang commented on SPARK-29569: -- [~sowen][~dongjoon] Can you take a look at

[jira] [Updated] (SPARK-29569) doc build fails with `/api/scala/lib/jquery.js` doesn't exist

2019-10-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29569: - Attachment: Screen Shot 2019-10-23 at 8.25.01 PM.png > doc build fails with `/api/scala/lib/jque

[jira] [Commented] (SPARK-29569) doc build fails with `/api/scala/lib/jquery.js` doesn't exist

2019-10-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957772#comment-16957772 ] Hyukjin Kwon commented on SPARK-29569: -- I attached the ScalaDoc output from the cur

[jira] [Assigned] (SPARK-29542) [SQL][DOC] The descriptions of `spark.sql.files.*` are confused.

2019-10-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-29542: Assignee: feiwang > [SQL][DOC] The descriptions of `spark.sql.files.*` are confused. > --

[jira] [Resolved] (SPARK-29542) [SQL][DOC] The descriptions of `spark.sql.files.*` are confused.

2019-10-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29542?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-29542. -- Fix Version/s: 3.0.0 Resolution: Fixed Fixed in https://github.com/apache/spark/pull/26

[jira] [Updated] (SPARK-29569) doc build fails with `/api/scala/lib/jquery.js` doesn't exist

2019-10-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29569?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-29569: - Component/s: docs > doc build fails with `/api/scala/lib/jquery.js` doesn't exist >

[jira] [Commented] (SPARK-29569) doc build fails with `/api/scala/lib/jquery.js` doesn't exist

2019-10-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957785#comment-16957785 ] Hyukjin Kwon commented on SPARK-29569: -- This seems to start to happen after Scala 2

[jira] [Resolved] (SPARK-29499) Add mapPartitionsWithIndex for RDDBarrier

2019-10-23 Thread Xingbo Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xingbo Jiang resolved SPARK-29499. -- Assignee: Xianyang Liu Resolution: Fixed > Add mapPartitionsWithIndex for RDDBarrier > -

[jira] [Created] (SPARK-29570) Improve tooltip for Executor Tab for Shuffle Write,Blacklisted,Logs,Threaddump columns

2019-10-23 Thread ABHISHEK KUMAR GUPTA (Jira)
ABHISHEK KUMAR GUPTA created SPARK-29570: Summary: Improve tooltip for Executor Tab for Shuffle Write,Blacklisted,Logs,Threaddump columns Key: SPARK-29570 URL: https://issues.apache.org/jira/browse/SPARK-2

[jira] [Commented] (SPARK-29570) Improve tooltip for Executor Tab for Shuffle Write,Blacklisted,Logs,Threaddump columns

2019-10-23 Thread Ankit Raj Boudh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29570?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957897#comment-16957897 ] Ankit Raj Boudh commented on SPARK-29570: - I will fix this issue > Improve tool

[jira] [Commented] (SPARK-29571) Fix UT in AllExecutionsPageSuite class

2019-10-23 Thread Ankit Raj Boudh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957898#comment-16957898 ] Ankit Raj Boudh commented on SPARK-29571: - i will raise the PR soon > Fix UT in

[jira] [Created] (SPARK-29571) Fix UT in AllExecutionsPageSuite class

2019-10-23 Thread Ankit Raj Boudh (Jira)
Ankit Raj Boudh created SPARK-29571: --- Summary: Fix UT in AllExecutionsPageSuite class Key: SPARK-29571 URL: https://issues.apache.org/jira/browse/SPARK-29571 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-15348) Hive ACID

2019-10-23 Thread Abhishek Somani (Jira)
[ https://issues.apache.org/jira/browse/SPARK-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957904#comment-16957904 ] Abhishek Somani commented on SPARK-15348: - [~Kelvin.FE] This seems to be happeni

[jira] [Created] (SPARK-29572) add v1 read fallback API in DS v2

2019-10-23 Thread Wenchen Fan (Jira)
Wenchen Fan created SPARK-29572: --- Summary: add v1 read fallback API in DS v2 Key: SPARK-29572 URL: https://issues.apache.org/jira/browse/SPARK-29572 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-29573) Spark should work as PostgreSQL when using + Operator

2019-10-23 Thread ABHISHEK KUMAR GUPTA (Jira)
ABHISHEK KUMAR GUPTA created SPARK-29573: Summary: Spark should work as PostgreSQL when using + Operator Key: SPARK-29573 URL: https://issues.apache.org/jira/browse/SPARK-29573 Project: Spark

[jira] [Commented] (SPARK-21287) Cannot use Int.MIN_VALUE as Spark SQL fetchsize

2019-10-23 Thread Hu Fuwang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-21287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957939#comment-16957939 ] Hu Fuwang commented on SPARK-21287: --- [~smilegator]  [~srowen] Just submitted a PR for

[jira] [Updated] (SPARK-29557) Upgrade dropwizard metrics library to 3.2.6

2019-10-23 Thread Luca Canali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Luca Canali updated SPARK-29557: Summary: Upgrade dropwizard metrics library to 3.2.6 (was: Upgrade dropwizard metrics library to

[jira] [Commented] (SPARK-29557) Upgrade dropwizard metrics library to 3.2.6

2019-10-23 Thread Luca Canali (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29557?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957945#comment-16957945 ] Luca Canali commented on SPARK-29557: - Upgrading Apache Spark to use dropwizard/coda

[jira] [Resolved] (SPARK-29513) REFRESH TABLE should look up catalog/table like v2 commands

2019-10-23 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh resolved SPARK-29513. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26183 [https://gith

[jira] [Assigned] (SPARK-29513) REFRESH TABLE should look up catalog/table like v2 commands

2019-10-23 Thread L. C. Hsieh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] L. C. Hsieh reassigned SPARK-29513: --- Assignee: Terry Kim > REFRESH TABLE should look up catalog/table like v2 commands > ---

[jira] [Created] (SPARK-29574) spark with user provided hadoop doesn't work on kubernetes

2019-10-23 Thread Jira
Michał Wesołowski created SPARK-29574: - Summary: spark with user provided hadoop doesn't work on kubernetes Key: SPARK-29574 URL: https://issues.apache.org/jira/browse/SPARK-29574 Project: Spark

[jira] [Commented] (SPARK-29415) Stage Level Sched: Add base ResourceProfile and Request classes

2019-10-23 Thread Thomas Graves (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16957974#comment-16957974 ] Thomas Graves commented on SPARK-29415: --- >From a high level design point, this is

[jira] [Commented] (SPARK-29106) Add jenkins arm test for spark

2019-10-23 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958019#comment-16958019 ] Shane Knapp commented on SPARK-29106: - [~huangtianhua]: > we don't have to download

[jira] [Created] (SPARK-29575) from_json can produce nulls for fields which are marked as non-nullable

2019-10-23 Thread Victor Lopez (Jira)
Victor Lopez created SPARK-29575: Summary: from_json can produce nulls for fields which are marked as non-nullable Key: SPARK-29575 URL: https://issues.apache.org/jira/browse/SPARK-29575 Project: Spar

[jira] [Resolved] (SPARK-29503) MapObjects doesn't copy Unsafe data when nested under Safe data

2019-10-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29503. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26173 [https://gith

[jira] [Assigned] (SPARK-29503) MapObjects doesn't copy Unsafe data when nested under Safe data

2019-10-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29503?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29503: --- Assignee: Jungtaek Lim > MapObjects doesn't copy Unsafe data when nested under Safe data >

[jira] [Resolved] (SPARK-29552) Fix the flaky test failed in AdaptiveQueryExecSuite # multiple joins

2019-10-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-29552. - Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26207 [https://gith

[jira] [Assigned] (SPARK-29552) Fix the flaky test failed in AdaptiveQueryExecSuite # multiple joins

2019-10-23 Thread Wenchen Fan (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29552?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-29552: --- Assignee: Ke Jia > Fix the flaky test failed in AdaptiveQueryExecSuite # multiple joins > -

[jira] [Commented] (SPARK-29106) Add jenkins arm test for spark

2019-10-23 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958056#comment-16958056 ] Shane Knapp commented on SPARK-29106: - > For pyspark test, you mentioned we didn't i

[jira] [Updated] (SPARK-29106) Add jenkins arm test for spark

2019-10-23 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shane Knapp updated SPARK-29106: Attachment: R-libs.txt R-ansible.yml > Add jenkins arm test for spark > --

[jira] [Updated] (SPARK-29575) from_json can produce nulls for fields which are marked as non-nullable

2019-10-23 Thread Victor Lopez (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victor Lopez updated SPARK-29575: - Description: I believe this issue was resolved elsewhere (https://issues.apache.org/jira/browse

[jira] [Updated] (SPARK-29575) from_json can produce nulls for fields which are marked as non-nullable

2019-10-23 Thread Victor Lopez (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victor Lopez updated SPARK-29575: - Description: I believe this issue was resolved elsewhere (https://issues.apache.org/jira/browse

[jira] [Updated] (SPARK-29575) from_json can produce nulls for fields which are marked as non-nullable

2019-10-23 Thread Victor Lopez (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victor Lopez updated SPARK-29575: - Description: I believe this issue was resolved elsewhere (https://issues.apache.org/jira/browse

[jira] [Comment Edited] (SPARK-29106) Add jenkins arm test for spark

2019-10-23 Thread Shane Knapp (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29106?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958056#comment-16958056 ] Shane Knapp edited comment on SPARK-29106 at 10/23/19 5:29 PM: ---

[jira] [Updated] (SPARK-29575) from_json can produce nulls for fields which are marked as non-nullable

2019-10-23 Thread Victor Lopez (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29575?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Victor Lopez updated SPARK-29575: - Description: I believe this issue was resolved elsewhere (https://issues.apache.org/jira/browse

[jira] [Commented] (SPARK-29569) doc build fails with `/api/scala/lib/jquery.js` doesn't exist

2019-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958065#comment-16958065 ] Dongjoon Hyun commented on SPARK-29569: --- Sorry for being late to the party! Thank

[jira] [Assigned] (SPARK-29557) Upgrade dropwizard metrics library to 3.2.6

2019-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29557: - Assignee: Luca Canali > Upgrade dropwizard metrics library to 3.2.6 > -

[jira] [Resolved] (SPARK-29557) Upgrade dropwizard metrics library to 3.2.6

2019-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29557. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26212 [https://

[jira] [Updated] (SPARK-29557) Upgrade dropwizard metrics library to 3.2.6

2019-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29557?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29557: -- Description: This proposes to upgrade the dropwizard/codahale metrics library version used by

[jira] [Commented] (SPARK-29569) doc build fails with `/api/scala/lib/jquery.js` doesn't exist

2019-10-23 Thread Xingbo Jiang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958166#comment-16958166 ] Xingbo Jiang commented on SPARK-29569: -- [~dongjoon] Not yet, the release script is

[jira] [Commented] (SPARK-29571) Fix UT in AllExecutionsPageSuite class

2019-10-23 Thread shahid (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958171#comment-16958171 ] shahid commented on SPARK-29571: Could you clarify which UT is failing? > Fix UT in Al

[jira] [Commented] (SPARK-15348) Hive ACID

2019-10-23 Thread Georg Heiler (Jira)
[ https://issues.apache.org/jira/browse/SPARK-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958225#comment-16958225 ] Georg Heiler commented on SPARK-15348: -- Another workaround could be external tables

[jira] [Resolved] (SPARK-29538) Test failure: org.apache.spark.sql.execution.adaptive.AdaptiveQueryExecSuite.multiple joins

2019-10-23 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29538?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim resolved SPARK-29538. -- Resolution: Duplicate SPARK-29552 dealt with this. Will reopen if it is still flaky. > Test f

[jira] [Commented] (SPARK-29571) Fix UT in AllExecutionsPageSuite class

2019-10-23 Thread Ankit raj boudh (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958237#comment-16958237 ] Ankit raj boudh commented on SPARK-29571: - Assert condition is wrong in AllExecu

[jira] [Created] (SPARK-29576) Use Spark's CompressionCodec for Ser/Deser of MapOutputStatus

2019-10-23 Thread DB Tsai (Jira)
DB Tsai created SPARK-29576: --- Summary: Use Spark's CompressionCodec for Ser/Deser of MapOutputStatus Key: SPARK-29576 URL: https://issues.apache.org/jira/browse/SPARK-29576 Project: Spark Issue Ty

[jira] [Created] (SPARK-29577) Implement p-value simulation and unit tests for chi2 test

2019-10-23 Thread Alexander Tronchin-James (Jira)
Alexander Tronchin-James created SPARK-29577: Summary: Implement p-value simulation and unit tests for chi2 test Key: SPARK-29577 URL: https://issues.apache.org/jira/browse/SPARK-29577 Proj

[jira] [Created] (SPARK-29578) JDK 1.8.0_232 timezone updates cause "Kwajalein" test failures again

2019-10-23 Thread Sean R. Owen (Jira)
Sean R. Owen created SPARK-29578: Summary: JDK 1.8.0_232 timezone updates cause "Kwajalein" test failures again Key: SPARK-29578 URL: https://issues.apache.org/jira/browse/SPARK-29578 Project: Spark

[jira] [Updated] (SPARK-28870) Snapshot event log files to support incremental reading

2019-10-23 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-28870: - Summary: Snapshot event log files to support incremental reading (was: Snapshot old event log f

[jira] [Commented] (SPARK-28870) Snapshot old event log files to support compaction

2019-10-23 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958300#comment-16958300 ] Jungtaek Lim commented on SPARK-28870: -- Discussed with Marcelo/Imran offline: I'm c

[jira] [Updated] (SPARK-28870) Snapshot event log files to support incremental reading

2019-10-23 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-28870: - Description: This issue tracks the effort on compacting event log files into snapshot and enabl

[jira] [Updated] (SPARK-28870) Snapshot event log files to support incremental reading

2019-10-23 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-28870: - Description: This issue tracks the effort on compacting event log files into snapshot and enabl

[jira] [Updated] (SPARK-28870) Snapshot event log files to support incremental reading

2019-10-23 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-28870: - Description: This issue tracks the effort on compacting event log files into snapshot and enabl

[jira] [Updated] (SPARK-29111) Support snapshot/restore of KVStore

2019-10-23 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29111?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-29111: - Description: This issue tracks the effort of supporting snapshot/restore from/to KVStore. Note

[jira] [Updated] (SPARK-29261) Support recover live entities from KVStore for (SQL)AppStatusListener

2019-10-23 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29261?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Jungtaek Lim updated SPARK-29261: - Description: To achieve incremental reply goal in SHS, we need to support recover live entities

[jira] [Created] (SPARK-29579) Guarantee compatibility of snapshot (live entities, KVstore entities)

2019-10-23 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-29579: Summary: Guarantee compatibility of snapshot (live entities, KVstore entities) Key: SPARK-29579 URL: https://issues.apache.org/jira/browse/SPARK-29579 Project: Spark

[jira] [Created] (SPARK-29580) KafkaDelegationTokenSuite fails to create new KafkaAdminClient

2019-10-23 Thread Dongjoon Hyun (Jira)
Dongjoon Hyun created SPARK-29580: - Summary: KafkaDelegationTokenSuite fails to create new KafkaAdminClient Key: SPARK-29580 URL: https://issues.apache.org/jira/browse/SPARK-29580 Project: Spark

[jira] [Commented] (SPARK-29580) KafkaDelegationTokenSuite fails to create new KafkaAdminClient

2019-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958372#comment-16958372 ] Dongjoon Hyun commented on SPARK-29580: --- This is a different failure from SPARK-29

[jira] [Updated] (SPARK-29580) KafkaDelegationTokenSuite fails to create new KafkaAdminClient

2019-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-29580: -- Issue Type: Bug (was: Improvement) > KafkaDelegationTokenSuite fails to create new KafkaAdmin

[jira] [Commented] (SPARK-29580) KafkaDelegationTokenSuite fails to create new KafkaAdminClient

2019-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958374#comment-16958374 ] Dongjoon Hyun commented on SPARK-29580: --- Hi, [~gsomogyi]. Could you take a look at

[jira] [Created] (SPARK-29581) Enable cleanup old event log files

2019-10-23 Thread Jungtaek Lim (Jira)
Jungtaek Lim created SPARK-29581: Summary: Enable cleanup old event log files Key: SPARK-29581 URL: https://issues.apache.org/jira/browse/SPARK-29581 Project: Spark Issue Type: Sub-task

[jira] [Resolved] (SPARK-29567) Update JDBC Integration Test Docker Images

2019-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29567. --- Fix Version/s: 3.0.0 Resolution: Fixed Issue resolved by pull request 26224 [https://

[jira] [Assigned] (SPARK-29567) Update JDBC Integration Test Docker Images

2019-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29567?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29567: - Assignee: Dongjoon Hyun > Update JDBC Integration Test Docker Images >

[jira] [Commented] (SPARK-29569) doc build fails with `/api/scala/lib/jquery.js` doesn't exist

2019-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29569?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958410#comment-16958410 ] Dongjoon Hyun commented on SPARK-29569: --- Oh.. Too bad. Got it. Thank you for updat

[jira] [Commented] (SPARK-28594) Allow event logs for running streaming apps to be rolled over.

2019-10-23 Thread Jungtaek Lim (Jira)
[ https://issues.apache.org/jira/browse/SPARK-28594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958412#comment-16958412 ] Jungtaek Lim commented on SPARK-28594: -- Please note that SPARK-29579 and SPARK-2958

[jira] [Assigned] (SPARK-29576) Use Spark's CompressionCodec for Ser/Deser of MapOutputStatus

2019-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun reassigned SPARK-29576: - Assignee: DB Tsai > Use Spark's CompressionCodec for Ser/Deser of MapOutputStatus > ---

[jira] [Resolved] (SPARK-29576) Use Spark's CompressionCodec for Ser/Deser of MapOutputStatus

2019-10-23 Thread Dongjoon Hyun (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29576?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-29576. --- Resolution: Fixed Issue resolved by pull request 26235 [https://github.com/apache/spark/pull

[jira] [Commented] (SPARK-15348) Hive ACID

2019-10-23 Thread Zhaoyang Qin (Jira)
[ https://issues.apache.org/jira/browse/SPARK-15348?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958471#comment-16958471 ] Zhaoyang Qin commented on SPARK-15348: -- [~asomani]  [~georg.kf.hei...@gmail.com] Th

[jira] [Created] (SPARK-29582) Unify the behavior of pyspark.TaskContext with spark core

2019-10-23 Thread Xianyang Liu (Jira)
Xianyang Liu created SPARK-29582: Summary: Unify the behavior of pyspark.TaskContext with spark core Key: SPARK-29582 URL: https://issues.apache.org/jira/browse/SPARK-29582 Project: Spark Iss

[jira] [Created] (SPARK-29583) extract support interval type

2019-10-23 Thread Yuming Wang (Jira)
Yuming Wang created SPARK-29583: --- Summary: extract support interval type Key: SPARK-29583 URL: https://issues.apache.org/jira/browse/SPARK-29583 Project: Spark Issue Type: Sub-task Co

[jira] [Updated] (SPARK-29583) extract support interval type

2019-10-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29583?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-29583: Description: {code:sql} postgres=# select extract(minute from INTERVAL '1 YEAR 10 DAYS 50 MINUTES'

[jira] [Commented] (SPARK-29583) extract support interval type

2019-10-23 Thread Yuming Wang (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29583?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958479#comment-16958479 ] Yuming Wang commented on SPARK-29583: - cc [~maxgekk] > extract support interval typ

[jira] [Updated] (SPARK-16483) Unifying struct fields and columns

2019-10-23 Thread Hyukjin Kwon (Jira)
[ https://issues.apache.org/jira/browse/SPARK-16483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-16483: - Labels: sql (was: bulk-closed sql) > Unifying struct fields and columns > -

[jira] [Created] (SPARK-29584) NOT NULL is not supported in Spark

2019-10-23 Thread ABHISHEK KUMAR GUPTA (Jira)
ABHISHEK KUMAR GUPTA created SPARK-29584: Summary: NOT NULL is not supported in Spark Key: SPARK-29584 URL: https://issues.apache.org/jira/browse/SPARK-29584 Project: Spark Issue Type

[jira] [Commented] (SPARK-29565) OneHotEncoder should support single-column input/ouput

2019-10-23 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29565?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958537#comment-16958537 ] Huaxin Gao commented on SPARK-29565: I will work on this. Thanks for ping me [~podon

[jira] [Commented] (SPARK-29566) Imputer should support single-column input/ouput

2019-10-23 Thread Huaxin Gao (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29566?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958539#comment-16958539 ] Huaxin Gao commented on SPARK-29566: I will work on this. Thanks! [~podongfeng] > I

[jira] [Commented] (SPARK-29584) NOT NULL is not supported in Spark

2019-10-23 Thread pavithra ramachandran (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16958540#comment-16958540 ] pavithra ramachandran commented on SPARK-29584: --- i shall work on this > N

[jira] [Created] (SPARK-29585) Duration in stagePage does not match Duration in Summary Metrics for Completed Tasks

2019-10-23 Thread teeyog (Jira)
teeyog created SPARK-29585: -- Summary: Duration in stagePage does not match Duration in Summary Metrics for Completed Tasks Key: SPARK-29585 URL: https://issues.apache.org/jira/browse/SPARK-29585 Project: Spa

[jira] [Updated] (SPARK-29585) Duration in stagePage does not match Duration in Summary Metrics for Completed Tasks

2019-10-23 Thread teeyog (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] teeyog updated SPARK-29585: --- Description: Summary Metrics for Completed Tasks uses  executorRunTime, and Duration in Task details uses ex

[jira] [Updated] (SPARK-29585) Duration in stagePage does not match Duration in Summary Metrics for Completed Tasks

2019-10-23 Thread teeyog (Jira)
[ https://issues.apache.org/jira/browse/SPARK-29585?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] teeyog updated SPARK-29585: --- Description: Summary Metrics for Completed Tasks uses  executorRunTime, and Duration in Task details uses e

  1   2   >