[jira] [Comment Edited] (SPARK-23115) SparkR 2.3 QA: New R APIs and API docs

2018-01-23 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23115?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333677#comment-16333677 ] Felix Cheung edited comment on SPARK-23115 at 1/24/18 7:17 AM:

[jira] [Commented] (SPARK-20906) Constrained Logistic Regression for SparkR

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16337051#comment-16337051 ] Apache Spark commented on SPARK-20906: -- User 'felixcheung' has created a pull reques

[jira] [Commented] (SPARK-23177) PySpark parameter-less UDFs raise exception if applied after distinct

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16337046#comment-16337046 ] Apache Spark commented on SPARK-23177: -- User 'viirya' has created a pull request for

[jira] [Commented] (SPARK-17147) Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets (i.e. Log Compaction)

2018-01-23 Thread chao.wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16337027#comment-16337027 ] chao.wu commented on SPARK-17147: - While the  topic clean config is delete "cleanup.polic

[jira] [Resolved] (SPARK-21727) Operating on an ArrayType in a SparkR DataFrame throws error

2018-01-23 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21727?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-21727. -- Resolution: Fixed Fix Version/s: 2.4.0 2.3.0 Target Versi

[jira] [Commented] (SPARK-22809) pyspark is sensitive to imports with dots

2018-01-23 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336983#comment-16336983 ] holdenk commented on SPARK-22809: - oh wait it should work in 0.4.2, I'll poke at that PR

[jira] [Commented] (SPARK-22809) pyspark is sensitive to imports with dots

2018-01-23 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336979#comment-16336979 ] holdenk commented on SPARK-22809: - [~ueshin]: we can push this out to 2.3.1 given we are

[jira] [Updated] (SPARK-22809) pyspark is sensitive to imports with dots

2018-01-23 Thread holdenk (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22809?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk updated SPARK-22809: Target Version/s: 2.3.1, 2.4.0 (was: 2.3.0) > pyspark is sensitive to imports with dots >

[jira] [Comment Edited] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-01-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336957#comment-16336957 ] Kazuaki Ishizaki edited comment on SPARK-18016 at 1/24/18 5:49 AM:

[jira] [Commented] (SPARK-18016) Code Generation: Constant Pool Past Limit for Wide/Nested Dataset

2018-01-23 Thread Kazuaki Ishizaki (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18016?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336957#comment-16336957 ] Kazuaki Ishizaki commented on SPARK-18016: -- I cannot reproduce this issue using

[jira] [Commented] (SPARK-11222) Add style checker rules to validate doc tests aren't included in docs

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336934#comment-16336934 ] Apache Spark commented on SPARK-11222: -- User 'rekhajoshm' has created a pull request

[jira] [Commented] (SPARK-17088) IsolatedClientLoader fails to load Hive client when sharesHadoopClasses is false

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336913#comment-16336913 ] Apache Spark commented on SPARK-17088: -- User 'gatorsmile' has created a pull request

[jira] [Commented] (SPARK-22809) pyspark is sensitive to imports with dots

2018-01-23 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336881#comment-16336881 ] Takuya Ueshin commented on SPARK-22809: --- I agree with [~dongjoon]. [~CricketScienc

[jira] [Commented] (SPARK-23020) Re-enable Flaky Test: org.apache.spark.launcher.SparkLauncherSuite.testInProcessLauncher

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23020?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336873#comment-16336873 ] Apache Spark commented on SPARK-23020: -- User 'ueshin' has created a pull request for

[jira] [Assigned] (SPARK-23199) improved Removes repetition from group expressions in Aggregate

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23199: Assignee: Apache Spark > improved Removes repetition from group expressions in Aggregate >

[jira] [Commented] (SPARK-23199) improved Removes repetition from group expressions in Aggregate

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23199?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336833#comment-16336833 ] Apache Spark commented on SPARK-23199: -- User 'heary-cao' has created a pull request

[jira] [Assigned] (SPARK-23199) improved Removes repetition from group expressions in Aggregate

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23199?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23199: Assignee: (was: Apache Spark) > improved Removes repetition from group expressions in

[jira] [Created] (SPARK-23199) improved Removes repetition from group expressions in Aggregate

2018-01-23 Thread caoxuewen (JIRA)
caoxuewen created SPARK-23199: - Summary: improved Removes repetition from group expressions in Aggregate Key: SPARK-23199 URL: https://issues.apache.org/jira/browse/SPARK-23199 Project: Spark Is

[jira] [Assigned] (SPARK-23177) PySpark parameter-less UDFs raise exception if applied after distinct

2018-01-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon reassigned SPARK-23177: Assignee: Liang-Chi Hsieh > PySpark parameter-less UDFs raise exception if applied after d

[jira] [Updated] (SPARK-23177) PySpark parameter-less UDFs raise exception if applied after distinct

2018-01-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-23177: - Fix Version/s: 2.4.0 > PySpark parameter-less UDFs raise exception if applied after distinct > --

[jira] [Resolved] (SPARK-23177) PySpark parameter-less UDFs raise exception if applied after distinct

2018-01-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23177?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon resolved SPARK-23177. -- Resolution: Fixed Fixed in https://github.com/apache/spark/pull/20360 > PySpark parameter-less

[jira] [Commented] (SPARK-23194) from_json in FAILFAST mode doesn't fail fast, instead it just returns nulls

2018-01-23 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336709#comment-16336709 ] Hyukjin Kwon commented on SPARK-23194: -- Yup, I think we don't support the parse mode

[jira] [Commented] (SPARK-23198) Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to test ContinuousExecution

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23198?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336707#comment-16336707 ] Apache Spark commented on SPARK-23198: -- User 'dongjoon-hyun' has created a pull requ

[jira] [Assigned] (SPARK-23198) Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to test ContinuousExecution

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23198: Assignee: (was: Apache Spark) > Fix KafkaContinuousSourceStressForDontFailOnDataLossSu

[jira] [Assigned] (SPARK-23198) Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to test ContinuousExecution

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23198?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23198: Assignee: Apache Spark > Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to test

[jira] [Created] (SPARK-23198) Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to test ContinuousExecution

2018-01-23 Thread Dongjoon Hyun (JIRA)
Dongjoon Hyun created SPARK-23198: - Summary: Fix KafkaContinuousSourceStressForDontFailOnDataLossSuite to test ContinuousExecution Key: SPARK-23198 URL: https://issues.apache.org/jira/browse/SPARK-23198

[jira] [Commented] (SPARK-20928) SPIP: Continuous Processing Mode for Structured Streaming

2018-01-23 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336699#comment-16336699 ] Tathagata Das commented on SPARK-20928: --- I accidentally closed the issue without se

[jira] [Updated] (SPARK-20928) SPIP: Continuous Processing Mode for Structured Streaming

2018-01-23 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das updated SPARK-20928: -- Target Version/s: (was: 2.3.0) > SPIP: Continuous Processing Mode for Structured Streaming >

[jira] [Reopened] (SPARK-20928) SPIP: Continuous Processing Mode for Structured Streaming

2018-01-23 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reopened SPARK-20928: --- > SPIP: Continuous Processing Mode for Structured Streaming > ---

[jira] [Assigned] (SPARK-22863) Make MicroBatchExecution also support MicroBatchRead/WriteSupport

2018-01-23 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22863?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-22863: - Assignee: Jose Torres > Make MicroBatchExecution also support MicroBatchRead/WriteSuppor

[jira] [Assigned] (SPARK-23033) disable task-level retry for continuous execution

2018-01-23 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das reassigned SPARK-23033: - Assignee: Jose Torres > disable task-level retry for continuous execution >

[jira] [Resolved] (SPARK-20928) SPIP: Continuous Processing Mode for Structured Streaming

2018-01-23 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20928?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-20928. --- Resolution: Done Assignee: Jose Torres > SPIP: Continuous Processing Mode for Structure

[jira] [Updated] (SPARK-23195) Hint of cached data is lost

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23195: -- Fix Version/s: (was: 2.3.0) 2.3.1 > Hint of cached data is lost > --

[jira] [Resolved] (SPARK-23197) Flaky test: spark.streaming.ReceiverSuite."receiver_life_cycle"

2018-01-23 Thread Tathagata Das (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Tathagata Das resolved SPARK-23197. --- Resolution: Fixed Fix Version/s: 2.3.0 3.0.0 Issue resolved by pull

[jira] [Resolved] (SPARK-23195) Hint of cached data is lost

2018-01-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23195. - Resolution: Fixed Fix Version/s: 2.3.0 > Hint of cached data is lost > ---

[jira] [Commented] (SPARK-23195) Hint of cached data is lost

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336615#comment-16336615 ] Dongjoon Hyun commented on SPARK-23195: --- I'm sure that RC3 will be made soon, but l

[jira] [Commented] (SPARK-22739) Additional Expression Support for Objects

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336616#comment-16336616 ] Dongjoon Hyun commented on SPARK-22739: --- Thank you! > Additional Expression Suppor

[jira] [Commented] (SPARK-23192) Hint is lost after using cached data

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336612#comment-16336612 ] Dongjoon Hyun commented on SPARK-23192: --- Sorry, but this should be marked as 2.3.1

[jira] [Updated] (SPARK-23195) Hint of cached data is lost

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23195: -- Target Version/s: 2.3.1 (was: 2.3.0) > Hint of cached data is lost > -

[jira] [Updated] (SPARK-23192) Hint is lost after using cached data

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23192: -- Fix Version/s: (was: 2.3.0) 2.3.1 > Hint is lost after using cached data

[jira] [Assigned] (SPARK-23159) Update Cloudpickle to match version 0.4.2

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23159: Assignee: Apache Spark > Update Cloudpickle to match version 0.4.2 > -

[jira] [Assigned] (SPARK-23159) Update Cloudpickle to match version 0.4.2

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23159?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23159: Assignee: (was: Apache Spark) > Update Cloudpickle to match version 0.4.2 > --

[jira] [Commented] (SPARK-23159) Update Cloudpickle to match version 0.4.2

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336583#comment-16336583 ] Apache Spark commented on SPARK-23159: -- User 'BryanCutler' has created a pull reques

[jira] [Updated] (SPARK-22739) Additional Expression Support for Objects

2018-01-23 Thread Sameer Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sameer Agarwal updated SPARK-22739: --- Target Version/s: 2.4.0 (was: 2.3.0) > Additional Expression Support for Objects > -

[jira] [Commented] (SPARK-22739) Additional Expression Support for Objects

2018-01-23 Thread Sameer Agarwal (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336565#comment-16336565 ] Sameer Agarwal commented on SPARK-22739: Agree, I'll re-target this for 2.4.0 for

[jira] [Resolved] (SPARK-23192) Hint is lost after using cached data

2018-01-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-23192. - Resolution: Fixed Fix Version/s: 2.3.0 > Hint is lost after using cached data > --

[jira] [Commented] (SPARK-23197) Flaky test: spark.streaming.ReceiverSuite."receiver_life_cycle"

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23197?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336499#comment-16336499 ] Apache Spark commented on SPARK-23197: -- User 'tdas' has created a pull request for t

[jira] [Assigned] (SPARK-23197) Flaky test: spark.streaming.ReceiverSuite."receiver_life_cycle"

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23197: Assignee: Tathagata Das (was: Apache Spark) > Flaky test: spark.streaming.ReceiverSuite."

[jira] [Assigned] (SPARK-23197) Flaky test: spark.streaming.ReceiverSuite."receiver_life_cycle"

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23197?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23197: Assignee: Apache Spark (was: Tathagata Das) > Flaky test: spark.streaming.ReceiverSuite."

[jira] [Created] (SPARK-23197) Flaky test: spark.streaming.ReceiverSuite."receiver_life_cycle"

2018-01-23 Thread Tathagata Das (JIRA)
Tathagata Das created SPARK-23197: - Summary: Flaky test: spark.streaming.ReceiverSuite."receiver_life_cycle" Key: SPARK-23197 URL: https://issues.apache.org/jira/browse/SPARK-23197 Project: Spark

[jira] [Commented] (SPARK-22809) pyspark is sensitive to imports with dots

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336494#comment-16336494 ] Dongjoon Hyun commented on SPARK-22809: --- Since this is not a regression, can we mov

[jira] [Commented] (SPARK-22739) Additional Expression Support for Objects

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22739?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336487#comment-16336487 ] Dongjoon Hyun commented on SPARK-22739: --- Since this is a new feature and the PR is

[jira] [Assigned] (SPARK-23196) Unify continuous and microbatch V2 sinks

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23196: Assignee: (was: Apache Spark) > Unify continuous and microbatch V2 sinks > ---

[jira] [Assigned] (SPARK-23196) Unify continuous and microbatch V2 sinks

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23196: Assignee: Apache Spark > Unify continuous and microbatch V2 sinks > --

[jira] [Commented] (SPARK-23196) Unify continuous and microbatch V2 sinks

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23196?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336485#comment-16336485 ] Apache Spark commented on SPARK-23196: -- User 'jose-torres' has created a pull reques

[jira] [Resolved] (SPARK-23193) Insert into Spark Table statement cannot specify column names

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun resolved SPARK-23193. --- Resolution: Duplicate Hi, [~zwu@gmail.com]. Thank you for your suggestion, but this is tr

[jira] [Closed] (SPARK-23193) Insert into Spark Table statement cannot specify column names

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun closed SPARK-23193. - > Insert into Spark Table statement cannot specify column names > ---

[jira] [Created] (SPARK-23196) Unify continuous and microbatch V2 sinks

2018-01-23 Thread Jose Torres (JIRA)
Jose Torres created SPARK-23196: --- Summary: Unify continuous and microbatch V2 sinks Key: SPARK-23196 URL: https://issues.apache.org/jira/browse/SPARK-23196 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-23189) reflect stage level blacklisting on executor tab

2018-01-23 Thread Attila Zsolt Piros (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336372#comment-16336372 ] Attila Zsolt Piros commented on SPARK-23189: [~tgraves] I see so only active

[jira] [Assigned] (SPARK-17088) IsolatedClientLoader fails to load Hive client when sharesHadoopClasses is false

2018-01-23 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin reassigned SPARK-17088: -- Assignee: Marcelo Vanzin > IsolatedClientLoader fails to load Hive client when sharesH

[jira] [Resolved] (SPARK-17088) IsolatedClientLoader fails to load Hive client when sharesHadoopClasses is false

2018-01-23 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-17088. Resolution: Fixed Fix Version/s: 2.4.0 Issue resolved by pull request 20169 [https:/

[jira] [Assigned] (SPARK-23195) Hint of cached data is lost

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23195: Assignee: Xiao Li (was: Apache Spark) > Hint of cached data is lost > ---

[jira] [Assigned] (SPARK-23195) Hint of cached data is lost

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23195?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23195: Assignee: Apache Spark (was: Xiao Li) > Hint of cached data is lost > ---

[jira] [Commented] (SPARK-23195) Hint of cached data is lost

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23195?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336356#comment-16336356 ] Apache Spark commented on SPARK-23195: -- User 'gatorsmile' has created a pull request

[jira] [Created] (SPARK-23195) Hint of cached data is lost

2018-01-23 Thread Xiao Li (JIRA)
Xiao Li created SPARK-23195: --- Summary: Hint of cached data is lost Key: SPARK-23195 URL: https://issues.apache.org/jira/browse/SPARK-23195 Project: Spark Issue Type: Bug Components: SQL

[jira] [Updated] (SPARK-22389) partitioning reporting

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-22389: -- Fix Version/s: (was: 2.3.0) 2.3.1 > partitioning reporting > ---

[jira] [Commented] (SPARK-22389) partitioning reporting

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336349#comment-16336349 ] Dongjoon Hyun commented on SPARK-22389: --- Hi, [~cloud_fan] and [~sameerag]. Since th

[jira] [Commented] (SPARK-23189) reflect stage level blacklisting on executor tab

2018-01-23 Thread Thomas Graves (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336332#comment-16336332 ] Thomas Graves commented on SPARK-23189: --- Personally I use the executors page a lot

[jira] [Commented] (SPARK-23189) reflect stage level blacklisting on executor tab

2018-01-23 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23189?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336326#comment-16336326 ] Alex Bozarth commented on SPARK-23189: -- The tooltip feature does exist, but I agree

[jira] [Commented] (SPARK-23166) Add maxDF Parameter to CountVectorizer

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336264#comment-16336264 ] Apache Spark commented on SPARK-23166: -- User 'ymazari' has created a pull request fo

[jira] [Commented] (SPARK-22522) Convert to apache-release to publish Maven artifacts

2018-01-23 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336254#comment-16336254 ] Felix Cheung commented on SPARK-22522: -- It’s to the right place but not the supporte

[jira] [Reopened] (SPARK-22522) Convert to apache-release to publish Maven artifacts

2018-01-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reopened SPARK-22522: --- Oh was reading too fast. Not done. I thought it was publishing yo the right place ? Seemed so > Convert

[jira] [Resolved] (SPARK-22522) Convert to apache-release to publish Maven artifacts

2018-01-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-22522. --- Resolution: Done > Convert to apache-release to publish Maven artifacts > --

[jira] [Commented] (SPARK-22522) Convert to apache-release to publish Maven artifacts

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336221#comment-16336221 ] Dongjoon Hyun commented on SPARK-22522: --- Thank you for confirming, [~felixcheung].

[jira] [Updated] (SPARK-22522) Convert to apache-release to publish Maven artifacts

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-22522: -- Target Version/s: (was: 2.3.0) > Convert to apache-release to publish Maven artifacts >

[jira] [Updated] (SPARK-23192) Hint is lost after using cached data

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-23192: -- Target Version/s: (was: 2.3.0) > Hint is lost after using cached data > -

[jira] [Commented] (SPARK-23192) Hint is lost after using cached data

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336217#comment-16336217 ] Dongjoon Hyun commented on SPARK-23192: --- Thank you for confirming, [~smilegator]!

[jira] [Commented] (SPARK-22522) Convert to apache-release to publish Maven artifacts

2018-01-23 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336204#comment-16336204 ] Felix Cheung commented on SPARK-22522: -- No it’s not done AFAIK > Convert to apache

[jira] [Commented] (SPARK-22522) Convert to apache-release to publish Maven artifacts

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22522?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336198#comment-16336198 ] Dongjoon Hyun commented on SPARK-22522: --- Hi, [~felixcheung] and [~srowen]. Is this

[jira] [Commented] (SPARK-23192) Hint is lost after using cached data

2018-01-23 Thread Xiao Li (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336196#comment-16336196 ] Xiao Li commented on SPARK-23192: - This is not a regression. It does not block 2.3.0 rele

[jira] [Commented] (SPARK-21960) Spark Streaming Dynamic Allocation should respect spark.executor.instances

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336192#comment-16336192 ] Dongjoon Hyun commented on SPARK-21960: --- Hi, [~Karthik Palaniappan]. This is an imp

[jira] [Updated] (SPARK-21960) Spark Streaming Dynamic Allocation should respect spark.executor.instances

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21960?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21960: -- Target Version/s: (was: 2.3.0) > Spark Streaming Dynamic Allocation should respect spark.exec

[jira] [Commented] (SPARK-21646) Add new type coercion rules to compatible with Hive

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21646?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336189#comment-16336189 ] Dongjoon Hyun commented on SPARK-21646: --- Hi, [~smilegator] and [~q79969786]. Since

[jira] [Updated] (SPARK-21646) Add new type coercion rules to compatible with Hive

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21646?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-21646: -- Target Version/s: (was: 2.3.0) > Add new type coercion rules to compatible with Hive > --

[jira] [Commented] (SPARK-23194) from_json in FAILFAST mode doesn't fail fast, instead it just returns nulls

2018-01-23 Thread Burak Yavuz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23194?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336183#comment-16336183 ] Burak Yavuz commented on SPARK-23194: - cc [~cloud_fan] seems like you touched that co

[jira] [Created] (SPARK-23194) from_json in FAILFAST mode doesn't fail fast, instead it just returns nulls

2018-01-23 Thread Burak Yavuz (JIRA)
Burak Yavuz created SPARK-23194: --- Summary: from_json in FAILFAST mode doesn't fail fast, instead it just returns nulls Key: SPARK-23194 URL: https://issues.apache.org/jira/browse/SPARK-23194 Project: Sp

[jira] [Updated] (SPARK-19989) Flaky Test: org.apache.spark.sql.kafka010.KafkaSourceStressSuite

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-19989: -- Target Version/s: (was: 2.3.0) > Flaky Test: org.apache.spark.sql.kafka010.KafkaSourceStressS

[jira] [Commented] (SPARK-19989) Flaky Test: org.apache.spark.sql.kafka010.KafkaSourceStressSuite

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19989?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336179#comment-16336179 ] Dongjoon Hyun commented on SPARK-19989: --- This test case doesn't fail during the lat

[jira] [Updated] (SPARK-22731) Add a test for ROWID type to OracleIntegrationSuite

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22731?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-22731: -- Target Version/s: (was: 2.3.0) > Add a test for ROWID type to OracleIntegrationSuite > --

[jira] [Commented] (SPARK-22731) Add a test for ROWID type to OracleIntegrationSuite

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22731?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336167#comment-16336167 ] Dongjoon Hyun commented on SPARK-22731: --- Since this is not a regression, I'll remov

[jira] [Commented] (SPARK-23192) Hint is lost after using cached data

2018-01-23 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336163#comment-16336163 ] Dongjoon Hyun commented on SPARK-23192: --- Hi, [~smilegator]. I'm confused. You gave

[jira] [Commented] (SPARK-23166) Add maxDF Parameter to CountVectorizer

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23166?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336157#comment-16336157 ] Apache Spark commented on SPARK-23166: -- User 'ymazari' has created a pull request fo

[jira] [Resolved] (SPARK-21571) Spark history server leaves incomplete or unreadable history files around forever.

2018-01-23 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21571?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-21571. Resolution: Fixed Assignee: Eric Vandenberg Fix Version/s: 2.3.0 > Spark hi

[jira] [Commented] (SPARK-21571) Spark history server leaves incomplete or unreadable history files around forever.

2018-01-23 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21571?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336150#comment-16336150 ] Marcelo Vanzin commented on SPARK-21571: Fixed with the patch for SPARK-20664 (wh

[jira] [Updated] (SPARK-23193) Insert into Spark Table statement cannot specify column names

2018-01-23 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-23193: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) I don't see that syntax is supp

[jira] [Updated] (SPARK-23193) Insert into Spark Table statement cannot specify column names

2018-01-23 Thread Paul Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Paul Wu updated SPARK-23193: Summary: Insert into Spark Table statement cannot specify column names (was: Insert into Spark Table canno

[jira] [Created] (SPARK-23193) Insert into Spark Table cannot specify column names

2018-01-23 Thread Paul Wu (JIRA)
Paul Wu created SPARK-23193: --- Summary: Insert into Spark Table cannot specify column names Key: SPARK-23193 URL: https://issues.apache.org/jira/browse/SPARK-23193 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-22809) pyspark is sensitive to imports with dots

2018-01-23 Thread Cricket Temple (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22809?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336070#comment-16336070 ] Cricket Temple commented on SPARK-22809: You've got to run it in ipython or zeppe

[jira] [Commented] (SPARK-23192) Hint is lost after using cached data

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23192?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16336055#comment-16336055 ] Apache Spark commented on SPARK-23192: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-23192) Hint is lost after using cached data

2018-01-23 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23192?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-23192: Assignee: Xiao Li (was: Apache Spark) > Hint is lost after using cached data > --

  1   2   >