[jira] [Reopened] (SPARK-19840) Disallow creating permanent functions with invalid class names

[ https://issues.apache.org/jira/browse/SPARK-19840?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li reopened SPARK-19840: - > Disallow creating permanent functions with invalid class names > --

[jira] [Commented] (SPARK-20000) Spark Hive tests aborted due to lz4-java on ppc64le

[ https://issues.apache.org/jira/browse/SPARK-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963922#comment-15963922 ] Ayappan commented on SPARK-2: - Yup! that's right. > Spark Hive tests aborted due to

[jira] [Commented] (SPARK-20287) Kafka Consumer should be able to subscribe to more than one topic partition

[ https://issues.apache.org/jira/browse/SPARK-20287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963920#comment-15963920 ] Sean Owen commented on SPARK-20287: --- Spark has a different execution model though where

[jira] [Resolved] (SPARK-20279) In web ui,'Only showing 200' should be changed to 'only showing last 200'.

[ https://issues.apache.org/jira/browse/SPARK-20279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20279. --- Resolution: Won't Fix > In web ui,'Only showing 200' should be changed to 'only showing last 200'. >

[jira] [Commented] (SPARK-5928) Remote Shuffle Blocks cannot be more than 2 GB

[ https://issues.apache.org/jira/browse/SPARK-5928?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963898#comment-15963898 ] Thomas Decaux commented on SPARK-5928: -- Since there is lot of issues with >2G partiti

[jira] [Resolved] (SPARK-14385) Use FunctionIdentifier in FunctionRegistry/SessionCatalog

[ https://issues.apache.org/jira/browse/SPARK-14385?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-14385. - Resolution: Fixed Fix Version/s: 2.0.0 > Use FunctionIdentifier in FunctionRegistry/SessionCatalog

[jira] [Commented] (SPARK-20288) Improve BasicSchedulerIntegrationSuite "multi-stage job"

[ https://issues.apache.org/jira/browse/SPARK-20288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963884#comment-15963884 ] Apache Spark commented on SPARK-20288: -- User 'jinxing64' has created a pull request

[jira] [Assigned] (SPARK-20288) Improve BasicSchedulerIntegrationSuite "multi-stage job"

[ https://issues.apache.org/jira/browse/SPARK-20288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20288: Assignee: Apache Spark > Improve BasicSchedulerIntegrationSuite "multi-stage job" > --

[jira] [Assigned] (SPARK-20288) Improve BasicSchedulerIntegrationSuite "multi-stage job"

[ https://issues.apache.org/jira/browse/SPARK-20288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20288: Assignee: (was: Apache Spark) > Improve BasicSchedulerIntegrationSuite "multi-stage jo

[jira] [Updated] (SPARK-20288) Improve BasicSchedulerIntegrationSuite "multi-stage job"

[ https://issues.apache.org/jira/browse/SPARK-20288?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] jin xing updated SPARK-20288: - Description: ShuffleId is determined before job submitted. But it's hard to predict stageId by shuffleId

[jira] [Created] (SPARK-20288) Improve BasicSchedulerIntegrationSuite "multi-stage job"

jin xing created SPARK-20288: Summary: Improve BasicSchedulerIntegrationSuite "multi-stage job" Key: SPARK-20288 URL: https://issues.apache.org/jira/browse/SPARK-20288 Project: Spark Issue Type:

[jira] [Commented] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963815#comment-15963815 ] Wenchen Fan commented on SPARK-12837: - [~Tagar] due to some implementation limitation

[jira] [Commented] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963812#comment-15963812 ] Ruslan Dautkhanov commented on SPARK-12837: --- [~cloud_fan] I didn't realize torr

[jira] [Resolved] (SPARK-17564) Flaky RequestTimeoutIntegrationSuite, furtherRequestsDelay

[ https://issues.apache.org/jira/browse/SPARK-17564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17564. - Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.2.0 2

[jira] [Commented] (SPARK-3383) DecisionTree aggregate size could be smaller

[ https://issues.apache.org/jira/browse/SPARK-3383?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963741#comment-15963741 ] Yan Facai (颜发才) commented on SPARK-3383: I think the task contains two subtask: 1

[jira] [Comment Edited] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963705#comment-15963705 ] Wenchen Fan edited comment on SPARK-12837 at 4/11/17 1:06 AM: -

[jira] [Commented] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963705#comment-15963705 ] Wenchen Fan commented on SPARK-12837: - [~Tagar] I think this may be because the size

[jira] [Created] (SPARK-20287) Kafka Consumer should be able to subscribe to more than one topic partition

Stephane Maarek created SPARK-20287: --- Summary: Kafka Consumer should be able to subscribe to more than one topic partition Key: SPARK-20287 URL: https://issues.apache.org/jira/browse/SPARK-20287 Pro

[jira] [Commented] (SPARK-18555) na.fill miss up original values in long integers

[ https://issues.apache.org/jira/browse/SPARK-18555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963685#comment-15963685 ] Apache Spark commented on SPARK-18555: -- User 'dbtsai' has created a pull request for

[jira] [Commented] (SPARK-18555) na.fill miss up original values in long integers

[ https://issues.apache.org/jira/browse/SPARK-18555?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963684#comment-15963684 ] Apache Spark commented on SPARK-18555: -- User 'dbtsai' has created a pull request for

[jira] [Updated] (SPARK-20270) na.fill will change the values in long or integer when the default value is in double

[ https://issues.apache.org/jira/browse/SPARK-20270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-20270: Fix Version/s: 2.1.1 2.0.3 > na.fill will change the values in long or integer when the

[jira] [Updated] (SPARK-18555) na.fill miss up original values in long integers

[ https://issues.apache.org/jira/browse/SPARK-18555?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] DB Tsai updated SPARK-18555: Fix Version/s: 2.1.1 2.0.3 Component/s: SQL > na.fill miss up original values in l

[jira] [Assigned] (SPARK-17564) Flaky RequestTimeoutIntegrationSuite, furtherRequestsDelay

[ https://issues.apache.org/jira/browse/SPARK-17564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17564: Assignee: Apache Spark > Flaky RequestTimeoutIntegrationSuite, furtherRequestsDelay >

[jira] [Assigned] (SPARK-17564) Flaky RequestTimeoutIntegrationSuite, furtherRequestsDelay

[ https://issues.apache.org/jira/browse/SPARK-17564?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17564: Assignee: (was: Apache Spark) > Flaky RequestTimeoutIntegrationSuite, furtherRequestsD

[jira] [Commented] (SPARK-17564) Flaky RequestTimeoutIntegrationSuite, furtherRequestsDelay

[ https://issues.apache.org/jira/browse/SPARK-17564?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963663#comment-15963663 ] Apache Spark commented on SPARK-17564: -- User 'zsxwing' has created a pull request fo

[jira] [Commented] (SPARK-18084) write.partitionBy() does not recognize nested columns that select() can access

[ https://issues.apache.org/jira/browse/SPARK-18084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963621#comment-15963621 ] Rupesh Mane commented on SPARK-18084: - Any update on when this will be fixed? > writ

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963601#comment-15963601 ] Michael Gummelt commented on SPARK-16742: - bq. So, assuming that Mesos is configu

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963592#comment-15963592 ] Marcelo Vanzin commented on SPARK-16742: bq. It authenticates the Mesos principal

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963583#comment-15963583 ] Michael Gummelt commented on SPARK-16742: - bq. That sounds problematic. The way Y

[jira] [Updated] (SPARK-20284) Make SerializationStream and DeserializationStream extend Closeable

[ https://issues.apache.org/jira/browse/SPARK-20284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-20284: -- Priority: Trivial (was: Minor) It might be fine to add, but, does it help anything? try-with-resources

[jira] [Created] (SPARK-20286) dynamicAllocation.executorIdleTimeout is ignored after unpersist

Miguel Pérez created SPARK-20286: Summary: dynamicAllocation.executorIdleTimeout is ignored after unpersist Key: SPARK-20286 URL: https://issues.apache.org/jira/browse/SPARK-20286 Project: Spark

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963559#comment-15963559 ] Marcelo Vanzin commented on SPARK-16742: bq. But in Spark, this isn't currently d

[jira] [Comment Edited] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963510#comment-15963510 ] Ruslan Dautkhanov edited comment on SPARK-12837 at 4/10/17 9:29 PM: ---

[jira] [Commented] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963510#comment-15963510 ] Ruslan Dautkhanov commented on SPARK-12837: --- It might be a bug in broadcast joi

[jira] [Resolved] (SPARK-20283) Add preOptimizationBatches

[ https://issues.apache.org/jira/browse/SPARK-20283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-20283. - Resolution: Fixed Fix Version/s: 2.2.0 > Add preOptimizationBatches >

[jira] [Resolved] (SPARK-20282) Flaky test: org.apache.spark.sql.streaming/StreamingQuerySuite/OneTime_trigger__commit_log__and_exception

[ https://issues.apache.org/jira/browse/SPARK-20282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-20282. -- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.2.0 > Flaky test: >

[jira] [Updated] (SPARK-20285) Flaky test: pyspark.streaming.tests.BasicOperationTests.test_cogroup

[ https://issues.apache.org/jira/browse/SPARK-20285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-20285: - Affects Version/s: 2.1.1 2.0.3 > Flaky test: pyspark.streaming.tests.Basic

[jira] [Updated] (SPARK-20285) Flaky test: pyspark.streaming.tests.BasicOperationTests.test_cogroup

[ https://issues.apache.org/jira/browse/SPARK-20285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-20285: - Affects Version/s: (was: 2.1.1) (was: 2.0.3)

[jira] [Resolved] (SPARK-20285) Flaky test: pyspark.streaming.tests.BasicOperationTests.test_cogroup

[ https://issues.apache.org/jira/browse/SPARK-20285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-20285. -- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 2.0.3

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963469#comment-15963469 ] Michael Gummelt commented on SPARK-16742: - [~jerryshao] Great! The current RPC us

[jira] [Comment Edited] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963446#comment-15963446 ] Michael Gummelt edited comment on SPARK-16742 at 4/10/17 8:35 PM: -

[jira] [Comment Edited] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963446#comment-15963446 ] Michael Gummelt edited comment on SPARK-16742 at 4/10/17 8:20 PM: -

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963446#comment-15963446 ] Michael Gummelt commented on SPARK-16742: - bq. The most basic feature needed for

[jira] [Comment Edited] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963446#comment-15963446 ] Michael Gummelt edited comment on SPARK-16742 at 4/10/17 8:18 PM: -

[jira] [Resolved] (SPARK-20280) SharedInMemoryCache Weigher integer overflow

[ https://issues.apache.org/jira/browse/SPARK-20280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-20280. --- Resolution: Fixed Assignee: Bogdan Raducanu Fix Version/s: 2.2.0

[jira] [Commented] (SPARK-20285) Flaky test: pyspark.streaming.tests.BasicOperationTests.test_cogroup

[ https://issues.apache.org/jira/browse/SPARK-20285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963422#comment-15963422 ] Shixiong Zhu commented on SPARK-20285: -- https://github.com/apache/spark/pull/17597

[jira] [Issue Comment Deleted] (SPARK-20285) Flaky test: pyspark.streaming.tests.BasicOperationTests.test_cogroup

[ https://issues.apache.org/jira/browse/SPARK-20285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-20285: - Comment: was deleted (was: https://github.com/apache/spark/pull/17597) > Flaky test: pyspark.str

[jira] [Assigned] (SPARK-20284) Make SerializationStream and DeserializationStream extend Closeable

[ https://issues.apache.org/jira/browse/SPARK-20284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20284: Assignee: (was: Apache Spark) > Make SerializationStream and DeserializationStream ext

[jira] [Commented] (SPARK-20284) Make SerializationStream and DeserializationStream extend Closeable

[ https://issues.apache.org/jira/browse/SPARK-20284?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963419#comment-15963419 ] Apache Spark commented on SPARK-20284: -- User 'superbobry' has created a pull request

[jira] [Assigned] (SPARK-20284) Make SerializationStream and DeserializationStream extend Closeable

[ https://issues.apache.org/jira/browse/SPARK-20284?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20284: Assignee: Apache Spark > Make SerializationStream and DeserializationStream extend Closeab

[jira] [Assigned] (SPARK-20285) Flaky test: pyspark.streaming.tests.BasicOperationTests.test_cogroup

[ https://issues.apache.org/jira/browse/SPARK-20285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20285: Assignee: Shixiong Zhu (was: Apache Spark) > Flaky test: pyspark.streaming.tests.BasicOpe

[jira] [Commented] (SPARK-20285) Flaky test: pyspark.streaming.tests.BasicOperationTests.test_cogroup

[ https://issues.apache.org/jira/browse/SPARK-20285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963412#comment-15963412 ] Apache Spark commented on SPARK-20285: -- User 'zsxwing' has created a pull request fo

[jira] [Assigned] (SPARK-20285) Flaky test: pyspark.streaming.tests.BasicOperationTests.test_cogroup

[ https://issues.apache.org/jira/browse/SPARK-20285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20285: Assignee: Apache Spark (was: Shixiong Zhu) > Flaky test: pyspark.streaming.tests.BasicOpe

[jira] [Created] (SPARK-20285) Flaky test:

Shixiong Zhu created SPARK-20285: Summary: Flaky test: Key: SPARK-20285 URL: https://issues.apache.org/jira/browse/SPARK-20285 Project: Spark Issue Type: Bug Components: Tests

[jira] [Updated] (SPARK-20285) Flaky test: pyspark.streaming.tests.BasicOperationTests.test_cogroup

[ https://issues.apache.org/jira/browse/SPARK-20285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-20285: - Summary: Flaky test: pyspark.streaming.tests.BasicOperationTests.test_cogroup (was: Flaky test:

[jira] [Commented] (SPARK-20156) Java String toLowerCase "Turkish locale bug" causes Spark problems

[ https://issues.apache.org/jira/browse/SPARK-20156?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963406#comment-15963406 ] Serkan Taş commented on SPARK-20156: Thank you Sean, it was more than i expected i th

[jira] [Assigned] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12837: Assignee: Apache Spark (was: Wenchen Fan) > Spark driver requires large memory space for

[jira] [Commented] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963402#comment-15963402 ] Apache Spark commented on SPARK-12837: -- User 'cloud-fan' has created a pull request

[jira] [Assigned] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12837: Assignee: Wenchen Fan (was: Apache Spark) > Spark driver requires large memory space for

[jira] [Created] (SPARK-20284) Make SerializationStream and DeserializationStream extend Closeable

Sergei Lebedev created SPARK-20284: -- Summary: Make SerializationStream and DeserializationStream extend Closeable Key: SPARK-20284 URL: https://issues.apache.org/jira/browse/SPARK-20284 Project: Spar

[jira] [Resolved] (SPARK-20156) Java String toLowerCase "Turkish locale bug" causes Spark problems

[ https://issues.apache.org/jira/browse/SPARK-20156?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-20156. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17527 [https://github.co

[jira] [Commented] (SPARK-19352) Sorting issues on relatively big datasets

[ https://issues.apache.org/jira/browse/SPARK-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963367#comment-15963367 ] Charles Pritchard commented on SPARK-19352: --- Does this fix the issue in SPARK-1

[jira] [Commented] (SPARK-19352) Sorting issues on relatively big datasets

[ https://issues.apache.org/jira/browse/SPARK-19352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963365#comment-15963365 ] Charles Pritchard commented on SPARK-19352: --- [~cloud_fan] Yes, Hive relies on s

[jira] [Commented] (SPARK-19067) mapGroupsWithState - arbitrary stateful operations with Structured Streaming (similar to DStream.mapWithState)

[ https://issues.apache.org/jira/browse/SPARK-19067?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963353#comment-15963353 ] Michael Armbrust commented on SPARK-19067: -- No, this will be available in Spark

[jira] [Commented] (SPARK-20283) Add preOptimizationBatches

[ https://issues.apache.org/jira/browse/SPARK-20283?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963351#comment-15963351 ] Apache Spark commented on SPARK-20283: -- User 'rxin' has created a pull request for t

[jira] [Assigned] (SPARK-20283) Add preOptimizationBatches

[ https://issues.apache.org/jira/browse/SPARK-20283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20283: Assignee: Reynold Xin (was: Apache Spark) > Add preOptimizationBatches >

[jira] [Assigned] (SPARK-20283) Add preOptimizationBatches

[ https://issues.apache.org/jira/browse/SPARK-20283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20283: Assignee: Apache Spark (was: Reynold Xin) > Add preOptimizationBatches >

[jira] [Updated] (SPARK-20283) Add preOptimizationBatches

[ https://issues.apache.org/jira/browse/SPARK-20283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-20283: Description: We currently have postHocOptimizationBatches, but not preOptimizationBatches. This pa

[jira] [Created] (SPARK-20283) Add preOptimizationBatches

Reynold Xin created SPARK-20283: --- Summary: Add preOptimizationBatches Key: SPARK-20283 URL: https://issues.apache.org/jira/browse/SPARK-20283 Project: Spark Issue Type: New Feature Co

[jira] [Assigned] (SPARK-20282) Flaky test: org.apache.spark.sql.streaming/StreamingQuerySuite/OneTime_trigger__commit_log__and_exception

[ https://issues.apache.org/jira/browse/SPARK-20282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20282: Assignee: Apache Spark > Flaky test: > org.apache.spark.sql.streaming/StreamingQuerySuite

[jira] [Assigned] (SPARK-20282) Flaky test: org.apache.spark.sql.streaming/StreamingQuerySuite/OneTime_trigger__commit_log__and_exception

[ https://issues.apache.org/jira/browse/SPARK-20282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20282: Assignee: (was: Apache Spark) > Flaky test: > org.apache.spark.sql.streaming/Streamin

[jira] [Commented] (SPARK-20282) Flaky test: org.apache.spark.sql.streaming/StreamingQuerySuite/OneTime_trigger__commit_log__and_exception

[ https://issues.apache.org/jira/browse/SPARK-20282?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963341#comment-15963341 ] Apache Spark commented on SPARK-20282: -- User 'zsxwing' has created a pull request fo

[jira] [Updated] (SPARK-20282) Flaky test: org.apache.spark.sql.streaming/StreamingQuerySuite/OneTime_trigger__commit_log__and_exception

[ https://issues.apache.org/jira/browse/SPARK-20282?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-20282: - Issue Type: Test (was: Bug) > Flaky test: > org.apache.spark.sql.streaming/StreamingQuerySuite/

[jira] [Created] (SPARK-20282) Flaky test: org.apache.spark.sql.streaming/StreamingQuerySuite/OneTime_trigger__commit_log__and_exception

Shixiong Zhu created SPARK-20282: Summary: Flaky test: org.apache.spark.sql.streaming/StreamingQuerySuite/OneTime_trigger__commit_log__and_exception Key: SPARK-20282 URL: https://issues.apache.org/jira/browse/SPAR

[jira] [Comment Edited] (SPARK-650) Add a "setup hook" API for running initialization code on each executor

[ https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963304#comment-15963304 ] Ryan Williams edited comment on SPARK-650 at 4/10/17 6:42 PM: --

[jira] [Commented] (SPARK-650) Add a "setup hook" API for running initialization code on each executor

[ https://issues.apache.org/jira/browse/SPARK-650?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963304#comment-15963304 ] Ryan Williams commented on SPARK-650: - Both suggested workarounds here are lacking or b

[jira] [Commented] (SPARK-16742) Kerberos support for Spark on Mesos

[ https://issues.apache.org/jira/browse/SPARK-16742?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15963136#comment-15963136 ] Marcelo Vanzin commented on SPARK-16742: bq. The problem is then that a kerberos-

[jira] [Resolved] (SPARK-20273) Disallow Non-deterministic Filter push-down into Join Conditions

[ https://issues.apache.org/jira/browse/SPARK-20273?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-20273. - Resolution: Fixed Fix Version/s: 2.2.0 > Disallow Non-deterministic Filter push-down into Join Con

[jira] [Resolved] (SPARK-19518) IGNORE NULLS in first_value / last_value should be supported in SQL statements

[ https://issues.apache.org/jira/browse/SPARK-19518?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19518. --- Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.2.0 > IGNOR

[jira] [Resolved] (SPARK-20243) DebugFilesystem.assertNoOpenStreams thread race

[ https://issues.apache.org/jira/browse/SPARK-20243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-20243. --- Resolution: Fixed Assignee: Bogdan Raducanu Fix Version/s: 2.2.0 > De

[jira] [Updated] (SPARK-20000) Spark Hive tests aborted due to lz4-java on ppc64le

[ https://issues.apache.org/jira/browse/SPARK-2?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-2: -- Priority: Minor (was: Major) Issue Type: Improvement (was: Bug) OK so this is about updating lz

[jira] [Commented] (SPARK-20202) Remove references to org.spark-project.hive

[ https://issues.apache.org/jira/browse/SPARK-20202?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15962903#comment-15962903 ] Steve Loughran commented on SPARK-20202: One thing I do recall as trouble here wa

[jira] [Commented] (SPARK-20000) Spark Hive tests aborted due to lz4-java on ppc64le

[ https://issues.apache.org/jira/browse/SPARK-2?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15962896#comment-15962896 ] Ayappan commented on SPARK-2: - https://github.com/lz4/lz4-java/pull/84 > Spark Hive

[jira] [Commented] (SPARK-20277) Allow Spark on YARN to be launched with Docker

[ https://issues.apache.org/jira/browse/SPARK-20277?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15962787#comment-15962787 ] Zhankun Tang commented on SPARK-20277: -- It requires at least hadoop 2.8 Alpha1 from

[jira] [Updated] (SPARK-20279) In web ui,'Only showing 200' should be changed to 'only showing last 200'.

[ https://issues.apache.org/jira/browse/SPARK-20279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte updated SPARK-20279: --- Description: In web ui,'Only showing 200' should be changed to 'only showing last 200' in th

[jira] [Updated] (SPARK-20279) In web ui,'Only showing 200' should be changed to 'only showing last 200'.

[ https://issues.apache.org/jira/browse/SPARK-20279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte updated SPARK-20279: --- Attachment: stages.png jobs.png > In web ui,'Only showing 200' should be chan

[jira] [Assigned] (SPARK-20279) In web ui,'Only showing 200' should be changed to 'only showing last 200'.

[ https://issues.apache.org/jira/browse/SPARK-20279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20279: Assignee: (was: Apache Spark) > In web ui,'Only showing 200' should be changed to 'onl

[jira] [Assigned] (SPARK-20279) In web ui,'Only showing 200' should be changed to 'only showing last 200'.

[ https://issues.apache.org/jira/browse/SPARK-20279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20279: Assignee: Apache Spark > In web ui,'Only showing 200' should be changed to 'only showing l

[jira] [Commented] (SPARK-20279) In web ui,'Only showing 200' should be changed to 'only showing last 200'.

[ https://issues.apache.org/jira/browse/SPARK-20279?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15962763#comment-15962763 ] Apache Spark commented on SPARK-20279: -- User 'guoxiaolongzte' has created a pull req

[jira] [Assigned] (SPARK-20243) DebugFilesystem.assertNoOpenStreams thread race

[ https://issues.apache.org/jira/browse/SPARK-20243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20243: Assignee: (was: Apache Spark) > DebugFilesystem.assertNoOpenStreams thread race >

[jira] [Assigned] (SPARK-20243) DebugFilesystem.assertNoOpenStreams thread race

[ https://issues.apache.org/jira/browse/SPARK-20243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20243: Assignee: Apache Spark > DebugFilesystem.assertNoOpenStreams thread race > ---

[jira] [Commented] (SPARK-20243) DebugFilesystem.assertNoOpenStreams thread race

[ https://issues.apache.org/jira/browse/SPARK-20243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15962761#comment-15962761 ] Apache Spark commented on SPARK-20243: -- User 'bogdanrdc' has created a pull request

[jira] [Created] (SPARK-20281) Table-valued function range in SQL should use the same number of partitions as spark.range

Jacek Laskowski created SPARK-20281: --- Summary: Table-valued function range in SQL should use the same number of partitions as spark.range Key: SPARK-20281 URL: https://issues.apache.org/jira/browse/SPARK-20281

[jira] [Updated] (SPARK-20279) In web ui,'Only showing 200' should be changed to 'only showing last 200'.

[ https://issues.apache.org/jira/browse/SPARK-20279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte updated SPARK-20279: --- Summary: In web ui,'Only showing 200' should be changed to 'only showing last 200'. (was: In

[jira] [Updated] (SPARK-20279) In web ui,'Only showing 200' should be changed to 'only showing last 200' in the page of 'jobs' or'stages'.

[ https://issues.apache.org/jira/browse/SPARK-20279?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] guoxiaolongzte updated SPARK-20279: --- Summary: In web ui,'Only showing 200' should be changed to 'only showing last 200' in the pag

[jira] [Assigned] (SPARK-20280) SharedInMemoryCache Weigher integer overflow

[ https://issues.apache.org/jira/browse/SPARK-20280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20280: Assignee: (was: Apache Spark) > SharedInMemoryCache Weigher integer overflow > ---

[jira] [Commented] (SPARK-20280) SharedInMemoryCache Weigher integer overflow

[ https://issues.apache.org/jira/browse/SPARK-20280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15962751#comment-15962751 ] Apache Spark commented on SPARK-20280: -- User 'bogdanrdc' has created a pull request

[jira] [Assigned] (SPARK-20280) SharedInMemoryCache Weigher integer overflow

[ https://issues.apache.org/jira/browse/SPARK-20280?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-20280: Assignee: Apache Spark > SharedInMemoryCache Weigher integer overflow > --

[jira] [Commented] (SPARK-20280) SharedInMemoryCache Weigher integer overflow

[ https://issues.apache.org/jira/browse/SPARK-20280?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15962750#comment-15962750 ] Sean Owen commented on SPARK-20280: --- I guess cap it at {{Int.MaxValue}}? > SharedInMem

[jira] [Comment Edited] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15962731#comment-15962731 ] Wenchen Fan edited comment on SPARK-12837 at 4/10/17 11:54 AM:

  1   2   >