[jira] [Updated] (SPARK-19816) DataFrameCallbackSuite doesn't recover the log level

[ https://issues.apache.org/jira/browse/SPARK-19816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19816: - Affects Version/s: (was: 2.2.0) > DataFrameCallbackSuite doesn't recover the log level >

[jira] [Updated] (SPARK-19816) DataFrameCallbackSuite doesn't recover the log level

[ https://issues.apache.org/jira/browse/SPARK-19816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-19816: - Affects Version/s: 2.1.0 > DataFrameCallbackSuite doesn't recover the log level >

[jira] [Commented] (SPARK-19815) Not orderable should be applied to right key instead of left key

[ https://issues.apache.org/jira/browse/SPARK-19815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895522#comment-15895522 ] Zhan Zhang commented on SPARK-19815: I am thinking the logic again. On the surface, the logic may be

[jira] [Commented] (SPARK-19659) Fetch big blocks to disk when shuffle-read

[ https://issues.apache.org/jira/browse/SPARK-19659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895502#comment-15895502 ] Imran Rashid commented on SPARK-19659: -- I think Reynold has a good point. I really don't like the

[jira] [Assigned] (SPARK-19701) the `in` operator in pyspark is broken

[ https://issues.apache.org/jira/browse/SPARK-19701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19701: Assignee: (was: Apache Spark) > the `in` operator in pyspark is broken >

[jira] [Commented] (SPARK-19701) the `in` operator in pyspark is broken

[ https://issues.apache.org/jira/browse/SPARK-19701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895494#comment-15895494 ] Apache Spark commented on SPARK-19701: -- User 'HyukjinKwon' has created a pull request for this

[jira] [Assigned] (SPARK-19701) the `in` operator in pyspark is broken

[ https://issues.apache.org/jira/browse/SPARK-19701?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19701: Assignee: Apache Spark > the `in` operator in pyspark is broken >

[jira] [Updated] (SPARK-19816) DataFrameCallbackSuite doesn't recover the log level

[ https://issues.apache.org/jira/browse/SPARK-19816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-19816: - Fix Version/s: 2.1.1 > DataFrameCallbackSuite doesn't recover the log level >

[jira] [Updated] (SPARK-19816) DataFrameCallbackSuite doesn't recover the log level

[ https://issues.apache.org/jira/browse/SPARK-19816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li updated SPARK-19816: Fix Version/s: 2.2.0 > DataFrameCallbackSuite doesn't recover the log level >

[jira] [Resolved] (SPARK-19816) DataFrameCallbackSuite doesn't recover the log level

[ https://issues.apache.org/jira/browse/SPARK-19816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-19816. - Resolution: Fixed > DataFrameCallbackSuite doesn't recover the log level >

[jira] [Assigned] (SPARK-19818) SparkR union should check for name consistency of input data frames

[ https://issues.apache.org/jira/browse/SPARK-19818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19818: Assignee: (was: Apache Spark) > SparkR union should check for name consistency of

[jira] [Assigned] (SPARK-19818) SparkR union should check for name consistency of input data frames

[ https://issues.apache.org/jira/browse/SPARK-19818?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19818: Assignee: Apache Spark > SparkR union should check for name consistency of input data

[jira] [Commented] (SPARK-19818) SparkR union should check for name consistency of input data frames

[ https://issues.apache.org/jira/browse/SPARK-19818?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895468#comment-15895468 ] Apache Spark commented on SPARK-19818: -- User 'actuaryzhang' has created a pull request for this

[jira] [Created] (SPARK-19818) SparkR union should check for name consistency of input data frames

Wayne Zhang created SPARK-19818: --- Summary: SparkR union should check for name consistency of input data frames Key: SPARK-19818 URL: https://issues.apache.org/jira/browse/SPARK-19818 Project: Spark

[jira] [Resolved] (SPARK-19804) HiveClientImpl does not work with Hive 2.2.0 metastore

[ https://issues.apache.org/jira/browse/SPARK-19804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Xiao Li resolved SPARK-19804. - Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.2.0 > HiveClientImpl does

[jira] [Comment Edited] (SPARK-19804) HiveClientImpl does not work with Hive 2.2.0 metastore

[ https://issues.apache.org/jira/browse/SPARK-19804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895459#comment-15895459 ] Xiao Li edited comment on SPARK-19804 at 3/4/17 2:47 AM: - Resolved by

[jira] [Commented] (SPARK-19804) HiveClientImpl does not work with Hive 2.2.0 metastore

[ https://issues.apache.org/jira/browse/SPARK-19804?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895459#comment-15895459 ] Xiao Li commented on SPARK-19804: - https://github.com/apache/spark/pull/17154 > HiveClientImpl does not

[jira] [Commented] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895452#comment-15895452 ] Apache Spark commented on SPARK-16845: -- User 'ueshin' has created a pull request for this issue:

[jira] [Commented] (SPARK-16845) org.apache.spark.sql.catalyst.expressions.GeneratedClass$SpecificOrdering" grows beyond 64 KB

[ https://issues.apache.org/jira/browse/SPARK-16845?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895451#comment-15895451 ] Apache Spark commented on SPARK-16845: -- User 'ueshin' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-19659) Fetch big blocks to disk when shuffle-read

[ https://issues.apache.org/jira/browse/SPARK-19659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895411#comment-15895411 ] jin xing edited comment on SPARK-19659 at 3/4/17 2:11 AM: -- [~rxin] Thanks a lot

[jira] [Commented] (SPARK-19659) Fetch big blocks to disk when shuffle-read

[ https://issues.apache.org/jira/browse/SPARK-19659?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895411#comment-15895411 ] jin xing commented on SPARK-19659: -- [~rxin] Thanks a lot for comment. Tracking average size and also the

[jira] [Updated] (SPARK-19817) make it clear that `timeZone` option is a general option in DataFrameReader/Writer

[ https://issues.apache.org/jira/browse/SPARK-19817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-19817: Description: As timezone setting can also affect partition values, it works for all formats, we

[jira] [Updated] (SPARK-19817) make it clear that `timeZone` option is a general option in DataFrameReader/Writer

[ https://issues.apache.org/jira/browse/SPARK-19817?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-19817: Summary: make it clear that `timeZone` option is a general option in DataFrameReader/Writer (was:

[jira] [Created] (SPARK-19817) support timeZone option for all formats in `DataFrameReader/Writer`

Wenchen Fan created SPARK-19817: --- Summary: support timeZone option for all formats in `DataFrameReader/Writer` Key: SPARK-19817 URL: https://issues.apache.org/jira/browse/SPARK-19817 Project: Spark

[jira] [Reopened] (SPARK-18350) Support session local timezone

[ https://issues.apache.org/jira/browse/SPARK-18350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reopened SPARK-18350: - > Support session local timezone > -- > > Key:

[jira] [Resolved] (SPARK-19718) Fix flaky test: org.apache.spark.sql.kafka010.KafkaSourceStressForDontFailOnDataLossSuite: stress test for failOnDataLoss=false

[ https://issues.apache.org/jira/browse/SPARK-19718?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19718. -- Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.2.0 > Fix flaky

[jira] [Assigned] (SPARK-19816) DataFrameCallbackSuite doesn't recover the log level

[ https://issues.apache.org/jira/browse/SPARK-19816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19816: Assignee: Apache Spark (was: Shixiong Zhu) > DataFrameCallbackSuite doesn't recover the

[jira] [Assigned] (SPARK-19816) DataFrameCallbackSuite doesn't recover the log level

[ https://issues.apache.org/jira/browse/SPARK-19816?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19816: Assignee: Shixiong Zhu (was: Apache Spark) > DataFrameCallbackSuite doesn't recover the

[jira] [Commented] (SPARK-19816) DataFrameCallbackSuite doesn't recover the log level

[ https://issues.apache.org/jira/browse/SPARK-19816?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895348#comment-15895348 ] Apache Spark commented on SPARK-19816: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Comment Edited] (SPARK-19811) sparksql 2.1 can not prune hive partition

[ https://issues.apache.org/jira/browse/SPARK-19811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895346#comment-15895346 ] sydt edited comment on SPARK-19811 at 3/4/17 1:04 AM: -- this is not a problem because

[jira] [Commented] (SPARK-19811) sparksql 2.1 can not prune hive partition

[ https://issues.apache.org/jira/browse/SPARK-19811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895346#comment-15895346 ] sydt commented on SPARK-19811: -- this is not a problem because it can be resolved by change partition

[jira] [Created] (SPARK-19816) DataFrameCallbackSuite doesn't recover the log level

Shixiong Zhu created SPARK-19816: Summary: DataFrameCallbackSuite doesn't recover the log level Key: SPARK-19816 URL: https://issues.apache.org/jira/browse/SPARK-19816 Project: Spark Issue

[jira] [Assigned] (SPARK-13446) Spark need to support reading data from Hive 2.0.0 metastore

[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-13446: --- Assignee: Xiao Li > Spark need to support reading data from Hive 2.0.0 metastore >

[jira] [Resolved] (SPARK-13446) Spark need to support reading data from Hive 2.0.0 metastore

[ https://issues.apache.org/jira/browse/SPARK-13446?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-13446. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17061

[jira] [Updated] (SPARK-19348) pyspark.ml.Pipeline gets corrupted under multi threaded use

[ https://issues.apache.org/jira/browse/SPARK-19348?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-19348: -- Fix Version/s: 2.2.0 > pyspark.ml.Pipeline gets corrupted under multi threaded use >

[jira] [Resolved] (SPARK-18350) Support session local timezone

[ https://issues.apache.org/jira/browse/SPARK-18350?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18350. - Resolution: Fixed Assignee: Takuya Ueshin Fix Version/s: 2.2.0 > Support session

[jira] [Assigned] (SPARK-18939) Timezone support in partition values.

[ https://issues.apache.org/jira/browse/SPARK-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan reassigned SPARK-18939: --- Assignee: Takuya Ueshin > Timezone support in partition values. >

[jira] [Resolved] (SPARK-18939) Timezone support in partition values.

[ https://issues.apache.org/jira/browse/SPARK-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-18939. - Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17053

[jira] [Updated] (SPARK-19815) Not orderable should be applied to right key instead of left key

[ https://issues.apache.org/jira/browse/SPARK-19815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Zhan Zhang updated SPARK-19815: --- Summary: Not orderable should be applied to right key instead of left key (was: Not order able

[jira] [Assigned] (SPARK-19815) Not order able should be applied to right key instead of left key

[ https://issues.apache.org/jira/browse/SPARK-19815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19815: Assignee: (was: Apache Spark) > Not order able should be applied to right key instead

[jira] [Commented] (SPARK-19815) Not order able should be applied to right key instead of left key

[ https://issues.apache.org/jira/browse/SPARK-19815?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895250#comment-15895250 ] Apache Spark commented on SPARK-19815: -- User 'zhzhan' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19815) Not order able should be applied to right key instead of left key

[ https://issues.apache.org/jira/browse/SPARK-19815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19815: Assignee: Apache Spark > Not order able should be applied to right key instead of left

[jira] [Created] (SPARK-19815) Not order able should be applied to right key instead of left key

Zhan Zhang created SPARK-19815: -- Summary: Not order able should be applied to right key instead of left key Key: SPARK-19815 URL: https://issues.apache.org/jira/browse/SPARK-19815 Project: Spark

[jira] [Commented] (SPARK-19084) conditional function: field

[ https://issues.apache.org/jira/browse/SPARK-19084?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895154#comment-15895154 ] Apache Spark commented on SPARK-19084: -- User 'vanzin' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19813) maxFilesPerTrigger combo latestFirst may miss old files in combination with maxFileAge in FileStreamSource

[ https://issues.apache.org/jira/browse/SPARK-19813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19813: Assignee: Burak Yavuz (was: Apache Spark) > maxFilesPerTrigger combo latestFirst may

[jira] [Commented] (SPARK-19813) maxFilesPerTrigger combo latestFirst may miss old files in combination with maxFileAge in FileStreamSource

[ https://issues.apache.org/jira/browse/SPARK-19813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15895110#comment-15895110 ] Apache Spark commented on SPARK-19813: -- User 'brkyvz' has created a pull request for this issue:

[jira] [Assigned] (SPARK-19813) maxFilesPerTrigger combo latestFirst may miss old files in combination with maxFileAge in FileStreamSource

[ https://issues.apache.org/jira/browse/SPARK-19813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19813: Assignee: Apache Spark (was: Burak Yavuz) > maxFilesPerTrigger combo latestFirst may

[jira] [Commented] (SPARK-19814) Spark History Server Out Of Memory / Extreme GC

[ https://issues.apache.org/jira/browse/SPARK-19814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894952#comment-15894952 ] Sean Owen commented on SPARK-19814: --- Yes, that already describes further optimizations. I would close

[jira] [Commented] (SPARK-19814) Spark History Server Out Of Memory / Extreme GC

[ https://issues.apache.org/jira/browse/SPARK-19814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894948#comment-15894948 ] Simon King commented on SPARK-19814: Sean, I think that giving more memory only delays the problem,

[jira] [Commented] (SPARK-19814) Spark History Server Out Of Memory / Extreme GC

[ https://issues.apache.org/jira/browse/SPARK-19814?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894942#comment-15894942 ] Sean Owen commented on SPARK-19814: --- I'm not sure if this is a bug. It depends on how much memory you

[jira] [Updated] (SPARK-19814) Spark History Server Out Of Memory / Extreme GC

[ https://issues.apache.org/jira/browse/SPARK-19814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon King updated SPARK-19814: --- Attachment: SparkHistoryCPUandRAM.png Graph showing CPU usage (top) and RSS RAM (bottom). Note the

[jira] [Updated] (SPARK-19813) maxFilesPerTrigger combo latestFirst may miss old files in combination with maxFileAge in FileStreamSource

[ https://issues.apache.org/jira/browse/SPARK-19813?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-19813: - Target Version/s: 2.2.0 > maxFilesPerTrigger combo latestFirst may miss old files in

[jira] [Updated] (SPARK-19814) Spark History Server Out Of Memory / Extreme GC

[ https://issues.apache.org/jira/browse/SPARK-19814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Simon King updated SPARK-19814: --- Description: Spark History Server runs out of memory, gets into GC thrash and eventually becomes

[jira] [Created] (SPARK-19814) Spark History Server Out Of Memory / Extreme GC

Simon King created SPARK-19814: -- Summary: Spark History Server Out Of Memory / Extreme GC Key: SPARK-19814 URL: https://issues.apache.org/jira/browse/SPARK-19814 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-19690) Join a streaming DataFrame with a batch DataFrame may not work

[ https://issues.apache.org/jira/browse/SPARK-19690?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-19690: - Priority: Critical (was: Major) > Join a streaming DataFrame with a batch DataFrame may

[jira] [Updated] (SPARK-18258) Sinks need access to offset representation

[ https://issues.apache.org/jira/browse/SPARK-18258?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust updated SPARK-18258: - Target Version/s: (was: 2.2.0) > Sinks need access to offset representation >

[jira] [Created] (SPARK-19813) maxFilesPerTrigger combo latestFirst may miss old files in combination with maxFileAge in FileStreamSource

Burak Yavuz created SPARK-19813: --- Summary: maxFilesPerTrigger combo latestFirst may miss old files in combination with maxFileAge in FileStreamSource Key: SPARK-19813 URL:

[jira] [Resolved] (SPARK-19774) StreamExecution should call stop() on sources when a stream fails

[ https://issues.apache.org/jira/browse/SPARK-19774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-19774. -- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > StreamExecution

[jira] [Commented] (SPARK-19701) the `in` operator in pyspark is broken

[ https://issues.apache.org/jira/browse/SPARK-19701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894796#comment-15894796 ] Wenchen Fan commented on SPARK-19701: - let's remove it then > the `in` operator in pyspark is broken

[jira] [Commented] (SPARK-18085) Better History Server scalability for many / large applications

[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894768#comment-15894768 ] Marcelo Vanzin commented on SPARK-18085: bq. does this local db will delete the data as

[jira] [Commented] (SPARK-18278) Support native submission of spark jobs to a kubernetes cluster

[ https://issues.apache.org/jira/browse/SPARK-18278?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894701#comment-15894701 ] Apache Spark commented on SPARK-18278: -- User 'erikerlandson' has created a pull request for this

[jira] [Resolved] (SPARK-19710) Test Failures in SQLQueryTests on big endian platforms

[ https://issues.apache.org/jira/browse/SPARK-19710?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19710. --- Resolution: Fixed Assignee: Pete Robbins Fix Version/s: 2.2.0 > Test

[jira] [Commented] (SPARK-19812) YARN shuffle service fails to relocate recovery DB directories

[ https://issues.apache.org/jira/browse/SPARK-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894584#comment-15894584 ] Thomas Graves commented on SPARK-19812: --- note that it will go ahead and start using the recovery

[jira] [Updated] (SPARK-19812) YARN shuffle service fails to relocate recovery DB directories

[ https://issues.apache.org/jira/browse/SPARK-19812?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Thomas Graves updated SPARK-19812: -- Summary: YARN shuffle service fails to relocate recovery DB directories (was: YARN shuffle

[jira] [Created] (SPARK-19812) YARN shuffle service fix moving recovery DB directories

Thomas Graves created SPARK-19812: - Summary: YARN shuffle service fix moving recovery DB directories Key: SPARK-19812 URL: https://issues.apache.org/jira/browse/SPARK-19812 Project: Spark

[jira] [Assigned] (SPARK-18389) Disallow cyclic view reference

[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18389: Assignee: (was: Apache Spark) > Disallow cyclic view reference >

[jira] [Assigned] (SPARK-18389) Disallow cyclic view reference

[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18389: Assignee: Apache Spark > Disallow cyclic view reference > --

[jira] [Commented] (SPARK-18389) Disallow cyclic view reference

[ https://issues.apache.org/jira/browse/SPARK-18389?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894571#comment-15894571 ] Apache Spark commented on SPARK-18389: -- User 'jiangxb1987' has created a pull request for this

[jira] [Resolved] (SPARK-19758) Casting string to timestamp in inline table definition fails with AnalysisException

[ https://issues.apache.org/jira/browse/SPARK-19758?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-19758. --- Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.2.0 >

[jira] [Commented] (SPARK-15797) To expose groupingSets for DataFrame

[ https://issues.apache.org/jira/browse/SPARK-15797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894509#comment-15894509 ] Pau Tallada Crespí commented on SPARK-15797: Hi, any progress on this? :P > To expose

[jira] [Commented] (SPARK-19503) Execution Plan Optimizer: avoid sort or shuffle when it does not change end result such as df.sort(...).count()

[ https://issues.apache.org/jira/browse/SPARK-19503?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894496#comment-15894496 ] Herman van Hovell commented on SPARK-19503: --- We do not prune local sorts yet; however a user

[jira] [Updated] (SPARK-19764) Executors hang with supposedly running task that are really finished.

[ https://issues.apache.org/jira/browse/SPARK-19764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Ari Gesher updated SPARK-19764: --- There's nothing output in the driver. It just appears hung. > Executors hang with supposedly running

[jira] [Commented] (SPARK-16599) java.util.NoSuchElementException: None.get at at org.apache.spark.storage.BlockInfoManager.releaseAllLocksForTask(BlockInfoManager.scala:343)

[ https://issues.apache.org/jira/browse/SPARK-16599?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894402#comment-15894402 ] Jakub Dubovsky commented on SPARK-16599: [~srowen] I tried to create a custom spark build with

[jira] [Assigned] (SPARK-19810) Remove support for Scala 2.10

[ https://issues.apache.org/jira/browse/SPARK-19810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19810: Assignee: Apache Spark (was: Sean Owen) > Remove support for Scala 2.10 >

[jira] [Assigned] (SPARK-19810) Remove support for Scala 2.10

[ https://issues.apache.org/jira/browse/SPARK-19810?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19810: Assignee: Sean Owen (was: Apache Spark) > Remove support for Scala 2.10 >

[jira] [Commented] (SPARK-19810) Remove support for Scala 2.10

[ https://issues.apache.org/jira/browse/SPARK-19810?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894267#comment-15894267 ] Apache Spark commented on SPARK-19810: -- User 'srowen' has created a pull request for this issue:

[jira] [Created] (SPARK-19811) sparksql 2.1 can not prune hive partition

sydt created SPARK-19811: Summary: sparksql 2.1 can not prune hive partition Key: SPARK-19811 URL: https://issues.apache.org/jira/browse/SPARK-19811 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-19810) Remove support for Scala 2.10

Sean Owen created SPARK-19810: - Summary: Remove support for Scala 2.10 Key: SPARK-19810 URL: https://issues.apache.org/jira/browse/SPARK-19810 Project: Spark Issue Type: Task

[jira] [Resolved] (SPARK-16773) Post Spark 2.0 deprecation & warnings cleanup

[ https://issues.apache.org/jira/browse/SPARK-16773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16773. --- Resolution: Done > Post Spark 2.0 deprecation & warnings cleanup >

[jira] [Commented] (SPARK-16775) Reduce internal warnings from deprecated accumulator API

[ https://issues.apache.org/jira/browse/SPARK-16775?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894209#comment-15894209 ] Sean Owen commented on SPARK-16775: --- Are there still areas where uses of deprecated accumulators can be

[jira] [Updated] (SPARK-16775) Reduce internal warnings from deprecated accumulator API

[ https://issues.apache.org/jira/browse/SPARK-16775?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-16775: -- Affects Version/s: 2.1.0 Issue Type: Improvement (was: Sub-task) Parent:

[jira] [Comment Edited] (SPARK-19807) Add reason for cancellation when a stage is killed using web UI

[ https://issues.apache.org/jira/browse/SPARK-19807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894185#comment-15894185 ] Genmao Yu edited comment on SPARK-19807 at 3/3/17 11:35 AM:

[jira] [Commented] (SPARK-19807) Add reason for cancellation when a stage is killed using web UI

[ https://issues.apache.org/jira/browse/SPARK-19807?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894185#comment-15894185 ] Genmao Yu commented on SPARK-19807: ---

[jira] [Resolved] (SPARK-19782) Spark query available cores from application

[ https://issues.apache.org/jira/browse/SPARK-19782?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19782. --- Resolution: Not A Problem > Spark query available cores from application >

[jira] [Commented] (SPARK-19701) the `in` operator in pyspark is broken

[ https://issues.apache.org/jira/browse/SPARK-19701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894159#comment-15894159 ] Hyukjin Kwon commented on SPARK-19701: -- I was thinking a way to work around (e.g., hijacking..) but

[jira] [Commented] (SPARK-19701) the `in` operator in pyspark is broken

[ https://issues.apache.org/jira/browse/SPARK-19701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894155#comment-15894155 ] Hyukjin Kwon commented on SPARK-19701: -- [~cloud_fan], I took a look this for my curiosity. It seems

[jira] [Assigned] (SPARK-19801) Remove JDK7 from Travis CI

[ https://issues.apache.org/jira/browse/SPARK-19801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-19801: - Assignee: Dongjoon Hyun > Remove JDK7 from Travis CI > -- > >

[jira] [Resolved] (SPARK-19801) Remove JDK7 from Travis CI

[ https://issues.apache.org/jira/browse/SPARK-19801?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19801. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17143

[jira] [Updated] (SPARK-19792) In the Master Page,the column named “Memory per Node” ,I think it is not all right

[ https://issues.apache.org/jira/browse/SPARK-19792?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19792: -- Priority: Trivial (was: Major) Hm, I'm honestly not sure. Does this refer to the memory allocated to

[jira] [Assigned] (SPARK-19797) ML pipelines document error

[ https://issues.apache.org/jira/browse/SPARK-19797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-19797: - Assignee: Zhe Sun > ML pipelines document error > --- > >

[jira] [Resolved] (SPARK-19797) ML pipelines document error

[ https://issues.apache.org/jira/browse/SPARK-19797?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19797. --- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 Issue resolved by pull

[jira] [Resolved] (SPARK-19339) StatFunctions.multipleApproxQuantiles can give NoSuchElementException: next on empty iterator

[ https://issues.apache.org/jira/browse/SPARK-19339?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19339. --- Resolution: Duplicate > StatFunctions.multipleApproxQuantiles can give NoSuchElementException: next

[jira] [Resolved] (SPARK-19739) SparkHadoopUtil.appendS3AndSparkHadoopConfigurations to propagate full set of AWS env vars

[ https://issues.apache.org/jira/browse/SPARK-19739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19739. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 17080

[jira] [Assigned] (SPARK-19739) SparkHadoopUtil.appendS3AndSparkHadoopConfigurations to propagate full set of AWS env vars

[ https://issues.apache.org/jira/browse/SPARK-19739?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen reassigned SPARK-19739: - Assignee: Genmao Yu > SparkHadoopUtil.appendS3AndSparkHadoopConfigurations to propagate full

[jira] [Resolved] (SPARK-19794) Release HDFS Client after read/write checkpoint

[ https://issues.apache.org/jira/browse/SPARK-19794?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19794. --- Resolution: Not A Problem See PR > Release HDFS Client after read/write checkpoint >

[jira] [Commented] (SPARK-19808) About the default blocking arg in unpersist

[ https://issues.apache.org/jira/browse/SPARK-19808?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894087#comment-15894087 ] Sean Owen commented on SPARK-19808: --- (Maybe you can rewrite this as a proposed change rather than

[jira] [Commented] (SPARK-18085) Better History Server scalability for many / large applications

[ https://issues.apache.org/jira/browse/SPARK-18085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894074#comment-15894074 ] DjvuLee commented on SPARK-18085: - [~vanzin] This is a nice design. There is not much information about

[jira] [Commented] (SPARK-19257) The type of CatalogStorageFormat.locationUri should be java.net.URI instead of String

[ https://issues.apache.org/jira/browse/SPARK-19257?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15894039#comment-15894039 ] Apache Spark commented on SPARK-19257: -- User 'windpiger' has created a pull request for this issue:

[jira] [Created] (SPARK-19809) NullPointerException on empty ORC file

Michał Dawid created SPARK-19809: Summary: NullPointerException on empty ORC file Key: SPARK-19809 URL: https://issues.apache.org/jira/browse/SPARK-19809 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-19808) About the default blocking arg in unpersist

zhengruifeng created SPARK-19808: Summary: About the default blocking arg in unpersist Key: SPARK-19808 URL: https://issues.apache.org/jira/browse/SPARK-19808 Project: Spark Issue Type:

  1   2   >