[jira] [Updated] (SPARK-17256) spark-submit.cmd cannot work if path has space and cut off double-quoted arguments

2016-08-25 Thread Quanmao Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quanmao Liu updated SPARK-17256: Remaining Estimate: 120h (was: 5m) Original Estimate: 120h (was: 5m) > spark-submit.cmd cann

[jira] [Updated] (SPARK-17256) spark-submit.cmd cannot work if path has space and cut off double-quoted arguments

2016-08-25 Thread Quanmao Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Quanmao Liu updated SPARK-17256: Description: The key problem is { cmd /V /E /C "~%dp0spark-xxx.cmd" } cannot accept arguments whi

[jira] [Assigned] (SPARK-17256) spark-submit.cmd cannot work if path has space and cut off double-quoted arguments

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17256: Assignee: (was: Apache Spark) > spark-submit.cmd cannot work if path has space and cut

[jira] [Assigned] (SPARK-17256) spark-submit.cmd cannot work if path has space and cut off double-quoted arguments

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17256: Assignee: Apache Spark > spark-submit.cmd cannot work if path has space and cut off double

[jira] [Commented] (SPARK-17256) spark-submit.cmd cannot work if path has space and cut off double-quoted arguments

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438536#comment-15438536 ] Apache Spark commented on SPARK-17256: -- User 'qualiu' has created a pull request for

[jira] [Commented] (SPARK-17256) spark-submit.cmd cannot work if path has space and cut off double-quoted arguments

2016-08-25 Thread Quanmao Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17256?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438534#comment-15438534 ] Quanmao Liu commented on SPARK-17256: - I've pushed a pull request : https://github.co

[jira] [Created] (SPARK-17256) spark-submit.cmd cannot work if path has space and cut off double-quoted arguments

2016-08-25 Thread Quanmao Liu (JIRA)
Quanmao Liu created SPARK-17256: --- Summary: spark-submit.cmd cannot work if path has space and cut off double-quoted arguments Key: SPARK-17256 URL: https://issues.apache.org/jira/browse/SPARK-17256 Proj

[jira] [Created] (SPARK-17255) Spark queries inside Futures occasionally fail due to missing class definitions

2016-08-25 Thread Jukka Vaaramaki (JIRA)
Jukka Vaaramaki created SPARK-17255: --- Summary: Spark queries inside Futures occasionally fail due to missing class definitions Key: SPARK-17255 URL: https://issues.apache.org/jira/browse/SPARK-17255

[jira] [Commented] (SPARK-17187) Support using arbitrary Java object as internal aggregation buffer object

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17187?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438525#comment-15438525 ] Apache Spark commented on SPARK-17187: -- User 'cloud-fan' has created a pull request

[jira] [Commented] (SPARK-12394) Support writing out pre-hash-partitioned data and exploit that in join optimizations to avoid shuffle (i.e. bucketing in Hive)

2016-08-25 Thread Alexander Tronchin-James (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438461#comment-15438461 ] Alexander Tronchin-James commented on SPARK-12394: -- Awesome news Tejas!

[jira] [Comment Edited] (SPARK-14927) DataFrame. saveAsTable creates RDD partitions but not Hive partitions

2016-08-25 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438457#comment-15438457 ] Xin Wu edited comment on SPARK-14927 at 8/26/16 4:46 AM: - [~smile

[jira] [Commented] (SPARK-14927) DataFrame. saveAsTable creates RDD partitions but not Hive partitions

2016-08-25 Thread Xin Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438457#comment-15438457 ] Xin Wu commented on SPARK-14927: [~smilegator] Do you think what you are working on regar

[jira] [Resolved] (SPARK-17242) Update links of external dstream projects

2016-08-25 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-17242. - Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Update links of external

[jira] [Commented] (SPARK-16998) select($"column1", explode($"column2")) is extremely slow

2016-08-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438429#comment-15438429 ] Takeshi Yamamuro commented on SPARK-16998: -- If no problem, I'll pick up the pr.

[jira] [Commented] (SPARK-12394) Support writing out pre-hash-partitioned data and exploit that in join optimizations to avoid shuffle (i.e. bucketing in Hive)

2016-08-25 Thread Tejas Patil (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438423#comment-15438423 ] Tejas Patil commented on SPARK-12394: - [~alex.n.ja...@gmail.com] [~kotim...@gmail.com

[jira] [Created] (SPARK-17254) Filter operator should have “stop if false” semantics for sorted data

2016-08-25 Thread Tejas Patil (JIRA)
Tejas Patil created SPARK-17254: --- Summary: Filter operator should have “stop if false” semantics for sorted data Key: SPARK-17254 URL: https://issues.apache.org/jira/browse/SPARK-17254 Project: Spark

[jira] [Created] (SPARK-17253) Left join where ON clause does not reference the right table produces analysis error

2016-08-25 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17253: -- Summary: Left join where ON clause does not reference the right table produces analysis error Key: SPARK-17253 URL: https://issues.apache.org/jira/browse/SPARK-17253 Proj

[jira] [Created] (SPARK-17252) Performing arithmetic in VALUES can lead to ClassCastException / MatchErrors during query parsing

2016-08-25 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17252: -- Summary: Performing arithmetic in VALUES can lead to ClassCastException / MatchErrors during query parsing Key: SPARK-17252 URL: https://issues.apache.org/jira/browse/SPARK-17252

[jira] [Created] (SPARK-17251) "ClassCastException: OuterReference cannot be cast to NamedExpression" for correlated subquery on the RHS of an IN operator

2016-08-25 Thread Josh Rosen (JIRA)
Josh Rosen created SPARK-17251: -- Summary: "ClassCastException: OuterReference cannot be cast to NamedExpression" for correlated subquery on the RHS of an IN operator Key: SPARK-17251 URL: https://issues.apache.org/ji

[jira] [Commented] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

2016-08-25 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438331#comment-15438331 ] Sun Rui commented on SPARK-13525: - sorry, due to missing log information in such point, i

[jira] [Updated] (SPARK-16283) Implement percentile_approx SQL function

2016-08-25 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16283: --- Assignee: (was: Sean Zhong) > Implement percentile_approx SQL function >

[jira] [Commented] (SPARK-12394) Support writing out pre-hash-partitioned data and exploit that in join optimizations to avoid shuffle (i.e. bucketing in Hive)

2016-08-25 Thread Darren Fu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12394?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438317#comment-15438317 ] Darren Fu commented on SPARK-12394: --- Any luck to see this feature implemented in v2.0?

[jira] [Commented] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

2016-08-25 Thread Sun Rui (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438316#comment-15438316 ] Sun Rui commented on SPARK-13525: - I checked the code and realized that SocketTimeoutExce

[jira] [Updated] (SPARK-16283) Implement percentile_approx SQL function

2016-08-25 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16283?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-16283: --- Assignee: Sean Zhong > Implement percentile_approx SQL function > ---

[jira] [Updated] (SPARK-16948) Support empty orc table when converting hive serde table to data source table

2016-08-25 Thread Rajesh Balamohan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Rajesh Balamohan updated SPARK-16948: - Summary: Support empty orc table when converting hive serde table to data source table (

[jira] [Commented] (SPARK-17250) Remove HiveClient and setCurrentDatabase from HiveSessionCatalog

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17250?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438233#comment-15438233 ] Apache Spark commented on SPARK-17250: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-17250) Remove HiveClient and setCurrentDatabase from HiveSessionCatalog

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17250?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17250: Assignee: Apache Spark > Remove HiveClient and setCurrentDatabase from HiveSessionCatalog

[jira] [Created] (SPARK-17250) Remove HiveClient and setCurrentDatabase from HiveSessionCatalog

2016-08-25 Thread Xiao Li (JIRA)
Xiao Li created SPARK-17250: --- Summary: Remove HiveClient and setCurrentDatabase from HiveSessionCatalog Key: SPARK-17250 URL: https://issues.apache.org/jira/browse/SPARK-17250 Project: Spark Issue

[jira] [Updated] (SPARK-17212) TypeCoercion support widening conversion between DateType and TimestampType

2016-08-25 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-17212: Assignee: Hyukjin Kwon > TypeCoercion support widening conversion between DateType and TimestampTyp

[jira] [Resolved] (SPARK-17212) TypeCoercion support widening conversion between DateType and TimestampType

2016-08-25 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17212?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-17212. - Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14786 [https://githu

[jira] [Created] (SPARK-17249) java.lang.IllegalStateException: Did not find registered driver with class org.apache.spark.sql.execution.datasources.jdbc.DriverWrapper

2016-08-25 Thread Graeme Edwards (JIRA)
Graeme Edwards created SPARK-17249: -- Summary: java.lang.IllegalStateException: Did not find registered driver with class org.apache.spark.sql.execution.datasources.jdbc.DriverWrapper Key: SPARK-17249 URL: https:

[jira] [Created] (SPARK-17248) Add native Scala enum support to Dataset Encoders

2016-08-25 Thread Silvio Fiorito (JIRA)
Silvio Fiorito created SPARK-17248: -- Summary: Add native Scala enum support to Dataset Encoders Key: SPARK-17248 URL: https://issues.apache.org/jira/browse/SPARK-17248 Project: Spark Issue T

[jira] [Assigned] (SPARK-17246) Support BigDecimal literal parsing

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17246: Assignee: Herman van Hovell (was: Apache Spark) > Support BigDecimal literal parsing > --

[jira] [Assigned] (SPARK-17246) Support BigDecimal literal parsing

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17246?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17246: Assignee: Apache Spark (was: Herman van Hovell) > Support BigDecimal literal parsing > --

[jira] [Commented] (SPARK-17246) Support BigDecimal literal parsing

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17246?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438152#comment-15438152 ] Apache Spark commented on SPARK-17246: -- User 'hvanhovell' has created a pull request

[jira] [Updated] (SPARK-17187) Support using arbitrary Java object as internal aggregation buffer object

2016-08-25 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17187: - Assignee: Sean Zhong > Support using arbitrary Java object as internal aggregation buffer object > --

[jira] [Resolved] (SPARK-17187) Support using arbitrary Java object as internal aggregation buffer object

2016-08-25 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17187?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-17187. -- Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 14753 [https://github.com/

[jira] [Resolved] (SPARK-17231) Avoid building debug or trace log messages unless the respective log level is enabled

2016-08-25 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-17231. -- Resolution: Fixed Assignee: Michael Allman Fix Version/s: 2.1.0

[jira] [Comment Edited] (SPARK-7007) Add metrics source for ExecutorAllocationManager to expose internal status

2016-08-25 Thread Erik Erlandson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-7007?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437325#comment-15437325 ] Erik Erlandson edited comment on SPARK-7007 at 8/25/16 11:25 PM: ---

[jira] [Commented] (SPARK-17157) Add multiclass logistic regression SparkR Wrapper

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17157?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15438054#comment-15438054 ] Apache Spark commented on SPARK-17157: -- User 'wangmiao1981' has created a pull reque

[jira] [Assigned] (SPARK-17157) Add multiclass logistic regression SparkR Wrapper

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17157: Assignee: (was: Apache Spark) > Add multiclass logistic regression SparkR Wrapper > --

[jira] [Resolved] (SPARK-17240) SparkConf is Serializable but contains a non-serializable field

2016-08-25 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17240?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-17240. Resolution: Fixed Assignee: Marcelo Vanzin Fix Version/s: 2.1.0 > SparkConf

[jira] [Assigned] (SPARK-17157) Add multiclass logistic regression SparkR Wrapper

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17157?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17157: Assignee: Apache Spark > Add multiclass logistic regression SparkR Wrapper > -

[jira] [Closed] (SPARK-16627) --jars doesn't work in Mesos mode

2016-08-25 Thread Michael Gummelt (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Gummelt closed SPARK-16627. --- Resolution: Won't Fix > --jars doesn't work in Mesos mode > -

[jira] [Commented] (SPARK-14927) DataFrame. saveAsTable creates RDD partitions but not Hive partitions

2016-08-25 Thread Corentin Kerisit (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14927?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437861#comment-15437861 ] Corentin Kerisit commented on SPARK-14927: -- Any guideline on how we could help g

[jira] [Assigned] (SPARK-17247) when fall back to hdfs is enabled for stats calculation, the hdfs listing and size calcuation should be terminated as soon as total size > broadcast threshold

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17247: Assignee: Apache Spark > when fall back to hdfs is enabled for stats calculation, the hdfs

[jira] [Assigned] (SPARK-17247) when fall back to hdfs is enabled for stats calculation, the hdfs listing and size calcuation should be terminated as soon as total size > broadcast threshold

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17247?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17247: Assignee: (was: Apache Spark) > when fall back to hdfs is enabled for stats calculatio

[jira] [Commented] (SPARK-17247) when fall back to hdfs is enabled for stats calculation, the hdfs listing and size calcuation should be terminated as soon as total size > broadcast threshold

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17247?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437860#comment-15437860 ] Apache Spark commented on SPARK-17247: -- User 'Parth-Brahmbhatt' has created a pull r

[jira] [Commented] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

2016-08-25 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437856#comment-15437856 ] Shivaram Venkataraman commented on SPARK-13525: --- Yeah this is related but a

[jira] [Created] (SPARK-17247) when fall back to hdfs is enabled for stats calculation, the hdfs listing and size calcuation should be terminated as soon as total size > broadcast threshold

2016-08-25 Thread Parth Brahmbhatt (JIRA)
Parth Brahmbhatt created SPARK-17247: Summary: when fall back to hdfs is enabled for stats calculation, the hdfs listing and size calcuation should be terminated as soon as total size > broadcast threshold Key: SPARK-17247

[jira] [Resolved] (SPARK-17205) Literal.sql does not properly convert NaN and Infinity literals

2016-08-25 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17205?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17205. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.1 > Literal.sql

[jira] [Created] (SPARK-17246) Support BigDecimal literal parsing

2016-08-25 Thread Herman van Hovell (JIRA)
Herman van Hovell created SPARK-17246: - Summary: Support BigDecimal literal parsing Key: SPARK-17246 URL: https://issues.apache.org/jira/browse/SPARK-17246 Project: Spark Issue Type: Impr

[jira] [Commented] (SPARK-17245) NPE thrown by ClientWrapper.conf

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17245?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437809#comment-15437809 ] Apache Spark commented on SPARK-17245: -- User 'yhuai' has created a pull request for

[jira] [Assigned] (SPARK-17245) NPE thrown by ClientWrapper.conf

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17245: Assignee: (was: Apache Spark) > NPE thrown by ClientWrapper.conf > ---

[jira] [Assigned] (SPARK-17245) NPE thrown by ClientWrapper.conf

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17245: Assignee: Apache Spark > NPE thrown by ClientWrapper.conf > --

[jira] [Updated] (SPARK-16700) StructType doesn't accept Python dicts anymore

2016-08-25 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16700?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-16700: - Fix Version/s: 2.0.1 > StructType doesn't accept Python dicts anymore > -

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-25 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437764#comment-15437764 ] Alex Bozarth commented on SPARK-17243: -- Thanks, that'll help when I look into it >

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-25 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437768#comment-15437768 ] Alex Bozarth commented on SPARK-17243: -- Sorry, my misunderstanding of your problem,

[jira] [Commented] (SPARK-11085) Add support for HTTP proxy

2016-08-25 Thread SURESH CHAGANTI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11085?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437727#comment-15437727 ] SURESH CHAGANTI commented on SPARK-11085: - Hi All, I have made the code changes t

[jira] [Commented] (SPARK-17241) SparkR spark.glm should have configurable regularization parameter

2016-08-25 Thread Junyang Qian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437717#comment-15437717 ] Junyang Qian commented on SPARK-17241: -- I'll take a closer look and see if we can ad

[jira] [Updated] (SPARK-17245) NPE thrown by ClientWrapper.conf

2016-08-25 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17245?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-17245: - Summary: NPE thrown by ClientWrapper.conf (was: NPE thrown by ClientWrapper ) > NPE thrown by ClientWra

[jira] [Created] (SPARK-17245) NPE thrown by ClientWrapper

2016-08-25 Thread Yin Huai (JIRA)
Yin Huai created SPARK-17245: Summary: NPE thrown by ClientWrapper Key: SPARK-17245 URL: https://issues.apache.org/jira/browse/SPARK-17245 Project: Spark Issue Type: Bug Components: SQ

[jira] [Resolved] (SPARK-17229) Postgres JDBC dialect should not widen float and short types during reads

2016-08-25 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Herman van Hovell resolved SPARK-17229. --- Resolution: Fixed Fix Version/s: 2.1.0 > Postgres JDBC dialect should not wide

[jira] [Commented] (SPARK-17241) SparkR spark.glm should have configurable regularization parameter

2016-08-25 Thread Junyang Qian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437692#comment-15437692 ] Junyang Qian commented on SPARK-17241: -- [~shivaram] It seems that spark has it for l

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-25 Thread Gang Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437691#comment-15437691 ] Gang Wu commented on SPARK-17243: - This doesn't work. This is for the cache of WEB UIs no

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-25 Thread Gang Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437688#comment-15437688 ] Gang Wu commented on SPARK-17243: - Hi Alex, I think in Spark 1.5 history server obtains a

[jira] [Comment Edited] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-25 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437682#comment-15437682 ] Alex Bozarth edited comment on SPARK-17243 at 8/25/16 9:13 PM:

[jira] [Comment Edited] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-25 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437682#comment-15437682 ] Alex Bozarth edited comment on SPARK-17243 at 8/25/16 9:13 PM:

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-25 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437682#comment-15437682 ] Alex Bozarth commented on SPARK-17243: -- [~wgtmac] until this is fixed you can limit

[jira] [Commented] (SPARK-17147) Spark Streaming Kafka 0.10 Consumer Can't Handle Non-consecutive Offsets

2016-08-25 Thread Sean McKibben (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17147?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437672#comment-15437672 ] Sean McKibben commented on SPARK-17147: --- I tried Robert's changes, but the performa

[jira] [Comment Edited] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-25 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437668#comment-15437668 ] Alex Bozarth edited comment on SPARK-17243 at 8/25/16 9:05 PM:

[jira] [Updated] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-17243: -- Priority: Major (was: Blocker) Yes, should not be assigned as a Blocker. > Spark 2.0 history server s

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-25 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437668#comment-15437668 ] Alex Bozarth commented on SPARK-17243: -- I'm not sure I agree that this should be a b

[jira] [Commented] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-25 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437653#comment-15437653 ] Sean Owen commented on SPARK-17243: --- Related, but not identical: https://issues.apache.

[jira] [Assigned] (SPARK-17244) Joins should not pushdown non-deterministic conditions

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17244: Assignee: Apache Spark > Joins should not pushdown non-deterministic conditions >

[jira] [Commented] (SPARK-17244) Joins should not pushdown non-deterministic conditions

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17244?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437649#comment-15437649 ] Apache Spark commented on SPARK-17244: -- User 'sameeragarwal' has created a pull requ

[jira] [Assigned] (SPARK-17244) Joins should not pushdown non-deterministic conditions

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17244: Assignee: (was: Apache Spark) > Joins should not pushdown non-deterministic conditions

[jira] [Commented] (SPARK-17163) Merge MLOR into a single LOR interface

2016-08-25 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437642#comment-15437642 ] DB Tsai commented on SPARK-17163: - Maybe we can store them as the same format, and for sc

[jira] [Created] (SPARK-17244) Joins should not pushdown non-deterministic conditions

2016-08-25 Thread Sameer Agarwal (JIRA)
Sameer Agarwal created SPARK-17244: -- Summary: Joins should not pushdown non-deterministic conditions Key: SPARK-17244 URL: https://issues.apache.org/jira/browse/SPARK-17244 Project: Spark Is

[jira] [Updated] (SPARK-17243) Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-25 Thread Gang Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Wu updated SPARK-17243: Summary: Spark 2.0 history server summary page gets stuck at "loading history summary" with 10K+ applicatio

[jira] [Created] (SPARK-17243) Spark history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-25 Thread Gang Wu (JIRA)
Gang Wu created SPARK-17243: --- Summary: Spark history server summary page gets stuck at "loading history summary" with 10K+ application history Key: SPARK-17243 URL: https://issues.apache.org/jira/browse/SPARK-17243

[jira] [Updated] (SPARK-17243) Spark history server summary page gets stuck at "loading history summary" with 10K+ application history

2016-08-25 Thread Gang Wu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17243?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Gang Wu updated SPARK-17243: Description: The summary page of Spark 2.0 history server web UI keep displaying "Loading history summary.

[jira] [Commented] (SPARK-14241) Output of monotonically_increasing_id lacks stable relation with rows of DataFrame

2016-08-25 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437616#comment-15437616 ] Nicholas Chammas commented on SPARK-14241: -- [~marmbrus] - Would it be tough to m

[jira] [Commented] (SPARK-17163) Merge MLOR into a single LOR interface

2016-08-25 Thread Seth Hendrickson (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437606#comment-15437606 ] Seth Hendrickson commented on SPARK-17163: -- If we store the binomial case with 2

[jira] [Commented] (SPARK-13525) SparkR: java.net.SocketTimeoutException: Accept timed out when running any dataframe function

2016-08-25 Thread Arihanth Jain (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13525?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437605#comment-15437605 ] Arihanth Jain commented on SPARK-13525: --- [~shivaram] I have similar trace, please s

[jira] [Comment Edited] (SPARK-16998) select($"column1", explode($"column2")) is extremely slow

2016-08-25 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437598#comment-15437598 ] Herman van Hovell edited comment on SPARK-16998 at 8/25/16 8:27 PM: ---

[jira] [Commented] (SPARK-16998) select($"column1", explode($"column2")) is extremely slow

2016-08-25 Thread Herman van Hovell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16998?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437598#comment-15437598 ] Herman van Hovell commented on SPARK-16998: --- I still have a code generation PR

[jira] [Issue Comment Deleted] (SPARK-11085) Add support for HTTP proxy

2016-08-25 Thread SURESH CHAGANTI (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11085?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] SURESH CHAGANTI updated SPARK-11085: Comment: was deleted (was: The following Script accepts the "--proxy_host_port" argument

[jira] [Commented] (SPARK-17241) SparkR spark.glm should have configurable regularization parameter

2016-08-25 Thread Xin Ren (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437588#comment-15437588 ] Xin Ren commented on SPARK-17241: - I can work on this one :) > SparkR spark.glm should h

[jira] [Assigned] (SPARK-17242) Update links of external dstream projects

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17242: Assignee: Apache Spark (was: Shixiong Zhu) > Update links of external dstream projects >

[jira] [Commented] (SPARK-17242) Update links of external dstream projects

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17242?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437584#comment-15437584 ] Apache Spark commented on SPARK-17242: -- User 'zsxwing' has created a pull request fo

[jira] [Assigned] (SPARK-17242) Update links of external dstream projects

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17242: Assignee: Shixiong Zhu (was: Apache Spark) > Update links of external dstream projects >

[jira] [Created] (SPARK-17242) Update links of external dstream projects

2016-08-25 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-17242: Summary: Update links of external dstream projects Key: SPARK-17242 URL: https://issues.apache.org/jira/browse/SPARK-17242 Project: Spark Issue Type: Documen

[jira] [Commented] (SPARK-14586) SparkSQL doesn't parse decimal like Hive

2016-08-25 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14586?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437548#comment-15437548 ] Dongjoon Hyun commented on SPARK-14586: --- Hi, [~stephane.maa...@gmail.com] and [~tsu

[jira] [Commented] (SPARK-16501) spark.mesos.secret exposed on UI and command line

2016-08-25 Thread Alex Bozarth (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16501?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437539#comment-15437539 ] Alex Bozarth commented on SPARK-16501: -- The first problem (web ui) was fixed by htt

[jira] [Comment Edited] (SPARK-17163) Merge MLOR into a single LOR interface

2016-08-25 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437535#comment-15437535 ] DB Tsai edited comment on SPARK-17163 at 8/25/16 7:58 PM: -- BTW,

[jira] [Commented] (SPARK-17163) Merge MLOR into a single LOR interface

2016-08-25 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437535#comment-15437535 ] DB Tsai commented on SPARK-17163: - BTW, having the name as `coefficientMatrix` doesn't lo

[jira] [Commented] (SPARK-17241) SparkR spark.glm should have configurable regularization parameter

2016-08-25 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17241?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437536#comment-15437536 ] Shivaram Venkataraman commented on SPARK-17241: --- +1 - This would be good to

[jira] [Commented] (SPARK-17163) Merge MLOR into a single LOR interface

2016-08-25 Thread DB Tsai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17163?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437531#comment-15437531 ] DB Tsai commented on SPARK-17163: - Why do we need `_intercept`? Also, for intercept, we m

[jira] [Commented] (SPARK-17240) SparkConf is Serializable but contains a non-serializable field

2016-08-25 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17240?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15437524#comment-15437524 ] Apache Spark commented on SPARK-17240: -- User 'vanzin' has created a pull request for

  1   2   3   >