[jira] [Resolved] (SPARK-12872) Support to specify the option for compression codec for JSON datasource.

[ https://issues.apache.org/jira/browse/SPARK-12872?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12872. - Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.0.0 > Support to specif

[jira] [Commented] (SPARK-12968) Implement command to set current database

[ https://issues.apache.org/jira/browse/SPARK-12968?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113631#comment-15113631 ] Reynold Xin commented on SPARK-12968: - cc [~viirya] / [~hvanhovell] this is probably

[jira] [Created] (SPARK-12968) Implement command to set current database

Reynold Xin created SPARK-12968: --- Summary: Implement command to set current database Key: SPARK-12968 URL: https://issues.apache.org/jira/browse/SPARK-12968 Project: Spark Issue Type: Sub-task

[jira] [Closed] (SPARK-12151) Improve PySpark MLLib prediction performance when using pickled vectors

[ https://issues.apache.org/jira/browse/SPARK-12151?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] holdenk closed SPARK-12151. --- Resolution: Not A Problem Checked the models, all of the ones not doing these were doing there prediction in

[jira] [Commented] (SPARK-12904) Strength reduction for integer/decimal comparisons

[ https://issues.apache.org/jira/browse/SPARK-12904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113625#comment-15113625 ] Apache Spark commented on SPARK-12904: -- User 'rxin' has created a pull request for t

[jira] [Updated] (SPARK-12950) Improve performance of BytesToBytesMap

[ https://issues.apache.org/jira/browse/SPARK-12950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12950: --- Description: When benchmark generated aggregate with grouping keys, the profiling show that lookup i

[jira] [Resolved] (SPARK-5293) Enable Spark user applications to use different versions of Akka

[ https://issues.apache.org/jira/browse/SPARK-5293?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-5293. Resolution: Fixed Fix Version/s: 2.0.0 > Enable Spark user applications to use different vers

[jira] [Updated] (SPARK-11095) Simplify Netty RPC implementation by using a separate thread pool for each endpoint

[ https://issues.apache.org/jira/browse/SPARK-11095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-11095: Issue Type: Improvement (was: Sub-task) Parent: (was: SPARK-5293) > Simplify Netty RPC

[jira] [Resolved] (SPARK-7997) Remove the developer api SparkEnv.actorSystem and AkkaUtils

[ https://issues.apache.org/jira/browse/SPARK-7997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-7997. Resolution: Fixed Assignee: Shixiong Zhu Fix Version/s: 2.0.0 > Remove the developer

[jira] [Closed] (SPARK-12953) RDDRelation write set mode will be better to avoid error "pair.parquet already exists"

[ https://issues.apache.org/jira/browse/SPARK-12953?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-12953. --- Resolution: Not A Problem > RDDRelation write set mode will be better to avoid error "pair.parquet >

[jira] [Commented] (SPARK-11075) Spark SQL Thrift Server authentication issue on kerberized yarn cluster

[ https://issues.apache.org/jira/browse/SPARK-11075?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113560#comment-15113560 ] Zhan Zhang commented on SPARK-11075: Duplicated to SPARK-5159? > Spark SQL Thrift Se

[jira] [Commented] (SPARK-6847) Stack overflow on updateStateByKey which followed by a dstream with checkpoint set

[ https://issues.apache.org/jira/browse/SPARK-6847?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113490#comment-15113490 ] Shixiong Zhu commented on SPARK-6847: - It has not yet been released. You need to use t

[jira] [Commented] (SPARK-5159) Thrift server does not respect hive.server2.enable.doAs=true

[ https://issues.apache.org/jira/browse/SPARK-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113469#comment-15113469 ] Zhan Zhang commented on SPARK-5159: --- [~luciano resende] Given the current code base, I d

[jira] [Commented] (SPARK-5159) Thrift server does not respect hive.server2.enable.doAs=true

[ https://issues.apache.org/jira/browse/SPARK-5159?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113430#comment-15113430 ] Luciano Resende commented on SPARK-5159: I was able to validate doAs working via t

[jira] [Commented] (SPARK-12143) When column type is binary, select occurs ClassCastExcption in Beeline.

[ https://issues.apache.org/jira/browse/SPARK-12143?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113337#comment-15113337 ] Russell Alexander Spitzer commented on SPARK-12143: --- I see this as well

[jira] [Commented] (SPARK-12967) NettyRPC races with SparkContext.stop() and throws exception

[ https://issues.apache.org/jira/browse/SPARK-12967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113276#comment-15113276 ] Apache Spark commented on SPARK-12967: -- User 'nishkamravi2' has created a pull reque

[jira] [Assigned] (SPARK-12967) NettyRPC races with SparkContext.stop() and throws exception

[ https://issues.apache.org/jira/browse/SPARK-12967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12967: Assignee: Apache Spark > NettyRPC races with SparkContext.stop() and throws exception > --

[jira] [Assigned] (SPARK-12967) NettyRPC races with SparkContext.stop() and throws exception

[ https://issues.apache.org/jira/browse/SPARK-12967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12967: Assignee: (was: Apache Spark) > NettyRPC races with SparkContext.stop() and throws exc

[jira] [Commented] (SPARK-12963) In cluster mode,spark_local_ip will cause driver exception:Service 'Driver' failed after 16 retries!

[ https://issues.apache.org/jira/browse/SPARK-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113260#comment-15113260 ] Marcelo Vanzin commented on SPARK-12963: I remember seeing this a long time ago,

[jira] [Created] (SPARK-12967) NettyRPC races with SparkContext.stop() and throws exception

Nishkam Ravi created SPARK-12967: Summary: NettyRPC races with SparkContext.stop() and throws exception Key: SPARK-12967 URL: https://issues.apache.org/jira/browse/SPARK-12967 Project: Spark

[jira] [Commented] (SPARK-12826) Spark Workers do not attempt reconnect or exit on connection failure.

[ https://issues.apache.org/jira/browse/SPARK-12826?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113180#comment-15113180 ] Alan Braithwaite commented on SPARK-12826: -- Update to this: We moved the spark-

[jira] [Commented] (SPARK-12965) Indexer setInputCol() doesn't resolve column names like DataFrame.col()

[ https://issues.apache.org/jira/browse/SPARK-12965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15113097#comment-15113097 ] Joshua Taylor commented on SPARK-12965: --- Actually, this seems to go a bit farther t

[jira] [Updated] (SPARK-12966) Postgres JDBC ArrayType(DecimalType) 'Unable to find server array type'

[ https://issues.apache.org/jira/browse/SPARK-12966?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Bradley updated SPARK-12966: Description: Similar to SPARK-12747 but for DecimalType. Do we need to handle precision an

[jira] [Created] (SPARK-12966) Postgres JDBC ArrayType(DecimalType) 'Unable to find server array type'

Brandon Bradley created SPARK-12966: --- Summary: Postgres JDBC ArrayType(DecimalType) 'Unable to find server array type' Key: SPARK-12966 URL: https://issues.apache.org/jira/browse/SPARK-12966 Project

[jira] [Commented] (SPARK-12965) Indexer setInputCol() doesn't resolve column names like DataFrame.col()

[ https://issues.apache.org/jira/browse/SPARK-12965?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112922#comment-15112922 ] Joshua Taylor commented on SPARK-12965: --- I asked about this issue on Stack Overflow

[jira] [Updated] (SPARK-12965) Indexer setInputCol() doesn't resolve column names like DataFrame.col()

[ https://issues.apache.org/jira/browse/SPARK-12965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua Taylor updated SPARK-12965: -- Description: The setInputCol() method doesn't seem to resolve column names in the same way tha

[jira] [Updated] (SPARK-12965) Indexer setInputCol() doesn't resolve column names like DataFrame.col()

[ https://issues.apache.org/jira/browse/SPARK-12965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua Taylor updated SPARK-12965: -- Description: The setInputCol() method doesn't seem to resolve column names in the same way tha

[jira] [Updated] (SPARK-12965) Indexer setInputCol() doesn't resolve column names like DataFrame.col()

[ https://issues.apache.org/jira/browse/SPARK-12965?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joshua Taylor updated SPARK-12965: -- Attachment: SparkMLDotColumn.java > Indexer setInputCol() doesn't resolve column names like Dat

[jira] [Created] (SPARK-12965) Indexer setInputCol() doesn't resolve column names like DataFrame.col()

Joshua Taylor created SPARK-12965: - Summary: Indexer setInputCol() doesn't resolve column names like DataFrame.col() Key: SPARK-12965 URL: https://issues.apache.org/jira/browse/SPARK-12965 Project: Sp

[jira] [Updated] (SPARK-12940) Partition field in Spark SQL WHERE clause causing Exception

[ https://issues.apache.org/jira/browse/SPARK-12940?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brian Wheeler updated SPARK-12940: -- Environment: AWS EMR 4.2, OSX (was: AWS EMR 4.2) Summary: Partition field in Spark SQL

[jira] [Commented] (SPARK-12958) Map accumulator in spark

[ https://issues.apache.org/jira/browse/SPARK-12958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112843#comment-15112843 ] Souri commented on SPARK-12958: --- I should have been more specific.I wanted to have Map[Stri

[jira] [Resolved] (SPARK-12629) SparkR: DataFrame's saveAsTable method has issues with the signature and HiveContext

[ https://issues.apache.org/jira/browse/SPARK-12629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-12629. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by https://githu

[jira] [Updated] (SPARK-12629) SparkR: DataFrame's saveAsTable method has issues with the signature and HiveContext

[ https://issues.apache.org/jira/browse/SPARK-12629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-12629: -- Assignee: Narine Kokhlikyan > SparkR: DataFrame's saveAsTable method has issues

[jira] [Commented] (SPARK-12177) Update KafkaDStreams to new Kafka 0.9 Consumer API

[ https://issues.apache.org/jira/browse/SPARK-12177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112785#comment-15112785 ] Mark Grover commented on SPARK-12177: - Hi Mario, Thanks for checking. I was still hop

[jira] [Commented] (SPARK-12747) Postgres JDBC ArrayType(DoubleType) 'Unable to find server array type'

[ https://issues.apache.org/jira/browse/SPARK-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112665#comment-15112665 ] Reynold Xin commented on SPARK-12747: - [~blbradley] please don't reopen issues this w

[jira] [Commented] (SPARK-12831) akka.remote.OversizedPayloadException on DirectTaskResult

[ https://issues.apache.org/jira/browse/SPARK-12831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112676#comment-15112676 ] Brett Stime commented on SPARK-12831: - Actually, even with more conservative timeouts

[jira] [Resolved] (SPARK-12747) Postgres JDBC ArrayType(DoubleType) 'Unable to find server array type'

[ https://issues.apache.org/jira/browse/SPARK-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12747. - Resolution: Fixed > Postgres JDBC ArrayType(DoubleType) 'Unable to find server array type' >

[jira] [Commented] (SPARK-12947) Spark with Swift throws EOFException when reading parquet file

[ https://issues.apache.org/jira/browse/SPARK-12947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112603#comment-15112603 ] Gil Vernik commented on SPARK-12947: What is the Hadoop version you use? I work with

[jira] [Commented] (SPARK-12958) Map accumulator in spark

[ https://issues.apache.org/jira/browse/SPARK-12958?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112565#comment-15112565 ] Sean Owen commented on SPARK-12958: --- You can make your own {{Accumulator}} -- have a lo

[jira] [Commented] (SPARK-3830) Implement genetic algorithms in MLLib

[ https://issues.apache.org/jira/browse/SPARK-3830?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112535#comment-15112535 ] John Muller commented on SPARK-3830: [~ipv6guru] I see from the PR that your patch was

[jira] [Commented] (SPARK-11219) Make Parameter Description Format Consistent in PySpark.MLlib

[ https://issues.apache.org/jira/browse/SPARK-11219?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112452#comment-15112452 ] Vijay Kiran commented on SPARK-11219: - Hi All - sorry for the delay, I updated the 3

[jira] [Commented] (SPARK-12947) Spark with Swift throws EOFException when reading parquet file

[ https://issues.apache.org/jira/browse/SPARK-12947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112446#comment-15112446 ] Apurva Nandan commented on SPARK-12947: --- Yes. I have been facing the same issue. It

[jira] [Reopened] (SPARK-12747) Postgres JDBC ArrayType(DoubleType) 'Unable to find server array type'

[ https://issues.apache.org/jira/browse/SPARK-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Brandon Bradley reopened SPARK-12747: - Reopened until question answered. > Postgres JDBC ArrayType(DoubleType) 'Unable to find serv

[jira] [Commented] (SPARK-12963) In cluster mode,spark_local_ip will cause driver exception:Service 'Driver' failed after 16 retries!

[ https://issues.apache.org/jira/browse/SPARK-12963?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112423#comment-15112423 ] Sean Owen commented on SPARK-12963: --- Hm, how would the env setting from one machine aff

[jira] [Commented] (SPARK-12747) Postgres JDBC ArrayType(DoubleType) 'Unable to find server array type'

[ https://issues.apache.org/jira/browse/SPARK-12747?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112413#comment-15112413 ] Brandon Bradley commented on SPARK-12747: - Should this issue cover any Postgres t

[jira] [Commented] (SPARK-12947) Spark with Swift throws EOFException when reading parquet file

[ https://issues.apache.org/jira/browse/SPARK-12947?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112408#comment-15112408 ] Sam Stoelinga commented on SPARK-12947: --- I've been hitting this issue constantly cu

[jira] [Updated] (SPARK-12964) SparkContext.localProperties leaked

[ https://issues.apache.org/jira/browse/SPARK-12964?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Daniel Darabos updated SPARK-12964: --- Description: I have a non-deterministic but quite reliable reproduction for a case where {{s

[jira] [Created] (SPARK-12964) SparkContext.localProperties leaked

Daniel Darabos created SPARK-12964: -- Summary: SparkContext.localProperties leaked Key: SPARK-12964 URL: https://issues.apache.org/jira/browse/SPARK-12964 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-11811) Database can't be changed if it is specified in url

[ https://issues.apache.org/jira/browse/SPARK-11811?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112299#comment-15112299 ] Ajesh Kumar commented on SPARK-11811: - beeline script is invoking "org.apache.hive.be

[jira] [Commented] (SPARK-12843) Spark should avoid scanning all partitions when limit is set

[ https://issues.apache.org/jira/browse/SPARK-12843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112286#comment-15112286 ] Maciej Bryński commented on SPARK-12843: >From the Spark UI. I can observe time o

[jira] [Commented] (SPARK-12843) Spark should avoid scanning all partitions when limit is set

[ https://issues.apache.org/jira/browse/SPARK-12843?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112280#comment-15112280 ] dileep commented on SPARK-12843: Ok...Got it. But how you reach in to conclusion that its

[jira] [Commented] (SPARK-12950) Improve performance of BytesToBytesMap

[ https://issues.apache.org/jira/browse/SPARK-12950?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15112127#comment-15112127 ] Apache Spark commented on SPARK-12950: -- User 'viirya' has created a pull request for

[jira] [Assigned] (SPARK-12950) Improve performance of BytesToBytesMap

[ https://issues.apache.org/jira/browse/SPARK-12950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12950: Assignee: Apache Spark > Improve performance of BytesToBytesMap >

[jira] [Assigned] (SPARK-12950) Improve performance of BytesToBytesMap

[ https://issues.apache.org/jira/browse/SPARK-12950?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-12950: Assignee: (was: Apache Spark) > Improve performance of BytesToBytesMap > -

[jira] [Resolved] (SPARK-12959) Writing Bucketed Data with Disabled Bucketing in SQLConf

[ https://issues.apache.org/jira/browse/SPARK-12959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin resolved SPARK-12959. - Resolution: Fixed Assignee: Xiao Li Fix Version/s: 2.0.0 > Writing Bucketed Data