[jira] [Updated] (SPARK-19237) SparkR package install stuck when no java is found

2017-01-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19237: - Description: When installing SparkR as a R package (install.packages), it will check for Spark di

[jira] [Created] (SPARK-19237) SparkR package stuck when no java is found

2017-01-15 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-19237: Summary: SparkR package stuck when no java is found Key: SPARK-19237 URL: https://issues.apache.org/jira/browse/SPARK-19237 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-19237) SparkR package install stuck when no java is found

2017-01-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19237?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19237: - Summary: SparkR package install stuck when no java is found (was: SparkR package stuck when no j

[jira] [Created] (SPARK-19236) Add createOrReplaceGlobalTempView

2017-01-15 Thread Arman Yazdani (JIRA)
Arman Yazdani created SPARK-19236: - Summary: Add createOrReplaceGlobalTempView Key: SPARK-19236 URL: https://issues.apache.org/jira/browse/SPARK-19236 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-19082) The config ignoreCorruptFiles doesn't work for Parquet

2017-01-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19082?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19082. - Resolution: Fixed Assignee: Liang-Chi Hsieh Fix Version/s: 2.2.0

[jira] [Assigned] (SPARK-17078) show estimated stats when doing explain

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17078: Assignee: Apache Spark > show estimated stats when doing explain > ---

[jira] [Assigned] (SPARK-17078) show estimated stats when doing explain

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-17078: Assignee: (was: Apache Spark) > show estimated stats when doing explain >

[jira] [Commented] (SPARK-17078) show estimated stats when doing explain

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17078?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823567#comment-15823567 ] Apache Spark commented on SPARK-17078: -- User 'wzhfy' has created a pull request for

[jira] [Commented] (SPARK-19153) DataFrameWriter.saveAsTable should work with hive format to create partitioned table

2017-01-15 Thread Shuai Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823552#comment-15823552 ] Shuai Lin commented on SPARK-19153: --- [~windpiger] I planned to sent a PR today, just to

[jira] [Assigned] (SPARK-19153) DataFrameWriter.saveAsTable should work with hive format to create partitioned table

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19153: Assignee: Apache Spark > DataFrameWriter.saveAsTable should work with hive format to creat

[jira] [Commented] (SPARK-19153) DataFrameWriter.saveAsTable should work with hive format to create partitioned table

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823520#comment-15823520 ] Apache Spark commented on SPARK-19153: -- User 'windpiger' has created a pull request

[jira] [Assigned] (SPARK-19153) DataFrameWriter.saveAsTable should work with hive format to create partitioned table

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19153?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19153: Assignee: (was: Apache Spark) > DataFrameWriter.saveAsTable should work with hive form

[jira] [Commented] (SPARK-19208) MaxAbsScaler and MinMaxScaler are very inefficient

2017-01-15 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823517#comment-15823517 ] zhengruifeng commented on SPARK-19208: -- cc [~josephkb] [~yanboliang] > MaxAbsScaler

[jira] [Updated] (SPARK-19208) MaxAbsScaler and MinMaxScaler are very inefficient

2017-01-15 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19208?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] zhengruifeng updated SPARK-19208: - Attachment: Tests.pdf > MaxAbsScaler and MinMaxScaler are very inefficient >

[jira] [Commented] (SPARK-19208) MaxAbsScaler and MinMaxScaler are very inefficient

2017-01-15 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823508#comment-15823508 ] zhengruifeng commented on SPARK-19208: -- I do tests on a dataset with 6,000,000 insta

[jira] [Commented] (SPARK-19235) Enable Test Cases in DDLSuite with Hive Metastore

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19235?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823475#comment-15823475 ] Apache Spark commented on SPARK-19235: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-19235) Enable Test Cases in DDLSuite with Hive Metastore

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19235: Assignee: Apache Spark (was: Xiao Li) > Enable Test Cases in DDLSuite with Hive Metastore

[jira] [Assigned] (SPARK-19235) Enable Test Cases in DDLSuite with Hive Metastore

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19235?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19235: Assignee: Xiao Li (was: Apache Spark) > Enable Test Cases in DDLSuite with Hive Metastore

[jira] [Created] (SPARK-19235) Enable Test Cases in DDLSuite with Hive Metastore

2017-01-15 Thread Xiao Li (JIRA)
Xiao Li created SPARK-19235: --- Summary: Enable Test Cases in DDLSuite with Hive Metastore Key: SPARK-19235 URL: https://issues.apache.org/jira/browse/SPARK-19235 Project: Spark Issue Type: Test

[jira] [Updated] (SPARK-19234) AFTSurvivalRegression chokes silently or with confusing errors when any labels are zero

2017-01-15 Thread Andrew MacKinlay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew MacKinlay updated SPARK-19234: - Description: If you try and use AFTSurvivalRegression and any label in your input data is

[jira] [Updated] (SPARK-19234) AFTSurvivalRegression chokes silently or with confusing errors when any labels are zero

2017-01-15 Thread Andrew MacKinlay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19234?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew MacKinlay updated SPARK-19234: - Attachment: spark-aft-failure.txt Show failure case (a single zero-valued label, causing

[jira] [Created] (SPARK-19234) AFTSurvivalRegression chokes silently or with confusing errors when any labels are zero

2017-01-15 Thread Andrew MacKinlay (JIRA)
Andrew MacKinlay created SPARK-19234: Summary: AFTSurvivalRegression chokes silently or with confusing errors when any labels are zero Key: SPARK-19234 URL: https://issues.apache.org/jira/browse/SPARK-19234

[jira] [Comment Edited] (SPARK-19217) Offer easy cast from vector to array

2017-01-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823439#comment-15823439 ] Hyukjin Kwon edited comment on SPARK-19217 at 1/16/17 4:18 AM:

[jira] [Commented] (SPARK-19217) Offer easy cast from vector to array

2017-01-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823439#comment-15823439 ] Hyukjin Kwon commented on SPARK-19217: -- Up to my knowledge, the vectors in SQL becom

[jira] [Commented] (SPARK-19222) Limit Query Performance issue

2017-01-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823427#comment-15823427 ] Hyukjin Kwon commented on SPARK-19222: -- (I just simply inserted \{code\} ... \{code\

[jira] [Updated] (SPARK-19222) Limit Query Performance issue

2017-01-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-19222: - Description: Performance/memory bottle neck occurs in the below mentioned query case 1: {code} cr

[jira] [Updated] (SPARK-19222) Limit Query Performance issue

2017-01-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hyukjin Kwon updated SPARK-19222: - Description: Performance/memory bottle neck occurs in the below mentioned query case 1: create ta

[jira] [Commented] (SPARK-19208) MaxAbsScaler and MinMaxScaler are very inefficient

2017-01-15 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19208?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823422#comment-15823422 ] zhengruifeng commented on SPARK-19208: -- The code in {{MinMaxScaler}} are copied from

[jira] [Commented] (SPARK-19228) inferSchema function processed csv date column as string and "dateFormat" DataSource option is ignored

2017-01-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823419#comment-15823419 ] Hyukjin Kwon commented on SPARK-19228: -- Yes, inferring {{DateType}} is currently not

[jira] [Updated] (SPARK-19092) Save() API of DataFrameWriter should not scan all the saved files

2017-01-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan updated SPARK-19092: Fix Version/s: 2.1.1 > Save() API of DataFrameWriter should not scan all the saved files >

[jira] [Comment Edited] (SPARK-19222) Limit Query Performance issue

2017-01-15 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823389#comment-15823389 ] Yadong Qi edited comment on SPARK-19222 at 1/16/17 2:56 AM: H

[jira] [Commented] (SPARK-19222) Limit Query Performance issue

2017-01-15 Thread Yadong Qi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19222?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823389#comment-15823389 ] Yadong Qi commented on SPARK-19222: --- Hi [~maropu], sample means `TABLESAMPLE(x ROWS)` o

[jira] [Commented] (SPARK-19227) Typo in `org.apache.spark.internal.config.ConfigEntry`

2017-01-15 Thread Biao Ma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823366#comment-15823366 ] Biao Ma commented on SPARK-19227: - Its not outdated comment, simply for its sub-class. >

[jira] [Commented] (SPARK-19233) Inconsistent Behaviour of Spark Streaming Checkpoint

2017-01-15 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823364#comment-15823364 ] Nan Zhu commented on SPARK-19233: - [~zsxwing] so, another potential issue I found in Spar

[jira] [Commented] (SPARK-19233) Inconsistent Behaviour of Spark Streaming Checkpoint

2017-01-15 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19233?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823359#comment-15823359 ] Nan Zhu commented on SPARK-19233: - The category of this issue is Improvement which is sub

[jira] [Assigned] (SPARK-19227) Typo in `org.apache.spark.internal.config.ConfigEntry`

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19227: Assignee: Apache Spark > Typo in `org.apache.spark.internal.config.ConfigEntry` > ---

[jira] [Assigned] (SPARK-19227) Typo in `org.apache.spark.internal.config.ConfigEntry`

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19227: Assignee: (was: Apache Spark) > Typo in `org.apache.spark.internal.config.ConfigEntry

[jira] [Commented] (SPARK-19227) Typo in `org.apache.spark.internal.config.ConfigEntry`

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19227?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823356#comment-15823356 ] Apache Spark commented on SPARK-19227: -- User 'uncleGen' has created a pull request f

[jira] [Commented] (SPARK-19153) DataFrameWriter.saveAsTable should work with hive format to create partitioned table

2017-01-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823350#comment-15823350 ] Wenchen Fan commented on SPARK-19153: - > Now that we can create partitioned table usi

[jira] [Commented] (SPARK-19225) Spark SQL round constant double return null

2017-01-15 Thread discipleforteen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19225?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823348#comment-15823348 ] discipleforteen commented on SPARK-19225: - spark 1.4.1 > select round(4.4, 2); 2

[jira] [Created] (SPARK-19233) Inconsistent Behaviour of Spark Streaming Checkpoint

2017-01-15 Thread Nan Zhu (JIRA)
Nan Zhu created SPARK-19233: --- Summary: Inconsistent Behaviour of Spark Streaming Checkpoint Key: SPARK-19233 URL: https://issues.apache.org/jira/browse/SPARK-19233 Project: Spark Issue Type: Improv

[jira] [Assigned] (SPARK-19232) SparkR distribution cache location is wrong on Windows

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19232: Assignee: Apache Spark (was: Felix Cheung) > SparkR distribution cache location is wrong

[jira] [Commented] (SPARK-19232) SparkR distribution cache location is wrong on Windows

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19232?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1582#comment-1582 ] Apache Spark commented on SPARK-19232: -- User 'felixcheung' has created a pull reques

[jira] [Assigned] (SPARK-19232) SparkR distribution cache location is wrong on Windows

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19232: Assignee: Felix Cheung (was: Apache Spark) > SparkR distribution cache location is wrong

[jira] [Updated] (SPARK-19232) SparkR distribution cache location is wrong on Windows

2017-01-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19232: - Description: On Linux: {code} ~/.cache/spark# ls -lart total 12 drwxr-xr-x 12 500 500 4096 Dec

[jira] [Updated] (SPARK-19232) SparkR distribution cache location is wrong on Windows

2017-01-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19232: - Priority: Trivial (was: Major) > SparkR distribution cache location is wrong on Windows > --

[jira] [Updated] (SPARK-19232) SparkR distribution cache location is wrong on Windows

2017-01-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19232: - Description: On Linux: {code} ~/.cache/spark# ls -lart total 12 drwxr-xr-x 12 500 500 4096 Dec

[jira] [Updated] (SPARK-19232) SparkR distribution cache location is wrong on Windows

2017-01-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19232: - Description: On Linux: {code} ~/.cache/spark# ls -lart total 12 drwxr-xr-x 12 500 500 4096 Dec

[jira] [Updated] (SPARK-19232) SparkR distribution cache location is wrong on Windows

2017-01-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19232?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19232: - Summary: SparkR distribution cache location is wrong on Windows (was: SparkR distribution cache

[jira] [Assigned] (SPARK-19231) SparkR hangs when there is download or untar failure

2017-01-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung reassigned SPARK-19231: Assignee: Felix Cheung > SparkR hangs when there is download or untar failure > --

[jira] [Created] (SPARK-19232) SparkR distribution cache location is wrong

2017-01-15 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-19232: Summary: SparkR distribution cache location is wrong Key: SPARK-19232 URL: https://issues.apache.org/jira/browse/SPARK-19232 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-19231) SparkR hangs when there is download or untar failure

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19231?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823324#comment-15823324 ] Apache Spark commented on SPARK-19231: -- User 'felixcheung' has created a pull reques

[jira] [Assigned] (SPARK-19231) SparkR hangs when there is download or untar failure

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19231: Assignee: Apache Spark > SparkR hangs when there is download or untar failure > --

[jira] [Assigned] (SPARK-19231) SparkR hangs when there is download or untar failure

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19231: Assignee: (was: Apache Spark) > SparkR hangs when there is download or untar failure >

[jira] [Updated] (SPARK-19231) SparkR hangs when there is download or untar failure

2017-01-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19231: - Description: When there is any partial download, or download error it is not cleaned up, and spa

[jira] [Updated] (SPARK-19231) SparkR hangs when there is download or untar failure

2017-01-15 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19231?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-19231: - Description: When there is any partial download, or download error it is not cleaned up, and spa

[jira] [Created] (SPARK-19231) SparkR hangs when there is download or untar failure

2017-01-15 Thread Felix Cheung (JIRA)
Felix Cheung created SPARK-19231: Summary: SparkR hangs when there is download or untar failure Key: SPARK-19231 URL: https://issues.apache.org/jira/browse/SPARK-19231 Project: Spark Issue Ty

[jira] [Commented] (SPARK-14560) Cooperative Memory Management for Spillables

2017-01-15 Thread Lenni Kuff (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823297#comment-15823297 ] Lenni Kuff commented on SPARK-14560: Hi,I will be out of the office from 12/19/16 - 1

[jira] [Commented] (SPARK-14560) Cooperative Memory Management for Spillables

2017-01-15 Thread Morten Hornbech (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14560?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823296#comment-15823296 ] Morten Hornbech commented on SPARK-14560: - I have also observed this error sporad

[jira] [Created] (SPARK-19230) View creation in Derby gets SQLDataException because definition gets very big

2017-01-15 Thread Ohad Raviv (JIRA)
Ohad Raviv created SPARK-19230: -- Summary: View creation in Derby gets SQLDataException because definition gets very big Key: SPARK-19230 URL: https://issues.apache.org/jira/browse/SPARK-19230 Project: Sp

[jira] [Commented] (SPARK-19092) Save() API of DataFrameWriter should not scan all the saved files

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19092?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823229#comment-15823229 ] Apache Spark commented on SPARK-19092: -- User 'gatorsmile' has created a pull request

[jira] [Assigned] (SPARK-19229) Disallow Creating Hive Source Tables when Hive Support is Not Enabled

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19229: Assignee: Apache Spark (was: Xiao Li) > Disallow Creating Hive Source Tables when Hive Su

[jira] [Assigned] (SPARK-19229) Disallow Creating Hive Source Tables when Hive Support is Not Enabled

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19229?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-19229: Assignee: Xiao Li (was: Apache Spark) > Disallow Creating Hive Source Tables when Hive Su

[jira] [Commented] (SPARK-19229) Disallow Creating Hive Source Tables when Hive Support is Not Enabled

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19229?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823227#comment-15823227 ] Apache Spark commented on SPARK-19229: -- User 'gatorsmile' has created a pull request

[jira] [Created] (SPARK-19229) Disallow Creating Hive Source Tables when Hive Support is Not Enabled

2017-01-15 Thread Xiao Li (JIRA)
Xiao Li created SPARK-19229: --- Summary: Disallow Creating Hive Source Tables when Hive Support is Not Enabled Key: SPARK-19229 URL: https://issues.apache.org/jira/browse/SPARK-19229 Project: Spark

[jira] [Updated] (SPARK-19227) Typo in `org.apache.spark.internal.config.ConfigEntry`

2017-01-15 Thread Biao Ma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Biao Ma updated SPARK-19227: Summary: Typo in `org.apache.spark.internal.config.ConfigEntry` (was: Modify some typo in `ConfigEntry`)

[jira] [Updated] (SPARK-19227) Modify some typo in `ConfigEntry`

2017-01-15 Thread Biao Ma (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19227?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Biao Ma updated SPARK-19227: Description: The parameter `defaultValue` is not exists in class `org.apache.spark.internal.config.ConfigEn

[jira] [Resolved] (SPARK-11353) Writing to S3 buckets, which only support AWS4-HMAC-SHA256 fails with s3n URLs

2017-01-15 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran resolved SPARK-11353. Resolution: Duplicate This is a duplicate of SPARK-13044; that's transitive a WONTFIX due t

[jira] [Updated] (SPARK-19228) inferSchema function processed csv date column as string and "dateFormat" DataSource option is ignored

2017-01-15 Thread Sergey Rubtsov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sergey Rubtsov updated SPARK-19228: --- Description: I need to process user.csv like this: {code} id,project,started,ended sergey.rub

[jira] [Updated] (SPARK-11353) Writing to S3 buckets, which only support AWS4-HMAC-SHA256 fails with s3n URLs

2017-01-15 Thread Steve Loughran (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Steve Loughran updated SPARK-11353: --- Summary: Writing to S3 buckets, which only support AWS4-HMAC-SHA256 fails with s3n URLs (was

[jira] [Created] (SPARK-19228) inferSchema function processed csv date column as string and "dateFormat" DataSource option is ignored

2017-01-15 Thread Sergey Rubtsov (JIRA)
Sergey Rubtsov created SPARK-19228: -- Summary: inferSchema function processed csv date column as string and "dateFormat" DataSource option is ignored Key: SPARK-19228 URL: https://issues.apache.org/jira/browse/SPA

[jira] [Commented] (SPARK-19185) ConcurrentModificationExceptions with CachedKafkaConsumers when Windowing

2017-01-15 Thread Cody Koeninger (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19185?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823164#comment-15823164 ] Cody Koeninger commented on SPARK-19185: This is a good error report, sorry it's

[jira] [Updated] (SPARK-19226) Report failure reason from Reporter Thread

2017-01-15 Thread Maheedhar Reddy Chappidi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Maheedhar Reddy Chappidi updated SPARK-19226: - Description: With the exponential[1] increase in executor count the Repor

[jira] [Created] (SPARK-19227) Remove non-existent parameter annotation in `ConfigEntry`

2017-01-15 Thread Biao Ma (JIRA)
Biao Ma created SPARK-19227: --- Summary: Remove non-existent parameter annotation in `ConfigEntry` Key: SPARK-19227 URL: https://issues.apache.org/jira/browse/SPARK-19227 Project: Spark Issue Type: D

[jira] [Created] (SPARK-19226) Report failure reason from Reporter Thread

2017-01-15 Thread Maheedhar Reddy Chappidi (JIRA)
Maheedhar Reddy Chappidi created SPARK-19226: Summary: Report failure reason from Reporter Thread Key: SPARK-19226 URL: https://issues.apache.org/jira/browse/SPARK-19226 Project: Spark

[jira] [Commented] (SPARK-18922) Fix more resource-closing-related and path-related test failures in identified ones on Windows

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18922?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823150#comment-15823150 ] Apache Spark commented on SPARK-18922: -- User 'HyukjinKwon' has created a pull reques

[jira] [Commented] (SPARK-19117) script transformation does not work on Windows due to fixed bash executable location

2017-01-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19117?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823149#comment-15823149 ] Apache Spark commented on SPARK-19117: -- User 'HyukjinKwon' has created a pull reques

[jira] [Commented] (SPARK-19153) DataFrameWriter.saveAsTable should work with hive format to create partitioned table

2017-01-15 Thread Shuai Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823141#comment-15823141 ] Shuai Lin commented on SPARK-19153: --- bq. To clarify, we want this feature in DataFrameW

[jira] [Resolved] (SPARK-13735) Log for parquet relation reading files is too verbose

2017-01-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13735?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-13735. --- Resolution: Duplicate > Log for parquet relation reading files is too verbose > -

[jira] [Commented] (SPARK-13735) Log for parquet relation reading files is too verbose

2017-01-15 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13735?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823118#comment-15823118 ] Hyukjin Kwon commented on SPARK-13735: -- Is this a duplicate of SPARK-8118? > Log fo

[jira] [Resolved] (SPARK-19120) Returned an Empty Result after Loading a Hive Table

2017-01-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19120?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Wenchen Fan resolved SPARK-19120. - Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 Issue resolved by pull req

[jira] [Commented] (SPARK-19153) DataFrameWriter.saveAsTable should work with hive format to create partitioned table

2017-01-15 Thread Wenchen Fan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823116#comment-15823116 ] Wenchen Fan commented on SPARK-19153: - To clarify, we want this feature in DataFrameW

[jira] [Commented] (SPARK-18492) GeneratedIterator grows beyond 64 KB

2017-01-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823110#comment-15823110 ] Sean Owen commented on SPARK-18492: --- 64K is a JVM limit as far as I know. And there are

[jira] [Updated] (SPARK-19206) Update outdated parameter descriptions in external-kafka module

2017-01-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19206: -- Assignee: Genmao Yu > Update outdated parameter descriptions in external-kafka module > ---

[jira] [Resolved] (SPARK-19206) Update outdated parameter descriptions in external-kafka module

2017-01-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19206?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19206. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16569 [https://github.co

[jira] [Updated] (SPARK-18971) Netty issue may cause the shuffle client hang

2017-01-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-18971: -- Assignee: Shixiong Zhu Priority: Minor (was: Major) > Netty issue may cause the shuffle client han

[jira] [Resolved] (SPARK-18971) Netty issue may cause the shuffle client hang

2017-01-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18971?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-18971. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16568 [https://github.co

[jira] [Updated] (SPARK-19042) Remove query string from jar url for executor

2017-01-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19042: -- Assignee: hustfxj Priority: Minor (was: Major) > Remove query string from jar url for executor > -

[jira] [Resolved] (SPARK-19042) Remove query string from jar url for executor

2017-01-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19042?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19042. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16509 [https://github.co

[jira] [Resolved] (SPARK-19207) LocalSparkSession should use Slf4JLoggerFactory.INSTANCE instead of creating new object via constructor

2017-01-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-19207. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 16570 [https://github.co

[jira] [Updated] (SPARK-19207) LocalSparkSession should use Slf4JLoggerFactory.INSTANCE instead of creating new object via constructor

2017-01-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19207?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19207: -- Assignee: Tsuyoshi Ozawa Priority: Trivial (was: Major) > LocalSparkSession should use Slf4JLogger

[jira] [Commented] (SPARK-19217) Offer easy cast from vector to array

2017-01-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823105#comment-15823105 ] Sean Owen commented on SPARK-19217: --- Yes it's not hard to do this with a UDF, but it se

[jira] [Commented] (SPARK-19153) DataFrameWriter.saveAsTable should work with hive format to create partitioned table

2017-01-15 Thread Shuai Lin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19153?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15823103#comment-15823103 ] Shuai Lin commented on SPARK-19153: --- I find it's quite straight forward to remove the r

[jira] [Updated] (SPARK-19225) Spark SQL round constant double return null

2017-01-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19225?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen updated SPARK-19225: -- Target Version/s: (was: 2.1.1) Fix Version/s: (was: 2.1.1) Read http://spark.apache.org/co

[jira] [Closed] (SPARK-10890) "Column count does not match; SQL statement:" error in JDBCWriteSuite

2017-01-15 Thread Christian Kadner (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Christian Kadner closed SPARK-10890. Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 This is issue is no