[jira] [Commented] (SPARK-18877) Unable to read given csv data. Excepion: java.lang.IllegalArgumentException: requirement failed: Decimal precision 28 exceeds max precision 20

2016-12-16 Thread Navya Krishnappa (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756554#comment-15756554 ] Navya Krishnappa commented on SPARK-18877: -- Thank you for replying [~dongjoon]. Can you help me

[jira] [Commented] (SPARK-18813) MLlib 2.2 Roadmap

2016-12-16 Thread zhengruifeng (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18813?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756480#comment-15756480 ] zhengruifeng commented on SPARK-18813: -- prediction on single instance & Set initial model +1

[jira] [Assigned] (SPARK-18910) Can't use UDF that jar file in hdfs

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18910: Assignee: Apache Spark > Can't use UDF that jar file in hdfs >

[jira] [Assigned] (SPARK-18910) Can't use UDF that jar file in hdfs

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18910: Assignee: (was: Apache Spark) > Can't use UDF that jar file in hdfs >

[jira] [Commented] (SPARK-18910) Can't use UDF that jar file in hdfs

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756433#comment-15756433 ] Apache Spark commented on SPARK-18910: -- User 'shenh062326' has created a pull request for this

[jira] [Closed] (SPARK-17997) Aggregation function for counting distinct values for multiple intervals

2016-12-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin closed SPARK-17997. --- Resolution: Later > Aggregation function for counting distinct values for multiple intervals >

[jira] [Updated] (SPARK-17997) Aggregation function for counting distinct values for multiple intervals

2016-12-16 Thread Reynold Xin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17997?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Reynold Xin updated SPARK-17997: Issue Type: Sub-task (was: New Feature) Parent: SPARK-16026 > Aggregation function for

[jira] [Commented] (SPARK-18817) Ensure nothing is written outside R's tempdir() by default

2016-12-16 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18817?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756403#comment-15756403 ] Shivaram Venkataraman commented on SPARK-18817: --- [~felixcheung] What do you think are the

[jira] [Assigned] (SPARK-18911) Decouple Statistics and CatalogTable

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18911: Assignee: (was: Apache Spark) > Decouple Statistics and CatalogTable >

[jira] [Commented] (SPARK-18911) Decouple Statistics and CatalogTable

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18911?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756389#comment-15756389 ] Apache Spark commented on SPARK-18911: -- User 'wzhfy' has created a pull request for this issue:

[jira] [Updated] (SPARK-18910) Can't use UDF that jar file in hdfs

2016-12-16 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated SPARK-18910: -- Description: When I create a UDF that jar file in hdfs, I can't use the UDF. {code} spark-sql> create

[jira] [Updated] (SPARK-18910) Can't use UDF that jar file in hdfs

2016-12-16 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated SPARK-18910: -- Description: When I create a UDF that jar file in hdfs, I can't use the UDF. {code} spark-sql> create

[jira] [Assigned] (SPARK-18911) Decouple Statistics and CatalogTable

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18911?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18911: Assignee: Apache Spark > Decouple Statistics and CatalogTable >

[jira] [Updated] (SPARK-18910) Can't use UDF that jar file in hdfs

2016-12-16 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated SPARK-18910: -- Description: When I create a UDF that jar file in hdfs, I can't use the UDF. spark-sql> create

[jira] [Commented] (SPARK-18910) Can't use UDF that jar file in hdfs

2016-12-16 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756385#comment-15756385 ] Hong Shen commented on SPARK-18910: --- Should I add a pull request to resolve this problem? > Can't use

[jira] [Resolved] (SPARK-18895) Fix resource-closing-related and path-related test failures in identified ones on Windows

2016-12-16 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18895?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-18895. --- Resolution: Fixed Assignee: Hyukjin Kwon Fix Version/s: 2.2.0

[jira] [Updated] (SPARK-18910) Can't use UDF that jar file in hdfs

2016-12-16 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated SPARK-18910: -- Description: When I create a UDF that jar file in hdfs, I can't use the UDF. spark-sql> create

[jira] [Commented] (SPARK-18903) uiWebUrl is not accessible to SparkR

2016-12-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756379#comment-15756379 ] Felix Cheung commented on SPARK-18903: -- this sounds like a reasonable ask, I'll take a look. >

[jira] [Created] (SPARK-18911) Decouple Statistics and CatalogTable

2016-12-16 Thread Zhenhua Wang (JIRA)
Zhenhua Wang created SPARK-18911: Summary: Decouple Statistics and CatalogTable Key: SPARK-18911 URL: https://issues.apache.org/jira/browse/SPARK-18911 Project: Spark Issue Type: Sub-task

[jira] [Updated] (SPARK-18903) uiWebUrl is not accessible to SparkR

2016-12-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung updated SPARK-18903: - Component/s: (was: Java API) > uiWebUrl is not accessible to SparkR >

[jira] [Resolved] (SPARK-18902) Include Apache License in R source Package

2016-12-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung resolved SPARK-18902. -- Resolution: Not A Problem > Include Apache License in R source Package >

[jira] [Commented] (SPARK-18902) Include Apache License in R source Package

2016-12-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756369#comment-15756369 ] Felix Cheung commented on SPARK-18902: -- We have the license in DESCRIPTION file as required for R

[jira] [Closed] (SPARK-18902) Include Apache License in R source Package

2016-12-16 Thread Felix Cheung (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Felix Cheung closed SPARK-18902. Assignee: Felix Cheung > Include Apache License in R source Package >

[jira] [Updated] (SPARK-18910) Can't use UDF that jar file in hdfs

2016-12-16 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated SPARK-18910: -- Summary: Can't use UDF that jar file in hdfs (was: Can't use UDF that source file in hdfs) > Can't

[jira] [Updated] (SPARK-18910) Can't use UDF that source file in hdfs

2016-12-16 Thread Hong Shen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Hong Shen updated SPARK-18910: -- Description: When I create a UDF that jar spark-sql> create function trans_array as

[jira] [Created] (SPARK-18910) Can't use UDF that source file in hdfs

2016-12-16 Thread Hong Shen (JIRA)
Hong Shen created SPARK-18910: - Summary: Can't use UDF that source file in hdfs Key: SPARK-18910 URL: https://issues.apache.org/jira/browse/SPARK-18910 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-18909) The error message in `ExpressionEncoder.toRow` and `fromRow` is too verbose

2016-12-16 Thread Wenchen Fan (JIRA)
Wenchen Fan created SPARK-18909: --- Summary: The error message in `ExpressionEncoder.toRow` and `fromRow` is too verbose Key: SPARK-18909 URL: https://issues.apache.org/jira/browse/SPARK-18909 Project:

[jira] [Commented] (SPARK-18906) CSV parser should return null for empty (or with "") numeric columns.

2016-12-16 Thread Hyukjin Kwon (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756184#comment-15756184 ] Hyukjin Kwon commented on SPARK-18906: -- Yes, we should be able to set multiple values in

[jira] [Commented] (SPARK-18886) Delay scheduling should not delay some executors indefinitely if one task is scheduled before delay timeout

2016-12-16 Thread Mridul Muralidharan (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18886?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756179#comment-15756179 ] Mridul Muralidharan commented on SPARK-18886: - - Delay 'using up' all resources - another

[jira] [Assigned] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18699: Assignee: Apache Spark > Spark CSV parsing types other than String throws exception when

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756175#comment-15756175 ] Apache Spark commented on SPARK-18699: -- User 'kubatyszko' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18699: Assignee: (was: Apache Spark) > Spark CSV parsing types other than String throws

[jira] [Commented] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

2016-12-16 Thread Kuba Tyszko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756170#comment-15756170 ] Kuba Tyszko commented on SPARK-18699: - Fixed in PR https://github.com/apache/spark/pull/16319 Enjoy

[jira] [Comment Edited] (SPARK-18699) Spark CSV parsing types other than String throws exception when malformed

2016-12-16 Thread Kuba Tyszko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18699?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15756170#comment-15756170 ] Kuba Tyszko edited comment on SPARK-18699 at 12/17/16 3:05 AM: --- Fixed in PR

[jira] [Closed] (SPARK-18906) CSV parser should return null for empty (or with "") numeric columns.

2016-12-16 Thread Kuba Tyszko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuba Tyszko closed SPARK-18906. --- Resolution: Duplicate > CSV parser should return null for empty (or with "") numeric columns. >

[jira] [Updated] (SPARK-18827) Cann't read broadcast if broadcast blocks are stored on-disk

2016-12-16 Thread Yuming Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yuming Wang updated SPARK-18827: Attachment: NoSuchElementException4722.gif > Cann't read broadcast if broadcast blocks are stored

[jira] [Assigned] (SPARK-18908) It's hard for the user to see the failure if StreamExecution fails to create the logical plan

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18908: Assignee: Apache Spark (was: Shixiong Zhu) > It's hard for the user to see the failure

[jira] [Commented] (SPARK-18908) It's hard for the user to see the failure if StreamExecution fails to create the logical plan

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755992#comment-15755992 ] Apache Spark commented on SPARK-18908: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18908) It's hard for the user to see the failure if StreamExecution fails to create the logical plan

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18908: Assignee: Shixiong Zhu (was: Apache Spark) > It's hard for the user to see the failure

[jira] [Assigned] (SPARK-18908) It's hard for the user to see the failure if StreamExecution fails to create the logical plan

2016-12-16 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-18908: Assignee: Shixiong Zhu > It's hard for the user to see the failure if StreamExecution

[jira] [Updated] (SPARK-18908) It's hard for the user to see the failure if StreamExecution fails to create the logical plan

2016-12-16 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18908: - Description: If the logical plan fails to create, e.g., some Source options are invalid, the

[jira] [Updated] (SPARK-18908) It's hard for the user to see the failure if StreamExecution fails to create the logical plan

2016-12-16 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18908: - Description: If the logical plan fails to create, e.g., some options are invalid, the user

[jira] [Updated] (SPARK-18908) It's hard for the user to see the failure if StreamExecution fails to create the logical plan

2016-12-16 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18908: - Priority: Blocker (was: Major) > It's hard for the user to see the failure if StreamExecution

[jira] [Created] (SPARK-18908) It's hard for the user to see the failure if StreamExecution fails to create the logical plan

2016-12-16 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-18908: Summary: It's hard for the user to see the failure if StreamExecution fails to create the logical plan Key: SPARK-18908 URL: https://issues.apache.org/jira/browse/SPARK-18908

[jira] [Updated] (SPARK-18908) It's hard for the user to see the failure if StreamExecution fails to create the logical plan

2016-12-16 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18908: - Affects Version/s: 2.1.0 > It's hard for the user to see the failure if StreamExecution fails to

[jira] [Updated] (SPARK-18907) Fix flaky test: o.a.s.sql.streaming.FileStreamSourceSuite max files per trigger - incorrect values

2016-12-16 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18907?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18907: - Priority: Minor (was: Major) > Fix flaky test: o.a.s.sql.streaming.FileStreamSourceSuite max

[jira] [Created] (SPARK-18907) Fix flaky test: o.a.s.sql.streaming.FileStreamSourceSuite max files per trigger - incorrect values

2016-12-16 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-18907: Summary: Fix flaky test: o.a.s.sql.streaming.FileStreamSourceSuite max files per trigger - incorrect values Key: SPARK-18907 URL:

[jira] [Commented] (SPARK-18031) Flaky test: org.apache.spark.streaming.scheduler.ExecutorAllocationManagerSuite basic functionality

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18031?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755839#comment-15755839 ] Apache Spark commented on SPARK-18031: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18031) Flaky test: org.apache.spark.streaming.scheduler.ExecutorAllocationManagerSuite basic functionality

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18031: Assignee: Apache Spark (was: Shixiong Zhu) > Flaky test: >

[jira] [Assigned] (SPARK-18031) Flaky test: org.apache.spark.streaming.scheduler.ExecutorAllocationManagerSuite basic functionality

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18031: Assignee: Shixiong Zhu (was: Apache Spark) > Flaky test: >

[jira] [Assigned] (SPARK-18031) Flaky test: org.apache.spark.streaming.scheduler.ExecutorAllocationManagerSuite basic functionality

2016-12-16 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu reassigned SPARK-18031: Assignee: Shixiong Zhu > Flaky test: >

[jira] [Commented] (SPARK-18884) Support Array[_] in ScalaUDF

2016-12-16 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18884?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755778#comment-15755778 ] Dongjoon Hyun commented on SPARK-18884: --- +1 for the idea! > Support Array[_] in ScalaUDF >

[jira] [Commented] (SPARK-18255) SQLContext.getOrCreate always returns a SQLContext even if a user originally created a HiveContext

2016-12-16 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18255?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755762#comment-15755762 ] Dongjoon Hyun commented on SPARK-18255: --- Hi, [~yhuai]. I have a question about HiveContext.

[jira] [Resolved] (SPARK-18904) Merge two FileStreamSourceSuite files

2016-12-16 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu resolved SPARK-18904. -- Resolution: Fixed Fix Version/s: 2.2.0 2.1.1 > Merge two

[jira] [Commented] (SPARK-18492) GeneratedIterator grows beyond 64 KB

2016-12-16 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18492?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755742#comment-15755742 ] Nicholas Chammas commented on SPARK-18492: -- I'm hitting this problem as well when I try to apply

[jira] [Assigned] (SPARK-18877) Unable to read given csv data. Excepion: java.lang.IllegalArgumentException: requirement failed: Decimal precision 28 exceeds max precision 20

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18877: Assignee: Apache Spark > Unable to read given csv data. Excepion:

[jira] [Assigned] (SPARK-18877) Unable to read given csv data. Excepion: java.lang.IllegalArgumentException: requirement failed: Decimal precision 28 exceeds max precision 20

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18877: Assignee: (was: Apache Spark) > Unable to read given csv data. Excepion:

[jira] [Commented] (SPARK-18877) Unable to read given csv data. Excepion: java.lang.IllegalArgumentException: requirement failed: Decimal precision 28 exceeds max precision 20

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755691#comment-15755691 ] Apache Spark commented on SPARK-18877: -- User 'dongjoon-hyun' has created a pull request for this

[jira] [Updated] (SPARK-18877) Unable to read given csv data. Excepion: java.lang.IllegalArgumentException: requirement failed: Decimal precision 28 exceeds max precision 20

2016-12-16 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18877?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Dongjoon Hyun updated SPARK-18877: -- Component/s: SQL > Unable to read given csv data. Excepion:

[jira] [Comment Edited] (SPARK-18906) CSV parser should return null for empty (or with "") numeric columns.

2016-12-16 Thread Kuba Tyszko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755597#comment-15755597 ] Kuba Tyszko edited comment on SPARK-18906 at 12/16/16 9:53 PM: --- Well, in

[jira] [Updated] (SPARK-18906) CSV parser should return null for empty (or with "") numeric columns.

2016-12-16 Thread Kuba Tyszko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuba Tyszko updated SPARK-18906: Description: Spark allows user to set a nullValue that will indicate certain value's translation

[jira] [Updated] (SPARK-18901) Require in LR LogisticAggregator is redundant

2016-12-16 Thread yuhao yang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] yuhao yang updated SPARK-18901: --- Component/s: ML > Require in LR LogisticAggregator is redundant >

[jira] [Assigned] (SPARK-18906) CSV parser should return null for empty (or with "") numeric columns.

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18906: Assignee: Apache Spark > CSV parser should return null for empty (or with "") numeric

[jira] [Commented] (SPARK-18906) CSV parser should return null for empty (or with "") numeric columns.

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755604#comment-15755604 ] Apache Spark commented on SPARK-18906: -- User 'kubatyszko' has created a pull request for this issue:

[jira] [Assigned] (SPARK-18906) CSV parser should return null for empty (or with "") numeric columns.

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18906: Assignee: (was: Apache Spark) > CSV parser should return null for empty (or with "")

[jira] [Commented] (SPARK-18906) CSV parser should return null for empty (or with "") numeric columns.

2016-12-16 Thread Kuba Tyszko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755597#comment-15755597 ] Kuba Tyszko commented on SPARK-18906: - Well, in csv null can either be an empty field or in this case

[jira] [Updated] (SPARK-18906) CSV parser should return null for empty (or with "") numeric columns.

2016-12-16 Thread Kuba Tyszko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Kuba Tyszko updated SPARK-18906: Description: Spark allows user to set a nullValue that will indicate certain value's translation

[jira] [Commented] (SPARK-18906) CSV parser should return null for empty (or with "") numeric columns.

2016-12-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18906?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755581#comment-15755581 ] Sean Owen commented on SPARK-18906: --- It sounds like you're saying your data has many types of "null" --

[jira] [Commented] (SPARK-18877) Unable to read given csv data. Excepion: java.lang.IllegalArgumentException: requirement failed: Decimal precision 28 exceeds max precision 20

2016-12-16 Thread Dongjoon Hyun (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18877?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755579#comment-15755579 ] Dongjoon Hyun commented on SPARK-18877: --- Yes. I reproduced the bug and found the root cause on

[jira] [Created] (SPARK-18906) CSV parser should return null for empty (or with "") numeric columns.

2016-12-16 Thread Kuba Tyszko (JIRA)
Kuba Tyszko created SPARK-18906: --- Summary: CSV parser should return null for empty (or with "") numeric columns. Key: SPARK-18906 URL: https://issues.apache.org/jira/browse/SPARK-18906 Project: Spark

[jira] [Commented] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

2016-12-16 Thread Ruslan Dautkhanov (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755560#comment-15755560 ] Ruslan Dautkhanov commented on SPARK-12837: --- Yep, we continue to see this issue in Spark 2.. >

[jira] [Commented] (SPARK-18898) Exception not failing Scala applications (in yarn)

2016-12-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18898?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755475#comment-15755475 ] Sean Owen commented on SPARK-18898: --- Probably the same as SPARK-15955 > Exception not failing Scala

[jira] [Resolved] (SPARK-17769) Some FetchFailure refactoring in the DAGScheduler

2016-12-16 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17769?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Marcelo Vanzin resolved SPARK-17769. Resolution: Fixed Fix Version/s: 2.2.0 > Some FetchFailure refactoring in the

[jira] [Updated] (SPARK-18905) Potential Issue of Semantics of BatchCompleted

2016-12-16 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-18905: Description: the current implementation of Spark streaming considers a batch is completed no matter the

[jira] [Updated] (SPARK-18905) Potential Issue of Semantics of BatchCompleted

2016-12-16 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-18905: Description: the current implementation of Spark streaming considers a batch is completed no matter the

[jira] [Updated] (SPARK-18905) Potential Issue of Semantics of BatchCompleted

2016-12-16 Thread Nan Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18905?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Nan Zhu updated SPARK-18905: Description: the current implementation of Spark streaming considers a batch is completed no matter the

[jira] [Created] (SPARK-18905) Potential Issue of Semantics of BatchCompleted

2016-12-16 Thread Nan Zhu (JIRA)
Nan Zhu created SPARK-18905: --- Summary: Potential Issue of Semantics of BatchCompleted Key: SPARK-18905 URL: https://issues.apache.org/jira/browse/SPARK-18905 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-18903) uiWebUrl is not accessible to SparkR

2016-12-16 Thread Diogo Munaro Vieira (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755440#comment-15755440 ] Diogo Munaro Vieira commented on SPARK-18903: - Hey Sean, I don't know a way to get what port

[jira] [Updated] (SPARK-18904) Merge two FileStreamSourceSuite files

2016-12-16 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18904: - Priority: Minor (was: Trivial) > Merge two FileStreamSourceSuite files >

[jira] [Updated] (SPARK-18904) Merge two FileStreamSourceSuite files

2016-12-16 Thread Shixiong Zhu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shixiong Zhu updated SPARK-18904: - Priority: Trivial (was: Major) > Merge two FileStreamSourceSuite files >

[jira] [Assigned] (SPARK-18904) Merge two FileStreamSourceSuite files

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18904: Assignee: Apache Spark (was: Shixiong Zhu) > Merge two FileStreamSourceSuite files >

[jira] [Assigned] (SPARK-18904) Merge two FileStreamSourceSuite files

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18904?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-18904: Assignee: Shixiong Zhu (was: Apache Spark) > Merge two FileStreamSourceSuite files >

[jira] [Commented] (SPARK-18904) Merge two FileStreamSourceSuite files

2016-12-16 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18904?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755436#comment-15755436 ] Apache Spark commented on SPARK-18904: -- User 'zsxwing' has created a pull request for this issue:

[jira] [Created] (SPARK-18904) Merge two FileStreamSourceSuite files

2016-12-16 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-18904: Summary: Merge two FileStreamSourceSuite files Key: SPARK-18904 URL: https://issues.apache.org/jira/browse/SPARK-18904 Project: Spark Issue Type: Test

[jira] [Commented] (SPARK-18902) Include Apache License in R source Package

2016-12-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755431#comment-15755431 ] Sean Owen commented on SPARK-18902: --- Oh, I missed DESCRIPTION. If that's the standard place to put it,

[jira] [Commented] (SPARK-18903) uiWebUrl is not accessible to SparkR

2016-12-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18903?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755428#comment-15755428 ] Sean Owen commented on SPARK-18903: --- (Does it really need to be?) > uiWebUrl is not accessible to

[jira] [Updated] (SPARK-18903) uiWebUrl is not accessible to SparkR

2016-12-16 Thread Diogo Munaro Vieira (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18903?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Diogo Munaro Vieira updated SPARK-18903: Description: Like https://issues.apache.org/jira/browse/SPARK-17437 uiWebUrl is

[jira] [Created] (SPARK-18903) uiWebUrl is not accessible to SparkR

2016-12-16 Thread Diogo Munaro Vieira (JIRA)
Diogo Munaro Vieira created SPARK-18903: --- Summary: uiWebUrl is not accessible to SparkR Key: SPARK-18903 URL: https://issues.apache.org/jira/browse/SPARK-18903 Project: Spark Issue

[jira] [Commented] (SPARK-18902) Include Apache License in R source Package

2016-12-16 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755386#comment-15755386 ] Shivaram Venkataraman commented on SPARK-18902: --- A couple of points - We already include a

[jira] [Commented] (SPARK-18902) Include Apache License in R source Package

2016-12-16 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755385#comment-15755385 ] Shivaram Venkataraman commented on SPARK-18902: --- A couple of points - We already include a

[jira] [Commented] (SPARK-17073) generate basic stats for column

2016-12-16 Thread Ioana Delaney (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17073?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755367#comment-15755367 ] Ioana Delaney commented on SPARK-17073: --- [~mikewzh] FYI, I ran the ANALYZE COMPUTE STATISTICS FOR

[jira] [Created] (SPARK-18902) Include Apache License in R source Package

2016-12-16 Thread Shivaram Venkataraman (JIRA)
Shivaram Venkataraman created SPARK-18902: - Summary: Include Apache License in R source Package Key: SPARK-18902 URL: https://issues.apache.org/jira/browse/SPARK-18902 Project: Spark

[jira] [Commented] (SPARK-18648) spark-shell --jars option does not add jars to classpath on windows

2016-12-16 Thread Michel Lemay (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755317#comment-15755317 ] Michel Lemay commented on SPARK-18648: -- It''s related because the path given to textFile is on a

[jira] [Updated] (SPARK-18897) Fix SparkR SQL Test to drop test table

2016-12-16 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-18897: -- Fix Version/s: (was: 2.1.0) 2.2.0

[jira] [Updated] (SPARK-18897) Fix SparkR SQL Test to drop test table

2016-12-16 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman updated SPARK-18897: -- Assignee: Dongjoon Hyun > Fix SparkR SQL Test to drop test table >

[jira] [Resolved] (SPARK-18897) Fix SparkR SQL Test to drop test table

2016-12-16 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Shivaram Venkataraman resolved SPARK-18897. --- Resolution: Fixed Fix Version/s: 2.1.0 2.0.3 Issue

[jira] [Created] (SPARK-18901) Require in LR LogisticAggregator is redundant

2016-12-16 Thread yuhao yang (JIRA)
yuhao yang created SPARK-18901: -- Summary: Require in LR LogisticAggregator is redundant Key: SPARK-18901 URL: https://issues.apache.org/jira/browse/SPARK-18901 Project: Spark Issue Type:

[jira] [Commented] (SPARK-18648) spark-shell --jars option does not add jars to classpath on windows

2016-12-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18648?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755223#comment-15755223 ] Sean Owen commented on SPARK-18648: --- That is unrelated though. It's an error from the path you pass to

[jira] [Commented] (SPARK-16032) Audit semantics of various insertion operations related to partitioned tables

2016-12-16 Thread Ryan Blue (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15755221#comment-15755221 ] Ryan Blue commented on SPARK-16032: --- +1 > Audit semantics of various insertion operations related to

[jira] [Resolved] (SPARK-16032) Audit semantics of various insertion operations related to partitioned tables

2016-12-16 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-16032?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Sean Owen resolved SPARK-16032. --- Resolution: Done > Audit semantics of various insertion operations related to partitioned tables >

  1   2   >