[jira] [Updated] (SPARK-3468) Provide timeline view in Job and Stage pages

2015-04-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3468: --- Summary: Provide timeline view in Job and Stage pages (was: WebUI Timeline-View feature) > P

[jira] [Updated] (SPARK-3468) Provide timeline view in Job and Stage UI pages

2015-04-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3468: --- Summary: Provide timeline view in Job and Stage UI pages (was: Provide timeline view in Job a

[jira] [Created] (SPARK-6943) Graphically show RDD's included in a stage

2015-04-15 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-6943: -- Summary: Graphically show RDD's included in a stage Key: SPARK-6943 URL: https://issues.apache.org/jira/browse/SPARK-6943 Project: Spark Issue Type: Sub-

[jira] [Updated] (SPARK-3468) WebUI Timeline-View feature

2015-04-15 Thread Patrick Wendell (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3468?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Patrick Wendell updated SPARK-3468: --- Issue Type: Sub-task (was: New Feature) Parent: SPARK-6942 > WebUI Timeline-View feat

[jira] [Commented] (SPARK-6893) Better handling of pipeline parameters in PySpark

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6893?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496853#comment-14496853 ] Apache Spark commented on SPARK-6893: - User 'mengxr' has created a pull request for th

[jira] [Assigned] (SPARK-6893) Better handling of pipeline parameters in PySpark

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6893: --- Assignee: Xiangrui Meng (was: Apache Spark) > Better handling of pipeline parameters in PySp

[jira] [Created] (SPARK-6942) Umbrella: UI Visualizations for Core and Dataframes

2015-04-15 Thread Patrick Wendell (JIRA)
Patrick Wendell created SPARK-6942: -- Summary: Umbrella: UI Visualizations for Core and Dataframes Key: SPARK-6942 URL: https://issues.apache.org/jira/browse/SPARK-6942 Project: Spark Issue

[jira] [Assigned] (SPARK-6893) Better handling of pipeline parameters in PySpark

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6893?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6893: --- Assignee: Apache Spark (was: Xiangrui Meng) > Better handling of pipeline parameters in PySp

[jira] [Resolved] (SPARK-6844) Memory leak occurs when register temp table with cache table on

2015-04-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6844?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6844. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5475 [https:/

[jira] [Commented] (SPARK-4414) SparkContext.wholeTextFiles Doesn't work with S3 Buckets

2015-04-15 Thread Eldon Stegall (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4414?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496829#comment-14496829 ] Eldon Stegall commented on SPARK-4414: -- I am currently seeing this issue with spark 1

[jira] [Resolved] (SPARK-6638) optimize StringType in SQL

2015-04-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6638?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6638. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5350 [https:/

[jira] [Updated] (SPARK-5634) History server shows misleading message when there are no incomplete apps

2015-04-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5634?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen updated SPARK-5634: -- Fix Version/s: (was: 1.2.3) > History server shows misleading message when there are no incomplete a

[jira] [Resolved] (SPARK-6887) ColumnBuilder misses FloatType

2015-04-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6887. - Resolution: Fixed Issue resolved by pull request 5499 [https://github.com/apache/spark/pul

[jira] [Closed] (SPARK-6745) Develop a general filter function to be used in PrunedFilteredScan and CatalystScan

2015-04-15 Thread Alex Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6745?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Alex Liu closed SPARK-6745. --- Resolution: Won't Fix I close it if some one is interested in it, please reopen it. > Develop a general filte

[jira] [Resolved] (SPARK-6800) Reading from JDBC with SQLContext, using lower/upper bounds and numPartitions gives incorrect results.

2015-04-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6800?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6800. - Resolution: Fixed Fix Version/s: 1.4.0 1.3.1 Issue resolved by p

[jira] [Resolved] (SPARK-6730) Can't have table as identifier in OPTIONS

2015-04-15 Thread Michael Armbrust (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6730?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Michael Armbrust resolved SPARK-6730. - Resolution: Fixed Fix Version/s: 1.4.0 Issue resolved by pull request 5520 [https:/

[jira] [Resolved] (SPARK-5720) `Create Table Like` in HiveContext need support `like registered temporary table`

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5720?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-5720. - Resolution: Duplicate Seems SPARK-4944 and this one are for the same issue. I am resolving it. > `Create

[jira] [Commented] (SPARK-4892) java.io.FileNotFound exceptions when creating EXTERNAL hive tables

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4892?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496744#comment-14496744 ] Yin Huai commented on SPARK-4892: - https://issues.apache.org/jira/browse/HIVE-7633 is the

[jira] [Commented] (SPARK-6217) insertInto doesn't work in PySpark

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496706#comment-14496706 ] Yin Huai commented on SPARK-6217: - SPARK-6941 is used to track the work of better error me

[jira] [Updated] (SPARK-6941) Provide a better error message to explain that tables created from RDDs are immutable

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6941?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6941: Description: We should explicitly let users know that tables created from RDDs are immutable and new rows c

[jira] [Created] (SPARK-6941) Provide a better error message to explain that tables created from RDDs are immutable

2015-04-15 Thread Yin Huai (JIRA)
Yin Huai created SPARK-6941: --- Summary: Provide a better error message to explain that tables created from RDDs are immutable Key: SPARK-6941 URL: https://issues.apache.org/jira/browse/SPARK-6941 Project: Sp

[jira] [Commented] (SPARK-6831) Document how to use external data sources

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496700#comment-14496700 ] Yin Huai commented on SPARK-6831: - OK, makes sense. Let's use this JIRA to track the doc i

[jira] [Updated] (SPARK-6831) Document how to use external data sources

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6831: Priority: Blocker (was: Critical) > Document how to use external data sources > ---

[jira] [Resolved] (SPARK-5003) cast support date data type

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5003?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-5003. - Resolution: Fixed This issue has been resolved. > cast support date data type >

[jira] [Commented] (SPARK-6940) PySpark ML.Tuning Wrappers are missing

2015-04-15 Thread Omede Firouz (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6940?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496672#comment-14496672 ] Omede Firouz commented on SPARK-6940: - I'm beginning work on this ticket, please let m

[jira] [Created] (SPARK-6940) PySpark ML.Tuning Wrappers are missing

2015-04-15 Thread Omede Firouz (JIRA)
Omede Firouz created SPARK-6940: --- Summary: PySpark ML.Tuning Wrappers are missing Key: SPARK-6940 URL: https://issues.apache.org/jira/browse/SPARK-6940 Project: Spark Issue Type: Improvement

[jira] [Closed] (SPARK-4967) File name with comma will cause exception for SQLContext.parquetFile

2015-04-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4967?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Hao closed SPARK-4967. Resolution: Won't Fix > File name with comma will cause exception for SQLContext.parquetFile > -

[jira] [Commented] (SPARK-4521) Parquet fails to read columns with spaces in the name

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496646#comment-14496646 ] Yin Huai commented on SPARK-4521: - https://github.com/apache/spark/pull/5263 is for Spark-

[jira] [Commented] (SPARK-4967) File name with comma will cause exception for SQLContext.parquetFile

2015-04-15 Thread Cheng Hao (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496645#comment-14496645 ] Cheng Hao commented on SPARK-4967: -- Thanks for explanation, I will close this issue. > F

[jira] [Commented] (SPARK-4944) Table Not Found exception in "Create Table Like registered RDD table"

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496641#comment-14496641 ] Yin Huai commented on SPARK-4944: - Seems we have to handle Create Table Like in Spark SQL

[jira] [Commented] (SPARK-4967) File name with comma will cause exception for SQLContext.parquetFile

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4967?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496635#comment-14496635 ] Yin Huai commented on SPARK-4967: - For new parquet, because we only accept a single path (

[jira] [Updated] (SPARK-5741) Support the path contains comma in HiveContext

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-5741: Assignee: Yadong Qi (was: Yin Huai) > Support the path contains comma in HiveContext >

[jira] [Commented] (SPARK-6217) insertInto doesn't work in PySpark

2015-04-15 Thread Charles Cloud (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496626#comment-14496626 ] Charles Cloud commented on SPARK-6217: -- Was there a more informative error message ad

[jira] [Assigned] (SPARK-5741) Support the path contains comma in HiveContext

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5741?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai reassigned SPARK-5741: --- Assignee: Yin Huai > Support the path contains comma in HiveContext > ---

[jira] [Commented] (SPARK-5133) Feature Importance for Decision Tree (Ensembles)

2015-04-15 Thread Parv Oberoi (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5133?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496625#comment-14496625 ] Parv Oberoi commented on SPARK-5133: this would be a really useful feature to have in

[jira] [Updated] (SPARK-6937) Tiny bug in PowerIterationClusteringExample in which radius not accepted from command line

2015-04-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6937: - Summary: Tiny bug in PowerIterationClusteringExample in which radius not accepted from com

[jira] [Updated] (SPARK-6915) VectorIndexer improvements

2015-04-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6915?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley updated SPARK-6915: - Description: This covers several improvements to VectorIndexer. They could be handled se

[jira] [Closed] (SPARK-6916) StringIndexer should preserve non-ML metadata

2015-04-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6916?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Joseph K. Bradley closed SPARK-6916. Resolution: Not A Problem Target Version/s: (was: 1.4.0) > StringIndexer should

[jira] [Assigned] (SPARK-6939) Refactoring existing batch statistics into the new UI

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6939: --- Assignee: Apache Spark > Refactoring existing batch statistics into the new UI >

[jira] [Assigned] (SPARK-6939) Refactoring existing batch statistics into the new UI

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6939?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6939: --- Assignee: (was: Apache Spark) > Refactoring existing batch statistics into the new UI > -

[jira] [Commented] (SPARK-6939) Refactoring existing batch statistics into the new UI

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496608#comment-14496608 ] Apache Spark commented on SPARK-6939: - User 'zsxwing' has created a pull request for t

[jira] [Resolved] (SPARK-2984) FileNotFoundException on _temporary directory

2015-04-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2984?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Josh Rosen resolved SPARK-2984. --- Resolution: Fixed Fix Version/s: 1.3.0 Assignee: Josh Rosen For speculative tasks, thi

[jira] [Created] (SPARK-6939) Refactoring existing batch statistics into the new UI

2015-04-15 Thread Shixiong Zhu (JIRA)
Shixiong Zhu created SPARK-6939: --- Summary: Refactoring existing batch statistics into the new UI Key: SPARK-6939 URL: https://issues.apache.org/jira/browse/SPARK-6939 Project: Spark Issue Type:

[jira] [Created] (SPARK-6938) Add informative error messages to require statements.

2015-04-15 Thread Juliet Hougland (JIRA)
Juliet Hougland created SPARK-6938: -- Summary: Add informative error messages to require statements. Key: SPARK-6938 URL: https://issues.apache.org/jira/browse/SPARK-6938 Project: Spark Issue

[jira] [Resolved] (SPARK-4804) StringContext method to allow using Strings for column names in catalyst DSL

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4804?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-4804. - Resolution: Duplicate I has been resolved by SPARK-5040. > StringContext method to allow using Strings fo

[jira] [Assigned] (SPARK-6938) Add informative error messages to require statements.

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6938: --- Assignee: (was: Apache Spark) > Add informative error messages to require statements. > -

[jira] [Commented] (SPARK-6938) Add informative error messages to require statements.

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6938?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496575#comment-14496575 ] Apache Spark commented on SPARK-6938: - User 'jhlch' has created a pull request for thi

[jira] [Assigned] (SPARK-6938) Add informative error messages to require statements.

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6938?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6938: --- Assignee: Apache Spark > Add informative error messages to require statements. >

[jira] [Resolved] (SPARK-6060) List type missing for catalyst's package.scala

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6060?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-6060. - Resolution: Not A Problem [~stedre] Unfortunately, sometimes we need to do {{clean}} first. I am resolvin

[jira] [Updated] (SPARK-4176) Support decimals with precision > 18 in Parquet

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4176: -- Priority: Major (was: Critical) > Support decimals with precision > 18 in Parquet > ---

[jira] [Updated] (SPARK-6694) SparkSQL CLI must be able to specify an option --database on the command line.

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6694?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6694: -- Priority: Critical (was: Major) > SparkSQL CLI must be able to specify an option --database on the comm

[jira] [Updated] (SPARK-4176) Support decimals with precision > 18 in Parquet

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4176: -- Priority: Critical (was: Major) > Support decimals with precision > 18 in Parquet > ---

[jira] [Updated] (SPARK-6869) Pass PYTHONPATH to executor, so that executor can read pyspark file from local file system on executor node

2015-04-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6869: - Affects Version/s: (was: 1.1.0) 1.0.0 > Pass PYTHONPATH to executor, so that ex

[jira] [Resolved] (SPARK-6657) Fix Python doc build warnings

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-6657. - Resolution: Fixed Fix Version/s: 1.3.1 The pr has been merged. I am resolving it. > Fix Python doc

[jira] [Updated] (SPARK-6869) Pass PYTHONPATH to executor, so that executor can read pyspark file from local file system on executor node

2015-04-15 Thread Andrew Or (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6869?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Andrew Or updated SPARK-6869: - Affects Version/s: 1.1.0 > Pass PYTHONPATH to executor, so that executor can read pyspark file from > loc

[jira] [Updated] (SPARK-6774) Implement Parquet complex types backwards-compatiblity rules

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6774?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6774: -- Priority: Critical (was: Major) > Implement Parquet complex types backwards-compatiblity rules > --

[jira] [Commented] (SPARK-6831) Document how to use external data sources

2015-04-15 Thread Shivaram Venkataraman (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496548#comment-14496548 ] Shivaram Venkataraman commented on SPARK-6831: -- I think we should give an exa

[jira] [Assigned] (SPARK-4176) Support decimals with precision > 18 in Parquet

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4176?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-4176: - Assignee: Cheng Lian > Support decimals with precision > 18 in Parquet >

[jira] [Updated] (SPARK-6831) Document how to use external data sources

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai updated SPARK-6831: Component/s: Documentation > Document how to use external data sources > ---

[jira] [Commented] (SPARK-4854) Custom UDTF with Lateral View throws ClassNotFound exception in Spark SQL CLI

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496554#comment-14496554 ] Cheng Lian commented on SPARK-4854: --- I guess this one duplicates SPARK-6835. > Custom U

[jira] [Commented] (SPARK-3937) Unsafe memory access inside of Snappy library

2015-04-15 Thread Josh Rosen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496550#comment-14496550 ] Josh Rosen commented on SPARK-3937: --- [~witgo], is there any way to reproduce this withou

[jira] [Commented] (SPARK-4854) Custom UDTF with Lateral View throws ClassNotFound exception in Spark SQL CLI

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4854?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496552#comment-14496552 ] Cheng Lian commented on SPARK-4854: --- [~wanshenghua] Would you mind to verify whether [PR

[jira] [Updated] (SPARK-4521) Parquet fails to read columns with spaces in the name

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4521: -- Description: I think this is actually a bug in parquet, but it would be good to track it here as well.

[jira] [Resolved] (SPARK-6217) insertInto doesn't work in PySpark

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Yin Huai resolved SPARK-6217. - Resolution: Not A Problem We have not implemented the support for inserting into a table created from col

[jira] [Updated] (SPARK-4944) Table Not Found exception in "Create Table Like registered RDD table"

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4944?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4944: -- Description: {code} rdd_table.saveAsParquetFile("/user/spark/my_data.parquet") hiveContext.registerRDDAs

[jira] [Commented] (SPARK-4521) Parquet fails to read columns with spaces in the name

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496542#comment-14496542 ] Cheng Lian commented on SPARK-4521: --- This is because Parquet {{MessageTypeParser}} doesn

[jira] [Updated] (SPARK-4629) Spark SQL uses Hadoop Configuration in a thread-unsafe manner when writing Parquet files

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-4629: -- Priority: Critical (was: Major) > Spark SQL uses Hadoop Configuration in a thread-unsafe manner when wr

[jira] [Assigned] (SPARK-4629) Spark SQL uses Hadoop Configuration in a thread-unsafe manner when writing Parquet files

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4629?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-4629: - Assignee: Cheng Lian > Spark SQL uses Hadoop Configuration in a thread-unsafe manner when writing

[jira] [Assigned] (SPARK-5251) Using `tableIdentifier` in hive metastore

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5251: --- Assignee: (was: Apache Spark) > Using `tableIdentifier` in hive metastore >

[jira] [Assigned] (SPARK-5251) Using `tableIdentifier` in hive metastore

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5251?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-5251: --- Assignee: Apache Spark > Using `tableIdentifier` in hive metastore > ---

[jira] [Assigned] (SPARK-6937) [MLLIB] Tiny bug in PowerIterationClusteringExample in which radius not accepted from command line

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6937: --- Assignee: Apache Spark > [MLLIB] Tiny bug in PowerIterationClusteringExample in which radius

[jira] [Assigned] (SPARK-6937) [MLLIB] Tiny bug in PowerIterationClusteringExample in which radius not accepted from command line

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6937?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Apache Spark reassigned SPARK-6937: --- Assignee: (was: Apache Spark) > [MLLIB] Tiny bug in PowerIterationClusteringExample in wh

[jira] [Commented] (SPARK-6937) [MLLIB] Tiny bug in PowerIterationClusteringExample in which radius not accepted from command line

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496523#comment-14496523 ] Apache Spark commented on SPARK-6937: - User 'javadba' has created a pull request for t

[jira] [Updated] (SPARK-6123) Parquet reader should use the schema of every file to create converter

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6123: -- Priority: Critical (was: Major) > Parquet reader should use the schema of every file to create converte

[jira] [Created] (SPARK-6937) [MLLIB] Tiny bug in PowerIterationClusteringExample in which radius not accepted from command line

2015-04-15 Thread Stephen Boesch (JIRA)
Stephen Boesch created SPARK-6937: - Summary: [MLLIB] Tiny bug in PowerIterationClusteringExample in which radius not accepted from command line Key: SPARK-6937 URL: https://issues.apache.org/jira/browse/SPARK-6937

[jira] [Commented] (SPARK-6217) insertInto doesn't work in PySpark

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496522#comment-14496522 ] Yin Huai commented on SPARK-6217: - [~cpcloud] Right now, we do not support inserting into

[jira] [Updated] (SPARK-5947) First class partitioning support in data sources API

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5947?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-5947: -- Priority: Blocker (was: Major) > First class partitioning support in data sources API > ---

[jira] [Updated] (SPARK-5948) Support writing to partitioned table for the Parquet data source

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5948?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-5948: -- Priority: Blocker (was: Major) > Support writing to partitioned table for the Parquet data source > ---

[jira] [Assigned] (SPARK-6123) Parquet reader should use the schema of every file to create converter

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6123?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-6123: - Assignee: Cheng Lian > Parquet reader should use the schema of every file to create converter > -

[jira] [Commented] (SPARK-6889) Streamline contribution process with update to Contribution wiki, JIRA rules

2015-04-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496481#comment-14496481 ] Sean Owen commented on SPARK-6889: -- We can take away old docs that encourage people to he

[jira] [Commented] (SPARK-6548) Adding stddev to DataFrame functions

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6548?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496477#comment-14496477 ] Cheng Lian commented on SPARK-6548: --- Hey [~dreamquster], are you still working on this?

[jira] [Commented] (SPARK-6113) Stabilize DecisionTree and ensembles APIs

2015-04-15 Thread Apache Spark (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6113?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496475#comment-14496475 ] Apache Spark commented on SPARK-6113: - User 'jkbradley' has created a pull request for

[jira] [Commented] (SPARK-6933) Thrift Server couldn't strip .inprogress suffix after being stopped

2015-04-15 Thread Marcelo Vanzin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496474#comment-14496474 ] Marcelo Vanzin commented on SPARK-6933: --- I think this is actually caused by the same

[jira] [Created] (SPARK-6936) SQLContext.sql() caused deadlock in multi-thread env

2015-04-15 Thread Paul Wu (JIRA)
Paul Wu created SPARK-6936: -- Summary: SQLContext.sql() caused deadlock in multi-thread env Key: SPARK-6936 URL: https://issues.apache.org/jira/browse/SPARK-6936 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-6012) Deadlock when asking for partitions from CoalescedRDD on top of a TakeOrdered operator

2015-04-15 Thread Yin Huai (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6012?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496469#comment-14496469 ] Yin Huai commented on SPARK-6012: - [~maxseiden] Can you try 1.3 and see if this issue has

[jira] [Updated] (SPARK-6432) Cannot load parquet data with partitions if not all partition columns match data columns

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6432?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6432: -- Priority: Critical (was: Major) > Cannot load parquet data with partitions if not all partition columns

[jira] [Commented] (SPARK-6916) StringIndexer should preserve non-ML metadata

2015-04-15 Thread Joseph K. Bradley (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6916?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496456#comment-14496456 ] Joseph K. Bradley commented on SPARK-6916: -- Yeah, I guess I don't know what users

[jira] [Updated] (SPARK-6935) spark/spark-ec2.py add parameters to give different instance types for master and slaves

2015-04-15 Thread Oleksii Mandrychenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6935?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Oleksii Mandrychenko updated SPARK-6935: Description: I want to start a cluster where I give beefy AWS instances to slaves, s

[jira] [Commented] (SPARK-6935) spark/spark-ec2.py add parameters to give different instance types for master and slaves

2015-04-15 Thread Oleksii Mandrychenko (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496453#comment-14496453 ] Oleksii Mandrychenko commented on SPARK-6935: - Added support for default flag

[jira] [Commented] (SPARK-6935) spark/spark-ec2.py add parameters to give different instance types for master and slaves

2015-04-15 Thread Sean Owen (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6935?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496443#comment-14496443 ] Sean Owen commented on SPARK-6935: -- Sounds pretty reasonable, though you may need to supp

[jira] [Commented] (SPARK-6933) Thrift Server couldn't strip .inprogress suffix after being stopped

2015-04-15 Thread Tao Wang (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6933?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496437#comment-14496437 ] Tao Wang commented on SPARK-6933: - P.P.S: Tested with SparkPi, it worked fine. Now this is

[jira] [Updated] (SPARK-6482) Remove synchronization of Hive Native commands

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6482: -- Priority: Critical (was: Major) > Remove synchronization of Hive Native commands >

[jira] [Updated] (SPARK-6570) Spark SQL arrays: "explode()" fails and cannot save array type to Parquet

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6570: -- Priority: Critical (was: Major) > Spark SQL arrays: "explode()" fails and cannot save array type to Par

[jira] [Assigned] (SPARK-6570) Spark SQL arrays: "explode()" fails and cannot save array type to Parquet

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6570?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-6570: - Assignee: Cheng Lian > Spark SQL arrays: "explode()" fails and cannot save array type to Parquet

[jira] [Updated] (SPARK-6581) Metadata is missing when saving parquet file using hadoop 1.0.4

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6581: -- Priority: Critical (was: Major) > Metadata is missing when saving parquet file using hadoop 1.0.4 > ---

[jira] [Assigned] (SPARK-6581) Metadata is missing when saving parquet file using hadoop 1.0.4

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6581?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian reassigned SPARK-6581: - Assignee: Cheng Lian > Metadata is missing when saving parquet file using hadoop 1.0.4 >

[jira] [Updated] (SPARK-6759) Do not borrow/release a kryo instance for every value in a complex type value when doing serialization/deserialization in in-memory columnar store

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6759?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6759: -- Priority: Critical (was: Major) > Do not borrow/release a kryo instance for every value in a complex ty

[jira] [Updated] (SPARK-6777) Implement backwards-compatibility rules in Parquet schema converters

2015-04-15 Thread Cheng Lian (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6777?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Cheng Lian updated SPARK-6777: -- Priority: Critical (was: Major) > Implement backwards-compatibility rules in Parquet schema converters

[jira] [Created] (SPARK-6935) spark/spark-ec2.py add parameters to give different instance types for master and slaves

2015-04-15 Thread Oleksii Mandrychenko (JIRA)
Oleksii Mandrychenko created SPARK-6935: --- Summary: spark/spark-ec2.py add parameters to give different instance types for master and slaves Key: SPARK-6935 URL: https://issues.apache.org/jira/browse/SPARK-69

[jira] [Commented] (SPARK-6889) Streamline contribution process with update to Contribution wiki, JIRA rules

2015-04-15 Thread Nicholas Chammas (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6889?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14496373#comment-14496373 ] Nicholas Chammas commented on SPARK-6889: - {quote} I think that really the most im

<    1   2   3   >