[jira] [Commented] (SPARK-9725) spark sql query string field return empty/garbled string

[ https://issues.apache.org/jira/browse/SPARK-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697426#comment-14697426 ] Davies Liu commented on SPARK-9725: --- I can reproduce this issue now (<32G for executor,

[jira] [Commented] (SPARK-9725) spark sql query string field return empty/garbled string

[ https://issues.apache.org/jira/browse/SPARK-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14697378#comment-14697378 ] Davies Liu commented on SPARK-9725: --- So the problem came from serialization of UTF8Strin

[jira] [Resolved] (SPARK-9945) pageSize is calculated from driver.memory instead of executor.memory

[ https://issues.apache.org/jira/browse/SPARK-9945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9945. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8175 [https://github.com/

[jira] [Resolved] (SPARK-9943) Failed to serialize a deserialized UnsafeHashedRelation

[ https://issues.apache.org/jira/browse/SPARK-9943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9943. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8174 [https://github.com/

[jira] [Resolved] (SPARK-8976) Python 3 crash: ValueError: invalid mode 'a+' (only r, w, b allowed)

[ https://issues.apache.org/jira/browse/SPARK-8976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8976. --- Resolution: Fixed Fix Version/s: 1.5.0 1.4.2 Issue resolved by pull request

[jira] [Created] (SPARK-9955) TPCDS Q8 failed in 1.5

Davies Liu created SPARK-9955: - Summary: TPCDS Q8 failed in 1.5 Key: SPARK-9955 URL: https://issues.apache.org/jira/browse/SPARK-9955 Project: Spark Issue Type: Bug Components: SQL

[jira] [Commented] (SPARK-9725) spark sql query string field return empty/garbled string

[ https://issues.apache.org/jira/browse/SPARK-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14696043#comment-14696043 ] Davies Liu commented on SPARK-9725: --- This PR is not for this JIRA, was linked by mistake

[jira] [Updated] (SPARK-9946) NPE in TaskMemoryManager

[ https://issues.apache.org/jira/browse/SPARK-9946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9946: -- Summary: NPE in TaskMemoryManager (was: TaskMemoryManager is not thread safe) > NPE in TaskMemoryManag

[jira] [Created] (SPARK-9946) TaskMemoryManager is not thread safe

Davies Liu created SPARK-9946: - Summary: TaskMemoryManager is not thread safe Key: SPARK-9946 URL: https://issues.apache.org/jira/browse/SPARK-9946 Project: Spark Issue Type: Bug Compon

[jira] [Assigned] (SPARK-9945) pageSize is calculated from driver.memory instead of executor.memory

[ https://issues.apache.org/jira/browse/SPARK-9945?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-9945: - Assignee: Davies Liu > pageSize is calculated from driver.memory instead of executor.memory > ---

[jira] [Created] (SPARK-9945) pageSize is calculated from driver.memory instead of executor.memory

Davies Liu created SPARK-9945: - Summary: pageSize is calculated from driver.memory instead of executor.memory Key: SPARK-9945 URL: https://issues.apache.org/jira/browse/SPARK-9945 Project: Spark

[jira] [Created] (SPARK-9943) Failed to serialize a deserialized UnsafeHashedRelation

Davies Liu created SPARK-9943: - Summary: Failed to serialize a deserialized UnsafeHashedRelation Key: SPARK-9943 URL: https://issues.apache.org/jira/browse/SPARK-9943 Project: Spark Issue Type: B

[jira] [Created] (SPARK-9942) Broken pandas could crash PySpark SQL

Davies Liu created SPARK-9942: - Summary: Broken pandas could crash PySpark SQL Key: SPARK-9942 URL: https://issues.apache.org/jira/browse/SPARK-9942 Project: Spark Issue Type: Bug Compo

[jira] [Commented] (SPARK-9725) spark sql query string field return empty/garbled string

[ https://issues.apache.org/jira/browse/SPARK-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14695652#comment-14695652 ] Davies Liu commented on SPARK-9725: --- [~lian cheng] [~yhuai] Can you reproduce this issue

[jira] [Commented] (SPARK-9604) Unsafe ArrayData and MapData is very very slow

[ https://issues.apache.org/jira/browse/SPARK-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14695617#comment-14695617 ] Davies Liu commented on SPARK-9604: --- [~cloud_fan] Yeah, The test looks much better now.

[jira] [Commented] (SPARK-7379) pickle.loads expects a string instead of bytes in Python 3.

[ https://issues.apache.org/jira/browse/SPARK-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14695614#comment-14695614 ] Davies Liu commented on SPARK-7379: --- Could you post the full stacktrace or a way to repr

[jira] [Assigned] (SPARK-7379) pickle.loads expects a string instead of bytes in Python 3.

[ https://issues.apache.org/jira/browse/SPARK-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-7379: - Assignee: Davies Liu > pickle.loads expects a string instead of bytes in Python 3. >

[jira] [Resolved] (SPARK-9499) Possible file handle leak in spilling/sort code

[ https://issues.apache.org/jira/browse/SPARK-9499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9499. --- Resolution: Duplicate Assignee: Davies Liu (was: Josh Rosen) Fix Version/s: 1.5.0 > P

[jira] [Updated] (SPARK-9931) Flaky test: mllib/tests.py StreamingLogisticRegressionWithSGDTests. test_training_and_prediction

[ https://issues.apache.org/jira/browse/SPARK-9931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9931: -- Priority: Critical (was: Major) > Flaky test: mllib/tests.py StreamingLogisticRegressionWithSGDTests.

[jira] [Created] (SPARK-9931) Flaky test: mllib/tests.py StreamingLogisticRegressionWithSGDTests. test_training_and_prediction

Davies Liu created SPARK-9931: - Summary: Flaky test: mllib/tests.py StreamingLogisticRegressionWithSGDTests. test_training_and_prediction Key: SPARK-9931 URL: https://issues.apache.org/jira/browse/SPARK-9931

[jira] [Commented] (SPARK-9868) auto_sortmerge_join_8 fails non-deterministically in Jenkins

[ https://issues.apache.org/jira/browse/SPARK-9868?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14694716#comment-14694716 ] Davies Liu commented on SPARK-9868: --- [~joshrosen]I can look into this one, since you are

[jira] [Assigned] (SPARK-9868) auto_sortmerge_join_8 fails non-deterministically in Jenkins

[ https://issues.apache.org/jira/browse/SPARK-9868?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-9868: - Assignee: Davies Liu > auto_sortmerge_join_8 fails non-deterministically in Jenkins > ---

[jira] [Resolved] (SPARK-9845) Add built-in UDF

[ https://issues.apache.org/jira/browse/SPARK-9845?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9845. --- Resolution: Duplicate Fix Version/s: 1.5.0 We already added nearly 100+ functions in 1.5 releas

[jira] [Updated] (SPARK-9887) After recent hive patches PySpark fails with IllegalArgumentException: Wrong FS: hdfs:

[ https://issues.apache.org/jira/browse/SPARK-9887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9887: -- Assignee: Marcelo Vanzin > After recent hive patches PySpark fails with IllegalArgumentException: Wrong

[jira] [Resolved] (SPARK-9887) After recent hive patches PySpark fails with IllegalArgumentException: Wrong FS: hdfs:

[ https://issues.apache.org/jira/browse/SPARK-9887?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9887. --- Resolution: Duplicate Fix Version/s: 1.5.0 > After recent hive patches PySpark fails with Illeg

[jira] [Resolved] (SPARK-9921) Too many open files in Spark SQL

[ https://issues.apache.org/jira/browse/SPARK-9921?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9921. --- Resolution: Duplicate Assignee: Davies Liu Fix Version/s: 1.5.0 > Too many open files

[jira] [Assigned] (SPARK-9827) Too many open files in TungstenExchange

[ https://issues.apache.org/jira/browse/SPARK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-9827: - Assignee: Davies Liu (was: Josh Rosen) > Too many open files in TungstenExchange > -

[jira] [Updated] (SPARK-9908) TPCDS Q98 failed when tungsten is off

[ https://issues.apache.org/jira/browse/SPARK-9908?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9908: -- Description: {code} org.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage

[jira] [Created] (SPARK-9908) TPCDS Q98 failed when tungsten is off

Davies Liu created SPARK-9908: - Summary: TPCDS Q98 failed when tungsten is off Key: SPARK-9908 URL: https://issues.apache.org/jira/browse/SPARK-9908 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-9832) TPCDS Q98 Fails

[ https://issues.apache.org/jira/browse/SPARK-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14693982#comment-14693982 ] Davies Liu commented on SPARK-9832: --- [~marmbrus] It also fail if unsafe and codegen are

[jira] [Resolved] (SPARK-9831) TPCDS Q73 Fails

[ https://issues.apache.org/jira/browse/SPARK-9831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9831. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8117 [https://github.com/

[jira] [Created] (SPARK-9865) Flaky SparkR test: test_sparkSQL.R: sample on a DataFrame

Davies Liu created SPARK-9865: - Summary: Flaky SparkR test: test_sparkSQL.R: sample on a DataFrame Key: SPARK-9865 URL: https://issues.apache.org/jira/browse/SPARK-9865 Project: Spark Issue Type:

[jira] [Assigned] (SPARK-9831) TPCDS Q73 Fails

[ https://issues.apache.org/jira/browse/SPARK-9831?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-9831: - Assignee: Davies Liu > TPCDS Q73 Fails > --- > > Key: SPARK-9831 >

[jira] [Assigned] (SPARK-9832) TPCDS Q98 Fails

[ https://issues.apache.org/jira/browse/SPARK-9832?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-9832: - Assignee: Davies Liu > TPCDS Q98 Fails > --- > > Key: SPARK-9832 >

[jira] [Updated] (SPARK-9829) peakExecutionMemory is not correct

[ https://issues.apache.org/jira/browse/SPARK-9829?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9829: -- Description: When run a query with 8G memory, the peakExecutionMemory in WebUI said that 40344371200 (4

[jira] [Created] (SPARK-9829) peakExecutionMemory is not correct

Davies Liu created SPARK-9829: - Summary: peakExecutionMemory is not correct Key: SPARK-9829 URL: https://issues.apache.org/jira/browse/SPARK-9829 Project: Spark Issue Type: Bug Componen

[jira] [Updated] (SPARK-9827) Too many open files in TungstenExchange

[ https://issues.apache.org/jira/browse/SPARK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9827: -- Description: When run q19 on TPCDS (scale=5) dataset with 8G memory, it open 10k shuffle files, crash m

[jira] [Updated] (SPARK-9827) Too many open files in TungstenExchange

[ https://issues.apache.org/jira/browse/SPARK-9827?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9827: -- Description: When run q19 on TPCDS (scale=5) dataset with 8G memory, it open 10k shuffle files, crash m

[jira] [Created] (SPARK-9827) Too many open files in TungstenExchange

Davies Liu created SPARK-9827: - Summary: Too many open files in TungstenExchange Key: SPARK-9827 URL: https://issues.apache.org/jira/browse/SPARK-9827 Project: Spark Issue Type: Bug Com

[jira] [Updated] (SPARK-8976) Python 3 crash: ValueError: invalid mode 'a+' (only r, w, b allowed)

[ https://issues.apache.org/jira/browse/SPARK-8976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-8976: -- Assignee: Davies Liu Target Version/s: 1.5.0 > Python 3 crash: ValueError: invalid mode 'a+'

[jira] [Updated] (SPARK-9822) Update doc about supported Python versions

[ https://issues.apache.org/jira/browse/SPARK-9822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9822: -- Description: Not sure if here's the right place to post this, but the documentation on the official web

[jira] [Created] (SPARK-9822) Update doc about supported Python versions

Davies Liu created SPARK-9822: - Summary: Update doc about supported Python versions Key: SPARK-9822 URL: https://issues.apache.org/jira/browse/SPARK-9822 Project: Spark Issue Type: Documentation

[jira] [Resolved] (SPARK-9815) Rename PlatformDependent.UNSAFE -> Platform

[ https://issues.apache.org/jira/browse/SPARK-9815?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9815. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8094 [https://github.com/

[jira] [Updated] (SPARK-9633) SBT download locations outdated; need an update

[ https://issues.apache.org/jira/browse/SPARK-9633?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9633: -- Fix Version/s: 1.2.3 1.1.2 1.0.3 > SBT download locations outdated

[jira] [Commented] (SPARK-9633) SBT download locations outdated; need an update

[ https://issues.apache.org/jira/browse/SPARK-9633?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14680696#comment-14680696 ] Davies Liu commented on SPARK-9633: --- cherry-pick this into 1.2, 1.1 and 1.0 branch > SB

[jira] [Updated] (SPARK-9686) Spark hive jdbc client cannot get table from metadata store

[ https://issues.apache.org/jira/browse/SPARK-9686?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9686: -- Assignee: Cheng Lian > Spark hive jdbc client cannot get table from metadata store > ---

[jira] [Created] (SPARK-9759) Improve performance of Decimal.times() and casting from integral

Davies Liu created SPARK-9759: - Summary: Improve performance of Decimal.times() and casting from integral Key: SPARK-9759 URL: https://issues.apache.org/jira/browse/SPARK-9759 Project: Spark Iss

[jira] [Resolved] (SPARK-6902) Row() object can be mutated even though it should be immutable

[ https://issues.apache.org/jira/browse/SPARK-6902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-6902. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8009 [https://github.com/

[jira] [Updated] (SPARK-4561) PySparkSQL's Row.asDict() should convert nested rows to dictionaries

[ https://issues.apache.org/jira/browse/SPARK-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-4561: -- Target Version/s: 1.5.0 (was: 1.6.0) > PySparkSQL's Row.asDict() should convert nested rows to dictiona

[jira] [Resolved] (SPARK-4561) PySparkSQL's Row.asDict() should convert nested rows to dictionaries

[ https://issues.apache.org/jira/browse/SPARK-4561?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-4561. --- Resolution: Fixed Fix Version/s: 1.5.0 > PySparkSQL's Row.asDict() should convert nested rows t

[jira] [Resolved] (SPARK-9738) remove FromUnsafe and add its codegen version to GenerateSafe

[ https://issues.apache.org/jira/browse/SPARK-9738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9738. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8029 [https://github.com/

[jira] [Resolved] (SPARK-9614) InternalRow representation during executionPlan.toRdd.aggregete possibly problematic

[ https://issues.apache.org/jira/browse/SPARK-9614?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9614. --- Resolution: Fixed Assignee: Burak Yavuz Fix Version/s: 1.5.0 I'd like to mark this as

[jira] [Commented] (SPARK-9701) allow not automatically using HiveContext with spark-shell when hive support built in

[ https://issues.apache.org/jira/browse/SPARK-9701?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14662115#comment-14662115 ] Davies Liu commented on SPARK-9701: --- [~lian cheng] [~yhuai] Should we target this for 1.

[jira] [Resolved] (SPARK-9683) copy UTF8String when convert unsafe array/map to safe

[ https://issues.apache.org/jira/browse/SPARK-9683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9683. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7990 [https://github.com/

[jira] [Commented] (SPARK-9714) Cannot insert into a table using pySpark

[ https://issues.apache.org/jira/browse/SPARK-9714?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14661406#comment-14661406 ] Davies Liu commented on SPARK-9714: --- The failure seems not related to Python, should als

[jira] [Updated] (SPARK-9714) Cannot insert into a table using pySpark

[ https://issues.apache.org/jira/browse/SPARK-9714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9714: -- Assignee: Yin Huai > Cannot insert into a table using pySpark >

[jira] [Updated] (SPARK-9714) Cannot insert into a table using pySpark

[ https://issues.apache.org/jira/browse/SPARK-9714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9714: -- Component/s: SQL > Cannot insert into a table using pySpark > >

[jira] [Updated] (SPARK-9714) Cannot insert into a table using pySpark

[ https://issues.apache.org/jira/browse/SPARK-9714?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9714: -- Target Version/s: 1.5.0 Priority: Blocker (was: Critical) > Cannot insert into a table usin

[jira] [Commented] (SPARK-9734) java.lang.IllegalArgumentException: Don't know how to save StructField(sal,DecimalType(7,2),true) to JDBC

[ https://issues.apache.org/jira/browse/SPARK-9734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14661384#comment-14661384 ] Davies Liu commented on SPARK-9734: --- This is fixed in master and 1.5. Because there many

[jira] [Commented] (SPARK-9614) InternalRow representation during executionPlan.toRdd.aggregete possibly problematic

[ https://issues.apache.org/jira/browse/SPARK-9614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14661376#comment-14661376 ] Davies Liu commented on SPARK-9614: --- I had visited all the use of toRdd in our codebase,

[jira] [Commented] (SPARK-9725) spark sql query string field return empty/garbled string

[ https://issues.apache.org/jira/browse/SPARK-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14661333#comment-14661333 ] Davies Liu commented on SPARK-9725: --- I tried with latest master in standalone mode : {co

[jira] [Commented] (SPARK-9614) InternalRow representation during executionPlan.toRdd.aggregete possibly problematic

[ https://issues.apache.org/jira/browse/SPARK-9614?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14661268#comment-14661268 ] Davies Liu commented on SPARK-9614: --- How to reproduce this issue? > InternalRow represe

[jira] [Assigned] (SPARK-9725) spark sql query string field return empty/garbled string

[ https://issues.apache.org/jira/browse/SPARK-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-9725: - Assignee: Davies Liu > spark sql query string field return empty/garbled string > ---

[jira] [Updated] (SPARK-9725) spark sql query string field return empty/garbled string

[ https://issues.apache.org/jira/browse/SPARK-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9725: -- Labels: 1.5.0 (was: ) > spark sql query string field return empty/garbled string >

[jira] [Resolved] (SPARK-9228) Combine unsafe and codegen into a single option

[ https://issues.apache.org/jira/browse/SPARK-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9228. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7998 [https://github.com/

[jira] [Updated] (SPARK-9627) SQL job failed if the dataframe is cached

[ https://issues.apache.org/jira/browse/SPARK-9627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9627: -- Priority: Blocker (was: Critical) > SQL job failed if the dataframe is cached > ---

[jira] [Commented] (SPARK-9627) SQL job failed if the dataframe is cached

[ https://issues.apache.org/jira/browse/SPARK-9627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14661083#comment-14661083 ] Davies Liu commented on SPARK-9627: --- This is only failed when codegen is enabled, [~lian

[jira] [Updated] (SPARK-9627) SQL job failed if the dataframe is cached

[ https://issues.apache.org/jira/browse/SPARK-9627?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9627: -- Assignee: Cheng Lian > SQL job failed if the dataframe is cached > -

[jira] [Resolved] (SPARK-4302) Support fixed precision decimal type in JsonParser

[ https://issues.apache.org/jira/browse/SPARK-4302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-4302. --- Resolution: Fixed This is fixed in 1.5 > Support fixed precision decimal type in JsonParser > ---

[jira] [Updated] (SPARK-4302) Support fixed precision decimal type in JsonParser

[ https://issues.apache.org/jira/browse/SPARK-4302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-4302: -- Target Version/s: 1.5.0 (was: 1.6.0) Fix Version/s: 1.5.0 > Support fixed precision decimal type

[jira] [Commented] (SPARK-4302) Support fixed precision decimal type in JsonParser

[ https://issues.apache.org/jira/browse/SPARK-4302?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660984#comment-14660984 ] Davies Liu commented on SPARK-4302: --- Right now, we use fixed DecimalType for big integer

[jira] [Assigned] (SPARK-4302) Support fixed precision decimal type in JsonParser

[ https://issues.apache.org/jira/browse/SPARK-4302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-4302: - Assignee: Davies Liu > Support fixed precision decimal type in JsonParser > -

[jira] [Updated] (SPARK-7897) Column with an unsigned bigint should be treated as DecimalType in JDBCRDD

[ https://issues.apache.org/jira/browse/SPARK-7897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-7897: -- Fix Version/s: (was: 1.4.2) > Column with an unsigned bigint should be treated as DecimalType in JDB

[jira] [Resolved] (SPARK-7897) Column with an unsigned bigint should be treated as DecimalType in JDBCRDD

[ https://issues.apache.org/jira/browse/SPARK-7897?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-7897. --- Resolution: Fixed Fix Version/s: (was: 1.4.0) 1.5.0 1.

[jira] [Commented] (SPARK-3033) [Hive] java.math.BigDecimal cannot be cast to org.apache.hadoop.hive.common.type.HiveDecimal

[ https://issues.apache.org/jira/browse/SPARK-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660975#comment-14660975 ] Davies Liu commented on SPARK-3033: --- [~yhuai] Could you help to confirm this bug is stil

[jira] [Updated] (SPARK-3033) [Hive] java.math.BigDecimal cannot be cast to org.apache.hadoop.hive.common.type.HiveDecimal

[ https://issues.apache.org/jira/browse/SPARK-3033?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-3033: -- Assignee: Yin Huai > [Hive] java.math.BigDecimal cannot be cast to > org.apache.hadoop.hive.common.type

[jira] [Resolved] (SPARK-6360) For Spark 1.1 and 1.2, after any RDD transformations, calling saveAsParquetFile over a SchemaRDD with decimal or UDT column throws

[ https://issues.apache.org/jira/browse/SPARK-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-6360. --- Resolution: Fixed Fix Version/s: 1.5.0 This should be fixed in 1.5 (will not back porting to ot

[jira] [Updated] (SPARK-6360) For Spark 1.1 and 1.2, after any RDD transformations, calling saveAsParquetFile over a SchemaRDD with decimal or UDT column throws

[ https://issues.apache.org/jira/browse/SPARK-6360?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-6360: -- Target Version/s: (was: 1.2.3) > For Spark 1.1 and 1.2, after any RDD transformations, calling > save

[jira] [Resolved] (SPARK-4988) "Create table ..as select ..from..order by .. limit 10" report error when one col is a Decimal

[ https://issues.apache.org/jira/browse/SPARK-4988?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-4988. --- Resolution: Fixed Assignee: Davies Liu Fix Version/s: 1.5.0 This should be fixed after

[jira] [Commented] (SPARK-4549) Support BigInt -> Decimal in convertToCatalyst in SparkSQL

[ https://issues.apache.org/jira/browse/SPARK-4549?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660959#comment-14660959 ] Davies Liu commented on SPARK-4549: --- When convert a BigInt into Decimal, what's the scal

[jira] [Resolved] (SPARK-8272) BigDecimal in parquet not working

[ https://issues.apache.org/jira/browse/SPARK-8272?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8272. --- Resolution: Fixed Assignee: Davies Liu Fix Version/s: 1.5.0 This should be fixed 1.5 (

[jira] [Resolved] (SPARK-9362) Exception when using DataFrame groupby().sum on Decimal type in Python

[ https://issues.apache.org/jira/browse/SPARK-9362?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9362. --- Resolution: Fixed Assignee: Davies Liu Fix Version/s: 1.5.0 > Exception when using Dat

[jira] [Closed] (SPARK-8802) Decimal.apply(BigDecimal).toBigDecimal may throw NumberFormatException

[ https://issues.apache.org/jira/browse/SPARK-8802?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-8802. - Resolution: Won't Fix Target Version/s: (was: 1.6.0) > Decimal.apply(BigDecimal).toBigDecimal

[jira] [Commented] (SPARK-8802) Decimal.apply(BigDecimal).toBigDecimal may throw NumberFormatException

[ https://issues.apache.org/jira/browse/SPARK-8802?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14660951#comment-14660951 ] Davies Liu commented on SPARK-8802: --- In 1.5, we only support precision and scale that <

[jira] [Updated] (SPARK-9683) deep copy UTF8String when convert unsafe row to safe row

[ https://issues.apache.org/jira/browse/SPARK-9683?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9683: -- Assignee: Wenchen Fan > deep copy UTF8String when convert unsafe row to safe row > -

[jira] [Created] (SPARK-9700) Decrease the default page size

Davies Liu created SPARK-9700: - Summary: Decrease the default page size Key: SPARK-9700 URL: https://issues.apache.org/jira/browse/SPARK-9700 Project: Spark Issue Type: Improvement Comp

[jira] [Assigned] (SPARK-9453) Support records larger than default page size in UnsafeShuffleExternalSorter

[ https://issues.apache.org/jira/browse/SPARK-9453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-9453: - Assignee: Davies Liu (was: Josh Rosen) > Support records larger than default page size in Unsafe

[jira] [Updated] (SPARK-9453) Support records larger than default page size in UnsafeShuffleExternalSorter

[ https://issues.apache.org/jira/browse/SPARK-9453?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9453: -- Target Version/s: 1.5.0 (was: 1.6.0) > Support records larger than default page size in UnsafeShuffleEx

[jira] [Resolved] (SPARK-7083) Binary processing dimensional join

[ https://issues.apache.org/jira/browse/SPARK-7083?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-7083. --- Resolution: Fixed Assignee: Davies Liu Fix Version/s: 1.5.0 All hash join support Unsa

[jira] [Updated] (SPARK-8890) Reduce memory consumption for dynamic partition insert

[ https://issues.apache.org/jira/browse/SPARK-8890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-8890: -- Assignee: Michael Armbrust (was: Davies Liu) > Reduce memory consumption for dynamic partition insert >

[jira] [Assigned] (SPARK-8890) Reduce memory consumption for dynamic partition insert

[ https://issues.apache.org/jira/browse/SPARK-8890?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-8890: - Assignee: Davies Liu > Reduce memory consumption for dynamic partition insert > -

[jira] [Resolved] (SPARK-9482) Broadcast join projection is not thread safe

[ https://issues.apache.org/jira/browse/SPARK-9482?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9482. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7940 [https://github.com/

[jira] [Updated] (SPARK-8266) string function: translate

[ https://issues.apache.org/jira/browse/SPARK-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-8266: -- Assignee: zhichao-li (was: Cheng Hao) > string function: translate > -- > >

[jira] [Resolved] (SPARK-9644) Support update DecimalType with precision > 18 in UnsafeRow

[ https://issues.apache.org/jira/browse/SPARK-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9644. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7978 [https://github.com/

[jira] [Updated] (SPARK-8266) string function: translate

[ https://issues.apache.org/jira/browse/SPARK-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-8266: -- Target Version/s: 1.5.0 (was: 1.6.0) > string function: translate > -- > >

[jira] [Resolved] (SPARK-8266) string function: translate

[ https://issues.apache.org/jira/browse/SPARK-8266?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8266. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7709 [https://github.com/

[jira] [Resolved] (SPARK-6591) Python data source load options should auto convert common types into strings

[ https://issues.apache.org/jira/browse/SPARK-6591?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-6591. --- Resolution: Fixed Fix Version/s: 1.5.0 1.4.2 Issue resolved by pull request

[jira] [Updated] (SPARK-9644) Support update DecimalType with precision > 18 in UnsafeRow

[ https://issues.apache.org/jira/browse/SPARK-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9644: -- Issue Type: Improvement (was: Bug) > Support update DecimalType with precision > 18 in UnsafeRow >

[jira] [Resolved] (SPARK-9403) Implement code generation for In / InSet

[ https://issues.apache.org/jira/browse/SPARK-9403?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9403. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 7893 [https://github.com/

[jira] [Created] (SPARK-9644) Support update DecimalType with precision > 18 in UnsafeRow

Davies Liu created SPARK-9644: - Summary: Support update DecimalType with precision > 18 in UnsafeRow Key: SPARK-9644 URL: https://issues.apache.org/jira/browse/SPARK-9644 Project: Spark Issue Ty

<    10   11   12   13   14   15   16   17   18   19   >