[jira] [Commented] (SPARK-8632) Poor Python UDF performance because of RDD caching

[ https://issues.apache.org/jira/browse/SPARK-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14735644#comment-14735644 ] Davies Liu commented on SPARK-8632: --- The upstream means child of current SparkPlan, could have other

[jira] [Created] (SPARK-10494) Multiple Python UDFs together with aggregation or sort merge join may cause OOM (failed to acquire memory)

Davies Liu created SPARK-10494: -- Summary: Multiple Python UDFs together with aggregation or sort merge join may cause OOM (failed to acquire memory) Key: SPARK-10494 URL:

[jira] [Commented] (SPARK-10466) UnsafeRow exception in Sort-Based Shuffle with data spill

[ https://issues.apache.org/jira/browse/SPARK-10466?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14735776#comment-14735776 ] Davies Liu commented on SPARK-10466: [~chenghao] I tried your test case, it passed in master. Is

[jira] [Commented] (SPARK-10309) Some tasks failed with Unable to acquire memory

[ https://issues.apache.org/jira/browse/SPARK-10309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14735903#comment-14735903 ] Davies Liu commented on SPARK-10309: [~nadenf] Could you post the physical plan here? That could help

[jira] [Commented] (SPARK-10309) Some tasks failed with Unable to acquire memory

[ https://issues.apache.org/jira/browse/SPARK-10309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14735908#comment-14735908 ] Davies Liu commented on SPARK-10309: This also could be related to

[jira] [Created] (SPARK-10459) PythonUDF could process UnsafeRow

Davies Liu created SPARK-10459: -- Summary: PythonUDF could process UnsafeRow Key: SPARK-10459 URL: https://issues.apache.org/jira/browse/SPARK-10459 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-8632) Poor Python UDF performance because of RDD caching

[ https://issues.apache.org/jira/browse/SPARK-8632?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-8632: - Assignee: Davies Liu > Poor Python UDF performance because of RDD caching >

[jira] [Commented] (SPARK-10434) Parquet compatibility with 1.4 is broken when writing arrays that may contain nulls

[ https://issues.apache.org/jira/browse/SPARK-10434?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14729445#comment-14729445 ] Davies Liu commented on SPARK-10434: [~lian cheng] I think it's hard to guarantee forward

[jira] [Updated] (SPARK-10434) Parquet compatibility with 1.4 is broken when writing arrays that may contain nulls

[ https://issues.apache.org/jira/browse/SPARK-10434?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-10434: --- Priority: Minor (was: Critical) > Parquet compatibility with 1.4 is broken when writing arrays that

[jira] [Commented] (SPARK-10425) Add a regression test for SPARK-10379

[ https://issues.apache.org/jira/browse/SPARK-10425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14729421#comment-14729421 ] Davies Liu commented on SPARK-10425: [~sowen] Thanks for your comment, The reason that PR didn't have

[jira] [Updated] (SPARK-10436) spark-submit overwrites spark.files defaults with the job script filename

[ https://issues.apache.org/jira/browse/SPARK-10436?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-10436: --- Target Version/s: 1.6.0 > spark-submit overwrites spark.files defaults with the job script filename

[jira] [Created] (SPARK-10424) ShuffleHashOuterJoin should consider condition

Davies Liu created SPARK-10424: -- Summary: ShuffleHashOuterJoin should consider condition Key: SPARK-10424 URL: https://issues.apache.org/jira/browse/SPARK-10424 Project: Spark Issue Type: New

[jira] [Created] (SPARK-10425) Add a regression test for SPARK-10379

Davies Liu created SPARK-10425: -- Summary: Add a regression test for SPARK-10379 Key: SPARK-10425 URL: https://issues.apache.org/jira/browse/SPARK-10425 Project: Spark Issue Type: New Feature

[jira] [Updated] (SPARK-10424) ShuffleHashOuterJoin should consider condition

[ https://issues.apache.org/jira/browse/SPARK-10424?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-10424: --- Priority: Blocker (was: Major) > ShuffleHashOuterJoin should consider condition >

[jira] [Resolved] (SPARK-10422) String column in InMemoryColumnarCache needs to override clone method

[ https://issues.apache.org/jira/browse/SPARK-10422?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10422. Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8578

[jira] [Resolved] (SPARK-10417) Iterating through Column results in infinite loop

[ https://issues.apache.org/jira/browse/SPARK-10417?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10417. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8574

[jira] [Created] (SPARK-10403) UnsafeRowSerializer can't work with UnsafeShuffleManager (tungsten-sort)

Davies Liu created SPARK-10403: -- Summary: UnsafeRowSerializer can't work with UnsafeShuffleManager (tungsten-sort) Key: SPARK-10403 URL: https://issues.apache.org/jira/browse/SPARK-10403 Project: Spark

[jira] [Updated] (SPARK-10379) UnsafeShuffleExternalSorter should preserve first page

[ https://issues.apache.org/jira/browse/SPARK-10379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-10379: --- Target Version/s: 1.6.0, 1.5.1 (was: 1.5.0) > UnsafeShuffleExternalSorter should preserve first

[jira] [Resolved] (SPARK-10392) Pyspark - Wrong DateType support on JDBC connection

[ https://issues.apache.org/jira/browse/SPARK-10392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10392. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8556

[jira] [Resolved] (SPARK-10162) PySpark filters with datetimes mess up when datetimes have timezones.

[ https://issues.apache.org/jira/browse/SPARK-10162?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10162. Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8555

[jira] [Created] (SPARK-10404) Worker should terminate previous executor before launch new one

Davies Liu created SPARK-10404: -- Summary: Worker should terminate previous executor before launch new one Key: SPARK-10404 URL: https://issues.apache.org/jira/browse/SPARK-10404 Project: Spark

[jira] [Updated] (SPARK-10392) Pyspark - Wrong DateType support on JDBC connection

[ https://issues.apache.org/jira/browse/SPARK-10392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-10392: --- Fix Version/s: 1.5.1 > Pyspark - Wrong DateType support on JDBC connection >

[jira] [Commented] (SPARK-10373) Move @since annotator to pyspark to be shared by all components

[ https://issues.apache.org/jira/browse/SPARK-10373?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=14724009#comment-14724009 ] Davies Liu commented on SPARK-10373: [~mengxr] Do we want to add @since for the MLLib APIs in 1.5

[jira] [Created] (SPARK-10379) UnsafeShuffleExternalSorter should preserve first page

Davies Liu created SPARK-10379: -- Summary: UnsafeShuffleExternalSorter should preserve first page Key: SPARK-10379 URL: https://issues.apache.org/jira/browse/SPARK-10379 Project: Spark Issue

[jira] [Updated] (SPARK-10341) SMJ fail with unable to acquire memory

[ https://issues.apache.org/jira/browse/SPARK-10341?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-10341: --- Target Version/s: 1.5.0 (was: 1.5.1) SMJ fail with unable to acquire memory

[jira] [Resolved] (SPARK-10323) NPE in code-gened In expression

[ https://issues.apache.org/jira/browse/SPARK-10323?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10323. Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8492

[jira] [Created] (SPARK-10341) SMJ fail with unable to acquire memory

Davies Liu created SPARK-10341: -- Summary: SMJ fail with unable to acquire memory Key: SPARK-10341 URL: https://issues.apache.org/jira/browse/SPARK-10341 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-10342) Cooperative memory management

Davies Liu created SPARK-10342: -- Summary: Cooperative memory management Key: SPARK-10342 URL: https://issues.apache.org/jira/browse/SPARK-10342 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-10342) Cooperative memory management

[ https://issues.apache.org/jira/browse/SPARK-10342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-10342: --- Issue Type: Improvement (was: Story) Cooperative memory management -

[jira] [Created] (SPARK-10343) Consider nullability of expression in codegen

Davies Liu created SPARK-10343: -- Summary: Consider nullability of expression in codegen Key: SPARK-10343 URL: https://issues.apache.org/jira/browse/SPARK-10343 Project: Spark Issue Type:

[jira] [Created] (SPARK-10345) Flaky test: HiveCompatibilitySuite.nonblock_op_deduplicate

Davies Liu created SPARK-10345: -- Summary: Flaky test: HiveCompatibilitySuite.nonblock_op_deduplicate Key: SPARK-10345 URL: https://issues.apache.org/jira/browse/SPARK-10345 Project: Spark Issue

[jira] [Assigned] (SPARK-10321) OrcRelation doesn't override sizeInBytes

[ https://issues.apache.org/jira/browse/SPARK-10321?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-10321: -- Assignee: Davies Liu OrcRelation doesn't override sizeInBytes

[jira] [Commented] (SPARK-9228) Combine unsafe and codegen into a single option

[ https://issues.apache.org/jira/browse/SPARK-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14715512#comment-14715512 ] Davies Liu commented on SPARK-9228: --- [~jameszhouyi] unsafe.offHeap is another option

[jira] [Closed] (SPARK-10302) NPE while save a DataFrame as ORC

[ https://issues.apache.org/jira/browse/SPARK-10302?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-10302. -- Resolution: Duplicate Fix Version/s: 1.5.0 NPE while save a DataFrame as ORC

[jira] [Created] (SPARK-10302) NPE while save a DataFrame as ORC

Davies Liu created SPARK-10302: -- Summary: NPE while save a DataFrame as ORC Key: SPARK-10302 URL: https://issues.apache.org/jira/browse/SPARK-10302 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-10305) PySpark createDataFrame on list of LabeledPoints fails (regression)

[ https://issues.apache.org/jira/browse/SPARK-10305?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10305. Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8470

[jira] [Created] (SPARK-10309) Some tasks failed with Unable to acquire memory

Davies Liu created SPARK-10309: -- Summary: Some tasks failed with Unable to acquire memory Key: SPARK-10309 URL: https://issues.apache.org/jira/browse/SPARK-10309 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-10196) Failed to save json data with a decimal type in the schema

[ https://issues.apache.org/jira/browse/SPARK-10196?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10196. Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8408

[jira] [Commented] (SPARK-10215) Div of Decimal returns null

[ https://issues.apache.org/jira/browse/SPARK-10215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14710696#comment-14710696 ] Davies Liu commented on SPARK-10215: I think we have not enough time to figure out

[jira] [Created] (SPARK-10245) SQLContext can't parse literal less than 0.1 ( 0.01)

Davies Liu created SPARK-10245: -- Summary: SQLContext can't parse literal less than 0.1 ( 0.01) Key: SPARK-10245 URL: https://issues.apache.org/jira/browse/SPARK-10245 Project: Spark Issue Type:

[jira] [Commented] (SPARK-10177) Parquet support interprets timestamp values differently from Hive 0.14.0+

[ https://issues.apache.org/jira/browse/SPARK-10177?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14709878#comment-14709878 ] Davies Liu commented on SPARK-10177: [~lian cheng] After some investigation, just

[jira] [Closed] (SPARK-7506) pyspark.sql.types.StructType.fromJson() is incorrectly named

[ https://issues.apache.org/jira/browse/SPARK-7506?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-7506. - Resolution: Won't Fix Assignee: (was: Davies Liu) These functions could be used by users (even

[jira] [Resolved] (SPARK-8580) Test Parquet interoperability and compatibility with other libraries/systems

[ https://issues.apache.org/jira/browse/SPARK-8580?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8580. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8392

[jira] [Resolved] (SPARK-9401) Fully implement code generation for ConcatWs

[ https://issues.apache.org/jira/browse/SPARK-9401?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9401. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8353

[jira] [Commented] (SPARK-9228) Combine unsafe and codegen into a single option

[ https://issues.apache.org/jira/browse/SPARK-9228?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14706289#comment-14706289 ] Davies Liu commented on SPARK-9228: --- Right now, it's an internal configuration (could be

[jira] [Reopened] (SPARK-3533) Add saveAsTextFileByKey() method to RDDs

[ https://issues.apache.org/jira/browse/SPARK-3533?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reopened SPARK-3533: --- Re-open this for discussion. Add saveAsTextFileByKey() method to RDDs

[jira] [Resolved] (SPARK-9400) Implement code generation for StringLocate

[ https://issues.apache.org/jira/browse/SPARK-9400?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9400. --- Resolution: Fixed Fix Version/s: 1.6.0 Issue resolved by pull request 8330

[jira] [Commented] (SPARK-9644) Support update DecimalType with precision 18 in UnsafeRow

[ https://issues.apache.org/jira/browse/SPARK-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703474#comment-14703474 ] Davies Liu commented on SPARK-9644: --- [~robbinspg] It's fixed by

[jira] [Commented] (SPARK-7379) pickle.loads expects a string instead of bytes in Python 3.

[ https://issues.apache.org/jira/browse/SPARK-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14703583#comment-14703583 ] Davies Liu commented on SPARK-7379: --- [~mengxr] This is a known issue that how to

[jira] [Created] (SPARK-10129) math function: stddev_samp

Davies Liu created SPARK-10129: -- Summary: math function: stddev_samp Key: SPARK-10129 URL: https://issues.apache.org/jira/browse/SPARK-10129 Project: Spark Issue Type: New Feature

[jira] [Commented] (SPARK-6798) Fix Date serialization in SparkR

[ https://issues.apache.org/jira/browse/SPARK-6798?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702625#comment-14702625 ] Davies Liu commented on SPARK-6798: --- Not a bug, it's just not efficient. Fix Date

[jira] [Updated] (SPARK-6798) Fix Date serialization in SparkR

[ https://issues.apache.org/jira/browse/SPARK-6798?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-6798: -- Issue Type: Improvement (was: Bug) Fix Date serialization in SparkR

[jira] [Created] (SPARK-10107) NPE in format_number

Davies Liu created SPARK-10107: -- Summary: NPE in format_number Key: SPARK-10107 URL: https://issues.apache.org/jira/browse/SPARK-10107 Project: Spark Issue Type: Bug Components: SQL

[jira] [Created] (SPARK-10090) After division, Decimal may have longer precision than expected

Davies Liu created SPARK-10090: -- Summary: After division, Decimal may have longer precision than expected Key: SPARK-10090 URL: https://issues.apache.org/jira/browse/SPARK-10090 Project: Spark

[jira] [Created] (SPARK-10095) Should not use the private field of BigInteger

Davies Liu created SPARK-10095: -- Summary: Should not use the private field of BigInteger Key: SPARK-10095 URL: https://issues.apache.org/jira/browse/SPARK-10095 Project: Spark Issue Type:

[jira] [Commented] (SPARK-10056) PySpark Row - Support for row[columnName] syntax

[ https://issues.apache.org/jira/browse/SPARK-10056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14701883#comment-14701883 ] Davies Liu commented on SPARK-10056: [~maver1ck], Yes, changes in Row class and unit

[jira] [Commented] (SPARK-9644) Support update DecimalType with precision 18 in UnsafeRow

[ https://issues.apache.org/jira/browse/SPARK-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14701891#comment-14701891 ] Davies Liu commented on SPARK-9644: --- [~robbinspg] Thanks for point this out, would is be

[jira] [Commented] (SPARK-9644) Support update DecimalType with precision 18 in UnsafeRow

[ https://issues.apache.org/jira/browse/SPARK-9644?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14701974#comment-14701974 ] Davies Liu commented on SPARK-9644: --- In a benchmark of doing aggregation on a decimal

[jira] [Commented] (SPARK-9627) SQL job failed if the dataframe with string columns is cached

[ https://issues.apache.org/jira/browse/SPARK-9627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702390#comment-14702390 ] Davies Liu commented on SPARK-9627: --- I can reproduce it with latest master. SQL job

[jira] [Commented] (SPARK-9627) SQL job failed if the dataframe with string columns is cached

[ https://issues.apache.org/jira/browse/SPARK-9627?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14702387#comment-14702387 ] Davies Liu commented on SPARK-9627: --- The `df.show()` will succeed, but

[jira] [Resolved] (SPARK-10095) Should not use the private field of BigInteger

[ https://issues.apache.org/jira/browse/SPARK-10095?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-10095. Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8286

[jira] [Commented] (SPARK-10079) Make `column` and `col` functions be S4 functions

[ https://issues.apache.org/jira/browse/SPARK-10079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14701877#comment-14701877 ] Davies Liu commented on SPARK-10079: There are two public col/column function in

[jira] [Closed] (SPARK-5901) [PySpark] pickle classes in main module

[ https://issues.apache.org/jira/browse/SPARK-5901?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-5901. - Resolution: Invalid Target Version/s: (was: 1.5.0) couldpickle does support to serialize

[jira] [Comment Edited] (SPARK-9982) SparkR DataFrame fail to return data of Decimal type

[ https://issues.apache.org/jira/browse/SPARK-9982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14699861#comment-14699861 ] Davies Liu edited comment on SPARK-9982 at 8/17/15 5:21 PM: R

[jira] [Commented] (SPARK-9982) SparkR DataFrame fail to return data of Decimal type

[ https://issues.apache.org/jira/browse/SPARK-9982?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14699861#comment-14699861 ] Davies Liu commented on SPARK-9982: --- R has numeric (similar to double), no decimal, is

[jira] [Created] (SPARK-10059) Broken test: YarnClusterSuite

Davies Liu created SPARK-10059: -- Summary: Broken test: YarnClusterSuite Key: SPARK-10059 URL: https://issues.apache.org/jira/browse/SPARK-10059 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-10058) Flaky test: HeartbeatReceiverSuite: normal heartbeat

Davies Liu created SPARK-10058: -- Summary: Flaky test: HeartbeatReceiverSuite: normal heartbeat Key: SPARK-10058 URL: https://issues.apache.org/jira/browse/SPARK-10058 Project: Spark Issue Type:

[jira] [Created] (SPARK-10057) Faill to load class org.slf4j.impl.StaticLoggerBinder

Davies Liu created SPARK-10057: -- Summary: Faill to load class org.slf4j.impl.StaticLoggerBinder Key: SPARK-10057 URL: https://issues.apache.org/jira/browse/SPARK-10057 Project: Spark Issue

[jira] [Commented] (SPARK-9427) Add expression functions in SparkR

[ https://issues.apache.org/jira/browse/SPARK-9427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14699843#comment-14699843 ] Davies Liu commented on SPARK-9427: --- [~yu_ishikawa] `rand` does work in PySpark (Python

[jira] [Closed] (SPARK-9822) Update doc about supported Python versions

[ https://issues.apache.org/jira/browse/SPARK-9822?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-9822. - Resolution: Duplicate Update doc about supported Python versions

[jira] [Commented] (SPARK-9705) outdated Python 3 and IPython information

[ https://issues.apache.org/jira/browse/SPARK-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14699983#comment-14699983 ] Davies Liu commented on SPARK-9705: --- The PyLab thing is already fixed by

[jira] [Assigned] (SPARK-10038) TungstenProject code generation fails when applied to arraybinary

[ https://issues.apache.org/jira/browse/SPARK-10038?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-10038: -- Assignee: Davies Liu TungstenProject code generation fails when applied to arraybinary

[jira] [Commented] (SPARK-9705) outdated Python 3 and IPython information

[ https://issues.apache.org/jira/browse/SPARK-9705?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14699973#comment-14699973 ] Davies Liu commented on SPARK-9705: --- As the exception said, PySpark cannot run with

[jira] [Commented] (SPARK-10056) PySpark Row - Support for row[columnName] syntax

[ https://issues.apache.org/jira/browse/SPARK-10056?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14700028#comment-14700028 ] Davies Liu commented on SPARK-10056: We support accessing nested column by

[jira] [Commented] (SPARK-9427) Add expression functions in SparkR

[ https://issues.apache.org/jira/browse/SPARK-9427?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14700044#comment-14700044 ] Davies Liu commented on SPARK-9427: --- Should we target this for 1.6? Add expression

[jira] [Created] (SPARK-10065) Avoid triple copy of var-length objects in Array in tungsten projection

Davies Liu created SPARK-10065: -- Summary: Avoid triple copy of var-length objects in Array in tungsten projection Key: SPARK-10065 URL: https://issues.apache.org/jira/browse/SPARK-10065 Project: Spark

[jira] [Resolved] (SPARK-9946) NPE in TaskMemoryManager

[ https://issues.apache.org/jira/browse/SPARK-9946?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9946. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8177

[jira] [Assigned] (SPARK-9978) Window functions require partitionBy to work as expected

[ https://issues.apache.org/jira/browse/SPARK-9978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-9978: - Assignee: Davies Liu Window functions require partitionBy to work as expected

[jira] [Resolved] (SPARK-9589) Flaky test: HiveCompatibilitySuite.groupby8

[ https://issues.apache.org/jira/browse/SPARK-9589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9589. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8177

[jira] [Comment Edited] (SPARK-9725) spark sql query string field return empty/garbled string

[ https://issues.apache.org/jira/browse/SPARK-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697426#comment-14697426 ] Davies Liu edited comment on SPARK-9725 at 8/14/15 5:36 PM: I

[jira] [Assigned] (SPARK-9589) Flaky test: HiveCompatibilitySuite.groupby8

[ https://issues.apache.org/jira/browse/SPARK-9589?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-9589: - Assignee: Davies Liu (was: Josh Rosen) Flaky test: HiveCompatibilitySuite.groupby8

[jira] [Commented] (SPARK-9725) spark sql query string field return empty/garbled string

[ https://issues.apache.org/jira/browse/SPARK-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697378#comment-14697378 ] Davies Liu commented on SPARK-9725: --- So the problem came from serialization of

[jira] [Commented] (SPARK-9725) spark sql query string field return empty/garbled string

[ https://issues.apache.org/jira/browse/SPARK-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697426#comment-14697426 ] Davies Liu commented on SPARK-9725: --- I can reproduce this issue now (32G for executor,

[jira] [Commented] (SPARK-9971) MaxFunction not working correctly with columns containing Double.NaN

[ https://issues.apache.org/jira/browse/SPARK-9971?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697868#comment-14697868 ] Davies Liu commented on SPARK-9971: --- We had a long discussion about how to support NaN,

[jira] [Resolved] (SPARK-9725) spark sql query string field return empty/garbled string

[ https://issues.apache.org/jira/browse/SPARK-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9725. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8210

[jira] [Commented] (SPARK-9734) java.lang.IllegalArgumentException: Don't know how to save StructField(sal,DecimalType(7,2),true) to JDBC

[ https://issues.apache.org/jira/browse/SPARK-9734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14698121#comment-14698121 ] Davies Liu commented on SPARK-9734: --- [~rmullapudi] It's fixed by

[jira] [Commented] (SPARK-9734) java.lang.IllegalArgumentException: Don't know how to save StructField(sal,DecimalType(7,2),true) to JDBC

[ https://issues.apache.org/jira/browse/SPARK-9734?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14697998#comment-14697998 ] Davies Liu commented on SPARK-9734: --- [~rmullapudi] Good catch! I will sending out a

[jira] [Created] (SPARK-9931) Flaky test: mllib/tests.py StreamingLogisticRegressionWithSGDTests. test_training_and_prediction

Davies Liu created SPARK-9931: - Summary: Flaky test: mllib/tests.py StreamingLogisticRegressionWithSGDTests. test_training_and_prediction Key: SPARK-9931 URL: https://issues.apache.org/jira/browse/SPARK-9931

[jira] [Updated] (SPARK-9931) Flaky test: mllib/tests.py StreamingLogisticRegressionWithSGDTests. test_training_and_prediction

[ https://issues.apache.org/jira/browse/SPARK-9931?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-9931: -- Priority: Critical (was: Major) Flaky test: mllib/tests.py StreamingLogisticRegressionWithSGDTests.

[jira] [Created] (SPARK-9942) Broken pandas could crash PySpark SQL

Davies Liu created SPARK-9942: - Summary: Broken pandas could crash PySpark SQL Key: SPARK-9942 URL: https://issues.apache.org/jira/browse/SPARK-9942 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-9943) Failed to serialize a deserialized UnsafeHashedRelation

Davies Liu created SPARK-9943: - Summary: Failed to serialize a deserialized UnsafeHashedRelation Key: SPARK-9943 URL: https://issues.apache.org/jira/browse/SPARK-9943 Project: Spark Issue Type:

[jira] [Commented] (SPARK-9604) Unsafe ArrayData and MapData is very very slow

[ https://issues.apache.org/jira/browse/SPARK-9604?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14695617#comment-14695617 ] Davies Liu commented on SPARK-9604: --- [~cloud_fan] Yeah, The test looks much better now.

[jira] [Assigned] (SPARK-7379) pickle.loads expects a string instead of bytes in Python 3.

[ https://issues.apache.org/jira/browse/SPARK-7379?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-7379: - Assignee: Davies Liu pickle.loads expects a string instead of bytes in Python 3.

[jira] [Resolved] (SPARK-9499) Possible file handle leak in spilling/sort code

[ https://issues.apache.org/jira/browse/SPARK-9499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9499. --- Resolution: Duplicate Assignee: Davies Liu (was: Josh Rosen) Fix Version/s: 1.5.0

[jira] [Commented] (SPARK-9725) spark sql query string field return empty/garbled string

[ https://issues.apache.org/jira/browse/SPARK-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14695652#comment-14695652 ] Davies Liu commented on SPARK-9725: --- [~lian cheng] [~yhuai] Can you reproduce this

[jira] [Commented] (SPARK-9725) spark sql query string field return empty/garbled string

[ https://issues.apache.org/jira/browse/SPARK-9725?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14696043#comment-14696043 ] Davies Liu commented on SPARK-9725: --- This PR is not for this JIRA, was linked by

[jira] [Created] (SPARK-9955) TPCDS Q8 failed in 1.5

Davies Liu created SPARK-9955: - Summary: TPCDS Q8 failed in 1.5 Key: SPARK-9955 URL: https://issues.apache.org/jira/browse/SPARK-9955 Project: Spark Issue Type: Bug Components: SQL

[jira] [Resolved] (SPARK-9943) Failed to serialize a deserialized UnsafeHashedRelation

[ https://issues.apache.org/jira/browse/SPARK-9943?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-9943. --- Resolution: Fixed Fix Version/s: 1.5.0 Issue resolved by pull request 8174

[jira] [Resolved] (SPARK-8976) Python 3 crash: ValueError: invalid mode 'a+' (only r, w, b allowed)

[ https://issues.apache.org/jira/browse/SPARK-8976?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-8976. --- Resolution: Fixed Fix Version/s: 1.5.0 1.4.2 Issue resolved by pull request

[jira] [Created] (SPARK-9946) TaskMemoryManager is not thread safe

Davies Liu created SPARK-9946: - Summary: TaskMemoryManager is not thread safe Key: SPARK-9946 URL: https://issues.apache.org/jira/browse/SPARK-9946 Project: Spark Issue Type: Bug

<    9   10   11   12   13   14   15   16   17   18   >