[jira] [Updated] (SPARK-15441) dataset outer join seems to return incorrect result

2016-05-21 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15441: --- Assignee: Wenchen Fan > dataset outer join seems to return incorrect result > ---

[jira] [Commented] (SPARK-15441) dataset outer join seems to return incorrect result

2016-05-21 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15441?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294807#comment-15294807 ] Davies Liu commented on SPARK-15441: How to we represent a null in Dataset? If it's a

[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294785#comment-15294785 ] Davies Liu commented on SPARK-15285: [~kiszk] Go ahead, don't know why I can't assign

[jira] [Updated] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15285: --- Assignee: (was: Wenchen Fan) > Generated SpecificSafeProjection.apply method grows beyond 64 KB >

[jira] [Resolved] (SPARK-15078) Add all TPCDS 1.4 benchmark queries for SparkSQL

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15078?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15078. Resolution: Fixed Fix Version/s: 2.1.0 Issue resolved by pull request 13188 [https://github.

[jira] [Assigned] (SPARK-15327) Catalyst code generation fails with complex data structure

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15327?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15327: -- Assignee: Davies Liu > Catalyst code generation fails with complex data structure > --

[jira] [Commented] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294178#comment-15294178 ] Davies Liu commented on SPARK-15285: cc [~cloud_fan] > Generated SpecificSafeProject

[jira] [Updated] (SPARK-15285) Generated SpecificSafeProjection.apply method grows beyond 64 KB

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15285?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15285: --- Assignee: Wenchen Fan > Generated SpecificSafeProjection.apply method grows beyond 64 KB > --

[jira] [Commented] (SPARK-14331) Exceptions saving to parquetFile after join from dataframes in master

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14331?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15294168#comment-15294168 ] Davies Liu commented on SPARK-14331: Could you post the full stacktrace? This excepti

[jira] [Closed] (SPARK-15448) Flaky test:pyspark.ml.tests.DefaultValuesTests.test_java_params

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-15448. -- Resolution: Duplicate Fix Version/s: 2.0.0 > Flaky test:pyspark.ml.tests.DefaultValuesTests.test

[jira] [Assigned] (SPARK-14031) Dataframe to csv IO, system performance enters high CPU state and write operation takes 1 hour to complete

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14031?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-14031: -- Assignee: Davies Liu > Dataframe to csv IO, system performance enters high CPU state and write

[jira] [Updated] (SPARK-15448) Flaky test:pyspark.ml.tests.DefaultValuesTests.test_java_params

2016-05-20 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15448?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15448: --- Description: {code} == FAIL [1.2

[jira] [Created] (SPARK-15448) Flaky test:pyspark.ml.tests.DefaultValuesTests.test_java_params

2016-05-20 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15448: -- Summary: Flaky test:pyspark.ml.tests.DefaultValuesTests.test_java_params Key: SPARK-15448 URL: https://issues.apache.org/jira/browse/SPARK-15448 Project: Spark

[jira] [Created] (SPARK-15438) Improve the explain of whole-stage codegen

2016-05-19 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15438: -- Summary: Improve the explain of whole-stage codegen Key: SPARK-15438 URL: https://issues.apache.org/jira/browse/SPARK-15438 Project: Spark Issue Type: Improvemen

[jira] [Created] (SPARK-15432) Two executors with same id in Spark UI

2016-05-19 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15432: -- Summary: Two executors with same id in Spark UI Key: SPARK-15432 URL: https://issues.apache.org/jira/browse/SPARK-15432 Project: Spark Issue Type: Bug Affect

[jira] [Updated] (SPARK-14959) ​Problem Reading partitioned ORC or Parquet files

2016-05-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-14959: --- Priority: Blocker (was: Major) > ​Problem Reading partitioned ORC or Parquet files > ---

[jira] [Updated] (SPARK-14959) ​Problem Reading partitioned ORC or Parquet files

2016-05-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14959?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-14959: --- Target Version/s: 2.0.0 > ​Problem Reading partitioned ORC or Parquet files > ---

[jira] [Updated] (SPARK-15396) [Spark] [SQL] [DOC] It can't connect hive metastore database

2016-05-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15396?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15396: --- Target Version/s: 2.0.0 Issue Type: Documentation (was: Bug) Summary: [Spark]

[jira] [Updated] (SPARK-15393) Writing empty Dataframes doesn't save any _metadata files

2016-05-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15393?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15393: --- Priority: Critical (was: Major) > Writing empty Dataframes doesn't save any _metadata files > --

[jira] [Commented] (SPARK-15332) OutOfMemory in TimSort

2016-05-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15332?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15292143#comment-15292143 ] Davies Liu commented on SPARK-15332: It only happen in some corner cases, could not

[jira] [Updated] (SPARK-14343) Dataframe operations on a partitioned dataset (using partition discovery) return invalid results

2016-05-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-14343: --- Priority: Critical (was: Major) > Dataframe operations on a partitioned dataset (using partition dis

[jira] [Commented] (SPARK-14343) Dataframe operations on a partitioned dataset (using partition discovery) return invalid results

2016-05-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14343?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15292139#comment-15292139 ] Davies Liu commented on SPARK-14343: [~jurriaanpruis] Since you fixed the SPARK-14463

[jira] [Closed] (SPARK-15415) Marking partitions for broadcast broken

2016-05-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15415?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-15415. -- Resolution: Won't Fix Assignee: Davies Liu > Marking partitions for broadcast broken > --

[jira] [Commented] (SPARK-15415) Marking partitions for broadcast broken

2016-05-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15415?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15292134#comment-15292134 ] Davies Liu commented on SPARK-15415: [~jurriaanpruis] The implementation of broadcast

[jira] [Updated] (SPARK-13513) add some tests for leap year handling in catalyst

2016-05-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13513: --- Affects Version/s: (was: 2.0.0) > add some tests for leap year handling in catalyst > ---

[jira] [Updated] (SPARK-13513) add some tests for leap year handling in catalyst

2016-05-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13513?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13513: --- Priority: Minor (was: Major) > add some tests for leap year handling in catalyst > -

[jira] [Updated] (SPARK-15390) Memory management issue in complex DataFrame join and filter

2016-05-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15390: --- Assignee: Davies Liu > Memory management issue in complex DataFrame join and filter > ---

[jira] [Resolved] (SPARK-15390) Memory management issue in complex DataFrame join and filter

2016-05-19 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15390?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15390. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13182 [https://github.

[jira] [Resolved] (SPARK-15381) physical object operator should define `reference` correctly

2016-05-18 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15381?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15381. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13167 [https://github.

[jira] [Assigned] (SPARK-15392) The default value of size estimation is not good

2016-05-18 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15392?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15392: -- Assignee: Davies Liu > The default value of size estimation is not good >

[jira] [Created] (SPARK-15392) The default value of size estimation is not good

2016-05-18 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15392: -- Summary: The default value of size estimation is not good Key: SPARK-15392 URL: https://issues.apache.org/jira/browse/SPARK-15392 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-15342) PySpark test for non ascii column name does not actually test with unicode column name

2016-05-18 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15342. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13134 [https://github.

[jira] [Updated] (SPARK-15342) PySpark test for non ascii column name does not actually test with unicode column name

2016-05-18 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15342?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15342: --- Assignee: Liang-Chi Hsieh > PySpark test for non ascii column name does not actually test with unicod

[jira] [Resolved] (SPARK-15357) Cooperative spilling should check consumer memory mode

2016-05-18 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15357. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13151 [https://github.

[jira] [Resolved] (SPARK-15244) Type of column name created with sqlContext.createDataFrame() is not consistent.

2016-05-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15244. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13097 [https://github.

[jira] [Updated] (SPARK-15244) Type of column name created with sqlContext.createDataFrame() is not consistent.

2016-05-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15244?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15244: --- Assignee: Dongjoon Hyun > Type of column name created with sqlContext.createDataFrame() is not > con

[jira] [Assigned] (SPARK-15357) Cooperative spilling should check consumer memory mode

2016-05-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15357?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-15357: -- Assignee: Davies Liu > Cooperative spilling should check consumer memory mode > --

[jira] [Updated] (SPARK-15165) Codegen can break because toCommentSafeString is not actually safe

2016-05-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15165: --- Fix Version/s: 2.0.0 > Codegen can break because toCommentSafeString is not actually safe > -

[jira] [Resolved] (SPARK-15165) Codegen can break because toCommentSafeString is not actually safe

2016-05-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15165. Resolution: Fixed > Codegen can break because toCommentSafeString is not actually safe > --

[jira] [Updated] (SPARK-15165) Codegen can break because toCommentSafeString is not actually safe

2016-05-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15165: --- Assignee: Kousuke Saruta > Codegen can break because toCommentSafeString is not actually safe > -

[jira] [Updated] (SPARK-15165) Codegen can break because toCommentSafeString is not actually safe

2016-05-17 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15165?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15165: --- Affects Version/s: 1.5.2 1.6.1 > Codegen can break because toCommentSafeString

[jira] [Created] (SPARK-15332) OutOfMemory in TimSort

2016-05-15 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15332: -- Summary: OutOfMemory in TimSort Key: SPARK-15332 URL: https://issues.apache.org/jira/browse/SPARK-15332 Project: Spark Issue Type: Bug Components: Spa

[jira] [Updated] (SPARK-13866) Handle decimal type in CSV inference

2016-05-12 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13866: --- Assignee: Hyukjin Kwon > Handle decimal type in CSV inference >

[jira] [Resolved] (SPARK-13866) Handle decimal type in CSV inference

2016-05-12 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13866?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13866. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11724 [https://github.

[jira] [Created] (SPARK-15307) Super slow to load a partitioned table from local disks

2016-05-12 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15307: -- Summary: Super slow to load a partitioned table from local disks Key: SPARK-15307 URL: https://issues.apache.org/jira/browse/SPARK-15307 Project: Spark Issue Typ

[jira] [Commented] (SPARK-15307) Super slow to load a partitioned table from local disks

2016-05-12 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15282270#comment-15282270 ] Davies Liu commented on SPARK-15307: cc [~liancheng] > Super slow to load a partitio

[jira] [Closed] (SPARK-15287) Spark SQL partition filter clause with different literal type will scan all hive partitions

2016-05-12 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15287?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-15287. -- Resolution: Won't Fix > Spark SQL partition filter clause with different literal type will scan all >

[jira] [Commented] (SPARK-15287) Spark SQL partition filter clause with different literal type will scan all hive partitions

2016-05-12 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15287?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15282247#comment-15282247 ] Davies Liu commented on SPARK-15287: By design, we can't compare two expression with

[jira] [Updated] (SPARK-15300) Can't remove a block if it's under evicting

2016-05-12 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15300?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15300: --- Summary: Can't remove a block if it's under evicting (was: Can't remove a block if it's under evitin

[jira] [Created] (SPARK-15300) Can't remove a block if it's under eviting

2016-05-12 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15300: -- Summary: Can't remove a block if it's under eviting Key: SPARK-15300 URL: https://issues.apache.org/jira/browse/SPARK-15300 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-13522) Executor should kill itself when it's unable to heartbeat to the driver more than N times

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13522: --- Fix Version/s: (was: 1.6.2) > Executor should kill itself when it's unable to heartbeat to the dr

[jira] [Updated] (SPARK-15260) UnifiedMemoryManager could be in bad state if any exception happen while evicting blocks

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15260: --- Fix Version/s: 1.6.2 > UnifiedMemoryManager could be in bad state if any exception happen while > ev

[jira] [Updated] (SPARK-13522) Executor should kill itself when it's unable to heartbeat to the driver more than N times

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13522?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13522: --- Fix Version/s: 1.6.2 > Executor should kill itself when it's unable to heartbeat to the driver more

[jira] [Updated] (SPARK-15256) Clarify the docstring for DataFrameReader.jdbc()

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15256: --- Assignee: Nicholas Chammas > Clarify the docstring for DataFrameReader.jdbc() > -

[jira] [Resolved] (SPARK-15256) Clarify the docstring for DataFrameReader.jdbc()

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15256?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15256. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13034 [https://github.

[jira] [Resolved] (SPARK-15278) Remove experimental tag from Python DataFrame

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15278?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15278. Resolution: Fixed Issue resolved by pull request 13062 [https://github.com/apache/spark/pull/13062]

[jira] [Resolved] (SPARK-15270) Creating HiveContext does not work

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15270?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15270. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13056 [https://github.

[jira] [Resolved] (SPARK-15260) UnifiedMemoryManager could be in bad state if any exception happen while evicting blocks

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15260?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15260. Resolution: Fixed Fix Version/s: 2.0.0 > UnifiedMemoryManager could be in bad state if any e

[jira] [Resolved] (SPARK-15259) Sort time metric should not include spill and record insertion time

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15259. Resolution: Fixed Fix Version/s: 2.0.0 > Sort time metric should not include spill and recor

[jira] [Updated] (SPARK-15259) Sort time metric should not include spill and record insertion time

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15259?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15259: --- Assignee: Eric Liang > Sort time metric should not include spill and record insertion time >

[jira] [Resolved] (SPARK-15241) support scala decimal in external row

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15241?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15241. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13019 [https://github.

[jira] [Resolved] (SPARK-15242) keep decimal precision and scale when convert external decimal to catalyst decimal

2016-05-11 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15242. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13019 [https://github.

[jira] [Created] (SPARK-15260) UnifiedMemoryManager could be in bad state if any exception happen while evicting blocks

2016-05-10 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15260: -- Summary: UnifiedMemoryManager could be in bad state if any exception happen while evicting blocks Key: SPARK-15260 URL: https://issues.apache.org/jira/browse/SPARK-15260

[jira] [Updated] (SPARK-12661) Drop Python 2.6 support in PySpark

2016-05-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12661?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12661: --- Target Version/s: 2.1.0 (was: 2.0.0) > Drop Python 2.6 support in PySpark >

[jira] [Commented] (SPARK-12661) Drop Python 2.6 support in PySpark

2016-05-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12661?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15278630#comment-15278630 ] Davies Liu commented on SPARK-12661: I think the goal is clear we did not enough to d

[jira] [Resolved] (SPARK-14560) Cooperative Memory Management for Spillables

2016-05-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14560?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14560. Resolution: Fixed Assignee: Lianhui Wang (was: Imran Rashid) Fix Version/s: 2.0.0

[jira] [Updated] (SPARK-15179) Enable SQL generation for subqueries

2016-05-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15179: --- Assignee: Herman van Hovell > Enable SQL generation for subqueries >

[jira] [Resolved] (SPARK-14773) Enable the tests in HiveCompatibilitySuite for subquery

2016-05-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14773?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14773. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12988 [https://github.

[jira] [Resolved] (SPARK-15179) Enable SQL generation for subqueries

2016-05-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15179?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15179. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12988 [https://github.

[jira] [Updated] (SPARK-15154) LongHashedRelation test fails on Big Endian platform

2016-05-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15154: --- Assignee: Pete Robbins > LongHashedRelation test fails on Big Endian platform > -

[jira] [Resolved] (SPARK-15154) LongHashedRelation test fails on Big Endian platform

2016-05-10 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15154?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15154. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 13009 [https://github.

[jira] [Resolved] (SPARK-14972) Improve performance of JSON schema inference's inferField step

2016-05-09 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14972?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14972. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12750 [https://github.

[jira] [Commented] (SPARK-14946) Spark 2.0 vs 1.6.1 Query Time(out)

2016-05-09 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15276743#comment-15276743 ] Davies Liu commented on SPARK-14946: [~raymond.honderd...@sizmek.com] It seems that t

[jira] [Resolved] (SPARK-15122) TPC-DS Qury 41 fails with The correlated scalar subquery can only contain equality predicates

2016-05-06 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15122?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15122. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12954 [https://github.

[jira] [Resolved] (SPARK-1239) Improve fetching of map output statuses

2016-05-06 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-1239?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-1239. --- Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12113 [https://github.com

[jira] [Resolved] (SPARK-14512) Add python example for QuantileDiscretizer

2016-05-06 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14512?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14512. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12281 [https://github.

[jira] [Resolved] (SPARK-15110) SparkR - Implement repartitionByColumn on DataFrame

2016-05-05 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15110?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15110. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12887 [https://github.

[jira] [Commented] (SPARK-14946) Spark 2.0 vs 1.6.1 Query Time(out)

2016-05-05 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272656#comment-15272656 ] Davies Liu commented on SPARK-14946: The screen shot of 2.0 seemed that the second jo

[jira] [Commented] (SPARK-14946) Spark 2.0 vs 1.6.1 Query Time(out)

2016-05-05 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15272646#comment-15272646 ] Davies Liu commented on SPARK-14946: It will be great to narrow down to this issue, I

[jira] [Updated] (SPARK-15045) Remove dead code in TaskMemoryManager.cleanUpAllAllocatedMemory for pageTable

2016-05-04 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-15045: --- Assignee: Jacek Lewandowski > Remove dead code in TaskMemoryManager.cleanUpAllAllocatedMemory for pag

[jira] [Resolved] (SPARK-15045) Remove dead code in TaskMemoryManager.cleanUpAllAllocatedMemory for pageTable

2016-05-04 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15045?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15045. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12829 [https://github.

[jira] [Commented] (SPARK-14946) Spark 2.0 vs 1.6.1 Query Time(out)

2016-05-04 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14946?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15271332#comment-15271332 ] Davies Liu commented on SPARK-14946: [~raymond.honderd...@sizmek.com] Could you try t

[jira] [Updated] (SPARK-14951) Subexpression elimination in wholestage codegen version of TungstenAggregate

2016-05-04 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-14951: --- Assignee: Liang-Chi Hsieh > Subexpression elimination in wholestage codegen version of TungstenAggreg

[jira] [Resolved] (SPARK-14951) Subexpression elimination in wholestage codegen version of TungstenAggregate

2016-05-04 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14951?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14951. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12729 [https://github.

[jira] [Created] (SPARK-15105) Remove HiveSessionHook from ThriftServer

2016-05-03 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15105: -- Summary: Remove HiveSessionHook from ThriftServer Key: SPARK-15105 URL: https://issues.apache.org/jira/browse/SPARK-15105 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-15102) remove delegation token from ThriftServer

2016-05-03 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15102: -- Summary: remove delegation token from ThriftServer Key: SPARK-15102 URL: https://issues.apache.org/jira/browse/SPARK-15102 Project: Spark Issue Type: Bug

[jira] [Resolved] (SPARK-11316) coalesce doesn't handle UnionRDD with partial locality properly

2016-05-03 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11316?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-11316. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11327 [https://github.

[jira] [Resolved] (SPARK-14521) StackOverflowError in Kryo when executing TPC-DS

2016-05-03 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14521?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14521. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12598 [https://github.

[jira] [Created] (SPARK-15095) Drop binary mode in ThriftServer

2016-05-03 Thread Davies Liu (JIRA)
Davies Liu created SPARK-15095: -- Summary: Drop binary mode in ThriftServer Key: SPARK-15095 URL: https://issues.apache.org/jira/browse/SPARK-15095 Project: Spark Issue Type: Bug Compon

[jira] [Closed] (SPARK-14226) Caching a table with 1,100 columns and a few million rows fails

2016-05-03 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14226?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-14226. -- Resolution: Duplicate > Caching a table with 1,100 columns and a few million rows fails > -

[jira] [Updated] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

2016-05-03 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12837: --- Target Version/s: 2.0.0 Priority: Critical (was: Major) > Spark driver requires large me

[jira] [Updated] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

2016-05-03 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12837: --- Assignee: Wenchen Fan > Spark driver requires large memory space for serialized results even there >

[jira] [Commented] (SPARK-12837) Spark driver requires large memory space for serialized results even there are no data collected to the driver

2016-05-03 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12837?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15269132#comment-15269132 ] Davies Liu commented on SPARK-12837: With spark.driver.maxResultSize=1m, the simply j

[jira] [Resolved] (SPARK-14992) Flaky test: BucketedReadSuite.only shuffle one side when join bucketed table and non-bucketed table

2016-05-03 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14992?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14992. Resolution: Fixed Fix Version/s: 2.0.0 Fixed by https://github.com/apache/spark/pull/12773

[jira] [Resolved] (SPARK-15088) Remove SparkSqlSerializer

2016-05-03 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15088?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-15088. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12864 [https://github.

[jira] [Resolved] (SPARK-12540) Support all TPCDS queries

2016-05-02 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12540?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12540. Resolution: Fixed Fix Version/s: 2.0.0 > Support all TPCDS queries > ---

[jira] [Commented] (SPARK-12540) Support all TPCDS queries

2016-05-02 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12540?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15267739#comment-15267739 ] Davies Liu commented on SPARK-12540: We made it into Spark 2.0 finally, bingo! > Sup

[jira] [Updated] (SPARK-14785) Support correlated scalar subquery

2016-05-02 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-14785: --- Assignee: Herman van Hovell > Support correlated scalar subquery > --

[jira] [Resolved] (SPARK-14785) Support correlated scalar subquery

2016-05-02 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14785?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14785. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 12822 [https://github.

[jira] [Commented] (SPARK-13753) Column nullable is derived incorrectly

2016-05-02 Thread Davies Liu (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15267689#comment-15267689 ] Davies Liu commented on SPARK-13753: After looking at the query, the bug is caused by

<    6   7   8   9   10   11   12   13   14   15   >