[jira] [Created] (SPARK-14175) Simplify whole stage codegen interface

Davies Liu created SPARK-14175: -- Summary: Simplify whole stage codegen interface Key: SPARK-14175 URL: https://issues.apache.org/jira/browse/SPARK-14175 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-14137) Conflict between NullPropagation and InferFiltersFromConstraints

[ https://issues.apache.org/jira/browse/SPARK-14137?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14137. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11953

[jira] [Resolved] (SPARK-14150) Infer IsNotNull constraints from non-nullable attributes

[ https://issues.apache.org/jira/browse/SPARK-14150?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14150. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11953

[jira] [Resolved] (SPARK-13919) Resolving the Conflicts of ColumnPruning and PushPredicateThroughProject

[ https://issues.apache.org/jira/browse/SPARK-13919?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13919. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11828

[jira] [Created] (SPARK-14137) Conflict between NullPropagation and InferFiltersFromConstraints

Davies Liu created SPARK-14137: -- Summary: Conflict between NullPropagation and InferFiltersFromConstraints Key: SPARK-14137 URL: https://issues.apache.org/jira/browse/SPARK-14137 Project: Spark

[jira] [Resolved] (SPARK-14092) Performance regression when aggregation on parquet scan

[ https://issues.apache.org/jira/browse/SPARK-14092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14092. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11912

[jira] [Updated] (SPARK-13806) SQL round() produces incorrect results for negative values

[ https://issues.apache.org/jira/browse/SPARK-13806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13806: --- Fix Version/s: (was: 2.1.0) 2.0.0 > SQL round() produces incorrect results

[jira] [Assigned] (SPARK-14092) Performance regression when aggregation on parquet scan

[ https://issues.apache.org/jira/browse/SPARK-14092?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-14092: -- Assignee: Davies Liu > Performance regression when aggregation on parquet scan >

[jira] [Created] (SPARK-14092) Performance regression when aggregation on parquet scan

Davies Liu created SPARK-14092: -- Summary: Performance regression when aggregation on parquet scan Key: SPARK-14092 URL: https://issues.apache.org/jira/browse/SPARK-14092 Project: Spark Issue

[jira] [Resolved] (SPARK-13806) SQL round() produces incorrect results for negative values

[ https://issues.apache.org/jira/browse/SPARK-13806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13806. Resolution: Fixed Fix Version/s: 1.6.2 1.5.3 2.1.0

[jira] [Commented] (SPARK-14079) Limit the number of queries on SQL UI

[ https://issues.apache.org/jira/browse/SPARK-14079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207349#comment-15207349 ] Davies Liu commented on SPARK-14079: Yes, that's what I meant. > Limit the number of queries on SQL

[jira] [Updated] (SPARK-14079) Limit the number of queries on SQL UI

[ https://issues.apache.org/jira/browse/SPARK-14079?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-14079: --- Description: The SQL UI become very very slow if there are hundreds of SQL queries on it. > Limit

[jira] [Commented] (SPARK-14079) Limit the number of queries on SQL UI

[ https://issues.apache.org/jira/browse/SPARK-14079?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15207231#comment-15207231 ] Davies Liu commented on SPARK-14079: cc [~zsxwing] [~andrewor14] > Limit the number of queries on

[jira] [Created] (SPARK-14079) Limit the number of queries on SQL UI

Davies Liu created SPARK-14079: -- Summary: Limit the number of queries on SQL UI Key: SPARK-14079 URL: https://issues.apache.org/jira/browse/SPARK-14079 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-14052) Build BytesToBytesMap in HashedRelation

Davies Liu created SPARK-14052: -- Summary: Build BytesToBytesMap in HashedRelation Key: SPARK-14052 URL: https://issues.apache.org/jira/browse/SPARK-14052 Project: Spark Issue Type: Improvement

[jira] [Assigned] (SPARK-13806) SQL round() produces incorrect results for negative values

[ https://issues.apache.org/jira/browse/SPARK-13806?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-13806: -- Assignee: Davies Liu > SQL round() produces incorrect results for negative values >

[jira] [Resolved] (SPARK-13805) Direct consume ColumnVector in generated code when ColumnarBatch is used

[ https://issues.apache.org/jira/browse/SPARK-13805?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13805. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11636

[jira] [Commented] (SPARK-13806) SQL round() produces incorrect results for negative values

[ https://issues.apache.org/jira/browse/SPARK-13806?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15205103#comment-15205103 ] Davies Liu commented on SPARK-13806: This is because round() in Java/Scala have different sematics

[jira] [Resolved] (SPARK-14007) Manage the memory for hash map for shuffle hash join

[ https://issues.apache.org/jira/browse/SPARK-14007?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14007. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11826

[jira] [Commented] (SPARK-10309) Some tasks failed with Unable to acquire memory

[ https://issues.apache.org/jira/browse/SPARK-10309?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15203737#comment-15203737 ] Davies Liu commented on SPARK-10309: [~java8964] This patch is huge, also depends on other changes,

[jira] [Created] (SPARK-14007) Manage the memory for hash map for shuffle hash join

Davies Liu created SPARK-14007: -- Summary: Manage the memory for hash map for shuffle hash join Key: SPARK-14007 URL: https://issues.apache.org/jira/browse/SPARK-14007 Project: Spark Issue Type:

[jira] [Updated] (SPARK-14010) ColumnPruning is conflict with PushPredicateThroughProject

[ https://issues.apache.org/jira/browse/SPARK-14010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-14010: --- Description: ColumnPruning will insert a Project before Filter, but > ColumnPruning is conflict

[jira] [Updated] (SPARK-14010) ColumnPruning is conflict with PushPredicateThroughProject

[ https://issues.apache.org/jira/browse/SPARK-14010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-14010: --- Description: ColumnPruning will insert a Project before Filter, but PushPredicateThroughProject will

[jira] [Created] (SPARK-14009) Fail the tests if the any catalyst rule reach max number of iteration.

Davies Liu created SPARK-14009: -- Summary: Fail the tests if the any catalyst rule reach max number of iteration. Key: SPARK-14009 URL: https://issues.apache.org/jira/browse/SPARK-14009 Project: Spark

[jira] [Updated] (SPARK-13983) HiveThriftServer2 can not get "--hiveconf" or ''--hivevar" variables since 1.6 version (both multi-session and single session)

[ https://issues.apache.org/jira/browse/SPARK-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13983: --- Assignee: Cheng Lian > HiveThriftServer2 can not get "--hiveconf" or ''--hivevar" variables since >

[jira] [Resolved] (SPARK-13989) Remove non-vectorized/unsafe-row parquet record reader

[ https://issues.apache.org/jira/browse/SPARK-13989?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13989. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11799

[jira] [Resolved] (SPARK-13873) Avoid the copy in whole stage codegen when there is no joins

[ https://issues.apache.org/jira/browse/SPARK-13873?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13873. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11740

[jira] [Resolved] (SPARK-14010) ColumnPruning is conflict with PushPredicateThroughProject

[ https://issues.apache.org/jira/browse/SPARK-14010?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-14010. Resolution: Duplicate > ColumnPruning is conflict with PushPredicateThroughProject >

[jira] [Resolved] (SPARK-13930) Apply fast serialization on collect limit

[ https://issues.apache.org/jira/browse/SPARK-13930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13930. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11759

[jira] [Commented] (SPARK-13831) TPC-DS Query 35 fails with the following compile error

[ https://issues.apache.org/jira/browse/SPARK-13831?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15198827#comment-15198827 ] Davies Liu commented on SPARK-13831: Yes, Exists is not supported in Spark SQL yet. > TPC-DS Query

[jira] [Commented] (SPARK-13983) HiveThriftServer2 can not get "--hiveconf" or ''--hivevar" variables since 1.6 version (both multi-session and single session)

[ https://issues.apache.org/jira/browse/SPARK-13983?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15200328#comment-15200328 ] Davies Liu commented on SPARK-13983: [~lian cheng] Could you help to look at this one? >

[jira] [Created] (SPARK-13977) Bring back ShuffledHashJoin

Davies Liu created SPARK-13977: -- Summary: Bring back ShuffledHashJoin Key: SPARK-13977 URL: https://issues.apache.org/jira/browse/SPARK-13977 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-14010) ColumnPruning is conflict with PushPredicateThroughProject

Davies Liu created SPARK-14010: -- Summary: ColumnPruning is conflict with PushPredicateThroughProject Key: SPARK-14010 URL: https://issues.apache.org/jira/browse/SPARK-14010 Project: Spark Issue

[jira] [Resolved] (SPARK-13958) Executor OOM due to unbounded growth of pointer array in Sorter

[ https://issues.apache.org/jira/browse/SPARK-13958?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13958. Resolution: Fixed Fix Version/s: 1.6.2 2.0.0 Issue resolved by pull

[jira] [Created] (SPARK-13950) Generate code for sort merge left/right outer join

Davies Liu created SPARK-13950: -- Summary: Generate code for sort merge left/right outer join Key: SPARK-13950 URL: https://issues.apache.org/jira/browse/SPARK-13950 Project: Spark Issue Type:

[jira] [Resolved] (SPARK-13838) Clear variable code to prevent it to be re-evaluated in BoundAttribute

[ https://issues.apache.org/jira/browse/SPARK-13838?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13838. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11674

[jira] [Resolved] (SPARK-13977) Bring back ShuffledHashJoin

[ https://issues.apache.org/jira/browse/SPARK-13977?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13977. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11788

[jira] [Commented] (SPARK-13820) TPC-DS Query 10 fails to compile

[ https://issues.apache.org/jira/browse/SPARK-13820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15201896#comment-15201896 ] Davies Liu commented on SPARK-13820: May be 2.1? I had an prototype for that:

[jira] [Resolved] (SPARK-13917) Generate code for broadcast left semi join

[ https://issues.apache.org/jira/browse/SPARK-13917?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13917. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11742

[jira] [Commented] (SPARK-13820) TPC-DS Query 10 fails to compile

[ https://issues.apache.org/jira/browse/SPARK-13820?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15196430#comment-15196430 ] Davies Liu commented on SPARK-13820: Unfortunately, subquery is not target for 2.0. > TPC-DS Query

[jira] [Created] (SPARK-13918) merge SortMergeJoin and SortMergeOuterJoin

Davies Liu created SPARK-13918: -- Summary: merge SortMergeJoin and SortMergeOuterJoin Key: SPARK-13918 URL: https://issues.apache.org/jira/browse/SPARK-13918 Project: Spark Issue Type: Task

[jira] [Created] (SPARK-13917) Generate code for broadcast left semi join

Davies Liu created SPARK-13917: -- Summary: Generate code for broadcast left semi join Key: SPARK-13917 URL: https://issues.apache.org/jira/browse/SPARK-13917 Project: Spark Issue Type: New

[jira] [Reopened] (SPARK-13135) Don't print expressions recursively in generated code

[ https://issues.apache.org/jira/browse/SPARK-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reopened SPARK-13135: That PR is not merged. > Don't print expressions recursively in generated code >

[jira] [Resolved] (SPARK-13878) Window functions failed in cluster

[ https://issues.apache.org/jira/browse/SPARK-13878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13878. Resolution: Duplicate > Window functions failed in cluster > -- >

[jira] [Updated] (SPARK-13878) Window functions failed in cluster

[ https://issues.apache.org/jira/browse/SPARK-13878?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13878: --- Description: When cume_dist is used, we got the following error. {code}

[jira] [Created] (SPARK-13878) Window functions failed in cluster

Davies Liu created SPARK-13878: -- Summary: Window functions failed in cluster Key: SPARK-13878 URL: https://issues.apache.org/jira/browse/SPARK-13878 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-13135) Don't print expressions recursively in generated code

[ https://issues.apache.org/jira/browse/SPARK-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13135: --- Fix Version/s: (was: 2.0.0) > Don't print expressions recursively in generated code >

[jira] [Updated] (SPARK-13135) Don't print expressions recursively in generated code

[ https://issues.apache.org/jira/browse/SPARK-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13135: --- Assignee: (was: Reynold Xin) > Don't print expressions recursively in generated code >

[jira] [Resolved] (SPARK-13135) Don't print expressions recursively in generated code

[ https://issues.apache.org/jira/browse/SPARK-13135?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13135. Resolution: Fixed Fix Version/s: 2.0.0 > Don't print expressions recursively in generated

[jira] [Created] (SPARK-13873) Avoid the copy in whole stage codegen when there is no joins

Davies Liu created SPARK-13873: -- Summary: Avoid the copy in whole stage codegen when there is no joins Key: SPARK-13873 URL: https://issues.apache.org/jira/browse/SPARK-13873 Project: Spark

[jira] [Updated] (SPARK-13353) Fast serialization for collecting DataFrame

[ https://issues.apache.org/jira/browse/SPARK-13353?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13353: --- Summary: Fast serialization for collecting DataFrame (was: Use UnsafeRowSerializer to collect

[jira] [Resolved] (SPARK-13249) Filter null keys for inner join

[ https://issues.apache.org/jira/browse/SPARK-13249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13249. Resolution: Fixed Fix Version/s: 2.0.0 https://github.com/apache/spark/pull/11372 > Filter

[jira] [Updated] (SPARK-13249) Filter null keys for inner join

[ https://issues.apache.org/jira/browse/SPARK-13249?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13249: --- Assignee: Sameer Agarwal > Filter null keys for inner join > --- > >

[jira] [Resolved] (SPARK-13814) Delete unnecessary imports in python examples files

[ https://issues.apache.org/jira/browse/SPARK-13814?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13814. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11651

[jira] [Created] (SPARK-13830) Fetch large directly result from executor is very slow

Davies Liu created SPARK-13830: -- Summary: Fetch large directly result from executor is very slow Key: SPARK-13830 URL: https://issues.apache.org/jira/browse/SPARK-13830 Project: Spark Issue

[jira] [Resolved] (SPARK-13751) Generate better code for Filter

[ https://issues.apache.org/jira/browse/SPARK-13751?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13751. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11585

[jira] [Resolved] (SPARK-13790) Speed up ColumnVector's getDecimal

[ https://issues.apache.org/jira/browse/SPARK-13790?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13790. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11624

[jira] [Resolved] (SPARK-13636) Direct consume UnsafeRow in wholestage codegen plans

[ https://issues.apache.org/jira/browse/SPARK-13636?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13636. Resolution: Fixed Assignee: Liang-Chi Hsieh > Direct consume UnsafeRow in wholestage codegen

[jira] [Resolved] (SPARK-13242) Moderately complex `when` expression causes code generation failure

[ https://issues.apache.org/jira/browse/SPARK-13242?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13242. Resolution: Fixed Fix Version/s: 1.6.2 2.0.0 Issue resolved by pull

[jira] [Resolved] (SPARK-13523) Reuse the exchanges in a query

[ https://issues.apache.org/jira/browse/SPARK-13523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13523. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11403

[jira] [Resolved] (SPARK-13593) improve the `createDataFrame` method to accept data type string and verify the data

[ https://issues.apache.org/jira/browse/SPARK-13593?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13593. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11444

[jira] [Assigned] (SPARK-13750) Fix sizeInBytes for HadoopFSRelation

[ https://issues.apache.org/jira/browse/SPARK-13750?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-13750: -- Assignee: Davies Liu > Fix sizeInBytes for HadoopFSRelation >

[jira] [Created] (SPARK-13751) Generate better code for Filter

Davies Liu created SPARK-13751: -- Summary: Generate better code for Filter Key: SPARK-13751 URL: https://issues.apache.org/jira/browse/SPARK-13751 Project: Spark Issue Type: Improvement

[jira] [Resolved] (SPARK-13657) Support parsing very long AND/OR expression

[ https://issues.apache.org/jira/browse/SPARK-13657?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13657. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11501

[jira] [Resolved] (SPARK-13404) Create the variables for input when it's used

[ https://issues.apache.org/jira/browse/SPARK-13404?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13404. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11274

[jira] [Resolved] (SPARK-13651) Generator outputs are not resolved correctly resulting in runtime error

[ https://issues.apache.org/jira/browse/SPARK-13651?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13651. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11497

[jira] [Resolved] (SPARK-13697) TransformFunctionSerializer.loads doesn't restore the function's module name if it's '__main__'

[ https://issues.apache.org/jira/browse/SPARK-13697?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13697. Resolution: Fixed Fix Version/s: 1.4.2 1.6.1 1.5.3

[jira] [Reopened] (SPARK-13523) Reuse the exchanges in a query

[ https://issues.apache.org/jira/browse/SPARK-13523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reopened SPARK-13523: SPARK-11838 is different than this, that is about cache fragment across queries. > Reuse the

[jira] [Assigned] (SPARK-13523) Reuse the exchanges in a query

[ https://issues.apache.org/jira/browse/SPARK-13523?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu reassigned SPARK-13523: -- Assignee: Davies Liu > Reuse the exchanges in a query > -- > >

[jira] [Resolved] (SPARK-13255) Integrate vectorized parquet scan with whole stage codegen.

[ https://issues.apache.org/jira/browse/SPARK-13255?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13255. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11435

[jira] [Created] (SPARK-13671) Use different physical plan for existing RDD and data sources

Davies Liu created SPARK-13671: -- Summary: Use different physical plan for existing RDD and data sources Key: SPARK-13671 URL: https://issues.apache.org/jira/browse/SPARK-13671 Project: Spark

[jira] [Resolved] (SPARK-13647) also check if numeric value is within allowed range in _verify_type

[ https://issues.apache.org/jira/browse/SPARK-13647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13647. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11492

[jira] [Created] (SPARK-13661) Avoid the copy of UnsafeRow in HashedRelation

Davies Liu created SPARK-13661: -- Summary: Avoid the copy of UnsafeRow in HashedRelation Key: SPARK-13661 URL: https://issues.apache.org/jira/browse/SPARK-13661 Project: Spark Issue Type:

[jira] [Updated] (SPARK-13635) Enable LimitPushdown optimizer rule because we have whole-stage codegen for Limit

[ https://issues.apache.org/jira/browse/SPARK-13635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13635: --- Assignee: Liang-Chi Hsieh > Enable LimitPushdown optimizer rule because we have whole-stage codegen

[jira] [Created] (SPARK-13658) BooleanSimplification rule is slow with large boolean expressions

Davies Liu created SPARK-13658: -- Summary: BooleanSimplification rule is slow with large boolean expressions Key: SPARK-13658 URL: https://issues.apache.org/jira/browse/SPARK-13658 Project: Spark

[jira] [Created] (SPARK-13657) Can't parse very long AND/OR expression

Davies Liu created SPARK-13657: -- Summary: Can't parse very long AND/OR expression Key: SPARK-13657 URL: https://issues.apache.org/jira/browse/SPARK-13657 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-13466) Remove redundant project in colum pruning rule

[ https://issues.apache.org/jira/browse/SPARK-13466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13466: --- Issue Type: Improvement (was: Bug) Summary: Remove redundant project in colum pruning rule

[jira] [Updated] (SPARK-13466) Don't introduce redundant project with colum pruning rule

[ https://issues.apache.org/jira/browse/SPARK-13466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13466: --- Assignee: Liang-Chi Hsieh > Don't introduce redundant project with colum pruning rule >

[jira] [Resolved] (SPARK-13466) Don't introduce redundant project with colum pruning rule

[ https://issues.apache.org/jira/browse/SPARK-13466?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13466. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11341

[jira] [Resolved] (SPARK-13635) Enable LimitPushdown optimizer rule because we have whole-stage codegen for Limit

[ https://issues.apache.org/jira/browse/SPARK-13635?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13635. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11483

[jira] [Resolved] (SPARK-13574) Improve parquet dictionary decoding for strings

[ https://issues.apache.org/jira/browse/SPARK-13574?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13574. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11454

[jira] [Resolved] (SPARK-13601) Invoke task failure callbacks before calling outputstream.close()

[ https://issues.apache.org/jira/browse/SPARK-13601?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13601. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11450

[jira] [Commented] (SPARK-13353) Use UnsafeRowSerializer to collect DataFrame

[ https://issues.apache.org/jira/browse/SPARK-13353?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176598#comment-15176598 ] Davies Liu commented on SPARK-13353: [~rxin] Any idea on this one? One workaround could be packing

[jira] [Commented] (SPARK-13352) BlockFetch does not scale well on large block

[ https://issues.apache.org/jira/browse/SPARK-13352?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15176589#comment-15176589 ] Davies Liu commented on SPARK-13352: [~rxin] Can someone help to look into this one? This is one of

[jira] [Resolved] (SPARK-12738) GROUPING__ID is wrong

[ https://issues.apache.org/jira/browse/SPARK-12738?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-12738. Resolution: Fixed Assignee: Davies Liu Fix Version/s: 2.0.0

[jira] [Updated] (SPARK-13511) Add wholestage codegen for limit

[ https://issues.apache.org/jira/browse/SPARK-13511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13511: --- Assignee: Liang-Chi Hsieh > Add wholestage codegen for limit > > >

[jira] [Resolved] (SPARK-13609) Support Column Pruning for MapPartitions

[ https://issues.apache.org/jira/browse/SPARK-13609?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13609. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11460

[jira] [Resolved] (SPARK-13598) Remove LeftSemiJoinBNL

[ https://issues.apache.org/jira/browse/SPARK-13598?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13598. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11448

[jira] [Created] (SPARK-13603) SQL generation for subquery

Davies Liu created SPARK-13603: -- Summary: SQL generation for subquery Key: SPARK-13603 URL: https://issues.apache.org/jira/browse/SPARK-13603 Project: Spark Issue Type: Sub-task

[jira] [Commented] (SPARK-13307) TPCDS query 66 degraded by 30% in 1.6.0 compared to 1.4.1

[ https://issues.apache.org/jira/browse/SPARK-13307?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174473#comment-15174473 ] Davies Liu commented on SPARK-13307: In the plan, I saw that the column pruning does not work well,

[jira] [Commented] (SPARK-13174) Add API and options for csv data sources

[ https://issues.apache.org/jira/browse/SPARK-13174?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174451#comment-15174451 ] Davies Liu commented on SPARK-13174: We may still need Python and R API, also some convenient

[jira] [Closed] (SPARK-10712) JVM crashes with spark.sql.tungsten.enabled = true

[ https://issues.apache.org/jira/browse/SPARK-10712?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu closed SPARK-10712. -- Resolution: Cannot Reproduce > JVM crashes with spark.sql.tungsten.enabled = true >

[jira] [Updated] (SPARK-13499) Optimize vectorized parquet reader for dictionary encoded data and RLE decoding

[ https://issues.apache.org/jira/browse/SPARK-13499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13499: --- Assignee: Davies Liu (was: Nong Li) > Optimize vectorized parquet reader for dictionary encoded

[jira] [Updated] (SPARK-13499) Optimize vectorized parquet reader for dictionary encoded data and RLE decoding

[ https://issues.apache.org/jira/browse/SPARK-13499?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-13499: --- Assignee: Nong Li (was: Davies Liu) > Optimize vectorized parquet reader for dictionary encoded

[jira] [Commented] (SPARK-13463) Support Column pruning for Dataset logical plan

[ https://issues.apache.org/jira/browse/SPARK-13463?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15174445#comment-15174445 ] Davies Liu commented on SPARK-13463: [~smilegator] It could be true, then we just remove the TODO,

[jira] [Updated] (SPARK-12313) getPartitionsByFilter doesnt handle predicates on all / multiple Partition Columns

[ https://issues.apache.org/jira/browse/SPARK-12313?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu updated SPARK-12313: --- Assignee: Harsh Gupta > getPartitionsByFilter doesnt handle predicates on all / multiple Partition

[jira] [Created] (SPARK-13601) Invoke task failure callbacks before calling outputstream.close()

Davies Liu created SPARK-13601: -- Summary: Invoke task failure callbacks before calling outputstream.close() Key: SPARK-13601 URL: https://issues.apache.org/jira/browse/SPARK-13601 Project: Spark

[jira] [Resolved] (SPARK-13582) Improve performance of parquet reader with dictionary encoding

[ https://issues.apache.org/jira/browse/SPARK-13582?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13582. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11437

[jira] [Created] (SPARK-13598) Remove LeftSemiJoinBNL

Davies Liu created SPARK-13598: -- Summary: Remove LeftSemiJoinBNL Key: SPARK-13598 URL: https://issues.apache.org/jira/browse/SPARK-13598 Project: Spark Issue Type: Task Components:

[jira] [Resolved] (SPARK-13511) Add wholestage codegen for limit

[ https://issues.apache.org/jira/browse/SPARK-13511?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Davies Liu resolved SPARK-13511. Resolution: Fixed Fix Version/s: 2.0.0 Issue resolved by pull request 11391

<    2   3   4   5   6   7   8   9   10   11   >