[jira] [Commented] (SPARK-15467) Getting stack overflow when attempting to query a wide Dataset (>200 fields)

2018-01-20 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15467?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1694#comment-1694 ] Takeshi Yamamuro commented on SPARK-15467: -- I think this should be fixed in the

[jira] [Commented] (SPARK-19217) Offer easy cast from vector to array

2018-01-20 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1698#comment-1698 ] Takeshi Yamamuro commented on SPARK-19217: -- If it still makes some sense and nob

[jira] [Comment Edited] (SPARK-19217) Offer easy cast from vector to array

2018-01-20 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1698#comment-1698 ] Takeshi Yamamuro edited comment on SPARK-19217 at 1/21/18 5:32 AM:

[jira] [Created] (SPARK-23168) Hints for fact tables and unique columns

2018-01-20 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-23168: Summary: Hints for fact tables and unique columns Key: SPARK-23168 URL: https://issues.apache.org/jira/browse/SPARK-23168 Project: Spark Issue Type:

[jira] [Updated] (SPARK-23168) Hints for fact tables and unique columns

2018-01-20 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-23168: - Environment: (was: We already have fact table and unique column inferences in StarSch

[jira] [Commented] (SPARK-23168) Hints for fact tables and unique columns

2018-01-20 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333402#comment-16333402 ] Takeshi Yamamuro commented on SPARK-23168: -- An example code is like: [https://g

[jira] [Updated] (SPARK-23168) Hints for fact tables and unique columns

2018-01-20 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-23168: - Description: We already have fact table and unique column inferences in StarSchemaDetecti

[jira] [Commented] (SPARK-23167) Update TPCDS queries from v1.4 to v2.7 (latest)

2018-01-21 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333536#comment-16333536 ] Takeshi Yamamuro commented on SPARK-23167: -- ok, will do. > Update TPCDS queries

[jira] [Commented] (SPARK-23171) Reduce the time costs of the rule runs that do not change the plans

2018-01-21 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23171?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333551#comment-16333551 ] Takeshi Yamamuro commented on SPARK-23171: -- ok, I'll check code based on these m

[jira] [Commented] (SPARK-23168) Hints for fact tables and unique columns

2018-01-21 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23168?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333572#comment-16333572 ] Takeshi Yamamuro commented on SPARK-23168: -- ok > Hints for fact tables and uniq

[jira] [Updated] (SPARK-23168) Hints for fact tables and unique columns

2018-01-21 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23168?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-23168: - Issue Type: Sub-task (was: New Feature) Parent: SPARK-19842 > Hints for fact tab

[jira] [Commented] (SPARK-19842) Informational Referential Integrity Constraints Support in Spark

2018-01-21 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333576#comment-16333576 ] Takeshi Yamamuro commented on SPARK-19842: -- What's the status of this tickets no

[jira] [Created] (SPARK-23172) Respect Project nodes in ReorderJoin

2018-01-21 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-23172: Summary: Respect Project nodes in ReorderJoin Key: SPARK-23172 URL: https://issues.apache.org/jira/browse/SPARK-23172 Project: Spark Issue Type: Impr

[jira] [Commented] (SPARK-19217) Offer easy cast from vector to array

2018-01-21 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333959#comment-16333959 ] Takeshi Yamamuro commented on SPARK-19217: -- If we can, I think it's the best to

[jira] [Commented] (SPARK-19217) Offer easy cast from vector to array

2018-01-22 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19217?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16333987#comment-16333987 ] Takeshi Yamamuro commented on SPARK-19217: -- ok, I'll reconsider this. > Offer e

[jira] [Updated] (SPARK-23172) Expand the ReorderJoin rule to handle Project nodes

2018-01-22 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23172?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-23172: - Summary: Expand the ReorderJoin rule to handle Project nodes (was: Respect Project nodes

[jira] [Created] (SPARK-23264) Support interval values without INTERVAL clauses

2018-01-29 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-23264: Summary: Support interval values without INTERVAL clauses Key: SPARK-23264 URL: https://issues.apache.org/jira/browse/SPARK-23264 Project: Spark Issu

[jira] [Commented] (SPARK-20425) Support an extended display mode to print a column data per line

2018-02-05 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20425?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16352271#comment-16352271 ] Takeshi Yamamuro commented on SPARK-20425: -- yea, ok. If nobody takes on this, I'

[jira] [Commented] (SPARK-26077) Reserved SQL words are not escaped by JDBC writer for table name

2018-11-19 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26077?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16692546#comment-16692546 ] Takeshi Yamamuro commented on SPARK-26077: -- Thanks for your reporting! It seems

[jira] [Updated] (SPARK-23264) Support interval values without INTERVAL clauses

2018-12-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-23264: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-26217 > Support interval

[jira] [Reopened] (SPARK-23264) Support interval values without INTERVAL clauses

2018-12-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23264?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro reopened SPARK-23264: -- > Support interval values without INTERVAL clauses > -

[jira] [Commented] (SPARK-23264) Support interval values without INTERVAL clauses

2018-12-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23264?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16707006#comment-16707006 ] Takeshi Yamamuro commented on SPARK-23264: -- Since the next release version is v

[jira] [Commented] (SPARK-26215) define reserved keywords after SQL standard

2018-12-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16707095#comment-16707095 ] Takeshi Yamamuro commented on SPARK-26215: -- These reserved words should be hand

[jira] [Commented] (SPARK-26215) define reserved keywords after SQL standard

2018-12-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26215?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16707099#comment-16707099 ] Takeshi Yamamuro commented on SPARK-26215: -- I found some useful documents about

[jira] [Created] (SPARK-26262) Run SQLQueryTestSuite with WHOLESTAGE_CODEGEN_ENABLED=false

2018-12-03 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-26262: Summary: Run SQLQueryTestSuite with WHOLESTAGE_CODEGEN_ENABLED=false Key: SPARK-26262 URL: https://issues.apache.org/jira/browse/SPARK-26262 Project: Spark

[jira] [Updated] (SPARK-26262) Runs SQLQueryTestSuite on mixed config sets: WHOLESTAGE_CODEGEN_ENABLED and CODEGEN_FACTORY_MODE

2018-12-04 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-26262: - Description: For better test coverage, we need to run `SQLQueryTestSuite` on 4 mixed con

[jira] [Updated] (SPARK-26262) Runs SQLQueryTestSuite on mixed config sets: WHOLESTAGE_CODEGEN_ENABLED and CODEGEN_FACTORY_MODE

2018-12-04 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-26262: - Summary: Runs SQLQueryTestSuite on mixed config sets: WHOLESTAGE_CODEGEN_ENABLED and COD

[jira] [Updated] (SPARK-26262) Runs SQLQueryTestSuite on mixed config sets: WHOLESTAGE_CODEGEN_ENABLED and CODEGEN_FACTORY_MODE

2018-12-04 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26262?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-26262: - Description: For better test coverage, we need to run `SQLQueryTestSuite` on 4 mixed con

[jira] [Updated] (SPARK-26182) Cost increases when optimizing scalaUDF

2018-12-05 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26182?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-26182: - Issue Type: Improvement (was: Bug) > Cost increases when optimizing scalaUDF >

[jira] [Commented] (SPARK-26182) Cost increases when optimizing scalaUDF

2018-12-05 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26182?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16711014#comment-16711014 ] Takeshi Yamamuro commented on SPARK-26182: -- This is an expected behaviour and

[jira] [Updated] (SPARK-26224) Results in stackOverFlowError when trying to add 3000 new columns using withColumn function of dataframe.

2018-12-07 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-26224?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-26224: - Component/s: (was: Spark Core) SQL > Results in stackOverFlowError

[jira] [Commented] (SPARK-13337) DataFrame join-on-columns function should support null-safe equal

2016-02-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15152225#comment-15152225 ] Takeshi Yamamuro commented on SPARK-13337: -- Is it not enough to use `df1.join(df

[jira] [Commented] (SPARK-8000) SQLContext.read.load() should be able to auto-detect input data

2016-02-19 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15154044#comment-15154044 ] Takeshi Yamamuro commented on SPARK-8000: - Oh, it is a simple and good approach to

[jira] [Commented] (SPARK-13484) Filter outer joined result using a non-nullable column from the right table

2016-02-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15167344#comment-15167344 ] Takeshi Yamamuro commented on SPARK-13484: -- Im working on this, and almost finis

[jira] [Commented] (SPARK-13484) Filter outer joined result using a non-nullable column from the right table

2016-02-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15168246#comment-15168246 ] Takeshi Yamamuro commented on SPARK-13484: -- Thanks! > Filter outer joined resul

[jira] [Commented] (SPARK-11691) Allow to specify compression codec in HadoopFsRelation when saving

2016-02-26 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15170333#comment-15170333 ] Takeshi Yamamuro commented on SPARK-11691: -- I think it's okay to close this tick

[jira] [Comment Edited] (SPARK-11691) Allow to specify compression codec in HadoopFsRelation when saving

2016-02-26 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11691?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15170333#comment-15170333 ] Takeshi Yamamuro edited comment on SPARK-11691 at 2/27/16 3:38 AM:

[jira] [Created] (SPARK-13528) Make the short names of compression codecs consistent in spark

2016-02-26 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-13528: Summary: Make the short names of compression codecs consistent in spark Key: SPARK-13528 URL: https://issues.apache.org/jira/browse/SPARK-13528 Project: Spark

[jira] [Created] (SPARK-13607) Improves compression performance for integer-typed values on cache to reduce GC pressure

2016-03-01 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-13607: Summary: Improves compression performance for integer-typed values on cache to reduce GC pressure Key: SPARK-13607 URL: https://issues.apache.org/jira/browse/SPARK-13607

[jira] [Commented] (SPARK-13337) DataFrame join-on-columns function should support null-safe equal

2016-03-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15175548#comment-15175548 ] Takeshi Yamamuro commented on SPARK-13337: -- ISTM an interface to get TableC dire

[jira] [Commented] (SPARK-13656) Delte spark.sql.parquet.cacheMetadata

2016-03-07 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15182872#comment-15182872 ] Takeshi Yamamuro commented on SPARK-13656: -- Is it okay to simply remove the rela

[jira] [Commented] (SPARK-13656) Delte spark.sql.parquet.cacheMetadata

2016-03-07 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15182949#comment-15182949 ] Takeshi Yamamuro commented on SPARK-13656: -- I looked over the related codes; doe

[jira] [Commented] (SPARK-13337) DataFrame join-on-columns function should support null-safe equal

2016-03-07 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13337?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15182962#comment-15182962 ] Takeshi Yamamuro commented on SPARK-13337: -- Oh, I got your point ;) However, it

[jira] [Commented] (SPARK-13656) Delete spark.sql.parquet.cacheMetadata

2016-03-07 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15184110#comment-15184110 ] Takeshi Yamamuro commented on SPARK-13656: -- Okay, I'll make a pr in a day. > De

[jira] [Commented] (SPARK-13656) Delete spark.sql.parquet.cacheMetadata

2016-03-22 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207941#comment-15207941 ] Takeshi Yamamuro commented on SPARK-13656: -- [~yhuai] I looked over the codes, an

[jira] [Comment Edited] (SPARK-13656) Delete spark.sql.parquet.cacheMetadata

2016-03-22 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15207941#comment-15207941 ] Takeshi Yamamuro edited comment on SPARK-13656 at 3/23/16 6:11 AM:

[jira] [Updated] (SPARK-6747) Support List<> as a return type in Hive UDF

2015-06-24 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-6747: Affects Version/s: 1.4.0 > Support List<> as a return type in Hive UDF > ---

[jira] [Updated] (SPARK-6747) Support List<> as a return type in Hive UDF

2015-06-24 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-6747: Labels: 1.5.0 (was: ) > Support List<> as a return type in Hive UDF > -

[jira] [Updated] (SPARK-6747) Support List<> as a return type in Hive UDF

2015-06-24 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-6747: Labels: (was: 1.5.0) > Support List<> as a return type in Hive UDF > -

[jira] [Updated] (SPARK-6912) Support Map as a return type in Hive UDF

2015-06-24 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-6912: Affects Version/s: 1.4.0 > Support Map as a return type in Hive UDF > --

[jira] [Updated] (SPARK-6747) Throw an AnalysisException when unsupported Java list types used in Hive UDF

2015-07-06 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-6747: Summary: Throw an AnalysisException when unsupported Java list types used in Hive UDF (was:

[jira] [Updated] (SPARK-6747) Throw an AnalysisException when unsupported Java list types used in Hive UDF

2015-07-06 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6747?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-6747: Description: The current implementation can't handle List<> as a return type in Hive UDF and

[jira] [Updated] (SPARK-6912) Throw an AnalysisException when unsupported Java Map types used in Hive UDF

2015-07-07 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6912?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-6912: Summary: Throw an AnalysisException when unsupported Java Map types used in Hive UDF (was:

[jira] [Created] (SPARK-8930) Support a star '*' in the generator function argument

2015-07-08 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-8930: --- Summary: Support a star '*' in the generator function argument Key: SPARK-8930 URL: https://issues.apache.org/jira/browse/SPARK-8930 Project: Spark Iss

[jira] [Updated] (SPARK-8930) Support a star '*' in generator function arguments

2015-07-08 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-8930: Summary: Support a star '*' in generator function arguments (was: Support a star '*' in the

[jira] [Created] (SPARK-8955) Replace a duplicated initialize() in HiveGenericUDTF with new one

2015-07-09 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-8955: --- Summary: Replace a duplicated initialize() in HiveGenericUDTF with new one Key: SPARK-8955 URL: https://issues.apache.org/jira/browse/SPARK-8955 Project: Spark

[jira] [Updated] (SPARK-8930) Support a star '*' in generator function arguments

2015-07-09 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-8930: Description: The current implementation throws an exception if generators contain a star '*'

[jira] [Updated] (SPARK-8930) Support a star '*' in generator function arguments

2015-07-09 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-8930: Description: The current implementation throws an exception if generators contain a star '*'

[jira] [Updated] (SPARK-8930) Support a star '*' in generator function arguments

2015-07-09 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8930?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-8930: Description: The current implementation throws an exception if generators contain a star '*'

[jira] [Created] (SPARK-9034) Reflect field names defined in GenericUDTF

2015-07-14 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-9034: --- Summary: Reflect field names defined in GenericUDTF Key: SPARK-9034 URL: https://issues.apache.org/jira/browse/SPARK-9034 Project: Spark Issue Type: Bu

[jira] [Updated] (SPARK-9034) Reflect field names defined in GenericUDTF

2015-07-14 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-9034: Description: Hive GenericUDTF#initialize() defines field names in a returned schema though,

[jira] [Updated] (SPARK-9034) Reflect field names defined in GenericUDTF

2015-07-14 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-9034: Description: GenericUDTF#initialize() in Hive defines field names in a returned schema thou

[jira] [Commented] (SPARK-9034) Reflect field names defined in GenericUDTF

2015-07-14 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-9034?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=14626279#comment-14626279 ] Takeshi Yamamuro commented on SPARK-9034: - I'll make a PR for this after SPARK-895

[jira] [Commented] (SPARK-11704) Optimize the Cartesian Join

2015-11-13 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005111#comment-15005111 ] Takeshi Yamamuro commented on SPARK-11704: -- ISTM that some earlier stages in rdd

[jira] [Commented] (SPARK-11704) Optimize the Cartesian Join

2015-11-13 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-11704?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15005217#comment-15005217 ] Takeshi Yamamuro commented on SPARK-11704: -- You're right; they're not automatica

[jira] [Commented] (SPARK-6521) Bypass network shuffle read if both endpoints are local

2015-11-30 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15033016#comment-15033016 ] Takeshi Yamamuro commented on SPARK-6521: - Performance of the current Spark heavil

[jira] [Comment Edited] (SPARK-6521) Bypass network shuffle read if both endpoints are local

2015-12-15 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-6521?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15033016#comment-15033016 ] Takeshi Yamamuro edited comment on SPARK-6521 at 12/15/15 9:23 AM: -

[jira] [Created] (SPARK-12392) Optimize a location order of broadcast blocks by considering preferred local hosts

2015-12-16 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-12392: Summary: Optimize a location order of broadcast blocks by considering preferred local hosts Key: SPARK-12392 URL: https://issues.apache.org/jira/browse/SPARK-12392

[jira] [Created] (SPARK-12446) Add unit tests for JDBCRDD internal functions

2015-12-20 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-12446: Summary: Add unit tests for JDBCRDD internal functions Key: SPARK-12446 URL: https://issues.apache.org/jira/browse/SPARK-12446 Project: Spark Issue T

[jira] [Created] (SPARK-12476) Implement JdbcRelation#unhandledFilters for removing unnecessary Spark Fileter

2015-12-21 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-12476: Summary: Implement JdbcRelation#unhandledFilters for removing unnecessary Spark Fileter Key: SPARK-12476 URL: https://issues.apache.org/jira/browse/SPARK-12476

[jira] [Updated] (SPARK-12476) Implement JdbcRelation#unhandledFilters for removing unnecessary Spark Fileter

2015-12-21 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-12476: - Description: Input: SELECT * FROM jdbcTable WHERE col0 = 'xxx' {code} Current plan: == O

[jira] [Updated] (SPARK-12476) Implement JdbcRelation#unhandledFilters for removing unnecessary Spark Filter

2015-12-21 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-12476: - Description: Input: SELECT * FROM jdbcTable WHERE col0 = 'xxx' Current plan: {code} == O

[jira] [Updated] (SPARK-12476) Implement JdbcRelation#unhandledFilters for removing unnecessary Spark Filter

2015-12-21 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-12476: - Summary: Implement JdbcRelation#unhandledFilters for removing unnecessary Spark Filter (

[jira] [Updated] (SPARK-12476) Implement JdbcRelation#unhandledFilters for removing unnecessary Spark Fileter

2015-12-21 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-12476: - Description: Input: SELECT * FROM jdbcTable WHERE col0 = 'xxx' {code} Current plan: == O

[jira] [Updated] (SPARK-12476) Implement JdbcRelation#unhandledFilters for removing unnecessary Spark Filter

2015-12-21 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12476?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-12476: - Description: Input: SELECT * FROM jdbcTable WHERE col0 = 'xxx' Current plan: {code} == O

[jira] [Commented] (SPARK-5753) add basic support to JDBCRDD for postgresql types: uuid, hstore, and array

2015-12-24 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5753?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15070822#comment-15070822 ] Takeshi Yamamuro commented on SPARK-5753: - [~rxin] This ticket should be closed be

[jira] [Created] (SPARK-12581) Support case-sensitive table names in postgresql

2015-12-30 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-12581: Summary: Support case-sensitive table names in postgresql Key: SPARK-12581 URL: https://issues.apache.org/jira/browse/SPARK-12581 Project: Spark Issu

[jira] [Created] (SPARK-12686) Support group-by push down into data sources

2016-01-06 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-12686: Summary: Support group-by push down into data sources Key: SPARK-12686 URL: https://issues.apache.org/jira/browse/SPARK-12686 Project: Spark Issue Ty

[jira] [Created] (SPARK-12978) Skip unnecessary final group-by when input data already clustered with group-by keys

2016-01-25 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-12978: Summary: Skip unnecessary final group-by when input data already clustered with group-by keys Key: SPARK-12978 URL: https://issues.apache.org/jira/browse/SPARK-12978

[jira] [Updated] (SPARK-12978) Skip unnecessary final group-by when input data already clustered with group-by keys

2016-01-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-12978: - Description: This pr enables the optimization below; Without opt.: ``` == Physical Plan

[jira] [Updated] (SPARK-12978) Skip unnecessary final group-by when input data already clustered with group-by keys

2016-01-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-12978: - Description: This pr enables the optimization to skip an unnecessary group-by operation

[jira] [Updated] (SPARK-12978) Skip unnecessary final group-by when input data already clustered with group-by keys

2016-01-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-12978: - Description: This pr enables the optimization below; Without opt.: {code} == Physical Pl

[jira] [Updated] (SPARK-12978) Skip unnecessary final group-by when input data already clustered with group-by keys

2016-01-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-12978: - Description: This ticket targets the optimization to skip an unnecessary group-by operati

[jira] [Commented] (SPARK-12890) Spark SQL query related to only partition fields should not scan the whole data.

2016-01-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115076#comment-15115076 ] Takeshi Yamamuro commented on SPARK-12890: -- I looked over the related codes; IST

[jira] [Comment Edited] (SPARK-12890) Spark SQL query related to only partition fields should not scan the whole data.

2016-01-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115076#comment-15115076 ] Takeshi Yamamuro edited comment on SPARK-12890 at 1/25/16 11:39 AM: ---

[jira] [Comment Edited] (SPARK-12890) Spark SQL query related to only partition fields should not scan the whole data.

2016-01-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115076#comment-15115076 ] Takeshi Yamamuro edited comment on SPARK-12890 at 1/25/16 1:32 PM:

[jira] [Commented] (SPARK-12890) Spark SQL query related to only partition fields should not scan the whole data.

2016-01-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12890?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15115257#comment-15115257 ] Takeshi Yamamuro commented on SPARK-12890: -- Ah, I see. > Spark SQL query relate

[jira] [Created] (SPARK-12995) Remove deprecate APIs from Pregel

2016-01-25 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-12995: Summary: Remove deprecate APIs from Pregel Key: SPARK-12995 URL: https://issues.apache.org/jira/browse/SPARK-12995 Project: Spark Issue Type: Improve

[jira] [Updated] (SPARK-12995) Remove deprecate APIs from Pregel

2016-01-25 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12995?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-12995: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-11806 > Remove deprecate A

[jira] [Created] (SPARK-13057) Add benchmark codes and the performance results for implemented compression schemes for InMemoryRelation

2016-01-27 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-13057: Summary: Add benchmark codes and the performance results for implemented compression schemes for InMemoryRelation Key: SPARK-13057 URL: https://issues.apache.org/jira/brow

[jira] [Created] (SPARK-13158) Show the information of broadcast blocks in WebUI

2016-02-03 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-13158: Summary: Show the information of broadcast blocks in WebUI Key: SPARK-13158 URL: https://issues.apache.org/jira/browse/SPARK-13158 Project: Spark Iss

[jira] [Created] (SPARK-13361) Add benchmark codes for Encoder#compress() in CompressionSchemeBenchmark

2016-02-17 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-13361: Summary: Add benchmark codes for Encoder#compress() in CompressionSchemeBenchmark Key: SPARK-13361 URL: https://issues.apache.org/jira/browse/SPARK-13361 Proj

[jira] [Commented] (SPARK-8000) SQLContext.read.load() should be able to auto-detect input data

2016-02-17 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-8000?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15150348#comment-15150348 ] Takeshi Yamamuro commented on SPARK-8000: - This ticket covers files exported by ot

[jira] [Commented] (SPARK-12449) Pushing down arbitrary logical plans to data sources

2016-02-17 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12449?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15150372#comment-15150372 ] Takeshi Yamamuro commented on SPARK-12449: -- I agree though, there are many jira

[jira] [Commented] (SPARK-13656) Delete spark.sql.parquet.cacheMetadata

2016-03-23 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13656?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15209561#comment-15209561 ] Takeshi Yamamuro commented on SPARK-13656: -- Okay, I'll make a new ticket with a

[jira] [Created] (SPARK-14193) Skip unnecessary sorts if input data have been already ordered in InMemoryRelation

2016-03-28 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-14193: Summary: Skip unnecessary sorts if input data have been already ordered in InMemoryRelation Key: SPARK-14193 URL: https://issues.apache.org/jira/browse/SPARK-14193

[jira] [Updated] (SPARK-14193) Skip unnecessary sorts if input data have been already ordered in InMemoryRelation

2016-03-28 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-14193: - Description: This ticket is to skip unnecessary sorts if input data have been already or

[jira] [Updated] (SPARK-14193) Skip unnecessary sorts if input data have been already ordered in InMemoryRelation

2016-03-28 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-14193: - Description: This ticket is to skip unnecessary sorts if input data have been already or

[jira] [Updated] (SPARK-14193) Skip unnecessary sorts if input data have been already ordered in InMemoryRelation

2016-03-28 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14193?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-14193: - Description: This ticket is to skip unnecessary sorts if input data have been already or

[jira] [Commented] (SPARK-14260) Increase default value for maxCharsPerColumn

2016-03-30 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14260?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15217664#comment-15217664 ] Takeshi Yamamuro commented on SPARK-14260: -- In addition to the performance issue

<    3   4   5   6   7   8   9   10   11   12   >