[jira] [Commented] (SPARK-19783) Treat shorter/longer lengths of tokens as malformed records in CSV parser

2017-03-01 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19783?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15889886#comment-15889886 ] Takeshi Yamamuro commented on SPARK-19783: -- I think we must treat longer length

[jira] [Commented] (SPARK-19754) Casting to int from a JSON-parsed float rounds instead of truncating

2017-03-01 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890261#comment-15890261 ] Takeshi Yamamuro commented on SPARK-19754: -- Aha, I see. I also checked this in v

[jira] [Commented] (SPARK-19754) Casting to int from a JSON-parsed float rounds instead of truncating

2017-03-01 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15890287#comment-15890287 ] Takeshi Yamamuro commented on SPARK-19754: -- Good. > Casting to int from a JSON-

[jira] [Commented] (SPARK-19754) Casting to int from a JSON-parsed float rounds instead of truncating

2017-03-01 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19754?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15891422#comment-15891422 ] Takeshi Yamamuro commented on SPARK-19754: -- cc: [~cloud_fan] > Casting to int f

[jira] [Created] (SPARK-25358) MutableProjection supports fallback to an interpreted mode

2018-09-06 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-25358: Summary: MutableProjection supports fallback to an interpreted mode Key: SPARK-25358 URL: https://issues.apache.org/jira/browse/SPARK-25358 Project: Spark

[jira] [Updated] (SPARK-25358) MutableProjection supports fallback to an interpreted mode

2018-09-06 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25358?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-25358: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-23580 > MutableProjection

[jira] [Created] (SPARK-25374) SafeProjection supports fallback to an interpreted mode

2018-09-07 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-25374: Summary: SafeProjection supports fallback to an interpreted mode Key: SPARK-25374 URL: https://issues.apache.org/jira/browse/SPARK-25374 Project: Spark

[jira] [Commented] (SPARK-25374) SafeProjection supports fallback to an interpreted mode

2018-09-07 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16607837#comment-16607837 ] Takeshi Yamamuro commented on SPARK-25374: -- WIP: https://github.com/maropu/spa

[jira] [Updated] (SPARK-25374) SafeProjection supports fallback to an interpreted mode

2018-09-07 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25374?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-25374: - Issue Type: Sub-task (was: Improvement) Parent: SPARK-23580 > SafeProjection su

[jira] [Created] (SPARK-25426) Handles subexpression elimination config inside CodeGeneratorWithInterpretedFallback

2018-09-13 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-25426: Summary: Handles subexpression elimination config inside CodeGeneratorWithInterpretedFallback Key: SPARK-25426 URL: https://issues.apache.org/jira/browse/SPARK-25426

[jira] [Updated] (SPARK-25426) Remove the duplicate fallback logic in UnsafeProjection

2018-09-14 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25426?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-25426: - Summary: Remove the duplicate fallback logic in UnsafeProjection (was: Handles subexpre

[jira] [Commented] (SPARK-25374) SafeProjection supports fallback to an interpreted mode

2018-09-17 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25374?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16617437#comment-16617437 ] Takeshi Yamamuro commented on SPARK-25374: -- I do not have a strong opinion thou

[jira] [Commented] (SPARK-25453) OracleIntegrationSuite IllegalArgumentException: Timestamp format must be yyyy-mm-dd hh:mm:ss[.fffffffff]

2018-09-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25453?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16619121#comment-16619121 ] Takeshi Yamamuro commented on SPARK-25453: -- oh, thanks. Can you fix this? > Or

[jira] [Resolved] (SPARK-25462) hive on spark - got a weird output when count(*) from this script

2018-09-19 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25462?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-25462. -- Resolution: Invalid > hive on spark - got a weird output when count(*) from this scri

***UNCHECKED*** [jira] [Commented] (SPARK-25462) hive on spark - got a weird output when count(*) from this script

2018-09-19 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25462?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620460#comment-16620460 ] Takeshi Yamamuro commented on SPARK-25462: -- You should first ask in the spark-u

***UNCHECKED*** [jira] [Commented] (SPARK-20937) Describe spark.sql.parquet.writeLegacyFormat property in Spark SQL, DataFrames and Datasets Guide

2018-09-19 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20937?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620468#comment-16620468 ] Takeshi Yamamuro commented on SPARK-20937: -- +1, too. Do you have time to write

[jira] [Commented] (SPARK-12978) Skip unnecessary final group-by when input data already clustered with group-by keys

2018-09-19 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12978?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16620605#comment-16620605 ] Takeshi Yamamuro commented on SPARK-12978: -- yea, I think so and we need to disc

[jira] [Created] (SPARK-25498) Fix SQLQueryTestSuite failures when the interpreter mode enabled

2018-09-20 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-25498: Summary: Fix SQLQueryTestSuite failures when the interpreter mode enabled Key: SPARK-25498 URL: https://issues.apache.org/jira/browse/SPARK-25498 Project: Spa

[jira] [Resolved] (SPARK-25469) Eval methods of Concat, Reverse and ElementAt should use pattern matching only once

2018-09-21 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25469?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-25469. -- Resolution: Fixed Assignee: Marek Novotny Fix Version/s: 2.4.0 Issue r

[jira] [Commented] (SPARK-25728) SPIP: Structured Intermediate Representation (Tungsten IR) for generating Java code

2018-10-14 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-25728?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16649681#comment-16649681 ] Takeshi Yamamuro commented on SPARK-25728: -- Cool! I'll join this activity. Anyw

[jira] [Created] (SPARK-25734) Literal should have a value corresponding to dataType

2018-10-15 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-25734: Summary: Literal should have a value corresponding to dataType Key: SPARK-25734 URL: https://issues.apache.org/jira/browse/SPARK-25734 Project: Spark

[jira] [Commented] (SPARK-5484) Pregel should checkpoint periodically to avoid StackOverflowError

2016-09-16 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-5484?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15495984#comment-15495984 ] Takeshi Yamamuro commented on SPARK-5484: - This component seems very inactive, so

[jira] [Created] (SPARK-17683) Support ArrayType in Literal.apply

2016-09-27 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-17683: Summary: Support ArrayType in Literal.apply Key: SPARK-17683 URL: https://issues.apache.org/jira/browse/SPARK-17683 Project: Spark Issue Type: Improv

[jira] [Commented] (SPARK-14393) monotonicallyIncreasingId not monotonically increasing with downstream coalesce

2016-10-09 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15560941#comment-15560941 ] Takeshi Yamamuro commented on SPARK-14393: -- [~hyukjin.kwon] What do u think the

[jira] [Commented] (SPARK-14393) monotonicallyIncreasingId not monotonically increasing with downstream coalesce

2016-10-09 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15561008#comment-15561008 ] Takeshi Yamamuro commented on SPARK-14393: -- yea, since this ticket is totally in

[jira] [Commented] (SPARK-14393) monotonicallyIncreasingId not monotonically increasing with downstream coalesce

2016-10-10 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14393?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=15564350#comment-15564350 ] Takeshi Yamamuro commented on SPARK-14393: -- Since coalesce() just after monotoni

[jira] [Commented] (SPARK-23580) Interpreted mode fallback should be implemented for all expressions & projections

2018-03-05 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23580?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387315#comment-16387315 ] Takeshi Yamamuro commented on SPARK-23580: -- I'll join this work, too. > Interpr

[jira] [Commented] (SPARK-23594) Add interpreted execution for GetExternalRowField expression

2018-03-05 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23594?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387334#comment-16387334 ] Takeshi Yamamuro commented on SPARK-23594: -- If nobody works on this, I'll take t

[jira] [Commented] (SPARK-23611) Extend ExpressionEvalHelper harness to also test failures

2018-03-06 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23611?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16387633#comment-16387633 ] Takeshi Yamamuro commented on SPARK-23611: -- I'll take it. > Extend ExpressionEv

[jira] [Commented] (SPARK-23595) Add interpreted execution for ValidateExternalType expression

2018-03-06 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16388114#comment-16388114 ] Takeshi Yamamuro commented on SPARK-23595: -- [~DylanGuedes] oh, I'm already worki

[jira] [Commented] (SPARK-23595) Add interpreted execution for ValidateExternalType expression

2018-03-06 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23595?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16388707#comment-16388707 ] Takeshi Yamamuro commented on SPARK-23595: -- ok, If you need help in other ticket

[jira] [Commented] (SPARK-23584) Add interpreted execution to NewInstance expression

2018-03-06 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23584?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16389140#comment-16389140 ] Takeshi Yamamuro commented on SPARK-23584: -- I'm working on it. > Add interprete

[jira] [Commented] (SPARK-23589) Add interpreted execution for ExternalMapToCatalyst expression

2018-03-08 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23589?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392536#comment-16392536 ] Takeshi Yamamuro commented on SPARK-23589: -- I'll make a pr after I finish other

[jira] [Commented] (SPARK-23588) Add interpreted execution for CatalystToExternalMap expression

2018-03-09 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16392768#comment-16392768 ] Takeshi Yamamuro commented on SPARK-23588: -- I'm working on this. > Add interpre

[jira] [Commented] (SPARK-23634) AttributeReferences may be too conservative wrt nullability after optimization

2018-03-11 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23634?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394777#comment-16394777 ] Takeshi Yamamuro commented on SPARK-23634: -- Probably, this ticket is duplicate t

[jira] [Commented] (SPARK-18134) SQL: MapType in Group BY and Joins not working

2018-03-11 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18134?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394787#comment-16394787 ] Takeshi Yamamuro commented on SPARK-18134: -- Sorry for interrupting you, but the

[jira] [Commented] (SPARK-23588) Add interpreted execution for CatalystToExternalMap expression

2018-03-11 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16394827#comment-16394827 ] Takeshi Yamamuro commented on SPARK-23588: -- I'll make a pr later: https://githu

[jira] [Commented] (SPARK-23655) Add support for type aclitem (PostgresDialect)

2018-03-12 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23655?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16396537#comment-16396537 ] Takeshi Yamamuro commented on SPARK-23655: -- Is it bad to cast `aclitem` to strin

[jira] [Created] (SPARK-23662) Support selective tests in SQLQueryTestSuite

2018-03-12 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-23662: Summary: Support selective tests in SQLQueryTestSuite Key: SPARK-23662 URL: https://issues.apache.org/jira/browse/SPARK-23662 Project: Spark Issue Ty

[jira] [Commented] (SPARK-23665) Add adaptive algorithm to select query result collect method

2018-03-14 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16398358#comment-16398358 ] Takeshi Yamamuro commented on SPARK-23665: -- IMHO it's ok to do so in application

[jira] [Commented] (SPARK-19842) Informational Referential Integrity Constraints Support in Spark

2018-03-14 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399586#comment-16399586 ] Takeshi Yamamuro commented on SPARK-19842: -- I think he just want to make the goo

[jira] [Commented] (SPARK-23666) Undeterministic column name with UDFs

2018-03-14 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23666?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399633#comment-16399633 ] Takeshi Yamamuro commented on SPARK-23666: -- This is just a bug, so I'll make a p

[jira] [Comment Edited] (SPARK-19842) Informational Referential Integrity Constraints Support in Spark

2018-03-14 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399586#comment-16399586 ] Takeshi Yamamuro edited comment on SPARK-19842 at 3/15/18 2:41 AM:

[jira] [Commented] (SPARK-23677) Selecting columns from joined DataFrames with the same origin yields wrong results

2018-03-14 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23677?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16399969#comment-16399969 ] Takeshi Yamamuro commented on SPARK-23677: -- You mean this ticket? SPARK-14948. I

[jira] [Resolved] (SPARK-23662) Support selective tests in SQLQueryTestSuite

2018-03-16 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23662?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-23662. -- Resolution: Not A Problem > Support selective tests in SQLQueryTestSuite >

[jira] [Commented] (SPARK-23665) Add adaptive algorithm to select query result collect method

2018-03-16 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23665?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16401677#comment-16401677 ] Takeshi Yamamuro commented on SPARK-23665: -- I feel most database users explicitl

[jira] [Commented] (SPARK-23772) Provide an option to ignore column of all null values or empty map/array during JSON schema inference

2018-03-22 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23772?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16410791#comment-16410791 ] Takeshi Yamamuro commented on SPARK-23772: -- ya, I'll take this, thanks! > Provi

[jira] [Created] (SPARK-23793) Handle database names in spark.udf.register()

2018-03-26 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-23793: Summary: Handle database names in spark.udf.register() Key: SPARK-23793 URL: https://issues.apache.org/jira/browse/SPARK-23793 Project: Spark Issue T

[jira] [Commented] (SPARK-23793) Handle database names in spark.udf.register()

2018-03-26 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23793?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16413542#comment-16413542 ] Takeshi Yamamuro commented on SPARK-23793: -- [~smilegator] This behaviour is expe

[jira] [Commented] (SPARK-23839) consider bucket join in cost-based JoinReorder rule

2018-04-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422026#comment-16422026 ] Takeshi Yamamuro commented on SPARK-23839: -- Is it not enough to add a simple opt

[jira] [Commented] (SPARK-19842) Informational Referential Integrity Constraints Support in Spark

2018-04-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422041#comment-16422041 ] Takeshi Yamamuro commented on SPARK-19842: -- Probably, the proposed might comply

[jira] [Commented] (SPARK-23791) Sub-optimal generated code for sum aggregating

2018-04-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422062#comment-16422062 ] Takeshi Yamamuro commented on SPARK-23791: -- Probably, this issue might to be rel

[jira] [Commented] (SPARK-23791) Sub-optimal generated code for sum aggregating

2018-04-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422074#comment-16422074 ] Takeshi Yamamuro commented on SPARK-23791: -- sure, I will. > Sub-optimal generat

[jira] [Commented] (SPARK-23797) SparkSQL performance on small TPCDS tables is very low when compared to Drill or Presto

2018-04-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23797?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422078#comment-16422078 ] Takeshi Yamamuro commented on SPARK-23797: -- I think this ticket seems to be inva

[jira] [Reopened] (SPARK-21870) Split codegen'd aggregation code into small functions for the HotSpot

2018-04-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21870?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro reopened SPARK-21870: -- > Split codegen'd aggregation code into small functions for the HotSpot > -

[jira] [Commented] (SPARK-21870) Split codegen'd aggregation code into small functions for the HotSpot

2018-04-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21870?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422093#comment-16422093 ] Takeshi Yamamuro commented on SPARK-21870: -- Reopened by the suggestion of [~mgai

[jira] [Commented] (SPARK-23791) Sub-optimal generated code for sum aggregating

2018-04-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422095#comment-16422095 ] Takeshi Yamamuro commented on SPARK-23791: -- Reopend (https://issues.apache.org/j

[jira] [Commented] (SPARK-23839) consider bucket join in cost-based JoinReorder rule

2018-04-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23839?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422177#comment-16422177 ] Takeshi Yamamuro commented on SPARK-23839: -- Yea, I know the case. I just suggest

[jira] [Commented] (SPARK-21363) Prevent column name duplication in temporary view

2018-04-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21363?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16423264#comment-16423264 ] Takeshi Yamamuro commented on SPARK-21363: -- What's the concrete case? I think is

[jira] [Commented] (SPARK-23791) Sub-optimal generated code for sum aggregating

2018-04-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23791?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16423312#comment-16423312 ] Takeshi Yamamuro commented on SPARK-23791: -- I checked the root cause of this tic

[jira] [Resolved] (SPARK-23791) Sub-optimal generated code for sum aggregating

2018-04-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23791?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro resolved SPARK-23791. -- Resolution: Duplicate > Sub-optimal generated code for sum aggregating > --

[jira] [Commented] (SPARK-23856) Spark jdbc setQueryTimeout option

2018-04-03 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23856?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16423848#comment-16423848 ] Takeshi Yamamuro commented on SPARK-23856: -- It sounds some reasonable to me. It

[jira] [Comment Edited] (SPARK-19842) Informational Referential Integrity Constraints Support in Spark

2018-04-04 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16422041#comment-16422041 ] Takeshi Yamamuro edited comment on SPARK-19842 at 4/5/18 12:29 AM:

[jira] [Commented] (SPARK-19842) Informational Referential Integrity Constraints Support in Spark

2018-04-04 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19842?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16426369#comment-16426369 ] Takeshi Yamamuro commented on SPARK-19842: -- Sorry, my bad. I just suggested we m

[jira] [Commented] (SPARK-23880) table cache should be lazy and don't trigger any jobs.

2018-04-05 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23880?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16428018#comment-16428018 ] Takeshi Yamamuro commented on SPARK-23880: -- ya, I'll look into this. Thank! > t

[jira] [Commented] (SPARK-12105) Add a DataFrame.show() with argument for output PrintStream

2018-04-11 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12105?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16435019#comment-16435019 ] Takeshi Yamamuro commented on SPARK-12105: -- I feel it is enough to use Console?

[jira] [Commented] (SPARK-23711) Add fallback to interpreted execution logic

2018-04-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23711?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16443588#comment-16443588 ] Takeshi Yamamuro commented on SPARK-23711: -- +1, I also think so. > Add fallback

[jira] [Commented] (SPARK-24025) Join of bucketed and non-bucketed tables can give two exchanges and sorts for non-bucketed side

2018-04-19 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24025?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16444911#comment-16444911 ] Takeshi Yamamuro commented on SPARK-24025: -- This is the same with https://issues

[jira] [Commented] (SPARK-24043) InterpretedPredicate.eval fails if expression tree contains Nondeterministic expressions

2018-04-22 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24043?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447614#comment-16447614 ] Takeshi Yamamuro commented on SPARK-24043: -- I tried this in the master and v2.3

[jira] [Commented] (SPARK-24030) SparkSQL percentile_approx function is too slow for over 1,060,000 records.

2018-04-22 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-24030?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16447626#comment-16447626 ] Takeshi Yamamuro commented on SPARK-24030: -- I quickly tried this at least in the

[jira] [Commented] (SPARK-22771) SQL concat for binary

2017-12-13 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16289539#comment-16289539 ] Takeshi Yamamuro commented on SPARK-22771: -- fyi: postgresql has the behaiouvr as

[jira] [Commented] (SPARK-22771) SQL concat for binary

2017-12-13 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16290101#comment-16290101 ] Takeshi Yamamuro commented on SPARK-22771: -- [~smilegator] It's worth fixing this

[jira] [Commented] (SPARK-22771) SQL concat for binary

2017-12-13 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22771?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16290174#comment-16290174 ] Takeshi Yamamuro commented on SPARK-22771: -- ok, I'll take > SQL concat for bina

[jira] [Created] (SPARK-22800) Add a SSB query suite

2017-12-15 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-22800: Summary: Add a SSB query suite Key: SPARK-22800 URL: https://issues.apache.org/jira/browse/SPARK-22800 Project: Spark Issue Type: Test Comp

[jira] [Commented] (SPARK-17647) SQL LIKE does not handle backslashes correctly

2017-12-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295976#comment-16295976 ] Takeshi Yamamuro commented on SPARK-17647: -- I'm looking into the code and I'll

[jira] [Commented] (SPARK-17647) SQL LIKE does not handle backslashes correctly

2017-12-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16295992#comment-16295992 ] Takeshi Yamamuro commented on SPARK-17647: -- It seems the master still handle ''%

[jira] [Issue Comment Deleted] (SPARK-17647) SQL LIKE does not handle backslashes correctly

2017-12-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takeshi Yamamuro updated SPARK-17647: - Comment: was deleted (was: It seems the master still handle ''%\\%' as `(?s).*\Q%\E` inst

[jira] [Comment Edited] (SPARK-17647) SQL LIKE does not handle backslashes correctly

2017-12-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16296234#comment-16296234 ] Takeshi Yamamuro edited comment on SPARK-17647 at 12/19/17 5:29 AM: ---

[jira] [Commented] (SPARK-17647) SQL LIKE does not handle backslashes correctly

2017-12-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17647?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16296234#comment-16296234 ] Takeshi Yamamuro commented on SPARK-17647: -- Probably, is it okay to set spark.sq

[jira] [Commented] (SPARK-22825) Incorrect results of Casting Array to String

2017-12-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16296304#comment-16296304 ] Takeshi Yamamuro commented on SPARK-22825: -- [~smilegator] you're taking on this

[jira] [Commented] (SPARK-22825) Incorrect results of Casting Array to String

2017-12-18 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22825?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16296308#comment-16296308 ] Takeshi Yamamuro commented on SPARK-22825: -- ok > Incorrect results of Casting A

[jira] [Commented] (SPARK-22553) Drop FROM in nonReserved

2018-01-01 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22553?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16307469#comment-16307469 ] Takeshi Yamamuro commented on SPARK-22553: -- Look fine to me > Drop FROM in nonR

[jira] [Created] (SPARK-22937) SQL elt for binary inputs

2018-01-02 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-22937: Summary: SQL elt for binary inputs Key: SPARK-22937 URL: https://issues.apache.org/jira/browse/SPARK-22937 Project: Spark Issue Type: Improvement

[jira] [Commented] (SPARK-21687) Spark SQL should set createTime for Hive partition

2018-01-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308153#comment-16308153 ] Takeshi Yamamuro commented on SPARK-21687: -- I feel this make some sense (But, th

[jira] [Comment Edited] (SPARK-21687) Spark SQL should set createTime for Hive partition

2018-01-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21687?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308153#comment-16308153 ] Takeshi Yamamuro edited comment on SPARK-21687 at 1/2/18 10:26 PM:

[jira] [Commented] (SPARK-22942) Spark Sql UDF throwing NullPointer when adding a filter on a columns that uses that UDF

2018-01-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16308830#comment-16308830 ] Takeshi Yamamuro commented on SPARK-22942: -- I think you just need NULL checks; {

[jira] [Commented] (SPARK-22942) Spark Sql UDF throwing NullPointer when adding a filter on a columns that uses that UDF

2018-01-02 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-22942?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16309037#comment-16309037 ] Takeshi Yamamuro commented on SPARK-22942: -- Since spark passes null to udfs in o

[jira] [Created] (SPARK-22973) Incorrect results of casting Map to String

2018-01-05 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-22973: Summary: Incorrect results of casting Map to String Key: SPARK-22973 URL: https://issues.apache.org/jira/browse/SPARK-22973 Project: Spark Issue Type

[jira] [Created] (SPARK-22981) Incorrect results of casting Struct to String

2018-01-06 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-22981: Summary: Incorrect results of casting Struct to String Key: SPARK-22981 URL: https://issues.apache.org/jira/browse/SPARK-22981 Project: Spark Issue T

[jira] [Created] (SPARK-23023) Incorrect results of printing Array/Map/Struct in showString

2018-01-09 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-23023: Summary: Incorrect results of printing Array/Map/Struct in showString Key: SPARK-23023 URL: https://issues.apache.org/jira/browse/SPARK-23023 Project: Spark

[jira] [Commented] (SPARK-23027) optimizer a simple query using a non-existent data is too slow

2018-01-10 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23027?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16321487#comment-16321487 ] Takeshi Yamamuro commented on SPARK-23027: -- You tried v2.1? https://databricks.

[jira] [Commented] (SPARK-23021) AnalysisBarrier should not cut off the explain output for Parsed Logical Plan

2018-01-11 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323599#comment-16323599 ] Takeshi Yamamuro commented on SPARK-23021: -- hi, kris, you're working on it? I th

[jira] [Commented] (SPARK-23021) AnalysisBarrier should not cut off the explain output for Parsed Logical Plan

2018-01-11 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323623#comment-16323623 ] Takeshi Yamamuro commented on SPARK-23021: -- yea, sure, I have bandwidth to do :)

[jira] [Created] (SPARK-23054) Incorrect results of casting UserDefinedType to String

2018-01-11 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-23054: Summary: Incorrect results of casting UserDefinedType to String Key: SPARK-23054 URL: https://issues.apache.org/jira/browse/SPARK-23054 Project: Spark

[jira] [Commented] (SPARK-23021) AnalysisBarrier should not cut off the explain output for Parsed Logical Plan

2018-01-12 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23021?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16323822#comment-16323822 ] Takeshi Yamamuro commented on SPARK-23021: -- ok, I'll make a pr that way > Analy

[jira] [Commented] (SPARK-21274) Implement EXCEPT ALL and INTERSECT ALL

2018-01-15 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16326119#comment-16326119 ] Takeshi Yamamuro commented on SPARK-21274: -- Nobody takes this? If so, I'll do (P

[jira] [Commented] (SPARK-21274) Implement EXCEPT ALL and INTERSECT ALL

2018-01-16 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16328028#comment-16328028 ] Takeshi Yamamuro commented on SPARK-21274: -- yea, I tried though, I couldn't find

[jira] [Comment Edited] (SPARK-21274) Implement EXCEPT ALL and INTERSECT ALL

2018-01-16 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-21274?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=16328028#comment-16328028 ] Takeshi Yamamuro edited comment on SPARK-21274 at 1/17/18 12:05 AM: ---

[jira] [Created] (SPARK-23167) Update TPCDS queries from v1.4 to v2.7 (latest)

2018-01-20 Thread Takeshi Yamamuro (JIRA)
Takeshi Yamamuro created SPARK-23167: Summary: Update TPCDS queries from v1.4 to v2.7 (latest) Key: SPARK-23167 URL: https://issues.apache.org/jira/browse/SPARK-23167 Project: Spark Issue

[jira] [Commented] (SPARK-23167) Update TPCDS queries from v1.4 to v2.7 (latest)

2018-01-20 Thread Takeshi Yamamuro (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-23167?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel&focusedCommentId=1692#comment-1692 ] Takeshi Yamamuro commented on SPARK-23167: -- cc: [~smilegator] > Update TPCDS qu

<    2   3   4   5   6   7   8   9   10   11   >