[jira] [Resolved] (SPARK-17914) Spark SQL casting to TimestampType with nanosecond results in incorrect timestamp

2017-06-12 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-17914. --- Resolution: Fixed Fix Version/s: 2.3.0 2.2.1 Issue resolved by

[jira] [Assigned] (SPARK-17914) Spark SQL casting to TimestampType with nanosecond results in incorrect timestamp

2017-06-12 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17914?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-17914: - Assignee: Anton Okolnychyi > Spark SQL casting to TimestampType with nanosecond results

[jira] [Assigned] (SPARK-19732) DataFrame.fillna() does not work for bools in PySpark

2017-06-03 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-19732: - Assignee: Ruben Berenguel > DataFrame.fillna() does not work for bools in PySpark >

[jira] [Resolved] (SPARK-19732) DataFrame.fillna() does not work for bools in PySpark

2017-06-02 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-19732?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-19732. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18164

[jira] [Resolved] (SPARK-20910) Build-in SQL Function Support - UUID

2017-06-01 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-20910. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18136

[jira] [Assigned] (SPARK-20910) Build-in SQL Function Support - UUID

2017-06-01 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20910?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin reassigned SPARK-20910: - Assignee: Yuming Wang > Build-in SQL Function Support - UUID >

[jira] [Resolved] (SPARK-20909) Build-in SQL Function Support - DAYOFWEEK

2017-05-30 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20909?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-20909. --- Resolution: Fixed Fix Version/s: 2.3.0 Issue resolved by pull request 18134

[jira] [Created] (SPARK-20668) Modify ScalaUDF to handle nullability.

2017-05-08 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-20668: - Summary: Modify ScalaUDF to handle nullability. Key: SPARK-20668 URL: https://issues.apache.org/jira/browse/SPARK-20668 Project: Spark Issue Type:

[jira] [Commented] (SPARK-20588) from_utc_timestamp causes bottleneck

2017-05-07 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-20588?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000290#comment-16000290 ] Takuya Ueshin commented on SPARK-20588: --- I agree with caching per thread for now. >

[jira] [Commented] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

2017-05-07 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=16000213#comment-16000213 ] Takuya Ueshin commented on SPARK-12297: --- Issue resolved by pull request 16781

[jira] [Resolved] (SPARK-12297) Add work-around for Parquet/Hive int96 timestamp bug.

2017-05-07 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-12297?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-12297. --- Resolution: Fixed Fix Version/s: 2.3.0 > Add work-around for Parquet/Hive int96

[jira] [Resolved] (SPARK-10364) Support Parquet logical type TIMESTAMP_MILLIS

2017-04-03 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-10364?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin resolved SPARK-10364. --- Resolution: Fixed Fix Version/s: 2.2.0 Issue resolved by pull request 15332

[jira] [Created] (SPARK-20201) Flaky Test: org.apache.spark.sql.catalyst.expressions.OrderingSuite

2017-04-03 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-20201: - Summary: Flaky Test: org.apache.spark.sql.catalyst.expressions.OrderingSuite Key: SPARK-20201 URL: https://issues.apache.org/jira/browse/SPARK-20201 Project: Spark

[jira] [Created] (SPARK-20200) Flaky Test: org.apache.spark.rdd.LocalCheckpointSuite

2017-04-03 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-20200: - Summary: Flaky Test: org.apache.spark.rdd.LocalCheckpointSuite Key: SPARK-20200 URL: https://issues.apache.org/jira/browse/SPARK-20200 Project: Spark

[jira] [Created] (SPARK-20131) Flaky Test: org.apache.spark.streaming.StreamingContextSuite

2017-03-28 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-20131: - Summary: Flaky Test: org.apache.spark.streaming.StreamingContextSuite Key: SPARK-20131 URL: https://issues.apache.org/jira/browse/SPARK-20131 Project: Spark

[jira] [Commented] (SPARK-18939) Timezone support in partition values.

2016-12-19 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-18939?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15763375#comment-15763375 ] Takuya Ueshin commented on SPARK-18939: --- Yes, I think so. > Timezone support in partition values.

[jira] [Created] (SPARK-18939) Timezone support in partition values.

2016-12-19 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-18939: - Summary: Timezone support in partition values. Key: SPARK-18939 URL: https://issues.apache.org/jira/browse/SPARK-18939 Project: Spark Issue Type: Sub-task

[jira] [Created] (SPARK-18623) Add `returnNullable` to `StaticInvoke` and modify it to handle properly.

2016-11-29 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-18623: - Summary: Add `returnNullable` to `StaticInvoke` and modify it to handle properly. Key: SPARK-18623 URL: https://issues.apache.org/jira/browse/SPARK-18623 Project:

[jira] [Created] (SPARK-18585) Use `ev.isNull = "false"` if possible for Janino to have a chance to optimize.

2016-11-24 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-18585: - Summary: Use `ev.isNull = "false"` if possible for Janino to have a chance to optimize. Key: SPARK-18585 URL: https://issues.apache.org/jira/browse/SPARK-18585

[jira] [Created] (SPARK-18583) Fix nullability of InputFileName.

2016-11-24 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-18583: - Summary: Fix nullability of InputFileName. Key: SPARK-18583 URL: https://issues.apache.org/jira/browse/SPARK-18583 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-18500) Make GenericStrategy be able to prune plans by itself after placeholders are replaced.

2016-11-17 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-18500: - Summary: Make GenericStrategy be able to prune plans by itself after placeholders are replaced. Key: SPARK-18500 URL: https://issues.apache.org/jira/browse/SPARK-18500

[jira] [Created] (SPARK-18467) Refactor StaticInvoke, Invoke and NewInstance.

2016-11-15 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-18467: - Summary: Refactor StaticInvoke, Invoke and NewInstance. Key: SPARK-18467 URL: https://issues.apache.org/jira/browse/SPARK-18467 Project: Spark Issue Type:

[jira] [Created] (SPARK-18442) Fix nullability of WrapOption.

2016-11-14 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-18442: - Summary: Fix nullability of WrapOption. Key: SPARK-18442 URL: https://issues.apache.org/jira/browse/SPARK-18442 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-18398) Fix nullabilities of MapObjects and optimize not to check null if lambda is not nullable.

2016-11-10 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-18398: - Summary: Fix nullabilities of MapObjects and optimize not to check null if lambda is not nullable. Key: SPARK-18398 URL: https://issues.apache.org/jira/browse/SPARK-18398

[jira] [Created] (SPARK-17985) Bump commons-lang3 version to 3.5.

2016-10-17 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-17985: - Summary: Bump commons-lang3 version to 3.5. Key: SPARK-17985 URL: https://issues.apache.org/jira/browse/SPARK-17985 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-17703) Add unnamed version of addReferenceObj for minor objects.

2016-09-28 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-17703?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-17703: -- Description: There are many `minor` objects in references, which are extracted to the

[jira] [Created] (SPARK-17703) Add unnamed version of addReferenceObj for minor objects.

2016-09-28 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-17703: - Summary: Add unnamed version of addReferenceObj for minor objects. Key: SPARK-17703 URL: https://issues.apache.org/jira/browse/SPARK-17703 Project: Spark

[jira] [Created] (SPARK-17702) Code generation including too many mutable states exceeds JVM size limit.

2016-09-27 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-17702: - Summary: Code generation including too many mutable states exceeds JVM size limit. Key: SPARK-17702 URL: https://issues.apache.org/jira/browse/SPARK-17702 Project:

[jira] [Created] (SPARK-16189) Add ExistingRDD logical plan for input with RDD to have a chance to eliminate serialize/deserialize.

2016-06-24 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-16189: - Summary: Add ExistingRDD logical plan for input with RDD to have a chance to eliminate serialize/deserialize. Key: SPARK-16189 URL:

[jira] [Closed] (SPARK-15980) Add PushPredicateThroughObjectConsumer rule to Optimizer.

2016-06-24 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15980?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin closed SPARK-15980. - Resolution: Duplicate > Add PushPredicateThroughObjectConsumer rule to Optimizer. >

[jira] [Created] (SPARK-16167) RowEncoder should preserve array/map type nullability.

2016-06-23 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-16167: - Summary: RowEncoder should preserve array/map type nullability. Key: SPARK-16167 URL: https://issues.apache.org/jira/browse/SPARK-16167 Project: Spark

[jira] [Created] (SPARK-15980) Add PushPredicateThroughObjectConsumer rule to Optimizer.

2016-06-15 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-15980: - Summary: Add PushPredicateThroughObjectConsumer rule to Optimizer. Key: SPARK-15980 URL: https://issues.apache.org/jira/browse/SPARK-15980 Project: Spark

[jira] [Created] (SPARK-15915) CacheManager should use canonicalized plan for planToCache.

2016-06-13 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-15915: - Summary: CacheManager should use canonicalized plan for planToCache. Key: SPARK-15915 URL: https://issues.apache.org/jira/browse/SPARK-15915 Project: Spark

[jira] [Created] (SPARK-15870) DataFrame can't execute after uncacheTable.

2016-06-10 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-15870: - Summary: DataFrame can't execute after uncacheTable. Key: SPARK-15870 URL: https://issues.apache.org/jira/browse/SPARK-15870 Project: Spark Issue Type:

[jira] [Commented] (SPARK-14752) LazilyGenerateOrdering throws NullPointerException

2016-06-02 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-14752?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15312215#comment-15312215 ] Takuya Ueshin commented on SPARK-14752: --- I agree that we should avoid null checks or lazy vals.

[jira] [Updated] (SPARK-15483) IncrementalExecution should use extra strategies.

2016-05-23 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15483?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-15483: -- Description: Extra strategies does not work for streams because {{IncrementalExecution}} uses

[jira] [Created] (SPARK-15483) IncrementalExecution should use extra strategies.

2016-05-23 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-15483: - Summary: IncrementalExecution should use extra strategies. Key: SPARK-15483 URL: https://issues.apache.org/jira/browse/SPARK-15483 Project: Spark Issue

[jira] [Created] (SPARK-15400) CreateNamedStruct and CreateNamedStructUnsafe should preserve metadata of value expressions if it is NamedExpression.

2016-05-19 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-15400: - Summary: CreateNamedStruct and CreateNamedStructUnsafe should preserve metadata of value expressions if it is NamedExpression. Key: SPARK-15400 URL:

[jira] [Created] (SPARK-15313) EmbedSerializerInFilter rule should keep exprIds of output of surrounded SerializeFromObject.

2016-05-13 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-15313: - Summary: EmbedSerializerInFilter rule should keep exprIds of output of surrounded SerializeFromObject. Key: SPARK-15313 URL: https://issues.apache.org/jira/browse/SPARK-15313

[jira] [Updated] (SPARK-15308) RowEncoder should preserve nested column name.

2016-05-12 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-15308?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-15308: -- Description: The following code generates wrong schema: {code} val schema = new

[jira] [Created] (SPARK-15308) RowEncoder should preserve nested column name.

2016-05-12 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-15308: - Summary: RowEncoder should preserve nested column name. Key: SPARK-15308 URL: https://issues.apache.org/jira/browse/SPARK-15308 Project: Spark Issue Type:

[jira] [Updated] (SPARK-13902) Make DAGScheduler not to create duplicate stage.

2016-05-08 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-13902: -- Description: {{DAGScheduler}} sometimes generate incorrect stage graph. Suppose you have the

[jira] [Updated] (SPARK-13902) Make DAGScheduler not to create duplicate stage.

2016-05-08 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-13902: -- Description: {{DAGScheduler}} sometimes generate incorrect stage graph. Suppose you have the

[jira] [Commented] (SPARK-13902) Make DAGScheduler.getAncestorShuffleDependencies() return in topological order to ensure building ancestor stages first.

2016-04-26 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15257721#comment-15257721 ] Takuya Ueshin commented on SPARK-13902: --- I found that the example above does not reproduce the

[jira] [Commented] (SPARK-13902) Make DAGScheduler.getAncestorShuffleDependencies() return in topological order to ensure building ancestor stages first.

2016-04-25 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15257496#comment-15257496 ] Takuya Ueshin commented on SPARK-13902: --- Yes, exactly. > Make

[jira] [Updated] (SPARK-13902) Make DAGScheduler.getAncestorShuffleDependencies() return in topological order to ensure building ancestor stages first.

2016-04-25 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-13902: -- Description: {{DAGScheduler}} sometimes generate incorrect stage graph. Some stages are

[jira] [Commented] (SPARK-13902) Make DAGScheduler.getAncestorShuffleDependencies() return in topological order to ensure building ancestor stages first.

2016-04-25 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13902?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=15257481#comment-15257481 ] Takuya Ueshin commented on SPARK-13902: --- I'm sorry, I made a mistake. I should have written the

[jira] [Updated] (SPARK-13902) Make DAGScheduler.getAncestorShuffleDependencies() return in topological order to ensure building ancestor stages first.

2016-04-24 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-13902: -- Description: {{DAGScheduler}} sometimes generate incorrect stage graph. Some stages are

[jira] [Updated] (SPARK-13902) Make DAGScheduler.getAncestorShuffleDependencies() return in topological order to ensure building ancestor stages first.

2016-04-24 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-13902: -- Description: {{DAGScheduler}} sometimes generate incorrect stage graph. Some stages are

[jira] [Updated] (SPARK-13902) Make DAGScheduler.getAncestorShuffleDependencies() return in topological order to ensure building ancestor stages first.

2016-04-24 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-13902: -- Description: {{DAGScheduler}} sometimes generate incorrect stage graph. Some stages are

[jira] [Created] (SPARK-14793) Code generation for large complex type exceeds JVM size limit.

2016-04-20 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-14793: - Summary: Code generation for large complex type exceeds JVM size limit. Key: SPARK-14793 URL: https://issues.apache.org/jira/browse/SPARK-14793 Project: Spark

[jira] [Created] (SPARK-14269) Eliminate unnecessary submitStage() call.

2016-03-30 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-14269: - Summary: Eliminate unnecessary submitStage() call. Key: SPARK-14269 URL: https://issues.apache.org/jira/browse/SPARK-14269 Project: Spark Issue Type:

[jira] [Updated] (SPARK-13902) Make DAGScheduler.getAncestorShuffleDependencies() return in topological order to ensure building ancestor stages first.

2016-03-15 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-13902?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-13902: -- Description: {{DAGScheduler}} sometimes generate incorrect stage graph. Some stages are

[jira] [Created] (SPARK-13902) Make DAGScheduler.getAncestorShuffleDependencies() return in topological order to ensure building ancestor stages first.

2016-03-15 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-13902: - Summary: Make DAGScheduler.getAncestorShuffleDependencies() return in topological order to ensure building ancestor stages first. Key: SPARK-13902 URL:

[jira] [Created] (SPARK-13640) Synchronize ScalaReflection.mirror method.

2016-03-02 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-13640: - Summary: Synchronize ScalaReflection.mirror method. Key: SPARK-13640 URL: https://issues.apache.org/jira/browse/SPARK-13640 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-13357) Use generated projection and ordering for TakeOrderedAndProjectNode

2016-02-16 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-13357: - Summary: Use generated projection and ordering for TakeOrderedAndProjectNode Key: SPARK-13357 URL: https://issues.apache.org/jira/browse/SPARK-13357 Project: Spark

[jira] [Created] (SPARK-12976) Add LazilyGenerateOrdering and use it for RangePartitioner of Exchange.

2016-01-24 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-12976: - Summary: Add LazilyGenerateOrdering and use it for RangePartitioner of Exchange. Key: SPARK-12976 URL: https://issues.apache.org/jira/browse/SPARK-12976 Project:

[jira] [Created] (SPARK-8476) Setters inc/decDiskBytesSpilled in TaskMetrics should also be private.

2015-06-18 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-8476: Summary: Setters inc/decDiskBytesSpilled in TaskMetrics should also be private. Key: SPARK-8476 URL: https://issues.apache.org/jira/browse/SPARK-8476 Project: Spark

[jira] [Created] (SPARK-7826) Suppress extra calling getCacheLocs.

2015-05-22 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-7826: Summary: Suppress extra calling getCacheLocs. Key: SPARK-7826 URL: https://issues.apache.org/jira/browse/SPARK-7826 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-5457) Add missing DSL for ApproxCountDistinct.

2015-01-28 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-5457: Summary: Add missing DSL for ApproxCountDistinct. Key: SPARK-5457 URL: https://issues.apache.org/jira/browse/SPARK-5457 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-4720) Remainder should also return null if the divider is 0.

2014-12-03 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-4720: Summary: Remainder should also return null if the divider is 0. Key: SPARK-4720 URL: https://issues.apache.org/jira/browse/SPARK-4720 Project: Spark Issue

[jira] [Closed] (SPARK-4418) Remove restriction of binary arithmetic operations for fixed-precision decimal.

2014-11-18 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4418?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin closed SPARK-4418. Resolution: Not a Problem Remove restriction of binary arithmetic operations for fixed-precision

[jira] [Created] (SPARK-4425) Handle NaN cast to Timestamp correctly

2014-11-15 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-4425: Summary: Handle NaN cast to Timestamp correctly Key: SPARK-4425 URL: https://issues.apache.org/jira/browse/SPARK-4425 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-4425) Handle NaN or Infinity cast to Timestamp correctly

2014-11-15 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-4425?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-4425: - Summary: Handle NaN or Infinity cast to Timestamp correctly (was: Handle NaN cast to Timestamp

[jira] [Created] (SPARK-4428) Use ${scala.binary.version} property for artifactId.

2014-11-15 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-4428: Summary: Use ${scala.binary.version} property for artifactId. Key: SPARK-4428 URL: https://issues.apache.org/jira/browse/SPARK-4428 Project: Spark Issue

[jira] [Created] (SPARK-4429) Build for Scala 2.11 using sbt fails.

2014-11-15 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-4429: Summary: Build for Scala 2.11 using sbt fails. Key: SPARK-4429 URL: https://issues.apache.org/jira/browse/SPARK-4429 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-4420) Change nullability of Cast from DoubleType/FloatType to DecimalType.

2014-11-14 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-4420: Summary: Change nullability of Cast from DoubleType/FloatType to DecimalType. Key: SPARK-4420 URL: https://issues.apache.org/jira/browse/SPARK-4420 Project: Spark

[jira] [Created] (SPARK-4318) Fix empty sum distinct.

2014-11-10 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-4318: Summary: Fix empty sum distinct. Key: SPARK-4318 URL: https://issues.apache.org/jira/browse/SPARK-4318 Project: Spark Issue Type: Bug Components:

[jira] [Created] (SPARK-4319) Enable an ignored test null count.

2014-11-10 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-4319: Summary: Enable an ignored test null count. Key: SPARK-4319 URL: https://issues.apache.org/jira/browse/SPARK-4319 Project: Spark Issue Type: Test

[jira] [Created] (SPARK-4270) Fix Cast from DateType to DecimalType.

2014-11-06 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-4270: Summary: Fix Cast from DateType to DecimalType. Key: SPARK-4270 URL: https://issues.apache.org/jira/browse/SPARK-4270 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-4293) Make Cast be able to handle complex types.

2014-11-06 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-4293: Summary: Make Cast be able to handle complex types. Key: SPARK-4293 URL: https://issues.apache.org/jira/browse/SPARK-4293 Project: Spark Issue Type:

[jira] [Created] (SPARK-4245) Fix containsNull of the result ArrayType of CreateArray expression.

2014-11-05 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-4245: Summary: Fix containsNull of the result ArrayType of CreateArray expression. Key: SPARK-4245 URL: https://issues.apache.org/jira/browse/SPARK-4245 Project: Spark

[jira] [Created] (SPARK-4193) Disable doclint in Java 8 to prevent from build error.

2014-11-01 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-4193: Summary: Disable doclint in Java 8 to prevent from build error. Key: SPARK-4193 URL: https://issues.apache.org/jira/browse/SPARK-4193 Project: Spark Issue

[jira] [Created] (SPARK-3969) Optimizer should have a super class as an interface.

2014-10-16 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-3969: Summary: Optimizer should have a super class as an interface. Key: SPARK-3969 URL: https://issues.apache.org/jira/browse/SPARK-3969 Project: Spark Issue

[jira] [Created] (SPARK-3986) Fix package names to fit their directory names.

2014-10-16 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-3986: Summary: Fix package names to fit their directory names. Key: SPARK-3986 URL: https://issues.apache.org/jira/browse/SPARK-3986 Project: Spark Issue Type:

[jira] [Created] (SPARK-3966) Fix nullabilities of Cast related to DateType.

2014-10-15 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-3966: Summary: Fix nullabilities of Cast related to DateType. Key: SPARK-3966 URL: https://issues.apache.org/jira/browse/SPARK-3966 Project: Spark Issue Type: Bug

[jira] [Created] (SPARK-3764) Invalid dependencies of artifacts in Maven Central Repository.

2014-10-02 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-3764: Summary: Invalid dependencies of artifacts in Maven Central Repository. Key: SPARK-3764 URL: https://issues.apache.org/jira/browse/SPARK-3764 Project: Spark

[jira] [Commented] (SPARK-3764) Invalid dependencies of artifacts in Maven Central Repository.

2014-10-02 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14156576#comment-14156576 ] Takuya Ueshin commented on SPARK-3764: -- Ah, so, these artifacts are only for hadoop-2

[jira] [Commented] (SPARK-3764) Invalid dependencies of artifacts in Maven Central Repository.

2014-10-02 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14156578#comment-14156578 ] Takuya Ueshin commented on SPARK-3764: -- Now I found the instruction

[jira] [Commented] (SPARK-3764) Invalid dependencies of artifacts in Maven Central Repository.

2014-10-02 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14156588#comment-14156588 ] Takuya Ueshin commented on SPARK-3764: -- But there are some codes using

[jira] [Commented] (SPARK-3764) Invalid dependencies of artifacts in Maven Central Repository.

2014-10-02 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14156599#comment-14156599 ] Takuya Ueshin commented on SPARK-3764: -- Ah, I see that {{context.getTaskAttemptID}}

[jira] [Commented] (SPARK-3764) Invalid dependencies of artifacts in Maven Central Repository.

2014-10-02 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14157418#comment-14157418 ] Takuya Ueshin commented on SPARK-3764: -- {{AppendingParquetOutputFormat}} is using

[jira] [Created] (SPARK-3771) AppendingParquetOutputFormat should use reflection to prevent breaking binary-compatibility.

2014-10-02 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-3771: Summary: AppendingParquetOutputFormat should use reflection to prevent breaking binary-compatibility. Key: SPARK-3771 URL: https://issues.apache.org/jira/browse/SPARK-3771

[jira] [Commented] (SPARK-3764) Invalid dependencies of artifacts in Maven Central Repository.

2014-10-02 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14157447#comment-14157447 ] Takuya Ueshin commented on SPARK-3764: -- I filed a new issue SPARK-3771 and close

[jira] [Closed] (SPARK-3764) Invalid dependencies of artifacts in Maven Central Repository.

2014-10-02 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3764?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin closed SPARK-3764. Resolution: Fixed Invalid dependencies of artifacts in Maven Central Repository.

[jira] [Updated] (SPARK-3771) AppendingParquetOutputFormat should use reflection to prevent from breaking binary-compatibility.

2014-10-02 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3771?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-3771: - Summary: AppendingParquetOutputFormat should use reflection to prevent from breaking

[jira] [Commented] (SPARK-3341) The dataType of Sqrt expression should be DoubleType.

2014-09-02 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3341?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14119273#comment-14119273 ] Takuya Ueshin commented on SPARK-3341: -- Hi, this issue's Fix Version should be the

[jira] [Created] (SPARK-3204) MaxOf would be foldable if both left and right are foldable.

2014-08-25 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-3204: Summary: MaxOf would be foldable if both left and right are foldable. Key: SPARK-3204 URL: https://issues.apache.org/jira/browse/SPARK-3204 Project: Spark

[jira] [Commented] (SPARK-3036) Add MapType containing null value support to Parquet.

2014-08-20 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-3036?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14105012#comment-14105012 ] Takuya Ueshin commented on SPARK-3036: -- Ah, that's right. It was my mistake. Newer

[jira] [Created] (SPARK-3063) ExistingRdd should convert Map to catalyst Map.

2014-08-15 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-3063: Summary: ExistingRdd should convert Map to catalyst Map. Key: SPARK-3063 URL: https://issues.apache.org/jira/browse/SPARK-3063 Project: Spark Issue Type:

[jira] [Created] (SPARK-3036) Add MapType containing null value support to Parquet.

2014-08-14 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-3036: Summary: Add MapType containing null value support to Parquet. Key: SPARK-3036 URL: https://issues.apache.org/jira/browse/SPARK-3036 Project: Spark Issue

[jira] [Updated] (SPARK-2969) Make ScalaReflection be able to handle ArrayType.containsNull and MapType.valueContainsNull.

2014-08-13 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-2969: - Summary: Make ScalaReflection be able to handle ArrayType.containsNull and

[jira] [Created] (SPARK-2965) Fix HashOuterJoin output nullabilities.

2014-08-11 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-2965: Summary: Fix HashOuterJoin output nullabilities. Key: SPARK-2965 URL: https://issues.apache.org/jira/browse/SPARK-2965 Project: Spark Issue Type: Bug

[jira] [Updated] (SPARK-2969) Make ScalaReflection be able to handle MapType.containsNull and MapType.valueContainsNull.

2014-08-11 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2969?page=com.atlassian.jira.plugin.system.issuetabpanels:all-tabpanel ] Takuya Ueshin updated SPARK-2969: - Description: Make {{ScalaReflection}} be able to handle like: - Seq\[Int] as

[jira] [Created] (SPARK-2969) Make ScalaReflection be able to handle MapType.containsNull and MapType.valueContainsNull.

2014-08-11 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-2969: Summary: Make ScalaReflection be able to handle MapType.containsNull and MapType.valueContainsNull. Key: SPARK-2969 URL: https://issues.apache.org/jira/browse/SPARK-2969

[jira] [Created] (SPARK-2588) Add some more DSLs.

2014-07-18 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-2588: Summary: Add some more DSLs. Key: SPARK-2588 URL: https://issues.apache.org/jira/browse/SPARK-2588 Project: Spark Issue Type: Improvement

[jira] [Created] (SPARK-2518) Fix foldability of Substring expression.

2014-07-16 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-2518: Summary: Fix foldability of Substring expression. Key: SPARK-2518 URL: https://issues.apache.org/jira/browse/SPARK-2518 Project: Spark Issue Type: Bug

[jira] [Commented] (SPARK-2518) Fix foldability of Substring expression.

2014-07-16 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2518?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14063212#comment-14063212 ] Takuya Ueshin commented on SPARK-2518: -- PRed:

[jira] [Created] (SPARK-2535) Add StringComparison case to NullPropagation.

2014-07-16 Thread Takuya Ueshin (JIRA)
Takuya Ueshin created SPARK-2535: Summary: Add StringComparison case to NullPropagation. Key: SPARK-2535 URL: https://issues.apache.org/jira/browse/SPARK-2535 Project: Spark Issue Type:

[jira] [Commented] (SPARK-2535) Add StringComparison case to NullPropagation.

2014-07-16 Thread Takuya Ueshin (JIRA)
[ https://issues.apache.org/jira/browse/SPARK-2535?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanelfocusedCommentId=14064381#comment-14064381 ] Takuya Ueshin commented on SPARK-2535: -- PR: https://github.com/apache/spark/pull/1451

<    3   4   5   6   7   8   9   >