[GitHub] spark pull request #22039: [SPARK-25036][SQL][FOLLOW-UP] Avoid match may not...

2018-08-08 Thread asfgit 
Github user asfgit closed the pull request at:

https://github.com/apache/spark/pull/22039


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22039: [SPARK-25036][SQL][FOLLOW-UP] Avoid match may not...

2018-08-08 Thread kiszk 
Github user kiszk commented on a diff in the pull request:

https://github.com/apache/spark/pull/22039#discussion_r208666588
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/SymmetricHashJoinStateManager.scala
 ---
@@ -273,6 +273,9 @@ class SymmetricHashJoinStateManager(
   s.copy(desc = newDesc(desc)) -> value
 case (s @ StateStoreCustomTimingMetric(_, desc), value) =>
   s.copy(desc = newDesc(desc)) -> value
+case (s, _) =>
+  throw new IllegalArgumentException(
+  s"Unknown state store custom metric is found at metrics: $s")
--- End diff --

nit: 2 more spaces?


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22039: [SPARK-25036][SQL][FOLLOW-UP] Avoid match may not...

2018-08-08 Thread kiszk 
Github user kiszk commented on a diff in the pull request:

https://github.com/apache/spark/pull/22039#discussion_r208666245
  
--- Diff: 
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala
 ---
@@ -394,6 +394,9 @@ class MicroBatchExecution(
   case (src: Source, off) => src.commit(off)
   case (reader: MicroBatchReader, off) =>
 reader.commit(reader.deserializeOffset(off.json))
+  case (src, _) =>
+throw new IllegalArgumentException(
+  s"Unknows source is found at constructNextBatch: $src")
--- End diff --

nit: `Unknows` -> `Unknown`


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #22039: [SPARK-25036][SQL][FOLLOW-UP] Avoid match may not...

2018-08-08 Thread ueshin 
GitHub user ueshin opened a pull request:

https://github.com/apache/spark/pull/22039

[SPARK-25036][SQL][FOLLOW-UP] Avoid match may not be exhaustive in 
Scala-2.12.

## What changes were proposed in this pull request?

This is a follow-up pr of #22014.

We still have some more compilation errors in scala-2.12 with sbt:

```
[error] [warn] 
/.../sql/core/src/main/scala/org/apache/spark/sql/DataFrameNaFunctions.scala:493:
 match may not be exhaustive.
[error] It would fail on the following input: (_, _)
[error] [warn]   val typeMatches = (targetType, f.dataType) match {
[error] [warn] 
[error] [warn] 
/.../sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/MicroBatchExecution.scala:393:
 match may not be exhaustive.
[error] It would fail on the following input: (_, _)
[error] [warn] 
prevBatchOff.get.toStreamProgress(sources).foreach {
[error] [warn] 
[error] [warn] 
/.../sql/core/src/main/scala/org/apache/spark/sql/execution/aggregate/AggUtils.scala:173:
 match may not be exhaustive.
[error] It would fail on the following input: AggregateExpression(_, _, 
false, _)
[error] [warn] val rewrittenDistinctFunctions = 
functionsWithDistinct.map {
[error] [warn] 
[error] [warn] 
/.../sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/state/SymmetricHashJoinStateManager.scala:271:
 match may not be exhaustive.
[error] It would fail on the following input: (_, _)
[error] [warn]   keyWithIndexToValueMetrics.customMetrics.map {
[error] [warn] 
[error] [warn] 
/.../sql/core/src/main/scala/org/apache/spark/sql/execution/command/tables.scala:959:
 match may not be exhaustive.
[error] It would fail on the following input: CatalogTableType(_)
[error] [warn] val tableTypeString = metadata.tableType match {
[error] [warn] 
[error] [warn] 
/.../sql/hive/src/main/scala/org/apache/spark/sql/hive/client/HiveClientImpl.scala:923:
 match may not be exhaustive.
[error] It would fail on the following input: CatalogTableType(_)
[error] [warn] hiveTable.setTableType(table.tableType match {
[error] [warn]
```

## How was this patch tested?

Manually build with Scala-2.12.


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/ueshin/apache-spark 
issues/SPARK-25036/fix_match

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/22039.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #22039


commit e9220b4e68178a6cba4e02bf34f0623a80dbc31f
Author: Takuya UESHIN 
Date:   2018-08-08T15:59:25Z

Add default case to match.




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org