spark git commit: [SPARK-18175][SQL] Improve the test case coverage of implicit type casting

Repository: spark Updated Branches: refs/heads/master 7eb2ca8e3 -> 9ddec8636 [SPARK-18175][SQL] Improve the test case coverage of implicit type casting ### What changes were proposed in this pull request? So far, we have limited test case coverage about implicit type casting. We need to

spark git commit: [SPARK-18175][SQL] Improve the test case coverage of implicit type casting

Repository: spark Updated Branches: refs/heads/branch-2.1 1e29f0a0d -> 2cf39d638 [SPARK-18175][SQL] Improve the test case coverage of implicit type casting ### What changes were proposed in this pull request? So far, we have limited test case coverage about implicit type casting. We need to

[2/3] spark git commit: [SPARK-17963][SQL][DOCUMENTATION] Add examples (extend) in each expression and improve documentation

http://git-wip-us.apache.org/repos/asf/spark/blob/1e29f0a0/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala -- diff --git

[3/3] spark git commit: [SPARK-17963][SQL][DOCUMENTATION] Add examples (extend) in each expression and improve documentation

[SPARK-17963][SQL][DOCUMENTATION] Add examples (extend) in each expression and improve documentation ## What changes were proposed in this pull request? This PR proposes to change the documentation for functions. Please refer the discussion from https://github.com/apache/spark/pull/15513 The

[1/3] spark git commit: [SPARK-17963][SQL][DOCUMENTATION] Add examples (extend) in each expression and improve documentation

Repository: spark Updated Branches: refs/heads/branch-2.1 5ea2f9e5e -> 1e29f0a0d http://git-wip-us.apache.org/repos/asf/spark/blob/1e29f0a0/sql/core/src/main/scala/org/apache/spark/sql/execution/command/functions.scala -- diff

[2/3] spark git commit: [SPARK-17963][SQL][DOCUMENTATION] Add examples (extend) in each expression and improve documentation

http://git-wip-us.apache.org/repos/asf/spark/blob/7eb2ca8e/sql/catalyst/src/main/scala/org/apache/spark/sql/catalyst/expressions/mathExpressions.scala -- diff --git

[1/3] spark git commit: [SPARK-17963][SQL][DOCUMENTATION] Add examples (extend) in each expression and improve documentation

Repository: spark Updated Branches: refs/heads/master 3a1bc6f47 -> 7eb2ca8e3 http://git-wip-us.apache.org/repos/asf/spark/blob/7eb2ca8e/sql/core/src/main/scala/org/apache/spark/sql/execution/command/functions.scala -- diff

[3/3] spark git commit: [SPARK-17963][SQL][DOCUMENTATION] Add examples (extend) in each expression and improve documentation

[SPARK-17963][SQL][DOCUMENTATION] Add examples (extend) in each expression and improve documentation ## What changes were proposed in this pull request? This PR proposes to change the documentation for functions. Please refer the discussion from https://github.com/apache/spark/pull/15513 The

spark git commit: [SPARK-17470][SQL] unify path for data source table and locationUri for hive serde table

Repository: spark Updated Branches: refs/heads/branch-2.1 2aff2ea81 -> 5ea2f9e5e [SPARK-17470][SQL] unify path for data source table and locationUri for hive serde table ## What changes were proposed in this pull request? Due to a limitation of hive metastore(table location must be

spark git commit: [SPARK-17470][SQL] unify path for data source table and locationUri for hive serde table

Repository: spark Updated Branches: refs/heads/master fd90541c3 -> 3a1bc6f47 [SPARK-17470][SQL] unify path for data source table and locationUri for hive serde table ## What changes were proposed in this pull request? Due to a limitation of hive metastore(table location must be directory

spark git commit: [SPARK-18214][SQL] Simplify RuntimeReplaceable type coercion

Repository: spark Updated Branches: refs/heads/branch-2.1 1eef8e5cd -> 2aff2ea81 [SPARK-18214][SQL] Simplify RuntimeReplaceable type coercion ## What changes were proposed in this pull request? RuntimeReplaceable is used to create aliases for expressions, but the way it deals with type

spark git commit: [SPARK-18214][SQL] Simplify RuntimeReplaceable type coercion

Repository: spark Updated Branches: refs/heads/master 37d95227a -> fd90541c3 [SPARK-18214][SQL] Simplify RuntimeReplaceable type coercion ## What changes were proposed in this pull request? RuntimeReplaceable is used to create aliases for expressions, but the way it deals with type coercion

[1/2] spark git commit: Preparing Spark release v1.6.3-rc2

Repository: spark Updated Branches: refs/heads/branch-1.6 82e98f126 -> 9136e2693 Preparing Spark release v1.6.3-rc2 Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/1e860747 Tree:

[2/2] spark git commit: Preparing development version 1.6.4-SNAPSHOT

Preparing development version 1.6.4-SNAPSHOT Project: http://git-wip-us.apache.org/repos/asf/spark/repo Commit: http://git-wip-us.apache.org/repos/asf/spark/commit/9136e269 Tree: http://git-wip-us.apache.org/repos/asf/spark/tree/9136e269 Diff:

[spark] Git Push Summary

Repository: spark Updated Tags: refs/tags/v1.6.3-rc2 [created] 1e8607474 - To unsubscribe, e-mail: commits-unsubscr...@spark.apache.org For additional commands, e-mail: commits-h...@spark.apache.org

spark git commit: [SPARK-17058][BUILD] Add maven snapshots-and-staging profile to build/test against staging artifacts

Repository: spark Updated Branches: refs/heads/branch-2.1 bd3ea6595 -> 1eef8e5cd [SPARK-17058][BUILD] Add maven snapshots-and-staging profile to build/test against staging artifacts ## What changes were proposed in this pull request? Adds a `snapshots-and-staging profile` so that RCs of

spark git commit: [SPARK-17058][BUILD] Add maven snapshots-and-staging profile to build/test against staging artifacts

Repository: spark Updated Branches: refs/heads/master 3c24299b7 -> 37d95227a [SPARK-17058][BUILD] Add maven snapshots-and-staging profile to build/test against staging artifacts ## What changes were proposed in this pull request? Adds a `snapshots-and-staging profile` so that RCs of

spark git commit: [SPARK-18111][SQL] Wrong approximate quantile answer when multiple records have the minimum value(for branch 2.0)

Repository: spark Updated Branches: refs/heads/branch-2.0 1696bcfad -> 3253ae7f7 [SPARK-18111][SQL] Wrong approximate quantile answer when multiple records have the minimum value(for branch 2.0) ## What changes were proposed in this pull request? When multiple records have the minimum value,

spark git commit: [SPARK-18160][CORE][YARN] spark.files & spark.jars should not be passed to driver in yarn mode

Repository: spark Updated Branches: refs/heads/branch-2.0 eb790c5b1 -> 1696bcfad [SPARK-18160][CORE][YARN] spark.files & spark.jars should not be passed to driver in yarn mode ## What changes were proposed in this pull request? spark.files is still passed to driver in yarn mode, so

spark git commit: [SPARK-18160][CORE][YARN] spark.files & spark.jars should not be passed to driver in yarn mode

Repository: spark Updated Branches: refs/heads/branch-2.1 0093257ea -> bd3ea6595 [SPARK-18160][CORE][YARN] spark.files & spark.jars should not be passed to driver in yarn mode ## What changes were proposed in this pull request? spark.files is still passed to driver in yarn mode, so

spark git commit: [SPARK-18160][CORE][YARN] spark.files & spark.jars should not be passed to driver in yarn mode

Repository: spark Updated Branches: refs/heads/master 02f203107 -> 3c24299b7 [SPARK-18160][CORE][YARN] spark.files & spark.jars should not be passed to driver in yarn mode ## What changes were proposed in this pull request? spark.files is still passed to driver in yarn mode, so SparkContext

spark git commit: [SPARK-17895] Improve doc for rangeBetween and rowsBetween

Repository: spark Updated Branches: refs/heads/master 4af0ce2d9 -> 742e0fea5 [SPARK-17895] Improve doc for rangeBetween and rowsBetween ## What changes were proposed in this pull request? Copied description for row and range based frame boundary from

spark git commit: [SPARK-14393][SQL] values generated by non-deterministic functions shouldn't change after coalesce or union

Repository: spark Updated Branches: refs/heads/branch-2.1 a885d5bbc -> 0093257ea [SPARK-14393][SQL] values generated by non-deterministic functions shouldn't change after coalesce or union ## What changes were proposed in this pull request? When a user appended a column using a

spark git commit: [SPARK-14393][SQL] values generated by non-deterministic functions shouldn't change after coalesce or union

Repository: spark Updated Branches: refs/heads/master 742e0fea5 -> 02f203107 [SPARK-14393][SQL] values generated by non-deterministic functions shouldn't change after coalesce or union ## What changes were proposed in this pull request? When a user appended a column using a

spark git commit: [SPARK-17895] Improve doc for rangeBetween and rowsBetween

Repository: spark Updated Branches: refs/heads/branch-2.1 9be069125 -> a885d5bbc [SPARK-17895] Improve doc for rangeBetween and rowsBetween ## What changes were proposed in this pull request? Copied description for row and range based frame boundary from

spark git commit: [SPARK-17683][SQL] Support ArrayType in Literal.apply

Repository: spark Updated Branches: refs/heads/branch-2.1 41491e540 -> 9be069125 [SPARK-17683][SQL] Support ArrayType in Literal.apply ## What changes were proposed in this pull request? This pr is to add pattern-matching entries for array data in `Literal.apply`. ## How was this patch

spark git commit: [SPARK-17683][SQL] Support ArrayType in Literal.apply

Repository: spark Updated Branches: refs/heads/master f151bd1af -> 4af0ce2d9 [SPARK-17683][SQL] Support ArrayType in Literal.apply ## What changes were proposed in this pull request? This pr is to add pattern-matching entries for array data in `Literal.apply`. ## How was this patch tested?

spark git commit: [SPARK-16796][WEB UI] Mask spark.authenticate.secret on Spark environ…

Repository: spark Updated Branches: refs/heads/branch-2.0 09178b6ee -> eb790c5b1 [SPARK-16796][WEB UI] Mask spark.authenticate.secret on Spark environ… ## What changes were proposed in this pull request? Mask `spark.authenticate.secret` on Spark environment page (Web UI). This is addition

spark git commit: [SPARK-16839][SQL] Simplify Struct creation code path

Repository: spark Updated Branches: refs/heads/branch-2.1 176afa5e8 -> 41491e540 [SPARK-16839][SQL] Simplify Struct creation code path ## What changes were proposed in this pull request? Simplify struct creation, especially the aspect of `CleanupAliases` which missed some aliases when

spark git commit: [SPARK-16839][SQL] Simplify Struct creation code path

Repository: spark Updated Branches: refs/heads/master 9c8deef64 -> f151bd1af [SPARK-16839][SQL] Simplify Struct creation code path ## What changes were proposed in this pull request? Simplify struct creation, especially the aspect of `CleanupAliases` which missed some aliases when handling

spark git commit: [SPARK-18076][CORE][SQL] Fix default Locale used in DateFormat, NumberFormat to Locale.US

Repository: spark Updated Branches: refs/heads/branch-2.1 ab8da1413 -> 176afa5e8 [SPARK-18076][CORE][SQL] Fix default Locale used in DateFormat, NumberFormat to Locale.US ## What changes were proposed in this pull request? Fix `Locale.US` for all usages of `DateFormat`, `NumberFormat` ##

spark git commit: [SPARK-18198][DOC][STREAMING] Highlight code snippets

Repository: spark Updated Branches: refs/heads/branch-2.1 3b624bedf -> ab8da1413 [SPARK-18198][DOC][STREAMING] Highlight code snippets ## What changes were proposed in this pull request? This patch uses `{% highlight lang %}...{% endhighlight %}` to highlight code snippets in the

spark git commit: [SPARK-18076][CORE][SQL] Fix default Locale used in DateFormat, NumberFormat to Locale.US

Repository: spark Updated Branches: refs/heads/master 70a5db7bb -> 9c8deef64 [SPARK-18076][CORE][SQL] Fix default Locale used in DateFormat, NumberFormat to Locale.US ## What changes were proposed in this pull request? Fix `Locale.US` for all usages of `DateFormat`, `NumberFormat` ## How

spark git commit: [SPARK-18133][BRANCH-2.0][EXAMPLES][ML] Python ML Pipeline Exampl…

Repository: spark Updated Branches: refs/heads/branch-2.0 81f080425 -> 09178b6ee [SPARK-18133][BRANCH-2.0][EXAMPLES][ML] Python ML Pipeline Exampl… ## What changes were proposed in this pull request? [Fix] [branch-2.0] In Python 3, there is only one integer type (i.e., int), which mostly

spark git commit: [SPARK-18204][WEBUI] Remove SparkUI.appUIAddress

Repository: spark Updated Branches: refs/heads/master 98ede4949 -> 70a5db7bb [SPARK-18204][WEBUI] Remove SparkUI.appUIAddress ## What changes were proposed in this pull request? Removing `appUIAddress` attribute since it is no longer in use. ## How was this patch tested? Local build

spark git commit: [SPARK-18198][DOC][STREAMING] Highlight code snippets

Repository: spark Updated Branches: refs/heads/master bcbe0 -> 98ede4949 [SPARK-18198][DOC][STREAMING] Highlight code snippets ## What changes were proposed in this pull request? This patch uses `{% highlight lang %}...{% endhighlight %}` to highlight code snippets in the `Structured

spark git commit: [MINOR] Use <= for clarity in Pi examples' Monte Carlo process

Repository: spark Updated Branches: refs/heads/master 2dc048081 -> bcbe0 [MINOR] Use <= for clarity in Pi examples' Monte Carlo process ## What changes were proposed in this pull request? If my understanding is correct we should be rather looking at closed disk than the opened one. ##

spark git commit: [SPARK-17532] Add lock debugging info to thread dumps.

Repository: spark Updated Branches: refs/heads/branch-2.1 4c4bf87ac -> 3b624bedf [SPARK-17532] Add lock debugging info to thread dumps. ## What changes were proposed in this pull request? This adds information to the web UI thread dump page about the JVM locks held by threads and the locks

spark git commit: [SPARK-17532] Add lock debugging info to thread dumps.

Repository: spark Updated Branches: refs/heads/master 85c5424d4 -> 2dc048081 [SPARK-17532] Add lock debugging info to thread dumps. ## What changes were proposed in this pull request? This adds information to the web UI thread dump page about the JVM locks held by threads and the locks that

spark git commit: [SPARK-18144][SQL] logging StreamingQueryListener$QueryStartedEvent

Repository: spark Updated Branches: refs/heads/branch-2.1 85dd07374 -> 4c4bf87ac [SPARK-18144][SQL] logging StreamingQueryListener$QueryStartedEvent ## What changes were proposed in this pull request? The PR fixes the bug that the QueryStartedEvent is not logged the postToAll() in the

spark git commit: [SPARK-18144][SQL] logging StreamingQueryListener$QueryStartedEvent

Repository: spark Updated Branches: refs/heads/branch-2.0 d401a74d4 -> 81f080425 [SPARK-18144][SQL] logging StreamingQueryListener$QueryStartedEvent ## What changes were proposed in this pull request? The PR fixes the bug that the QueryStartedEvent is not logged the postToAll() in the

spark git commit: [SPARK-18144][SQL] logging StreamingQueryListener$QueryStartedEvent

Repository: spark Updated Branches: refs/heads/master a36653c5b -> 85c5424d4 [SPARK-18144][SQL] logging StreamingQueryListener$QueryStartedEvent ## What changes were proposed in this pull request? The PR fixes the bug that the QueryStartedEvent is not logged the postToAll() in the original

spark git commit: [SPARK-18192] Support all file formats in structured streaming

Repository: spark Updated Branches: refs/heads/branch-2.1 e6509c245 -> 85dd07374 [SPARK-18192] Support all file formats in structured streaming ## What changes were proposed in this pull request? This patch adds support for all file formats in structured streaming sinks. This is actually a

spark git commit: [SPARK-18192] Support all file formats in structured streaming

Repository: spark Updated Branches: refs/heads/master abefe2ec4 -> a36653c5b [SPARK-18192] Support all file formats in structured streaming ## What changes were proposed in this pull request? This patch adds support for all file formats in structured streaming sinks. This is actually a very

spark git commit: [SPARK-18183][SPARK-18184] Fix INSERT [INTO|OVERWRITE] TABLE ... PARTITION for Datasource tables

Repository: spark Updated Branches: refs/heads/branch-2.1 39d2fdb51 -> e6509c245 [SPARK-18183][SPARK-18184] Fix INSERT [INTO|OVERWRITE] TABLE ... PARTITION for Datasource tables There are a couple issues with the current 2.1 behavior when inserting into Datasource tables with partitions

spark git commit: [SPARK-18183][SPARK-18184] Fix INSERT [INTO|OVERWRITE] TABLE ... PARTITION for Datasource tables

Repository: spark Updated Branches: refs/heads/master 620da3b48 -> abefe2ec4 [SPARK-18183][SPARK-18184] Fix INSERT [INTO|OVERWRITE] TABLE ... PARTITION for Datasource tables ## What changes were proposed in this pull request? There are a couple issues with the current 2.1 behavior when

spark git commit: [SPARK-17475][STREAMING] Delete CRC files if the filesystem doesn't use checksum files

Repository: spark Updated Branches: refs/heads/branch-2.1 1bbf9ff63 -> 39d2fdb51 [SPARK-17475][STREAMING] Delete CRC files if the filesystem doesn't use checksum files ## What changes were proposed in this pull request? When the metadata logs for various parts of Structured Streaming are

spark git commit: [SPARK-17475][STREAMING] Delete CRC files if the filesystem doesn't use checksum files

Repository: spark Updated Branches: refs/heads/master 1bbf9ff63 -> 620da3b48 [SPARK-17475][STREAMING] Delete CRC files if the filesystem doesn't use checksum files ## What changes were proposed in this pull request? When the metadata logs for various parts of Structured Streaming are stored