[GitHub] spark pull request #20832: [SPARK-20536][SQL] Extend ColumnName to create St...

[efimpoberezkin]

Github user efimpoberezkin closed the pull request at:

https://github.com/apache/spark/pull/20832


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20832: [SPARK-20536][SQL] Extend ColumnName to create St...

[gatorsmile]

Github user gatorsmile commented on a diff in the pull request:

https://github.com/apache/spark/pull/20832#discussion_r223164144
  
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Column.scala ---
@@ -1208,85 +1208,172 @@ class ColumnName(name: String) extends 
Column(name) {
*/
   def boolean: StructField = StructField(name, BooleanType)
 
+  /**
+   * Creates a new `StructField` of type boolean.
+   * @since 2.4.0
+   */
+  def boolean(nullable: Boolean): StructField = StructField(name, 
BooleanType, nullable)
--- End diff --

The NULL hints are not enforced. Thus, it is kind of risky to expose this 
to end users since it could generate a wrong result. We plan to ignore the 
user-specified NULL hints in the upcoming release. 


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20832: [SPARK-20536][SQL] Extend ColumnName to create St...

[efimpoberezkin]

Github user efimpoberezkin commented on a diff in the pull request:

https://github.com/apache/spark/pull/20832#discussion_r174706413
  
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Column.scala ---
@@ -1208,85 +1208,172 @@ class ColumnName(name: String) extends 
Column(name) {
*/
   def boolean: StructField = StructField(name, BooleanType)
 
+  /**
+   * Creates a new `StructField` of type boolean.
+   * @since 2.3.0
--- End diff --

Done


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20832: [SPARK-20536][SQL] Extend ColumnName to create St...

[jaceklaskowski]

Github user jaceklaskowski commented on a diff in the pull request:

https://github.com/apache/spark/pull/20832#discussion_r174700327
  
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Column.scala ---
@@ -1208,85 +1208,172 @@ class ColumnName(name: String) extends 
Column(name) {
*/
   def boolean: StructField = StructField(name, BooleanType)
 
+  /**
+   * Creates a new `StructField` of type boolean.
+   * @since 2.3.0
+   */
+  def boolean(nullable: Boolean): StructField = StructField(name, 
BooleanType, nullable)
+
   /**
* Creates a new `StructField` of type byte.
* @since 1.3.0
*/
   def byte: StructField = StructField(name, ByteType)
 
+  /**
+   * Creates a new `StructField` of type byte.
+   * @since 2.3.0
--- End diff --

`2.4.0` and in the other places too (unless they patch 2.3.0 and becomes 
2.3.1).


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20832: [SPARK-20536][SQL] Extend ColumnName to create St...

[jaceklaskowski]

Github user jaceklaskowski commented on a diff in the pull request:

https://github.com/apache/spark/pull/20832#discussion_r174699743
  
--- Diff: sql/core/src/main/scala/org/apache/spark/sql/Column.scala ---
@@ -1208,85 +1208,172 @@ class ColumnName(name: String) extends 
Column(name) {
*/
   def boolean: StructField = StructField(name, BooleanType)
 
+  /**
+   * Creates a new `StructField` of type boolean.
+   * @since 2.3.0
--- End diff --

`2.4.0` I think


---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org

[GitHub] spark pull request #20832: [SPARK-20536][SQL] Extend ColumnName to create St...

[efimpoberezkin]

GitHub user efimpoberezkin opened a pull request:

https://github.com/apache/spark/pull/20832

[SPARK-20536][SQL] Extend ColumnName to create StructFields with explicit 
nullable

## What changes were proposed in this pull request?

Extended ColumnName with methods that create StructFields with explicit 
nullable property

## How was this patch tested?

Existing tests


You can merge this pull request into a Git repository by running:

$ git pull https://github.com/efimpoberezkin/spark pr/extend-ColumnName

Alternatively you can review and apply these changes as the patch at:

https://github.com/apache/spark/pull/20832.patch

To close this pull request, make a commit to your master/trunk branch
with (at least) the following in the commit message:

This closes #20832


commit 101f3451e07d64cb650699e3b49b1c35fc7cce20
Author: Efim Poberezkin 
Date:   2018-03-14T10:22:59Z

[SPARK-20536][SQL] Extend ColumnName to create StructFields with explicit 
nullable




---

-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org