Github user xysun commented on the issue:
https://github.com/apache/spark/pull/20649
Latest code review fixes pushed. cc @HyukjinKwon
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user xysun commented on a diff in the pull request:
https://github.com/apache/spark/pull/20649#discussion_r173682838
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala ---
@@ -271,7 +271,9 @@ case class StructType(fields: Array[StructField
Github user xysun commented on the issue:
https://github.com/apache/spark/pull/20649
hey @HyukjinKwon @gatorsmile @mgaido91 anything else you'd like to
comment/change? else are we good to merge?
Thanks
Github user xysun commented on the issue:
https://github.com/apache/spark/pull/20649
:( can anyone help issue the retest command? @mgaido91 @gatorsmile thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user xysun commented on a diff in the pull request:
https://github.com/apache/spark/pull/20649#discussion_r172104069
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala ---
@@ -284,7 +286,8 @@ case class StructType(fields: Array[StructField
Github user xysun commented on a diff in the pull request:
https://github.com/apache/spark/pull/20649#discussion_r172084508
--- Diff:
sql/catalyst/src/main/scala/org/apache/spark/sql/types/StructType.scala ---
@@ -284,7 +286,8 @@ case class StructType(fields: Array[StructField
Github user xysun commented on the issue:
https://github.com/apache/spark/pull/20649
hey @gatorsmile @kiszk @jiangxb1987 @HyukjinKwon any more comments please?
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user xysun commented on a diff in the pull request:
https://github.com/apache/spark/pull/20649#discussion_r171182128
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/StructTypeSuite.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user xysun commented on a diff in the pull request:
https://github.com/apache/spark/pull/20649#discussion_r171181994
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/StructTypeSuite.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user xysun commented on a diff in the pull request:
https://github.com/apache/spark/pull/20649#discussion_r171182024
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/StructTypeSuite.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user xysun commented on the issue:
https://github.com/apache/spark/pull/20649
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user xysun commented on the issue:
https://github.com/apache/spark/pull/20649
I'm getting some weird compile errors on local even when based off previous
successful commit.. checking..
---
-
To unsubscribe
Github user xysun commented on a diff in the pull request:
https://github.com/apache/spark/pull/20649#discussion_r169850843
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/StructTypeSuite.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user xysun commented on a diff in the pull request:
https://github.com/apache/spark/pull/20649#discussion_r169850853
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/StructTypeSuite.scala ---
@@ -0,0 +1,43 @@
+/*
+ * Licensed to the Apache Software Foundation
Github user xysun commented on the issue:
https://github.com/apache/spark/pull/20649
@gatorsmile please review. Thanks!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e
GitHub user xysun opened a pull request:
https://github.com/apache/spark/pull/20649
[SPARK-23452][SQL] improve missing field error message in `StructType`
## What changes were proposed in this pull request?
The error message ```s"""Field "$
Github user xysun commented on the issue:
https://github.com/apache/spark/pull/19247
retest this please
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user xysun commented on the issue:
https://github.com/apache/spark/pull/19247
Hi @zsxwing I have pushed latest changes (for file sink I'll be honest I
simply copied your code =p)
I also verified that both tests would fail without the fix.
Please review. Thanks
Github user xysun commented on a diff in the pull request:
https://github.com/apache/spark/pull/19247#discussion_r161932613
--- Diff:
sql/core/src/test/scala/org/apache/spark/sql/streaming/FileStreamSourceSuite.scala
---
@@ -86,6 +86,18 @@ abstract class FileStreamSourceTest
Github user xysun commented on the issue:
https://github.com/apache/spark/pull/19247
hi @HyukjinKwon @zsxwing @mgaido91 i have updated code according to the
comments, also merged with latest master. Please review. Thanks
Github user xysun commented on a diff in the pull request:
https://github.com/apache/spark/pull/19247#discussion_r161430294
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/streaming/FileStreamSource.scala
---
@@ -233,7 +233,7 @@ class FileStreamSource
Github user xysun commented on the issue:
https://github.com/apache/spark/pull/19247
@joseph-torres @brkyvz @lw-lin can you please take a look? (sorry for
uninvited mentions but i just took the latest commits on `FileStreamSource
Github user xysun commented on the issue:
https://github.com/apache/spark/pull/19247
To handle file names with special characters, we should use `URI.getPath`
to get decoded path, instead of using `toString`, which may contain other
characters different from original path
GitHub user xysun opened a pull request:
https://github.com/apache/spark/pull/19247
[Spark-21996][SQL] read files with space in name for streaming
## What changes were proposed in this pull request?
Structured streaming is now able to read files with space in file name
24 matches
Mail list logo
