Github user zheh12 closed the pull request at:
https://github.com/apache/spark/pull/21554
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h...@spark.apache.org
Github user zheh12 commented on the issue:
https://github.com/apache/spark/pull/21554
I know this sql standard.
But I wonder If I use `query.schema`, how it will affect the logic of
by-position.
I think we should let datasource implement has the ability to decide use
Github user zheh12 commented on the issue:
https://github.com/apache/spark/pull/21554
cc @cloud-fan @jiangxb1987, please give me some advise.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
GitHub user zheh12 opened a pull request:
https://github.com/apache/spark/pull/21554
[SPARK-24546] InsertIntoDataSourceCommand make data frame with wrong schema
when use kudu.
## What changes were proposed in this pull request?
I have a hdfs table
Github user zheh12 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21257#discussion_r189422931
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala
---
@@ -163,6 +170,15 @@ class HadoopMapReduceCommitProtocol
Github user zheh12 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21257#discussion_r188211717
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/FileCommitProtocol.scala ---
@@ -120,7 +120,8 @@ abstract class FileCommitProtocol
Github user zheh12 commented on the issue:
https://github.com/apache/spark/pull/21286
I think I may not have described this issue clearly.
First of all,the scene of the problem is this.
When multiple applications simultaneously append data to the same parquet
Github user zheh12 commented on the issue:
https://github.com/apache/spark/pull/21286
Thanks @cloud-fan @steveloughran for your reply, I will look more detail on
this problem.
---
-
To unsubscribe, e-mail: reviews
Github user zheh12 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21257#discussion_r187960677
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/FileCommitProtocol.scala ---
@@ -120,7 +120,8 @@ abstract class FileCommitProtocol
Github user zheh12 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21257#discussion_r187959698
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala
---
@@ -163,6 +169,12 @@ class HadoopMapReduceCommitProtocol
Github user zheh12 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21257#discussion_r187930560
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala
---
@@ -163,6 +169,12 @@ class HadoopMapReduceCommitProtocol
Github user zheh12 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21257#discussion_r187929156
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/command/ddl.scala ---
@@ -898,4 +898,12 @@ object DDLUtils {
"Cannot over
Github user zheh12 commented on the issue:
https://github.com/apache/spark/pull/21286
I think the Hadoop design does not allow two jobs to share the same output
folder.
Hadoop has a related patch that can partially solve this problem. You can
configure the parameters
Github user zheh12 commented on the issue:
https://github.com/apache/spark/pull/21286
relates to #21257
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user zheh12 commented on the issue:
https://github.com/apache/spark/pull/21286
cc @cloud-fan @jiangxb1987
Is there some drawbacks for this idea? Please give some advice when you
have time
GitHub user zheh12 opened a pull request:
https://github.com/apache/spark/pull/21286
[SPARK-24194] HadoopFsRelation cannot overwrite a path that is also bâ¦
## What changes were proposed in this pull request?
When there are multiple tasks at the same time append
Github user zheh12 commented on the issue:
https://github.com/apache/spark/pull/21257
cc @cloud-fan, Jenkins has some error, please help me retest, thanks
---
-
To unsubscribe, e-mail: reviews-unsubscr
Github user zheh12 commented on the issue:
https://github.com/apache/spark/pull/21257
Jenkins, retest this please.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail
Github user zheh12 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21257#discussion_r186936032
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala
---
@@ -207,9 +207,25 @@ case class
Github user zheh12 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21257#discussion_r186915569
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala
---
@@ -207,9 +207,25 @@ case class
Github user zheh12 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21257#discussion_r186719888
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala
---
@@ -207,9 +207,25 @@ case class
Github user zheh12 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21257#discussion_r186609102
--- Diff:
core/src/main/scala/org/apache/spark/internal/io/HadoopMapReduceCommitProtocol.scala
---
@@ -235,4 +247,20 @@ class HadoopMapReduceCommitProtocol
Github user zheh12 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21257#discussion_r186608143
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala
---
@@ -207,9 +207,25 @@ case class
Github user zheh12 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21257#discussion_r186603145
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala
---
@@ -207,9 +207,25 @@ case class
Github user zheh12 commented on a diff in the pull request:
https://github.com/apache/spark/pull/21257#discussion_r186599760
--- Diff:
sql/core/src/main/scala/org/apache/spark/sql/execution/datasources/InsertIntoHadoopFsRelationCommand.scala
---
@@ -207,9 +207,25 @@ case class
Github user zheh12 commented on the issue:
https://github.com/apache/spark/pull/21257
cc @rxin @cloud-fan
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews
GitHub user zheh12 opened a pull request:
https://github.com/apache/spark/pull/21257
[SPARK-24194] HadoopFsRelation cannot overwrite a path that is also bâ¦
## What changes were proposed in this pull request?
When insert overwrite in a parquet table. There will be a error
27 matches
Mail list logo
