GitHub user SongYadong opened a pull request:
https://github.com/apache/spark/pull/22706
[SPARK-25716][SQL][MINOR] remove unnecessary collection operation in valid
constraints generation
## What changes were proposed in this pull request?
Project logical operator generates
Github user SongYadong commented on the issue:
https://github.com/apache/spark/pull/22348
Sounds reasonable. I will close this PR. Thank you!
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user SongYadong closed the pull request at:
https://github.com/apache/spark/pull/22348
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
Github user SongYadong commented on the issue:
https://github.com/apache/spark/pull/22348
@dongjoon-hyun . You are right, DataSourceReadBenchmark result show the
benefit is too small even in some cases is covered up by fluctuation.
Java HotSpot(TM) 64-Bit Server VM
Github user SongYadong commented on the issue:
https://github.com/apache/spark/pull/22348
Could I ask you to review this PR if you have the time? cc @dongjoon-hyun
@cloud-fan
---
-
To unsubscribe, e-mail: reviews
Github user SongYadong commented on a diff in the pull request:
https://github.com/apache/spark/pull/22348#discussion_r215870108
--- Diff:
sql/core/src/main/java/org/apache/spark/sql/execution/datasources/parquet/VectorizedParquetRecordReader.java
---
@@ -154,8 +159,6 @@ public
Github user SongYadong commented on the issue:
https://github.com/apache/spark/pull/22348
In my test, total executing time of 4885 spark applications:
origin: 447213 seconds
after : 444584 seconds
time saved : about 0.6
GitHub user SongYadong opened a pull request:
https://github.com/apache/spark/pull/22348
Reduce unneeded operation in nextKeyValue process of parquet vectorized
record reader
## What changes were proposed in this pull request?
this PR do following
Github user SongYadong commented on the issue:
https://github.com/apache/spark/pull/22254
OK, I will close it. thanks for your review.
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
Github user SongYadong closed the pull request at:
https://github.com/apache/spark/pull/22254
---
-
To unsubscribe, e-mail: reviews-unsubscr...@spark.apache.org
For additional commands, e-mail: reviews-h
GitHub user SongYadong opened a pull request:
https://github.com/apache/spark/pull/22254
[CATALYST] Add a call to apply method explicitly in combineFilterFunction
## What changes were proposed in this pull request?
This PR do:
add an apply method explicitly
Github user SongYadong commented on the issue:
https://github.com/apache/spark/pull/21836
I'm sorry for the scala style checks failure, i will read scala style guide
doc again.
---
-
To unsubscribe, e-mail: reviews
Github user SongYadong commented on a diff in the pull request:
https://github.com/apache/spark/pull/21836#discussion_r204290930
--- Diff:
sql/hive/src/main/scala/org/apache/spark/sql/hive/TableReader.scala ---
@@ -111,7 +111,8 @@ class HadoopTableReader(
filterOpt
GitHub user SongYadong opened a pull request:
https://github.com/apache/spark/pull/21836
[SQL][HIVE] Correct an assert message in function makeRDDForTable
## What changes were proposed in this pull request?
according to the context, "makeRDDForTablePartitions" in asse
14 matches
Mail list logo
