EnricoMi commented on PR #35965:
URL: https://github.com/apache/spark/pull/35965#issuecomment-1162082945
Thanks @sunchao
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To
EnricoMi commented on PR #35965:
URL: https://github.com/apache/spark/pull/35965#issuecomment-1151468995
@sunchao @cloud-fan all comments addressed, all tests green
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
U
EnricoMi commented on PR #35965:
URL: https://github.com/apache/spark/pull/35965#issuecomment-1127488590
@sunchao @HyukjinKwon @aokolnychyi @cloud-fan I have addressed comments and
rebased.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log
EnricoMi commented on PR #35965:
URL: https://github.com/apache/spark/pull/35965#issuecomment-1120977044
@sunchao I have pushed all my changes
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the
EnricoMi commented on PR #35965:
URL: https://github.com/apache/spark/pull/35965#issuecomment-1110217254
@cloud-fan who would be best to review this?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go
EnricoMi commented on PR #35965:
URL: https://github.com/apache/spark/pull/35965#issuecomment-1102712930
@cloud-fan thanks for clarification, I have incorporated this distinction
into the documentation of `SupportsReportOrdering`.
--
This is an automated message from the Apache Git Servic
EnricoMi commented on PR #35965:
URL: https://github.com/apache/spark/pull/35965#issuecomment-1097249916
So you are saying ordering is seen as being part of partitioning. This kind
of makes sense.
Are you foreseeing that partitioning can transport in-partition order,
rather than glob
EnricoMi commented on PR #35965:
URL: https://github.com/apache/spark/pull/35965#issuecomment-1096834886
@sunchao #35657 says `"have V2 data sources to report distribution and
ordering to Spark on read path"`: How do V2 sources report their ordering?
--
This is an automated message from t
EnricoMi commented on PR #35965:
URL: https://github.com/apache/spark/pull/35965#issuecomment-1096810002
I have found the issue, option `spark.sql.sources.v2.bucketing.enabled`
moved to default `false`, so I have to enable that to have order and
partitioning being reused.
--
This is an a
EnricoMi commented on PR #35965:
URL: https://github.com/apache/spark/pull/35965#issuecomment-1096802477
@sunchao @dongjoon-hyun looks like #35657 (20ffbf7b) broke reusing
**partitioning and order** provided by DS V2 in my tests, which worked before
rebase. The query planner now introduces
10 matches
Mail list logo
