Re: [PR] [FLINK-34132][runtime] Correct the error message and doc of AdaptiveBatch only supports all edges being BLOCKING or HYBRID_FULL/HYBRID_SELECTIVE. [flink]
zhuzhurk closed pull request #24118: [FLINK-34132][runtime] Correct the error message and doc of AdaptiveBatch only supports all edges being BLOCKING or HYBRID_FULL/HYBRID_SELECTIVE. URL: https://github.com/apache/flink/pull/24118 -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34132][runtime] Correct the error message and doc of AdaptiveBatch only supports all edges being BLOCKING or HYBRID_FULL/HYBRID_SELECTIVE. [flink]
zhuzhurk commented on code in PR #24118:
URL: https://github.com/apache/flink/pull/24118#discussion_r1472843285
##
docs/content/docs/deployment/elastic_scaling.md:
##
@@ -238,7 +238,7 @@ In addition, there are several related configuration
options that may need adjus
### Limitations
- **Batch jobs only**: Adaptive Batch Scheduler only supports batch jobs.
Exception will be thrown if a streaming job is submitted.
-- **BLOCKING or HYBRID jobs only**: At the moment, Adaptive Batch Scheduler
only supports jobs whose [shuffle mode]({{< ref "docs/deployment/config"
>}}#execution-batch-shuffle-mode) is `ALL_EXCHANGES_BLOCKING /
ALL_EXCHANGES_HYBRID_FULL / ALL_EXCHANGES_HYBRID_SELECTIVE`.
+- **BLOCKING or HYBRID jobs only**: At the moment, Adaptive Batch Scheduler
only supports DataStream jobs whose [shuffle mode]({{< ref
"docs/deployment/config" >}}#execution-batch-shuffle-mode) is
`ALL_EXCHANGES_BLOCKING / ALL_EXCHANGES_HYBRID_FULL /
ALL_EXCHANGES_HYBRID_SELECTIVE` and DataSet jobs whose `ExecutionMode` is
`BATCH_FORCED`.
Review Comment:
Maybe keep the original statement and add an extra statement like "Note that
for DataSet jobs which do not recognize the aforementioned shuffle mode, the
`ExecutionMode` needs to be `BATCH_FORCED` to force BLOCKING shuffle".
Later in Flink 2.0, we can directly remove the extra statement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [FLINK-34132][runtime] Correct the error message and doc of AdaptiveBatch only supports all edges being BLOCKING or HYBRID_FULL/HYBRID_SELECTIVE. [flink]
zhuzhurk commented on code in PR #24118:
URL: https://github.com/apache/flink/pull/24118#discussion_r1472843285
##
docs/content/docs/deployment/elastic_scaling.md:
##
@@ -238,7 +238,7 @@ In addition, there are several related configuration
options that may need adjus
### Limitations
- **Batch jobs only**: Adaptive Batch Scheduler only supports batch jobs.
Exception will be thrown if a streaming job is submitted.
-- **BLOCKING or HYBRID jobs only**: At the moment, Adaptive Batch Scheduler
only supports jobs whose [shuffle mode]({{< ref "docs/deployment/config"
>}}#execution-batch-shuffle-mode) is `ALL_EXCHANGES_BLOCKING /
ALL_EXCHANGES_HYBRID_FULL / ALL_EXCHANGES_HYBRID_SELECTIVE`.
+- **BLOCKING or HYBRID jobs only**: At the moment, Adaptive Batch Scheduler
only supports DataStream jobs whose [shuffle mode]({{< ref
"docs/deployment/config" >}}#execution-batch-shuffle-mode) is
`ALL_EXCHANGES_BLOCKING / ALL_EXCHANGES_HYBRID_FULL /
ALL_EXCHANGES_HYBRID_SELECTIVE` and DataSet jobs whose `ExecutionMode` is
`BATCH_FORCED`.
Review Comment:
Maybe keep the original statement and add an extra statement like "Note that
for DataSet jobs, the `ExecutionMode` needs to be `BATCH_FORCED` since they do
not use the aforementioned shuffle mode".
Later in Flink 2.0, we can directly remove the extra statement.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [FLINK-34132][runtime] Correct the error message and doc of AdaptiveBatch only supports all edges being BLOCKING or HYBRID_FULL/HYBRID_SELECTIVE. [flink]
wanglijie95 commented on code in PR #24118:
URL: https://github.com/apache/flink/pull/24118#discussion_r1464226470
##
docs/content/docs/deployment/elastic_scaling.md:
##
@@ -238,7 +238,7 @@ In addition, there are several related configuration
options that may need adjus
### Limitations
- **Batch jobs only**: Adaptive Batch Scheduler only supports batch jobs.
Exception will be thrown if a streaming job is submitted.
-- **BLOCKING or HYBRID jobs only**: At the moment, Adaptive Batch Scheduler
only supports jobs whose [shuffle mode]({{< ref "docs/deployment/config"
>}}#execution-batch-shuffle-mode) is `ALL_EXCHANGES_BLOCKING /
ALL_EXCHANGES_HYBRID_FULL / ALL_EXCHANGES_HYBRID_SELECTIVE`.
+- **BLOCKING or HYBRID jobs only**: At the moment, Adaptive Batch Scheduler
only supports DataStream jobs whose [shuffle mode]({{< ref
"docs/deployment/config" >}}#execution-batch-shuffle-mode) is
`ALL_EXCHANGES_BLOCKING / ALL_EXCHANGES_HYBRID_FULL /
ALL_EXCHANGES_HYBRID_SELECTIVE` and DataSet jobs whose `ExecutionMode` is
`BATCH_FORCED`.
Review Comment:
I personlly think that `DataStream API` does not include `SQL/Table API`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [FLINK-34132][runtime] Correct the error message and doc of AdaptiveBatch only supports all edges being BLOCKING or HYBRID_FULL/HYBRID_SELECTIVE. [flink]
JunRuiLee commented on code in PR #24118:
URL: https://github.com/apache/flink/pull/24118#discussion_r1464218412
##
docs/content/docs/deployment/elastic_scaling.md:
##
@@ -238,7 +238,7 @@ In addition, there are several related configuration
options that may need adjus
### Limitations
- **Batch jobs only**: Adaptive Batch Scheduler only supports batch jobs.
Exception will be thrown if a streaming job is submitted.
-- **BLOCKING or HYBRID jobs only**: At the moment, Adaptive Batch Scheduler
only supports jobs whose [shuffle mode]({{< ref "docs/deployment/config"
>}}#execution-batch-shuffle-mode) is `ALL_EXCHANGES_BLOCKING /
ALL_EXCHANGES_HYBRID_FULL / ALL_EXCHANGES_HYBRID_SELECTIVE`.
+- **BLOCKING or HYBRID jobs only**: At the moment, Adaptive Batch Scheduler
only supports DataStream jobs whose [shuffle mode]({{< ref
"docs/deployment/config" >}}#execution-batch-shuffle-mode) is
`ALL_EXCHANGES_BLOCKING / ALL_EXCHANGES_HYBRID_FULL /
ALL_EXCHANGES_HYBRID_SELECTIVE` and DataSet jobs whose `ExecutionMode` is
`BATCH_FORCED`.
Review Comment:
How about `DataStream API` and `DataSet API`, which are used in many FLIP
content ?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [FLINK-34132][runtime] Correct the error message and doc of AdaptiveBatch only supports all edges being BLOCKING or HYBRID_FULL/HYBRID_SELECTIVE. [flink]
wanglijie95 commented on code in PR #24118:
URL: https://github.com/apache/flink/pull/24118#discussion_r1464214787
##
docs/content/docs/deployment/elastic_scaling.md:
##
@@ -238,7 +238,7 @@ In addition, there are several related configuration
options that may need adjus
### Limitations
- **Batch jobs only**: Adaptive Batch Scheduler only supports batch jobs.
Exception will be thrown if a streaming job is submitted.
-- **BLOCKING or HYBRID jobs only**: At the moment, Adaptive Batch Scheduler
only supports jobs whose [shuffle mode]({{< ref "docs/deployment/config"
>}}#execution-batch-shuffle-mode) is `ALL_EXCHANGES_BLOCKING /
ALL_EXCHANGES_HYBRID_FULL / ALL_EXCHANGES_HYBRID_SELECTIVE`.
+- **BLOCKING or HYBRID jobs only**: At the moment, Adaptive Batch Scheduler
only supports DataStream jobs whose [shuffle mode]({{< ref
"docs/deployment/config" >}}#execution-batch-shuffle-mode) is
`ALL_EXCHANGES_BLOCKING / ALL_EXCHANGES_HYBRID_FULL /
ALL_EXCHANGES_HYBRID_SELECTIVE` and DataSet jobs whose `ExecutionMode` is
`BATCH_FORCED`.
Review Comment:
I think we can not directly use `DataStream jobs` here, because it does not
include SQL/Table jobs.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [FLINK-34132][runtime] Correct the error message and doc of AdaptiveBatch only supports all edges being BLOCKING or HYBRID_FULL/HYBRID_SELECTIVE. [flink]
JunRuiLee commented on PR #24118: URL: https://github.com/apache/flink/pull/24118#issuecomment-1899838542 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34132][runtime] Correct the error message and doc of AdaptiveBatch only supports all edges being BLOCKING or HYBRID_FULL/HYBRID_SELECTIVE. [flink]
JunRuiLee commented on PR #24118: URL: https://github.com/apache/flink/pull/24118#issuecomment-1897703554 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34132][runtime] Correct the error message and doc of AdaptiveBatch only supports all edges being BLOCKING or HYBRID_FULL/HYBRID_SELECTIVE. [flink]
JunRuiLee commented on code in PR #24118:
URL: https://github.com/apache/flink/pull/24118#discussion_r1456765220
##
docs/content/docs/deployment/elastic_scaling.md:
##
@@ -238,7 +238,7 @@ In addition, there are several related configuration
options that may need adjus
### Limitations
- **Batch jobs only**: Adaptive Batch Scheduler only supports batch jobs.
Exception will be thrown if a streaming job is submitted.
-- **BLOCKING or HYBRID jobs only**: At the moment, Adaptive Batch Scheduler
only supports jobs whose [shuffle mode]({{< ref "docs/deployment/config"
>}}#execution-batch-shuffle-mode) is `ALL_EXCHANGES_BLOCKING /
ALL_EXCHANGES_HYBRID_FULL / ALL_EXCHANGES_HYBRID_SELECTIVE`.
+- **BLOCKING or HYBRID jobs only**: At the moment, Adaptive Batch Scheduler
only supports DataStream jobs whose [shuffle mode]({{< ref
"docs/deployment/config" >}}#execution-batch-shuffle-mode) is
`ALL_EXCHANGES_BLOCKING / ALL_EXCHANGES_HYBRID_FULL /
ALL_EXCHANGES_HYBRID_SELECTIVE` and DataSet jobs whose `ExecutionMode` is
`BATCH_FORCED`.
Review Comment:
I would prefer to clarify here on how to enforce the ALL BLOCKING for
DataSet jobs, considering that the usage of AdaptiveBatch with DataSet jobs has
already been detailed in the official documentation for users.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [FLINK-34132][runtime] Correct the error message and doc of AdaptiveBatch only supports all edges being BLOCKING or HYBRID_FULL/HYBRID_SELECTIVE. [flink]
wanglijie95 commented on code in PR #24118:
URL: https://github.com/apache/flink/pull/24118#discussion_r1456018455
##
docs/content/docs/deployment/elastic_scaling.md:
##
@@ -238,7 +238,7 @@ In addition, there are several related configuration
options that may need adjus
### Limitations
- **Batch jobs only**: Adaptive Batch Scheduler only supports batch jobs.
Exception will be thrown if a streaming job is submitted.
-- **BLOCKING or HYBRID jobs only**: At the moment, Adaptive Batch Scheduler
only supports jobs whose [shuffle mode]({{< ref "docs/deployment/config"
>}}#execution-batch-shuffle-mode) is `ALL_EXCHANGES_BLOCKING /
ALL_EXCHANGES_HYBRID_FULL / ALL_EXCHANGES_HYBRID_SELECTIVE`.
+- **BLOCKING or HYBRID jobs only**: At the moment, Adaptive Batch Scheduler
only supports DataStream jobs whose [shuffle mode]({{< ref
"docs/deployment/config" >}}#execution-batch-shuffle-mode) is
`ALL_EXCHANGES_BLOCKING / ALL_EXCHANGES_HYBRID_FULL /
ALL_EXCHANGES_HYBRID_SELECTIVE` and DataSet jobs whose `ExecutionMode` is
`BATCH_FORCED`.
Review Comment:
As we discussed offline, we should state that adaptive batch scheduler does
not support `DataSet` jobs here?
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org
For queries about this service, please contact Infrastructure at:
us...@infra.apache.org
Re: [PR] [FLINK-34132][runtime] Correct the error message and doc of AdaptiveBatch only supports all edges being BLOCKING or HYBRID_FULL/HYBRID_SELECTIVE. [flink]
JunRuiLee commented on PR #24118: URL: https://github.com/apache/flink/pull/24118#issuecomment-1897604825 @flinkbot run azure -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34132][runtime] Correct the error message and doc of AdaptiveBatch only supports all edges being BLOCKING or HYBRID_FULL/HYBRID_SELECTIVE. [flink]
flinkbot commented on PR #24118: URL: https://github.com/apache/flink/pull/24118#issuecomment-1895778440 ## CI report: * 8f82d8dde069faa3f74276493dc374679e4ba4a2 UNKNOWN Bot commands The @flinkbot bot supports the following commands: - `@flinkbot run azure` re-run the last Azure build -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
Re: [PR] [FLINK-34132][runtime] Correct the error message and doc of AdaptiveBatch only supports all edges being BLOCKING or HYBRID_FULL/HYBRID_SELECTIVE. [flink]
JunRuiLee commented on PR #24118: URL: https://github.com/apache/flink/pull/24118#issuecomment-1895750981 @wanglijie95 Could you help to review this change when you're free? Thanks a lot! -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
[PR] [FLINK-34132][runtime] Correct the error message and doc of AdaptiveBatch only supports all edges being BLOCKING or HYBRID_FULL/HYBRID_SELECTIVE. [flink]
JunRuiLee opened a new pull request, #24118: URL: https://github.com/apache/flink/pull/24118 ## What is the purpose of the change Correct the error message and doc of AdaptiveBatch only supports all edges being BLOCKING or HYBRID_FULL/HYBRID_SELECTIVE. ## Brief change log Correct the error message and doc of AdaptiveBatch only supports all edges being BLOCKING or HYBRID_FULL/HYBRID_SELECTIVE. ## Verifying this change This change is a trivial rework / code cleanup without any test coverage. ## Does this pull request potentially affect one of the following parts: - Dependencies (does it add or upgrade a dependency): (yes / **no**) - The public API, i.e., is any changed class annotated with `@Public(Evolving)`: (yes / **no**) - The serializers: (yes / **no** / don't know) - The runtime per-record code paths (performance sensitive): (yes / **no** / don't know) - Anything that affects deployment or recovery: JobManager (and its components), Checkpointing, Kubernetes/Yarn, ZooKeeper: (yes / **no** / don't know) - The S3 file system connector: (yes / **no** / don't know) ## Documentation - Does this pull request introduce a new feature? (yes / **no**) - If yes, how is the feature documented? (**not applicable** / docs / JavaDocs / not documented) -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: issues-unsubscr...@flink.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
