shunping commented on issue #36222:
URL: https://github.com/apache/beam/issues/36222#issuecomment-3315188018
I did a quick runtime comparison between Prism and FnAPI Runner using a very
simple workflow.
```python
import logging
import apache_beam as beam
from apache_beam.options.pipeline_options import PipelineOptions
from apache_beam.transforms import trigger
from apache_beam.transforms import window
logging.basicConfig(level=logging.INFO)
# fnapi works
options = PipelineOptions([
"--streaming",
"--runner=FnAPIRunner",
])
# # prism runner option
options = PipelineOptions([
"--streaming",
"--environment_type=LOOPBACK",
"--runner=PrismRunner",
#"--prism_beam_version_override=v2.68.0rc2",
#"--runner=PortableRunner", "--job_endpoint=localhost:8073",
#"--experiments=prism_enable_rtc",
"--prism_log_level=info",
])
with beam.Pipeline(options=options) as p:
result = (
p | beam.Create([1, 2])
| beam.WindowInto(
window.GlobalWindows(),
trigger=trigger.Repeatedly(trigger.AfterCount(1)),
accumulation_mode=trigger.AccumulationMode.ACCUMULATING,
# accumulation_mode=trigger.AccumulationMode.DISCARDING,
allowed_lateness=0,
)
| beam.WithKeys(0)
| beam.GroupByKey()
| beam.Values()
| beam.LogElements(
level=logging.WARNING,
with_timestamp=True,
with_window=True,
with_pane_info=True,
use_epoch_time=True))
```
- FnAPI Runner:
- `0.85s user 0.14s system 98% cpu 0.999 total`
- `0.86s user 0.14s system 98% cpu 1.013 total`
- `1.08s user 0.18s system 99% cpu 1.275 total`
- Prism Runner with cold start. The runtime variates a bit.
- `2.35s user 1.44s system 76% cpu 4.974 total`
- `1.90s user 1.84s system 87% cpu 4.266 total`
- `3.27s user 2.01s system 87% cpu 6.009 total`
- Prism Runner with started runner (Portable Runner mode).
- `1.38s user 0.24s system 96% cpu 1.669 total`
- `1.36s user 0.23s system 97% cpu 1.646 total`
- `1.38s user 0.24s system 97% cpu 1.669 total`
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
