bvolpato opened a new pull request, #40220:
URL: https://github.com/apache/beam/pull/40220
## Summary
This draft explores **decision models** as a small, typed Beam Python
integration. The main demo uses Jev Choice to route a finite message stream to
a model-selected BigQuery table. A separate Noul fraud-review example includes
an optional Score rubric. `DecisionModel` keeps the Beam caller independent of
Jev, so a local Laya or Kev adapter can implement the same question/answer
contract without changing the pipeline.
## Architecture
```mermaid
flowchart LR
A["TestStream or Pub/Sub"] --> B["RequestResponseIO<br/>DecisionModel
caller"]
B --> C["Choice: billing / technical / sales<br/>probabilities +
confidence"]
C --> D{"confidence gate"}
D -->|accepted| E["dynamic BigQuery table<br/>or local JSONL destination"]
D -->|uncertain| F["messages_review"]
F --> E
```
The transform owns the provider client lifecycle. The model chooses from a
fixed label set; Beam code validates the label and selects the destination. The
Noul example uses its yes probability for a visible review threshold. Score is
a separate rubric question and can run beside Noul in one Jev request. The
offline rule adapter exercises the same Beam graph without network access. It
is a demo fixture, not a trained model.
The
[README](https://github.com/bvolpato/beam/blob/dd3428debcbe98a896a54421c05fbf9df4b2d3be/sdks/python/apache_beam/examples/inference/decision_models/README.md)
includes commands, limitations, and a [graph rendered from the Python
pipeline](https://github.com/bvolpato/beam/blob/dd3428debcbe98a896a54421c05fbf9df4b2d3be/sdks/python/apache_beam/examples/inference/decision_models/message_router.svg)
using Beam `RenderRunner`. The graph-only BigQuery path does not write to
BigQuery.
## Verification
- Three focused policy tests pass. YAPF, pycodestyle, compilation, and `git
diff --check` pass.
- DirectRunner local runs wrote one JSONL row each to billing, technical,
and sales destinations. The local Noul/Score demo completed.
- A short live Jev DirectRunner run wrote the same three destinations.
Caller wall times were **406.26 ms**, **249.70 ms**, and **226.34 ms** for
billing, technical, and sales respectively (`jev-1.13.0`, `n=3`, local JSONL
sink). This is a smoke observation, not a throughput or latency guarantee.
- A separate live Jev Noul+Score run flagged only the synthetic
bypass-verification message (Noul `0.97`; review threshold `0.7`).
- The dynamic BigQuery transform expands and renders with Beam's GCP extra.
A live BigQuery write and a Pub/Sub subscription run require project
credentials and were not performed.
The supplied Jev key was used only in process memory for the short live
checks. It is absent from the commit, graph, logs, and PR body. Production use
needs a worker secret path, quota policy, idempotency, and failed-row handling.
------------------------
Thank you for your contribution! Follow this checklist to help us
incorporate your contribution quickly and easily:
- [ ] Mention the appropriate issue in your description (for example:
`addresses #123`), if applicable. This will automatically add a link to the
pull request in the issue. If you would like the issue to automatically close
on merging the pull request, comment `fixes #<ISSUE NUMBER>` instead.
- [ ] Update `CHANGES.md` with noteworthy changes.
- [ ] If this contribution is large, please file an Apache [Individual
Contributor License Agreement](https://www.apache.org/licenses/icla.pdf).
This is an example-only draft without an associated issue or release note.
ICLA status is left for contributor review.
See the [Contributor Guide](https://beam.apache.org/contribute) for more
tips on [how to make review process
smoother](https://github.com/apache/beam/blob/master/CONTRIBUTING.md#make-the-reviewers-job-easier).
To check the build health, please visit
[https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md](https://github.com/apache/beam/blob/master/.test-infra/BUILD_STATUS.md)
GitHub Actions Tests Status (on master branch)
------------------------------------------------------------------------------------------------
[](https://github.com/apache/beam/actions?query=workflow%3A%22Build+python+source+distribution+and+wheels%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Python+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Java+Tests%22+branch%3Amaster+event%3Aschedule)
[](https://github.com/apache/beam/actions?query=workflow%3A%22Go+tests%22+branch%3Amaster+event%3Aschedule)
See [CI.md](https://github.com/apache/beam/blob/master/CI.md) for more
information about GitHub Actions CI or the [workflows
README](https://github.com/apache/beam/blob/master/.github/workflows/README.md)
to see a list of phrases to trigger workflows.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
