nikitagrover19 commented on issue #36201:
URL: https://github.com/apache/beam/issues/36201#issuecomment-4019731231
Hi @tvalentyn apologies for the delay , had to take some time to properly
investigate the XLang approach you suggested.
I went ahead and tested it, and the good news is it works. I also ended up
implementing `publish_with_ordering_key` through the XLang bridge while I was
at it, so we have something concrete to look at.
**What I tested:**
- Basic XLang PubSub write end-to-end ✅
- `publish_with_ordering_key=True` with `with_attributes=True` — ordering
keys preserved end-to-end ✅
- Default path (no ordering key) — no regression ✅
**Changes made:**
- `ExternalWrite.java`: added `publishWithOrderingKey` to `Configuration`,
wired to existing Java `withOrderingKey()`
- `external/gcp/pubsub.py`: added `publish_with_ordering_key=False` param
to `WriteToPubSub`
- `PubsubIO.java`: fixed a pre-existing bug where
`PubsubBoundedWriter.startBundle` ignored the transform-level `pubsubRootUrl`,
always falling back to pipeline options
- `external/gcp/pubsub.py`: also fixed a pre-existing bug where
`with_attributes=True` referenced a non-existent method and would crash for any
user
I'm happy to go with this approach instead of the Direct Runner
implementation. Just let me know if you'd like me to update the PR to drop the
native Python changes in favor of the XLang path, or if you'd prefer to keep
both.
--
This is an automated message from the Apache Git Service.
To respond to the message, please log on to GitHub and use the
URL above to go to the specific comment.
To unsubscribe, e-mail: [email protected]
For queries about this service, please contact Infrastructure at:
[email protected]
