chamikaramj opened a new issue, #29424: URL: https://github.com/apache/beam/issues/29424
### What happened? I recently upgraded to Mac OS Sonoma and seems like x-lang jobs are getting stuck during job submission when run with this OS. The job seems to be getting stuck when connecting to Java expansion service using gRPC (see below for the stack-trace). I tried following scenarios: * Running job with Beam HEAD on Mac OS Sonoma - job gets stuck * Running job with Beam 2.51.0 and dependencies constrained to ```https://github.com/apache/beam/blob/release-2.51.0/sdks/python/container/py38/base_image_requirements.txt``` on Mac OS Sonoma - job gets stuck * Running job with Beam HEAD on Linux (Debian) - job runs fine ``` INFO:apache_beam.utils.subprocess_server:INFO: Registering external transforms: [beam:transform:org.apache.beam:kafka_read_with_metadata:v1, beam:transform:org.apache.beam:kafka_read_without_metadata:v1, beam:transform:org.apache.beam:kafka_write:v1, beam:external:java:generate_sequence:v1] INFO:apache_beam.utils.subprocess_server: INFO:apache_beam.utils.subprocess_server:Registered transforms: INFO:apache_beam.utils.subprocess_server: beam:transform:org.apache.beam:kafka_read_with_metadata:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@71075444 INFO:apache_beam.utils.subprocess_server: beam:transform:org.apache.beam:kafka_read_without_metadata:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@4f32a3ad INFO:apache_beam.utils.subprocess_server: beam:transform:org.apache.beam:kafka_write:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@6b695b06 INFO:apache_beam.utils.subprocess_server: beam:external:java:generate_sequence:v1: org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader$1@4d1bf319 INFO:apache_beam.utils.subprocess_server: INFO:apache_beam.utils.subprocess_server:Registered SchemaTransformProviders: INFO:apache_beam.utils.subprocess_server: beam:schematransform:org.apache.beam:kafka_read:v1 INFO:apache_beam.utils.subprocess_server: beam:schematransform:org.apache.beam:kafka_write:v1 INFO:apache_beam.utils.subprocess_server:Nov 13, 2023 4:23:36 PM org.apache.beam.sdk.expansion.service.ExpansionService expand INFO:apache_beam.utils.subprocess_server:INFO: Expanding 'WriteToKafka(beam:transform:org.apache.beam:kafka_write:v1)' with URN 'beam:transform:org.apache.beam:kafka_write:v1' INFO:apache_beam.utils.subprocess_server:Dependencies list: {} INFO:apache_beam.utils.subprocess_server:Nov 13, 2023 4:23:36 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.io.kafka.KafkaIO$Write$External$Configuration' has no schema registered. Attempting to construct with setter approach. INFO:apache_beam.utils.subprocess_server:Nov 13, 2023 4:23:36 PM org.apache.beam.sdk.expansion.service.ExpansionService$ExternalTransformRegistrarLoader payloadToConfig INFO:apache_beam.utils.subprocess_server:WARNING: Configuration class 'org.apache.beam.sdk.io.kafka.KafkaIO$Write$External$Configuration' has no schema registered. Attempting to construct with setter approach. /Users/chamikara/testing/xlang_job_stuck/env_2_51_0_with_constraints/lib/python3.8/site-packages/apache_beam/transforms/external.py:725: BeamDeprecationWarning: options is deprecated since First stable release. References to <pipeline>.options will not be supported self._expansion_service, pipeline.options) ``` ### Issue Priority Priority: 2 (default / most bugs should be filed as P2) ### Issue Components - [ ] Component: Python SDK - [ ] Component: Java SDK - [ ] Component: Go SDK - [ ] Component: Typescript SDK - [ ] Component: IO connector - [ ] Component: Beam YAML - [ ] Component: Beam examples - [ ] Component: Beam playground - [ ] Component: Beam katas - [ ] Component: Website - [ ] Component: Spark Runner - [ ] Component: Flink Runner - [ ] Component: Samza Runner - [ ] Component: Twister2 Runner - [ ] Component: Hazelcast Jet Runner - [ ] Component: Google Cloud Dataflow Runner -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: [email protected] For queries about this service, please contact Infrastructure at: [email protected]
