Abacn commented on issue #34747: URL: https://github.com/apache/beam/issues/34747#issuecomment-2985556592
The Python transform seems not working. PostComit Python was tested on https://github.com/apache/beam/pull/35232, but fails after https://github.com/apache/beam/pull/34763 gets merged. Error is ``` E RuntimeError: Pipeline BeamApp-yathu-0618194235-a8e76e10_e3353e19-3264-4ece-ad50-733f5e48f45f failed in state FAILED: java.lang.RuntimeException: Error received from SDK harness for instruction 2: org.apache.beam.sdk.util.UserCodeException: java.lang.RuntimeException: Error occurred when consuming changes from Database. E at org.apache.beam.sdk.util.UserCodeException.wrap(UserCodeException.java:39) E at org.apache.beam.io.debezium.KafkaSourceConsumerFn$DoFnInvoker.invokeProcessElement(Unknown Source) E at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForWindowObservingSizedElementAndRestriction(FnApiDoFnRunner.java:871) ... E Caused by: java.lang.RuntimeException: Error occurred when consuming changes from Database. E at org.apache.beam.io.debezium.KafkaSourceConsumerFn.process(KafkaSourceConsumerFn.java:278) E Caused by: org.apache.kafka.connect.errors.ConnectException: Error configuring an instance of PostgresConnectorTask; check the logs for details E at io.debezium.connector.common.BaseSourceTask.start(BaseSourceTask.java:243) E at org.apache.beam.io.debezium.KafkaSourceConsumerFn.process(KafkaSourceConsumerFn.java:247) E at org.apache.beam.io.debezium.KafkaSourceConsumerFn$DoFnInvoker.invokeProcessElement(Unknown Source) E at org.apache.beam.fn.harness.FnApiDoFnRunner.processElementForWindowObservingSizedElementAndRestriction(FnApiDoFnRunner.java:871) ``` However not sure where I can find "Error configuring an instance of PostgresConnectorTask; check the logs for details" @tkaymak is this error familiar to you? --------- To reproduce locally, here is my command Prerequisite: - Path Java is Java 11 - Has Java 21 installed somewhere (set JAVA21_HOME=...) - Has a Python 3.11 virtualenv - In Beam root (BEAM_ROOT=...) Steps 1. `./gradlew :sdks:java:io:debezium:expansion-service:shadowJar :runners:flink:1.19:job-server:shadowJar -Pjava21Home=$JAVA21_HOME -info` 2. `./gradlew :sdks:java:container:java21:docker` 3. `cd sdks/python` 4. `pip install .[gcp,test]` 5. ``` JAVA_HOME=$JAVA21_HOME ./scripts/run_integration_test.sh --test_opts \ 'apache_beam/io/external/xlang_debeziumio_it_test.py --log-cli-level=INFO' \ --suite postCommitIT-flink-py311 --pipeline_opts \ "--runner=FlinkRunner --project=apache-beam-testing --environment_type=LOOPBACK --temp_location=gs://temp-storage-for-end-to-end-tests/temp-it --flink_job_server_jar=$BEAM_ROOT/runners/flink/1.19/job-server/build/libs/beam-runners-flink-1.19-job-server-2.67.0-SNAPSHOT.jar --flink_conf_dir=$BEAM_ROOT/runners/flink/src/test/resources/" ``` -- This is an automated message from the Apache Git Service. To respond to the message, please log on to GitHub and use the URL above to go to the specific comment. To unsubscribe, e-mail: github-unsubscr...@beam.apache.org For queries about this service, please contact Infrastructure at: us...@infra.apache.org
