(moving this to the user list, dev list to bcc.) Adding relevant people: @John Casey <johnjca...@google.com>.
(Keshav, for Dataflow issues you could also reach out to Dataflow support: https://cloud.google.com/dataflow/docs/support) On Mon, Oct 31, 2022 at 1:23 PM Chennakeshavlu Maddela < chennakeshavlu.madd...@davita.com> wrote: > Hi Team, > > > > We are setting up avro write to a kafka topic with confluent schema > registry on SSL, its throwing (below) error. > > > > We are using SASL_SSL with PEM certificate for connecting Kafka broker, > which is working fine with non-avro kafka topics. Can you please help us > with configuring SSL for schema registry? (we are using dataflow runner) > > > > Please let me know if you need more details. > > > > Thank you, > > Keshav > > > > *Exception:* > > Failed to send HTTP request to endpoint: > https://confluent-schemaregistry-xxxx.com/subjects/topic-value?deleted=false > > > > javax.net.ssl.SSLHandshakeException: PKIX path building failed: > sun.security.provider.certpath.SunCertPathBuilderException: unable to find > valid certification path to requested target > > at > java.base/sun.security.ssl.Alert.createSSLException(Alert.java:131) > > at > java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:350) > > at > java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:293) > > at > java.base/sun.security.ssl.TransportContext.fatal(TransportContext.java:288) > > at > java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1356) > > at > java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.onConsumeCertificate(CertificateMessage.java:1231) > > at > java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.consume(CertificateMessage.java:1174) > > at > java.base/sun.security.ssl.SSLHandshake.consume(SSLHandshake.java:392) > > at > java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:444) > > at > java.base/sun.security.ssl.HandshakeContext.dispatch(HandshakeContext.java:422) > > at > java.base/sun.security.ssl.TransportContext.dispatch(TransportContext.java:183) > > at > java.base/sun.security.ssl.SSLTransport.decode(SSLTransport.java:171) > > at > java.base/sun.security.ssl.SSLSocketImpl.decode(SSLSocketImpl.java:1408) > > at > java.base/sun.security.ssl.SSLSocketImpl.readHandshakeRecord(SSLSocketImpl.java:1314) > > at > java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:440) > > at > java.base/sun.security.ssl.SSLSocketImpl.startHandshake(SSLSocketImpl.java:411) > > at > java.base/sun.net.www.protocol.https.HttpsClient.afterConnect(HttpsClient.java:567) > > at > java.base/sun.net.www.protocol.https.AbstractDelegateHttpsURLConnection.connect(AbstractDelegateHttpsURLConnection.java:185) > > at > java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream0(HttpURLConnection.java:1367) > > at > java.base/sun.net.www.protocol.http.HttpURLConnection.getOutputStream(HttpURLConnection.java:1342) > > at > java.base/sun.net.www.protocol.https.HttpsURLConnectionImpl.getOutputStream(HttpsURLConnectionImpl.java:246) > > at > io.confluent.kafka.schemaregistry.client.rest.RestService.sendHttpRequest(RestService.java:199) > > at > io.confluent.kafka.schemaregistry.client.rest.RestService.httpRequest(RestService.java:256) > > at > io.confluent.kafka.schemaregistry.client.rest.RestService.lookUpSubjectVersion(RestService.java:323) > > at > io.confluent.kafka.schemaregistry.client.rest.RestService.lookUpSubjectVersion(RestService.java:311) > > at > io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.getIdFromRegistry(CachedSchemaRegistryClient.java:191) > > at > io.confluent.kafka.schemaregistry.client.CachedSchemaRegistryClient.getId(CachedSchemaRegistryClient.java:323) > > at > io.confluent.kafka.serializers.AbstractKafkaAvroSerializer.serializeImpl(AbstractKafkaAvroSerializer.java:73) > > at > io.confluent.kafka.serializers.KafkaAvroSerializer.serialize(KafkaAvroSerializer.java:53) > > at > org.apache.kafka.common.serialization.Serializer.serialize(Serializer.java:62) > > at > org.apache.kafka.clients.producer.KafkaProducer.doSend(KafkaProducer.java:952) > > at > org.apache.kafka.clients.producer.KafkaProducer.send(KafkaProducer.java:912) > > at > com.davita.cwow.pmt.transformations.PmtAvroKafkaWriter$KafkaWriteEvaluationFn.processElement(PmtAvroKafkaWriter.java:86) > > at > com.davita.cwow.pmt.transformations.PmtAvroKafkaWriter$KafkaWriteEvaluationFn$DoFnInvoker.invokeProcessElement(Unknown > Source) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:211) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188) > > at > org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:340) > > at > org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) > > at > org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) > > at > org.apache.beam.runners.dataflow.worker.SimpleParDoFn$1.output(SimpleParDoFn.java:285) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.outputWindowedValue(SimpleDoFnRunner.java:275) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$900(SimpleDoFnRunner.java:85) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:423) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:411) > > at > com.davita.cwow.pmt.transformations.PmtAvroKafkaWriter$EvaluationAvroToProducerRecordFn.processElement(PmtAvroKafkaWriter.java:67) > > at > com.davita.cwow.pmt.transformations.PmtAvroKafkaWriter$EvaluationAvroToProducerRecordFn$DoFnInvoker.invokeProcessElement(Unknown > Source) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:211) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188) > > at > org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:340) > > at > org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) > > at > org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) > > at > org.apache.beam.runners.dataflow.worker.SimpleParDoFn$1.output(SimpleParDoFn.java:285) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.outputWindowedValue(SimpleDoFnRunner.java:275) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$900(SimpleDoFnRunner.java:85) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:423) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:411) > > at > com.davita.cwow.pmt.transformations.BatchEvaluationTransform$2.process(BatchEvaluationTransform.java:100) > > at > com.davita.cwow.pmt.transformations.BatchEvaluationTransform$2$DoFnInvoker.invokeProcessElement(Unknown > Source) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:211) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188) > > at > org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:340) > > at > org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) > > at > org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) > > at > org.apache.beam.runners.dataflow.worker.SimpleParDoFn$1.output(SimpleParDoFn.java:285) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.outputWindowedValue(SimpleDoFnRunner.java:275) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$900(SimpleDoFnRunner.java:85) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:423) > > at > com.davita.cwow.pmt.transformations.PatientProtocolEvaluationResultMutationTransform$ToEntityFn.processElement(PatientProtocolEvaluationResultMutationTransform.java:194) > > at > com.davita.cwow.pmt.transformations.PatientProtocolEvaluationResultMutationTransform$ToEntityFn$DoFnInvoker.invokeProcessElement(Unknown > Source) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:211) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:185) > > at > org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:340) > > at > org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) > > at > org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) > > at > org.apache.beam.runners.dataflow.worker.SimpleParDoFn$1.output(SimpleParDoFn.java:285) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.outputWindowedValue(SimpleDoFnRunner.java:275) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$900(SimpleDoFnRunner.java:85) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:423) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:411) > > at > java.base/java.util.ArrayList$ArrayListSpliterator.forEachRemaining(ArrayList.java:1655) > > at > java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:658) > > at > com.davita.cwow.pmt.transformations.ProtocolEvaluatorTransform$5.process(ProtocolEvaluatorTransform.java:248) > > at > com.davita.cwow.pmt.transformations.ProtocolEvaluatorTransform$5$DoFnInvoker.invokeProcessElement(Unknown > Source) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:211) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188) > > at > org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:340) > > at > org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) > > at > org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) > > at > org.apache.beam.runners.dataflow.worker.SimpleParDoFn$1.output(SimpleParDoFn.java:285) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.outputWindowedValue(SimpleDoFnRunner.java:275) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.access$900(SimpleDoFnRunner.java:85) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:423) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner$DoFnProcessContext.output(SimpleDoFnRunner.java:411) > > at > org.apache.beam.sdk.transforms.join.CoGroupByKey$ConstructCoGbkResultFn.processElement(CoGroupByKey.java:192) > > at > org.apache.beam.sdk.transforms.join.CoGroupByKey$ConstructCoGbkResultFn$DoFnInvoker.invokeProcessElement(Unknown > Source) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.invokeProcessElement(SimpleDoFnRunner.java:211) > > at > org.apache.beam.runners.dataflow.worker.repackaged.org.apache.beam.runners.core.SimpleDoFnRunner.processElement(SimpleDoFnRunner.java:188) > > at > org.apache.beam.runners.dataflow.worker.SimpleParDoFn.processElement(SimpleParDoFn.java:340) > > at > org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) > > at > org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) > > at > org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn$1.output(GroupAlsoByWindowsParDoFn.java:185) > > at > org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner$1.outputWindowedValue(GroupAlsoByWindowFnRunner.java:108) > > at > org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:128) > > at > org.apache.beam.runners.dataflow.worker.util.BatchGroupAlsoByWindowViaIteratorsFn.processElement(BatchGroupAlsoByWindowViaIteratorsFn.java:56) > > at > org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.invokeProcessElement(GroupAlsoByWindowFnRunner.java:121) > > at > org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowFnRunner.processElement(GroupAlsoByWindowFnRunner.java:73) > > at > org.apache.beam.runners.dataflow.worker.GroupAlsoByWindowsParDoFn.processElement(GroupAlsoByWindowsParDoFn.java:117) > > at > org.apache.beam.runners.dataflow.worker.util.common.worker.ParDoOperation.process(ParDoOperation.java:44) > > at > org.apache.beam.runners.dataflow.worker.util.common.worker.OutputReceiver.process(OutputReceiver.java:49) > > at > org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.runReadLoop(ReadOperation.java:218) > > at > org.apache.beam.runners.dataflow.worker.util.common.worker.ReadOperation.start(ReadOperation.java:169) > > at > org.apache.beam.runners.dataflow.worker.util.common.worker.MapTaskExecutor.execute(MapTaskExecutor.java:83) > > at > org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.executeWork(BatchDataflowWorker.java:420) > > at > org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.doWork(BatchDataflowWorker.java:389) > > at > org.apache.beam.runners.dataflow.worker.BatchDataflowWorker.getAndPerformWork(BatchDataflowWorker.java:314) > > at > org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.doWork(DataflowBatchWorkerHarness.java:140) > > at > org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:120) > > at > org.apache.beam.runners.dataflow.worker.DataflowBatchWorkerHarness$WorkerThread.call(DataflowBatchWorkerHarness.java:107) > > at > java.base/java.util.concurrent.FutureTask.run(FutureTask.java:264) > > at > java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128) > > at > java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628) > > at java.base/java.lang.Thread.run(Thread.java:834) > > Caused by: sun.security.validator.ValidatorException: PKIX path building > failed: sun.security.provider.certpath.SunCertPathBuilderException: unable > to find valid certification path to requested target > > at > java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:439) > > at > java.base/sun.security.validator.PKIXValidator.engineValidate(PKIXValidator.java:306) > > at > java.base/sun.security.validator.Validator.validate(Validator.java:264) > > at > java.base/sun.security.ssl.X509TrustManagerImpl.validate(X509TrustManagerImpl.java:313) > > at > java.base/sun.security.ssl.X509TrustManagerImpl.checkTrusted(X509TrustManagerImpl.java:222) > > at > java.base/sun.security.ssl.X509TrustManagerImpl.checkServerTrusted(X509TrustManagerImpl.java:129) > > at > java.base/sun.security.ssl.CertificateMessage$T13CertificateConsumer.checkServerCerts(CertificateMessage.java:1340) > > ... 117 more > > Caused by: sun.security.provider.certpath.SunCertPathBuilderException: > unable to find valid certification path to requested target > > at > java.base/sun.security.provider.certpath.SunCertPathBuilder.build(SunCertPathBuilder.java:141) > > at > java.base/sun.security.provider.certpath.SunCertPathBuilder.engineBuild(SunCertPathBuilder.java:126) > > at > java.base/java.security.cert.CertPathBuilder.build(CertPathBuilder.java:297) > > at > java.base/sun.security.validator.PKIXValidator.doBuild(PKIXValidator.java:434) > > ... 123 more > > CONFIDENTIALITY NOTICE: THIS MESSAGE IS CONFIDENTIAL, INTENDED FOR THE > NAMED RECIPIENT(S) AND MAY CONTAIN INFORMATION THAT IS (I) PROPRIETARY TO > THE SENDER, AND/OR, (II) PRIVILEGED, CONFIDENTIAL, AND/OR OTHERWISE EXEMPT > FROM DISCLOSURE UNDER APPLICABLE STATE AND FEDERAL LAW, INCLUDING, BUT NOT > LIMITED TO, PRIVACY STANDARDS IMPOSED PURSUANT TO THE FEDERAL HEALTH > INSURANCE PORTABILITY AND ACCOUNTABILITY ACT OF 1996 ("HIPAA"). IF YOU ARE > NOT THE INTENDED RECIPIENT, OR THE EMPLOYEE OR AGENT RESPONSIBLE FOR > DELIVERING THE MESSAGE TO THE INTENDED RECIPIENT, YOU ARE HEREBY NOTIFIED > THAT ANY DISSEMINATION, DISTRIBUTION OR COPYING OF THIS COMMUNICATION IS > STRICTLY PROHIBITED. IF YOU HAVE RECEIVED THIS TRANSMISSION IN ERROR, > PLEASE (I) NOTIFY US IMMEDIATELY BY REPLY E-MAIL OR BY TELEPHONE AT > (855.472.9822 <(855)%20472-9822>), (II) REMOVE IT FROM YOUR SYSTEM, AND > (III) DESTROY THE ORIGINAL TRANSMISSION AND ITS ATTACHMENTS WITHOUT READING > OR SAVING THEM. THANK YOU. > > -DaVita Inc- >
