[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17052431#comment-17052431 ] Luke Cwik commented on BEAM-9288: - Release vote thread for RC #3: [https://lists.apache.org/thread.html/rea4a27c47529a27936ab2c51162c8e532b8b625c4d70c4f7f485c7cd%40%3Cdev.beam.apache.org%3E] > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 6h > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17051631#comment-17051631 ] Luke Cwik commented on BEAM-9288: - Yes to your steps: My understanding to the follow up of this Jira is: 1. "stop adding the conscrypt native libraries as part in META-INF" 2. re-vote vendored grpc release, and release it if it passes votes. Is the above all needed be done to close this Jira? Yes to the fact that pr/10940 did not address all the goals. > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 4h 40m > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17051619#comment-17051619 ] Rui Wang commented on BEAM-9288: A followup question is, is "still are adding the conscrypt native libraries as part in META-INF" related to https://github.com/apache/beam/pull/10940? E.g. is META-INF is a separate issue or is it supposed to be fixed by #10940 but that PR didn't achieve this goal? > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 4h 40m > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17051602#comment-17051602 ] Rui Wang commented on BEAM-9288: [~lcwik] My understanding to the follow up of this Jira is: 1. "adding the conscrypt native libraries as part in META-INF" 2. re-vote vendored grpc release, and release it if it passes votes. Is the above all needed be done to close this Jira. > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 4h 40m > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050643#comment-17050643 ] Luke Cwik commented on BEAM-9288: - It seems we still are adding the conscrypt native libraries as part in META-INF $ jar tvf beam-vendor-grpc-1_26_0-0.3.jar | grep conscrypt 2218176 Mon Sep 17 10:36:24 CEST 2018 META-INF/native/libconscrypt_openjdk_jni-linux-x86_64.so 1720832 Mon Sep 17 10:36:24 CEST 2018 META-INF/native/conscrypt_openjdk_jni-windows-x86.dll 2637496 Mon Sep 17 10:36:24 CEST 2018 META-INF/native/libconscrypt_openjdk_jni-osx-x86_64.dylib 2557952 Mon Sep 17 10:36:24 CEST 2018 META-INF/native/conscrypt_openjdk_jni-windows-x86_64.dll > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 4h 40m > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050468#comment-17050468 ] Luke Cwik commented on BEAM-9288: - RC #1 vote thread: [https://lists.apache.org/thread.html/r9398fd720cc2372dab5ca2598453c9163fedf05dd23eacdef4538ef4%40%3Cdev.beam.apache.org%3E] > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 4h 40m > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17050052#comment-17050052 ] Ismaël Mejía commented on BEAM-9288: As mentioned by [~lukec...@gmail.com] If we cherry-pick #10940 we still need a proper vote/release from the vendored dependency, so better start with this. So better start with the vote [~sunjincheng121] or [~lukec...@gmail.com] can you start to roll the ball for the vendored grpc release? > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049607#comment-17049607 ] Rui Wang commented on BEAM-9288: I will cherry-pick #10940 and make 2.20.0 released. After that you could update the version in Beam. Does it sounds good? > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049540#comment-17049540 ] Luke Cwik commented on BEAM-9288: - A release needs to happen and the version used in Apache beam needs to be updated to match. > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17049537#comment-17049537 ] Rui Wang commented on BEAM-9288: [~sunjincheng121] will this Jira be considered finished? > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 3h 20m > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047818#comment-17047818 ] Kenneth Knowles commented on BEAM-9288: --- Oh, and yea Beam definitely needs this. Performance without it is very bad. > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 2.5h > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047817#comment-17047817 ] Kenneth Knowles commented on BEAM-9288: --- If it is not a regression then I would not block the release. I think though that the fix for BEAM-9030 _is_ a regression so it is a blocker. Jincheng already has a fix up so maybe the timing for everything will be OK? > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 2.5h > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047343#comment-17047343 ] Ismaël Mejía commented on BEAM-9288: Oh I see that's clearly a mistake I will unlink the issues because they are not related. BEAM-9252 was all about users being unable to shade the Jar and this is solved. This issue is different. My comment about Beam's use of the dependency is related to the fact that it looks like a transitive dependency that gRPC uses, but that our use of gRPC might not use. If that's the case we can go with it even if it is badly shaded, but well I understand if you guys prefer to do it properly, I just wanted to avoid delaying the release if we can go with it as it is. > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 2h 10m > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047061#comment-17047061 ] Kenneth Knowles commented on BEAM-9288: --- But since you filed BEAM-9252 (resolved) but this is not resolved even though it says it is blocking BEAM-9252 perhaps they are being confused. > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 2h > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17047060#comment-17047060 ] Kenneth Knowles commented on BEAM-9288: --- I think leaving it as a residual dependency is necessary. Jincheng has the PR ready for that. After that, solving actual shading of conscrypt is more determined by whether we hit diamond dep issues, like with any Java library. > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 2h > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17046992#comment-17046992 ] Ismaël Mejía commented on BEAM-9288: Is this a real issue or just a nice to have, because if it is the former this implies a new release of the vendored grpc dependency with the respective 3 day vote, is this really important or can we just wait and do it for 2.21.0 ? > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17043967#comment-17043967 ] Rui Wang commented on BEAM-9288: I am going to cut 2.20.0 branch on 02/26/2020. Can this Jira be finished by that date? > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 1h 50m > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17042601#comment-17042601 ] sunjincheng commented on BEAM-9288: --- I agree that we should exclude it from the vendored grpc for now. I will submit a PR to address this issue. > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17041366#comment-17041366 ] Kenneth Knowles commented on BEAM-9288: --- In the near term, I think we can exclude it from vendor/grpc. Then vendor/grpc will have a transitive dependency that users can be aware of, instead of relocated and bundled. It still can have diamond dependency problems, but at least they are exposed and "normal". > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Critical > Fix For: 2.20.0 > > Time Spent: 10m > Remaining Estimate: 0h > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034131#comment-17034131 ] Esun Kim commented on BEAM-9288: Gcsio starts depending on gRPC along with conscrypt and we're considering if it's feasible to shade it in gcsio. Since Conscrypt is not ready for being shaded, however, shading isn't considered as an option for now. I think it shouldn't shade it until Conscrypt becomes fully aware of shading. > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Major > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034076#comment-17034076 ] Luke Cwik commented on BEAM-9288: - If gcs io is shading conscrypt, then it should fix it's shading. It should rely on a provided conscrypt unless gcs io marks the dependency as provided. Apache beam should fix it's own usage to not interfere with others usage of it and not have users depend on it. > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Major > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034066#comment-17034066 ] Igor Dvorzhak commented on BEAM-9288: - Will it be better to exclude Concrypt from shaded GCS IO jar and rely on Conscrypt installed system-wide, if any? > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Major > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034021#comment-17034021 ] Luke Cwik commented on BEAM-9288: - I filed [https://github.com/google/conscrypt/issues/811] for "auto" discovery of package path. > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Major > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034017#comment-17034017 ] Luke Cwik commented on BEAM-9288: - The idea would be to use a solution similar to tcnative where the library path can be "parsed" to provide the appropriate package prefix: https://github.com/netty/netty-tcnative/blob/9fd38de8a07cc413b83cacf39470f2845a68e26c/openssl-dynamic/src/main/c/jnilib.c#L233 > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Major > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034010#comment-17034010 ] Luke Cwik commented on BEAM-9288: - cc: [~sunjincheng121] Conscrypt can be shaded properly by building a special version of the so file with a JNI_JARJAR_PREFIX defined. It would be best if conscrypt was updated to support auto-detection of post relocation path instead of needing everyone to build their own version. > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Assignee: sunjincheng >Priority: Major > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034008#comment-17034008 ] Chamikara Madhusanka Jayalath commented on BEAM-9288: - Is there a way we can upgrade gRPC without shading Conscrypt ? @sunjincheng121 can you look into this ? > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Priority: Major > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
[jira] [Commented] (BEAM-9288) Conscrypt shaded dependency
[ https://issues.apache.org/jira/browse/BEAM-9288?page=com.atlassian.jira.plugin.system.issuetabpanels:comment-tabpanel=17034007#comment-17034007 ] Chamikara Madhusanka Jayalath commented on BEAM-9288: - cc: [~lcwik] [~medb] > Conscrypt shaded dependency > --- > > Key: BEAM-9288 > URL: https://issues.apache.org/jira/browse/BEAM-9288 > Project: Beam > Issue Type: Bug > Components: build-system >Reporter: Esun Kim >Priority: Major > > Conscrypt is not designed to be shaded properly mainly because of so files. I > happened to see BEAM-9030 (*1) creating a new vendored gRPC shading Conscrypt > (*2) in it. I think this could make a problem when new Conscrypt is brought > by new gcsio depending on gRPC-alts (*4) in a dependency chain. (*5) In this > case, it may have a conflict when finding proper so files for Conscrypt. > *1: https://issues.apache.org/jira/browse/BEAM-9030 > *2: > [https://github.com/apache/beam/blob/e24d1e51cbabe27cb3cc381fd95b334db639c45d/buildSrc/src/main/groovy/org/apache/beam/gradle/GrpcVendoring_1_26_0.groovy#L78] > *3: https://issues.apache.org/jira/browse/BEAM-6136 > *4: [https://mvnrepository.com/artifact/io.grpc/grpc-alts/1.27.0] > *5: https://issues.apache.org/jira/browse/BEAM-8889 > -- This message was sent by Atlassian Jira (v8.3.4#803005)
