Thanks for update, Brian! I’ve just checked the latest version and it still fails in the same way. Well, maybe it’s somehow related to my environment.
> On 12 Jun 2021, at 03:27, Brian Hulette <[email protected]> wrote: > > FYI Fernando recently contributed a setup script for configuring a local > development environment [1] that is continuously verified on mac and ubuntu > [2]. > > [1] https://github.com/apache/beam/pull/14584 > <https://github.com/apache/beam/pull/14584> > [2] https://github.com/apache/beam/actions/workflows/local_env_tests.yml > <https://github.com/apache/beam/actions/workflows/local_env_tests.yml> > On Wed, May 26, 2021 at 2:17 PM Alexey Romanenko <[email protected] > <mailto:[email protected]>> wrote: > Thanks for the link! > > Well, going back to initial problem - seems that it’s MacOS-related issue > about permissions to “/var/run/docker.sock” and quick workaround, like “sudo > chmod 666 /var/run/docker.sock” fixes the problem. > Though, I’m not sure we need to do this when building a container (since it’s > not good in terms of security). Perhaps, there is another proper way how to > fix it. > > — > Alexey > >> On 25 May 2021, at 19:27, Brian Hulette <[email protected] >> <mailto:[email protected]>> wrote: >> >> Yes this came up before, see BEAM-12149 [1]. I'm not sure what causes it, I >> wasn't able to repro the issue. >> >> [1] https://issues.apache.org/jira/browse/BEAM-12149 >> <https://issues.apache.org/jira/browse/BEAM-12149> >> On Tue, May 25, 2021 at 10:21 AM Alexey Romanenko <[email protected] >> <mailto:[email protected]>> wrote: >> I checked it again and I have failed Docker-dependent tests: >> >> > Task :sdks:java:io:clickhouse:test >> >> org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED >> java.lang.IllegalStateException at DockerClientProviderStrategy.java:215 >> >> org.apache.beam.sdk.io.clickhouse.AtomicInsertTest > classMethod FAILED >> java.lang.NullPointerException at BaseClickHouseTest.java:134 >> >> org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED >> java.lang.IllegalStateException at DockerClientProviderStrategy.java:109 >> >> org.apache.beam.sdk.io.clickhouse.ClickHouseIOTest > classMethod FAILED >> java.lang.NullPointerException at BaseClickHouseTest.java:134 >> >> 29 tests completed, 4 failed >> >> > Task :sdks:java:io:clickhouse:test FAILED >> >> >> >> Btw, while running it for the first time once a container was started, I had >> an issue with file permissions that I foxed manually. Is it a known issue? >> >> $ ./gradlew -p sdks/java/io/clickhouse/ check >> Starting a Gradle Daemon, 1 incompatible and 2 stopped Daemons could not be >> reused, use --status for details >> Configuration on demand is an incubating feature. >> > Task :model:pipeline:generateProto FAILED >> >> FAILURE: Build failed with an exception. >> >> * What went wrong: >> Execution failed for task ':model:pipeline:generateProto'. >> > protoc: stdout: . stderr: >> > /home/aromanenko/.gradle/caches/modules-2/files-2.1/io.grpc/protoc-gen-grpc-java/1.26.0/4f8bb54a74ab655cf2691e6eaa513fccc4c605d5/protoc-gen-grpc-java-1.26.0-linux-x86_64.exe: >> > program not found or is not executable >> Please specify a program using absolute path or make sure the program is >> available in your PATH system variable >> --grpc_out: protoc-gen-grpc: Plugin failed with status code 1. >> >> $ ll >> /home/aromanenko/.gradle/caches/modules-2/files-2.1/io.grpc/protoc-gen-grpc-java/1.26.0/4f8bb54a74ab655cf2691e6eaa513fccc4c605d5/protoc-gen-grpc-java-1.26.0-linux-x86_64.exe >> >> -rw-r--r-- 1 aromanenko users 3009112 May 21 17:35 >> /home/aromanenko/.gradle/caches/modules-2/files-2.1/io.grpc/protoc-gen-grpc-java/1.26.0/4f8bb54a74ab655cf2691e6eaa513fccc4c605d5/protoc-gen-grpc-java-1.26.0-linux-x86_64.exe >> >> >> — >> Alexey >> >>> On 21 May 2021, at 18:06, Brian Hulette <[email protected] >>> <mailto:[email protected]>> wrote: >>> >>> I think the build environment was set up with that configured: >>> https://github.com/apache/beam/blob/40326dd0a2a1c9b5dcbbcd6486a43e3875a64a43/start-build-env.sh#L110 >>> >>> <https://github.com/apache/beam/blob/40326dd0a2a1c9b5dcbbcd6486a43e3875a64a43/start-build-env.sh#L110> >>> Could there be something about your environment preventing that from >>> working? >>> >>> Brian >>> >>> On Fri, May 21, 2021 at 3:34 AM Gleb Kanterov <[email protected] >>> <mailto:[email protected]>> wrote: >>> Is it possible to mount the Docker socket inside the build-env Docker >>> container? We run a lot of similar tests in CI, and it always worked: >>> >>> --mount type=bind,source=/var/run/docker.sock,target=/var/run/docker.sock >>> >>> On Fri, May 21, 2021 at 12:26 PM Alexey Romanenko <[email protected] >>> <mailto:[email protected]>> wrote: >>> Hello, >>> >>> Beam provides a very cool feature to run a local development environment >>> via Docker [1]. In the same time, some unit tests require to run Docker >>> containers to test against “real” instances (for example, >>> ClickHouseIOTest). So, it will end up with “docker-in-docker” issue and >>> such tests will fail. >>> >>> What would be a proper solution for that? Annotate these tests with a >>> specific “DockerImageRequired” annotation and skip them when running from >>> inside container or something else? Any ideas on this? >>> >>> Thanks, >>> Alexey >>> >>> >>> [1] https://github.com/apache/beam/blob/master/start-build-env.sh >>> <https://github.com/apache/beam/blob/master/start-build-env.sh> >
