Ah okay then. I commented out the goavro line and the image does finish
building. It seems that the python still has be built which takes a bit
(using ` pip install -e .[gcp,test]` inside the sdks/python directory). It
looks like the source isn't synced with the host system it doesn't look
like there's a git repo inside the container either so maybe the container
env isn't worth the effort (especially if no one else is using or helping
maintaining it)
Going back to trying to get a local python environment working... I tried
doing this a few months ago and eventually just gave up. I ran
`./local-env-setup.sh` and then following the instructions in the python
developer guide in the wiki[1], I ran `pip install -e .[gcp,test]`. I
eventually hit an issue in what looks cythonizing windowed_value.py
```
[14/14] Cythonizing apache_beam/utils/windowed_value.py
```
There's a lot of messaging but it looks like the crux of the issue is:
```
clang: error: invalid arch name '-arch
-I/private/var/folders/n1/6qk3ljm97h32j1g7qg....
```
Googling around it seems like it might be some kind of mac M2 issue, but
there aren't any obvious solutions. Has anyone that works on the python sdk
with a new-ish mac run into this problem?
Best,
Joey
[1] https://cwiki.apache.org/confluence/display/BEAM/Python+Tips
On Thu, Sep 19, 2024 at 5:23 PM Robert Burke <rob...@frantil.com> wrote:
> To my knowledge, the "dev" docker image is unmaintained, and has been for
> quite some time, hence not having been moved to Go Modules and similar.
>
> I personally don't have time to delve into what's going on with it, but
> IIRC, deleting the weird goavro lines would be a good first step.
>
> I'd be happy to review any PRs to help fix it though, if you get it
> working for yourself. (Tag lostluck in GitHub).
>
> I think if you search the dev list, you might find previous discussions on
> the topic of the dev container.
>
> Robert Burke
>
>
> On Thu, Sep 19, 2024, 2:17 PM Joey Tran <joey.t...@schrodinger.com> wrote:
>
>> Hi,
>>
>> I'm trying again to get a working development image of beam. Working from
>> the beam repo CONTRIBUTING.md, I ran `./start-build-env.sh`
>>
>> At image build time after getting a few steps in, the build fails with:
>>
>> => ERROR [7/8] RUN go get github.com/linkedin/goavro/v2
>>
>> > [7/8] RUN go get github.com/linkedin/goavro/v2:
>>
>>
>> 0.134 go: go.mod file not found in current directory or any parent
>> directory.
>> 0.134 'go get' is no longer supported outside a module.
>>
>> 0.134 To build and install a command, use 'go install' with a version,
>>
>> 0.134 like 'go install example.com/cmd@latest'
>>
>> 0.134 For more information, see
>> https://golang.org/doc/go-get-install-deprecation
>> 0.134 or run 'go help get' or 'go help install'.
>> ------
>> 2 warnings found (use docker --debug to expand):
>> - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy
>> "ENV key value" format (line 8)
>> - LegacyKeyValueFormat: "ENV key=value" should be used instead of legacy
>> "ENV key value" format (line 7)
>> Dockerfile:10
>> --------------------
>> 8 | ENV GOPATH
>> /home/jtran/beam/sdks/go/examples/.gogradle/project_gopath
>> 9 | # This next command still runs as root causing the
>> ~/.cache/go-build to be owned by root
>> 10 | >>> RUN go get github.com/linkedin/goavro/v2
>> 11 | RUN chown -R jtran:100 /home/jtran/.cache
>> 12 |
>> --------------------
>> ERROR: failed to solve: process "/bin/bash -o pipefail -c go get
>> github.com/linkedin/goavro/v2" did not complete successfully: exit code:
>> 1
>>
>>
>> Any tips?
>> Cheers,
>> --
>>
>> Joey Tran
>>
>> [image: Schrödinger, Inc.] <https://schrodinger.com/>
>>
>
