Hello Divyansh, First, and foremost, I would like to call your attention to what Nic said here:
It looks like you're trying to set up > a Docker image based on one of our CI jobs which builds the docs for > multiple Arrow implementations, which I wouldn't recommend, as it involves > a lot more steps than you need for an R dev setup, and probably won't be > that useful for the specific task of getting a working R dev setup on > Docker. This is absolutely correct — the docker-compose file that you're looking at is intended for our CI and if what you're trying to do is update the docs, getting the ubuntu-cpp docker image to build locally will both be way more complicated than necessary and might not even accomplish what you're looking for. I highly encourage you to follow Nic's advice here. They are trying to help and ignoring that help is not going to get you where you want to go. As for the gist, I see a number of problems: (1) It looks like you never ran `docker-compose build ubuntu-docs` which might be the first, biggest issue there. You allude to having built it in your most recent email, but it's not in the gist, so I'm not sure if you did and didn't include it or what. (2) It looks like you've taken only the `ubuntu-docs` service from the docker-compose.yml file from the root of the arrow directory and placed it in its own separate docker-compose.yml file. This isn't going to work because there are a number of dependencies on other services (see the usage section where it says to build `ubuntu-cpp` and `ubuntu-python` first? that's what that's about). Editing the `docker-compose.yml` file takes a lot of care + detailed knowledge of our setup as well as the project generally. (3) Like I noted in (2), I don't see any references you make to building the dependent services `ubuntu-cpp` and `ubuntu-python` which are required for `ubuntu-docs`. This is also alluded to in the last section of the linked documentation [1]. If you're going to pursue using the docker images for CI locally, I highly encourage you to learn how to use archery to do so. (but see below, I like others in this thread do not encourage this!) (4) I've just finished building `ubuntu-cpp` locally using the commands in the usage section [2] and it completed successfully through the R documentation [6] — so the docker-compose file that is there + those services work if you follow the usage section using the unedited `docker-compose.yml` file. But again, like Nic says: I highly encourage you not to take this approach of trying to run the CI docker containers. Instead, I recommend you follow the other developer documentation [3], [4], [5] especially that first one for the issue of re-rendering the R documentation. Finally, on an etiquette point: I noticed a second thread on the mailing list pointing to this same issue. I know that you're not super familiar with the community yet, but starting multiple threads like that is not something we do here. It's confusing because different people might respond to each thread and not realize that an issue has already been resolved. [1] – https://arrow.apache.org/docs/r/articles/developers/docker.html#example---using-archery [2] – https://github.com/apache/arrow/blob/cda7ffc67abf6c8b9170cc2606acf1a704b161eb/docker-compose.yml#L1755-L1758 [3] – https://arrow.apache.org/docs/r/articles/developers/workflow.html#rebuilding-the-documentation [4] – https://arrow.apache.org/docs/dev/r/articles/developers/setup.html (note: this is a link to our dev site because this process has changed slightly recently — which should be much improved and easier to use now!) [5] – https://arrow.apache.org/docs/r/articles/developing.html [6] – There was an issue with a connection reset in maven and the java build at the very end, but that's not relevant to what you're doing here. -Jon On Fri, Oct 20, 2023 at 6:30 AM Divyansh Khatri <divyanshkhatri...@gmail.com> wrote: > This is a different issue from the previous one Nic Crane I think I > figured out the one with the R setup but this one is related to the setup > of the Ubuntu-docs which i setup up with the help of the docker-compose.yml > file as mentioned in the gist.I have built the image but for some reason am > not able to deploy the container.Please refer to the gist for the > understanding the whole issue. > > On Fri, 20 Oct 2023 at 14:09, Nic Crane <thisis...@gmail.com> wrote: > > > Hi Divyansh, > > > > You mentioned previously that you were trying to get an R dev setup in > > Docker, but that gist looks quite different to what's recommended in the > R > > dev setup instructions linked above. It looks like you're trying to set > up > > a Docker image based on one of our CI jobs which builds the docs for > > multiple Arrow implementations, which I wouldn't recommend, as it > involves > > a lot more steps than you need for an R dev setup, and probably won't be > > that useful for the specific task of getting a working R dev setup on > > Docker. > > > > I'd recommend instead looking at the article mentioned by me, Bryce, and > > Jon [1]. Happy to answer any questions if any issues come up with those > > instructions, as they could potentially be made more clear, and it's > always > > useful to get feedback on docs like these. > > > > Nic > > > > [1] https://arrow.apache.org/docs/r/articles/developers/docker.html > > > > On Fri, 20 Oct 2023 at 08:13, Divyansh Khatri < > divyanshkhatri...@gmail.com > > > > > wrote: > > > > > please see this and help me resolve the issue > > > > https://gist.github.com/Divyansh200102/3ba4f5e391d8e62307f8b584a5a659d8 > > > > > > On Wed, 18 Oct 2023 at 19:14, Jonathan Keane <jke...@gmail.com> wrote: > > > > > > > For development of the R package with docker containers, the link [1] > > > that > > > > Nic sent in this same thread is the place to go. In addition to that > > > > docker-focused one, there are a handful of others that might prove > > useful > > > > to you in getting your development environment setup [2]. > > > > > > > > If you run into any issues, feel free to post here, but it's helpful > to > > > do > > > > so with debugging mode on (i.e. set the env var ARROW_DEV to true) > and > > to > > > > provide the exact commands you sent along with the output you're > seeing > > > so > > > > we can help diagnose what's going wrong. > > > > > > > > [1] – > https://arrow.apache.org/docs/r/articles/developers/docker.html > > > > [2] – > > > https://arrow.apache.org/docs/r/articles/index.html#developer-guides > > > > > > > > -Jon > > > > > > > > > > > > On Wed, Oct 18, 2023 at 2:48 AM Divyansh Khatri < > > > > divyanshkhatri...@gmail.com> > > > > wrote: > > > > > > > > > I am trying to contribute to the arrow project.so i am trying to > > setup > > > > the > > > > > project on locally. > > > > > > > > > > On Tue, 17 Oct 2023 at 05:14, Bryce Mecum <bryceme...@gmail.com> > > > wrote: > > > > > > > > > > > That error makes it look like you're running `docker compose up` > > from > > > > > > the root of the Arrow source tree which is likely not what you > > want. > > > > > > Are you trying to use the Arrow R package in a Docker container > or > > > are > > > > > > you trying to contribute to it by developing inside of a Docker > > > > > > container? Nic's link [1] is a good starting point. > > > > > > > > > > > > [1] > > https://arrow.apache.org/docs/r/articles/developers/docker.html > > > > > > > > > > > > On Mon, Oct 16, 2023 at 4:31 AM Divyansh Khatri > > > > > > <divyanshkhatri...@gmail.com> wrote: > > > > > > > > > > > > > > Hi,so i am basically using the docker cmd 'docker compose up > -d' > > in > > > > the > > > > > > > docker-compose.yml but i am encountering this error(Error > > response > > > > from > > > > > > > daemon: manifest for amd64/maven:3.5.4-eclipse-temurin-8 not > > found: > > > > > > > manifest unknown: manifest unknown)so i am not sure how to > > proceed > > > > from > > > > > > > here? > > > > > > > > > > > > > > On Mon, 16 Oct 2023 at 14:17, Nic Crane <thisis...@gmail.com> > > > wrote: > > > > > > > > > > > > > > > Hi Divyansh, > > > > > > > > > > > > > > > > There are instructions for creating a R package dev setup > here: > > > > > > > > > https://arrow.apache.org/docs/r/articles/developers/setup.html > > > > > > > > > > > > > > > > If you can explain a bit more about what you've tried so far > > and > > > > > > what's not > > > > > > > > working, we may be able to advise. > > > > > > > > > > > > > > > > Best wishes, > > > > > > > > > > > > > > > > Nic > > > > > > > > > > > > > > > > On Mon, 16 Oct 2023 at 06:02, Divyansh Khatri < > > > > > > divyanshkhatri...@gmail.com > > > > > > > > > > > > > > > > > wrote: > > > > > > > > > > > > > > > > > I am having problems regarding setting up the r package > using > > > > > docker > > > > > > of > > > > > > > > the > > > > > > > > > apache arrow.Can you give me the step by step process of > how > > > do i > > > > > > setup > > > > > > > > the > > > > > > > > > r package in my vs code system using docker. > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > > >
