Re: [go-cd] Mount .ssh from EFS to the container (from the image gocd/gocd-server:v22.3.0)
Thank you Sriram.
So, ".ssh" folder mounting will be separate from the rest of the data
(/godata, for plugins, pipelines, db etc)...so there would be two separate
mount points into the container ?
I'm using ECS at the moment and not kubernetes, so my task definition will
have two mount points like below:
```
"mountPoints": [
{
"sourceVolume": "efs_id:/godata",
"containerPath": "/godata"
},
{
"sourceVolume": "efs_id:/godata/.ssh",
"containerPath": "/home/go/.ssh"
}
],
```
So mounting /godata and efs_id:/godata/.ssh from EFS into the
container at /godata
and /home/go/.ssh locations respectively (per above code) seems to work.
In this case entry_point.sh from the base image is able to map/consider and
execute them properly, hence the server is up and running and
functioning properly.
Is that the way it has to be, I think the github repo for gocd server says
that I guess, but perhaps I feel that extra mount point just for .ssh is
overkill and if .ssh can also be entertained by entry_point.sh from one
single mount point /godata in my case, that would be great ?
If I do not mount .ssh into /home/go/.ssh separately into the container -
things seem to fail complaining that "key verification failed", I'm not
sure whether I'm still missing something here.
Many thanks
Satya
On Thu, Apr 25, 2024 at 3:31 PM Sriram Narayanan
wrote:
>
>
> On Thu, Apr 25, 2024 at 10:16 PM Satya Elipe
> wrote:
>
>> Hi all
>>
>> Wonder, what's the way around to mount .ssh from EFS into the gocd base
>> container (from the image gocd/gocd-server:v22.3.0).
>>
>>
>> We have saved all our content into EFS under /godata and maps that into
>> the container as /godata.
>>
>>
>> We are using gocd/gocd-server:v22.3.0.
>>
>>
>> It all runs good, mapping was fine too but just one thing that’s not
>> happening is “.ssh” folder.
>>
>>
>> I have .ssh with all required keys in EFS under /godata and /godata
>> within the container also has .ssh but not /go-working-dir.
>>
>>
>> Is that supported, am I mis-configuring it, or do we need to handle that
>> outside of the base image ?
>>
>
> At a high level, the .ssh folder should be mounted into /home/go.
> e.g. docker run -v /path/to/godata:/godata -v /path/to/home-dir:/home/go
> gocd/gocd-server:v23.5.0
> IMPORTANT: You must set the user ID of the files within .ssh to 1000. This
> is the user ID of the gocd process within the container.
>
> See:
> https://github.com/gocd/docker-gocd-server?tab=readme-ov-file#mounting-volumes
>
> Given that you are using Kubernetes, please see the Helm chart
> documentation here
> https://github.com/gocd/helm-chart/blob/master/gocd/README.md
>
> It provides info on just about every configurable attribute for the GoCD
> server and the agent.
>
> Of particular importance for you are these two attributes:
> server.persistence.subpath.homego
> agent.persistence.subpath.homego
>
> Please see that document and jot down your action plan since you will need
> to provide the SSH keys to the server _and_ the agent containers.
>
> IMPORTANT: You must set the user ID of the files within .ssh to 1000. This
> is the user ID of the gocd process within the container.
>
>
>
>>
>> Many thanks in advance !
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "go-cd" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/CADKEDRrQOX11i951ZPiUYeOdMqThbCoZG7_WAqgBJFg1BxqxfQ%40mail.gmail.com
>> <https://groups.google.com/d/msgid/go-cd/CADKEDRrQOX11i951ZPiUYeOdMqThbCoZG7_WAqgBJFg1BxqxfQ%40mail.gmail.com?utm_medium=email&utm_source=footer>
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CANiY96aM47Ck0vc%3D1BnjnMd%2BT9eu4BKokLqLXMG0mNAezT2V_A%40mail.gmail.com
> <https://groups.google.com/d/msgid/go-cd/CANiY96aM47Ck0vc%3D1BnjnMd%2BT9eu4BKokLqLXMG0mNAezT2V_A%40mail.gmail.com?utm_medium=email&utm_source=footer>
> .
>
--
You received this message because you are subscribed to the Google Groups
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit
https://groups.google.com/d/msgid/go-cd/CADKEDRoj%2BrFqeT%2B3%2BF_TYnOn6C03kTJyzDzdtChoDOEc_BWxzg%40mail.gmail.com.
Re: [go-cd] Elastic agents plugin usage
of our pipelines, if you trigger the build job that in turn triggers the second and second triggers the third. Attached is the snippet from the dashboard. Please let me know if I have not covered any point or if you need more details on any specific thing, thank you. Regards Satya On Thu, Apr 25, 2024 at 3:21 PM Sriram Narayanan wrote: > > > On Thu, Apr 25, 2024 at 10:01 PM Satya Elipe > wrote: > >> Hi All >> >> I'm encountering some issues with the way Elastic agents are launched, >> assigned, and terminated. Despite setting the maximum agent count to two, >> both agents launch sequentially, with only the first being assigned to the >> job. >> > > Do you want the job to run on both the agents? If so, then these > instructions will help you: > https://docs.gocd.org/current/advanced_usage/admin_spawn_multiple_jobs.html > > >> >> Here's where it gets tricky: when the staging job completes and triggers >> the production job, I expect one of the active agents to take over. >> Instead, the production job attempts to launch new agents, fails due to the >> max count limit, and runs without any agents, leading to failure. >> >> >> > Do the various jobs have an elastic profile ID set? > > What is the error that you see due to the max count limit? > > When you say "staging job", do you have a stage in a pipeline called > "staging" with one job in it? Or do you have a stage in a pipeline with one > job called "staging" and the other called "prod"? > > Could you share how your pipelines are composed? I'm especially asking > this since many new users tend to use GoCD after using other tools and > carry over some of the terminology but also the constraints. If you share > your pipeline structure and what you want to achieve, then we can design > something together. > > >> Additionally, some agent instances remain active for an extended period, >> requiring manual termination. This disrupts the workflow significantly. >> >> >> > On our cluster, we see the pods being activated upon need, then the > relevant job runs in the pod, and the pod is then deactivated. We are > sticking to the default of "10 pods" right now, and will be increasing the > limit after certain parallel-load reviews. > > Could you share your Cluster Profile and the Elastic Profile? Please take > care to obfuscate any org-specific information such as IP addresses, > hostnames, AWS ARNs, URLs, etc. > > >> Has anyone experienced similar issues, or anyone has any suggestions for >> a workaround? >> >> >> Thanks in advance ! >> >> -- >> You received this message because you are subscribed to the Google Groups >> "go-cd" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to go-cd+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/go-cd/CADKEDRo_0yJjA0y31vOkXzgtVA_MOiSPQEc_uB3fE%3DfguO-wWQ%40mail.gmail.com >> <https://groups.google.com/d/msgid/go-cd/CADKEDRo_0yJjA0y31vOkXzgtVA_MOiSPQEc_uB3fE%3DfguO-wWQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "go-cd" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to go-cd+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/go-cd/CANiY96Yvww_OoK3tzZ%3DogcDjN82FGfydkOoFvxxpCDwHRzKV9A%40mail.gmail.com > <https://groups.google.com/d/msgid/go-cd/CANiY96Yvww_OoK3tzZ%3DogcDjN82FGfydkOoFvxxpCDwHRzKV9A%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CADKEDRpY%2ByC2YiN-nE2ZTY9DNSwYrAVmNaFZynDnAD0yXDP6kg%40mail.gmail.com.
[go-cd] Mount .ssh from EFS to the container (from the image gocd/gocd-server:v22.3.0)
Hi all Wonder, what's the way around to mount .ssh from EFS into the gocd base container (from the image gocd/gocd-server:v22.3.0). We have saved all our content into EFS under /godata and maps that into the container as /godata. We are using gocd/gocd-server:v22.3.0. It all runs good, mapping was fine too but just one thing that’s not happening is “.ssh” folder. I have .ssh with all required keys in EFS under /godata and /godata within the container also has .ssh but not /go-working-dir. Is that supported, am I mis-configuring it, or do we need to handle that outside of the base image ? Many thanks in advance ! -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CADKEDRrQOX11i951ZPiUYeOdMqThbCoZG7_WAqgBJFg1BxqxfQ%40mail.gmail.com.
[go-cd] Elastic agents plugin usage
Hi All I'm encountering some issues with the way Elastic agents are launched, assigned, and terminated. Despite setting the maximum agent count to two, both agents launch sequentially, with only the first being assigned to the job. Here's where it gets tricky: when the staging job completes and triggers the production job, I expect one of the active agents to take over. Instead, the production job attempts to launch new agents, fails due to the max count limit, and runs without any agents, leading to failure. Additionally, some agent instances remain active for an extended period, requiring manual termination. This disrupts the workflow significantly. Has anyone experienced similar issues, or anyone has any suggestions for a workaround? Thanks in advance ! -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CADKEDRo_0yJjA0y31vOkXzgtVA_MOiSPQEc_uB3fE%3DfguO-wWQ%40mail.gmail.com.
Re: [go-cd] "/godata/db/h2db/cruise.mv.db" is locked
Sure, I will. Thank you . On Thu, Apr 25, 2024 at 2:56 PM Sriram Narayanan wrote: > > > On Thu, Apr 25, 2024 at 9:53 PM Satya Elipe wrote: > >> Thank you, Sriram. >> >> >> I managed to launch the new container alongside the old one. >> > > Good! > > >> >> As you use the Elastic Agents plugin, can I ask you a question please. >> > > Could you post this as a separate email question? That will help > improve discoverability. Meanwhile, I'll read your email a bit and then > have a response ready as well. > > >> >> I'm encountering some issues with the way agents are launched, assigned, >> and terminated. Despite setting the maximum agent count to two, both agents >> launch sequentially, with only the first being assigned to the job. >> >> >> Here's where it gets tricky: when the staging job completes and triggers >> the production job, I expect one of the active agents to take over. >> Instead, the production job attempts to launch new agents, fails due to the >> max count limit, and runs without any agents, leading to failure. >> >> >> Additionally, some agent instances remain active for an extended period, >> requiring manual termination. This disrupts the workflow significantly. >> >> >> Have you experienced similar issues, or do you have any suggestions for a >> workaround? >> >> >> Thanks in advance ! >> >> >> >> >> On Tue, Apr 23, 2024 at 4:13 PM Sriram Narayanan >> wrote: >> >>> >>> >>> On Tue, Apr 23, 2024 at 10:22 PM Satya Elipe >>> wrote: >>> >>>> Hi All >>>> >>>> I'm dockerizing our static/standalone GoCD server. >>>> >>>> Hence copied the content from the static GoCD server to EFS and mounted >>>> that EFS content into the container (GoCD server container). >>>> >>>> And with that, container starts and runs but it can't fetch any data, >>>> and I see the below in the log: >>>> >>>> ```jvm 2| Caused by: >>>> org.h2.jdbc.JdbcSQLNonTransientConnectionException: Database may be already >>>> in use: null. Possible solutions: close all other connection(s); use the >>>> server mode [90020-200] >>>> >>>> jvm 2| Caused by: java.lang.IllegalStateException: The file is >>>> locked: nio:/godata/db/h2db/cruise.mv.db [1.4.200/7]``` >>>> >>>> Looks like the static/old server acquired the lock and when the data >>>> from that server copied to EFS lock is still maintained and hence the new >>>> server has the db file locked and couldn't fetch any data. >>>> >>>> Wonder, how are the people handling this situation ? >>>> We need the old server running as well when the new server is being >>>> brought up within the docker container. >>>> >>>> Any inputs will be of great help. >>>> >>> >>> If this is an important GoCD instance, then I urge you to move off H2DB >>> to PostgreSQL. >>> >>> We recently migrated a GoCD instance that has about 6 years of data in >>> H2DB to use RDS and hosted GoCD itself as a Container. We are running on an >>> EKS cluster and are making use of the Elastic Agent plugin. >>> >>> The H2DB that GoCD uses does not allow multiple processes to share the >>> same database file. I recommend that you stop the original GoCD process >>> (the jobs will keep running and will wait for the server to come back), >>> rsync the DB contents to the EFS store to speed up the copy and remove the >>> locks, and then start the new GoCD instance as a container with the DB >>> mounted via EFS. >>> >>> If you could consider moving off H2DB and using postgres, then use the >>> DB Migrator tool and do so. You process would then be: >>> 1. Stop the older GoCD server >>> 2. Use the DB Migrator tool to migrate from the H2DB file to a >>> PostgreSQL instance. >>> 3. Start the GoCD instance as a Container process with the configuration >>> to point it to the PostgreSQL instance. >>> >>> You may want to triel this once and then perform the actual switch. >>> >>> >>> >>>> >>>> Many thanks, >>>> Satya >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "go-cd"
Re: [go-cd] "/godata/db/h2db/cruise.mv.db" is locked
Thank you, Sriram. I managed to launch the new container alongside the old one. As you use the Elastic Agents plugin, can I ask you a question please. I'm encountering some issues with the way agents are launched, assigned, and terminated. Despite setting the maximum agent count to two, both agents launch sequentially, with only the first being assigned to the job. Here's where it gets tricky: when the staging job completes and triggers the production job, I expect one of the active agents to take over. Instead, the production job attempts to launch new agents, fails due to the max count limit, and runs without any agents, leading to failure. Additionally, some agent instances remain active for an extended period, requiring manual termination. This disrupts the workflow significantly. Have you experienced similar issues, or do you have any suggestions for a workaround? Thanks in advance ! On Tue, Apr 23, 2024 at 4:13 PM Sriram Narayanan wrote: > > > On Tue, Apr 23, 2024 at 10:22 PM Satya Elipe > wrote: > >> Hi All >> >> I'm dockerizing our static/standalone GoCD server. >> >> Hence copied the content from the static GoCD server to EFS and mounted >> that EFS content into the container (GoCD server container). >> >> And with that, container starts and runs but it can't fetch any data, and >> I see the below in the log: >> >> ```jvm 2| Caused by: >> org.h2.jdbc.JdbcSQLNonTransientConnectionException: Database may be already >> in use: null. Possible solutions: close all other connection(s); use the >> server mode [90020-200] >> >> jvm 2| Caused by: java.lang.IllegalStateException: The file is >> locked: nio:/godata/db/h2db/cruise.mv.db [1.4.200/7]``` >> >> Looks like the static/old server acquired the lock and when the data from >> that server copied to EFS lock is still maintained and hence the new server >> has the db file locked and couldn't fetch any data. >> >> Wonder, how are the people handling this situation ? >> We need the old server running as well when the new server is being >> brought up within the docker container. >> >> Any inputs will be of great help. >> > > If this is an important GoCD instance, then I urge you to move off H2DB to > PostgreSQL. > > We recently migrated a GoCD instance that has about 6 years of data in > H2DB to use RDS and hosted GoCD itself as a Container. We are running on an > EKS cluster and are making use of the Elastic Agent plugin. > > The H2DB that GoCD uses does not allow multiple processes to share the > same database file. I recommend that you stop the original GoCD process > (the jobs will keep running and will wait for the server to come back), > rsync the DB contents to the EFS store to speed up the copy and remove the > locks, and then start the new GoCD instance as a container with the DB > mounted via EFS. > > If you could consider moving off H2DB and using postgres, then use the DB > Migrator tool and do so. You process would then be: > 1. Stop the older GoCD server > 2. Use the DB Migrator tool to migrate from the H2DB file to a PostgreSQL > instance. > 3. Start the GoCD instance as a Container process with the configuration > to point it to the PostgreSQL instance. > > You may want to triel this once and then perform the actual switch. > > > >> >> Many thanks, >> Satya >> >> -- >> You received this message because you are subscribed to the Google Groups >> "go-cd" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to go-cd+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/go-cd/CADKEDRoqGkWLv6%3DUG8CA-NAD6sRETHiE6bB5Q2crf2Ezbn-cMw%40mail.gmail.com >> <https://groups.google.com/d/msgid/go-cd/CADKEDRoqGkWLv6%3DUG8CA-NAD6sRETHiE6bB5Q2crf2Ezbn-cMw%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "go-cd" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to go-cd+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/go-cd/CANiY96af882MjPPSVFowz9hNqakLNS7xF%2Bu7mLWb_2FXQ8fQrg%40mail.gmail.com > <https://groups.google.com/d/msgid/go-cd/CANiY96af882MjPPSVFowz9hNqakLNS7xF%2Bu7mLWb_2FXQ8fQrg%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CADKEDRpg_6YdfCGt7_sOp%3DkOyb6DVSf19mAuJdzeXbvgk76VAA%40mail.gmail.com.
[go-cd] "/godata/db/h2db/cruise.mv.db" is locked
Hi All I'm dockerizing our static/standalone GoCD server. Hence copied the content from the static GoCD server to EFS and mounted that EFS content into the container (GoCD server container). And with that, container starts and runs but it can't fetch any data, and I see the below in the log: ```jvm 2| Caused by: org.h2.jdbc.JdbcSQLNonTransientConnectionException: Database may be already in use: null. Possible solutions: close all other connection(s); use the server mode [90020-200] jvm 2| Caused by: java.lang.IllegalStateException: The file is locked: nio:/godata/db/h2db/cruise.mv.db [1.4.200/7]``` Looks like the static/old server acquired the lock and when the data from that server copied to EFS lock is still maintained and hence the new server has the db file locked and couldn't fetch any data. Wonder, how are the people handling this situation ? We need the old server running as well when the new server is being brought up within the docker container. Any inputs will be of great help. Many thanks, Satya -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CADKEDRoqGkWLv6%3DUG8CA-NAD6sRETHiE6bB5Q2crf2Ezbn-cMw%40mail.gmail.com.
Re: [go-cd] Help Needed with GoCD Docker Setup and cruise-config.xml Configuration
Just to add: I see the right content in the file in the /godata/: ``` bash-5.1$ ls -al /godata/cruise-config.xml -rwxr-xr-x1 root root 14950 Apr 11 09:38 /godata/cruise-config.xml bash-5.1$ cat /godata/cruise-config.xml http://www.w3.org/2001/XMLSchema-instance"; xsi:noNamespaceSchemaLocation="cruise-config.xsd" schemaVersion="139"> https://gocd-brxt.corp.mendeley.com/ https://gocd-brxt.corp.mendeley.com/ PasswordFilePath /etc/go/users.passwd api_user artifacts ``` But then looks like the file getting changed after the sym links creation: ``` bash-5.1$ ls -al /go-working-dir/ total 107804 drwxrwxr-x1 go root 16384 Apr 11 09:42 . drwxr-xr-x1 root root90 Apr 11 09:41 .. lrwxrwxrwx1 go root17 Apr 11 09:41 artifacts -> /godata/artifacts lrwxrwxrwx1 go root14 Apr 11 09:41 bin -> /go-server/bin lrwxrwxrwx1 go root14 Apr 11 09:41 config -> /godata/config -rw-r--r--1 go root 110355285 Apr 11 09:41 cruise.war lrwxrwxrwx1 go root10 Apr 11 09:41 db -> /godata/db drwxr-xr-x 11 go root 141 Apr 11 09:42 felix-cache lrwxrwxrwx1 go root14 Apr 11 09:41 lib -> /go-server/lib lrwxrwxrwx1 go root12 Apr 11 09:41 logs -> /godata/logs lrwxrwxrwx1 go root15 Apr 11 09:41 plugins -> /godata/plugins drwxr-xr-x 11 go root 16384 Apr 11 09:42 plugins_work lrwxrwxrwx1 go root14 Apr 11 09:41 run -> /go-server/run drwxr-xr-x3 go root52 Apr 11 09:41 work lrwxrwxrwx1 go root18 Apr 11 09:41 wrapper -> /go-server/wrapper lrwxrwxrwx1 go root25 Apr 11 09:41 wrapper-config -> /go-server/wrapper-config bash-5.1$ cat /godata/config/cruise-config.xml http://www.w3.org/2001/XMLSchema-instance"; xsi:noNamespaceSchemaLocation="cruise-config.xsd" schemaVersion="139"> artifacts ``` Is that been happening by the entrypoint script as such ? For information, here is my dockerfile: ``` # Use the official GoCD server image from gocd.org as the base FROM gocd/gocd-server:v22.3.0 # Environment variables can be set if needed ENV GO_SERVER_SYSTEM_PROPERTIES="" # Switch to root user to install additional packages USER root # Install additional packages required for your setup RUN apk update && apk add --no-cache \ unzip \ git \ openssh-client \ jq \ ca-certificates \ curl \ zip \ && rm -rf /var/cache/apk/* /var/lib/apt/lists/* # No custom entrypoint script is used, rely on the base image's entrypoint # Switch back to the GoCD user for safety USER go # Volumes where GoCD expects data to persist, you will mount your EFS to these locations VOLUME ["/var/lib/go-server", "/etc/go"] # Expose ports EXPOSE 8153 8154 # The ENTRYPOINT from the base image is used, ensuring GoCD starts up as expected``` Many thanks Satya On Thu, Apr 11, 2024 at 11:11 AM Satya Elipe wrote: > Thank you Chad. > > I see my next issue with that, wherein cruise-config.xml is still not seen > by the container and uses default or generates its own. > > Here is the data on the EFS mount: > ``` > [ec2-user@ip-10-154-168-49 ~]$ ls -al /mnt/gocd_efs > total 108252 > drwxr-xr-x. 15 ec2-user root 6144 Apr 11 09:38 . > drwxr-xr-x. 3 root root22 Apr 10 15:57 .. > drwxr-xr-x. 4 ec2-user root 6144 Dec 16 2020 artifacts > -rwxr-xr-x. 1 root root32 Apr 11 09:38 cipher.aes > -rwxr-xr-x. 1 root root32 Apr 11 09:38 > cipher.aes.16540.2020-12-16@14:08:08~ > drwxr-xr-x. 2 ec2-user root 6144 Apr 11 08:10 config > -rwxr-xr-x. 1 root root 14950 Apr 11 09:38 cruise-config.xml > -rwxr-xr-x. 1 root root 7382 Apr 11 09:38 > cruise-config.xml.1125.2021-01-19@11:20:26~ > -rwxr-xr-x. 1 root root 10460 Apr 11 09:38 > cruise-config.xml.invalid.2021-09-27-16-03-30 > -rwxr-xr-x. 1 root root 47442 Apr 11 09:38 cruise-config.xsd > -rwxr-xr-x. 1 ec2-user root 110355285 Apr 10 09:28 cruise.war > drwxr-xr-x. 3 ec2-user root 6144 Dec 16 2020 data > drwxr-xr-x. 5 ec2-user root 6144 Jul 7 2022 db > drwxr-xr-x. 3 ec2-user root 6144 Apr 10 18:17 etc_go > drwxr-xr-x. 12 ec2-user root 6144 Apr 10 09:29 felix-cache > -rwxr-xr-x. 1 root root 799 Apr 11 09:38 go_update_server.pub > drwxr-xr-x. 3 root root 6144 Apr 11 09:38 godata > -rwxr-xr-x.
Re: [go-cd] Help Needed with GoCD Docker Setup and cruise-config.xml Configuration
ker-gocd-server/blob/master/docker-entrypoint.sh>), > so these things need to be mounted together and consistent. > > I don't think your mounts to /var/lib/go-server or /etc/go will be doing > anything, as these folders are not used for *off-the-shelf* docker server > images, since the server is not installed as an rpm/deb package when > creating a container. You can/should likely remove these. > > In a general sense, all you should need to do is mount a location in EFS > to /godata, similar to what the Helm chart does: > https://github.com/gocd/helm-chart/blob/c734ad2263b1d2885229d00267c428e88f868504/gocd/templates/gocd-server-deployment.yaml#L122-L124 > That will put all logs, artifacts, databases and config on your volume. > Some folks decide to also mount /home/go if they want to use external > storage for things like Git SSH keys or other shell defaults affecting the > *go* user, but that's optional. > > -Chad > > On Thu, Apr 11, 2024 at 5:34 AM Satya Elipe wrote: > >> >> Dear GoCD Support Team, >> >> >> I hope this message finds you well. I am currently encountering an issue >> with setting up GoCD on Docker and specifically with configuring the >> cruise-config.xml file to be recognized correctly in my setup. Despite >> following the official documentation and trying various configurations, >> I've hit a stumbling block that I hope you can help me with. >> >> >> *Issue Summary:* >> >> I have a GoCD server running in a Docker container, and I'm attempting to >> ensure that the cruise-config.xml file is correctly picked up from a >> specified location. My goal is to mount this configuration file from an >> external volume into the GoCD server container so that the server uses this >> configuration instead of the default one. >> >> *Configuration Details:* >> >>- *GoCD Server Image Version:* gocd/gocd-server:v22.3.0 >>- *Docker Version:* Docker version 25.0.3, build 4debf41 >>- *Host Operating System:* Amazon Linux 2023 >> >> *Docker Run Command:* >> >> docker run -d \ >> >> --name gocd-server \ >> >> -p 8153:8153 \ >> >> -p 8154:8154 \ >> >> -v /mnt/gocd_efs:/var/lib/go-server \ >> >> -v >> /mnt/gocd_efs/etc_go/cruise-config.xml:/go-working-dir/config/cruise-config.xml >> \ >> >> -v /mnt/gocd_efs/etc_go:/etc/go \ >> >> custom-gocd-server >> >> >> >> *Issue Encountered:* >> >> When including the -v >> /mnt/gocd_efs/etc_go/cruise-config.xml:/go-working-dir/config/cruise-config.xml >> volume mount, the GoCD server fails to start correctly, with logs >> indicating an inability to create or copy necessary files within >> /go-working-dir/config. >> >> >> Without this mount, the server starts but does not load the desired >> configuration, defaulting instead to the initial configuration without any >> of our pipeline configurations. >> >> >> Log snippet: >> ``` >> jvm 1| 2024-04-10 20:55:17,429 ERROR [Thread-79] >> GoFileConfigDataSource:436 - Unable to load config file: >> /go-working-dir/config/cruise-config.xml The value of 'serverId' uniquely >> identifies a Go server instance. This field cannot be modified. >> >> ``` >> >> >> >> *Questions:* >> >>- Is there a recommended approach to ensure cruise-config.xml is >>correctly recognized and used by the GoCD server when running in Docker? >>- Could this issue be related to how volumes are mounted or >>permissions within the container? >> >> >> Any assistance or insights you could provide on this matter would be >> greatly appreciated. I am happy to provide any further information or logs >> as needed. >> >> Thank you for your time and support. >> >> >> >> Best regards, >> >> Satya >> >> -- >> You received this message because you are subscribed to the Google Groups >> "go-cd" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to go-cd+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/go-cd/CADKEDRrF7KRFP9jNsg631STyokXS1Njqutdshd0ZKdwHMRTy6g%40mail.gmail.com >> <https://groups.google.com/d/msgid/go-cd/CADKEDRrF7KRFP9jNsg631STyokXS1Njqutdshd0ZKdwHMRTy6g%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "go-cd" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to go-cd+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/go-cd/CAA1RwH-%3DFtT%3DDoPvoCekei%2BncFGgC2CSxu7DS7QgZWQ_QH3%3DwQ%40mail.gmail.com > <https://groups.google.com/d/msgid/go-cd/CAA1RwH-%3DFtT%3DDoPvoCekei%2BncFGgC2CSxu7DS7QgZWQ_QH3%3DwQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CADKEDRqsXON0uNvZ3-a1EL1sqYMtroePSP5GguD13pzVsFQhnQ%40mail.gmail.com.
[go-cd] Help Needed with GoCD Docker Setup and cruise-config.xml Configuration
Dear GoCD Support Team, I hope this message finds you well. I am currently encountering an issue with setting up GoCD on Docker and specifically with configuring the cruise-config.xml file to be recognized correctly in my setup. Despite following the official documentation and trying various configurations, I've hit a stumbling block that I hope you can help me with. *Issue Summary:* I have a GoCD server running in a Docker container, and I'm attempting to ensure that the cruise-config.xml file is correctly picked up from a specified location. My goal is to mount this configuration file from an external volume into the GoCD server container so that the server uses this configuration instead of the default one. *Configuration Details:* - *GoCD Server Image Version:* gocd/gocd-server:v22.3.0 - *Docker Version:* Docker version 25.0.3, build 4debf41 - *Host Operating System:* Amazon Linux 2023 *Docker Run Command:* docker run -d \ --name gocd-server \ -p 8153:8153 \ -p 8154:8154 \ -v /mnt/gocd_efs:/var/lib/go-server \ -v /mnt/gocd_efs/etc_go/cruise-config.xml:/go-working-dir/config/cruise-config.xml \ -v /mnt/gocd_efs/etc_go:/etc/go \ custom-gocd-server *Issue Encountered:* When including the -v /mnt/gocd_efs/etc_go/cruise-config.xml:/go-working-dir/config/cruise-config.xml volume mount, the GoCD server fails to start correctly, with logs indicating an inability to create or copy necessary files within /go-working-dir/config. Without this mount, the server starts but does not load the desired configuration, defaulting instead to the initial configuration without any of our pipeline configurations. Log snippet: ``` jvm 1| 2024-04-10 20:55:17,429 ERROR [Thread-79] GoFileConfigDataSource:436 - Unable to load config file: /go-working-dir/config/cruise-config.xml The value of 'serverId' uniquely identifies a Go server instance. This field cannot be modified. ``` *Questions:* - Is there a recommended approach to ensure cruise-config.xml is correctly recognized and used by the GoCD server when running in Docker? - Could this issue be related to how volumes are mounted or permissions within the container? Any assistance or insights you could provide on this matter would be greatly appreciated. I am happy to provide any further information or logs as needed. Thank you for your time and support. Best regards, Satya -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CADKEDRrF7KRFP9jNsg631STyokXS1Njqutdshd0ZKdwHMRTy6g%40mail.gmail.com.
Re: [go-cd] Agent docker image
Thank you Sriram. My comments inline please. Are you planning to run the go server as an ECS instance and The go agent as a ECS instance? Satya] Yes, server will be part of ECS (ECS instance) and it will launch the agents dynamically when theres a job to perform and once the job is done the agent is terminated You should start by understanding the following: - how the containerised server and agent work Satya] If Im not wrong, this goes with docker elastic plugin, configuring cluster and agent profiles, setting up agentAutoRegisterKey in cruise-config.yml and the below configuration in the cruise-config.yml. Agent: this would go into agent container ``` ec2_user_data echo "agent.auto.register.environments=production,sandbox" >> /var/lib/go-agent/config/autoregister.properties ``` So, we should have this file updated with the required details including the agentAutoRegisterKey (as given here <https://docs.gocd.org/current/advanced_usage/agent_auto_register.html> ), so agent would know what server it is working with, correct me if I am wrong please. - auto registration of the agent Satya] as mentioned above. - scaling ECS instances ( for the agent) - addressing a server instance via an ALB - the security group settings needed to enable the agent ECS cluster to connect to the server on port 8153. - using RDS for the server data ( including migrating from the H2DB to RDS) - storing files on EFS and making available to the go server instance ( artifacts, configuration, logs) - sending logs to cloud watch Satya] Yes, mostly set up is the same, but just that whether H2DB or RDS . Thank you Satya On Fri, Nov 10, 2023 at 4:31 PM Sriram Narayanan wrote: > > > On Sat, 11 Nov 2023 at 12:00 AM, Satya Elipe > wrote: > >> Hi All >> >> Any recommendation for go-agent docker image like we have for go-server >> (gocd/go-server:v22.3.0) ? >> >> We are dockerizing a stand-alone g-server runs as an ec2 at the moment >> but would like to move it to ECS. >> >> any inputs will be appreciated. >> > > Are you planning to run the go server as an ECS instance and The go agent > as a ECS instance? > > You should start by understanding the following: > - how the containerised server and agent work > - auto registration of the agent > - scaling ECS instances ( for the agent) > - addressing a server instance via an ALB > - the security group settings needed to enable the agent ECS cluster to > connect to the server on port 8153. > - using RDS for the server data ( including migrating from the H2DB to RDS) > - storing files on EFS and making available to the go server instance ( > artifacts, configuration, logs) > - sending logs to cloud watch > > You could also consider using the Elastic Agent support to use EKS based > agents instead of ECS. See: > https://github.com/gocd/kubernetes-elastic-agents > > — Sriram > > > >> Many thanks >> Satya >> >> -- >> You received this message because you are subscribed to the Google Groups >> "go-cd" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to go-cd+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/go-cd/CADKEDRrGqba1Oi3Wcu7%2BTJEuKMENkWOc-JbMcKb2XG1Px6BJGQ%40mail.gmail.com >> <https://groups.google.com/d/msgid/go-cd/CADKEDRrGqba1Oi3Wcu7%2BTJEuKMENkWOc-JbMcKb2XG1Px6BJGQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "go-cd" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to go-cd+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/go-cd/CANiY96Y8ZwK5fcVeSsySGLXxdLdPNG356_f-BGVWkaa%3DAA%2BX3w%40mail.gmail.com > <https://groups.google.com/d/msgid/go-cd/CANiY96Y8ZwK5fcVeSsySGLXxdLdPNG356_f-BGVWkaa%3DAA%2BX3w%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CADKEDRoydXAcRcy0fC2rQkftuYuuHjF7SJKWDCS21biFWnzUhA%40mail.gmail.com.
[go-cd] Docker Elastic agent plugin .jar location (to run it in the ECS)
Hi All Im containerizing my standalone GoCD server v22.3.0 with elastic agents, and looking for the elastic agents plugin jar file location for the version that works with server version v22.3.0 and use it in the container something like: docker run -d -p 8153:8153 -p 8154:8154 \ -v /etc/go:/etc/go \ -v /var/lib/go-server:/var/lib/go-server \ -e GOCD_PLUGIN_INSTALL_docker-elastic-agents= https://github.com/gocd-contrib/docker-elastic-agents/releases/download/v2.2.0-218/docker-elastic-agents-2.2.0-218.jar \ --name gocd-server \ gocd/gocd-server:v22.3.0 The above command fails with the below error: ```$ mkdir -p /godata/plugins/external $ curl --silent --location --fail --retry 3 https://github.com/gocd-contrib/docker-elastic-agents/releases/download/v2.2.0-218/docker-elastic-agents-2.2.0-218.jar --output /godata/plugins/external/docker-elastic-agents.jar /usr/local/sbin/install-gocd-plugins: cannot curl --silent --location --fail --retry 3 https://github.com/gocd-contrib/docker-elastic-agents/releases/download/v2.2.0-218/docker-elastic-agents-2.2.0-218.jar --output /godata/plugins/external/docker-elastic-agents.jar``` I see the documentation available for the same, my bad that I couldn't locate the jar file: https://github.com/gocd-contrib/docker-elastic-agents-plugin https://github.com/gocd-contrib/docker-elastic-agents-plugin/blob/master/INSTALL.md Any inputs will be of great help. Many thanks Satya [P.S: Right now the server runs as a standalone ec2 on AWS, plan is to run it as part of ECS.] -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CADKEDRpTctJDBtWXSJWoEW81ryUwL7cAzeePbuZd4wi7LDmQFg%40mail.gmail.com.
[go-cd] Agent docker image
Hi All Any recommendation for go-agent docker image like we have for go-server (gocd/go-server:v22.3.0) ? We are dockerizing a stand-alone g-server runs as an ec2 at the moment but would like to move it to ECS. any inputs will be appreciated. Many thanks Satya -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CADKEDRrGqba1Oi3Wcu7%2BTJEuKMENkWOc-JbMcKb2XG1Px6BJGQ%40mail.gmail.com.
Re: [go-cd] Where can I find the end-of-support documentation for GoCD releases?
Thank you Sriram and Chad. On Tue, Nov 7, 2023 at 2:30 PM Chad Wilson wrote: > Other than Ram's (accurate) comments about non-use of semantic versioning, > please see https://github.com/gocd/gocd/blob/master/SECURITY.md > > Upgrading from 22.3.0 through to 23.3.0 or 23.4.0 is likely to be very low > risk/impact given the slow rate of change in GoCD over the last year. > > -Chad > > On Wed, 8 Nov 2023, 00:50 Satya Elipe, wrote: > >> Hi All >> >> We are currently on GoCD version 22.3.0 and wonder when is the right time >> to upgrade, so the documentation for end-of-support dates would be a help >> here, many thanks. >> >> Regards >> Satya >> >> -- >> You received this message because you are subscribed to the Google Groups >> "go-cd" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to go-cd+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/go-cd/CADKEDRr_8jWerqD8enn%2BcJP7AEVCAs%2BMQijXQ7%2BmZARqeV94YQ%40mail.gmail.com >> <https://groups.google.com/d/msgid/go-cd/CADKEDRr_8jWerqD8enn%2BcJP7AEVCAs%2BMQijXQ7%2BmZARqeV94YQ%40mail.gmail.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "go-cd" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to go-cd+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/go-cd/CAA1RwH_dqivHCgNH2LzXMwGLx0q_t3QhvU7Aei7hg-UmtNR7gQ%40mail.gmail.com > <https://groups.google.com/d/msgid/go-cd/CAA1RwH_dqivHCgNH2LzXMwGLx0q_t3QhvU7Aei7hg-UmtNR7gQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CADKEDRrYuHWVzSeQreajSg%2BMsPCK1vOE%2B7DJ%3D0q%2B69t4XcdUNA%40mail.gmail.com.
[go-cd] Where can I find the end-of-support documentation for GoCD releases?
Hi All We are currently on GoCD version 22.3.0 and wonder when is the right time to upgrade, so the documentation for end-of-support dates would be a help here, many thanks. Regards Satya -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CADKEDRr_8jWerqD8enn%2BcJP7AEVCAs%2BMQijXQ7%2BmZARqeV94YQ%40mail.gmail.com.
[go-cd] Re: Dynamic agents are not getting terminated if the job is cancelled
On Wed, Sep 28, 2022 at 1:11 PM Satya Elipe wrote: > Hi All > Would like to seek some inputs here for the issue that Im facing at > the moment with dynamic agents > > Dynamic agents are not getting terminated when the job is cancelled before > completion due to some job/script issues. > > They stay back until I terminate them. > > Wonder if I'm missing something in my configuration ? Any suggestions > please, thank you ... > > Regards > Satya > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CADKEDRq0a%3DVpyCQ63-ui1kjTAcckFLtRU0YHSXHxM7s4vrcWUg%40mail.gmail.com.
[go-cd] Dynamic agents are not getting terminated if the job is cancelled
Hi All Would like to seek some inputs here for the issue that Im facing at the moment with dynamic agents Dynamic agents are not getting terminated when the job is cancelled before completion due to some job/script issues. They stay back until I terminate them. Wonder if I'm missing something in my configuration ? Any suggestions please, thank you ... Regards Satya -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CADKEDRpgbGhvxcwpoGS1BZxkxA%2BtLz_Nk40cRuRzV9%2BEVtc48Q%40mail.gmail.com.
[go-cd] Elastic Profile in GoCD groovy pipeline
Hi All I'm trying to use elasticProfileId in the same way as mentioned in here: ` https://github.com/gocd/pr-pipelines-workflow/blob/master/.gocd/build.gocd.groovy` Here is my code snippet, for some reason it doesn't take any effect and I do not receive any request to launch the instance in the console output that we normally see with elastic profiles. Wondering what's going on and causing this ? any suggestions please?, Many thanks Satya -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CADKEDRoXwM-8ezXzW3YuA43Ked9129dxmNVd1jScN1EjL%3D_5FQ%40mail.gmail.com.
Re: [go-cd] A new elastic agent is created for every 2 mins.
It would be great to have inputs from an author or anyone from https://github.com/iriusrisk/GoCD-EC2-Elastic-Agent-Plugin. @iriusrisk ? Many thanks in advance. Satya On Thu, Jul 14, 2022 at 4:58 PM Satya Elipe wrote: > Also, to add: > > "Check agent status" from the job gives all the details about the agent, > but agents tab shows nothing under "Elastic". > > > On Thu, Jul 14, 2022 at 4:52 PM Satya Elipe wrote: > >> Thanks Aravind. >> >> Yes, I use that plugin. >> >> Attached is the log snippet from /var/log/go-agent/go-agent.log >> Not sure, what does that mean? Agent not able to reach the severer, but >> we are able to see all the information about the agent on the server (from >> the job's agent status). >> >> Let me know if any other logs from the agent will be helpful . Thank you >> >> Best Wishes >> Satya >> >> On Thu, Jul 14, 2022 at 4:25 PM Aravind SV >> wrote: >> >>> Is there any information in the elastic agent logs? Not the GoCD server >>> logs. >>> >>> It looks like the plugin you’re using is: >>> https://github.com/iriusrisk/GoCD-EC2-Elastic-Agent-Plugin >>> >>> The author or someone there might know more. To me, it looks like some >>> kind of a timing issue and the agent not identifying itself properly to the >>> server, if it is able to connect. >>> >>> Cheers, >>> Aravind >>> >>> * From*: Satya Elipe >>> * Subject*: Re: [go-cd] A new elastic agent is created for every 2 mins. >>> * To*: go-cd@googlegroups.com, ketanpadegaon...@gmail.com >>> * Date*: Thu, 14 Jul 2022 12:28:03 +0100 >>> Hi Ketan >>> Thank you for the input. >>> >>> I could ssh into go-agent and look into some more details.. >>> 1. Server configuration from agent pc looks fine and I'm able to wget >>> the server info, attached the screenshot >>> 2. Elastic agent is actually visible on the server/job, attached are the >>> screenshots from the server/job. >>> 3. go-agent is installed and available on the AMI and is running/active >>> on the elastic agent pc; attached the screenshot; >>> 4. agent stderr.out file shows some errors about the cookies but finally >>> was able to set/get that info, attached the screenshot too. >>> >>> I think I'm missing something very obvious hence the agent not getting >>> assigned. >>> Any further inputs will be appreciated. Thank you. >>> >>> Regards >>> Satya >>> >>> >>> >>> >>> >>> On Wed, Jul 13, 2022 at 3:49 PM Ketan Padegaonkar < >>> ketanpadegaon...@gmail.com> wrote: >>> >>>> Most likely cause seems to be that the elastic agent is not registering >>>> with the server within a specified timeout. So the server attempts to >>>> create another elastic agent instance. >>>> >>>> Check if the profile supports a timeout configuration and see if you >>>> are able to increase it. Increasing the timeout might give you enough time >>>> to be able to SSH into the instance to determine what's going on. The >>>> plugin seems to expect that your AMI contains a gocd agent installed, it >>>> then sets up some user data >>>> <https://github.com/iriusrisk/GoCD-EC2-Elastic-Agent-Plugin/blob/92e5efd322ac9ae8bc574706e0503060bd1a9015/src/main/java/com/continuumsecurity/elasticagent/ec2/Ec2Instance.java#L73-L89> >>>> to configure the gocd agent, and start up the agent and wait for it to >>>> connect. >>>> >>>> - Ketan >>>> >>>> On Wed, Jul 13, 2022 at 7:55 PM Satya Elipe >>>> wrote: >>>> >>>>> Hi All >>>>> >>>>> I see that, for every 2 mins a new elastic agent is created in >>>>> contrast to my understanding that a single elastic agent comes up and >>>>> finishes the job and vanishes after the job is finished. >>>>> >>>>> Did I miss something? >>>>> >>>>> Many thanks in advance. >>>>> Satya >>>>> >>>>> -- >>>>> You received this message because you are subscribed to the Google >>>>> Groups "go-cd" group. >>>>> To unsubscribe from this group and stop receiving emails from it, send >>>>> an email to go-cd+unsubscr...@googlegroups.com. >>>>> To view this discussion on the we
Re: [go-cd] Associate job with an elastic agents
I see that definition in here, will try that out and let you know, thank you https://docs.gocd.org/current/gocd_on_kubernetes/sample_pipelines_explained.html Regards Satya On Fri, Jul 8, 2022 at 12:24 AM Satya Elipe wrote: > Hi All > > Wonder how can we associate a pipeline (as a code) to the elastic profile > id. > > "Elastic Agent Profile ID" is disabled in the job as seen in the > `disable-in-the-job.png` attachment and that makes sense to me as thats not > defined in the pipeline-as-code as shown in the code.png attachment. > > How can I enable it in the code ? Im looking for the syntax too please. > > Many thanks > Satya > > > -- > You received this message because you are subscribed to the Google Groups > "go-cd" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to go-cd+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/go-cd/2f6fb00d-f36c-4809-911b-3e403992e7cfn%40googlegroups.com > <https://groups.google.com/d/msgid/go-cd/2f6fb00d-f36c-4809-911b-3e403992e7cfn%40googlegroups.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CADKEDRquWxmtXCL3TLO_oK7iDj97Uc3Pbck-4XtZ-stKVAtx_w%40mail.gmail.com.
Re: [go-cd] Where is the jobs/pipelines information saved on the GoCD server ?
Thank you Chad, that was helpful and I could the info that I was looking for. On Thursday, July 7, 2022 at 3:04:59 PM UTC+1 Chad Wilson wrote: > The configuration for the server, plugins, along with all your > pipelines/jobs etc is stored in a single big config.xml (and versioned > controlled internally) - see reference at > https://docs.gocd.org/current/configuration/configuration_reference.html > or you can view it as an admin at /go/admin/config_xml. Storage location > differs by installation, see the docs > <https://docs.gocd.org/current/installation/installing_go_server.html> > for whatever your platform is. > >- Some of this configuration for pipelines and logical environments >can be externalised into your own source control repositories (pipelines >as code ><https://docs.gocd.org/current/advanced_usage/pipelines_as_code.html>) >as YAML, JSON or Groovy via plugins, and GoCD will take care of >synchronising this/merging it into the config.xml as it is updated. >- As for metadata about jobs/pipelines, that is stored inside a >database (on-disk H2 Database by default, PostgreSQL and MySQL as options). >- Artifacts and job run logs etc are stored on disk > > -Chad > > On Thu, Jul 7, 2022 at 9:14 PM Satya Elipe wrote: > >> Hi All >> >> Wodering whether the jobs/pipelines information saved on the server as an >> xml/json etc ? >> >> Many thanks >> Satya >> >> -- >> You received this message because you are subscribed to the Google Groups >> "go-cd" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to go-cd+un...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/go-cd/3fb9cbd1-5b65-4d67-839e-fb3412f5c392n%40googlegroups.com >> >> <https://groups.google.com/d/msgid/go-cd/3fb9cbd1-5b65-4d67-839e-fb3412f5c392n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/9f716524-0227-46bf-81d6-e8d3943e21cen%40googlegroups.com.
[go-cd] Where is the jobs/pipelines information saved on the GoCD server ?
Hi All Wodering whether the jobs/pipelines information saved on the server as an xml/json etc ? Many thanks Satya -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/3fb9cbd1-5b65-4d67-839e-fb3412f5c392n%40googlegroups.com.
Re: [go-cd] Configure "Elastic agents" on standalone Go server (v21.1.0)
Thank you Aravind, thats more clear now. On Mon, Jul 4, 2022 at 4:21 PM Aravind SV wrote: > I've updated > https://github.com/gocd/kubernetes-elastic-agents/blob/master/docs/install.md > to make it more clear where the plugin JAR file can be downloaded from. > > Cheers, > Aravind > > On Sat, Jul 2, 2022 at 10:15 PM Satya Elipe wrote: > >> Hi Chad >> Thank you for the input, appreciate it. >> So, as I mentioned, my server is neither dockerized nor running as a pod >> in the cluster, it's just an amzn2 box running on its own. >> So, I think the second approach in your email works for me; but I just >> wonder, where can I find this jar ( >> build/libs/kubernetes-elastic-agent-plugin-VERSION.jar) to download onto >> amzn2 ? (Ii think I'm terribly missing something finding it ) >> >> >> [>>You'd then need to follow >> https://github.com/gocd/kubernetes-elastic-agents/blob/master/docs/install.md >> to configure a cluster profile and 1+ elastic profiles, and then change >> your pipeline jobs to tell them to use a particular elastic profile ID to >> essentially tell them which agent container image to run, and what k8s >> resources the agents need.] >> >> We are also planning to dockerize the GoCD+Agents soon, then I think >> the better approach would be the first point in your mail. >> [I would consider deploying a GoCD server as a pod and then elastic >> agents will be out of the box available and will be ready with the required >> configuration, I guess. ] >> >> Thanks Again >> Satya >> >> On Fri, Jul 1, 2022 at 5:17 PM Chad Wilson >> wrote: >> >>> Hi Satya >>> >>> If you are deploying GoCD into Kubernetes itself via the Helm chart >>> <https://github.com/gocd/helm-chart/blob/master/gocd/README.md> into >>> your cluster without overriding the plugin configuration the "Kubernetes >>> elastic agents" plugin is installed out-of-the-box, if your intent is to >>> run the agents as ephemeral containers inside a cluster. >>> >>> You'd then need to follow >>> https://github.com/gocd/kubernetes-elastic-agents/blob/master/docs/install.md >>> to configure a cluster profile and 1+ elastic profiles, and then change >>> your pipeline jobs to tell them to use a particular elastic profile ID to >>> essentially tell them which agent container image to run, and what k8s >>> resources the agents need. >>> >>> -Chad >>> >>> On Fri, Jul 1, 2022 at 11:53 PM Satya Elipe >>> wrote: >>> >>>> Hi All >>>> >>>> I would like to know how can we configure elastic agents for a >>>> standalone Go server which is used to CI/CD for our k8s cluster ? >>>> >>>> Its been brought up long back with two static agents configured to it. >>>> As they are static they always run hence need guidence to turn the agents >>>> into elastic so to avoid idle costs. >>>> >>>> Let me know if more information is required. >>>> >>>> Thank you >>>> Satya >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "go-cd" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to go-cd+unsubscr...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/go-cd/e4a5098c-ed26-45e5-ac8c-d0ec5251da3dn%40googlegroups.com >>>> <https://groups.google.com/d/msgid/go-cd/e4a5098c-ed26-45e5-ac8c-d0ec5251da3dn%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >>> You received this message because you are subscribed to the Google >>> Groups "go-cd" group. >>> To unsubscribe from this group and stop receiving emails from it, send >>> an email to go-cd+unsubscr...@googlegroups.com. >>> To view this discussion on the web visit >>> https://groups.google.com/d/msgid/go-cd/CAA1RwH_vL3YW%3DGursKVBPC7E1Oz3b%2BOSdmfLX7L8FKYhzzz2TA%40mail.gmail.com >>> <https://groups.google.com/d/msgid/go-cd/CAA1RwH_vL3YW%3DGursKVBPC7E1Oz3b%2BOSdmfLX7L8FKYhzzz2TA%40mail.gmail.com?utm_medium=email&utm_source=footer> >>> . >>> >> -- >> You received this message because you are subscribed to the Google Groups >> "go-cd" group. >> To unsubscribe from this group and stop receiving emails from
Re: [go-cd] Configure "Elastic agents" on standalone Go server (v21.1.0)
Hi Chad Thank you for the input, appreciate it. So, as I mentioned, my server is neither dockerized nor running as a pod in the cluster, it's just an amzn2 box running on its own. So, I think the second approach in your email works for me; but I just wonder, where can I find this jar ( build/libs/kubernetes-elastic-agent-plugin-VERSION.jar) to download onto amzn2 ? (Ii think I'm terribly missing something finding it ) [>>You'd then need to follow https://github.com/gocd/kubernetes-elastic-agents/blob/master/docs/install.md to configure a cluster profile and 1+ elastic profiles, and then change your pipeline jobs to tell them to use a particular elastic profile ID to essentially tell them which agent container image to run, and what k8s resources the agents need.] We are also planning to dockerize the GoCD+Agents soon, then I think the better approach would be the first point in your mail. [I would consider deploying a GoCD server as a pod and then elastic agents will be out of the box available and will be ready with the required configuration, I guess. ] Thanks Again Satya On Fri, Jul 1, 2022 at 5:17 PM Chad Wilson wrote: > Hi Satya > > If you are deploying GoCD into Kubernetes itself via the Helm chart > <https://github.com/gocd/helm-chart/blob/master/gocd/README.md> into your > cluster without overriding the plugin configuration the "Kubernetes elastic > agents" plugin is installed out-of-the-box, if your intent is to run the > agents as ephemeral containers inside a cluster. > > You'd then need to follow > https://github.com/gocd/kubernetes-elastic-agents/blob/master/docs/install.md > to configure a cluster profile and 1+ elastic profiles, and then change > your pipeline jobs to tell them to use a particular elastic profile ID to > essentially tell them which agent container image to run, and what k8s > resources the agents need. > > -Chad > > On Fri, Jul 1, 2022 at 11:53 PM Satya Elipe wrote: > >> Hi All >> >> I would like to know how can we configure elastic agents for a standalone >> Go server which is used to CI/CD for our k8s cluster ? >> >> Its been brought up long back with two static agents configured to it. As >> they are static they always run hence need guidence to turn the agents into >> elastic so to avoid idle costs. >> >> Let me know if more information is required. >> >> Thank you >> Satya >> >> -- >> You received this message because you are subscribed to the Google Groups >> "go-cd" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to go-cd+unsubscr...@googlegroups.com. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/go-cd/e4a5098c-ed26-45e5-ac8c-d0ec5251da3dn%40googlegroups.com >> <https://groups.google.com/d/msgid/go-cd/e4a5098c-ed26-45e5-ac8c-d0ec5251da3dn%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "go-cd" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to go-cd+unsubscr...@googlegroups.com. > To view this discussion on the web visit > https://groups.google.com/d/msgid/go-cd/CAA1RwH_vL3YW%3DGursKVBPC7E1Oz3b%2BOSdmfLX7L8FKYhzzz2TA%40mail.gmail.com > <https://groups.google.com/d/msgid/go-cd/CAA1RwH_vL3YW%3DGursKVBPC7E1Oz3b%2BOSdmfLX7L8FKYhzzz2TA%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CADKEDRqB13wrvijiFYX0ma9e_JJse6M0Xhy57-c8tdKm154LsQ%40mail.gmail.com.
[go-cd] Configure "Elastic agents" on standalone Go server (v21.1.0)
Hi All I would like to know how can we configure elastic agents for a standalone Go server which is used to CI/CD for our k8s cluster ? Its been brought up long back with two static agents configured to it. As they are static they always run hence need guidence to turn the agents into elastic so to avoid idle costs. Let me know if more information is required. Thank you Satya -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/e4a5098c-ed26-45e5-ac8c-d0ec5251da3dn%40googlegroups.com.
