Re: [go-cd] Performance of popups in the gui

[Chad Wilson]

Great to hear. Really appreciate the feedback!

On Thu, 23 May 2024, 15:39 'Hans Dampf' via go-cd, 
wrote:

> So we have been running the new version for a couple of days now, and I
> can't see any further problems with it. The performance is great with >1000
> Pipelines now.
> It is a really great improvement.
>
> Wolfgang Achinger schrieb am Montag, 13. Mai 2024 um 10:30:42 UTC+2:
>
>> I'm currently in the process of upgrading the server and agents of all
>> our gocd setups. We will monitor it the next few days and I will come back
>> if we notice anything
>>
>> Am Mo., 13. Mai 2024 um 10:24 Uhr schrieb Chad Wilson <
>> ch...@thoughtworks.com>:
>>
>>> Great to hear - back to how it was *supposed* to behave! I hope it
>>> hasn't caused any other regressions 
>>>
>>> Now that this is cleaned up, let me know if it exposes any other
>>> unexpected weird slowness you can't get to the bottom of and I'll see if I
>>> can chip away at the other various niggles.
>>>
>>> -Chad
>>>
>>> On Mon, May 13, 2024 at 4:10 PM 'Wolfgang Achinger' via go-cd <
>>> go...@googlegroups.com> wrote:
>>>
>>>> Dude the fix is amazing !
>>>>
>>>> Am Mo., 13. Mai 2024 um 04:05 Uhr schrieb Chad Wilson <
>>>> ch...@thoughtworks.com>:
>>>>
>>>>> Apologies for the slow release of this (been rather busy personally)
>>>>> but 24.1.0 is out with what I think should be a fix for this issue.
>>>>>
>>>>> If you have any feedback it'd be appreciated.
>>>>>
>>>>> -Chad
>>>>>
>>>>>
>>>>> On Mon, 19 Feb 2024, 15:15 'Wolfgang Achinger' via go-cd, <
>>>>> go...@googlegroups.com> wrote:
>>>>>
>>>>>> Hello,
>>>>>>
>>>>>> this is actually incredible news for us. We will look forward to the
>>>>>> release with the fix.
>>>>>> Thanks for the support.
>>>>>>
>>>>>> The workarounds seem not to be viable for us, since we use the
>>>>>> environment for a lot of global variables and customizations.
>>>>>>
>>>>>> Regards
>>>>>>
>>>>>> Am Sa., 17. Feb. 2024 um 18:08 Uhr schrieb Chad Wilson <
>>>>>> ch...@thoughtworks.com>:
>>>>>>
>>>>>>> Hiya folks
>>>>>>>
>>>>>>> I've been able to replicate this problem and should be able to fix
>>>>>>> it for a subsequent release - thanks for the help debugging.
>>>>>>>
>>>>>>> The problem appears to arrive when there is a large number of
>>>>>>> pipelines* mapped to an environment;* and also a large number of
>>>>>>> agents for that environment. The logic for calculating the API response
>>>>>>> agents > environments is accidentally very, very inefficient (I think 
>>>>>>> it's
>>>>>>> O(n^2 x m^2) or something crazy. I replicated something similar to what 
>>>>>>> you
>>>>>>> describe with 5,000 pipelines and 60 or so agents, all mapped into the
>>>>>>> same, single, logical environment.
>>>>>>>
>>>>>>> [image: image.png]
>>>>>>>
>>>>>>>
>>>>>>> In your case if you have all say 1,690 pipelines mapped to a single
>>>>>>> environment (from your stats below), and all of your 95 agents are in 
>>>>>>> the
>>>>>>> same environment, you'd definitely trigger this issue. I can't tell 
>>>>>>> exactly
>>>>>>> from what you have shared how the pipelines and agents are mapped to
>>>>>>> environments, so this is a guess - can you confirm how many agents and
>>>>>>> pipelines are mapped to the environment below?
>>>>>>>
>>>>>>> "Number of pipelines": 1690,
>>>>>>> "Number of environments": 1,
>>>>>>> "Number of agents": 95,
>>>>>>>
>>>>>>>
>>>>>>> If it's the same problem, you will probably find that *untagging
>>>>>>> the agents from the environment *also has a similar speed-up effect
>>>>>>> to deleting all o

Re: [go-cd] GoCD Agent DinD Arm64 architecture

[Chad Wilson]

Right - this sounds like a reasonable use case :-)

On Thu, May 23, 2024 at 2:34 PM Komgrit Aneksri 
wrote:

> Dear Chad,
>
> Thank you so much again for your suggestions.
>
> As for question about Why I have to use GoCD agent with DinD?
>
> I use it because there are many build from source types.
>
> *Example my use case.*
> Ex1. If source code is Terraform, We will pull Terraform image for plan,
> apply,... infrastructure from terraform source code.
>
> Ex2. If source code python, node, ..., We will pull that programming
> languages  before run build the source code per that required from
> programming language.
>
> and also vulnerability scanning (sonarqube, dependency checker, OSV
> scanner, ..) in our pipelines.
>
> Best Regards,
> Komgrit
>
> On Sunday, May 19, 2024 at 7:21:41 PM UTC+7 Chad Wilson wrote:
>
>> No such thing is available right now - see
>> https://github.com/gocd/gocd/issues/11355
>>
>> The blocker right now is a sequence of issues:
>> 1) Official Docker DIND images are all *Alpine-based*. GoCD extends
>> these.
>> 2) Alpine is *supposed to be* only used with musl libc, (not glibc,
>> which software is often built for).
>> 3) GoCD uses wrapping native software called the *Tanuki Service Wrapper
>> which only supports glibc.*
>> 4) The hacky Alpine glibc package GoCD uses to make Tanuki and Java work
>> together on Alpine
>> <https://github.com/gocd/docker-gocd-agent-docker-dind/blob/master/Dockerfile>
>> is only *available for amd64*. Building it custom for aarch64 looks
>> painful, and I don't want to support it, as it's a bad solution already.
>>
>> The Tanuki folks have told me that they are working on a version for
>> Alpine/musl libc (both x64 and aarch64) which is the easiest way to resolve
>> this soon. However there is no defined release date.
>>
>> Do you actually *need* docker-in-docker behaviour that relies on this
>> particular agent image? I have found some people use the DIND image when
>> they actually do not need to. You'd need to upgrade your GoCD, but there
>> are aarch64 containers for every other platform except Alpine now.
>>
>> -Chad
>>
>> On Sun, May 19, 2024 at 7:46 PM Komgrit Aneksri 
>> wrote:
>>
>>> Dear GoCD Team,
>>>
>>> I am current using GoCD Server v23.1.0 and GoCD Agent DinD v22.1.0 amd64.
>>>
>>> I am looking for GoCD Agent Dind arm64 acrh version.
>>>
>>> Do we have GoCD Agent DinD Arm64 version support?
>>>
>>> or Have you any source for build it?
>>>
>>> Please suggest me.
>>>
>>> Best Regards,
>>> Komgrit
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+un...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/87bced9a-3411-47e8-a559-6cd40e9ad1dan%40googlegroups.com
>>> <https://groups.google.com/d/msgid/go-cd/87bced9a-3411-47e8-a559-6cd40e9ad1dan%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/0d5d5536-a869-4b11-a0bf-7d6ed9ff5dafn%40googlegroups.com
> <https://groups.google.com/d/msgid/go-cd/0d5d5536-a869-4b11-a0bf-7d6ed9ff5dafn%40googlegroups.com?utm_medium=email_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8r_28r2jo8d-Pfa6QE0KXN_1xkuPMbo9AUxYJB_Bi0%3Dg%40mail.gmail.com.

Re: [go-cd] GoCD Agent DinD Arm64 architecture

[Chad Wilson]

No such thing is available right now - see
https://github.com/gocd/gocd/issues/11355

The blocker right now is a sequence of issues:
1) Official Docker DIND images are all *Alpine-based*. GoCD extends these.
2) Alpine is *supposed to be* only used with musl libc, (not glibc, which
software is often built for).
3) GoCD uses wrapping native software called the *Tanuki Service Wrapper
which only supports glibc.*
4) The hacky Alpine glibc package GoCD uses to make Tanuki and Java work
together on Alpine

is only *available for amd64*. Building it custom for aarch64 looks
painful, and I don't want to support it, as it's a bad solution already.

The Tanuki folks have told me that they are working on a version for
Alpine/musl libc (both x64 and aarch64) which is the easiest way to resolve
this soon. However there is no defined release date.

Do you actually *need* docker-in-docker behaviour that relies on this
particular agent image? I have found some people use the DIND image when
they actually do not need to. You'd need to upgrade your GoCD, but there
are aarch64 containers for every other platform except Alpine now.

-Chad

On Sun, May 19, 2024 at 7:46 PM Komgrit Aneksri 
wrote:

> Dear GoCD Team,
>
> I am current using GoCD Server v23.1.0 and GoCD Agent DinD v22.1.0 amd64.
>
> I am looking for GoCD Agent Dind arm64 acrh version.
>
> Do we have GoCD Agent DinD Arm64 version support?
>
> or Have you any source for build it?
>
> Please suggest me.
>
> Best Regards,
> Komgrit
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/87bced9a-3411-47e8-a559-6cd40e9ad1dan%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8TLth%2BrGr17MUdEG_Pgu9YZZsu-_PLwRr%3DV%3DWH3Wry%2BA%40mail.gmail.com.

Re: [go-cd] How to Configure Redundant LDAP Authorization?

[Chad Wilson]

Minor correction, it's possible the LDAP authz plugin is validating the
certs in a way that the authentication plugin does not, despite both being
on old LDAP client API versions. Would need to dig deeper to validate.

https://github.com/gocd/gocd-ldap-authentication-plugin/blob/a0236ed52cc95646f30c72a20360893c548c/src/main/java/cd/go/apacheds/ConnectionConfiguration.java

Vs

https://github.com/gocd/gocd-ldap-authorization-plugin/blob/master/src/main/java/com/thoughtworks/gocd/authorization/ldap/apacheds/ConnectionConfiguration.java

-Chad

On Sat, 18 May 2024, 11:03 Chad Wilson,  wrote:

> I discovered recently that the plugins are on an ancient version of the
> Apache LDAP library that means they don't actually seem to validate the
> server certs fully by default (e.g on expiry), so may not validate the
> hostname either. But that's probably a bug, not a feature? Your call if you
> want to give it a go and rely on it for now.
>
> If the default is made more secure in future I'd like to think there's be
> an opt-out. But yeah - those LDAP plugins need some love.
>
> -Chad
>
> On Sat, 18 May 2024, 02:29 Jason Smyth,  wrote:
>
>> Hi Sriram,
>>
>>
>>
>> Thank you for the feedback.
>>
>>
>>
>> Do you know how the plugin handles SSL negotiation? We considered DNS
>> round-robin but ruled it a non-starter, under the assumption that LDAPS
>> would require that the hostname and certificate name match.
>>
>>
>>
>> Regards,
>>
>> *Jason Smyth*
>>
>>
>>
>> *From:* go-cd@googlegroups.com  *On Behalf Of *Sriram
>> Narayanan
>> *Sent:* Friday, May 17, 2024 12:46 PM
>> *To:* go-cd@googlegroups.com
>> *Subject:* Re: [go-cd] How to Configure Redundant LDAP Authorization?
>>
>>
>>
>>
>>
>>
>>
>> On Fri, 17 May 2024 at 10:53 PM, Jason Smyth  wrote:
>>
>> Hi everyone,
>>
>>
>>
>> We are looking to move from the bundled LDAP authentication plugin to the 
>> LDAP
>> authorization plugin
>> <https://github.com/gocd/gocd-ldap-authorization-plugin>.
>>
>>
>>
>> For redundancy, our current setup uses 2 LDAP connectors, each pointing
>> to a different Active Directory domain controller in the same domain.  If
>> we switch to the LDAP authorization plugin we can still create redundant
>> authentication links, but does this mean we will need to create duplicate
>> role configurations as well?
>>
>>
>>
>> Is there any documentation we should be referencing in terms of the
>> "right" way to set up a redundant connection to AD?
>>
>>
>>
>>
>>
>> For connection redundancy, I’ve used TCP load balancers. For on premise
>> setups, I’ve used DNS round robin to point to two different load balancer
>> instances.
>>
>>
>>
>>
>>
>>
>>
>> Any feedback is appreciated.
>>
>>
>>
>> Regards,
>>
>> Jason Smyth
>>
>>
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "go-cd" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/4b37a890-f442-4966-a053-0fb985f73e3cn%40googlegroups.com
>> <https://groups.google.com/d/msgid/go-cd/4b37a890-f442-4966-a053-0fb985f73e3cn%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "go-cd" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/go-cd/eEHCCj-vOuo/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/CANiY96ZECHfCOjUw5f-XS6kvsChV%2B8K%3Dry21%3DW%3DeOuFM011opw%40mail.gmail.com
>> <https://groups.google.com/d/msgid/go-cd/CANiY96ZECHfCOjUw5f-XS6kvsChV%2B8K%3Dry21%3DW%3DeOuFM011opw%40mail.gmail.com?utm_medium=email_source=footer>
>> .
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "go-cd" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/DM6PR16MB36713451AFA69A524EC80AB6CFEE2%40DM6PR16MB3671.namprd16.prod.outlook.com
>> <https://groups.google.com/d/msgid/go-cd/DM6PR16MB36713451AFA69A524EC80AB6CFEE2%40DM6PR16MB3671.namprd16.prod.outlook.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8AuQbTdpa37pBi3TaRGTa%3DKH%2Bq2X2UxKeOE3DazpdRSA%40mail.gmail.com.

Re: [go-cd] How to Configure Redundant LDAP Authorization?

[Chad Wilson]

I discovered recently that the plugins are on an ancient version of the
Apache LDAP library that means they don't actually seem to validate the
server certs fully by default (e.g on expiry), so may not validate the
hostname either. But that's probably a bug, not a feature? Your call if you
want to give it a go and rely on it for now.

If the default is made more secure in future I'd like to think there's be
an opt-out. But yeah - those LDAP plugins need some love.

-Chad

On Sat, 18 May 2024, 02:29 Jason Smyth,  wrote:

> Hi Sriram,
>
>
>
> Thank you for the feedback.
>
>
>
> Do you know how the plugin handles SSL negotiation? We considered DNS
> round-robin but ruled it a non-starter, under the assumption that LDAPS
> would require that the hostname and certificate name match.
>
>
>
> Regards,
>
> *Jason Smyth*
>
>
>
> *From:* go-cd@googlegroups.com  *On Behalf Of *Sriram
> Narayanan
> *Sent:* Friday, May 17, 2024 12:46 PM
> *To:* go-cd@googlegroups.com
> *Subject:* Re: [go-cd] How to Configure Redundant LDAP Authorization?
>
>
>
>
>
>
>
> On Fri, 17 May 2024 at 10:53 PM, Jason Smyth  wrote:
>
> Hi everyone,
>
>
>
> We are looking to move from the bundled LDAP authentication plugin to the LDAP
> authorization plugin
> .
>
>
>
> For redundancy, our current setup uses 2 LDAP connectors, each pointing to
> a different Active Directory domain controller in the same domain.  If we
> switch to the LDAP authorization plugin we can still create redundant
> authentication links, but does this mean we will need to create duplicate
> role configurations as well?
>
>
>
> Is there any documentation we should be referencing in terms of the
> "right" way to set up a redundant connection to AD?
>
>
>
>
>
> For connection redundancy, I’ve used TCP load balancers. For on premise
> setups, I’ve used DNS round robin to point to two different load balancer
> instances.
>
>
>
>
>
>
>
> Any feedback is appreciated.
>
>
>
> Regards,
>
> Jason Smyth
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/4b37a890-f442-4966-a053-0fb985f73e3cn%40googlegroups.com
> 
> .
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "go-cd" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/go-cd/eEHCCj-vOuo/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CANiY96ZECHfCOjUw5f-XS6kvsChV%2B8K%3Dry21%3DW%3DeOuFM011opw%40mail.gmail.com
> 
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/DM6PR16MB36713451AFA69A524EC80AB6CFEE2%40DM6PR16MB3671.namprd16.prod.outlook.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-R5GecTU1FUN5ORhbcK1KXXKtwDAaxY%2B4686PTEp5qpQ%40mail.gmail.com.

Re: [go-cd] Performance of popups in the gui

[Chad Wilson]

Great to hear - back to how it was *supposed* to behave! I hope it hasn't
caused any other regressions 

Now that this is cleaned up, let me know if it exposes any other unexpected
weird slowness you can't get to the bottom of and I'll see if I can chip
away at the other various niggles.

-Chad

On Mon, May 13, 2024 at 4:10 PM 'Wolfgang Achinger' via go-cd <
go-cd@googlegroups.com> wrote:

> Dude the fix is amazing !
>
> Am Mo., 13. Mai 2024 um 04:05 Uhr schrieb Chad Wilson <
> ch...@thoughtworks.com>:
>
>> Apologies for the slow release of this (been rather busy personally) but
>> 24.1.0 is out with what I think should be a fix for this issue.
>>
>> If you have any feedback it'd be appreciated.
>>
>> -Chad
>>
>>
>> On Mon, 19 Feb 2024, 15:15 'Wolfgang Achinger' via go-cd, <
>> go-cd@googlegroups.com> wrote:
>>
>>> Hello,
>>>
>>> this is actually incredible news for us. We will look forward to the
>>> release with the fix.
>>> Thanks for the support.
>>>
>>> The workarounds seem not to be viable for us, since we use the
>>> environment for a lot of global variables and customizations.
>>>
>>> Regards
>>>
>>> Am Sa., 17. Feb. 2024 um 18:08 Uhr schrieb Chad Wilson <
>>> ch...@thoughtworks.com>:
>>>
>>>> Hiya folks
>>>>
>>>> I've been able to replicate this problem and should be able to fix it
>>>> for a subsequent release - thanks for the help debugging.
>>>>
>>>> The problem appears to arrive when there is a large number of pipelines*
>>>> mapped to an environment;* and also a large number of agents for that
>>>> environment. The logic for calculating the API response agents >
>>>> environments is accidentally very, very inefficient (I think it's O(n^2 x
>>>> m^2) or something crazy. I replicated something similar to what you
>>>> describe with 5,000 pipelines and 60 or so agents, all mapped into the
>>>> same, single, logical environment.
>>>>
>>>> [image: image.png]
>>>>
>>>>
>>>> In your case if you have all say 1,690 pipelines mapped to a single
>>>> environment (from your stats below), and all of your 95 agents are in the
>>>> same environment, you'd definitely trigger this issue. I can't tell exactly
>>>> from what you have shared how the pipelines and agents are mapped to
>>>> environments, so this is a guess - can you confirm how many agents and
>>>> pipelines are mapped to the environment below?
>>>>
>>>> "Number of pipelines": 1690,
>>>> "Number of environments": 1,
>>>> "Number of agents": 95,
>>>>
>>>>
>>>> If it's the same problem, you will probably find that *untagging the
>>>> agents from the environment *also has a similar speed-up effect to
>>>> deleting all of the agents (although then the pipelines requiring that
>>>> environment won't schedule either, obviously).
>>>>
>>>> Another workaround in the meantime, *if you don't rely on the
>>>> environment*
>>>>
>>>>- to define environment variables/secure environment variables that
>>>>apply across all pipelines/jobs
>>>>- to affect whether jobs are scheduled to special agents
>>>>
>>>> ... may be to untag all pipelines and agents from the environment you
>>>> use and just use the default/empty environment.
>>>>
>>>> -Chad
>>>>
>>>> On Fri, Feb 16, 2024 at 3:40 PM 'Wolfgang Achinger' via go-cd <
>>>> go-cd@googlegroups.com> wrote:
>>>>
>>>>> > By 'resources' I am referring to the GoCD functionality where you
>>>>> can tag agents with resources that they offer, which are then matched to
>>>>> pipeline jobs that say they *require* those resources to run as part
>>>>> of agent assignment.
>>>>> 10 Agents have 5 resources attached
>>>>> 85 have 1 resource attached
>>>>>
>>>>> We use the resources to different special agents. They do the same as
>>>>> the rest, but they are placed in dedicated networks.
>>>>>
>>>>> > To make sure I understand you, are you saying that the problem has
>>>>> been here for the last year, perhaps gradually getting worse a story add
>>>>> more agents or pipelines - but n

Re: [go-cd] Performance of popups in the gui

[Chad Wilson]

Apologies for the slow release of this (been rather busy personally) but
24.1.0 is out with what I think should be a fix for this issue.

If you have any feedback it'd be appreciated.

-Chad


On Mon, 19 Feb 2024, 15:15 'Wolfgang Achinger' via go-cd, <
go-cd@googlegroups.com> wrote:

> Hello,
>
> this is actually incredible news for us. We will look forward to the
> release with the fix.
> Thanks for the support.
>
> The workarounds seem not to be viable for us, since we use the environment
> for a lot of global variables and customizations.
>
> Regards
>
> Am Sa., 17. Feb. 2024 um 18:08 Uhr schrieb Chad Wilson <
> ch...@thoughtworks.com>:
>
>> Hiya folks
>>
>> I've been able to replicate this problem and should be able to fix it for
>> a subsequent release - thanks for the help debugging.
>>
>> The problem appears to arrive when there is a large number of pipelines*
>> mapped to an environment;* and also a large number of agents for that
>> environment. The logic for calculating the API response agents >
>> environments is accidentally very, very inefficient (I think it's O(n^2 x
>> m^2) or something crazy. I replicated something similar to what you
>> describe with 5,000 pipelines and 60 or so agents, all mapped into the
>> same, single, logical environment.
>>
>> [image: image.png]
>>
>>
>> In your case if you have all say 1,690 pipelines mapped to a single
>> environment (from your stats below), and all of your 95 agents are in the
>> same environment, you'd definitely trigger this issue. I can't tell exactly
>> from what you have shared how the pipelines and agents are mapped to
>> environments, so this is a guess - can you confirm how many agents and
>> pipelines are mapped to the environment below?
>>
>> "Number of pipelines": 1690,
>> "Number of environments": 1,
>> "Number of agents": 95,
>>
>>
>> If it's the same problem, you will probably find that *untagging the
>> agents from the environment *also has a similar speed-up effect to
>> deleting all of the agents (although then the pipelines requiring that
>> environment won't schedule either, obviously).
>>
>> Another workaround in the meantime, *if you don't rely on the
>> environment*
>>
>>- to define environment variables/secure environment variables that
>>apply across all pipelines/jobs
>>- to affect whether jobs are scheduled to special agents
>>
>> ... may be to untag all pipelines and agents from the environment you use
>> and just use the default/empty environment.
>>
>> -Chad
>>
>> On Fri, Feb 16, 2024 at 3:40 PM 'Wolfgang Achinger' via go-cd <
>> go-cd@googlegroups.com> wrote:
>>
>>> > By 'resources' I am referring to the GoCD functionality where you can
>>> tag agents with resources that they offer, which are then matched to
>>> pipeline jobs that say they *require* those resources to run as part of
>>> agent assignment.
>>> 10 Agents have 5 resources attached
>>> 85 have 1 resource attached
>>>
>>> We use the resources to different special agents. They do the same as
>>> the rest, but they are placed in dedicated networks.
>>>
>>> > To make sure I understand you, are you saying that the problem has
>>> been here for the last year, perhaps gradually getting worse a story add
>>> more agents or pipelines - but not an issue suddenly created after a
>>> particular upgrade or change?
>>> That's correct. It's more an over-time issue than a sudden issue.
>>>
>>> I sent the additional information out, but not directly, they come from
>>> a different mail address over a secure transfer method.
>>>
>>> Am Do., 15. Feb. 2024 um 17:57 Uhr schrieb Chad Wilson <
>>> ch...@thoughtworks.com>:
>>>
>>>> Cool, thanks! Just trying to gather enough information to see if I can
>>>> replicate or find the issue in a dedicated chunk of time this weekend.
>>>>
>>>> You can email it to me, and/or encrypt with my GPG key if you'd like (
>>>> https://github.com/chadlwilson/chadlwilson/blob/main/gpg-public-key.asc
>>>> )
>>>>
>>>> By 'resources' I am referring to the GoCD functionality where you can
>>>> tag agents with resources that they offer, which are then matched to
>>>> pipeline jobs that say they *require* those resources to run as part
>>>> of agent assignment.
>>>>
>>>> > No we use this setup no for a

[go-cd] Release Announcement - 24.1.0

[Chad Wilson]

Hello everyone,

A new release of GoCD  (24.1.0) is
out.

This release is a relatively minor maintenance release, however does make
some potentially breaking changes to reduce maintenance overhead.

   - Links are no longer forced opened in new tabs
   - Java 17 is now the minimum supported version
   - Server container image is now based on Wolfi OS

As always, please review the release notes, and remember to take a backup
before upgrading.

To know more about the features and bug fixes in this release, see the release
notes  or head to the downloads page
 to try it. Feedback and ideas are always
welcome - we appreciate the discussion on issues you are having, and how we
can improve things.

Cheers,
Chad & Aravind

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH98HFhtNH4cXPMp3p%3DrX4COGBN8okW3jaLtAg8Hxgz59w%40mail.gmail.com.

Re: [go-cd] GOCD restart issue

[Chad Wilson]

Did someone recently change something in your compose volume mounts?

It's probably not a good idea to mount things directly into /go-working-dir
like the line "*-
./gocd/docker-elastic-agents-3.2.3-415.jar:/go-working-dir/plugins/external*
"

The container entrypoint expects to own and manage these locations with
symlinks, as Ram mentioned earlier. This is why the */go-working-dir* path
is not documented - it is internal and you shouldn't interfere with it or
couple your config to undocumented internals.

This mount being there at container start is probably going to prevent the
entrypoint from linking things to /godata correctly and you will probably
end up with a `plugins` directory which is not writable by GoCD and errors
like you had below.

Since you already have a /godata volume - put the plugin jars directly into
./gocd/data/plugins/external (on your host machine), and remove the
individual jar mount. Basically, please follow the documentation:
https://hub.docker.com/r/gocd/gocd-server

If removing that plugin mount still doesn't work, check the other mounts
and the permissions of those directories as seen by the container when it
starts by execing into it and exploring.

-Chad

On Tue, May 7, 2024 at 11:48 PM Vijayakumaran A. <
vijayakumara...@praniontech.com> wrote:

> We have started container using docker start servicename.
>
> Here is the composer file of gocd server part
>
> server:
> container_name: gocd-server
> image: gocd/gocd-server:v23.4.0
> restart: always
> ports:
>   - "8153:8153"
> volumes:
>   - ./gocd/data:/godata
>   - ./gocd/data/home:/home/go
>   - ./gocd/scripts/server:/docker-entrypoint.d
>   - ./gocd/scripts/shared:/shared
>   - ./gocd/passwd:/godata/config/passwd
>   -
> ./gocd/docker-elastic-agents-3.2.3-415.jar:/go-working-dir/plugins/external
>     networks:
>   - gocd
>
>
>
> On Tuesday, May 7, 2024 at 9:05:30 PM UTC+5:30 Chad Wilson wrote:
>
>> Please share how you are starting the container and what mounts you are
>> using, especially to /godata.
>>
>> Your container may have permissions issues writing to /godata or issues
>> mounting the filesystem in writeable mode. Is it some kind of NFS volume?
>> Is it mounted by another container/host already and thus is being mounted
>> in read-only mode? is this a Helm/Kubernetes install, or just a regular
>> container on a server?
>>
>> You can exec into the container and check /godata and /go-working-dir are
>> writable by the `go` user.
>>
>> -Chad
>>
>> On Tue, May 7, 2024 at 11:21 PM Sriram Narayanan 
>> wrote:
>>
>>>
>>>
>>> On Tue, May 7, 2024 at 10:33 PM Vijayakumaran A. <
>>> vijayak...@praniontech.com> wrote:
>>>
>>>> Team,we have just restarted the gocd container now gocd not started.
>>>>
>>>> Having below in docker logs please help
>>>> [image: go1.PNG][image: go2.PNG]
>>>>
>>>>
>>> Looks like something is wrong with the container system.
>>> /go-working-dir/ is a runtime symlink directory created here
>>> https://github.com/gocd/docker-gocd-server/blob/master/docker-entrypoint.sh#L41
>>>
>>> If a directory is not getting created, then there is an underlying issue
>>> unrelated to GoCD and entirely related to the underlying container system.
>>> Please see if a container restart helps you continue. You should also check
>>> the container system's logs. To share something that we saw at our project
>>> two weeks ago, one of the EKS clusters was still running at version 1.23
>>> and we would often see the error "Out of storage space" (sometimes once a
>>> day). When I'd exec into the container, I found that we were unable to make
>>> directories but were able to create files. We upgrades to EKS 1.24 and that
>>> error has not recurred since then.
>>>
>>> I also see that you are using GoCD 23.4 and the H2DB. If this is a GoCD
>>> setup that you are using for your project, then I urge you to use the
>>> postgres database instead. See
>>> https://github.com/gocd/gocd-database-migrator
>>>
>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "go-cd" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to go-cd+un...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/go-cd/5df023fd-f859-4a6b-99e7-92e3063653e7n%40googlegroups.co

Re: [go-cd] Mount .ssh from EFS to the container (from the image gocd/gocd-server:v22.3.0)

[Chad Wilson]

To add onto the many options here, if you only need the SSH keys to be used
by Git for clones etc, you can entirely customise how GIT uses SSH using
the *GIT_SSH_COMMAND* env var
;
set at the container level.

GIT_SSH_COMMAND="ssh -i /path/to/your/private/key"

Then you can put the private key anywhere you like (including /godata - not
just the home dir) which the GoCD server/agent has access to (as long as it
has the right file permissions (400 or 600) and is readable by
`go`/UID=1000 user, as Ram notes).

-Chad

On Tue, May 7, 2024 at 11:11 PM Sriram Narayanan 
wrote:

>
>
> On Mon, May 6, 2024 at 3:30 AM Jason Smyth  wrote:
>
>> Hi Satya,
>>
>> A possible workaround to the limitation is updating the server image and
>> adding a symlink that points ~/.ssh/ to wherever you want to actually mount
>> the data.
>>
>> I have never experimented with using a symlink for the .ssh directory,
>> though, so this may not work.
>>
>
> I haven't tried this yet, but one would explore adding a custom shell
> script at the /docker-entrypoint.d/ mount point which could create such a
> symlink
>
> Nice tip, Jason.
>
>
>>
>> Hope this helps,
>> Jason
>>
>>
>> On Sunday 28 April 2024 at 12:12:16 UTC-4 Sriram Narayanan wrote:
>>
>>> On Sat, Apr 27, 2024 at 7:10 PM Satya Elipe  wrote:
>>>
 Thank you Sriram.

 So, ".ssh" folder mounting will be separate from the rest of the data
 (/godata, for plugins, pipelines, db etc)...so there would be two separate
 mount points into the container ?

 I'm using ECS at the moment and not kubernetes, so my task definition
 will have two mount points like below:

 ```

 "mountPoints": [
 {
 "sourceVolume": "efs_id:/godata",

 "containerPath": "/godata"
 },
 {
 "sourceVolume": "efs_id:/godata/.ssh",

 "containerPath": "/home/go/.ssh"
 }
 ],

 ```

 So mounting /godata and efs_id:/godata/.ssh from EFS into the
 container at /godata and /home/go/.ssh locations respectively (per
 above code) seems to work.

 In this case entry_point.sh from the base image is able to
 map/consider and execute them properly, hence the server is up and running
 and functioning properly.

 Is that the way it has to be, I think the github repo for gocd server
 says that I guess, but perhaps I feel that extra mount point just for .ssh
 is overkill and if .ssh can also be entertained by entry_point.sh from one
 single mount point /godata in my case, that would be great ?

 If I do not mount .ssh into /home/go/.ssh separately into the container
 - things seem to fail complaining that "key verification failed", I'm not
 sure whether I'm still missing something here.

>>>
>>> Hey, I had got caught by surprise earlier during the "elastic agent"
>>> discussions and had assumed that you must be using EKS. Sorry, my bias had
>>> clouded my judgement then. Thankfully Chad and you cleared that up.
>>>
>>> ssh by default checks ~/.ssh/ for the keys. Within the GoCD server and
>>> agent containers, this home (~) is the /home/go directory, and hence we
>>> mount the .ssh folder there. There are use cases where the keys are made
>>> available via a different network share and not mixed with configurations
>>> that regular GoCD admins would have access to, and hence being able to
>>> mount from a separate place to ~/.ssh is helpful. You could always place
>>> the .ssh directory along side other directories that would get to godata,
>>> while also explicitly specifying a mount to /home/go. At present, GoCD does
>>> not have a configuration option to point it to a private key at a path
>>> other than ~/ssh
>>>
>>> https://docs.gocd.org/current/faq/docker_container_ssh_keys.html
>>>
>>>

 Many thanks
 Satya

 On Thu, Apr 25, 2024 at 3:31 PM Sriram Narayanan 
 wrote:

>
>
> On Thu, Apr 25, 2024 at 10:16 PM Satya Elipe 
> wrote:
>
>> Hi all
>>
>> Wonder, what's the way around to mount .ssh from EFS into the gocd
>> base container (from the image gocd/gocd-server:v22.3.0).
>>
>>
>> We have saved all our content into EFS under /godata and maps that
>> into the container as /godata.
>>
>>
>> We are using gocd/gocd-server:v22.3.0.
>>
>>
>> It all runs good, mapping was fine too but just one thing that’s not
>> happening is “.ssh” folder.
>>
>>
>> I have .ssh with all required keys in EFS under /godata and /godata
>> within the container also has .ssh but not /go-working-dir.
>>
>>
>> Is that supported, am I mis-configuring it, or do we need to handle
>> that outside of the base image ?
>>
>

Re: [go-cd] pipeline as code - stuck on waiting for agent

[Chad Wilson]

Perhaps you can share the agent(s) you are expecting it to run on, and the
job configuration of the jobs that aren't scheduling?

If the job isn't being assigned, it means there is no free agent[1] in the
desired pipeline-scope environment[2] that offers the requested job-scope
resources [3]. All three conditions need to be met, so either your agent or
the pipeline/job may be configured in a different way to your agents
(perhaps it is requesting a "resource")?

Keep in mind that you don't have to use job resource requirements or
environments if you don't want to, and you can configure it such that some
agents will be able to run jobs with no specific requested resource; or no
specific environment.

-Chad

On Sat, May 4, 2024 at 1:10 AM 'pan...@mammoth.io' via go-cd <
go-cd@googlegroups.com> wrote:

> Hi,
>
> I am trying to use the feature of pipeline as a code. In the config
> repository, I have configured an Allow rule for a specific environment.
> There are agents assigned to this environment. Having done this, I am now
> triggering the pipeline. The pipeline however just keeps waiting for an
> agent allocation. What step am I missing and where should this be
> configured.
> [image: Screenshot from 2024-05-03 22-38-39.png]
>
> Any help would be appreciated.
>
> Warm regards,
> Pankaj
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/a19f5206-b9e3-4321-9831-e3da5fdf4c92n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH9MsLA31XJ0HpS9s0Fw15owYt8x6r61JmpqLpoUx-RP%2Bw%40mail.gmail.com.

Re: [go-cd] Elastic agents plugin usage

[Chad Wilson]

It seems you are using the third party EC2 Elastic Agent Plugin
. This plugin
does not implement the plugin API correctly
 and
does not support environments correctly, which is presumably why you have
that user data hack.

It seems to me from a quick look at the code, this plugin only runs a
single job on an EC2 instance before terminating it so you shouldn't expect
re-use for multiple jobs. If you want to know more about how it is
designed, you are better to ask on their GitHub repo.

I don't know 100% why your agents aren't shutting down correctly, and you
probably need to look at the plugin logs (on both server and the agent
itself) to investigate.

However, since it looks like you have a ec2_user_data hack in place to get
some environment support with the plugin, you need to manually make sure
that the environments in the config agent.auto.register.
environments=staging,sandbox *exactly match the possible pipeline
environments for all possible jobs* you assign to this elastic agent
profile ID.

I also think having multiple environments registered here will possibly
cause chaos, because that is not how elastic agents manage environments
normally. They normally register only a single environment.

The problem will be that if *any single job* on your GoCD is assigned to
say, profile "*elastic_profile_staging*" with the autoregister config like
you have below, but then that job is configured for a pipeline inside a
GoCD environment called* "**other_env**" *an elastic agent will start but
then never get assigned the job. This is because it has registered only for
"*staging,sandbox*" via your hardcoded user_data, NOT "*other_env*".

This breaks the elastic agent plugin contract - GoCD thinks it has already
told the plugin to create an agent for "*other_env*", but it never does.
Now GoCD is confused as to what is happening with the agents. Thus the job
will likely never get assigned, and the plugin will never complete a job,
and it will never shut down the EC2 instance. Perhaps this is what is
happening to you? Might want to check if you have EC2 instances whose agent
logs don't show them doing any work or mismatched environments and elastic
profiles.

With a correctly behaving plugin GoCD tells the plugin a single environment
to register for (the one it needs a new agent to run a job on), and expects
the plugin to register for the environment it tells it to. This EC2 plugin
breaks that contract, which allows you to misconfigure things very easily
and create all sorts of problems. Personally I wouldn't use it if I am
using GoCD environments, but that's your decision to make.

-Chad

On Thu, Apr 25, 2024 at 10:48 PM Satya Elipe  wrote:

> Thank you Sriram.
> Please find my comments below.
>
> >Do the various jobs have an elastic profile ID set?
> Yes, I have two environments staging and prod, so we have separate
> profiles set for them.
>
> Here is pretty much what each profile has:
>
>1. ec2_ami
>2. ec2_instance_profile
>3. ec2_subnets
>4. ec2_instance_type
>5. ec2_key
>6. ec2_user_data
>*echo "agent.auto.register.environments=staging,sandbox" | sudo tee -a
>/var/lib/go-agent/config/autoregister.properties > /dev/null*
>7. ec2_sg
>
>
> >What is the error that you see due to the max count limit?
> ```
> [go] Received request to create an instance for
> brxt-config-service-deploy-production/19/prepare-deploy-stage/1/prepare-deploy-job
> at 2024-04-09 11:21:38 +00:00
> [go] Successfully created new instance i-093b44f70992505cc in
> subnet-555bba0d
> [go] Received request to create an instance for
> brxt-config-service-deploy-production/19/prepare-deploy-stage/1/prepare-deploy-job
> at 2024-04-09 11:23:38 +00:00
> [go] The number of instances currently running is currently at the maximum
> permissible limit, "2". Not creating more instances for jobs:
> brxt-core-service-deploy-staging/86/prepare-for-deploy-stage/1/prepare-for-deploy-job,
> brxt-core-service-deploy-staging/86/deploy-stage/1/deploy-job,
> brxt-core-service-deploy-staging/86/verify-stage/1/verify-job,
> brxt-config-service-deploy-staging/18/deploy-stage/1/deploy-job,
> brxt-config-service-deploy-production/19/prepare-deploy-stage/1/prepare-deploy-job.
> [go] Received request to create an instance for
> brxt-config-service-deploy-production/19/prepare-deploy-stage/1/prepare-deploy-job
> at 2024-04-09 11:25:39 +00:00
> [go] The number of instances currently running is currently at the maximum
> permissible limit, "2". Not creating more instances for jobs:
> brxt-core-service-deploy-staging/86/prepare-for-deploy-stage/1/prepare-for-deploy-job,
> brxt-core-service-deploy-staging/86/deploy-stage/1/deploy-job,
> brxt-core-service-deploy-staging/86/verify-stage/1/verify-job,
> brxt-config-service-deploy-staging/18/deploy-stage/1/deploy-job,
> 

Re: [go-cd] Elastic agents plugin usage

[Chad Wilson]

Can you be specific about the type of elastic agents you are creating and
the plugin you are using. Kubernetes? Docker? Something else? There are
many elastic agent plugins.


> Here's where it gets tricky: when the staging job completes and triggers
> the production job, I expect one of the active agents to take over.
> Instead, the production job attempts to launch new agents, fails due to the
> max count limit, and runs without any agents, leading to failure.
>

I believe elastic agents are generally launched sequentially - i.e a new
one won't be launched until there are no pending-launch ones - but this
depends on the specific elastic agent type.

If you are new to elastic agents, you'll want to be aware that in almost
all elastic agent plugin variants the elastic agents are
single-shot/single-job usage, and are not re-used. The specific type of
elastic agent and its plugin implementation defines how it handles such
things though, so need to know specifics to guess.

Look at the specific elastic agent plugin's log on the server to see what
it is doing. Perhaps your elastic agents are not shutting down
automatically for some reason due to a configuration issue or a problem
with the jobs you are running?

-Chad

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH9msgAM_RtZu%2BEYbzHoQge5wh62a1i-YV7ueE_KoAkxtQ%40mail.gmail.com.

Re: [go-cd] GOCD server Db issue

[Chad Wilson]

You may have killed GoCD while it was starting up? If you are sure that you
do not have multiple processes accessing the same database at the same time
and it is still stuck, you will have to try manually clearing the lock
.
Backup your database file before doing this.

Need to connect to the database and use this to clear the lock. You'll have
to do so using something like is done here

but *instead
of running an export/import* to recreate the DB you just want to

1) backup the cruise.mv.db file ()
2) find the h2-1.4.200.jar file
3) create clear-liquibase-lock.sql with UPDATE DATABASECHANGELOGLOCK SET
LOCKED=0
4) run something like this on the main DB file java -cp h2-1.4.200.jar
org.h2.tools.RunScript -url jdbc:h2:./cruise -user sa -script
clear-liquibase-lock.sql

-Chad


On Thu, Apr 25, 2024 at 7:22 PM Vijayakumaran A. <
vijayakumara...@praniontech.com> wrote:

> HI Team,
>
> When we restarting gocd server we have below error please advice us. We
> use h2db.
>
> 2024-04-25 16:22:33,896 INFO  [WrapperJarAppMain] Jetty9Server:199 -
> Configuring Jetty using /etc/go/jetty.xml
> 2024-04-25 16:22:33,958 WARN  [WrapperJarAppMain] Server:357 -
> ErrorPageMapper not supported for Server level Error Handling
> 2024-04-25 16:22:34,088 WARN  [WrapperJarAppMain] AbstractHandler:96 - No
> Server set for ResourceHandler@5904a259{STOPPED}
> 2024-04-25 16:22:40,659 WARN  [WrapperJarAppMain] ConnectionManager:117 -
> The file /etc/go/db.properties specified by `go.db.config` does not exist.
> 2024-04-25 16:22:41,319 INFO  [WrapperJarAppMain] DatabaseMigrator:40 -
> Upgrading database, this might take a while depending on the size of the
> database.
> 2024-04-25 16:22:41,320 INFO  [WrapperJarAppMain] DatabaseMigrator:49 -
> 
> 2024-04-25 16:22:41,320 INFO  [WrapperJarAppMain] DatabaseMigrator:49 -
> WARNING: Shutting down your server at this point will lead to a database
> corruption. Please wait until the database upgrade completes.
> 2024-04-25 16:22:41,320 INFO  [WrapperJarAppMain] DatabaseMigrator:49 -
> 
> 2024-04-25 16:27:42,107 ERROR [WrapperJarAppMain] DatabaseMigrator:65 -
> Unable to create database upgrade script for database. The problem was:
> Could not acquire change log lock.  Currently locked by 172.21.0.1
> (172.21.0.1) since 4/25/24, 12:19 PM
> liquibase.exception.LockException: Could not acquire change log lock.
> Currently locked by 172.21.0.1 (172.21.0.1) since 4/25/24, 12:19 PM
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/a1b7a304-1559-441e-b002-b385ae93184en%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH_UXfcQVRzEoksuCSCKvjPAvQ3%3D%2BmDm35ZLqHtwgK%2Bz_Q%40mail.gmail.com.

Re: [go-cd] Query re: Gitlab build status notifier

[Chad Wilson]

Hi Steve

I'm not familiar with use of either of these plugins with GitLab, however
it seems pretty clear that the gocd-build-status-notifier plugin does not
have gitlab support merged so the two plugins don't work together right
now. It seemed to be blocked by this comment here

which seems unresolved or not responded to.

The logs (if any) will be in plugin-specific logs rather than the main
server logs.

In any case, if you want to give it a go with v1.4.0-RC4 of the PR Builder
plugin, you can try the test build attached here
.
If it works OK with and without subgroups we can consider merging it. I
have not tested this myself, as I don't have a GitLab setup to test with
(nor time to do so). I'd still be a bit surprised if it works though, as
the Gitlab API interaction hasn't been changed on the notifier for many
years, and I suspect it has evolved

-Chad

On Mon, Apr 8, 2024 at 7:53 PM Steve  wrote:

> Hi,
>
> Got a question for the wonderful, knowledgable people here pls!
>
> I've got a copy of GoCD running in a docker container, GoCD v23.5.0, with
> Gitlab Pull Requests Builder v1.4.0-RC4, Gitlab Merge Requests status
> notifier v1.7.2-276.
>
> While I have got the Gitlab Pull Requests Builder working properly and
> responding to creation/changes in Gitlab Merge Requests with pipeline
> executions on the appropriate branch, I cannot:
>
> 1) get any Gitlab MR status notifications appearing in the MR nor
> 2) see any evidence of the plugin failing or being called at all in the
> GoCD server logs.
>
> Can anyone advise on what I'm doing wrong, if anything, please?
>
> Saw this
> https://github.com/gocd-contrib/gocd-build-status-notifier/pull/134,
> wondering if that's something to do with it (given that the notification
> plugin hasnt been updated in a few years), or maybe
> https://github.com/ashwanthkumar/gocd-build-github-pull-requests/issues/153
> (I'm using an enterprise Gitlab with subgroups that works for the builder
> but maybe it has a problem of some kind working with the notifier?)
>
> Thanks for any light that can be shed on this!
>
> Steve
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/a7936d1a-9a65-4d92-979d-36c886b1d150n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-mtcuqweJ_J5aOXdVAOsYCNKDKsoscYNFxWi21KH3%2BcQ%40mail.gmail.com.

Re: [go-cd] Help Needed with GoCD Docker Setup and cruise-config.xml Configuration

[Chad Wilson]

[Thread-79]
>> DefaultPluginJarChangeListener:74 - Plugin load finished:
>> /go-working-dir/plugins/bundled/gocd-filebased-authentication-plugin.jar
>> 2024-04-11 08:10:59,359 INFO  [Thread-79]
>> DefaultPluginJarChangeListener:67 - Plugin load starting:
>> /go-working-dir/plugins/bundled/gocd-yaml-config-plugin.jar
>> 2024-04-11 08:11:01,493 INFO  [Thread-79] ConfigRepositoryInitializer:108
>> - [Config Repository Initializer] Start initializing the config
>> repositories for plugin 'yaml.config.plugin'
>> 2024-04-11 08:11:01,499 INFO  [Thread-79] ConfigRepositoryInitializer:112
>> - [Config Repository Initializer] Done initializing the config repositories
>> for plugin 'yaml.config.plugin'
>> 2024-04-11 08:11:01,538 INFO  [Thread-79]
>> DefaultPluginJarChangeListener:74 - Plugin load finished:
>> /go-working-dir/plugins/bundled/gocd-yaml-config-plugin.jar
>> 2024-04-11 08:11:01,579 INFO  [Thread-79]
>> DefaultPluginJarChangeListener:67 - Plugin load starting:
>> /go-working-dir/plugins/external/gocd-slack-notifier-1.4.0.jar
>> 2024-04-11 08:11:01,710 ERROR [Thread-79] PluginLoader:121 - Failed to
>> load plugin: plugins_work/gocd-slack-notifier-1.4.0.jar. Plugin is invalid.
>> Reasons [Class [GoNotificationPlugin] is annotated with @Extension but
>> cannot be constructed. Reason: java.lang.RuntimeException: Unable to find
>> go_notify.conf. Please make sure you've set it up right.., No extensions
>> found in this plugin. Please check for @Extension annotations]
>> 2024-04-11 08:11:01,723 INFO  [Thread-79]
>> DefaultPluginJarChangeListener:74 - Plugin load finished:
>> /go-working-dir/plugins/external/gocd-slack-notifier-1.4.0.jar
>> 2024-04-11 08:11:01,763 INFO  [Thread-79]
>> DefaultPluginJarChangeListener:67 - Plugin load starting:
>> /go-working-dir/plugins/external/gocd-ec2-elastic-agent-plugin-2.2.2.jar
>> 2024-04-11 08:11:02,693 INFO  [Thread-79] GoConfigDao:108 - Config update
>> request by anonymous is in queue -
>> com.thoughtworks.go.config.update.ReplaceElasticAgentInformationCommand@424dd8d0
>> 2024-04-11 08:11:02,705 INFO  [Thread-79] GoConfigDao:111 - Config update
>> request
>> com.thoughtworks.go.config.update.ReplaceElasticAgentInformationCommand@424dd8d0
>> by anonymous is being processed
>> 2024-04-11 08:11:02,832 INFO  [Thread-79] MagicalGoConfigXmlWriter:85 -
>> [Serializing Config] Generating config partial.
>> 2024-04-11 08:11:02,908 INFO  [Thread-79] GoFileConfigDataSource:587 -
>> [Configuration Changed] Saving updated configuration.
>> 2024-04-11 08:11:02,936 INFO  [Thread-79] CachedGoConfig:223 - About to
>> notify config listeners
>> 2024-04-11 08:11:02,941 INFO  [Thread-79] BuildAssignmentService:250 -
>> [Configuration Changed] Removing jobs for pipelines that no longer exist in
>> configuration.
>> 2024-04-11 08:11:02,944 INFO  [Thread-79] CachedGoConfig:231 - Finished
>> notifying all listeners
>> 2024-04-11 08:11:02,945 INFO  [Thread-79] GoConfigDao:126 - Config update
>> request by anonymous is completed
>> 2024-04-11 08:11:02,985 WARN  [Thread-79] PluginSettingsMetadataLoader:63
>> - Failed to fetch plugin settings metadata for plugin
>> com.continuumsecurity.elasticagent.ec2. Maybe the plugin does not implement
>> plugin settings and view?
>> 2024-04-11 08:11:02,986 WARN  [Thread-79] PluginSettingsMetadataLoader:64
>> - Plugin: com.continuumsecurity.elasticagent.ec2 - Metadata load info:
>> [{extension='elastic-agent', configuration='null', view='null',
>> error='java.lang.NullPointerException: Cannot invoke
>> "com.continuumsecurity.elasticagent.ec2.Request.ordinal()" because the
>> return value of
>> "com.continuumsecurity.elasticagent.ec2.Request.fromString(String)" is
>> null'}]
>> 2024-04-11 08:11:02,986 WARN  [Thread-79] PluginSettingsMetadataLoader:65
>> - Not all plugins are required to implement the request above. This error
>> may be safe to ignore.
>> 2024-04-11 08:11:03,022 INFO  [Thread-79]
>> DefaultPluginJarChangeListener:74 - Plugin load finished:
>> /go-working-dir/plugins/external/gocd-ec2-elastic-agent-plugin-2.2.2.jar
>> 2024-04-11 08:11:03,057 INFO  [Thread-79]
>> DefaultPluginJarChangeListener:67 - Plugin load starting:
>> /go-working-dir/plugins/external/gocd-groovy-dsl-config-plugin-2.0.0-241.jar
>> 2024-04-11 08:11:03,972 INFO  [Thread-79] ConfigRepositoryInitializer:108
>> - [Config Repository Initializer] Start initializing the config
>> repositories for plugin 'cd.go.contrib.plugins.configrepo.groovy'
>> 2024-04-11 08:11:03,977 INFO  [Thread-79] ConfigRepositoryInitializer:112
>&g

Re: [go-cd] Help Needed with GoCD Docker Setup and cruise-config.xml Configuration

[Chad Wilson]

Hi Satya

When running a standard GoCD docker server image, you should mount a
writable volume to the entire */godata* directory which includes the config
as noted at https://hub.docker.com/r/gocd/gocd-server, but not directly to
an individual config file.

If you do not, even if you get the config working properly, you'll lose
other things such as artifacts at startup which you generally do not want.

You are getting this error likely because the server ID inside the go
config does not match what is inside the configuration history, which is
normally at /godata/db/config.git (The GoCD Docker entrypoint script
creates the symlinks into /go-working-dir within the docker-entrypoint.sh
),
so these things need to be mounted together and consistent.

I don't think your mounts to /var/lib/go-server or /etc/go will be doing
anything, as these folders are not used for *off-the-shelf* docker server
images, since the server is not installed as an rpm/deb package when
creating a container. You can/should likely remove these.

In a general sense, all you should need to do is mount a location in EFS to
/godata, similar to what the Helm chart does:
https://github.com/gocd/helm-chart/blob/c734ad2263b1d2885229d00267c428e88f868504/gocd/templates/gocd-server-deployment.yaml#L122-L124
That will put all logs, artifacts, databases and config on your volume.
Some folks decide to also mount /home/go if they want to use external
storage for things like Git SSH keys or other shell defaults affecting the
*go* user, but that's optional.

-Chad

On Thu, Apr 11, 2024 at 5:34 AM Satya Elipe  wrote:

>
> Dear GoCD Support Team,
>
>
> I hope this message finds you well. I am currently encountering an issue
> with setting up GoCD on Docker and specifically with configuring the
> cruise-config.xml file to be recognized correctly in my setup. Despite
> following the official documentation and trying various configurations,
> I've hit a stumbling block that I hope you can help me with.
>
>
> *Issue Summary:*
>
> I have a GoCD server running in a Docker container, and I'm attempting to
> ensure that the cruise-config.xml file is correctly picked up from a
> specified location. My goal is to mount this configuration file from an
> external volume into the GoCD server container so that the server uses this
> configuration instead of the default one.
>
> *Configuration Details:*
>
>- *GoCD Server Image Version:* gocd/gocd-server:v22.3.0
>- *Docker Version:* Docker version 25.0.3, build 4debf41
>- *Host Operating System:* Amazon Linux 2023
>
> *Docker Run Command:*
>
> docker run -d \
>
>   --name gocd-server \
>
>   -p 8153:8153 \
>
>   -p 8154:8154 \
>
>   -v /mnt/gocd_efs:/var/lib/go-server \
>
>   -v
> /mnt/gocd_efs/etc_go/cruise-config.xml:/go-working-dir/config/cruise-config.xml
> \
>
>   -v /mnt/gocd_efs/etc_go:/etc/go \
>
>   custom-gocd-server
>
>
>
> *Issue Encountered:*
>
> When including the -v
> /mnt/gocd_efs/etc_go/cruise-config.xml:/go-working-dir/config/cruise-config.xml
> volume mount, the GoCD server fails to start correctly, with logs
> indicating an inability to create or copy necessary files within
> /go-working-dir/config.
>
>
> Without this mount, the server starts but does not load the desired
> configuration, defaulting instead to the initial configuration without any
> of our pipeline configurations.
>
>
> Log snippet:
> ```
> jvm 1| 2024-04-10 20:55:17,429 ERROR [Thread-79]
> GoFileConfigDataSource:436 - Unable to load config file:
> /go-working-dir/config/cruise-config.xml The value of 'serverId' uniquely
> identifies a Go server instance. This field cannot be modified.
>
> ```
>
>
>
> *Questions:*
>
>- Is there a recommended approach to ensure cruise-config.xml is
>correctly recognized and used by the GoCD server when running in Docker?
>- Could this issue be related to how volumes are mounted or
>permissions within the container?
>
>
> Any assistance or insights you could provide on this matter would be
> greatly appreciated. I am happy to provide any further information or logs
> as needed.
>
> Thank you for your time and support.
>
>
>
> Best regards,
>
> Satya
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CADKEDRrF7KRFP9jNsg631STyokXS1Njqutdshd0ZKdwHMRTy6g%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this 

Re: [go-cd] SCM VALIDATE FAILED

[Chad Wilson]

 />\n  GOINPUTNAME[password].$error.server\">{{
> GOINPUTNAME[password].$error.server }}\n\n form_item_block\">\n Default Branch:\n  ng-model=\"defaultBranch\" ng-required=\"false\"/>\n  form_error\" ng-show=\"GOINPUTNAME[defaultBranch].$error.server\">{{
> GOINPUTNAME[defaultBranch].$error.server }}\n\n form_item_block\">\n Whitelisted branches:\n  text\" ng-model=\"branchwhitelist\" ng-required=\"false\"/>\n  \"form_error\" ng-show=\"GOINPUTNAME[branchwhitelist].$error.server\">{{
> GOINPUTNAME[branchwhitelist].$error.server }}\n\n \"form_item_block\">\n Blacklisted branches:\n  \"text\" ng-model=\"branchblacklist\" ng-required=\"false\"/>\n  class=\"form_error\" ng-show=\"GOINPUTNAME[branchblacklist].$error.server
> \">{{ GOINPUTNAME[branchblacklist].$error.server }}\n\n"
> }
> }
> } ]
> }, {
> "_links" : {
> "self" : {
> "href" : "https://serverurl.com/go/api/admin/plugin_info/github.pr;
> },
> "doc" : {
> "href" : "https://api.gocd.org/23.3.0/#plugin-info;
> },
> "find" : {
> "href" : "https://serverurl.com/go/api/admin/plugin_info/:id;
> }
> },
> "id" : "github.pr",
> "status" : {
> "state" : "active"
> },
> "plugin_file_location" :
> "/var/lib/go-server/plugins/external/github-pr-poller-1.4.0-RC4.jar",
> "bundled_plugin" : false,
> "about" : {
> "name" : "Github Pull Requests Builder",
> "version" : "1.4.0-RC4",
> "target_go_version" : "15.1.0",
> "description" : "Plugin that polls a GitHub repository for pull requests
> and triggers a build for each of them",
> "target_operating_systems" : [ ],
> "vendor" : {
> "name" : "Ashwanth Kumar",
> "url" : "https://github.com/ashwanthkumar/gocd-build-github-pull-requests;
> }
> },
> "extensions" : [ {
> "type" : "scm",
> "display_name" : "Github",
> "scm_settings" : {
> "configurations" : [ {
> "key" : "url",
> "metadata" : {
> "secure" : false,
> "required" : true,
> "part_of_identity" : true
> }
> }, {
> "key" : "username",
> "metadata" : {
> "secure" : false,
> "required" : false,
> "part_of_identity" : false
> }
> }, {
> "key" : "password",
> "metadata" : {
> "secure" : true,
> "required" : false,
> "part_of_identity" : false
> }
> }, {
> "key" : "defaultBranch",
> "metadata" : {
> "secure" : false,
> "required" : false,
> "part_of_identity" : false
> }
> }, {
> "key" : "shallowClone",
> "metadata" : {
> "secure" : false,
> "required" : false,
> "part_of_identity" : false
> }
> } ],
> "view" : {
> "template" : "\n URL: asterisk\">*\n  ng-required=\"true\"/>\n  GOINPUTNAME[url].$error.server\">{{ GOINPUTNAME[url].$error.server }}
> \n\n\n
> Username:\n  new-password\" ng-model=\"username\" ng-required=\"false\"/>\n  class=\"form_error\" ng-show=\"GOINPUTNAME[username].$error.server\">{{
> GOINPUTNAME[username].$error.server }}\n\n form_item_block\">\n Password:\n  autocomplete=\"new-password\" ng-model=\"password\" ng-required=\"false\"
> />\n  GOINPUTNAME[password].$error.server\">{{
> GOINPUTNAME[password].$error.server }}\n\n form_item_block\">\n Default Branch:\n  ng-model=\"defaultBranch\" ng-required=\"false\"/>\n  form_error\" ng-show=\"GOINPUTNAME[defaultBranch].$error.server\">{{
> GOINPUTNAME[defaultBranch].$error.server }}\n\n"
> }
> }
> } ]
> } ]
> }
> }%
> as you requested i shared the response here
> On Tuesday, March 19, 2024 at 12:58:39 PM UTC+5:30 Chad Wilson wrote:
>
>> Yes, the browser is not sending the correct data - probably for the same
>> reason I talked about. If the plugin, browser or server is not working
>> correctly, nothing you enter will work.
>>
>> Please share the response from */go/api/admin/plugin_info?type=scm* as I
>> requested below
>> *.*

Re: [go-cd] SCM VALIDATE FAILED

[Chad Wilson]

There is nothing you should *need* to configure that affects seeing those
raw template values like {{ GOINPUTNAME['url'].$error.server }} . You
should never see these - it means there is a problem in how your browser is
loading the plugin view and displaying plugin responses to you. We need to
find what is breaking the plugin view and fix that. When the plugin view is
working correctly you should see just errors like this:

[image: image.png]
I only have another couple of suggestions

   - Make sure you don't have multiple versions of the plugin installed in
   the server's plugin directory. If you remove duplicate plugin versions,
   restart the server afterwards and see if it's different.
   - If you are already in the network tab, perhaps you can share the JSON
   response from */go/api/admin/plugin_info?type=scm* when you click on the
   Admin > Pluggable SCMs tab (in the environment that is broken). There is
   nothing sensitive in that API response.

Something seems to be wrong with either the plugin, the way you installed
it, but without looking at the browser console and my earlier questions we
would just be guessing where the problem is.

-Chad

On Tue, Mar 19, 2024 at 2:14 PM Suthar robert 
wrote:

> i check using the inspect, in the response of the network tab it require
> key and value in it. In a working environment which is local host it has
> the key and value in it, But in the configured url which i mean i deployed
> the gocd into the specific domain url in the server, in that environment it
> didn't have key and value present in it. Also i check the source code of
> the plugin and check it, in that source code plugin they have scm template
> html file which show extact like this error in it but they didn't mention
> clarification about it how to config those
>
> On Tuesday, March 19, 2024 at 11:38:13 AM UTC+5:30 Chad Wilson wrote:
>
>> Version 1.4.0 of the plugin has never been officially released, so I am
>> not sure if that is stable, but I do not get those errors with that version
>> either. Please try and be specific about the version you are using in
>> different environments - do you mean "1.4.0-RC4" from here
>> <https://github.com/ashwanthkumar/gocd-build-github-pull-requests/releases/tag/v1.4.0-RC4>,
>> or did you build/compile the plugin yourself? Are both your localhost GoCD
>> server and the "real" server the same GoCD version 23.3.0?
>>
>> I don't understand what you mean by "so here the problem when i launched
>> the config gocd which is have own url in it for security purpose", sorry,
>> so cannot comment on this right now.
>>
>> Regardless, check the browser console for errors, and check the browser
>> debugger "network" tab to see if requests under the "JS" and "XHR" types
>> are being blocked or failing. Disable the browser cache and do a full
>> reload and check again.
>>
>> -Chad
>>
>> On Tue, Mar 19, 2024 at 1:56 PM Suthar robert 
>> wrote:
>>
>>> In the same browser when i use localhost gocd server it working fine.
>>> The version plugin is 1.4.0, so here the problem when i launched the config
>>> gocd which is have own url in it for security purpose. It display like
>>> this, whoever visit this url and use this scm plugin it display the error.
>>> i need to resolve it
>>>
>>>
>>> On Tuesday, March 19, 2024 at 11:13:05 AM UTC+5:30 Chad Wilson wrote:
>>>
>>>> I can't replicate this problem, but it looks like something is wrong
>>>> with the plugin or your browser. Try another browser (Edge, Safari,
>>>> Firefox) or Chrome Incognito mode (without plugins/extensions) to see if
>>>> you get the same errors.
>>>>
>>>> If it works in another browser, or without extensions/plugins, you may
>>>> also want to check your browser debugger javascript console for errors, as
>>>> this error seems to imply that the admin user interface is not creating the
>>>> plugin view correctly - this logic is done within the browser with
>>>> Javascript.
>>>>
>>>> Which version of the plugin did you install? e.g from
>>>> http://localhost:8153/go/admin/plugins
>>>>
>>>> [image: image.png]
>>>>
>>>> -Chad
>>>>
>>>> On Tue, Mar 19, 2024 at 12:36 PM Suthar robert 
>>>> wrote:
>>>>
>>>>> [image: image.png]
>>>>> Sorry for That
>>>>> Here it is
>>>>>
>>>>>
>>>>>
>>>>> On Monday, March 18, 2024 at 6:47:46 PM UTC+5:30 Chad Wilson wrote

Re: [go-cd] SCM VALIDATE FAILED

[Chad Wilson]

Version 1.4.0 of the plugin has never been officially released, so I am not
sure if that is stable, but I do not get those errors with that version
either. Please try and be specific about the version you are using in
different environments - do you mean "1.4.0-RC4" from here
<https://github.com/ashwanthkumar/gocd-build-github-pull-requests/releases/tag/v1.4.0-RC4>,
or did you build/compile the plugin yourself? Are both your localhost GoCD
server and the "real" server the same GoCD version 23.3.0?

I don't understand what you mean by "so here the problem when i launched
the config gocd which is have own url in it for security purpose", sorry,
so cannot comment on this right now.

Regardless, check the browser console for errors, and check the browser
debugger "network" tab to see if requests under the "JS" and "XHR" types
are being blocked or failing. Disable the browser cache and do a full
reload and check again.

-Chad

On Tue, Mar 19, 2024 at 1:56 PM Suthar robert 
wrote:

> In the same browser when i use localhost gocd server it working fine. The
> version plugin is 1.4.0, so here the problem when i launched the config
> gocd which is have own url in it for security purpose. It display like
> this, whoever visit this url and use this scm plugin it display the error.
> i need to resolve it
>
>
> On Tuesday, March 19, 2024 at 11:13:05 AM UTC+5:30 Chad Wilson wrote:
>
>> I can't replicate this problem, but it looks like something is wrong with
>> the plugin or your browser. Try another browser (Edge, Safari, Firefox) or
>> Chrome Incognito mode (without plugins/extensions) to see if you get the
>> same errors.
>>
>> If it works in another browser, or without extensions/plugins, you may
>> also want to check your browser debugger javascript console for errors, as
>> this error seems to imply that the admin user interface is not creating the
>> plugin view correctly - this logic is done within the browser with
>> Javascript.
>>
>> Which version of the plugin did you install? e.g from
>> http://localhost:8153/go/admin/plugins
>>
>> [image: image.png]
>>
>> -Chad
>>
>> On Tue, Mar 19, 2024 at 12:36 PM Suthar robert 
>> wrote:
>>
>>> [image: image.png]
>>> Sorry for That
>>> Here it is
>>>
>>>
>>>
>>> On Monday, March 18, 2024 at 6:47:46 PM UTC+5:30 Chad Wilson wrote:
>>>
>>>> Sorry, I don't think the screenshot came through correctly - please try
>>>> again?
>>>>
>>>> On Mon, Mar 18, 2024 at 7:39 PM Suthar robert 
>>>> wrote:
>>>>
>>>>> Sure here i am using a gocd versoin = 23.3.0, plugins is git featrure
>>>>> plugin which is the link i provide here
>>>>> https://github.com/ashwanthkumar/gocd-build-github-pull-requests
>>>>>
>>>>> the browser i am using chrome
>>>>>
>>>>> when i click a create new pluggable scm, it automatically arises like
>>>>> this, when i enter the url and save it, it still says the validation 
>>>>> failed
>>>>>
>>>>>
>>>>> Here i added the screenshot below for clarification
>>>>>
>>>>> [image: image.png]
>>>>>
>>>>>
>>>>>
>>>>> On Monday, March 18, 2024 at 3:36:18 PM UTC+5:30 Chad Wilson wrote:
>>>>>
>>>>>> It may help if you share screenshots of exactly what you are doing
>>>>>> and entering in the fields, as it's a bit difficult to understand what 
>>>>>> you
>>>>>> are seeing?
>>>>>>
>>>>>> Can you also share the versions of your GoCD server and plugins, plus
>>>>>> which browser you are using?
>>>>>>
>>>>>> On Mon, Mar 18, 2024 at 5:51 PM Suthar robert 
>>>>>> wrote:
>>>>>>
>>>>>>> Hi all,
>>>>>>> I am using a git feature plugin for my works here, for that i need
>>>>>>> to configure the scm plugin in it. So that i can able to use git feature
>>>>>>> plugin, i am a facing a problem here which is validate which return 
>>>>>>> without
>>>>>>> entering the value in the field which is like
>>>>>>>
>>>>>>> {{ GOINPUTNAME['url'].$error.server }}
>>>>>>> like this error for every input field. I don't know how to fix this.
>>>>>>>
>>>>>

Re: [go-cd] SCM VALIDATE FAILED

[Chad Wilson]

Sorry, I don't think the screenshot came through correctly - please try
again?

On Mon, Mar 18, 2024 at 7:39 PM Suthar robert 
wrote:

> Sure here i am using a gocd versoin = 23.3.0, plugins is git featrure
> plugin which is the link i provide here
> https://github.com/ashwanthkumar/gocd-build-github-pull-requests
>
> the browser i am using chrome
>
> when i click a create new pluggable scm, it automatically arises like
> this, when i enter the url and save it, it still says the validation failed
>
>
> Here i added the screenshot below for clarification
>
> [image: image.png]
>
>
>
> On Monday, March 18, 2024 at 3:36:18 PM UTC+5:30 Chad Wilson wrote:
>
>> It may help if you share screenshots of exactly what you are doing and
>> entering in the fields, as it's a bit difficult to understand what you are
>> seeing?
>>
>> Can you also share the versions of your GoCD server and plugins, plus
>> which browser you are using?
>>
>> On Mon, Mar 18, 2024 at 5:51 PM Suthar robert 
>> wrote:
>>
>>> Hi all,
>>> I am using a git feature plugin for my works here, for that i need to
>>> configure the scm plugin in it. So that i can able to use git feature
>>> plugin, i am a facing a problem here which is validate which return without
>>> entering the value in the field which is like
>>>
>>> {{ GOINPUTNAME['url'].$error.server }}
>>> like this error for every input field. I don't know how to fix this.
>>>
>>> i am using a pluing this one
>>> https://github.com/ashwanthkumar/gocd-build-github-pull-requests
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+un...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/fcdceb39-542b-41bd-ba34-244341f12a56n%40googlegroups.com
>>> <https://groups.google.com/d/msgid/go-cd/fcdceb39-542b-41bd-ba34-244341f12a56n%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/25f7ac83-a6a7-4f45-9eb7-133d456e94d8n%40googlegroups.com
> <https://groups.google.com/d/msgid/go-cd/25f7ac83-a6a7-4f45-9eb7-133d456e94d8n%40googlegroups.com?utm_medium=email_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH_TxK6kx4cbazfz1xccTorQbpei6wnwJ_%2BkSvfMG-0G4g%40mail.gmail.com.

Re: [go-cd] SCM VALIDATE FAILED

[Chad Wilson]

It may help if you share screenshots of exactly what you are doing and
entering in the fields, as it's a bit difficult to understand what you are
seeing?

Can you also share the versions of your GoCD server and plugins, plus which
browser you are using?

On Mon, Mar 18, 2024 at 5:51 PM Suthar robert 
wrote:

> Hi all,
> I am using a git feature plugin for my works here, for that i need to
> configure the scm plugin in it. So that i can able to use git feature
> plugin, i am a facing a problem here which is validate which return without
> entering the value in the field which is like
>
> {{ GOINPUTNAME['url'].$error.server }}
> like this error for every input field. I don't know how to fix this.
>
> i am using a pluing this one
> https://github.com/ashwanthkumar/gocd-build-github-pull-requests
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/fcdceb39-542b-41bd-ba34-244341f12a56n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH94XC00jXm2ADKxPUA-S5Pe7jm%3DVt4T-Ni-8Scc5yozCQ%40mail.gmail.com.

Re: [go-cd] git clone failures in gocd pipelines after sonoma upgrade

[Chad Wilson]

Is it always "kex_exchange_identification: Connection closed by remote
host" as the failure reason?

If so, it usually means the remote host is closing it before exchanging SSH
keys (maybe because of connection limits, or authentication problems), or
perhaps there is a firewall/network proxy or something else in between your
agent and the git repository host that is closing the connection from both
the git client and the git server's perspective.

I can't think of anything obvious that would have changed due just to the
MacOS upgrade, unless something has changed with firewall settings, or
other such software you have installed on the agents. Are you sure nothing
changed in the network environment your agents are deployed, or on the
Gitlab instance at the same time?

It looks like you are using SSH Git materials, so once you can replicate
with git commands directly on the agent you might need to use environment
variable GIT_SSH_COMMAND="ssh -vvv" to see all of the raw SSH debug
information (GIT_CURL_VERBOSE is for HTTPS connections). If you discover
something here you can probably google to debug it further.

*But the issue still exists. on the GoCD,  we are still using pre-installed
git, do I need install git with brew on the GoCD as well.*

If the problem is only on the agents, then no need to change things on the
servers, especially if using homebrew Git didn't change anything.

-Chad

On Sun, Mar 17, 2024 at 12:31 AM SATYA R  wrote:

> Hi Chad,
> Thank you for responding to my issue.
> Just wanted to give some basic info on the GoCD version.
> We are using GoCD 23.1.0 aarch version, all of our servers are Mac OS M1
> servers(for both GoCD and agents)
>
>- What are the specific clone errors you are getting?
>
>
>
>
>
>
>
>
>
>
> *we get the below error when the job tried to clone more the repositories,
> at first we thorught the git server is closing the connection, the gitlab
> team took the logs from ec2 instance where it says the source ip which is
> go-agent is requesting the disconnection.pipelines logs:STDERR: Cloning
> into
> '/Users//go-agent-23.1.0/pipelines/Git_Clone/daf-mock-server'...STDERR:
> kex_exchange_identification: Connection closed by remote hostSTDERR:
> Connection closed by 10.112.205.90 port 22STDERR: fatal: Could not read
> from remote repository.STDERR: STDERR: Please make sure you have the
> correct access rightsSTDERR: and the repository exists.Failed to run git
> clone command*
>
>
> *Logs from Git server: the Ip in the logs is go-agent ip.Mar  5 23:30:23
> ip-10-112-205-125 sshd[3315655]: pam_unix(sshd:session): session closed for
> user git Mar  5 23:30:23 ip-10-112-205-125 sshd[3315640]: Received
> disconnect from 10.254.56.146 port 63239:11: disconnected by user*
>
>- Have you tried running clones manually to see if you can replicate
>the problem? That way you can enable git debugging flags to debug the
>issue. (e.g GIT_CURL_VERBOSE=1)
>
> *I have not done this part, I can try this on the agent.*
>
>- Which exact git version are you using? A Mac pre-installed one, or
>something from Brew or MacPorts?
>
> *Before this issue, we used pre-installed git version on all go-agents. In
> order to resolve this issue, I have installed git with brew on the
> go-agents and started using it. the version is 2.44.0. But the issue still
> exists. on the GoCD,  we are still using pre-installed git, do I need
> install git with brew on the GoCD as well.*
>
>- Are the errors always on agents or always on the server, or both?
>
> *This error is occurring on the agents when the job starts cloning the
> repos mentioned in the materials.*
>
>- What's the filesystem path that GoCD is cloning into when there are
>failures?
>
> *The failures are happening on the agents. the agent path is
> /Users//go-agent-23.1.0/*
> *We installed GoCD in /Applications/.*
>
> *The artifacts path is /Volumes/Artifacts.*
> Please let me know if you need more information.
> Thank you.
>
> On Saturday, March 16, 2024 at 9:37:21 AM UTC-4 Chad Wilson wrote:
>
>> GoCD doesn't do anything special with Git - it just uses whatever git
>> version you have available to the agent/server on the path. If something
>> has changed after the Sonoma upgrade, it is probably affecting all Git
>> usage on the machine.
>>
>>- What are the specific clone errors you are getting?
>>- Have you tried running clones manually to see if you can replicate
>>the problem? That way you can enable git debugging flags to debug the
>>issue. (e.g GIT_CURL_VERBOSE=1)
>>- Which exact git version are you using? A Mac pre-installed one, or
>>something from Brew or MacPorts?
>>- Are the errors 

Re: [go-cd] git clone failures in gocd pipelines after sonoma upgrade

[Chad Wilson]

GoCD doesn't do anything special with Git - it just uses whatever git
version you have available to the agent/server on the path. If something
has changed after the Sonoma upgrade, it is probably affecting all Git
usage on the machine.

   - What are the specific clone errors you are getting?
   - Have you tried running clones manually to see if you can replicate the
   problem? That way you can enable git debugging flags to debug the issue.
   (e.g GIT_CURL_VERBOSE=1)
   - Which exact git version are you using? A Mac pre-installed one, or
   something from Brew or MacPorts?
   - Are the errors always on agents or always on the server, or both?
   - What's the filesystem path that GoCD is cloning into when there are
   failures?

-Chad

On Sat, Mar 16, 2024 at 2:51 AM SATYA R  wrote:

> Hi Team,
>
> Recently, we upgraded all of our go-agents to macos Sonoma from ventura.
> GoCD also runs on Sonoma version.
> After the upgrade, we are seeing intermittent git clone failures in the
> pipelines.
> Each pipeline has 3 or more repos to clone, when multiple pipelines around
> 20 pipelines try to clone the 3 or more repositories from git, we are
> facing this git clone issue.
> sometimes the pipeline are able to clone all the repos but sometimes they
> fail.
>
> is there anything that we need to particularly look into in gocd and
> go-agents configuration due to sonoma upgrade. do we need to make any
> changes in go-agent service  in order to support the sonoma version?
>
> Kindly help us.
>
> Thank you.
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/63efcffd-c537-4855-8039-9de9eafb885en%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH9B7ZCqMopamO9At-wHMcyGhZattbTipwzWToD4q25qeA%40mail.gmail.com.

Re: [go-cd] Is it possible for multiple GoCD servers to use the same database?

[Chad Wilson]

Unfortunately moving to an active-active state would be a huge
architectural change for GoCD, and unfortunately not consistent with its
original design - let alone its current maintainer/community state
 (alive, but not
thriving).

Previously, GoCD had a commercial "business continuity" extension to make
"active-passive" setups easier to automate and reduce "switchover" time.
This was a commercial extension later integrated into the core of GoCD
during its "full" open sourcing in 2020, however post-integration it
had a critical
security defect, and was disabled 
and later removed due to lack of interest/usage.

My current recommendation is generally to use the features of
Kubernetes/container orchestration, load balancers and/or cloud providers
to have an external database and both "shared" artfact file/cloud storage
(AWS EBS, EFS and the like) and database in a way that makes switching the
location the server runs as fast as possible, however making this even more
seamless could also do with some work.

In an ideal state of seamless active/passive switches, GoCD would have a
mode to be able to automatically turn on maintenance mode during shutdown,
gracefully shutdown after n-seconds (even if jobs are still running on some
agents), then start in an alternate location with limited human
intervention (utilising the ability for agents to continue in the
background without a server temporarily).

-Chad

On Fri, Mar 8, 2024 at 9:57 AM Lance Nehring 
wrote:

> Thank you.  The steps you are describing is almost exactly what I do
> today.  However, I think GoCD could see more widespread use, if it were
> possible to run multiple servers simultaneously.  If I had time, I'd try to
> help with such a feature.  Even with this limitation, I'd never go back to
> Jenkins.
>
> On Thu, Mar 7, 2024, 13:05 Sriram Narayanan  wrote:
>
>>
>>
>> On Thu, Mar 7, 2024 at 10:21 PM Lance Nehring 
>> wrote:
>>
>>> I'm wanting to deploy a GoCD server into 2 different K8S clusters and
>>> have them share a database. My goal is to be able performance
>>> maintenance on one K8S cluster without impacting the developers.
>>>
>>> *SO* the question is: can multiple GoCD servers share a database
>>> without interfering with each other?
>>>
>>
>> The two instances will have a unique server ID and should not point to
>> the same database. Please do not attempt this even by making them share the
>> same identity since they will be writing to the database.
>>
>> For higher availability, you'll need to think more about the agents
>> connecting to the server on the second K8s cluster. Agents can continue
>> with their job when a server is unavailable and can reconnect to the server
>> once it is back online.
>>
>> You can leverage this behaviour to achieve some form of HA by:
>> - keeping postgres external to the cluster
>> - stopping the GoCD server on the first cluster
>> - starting the GoCD server with the same configuration on the second
>> cluster
>> - either setting up an LB in front of the first with a fail over to the
>> second K8s cluster
>> OR setting up DNS round robin to refer to the second K8s
>> OR setting a really low TTL for the DNS name and making it point to the
>> second cluster for a switch over.
>>
>>
>>>
>>> Thanks.
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/448aff6b-4bbf-4a5f-a3a2-3289a228dfe6n%40googlegroups.com
>>> 
>>> .
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "go-cd" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/CANiY96basJe4Jiteh-GNsX%2BF5OJzDuCh82Dg-nBLmS%3DsPY7DiA%40mail.gmail.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CAMj%3DmE8akBXupyv4%2B9HbXt-n%3DFZaBfYOi%2B8jJx4hS5Ochnq4QA%40mail.gmail.com
> 
> .
>

-- 
You received this message 

Re: [go-cd] Is it possible for multiple GoCD servers to use the same database?

[Chad Wilson]

Unfortunately not if they are both running at the same time; no. GoCD does
not support active-active configurations like that.

If they otherwise share DB and storage and have the same identity, but one
deployment is scaled to 0 and the other scaled to 1 replica, that is
theoretically possible - but you'd need to co-ordinate very carefully, and
there'd still be brief downtime when you switched instances; similar to if
you just moved the server from one node to another within a single cluster
while doing "maintenance"; so in my personal view not worthwhile.

-Chad

On Fri, Mar 8, 2024 at 12:51 AM Lance Nehring 
wrote:

> I'm wanting to deploy a GoCD server into 2 different K8S clusters and have
> them share a database. My goal is to be able performance maintenance on
> one K8S cluster without impacting the developers.
>
> *SO* the question is: can multiple GoCD servers share a database without
> interfering with each other?
>
> Thanks.
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/448aff6b-4bbf-4a5f-a3a2-3289a228dfe6n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8-hpS44JxCH%3DX3bxX-k6TvzxbruYHRstk2ZWbpe%2BVSOA%40mail.gmail.com.

Re: [go-cd] Wehbook notification plugin

[Chad Wilson]

Good to hear!

On Tue, 20 Feb 2024, 07:06 Sylvain Fabre,  wrote:

> Thanks for your answer !
> After digging a bit, we discovered that Mattermost webhooks are compatible
> with Slack ones. We tested the plugin you mentionned above and ... it works
> :
>
> Thanks !
> (and so it means that the "Slack notification plugin" can also be tagged
> as "Mattermost notification plugin :) )
>
>
> Le lundi 19 février 2024 à 17:21:48 UTC+1, Chad Wilson a écrit :
>
>> That's not necessarily true. All the error tells us is that the plugin
>> couldn't handle a particular request type from the server. We need to know
>> which request type to know if that is a problem. From the gocd-server.log
>> you should look at the log lines before the stack trace - the ones with
>> timestamps and WARN/ERROR. Please include those, or we don't know the
>> context the server was in .
>>
>> More importantly, if the plugin isn't working, it's probably better to
>> describe what you actually observe rather than just what you interpret from
>> the logs. (when stage X completed, I expected Y to happen but instead I
>> observe Z)
>>
>> Keep in mind that a "generic" webhook notification plugin is only going
>> to work in some cases for certain target servers which expect a certain
>> payload being sent to their webhooks which match what the plugin does. To
>> my knowledge there is no such thing as a "generic" webhook standard format,
>> but I haven't looked into it in detail. Mattermost would need to
>> support/expect the same format as the plugin sends - so if the plugin is
>> 'not working' it might be that it's not compatible with Mattermost rather
>> than incompatible with GoCD. I suspect this is more likely to be your
>> problem (the plugin doesn't do what you need it to do) than a GoCD
>> compatibility issue - and that you need a .
>>
>> From a quick look at
>> https://developers.mattermost.com/integrate/webhooks/incoming/ the
>> format it accepts is specific to Mattermost and is not what this "generic"
>> plugin is sending. I believe Mattermost tries to adopt the same format as
>> Slack so you might be better to try
>> https://github.com/ashwanthkumar/gocd-slack-build-notifier (which
>> happens to be written by Ashwanth who also responded on this thread)
>> although no idea if it's Mattermost compatible.
>>
>> -Chad
>>
>> On Tue, Feb 20, 2024 at 12:00 AM Sylvain Fabre 
>> wrote:
>>
>>> Well in fact this log comes from the go-server.log file, but I have a
>>> full error in the log of the plugin itself :
>>>
>>>
>>> 2024-02-19 13:49:34,811 ERROR [qtp1928054064-42]
>>> WebhookNotifierPlugin:127 - Failed to refresh configuration
>>> java.lang.NullPointerException: Cannot invoke
>>> "net.getsentry.gocd.webhooknotifier.Request.ordinal()" because the return
>>> value of "net.getsentry.gocd.webhooknotifier.Request.fromString(String
>>> )" is null
>>>at
>>> net.getsentry.gocd.webhooknotifier.WebhookNotifierPlugin.handle(WebhookNotifierPlugin.java:53)
>>>
>>>at
>>> com.thoughtworks.go.plugin.infra.DefaultPluginManager.lambda$submitTo$0(DefaultPluginManager.java:134)
>>>
>>>at
>>> com.thoughtworks.go.plugin.infra.FelixGoPluginOSGiFramework.executeActionOnTheService(FelixGoPluginOSGiFramework.java:205)
>>>
>>>at
>>> com.thoughtworks.go.plugin.infra.FelixGoPluginOSGiFramework.doOn(FelixGoPluginOSGiFramework.java:164)
>>>
>>>at
>>> com.thoughtworks.go.plugin.infra.DefaultPluginManager.submitTo(DefaultPluginManager.java:131)
>>>
>>>at
>>> com.thoughtworks.go.plugin.access.PluginRequestHelper.submitRequest(PluginRequestHelper.java:49)
>>>
>>>at
>>> com.thoughtworks.go.plugin.access.common.AbstractExtension.notifyPluginSettingsChange(AbstractExtension.java:82)
>>>
>>>at
>>> com.thoughtworks.go.server.service.PluginService.notifyPluginSettingsChange(PluginService.java:191)
>>>
>>>at
>>> com.thoughtworks.go.server.service.PluginService.updatePluginSettingsAndNotifyPluginSettingsChangeListeners(PluginService.java:167)
>>>
>>>at
>>> com.thoughtworks.go.server.service.PluginService.updatePluginSettings(PluginService.java:118)
>>>
>>>at
>>> com.thoughtworks.go.apiv1.pluginsettings.PluginSettingsControllerV1.update(PluginSettingsControllerV1.java:115)
>>>
>>>at sp

Re: [go-cd] Wehbook notification plugin

[Chad Wilson]

That's not necessarily true. All the error tells us is that the plugin
couldn't handle a particular request type from the server. We need to know
which request type to know if that is a problem. From the gocd-server.log
you should look at the log lines before the stack trace - the ones with
timestamps and WARN/ERROR. Please include those, or we don't know the
context the server was in .

More importantly, if the plugin isn't working, it's probably better to
describe what you actually observe rather than just what you interpret from
the logs. (when stage X completed, I expected Y to happen but instead I
observe Z)

Keep in mind that a "generic" webhook notification plugin is only going to
work in some cases for certain target servers which expect a certain
payload being sent to their webhooks which match what the plugin does. To
my knowledge there is no such thing as a "generic" webhook standard format,
but I haven't looked into it in detail. Mattermost would need to
support/expect the same format as the plugin sends - so if the plugin is
'not working' it might be that it's not compatible with Mattermost rather
than incompatible with GoCD. I suspect this is more likely to be your
problem (the plugin doesn't do what you need it to do) than a GoCD
compatibility issue - and that you need a .

>From a quick look at
https://developers.mattermost.com/integrate/webhooks/incoming/ the format
it accepts is specific to Mattermost and is not what this "generic" plugin
is sending. I believe Mattermost tries to adopt the same format as Slack so
you might be better to try
https://github.com/ashwanthkumar/gocd-slack-build-notifier (which happens
to be written by Ashwanth who also responded on this thread) although no
idea if it's Mattermost compatible.

-Chad

On Tue, Feb 20, 2024 at 12:00 AM Sylvain Fabre 
wrote:

> Well in fact this log comes from the go-server.log file, but I have a full
> error in the log of the plugin itself :
>
>
> 2024-02-19 13:49:34,811 ERROR [qtp1928054064-42] WebhookNotifierPlugin:127
> - Failed to refresh configuration
> java.lang.NullPointerException: Cannot invoke
> "net.getsentry.gocd.webhooknotifier.Request.ordinal()" because the return
> value of "net.getsentry.gocd.webhooknotifier.Request.fromString(String
> )" is null
>at
> net.getsentry.gocd.webhooknotifier.WebhookNotifierPlugin.handle(WebhookNotifierPlugin.java:53)
>
>at
> com.thoughtworks.go.plugin.infra.DefaultPluginManager.lambda$submitTo$0(DefaultPluginManager.java:134)
>
>at
> com.thoughtworks.go.plugin.infra.FelixGoPluginOSGiFramework.executeActionOnTheService(FelixGoPluginOSGiFramework.java:205)
>
>at
> com.thoughtworks.go.plugin.infra.FelixGoPluginOSGiFramework.doOn(FelixGoPluginOSGiFramework.java:164)
>
>at
> com.thoughtworks.go.plugin.infra.DefaultPluginManager.submitTo(DefaultPluginManager.java:131)
>
>at
> com.thoughtworks.go.plugin.access.PluginRequestHelper.submitRequest(PluginRequestHelper.java:49)
>
>at
> com.thoughtworks.go.plugin.access.common.AbstractExtension.notifyPluginSettingsChange(AbstractExtension.java:82)
>
>at
> com.thoughtworks.go.server.service.PluginService.notifyPluginSettingsChange(PluginService.java:191)
>
>at
> com.thoughtworks.go.server.service.PluginService.updatePluginSettingsAndNotifyPluginSettingsChangeListeners(PluginService.java:167)
>
>at
> com.thoughtworks.go.server.service.PluginService.updatePluginSettings(PluginService.java:118)
>
>at
> com.thoughtworks.go.apiv1.pluginsettings.PluginSettingsControllerV1.update(PluginSettingsControllerV1.java:115)
>
>at spark.RouteImpl$1.handle(RouteImpl.java:72)
>
> I suspect this error confirms that the plugin is broken with the latest
> GoCD versions.
>
> Le lundi 19 février 2024 à 15:54:08 UTC+1, Chad Wilson a écrit :
>
>> If that error message comes after a log like WARN Error notifying plugin
>> -  with settings change but the plugin otherwise works OK, then you
>> can probably ignore it. There are some optional request types that some
>> plugins don't implement (and don't need to implement), but they don't
>> always handle them so elegantly so they can log errors like the below.
>>
>> If there are other concerns with the plugin or things we want to do with
>> it but it otherwise is working OK, we could consider forking it into the 
>> gocd-contrib
>> organisation <https://github.com/gocd-contrib> to maintain it, since the
>> Sentry folks don't appear to use/maintain it anymore.
>>
>> -Chad
>>
>> On Mon, Feb 19, 2024 at 10:29 PM Sylvain Fabre 
>> wrote:
>>
>>> Sure !
>>>
>

Re: [go-cd] Wehbook notification plugin

[Chad Wilson]

If that error message comes after a log like WARN Error notifying plugin -
 with settings change but the plugin otherwise works OK, then you can
probably ignore it. There are some optional request types that some plugins
don't implement (and don't need to implement), but they don't always handle
them so elegantly so they can log errors like the below.

If there are other concerns with the plugin or things we want to do with it
but it otherwise is working OK, we could consider forking it into the
gocd-contrib
organisation <https://github.com/gocd-contrib> to maintain it, since the
Sentry folks don't appear to use/maintain it anymore.

-Chad

On Mon, Feb 19, 2024 at 10:29 PM Sylvain Fabre 
wrote:

> Sure !
>
> Here is the log when we add a hook URL in the plugin configuration :
>
> Caused by: java.lang.RuntimeException: java.lang.NullPointerException:
> Cannot invoke "net.getsentry.gocd.webhooknotifier.Request.ordinal()"
> because the return value of "net.getsentry.gocd.we
> bhooknotifier.Request.fromString(String)" is null
>at
> net.getsentry.gocd.webhooknotifier.WebhookNotifierPlugin.handle(WebhookNotifierPlugin.java:72)
>
>at
> com.thoughtworks.go.plugin.infra.DefaultPluginManager.lambda$submitTo$0(DefaultPluginManager.java:134)
>
>at
> com.thoughtworks.go.plugin.infra.FelixGoPluginOSGiFramework.executeActionOnTheService(FelixGoPluginOSGiFramework.java:205)
>
>... 159 common frames omitted
> Caused by: java.lang.NullPointerException: Cannot invoke
> "net.getsentry.gocd.webhooknotifier.Request.ordinal()" because the return
> value of "net.getsentry.gocd.webhooknotifier.Request.fromSt
> ring(String)" is null
>at
> net.getsentry.gocd.webhooknotifier.WebhookNotifierPlugin.handle(WebhookNotifierPlugin.java:53)
>
>... 161 common frames omitted
>
>
> The webhook is in place, and has been tested independantly (and is working)
>
> Thanks for your help,
>
>
>
>
> Le lun. 19 févr. 2024 à 14:10, Chad Wilson  a
> écrit :
>
>> Does that plugin really not work?
>>
>> I note the repo has recently been archived, but the plugin was updated
>> relatively recently in 2023 and there haven't been any changes in those
>> plugin APIs for quite a while to my knowledge - nor removal of old versions
>> of the (plugin API) extension points.
>>
>> In case there's a simple answer, might be worth sharing in which way it
>> doesn't work?
>>
>> -Chad
>>
>>
>> On Mon, 19 Feb 2024, 20:46 Sylvain Fabre,  wrote:
>>
>>> Hi there !
>>>
>>> GoCD is a great tool, and we would like now to send Webhook
>>> notifications to our Mattermost server.
>>> It seems that this plugin
>>> https://github.com/getsentry/gocd-webhook-notification-plugin used to
>>> do the job, but it does not work anymore with latest GoCD releases.
>>>
>>> Do you know if there is another solution to push webhooks ? Is someone
>>> willing to update this plugin to the latest GoCD APIs  (sponsoring
>>> possible) ?
>>>
>>> Thanks !
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/f36cfd42-923e-44de-a5a6-68e12119bd5fn%40googlegroups.com
>>> <https://groups.google.com/d/msgid/go-cd/f36cfd42-923e-44de-a5a6-68e12119bd5fn%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "go-cd" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/go-cd/Wjwn06iKv-0/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/CAA1RwH8OgUjAGXVW%2Btd0C_%2BGsJigaqAJauyQnqt9iUeFvcQepA%40mail.gmail.com
>> <https://groups.google.com/d/msgid/go-cd/CAA1RwH8OgUjAGXVW%2Btd0C_%2BGsJigaqAJauyQnqt9iUeFvcQepA%40mail.gmail.com?utm_medium=email_source=footer>
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the w

Re: [go-cd] Wehbook notification plugin

[Chad Wilson]

Does that plugin really not work?

I note the repo has recently been archived, but the plugin was updated
relatively recently in 2023 and there haven't been any changes in those
plugin APIs for quite a while to my knowledge - nor removal of old versions
of the (plugin API) extension points.

In case there's a simple answer, might be worth sharing in which way it
doesn't work?

-Chad


On Mon, 19 Feb 2024, 20:46 Sylvain Fabre,  wrote:

> Hi there !
>
> GoCD is a great tool, and we would like now to send Webhook notifications
> to our Mattermost server.
> It seems that this plugin
> https://github.com/getsentry/gocd-webhook-notification-plugin used to do
> the job, but it does not work anymore with latest GoCD releases.
>
> Do you know if there is another solution to push webhooks ? Is someone
> willing to update this plugin to the latest GoCD APIs  (sponsoring
> possible) ?
>
> Thanks !
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/f36cfd42-923e-44de-a5a6-68e12119bd5fn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8OgUjAGXVW%2Btd0C_%2BGsJigaqAJauyQnqt9iUeFvcQepA%40mail.gmail.com.

Re: [go-cd] Performance of popups in the gui

[Chad Wilson]

Hiya folks

I've been able to replicate this problem and should be able to fix it for a
subsequent release - thanks for the help debugging.

The problem appears to arrive when there is a large number of pipelines*
mapped to an environment;* and also a large number of agents for that
environment. The logic for calculating the API response agents >
environments is accidentally very, very inefficient (I think it's O(n^2 x
m^2) or something crazy. I replicated something similar to what you
describe with 5,000 pipelines and 60 or so agents, all mapped into the
same, single, logical environment.

[image: image.png]


In your case if you have all say 1,690 pipelines mapped to a single
environment (from your stats below), and all of your 95 agents are in the
same environment, you'd definitely trigger this issue. I can't tell exactly
from what you have shared how the pipelines and agents are mapped to
environments, so this is a guess - can you confirm how many agents and
pipelines are mapped to the environment below?

"Number of pipelines": 1690,
"Number of environments": 1,
"Number of agents": 95,


If it's the same problem, you will probably find that *untagging the agents
from the environment *also has a similar speed-up effect to deleting all of
the agents (although then the pipelines requiring that environment won't
schedule either, obviously).

Another workaround in the meantime, *if you don't rely on the environment*

   - to define environment variables/secure environment variables that
   apply across all pipelines/jobs
   - to affect whether jobs are scheduled to special agents

... may be to untag all pipelines and agents from the environment you use
and just use the default/empty environment.

-Chad

On Fri, Feb 16, 2024 at 3:40 PM 'Wolfgang Achinger' via go-cd <
go-cd@googlegroups.com> wrote:

> > By 'resources' I am referring to the GoCD functionality where you can
> tag agents with resources that they offer, which are then matched to
> pipeline jobs that say they *require* those resources to run as part of
> agent assignment.
> 10 Agents have 5 resources attached
> 85 have 1 resource attached
>
> We use the resources to different special agents. They do the same as the
> rest, but they are placed in dedicated networks.
>
> > To make sure I understand you, are you saying that the problem has been
> here for the last year, perhaps gradually getting worse a story add more
> agents or pipelines - but not an issue suddenly created after a particular
> upgrade or change?
> That's correct. It's more an over-time issue than a sudden issue.
>
> I sent the additional information out, but not directly, they come from a
> different mail address over a secure transfer method.
>
> Am Do., 15. Feb. 2024 um 17:57 Uhr schrieb Chad Wilson <
> ch...@thoughtworks.com>:
>
>> Cool, thanks! Just trying to gather enough information to see if I can
>> replicate or find the issue in a dedicated chunk of time this weekend.
>>
>> You can email it to me, and/or encrypt with my GPG key if you'd like (
>> https://github.com/chadlwilson/chadlwilson/blob/main/gpg-public-key.asc)
>>
>> By 'resources' I am referring to the GoCD functionality where you can tag
>> agents with resources that they offer, which are then matched to pipeline
>> jobs that say they *require* those resources to run as part of agent
>> assignment.
>>
>> > No we use this setup no for about a year, patch the system on a regular
>> basis including the latest gocd stable version.
>>
>> To make sure I understand you, are you saying that the problem has been
>> here for the last year, perhaps gradually getting worse a story add more
>> agents or pipelines - but not an issue suddenly created after a particular
>> upgrade or change?
>>
>> -Chad
>>
>> On Fri, 16 Feb 2024, 00:29 'Wolfgang Achinger' via go-cd, <
>> go-cd@googlegroups.com> wrote:
>>
>>> > And how many resources are defined across the agents?
>>> What exactly do you mean here? System resources? XMS XMX Values of java ?
>>>
>>> - Is this a problem that has always been there, or something that has
>>> changed with a GoCD version or other change in environment?
>>> No we use this setup no for about a year, patch the system on a regular
>>> basis including the latest gocd stable version.
>>>
>>> - Is it faster when the server is restarted, and gets slower over time
>>> (or the same after a restart)?
>>> No a restart does not affect the speed at all. It stays constant
>>>
>>> - Why do you feel it is the # of jobs/stages the agents have processed
>>> that is a key factor, rather than simply the # of agents or some o

Re: [go-cd] Performance of popups in the gui

[Chad Wilson]

Cool, thanks! Just trying to gather enough information to see if I can
replicate or find the issue in a dedicated chunk of time this weekend.

You can email it to me, and/or encrypt with my GPG key if you'd like (
https://github.com/chadlwilson/chadlwilson/blob/main/gpg-public-key.asc)

By 'resources' I am referring to the GoCD functionality where you can tag
agents with resources that they offer, which are then matched to pipeline
jobs that say they *require* those resources to run as part of agent
assignment.

> No we use this setup no for about a year, patch the system on a regular
basis including the latest gocd stable version.

To make sure I understand you, are you saying that the problem has been
here for the last year, perhaps gradually getting worse a story add more
agents or pipelines - but not an issue suddenly created after a particular
upgrade or change?

-Chad

On Fri, 16 Feb 2024, 00:29 'Wolfgang Achinger' via go-cd, <
go-cd@googlegroups.com> wrote:

> > And how many resources are defined across the agents?
> What exactly do you mean here? System resources? XMS XMX Values of java ?
>
> - Is this a problem that has always been there, or something that has
> changed with a GoCD version or other change in environment?
> No we use this setup no for about a year, patch the system on a regular
> basis including the latest gocd stable version.
>
> - Is it faster when the server is restarted, and gets slower over time (or
> the same after a restart)?
> No a restart does not affect the speed at all. It stays constant
>
> - Why do you feel it is the # of jobs/stages the agents have processed
> that is a key factor, rather than simply the # of agents or some other
> agent configuration factor?
> I don't know it was more a wild guess. After later testing, i don't think
> this anymore. I cleaned up some tables and reduced the agent history
> visible in the GUI, but this did not affect the speed (Well, it increased
> the speed of the listing of the agent history itself but not the loading
> time of the popups).
>
> If it is ok i will send the support output directly your our mailadress
> so it will not get shared in the thread.
>
> Am Do., 15. Feb. 2024 um 15:50 Uhr schrieb Chad Wilson <
> ch...@thoughtworks.com>:
>
>> And how many resources are defined across the agents?
>>
>> Can you please answer the earlier questions I asked as well? It's rather
>> difficult to efficiently help if you don't respond to the questions that
>> characterise the problem from a maintainer perspective. :-)
>>
>> - Is this a problem that has always been there, or something that has
>> changed with a GoCD version or other change in environment?
>> - Is it faster when the server is restarted, and gets slower over time
>> (or the same after a restart)?
>> - Why do you feel it is the # of jobs/stages the agents have processed
>> that is a key factor, rather than simply the # of agents or some other
>> agent configuration factor?
>>
>> Additionally, can you share a redacted output from /go/api/support ? You
>> can enter the URL in the browser when logged in as an admin. Be careful of
>> the "Runtime Information" and "System Health Information" sections when
>> sharing. These are the two main places which might leak unintentional
>> information from your setup. Redact the individual values which feel
>> sensitive to you.
>>
>> -Chad
>>
>>
>> On Thu, Feb 15, 2024 at 10:17 PM 'Wolfgang Achinger' via go-cd <
>> go-cd@googlegroups.com> wrote:
>>
>>> 1 environment
>>> 164 materials
>>> 0 elastic agents
>>> 2 config repos
>>> 0 artifact stores
>>> 0 pluggable scms
>>>
>>> Am Do., 15. Feb. 2024 um 15:01 Uhr schrieb Chad Wilson <
>>> ch...@thoughtworks.com>:
>>>
>>>> How many distinct environments and resources do you have across these
>>>> 1200 pipelines, roughly?
>>>>
>>>> On Thu, Feb 15, 2024 at 5:38 PM 'Wolfgang Achinger' via go-cd <
>>>> go-cd@googlegroups.com> wrote:
>>>>
>>>>> Additional information
>>>>> Since the pipelines are configured via ~150 yaml files.
>>>>> I tested it now with one big, merged config file with all pipelines
>>>>> But this did not change anything,
>>>>> performance slow.
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "go-cd" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to go-cd+

Re: [go-cd] Performance of popups in the gui

[Chad Wilson]

And how many resources are defined across the agents?

Can you please answer the earlier questions I asked as well? It's rather
difficult to efficiently help if you don't respond to the questions that
characterise the problem from a maintainer perspective. :-)

- Is this a problem that has always been there, or something that has
changed with a GoCD version or other change in environment?
- Is it faster when the server is restarted, and gets slower over time (or
the same after a restart)?
- Why do you feel it is the # of jobs/stages the agents have processed that
is a key factor, rather than simply the # of agents or some other agent
configuration factor?

Additionally, can you share a redacted output from /go/api/support ? You
can enter the URL in the browser when logged in as an admin. Be careful of
the "Runtime Information" and "System Health Information" sections when
sharing. These are the two main places which might leak unintentional
information from your setup. Redact the individual values which feel
sensitive to you.

-Chad


On Thu, Feb 15, 2024 at 10:17 PM 'Wolfgang Achinger' via go-cd <
go-cd@googlegroups.com> wrote:

> 1 environment
> 164 materials
> 0 elastic agents
> 2 config repos
> 0 artifact stores
> 0 pluggable scms
>
> Am Do., 15. Feb. 2024 um 15:01 Uhr schrieb Chad Wilson <
> ch...@thoughtworks.com>:
>
>> How many distinct environments and resources do you have across these
>> 1200 pipelines, roughly?
>>
>> On Thu, Feb 15, 2024 at 5:38 PM 'Wolfgang Achinger' via go-cd <
>> go-cd@googlegroups.com> wrote:
>>
>>> Additional information
>>> Since the pipelines are configured via ~150 yaml files.
>>> I tested it now with one big, merged config file with all pipelines
>>> But this did not change anything,
>>> performance slow.
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/CANhjCLCY1Gsq8fef%2Bb0t8bHSfvgoZHdFHFK%2B1eWzBxYJFjqM3g%40mail.gmail.com
>>> <https://groups.google.com/d/msgid/go-cd/CANhjCLCY1Gsq8fef%2Bb0t8bHSfvgoZHdFHFK%2B1eWzBxYJFjqM3g%40mail.gmail.com?utm_medium=email_source=footer>
>>> .
>>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "go-cd" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/go-cd/c1n1Aq7hG1k/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/CAA1RwH8Vb0U5YPQNB4Qzf2d6kP8KiYRBsgXr1Jux3xEMEN_H5A%40mail.gmail.com
>> <https://groups.google.com/d/msgid/go-cd/CAA1RwH8Vb0U5YPQNB4Qzf2d6kP8KiYRBsgXr1Jux3xEMEN_H5A%40mail.gmail.com?utm_medium=email_source=footer>
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CANhjCLBiz4s%3DqpKr1Yrgw1TSyfRePkcdGXPesfrmAiu2e9aN6g%40mail.gmail.com
> <https://groups.google.com/d/msgid/go-cd/CANhjCLBiz4s%3DqpKr1Yrgw1TSyfRePkcdGXPesfrmAiu2e9aN6g%40mail.gmail.com?utm_medium=email_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-jix9BM1k24yCedriGXryO7zRiMtEXaOxOmqZ-PamU3g%40mail.gmail.com.

Re: [go-cd] Performance of popups in the gui

[Chad Wilson]

How many distinct environments and resources do you have across these 1200
pipelines, roughly?

On Thu, Feb 15, 2024 at 5:38 PM 'Wolfgang Achinger' via go-cd <
go-cd@googlegroups.com> wrote:

> Additional information
> Since the pipelines are configured via ~150 yaml files.
> I tested it now with one big, merged config file with all pipelines
> But this did not change anything,
> performance slow.
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CANhjCLCY1Gsq8fef%2Bb0t8bHSfvgoZHdFHFK%2B1eWzBxYJFjqM3g%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8Vb0U5YPQNB4Qzf2d6kP8KiYRBsgXr1Jux3xEMEN_H5A%40mail.gmail.com.

Re: [go-cd] Performance of popups in the gui

[Chad Wilson]

Hi Hans

Is this slowness specific to builds that are currently running? Or is the
slowness (and speed up with deleted agents) the same even with
builds/stages that are no longer running (even if the agents are)?

Anything majorly different about this GoCD server setup compared to the
others, e.g # of agents in the DB (deleted or not) or # of pipeline runs or
configuration/version - that type of thing?

To narrow things down a bit more, it's worth opening your browser Inspect
and looking at the network tab when clicking the button and seeing which of
the requests are slow:

[image: image.png]

On the assumption that it's the server that's slow rather than the UI, that
will at least tell us which API is slow, and thus which code path or DB
queries to look at contributing to the 5-6 seconds (or whether it is all of
them).

If it's much faster when the agents are deleted, my guess is that it could
be to do with the logic to essentially allow the agents to be linked, but I
am not sure intuitively which logic that might be.

-Chad

On Mon, Feb 12, 2024 at 11:19 PM 'Hans Dampf' via go-cd <
go-cd@googlegroups.com> wrote:

> Hello,
>
> we notice slow loads in the gui when you open the popup of a stage.  The
> loading animation in the popup takes 5-6 seconds on every stage, and we try
> to narrow down why.
>
> What we know so far is if we disable *and* delete all agents in the gui
> the loading is instant. With the agents registered (imported db table
> "agents"), the load time is back to several seconds.
>
> Gocd does the following sql statment in the database. 'update agents set
> deleted=true where uuid in("alot uuids")'
>
> If I execute the statement directly in the database, it does not increase
> the performance. You have to delete the agents via gui.
>
> The database backend is a postgres 14.
> We have several other gocd setups on the same databaseserver without this
> problem.
>
> The setup consists of 19 agent-servers with 5 agents each, overall 95
> agents.
> Every agent did process ~ 15000 stages so far.
>
> I haven+t figured out at the moment what exactly gocd does during the
> creation of the popup. But it must have something to do with the agents
> and/or the agent runtime history.
>
> Maybe someone can point me in the right direct where the bottleneck could
> be.
>
> Regards
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/502ce13f-1696-490c-aa91-6ca542f7e78cn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-wCHVD9UJQ_ZPXMkXWKRi6vsLRR9tMMFdAZMiTk6hySQ%40mail.gmail.com.

Re: [go-cd] API on pipeline instance returns 404

[Chad Wilson]

Hiya David

Did you look at https://api.gocd.org/19.1.0/#get-pipeline-instance ?

You need *GET /go/api/pipelines/:pipeline_name/instance/:pipeline_counter *to
get an instance of a pipeline run - that should work?

-Chad

On Mon, Feb 12, 2024 at 8:46 PM David DOS SANTOS 
wrote:

> Hello,
>
> I'm running an old version of GoCd Server (19.1.0), which I cannot upgrade
> for the moment.
> For the first time in years, I need to get specific info from pipelines,
> so I'm playing with GoCD API, and i really appreciate the documentation on
> this.
>
> So far, I tried with success the "pipeline history" endpoint :
> *GET /go/api/pipelines/:pipeline_name/history  *
>
> But when I need to get info on a specific instance :
> *GET /go/api/pipelines/:pipeline_name/:pipeline_counter*
>
>  I have the following error message:
> {
> "message": "The resource you requested was not found!"
> }
>
> I tried various indexes/counters numbers, which I know are correct, but I
> always get the same message.
> I also tried to suffix whith the stagename and stage_counter, but the
> error remains.
>
> I checked on web, but ressources are really low on this great DevOps tool.
>
> Thanks for you help.
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/07072bb1-208b-4d19-b154-fe1c262a524dn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-vnkdPc9Bp%2BZYLNWcMZLszY1Y7et63vnWmitRaOyhhZA%40mail.gmail.com.

Re: [go-cd] question on pipeline and view ordering

[Chad Wilson]

1) Pipelines on dashboard should be ordered for display by

- pipeline group | environment, in order that you see it in the config file
(may be different for pipelines-as-code implied groups|envs)
- then by pipeline as implied by
 - order it is in config file
 - pipelines-as-code: display_order for a pipeline
 - order implied by config repo definition (plugin dependent, normally
order of definition for yaml/json)

To my knowledge if not using pipelines-as-code, hacking the XML is the only
way to change the display order of groups and pipelines within that group -
which is irritating. /go/admin/pipelines doesn't seem to have this ability.

2) Views are user specific, not in the config file. If I recall correctly,
there is only a default view that effectively gets created for each user.
Users can change their view order around how they want themselves.

- Chad

On Thu, Jan 25, 2024 at 8:30 PM Josh  wrote:

> hey thanks again to all who help make gocd, including those on this forum.
>
> Two related questions, I believe I know answer to one and maybe it's the
> answer to the other.
>
>
>1. order of pipelines in the web ui
>2. order of the views in web ui (or default view)
>
> Is there a way to change the order?
>
> I'd guess it might be as simple as them being read in order, so if you
> edit the xml to change order the pipeline configurations are given, is this
> the best way to change pipeline order?
>
> Likewise for the pipeline views is there a way to make one view the
> default?
> I have not yet looked at the views in the config file, but are they
> similiar where i can just make my preferred view defined first in the gocd
> config?
>
> appreciate any help
>
> ps/aside last year reimplemented about majority of our pipelines using
> from-scratch templates.  was able to make them probably half as complex
> just from a tiny little bit of extra understanding of gocd (acquired
> through years and conversations, including this group).  amazing!
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/cd0eabb2-0849-4c3f-96d4-3f012a24cd52n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH9QTEFBUOeDuCgaNWbYX8h%3Dw3w-nyeRvhC4MqWyGXEFhQ%40mail.gmail.com.

Re: [go-cd] Are optional artifacts possible ?

[Chad Wilson]

It's a reasonable question. Possible different way of thinking about this:

When you have varying numbers of artifacts that are related to one another,
put them in a directory together and artifact the whole directory as a
single artifact (on the upload side). That way it is not dependent on any
individual log, result etc. Either use scripting to collect them together
post build (pre artifact) or change build scripting to ensure things all go
into "one place".

If I recall correctly, you can still pull sub-paths of a given artifact in
dependent pipelines rather than the "whole thing", in a way that the
dependent job will fail if the artifact didn't include a given file/folder.
(if I am wrong on this, this is not a great suggestion) This would allow to
weaken some of the "contract" mentioned below?

-Chad


On Thu, Jan 11, 2024 at 12:29 PM 'Alexey Savchkov' via go-cd <
go-cd@googlegroups.com> wrote:

> Can we come back to this point in 2024? Optional artifacts is a valid and
> natural case and are common in other CI systems. For example, when running
> tests I want to collect core dumps if they occur. Another example is a job
> which depending on its input parameters produces a varying number of
> artifacts (optional debug files, logs, reports or similar). The necessity
> to make dummy artifacts in all such cases only to satisfy GoCD internal
> logic is troublesome and disturbing. Could you please consider adding an
> "optional" attribute of the artifact element?
>
> четверг, 25 июня 2015 г. в 20:10:09 UTC+6, Aravind SV:
>
>> On Thu, Jun 25, 2015 at 7:19 AM, Pat McGrath  wrote:
>>
>>> So it seems unless you generate content the job will fail, is it
>>> possible to indicate that the artifact is optional ?
>>>
>>
>> No. :( The artifact declaration is a contract, something the downstream
>> pipelines can depend on.
>>
>> You can have a final task which creates dummy files, I suppose. But, that
>> feels like cheating. If a downstream pipeline uses it, then it will need to
>> validate that it is not a dummy file.
>>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/aa2762ac-18e9-4fff-a7bc-37a7a92be694n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8hmiS8%3DDidFVTqjAERs6-tG%3Dnmed_O5%2B5cuY%3DPzj0%2BpQ%40mail.gmail.com.

Re: [go-cd] Mac agent unable to connect to server

[Chad Wilson]

There's no error shown in that log file, which just means that the
bootstrapper/launcher is starting OK - but doesn't tell us much else.

You need to look at the other agent logs to see why it is failing to
register with the server. Look in the logs directory for the others (for
most problems you want to look at go-agent.log rather than
go-agent-bootstrapper.log or go-agent-launcher.log).

-Chad

On Sat, Jan 6, 2024 at 6:47 AM Manojsai katakam 
wrote:

> Hi all,
>
> We have an issue to Mac agent connecting to the gocd server.
> Go-server version: 20.5.0
> Go-agent version: 20.5.0
>
> Logs :
>
> *$hostname:Go Agent di$ ./bin/go-agent console*
> Running go-agent...
> wrapper  | --> Wrapper Started as Console
> wrapper  | Java Service Wrapper Standard Edition 64-bit 3.5.41
> wrapper  |   Copyright (C) 1999-2019 Tanuki Software, Ltd. All Rights
> Reserved.
> wrapper  | http://wrapper.tanukisoftware.com
> wrapper  |   Licensed to ThoughtWorks for GoCD Agent
> wrapper  |
> wrapper  | Launching a JVM...
> jvm 1| WrapperManager: Initializing...
> jvm 1| [Fri Jan 05 16:31:18 CST 2024] Starting process:
> jvm 1| [Fri Jan 05 16:31:18 CST 2024]   Working directory:
> /path/to/Go Agent
> jvm 1| [Fri Jan 05 16:31:18 CST 2024]   Application arguments:
> [-serverUrl, http://192.168.60.122:8154/go]
> jvm 1| [Fri Jan 05 16:31:18 CST 2024]GoCD Version:
> 20.5.0-11820
> jvm 1| [Fri Jan 05 16:31:18 CST 2024]Java Version: 11.0.21
> jvm 1| [Fri Jan 05 16:31:18 CST 2024]Operating System: Mac OS
> X(13.3.1)
> jvm 1| Could not find file `config/agent-bootstrapper-logback.xml'.
> Attempting to load from classpath.
> jvm 1| Using classpath resource
> `jar:onejar:lib/agent-bootstrapper-20.5.0-11820-classes.jar!/config/agent-bootstrapper-logback.xml'.
> jvm 1| SLF4J: Class path contains multiple SLF4J bindings.
> jvm 1| SLF4J: Found binding in
> [jar:onejar:lib/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> jvm 1| SLF4J: Found binding in
> [jar:file://path/to/Go%20Agent/data/deps-5225b8a899113f2c-agent-launcher.jar/logback-classic-1.2.3.jar!/org/slf4j/impl/StaticLoggerBinder.class]
> jvm 1| SLF4J: See http://www.slf4j.org/codes.html#multiple_bindings
> for an explanation.
> jvm 1| SLF4J: Actual binding is of type
> [ch.qos.logback.classic.util.ContextSelectorStaticBinder]
> jvm 1| Could not find file `config/agent-launcher-logback.xml'.
> Attempting to load from classpath.
> jvm 1| Using classpath resource
> `jar:file:/path/to/Go%20Agent/data/deps-5225b8a899113f2c-agent-launcher.jar/agent-launcher-20.5.0-11820-classes.jar!/config/agent-launcher-logback.xml
>
> Thanks & Regards,
> Manoj K
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CAEbMZ9sfq0GnY0n-1wDTLyjoinK4E89j2SNohkRqEuFNZuk7nA%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-Q5xYx3UMczrwDQzYhEgjiAJ0NkW1B26QYeU8%3DNpXQ1Q%40mail.gmail.com.

Re: [go-cd] Is possible to run task parallel

[Chad Wilson]

Indeed, you need the ability to have multiple agents running in parallel to
be able to do multiple jobs in parallel. The best way to achieve this
depends on 1) your installation/deployment choice 2) your agent
requirements 3) whether you are experimenting or thinking about a
production deployment.

If you want to use "static" agents directly as processes at OS level, there
is
https://docs.gocd.org/current/advanced_usage/admin_install_multiple_agents.html

Otherwise you can use "elastic" (dynamically created) agents
 via a
plugin , e.g to run multiple
agents on a single host within Docker (here
), or if you
are familiar with Kubernetes running GoCD or its agents inside Kubernetes
(use either multiple static agents, or elastic agents - even for
experimenting locally).

-Chad

On Thu, Jan 4, 2024 at 5:40 PM Sriram Narayanan  wrote:

> No
>
> On Thu, 4 Jan 2024 at 3:02 PM, Vijay A  wrote:
>
>>
>> I have installed GoCD server. And Installed one GoCD Agent.
>>
>> Now i created a new pipeline : Pipeline1
>>
>> Created one stage : Stage1
>>
>> Now two jobs: Job1 and Job2
>>
>> As i am having one GoAgent, does these two jobs will be executed
>> parallely.is there any way to achieve this ?
>>
>> Yes, add a second good agent. Ensure that both the go agents and The job
> are marked as the same “resource”.
>
>
> --
>> You received this message because you are subscribed to the Google Groups
>> "go-cd" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/b832fcba-6af7-4516-8420-3e97055bf486n%40googlegroups.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CANiY96aAwLpWERzOoeecEksG0NY1im3V4X6FQO9HReC%3DMXQYAA%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8QWsOFZ8SnevVkXScP01bCFKUt6g--vWpY5Q%2B9zXo%2BDg%40mail.gmail.com.

[go-cd] Release Announcement - 23.5.0

[Chad Wilson]

Hello everyone,

A new release of GoCD  (23.5.0) is
out.

This release is another relatively minor maintenance release, fixing a
minor UX regression from 23.4.0 when interacting with job console logs
alongside a few other older issues. As always, please remember to take a
backup before upgrading.

To know more about the features and bug fixes in this release, see the release
notes  or head to the downloads page
 to try it. Feedback and ideas are always
welcome - we appreciate the discussion on issues you are having, and how we
can improve things.

Cheers,
Chad & Aravind

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH_KHDo7bwSP8ebiygO0ow2D%3D9_Ct2xCb%3DU9kLx2rhcVrg%40mail.gmail.com.

Re: [go-cd] Repo Sync issues

[Chad Wilson]

Yeah, sorry, that artifact was probably unintentionally included in the
metadata. It'll be fixed in the next release when the metadata is corrected.

I'm not familiar with Foreman, but is it possible to filter out this
artifact (NessusAgent) with a content view or similar?

-Chad

On Wed, Dec 6, 2023 at 10:59 PM Miguel Santos  wrote:

> Hey guys,
>
> We would like to mirror your repository http://download.gocd.org, but
> your metadata file is pointing to files that do not have the correct
> permissions?
>
> We are trying to sync with Foreman http://download.gocd.org and the task
> fails because the first file you reference:
>
> http://linux.duke.edu/metadata/common; xmlns:rpm="
> http://linux.duke.edu/metadata/rpm; packages="161">
> 
>   NessusAgent
>   x86_64
>   
>pkgid="YES">e29737f08f303a268bcdb5f73dded015c7df9ba343a04182e4fd58665e764b47
>   Nessus Agent, supported by Tenable
>   The Nessus Client Agent
>   Renaud Deraison rderai...@tenablesecurity.com
> 
>   http://www.nessus.org
>   
>   
>   
>   
> Proprietary
> 
> Administration Tools
> ip-10-254-69-199.eu.build.aws.tenablesecurity.com
> 
> NessusAgent-10.1.0-amzn.src.rpm
> 
> 
>rel="amzn"/>
>ver="10.1.0" rel="amzn"/>
> 
> 
>   
>   
>   
>   
>   
>   
>   
>   
>   
>   
>   
>   
>   
>   
>   
>   
>   
>   
>   
>   
>   
>   
>   
>   
> 
> /etc/rc.d/init.d/nessusagent
> /opt/nessus_agent/bin/nasl
> /opt/nessus_agent/sbin/nessus-service
> /opt/nessus_agent/sbin/nessuscli
> /opt/nessus_agent/sbin/nessusd
>   
> 
>
>
> Gives a 403: tenable-agents/NessusAgent-10.1.0-amzn.x86_64.rpm
> 403 Forbidden
>
>- Code: AccessDenied
>- Message: Access Denied
>- RequestId: 6QRJB3V4160FY9SK
>- HostId:
>
> dA5S81S4IDNE1Ek981rVMIRH3+sKqLPRMOcjLCB2igR3PrHV+4ZrIBMRgfQnmWWX2hX7NH74D/w=
>
>
> Can you help us sort this out or confirm if you do not allow the mirroring
> of your repository as policy?
>
> Thank you.
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/f885e2f1-13eb-4d91-bf20-e45475b59ba0n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-O8KvTNDgJsv1bghewZNk73t%2BbwCwX6B2Ui9bRufWdAA%40mail.gmail.com.

Re: [go-cd] Re: GOCD Email server configuration

[Chad Wilson]

Do you have the SMTPS setting enabled? IIRC needs to be enabled for port
465 on Gmail but something might have changed.

And which GoCD version (if you have tried both and they don't work on port
465)

On Thu, 23 Nov 2023, 14:40 Vijayakumaran A., <
vijayakumara...@praniontech.com> wrote:

> HI PFA,
>
> jvm 1| 2023-11-23 06:39:41,213 ERROR [Thread-2818] GoSmtpMailSender:63
> - Sending failed for email [Go Email Notification] to [u...@gmail.com]
> jvm 1| jakarta.mail.MessagingException: Got bad greeting from SMTP
> host: smtp.gmail.com, port: 465, response: [EOF]
> jvm 1|  at
> org.eclipse.angus.mail.smtp.SMTPTransport.openServer(SMTPTransport.java:2231)
> jvm 1|  at
> org.eclipse.angus.mail.smtp.SMTPTransport.protocolConnect(SMTPTransport.java:729)
> jvm 1|  at jakarta.mail.Service.connect(Service.java:345)
> jvm 1|  at
> com.thoughtworks.go.config.GoSmtpMailSender.send(GoSmtpMailSender.java:58)
> jvm 1|  at
> com.thoughtworks.go.config.BackgroundMailSender.lambda$send$0(BackgroundMailSender.java:44)
> jvm 1|  at java.base/java.lang.Thread.run(Unknown Source)
>
> On Thursday, November 23, 2023 at 11:30:29 AM UTC+5:30
> ashwant...@gmail.com wrote:
>
>> Can you please share the relevant gocd-server logs that would likely have
>> more information which is useful for debugging.
>>
>> Thanks,
>>
>> On Thu, 23 Nov 2023 at 11:25, Vijayakumaran A. <
>> vijayak...@praniontech.com> wrote:
>>
>>> ps:Before 2 month back im used the same that time its works that was
>>> installed by apt ubuntu package.Now we are using GOCD by docker.
>>>
>>> On Thursday, November 23, 2023 at 11:17:14 AM UTC+5:30 Vijayakumaran A.
>>> wrote:
>>>
 Team,I was setting up an email server configuration with my smtp server
 credentials but its throw error.my credentials are correct. Why am I having
 this error ?

 There was an unknown error performing the operation. Possible reason
 (Not Acceptable)




 --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+un...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/5d47e887-8e96-45c0-8e0f-0ab02706278an%40googlegroups.com
>>> 
>>> .
>>>
>>
>>
>> --
>>
>> Ashwanth Kumar / ashwanthkumar.in
>>
>> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/1ee6c52b-b972-4918-b501-dde6f60bb41en%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-z4n-9yxa9t9%3DrTArZikzbtBw4Jpqf8OKfMFHQ_UiH5A%40mail.gmail.com.

Re: [go-cd] LDAP Group Authentication/Roles/Permissions

[Chad Wilson]

Editing agent attributes via the UI requires wider server administration
permissions. Don't think there is anything finer grained specifically for
agent administration.

Generally speaking, to automate tagging resources and environments to
agents it is done on the agent side configuration itself via "auto
registration":
https://docs.gocd.org/current/advanced_usage/agent_auto_register.html.

You can then subsequently control which jobs are allowed to use which
logical environments and resources (i.e which agents they are able to be
scheduled on) when using pipelines as code on the permissions for a config
repository - but I do not believe that finer grained control is available
if users use the GoCD UI or APIs to edit their pipelines/jobs (i.e they
have direct edit/admin permissions for pipeline groups).

-Chad

On Wed, Nov 22, 2023 at 10:11 PM  wrote:

> Do you know if this plugin allows any configuration for static agent
> modify/admin permissions? Its doing exactly what I was looking for and
> mapping permissions from roles, and also applying the role permissions for
> pipeline groups. I’m trying to see if I can give certain users permissions
> to add resource tags or assign environments to agents without giving them
> full admin access to the server.
>
>
>
> Thanks!
>
>
>
> *From:* chant...@gmail.com 
> *Sent:* Wednesday, November 8, 2023 2:00 PM
> *To:* go-cd@googlegroups.com
> *Subject:* RE: [go-cd] LDAP Group Authentication/Roles/Permissions
>
>
>
> Thanks! I’ll take a look. We are using the bundled version.
>
>
>
> *From:* go-cd@googlegroups.com  *On Behalf Of *Chad
> Wilson
> *Sent:* Wednesday, November 8, 2023 1:09 PM
> *To:* go-cd@googlegroups.com
> *Subject:* Re: [go-cd] LDAP Group Authentication/Roles/Permissions
>
>
>
> There are multiple LDAP plugins, so it depends which one you are referring
> to. Sounds like you might want to look at
> https://github.com/gocd/gocd-ldap-authorization-plugin rather than the
> bundled 'authentication-only' version?
>
>
>
> -Chad
>
>
>
> On Thu, 9 Nov 2023, 05:33 Funkycybermonk,  wrote:
>
> Hello!
>
> I'm trying to manage a pool of users that is going to change over time and
> their permissions across multiple GoCD servers. (regional server split)
>
> I can add a group into permissions using the LDAP plugin, but it doesn't
> seem initially like the user permissions are inherited or managed by that
> group membership. Is it possible to do group based permissions from AD or
> does it have to be per-user?
>
> I'm trying to minimize work since we'll have to manually replicate the
> roles and permissions across several servers.
>
> Thanks!
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/4183dab6-4dad-4fd3-9055-01333843d0dbn%40googlegroups.com
> <https://groups.google.com/d/msgid/go-cd/4183dab6-4dad-4fd3-9055-01333843d0dbn%40googlegroups.com?utm_medium=email_source=footer>
> .
>
> --
> You received this message because you are subscribed to a topic in the
> Google Groups "go-cd" group.
> To unsubscribe from this topic, visit
> https://groups.google.com/d/topic/go-cd/YXdA8U4UNEY/unsubscribe.
> To unsubscribe from this group and all its topics, send an email to
> go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CAA1RwH8fUnhEOODV7im%2BVU_xkTfkTEDkmpvsz_bhmDGcLrfWJA%40mail.gmail.com
> <https://groups.google.com/d/msgid/go-cd/CAA1RwH8fUnhEOODV7im%2BVU_xkTfkTEDkmpvsz_bhmDGcLrfWJA%40mail.gmail.com?utm_medium=email_source=footer>
> .
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/063a01da1d4d%24d23be980%2476b3bc80%24%40gmail.com
> <https://groups.google.com/d/msgid/go-cd/063a01da1d4d%24d23be980%2476b3bc80%24%40gmail.com?utm_medium=email_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8-uVFze56viifrbiYTUStdp6%2BG87EmZQadv4dyy6Q3yA%40mail.gmail.com.

Re: [go-cd] Ruby on Rails-Server-Version end-of-life

[Chad Wilson]

Hiya Erik

Plans, yes: https://github.com/gocd/gocd/pull/12077 - but it's not EOL
*quite* yet :P. Arguably there are riskier pieces of EOL software within
GoCD than Rails right now (Spring Security, Spring Framework) and if things
get messy I'm more inclined to focus on pieces with better risk/effort
ratios.

The main "messiness" blocker with Rails 7+ right now is that GoCD currently
relies on jruby-rack which is very lightly maintained and Rack's Rails
integration, and seems to be broken in some way with Rails 7. It's not
exactly my personal core expertise, so if anyone has some clue with
Ruby/Rails/Rack these days, help would be appreciated. What is confusing to
me may be simple to you!

Unfortunately, there are not too many "GoCD people" actively working on
things left to speak of :-)

-Chad



On Wed, Nov 22, 2023 at 4:42 PM Erik Wölfel  wrote:

> Dear Gocd-People,
>
> first of all we love gocd and admire your work :-)
>
> Our security system mentions gocd using end-of-life software having the
> ruby on rails-server on version 6.1.7.6 which will be out of security
> support in half a year (1st June 24, https://endoflife.date/rails)
>
> Are there any plans on upgrading to the latest version 7.1?
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/96cadc38-e56b-4812-8b37-27ac8fe189f3n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH_VACi8P%3D1yOXE9DxngGQOcY0JYmjmqOrWCsb0yNMRiTg%40mail.gmail.com.

Re: [go-cd] Material branch check exclusion?

[Chad Wilson]

I think it wouldn't be so much that the *material polling* on an upstream
would affect a downstream pipeline, but more that if a build runs upstream
(regardless of trigger type), when it triggers the downstream pipeline it
possibly triggers a material check on the other materials that pipeline has
as inputs, to determine which revision the other materials should use. Kind
of like when you manually trigger a pipeline (with the "play" button) GoCD
will trigger with the latest revisions of the materials on that pipeline,
regardless of whether you have polling enabled or disabled - meaning it
triggers a modification check.

However if the same *logical* Git material (generally determined by
identical URL and branch) is used in different pipelines whether connected
to each other or not, then the polling setting of one can definitely affect
others, I believe. Generally speaking GoCD complains at you when there are
conflicting values for the same logical material and asks you to make them
all the same, but I have seen cases where it gets confused or ends up
allowing inconsistent values here, and behaviour becomes indeterminate.
This is one of the testing challenges with fixing/allowing the change in
https://github.com/gocd/gocd/pull/11636 which would make it easier to
support such autoUpdate=false repos from pipelines-as-code. If the concern
is the triggering (rather than spurious errors from deleting branches as in
your case) the workaround some folks use for this is to put **/* material
deny lists on pipelines, so even if the material is polled and updated it
shouldn't cause a trigger since the denylists/allowlists are processed in
the context of an individual pipeline, rather than the (de-duplicated)
material itself.

But yes, the "default" is autoUpdate=true/polling.

-Chad

On Sat, Nov 18, 2023 at 1:27 AM 'Chris Gillatt' via go-cd <
go-cd@googlegroups.com> wrote:

> Thanks Chad
>
> I've done some more investigation and you're right - there's some upstream
> pipelines set to "Regularly fetch updates to this repository" rather than
> our company strategy of "Fetch updates only on webhook or manual trigger".
> I wasn't aware that upstream material configurations could affect
> downstream ones.  We rely on templates for the majority of our pipeline
> creations, and use an in-house tool (relying on GoCD APIs) that explicitly
> specify 'autoUpdate="false"'.  It's been a while since I've created a
> pipeline from the GoCD wizard or from scratch, so not sure if this has
> always been the case, but through the GoCD GUI Create Pipeline workflow,
> the material configuration is selected *before* a template is chosen,
> meaning templates cannot control advanced material configuration, such as
> autoUpdate="false".  Unless the user selects this from the material
> Advanced menu at the time, the default is essentially 'autoUpdate="true"'.
> I think that a bunch of these has crept in since I last cleared them up.
>
> For posterity for anyone else reading this, some contextual info is useful
> to know:
>
> 1. When "Regularly fetch updates to this repository" is selected, it is
> not explicitly expressed in the XML.  Its absence on the line beginning
>  simply delete the key value pair, and "Regularly fetch updates to this
> repository" is selected.
> 2. When "Fetch updates to this repository only on webhook or manual
> trigger" is selected, on the line beginning  3. This property must be the same over all pipelines which use the same
> material.  You cannot have one pipeline which pulls and one which receives
> a push for the same material.
> On Friday, November 17, 2023 at 2:54:32 AM UTC Chad Wilson wrote:
>
>> In theory if you have disabled polling on all usages of the material (via
>> its URL/branch combination), and there are no triggers from webhooks, APIs
>> or elsewhere then it shouldn't be trying to do modification checks that
>> generate such an error.
>>
>> [image: image.png]
>> Having said that, I wouldn't *entirely* be surprised if there is some
>> other kind of trigger for an update/modification check, e.g as a side
>> effect of an upstream pipeline completing (if there is one) or a server
>> restart, even if that is not intended.
>>
>> If you have disabled that polling everywhere and are still seeing the
>> error appear, it'd be interesting to know if you can figure out when it is
>> triggered.
>>
>> -Chad
>>
>>
>> On Fri, Nov 17, 2023 at 5:51 AM 'Chris Gillatt' via go-cd <
>> go...@googlegroups.com> wrote:
>>
>>> Hi all
>>>
>>> Is there a way to exclude a pipeline or pipeline group from the material
>>> checks?
>>>
>>> [image: S

Re: [go-cd] GOCD Version Upgrade Issue - Windows Agents Failing to Execute 'git pull'

[Chad Wilson]

Hi, the screenshot didn't come through correctly, so it would be good if
you can include that.

>From looking at the XML, If I recall correctly, it might be that you were
using the "legacy"/historical syntax for handling exec commands, but need
to dig a bit more. With this syntax it is somewhat indeterminate how spaces
should be handled. However, I don't think it was intentionally broken by
the changes I mentioned in 22.2.0 - although it looks like it might be the
case for you. I'd have to validate separately whether you were just using
incorrect configuration that happened to work. It should look like this on
the UI if you want to do it this way without using a separate script.

[image: image.png]
You might want to try editing the command via the UI, and then editing it
back in case the syntax needs converting, which will look like:

  
pull

  

-Chad

On Sat, Nov 11, 2023 at 2:38 AM Sri Sai Sruthi JAGANNADDAM <
sjagannad...@mdsol.com> wrote:

> Here is the screenshot
>
> git pull & mkdir reports 2nul & IF DEFINED APP_IR_GENERATOR
> (%APP_IR_GENERATOR%)" are failing as they are not able to recognize git &
> mkdir, its not a agent issue because the same agent when attached to the
> gocd server running on 21.2 version it works fine but not working on 23.3
> version, as you mentioned custom commands does not allow spaces in the new
> gocd version, what is best way to handle this, thanks for your inputs
> backend code:
>
> 
>   
> 
>   
>pipeline="" stage="Build" job="defaultJob">
> 
>   
>   
> 
>   
>   
> 
>   
>       
> 
>   
>   
>
> [image: image.png]
> On Friday, November 10, 2023 at 12:15:03 AM UTC-6 Chad Wilson wrote:
>
>> It's possibly a side effect of fixing issues with spaces in windows
>> commands in GoCD 22.2.0:  https://www.gocd.org/releases.html#22-2-0
>>
>> It may be that some of your command tasks were *technically* incorrectly
>> configured on earlier versions but worked as a side effect of these bugs in
>> GoCD, and GoCD is now stricter. Or it may be that there is a regression in
>> some circumstances.
>>
>> Can you please
>> - share a screenshot of the UI pipeline task configuration for the 'git
>> pull' example above?
>> - find that specific task's XML definition inside admin > config XML and
>> share the XML fragment for the task or job (you can redact any sensitive
>> text, I am mainly interested in the 'shape' of the XML)
>>
>> If my suspicion is correct, a workaround will certainly be possible to
>> correct the task config, but as there are a couple of historical ways tasks
>> could be configured, I'd like to understand what your config looked like
>> before you workaround the problem so I can see if this is avoidable.
>>
>> -Chad
>>
>>
>> On Fri, 10 Nov 2023, 15:55 Sri Sai Sruthi JAGANNADDAM, <
>> sjagan...@mdsol.com> wrote:
>>
>>>  Could any one please help with this - I am writing to report a
>>> challenge we’ve encountered following our recent upgrade from GOCD version
>>> 21.2 to version 23.3. During our testing phase, we observed that certain
>>> pipelines, which execute on Windows agents, are failing with the following
>>> error when run through "Custom Command"
>>>
>>>  [go] Task: git pull
>>> Took: 0.179s
>>> Exited: 1
>>> '"git pull"' is not recognized as an internal or external command,
>>> operable program or batch file.
>>>
>>>
>>> This issue seems to be isolated to the Windows agents as the pipelines
>>> running on Linux agents are performing without any issues.
>>> The same pipelines were functioning correctly on the previous version of
>>> the GOCD server,
>>> which suggests a potential configuration or environment variable path
>>> discrepancy in the new version.
>>>
>>>
>>> --
>>> The information in this email and any attachments are intended solely
>>> for the recipient(s) to whom it is addressed, and may be confidential
>>> and/or privileged. Any unauthorized distribution or copying of this
>>> transmittal or its attachments is prohibited. If you are not a named
>>> recipient or have received this email in error: (i) you should not read,
>>> disclose, or copy it, (ii) please noti

Re: [go-cd] Docker Elastic agent plugin .jar location (to run it in the ECS)

[Chad Wilson]

While getting a new plugin initially configured, I would use the UI to
begin with, yes.

Your `ls` command starts with 'gocd-' so is filtering out plugins that
don't start with the same (such as the docker elastic agent plugin). If the
version and list of plugins is correct on the admin > plugins list, you can
rely on that.

Your other thread mentions an intention to use ECS containers as agents.
Note that there is a different ECS-specific plugin to use if you want to
use ECS and have GoCD manage instances for agents to run on. The docker
elastic agent plugin will run agent instances as containers on an existing
individual local or remote host but is not intended for ECS usage which
require specific API usage to create task definitions etc. Just to be clear.

-Chad

On Wed, 15 Nov 2023, 10:14 Satya Elipe,  wrote:

> Thank you Chad, so what's the best approach to add the profiles, server
> console/UI ?
>
> Also, I'm trying to understand this:
> I see v3.2.3 on the console () and the container shows the old version
> that's 2.2.0 (ec2 plugin and not docker)
> Attached are the screenshots for both, am I missing something ?
>
> This is how I run the container and volume mounts are from the original
> backed-up server to the container.
> ```
> docker run -d -p 8153:8153 -p 8154:8154 \
>   -v /etc/go:/etc/go \
>   -v /var/lib/go-server:/var/lib/go-server \
>   -e GOCD_PLUGIN_INSTALL_docker-elastic-agents=
> https://github.com/gocd-contrib/docker-elastic-agents-plugin/releases/download/v3.2.3-399/docker-elastic-agents-3.2.3-399.jar
> \
>   --name gocd-server \
>   gocd/gocd-server:v22.3.0```
>
> Sorry about cruise-config.xml, looks like I have sent the old config.
>
> Best
> Satya
>
>
> On Tue, Nov 14, 2023 at 8:18 PM Chad Wilson 
> wrote:
>
>> The pluginId in that cluster profile fragment refers to a completely
>> different ec2 elastic agent plugin so not sure what you're trying to show
>> us there 
>>
>> Personally, I don't think it's a good idea to be directly configuring
>> cruise-config.xml especially for plugin-oriented config, since it
>> necessarily cannot be schema validated. Elastic agent config is typically
>> complex with validation rules implemented in the plugin. If automation is
>> the goal, using the APIs for automation is likely a better experience than
>> cruise-config.xml hacking.
>>
>> If you really want to control cruise config directly, start with a
>> working UI-driven configuration and work backwards to the required
>> cruise-config XML fragment.
>>
>> Otherwise you'll need to look at the logs and there's no guarantee as to
>> how friendly the error messages will be, as you are shortcutting all of the
>> UI-based validation assistance with that approach.
>>
>> -Chad
>>
>> On Wed, 15 Nov 2023, 04:16 Satya Elipe,  wrote:
>>
>>> Thank you Chad, as suggested I have used v3.2.3 in the container and it
>>> appears in the console.
>>>
>>> Im now into configuring cluster and agents profiles and trying to build
>>> them in cruise-config.xml (attached the code snippet), but doesn't seem to
>>> be working as those profiles doesn't appear on the Admin->Elastic Agents
>>> Configurations tab, but its asking for a fresh configuration, wonder what
>>> am I missing? Also, is it still a valid way of configuration ?
>>>
>>> Regards
>>> Satya
>>>
>>>
>>> On Mon, Nov 13, 2023 at 11:08 PM Chad Wilson 
>>> wrote:
>>>
>>>> There's no such release. You should use a non-experimental release from
>>>> https://github.com/gocd-contrib/docker-elastic-agents-plugin/releases
>>>> and find the link to the main jar file within that release.
>>>>
>>>> There should be no reason you cannot use the most recent release
>>>> <https://github.com/gocd-contrib/docker-elastic-agents-plugin/releases/tag/v3.2.3-399>:
>>>>
>>>> https://github.com/gocd-contrib/docker-elastic-agents-plugin/releases/download/v3.2.3-399/docker-elastic-agents-3.2.3-399.jar
>>>>
>>>> -Chad
>>>>
>>>> On Tue, Nov 14, 2023 at 10:50 AM Satya Elipe 
>>>> wrote:
>>>>
>>>>> Hi All
>>>>>
>>>>> Im containerizing my standalone GoCD server v22.3.0 with elastic
>>>>> agents, and looking for the elastic agents plugin jar file location for 
>>>>> the
>>>>> version that works with server version v22.3.0 and use it in the container
>>>>> something like:
>>>>>
>>>>> docker run -d -p 8153:8153 -p 

Re: [go-cd] Agent docker image

[Chad Wilson]

The default go-server docker image is Alpine based, so the equivalent image
would be https://hub.docker.com/r/gocd/gocd-agent-alpine-3.18

Having said this, most folks find they end up building their own agent
image using a GoCD official image as base image, so they can add on
required tooling of their own which they want to be available to jobs.

If you are moving from running builds directly on your EC2 instances you
might want to consider picking a base image closer to your normal Linux
ecosystem or package manager, especially if you do native compiles or
things like that which require a C/C++ compiler toolchain inside agents.
Your choices right now are Alpine, Debian, Ubuntu, CentOS Stream (other
than building your own) https://www.gocd.org/download/#docker

-Chad

On Wed, 15 Nov 2023, 06:27 Satya Elipe,  wrote:

> Thank you Sriram.
> My comments inline please.
>
> Are you planning to run the go server as an ECS instance and The go agent
> as a ECS instance?
> Satya] Yes, server will be part of ECS (ECS instance) and it will launch
> the agents dynamically when theres a job to perform and once the job is
> done the agent is terminated
>
> You should start by understanding the following:
> -  how the containerised server and agent work
> Satya] If Im not wrong, this goes with docker elastic plugin, configuring
> cluster and agent profiles, setting up agentAutoRegisterKey in
> cruise-config.yml and the below configuration in the cruise-config.yml.
>
> Agent: this would go into agent container
> ```
>   ec2_user_data
>   echo
> "agent.auto.register.environments=production,sandbox" 
> /var/lib/go-agent/config/autoregister.properties
> ```
>
> So, we should have this file updated with the required details including
> the agentAutoRegisterKey (as given here
> 
> ), so agent would know what server it is working with, correct me if I am
> wrong please.
>
> - auto registration of the agent
> Satya] as mentioned above.
>
> - scaling ECS instances ( for the agent)
> - addressing a server instance via an ALB
> - the security group settings needed to enable the agent ECS cluster to
> connect to the server on port 8153.
> - using RDS for the server data ( including migrating from the H2DB to RDS)
> - storing files on EFS and making available to the go server instance (
> artifacts, configuration, logs)
> - sending logs to cloud watch
>
> Satya] Yes, mostly set up is the same, but just that whether H2DB or RDS .
>
> Thank you
> Satya
>
> On Fri, Nov 10, 2023 at 4:31 PM Sriram Narayanan 
> wrote:
>
>>
>>
>> On Sat, 11 Nov 2023 at 12:00 AM, Satya Elipe 
>> wrote:
>>
>>> Hi All
>>>
>>> Any recommendation for go-agent docker image like we have for go-server
>>> (gocd/go-server:v22.3.0) ?
>>>
>>> We are dockerizing a stand-alone g-server runs as an ec2 at the moment
>>> but would like to move it to ECS.
>>>
>>> any inputs will be appreciated.
>>>
>>
>> Are you planning to run the go server as an ECS instance and The go agent
>> as a ECS instance?
>>
>> You should start by understanding the following:
>> -  how the containerised server and agent work
>> - auto registration of the agent
>> - scaling ECS instances ( for the agent)
>> - addressing a server instance via an ALB
>> - the security group settings needed to enable the agent ECS cluster to
>> connect to the server on port 8153.
>> - using RDS for the server data ( including migrating from the H2DB to
>> RDS)
>> - storing files on EFS and making available to the go server instance (
>> artifacts, configuration, logs)
>> - sending logs to cloud watch
>>
>> You could also consider using the Elastic Agent support to use EKS based
>> agents instead of ECS. See:
>> https://github.com/gocd/kubernetes-elastic-agents
>>
>> — Sriram
>>
>>
>>
>>> Many thanks
>>> Satya
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/CADKEDRrGqba1Oi3Wcu7%2BTJEuKMENkWOc-JbMcKb2XG1Px6BJGQ%40mail.gmail.com
>>> 
>>> .
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "go-cd" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/CANiY96Y8ZwK5fcVeSsySGLXxdLdPNG356_f-BGVWkaa%3DAA%2BX3w%40mail.gmail.com
>> 
>> .
>>
> --
> You received this message because 

Re: [go-cd] Docker Elastic agent plugin .jar location (to run it in the ECS)

[Chad Wilson]

The pluginId in that cluster profile fragment refers to a completely
different ec2 elastic agent plugin so not sure what you're trying to show
us there 

Personally, I don't think it's a good idea to be directly configuring
cruise-config.xml especially for plugin-oriented config, since it
necessarily cannot be schema validated. Elastic agent config is typically
complex with validation rules implemented in the plugin. If automation is
the goal, using the APIs for automation is likely a better experience than
cruise-config.xml hacking.

If you really want to control cruise config directly, start with a working
UI-driven configuration and work backwards to the required cruise-config
XML fragment.

Otherwise you'll need to look at the logs and there's no guarantee as to
how friendly the error messages will be, as you are shortcutting all of the
UI-based validation assistance with that approach.

-Chad

On Wed, 15 Nov 2023, 04:16 Satya Elipe,  wrote:

> Thank you Chad, as suggested I have used v3.2.3 in the container and it
> appears in the console.
>
> Im now into configuring cluster and agents profiles and trying to build
> them in cruise-config.xml (attached the code snippet), but doesn't seem to
> be working as those profiles doesn't appear on the Admin->Elastic Agents
> Configurations tab, but its asking for a fresh configuration, wonder what
> am I missing? Also, is it still a valid way of configuration ?
>
> Regards
> Satya
>
>
> On Mon, Nov 13, 2023 at 11:08 PM Chad Wilson 
> wrote:
>
>> There's no such release. You should use a non-experimental release from
>> https://github.com/gocd-contrib/docker-elastic-agents-plugin/releases
>> and find the link to the main jar file within that release.
>>
>> There should be no reason you cannot use the most recent release
>> <https://github.com/gocd-contrib/docker-elastic-agents-plugin/releases/tag/v3.2.3-399>:
>>
>> https://github.com/gocd-contrib/docker-elastic-agents-plugin/releases/download/v3.2.3-399/docker-elastic-agents-3.2.3-399.jar
>>
>> -Chad
>>
>> On Tue, Nov 14, 2023 at 10:50 AM Satya Elipe 
>> wrote:
>>
>>> Hi All
>>>
>>> Im containerizing my standalone GoCD server v22.3.0 with elastic agents,
>>> and looking for the elastic agents plugin jar file location for the version
>>> that works with server version v22.3.0 and use it in the container
>>> something like:
>>>
>>> docker run -d -p 8153:8153 -p 8154:8154 \
>>>   -v /etc/go:/etc/go \
>>>   -v /var/lib/go-server:/var/lib/go-server \
>>>   -e GOCD_PLUGIN_INSTALL_docker-elastic-agents=
>>> https://github.com/gocd-contrib/docker-elastic-agents/releases/download/v2.2.0-218/docker-elastic-agents-2.2.0-218.jar
>>> \
>>>   --name gocd-server \
>>>   gocd/gocd-server:v22.3.0
>>>
>>> The above command fails with the below error:
>>> ```$ mkdir -p /godata/plugins/external
>>> $ curl --silent --location --fail --retry 3
>>> https://github.com/gocd-contrib/docker-elastic-agents/releases/download/v2.2.0-218/docker-elastic-agents-2.2.0-218.jar
>>> --output /godata/plugins/external/docker-elastic-agents.jar
>>> /usr/local/sbin/install-gocd-plugins: cannot curl --silent --location
>>> --fail --retry 3
>>> https://github.com/gocd-contrib/docker-elastic-agents/releases/download/v2.2.0-218/docker-elastic-agents-2.2.0-218.jar
>>> --output /godata/plugins/external/docker-elastic-agents.jar```
>>>
>>>
>>> I see the documentation available for the same, my bad that I couldn't
>>> locate the jar file:
>>> https://github.com/gocd-contrib/docker-elastic-agents-plugin
>>>
>>> https://github.com/gocd-contrib/docker-elastic-agents-plugin/blob/master/INSTALL.md
>>>
>>> Any inputs will be of great help.
>>>
>>> Many thanks
>>> Satya
>>>
>>> [P.S: Right now the server runs as a standalone ec2 on AWS, plan is to
>>> run it as part of ECS.]
>>>
>>>
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/CADKEDRpTctJDBtWXSJWoEW81ryUwL7cAzeePbuZd4wi7LDmQFg%40mail.gmail.com
>>> <https://groups.google.com/d/msgid/go-cd/CADKEDRpTctJDBtWXSJWoEW81ryUwL7cAzeePbuZd4wi7LDmQFg%40mail.gmail.com?utm_medium=email_source=footer>
>>> .
>>>
>> --
>>

Re: [go-cd] Docker Elastic agent plugin .jar location (to run it in the ECS)

[Chad Wilson]

There's no such release. You should use a non-experimental release from
https://github.com/gocd-contrib/docker-elastic-agents-plugin/releases and
find the link to the main jar file within that release.

There should be no reason you cannot use the most recent release
:
https://github.com/gocd-contrib/docker-elastic-agents-plugin/releases/download/v3.2.3-399/docker-elastic-agents-3.2.3-399.jar

-Chad

On Tue, Nov 14, 2023 at 10:50 AM Satya Elipe  wrote:

> Hi All
>
> Im containerizing my standalone GoCD server v22.3.0 with elastic agents,
> and looking for the elastic agents plugin jar file location for the version
> that works with server version v22.3.0 and use it in the container
> something like:
>
> docker run -d -p 8153:8153 -p 8154:8154 \
>   -v /etc/go:/etc/go \
>   -v /var/lib/go-server:/var/lib/go-server \
>   -e GOCD_PLUGIN_INSTALL_docker-elastic-agents=
> https://github.com/gocd-contrib/docker-elastic-agents/releases/download/v2.2.0-218/docker-elastic-agents-2.2.0-218.jar
> \
>   --name gocd-server \
>   gocd/gocd-server:v22.3.0
>
> The above command fails with the below error:
> ```$ mkdir -p /godata/plugins/external
> $ curl --silent --location --fail --retry 3
> https://github.com/gocd-contrib/docker-elastic-agents/releases/download/v2.2.0-218/docker-elastic-agents-2.2.0-218.jar
> --output /godata/plugins/external/docker-elastic-agents.jar
> /usr/local/sbin/install-gocd-plugins: cannot curl --silent --location
> --fail --retry 3
> https://github.com/gocd-contrib/docker-elastic-agents/releases/download/v2.2.0-218/docker-elastic-agents-2.2.0-218.jar
> --output /godata/plugins/external/docker-elastic-agents.jar```
>
>
> I see the documentation available for the same, my bad that I couldn't
> locate the jar file:
> https://github.com/gocd-contrib/docker-elastic-agents-plugin
>
> https://github.com/gocd-contrib/docker-elastic-agents-plugin/blob/master/INSTALL.md
>
> Any inputs will be of great help.
>
> Many thanks
> Satya
>
> [P.S: Right now the server runs as a standalone ec2 on AWS, plan is to run
> it as part of ECS.]
>
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CADKEDRpTctJDBtWXSJWoEW81ryUwL7cAzeePbuZd4wi7LDmQFg%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH_-2KLLOTYUOuHHVuSNBLO1Ax24n6zyRHTpm6KftaUKkQ%40mail.gmail.com.

Re: [go-cd] GOCD Version Upgrade Issue - Windows Agents Failing to Execute 'git pull'

[Chad Wilson]

It's possibly a side effect of fixing issues with spaces in windows
commands in GoCD 22.2.0:  https://www.gocd.org/releases.html#22-2-0

It may be that some of your command tasks were *technically* incorrectly
configured on earlier versions but worked as a side effect of these bugs in
GoCD, and GoCD is now stricter. Or it may be that there is a regression in
some circumstances.

Can you please
- share a screenshot of the UI pipeline task configuration for the 'git
pull' example above?
- find that specific task's XML definition inside admin > config XML and
share the XML fragment for the task or job (you can redact any sensitive
text, I am mainly interested in the 'shape' of the XML)

If my suspicion is correct, a workaround will certainly be possible to
correct the task config, but as there are a couple of historical ways tasks
could be configured, I'd like to understand what your config looked like
before you workaround the problem so I can see if this is avoidable.

-Chad


On Fri, 10 Nov 2023, 15:55 Sri Sai Sruthi JAGANNADDAM, <
sjagannad...@mdsol.com> wrote:

>  Could any one please help with this - I am writing to report a challenge
> we’ve encountered following our recent upgrade from GOCD version 21.2 to
> version 23.3. During our testing phase, we observed that certain pipelines,
> which execute on Windows agents, are failing with the following error when
> run through "Custom Command"
>
>  [go] Task: git pull
> Took: 0.179s
> Exited: 1
> '"git pull"' is not recognized as an internal or external command,
> operable program or batch file.
>
>
> This issue seems to be isolated to the Windows agents as the pipelines
> running on Linux agents are performing without any issues.
> The same pipelines were functioning correctly on the previous version of
> the GOCD server,
> which suggests a potential configuration or environment variable path
> discrepancy in the new version.
>
>
> --
> The information in this email and any attachments are intended solely for
> the recipient(s) to whom it is addressed, and may be confidential and/or
> privileged. Any unauthorized distribution or copying of this transmittal or
> its attachments is prohibited. If you are not a named recipient or have
> received this email in error: (i) you should not read, disclose, or copy
> it, (ii) please notify the sender of your receipt by reply email and delete
> this email and all attachments.
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/08c6247d-e0c3-4d05-b449-e31c3b899e1dn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH_ky2xbJDscSAz%2BMvypJ_Zva_P-0_P%2BwtHPXc48CrgNVg%40mail.gmail.com.

Re: [go-cd] LDAP Group Authentication/Roles/Permissions

[Chad Wilson]

There are multiple LDAP plugins, so it depends which one you are referring
to. Sounds like you might want to look at
https://github.com/gocd/gocd-ldap-authorization-plugin rather than the
bundled 'authentication-only' version?

-Chad


On Thu, 9 Nov 2023, 05:33 Funkycybermonk,  wrote:

> Hello!
>
> I'm trying to manage a pool of users that is going to change over time and
> their permissions across multiple GoCD servers. (regional server split)
>
> I can add a group into permissions using the LDAP plugin, but it doesn't
> seem initially like the user permissions are inherited or managed by that
> group membership. Is it possible to do group based permissions from AD or
> does it have to be per-user?
>
> I'm trying to minimize work since we'll have to manually replicate the
> roles and permissions across several servers.
>
> Thanks!
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/4183dab6-4dad-4fd3-9055-01333843d0dbn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8fUnhEOODV7im%2BVU_xkTfkTEDkmpvsz_bhmDGcLrfWJA%40mail.gmail.com.

Re: [go-cd] Where can I find the end-of-support documentation for GoCD releases?

[Chad Wilson]

Other than Ram's (accurate) comments about non-use of semantic versioning,
please see https://github.com/gocd/gocd/blob/master/SECURITY.md

Upgrading from 22.3.0 through to 23.3.0 or 23.4.0 is likely to be very low
risk/impact given the slow rate of change in GoCD over the last year.

-Chad

On Wed, 8 Nov 2023, 00:50 Satya Elipe,  wrote:

> Hi All
>
> We are currently on GoCD version 22.3.0 and wonder when is the right time
> to upgrade, so the documentation for end-of-support dates would be a help
> here, many thanks.
>
> Regards
> Satya
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CADKEDRr_8jWerqD8enn%2BcJP7AEVCAs%2BMQijXQ7%2BmZARqeV94YQ%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH_dqivHCgNH2LzXMwGLx0q_t3QhvU7Aei7hg-UmtNR7gQ%40mail.gmail.com.

[go-cd] Release Announcement - 23.4.0

[Chad Wilson]

Hello everyone,

A new release of GoCD  (23.4.0) is
out.

This release is mainly a maintenance release. As always, please remember to
take a backup before upgrading.

Special thanks to Victor Sollerhed (MPV) and k-c-p for helping us debug a
(likely) rather old bug reported in various guises over time, which can
lead to server configuration values being toggled unexpectedly on a GoCD
restart. Hopefully we've nailed it this time!

To know more about the features and bug fixes in this release, see the release
notes  or head to the downloads page
 to try it. Feedback and ideas are always
welcome - we appreciate the discussion on issues you are having, and how we
can improve things.

Cheers,
Chad & Aravind

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH9Ho%3D%3D_7XO8PM_2hh4o9ZrAmePpdr3SjyaxHnZq5assUg%40mail.gmail.com.

Re: [go-cd] Kubernetes Elastic Agents - Pods can no longer be created

[Chad Wilson]

Thanks for sharing. Sounds like it could be a network policy thing or
something like that? Did you double check the cluster profile config to see
if anything is amiss there?

I can't see why you'd get a different error message with 23.1.0 (and the
same plugin version) unless there was something different about the cluster
profile config (API server url etc), so that might be an avenue for
investigation?

-Chad

On Sat, 28 Oct 2023, 05:16 Kim Pham,  wrote:

> Hi,
>
> I just wanted to give an update.  We're still unsure what's preventing
> agents from spinning up on our old cluster.  The pods aren't getting
> created at all and there's no logs in Kubernetes side indicating activity.
> I've tried with a regular Debian agent and the logs are the same.  I did
> roll back to GoCD 23.1.0 to see what the log output was and it's showing a
> SocketTimeoutException error.  We're still investigating but it's possible
> some configuration changed that we're not aware of with that cluster.
>
> I was able to spin up a new cluster and it works fine there.  That cluster
> is running:
>
>- GoCD Server: 23.3.0
>- GoCD DinD Agent: 23.3.0
>- Elastic Agent Node Pools: 1.25.10-gk3.2700
>
> I'll update if there's anything relevant to share.  Thanks for all the
> help!
>
> On Thu, Oct 26, 2023 at 10:55 AM Kim Pham  wrote:
>
>> Hi Chad,
>>
>> I was just reading up on the changes for Kubernetes.  Looks 1.24 moves to
>> containerd runtime images and we are still using DIND for our elastic
>> agents.  That seems like it could be the culprit.  I'll do some testing and
>> change our elastic agent images.  Thanks for pointing that out.
>>
>> On Thu, Oct 26, 2023 at 10:46 AM Chad Wilson 
>> wrote:
>>
>>> Unfortunately the error message is a bit mysterious and useless. Which
>>> agent image you are using? Anything special in the elastic agent pod spec
>>> that might no longer work as expected on Kubernetes 1.24? (e.g use of
>>> docker dind images)
>>>
>>> Does the pod get created (and fail) if you look at the events on the
>>> kubernetes side, or does it never get that far?
>>>
>>> -Chad
>>>
>>> On Thu, Oct 26, 2023 at 11:37 PM Chad Wilson 
>>> wrote:
>>>
>>>> Just curious - were the errors/stack traces on failure essentially
>>>> identical before and after you upgraded your gocd and elastic agent plugin
>>>> versions?
>>>>
>>>> On Thu, Oct 26, 2023 at 11:33 PM Kim Pham  wrote:
>>>>
>>>>> Hi Ashwanth,
>>>>>
>>>>> I checked the clusterrole of the service account it's using and it
>>>>> basically has full access atm.
>>>>>
>>>>> PolicyRule:
>>>>>   Resources   Non-Resource URLs  Resource Names  Verbs
>>>>>   -   -  --  -
>>>>>   events  [] []  [*]
>>>>>   namespaces  [] []  [*]
>>>>>   nodes   [] []  [*]
>>>>>   pods/log[] []  [*]
>>>>>   pods[] []  [*]
>>>>>
>>>>> On Thu, Oct 26, 2023 at 10:21 AM 'Ashwanth Kumar' via go-cd <
>>>>> go-cd@googlegroups.com> wrote:
>>>>>
>>>>>> A wild guess, anything changed on the service account side or a
>>>>>> custom role being added as part of the upgrade that is probably not
>>>>>> allowing the gocd plugin to create the pod?
>>>>>>
>>>>>> Thanks,
>>>>>>
>>>>>>
>>>>>> On Thu, 26 Oct 2023 at 20:27, Kim Pham  wrote:
>>>>>>
>>>>>>> Hi All,
>>>>>>>
>>>>>>> We recently began to encounter issues where pods were unable to be
>>>>>>> created.  Nothing has changed in terms of GoCD server, agent, and
>>>>>>> Kubernetes elastic agent plugin versions.  However, we did notice that 
>>>>>>> the
>>>>>>> cluster went through an automatic upgrade and updated gke version to
>>>>>>> 1.24.14.  GoCD is able to see the node pools through the 'Status Report'
>>>>>>> button.
>>>>>>>
>>>>>>> When attempting to create an agent on those node pools, I do see a
>>>>>>> 500 in the plugin logs and gocd-server logs.  Attached are logs.
>>>>

Re: [go-cd] goCD Yaml plugin doesn't pick-up code

[Chad Wilson]

As the error mentions please "Check the 'Rules' of this config repository"
- by default your config repo won't have rules/permissions to define
pipelines in any pipeline groups, environments etc. It is "secure by
default". You'll want to add a permission to do so.

e.g a blanket permission to allow a config repo to affect any environment
or pipeline group looked like: You can click the "learn more" link from the
config repo edit to read more about these rules.

[image: image.png]
-Chad

On Fri, Oct 27, 2023 at 1:27 PM vv-fork  wrote:

> Greetings colleagues,
>
> I am trying to get into Yaml plugin (*gocd-yaml-config-plugin.jar* v.
> *0.14.3-321*) with goCD v.23.3.0.
>
> The plugin is able to receive the yaml file but always throws the same
> error.
>
> I am stuck with something pretty simple, like this:
> format_version: 10
>
> pipelines:
>   mypipe:
> group: mygroup
> materials:
>   mygit:
> git: http://example.com/mygit.git
> stages:
>   - build:
>   jobs:
> build:
>   tasks:
>- exec:
>command: make
>
> environments:
>
> The message is:
> INVALID MERGED CONFIGURATION Number of errors: 1+ I. Rule Validation
> Errors: 1. Not allowed to refer to pipeline group 'mygroup'. Check the
> 'Rules' of this config repository. II. Config Validation Errors: - For
> Config Repo: https://github.com/...p at revision 03a639fd6444>>>74d5
>
> What am I doing wrong?
>
> Vlad.
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/01c04e91-0322-45fa-9021-58dd90424b80n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH9LXz3xBiN822wKt4GaF8m9232nnjsJvdwMjTf_2VsAiA%40mail.gmail.com.

Re: [go-cd] GoCD and private GitHub repo

[Chad Wilson]

Based on the error message it looks like the clone URL you are using is
still an HTTPS one - to use SSH auth, you need to change it to an ssh URL,
e.g g...@github.com:gocd/gocd.git - it's an entirely different git
transport, not just an auth mechanism so the URL needs to change
accordingly :-)

If you're new to using SSH to talk to a git repo manager, you might want to
try doing it separately on the command line with a git clone before getting
it to work with GoCD as adding in the GoCD server and agent adds some extra
complexity.

-Chad

On Fri, Oct 27, 2023 at 1:13 PM vv-fork  wrote:

> Thank you guys Sriram and Chad for answering those! Now it's getting
> clearer to me
>
> *I was able to connect using token. It's fine.*
>
> Though I* wasn't able* to connect using SSH Certificate.
> The key has been generated and installed:
> [image: unnamed.png]
>
> it seems i set proper permissions:
> [image: unnamed.png]
>
> but i still get that error message when i test connection from go-server:
> --- STANDARD ERR --- STDERR: fatal: could not read Username for '
> https://github.com': No such device or address ---
>
>
> So what am i doing wrong? May that be I have messed with permissions for *go
> user*?
>
>
> Vlad.
>
> On Thursday, October 26, 2023 at 8:41:33 PM UTC+13 Chad Wilson wrote:
>
>> To add on to Sriram's comments, the use of the
>> github-oauth-authorization-plugin doesn't have any relationship with access
>> to repository content on GitHub - it simply allows people to log onto GoCD
>> using their Github identity, and optionally to have access to GoCD pipeline
>> groups mapped to GitHub roles.
>>
>> This is because materials/repositories need to be accessed in an identity
>> known to the GoCD server/agents, not necessarily the individual user who
>> happens to be logged in to GoCD. So even if you use that authorization
>> plugin, you still need to decide how to provide GoCD itself access to
>> repositories on Github.
>>
>> You can use an SSH key linked to a GitHub user
>> <https://docs.github.com/en/authentication/connecting-to-github-with-ssh/adding-a-new-ssh-key-to-your-github-account>
>> if you wish to use SSH access - no restrictions for private repos unless
>> your GitHub org blocks use of SSH keys. If you instead wish to use HTTPS
>> access to repositories you have to fill in a username/"password" for each
>> material you configure. That "password" would be a personal access token
>> <https://docs.github.com/en/authentication/keeping-your-account-and-data-secure/managing-your-personal-access-tokens>
>> with at least read-only access to the 1 or more repositories you want to
>> use.
>>
>> If you want to share one personal access token across many materials
>> (perhaps a single token has read-only access to many repositories), the
>> easiest way is to use a GoCD Secrets Management plugin and refer to them in
>> the username/"password" fields of each material using the special secrets
>> interpolation syntax:
>> https://docs.gocd.org/current/configuration/secrets_management.html This
>> will work with either manually defined pipelines/materials, or those
>> defined externally in source control.
>>
>> -Chad
>>
>> On Thu, Oct 26, 2023 at 3:01 PM Sriram Narayanan 
>> wrote:
>>
>>> Please see:
>>>
>>> https://docs.github.com/en/authentication/connecting-to-github-with-ssh
>>>
>>> The gocd server runs as a particular user account. That user account
>>> needs access to the ssh private keys used to authenticate with GitHub.
>>>
>>> The go agent too needs the same access.
>>>
>>> Assuming you are on Linux and installer gocd via rpm, then you would set
>>> this key in the home directory (
>>> /var/lib/go-server/.ssh/myprivatekey.id_rsa)
>>>
>>> Permissions for .ssh would be 600, and for the key would be 400, with
>>> the gocd process user owning the directory and The identity file.
>>>
>>> — Sriram
>>>
>>>
>>> On Thu, 26 Oct 2023 at 12:00 PM, vv-fork  wrote:
>>>
>>>> Hello colleagues!
>>>>
>>>> What is the best way to connect on-prem goCD with GitHub private repo
>>>> in cloud? I was smoking docs and manuals for quite a while, but what people
>>>> say it’s to install ssh keys to both GitHub and goCD, which won’t work,
>>>> since I am using github.com, so i suppose i can’t install ssh key
>>>> there.
>>>>
>>>> I’ve installed github-oauth-authorization-plugin and set it as
>>>> des

Re: [go-cd] Kubernetes Elastic Agents - Pods can no longer be created

[Chad Wilson]

Unfortunately the error message is a bit mysterious and useless. Which
agent image you are using? Anything special in the elastic agent pod spec
that might no longer work as expected on Kubernetes 1.24? (e.g use of
docker dind images)

Does the pod get created (and fail) if you look at the events on the
kubernetes side, or does it never get that far?

-Chad

On Thu, Oct 26, 2023 at 11:37 PM Chad Wilson  wrote:

> Just curious - were the errors/stack traces on failure essentially
> identical before and after you upgraded your gocd and elastic agent plugin
> versions?
>
> On Thu, Oct 26, 2023 at 11:33 PM Kim Pham  wrote:
>
>> Hi Ashwanth,
>>
>> I checked the clusterrole of the service account it's using and it
>> basically has full access atm.
>>
>> PolicyRule:
>>   Resources   Non-Resource URLs  Resource Names  Verbs
>>   -   -  --  -
>>   events  [] []  [*]
>>   namespaces  [] []  [*]
>>   nodes   [] []  [*]
>>   pods/log[] []  [*]
>>   pods[] []  [*]
>>
>> On Thu, Oct 26, 2023 at 10:21 AM 'Ashwanth Kumar' via go-cd <
>> go-cd@googlegroups.com> wrote:
>>
>>> A wild guess, anything changed on the service account side or a custom
>>> role being added as part of the upgrade that is probably not allowing the
>>> gocd plugin to create the pod?
>>>
>>> Thanks,
>>>
>>>
>>> On Thu, 26 Oct 2023 at 20:27, Kim Pham  wrote:
>>>
>>>> Hi All,
>>>>
>>>> We recently began to encounter issues where pods were unable to be
>>>> created.  Nothing has changed in terms of GoCD server, agent, and
>>>> Kubernetes elastic agent plugin versions.  However, we did notice that the
>>>> cluster went through an automatic upgrade and updated gke version to
>>>> 1.24.14.  GoCD is able to see the node pools through the 'Status Report'
>>>> button.
>>>>
>>>> When attempting to create an agent on those node pools, I do see a 500
>>>> in the plugin logs and gocd-server logs.  Attached are logs.
>>>>
>>>> I've tried updating GoCD and the plugins to latest release versions.
>>>> Our static agents that are running on older gke versions aren't having any
>>>> issues.
>>>>
>>>> Has anyone encountered this?
>>>>
>>>> Thanks in advance.
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "go-cd" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to go-cd+unsubscr...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/go-cd/a6b8e99d-f415-4c18-b67d-e86c3df16733n%40googlegroups.com
>>>> <https://groups.google.com/d/msgid/go-cd/a6b8e99d-f415-4c18-b67d-e86c3df16733n%40googlegroups.com?utm_medium=email_source=footer>
>>>> .
>>>>
>>>
>>>
>>> --
>>>
>>> Ashwanth Kumar / ashwanthkumar.in
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/CAD9m7Cw2cK5yt_r9e5r1sxD%2B%2B%2Bjkd3%3DFTQe51vKqb081MunU%3DQ%40mail.gmail.com
>>> <https://groups.google.com/d/msgid/go-cd/CAD9m7Cw2cK5yt_r9e5r1sxD%2B%2B%2Bjkd3%3DFTQe51vKqb081MunU%3DQ%40mail.gmail.com?utm_medium=email_source=footer>
>>> .
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "go-cd" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/CA%2BnJfx4-Zks6-FOr1bYSOroxG2o4e2e4ir0OFUm3TsggWsrYpA%40mail.gmail.com
>> <https://groups.google.com/d/msgid/go-cd/CA%2BnJfx4-Zks6-FOr1bYSOroxG2o4e2e4ir0OFUm3TsggWsrYpA%40mail.gmail.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-tw%3D0MFCgWRQoT8ZwFYcyJsGT8T5pPxrBzzu46Nn960w%40mail.gmail.com.

Re: [go-cd] Kubernetes Elastic Agents - Pods can no longer be created

[Chad Wilson]

Just curious - were the errors/stack traces on failure essentially
identical before and after you upgraded your gocd and elastic agent plugin
versions?

On Thu, Oct 26, 2023 at 11:33 PM Kim Pham  wrote:

> Hi Ashwanth,
>
> I checked the clusterrole of the service account it's using and it
> basically has full access atm.
>
> PolicyRule:
>   Resources   Non-Resource URLs  Resource Names  Verbs
>   -   -  --  -
>   events  [] []  [*]
>   namespaces  [] []  [*]
>   nodes   [] []  [*]
>   pods/log[] []  [*]
>   pods[] []  [*]
>
> On Thu, Oct 26, 2023 at 10:21 AM 'Ashwanth Kumar' via go-cd <
> go-cd@googlegroups.com> wrote:
>
>> A wild guess, anything changed on the service account side or a custom
>> role being added as part of the upgrade that is probably not allowing the
>> gocd plugin to create the pod?
>>
>> Thanks,
>>
>>
>> On Thu, 26 Oct 2023 at 20:27, Kim Pham  wrote:
>>
>>> Hi All,
>>>
>>> We recently began to encounter issues where pods were unable to be
>>> created.  Nothing has changed in terms of GoCD server, agent, and
>>> Kubernetes elastic agent plugin versions.  However, we did notice that the
>>> cluster went through an automatic upgrade and updated gke version to
>>> 1.24.14.  GoCD is able to see the node pools through the 'Status Report'
>>> button.
>>>
>>> When attempting to create an agent on those node pools, I do see a 500
>>> in the plugin logs and gocd-server logs.  Attached are logs.
>>>
>>> I've tried updating GoCD and the plugins to latest release versions.
>>> Our static agents that are running on older gke versions aren't having any
>>> issues.
>>>
>>> Has anyone encountered this?
>>>
>>> Thanks in advance.
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/a6b8e99d-f415-4c18-b67d-e86c3df16733n%40googlegroups.com
>>> 
>>> .
>>>
>>
>>
>> --
>>
>> Ashwanth Kumar / ashwanthkumar.in
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "go-cd" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/CAD9m7Cw2cK5yt_r9e5r1sxD%2B%2B%2Bjkd3%3DFTQe51vKqb081MunU%3DQ%40mail.gmail.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CA%2BnJfx4-Zks6-FOr1bYSOroxG2o4e2e4ir0OFUm3TsggWsrYpA%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH9L%3DmBaJbvnXDfJ0d3J2_GuhC%2B38y4sMUiaMPDgAEd__Q%40mail.gmail.com.

Re: [go-cd] GoCD and private GitHub repo

[Chad Wilson]

To add on to Sriram's comments, the use of the
github-oauth-authorization-plugin doesn't have any relationship with access
to repository content on GitHub - it simply allows people to log onto GoCD
using their Github identity, and optionally to have access to GoCD pipeline
groups mapped to GitHub roles.

This is because materials/repositories need to be accessed in an identity
known to the GoCD server/agents, not necessarily the individual user who
happens to be logged in to GoCD. So even if you use that authorization
plugin, you still need to decide how to provide GoCD itself access to
repositories on Github.

You can use an SSH key linked to a GitHub user

if you wish to use SSH access - no restrictions for private repos unless
your GitHub org blocks use of SSH keys. If you instead wish to use HTTPS
access to repositories you have to fill in a username/"password" for each
material you configure. That "password" would be a personal access token

with at least read-only access to the 1 or more repositories you want to
use.

If you want to share one personal access token across many materials
(perhaps a single token has read-only access to many repositories), the
easiest way is to use a GoCD Secrets Management plugin and refer to them in
the username/"password" fields of each material using the special secrets
interpolation syntax:
https://docs.gocd.org/current/configuration/secrets_management.html This
will work with either manually defined pipelines/materials, or those
defined externally in source control.

-Chad

On Thu, Oct 26, 2023 at 3:01 PM Sriram Narayanan 
wrote:

> Please see:
>
> https://docs.github.com/en/authentication/connecting-to-github-with-ssh
>
> The gocd server runs as a particular user account. That user account needs
> access to the ssh private keys used to authenticate with GitHub.
>
> The go agent too needs the same access.
>
> Assuming you are on Linux and installer gocd via rpm, then you would set
> this key in the home directory (
> /var/lib/go-server/.ssh/myprivatekey.id_rsa)
>
> Permissions for .ssh would be 600, and for the key would be 400, with the
> gocd process user owning the directory and The identity file.
>
> — Sriram
>
>
> On Thu, 26 Oct 2023 at 12:00 PM, vv-fork  wrote:
>
>> Hello colleagues!
>>
>> What is the best way to connect on-prem goCD with GitHub private repo in
>> cloud? I was smoking docs and manuals for quite a while, but what people
>> say it’s to install ssh keys to both GitHub and goCD, which won’t work,
>> since I am using github.com, so i suppose i can’t install ssh key there.
>>
>> I’ve installed github-oauth-authorization-plugin and set it as described
>> (connection ok in authorisation configuration step), and restarted the
>> server, however it’s still throwing that standard error “fatal: could not
>> read Username for ‘https://github.com’ meaning that the access is still
>> closed.
>>
>> What else can be done as you think?
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "go-cd" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/ed3022b6-e1ec-4c3b-8ca3-3c5e6b7d72f4n%40googlegroups.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CANiY96azM2%3DaFO351d4PpExOatRCO%2BoaQju3Juvm2yAbQR2d5A%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-R9v39GDB_Rs98pXnh0x7xyMZKwanye7Mcq%3D7hSdA1tQ%40mail.gmail.com.

Re: [go-cd] POC setting up

[Chad Wilson]

Hiya Gowtham

It looks like you downloaded and are trying to run the *x64 version*
on an *Apple
Silicon* mac which will likely give errors like that. You probably want the
*aarch64* version from https://www.gocd.org/download/#osx

As a side note, the easiest way to POC GoCD locally is probably via the
Test Drive experience at https://www.gocd.org/test-drive-gocd.html which
downloads and runs both server and agent locally and includes some demo
pipelines.

If you have a reason to specifically try with a separate agent running on
EC2 rather than locally, you can run it manually like you are doing or
connect it to the test drive server. Getting the agent to connect back to
your locally running server will need you to use some tool to other tool
expose it locally or make it accessible to the agent via something
like cloudflared
tunnel --url http://localhost:8153, editing the servers "site URL" and
"server site URL" to the Cloudflare temporary domain, and then changing the
agent config to locate the server on the same URL.

-Chad

On Tue, Oct 10, 2023 at 6:36 AM Gowtham Ravipati 
wrote:

> Hello Community,
> Hope this was a great start of week for everyone..!
>
> I have been having hard time, installing the GoCD server on my MAC, I am
> tying to install server on my MAC and use and EC2 instance as an agent and
> try to get them live on my dashboard and use them accordingly.
> https://docs.gocd.org/current/installation/install/server/osx.html
> ./bin/go-server start
> Unable to locate any of the following binaries:
>
> /Users/P3142304/Downloads/go-server-23.3.0/bin/../wrapper/wrapper-macosx-arm-64
>
> /Users/P3142304/Downloads/go-server-23.3.0/bin/../wrapper/wrapper-macosx-arm-32
>   /Users/P3142304/Downloads/go-server-23.3.0/bin/../wrapper/wrapper
> I also tried with sudo and installing with the go-server install command...
> I was wondering if anyone else have faced this and see if I can get
> through this POC.
>
> Thanks
> Gowtham
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/c45cdc88-e047-4bb8-9edc-48ada4068bddn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8%3DqnF%3DqqE4qL_FGe3JmmPdS-LgynSMipXQbmEAJxOKzQ%40mail.gmail.com.

Re: [go-cd] Seeking clarification with GoCD Release Notes

[Chad Wilson]

For almost all users, what they think of as the 'agent' (as installed) is
actually just a bootstrapping process that downloads the 'real' agent code
from the current server version, thus matching that server version's code.

So sometimes it depends what one means by 'agent changes'. In a general
sense, almost all of the behaviour of an agent while running jobs is
defined and packaged within the server and its plugins which is probably
why it's not separated.

-Chad

On Fri, 6 Oct 2023, 01:01 Cory McKelvey,  wrote:

> No particular question, just looking for agent changes and needed
> clarification for internal skeptics like me.
>
> On Thursday, October 5, 2023 at 12:48:23 PM UTC-4 Chad Wilson wrote:
>
>> Yeah, that's correct.
>>
>> Any particular question you're trying to answer?
>>
>> -Chad
>>
>> On Fri, 6 Oct 2023, 00:36 Cory McKelvey,  wrote:
>>
>>> Does https://www.gocd.org/releases/ act as release notes for both
>>> gocd-server and gocd-agents?
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+un...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/9001d529-bc05-46ff-b238-d2b1cf5b76f7n%40googlegroups.com
>>> <https://groups.google.com/d/msgid/go-cd/9001d529-bc05-46ff-b238-d2b1cf5b76f7n%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/8a6165a0-55f2-46fa-bcea-c018aac23a98n%40googlegroups.com
> <https://groups.google.com/d/msgid/go-cd/8a6165a0-55f2-46fa-bcea-c018aac23a98n%40googlegroups.com?utm_medium=email_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8a1ZG4A8mJi7MDnaJTJHXTKBuisn9gNVixE%3Det%2B6E-hg%40mail.gmail.com.

Re: [go-cd] Seeking clarification with GoCD Release Notes

[Chad Wilson]

Yeah, that's correct.

Any particular question you're trying to answer?

-Chad

On Fri, 6 Oct 2023, 00:36 Cory McKelvey,  wrote:

> Does https://www.gocd.org/releases/ act as release notes for both
> gocd-server and gocd-agents?
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/9001d529-bc05-46ff-b238-d2b1cf5b76f7n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH9DcWM7ZgjwrQT2tasrPN7DAsskaj1qmMSp7SGQEJ2hTQ%40mail.gmail.com.

Re: [go-cd] Enterprise Support for GoCD

[Chad Wilson]

Not that I know of, if you are referring to classic "enterprise" support,
involving commitment to resolve issues, patch bugs etc. At the moment GoCD
is an entirely community project, meaning getting help from the community
and online resources.

Outside of community support or traditional enterprise product support
models, depending on their scale & needs, some folks may choose to work
with a generic managed services or software development vendor who is
familiar with GoCD and/or similar tools, of which Thoughtworks (my
employer) is one.

-Chad

On Mon, Oct 2, 2023 at 10:47 PM Gowtham Ravipati 
wrote:

> Hello Chad,
> Thank you so much for the response.
>
> So, if we were to get some support in the process even setting up a Demo
> or any thing while Migration as well...
> Is there any Point of Contact / Vendor for such Support?
>
> Regards
> Gowtham Ravipati
>
> On Saturday, September 30, 2023 at 3:38:58 AM UTC-6 Chad Wilson wrote:
>
>> Hi Gowtham - To my knowledge there has been no change in this situation.
>>
>> -Chad
>>
>> On Wed, 27 Sept 2023, 23:50 Gowtham Ravipati, 
>> wrote:
>>
>>> Hello
>>>
>>> Is this still the case for GOCD? or are there any vendors or from
>>> Thoughtworks itself that would have some Enterprise/Premiums support?
>>>
>>> Regards
>>> Gowtham Ravipati
>>> Charter Communications
>>> On Tuesday, June 28, 2022 at 9:20:56 AM UTC-6 Sriram Narayanan wrote:
>>>
>>>> On Tue, 28 Jun 2022 at 2:34 PM, 'Sarmistha Bhuyan' via go-cd <
>>>> go...@googlegroups.com> wrote:
>>>>
>>>>> Hi,  Thanks for the detail.
>>>>
>>>>
>>>> Please write to me : sriramNRN at gmail
>>>>
>>>>
>>>>>
>>>>> On Monday, June 27, 2022 at 4:05:23 PM UTC+5:30 Chad Wilson wrote:
>>>>>
>>>>>> Hi there
>>>>>>
>>>>>> Thoughtworks no longer provides commercial support or an enterprise
>>>>>> offering around GoCD - this ended 18 months ago in December 2020 with the
>>>>>> closure of Thoughtworks Studios, and the open sourcing of all the
>>>>>> previously "commercial" add-ons/plugins for GoCD. See
>>>>>> https://groups.google.com/g/go-cd/c/EXwfvZZeLrM/m/ONU1fjwGCgAJ for
>>>>>> background context.
>>>>>>
>>>>>> From a development perspective, it is currently a fully open source
>>>>>> community project and personally I'm not aware of folks providing
>>>>>> commercial support for it, or otherwise sponsoring development. You may
>>>>>> want to take that into account for your evaluation.
>>>>>>
>>>>>> -Chad
>>>>>>
>>>>>> PS: *Sent from my personal perspective as a GoCD contributor in my
>>>>>> spare time - not to be taken as an official statement of Thoughtworks*
>>>>>> .
>>>>>>
>>>>>> On Mon, Jun 27, 2022 at 5:56 PM 'Sarmistha Bhuyan' via go-cd <
>>>>>> go...@googlegroups.com> wrote:
>>>>>>
>>>>>>> We are evaluating GoCD as a continuous delivery and release
>>>>>>> orchestration tool.
>>>>>>>
>>>>>>> We would like to connect and understand the enterprise offerings in
>>>>>>> terms of support and professional services. I tried to write
>>>>>>> *sup...@thoughtworks.com* but did not get any response.
>>>>>>>
>>>>>>> Any idea how and whom to contact for this?
>>>>>>>
>>>>>>> --
>>>>>>> You received this message because you are subscribed to the Google
>>>>>>> Groups "go-cd" group.
>>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>>> send an email to go-cd+un...@googlegroups.com.
>>>>>>> To view this discussion on the web visit
>>>>>>> https://groups.google.com/d/msgid/go-cd/f1c56f07-8c09-4d9a-890b-2755bf9a8ed4n%40googlegroups.com
>>>>>>> <https://groups.google.com/d/msgid/go-cd/f1c56f07-8c09-4d9a-890b-2755bf9a8ed4n%40googlegroups.com?utm_medium=email_source=footer>
>>>>>>> .
>>>>>>>
>>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups

Re: [go-cd] Enterprise Support for GoCD

[Chad Wilson]

Hi Gowtham - To my knowledge there has been no change in this situation.

-Chad

On Wed, 27 Sept 2023, 23:50 Gowtham Ravipati, 
wrote:

> Hello
>
> Is this still the case for GOCD? or are there any vendors or from
> Thoughtworks itself that would have some Enterprise/Premiums support?
>
> Regards
> Gowtham Ravipati
> Charter Communications
> On Tuesday, June 28, 2022 at 9:20:56 AM UTC-6 Sriram Narayanan wrote:
>
>> On Tue, 28 Jun 2022 at 2:34 PM, 'Sarmistha Bhuyan' via go-cd <
>> go...@googlegroups.com> wrote:
>>
>>> Hi,  Thanks for the detail.
>>
>>
>> Please write to me : sriramNRN at gmail
>>
>>
>>>
>>> On Monday, June 27, 2022 at 4:05:23 PM UTC+5:30 Chad Wilson wrote:
>>>
>>>> Hi there
>>>>
>>>> Thoughtworks no longer provides commercial support or an enterprise
>>>> offering around GoCD - this ended 18 months ago in December 2020 with the
>>>> closure of Thoughtworks Studios, and the open sourcing of all the
>>>> previously "commercial" add-ons/plugins for GoCD. See
>>>> https://groups.google.com/g/go-cd/c/EXwfvZZeLrM/m/ONU1fjwGCgAJ for
>>>> background context.
>>>>
>>>> From a development perspective, it is currently a fully open source
>>>> community project and personally I'm not aware of folks providing
>>>> commercial support for it, or otherwise sponsoring development. You may
>>>> want to take that into account for your evaluation.
>>>>
>>>> -Chad
>>>>
>>>> PS: *Sent from my personal perspective as a GoCD contributor in my
>>>> spare time - not to be taken as an official statement of Thoughtworks*.
>>>>
>>>> On Mon, Jun 27, 2022 at 5:56 PM 'Sarmistha Bhuyan' via go-cd <
>>>> go...@googlegroups.com> wrote:
>>>>
>>>>> We are evaluating GoCD as a continuous delivery and release
>>>>> orchestration tool.
>>>>>
>>>>> We would like to connect and understand the enterprise offerings in
>>>>> terms of support and professional services. I tried to write
>>>>> *sup...@thoughtworks.com* but did not get any response.
>>>>>
>>>>> Any idea how and whom to contact for this?
>>>>>
>>>>> --
>>>>> You received this message because you are subscribed to the Google
>>>>> Groups "go-cd" group.
>>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>>> an email to go-cd+un...@googlegroups.com.
>>>>> To view this discussion on the web visit
>>>>> https://groups.google.com/d/msgid/go-cd/f1c56f07-8c09-4d9a-890b-2755bf9a8ed4n%40googlegroups.com
>>>>> <https://groups.google.com/d/msgid/go-cd/f1c56f07-8c09-4d9a-890b-2755bf9a8ed4n%40googlegroups.com?utm_medium=email_source=footer>
>>>>> .
>>>>>
>>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+un...@googlegroups.com.
>>>
>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/56ffb83a-7201-4a75-baf4-4e452e099fffn%40googlegroups.com
>>> <https://groups.google.com/d/msgid/go-cd/56ffb83a-7201-4a75-baf4-4e452e099fffn%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/89c568c8-30be-486d-9031-4361b08c6262n%40googlegroups.com
> <https://groups.google.com/d/msgid/go-cd/89c568c8-30be-486d-9031-4361b08c6262n%40googlegroups.com?utm_medium=email_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-5U6vJpSEA2UKa_QLPGOjS06eWGe3wKjHohiZiLYdT5g%40mail.gmail.com.

Re: [go-cd] Unable to run Microsoft Upgrade assistant

[Chad Wilson]

Which GoCD version are you using?

Can you share a screenshot from your task configuration? e.g something like
the below

[image: image.png]


Generally speaking you shouldn't need to do a "cmd /c" yourself, as GoCD
does this implicitly and it's possible doing this might be causing
something to go wrong with the arguments parsing/quoting. Just a guess
though.

Try configuring the task without it, e.g something like the below.

*command:* upgrade-assistant.exe
*arguments:*
upgrade
D:\Upgrade\GOPIPE\WEBAPP\SampleAPI\SampleEfAPI.csproj
--operation
Inplace
--targetFramework
net6.0
--non-interactive

-Chad

On Fri, Sep 15, 2023 at 5:58 PM Nitesh Kumar  wrote:

> Hi Team,
>
> Any help will be much appreciated. looking forward to hearing from you.
>
> On Thu, Sep 14, 2023 at 10:11 PM Nitesh Kumar 
> wrote:
>
>> Hi Aswanth,
>>
>> I have tried this option but no luck. can you pleas suggest anything else
>>
>> On Thu, Sep 14, 2023 at 4:05 AM 'Ashwanth Kumar' via go-cd <
>> go-cd@googlegroups.com> wrote:
>>
>>> Maybe try adding, "DOTNET_UPGRADEASSISTANT_TELEMETRY_OPTOUT" environment
>>> variable to "1"?
>>>
>>> Ref -
>>> https://learn.microsoft.com/en-us/dotnet/core/porting/upgrade-assistant-telemetry?tabs=console#disclosure
>>>
>>>
>>>
>>>
>>> On Wed, 13 Sept 2023 at 16:55, nitesh...@gmail.com <
>>> niteshcse...@gmail.com> wrote:
>>>
 Hi team,

 I am trying to run Microsoft upgrade assistant but its failing with
 below error
 Same command i am running through command prompt, it works as expected.

 i am running that it in non-interative mode so that microsoft upgrade
 assistant doesn't wait for user input on console. however running with GOCD
 task it still wait for user input and failing

 can you please help me with that


 [go] Task: cmd /c "upgrade-assistant.exe upgrade
 D:\Upgrade\GOPIPE\WEBAPP\SampleAPI\SampleEfAPI.csproj --operation Inplace
 --targetFramework net6.0 --non-interactive"

 Initializing and loading extensions...
 Telemetry
 Upgrade Assistant collects usage data in order to help us improve your
 experience. The data is collected by Microsoft and shared with the
 community.
 You can opt-out of telemetry by setting the
 DOTNET_UPGRADEASSISTANT_TELEMETRY_OPTOUT environment variable to '1' or
 'true'
 using your favorite shell.
 Read more about Upgrade Assistant telemetry:
 https://aka.ms/upgrade-assistant-telemetry
 Read more about .NET CLI Tools telemetry:
 https://aka.ms/dotnet-cli-telemetry
 Press any key to continue...
 System.InvalidOperationException: Cannot read keys when either
 application does
 not have a console or when console input has been redirected. Try
 Console.Read.
 at System.ConsolePal.ReadKey(Boolean intercept)
 at

 Microsoft.UpgradeAssistant.Cli.Startup.FirstUseStartup.StartupAsync(Cancellation
 Token cancellationToken) in

 D:\a\_work\1\s\src\Experiments\UpgradeAssistant\cli\Startup\FirstUseStartup.cs:l
 ine 37
 at

 Microsoft.UpgradeAssistant.Cli.Flow.Steps.Upgrade.StartupFlowStep.ValidateUserIn
 putAsync(IFlowContext context, CancellationToken cancellationToken) in

 D:\a\_work\1\s\src\Experiments\UpgradeAssistant\cli\Flow\Steps\Startup\StartupFl
 owStep .cs:line 34
 at Spectre.Console.Flow.FlowRunner.RunAsync(CancellationToken
 cancellationToken) in

 D:\a\_work\1\s\src\Experiments\UpgradeAssistant\spectre.flow\Flow\FlowRunner.cs:
 line 83

 --
 You received this message because you are subscribed to the Google
 Groups "go-cd" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to go-cd+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/go-cd/007cd4c6-639a-430f-bf49-53d6649b22d5n%40googlegroups.com
 
 .

>>>
>>>
>>> --
>>>
>>> Ashwanth Kumar / ashwanthkumar.in
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/CAD9m7CyUa7YbuSBB4gNjytKd%2BxccqXJy5cHVRMf2RUqJ9gv47Q%40mail.gmail.com
>>> 
>>> .
>>>
>>
>>
>> --
>> Thanks
>>
>> Nitesh kumar
>>
>
>
> --
> Thanks
>
> Nitesh kumar
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to 

Re: [go-cd] GoCD Agent in Arch Linux container - Can't start any job

[Chad Wilson]

Hi Jacques - that looks like a bug of some kind with the way the work for
the agent to do is being serialized by the server back to the agent. Have
never seen that before though.

Could you report at https://github.com/gocd/gocd/issues along with the
relevant versions of your server, agents, Java versions etc? Please include
other information such as when this started happening, whether it happens
for all jobs, or just a single git material etc would be useful.

I also wonder whether you have any special configuration of Java locales on
your server and/or agent machines/containers?

-Chad

On Wed, Sep 6, 2023 at 10:37 PM Jacques Progin 
wrote:

> Dear team,
>
> A pipeline has been triggered by a Git commit, and the GoCD server shows
> the first step as assigned to the agent, then waiting for agent again, then
> assigned again, in a never ending loop. But the console log stays empty.
> The agents view shows the agent as idle.
>
> The agent is running into a Arch Linux docker container, and the
> go-agent.log contains the following entries appearing in loop:
>
> 2023-09-06 14:03:35,976 ERROR [scheduler-1] AgentController:99 - [Agent
> Loop] Error occurred during loop:
> com.google.gson.JsonSyntaxException: Failed parsing 'Sep 2, 2023, 11:09:23
> AM' as Date; at path
> $.assignment.materialRevisions.revisions[0].modifications[0].modifiedTime
> at
> com.google.gson.internal.bind.DateTypeAdapter.deserializeToDate(DateTypeAdapter.java:90)
> at
> com.google.gson.internal.bind.DateTypeAdapter.read(DateTypeAdapter.java:75)
> at
> com.google.gson.internal.bind.DateTypeAdapter.read(DateTypeAdapter.java:46)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.readIntoField(ReflectiveTypeAdapterFactory.java:212)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$FieldReflectionAdapter.readField(ReflectiveTypeAdapterFactory.java:433)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:393)
> at
> com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:40)
> at
> com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:82)
> at
> com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:61)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.readIntoField(ReflectiveTypeAdapterFactory.java:212)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$FieldReflectionAdapter.readField(ReflectiveTypeAdapterFactory.java:433)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:393)
> at
> com.google.gson.internal.bind.TypeAdapterRuntimeTypeWrapper.read(TypeAdapterRuntimeTypeWrapper.java:40)
> at
> com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:82)
> at
> com.google.gson.internal.bind.CollectionTypeAdapterFactory$Adapter.read(CollectionTypeAdapterFactory.java:61)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.readIntoField(ReflectiveTypeAdapterFactory.java:212)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$FieldReflectionAdapter.readField(ReflectiveTypeAdapterFactory.java:433)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:393)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.readIntoField(ReflectiveTypeAdapterFactory.java:212)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$FieldReflectionAdapter.readField(ReflectiveTypeAdapterFactory.java:433)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:393)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$1.readIntoField(ReflectiveTypeAdapterFactory.java:212)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$FieldReflectionAdapter.readField(ReflectiveTypeAdapterFactory.java:433)
> at
> com.google.gson.internal.bind.ReflectiveTypeAdapterFactory$Adapter.read(ReflectiveTypeAdapterFactory.java:393)
> at com.google.gson.TypeAdapter.fromJsonTree(TypeAdapter.java:299)
> at
> com.thoughtworks.go.remote.adapter.RuntimeTypeAdapterFactory$1.read(RuntimeTypeAdapterFactory.java:258)
> at com.google.gson.TypeAdapter$1.read(TypeAdapter.java:204)
> at com.google.gson.Gson.fromJson(Gson.java:1227)
> at com.google.gson.Gson.fromJson(Gson.java:1137)
> at com.google.gson.Gson.fromJson(Gson.java:1047)
> at com.google.gson.Gson.fromJson(Gson.java:982)
> at
> com.thoughtworks.go.agent.RemotingClient.getWork(RemotingClient.java:77)
> at
> 

Re: [go-cd] GoCD Pipeline Views - Can they be copied across users?

[Chad Wilson]

Yeah, they are in the DB keyed to the user's id:
https://github.com/gocd/gocd/blob/0f58107c851cf2df6ce7c6902eebde796dc1f742/db-support/db-migration/src/main/resources/db-migration-scripts/initial/create-schema.xml#L291-L303

While it comes with some risks, it probably wouldn't be super difficult to
update the views/filters (PIPELINESELECTIONS) to point to the new user IDs
for each user with an appropriate query? IIRC there should be max one row
per user (can't recall if it's 1-to-(0,1) or 1-to-1 from USERS).

Not aware of any specific export or share support. The UI uses an internal
API */go/api/internal/pipeline_selection* to retrieve (GET) and update
(PUT) the views in one big block as an array of "filters", which in theory
you could use to get a JSON representation of your individual views if you
can still login with the old username. If one is more savvy, one could then
also PUT the collection back to the same API to update views when
authenticated with the new username - but obviously this is
undocumented/unsupported and would require some browser "Inspect" digging
:-)

-Chad

On Tue, Aug 22, 2023 at 4:21 PM 'Chris Gillatt' via go-cd <
go-cd@googlegroups.com> wrote:

> I think that the user-configurable pipeline views in GoCD are stored
> against the user in the DB.  I'm pretty sure that without messing around
> with the DB, migrating views (or sharing or exporting them) would not be
> possible.  Could anyone confirm this please?
>
> The reason I ask is that we've migrated from one auth plugin to another,
> and as a result, all users now have a new username (old: user.name, new:
> user.name@domain).  This means the users have lost their views.  It's not
> a biggie, and moving from LDAP to OIDC auth is a much bigger win than
> keeping views, but thought I'd just ask the question.
>
> Cheers
> Chris
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/8280b950-9e20-4291-845a-e35de44634ffn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8KaBor-GKKoVCQhWusc16nzfTQ1mg-LW-Gwm90PcTz0A%40mail.gmail.com.

Re: [go-cd] Config Repositories has always in fetch status

[Chad Wilson]

Thanks so much for sharing back! This does make sense. I have also
experienced issues with EFS related to things like this. I would have
suggested checking disk performance if I had realised a GoCD server
replacement+upgrade was part of what had changed :-)

While it has some challenges in terms of being AZ specific, generally I
have had better experience mounting EBS volumes for use by GoCD (rather
than network-based stores such as EFS), although that does limit which AZ
your GoCD server can run in (without manual intervention), so it depends on
your wider deployment architecture whether that is acceptable.

I can't think of any major reason the GoCD server version change on its own
would cause higher throughput usage than your older version GoCD Server.
One thing worth thinking about is that, in my recollection, EFS in bursting
mode does vary speeds a lot based on the size of the storage. If your *new
server *has much lower storage/use of EFS than your *old server* then the
limits may be different. (e.g if you wiped a lot of artifacts while
re-using the same EFS volume or created a new EFS volume which is a lot
smaller) I'd suggest comparing the AWS side metrics for your EFS throughput
between the two to compare what their usage, credits, and limits are per
https://docs.aws.amazon.com/efs/latest/ug/performance.html.

For small EFS volumes, the baseline throughput is pretty terrible (15 MiBps
read, 5 MiBps continuously), and GoCD servers tend to be rather write heavy
if you have heavy use of artifacts within GoCD itself.

I am not sure if use of https-git has any major implications for disk usage
on the git side of things compared to ssh , but I would not have thought
it'd majorly change the throughput requirements. If EFS volume size doesn't
explain issues, and you have changed all of the material URLs to https://,
perhaps you want to compare other aspects of your material configuration
for changes in

   - the # of distinct materials known to GoCD on the Materials tab (old vs
   new)
   - the # of these materials that are auto-updating (polling, the default)
   compared to having auto-update disabled (e.g if you also use Webhooks)

-Chad

On Wed, Aug 16, 2023 at 1:01 PM Komgrit Aneksri 
wrote:

> Hello Chad,
>
> Thank you for your suggestion. Just for your infomation.
>
> I had fixed this issue since last week.
>
> This root cause issue related to our EFS throughput for stored flyweight,
> /home/go, artifacts, 
>
> This issue solved by I changed EFS  throughput mode from bursting to
> elastic instead.
>
> But I am investigating about why the new our GoCD server (v23.1.0) use
> higher throughput than the current GoCD server (v22.1.0).
>
> So they are the same configuration but there are difference only GoCD
> version and new GOCD server use only git over https.
>
> Best Regards,
> Komgrit
>
> On Tuesday, August 8, 2023 at 11:23:25 PM UTC+7 Chad Wilson wrote:
>
>> That is quite a lot of forked git processes. If there are constantly the
>> same amount of forked git processes (over, say, a 1 minute period), that is
>> likely the server at maximum default "material updates" concurrency. This
>> may mean git operations are queued behind each other and you possibly can't
>> fetch/check for git updates fast enough. The server typically logs
>> something when this is happening - you might want to inspect the logs more
>> closely.
>>
>> Git operations being queued is possibly also what is happening to your
>> config repositories, which is why you see them constantly "refreshing".
>>
>> Since all the processes seem to be at low CPU usage, this implies to me
>> that they are probably waiting for the network or your GitLab server (or
>> theoretically the local disk in the container is slow, but less likely a
>> cause). As suggested earlier, I think you are going to need to analyze git
>> speed further, to see what is happening with your network connectivity to
>> the GitLab server, and possibly check the GitLab server metrics itself. If
>> you upgraded or changed something on GitLab, I'd suggest comparing its
>> metrics from before the change/upgrade to afterwards and that type of thing.
>>
>> -Chad
>>
>> On Tue, Aug 8, 2023 at 11:54 PM Komgrit Aneksri 
>> wrote:
>>
>>> Hello Chad,
>>>
>>> I look in top and ps but no any weird or stuck processes.
>>>
>>> As i attached pictures.
>>>
>>> [image: 1691509554118.jpg]
>>>
>>> [image: 1691509751182.jpg]
>>>
>>> BR,
>>> Komgrit
>>> On Tuesday, August 8, 2023 at 9:24:43 PM UTC+7 Chad Wilson wrote:
>>>
>>>> Hmm, given your description and the basic metrics you shared, the
>>>> behaviour so

Re: [go-cd] Config Repositories has always in fetch status

[Chad Wilson]

Hmm, given your description and the basic metrics you shared, the behaviour
sounds strange.

To step back slightly and confirm the issue, please look inside the
container at the process tree (via ps, top etc) and see whether there are a
large number of forked git processes. If there *are*, we want to see what
they are doing and focus there. If there *are not*, the problem may be
somewhere else inside GoCD causing the config repo loads to get stuck, and
we need to look in a different area.

If you changed nothing on GoCD or its hardware/host/config, that seems to
point to something outside GoCD as the source of the problem, unless it is
a problem that started as a side effect of restarting GoCD.

-Chad

On Tue, Aug 8, 2023 at 9:27 PM Komgrit Aneksri 
wrote:

> Hello Chad,
>
> What did you change around the time this behaviour started happening?
> -GoCD = no
> -Gitlab = upgrade from 16.2.0 to 16.2.3
>
> Which container image for the server are you using? (only
> gocd-server-centos-9 <https://hub.docker.com/r/gocd/gocd-server-centos-9/> is
> built for ARM64 - if you are trying to run emulated you will probably have
> many problems)
> - I am using official gocd/gocd-server-centos-9:v23.1.0
>
> What are the CPU requests and limits that you have assigned to the gocd
> server pod? And are you deploying with the standard GoCD helm chart? If the
> limits are too low, or the requests are low and other processes on the same
> node are using too much CPU you can still end up with CPU starvation, even
> if it looks like the pod isn't using much CPU, because it may be throttled.
> - I am using officail helm chart v 2.1.6. and no limit cpu and memory and
> set request CPU and Memory
>  resources:
> requests:
>   memory: 2048Mi
>   cpu: 1000m
> PS. GoCD server is running on c6g.large instance type and this below is
> top node result.
> NAME  CPU(cores)   CPU%
> MEMORY(bytes)   MEMORY%
> ip-xx-xxx-xx-xx.ap-southeast-1.compute.internal   74m  3%
> 2570Mi      81%
>
>
> Best Regards,
> Komgrit
>
> On Tuesday, August 8, 2023 at 4:11:18 PM UTC+7 Chad Wilson wrote:
>
>> If you are getting logs like that, it sounds like the container is
>> experiencing CPU starvation.
>>
>>- What did you change around the time this behaviour started
>>happening?
>>- Which container image for the server are you using? (only
>>gocd-server-centos-9
>><https://hub.docker.com/r/gocd/gocd-server-centos-9/> is built for
>>ARM64 - if you are trying to run emulated you will probably have many
>>problems)
>>- What are the CPU requests and limits that you have assigned to the
>>gocd server pod? And are you deploying with the standard GoCD helm chart?
>>If the limits are too low, or the requests are low and other processes on
>>the same node are using too much CPU you can still end up with CPU
>>starvation, even if it looks like the pod isn't using much CPU, because it
>>may be throttled.
>>
>> -Chad
>>
>> On Tue, Aug 8, 2023 at 4:06 PM Komgrit Aneksri 
>> wrote:
>>
>>> Hello Chad,
>>>
>>> Thank you for your advice.
>>>
>>> I tried run git clone with debug environment variables command in gocd
>>> server pod. So the command run successfully and there is no any abnormal
>>> log.
>>>
>>> And CPU and memory are not high usage.
>>>
>>> We tried restart gocd server pods many time but it is not help.
>>>
>>> And I dig to see timeout logs. There is log messages.
>>>  INFO   | wrapper  | 2023/08/08 07:43:48 | Wrapper Process has not
>>> received any CPU time for 22 seconds.  Extending timeouts.
>>>
>>> Best Regards,
>>> Komgrit
>>> On Tuesday, August 8, 2023 at 12:16:45 PM UTC+7 Chad Wilson wrote:
>>>
>>>> That error message is not relevant to the issue, you can ignore it. Are
>>>> there other errors or timeouts in the logs?
>>>>
>>>> To refresh config repos, GoCD forks regular git processes to clone and
>>>> then fetches. You might want to exec into the container and see what these
>>>> processes are doing (high CPU? stuck somehow?)
>>>>
>>>> There are also a few general suggestions on similar issues around which
>>>> you might want to check:
>>>>
>>>>- https://github.com/gocd/gocd/issues/10480
>>>>- https://github.com/gocd/gocd/issues/9588
>>>>- https://github.com/gocd/gocd/issues/8565
>>>>
>>>> If this is happening after a

Re: [go-cd] Log in Issues

[Chad Wilson]

I didn't say to delete the config file (I explained why that might not
work) - I said to *edit* the config file and remove the
... block from the XML content.

-Chad

On Tue, Aug 8, 2023 at 5:37 PM Alex Murphy 
wrote:

> I tried the way you told me, but nothing changed. I followed the above
> steps:
> 1. stop the GoCD server service
> 2. delete the config file
> 3. start the server service
> And then the config file was shown again and the content is same as the
> file was deleted.
>
> 在2023年8月8日星期二 UTC+8 11:11:21 写道：
>
>> Forgot to add that if the server seems to be restoring the "old" content,
>> make sure that the service is not running at the time you manually edit the
>> config on disk. The server may cache, flush and overwrite config for
>> various reasons so best to avoid this by ensuring it isn't running or
>> "competing" with your manual edits.
>>
>> On Tue, Aug 8, 2023 at 10:40 AM Chad Wilson 
>> wrote:
>>
>>> I can't recall exactly how it works, but perhaps if the config file is 
>>> *completely
>>> removed* like you say you did below, GoCD does some magic to restore
>>> the latest known version from the history git repo (that I referred to
>>> below) rather than starting from scratch.
>>>
>>> The most reliable way is to look at the config history in
>>> *db/config.git* (as I mentioned below) to see what your config was
>>> before you added the LDAP config and use the git history to restore the
>>> content.
>>>
>>> However if you just want to completely remove the security/login
>>> requirement and any "local" users you have marked as admins etc you should
>>> be able to edit the cruise-config.xml directly and remove the entire 
>>> 
>>> ... block, leaving other pieces unchanged. Configuration
>>> without a security block at all won't require any login.
>>>
>>> -Chad
>>>
>>> On Tue, Aug 8, 2023 at 10:23 AM Alex Murphy 
>>> wrote:
>>>
>>>>
>>>> Hi Chad,
>>>>
>>>> Yeah, I forgot to tell you the GoCD server is a Windows version, and I
>>>> located the file, but I tried to delete the content or this file, then I
>>>> restarted the GoCD Server Service, and the file will recover from before.
>>>> How can I change the login type to default that login without any
>>>> account or password?
>>>>
>>>> Thanks
>>>>
>>>> 在2023年8月7日星期一 UTC+8 16:27:15 写道：
>>>>
>>>>> The server configuration that controls this is plain XML, and has
>>>>> source controlled history using git itself so you can use standard git
>>>>> tools (git log, git show etc) to find the config revision as it was before
>>>>> your change and revert the security/login settings back to what they were
>>>>> before.
>>>>>
>>>>> Location of both the "current" config and the config history depends
>>>>> on how you are running/installing the GoCD server (Linux package install?
>>>>> Windows? containers?) The history should be a config.git subfolder of the
>>>>> 'db' folder. The latest config should be a cruise-config.xml inside the
>>>>> "GoCD Server configuration" folder.
>>>>>
>>>>>
>>>>> https://docs.gocd.org/current/installation/install/server/linux.html#location-of-gocd-server-files
>>>>>
>>>>> https://docs.gocd.org/current/installation/install/server/windows.html#location-of-gocd-server-files
>>>>> https://hub.docker.com/r/gocd/gocd-server/
>>>>>
>>>>> -Chad
>>>>>
>>>>> On Mon, Aug 7, 2023 at 4:09 PM Alex Murphy 
>>>>> wrote:
>>>>>
>>>>>> Hi guys,
>>>>>> I encountered a problem on login surface, the reason is I made a
>>>>>> mistake in the server config, I set the wrong LDAP and clicked save, and
>>>>>> now I cannot log in when I input any accounts and passwords.
>>>>>> And I tried to reinstall the GoCD Server and it was not work.
>>>>>> Can anyone give me some suggestions?
>>>>>>
>>>>>> --
>>>>>> You received this message because you are subscribed to the Google
>>>>>> Groups "go-cd" group.
>>>>>> To unsubscribe from this group and stop receiving emails from it,
>>>>>> send an ema

Re: [go-cd] Config Repositories has always in fetch status

[Chad Wilson]

If you are getting logs like that, it sounds like the container is
experiencing CPU starvation.

   - What did you change around the time this behaviour started happening?
   - Which container image for the server are you using? (only
   gocd-server-centos-9
   <https://hub.docker.com/r/gocd/gocd-server-centos-9/> is built for ARM64
   - if you are trying to run emulated you will probably have many problems)
   - What are the CPU requests and limits that you have assigned to the
   gocd server pod? And are you deploying with the standard GoCD helm chart?
   If the limits are too low, or the requests are low and other processes on
   the same node are using too much CPU you can still end up with CPU
   starvation, even if it looks like the pod isn't using much CPU, because it
   may be throttled.

-Chad

On Tue, Aug 8, 2023 at 4:06 PM Komgrit Aneksri 
wrote:

> Hello Chad,
>
> Thank you for your advice.
>
> I tried run git clone with debug environment variables command in gocd
> server pod. So the command run successfully and there is no any abnormal
> log.
>
> And CPU and memory are not high usage.
>
> We tried restart gocd server pods many time but it is not help.
>
> And I dig to see timeout logs. There is log messages.
>  INFO   | wrapper  | 2023/08/08 07:43:48 | Wrapper Process has not
> received any CPU time for 22 seconds.  Extending timeouts.
>
> Best Regards,
> Komgrit
> On Tuesday, August 8, 2023 at 12:16:45 PM UTC+7 Chad Wilson wrote:
>
>> That error message is not relevant to the issue, you can ignore it. Are
>> there other errors or timeouts in the logs?
>>
>> To refresh config repos, GoCD forks regular git processes to clone and
>> then fetches. You might want to exec into the container and see what these
>> processes are doing (high CPU? stuck somehow?)
>>
>> There are also a few general suggestions on similar issues around which
>> you might want to check:
>>
>>- https://github.com/gocd/gocd/issues/10480
>>- https://github.com/gocd/gocd/issues/9588
>>- https://github.com/gocd/gocd/issues/8565
>>
>> If this is happening after a restart of the GoCD server, or due to some
>> other change you've made it's possible Gitlab is throttling the requests.
>>
>> Depending on whether you are using https or ssh connections, you may want
>> to use standard git environment variables to debug what's happening 
>> (GIT_TRACE=1,
>> GIT_CURL_VERBOSE=1 etc).
>>
>> -Chad
>>
>> On Tue, Aug 8, 2023 at 12:29 PM Komgrit Aneksri 
>> wrote:
>>
>>> Hello GoCD team,
>>>
>>> I am facing issue in Config Repositories page.
>>>
>>> So All Config Repositories are always in fetching status for
>>> Gitlab(16.2.3)
>>> Our GoCD is 23.1.0 version on kubernetes and running ARM64.
>>>
>>> In server log, There is error message
>>>
>>> "jvm 1| 2023-08-08 04:18:17,151 ERROR [qtp1962126505-38]
>>> VariableReplacer:385 - function ${escape:} type 'escape:' not a valid type"
>>>
>>> when i did refresh or open the Config Repositories page.
>>> [image: 1691468556134.jpg]
>>>
>>> Please help us for fix it.
>>>
>>> Best Regards,
>>> Komgrit
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+un...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/af5f5dd4-6f27-47d3-8eb3-1a19c0675ae7n%40googlegroups.com
>>> <https://groups.google.com/d/msgid/go-cd/af5f5dd4-6f27-47d3-8eb3-1a19c0675ae7n%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/f5b73449-c62b-426c-8bf6-1a4911a8d521n%40googlegroups.com
> <https://groups.google.com/d/msgid/go-cd/f5b73449-c62b-426c-8bf6-1a4911a8d521n%40googlegroups.com?utm_medium=email_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH_Fh5m7Jzn3NUUKp85HVRvRrPCfdst%3DNPQGn%2BCRpBbZeA%40mail.gmail.com.

Re: [go-cd] Log in Issues

[Chad Wilson]

Forgot to add that if the server seems to be restoring the "old" content,
make sure that the service is not running at the time you manually edit the
config on disk. The server may cache, flush and overwrite config for
various reasons so best to avoid this by ensuring it isn't running or
"competing" with your manual edits.

On Tue, Aug 8, 2023 at 10:40 AM Chad Wilson  wrote:

> I can't recall exactly how it works, but perhaps if the config file is 
> *completely
> removed* like you say you did below, GoCD does some magic to restore the
> latest known version from the history git repo (that I referred to below)
> rather than starting from scratch.
>
> The most reliable way is to look at the config history in *db/config.git*
> (as I mentioned below) to see what your config was before you added the
> LDAP config and use the git history to restore the content.
>
> However if you just want to completely remove the security/login
> requirement and any "local" users you have marked as admins etc you should
> be able to edit the cruise-config.xml directly and remove the entire 
> 
> ... block, leaving other pieces unchanged. Configuration
> without a security block at all won't require any login.
>
> -Chad
>
> On Tue, Aug 8, 2023 at 10:23 AM Alex Murphy 
> wrote:
>
>>
>> Hi Chad,
>>
>> Yeah, I forgot to tell you the GoCD server is a Windows version, and I
>> located the file, but I tried to delete the content or this file, then I
>> restarted the GoCD Server Service, and the file will recover from before.
>> How can I change the login type to default that login without any account
>> or password?
>>
>> Thanks
>>
>> 在2023年8月7日星期一 UTC+8 16:27:15 写道：
>>
>>> The server configuration that controls this is plain XML, and has source
>>> controlled history using git itself so you can use standard git tools (git
>>> log, git show etc) to find the config revision as it was before your change
>>> and revert the security/login settings back to what they were before.
>>>
>>> Location of both the "current" config and the config history depends on
>>> how you are running/installing the GoCD server (Linux package install?
>>> Windows? containers?) The history should be a config.git subfolder of the
>>> 'db' folder. The latest config should be a cruise-config.xml inside the
>>> "GoCD Server configuration" folder.
>>>
>>>
>>> https://docs.gocd.org/current/installation/install/server/linux.html#location-of-gocd-server-files
>>>
>>> https://docs.gocd.org/current/installation/install/server/windows.html#location-of-gocd-server-files
>>> https://hub.docker.com/r/gocd/gocd-server/
>>>
>>> -Chad
>>>
>>> On Mon, Aug 7, 2023 at 4:09 PM Alex Murphy 
>>> wrote:
>>>
>>>> Hi guys,
>>>> I encountered a problem on login surface, the reason is I made a
>>>> mistake in the server config, I set the wrong LDAP and clicked save, and
>>>> now I cannot log in when I input any accounts and passwords.
>>>> And I tried to reinstall the GoCD Server and it was not work.
>>>> Can anyone give me some suggestions?
>>>>
>>>> --
>>>> You received this message because you are subscribed to the Google
>>>> Groups "go-cd" group.
>>>> To unsubscribe from this group and stop receiving emails from it, send
>>>> an email to go-cd+un...@googlegroups.com.
>>>> To view this discussion on the web visit
>>>> https://groups.google.com/d/msgid/go-cd/7cdbc223-66dd-4055-9c52-86071fedf1b7n%40googlegroups.com
>>>> <https://groups.google.com/d/msgid/go-cd/7cdbc223-66dd-4055-9c52-86071fedf1b7n%40googlegroups.com?utm_medium=email_source=footer>
>>>> .
>>>>
>>> --
>> You received this message because you are subscribed to the Google Groups
>> "go-cd" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/b00cfcd8-77fb-4200-b802-d39e775043bfn%40googlegroups.com
>> <https://groups.google.com/d/msgid/go-cd/b00cfcd8-77fb-4200-b802-d39e775043bfn%40googlegroups.com?utm_medium=email_source=footer>
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH_drR4%3Dk3KMD7sLqn%3DmsJCi3L24gNJeKFo2%3DkQub9NyNg%40mail.gmail.com.

Re: [go-cd] Log in Issues

[Chad Wilson]

I can't recall exactly how it works, but perhaps if the config file is
*completely
removed* like you say you did below, GoCD does some magic to restore the
latest known version from the history git repo (that I referred to below)
rather than starting from scratch.

The most reliable way is to look at the config history in *db/config.git*
(as I mentioned below) to see what your config was before you added the
LDAP config and use the git history to restore the content.

However if you just want to completely remove the security/login
requirement and any "local" users you have marked as admins etc you should
be able to edit the cruise-config.xml directly and remove the entire 
... block, leaving other pieces unchanged. Configuration without
a security block at all won't require any login.

-Chad

On Tue, Aug 8, 2023 at 10:23 AM Alex Murphy 
wrote:

>
> Hi Chad,
>
> Yeah, I forgot to tell you the GoCD server is a Windows version, and I
> located the file, but I tried to delete the content or this file, then I
> restarted the GoCD Server Service, and the file will recover from before.
> How can I change the login type to default that login without any account
> or password?
>
> Thanks
>
> 在2023年8月7日星期一 UTC+8 16:27:15 写道：
>
>> The server configuration that controls this is plain XML, and has source
>> controlled history using git itself so you can use standard git tools (git
>> log, git show etc) to find the config revision as it was before your change
>> and revert the security/login settings back to what they were before.
>>
>> Location of both the "current" config and the config history depends on
>> how you are running/installing the GoCD server (Linux package install?
>> Windows? containers?) The history should be a config.git subfolder of the
>> 'db' folder. The latest config should be a cruise-config.xml inside the
>> "GoCD Server configuration" folder.
>>
>>
>> https://docs.gocd.org/current/installation/install/server/linux.html#location-of-gocd-server-files
>>
>> https://docs.gocd.org/current/installation/install/server/windows.html#location-of-gocd-server-files
>> https://hub.docker.com/r/gocd/gocd-server/
>>
>> -Chad
>>
>> On Mon, Aug 7, 2023 at 4:09 PM Alex Murphy 
>> wrote:
>>
>>> Hi guys,
>>> I encountered a problem on login surface, the reason is I made a mistake
>>> in the server config, I set the wrong LDAP and clicked save, and now I
>>> cannot log in when I input any accounts and passwords.
>>> And I tried to reinstall the GoCD Server and it was not work.
>>> Can anyone give me some suggestions?
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+un...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/7cdbc223-66dd-4055-9c52-86071fedf1b7n%40googlegroups.com
>>> 
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/b00cfcd8-77fb-4200-b802-d39e775043bfn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH_sjxHMch6gcKoShfjJM5pOnk0HzF6S-g8CA-f2jUzzCA%40mail.gmail.com.

Re: [go-cd] Log in Issues

[Chad Wilson]

The server configuration that controls this is plain XML, and has source
controlled history using git itself so you can use standard git tools (git
log, git show etc) to find the config revision as it was before your change
and revert the security/login settings back to what they were before.

Location of both the "current" config and the config history depends on how
you are running/installing the GoCD server (Linux package install? Windows?
containers?) The history should be a config.git subfolder of the 'db'
folder. The latest config should be a cruise-config.xml inside the "GoCD
Server configuration" folder.

https://docs.gocd.org/current/installation/install/server/linux.html#location-of-gocd-server-files
https://docs.gocd.org/current/installation/install/server/windows.html#location-of-gocd-server-files
https://hub.docker.com/r/gocd/gocd-server/

-Chad

On Mon, Aug 7, 2023 at 4:09 PM Alex Murphy 
wrote:

> Hi guys,
> I encountered a problem on login surface, the reason is I made a mistake
> in the server config, I set the wrong LDAP and clicked save, and now I
> cannot log in when I input any accounts and passwords.
> And I tried to reinstall the GoCD Server and it was not work.
> Can anyone give me some suggestions?
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/7cdbc223-66dd-4055-9c52-86071fedf1b7n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-6ow%3DgxNL%3D8cjmAz4Nt3iDe1QE03VmRjL3h8Jwd%2Be6yA%40mail.gmail.com.

Re: [go-cd] Multi gocd server

[Chad Wilson]

Hiya - no it's not.

GoCD *previously* had the ability to run a warm standby ("business
continuity") instance against a replicated database, however the failover
was also manually triggered between the two. That functionality was removed
due to a major security issue being discovered, and lack of
support/requests from the community to add it back. Historically it was a
commercial feature before the full open sourcing of GoCD, so was probably
not as actively used by the wider community.

If the goal is to increase GoCD server reliability to minimise the chance
of downtime, I believe users are probably better to

   - ensure it is running on a "proper" external database (not default H2)
   - PostgreSQL recommended - with a proper DB backup/recovery strategy
   - if you are heavily reliant on "GoCD-stored" logs and artifacts from
   builds consider carefully the type of underlying file storage GoCD uses for
   its artifacts, and its replication or snapshots/backups to mitigate data
   loss issues around artifacts
   - otherwise monitor your GoCD server instance so you can size it proper;
   add more RAM and CPU where necessary, etc and react quickly to any issues
   - consider using containerised GoCD instances (with mounted persistent
   artifact storage) to make it "easier" to bring a new working instance up in
   the event of complete failure of hardware/VMs and get some implicit
   monitoring of that container from a container runtime or orchestration
   platform

-Chad

On Mon, Aug 7, 2023 at 12:15 PM Vijayakumaran A. <
vijayakumara...@praniontech.com> wrote:

> Hi Team,
>
> It is possible to setup multi master (gocd server) ?
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/6779d5e3-d6a0-4c35-a417-b893788ccc19n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8P-GLBDThD2feQ6tg%3DZKBq9ZJdKRcjw%2BA%3DgAa%2BYA8E6g%40mail.gmail.com.

Re: [go-cd] Rollback deployment method

[Chad Wilson]

It depends on the tooling you are using and what environment/stack you are
deploying to, alongside artifact storage relevant to your deployments.

Not all deployments can be rolled back without extra effort in tooling or
application design (e.g how will you roll back a DROP TABLE for a database?
Do you have rollback scripts that need to be invoked? Do you use DB
snapshots pre-deploy?)

GoCD isn't opinionated about this.

If you are using a declarative deployment tool or approach you can
sometimes trigger a re-run of the previous successful deployment's pipeline
instance/stage. Sometimes teams will roll forward (from a GoCD perspective)
by initialising a new pipeline run and populating some environment variable
to override the artifact versions to deploy. Or use a manual 'rollback'
stage in the same pipeline as the normal deploy.

As with most other pipeline automation approaches, all of these things
depend on your target platform, tooling and application design.

https://www.gocd.org/2017/06/20/hotfixes-rollback-rollforward.html may be
useful alongside the wider series at
https://www.gocd.org/tags/modeling-deployment-pipelines.html

-Chad

On Fri, 4 Aug 2023, 13:58 Vijayakumaran A., 
wrote:

> Hi Team,
>
> How to rollback to the previous deployment. Please any one share any
> ideas/Docs to refer.
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/933926d9-2ac7-4a34-b994-98a0f9a12c7an%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-gG18mKgqeUMkSz3NYwoHTe%2BWsb0ckSsB4uiPSt003AQ%40mail.gmail.com.

Re: [go-cd] Gocd refresh

[Chad Wilson]

Are you using Pipelines as Code via YAML Config Repositories - and this is
what you are talking about?

Assuming so, unless you've turned off auto updating for the underlying
repository, by default they should update within a minute. Admins can
review the configuration at Admin > Config Repositories.

If the repo and pipeline you changed is updated on the config repositories
view but you still don't see it on your dashboard, you might want to check
that you have at least permissions for the pipeline group (if adding a new
pipeline), that your dashboard view is not filtering any new pipeline and
if you are filtering pipelines you default it to include newly created
pipelines in the view.

-Chad



On Wed, Aug 2, 2023 at 7:43 PM Kowsik P  wrote:

> I updated existing yaml file. but still not reflected in gocd dashboard.
> any way we can refresh the dashboard.
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/e12f9406-97bb-4048-914d-c425e1a5dd7bn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8_BWHW%3Dei_hjFz0%3DG%2B2YRz%2B_rbtq%3D5NCQMO-1ZMJhMnQ%40mail.gmail.com.

Re: [go-cd] Regarding Gocd user management

[Chad Wilson]

Generally speaking, the permission required to pause a pipeline should be
the same as the permissions required to trigger it ("operate" or "admin"
permission to the pipeline or the pipeline group).
https://docs.gocd.org/current/configuration/dev_authorization.html

The only other thing I can guess you might have done to get this behaviour
is to override the permissions for a single *stage* within a pipeline to
allow it to be triggered by only user1 via PipelineConfig > stage >
permissions.

Normally this is intended to restrict (manual) stage approval triggers to a
different group than the overall pipeline as documented at
https://docs.gocd.org/current/configuration/dev_authorization.html#adding-authorization-to-approvals.
If you are configuring permissions for a pipeline *stage*, you may be doing
it in the wrong place to get your desired behaviour. Generally the
"pipeline group" is the normal unit of permissioning within GoCD and
stage-specific permissions are used for special cases.

Since *pausing* as an operation is at the overall pipeline scope (not for a
specific stage), the permissions need to be restricted from the pipeline
*group* which the pipeline belongs to.

If you still think something is not working as expected, perhaps share your
configuration of the pipeline group and the pipeline stages, and share some
more details/screenshots etc so someone else can reproduce what you are
seeing.

-Chad

On Mon, Jul 31, 2023 at 6:05 PM Vijayakumaran A. <
vijayakumara...@praniontech.com> wrote:

> Hi Team,
>
> I am new here and exploring gocd. i have setup one pipeline and created
> two users ex:(user1,user2).i have set specific permission only user1 able
> to trigger the pipeline. when i login as user2 im not able to trigger the
> pipeline its working but user2 able to pause the pipeline i don't want
> user2 pipeline pause access how to do this .im new to gocd pls help anyone.
>
> Thanks
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/b3c2fe15-9be2-47be-94f2-956a7b6e07c4n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH9p8ha7u3J3xkozYtVN6DvR7B9oz18uWbQOWdgHaaM9ng%40mail.gmail.com.

[go-cd] Release Announcement - 23.3.0

[Chad Wilson]

Hello everyone,

A new release of GoCD <https://www.gocd.org/releases/#23-3-0> (23.3.0)
fixing these minor regressions is out.

To know more about the bug fixes in this release, see the release notes
<https://www.gocd.org/releases/#23-3-0> or head to the downloads page
<https://www.gocd.org/download/> to try it.

Cheers,
Chad & Aravind

On Tue, Jul 25, 2023 at 12:41 PM Chad Wilson  wrote:

> Hi folks - unfortunately a problem sneaked through our automated
> regression in 23.2.0 which prevents UI navigation via Stage History to
> older stage runs in the history when you're already viewing the detail of
> an individual stage (Stage Details view).
>
> You can still navigate to these as normal from the Pipeline Activity view,
> or from a specific job in the history.
>
> Will look to release 23.3.0 with a fix for this shortly, but if you've
> upgraded already and have noticed any other problems, please let us know.
>
> https://github.com/gocd/gocd/milestone/82?closed=1
>
> -Chad
>
>
> On Sat, 22 Jul 2023, 16:38 Chad Wilson,  wrote:
>
>> Hello everyone,
>>
>> A new release of GoCD <https://www.gocd.org/releases/#23-2-0> (23.2.0)
>> is out.
>>
>> This release is mainly a minor maintenance release. As always, please
>> remember to take a backup before upgrading.
>>
>> To know more about the features and bug fixes in this release, see the 
>> release
>> notes <https://www.gocd.org/releases/#23-1-0> or head to the downloads
>> page <https://www.gocd.org/download/> to try it. Feedback and ideas are
>> always welcome - we appreciate the discussion on issues you are having, and
>> how we can improve things.
>>
>> Cheers,
>> Chad & Aravind
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH_bp00rcQfQEk%2B2Qcp1zW1AnvfhPOcjYTdoz2HgKawRgA%40mail.gmail.com.

Re: [go-cd] shouldn't required resources also be at the pipeline level?

[Chad Wilson]

Yeah, just semantics.  Like you can't know the parameters "name" or
anything like that in the script itself, or do things like "iterate on all
parameter names" - you can only know their injected *value*, so if you want
to make logic depend on a parameter's name, you have to assign it to a
local var in the script first and work with the variable known to the
script, e.g with GoCD *parameter *PIPE_RESOURCE_PARAM=fast and task/script
content

#!/bin/sh
agent_resource="#{PIPE_RESOURCE_PARAM}"

At runtime, the shell runtime execution environment will see the below,
with parameter name gone

#!/bin/sh
agent_resource="fast"

So you can't write logic that varies based on the parameters' *names*
defined in GoCD (that's what I mean by meta-programming). But can still
achieve most things you might like to with other workarounds 

By contrast, *with env vars* you can iterate on the entire env and see if
anything has been defined at GoCD level, e.g with GoCD *env var *
PIPE_RESOURCE_ENV_VAR=fast

#!/bin/sh
env # <--- will print PIPE_RESOURCE_ENV_VAR=fast (among other things)

So slightly different semantics due what the execution environment knows.

-Chad

On Tue, Jul 25, 2023 at 12:41 PM Joshua Franta  wrote:

>
> yes- i'm aware that parameters are replaced before the script is written
> into the agent and executed.
> while it's correct that scripts don't know where the information inside
> them comes from, this is true for any script not just ones used by gocd.
> perhaps these are more semantic level points.
>
> eg i don't know what you mean about meta programming-  you can put a
> pipeline parameter into a variable and do whatever you like with it, same
> as env var.
> but again that's little to do with gocd specifically.
>
> regardless- yes got what i needed!
>
> thx again to everybody who tried to help
>
>
>
> On Mon, Jul 24, 2023 at 11:27 PM Chad Wilson 
> wrote:
>
>> If you read Jason's message a bit more closely he is conveying that the
>> script's runtime environment has no knowledge of the parameters - not that
>> they can't be used at all.
>>
>> They are just tokens that have already been 'realized' or replaced into
>> the content by the time the script/task runs. So the scripting environment
>> itself doesn't know that there were parameters used to generate the content
>> to run/execute and you can't meta-program based on them inside the script's
>> logic. (unlike environment variables)
>>
>> I believe this is in reference to the earlier script-based example you
>> gave which is a little confusing.
>>
>> Anyway, seems you have a way forward here for your core requirement.
>>
>> On Tue, 25 Jul 2023, 11:39 Joshua Franta,  wrote:
>>
>>> Jason, your knowledge here is off. Parameters can be used in scripts,
>>> see a previous email I this thread that shows how it works.
>>>
>>> On Mon, Jul 24, 2023, 4:11 PM Jason Smyth  wrote:
>>>
>>>> Hi Josh,
>>>>
>>>> I think there may be some confusion here regarding GoCD terminology and
>>>> common concepts.
>>>>
>>>> > i think the main source of confusion is that I thought parameters
>>>> could only be referred to in scripts!
>>>> > I didn't know you could refer to them inside of other configuration
>>>> properties!
>>>>
>>>> To the best of my knowledge, Parameters (GoCD concept) cannot be
>>>> referenced in scripts. You can call a script that uses parameters
>>>> (scripting concept), but as far as I know, GoCD Parameters are not
>>>> persisted in the Agent's runtime environment unless they are somehow passed
>>>> in via the Task definition. Are you sure you aren't thinking of Environment
>>>> Variables (GoCD concept)? Environment Variables can be defined in a few
>>>> different places in GoCD. As the name suggests, these values are persisted
>>>> in the Agent's runtime environment when a Task is executed.
>>>>
>>>> > I still have a question about how this works in examples using
>>>> templates.
>>>> > If we didn't define the pipeline parameter by default, how would
>>>> gocd interpret what I'm guessing would be a blank resource?
>>>>
>>>> If a Template references a Parameter then every Pipeline that uses that
>>>> Template _must_ define that Parameter. Depending on how the Parameter is
>>>> used in the Template, leaving the Parameter Value blank may be valid.
>>>>
>>>> In the case of using Parameters to define Resources, my testing shows
>>>> tha

[go-cd] Re: Release Announcement - 23.2.0

[Chad Wilson]

Hi folks - unfortunately a problem sneaked through our automated regression
in 23.2.0 which prevents UI navigation via Stage History to older stage
runs in the history when you're already viewing the detail of an individual
stage (Stage Details view).

You can still navigate to these as normal from the Pipeline Activity view,
or from a specific job in the history.

Will look to release 23.3.0 with a fix for this shortly, but if you've
upgraded already and have noticed any other problems, please let us know.

https://github.com/gocd/gocd/milestone/82?closed=1

-Chad


On Sat, 22 Jul 2023, 16:38 Chad Wilson,  wrote:

> Hello everyone,
>
> A new release of GoCD <https://www.gocd.org/releases/#23-2-0> (23.2.0) is
> out.
>
> This release is mainly a minor maintenance release. As always, please
> remember to take a backup before upgrading.
>
> To know more about the features and bug fixes in this release, see the release
> notes <https://www.gocd.org/releases/#23-1-0> or head to the downloads
> page <https://www.gocd.org/download/> to try it. Feedback and ideas are
> always welcome - we appreciate the discussion on issues you are having, and
> how we can improve things.
>
> Cheers,
> Chad & Aravind
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH9NDOsiMviu%2BKD8TbRReKp0kDm920A%3D1Ojs9sqGpUfS6g%40mail.gmail.com.

Re: [go-cd] shouldn't required resources also be at the pipeline level?

[Chad Wilson]

If you read Jason's message a bit more closely he is conveying that the
script's runtime environment has no knowledge of the parameters - not that
they can't be used at all.

They are just tokens that have already been 'realized' or replaced into the
content by the time the script/task runs. So the scripting environment
itself doesn't know that there were parameters used to generate the content
to run/execute and you can't meta-program based on them inside the script's
logic. (unlike environment variables)

I believe this is in reference to the earlier script-based example you gave
which is a little confusing.

Anyway, seems you have a way forward here for your core requirement.

On Tue, 25 Jul 2023, 11:39 Joshua Franta,  wrote:

> Jason, your knowledge here is off. Parameters can be used in scripts, see
> a previous email I this thread that shows how it works.
>
> On Mon, Jul 24, 2023, 4:11 PM Jason Smyth  wrote:
>
>> Hi Josh,
>>
>> I think there may be some confusion here regarding GoCD terminology and
>> common concepts.
>>
>> > i think the main source of confusion is that I thought parameters
>> could only be referred to in scripts!
>> > I didn't know you could refer to them inside of other configuration
>> properties!
>>
>> To the best of my knowledge, Parameters (GoCD concept) cannot be
>> referenced in scripts. You can call a script that uses parameters
>> (scripting concept), but as far as I know, GoCD Parameters are not
>> persisted in the Agent's runtime environment unless they are somehow passed
>> in via the Task definition. Are you sure you aren't thinking of Environment
>> Variables (GoCD concept)? Environment Variables can be defined in a few
>> different places in GoCD. As the name suggests, these values are persisted
>> in the Agent's runtime environment when a Task is executed.
>>
>> > I still have a question about how this works in examples using
>> templates.
>> > If we didn't define the pipeline parameter by default, how would gocd
>> interpret what I'm guessing would be a blank resource?
>>
>> If a Template references a Parameter then every Pipeline that uses that
>> Template _must_ define that Parameter. Depending on how the Parameter is
>> used in the Template, leaving the Parameter Value blank may be valid.
>>
>> In the case of using Parameters to define Resources, my testing shows
>> that each Parameter must define a single, valid, Resource. That is, if you
>> want to specify multiple Parameterized Resources, you must use multiple
>> Parameters. You cannot, for example, provide a Parameter Value of "foo,
>> bar" to make your Pipeline's Job depend on the "foo" and "bar" Resources.
>> GoCD rejects the configuration as invalid if you try to save it. Similarly,
>> GoCD rejects the configuration as invalid if a Parameter is used in the
>> Resource field and you try to leave its Value blank.
>>
>> Regarding your specific use case, you can solve it using either
>> Environments or Resources. The right solution depends on your requirements
>> and how you want to reason about your environment.
>>
>> The way I understand it, in the context of this discussion you have 2
>> groups of Agents (Agents1 and Agents2) and 2 groups of Pipelines
>> (PipelinesA and PipelinesB). The Pipelines in PipelinesA can run on any
>> Agent, but the Pipelines in PipelinesB must run on the Agents in Agents2.
>> We will ignore the fact that Pipelines can contain multiple Stages and
>> multiple Jobs and assume either that all of the Pipelines contain a single
>> Stage with a single Job, or that the scheduling requirements are the same
>> for all Jobs in a given Pipeline. You have also talked about Pipeline
>> priority.
>>
>> Based on this, I assume your requirements are one of the following:
>>
>> 1. Agents should be used to the full extent possible; the workload in
>> PipelinesB is heavier so those Pipelines must not run on Agents1, or
>> 2. Pipelines in PipelinesB have a higher priority than those in
>> PipelinesA; Agents in Agents2 should take Jobs from PipelinesA only if
>> there are no pending Jobs for PipelinesB.
>>
>> GoCD supports the first scenario. You can achieve this by assigning 2
>> Resources/Environments. Pipelines in PipelinesA get 1 Resource/Environment;
>> Pipelines in PipelinesB get the other. Agents in Agents1 get the PipelinesA
>> Resource/Environment; Agents in Agents2 get both.
>>
>> GoCD does not support the concept of priority, so scenario 2 is not
>> supported. The best you could accomplish would be to map each group of
>> Pipelines to

Re: [go-cd] shouldn't required resources also be at the pipeline level?

[Chad Wilson]

On Mon, Jul 24, 2023 at 8:44 PM Joshua Franta  wrote:

>
> chad thanks for your answer.
>
> i think the main source of confusion is that I thought parameters could
> only be referred to in scripts!
> I didn't know you could refer to them inside of other configuration
> properties!
> Is this documented?   Regardless that's super useful, there's probably
> some other things that can be cleaned up knowing that.
>

https://docs.gocd.org/current/configuration/admin_use_parameters_in_configuration.html#rules-around-usage-of-parameters



> I tried this on a pipeline w/out any template and it worked as described.
>  Just put the parameter reference in resource- UI accepts as long as
> parameter exists and works.
>
> I still have a question about how this works in examples using templates.
> If we didn't define the pipeline parameter by default, how would gocd
> interpret what I'm guessing would be a blank resource?
>
> eg we have
>
>1. a pipeline template called FAST_OR_SLOW_PIPE
>2. every pipeline implementing this template defines a parameter
>called  PIPE_RESOURCE_PARAM
>
> What happens if somebody only defines PIPE_RESOURCE_PARAM when the
> pipeline is FAST?
> If it's left as empty for ANY-aka-SLOW resources, will gocd intepret this
> as a blank resource requirement and fail?
> Or will it ignore blank resources?
>

I'm not sure - perhaps just try it empirically? It could either fail or see
it as blank i.e "no resource requirement" - I don't think there's a strong
case for either behaviour being more correct.

-Chad


> On Sun, Jul 23, 2023 at 10:04 AM Chad Wilson 
> wrote:
>
>> With that description, if you want to use *environments
>> <https://docs.gocd.org/current/introduction/concepts_in_go.html#environment>*
>> rather than resources
>> <https://docs.gocd.org/current/introduction/concepts_in_go.html#resources>
>> (and assuming you don't use environments for any other purpose), I would
>>
>> *1) Create 2 environments "fast" and "any"*
>>
>> *2) Map agents to environments*
>> agents on GROUPA = machines that have less beefy hardware
>> *- declare environments "any" when registered*
>>
>> agents on GROUPB = more expensive machines
>> *- declare both environments "fast" and "any" when registered*
>>
>> *3) When configuring your pipelines*
>>
>>1. have a couple of pipelines only run on the more expensive machines *<--
>>add these pipelines to "fast" environment*
>>2. have all other pipelines run in either group (next available
>>agent) *<-- add these pipelines to "any" environment*
>>
>> This should give you roughly the semantics you say you want, but note it
>> won't *prioritise* the GROUPB agents for use by the "couple of pipelines
>> only run on the more expensive machines", it will just ensure they never
>> run on the slower machines/agents. Something equivalent could also be done
>> with resources
>> <https://docs.gocd.org/current/introduction/concepts_in_go.html#resources>
>> .
>>
>> There is no way to "try another agent" from inside the actual job's
>> tasks. In this sense, the contents of tasks/scripts aren't relevant to
>> scheduling. The GoCD resources and environments have to be known at
>> schedule time. When you use pipeline parameters, they are realised at
>> configuration time as when you create a pipeline from a template, it will
>> force you to set the parameter values.
>>
>> To clarify, when you talked earlier about "a resource requirement" are
>> you *actually* referring to GoCD's concept of resources, or were you
>> talking in a generic sense? The answers are assuming you are talking about
>> GoCD resources
>> <https://docs.gocd.org/current/introduction/concepts_in_go.html#resources>
>> but now I am more confused by your shell script. *If you want to use
>> resources* (rather than environments) to affect scheduling, while still
>> avoiding duplication of your templates, we are suggesting you use a
>> parameter like *this*, not put it into some task content. You are
>> setting the parameterized value into the field that determines the job's
>> scheduling, not something that happens at execution time like a task. But
>> again, if your goal is to control scheduling at pipeline level, for all
>> jobs in a pipeline, you don't need to use resources, and can just use
>> environments as in my earlier example above.
>>
>> [image: image.png]
>>
>> -Chad
>>
>>
>> On Sun, Jul 23, 2023 a

Re: [go-cd] shouldn't required resources also be at the pipeline level?

[Chad Wilson]

With that description, if you want to use *environments
<https://docs.gocd.org/current/introduction/concepts_in_go.html#environment>*
rather than resources
<https://docs.gocd.org/current/introduction/concepts_in_go.html#resources>
(and assuming you don't use environments for any other purpose), I would

*1) Create 2 environments "fast" and "any"*

*2) Map agents to environments*
agents on GROUPA = machines that have less beefy hardware
*- declare environments "any" when registered*

agents on GROUPB = more expensive machines
*- declare both environments "fast" and "any" when registered*

*3) When configuring your pipelines*

   1. have a couple of pipelines only run on the more expensive machines *<--
   add these pipelines to "fast" environment*
   2. have all other pipelines run in either group (next available agent) *<--
   add these pipelines to "any" environment*

This should give you roughly the semantics you say you want, but note it
won't *prioritise* the GROUPB agents for use by the "couple of pipelines
only run on the more expensive machines", it will just ensure they never
run on the slower machines/agents. Something equivalent could also be done
with resources
<https://docs.gocd.org/current/introduction/concepts_in_go.html#resources>.

There is no way to "try another agent" from inside the actual job's tasks.
In this sense, the contents of tasks/scripts aren't relevant to scheduling.
The GoCD resources and environments have to be known at schedule time. When
you use pipeline parameters, they are realised at configuration time as
when you create a pipeline from a template, it will force you to set the
parameter values.

To clarify, when you talked earlier about "a resource requirement" are you
*actually* referring to GoCD's concept of resources, or were you talking in
a generic sense? The answers are assuming you are talking about GoCD
resources
<https://docs.gocd.org/current/introduction/concepts_in_go.html#resources>
but now I am more confused by your shell script. *If you want to use
resources* (rather than environments) to affect scheduling, while still
avoiding duplication of your templates, we are suggesting you use a
parameter like *this*, not put it into some task content. You are setting
the parameterized value into the field that determines the job's
scheduling, not something that happens at execution time like a task. But
again, if your goal is to control scheduling at pipeline level, for all
jobs in a pipeline, you don't need to use resources, and can just use
environments as in my earlier example above.

[image: image.png]

-Chad


On Sun, Jul 23, 2023 at 8:21 PM Joshua Franta  wrote:

> appreciate so many responses.  i think we're a little apart so i'll take
> the suggestion to give our example:
>
> GROUPA = machines that have less beefy hardware
> GROUPB = more expensive machines
>
> we'd like to:
>
>1. have a couple of pipelines only run on the more expensive machines
>2. have all other pipelines run in either group (next available agent)
>
> perhaps this was not clear from my previous explanations, but a couple of
> people have suggested pipeline parameters.
>
> EXAMPLE
>
> a pipeline parameter is only going to be available to the job after the
> job has already been assigned an agent, right?
>
> so if i have a pipeline called 'Priority' w/a parameter  called "group-id"
> and the pipeline has a 'Job' that is a shell script:
>
> 
> ##!/bin/sh
>
> agent_resource="$GO_AGENT_RESOURCE_VARIABLE"
>
> if ! echo "#{group-id}" |grep -q "$agentr_esource"; then
>
>echo "agent can't run #{group-id} pipelines"
>
>## won't this will make my pipeline fail when I want it to simply try
> another agent?
> exit 1
> fi
>
> 
>
> or perhaps people saying this know of some environment variable that where
> we can request another agent?
>
> obviously pipeline parameters themseles don't do anything, so i'm confused
> how i can affect assignment in a job that requires an agent before it runs.
> this 2nd part is what i don't get above
>
> appreciate any clarifications or suggestions thx
>
>
> On Sat, Jul 22, 2023 at 9:58 AM Chad Wilson 
> wrote:
>
>>
>> On Sat, Jul 22, 2023 at 8:21 PM Joshua Franta 
>> wrote:
>>
>>> re: using environments rather than resources... environments can't be
>>> defined at the pipeline level either though?
>>>
>>
>> A pipeline is *assigned* to 0 or 1 environments (via the Admin  >
>> Environments UI if not using pipelines-as-code) - thus it's at the pipeline
>> level by definition. It defines a scheduling requirement for all jobs in
>>

Re: [go-cd] shouldn't required resources also be at the pipeline level?

[Chad Wilson]

On Sat, Jul 22, 2023 at 8:21 PM Joshua Franta  wrote:

> re: using environments rather than resources... environments can't be
> defined at the pipeline level either though?
>

A pipeline is *assigned* to 0 or 1 environments (via the Admin  >
Environments UI if not using pipelines-as-code) - thus it's at the pipeline
level by definition. It defines a scheduling requirement for all jobs in
that pipeline. Which seems what you asked for with "able to communicate a
resource requirement at the pipeline level" right?


or i guess it's more correct to say that using environments is a bit of a
> side-car feature, in that we use interact w/environments through a
> different prisim/ui/config (no biggie) but also seems it's mutually
> exclusive to maximizing overall usage of agents.for us if a given host
> can execute something (a pipeline, a job) it should.  and if it can't, it
> shouldn't.
> trying to force a hard divider can be useful for prod/qa staging, but it
> seems to limit just being able to have pipelines declare their needs.
> maybe i'm missing what you're saying but i don't think environments are
> functionally equivalent to resources?
>

I didn't imply they were functionally equivalent, but I did try to imply
they were a different mechanism of defining a requirement on a job's
scheduling, at the pipeline level. If a pipeline is assigned to an
"environment", its jobs must be scheduled on agents that also declare they
support that "environment". Similarly if a pipeline job declares a resource
requirement, the agent must also have that resource declared for it to be
assigned. This is a very similar, but different level of configuration of a
scheduling requirement, no?
https://docs.gocd.org/current/configuration/managing_environments.html

Anyway, perhaps I don't understand what you are trying to achieve. If you
are currently trying to "prioritise" pipelines by using resources you can
also "prioritise" pipelines by having pools of agents, say, dedicated to an
environment you call "high-priority". As I said, "Don't need to get hung up
on the name [environment]".



> we use template parameters extensively already.
> eg we even templatize further inside our own jobs by re-using scripts that
> interact with template parameters on most commonly used templates (eg our
> most popular template has maybe 10-15 pipelines).
> however this is more of a job specific thing since it's at the job level.
> if you're saying we could change every pipeline to read this at a pipeline
> level is a non trivial change to every job.
>

You said you had many templates that varied only by the "resources" field
for jobs. If that is the stated problem then parameters are a possible
solution to remove duplication, no?


> that's ok but i guess my overall question tho would be that if a given job
> decided it couldn't execute the pipeline parameters... it has no way to
> pass the job to another agent?
>

That's the same problem you have currently if the resource is typoed or
wrong inside the template, no? If the resource requirement has no available
agents, then it can't be scheduled.


> in such an example it would just fail the job, no?   again maybe i'm not
> following but this seems to not allow the business/value level to declare
> minimum needs
> (environments seem like they are more about maximimal requirements, but
> i'm no expert)
>

I'm not following what you're trying to say here, sorry.

Perhaps this would be easier if you gave a specific example of how you
achieve "have some pipelines that are given higher preferences for
agent/build resources" currently, rather than talking in abstract terms?

-Chad


On Sat, Jul 22, 2023 at 6:56 AM Chad Wilson  wrote:
>
>> Have you tried to use "environments" (or a mix of environments and
>> resources) to achieve what you are trying to?
>>
>> When scheduling jobs it's the combination of the resource and the
>> environment that are matched to an agent, but the relevant environment is
>> declared at the pipeline level like you refer to. Don't need to get hung up
>> on the name so much. Yes, you can have "environment variables" attached to
>> an environment and propagate those to all pipelines within it, but you
>> don't have to use them like that.
>>
>> Alternatively, to make the templates less duplicated and allow the
>> resource to flow from the pipeline *using* the template, you could try
>> using template parameters
>> <https://docs.gocd.org/current/configuration/admin_use_parameters_in_configuration.html>
>> in the resources field? e.g #{job-resoure-requirement}? If there are only a
>> small number of different resources used across the stages/jobs, you could
>> use

Re: [go-cd] shouldn't required resources also be at the pipeline level?

[Chad Wilson]

Have you tried to use "environments" (or a mix of environments and
resources) to achieve what you are trying to?

When scheduling jobs it's the combination of the resource and the
environment that are matched to an agent, but the relevant environment is
declared at the pipeline level like you refer to. Don't need to get hung up
on the name so much. Yes, you can have "environment variables" attached to
an environment and propagate those to all pipelines within it, but you
don't have to use them like that.

Alternatively, to make the templates less duplicated and allow the resource
to flow from the pipeline *using* the template, you could try using template
parameters

in the resources field? e.g #{job-resoure-requirement}? If there are only a
small number of different resources used across the stages/jobs, you could
use the parameters to "model" this I imagine.

-Chad

On Sat, Jul 22, 2023 at 6:54 PM Josh  wrote:

> QUESTION:
>
> Shouldn't we also be able to communicate a resource requirement at the
> pipeline level, and not just inside a single job?
>
> I get that it definately needs to be at the job level since that's the
> smallest unit of work and some machines can't execute certain tasks.
> But at the value-stream/pipeline/business level, you also want to be able
> to have some pipelines compiling on preferred resources, no?
>
>
> is there a better way to accomplish this?
> or perhaps this already is possible and i'm missing it.
> i looked closely at the config since sometimes you can do something simple
> that is not possible inside the UI, but I'm not seeing it.
>
> To restate use case:  We have some pipelines that are given higher
> preferences for agent/build resources.   Wanting to do a lot more of this,
> but it's tricky because resources can only be defined at the job level (in
> the UI). Also we use a lot of templates, so having resources at job
> level means we end up having lots of alsomost identical templates that only
> vary by the resources used (which somewhat defeats the point of the
> templates and the value of gocd in this respect).
>
> hoping there is a config hack or maybe i'm missinig something.
> also if this could be done in a plugin, any color there would be helpful
> (and i would make sure it's open sourced if need be).
>
> thx
>
> ps i keep using other ci/cd products and gocd is still one of the all
> around bests.
>
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/a9a4ba2c-b1c9-4202-9408-3e2566929b59n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8zGo6mu0ss0jCCyw0D7Hw4JOwEwfcfNu20yqo0aRRdWw%40mail.gmail.com.

[go-cd] Release Announcement - 23.2.0

[Chad Wilson]

Hello everyone,

A new release of GoCD  (23.2.0) is
out.

This release is mainly a minor maintenance release. As always, please
remember to take a backup before upgrading.

To know more about the features and bug fixes in this release, see the release
notes  or head to the downloads page
 to try it. Feedback and ideas are always
welcome - we appreciate the discussion on issues you are having, and how we
can improve things.

Cheers,
Chad & Aravind

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-7w%3D%3DFrVcrMT%3DKK0umoEOxCddgx0wcTNfdgh_A4xXn_g%40mail.gmail.com.

Re: [go-cd] Failed to find 'git' in path with Helm based installation

[Chad Wilson]

Yeah, the default images are Alpine based and not built for arm64/aarch64,
see this for nasty detail <https://github.com/gocd/gocd/issues/11355>.

I use colima rather than rancher but it is horrifically slow under QEMU
emulation, and still weirdly unstable like you experienced (although never
seen that git error as a result). Never bothered with Rosetta, but for some
reason qemu doesn't like the JVM very much on the GoCD images. It is
possibly a whole lot of problems mixed up together.

A *much higher performing and more stable* alternative for you will be to
add --set server.image.repository=gocd/gocd-server-centos-9 --set
agent.image.repository=gocd/gocd-agent-centos-9 or equivalent in values
overrides when you install/upgrade the chart. This switches to the
CentOS-based images which are a bit bigger, but perfectly stable and built
multi-arch including arm64. For the *agent,* all the non-alpine images
<https://www.gocd.org/download/#docker> (ubuntu, debian, centos) have been
built multi-arch since 23.1.0.

This is how I test/validate/develop locally, which is also on an Apple
Silicon Mac.

Probably could do with being better documented on the Helm Chart itself,
PRs welcome 

-Chad

On Wed, Jul 19, 2023 at 11:42 PM 'Andreas Hubert' via go-cd <
go-cd@googlegroups.com> wrote:

> Okay, so it did not worked in Rancher Desktop when I enabled Virtual
> Machine Emulation VZ with enabled Rosetta Option. It also did not worked
> with Virtual Machine Emulation QEMU. But finally it works now with VZ, but
> Rosetta option unchecked.
>
> Thanks for the hint Chad!
>
> Andreas Hubert schrieb am Mittwoch, 19. Juli 2023 um 16:58:39 UTC+2:
>
>> > At a guess, is this perhaps a local cluster on an M1 Mac?
>> Good guess ;)
>>
>> When I check the logs from the pod, I get this error upon checking
>> connection for sample Material:
>> jvm 1| 2023-07-19 14:52:28,756  INFO [166@MessageListener for
>> ServerPingListener] p.c.g.c.e.k.c.g.c.e.KubernetesPlugin:72
>> [plugin-cd.go.contrib.elasticagent.kubernetes] - [refresh-pod-state] Pod
>> information successfully synced. All(Running/Pending) pod count is 0.
>> jvm 1| 2023-07-19 14:52:30,015 ERROR [124@MessageListener for
>> MaterialUpdateListener] ProcessManager:102 - [Command Line] Failed
>> executing [git clone --branch master --no-checkout
>> https://github.com/gocd-contrib/getting-started-repo
>> /go-working-dir/pipelines/flyweight/8ad0eaec-5e2d-4f61-bfd6-dc26f7f67818]
>> jvm 1| 2023-07-19 14:52:30,015 ERROR [124@MessageListener for
>> MaterialUpdateListener] ProcessManager:103 - [Command Line] Agent's
>> Environment Variables: {GOCD_APP_SERVER_SERVICE_PORT_HTTP=8153,
>> PATH=/usr/local/sbin:/usr/local/bin:/usr/sbin:/usr/bin:/sbin:/bin,
>> WRAPPER_JAVA_VERSION_MINOR=0,
>> WRAPPER_HOSTNAME=gocd-app-server-5c9dd5b56c-646pn, WRAPPER_BITS=64,
>> WRAPPER_VERSION=3.5.51, WRAPPER_BASE_NAME=wrapper,
>> GOCD_APP_SERVER_SERVICE_PORT=8153,
>> WRAPPER_HOST_NAME=gocd-app-server-5c9dd5b56c-646pn,
>> WRAPPER_JAVA_VENDOR=OpenJDK, PWD=/, KUBERNETES_PORT_443_TCP=tcp://
>> 10.43.0.1:443, LANGUAGE=en_US:en,
>> GOCD_PLUGIN_INSTALL_docker-registry-artifact-plugin=
>> https://github.com/gocd/docker-registry-artifact-plugin/releases/download/v1.3.1-485/docker-registry-artifact-plugin-1.3.1-485.jar,
>> WRAPPER_EDITION=Standard, GOCD_APP_SERVER_PORT_8153_TCP_PROTO=tcp,
>> LC_ALL=en_US.UTF-8, WRAPPER_JAVA_VERSION_REVISION=6,
>> WRAPPER_JAVA_VERSION=17.0.6, KUBERNETES_SERVICE_PORT_HTTPS=443, SHLVL=1,
>> WRAPPER_PID=115, WRAPPER_WORKING_DIR=/go-working-dir, WRAPPER_OS=linux,
>> KUBERNETES_PORT=tcp://10.43.0.1:443,
>> GOCD_APP_SERVER_SERVICE_HOST=10.43.100.193,
>> KUBERNETES_SERVICE_HOST=10.43.0.1, LANG=en_US.UTF-8,
>> WRAPPER_BIN_DIR=/go-server/wrapper,
>> WRAPPER_CONF_DIR=/go-server/wrapper-config, WRAPPER_LANG=en,
>> GOCD_APP_SERVER_PORT_8153_TCP=tcp://10.43.100.193:8153,
>> WRAPPER_FILE_SEPARATOR=/, WRAPPER_INIT_DIR=/,
>> KUBERNETES_PORT_443_TCP_ADDR=10.43.0.1,
>> GOCD_APP_SERVER_PORT_8153_TCP_ADDR=10.43.100.193,
>> GOCD_PLUGIN_INSTALL_kubernetes-elastic-agents=
>> https://github.com/gocd/kubernetes-elastic-agents/releases/download/v3.9.0-442/kubernetes-elastic-agent-3.9.0-442.jar,
>> GO_JAVA_HOME=/gocd-jre, WRAPPER_PATH_SEPARATOR=:,
>> KUBERNETES_PORT_443_TCP_PROTO=tcp, KUBERNETES_SERVICE_PORT=443,
>> GOCD_APP_SERVER_PORT=tcp://10.43.100.193:8153,
>> HOSTNAME=gocd-app-server-5c9dd5b56c-646pn, WRAPPER_JAVA_VERSION_MAJOR=17,
>> WRAPPER_RUN_MODE=console, WRAPPER_ARCH=x86,
>> GOCD_APP_SERVER_PORT_8153_TCP_PORT=8153, KUBERNETES_PORT_443_TCP_PORT=443,
>> HOME=/home/go}
>>
>> Which is weird, because if I just run those commands directly with git,
&

Re: [go-cd] Failed to find 'git' in path with Helm based installation

[Chad Wilson]

The core error regarding git you are seeing is not directly related to the
agent not coming up, but they may have the same root cause.

What operating system, hardware architecture and Kubernetes variant are you
deploying the Helm chart to?

At a guess, is this perhaps a local cluster on an M1 Mac?

-Chad

On Wed, Jul 19, 2023 at 10:28 PM 'Andreas Hubert' via go-cd <
go-cd@googlegroups.com> wrote:

> Hi all!
> I just wanted to play and experiment a little bit with GoCD and tried to
> use the Helm chart for my own k8s cluster.
> But when I try to add Material or work with the sample Material and test
> connection, I get this error:
>
> Failed to find 'git' on your PATH. Please ensure 'git' is executable by
> the Go Server and on the Go Agents where this material will be used.
>
>
> If I check the resources in my namespace, it seems the agent is not coming
> up. Could this be related?
> NAME   READY   STATUSRESTARTS   AGE
> pod/gocd-app-server-5c9dd5b56c-646pn   1/1 Running   0  44m
>
> NAME  TYPE   CLUSTER-IP  EXTERNAL-IP   PORT(S)
>  AGE
> service/gocd-app-server   NodePort   10.43.100.193   
>  8153:30760/TCP   44m
>
> NAME  READY   UP-TO-DATE   AVAILABLE   AGE
> deployment.apps/gocd-app-agent0/0 00   44m
> deployment.apps/gocd-app-server   1/1 11   44m
>
> NAME DESIRED   CURRENT   READY
> AGE
> replicaset.apps/gocd-app-agent-54b5bdc7670 0 0
> 44m
> replicaset.apps/gocd-app-server-5c9dd5b56c   1 1 1
> 44m
>
> Thanks for any hint!
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/6ad3fb0c-a828-43fc-b103-e086cf7b293cn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8YuEqHEOJv_jSUknU4f98SEUZBPOrinH-7TXcWDgA40Q%40mail.gmail.com.

Re: [go-cd] source code compile problem

[Chad Wilson]

Maybe, GoCD is built with Node 18 right now.

It still uses Webpack 4, which might not work correctly with Node 20.

-Chad

On Mon, 17 Jul 2023, 09:59 jianhua guo,  wrote:

> After running the command "yarn cache clean" and "rm -rf
> server/src/main/webapp/WEB-INF/rails/node_modules", I compile the project
> again, then seeing the following exception, is it the nodejs version is too
> high?
>
> > Task :server:compileAssetsWebpackDev FAILED
> [/Users/guojianhua/gtja/projects/gocd/server/src/main/webapp/WEB-INF/rails]$
> yarn run webpack-dev --env
> outputDir=/Users/guojianhua/gtja/projects/gocd/server/src/main/webapp/WEB-INF/rails/public/assets/webpack
> --env
> licenseReportFile=/Users/guojianhua/gtja/projects/gocd/server/target/reports/yarn-license/license-report.json
> yarn run v1.22.19
> $ cross-env NODE_OPTIONS=--openssl-legacy-provider webpack --config
> webpack/config/webpack.config.ts --color --mode=development --env
> outputDir=/Users/guojianhua/gtja/projects/gocd/server/src/main/webapp/WEB-INF/rails/public/assets/webpack
> --env
> licenseReportFile=/Users/guojianhua/gtja/projects/gocd/server/target/reports/yarn-license/license-report.json
> /opt/homebrew/Cellar/node/20.3.0_1/bin/node: --openssl-legacy-provider is
> not allowed in NODE_OPTIONS
> error Command failed with exit code 9.
>
> 在2023年7月14日星期五 UTC+8 15:06:32 写道：
>
>> The commit is still at
>> https://github.com/dtabuenc/karma-html-reporter/commit/51ba3f91a6f19ef383c676431aa7f8c3fa73dab3
>> so it *should* work fine.
>>
>> Perhaps try yarn cache clean and rm -rf
>> server/src/main/webapp/WEB-INF/rails/node_modules and try again?
>>
>> If that doesn't work, what's your Yarn version? Do you have some
>> non-standard global git or Yarn configuration?
>>
>> -Chad
>>
>> On Fri, Jul 14, 2023 at 2:43 PM jianhua guo  wrote:
>>
>>> I use the following command './gradlew clean prepare' to build the gocd
>>> project. Failed with the following exception, can anyone help?
>>>
>>> error Couldn't find match for "51ba3f91a6f19ef383c676431aa7f8c3fa73dab3"
>>> in
>>> "refs/heads/dependabot/npm_and_yarn/grunt-1.5.3,refs/heads/dependabot/npm_and_yarn/hosted-git-info-2.8.9,refs/heads/dependabot/npm_and_yarn/minimatch-3.0.8,refs/heads/dependabot/npm_and_yarn/minimist-and-mkdirp-1.2.8,refs/heads/karma-0.11,refs/heads/master,refs/tags/0.1.2,refs/tags/v0.1.3,refs/tags/v0.2.2,refs/tags/v0.2.3"
>>> for "https://github.com/dtabuenc/karma-html-reporter.git;.
>>> info Visit https://yarnpkg.com/en/docs/cli/install for documentation
>>> about this command.
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+un...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/9a3d1fd9-4c6b-4eaf-a43a-c52986b0f038n%40googlegroups.com
>>> 
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/b7b17d17-10e8-4d2d-8ffc-3f1b32f13a9en%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH9D57sas_KV7k3H%2B7%3DHwUDn_Bc-tRGskKXD%2BV%3Dvdny4VQ%40mail.gmail.com.

Re: [go-cd] source code compile problem

[Chad Wilson]

The commit is still at
https://github.com/dtabuenc/karma-html-reporter/commit/51ba3f91a6f19ef383c676431aa7f8c3fa73dab3
so it *should* work fine.

Perhaps try yarn cache clean and rm -rf
server/src/main/webapp/WEB-INF/rails/node_modules and try again?

If that doesn't work, what's your Yarn version? Do you have some
non-standard global git or Yarn configuration?

-Chad

On Fri, Jul 14, 2023 at 2:43 PM jianhua guo  wrote:

> I use the following command './gradlew clean prepare' to build the gocd
> project. Failed with the following exception, can anyone help?
>
> error Couldn't find match for "51ba3f91a6f19ef383c676431aa7f8c3fa73dab3"
> in
> "refs/heads/dependabot/npm_and_yarn/grunt-1.5.3,refs/heads/dependabot/npm_and_yarn/hosted-git-info-2.8.9,refs/heads/dependabot/npm_and_yarn/minimatch-3.0.8,refs/heads/dependabot/npm_and_yarn/minimist-and-mkdirp-1.2.8,refs/heads/karma-0.11,refs/heads/master,refs/tags/0.1.2,refs/tags/v0.1.3,refs/tags/v0.2.2,refs/tags/v0.2.3"
> for "https://github.com/dtabuenc/karma-html-reporter.git;.
> info Visit https://yarnpkg.com/en/docs/cli/install for documentation
> about this command.
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/9a3d1fd9-4c6b-4eaf-a43a-c52986b0f038n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH_KE%2B1aWRB%2BS1y1fgYWKr6z0YNBCnFH2aaxrazj3jBhNg%40mail.gmail.com.

Re: [go-cd] Go-Agent || CVE-2022-42889

[Chad Wilson]

Hiya

GoCD has been using commons-text 1.10 (with the issue you refer to fixed)
since GoCD 22.3.0:
https://github.com/gocd/gocd/commit/293022076385c48c9fb41485b5674fa2e69c29c1

The agent *bootstrapper* doesn't use commons-text at all, however the agent
jar which is dynamically downloaded from the server and matches the
server's version does use commons-text. You might want to double check your
server is running GoCD version 22.3.0 or later?

-Chad

On Mon, Jul 10, 2023 at 11:06 PM Mai M. Khattab 
wrote:

> Hello There,
> Any idea how can if there a remediation for (CVE-2022-42889 -  Arbitrary
> code execution in Apache Commons Text · CVE-2022-42889 · GitHub Advisory
> Database   ) on
> (go-agent), please?
> I am using go-agent (v23.1) and I found it is using commons-text (v1.9)
> Regards,
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/29cd81fe-b404-41c8-8db4-260e1204d00cn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-MHbMNTohr%3DTODFWgg7CysPi5Y2Met-8%3D6rrjfV7id_g%40mail.gmail.com.

Re: [go-cd] Postgres backup not working on gocd server pod

[Chad Wilson]

Hi Alex

You would need to either
- build your own custom child GoCD server image which contains the relevant
postgres binaries for your Postgres version, push 8t to our own registry
and then override the repository/image:tag when installing the Helm chart,
or
- use an additional docker entrypoint script to dynamically install them
(or download from some internal artifact store) every time the container
starts, or
- mount an existing persistent volume containing the binaries using the
helm chart configuration and add it to the PATH

The official images so not contain these binaries as they are somewhat
database version dependent and it's not the 'default' configuration.

-Chad

On Tue, 20 Jun 2023, 19:41 Alexandru Cojocariu, <
cojocariualexand...@gmail.com> wrote:

> Hello,
>
> I am facing an issue regarding the GoCD postgres backup. I am using an
> external postgres server and I am trying to trigger a backup (using the
> GoCD API) with a curl command:
>
> curl 'https:///go/api/backups' -i -u
> ':' -H 'X-GoCD-Confirm: true' -H 'Accept:
> application/vnd.go.cd.v2+json' -X POST
>
> When inspecting the output in the gocd-server I get:
>
>
>
> *jvm 1| java.lang.RuntimeException: java.lang.RuntimeException: There
> was an error backing up the database using `pg_dump`. The `pg_dump` process
> failed with code 2.   jvm 1| Caused by:
> java.io.IOException: Cannot run program "pg_dump": error=2, No such file or
> directory*
> jvm 1|  at java.base/java.lang.ProcessBuilder.start(Unknown Source)
> jvm 1|  at java.base/java.lang.ProcessBuilder.start(Unknown Source)
> jvm 1|  at
> org.zeroturnaround.exec.ProcessExecutor.invokeStart(ProcessExecutor.java:997)
> jvm 1|  at
> org.zeroturnaround.exec.ProcessExecutor.startInternal(ProcessExecutor.java:970)
> jvm 1|  at
> org.zeroturnaround.exec.ProcessExecutor.execute(ProcessExecutor.java:906)
> jvm 1|  at
> com.thoughtworks.go.server.database.pg.PostgresqlBackupProcessor.backup(PostgresqlBackupProcessor.java:43)
> jvm 1|  ... 10 common frames omitted
> *jvm 1| Caused by: java.io.IOException: error=2, No such file or
> directory*
> jvm 1|  at java.base/java.lang.ProcessImpl.forkAndExec(Native
> Method)
> jvm 1|  at java.base/java.lang.ProcessImpl.(Unknown Source)
> jvm 1|  at java.base/java.lang.ProcessImpl.start(Unknown Source)
> jvm 1|  ... 16 common frames omitted
>
> The server is a k8s pod deployed through a helmchart. The helmchart used
> is: https://artifacthub.io/packages/helm/gocd/gocd - version 2.1.0
> The gocd image specified in the helmchart is: gocd-server v23.1.0
> The postgres server is connected according to the specifications from
> here:
> https://docs.gocd.org/current/installation/configuring_database/connection-properties.html
> My question is: can I configure somehow the helmchart to recognize the
> pg_dump command or the postgresql tools in general?
>
> Thanks,
> Alex
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/e384ce50-6e2d-4f81-b95f-3abb50c3cc2fn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH_NYV%2B2CE4_fvsy52ZfVKJxFDhzaM%3DsN-j95_Yx%3DbT5jQ%40mail.gmail.com.

Re: [go-cd] Config repository and github webhook

[Chad Wilson]

There might be some confusion here, triggering a config repo material
update from a webhook is via webhooks such as

https://gocd.somewhere.org/go/api/webhooks/github/config_repos/


I believe this has been available since GoCD 21.1.0 (it may work with
regular https://gocd.somewhere.org/go/api/webhooks/github/notify type of
webhooks too, if you are not trying to do anything fancy like pull
request/branch support). The special purpose config repo webhook is
possibly not properly documented
 outside the in-server
guidance (see below).

If you expand the config repo in the admin view, GoCD tells you the
specific webhook URL to use, e.g with a YAML config repo:

[image: image.png]

A "pluggable SCM" is an unrelated concept for source control materials
defined via a plugin, rather than built-in GoCD support (git, svn etc). If
you have nothing in Admin > Pluggable SCMs then you are not using pluggable
SCMs and that part is not relevant to you.

-Chad

On Thu, Jun 15, 2023 at 2:47 PM Marc  wrote:

> Hi,
> after reading https://github.com/gocd/gocd/issues/8170 I was thinking
> that the webhook notification for config repositories would work now.
> But I didn't get it working.
> I've registered the following URL in Github:
> https://gocd.somewhere.org/go/api/webhooks/github/notify?SCM_NAME=
> 
> I thought that the "NameOfConfigRepoConfig" would be correct. It was also
> shown in the logs when the webhook was called, but it didn't start a reread
> of the config repository.
> When reading the api docs I found the note: "The scm_name is the material
> name as defined in pipeline material page." But the config-repo isn't
> listed there. As such I've got no idea what I've got to do to schedule a
> rescan of the config repository. Or is it still not possible? :(
>
> thanks for your help...
> marc
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/1cea6e87-72b2-4fad-8d72-8fc7817caeefn%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH_Kf3LsdYgTigjORwbrYwXE0M-Y_6CMbvzH5SS%2BRCgShg%40mail.gmail.com.

Re: [go-cd] Queries on Webhook

[Chad Wilson]

I'm not sure I understand the question or what you are trying to achieve,
however.

Webhooks notify the GoCD server of a change to *materials*. The GoCD server
then finds the relevant material(s), and updates them from source to
determine the changes. It then follows normal operations (as it would with
polling) to determine which pipelines are affected by the changes to the
material (there may be one or more pipelines linked to a given material,
denylists/whitelists to consider, plugins to ask etc).

In some sense it sounds like you are thinking about this in slightly the
wrong way. In GoCD materials are a top level concept which manage incoming
triggers to pipelines. So you don't trigger a pipeline, you trigger a
"check" on a material. Pipelines see "stage triggered due to change in
material", not "pipeline triggered by webhook".

https://www.gocd.org/getting-started/part-1/#concept2

As to whether webhook events are "batched", I don't believe they are in any
deterministic way. If pipeline A has input materials X and material Y, and
there are 2 webhook events that trigger an update to both X and Y, you may
or may not get two pipeline runs - similar to what might happen if the
server was polling the materials in "normal" operation. I'm personally not
sure if it's 1:1 or not in the case of webhook-triggered material updates,
but speaking generally I doubt GoCD gives this guarantee. e.g if a
pipeline's *stage1* is already running, and then two webhooks are fired
that affect that pipeline's materials, I would *guess* that they would end
up batched into one pipeline instance when stage1 next triggers.

As to "status", I don't think the material subsystem exposes any particular
status to know what it is doing, how it is responding to webhook changes
etc. Probably you could enable some debug logging if trying to diagnose a
particular problem, but don't have the relevant config to hand.

-Chad

On Thu, Jun 15, 2023 at 9:55 AM Manikkumar K 
wrote:

> How do we know the status of the web hook, for example, Say Im firing more
> than one web hook to trigger a pipeline, is there a way we can query the
> queue status & pipeline instance of the web hook(s) it has picked up?
>
> Second query is above, if we would fire more than one web hook to a same
> pipeline, would go server aggregate all into one singe instance? Or is it
> like 1-1 mapping where each web hook corresponds to one pipeline instance?
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/98459935-967a-4c15-8a27-0d6afbe5fe72n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8RC4Xx4nEk%2BKzXoPRacrg23Vj%3DuzhcTYktbbnjAC_vDQ%40mail.gmail.com.

Re: [go-cd] Modify Fan in behaviour

[Chad Wilson]

So sometimes with one repo and multiple pipelines you want fan-in
behaviour, e.g if you are splitting the pipelines to run different logical
testing blocks in parallel and either

   - don't want to model them as parallel jobs within one pipeline or
   - where pipelines require multiple source code material inputs, and you
   don't want excessive triggering of unrelated pipeline work

Basically the classic cases documented at
https://www.gocd.org/getting-started/part-3/#fan_out_fan_in or
https://www.gocd.org/2017/04/17/build-propagation-using-fan-in-fan-out.html

Other times you have multiple things in one repo for ease of management,
but things are sufficiently segregated and you are OK with treating the
pieces independently. In this case fan-in might not be what you want/need,
despite sharing a repository/material.

Let's say you have some more classic mono repo A:

./deploy-scripts/
./service-common/
./service-A/
./frontend/

Perhaps you then have pipelines like

*repoA *--*fanout*--> frontend-build --> frontend-functional-test --*fanin*-->
frontend-plus-service-A-smoke-test --*fanout*--> (deploy-frontend-to-UAT +
deploy-service-A-to-UAT)
*repoA* --*fanout*--> service-A-build --> service-A-functional-test --
*fanin*--> frontend-plus-service-A-smoke-test --*fanout*-->
(deploy-frontend-to-UAT + deploy-service-A-to-UAT)

In a normal setup with GoCD, frontend-plus-service-A-smoke-test will only
trigger when all of the upstreams are built on the same revision from
repoA. If the frontend tests are perhaps far slower, this may unnecessarily
slow progress down for promoting service-A code changes to UAT, and require
triggering of everything.

You may be confident enough that they can be built and deployed
independently, but still want some confidence that the frontend and
service-A play nicely together which is why you have
"frontend-plus-service-A-smoke-test". In this case you may want to take
more responsibility on yourselves, and consider something like
https://github.com/TWChennai/gocd-git-path-material-plugin to allow the
deploys to happen independently, but without excessive triggering of
unrelated pieces, without splitting your repository. With something like
this you might instead model like

*repoA (path: deploy-scripts/ + frontend/*) --> frontend-build -->
frontend-functional-test --> frontend-plus-service-A-smoke-test --*fanout*-->
(deploy-frontend-to-UAT + deploy-service-A-to-UAT)
*repoA (path: deploy-scripts/ + service-common/ + service-A/)*  -->
service-A-build --> service-A-functional-test -->
frontend-plus-service-A-smoke-test --*fanout*--> (deploy-frontend-to-UAT +
deploy-service-A-to-UAT)

This would logically split the materials while allowing you to commit
together in one repo, essentially working around the limitations of
filtering noted here
<https://docs.gocd.org/current/advanced_usage/fan_in.html#limitations>.

Anyway, these are just different tools in the arsenal, depending on what
you are trying to achieve, and how you are modelling your pipelines and
splitting your source code, test/deploy scripting etc.

-Chad

On Tue, Jun 13, 2023 at 8:39 PM Manikkumar K 
wrote:

> You are spot on. We do share one of the repositories for the all
> microservice(aka upstreams).
> And was able to simulate and see where the fan-in didn't happen when
> upstreams do not share any of the materials.
>
> Thanks for prompt response,
> Mani
>
> On Tue, Jun 13, 2023 at 8:43 AM Chad Wilson 
> wrote:
>
>> If it's "not monorepo", can you clarify if you have multiple upstreams
>> that point to one repository/material? That is the scenario that triggers
>> fan-in, and that does at least indicate that the repo is somewhat being
>> treated as a monorepo in the sense that you have multiple pipelines
>> directly dependent on that repository.
>>
>> Otherwise you might need to share more specific examples of pipelines,
>> dependencies and the materials they have, along with the behaviour you
>> observe vs. desire.
>>
>> To successfully trigger a downstream, all upstreams only need to pass (on
>> a given material version) if they share a material. If they *do not*
>> share 1 or more materials then the behaviour should be as you describe -
>> the downstream will trigger with the last successful version of each
>> upstream stage (no fan-in required).
>>
>> -Chad
>>
>> On Tue, Jun 13, 2023 at 11:01 AM Manikkumar K <
>> manikkuma...@thoughtworks.com> wrote:
>>
>>> Ours is not monorepo. Is there any other alternative?
>>>
>>> On Tue, Jun 13, 2023, 07:18 Chad Wilson  wrote:
>>>
>>>> Doing that means removing the guarantees fan-in offers you as to
>>>> consistency of the material versions on upstream pipelines when triggering
>&

Re: [go-cd] Modify Fan in behaviour

[Chad Wilson]

If it's "not monorepo", can you clarify if you have multiple upstreams that
point to one repository/material? That is the scenario that triggers
fan-in, and that does at least indicate that the repo is somewhat being
treated as a monorepo in the sense that you have multiple pipelines
directly dependent on that repository.

Otherwise you might need to share more specific examples of pipelines,
dependencies and the materials they have, along with the behaviour you
observe vs. desire.

To successfully trigger a downstream, all upstreams only need to pass (on a
given material version) if they share a material. If they *do not* share 1
or more materials then the behaviour should be as you describe - the
downstream will trigger with the last successful version of each upstream
stage (no fan-in required).

-Chad

On Tue, Jun 13, 2023 at 11:01 AM Manikkumar K 
wrote:

> Ours is not monorepo. Is there any other alternative?
>
> On Tue, Jun 13, 2023, 07:18 Chad Wilson  wrote:
>
>> Doing that means removing the guarantees fan-in offers you as to
>> consistency of the material versions on upstream pipelines when triggering
>> a downstream.
>>
>> It's possible to turn off fan-in globally (although I have never done so
>> personally) but to my knowledge not for individual pipelines.
>>
>> Alternatively, if your upstreams are some sort of monorepo which are
>> segregated by upstream pipelines (e.g each pipeline operates on different
>> subdirectories or parts of the repo) you could consider using something
>> like https://github.com/TWChennai/gocd-git-path-material-plugin to
>> define separate materials for each of the upstreams that share a repo with
>> different subsets of the repo that they operate on so GoCD doesn't decide
>> they need fan-in.
>>
>> -Chad
>>
>> On Tue, 13 Jun 2023, 08:39 Manikkumar K, 
>> wrote:
>>
>>> We do have a fan-in where number upstream has exploded in the count,
>>>
>>> Instead of waiting for all upstream to pass, can we pick most recent
>>> successful upstream pipeline along with new pipeline instance of one(or
>>> more) of upstream?
>>>
>>> Thanks,
>>> Mani
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/4bcad889-0820-427b-ac85-d91e50f24e43n%40googlegroups.com
>>> <https://groups.google.com/d/msgid/go-cd/4bcad889-0820-427b-ac85-d91e50f24e43n%40googlegroups.com?utm_medium=email_source=footer>
>>> .
>>>
>> --
>> You received this message because you are subscribed to a topic in the
>> Google Groups "go-cd" group.
>> To unsubscribe from this topic, visit
>> https://groups.google.com/d/topic/go-cd/VVTYzU5OhV4/unsubscribe.
>> To unsubscribe from this group and all its topics, send an email to
>> go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/CAA1RwH-Pub%3DUtDr0QEc-yS5DoR3BbKbagC0Jk8rfSYFDbiXe5Q%40mail.gmail.com
>> <https://groups.google.com/d/msgid/go-cd/CAA1RwH-Pub%3DUtDr0QEc-yS5DoR3BbKbagC0Jk8rfSYFDbiXe5Q%40mail.gmail.com?utm_medium=email_source=footer>
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CACfO_cKcqovDwHrmD0G_yBP1cCYJJT%2BMs5hxV9KeTKrQmuf-OQ%40mail.gmail.com
> <https://groups.google.com/d/msgid/go-cd/CACfO_cKcqovDwHrmD0G_yBP1cCYJJT%2BMs5hxV9KeTKrQmuf-OQ%40mail.gmail.com?utm_medium=email_source=footer>
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH8n%2BOm%3D5L13B3srVAhNZAPnHdrPZKJMh2sC7ZQrCB%2BJBw%40mail.gmail.com.

Re: [go-cd] Modify Fan in behaviour

[Chad Wilson]

Doing that means removing the guarantees fan-in offers you as to
consistency of the material versions on upstream pipelines when triggering
a downstream.

It's possible to turn off fan-in globally (although I have never done so
personally) but to my knowledge not for individual pipelines.

Alternatively, if your upstreams are some sort of monorepo which are
segregated by upstream pipelines (e.g each pipeline operates on different
subdirectories or parts of the repo) you could consider using something
like https://github.com/TWChennai/gocd-git-path-material-plugin to define
separate materials for each of the upstreams that share a repo with
different subsets of the repo that they operate on so GoCD doesn't decide
they need fan-in.

-Chad

On Tue, 13 Jun 2023, 08:39 Manikkumar K, 
wrote:

> We do have a fan-in where number upstream has exploded in the count,
>
> Instead of waiting for all upstream to pass, can we pick most recent
> successful upstream pipeline along with new pipeline instance of one(or
> more) of upstream?
>
> Thanks,
> Mani
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/4bcad889-0820-427b-ac85-d91e50f24e43n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-Pub%3DUtDr0QEc-yS5DoR3BbKbagC0Jk8rfSYFDbiXe5Q%40mail.gmail.com.