Re: [go-cd] goCD Yaml plugin doesn't pick-up code

[Chad Wilson]

As the error mentions please "Check the 'Rules' of this config repository"
- by default your config repo won't have rules/permissions to define
pipelines in any pipeline groups, environments etc. It is "secure by
default". You'll want to add a permission to do so.

e.g a blanket permission to allow a config repo to affect any environment
or pipeline group looked like: You can click the "learn more" link from the
config repo edit to read more about these rules.

[image: image.png]
-Chad

On Fri, Oct 27, 2023 at 1:27 PM vv-fork  wrote:

> Greetings colleagues,
>
> I am trying to get into Yaml plugin (*gocd-yaml-config-plugin.jar* v.
> *0.14.3-321*) with goCD v.23.3.0.
>
> The plugin is able to receive the yaml file but always throws the same
> error.
>
> I am stuck with something pretty simple, like this:
> format_version: 10
>
> pipelines:
>   mypipe:
> group: mygroup
> materials:
>   mygit:
> git: http://example.com/mygit.git
> stages:
>   - build:
>   jobs:
> build:
>   tasks:
>- exec:
>command: make
>
> environments:
>
> The message is:
> INVALID MERGED CONFIGURATION Number of errors: 1+ I. Rule Validation
> Errors: 1. Not allowed to refer to pipeline group 'mygroup'. Check the
> 'Rules' of this config repository. II. Config Validation Errors: - For
> Config Repo: https://github.com/...p at revision 03a639fd6444>>>74d5
>
> What am I doing wrong?
>
> Vlad.
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/01c04e91-0322-45fa-9021-58dd90424b80n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH9LXz3xBiN822wKt4GaF8m9232nnjsJvdwMjTf_2VsAiA%40mail.gmail.com.

Re: [go-cd] GoCD and private GitHub repo

[Chad Wilson]

Based on the error message it looks like the clone URL you are using is
still an HTTPS one - to use SSH auth, you need to change it to an ssh URL,
e.g g...@github.com:gocd/gocd.git - it's an entirely different git
transport, not just an auth mechanism so the URL needs to change
accordingly :-)

If you're new to using SSH to talk to a git repo manager, you might want to
try doing it separately on the command line with a git clone before getting
it to work with GoCD as adding in the GoCD server and agent adds some extra
complexity.

-Chad

On Fri, Oct 27, 2023 at 1:13 PM vv-fork  wrote:

> Thank you guys Sriram and Chad for answering those! Now it's getting
> clearer to me
>
> *I was able to connect using token. It's fine.*
>
> Though I* wasn't able* to connect using SSH Certificate.
> The key has been generated and installed:
> [image: unnamed.png]
>
> it seems i set proper permissions:
> [image: unnamed.png]
>
> but i still get that error message when i test connection from go-server:
> --- STANDARD ERR --- STDERR: fatal: could not read Username for '
> https://github.com': No such device or address ---
>
>
> So what am i doing wrong? May that be I have messed with permissions for *go
> user*?
>
>
> Vlad.
>
> On Thursday, October 26, 2023 at 8:41:33 PM UTC+13 Chad Wilson wrote:
>
>> To add on to Sriram's comments, the use of the
>> github-oauth-authorization-plugin doesn't have any relationship with access
>> to repository content on GitHub - it simply allows people to log onto GoCD
>> using their Github identity, and optionally to have access to GoCD pipeline
>> groups mapped to GitHub roles.
>>
>> This is because materials/repositories need to be accessed in an identity
>> known to the GoCD server/agents, not necessarily the individual user who
>> happens to be logged in to GoCD. So even if you use that authorization
>> plugin, you still need to decide how to provide GoCD itself access to
>> repositories on Github.
>>
>> You can use an SSH key linked to a GitHub user
>> 
>> if you wish to use SSH access - no restrictions for private repos unless
>> your GitHub org blocks use of SSH keys. If you instead wish to use HTTPS
>> access to repositories you have to fill in a username/"password" for each
>> material you configure. That "password" would be a personal access token
>> 
>> with at least read-only access to the 1 or more repositories you want to
>> use.
>>
>> If you want to share one personal access token across many materials
>> (perhaps a single token has read-only access to many repositories), the
>> easiest way is to use a GoCD Secrets Management plugin and refer to them in
>> the username/"password" fields of each material using the special secrets
>> interpolation syntax:
>> https://docs.gocd.org/current/configuration/secrets_management.html This
>> will work with either manually defined pipelines/materials, or those
>> defined externally in source control.
>>
>> -Chad
>>
>> On Thu, Oct 26, 2023 at 3:01 PM Sriram Narayanan 
>> wrote:
>>
>>> Please see:
>>>
>>> https://docs.github.com/en/authentication/connecting-to-github-with-ssh
>>>
>>> The gocd server runs as a particular user account. That user account
>>> needs access to the ssh private keys used to authenticate with GitHub.
>>>
>>> The go agent too needs the same access.
>>>
>>> Assuming you are on Linux and installer gocd via rpm, then you would set
>>> this key in the home directory (
>>> /var/lib/go-server/.ssh/myprivatekey.id_rsa)
>>>
>>> Permissions for .ssh would be 600, and for the key would be 400, with
>>> the gocd process user owning the directory and The identity file.
>>>
>>> — Sriram
>>>
>>>
>>> On Thu, 26 Oct 2023 at 12:00 PM, vv-fork  wrote:
>>>
 Hello colleagues!

 What is the best way to connect on-prem goCD with GitHub private repo
 in cloud? I was smoking docs and manuals for quite a while, but what people
 say it’s to install ssh keys to both GitHub and goCD, which won’t work,
 since I am using github.com, so i suppose i can’t install ssh key
 there.

 I’ve installed github-oauth-authorization-plugin and set it as
 described (connection ok in authorisation configuration step), and
 restarted the server, however it’s still throwing that standard error
 “fatal: could not read Username for ‘https://github.com’ meaning that
 the access is still closed.

 What else can be done as you think?

 --
 You received this message because you are subscribed to the Google
 Groups "go-cd" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to go-cd+un...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/go-cd/ed3022b6-e1ec-4c3b-8ca3-3c

[go-cd] goCD Yaml plugin doesn't pick-up code

[vv-fork]

Greetings colleagues,

I am trying to get into Yaml plugin (*gocd-yaml-config-plugin.jar* v.
*0.14.3-321*) with goCD v.23.3.0. 

The plugin is able to receive the yaml file but always throws the same 
error. 

I am stuck with something pretty simple, like this:
format_version: 10

pipelines:
  mypipe:
group: mygroup
materials:
  mygit:
git: http://example.com/mygit.git
stages:
  - build:
  jobs:
build:
  tasks:
   - exec:
   command: make

environments:

The message is:
INVALID MERGED CONFIGURATION Number of errors: 1+ I. Rule Validation 
Errors: 1. Not allowed to refer to pipeline group 'mygroup'. Check the 
'Rules' of this config repository. II. Config Validation Errors: - For 
Config Repo: https://github.com/...p at revision 03a639fd6444>>>74d5

What am I doing wrong?

Vlad.

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/01c04e91-0322-45fa-9021-58dd90424b80n%40googlegroups.com.

Re: [go-cd] GoCD and private GitHub repo

[Sriram Narayanan]

On Fri, Oct 27, 2023 at 1:13 PM vv-fork  wrote:

> Thank you guys Sriram and Chad for answering those! Now it's getting
> clearer to me
>
> *I was able to connect using token. It's fine.*
>
> Though I* wasn't able* to connect using SSH Certificate.
> The key has been generated and installed:
> [image: unnamed.png]
>
> it seems i set proper permissions:
> [image: unnamed.png]
>
> but i still get that error message when i test connection from go-server:
> --- STANDARD ERR --- STDERR: fatal: could not read Username for '
> https://github.com': No such device or address ---
>
>
> So what am i doing wrong? May that be I have messed with permissions for *go
> user*?
>

Please change the key's owner to the user "go".



>
> Vlad.
>
> On Thursday, October 26, 2023 at 8:41:33 PM UTC+13 Chad Wilson wrote:
>
>> To add on to Sriram's comments, the use of the
>> github-oauth-authorization-plugin doesn't have any relationship with access
>> to repository content on GitHub - it simply allows people to log onto GoCD
>> using their Github identity, and optionally to have access to GoCD pipeline
>> groups mapped to GitHub roles.
>>
>> This is because materials/repositories need to be accessed in an identity
>> known to the GoCD server/agents, not necessarily the individual user who
>> happens to be logged in to GoCD. So even if you use that authorization
>> plugin, you still need to decide how to provide GoCD itself access to
>> repositories on Github.
>>
>> You can use an SSH key linked to a GitHub user
>> 
>> if you wish to use SSH access - no restrictions for private repos unless
>> your GitHub org blocks use of SSH keys. If you instead wish to use HTTPS
>> access to repositories you have to fill in a username/"password" for each
>> material you configure. That "password" would be a personal access token
>> 
>> with at least read-only access to the 1 or more repositories you want to
>> use.
>>
>> If you want to share one personal access token across many materials
>> (perhaps a single token has read-only access to many repositories), the
>> easiest way is to use a GoCD Secrets Management plugin and refer to them in
>> the username/"password" fields of each material using the special secrets
>> interpolation syntax:
>> https://docs.gocd.org/current/configuration/secrets_management.html This
>> will work with either manually defined pipelines/materials, or those
>> defined externally in source control.
>>
>> -Chad
>>
>> On Thu, Oct 26, 2023 at 3:01 PM Sriram Narayanan 
>> wrote:
>>
>>> Please see:
>>>
>>> https://docs.github.com/en/authentication/connecting-to-github-with-ssh
>>>
>>> The gocd server runs as a particular user account. That user account
>>> needs access to the ssh private keys used to authenticate with GitHub.
>>>
>>> The go agent too needs the same access.
>>>
>>> Assuming you are on Linux and installer gocd via rpm, then you would set
>>> this key in the home directory (
>>> /var/lib/go-server/.ssh/myprivatekey.id_rsa)
>>>
>>> Permissions for .ssh would be 600, and for the key would be 400, with
>>> the gocd process user owning the directory and The identity file.
>>>
>>> — Sriram
>>>
>>>
>>> On Thu, 26 Oct 2023 at 12:00 PM, vv-fork  wrote:
>>>
 Hello colleagues!

 What is the best way to connect on-prem goCD with GitHub private repo
 in cloud? I was smoking docs and manuals for quite a while, but what people
 say it’s to install ssh keys to both GitHub and goCD, which won’t work,
 since I am using github.com, so i suppose i can’t install ssh key
 there.

 I’ve installed github-oauth-authorization-plugin and set it as
 described (connection ok in authorisation configuration step), and
 restarted the server, however it’s still throwing that standard error
 “fatal: could not read Username for ‘https://github.com’ meaning that
 the access is still closed.

 What else can be done as you think?

 --
 You received this message because you are subscribed to the Google
 Groups "go-cd" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to go-cd+un...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/go-cd/ed3022b6-e1ec-4c3b-8ca3-3c5e6b7d72f4n%40googlegroups.com
 
 .

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+un...@googlegroups.com.
>>>
>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid

Re: [go-cd] GoCD and private GitHub repo

[vv-fork]

Thank you guys Sriram and Chad for answering those! Now it's getting 
clearer to me

*I was able to connect using token. It's fine.*

Though I* wasn't able* to connect using SSH Certificate. 
The key has been generated and installed:
[image: unnamed.png]

it seems i set proper permissions:
[image: unnamed.png]

but i still get that error message when i test connection from go-server:
--- STANDARD ERR --- STDERR: fatal: could not read Username for '
https://github.com': No such device or address ---


So what am i doing wrong? May that be I have messed with permissions for *go 
user*?


Vlad.

On Thursday, October 26, 2023 at 8:41:33 PM UTC+13 Chad Wilson wrote:

> To add on to Sriram's comments, the use of the 
> github-oauth-authorization-plugin doesn't have any relationship with access 
> to repository content on GitHub - it simply allows people to log onto GoCD 
> using their Github identity, and optionally to have access to GoCD pipeline 
> groups mapped to GitHub roles.
>
> This is because materials/repositories need to be accessed in an identity 
> known to the GoCD server/agents, not necessarily the individual user who 
> happens to be logged in to GoCD. So even if you use that authorization 
> plugin, you still need to decide how to provide GoCD itself access to 
> repositories on Github.
>
> You can use an SSH key linked to a GitHub user 
> 
>  
> if you wish to use SSH access - no restrictions for private repos unless 
> your GitHub org blocks use of SSH keys. If you instead wish to use HTTPS 
> access to repositories you have to fill in a username/"password" for each 
> material you configure. That "password" would be a personal access token 
> 
>  
> with at least read-only access to the 1 or more repositories you want to 
> use.
>
> If you want to share one personal access token across many materials 
> (perhaps a single token has read-only access to many repositories), the 
> easiest way is to use a GoCD Secrets Management plugin and refer to them in 
> the username/"password" fields of each material using the special secrets 
> interpolation syntax: 
> https://docs.gocd.org/current/configuration/secrets_management.html This 
> will work with either manually defined pipelines/materials, or those 
> defined externally in source control.
>
> -Chad
>
> On Thu, Oct 26, 2023 at 3:01 PM Sriram Narayanan  
> wrote:
>
>> Please see:
>>
>> https://docs.github.com/en/authentication/connecting-to-github-with-ssh
>>
>> The gocd server runs as a particular user account. That user account 
>> needs access to the ssh private keys used to authenticate with GitHub.
>>
>> The go agent too needs the same access.
>>
>> Assuming you are on Linux and installer gocd via rpm, then you would set 
>> this key in the home directory ( 
>> /var/lib/go-server/.ssh/myprivatekey.id_rsa)
>>
>> Permissions for .ssh would be 600, and for the key would be 400, with the 
>> gocd process user owning the directory and The identity file.
>>
>> — Sriram
>>
>>
>> On Thu, 26 Oct 2023 at 12:00 PM, vv-fork  wrote:
>>
>>> Hello colleagues!
>>>
>>> What is the best way to connect on-prem goCD with GitHub private repo in 
>>> cloud? I was smoking docs and manuals for quite a while, but what people 
>>> say it’s to install ssh keys to both GitHub and goCD, which won’t work, 
>>> since I am using github.com, so i suppose i can’t install ssh key there.
>>>
>>> I’ve installed github-oauth-authorization-plugin and set it as described 
>>> (connection ok in authorisation configuration step), and restarted the 
>>> server, however it’s still throwing that standard error “fatal: could not 
>>> read Username for ‘https://github.com’ meaning that the access is still 
>>> closed.
>>>
>>> What else can be done as you think?
>>>
>>> -- 
>>> You received this message because you are subscribed to the Google 
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send 
>>> an email to go-cd+un...@googlegroups.com.
>>> To view this discussion on the web visit 
>>> https://groups.google.com/d/msgid/go-cd/ed3022b6-e1ec-4c3b-8ca3-3c5e6b7d72f4n%40googlegroups.com
>>>  
>>> 
>>> .
>>>
>> -- 
>> You received this message because you are subscribed to the Google Groups 
>> "go-cd" group.
>> To unsubscribe from this group and stop receiving emails from it, send an 
>> email to go-cd+un...@googlegroups.com.
>>
> To view this discussion on the web visit 
>> https://groups.google.com/d/msgid/go-cd/CANiY96azM2%3DaFO351d4PpExOatRCO%2BoaQju3Juvm2yAbQR2d5A%40mail.gmail.com
>>  
>> 

Re: [go-cd] Kubernetes Elastic Agents - Pods can no longer be created

[Kim Pham]

Hi Chad,

I was just reading up on the changes for Kubernetes.  Looks 1.24 moves to
containerd runtime images and we are still using DIND for our elastic
agents.  That seems like it could be the culprit.  I'll do some testing and
change our elastic agent images.  Thanks for pointing that out.

On Thu, Oct 26, 2023 at 10:46 AM Chad Wilson  wrote:

> Unfortunately the error message is a bit mysterious and useless. Which
> agent image you are using? Anything special in the elastic agent pod spec
> that might no longer work as expected on Kubernetes 1.24? (e.g use of
> docker dind images)
>
> Does the pod get created (and fail) if you look at the events on the
> kubernetes side, or does it never get that far?
>
> -Chad
>
> On Thu, Oct 26, 2023 at 11:37 PM Chad Wilson 
> wrote:
>
>> Just curious - were the errors/stack traces on failure essentially
>> identical before and after you upgraded your gocd and elastic agent plugin
>> versions?
>>
>> On Thu, Oct 26, 2023 at 11:33 PM Kim Pham  wrote:
>>
>>> Hi Ashwanth,
>>>
>>> I checked the clusterrole of the service account it's using and it
>>> basically has full access atm.
>>>
>>> PolicyRule:
>>>   Resources   Non-Resource URLs  Resource Names  Verbs
>>>   -   -  --  -
>>>   events  [] []  [*]
>>>   namespaces  [] []  [*]
>>>   nodes   [] []  [*]
>>>   pods/log[] []  [*]
>>>   pods[] []  [*]
>>>
>>> On Thu, Oct 26, 2023 at 10:21 AM 'Ashwanth Kumar' via go-cd <
>>> go-cd@googlegroups.com> wrote:
>>>
 A wild guess, anything changed on the service account side or a custom
 role being added as part of the upgrade that is probably not allowing the
 gocd plugin to create the pod?

 Thanks,


 On Thu, 26 Oct 2023 at 20:27, Kim Pham  wrote:

> Hi All,
>
> We recently began to encounter issues where pods were unable to be
> created.  Nothing has changed in terms of GoCD server, agent, and
> Kubernetes elastic agent plugin versions.  However, we did notice that the
> cluster went through an automatic upgrade and updated gke version to
> 1.24.14.  GoCD is able to see the node pools through the 'Status Report'
> button.
>
> When attempting to create an agent on those node pools, I do see a 500
> in the plugin logs and gocd-server logs.  Attached are logs.
>
> I've tried updating GoCD and the plugins to latest release versions.
> Our static agents that are running on older gke versions aren't having any
> issues.
>
> Has anyone encountered this?
>
> Thanks in advance.
>
> --
> You received this message because you are subscribed to the Google
> Groups "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send
> an email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/a6b8e99d-f415-4c18-b67d-e86c3df16733n%40googlegroups.com
> 
> .
>


 --

 Ashwanth Kumar / ashwanthkumar.in

 --
 You received this message because you are subscribed to the Google
 Groups "go-cd" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to go-cd+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/go-cd/CAD9m7Cw2cK5yt_r9e5r1sxD%2B%2B%2Bjkd3%3DFTQe51vKqb081MunU%3DQ%40mail.gmail.com
 
 .

>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/CA%2BnJfx4-Zks6-FOr1bYSOroxG2o4e2e4ir0OFUm3TsggWsrYpA%40mail.gmail.com
>>> 
>>> .
>>>
>> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CAA1RwH-tw%3D0MFCgWRQoT8ZwFYcyJsGT8T5pPxrBzzu46Nn960w%40mail.gmail.com
> 

Re: [go-cd] Kubernetes Elastic Agents - Pods can no longer be created

[Chad Wilson]

Unfortunately the error message is a bit mysterious and useless. Which
agent image you are using? Anything special in the elastic agent pod spec
that might no longer work as expected on Kubernetes 1.24? (e.g use of
docker dind images)

Does the pod get created (and fail) if you look at the events on the
kubernetes side, or does it never get that far?

-Chad

On Thu, Oct 26, 2023 at 11:37 PM Chad Wilson  wrote:

> Just curious - were the errors/stack traces on failure essentially
> identical before and after you upgraded your gocd and elastic agent plugin
> versions?
>
> On Thu, Oct 26, 2023 at 11:33 PM Kim Pham  wrote:
>
>> Hi Ashwanth,
>>
>> I checked the clusterrole of the service account it's using and it
>> basically has full access atm.
>>
>> PolicyRule:
>>   Resources   Non-Resource URLs  Resource Names  Verbs
>>   -   -  --  -
>>   events  [] []  [*]
>>   namespaces  [] []  [*]
>>   nodes   [] []  [*]
>>   pods/log[] []  [*]
>>   pods[] []  [*]
>>
>> On Thu, Oct 26, 2023 at 10:21 AM 'Ashwanth Kumar' via go-cd <
>> go-cd@googlegroups.com> wrote:
>>
>>> A wild guess, anything changed on the service account side or a custom
>>> role being added as part of the upgrade that is probably not allowing the
>>> gocd plugin to create the pod?
>>>
>>> Thanks,
>>>
>>>
>>> On Thu, 26 Oct 2023 at 20:27, Kim Pham  wrote:
>>>
 Hi All,

 We recently began to encounter issues where pods were unable to be
 created.  Nothing has changed in terms of GoCD server, agent, and
 Kubernetes elastic agent plugin versions.  However, we did notice that the
 cluster went through an automatic upgrade and updated gke version to
 1.24.14.  GoCD is able to see the node pools through the 'Status Report'
 button.

 When attempting to create an agent on those node pools, I do see a 500
 in the plugin logs and gocd-server logs.  Attached are logs.

 I've tried updating GoCD and the plugins to latest release versions.
 Our static agents that are running on older gke versions aren't having any
 issues.

 Has anyone encountered this?

 Thanks in advance.

 --
 You received this message because you are subscribed to the Google
 Groups "go-cd" group.
 To unsubscribe from this group and stop receiving emails from it, send
 an email to go-cd+unsubscr...@googlegroups.com.
 To view this discussion on the web visit
 https://groups.google.com/d/msgid/go-cd/a6b8e99d-f415-4c18-b67d-e86c3df16733n%40googlegroups.com
 
 .

>>>
>>>
>>> --
>>>
>>> Ashwanth Kumar / ashwanthkumar.in
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/CAD9m7Cw2cK5yt_r9e5r1sxD%2B%2B%2Bjkd3%3DFTQe51vKqb081MunU%3DQ%40mail.gmail.com
>>> 
>>> .
>>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "go-cd" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/CA%2BnJfx4-Zks6-FOr1bYSOroxG2o4e2e4ir0OFUm3TsggWsrYpA%40mail.gmail.com
>> 
>> .
>>
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-tw%3D0MFCgWRQoT8ZwFYcyJsGT8T5pPxrBzzu46Nn960w%40mail.gmail.com.

Re: [go-cd] Kubernetes Elastic Agents - Pods can no longer be created

[Chad Wilson]

Just curious - were the errors/stack traces on failure essentially
identical before and after you upgraded your gocd and elastic agent plugin
versions?

On Thu, Oct 26, 2023 at 11:33 PM Kim Pham  wrote:

> Hi Ashwanth,
>
> I checked the clusterrole of the service account it's using and it
> basically has full access atm.
>
> PolicyRule:
>   Resources   Non-Resource URLs  Resource Names  Verbs
>   -   -  --  -
>   events  [] []  [*]
>   namespaces  [] []  [*]
>   nodes   [] []  [*]
>   pods/log[] []  [*]
>   pods[] []  [*]
>
> On Thu, Oct 26, 2023 at 10:21 AM 'Ashwanth Kumar' via go-cd <
> go-cd@googlegroups.com> wrote:
>
>> A wild guess, anything changed on the service account side or a custom
>> role being added as part of the upgrade that is probably not allowing the
>> gocd plugin to create the pod?
>>
>> Thanks,
>>
>>
>> On Thu, 26 Oct 2023 at 20:27, Kim Pham  wrote:
>>
>>> Hi All,
>>>
>>> We recently began to encounter issues where pods were unable to be
>>> created.  Nothing has changed in terms of GoCD server, agent, and
>>> Kubernetes elastic agent plugin versions.  However, we did notice that the
>>> cluster went through an automatic upgrade and updated gke version to
>>> 1.24.14.  GoCD is able to see the node pools through the 'Status Report'
>>> button.
>>>
>>> When attempting to create an agent on those node pools, I do see a 500
>>> in the plugin logs and gocd-server logs.  Attached are logs.
>>>
>>> I've tried updating GoCD and the plugins to latest release versions.
>>> Our static agents that are running on older gke versions aren't having any
>>> issues.
>>>
>>> Has anyone encountered this?
>>>
>>> Thanks in advance.
>>>
>>> --
>>> You received this message because you are subscribed to the Google
>>> Groups "go-cd" group.
>>> To unsubscribe from this group and stop receiving emails from it, send
>>> an email to go-cd+unsubscr...@googlegroups.com.
>>> To view this discussion on the web visit
>>> https://groups.google.com/d/msgid/go-cd/a6b8e99d-f415-4c18-b67d-e86c3df16733n%40googlegroups.com
>>> 
>>> .
>>>
>>
>>
>> --
>>
>> Ashwanth Kumar / ashwanthkumar.in
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "go-cd" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/CAD9m7Cw2cK5yt_r9e5r1sxD%2B%2B%2Bjkd3%3DFTQe51vKqb081MunU%3DQ%40mail.gmail.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CA%2BnJfx4-Zks6-FOr1bYSOroxG2o4e2e4ir0OFUm3TsggWsrYpA%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH9L%3DmBaJbvnXDfJ0d3J2_GuhC%2B38y4sMUiaMPDgAEd__Q%40mail.gmail.com.

Re: [go-cd] Kubernetes Elastic Agents - Pods can no longer be created

[Kim Pham]

Hi Ashwanth,

I checked the clusterrole of the service account it's using and it
basically has full access atm.

PolicyRule:
  Resources   Non-Resource URLs  Resource Names  Verbs
  -   -  --  -
  events  [] []  [*]
  namespaces  [] []  [*]
  nodes   [] []  [*]
  pods/log[] []  [*]
  pods[] []  [*]

On Thu, Oct 26, 2023 at 10:21 AM 'Ashwanth Kumar' via go-cd <
go-cd@googlegroups.com> wrote:

> A wild guess, anything changed on the service account side or a custom
> role being added as part of the upgrade that is probably not allowing the
> gocd plugin to create the pod?
>
> Thanks,
>
>
> On Thu, 26 Oct 2023 at 20:27, Kim Pham  wrote:
>
>> Hi All,
>>
>> We recently began to encounter issues where pods were unable to be
>> created.  Nothing has changed in terms of GoCD server, agent, and
>> Kubernetes elastic agent plugin versions.  However, we did notice that the
>> cluster went through an automatic upgrade and updated gke version to
>> 1.24.14.  GoCD is able to see the node pools through the 'Status Report'
>> button.
>>
>> When attempting to create an agent on those node pools, I do see a 500 in
>> the plugin logs and gocd-server logs.  Attached are logs.
>>
>> I've tried updating GoCD and the plugins to latest release versions.  Our
>> static agents that are running on older gke versions aren't having any
>> issues.
>>
>> Has anyone encountered this?
>>
>> Thanks in advance.
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "go-cd" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/a6b8e99d-f415-4c18-b67d-e86c3df16733n%40googlegroups.com
>> 
>> .
>>
>
>
> --
>
> Ashwanth Kumar / ashwanthkumar.in
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CAD9m7Cw2cK5yt_r9e5r1sxD%2B%2B%2Bjkd3%3DFTQe51vKqb081MunU%3DQ%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CA%2BnJfx4-Zks6-FOr1bYSOroxG2o4e2e4ir0OFUm3TsggWsrYpA%40mail.gmail.com.

Re: [go-cd] Kubernetes Elastic Agents - Pods can no longer be created

['Ashwanth Kumar' via go-cd]

A wild guess, anything changed on the service account side or a custom role
being added as part of the upgrade that is probably not allowing the gocd
plugin to create the pod?

Thanks,


On Thu, 26 Oct 2023 at 20:27, Kim Pham  wrote:

> Hi All,
>
> We recently began to encounter issues where pods were unable to be
> created.  Nothing has changed in terms of GoCD server, agent, and
> Kubernetes elastic agent plugin versions.  However, we did notice that the
> cluster went through an automatic upgrade and updated gke version to
> 1.24.14.  GoCD is able to see the node pools through the 'Status Report'
> button.
>
> When attempting to create an agent on those node pools, I do see a 500 in
> the plugin logs and gocd-server logs.  Attached are logs.
>
> I've tried updating GoCD and the plugins to latest release versions.  Our
> static agents that are running on older gke versions aren't having any
> issues.
>
> Has anyone encountered this?
>
> Thanks in advance.
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/a6b8e99d-f415-4c18-b67d-e86c3df16733n%40googlegroups.com
> 
> .
>


-- 

Ashwanth Kumar / ashwanthkumar.in

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAD9m7Cw2cK5yt_r9e5r1sxD%2B%2B%2Bjkd3%3DFTQe51vKqb081MunU%3DQ%40mail.gmail.com.

[go-cd] Kubernetes Elastic Agents - Pods can no longer be created

[Kim Pham]

Hi All,

We recently began to encounter issues where pods were unable to be 
created.  Nothing has changed in terms of GoCD server, agent, and 
Kubernetes elastic agent plugin versions.  However, we did notice that the 
cluster went through an automatic upgrade and updated gke version to 
1.24.14.  GoCD is able to see the node pools through the 'Status Report' 
button.  

When attempting to create an agent on those node pools, I do see a 500 in 
the plugin logs and gocd-server logs.  Attached are logs.  

I've tried updating GoCD and the plugins to latest release versions.  Our 
static agents that are running on older gke versions aren't having any 
issues.

Has anyone encountered this?

Thanks in advance.

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/a6b8e99d-f415-4c18-b67d-e86c3df16733n%40googlegroups.com.
java.lang.RuntimeException: The plugin sent a response that could not be 
understood by Go. Plugin returned with code '500' and the following response: 
'Failed to handle request cd.go.elastic-agent.create-agent'
at 
com.thoughtworks.go.plugin.access.PluginRequestHelper.submitRequest(PluginRequestHelper.java:58)
at 
com.thoughtworks.go.plugin.access.elastic.v5.ElasticAgentExtensionV5.createAgent(ElasticAgentExtensionV5.java:142)
at 
com.thoughtworks.go.plugin.access.elastic.ElasticAgentExtension.createAgent(ElasticAgentExtension.java:59)
at 
com.thoughtworks.go.plugin.access.elastic.ElasticAgentPluginRegistry.createAgent(ElasticAgentPluginRegistry.java:42)
at 
com.thoughtworks.go.server.messaging.elasticagents.CreateAgentListener.onMessage(CreateAgentListener.java:31)
at 
com.thoughtworks.go.server.messaging.elasticagents.CreateAgentListener.onMessage(CreateAgentListener.java:21)
at 
com.thoughtworks.go.server.messaging.activemq.JMSMessageListenerAdapter.runImpl(JMSMessageListenerAdapter.java:83)
at 
com.thoughtworks.go.server.messaging.activemq.JMSMessageListenerAdapter.run(JMSMessageListenerAdapter.java:63)
at java.base/java.lang.Thread.run(Unknown Source)2023-10-26 14:38:28,825 ERROR [237@MessageListener for CreateAgentListener] 
KubernetesPlugin:127 - Failed to handle request cd.go.elastic-agent.create-agent
io.fabric8.kubernetes.client.KubernetesClientException: An error has occurred.
at 
io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:129)
at 
io.fabric8.kubernetes.client.KubernetesClientException.launderThrowable(KubernetesClientException.java:122)
at 
io.fabric8.kubernetes.client.dsl.internal.CreateOnlyResourceOperation.create(CreateOnlyResourceOperation.java:44)
at 
io.fabric8.kubernetes.client.dsl.internal.BaseOperation.create(BaseOperation.java:1142)
at 
io.fabric8.kubernetes.client.dsl.internal.BaseOperation.create(BaseOperation.java:97)
at 
cd.go.contrib.elasticagent.KubernetesInstanceFactory.createKubernetesPod(KubernetesInstanceFactory.java:151)
at 
cd.go.contrib.elasticagent.KubernetesInstanceFactory.createUsingPodYaml(KubernetesInstanceFactory.java:246)
at 
cd.go.contrib.elasticagent.KubernetesInstanceFactory.create(KubernetesInstanceFactory.java:59)
at 
cd.go.contrib.elasticagent.KubernetesAgentInstances.createKubernetesInstance(KubernetesAgentInstances.java:91)
at 
cd.go.contrib.elasticagent.KubernetesAgentInstances.create(KubernetesAgentInstances.java:65)
at 
cd.go.contrib.elasticagent.KubernetesAgentInstances.create(KubernetesAgentInstances.java:36)
at 
cd.go.contrib.elasticagent.executors.CreateAgentRequestExecutor.execute(CreateAgentRequestExecutor.java:55)
at 
cd.go.contrib.elasticagent.KubernetesPlugin.handle(KubernetesPlugin.java:82)
at 
com.thoughtworks.go.plugin.infra.DefaultPluginManager.lambda$submitTo$0(DefaultPluginManager.java:134)
at 
com.thoughtworks.go.plugin.infra.FelixGoPluginOSGiFramework.executeActionOnTheService(FelixGoPluginOSGiFramework.java:205)
at 
com.thoughtworks.go.plugin.infra.FelixGoPluginOSGiFramework.doOn(FelixGoPluginOSGiFramework.java:164)
at 
com.thoughtworks.go.plugin.infra.DefaultPluginManager.submitTo(DefaultPluginManager.java:131)
at 
com.thoughtworks.go.plugin.access.PluginRequestHelper.submitRequest(PluginRequestHelper.java:49)
at 
com.thoughtworks.go.plugin.access.elastic.v5.ElasticAgentExtensionV5.createAgent(ElasticAgentExtensionV5.java:142)
at 
com.thoughtworks.go.plugin.access.elastic.ElasticAgentExtension.createAgent(ElasticAgentExtension.java:59)
at 
com.thoughtworks.go.plugin.access.elastic.ElasticAgentPluginRegistry.createAgent(ElasticAgentPluginRegistry.java:42)
at 
com.thoughtworks.go.ser

Re: [go-cd] GoCD and private GitHub repo

[Chad Wilson]

To add on to Sriram's comments, the use of the
github-oauth-authorization-plugin doesn't have any relationship with access
to repository content on GitHub - it simply allows people to log onto GoCD
using their Github identity, and optionally to have access to GoCD pipeline
groups mapped to GitHub roles.

This is because materials/repositories need to be accessed in an identity
known to the GoCD server/agents, not necessarily the individual user who
happens to be logged in to GoCD. So even if you use that authorization
plugin, you still need to decide how to provide GoCD itself access to
repositories on Github.

You can use an SSH key linked to a GitHub user

if you wish to use SSH access - no restrictions for private repos unless
your GitHub org blocks use of SSH keys. If you instead wish to use HTTPS
access to repositories you have to fill in a username/"password" for each
material you configure. That "password" would be a personal access token

with at least read-only access to the 1 or more repositories you want to
use.

If you want to share one personal access token across many materials
(perhaps a single token has read-only access to many repositories), the
easiest way is to use a GoCD Secrets Management plugin and refer to them in
the username/"password" fields of each material using the special secrets
interpolation syntax:
https://docs.gocd.org/current/configuration/secrets_management.html This
will work with either manually defined pipelines/materials, or those
defined externally in source control.

-Chad

On Thu, Oct 26, 2023 at 3:01 PM Sriram Narayanan 
wrote:

> Please see:
>
> https://docs.github.com/en/authentication/connecting-to-github-with-ssh
>
> The gocd server runs as a particular user account. That user account needs
> access to the ssh private keys used to authenticate with GitHub.
>
> The go agent too needs the same access.
>
> Assuming you are on Linux and installer gocd via rpm, then you would set
> this key in the home directory (
> /var/lib/go-server/.ssh/myprivatekey.id_rsa)
>
> Permissions for .ssh would be 600, and for the key would be 400, with the
> gocd process user owning the directory and The identity file.
>
> — Sriram
>
>
> On Thu, 26 Oct 2023 at 12:00 PM, vv-fork  wrote:
>
>> Hello colleagues!
>>
>> What is the best way to connect on-prem goCD with GitHub private repo in
>> cloud? I was smoking docs and manuals for quite a while, but what people
>> say it’s to install ssh keys to both GitHub and goCD, which won’t work,
>> since I am using github.com, so i suppose i can’t install ssh key there.
>>
>> I’ve installed github-oauth-authorization-plugin and set it as described
>> (connection ok in authorisation configuration step), and restarted the
>> server, however it’s still throwing that standard error “fatal: could not
>> read Username for ‘https://github.com’ meaning that the access is still
>> closed.
>>
>> What else can be done as you think?
>>
>> --
>> You received this message because you are subscribed to the Google Groups
>> "go-cd" group.
>> To unsubscribe from this group and stop receiving emails from it, send an
>> email to go-cd+unsubscr...@googlegroups.com.
>> To view this discussion on the web visit
>> https://groups.google.com/d/msgid/go-cd/ed3022b6-e1ec-4c3b-8ca3-3c5e6b7d72f4n%40googlegroups.com
>> 
>> .
>>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/CANiY96azM2%3DaFO351d4PpExOatRCO%2BoaQju3Juvm2yAbQR2d5A%40mail.gmail.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CAA1RwH-R9v39GDB_Rs98pXnh0x7xyMZKwanye7Mcq%3D7hSdA1tQ%40mail.gmail.com.

Re: [go-cd] GoCD and private GitHub repo

[Sriram Narayanan]

Please see:

https://docs.github.com/en/authentication/connecting-to-github-with-ssh

The gocd server runs as a particular user account. That user account needs
access to the ssh private keys used to authenticate with GitHub.

The go agent too needs the same access.

Assuming you are on Linux and installer gocd via rpm, then you would set
this key in the home directory (
/var/lib/go-server/.ssh/myprivatekey.id_rsa)

Permissions for .ssh would be 600, and for the key would be 400, with the
gocd process user owning the directory and The identity file.

— Sriram


On Thu, 26 Oct 2023 at 12:00 PM, vv-fork  wrote:

> Hello colleagues!
>
> What is the best way to connect on-prem goCD with GitHub private repo in
> cloud? I was smoking docs and manuals for quite a while, but what people
> say it’s to install ssh keys to both GitHub and goCD, which won’t work,
> since I am using github.com, so i suppose i can’t install ssh key there.
>
> I’ve installed github-oauth-authorization-plugin and set it as described
> (connection ok in authorisation configuration step), and restarted the
> server, however it’s still throwing that standard error “fatal: could not
> read Username for ‘https://github.com’ meaning that the access is still
> closed.
>
> What else can be done as you think?
>
> --
> You received this message because you are subscribed to the Google Groups
> "go-cd" group.
> To unsubscribe from this group and stop receiving emails from it, send an
> email to go-cd+unsubscr...@googlegroups.com.
> To view this discussion on the web visit
> https://groups.google.com/d/msgid/go-cd/ed3022b6-e1ec-4c3b-8ca3-3c5e6b7d72f4n%40googlegroups.com
> 
> .
>

-- 
You received this message because you are subscribed to the Google Groups 
"go-cd" group.
To unsubscribe from this group and stop receiving emails from it, send an email 
to go-cd+unsubscr...@googlegroups.com.
To view this discussion on the web visit 
https://groups.google.com/d/msgid/go-cd/CANiY96azM2%3DaFO351d4PpExOatRCO%2BoaQju3Juvm2yAbQR2d5A%40mail.gmail.com.