Re: [go-cd] Found Database issue

2024-09-12 Thread Sriram Narayanan
On Thu, Sep 12, 2024 at 6:33 PM Chad Wilson wrote: > The warnings on materials not matching and upstream pipelines may be able > to be ignored. Not relevant to this. Similar with the maximum backtracking > limit (unrelated problem to this issue). > > But > >1. is the GoCD server process being

Re: [go-cd] Found Database issue

2024-09-12 Thread Chad Wilson
The warnings on materials not matching and upstream pipelines may be able to be ignored. Not relevant to this. Similar with the maximum backtracking limit (unrelated problem to this issue). But 1. is the GoCD server process being restarted after these "The database has been closed" errors?

Re: [go-cd] Found Database issue

2024-09-12 Thread Komgrit Aneksri
Thank you Chad, I dig into GoCD Server logs before DB locked. *I always found many ERROR messages below.* 2024-09-10 08:14:09,413 WARN [118@MessageListener for ScheduleCheckListener] BuildCauseProducerService:175 - Error while scheduling pipeline: ar-eod-service-deploy-prod. Possible Reasons:

Re: [go-cd] Found Database issue

2024-09-11 Thread Chad Wilson
Before we get into memory stats, did you look at the logs to see if the server is being restarted internally as I described below? No point looking at memory stats unless we have evidence there is a memory problem. You generally cannot use OS-level stats to debug memory usage for Java applications

Re: [go-cd] Found Database issue

2024-09-10 Thread Komgrit Aneksri
Thank you Chad for help me to investigate and suggest. Here are more information about my GoCD server resources below. We are using worker node as c6g.2xlarge. Currently, CPU usage is 40 - 45 % After restarted GoCD has memory free around 3GB, Then GoCD server run After restart bash-5.1$ free

Re: [go-cd] Found Database issue

2024-09-10 Thread Chad Wilson
If this has never happened before, and only just started happening, then *something* must have changed. Might be worth figuring that out. A database becomes locked like this only when two instances are trying to connect to the same H2 database file, or one crashed somehow without releasing the loc

Re: [go-cd] Found Database issue

2024-09-10 Thread Komgrit Aneksri
I have no any change in configurations. But We have add new users and new pipelines every day. Pods is still running status and no restart/spawn/evicted. If we have to migrate h2 to postgresql. Do you have any migration documentations for K8s? Regards, Komgrit On Tuesday, September 10, 2024 at

Re: [go-cd] Found Database issue

2024-09-10 Thread Chad Wilson
What changed in your setup when this started happening? Is your GoCD server pod crashing and being automatically restarted? Are nodes it is running on dying and the pod being re-scheduled elsewhere? On Tue, 10 Sept 2024, 17:02 Komgrit Aneksri, wrote: > Hi team, > > I am facing issue about datab

Re: [go-cd] Found Database issue

2024-09-10 Thread Sriram Narayanan
On Tue, 10 Sep 2024 at 5:02 PM, Komgrit Aneksri wrote: > Hi team, > > I am facing issue about database. > > Error message below > Could not open JDBC Connection for transaction; nested exception is > org.h2.jdbc.JdbcSQLNonTransientConnectionException: Database may be already > in use: null. Possi

[go-cd] Found Database issue

2024-09-10 Thread Komgrit Aneksri
Hi team, I am facing issue about database. Error message below Could not open JDBC Connection for transaction; nested exception is org.h2.jdbc.JdbcSQLNonTransientConnectionException: Database may be already in use: null. Possible solutions: close all other connection(s); use the server mode [9