There could also be other opportunities for improvement such as: - moving from individual pipelines to templates - organising tasks ( what is a job in Jenkins) into jobs that can run in parallel in a stage, and then sequencing stages that must run together into a pipeline.
And of course, Chad has reminded below that instead of writing the commands as part of the pipeline, it is better to move those commands into a version controlled scripts that is checked out as a material for the pipeline with each script being appropriately referenced as a task in a job. — Sriram On Thu, 10 Mar 2022 at 10:17 PM, Chad Wilson <[email protected]> wrote: > OK, good to hear you have a way forward. > > I am curious as to whether the problem would have persisted with that > large a config file on one of the more recent versions, as I believe there > have been a number of JGit upgrades since then. Maybe we'll never know? :-) > > I also wonder a bit how your config got to 50MB and what is in it. With > 1500 pipelines, that seems to imply ~34kB per pipeline which seems a lot. > Perhaps you might want to look into the way pipelines are being modelled > and what the XML is composed of? > > Sometimes that can imply that perhaps folks are modelling pipelines using > many small stages or a large number of GoCD-modelled tasks for every step > in a job; rather than perhaps source controlling the build steps alongside > the actual source it's building using simple scripts or using build > automation tooling to assist - which can other advantages besides reducing > config overhead for GoCD. > > -Chad > > On Thu, Mar 10, 2022 at 8:12 PM Hugo Ferreira <[email protected]> > wrote: > >> Hi Chad, >> >> We were running the 19.10 version of GoCD. >> >> I checked your link and setting bigFileThreshold = 100 MiB on >> .git/config core section seems to have fixed it. >> >> In the meantime, we took this opportunity and split this Go Server to 2 >> new instances on the 21.2.0 version and haven't had any major issues since >> then. >> >> Cheers, >> >> A domingo, 6 de março de 2022 à(s) 08:57:38 UTC, Chad Wilson escreveu: >> >>> Hi Hugo >>> >>> Can you please report the specific GoCD version you are using and the >>> full stack trace of the error from the server logs? >>> >>> It's probably necessary to know the specific operation that failed and >>> the version you are using to know what limits might be applying. >>> >>> It may also help to know what git object type that is with git cat-file >>> -t 2506f3dcaf16646bf525968cd53bd023872d72c0 (will have to run inside >>> the config folder on your GoCD server). >>> >>> I believe 50MB might some kind of default limit where Git and JGit start >>> to treat some files differently, but not sure about which limit you are >>> hitting, nor the consequence of changing them (some may need to be adjusted >>> together, and will have other impacts on memory usage): >>> https://gerrit.googlesource.com/jgit/+/refs/heads/master/Documentation/config-options.md >>> >>> There are a few other similar issues reported against tools that use >>> JGit, but I didn't come to any specific conclusions about what caused them, >>> so probably need more specific details here. >>> >>> -Chad >>> >>> On Fri, Mar 4, 2022 at 2:23 AM Hugo Ferreira <[email protected]> wrote: >>> >>>> Hi all, >>>> >>>> Our GoCD stopped working with this error: >>>> >>>> org.eclipse.jgit.errors.LargeObjectException: >>>> 2506f3dcaf16646bf525968cd53bd023872d72c0 exceeds size limit >>>> >>>> We are running around 1500 pipelines and cruiser-config.xml has around >>>> 50mb. >>>> >>>> Any way we can bump this limit or workaround this issue? >>>> >>>> Cheers, >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "go-cd" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to [email protected]. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/go-cd/2857c3a4-ead0-40f2-b16c-d792fd57cf88n%40googlegroups.com >>>> <https://groups.google.com/d/msgid/go-cd/2857c3a4-ead0-40f2-b16c-d792fd57cf88n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "go-cd" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to [email protected]. >> To view this discussion on the web visit >> https://groups.google.com/d/msgid/go-cd/0b413d9d-e78b-4d32-a0f3-8f851b9658f0n%40googlegroups.com >> <https://groups.google.com/d/msgid/go-cd/0b413d9d-e78b-4d32-a0f3-8f851b9658f0n%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- > You received this message because you are subscribed to the Google Groups > "go-cd" group. > To unsubscribe from this group and stop receiving emails from it, send an > email to [email protected]. > To view this discussion on the web visit > https://groups.google.com/d/msgid/go-cd/CAA1RwH-%3D%3D-b%2Ba%3DvhMEQhOTLuSq%2B2mojLv0Qf-TgJrdct6QG0OQ%40mail.gmail.com > <https://groups.google.com/d/msgid/go-cd/CAA1RwH-%3D%3D-b%2Ba%3DvhMEQhOTLuSq%2B2mojLv0Qf-TgJrdct6QG0OQ%40mail.gmail.com?utm_medium=email&utm_source=footer> > . > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/CANiY96ZW_gB4pza8xhsHagNsr0gj-45q0%3DaPh41i_d-wece1Gg%40mail.gmail.com.
