Hi there, I think the best way to do this in GoCD is using build artifacts, pipeline materials and the Fetch Artifact task.
The build pipeline should publish the artifact to Artifactory and generate a GoCD build artifact that references the Artifactory location. The QA pipeline should have the build pipeline as a material and should fetch the GoCD artifact using the Fetch Artifact task with a Pipeline value of "build". The Prod pipeline should have the QA pipeline as a material and should fetch the GoCD artifact using the Fetch Artifact task with a Pipeline value of "build/QA". This ensures that it is impossible to deploy an artifact to production without first successfully deploying it to QA. Hope this helps, Jason On Wednesday, 14 December 2022 at 13:39:56 UTC-5 ma...@thoughtworks.com wrote: > You could upload it from every stage that passes, but I don't see a > benefit to doing that -- and as you said it's duplicative. What I'm > suggesting is that every downstream stage/pipeline can use the build stage > as a material so you can fetch artifacts from it. > > Something like: > > build-and-test (generates artifact) --> deploy QA --> validate QA --> > promote-to-prod-manual-gate --> deploy-prod > > Every stage that needs to use the built artifact can include > build-and-test as a material so that you can do a Fetch Artifact task. I > believe you had a concern/question that it meant stages downstream would > have access to an artifact that possibly didn't pass validations? I don't > think that's an issue because you would have more than 1 material. there's > an implicit dependency on the prior stage succeeding by default. For > instance, "deploy-prod" would only run if all the prior stages succeeded, I > believe, so it wouldn't actually deploy without going through all of your > validation stages (deploy-QA, validate-QA, promote-to-prod-manual-gate). > > So in other words, deploy-prod would need to depend on both > promote-to-prod-manual-gate and build-and-test. > > If you can't get it working, let me dust off my GoCD hat and look at it > tonight after working hours. > > Best, > > Marques > > On Wed, Dec 14, 2022 at 10:26 AM Anh Le <leth...@gmail.com> wrote: > >> 1. So you are suggesting there is no way to avoid temporarily duplicating >> the artifact between pipelines following my use case right? I'm still new >> to GoCD and haven't play much with plugins >> >> Not sure if I get your idea correctly, does that mean if I model my >> pipelines like this: >> build pipeline - deploy to QA pipeline - mark passed QA Gate pipeline - >> deploy to Production pipeline >> If I have a perfect artifact, I will need to Publishing artifacts and *Fetch >> Artifacts* for *each pipeline* >> >> Since storage is cheap, I'm perfectly fine with this approach. But having >> to upload the same file every time could sound not optimal >> >> 2. Is it possible to avoid that duplication? (but it could be >> over-engineer, I'm not sure, haven't spent much time doing this after >> reading the Continuous Delivery book) >> >> I'm thinking is there any vendor/tool/plugins that does artifact >> management strategy and could be integrated well with GoCD OOTB: >> - An artifact (it is a file generated from the build step anyway) should >> have some states: >> + Deployed to QA >> + Passed QA Gate >> Mark passed QA Gate pipeline and Deploy to Production >> pipeline should fetch artifacts based on that state from a central >> Artifactory >> On Wednesday, 14 December 2022 at 22:54:46 UTC+7 ma...@thoughtworks.com >> wrote: >> >>> It’s been a few years since I really looked at GoCD, so my memory is >>> hazy on this. >>> >>> You should be able to store artifacts on the GoCD server itself by >>> configuring a build artifact path. Then downstream stages or pipelines can >>> grab them from gocd server directly. There’s an S3 plugin as well if you >>> wanted to persist them in AWS. Then you can upload to artifactory when >>> you’ve had an artifact pass certain stages for confidence. >>> >>> You should be able to pass artifacts through stages so that a downstream >>> stage can fetch an artifact that passed the previous stage. If a stage >>> depends on the previous stage passing, it cannot run until those conditions >>> are met, so they won’t run with artifacts that didn’t succeed in upstream >>> stages. >>> >>> Once it’s passed production, you can then publish it to artifactory such >>> that artifactory only has production quality packages. >>> >>> Does that help? >>> >>> -Marques >>> >>> On Wed, Dec 14, 2022 at 7:33 AM Anh Le <leth...@gmail.com> wrote: >>> >>>> Hello everyone, >>>> >>>> I'm learning GoCD, I only know how to use the Files and Directories out >>>> of the box as Artifactory. >>>> I have few question about artifacts management best practices or your >>>> preferred approach >>>> Assumption: >>>> - Already has a build pipeline (to generate artifact) >>>> - Deploy to QA is manual deploy and depend on build pipeline success >>>> - Deploy to Prod is manual deploy and depend on Deploy to QA pipeline >>>> success >>>> I'm thinking about things below but something doesn't feel right: >>>> - In the build pipeline, after run build tasks, I push the artifact to >>>> an Artifactory >>>> + How to fetch artifact for "Deploy to QA" and "Deploy to >>>> Production"? if I add build as material (so I can get the artifact) >>>> somehow >>>> I feel it is weird because Production can access to something not already >>>> in QA. If I publish artifact again in QA to break the dependency of Prod >>>> to >>>> build then I feel that the artifact is duplicated >>>> >>>> What are your suggestions? >>>> >>>> -- >>>> You received this message because you are subscribed to the Google >>>> Groups "go-cd" group. >>>> To unsubscribe from this group and stop receiving emails from it, send >>>> an email to go-cd+un...@googlegroups.com. >>>> To view this discussion on the web visit >>>> https://groups.google.com/d/msgid/go-cd/d7155810-ffd1-4ecd-822a-0bddfb7ae419n%40googlegroups.com >>>> >>>> <https://groups.google.com/d/msgid/go-cd/d7155810-ffd1-4ecd-822a-0bddfb7ae419n%40googlegroups.com?utm_medium=email&utm_source=footer> >>>> . >>>> >>> -- >> You received this message because you are subscribed to the Google Groups >> "go-cd" group. >> To unsubscribe from this group and stop receiving emails from it, send an >> email to go-cd+un...@googlegroups.com. >> > To view this discussion on the web visit >> https://groups.google.com/d/msgid/go-cd/042f88ea-2e9e-4433-a2a8-af77cbbb294cn%40googlegroups.com >> >> <https://groups.google.com/d/msgid/go-cd/042f88ea-2e9e-4433-a2a8-af77cbbb294cn%40googlegroups.com?utm_medium=email&utm_source=footer> >> . >> > -- You received this message because you are subscribed to the Google Groups "go-cd" group. To unsubscribe from this group and stop receiving emails from it, send an email to go-cd+unsubscr...@googlegroups.com. To view this discussion on the web visit https://groups.google.com/d/msgid/go-cd/fe9f74dc-ae0e-439e-ac16-54fecc7f74b9n%40googlegroups.com.
