Hi Pierre, Why I need to keep the logs long term is:
1. It is a goal of the project long term I am working on 2. If the logHorizon ticket# 3572 <http://trac.buildbot.net/ticket/3572>was fixed I would not need to archive the logs Regards, Alvaro On Tue, Mar 28, 2017 at 4:00 AM, Pierre Tardy <[email protected]> wrote: > Hi Mike, > > Thanks for sharing this code. I think it is a very interesting approach. > > All, > I'd like to better understand the reasons why you want to archive those > build logs. > What do you want to do with them in the long term? > Why having them accessible from the buildbot UI/rest api is not enough? > Do we need a build report archiver reporter included in buildbot? > Pierre > > On Tue, Mar 28, 2017 at 12:49 PM Mike Alexeev <[email protected]> > wrote: > >> Hello Pierre, >> >> Thank you for advice. >> I will use reporter class instead of Build class customization. >> >> Alvaro, anyway here is an example of subclassing Build: >> https://pastebin.com/gfq1wQLG >> >> Maybe it will be helpful in other cases >> >> 2017-03-27 21:49 GMT+03:00 Pierre Tardy <[email protected]>: >> >> Hi all, >> Usually, the way to do that is to setup a custom reporter class. >> You can then use the data api to fetch as many information on the build >> as you need to build your html report, and push it to S3. >> >> The details of what you can get with rest api is documented here: >> https://docs.buildbot.net/0.9.5/developer/rest.html >> >> The best is to look at the example of reporters/utils https:// >> github.com/buildbot/buildbot/blob/master/master/buildbot/ >> reporters/utils.py >> >> If you are not confident enough with twisted and async, you can just put >> a MasterShellCommand with a python script in the end of your builds and use >> the REST api to fetch the data you need. >> In order to get raw logs, you can use the raw log endpoint (no need for >> logobservers!): >> >> e.g: https://nine.buildbot.net/api/v2/logs/357614/raw >> >> Note that buildbot store the stdout/stderr information, which is removed >> when you fetch it via raw endpoint. >> >> Here is an example on how to process the stream logs that we use for >> debugging our integration tests: >> https://github.com/buildbot/buildbot/blob/master/master/ >> buildbot/test/util/integration.py#L246 >> >> Regards >> Pierre >> >> >> Le lun. 27 mars 2017 à 18:41, Alvaro Erickson <[email protected]> >> a écrit : >> >> Hi Mike, >> >> I too would like to see what your solution for saving a log (into plain >> text, not html) for all steps after a build is finished. Specifically, if >> you could please share the subclassed Build class for log generation. Any >> examples would be very helpful. >> >> Background: On my work project, one of the goals is they want all the >> build logs to be archived. >> >> Regards, >> >> Alvaro >> >> On Mon, Mar 27, 2017 at 8:52 AM, Mike Alexeev <[email protected]> >> wrote: >> >> Hello, Koutheir >> >> >> I have also similar problem. >> I need to save log from all steps after build finished. >> No html just plain text. >> >> It seems that there is no ready solution. >> For example there is no file storage in buildbot for files (reports, logs >> and other build artifacts). >> So my solution is not universal though it can be helpful. >> >> >> For log generation I subclassed Build class. You can set your own custom >> Build class >> to build factory in buildbot config. >> >> I overrided buildFinished method and added there code to collect log from >> all steps. >> In build instance you have access to Data API and you can read logs from >> DB. >> I can share example if you want. >> >> For file serving I use nginx. >> I added links to log in build page template and substitute path to log >> based on build properties. >> >> So every build has link on its page to log. >> >> I think that saving log can also be implemented as step running on Local >> Worker >> (it runs on master). But I used custom build class because it looks >> cleaner (no odd steps in build). >> >> >> By the way, can be there default implementation for file storage in >> buildbot? >> For example default solution with twisted file serving >> and possibility to use server like nginx in bigger setups. Just like db >> setup - sqlite has no external >> processes and easy to setup, but also you can use postgre which much >> more faster. >> >> >> What about adding post-build hook in configuration? Log-saving method for >> example could be >> set there >> >> >> >> _______________________________________________ >> users mailing list >> [email protected] >> https://lists.buildbot.net/mailman/listinfo/users >> >> >> _______________________________________________ >> users mailing list >> [email protected] >> https://lists.buildbot.net/mailman/listinfo/users >> >> >>
_______________________________________________ users mailing list [email protected] https://lists.buildbot.net/mailman/listinfo/users
