Re: [Yade-dev] gitlab artifacts missing

2019-11-14 Thread Bruno Chareyre 
On Wed, 13 Nov 2019 at 20:33, Anton Gladky  wrote:

>
> could you please send me a script, which is doing the download/upload?
>
>
Please forgive in advance my naive bash crap and see below.

*cd /tmp*
*touch etag_last*






*wget --server-response --spider
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages

2>&1 | grep ETag > etag_remoteif ! cmp -s etag_remote etag_last; then
wget
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages

  unzip download?job=pagesrm -rf /var/www/doc/*mv /tmp/public
/var/www/docrm download?job=pages*
*mv etag_remote etag_last*
*fi*



> Actually, when I do
>
> "wget
> https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
> "
>
> I get right now the error code 8.
>

As I type this email that target works again (log at the end of this
email), yet for some reason (it occured at least twice in the last months,
probably only for a couple hours the first time).



> Actually, if you use the symbol &&, it will not execute the next step,
> if the previous one failed.
>
> Something like:
>
> wget
> https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
> &&
> 
> rm -rf /var/www/yade-dem.org && cp * /var/www/yade-dem.org



Sounds like the good fix, thanks for mentioning the syntax.
I think we will try that.
The question of why this happens remains but that's perfectly fine to
workaround it.
Cheers
Bruno




$ wget
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
--2019-11-13 17:55:13--
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
Resolving gitlab.com (gitlab.com)... 35.231.145.151
Connecting to gitlab.com (gitlab.com)|35.231.145.151|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2019-11-13 17:55:14 ERROR 404: Not Found.

$ wget
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
--2019-11-14 17:09:32--
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
Resolving gitlab.com (gitlab.com)... 35.231.145.151
Connecting to gitlab.com (gitlab.com)|35.231.145.151|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location:
https://gitlab.com/yade-dev/trunk/-/jobs/349059349/artifacts/download
[following]
--2019-11-14 17:09:34--
https://gitlab.com/yade-dev/trunk/-/jobs/349059349/artifacts/download
Reusing existing connection to gitlab.com:443.
HTTP request sent, awaiting response... 302 Found
Location:
https://storage.googleapis.com/gitlab-gprd-artifacts/5c/74/5c748985bd13444e59322d1dd55fa2a5091e45a49c1db75d94366184ad2a7fe9/2019_11_13/349059349/361212541/artifacts.zip?response-content-disposition=attachment%3B%20filename%3D%22artifacts.zip%22%3B%20filename%2A%3DUTF-8%27%27artifacts.zip=application%2Fzip=gitlab-object-storage-...@gitlab-production.iam.gserviceaccount.com=JpHFPisbZHmwBGSHy%2Bu%2BDN0Qyi8TQN%2F2J5DbGjSJiuKKO8T8XzyrxjfbEncC%0AgNSHmubYDcLB%2FScSvvS8m%2BJt%2F%2BHkerujryO%2B4FQMAzcnxg2ptUTE2hqR3F43%0AuJk6zo5OsJ8IFvzzfH43VRvhPXEiC2NwbC0FB7jIqvw%2FNevlghADoRxvG7BK%0A0m9yaeoywDilR%2FDu%2BB6qUegllIyp8gvc5XrjyDortOuSUM7wegpRGQ5vMBS1%0Arxn%2F1AvTyeUDaSkbyMeLmx9XxHBJsFdlBAPHdQ1nVgh1GmGU7uoIku7895nd%0AhxoDIfe4ANq5Qdj6O7LgKhr%2FHugAgkyPJ2B8VDf5ZQ%3D%3D=1573748375
[following]
--2019-11-14 17:09:35--
https://storage.googleapis.com/gitlab-gprd-artifacts/5c/74/5c748985bd13444e59322d1dd55fa2a5091e45a49c1db75d94366184ad2a7fe9/2019_11_13/349059349/361212541/artifacts.zip?response-content-disposition=attachment%3B%20filename%3D%22artifacts.zip%22%3B%20filename%2A%3DUTF-8%27%27artifacts.zip=application%2Fzip=gitlab-object-storage-...@gitlab-production.iam.gserviceaccount.com=JpHFPisbZHmwBGSHy%2Bu%2BDN0Qyi8TQN%2F2J5DbGjSJiuKKO8T8XzyrxjfbEncC%0AgNSHmubYDcLB%2FScSvvS8m%2BJt%2F%2BHkerujryO%2B4FQMAzcnxg2ptUTE2hqR3F43%0AuJk6zo5OsJ8IFvzzfH43VRvhPXEiC2NwbC0FB7jIqvw%2FNevlghADoRxvG7BK%0A0m9yaeoywDilR%2FDu%2BB6qUegllIyp8gvc5XrjyDortOuSUM7wegpRGQ5vMBS1%0Arxn%2F1AvTyeUDaSkbyMeLmx9XxHBJsFdlBAPHdQ1nVgh1GmGU7uoIku7895nd%0AhxoDIfe4ANq5Qdj6O7LgKhr%2FHugAgkyPJ2B8VDf5ZQ%3D%3D=1573748375
Resolving storage.googleapis.com (storage.googleapis.com)...
216.58.206.240, 2a00:1450:4007:817::2010
Connecting to storage.googleapis.com
(storage.googleapis.com)|216.58.206.240|:443...
connected.
HTTP request sent, awaiting response... 200 OK
Length: 18604798 (18M) [application/zip]
Saving to: ‘download?job=pages’

download?job=pages
100%[=>]  17.74M  23.0MB/s
 in 0.8s

>
> Am Mi., 13. Nov. 2019 um 18:05 Uhr schrieb Bruno Chareyre
> :
>
> >
> > Hi there,
> > yade-dem.org is currently down while yade-dev.gitlab.io/trunk/ is not.
> >
> > The reason is that yade server is downloading the gitlab artifacts

Re: [Yade-dev] gitlab artifacts missing

2019-11-14 Thread Janek Kozicki (yade) 
A very simple explanation of how && works: do this command:

ls && ls /wrongdir && ls

Then do this command:

ls && ls && ls

You will see that in the first case the failure of 'ls /wrongdir'
prevented execution of last 'ls'. That's how && works. The next
command is executed only if the previous one was a success.

cheers
Janek


Anton Gladky said: (by the date of Wed, 13 Nov 2019 20:33:14 +0100)

> Hallo Bruno,
> 
> could you please send me a script, which is doing the download/upload?
> 
> Actually, when I do
> 
> "wget 
> https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages;
> 
> I get right now the error code 8.
> 
> Actually, if you use the symbol &&, it will not execute the next step,
> if the previous one failed.
> 
> Something like:
> 
> wget 
> https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
> && rm -rf /var/www/yade-dem.org && cp * /var/www/yade-dem.org
> 
> should solve the problem. If wget fails, like in this case - rm and cp
> will not be executed. And we get old documentation,
> but in any case - not the empty pages.
> 
> I hope, my writings can be understood :)
> 
> Regards
> 
> Anton
> 
> Am Mi., 13. Nov. 2019 um 18:05 Uhr schrieb Bruno Chareyre
> :
> 
> >
> > Hi there,
> > yade-dem.org is currently down while yade-dev.gitlab.io/trunk/ is not.
> >
> > The reason is that yade server is downloading the gitlab artifacts like 
> > this:
> > wget 
> > https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages,
> > but at the moment that url returns nothing. Hence empty website.
> >
> > Obviously our script should test the output of wget, to not replace the 
> > content by nothing. We can fix this. Even so, I would like to understand 
> > why the artifacts are not there. They usually are. Is it because they 
> > expired on gitlab? Is it because of a failed pipeline?
> > Maybe we should form another url to try and get the content of 
> > yade-dev.gitlab.io/trunk/ (if possibe) instead of checking out an artifact.
> > I don't have time to check more right now so in case someone has 
> > inspiration, let me know:
> > - if you know how to make wget conditional
> > - if you know a better target url
> >
> > Cheers
> >
> > Bruno
> >
> > --
> > --
> > ___
> > Bruno Chareyre
> > Associate Professor
> > ENSE³ - Grenoble INP
> > Lab. 3SR
> > BP 53
> > 38041 Grenoble cedex 9
> > Tél : +33 4 56 52 86 21
> > 
> >
> > Email too brief?
> > Here's why: email charter
> > ___
> > Mailing list: https://launchpad.net/~yade-dev
> > Post to : yade-dev@lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~yade-dev
> > More help   : https://help.launchpad.net/ListHelp
> 
> ___
> Mailing list: https://launchpad.net/~yade-dev
> Post to : yade-dev@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~yade-dev
> More help   : https://help.launchpad.net/ListHelp


-- 
--
Janek Kozicki, PhD. DSc. Arch. Assoc. Prof.
Gdańsk University of Technology
Faculty of Applied Physics and Mathematics
Department of Theoretical Physics and Quantum Information
--
http://yade-dem.org/
http://pg.edu.pl/jkozicki (click English flag on top right)

___
Mailing list: https://launchpad.net/~yade-dev
Post to : yade-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yade-dev
More help   : https://help.launchpad.net/ListHelp