Re: [Yade-dev] gitlab artifacts missing

2019-11-14 Thread Bruno Chareyre 
The question of why this happens remains but that's perfectly fine to
> workaround it.
>

It's actually easy to suspect that latest "pages" artifact is empty when
the pipeline fails (especially if it fails at that particular "pages" step).
What's less clear to me, if that's really the problem: what's the url of
the latest _successful_ job?
Would be good to know but the previous workaround is ok.
Bruno
___
Mailing list: https://launchpad.net/~yade-dev
Post to : yade-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yade-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Yade-dev] gitlab artifacts missing

2019-11-14 Thread Bruno Chareyre 
On Wed, 13 Nov 2019 at 20:33, Anton Gladky  wrote:

>
> could you please send me a script, which is doing the download/upload?
>
>
Please forgive in advance my naive bash crap and see below.

*cd /tmp*
*touch etag_last*






*wget --server-response --spider
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages

2>&1 | grep ETag > etag_remoteif ! cmp -s etag_remote etag_last; then
wget
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages

  unzip download?job=pagesrm -rf /var/www/doc/*mv /tmp/public
/var/www/docrm download?job=pages*
*mv etag_remote etag_last*
*fi*



> Actually, when I do
>
> "wget
> https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
> "
>
> I get right now the error code 8.
>

As I type this email that target works again (log at the end of this
email), yet for some reason (it occured at least twice in the last months,
probably only for a couple hours the first time).



> Actually, if you use the symbol &&, it will not execute the next step,
> if the previous one failed.
>
> Something like:
>
> wget
> https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
> &&
> 
> rm -rf /var/www/yade-dem.org && cp * /var/www/yade-dem.org



Sounds like the good fix, thanks for mentioning the syntax.
I think we will try that.
The question of why this happens remains but that's perfectly fine to
workaround it.
Cheers
Bruno




$ wget
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
--2019-11-13 17:55:13--
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
Resolving gitlab.com (gitlab.com)... 35.231.145.151
Connecting to gitlab.com (gitlab.com)|35.231.145.151|:443... connected.
HTTP request sent, awaiting response... 404 Not Found
2019-11-13 17:55:14 ERROR 404: Not Found.

$ wget
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
--2019-11-14 17:09:32--
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
Resolving gitlab.com (gitlab.com)... 35.231.145.151
Connecting to gitlab.com (gitlab.com)|35.231.145.151|:443... connected.
HTTP request sent, awaiting response... 302 Found
Location:
https://gitlab.com/yade-dev/trunk/-/jobs/349059349/artifacts/download
[following]
--2019-11-14 17:09:34--
https://gitlab.com/yade-dev/trunk/-/jobs/349059349/artifacts/download
Reusing existing connection to gitlab.com:443.
HTTP request sent, awaiting response... 302 Found
Location:
https://storage.googleapis.com/gitlab-gprd-artifacts/5c/74/5c748985bd13444e59322d1dd55fa2a5091e45a49c1db75d94366184ad2a7fe9/2019_11_13/349059349/361212541/artifacts.zip?response-content-disposition=attachment%3B%20filename%3D%22artifacts.zip%22%3B%20filename%2A%3DUTF-8%27%27artifacts.zip&response-content-type=application%2Fzip&GoogleAccessId=gitlab-object-storage-...@gitlab-production.iam.gserviceaccount.com&Signature=JpHFPisbZHmwBGSHy%2Bu%2BDN0Qyi8TQN%2F2J5DbGjSJiuKKO8T8XzyrxjfbEncC%0AgNSHmubYDcLB%2FScSvvS8m%2BJt%2F%2BHkerujryO%2B4FQMAzcnxg2ptUTE2hqR3F43%0AuJk6zo5OsJ8IFvzzfH43VRvhPXEiC2NwbC0FB7jIqvw%2FNevlghADoRxvG7BK%0A0m9yaeoywDilR%2FDu%2BB6qUegllIyp8gvc5XrjyDortOuSUM7wegpRGQ5vMBS1%0Arxn%2F1AvTyeUDaSkbyMeLmx9XxHBJsFdlBAPHdQ1nVgh1GmGU7uoIku7895nd%0AhxoDIfe4ANq5Qdj6O7LgKhr%2FHugAgkyPJ2B8VDf5ZQ%3D%3D&Expires=1573748375
[following]
--2019-11-14 17:09:35--
https://storage.googleapis.com/gitlab-gprd-artifacts/5c/74/5c748985bd13444e59322d1dd55fa2a5091e45a49c1db75d94366184ad2a7fe9/2019_11_13/349059349/361212541/artifacts.zip?response-content-disposition=attachment%3B%20filename%3D%22artifacts.zip%22%3B%20filename%2A%3DUTF-8%27%27artifacts.zip&response-content-type=application%2Fzip&GoogleAccessId=gitlab-object-storage-...@gitlab-production.iam.gserviceaccount.com&Signature=JpHFPisbZHmwBGSHy%2Bu%2BDN0Qyi8TQN%2F2J5DbGjSJiuKKO8T8XzyrxjfbEncC%0AgNSHmubYDcLB%2FScSvvS8m%2BJt%2F%2BHkerujryO%2B4FQMAzcnxg2ptUTE2hqR3F43%0AuJk6zo5OsJ8IFvzzfH43VRvhPXEiC2NwbC0FB7jIqvw%2FNevlghADoRxvG7BK%0A0m9yaeoywDilR%2FDu%2BB6qUegllIyp8gvc5XrjyDortOuSUM7wegpRGQ5vMBS1%0Arxn%2F1AvTyeUDaSkbyMeLmx9XxHBJsFdlBAPHdQ1nVgh1GmGU7uoIku7895nd%0AhxoDIfe4ANq5Qdj6O7LgKhr%2FHugAgkyPJ2B8VDf5ZQ%3D%3D&Expires=1573748375
Resolving storage.googleapis.com (storage.googleapis.com)...
216.58.206.240, 2a00:1450:4007:817::2010
Connecting to storage.googleapis.com
(storage.googleapis.com)|216.58.206.240|:443...
connected.
HTTP request sent, awaiting response... 200 OK
Length: 18604798 (18M) [application/zip]
Saving to: ‘download?job=pages’

download?job=pages
100%[=>]  17.74M  23.0MB/s
 in 0.8s

>
> Am Mi., 13. Nov. 2019 um 18:05 Uhr schrieb Bruno Chareyre
> :
>
> >
> > Hi there,
> > yade-dem.org is currently down whil

Re: [Yade-dev] gitlab artifacts missing

2019-11-14 Thread Janek Kozicki (yade) 
A very simple explanation of how && works: do this command:

ls && ls /wrongdir && ls

Then do this command:

ls && ls && ls

You will see that in the first case the failure of 'ls /wrongdir'
prevented execution of last 'ls'. That's how && works. The next
command is executed only if the previous one was a success.

cheers
Janek


Anton Gladky said: (by the date of Wed, 13 Nov 2019 20:33:14 +0100)

> Hallo Bruno,
> 
> could you please send me a script, which is doing the download/upload?
> 
> Actually, when I do
> 
> "wget 
> https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages";
> 
> I get right now the error code 8.
> 
> Actually, if you use the symbol &&, it will not execute the next step,
> if the previous one failed.
> 
> Something like:
> 
> wget 
> https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
> && rm -rf /var/www/yade-dem.org && cp * /var/www/yade-dem.org
> 
> should solve the problem. If wget fails, like in this case - rm and cp
> will not be executed. And we get old documentation,
> but in any case - not the empty pages.
> 
> I hope, my writings can be understood :)
> 
> Regards
> 
> Anton
> 
> Am Mi., 13. Nov. 2019 um 18:05 Uhr schrieb Bruno Chareyre
> :
> 
> >
> > Hi there,
> > yade-dem.org is currently down while yade-dev.gitlab.io/trunk/ is not.
> >
> > The reason is that yade server is downloading the gitlab artifacts like 
> > this:
> > wget 
> > https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages,
> > but at the moment that url returns nothing. Hence empty website.
> >
> > Obviously our script should test the output of wget, to not replace the 
> > content by nothing. We can fix this. Even so, I would like to understand 
> > why the artifacts are not there. They usually are. Is it because they 
> > expired on gitlab? Is it because of a failed pipeline?
> > Maybe we should form another url to try and get the content of 
> > yade-dev.gitlab.io/trunk/ (if possibe) instead of checking out an artifact.
> > I don't have time to check more right now so in case someone has 
> > inspiration, let me know:
> > - if you know how to make wget conditional
> > - if you know a better target url
> >
> > Cheers
> >
> > Bruno
> >
> > --
> > --
> > ___
> > Bruno Chareyre
> > Associate Professor
> > ENSE³ - Grenoble INP
> > Lab. 3SR
> > BP 53
> > 38041 Grenoble cedex 9
> > Tél : +33 4 56 52 86 21
> > 
> >
> > Email too brief?
> > Here's why: email charter
> > ___
> > Mailing list: https://launchpad.net/~yade-dev
> > Post to : yade-dev@lists.launchpad.net
> > Unsubscribe : https://launchpad.net/~yade-dev
> > More help   : https://help.launchpad.net/ListHelp
> 
> ___
> Mailing list: https://launchpad.net/~yade-dev
> Post to : yade-dev@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~yade-dev
> More help   : https://help.launchpad.net/ListHelp


-- 
--
Janek Kozicki, PhD. DSc. Arch. Assoc. Prof.
Gdańsk University of Technology
Faculty of Applied Physics and Mathematics
Department of Theoretical Physics and Quantum Information
--
http://yade-dem.org/
http://pg.edu.pl/jkozicki (click English flag on top right)

___
Mailing list: https://launchpad.net/~yade-dev
Post to : yade-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yade-dev
More help   : https://help.launchpad.net/ListHelp

Re: [Yade-dev] gitlab artifacts missing

2019-11-13 Thread Anton Gladky 
Hallo Bruno,

could you please send me a script, which is doing the download/upload?

Actually, when I do

"wget 
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages";

I get right now the error code 8.

Actually, if you use the symbol &&, it will not execute the next step,
if the previous one failed.

Something like:

wget 
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
&& rm -rf /var/www/yade-dem.org && cp * /var/www/yade-dem.org

should solve the problem. If wget fails, like in this case - rm and cp
will not be executed. And we get old documentation,
but in any case - not the empty pages.

I hope, my writings can be understood :)

Regards

Anton

Am Mi., 13. Nov. 2019 um 18:05 Uhr schrieb Bruno Chareyre
:

>
> Hi there,
> yade-dem.org is currently down while yade-dev.gitlab.io/trunk/ is not.
>
> The reason is that yade server is downloading the gitlab artifacts like this:
> wget 
> https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages,
> but at the moment that url returns nothing. Hence empty website.
>
> Obviously our script should test the output of wget, to not replace the 
> content by nothing. We can fix this. Even so, I would like to understand why 
> the artifacts are not there. They usually are. Is it because they expired on 
> gitlab? Is it because of a failed pipeline?
> Maybe we should form another url to try and get the content of 
> yade-dev.gitlab.io/trunk/ (if possibe) instead of checking out an artifact.
> I don't have time to check more right now so in case someone has inspiration, 
> let me know:
> - if you know how to make wget conditional
> - if you know a better target url
>
> Cheers
>
> Bruno
>
> --
> --
> ___
> Bruno Chareyre
> Associate Professor
> ENSE³ - Grenoble INP
> Lab. 3SR
> BP 53
> 38041 Grenoble cedex 9
> Tél : +33 4 56 52 86 21
> 
>
> Email too brief?
> Here's why: email charter
> ___
> Mailing list: https://launchpad.net/~yade-dev
> Post to : yade-dev@lists.launchpad.net
> Unsubscribe : https://launchpad.net/~yade-dev
> More help   : https://help.launchpad.net/ListHelp

___
Mailing list: https://launchpad.net/~yade-dev
Post to : yade-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yade-dev
More help   : https://help.launchpad.net/ListHelp

[Yade-dev] gitlab artifacts missing

2019-11-13 Thread Bruno Chareyre 
Hi there,
yade-dem.org is currently down while yade-dev.gitlab.io/trunk/ is not.

The reason is that yade server is downloading the gitlab artifacts like
this:
*wget
https://gitlab.com/yade-dev/trunk/-/jobs/artifacts/master/download?job=pages
,*
but at the moment that url returns nothing. Hence empty website.

Obviously our script should test the output of wget, to not replace the
content by nothing. We can fix this. Even so, I would like to understand
why the artifacts are not there. They usually are. Is it because they
expired on gitlab? Is it because of a failed pipeline?
Maybe we should form another url to try and get the content of
yade-dev.gitlab.io/trunk/ (if possibe) instead of checking out an artifact.
I don't have time to check more right now so in case someone has
inspiration, let me know:
- if you know how to make wget conditional
- if you know a better target url

Cheers

Bruno

-- 
-- 
___
Bruno Chareyre
Associate Professor
ENSE³ - Grenoble INP
Lab. 3SR
BP 53
38041 Grenoble cedex 9
Tél : +33 4 56 52 86 21


Email too brief?
Here's why: email charter

___
Mailing list: https://launchpad.net/~yade-dev
Post to : yade-dev@lists.launchpad.net
Unsubscribe : https://launchpad.net/~yade-dev
More help   : https://help.launchpad.net/ListHelp