Re: [OpenStack-Infra] [openstack-dev] [all][stable][ptls] Tagging liberty as EOL

2017-02-14 Thread Tony Breeds 
On Wed, Feb 08, 2017 at 09:55:41AM -0800, Ihar Hrachyshka wrote:
> Has the liberty-eol cleanup happened? Because I still see
> stable/liberty branch in openstack/neutron repo, which gets in the way
> of some logic around proactive backports:
> https://review.openstack.org/#/c/427936/4/bugs-fixed-since.py@75, and
> I see backports proposed for the branch that is no longer supported
> and has broken gate setup.

It seems I setup the second wave and then neglected to ask the infra team to 
process them.

Infra tema (*cough* Josh *cough*) can you remove the branches marke Please EOL 
in

https://gist.githubusercontent.com/tbreeds/93cd346c37aa46269456f56649f0a4ac/raw/4f781426270cb2030f978f47b1f46dae6f5aecb4/liberty_eol_data.txt

Once that's done I can do a final pass for jobs that may break and we can drop
the devstack, devstack-gate and requirements branches.

Tony.


signature.asc
Description: PGP signature
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] PTG team dinner?

2017-02-14 Thread Joshua Hesketh 


>
> I'd say reserve for 11 now. You can always adjust it as the etherpad
> changes.
>
>
I'd suggest maybe adding a few extra as we might find people during the day
who want to tag along.
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] PTG team dinner?

2017-02-14 Thread Clark Boylan 
On Tue, Feb 14, 2017, at 08:45 AM, Clark Boylan wrote:
> On Wed, Feb 8, 2017, at 12:52 PM, Clark Boylan wrote:
> > On Tue, Feb 7, 2017, at 04:28 PM, Clark Boylan wrote:
> > > Hello,
> > > 
> > > We failed at doing a lunch or dinner together in Barcelona because
> > > ETOOMANYTHINGS. Wondering if there is interest in attempting to do a
> > > dinner during the PTG? Would likely have to be Monday night since I
> > > expect some people will leave late Tuesday.
> > > 
> > > Not sure how many of us are interested and will be at the PTG, but maybe
> > > we can try putting something together if we get a rough headcount?
> > > 
> > > I'm in for what its worth. Also noticed there is a Trader Vic's nearby,
> > > we could all dress in Hawaiian shirts like fungi :)
> > 
> > It was suggested that we could use an etherpad to get a headcount and
> > available times for people. I went ahead and quickly put
> > https://etherpad.openstack.org/p/2017-atl-ptg-infra-dinner together.
> > Please update that if interested and then we can figure out what will
> > work using that info.
> 
> Based on the etherpad's contents looks like Monday is indeed the best
> day for us. Let's go ahead and pencil in Monday as the evening for a
> team dinner. Apparently there will be an Atlanta guide published soon
> (or has been published) will look through that then suggest a better
> list of options for location and go from there.
> 
> If anyone else is interested please still get yourself on the etherpad
> so that we can get an accurate count of seats needed. For anyone that
> has organized one of these before is it typically helpful to try for a
> reservation with a couple extra seats regardless? Looks like we have 11
> interested currently.

With the help of the small restaurant guide on the wiki I have
discovered Poor Calvin's. This place looks neat, supposedly fusion of
Thai and Southern food and is well reviewed. It is 0.8 miles or ~15
minute walk from the Sheraton according to Google so should be walkable
for most. Rather than draw out restaurant selection I was thinking I
would go ahead and try to make a reservation for however many people are
signed up on the etherpad for as close to 7pm as I can manage Monday
night. I will wait until tomorrow before doing this in order to provide
veto time if this location does not work for someone, in which case
please do help suggest an alternative.

If I can't get a reservation for us at Poor Calvin's I was looking at
Sway as a backup since it also has southern food and seems to be well
reviewed despite being hotel restaurant. Again please veto if necessary
and suggest alternatives.

Lastly I did update the etherpad with links to these restaurants (and a
few others) if you need more info.

Thank you,
Clark

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] urllib3 import problem

2017-02-14 Thread Ian Cordasco 
-Original Message-
From: Carmine Annunziata 
Reply: Carmine Annunziata 
Date: February 14, 2017 at 10:56:21
To: openstack-infra@lists.openstack.org 
Subject:  [OpenStack-Infra] urllib3 import problem

> Hi,
> I upgraded ubuntu 14.04 to 16.04 LTS and now during the ./stack.sh i got
> this problem.
> I upgraded with:
> sudo pip install urllib3 --upgrade
> but the problem is still there:
>
> 2017-02-14 15:54:40.894 | Traceback (most recent call last):
> 2017-02-14 15:54:40.894 | File "/opt/stack/requirements/.venv/bin/pip",
> line 7, in
> 2017-02-14 15:54:40.894 | from pip import main
> 2017-02-14 15:54:40.894 | File "/opt/stack/requirements/.venv
> /local/lib/python2.7/site-packages/pip/__init__.py", line 21, in
> 2017-02-14 15:54:40.895 | from
> pip._vendor.requests.packages.urllib3.exceptions
> import DependencyWarning
> 2017-02-14 15:54:40.895 | File "/opt/stack/requirements/.venv
> /local/lib/python2.7/site-packages/pip/_vendor/requests/__init__.py", line
> 62, in
> 2017-02-14 15:54:40.895 | from .packages.urllib3.exceptions import
> DependencyWarning
> 2017-02-14 15:54:40.895 | File "/opt/stack/requirements/.venv
> /local/lib/python2.7/site-packages/pip/_vendor/requests/packages/__init__.py",
> line 29, in
> 2017-02-14 15:54:40.895 | import urllib3
> 2017-02-14 15:54:40.895 | ImportError: No module named urllib3
> 2017-02-14 15:54:40.904 | +inc/python:pip_install:1
> exit_trap
> 2017-02-14 15:54:40.908 | +./stack.sh:exit_trap:487 local
> r=1
> 2017-02-14 15:54:40.924 | ++./stack.sh:exit_trap:488 jobs
> -p
> 2017-02-14 15:54:40.935 | +./stack.sh:exit_trap:488 jobs=
> 2017-02-14 15:54:40.948 | +./stack.sh:exit_trap:491 [[ -n
> '' ]]
> 2017-02-14 15:54:40.952 | +./stack.sh:exit_trap:497
> kill_spinner
> 2017-02-14 15:54:40.967 | +./stack.sh:kill_spinner:383 '['
> '!' -z '' ']'
> 2017-02-14 15:54:40.982 | +./stack.sh:exit_trap:499 [[ 1
> -ne 0 ]]
> 2017-02-14 15:54:40.998 | +./stack.sh:exit_trap:500 echo
> 'Error on exit'
> 2017-02-14 15:54:40.999 | Error on exit
> 2017-02-14 15:54:41.002 | +./stack.sh:exit_trap:501
> generate-subunit 1487087613 67 fail
> 2017-02-14 15:54:41.817 | +./stack.sh:exit_trap:502 [[ -z
> /opt/stack/logs ]]
> 2017-02-14 15:54:41.821 | +./stack.sh:exit_trap:505
> /home/stack/devstack/tools/worlddump.py -d /opt/stack/logs
> 2017-02-14 15:54:43.088 | +./stack.sh:exit_trap:511 exit 1
> stack@coritel-VirtualBox:~/devstack$ sudo gedit
> /opt/stack/requirements/.venv/local/lib/python2.7/site-packa
> ges/pip/_vendor/requests/packages/__init__.py

Carmine,

I replied to you on openstack-dev. Please let's focus the conversation there.

--
Ian Cordasco

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

[OpenStack-Infra] Foundation mascot and logo treatments for Infra team (final version)

2017-02-14 Thread Jeremy Stanley 
As some of you may recall, Heidi Joy Tretheway has been wrangling
the OpenStack Foundation's effort to solicit mascots from each
project team and commission logo artists to produce a consistent set
of treatments which will be used on the foundation's Web properties
and which can also be reused by the community as desired (the
specific licensing is still being worked out, but that is the intent
anyway).

That effort has concluded, and until there's a more official place
to obtain their logo work for all the teams the following URL has
been provided for preliminary access to the various versions and
formats:

https://www.dropbox.com/sh/hjfhn8rev8dxqya/AACFbVuOjSYiSyPHVtsoO1Kda?dl=0

The foundation has printed some stickers with this design which
they'll have for us at the PTG, and will be incorporating it into
the door sign for our room there as well.

Also, Heidi Joy asked me to convey her personal thanks to everyone
who participated in this selection and review process.
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

[OpenStack-Infra] urllib3 import problem

2017-02-14 Thread Carmine Annunziata 
Hi,
I upgraded ubuntu 14.04 to 16.04 LTS and now during the ./stack.sh i got
this problem.
I upgraded with:
sudo pip install urllib3 --upgrade
but the problem is still there:

2017-02-14 15:54:40.894 | Traceback (most recent call last):
2017-02-14 15:54:40.894 |   File "/opt/stack/requirements/.venv/bin/pip",
line 7, in 
2017-02-14 15:54:40.894 | from pip import main
2017-02-14 15:54:40.894 |   File "/opt/stack/requirements/.venv
/local/lib/python2.7/site-packages/pip/__init__.py", line 21, in 
2017-02-14 15:54:40.895 | from
pip._vendor.requests.packages.urllib3.exceptions
import DependencyWarning
2017-02-14 15:54:40.895 |   File "/opt/stack/requirements/.venv
/local/lib/python2.7/site-packages/pip/_vendor/requests/__init__.py", line
62, in 
2017-02-14 15:54:40.895 | from .packages.urllib3.exceptions import
DependencyWarning
2017-02-14 15:54:40.895 |   File "/opt/stack/requirements/.venv
/local/lib/python2.7/site-packages/pip/_vendor/requests/packages/__init__.py",
line 29, in 
2017-02-14 15:54:40.895 | import urllib3
2017-02-14 15:54:40.895 | ImportError: No module named urllib3
2017-02-14 15:54:40.904 | +inc/python:pip_install:1
exit_trap
2017-02-14 15:54:40.908 | +./stack.sh:exit_trap:487  local
r=1
2017-02-14 15:54:40.924 | ++./stack.sh:exit_trap:488  jobs
-p
2017-02-14 15:54:40.935 | +./stack.sh:exit_trap:488  jobs=
2017-02-14 15:54:40.948 | +./stack.sh:exit_trap:491  [[ -n
'' ]]
2017-02-14 15:54:40.952 | +./stack.sh:exit_trap:497
kill_spinner
2017-02-14 15:54:40.967 | +./stack.sh:kill_spinner:383   '['
'!' -z '' ']'
2017-02-14 15:54:40.982 | +./stack.sh:exit_trap:499  [[ 1
-ne 0 ]]
2017-02-14 15:54:40.998 | +./stack.sh:exit_trap:500  echo
'Error on exit'
2017-02-14 15:54:40.999 | Error on exit
2017-02-14 15:54:41.002 | +./stack.sh:exit_trap:501
generate-subunit 1487087613 67 fail
2017-02-14 15:54:41.817 | +./stack.sh:exit_trap:502  [[ -z
/opt/stack/logs ]]
2017-02-14 15:54:41.821 | +./stack.sh:exit_trap:505
/home/stack/devstack/tools/worlddump.py -d /opt/stack/logs
2017-02-14 15:54:43.088 | +./stack.sh:exit_trap:511  exit 1
stack@coritel-VirtualBox:~/devstack$ sudo gedit
/opt/stack/requirements/.venv/local/lib/python2.7/site-packa
ges/pip/_vendor/requests/packages/__init__.py


thank you,
Carmine
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] The new openstack.cl...@gmail.com user

2017-02-14 Thread Jeremy Stanley 
On 2017-02-14 15:42:53 +0530 (+0530), Amrith Kumar wrote:
[...]
> I believe that to post reviews, one must register and sign a CLA.
[...]

Our OpenStack Individual Contributor License Agreement is about
provenance of source code contributions. Anyone can comment and even
vote on reviews without agreeing to a CLA, as reviewing is not
considered a copyrightable contribution to the source itself (even
though reviewers do include sample source code in their comments
from time to time).
-- 
Jeremy Stanley


signature.asc
Description: Digital signature
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] The new openstack.cl...@gmail.com user

2017-02-14 Thread Tom Fifield 

On 14/02/17 22:33, Paul Belanger wrote:

On Tue, Feb 14, 2017 at 03:42:53PM +0530, Amrith Kumar wrote:

It appears that a user registered as "openstack.cl...@gmail.com" has emerged
as a new bot that is posting a comment on numerous reviews with comments
like:[1]

"Patch Set 1: Code-Review-1

Not urgent to merge just before Ocata is released. And please save load to
the CI system."

"Patch Set 1: Code-Review-1

With more than 50 same topics. Such work seems to be for some marketing
slides. Why not submit the same corrections in one patch?"

"Patch Set 1: Code-Review-1

Such work seems to be for some marketing slides, not for OpenStack
technology. Why not submit the same corrections in one patch?"

Would someone in Infra or the TC please take a look into who this anonymous
user is? I believe that to post reviews, one must register and sign a CLA.
Could someone look into who may have signed this CLA?

Thanks,

-amrith

[1]
https://review.openstack.org/#/q/reviewer:%22OpenStack+Clean+%253Copenstack.
clean%2540gmail.com%253E%22



--
Amrith Kumar
amrith.ku...@gmail.com
+1-978-563-9590
GPG: 0x5e48849a9d21a29b


I've gone a head a disabled the account for now. Until we can look more into it.


I noticed that the account only comments on contributors from ZTE. Maybe 
it's something someone set up to help their colleagues there?



___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] [openstack-infra][fuel-ccp-designate] Repository empty core-group

2017-02-14 Thread Paul Belanger 
On Tue, Feb 14, 2017 at 01:06:08PM +0400, Peter Razumovsky wrote:
> No one from our team have no rights to merge patches to fuel-ccp-designate.
> Please, help us to land patches asap, it's very important! Gerrit patch
> with created project: https://review.openstack.org/#/c/425035/
> 
Looks like you are properly added to the group now.

> 2017-02-13 18:13 GMT+04:00 Peter Razumovsky :
> 
> >
> > -- Forwarded message --
> > From: Peter Razumovsky 
> > Date: 2017-02-13 13:21 GMT+04:00
> > Subject: [openstack-infra][fuel-ccp-designate] Repository empty core-group
> > To: "OpenStack Development Mailing List (not for usage questions)" <
> > openstack-...@lists.openstack.org>
> >
> >
> > Hi openstack-infra!
> >
> > I recently added new repo named fuel-ccp-designate [1], but we can't merge
> > any patch due to no one consists in fuel-ccp-designate core-group. Can you
> > add me to core-group, and then I'll add other cores to it. Thanks!
> >
> > [1] https://github.com/openstack/fuel-ccp-designate
> >
> > --
> > Best regards,
> > Peter Razumovsky
> >
> >
> >
> > --
> > Best regards,
> > Peter Razumovsky
> >
> 
> 
> 
> -- 
> Best regards,
> Peter Razumovsky

> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] The new openstack.cl...@gmail.com user

2017-02-14 Thread Paul Belanger 
On Tue, Feb 14, 2017 at 03:42:53PM +0530, Amrith Kumar wrote:
> It appears that a user registered as "openstack.cl...@gmail.com" has emerged
> as a new bot that is posting a comment on numerous reviews with comments
> like:[1]
> 
> "Patch Set 1: Code-Review-1
> 
> Not urgent to merge just before Ocata is released. And please save load to
> the CI system."
> 
> "Patch Set 1: Code-Review-1
> 
> With more than 50 same topics. Such work seems to be for some marketing
> slides. Why not submit the same corrections in one patch?"
> 
> "Patch Set 1: Code-Review-1
> 
> Such work seems to be for some marketing slides, not for OpenStack
> technology. Why not submit the same corrections in one patch?"
> 
> Would someone in Infra or the TC please take a look into who this anonymous
> user is? I believe that to post reviews, one must register and sign a CLA.
> Could someone look into who may have signed this CLA?
> 
> Thanks,
> 
> -amrith 
> 
> [1]
> https://review.openstack.org/#/q/reviewer:%22OpenStack+Clean+%253Copenstack.
> clean%2540gmail.com%253E%22
> 
> 
> 
> --
> Amrith Kumar
> amrith.ku...@gmail.com
> +1-978-563-9590
> GPG: 0x5e48849a9d21a29b
> 
I've gone a head a disabled the account for now. Until we can look more into it.

> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

[OpenStack-Infra] The new openstack.cl...@gmail.com user

2017-02-14 Thread Amrith Kumar 
It appears that a user registered as "openstack.cl...@gmail.com" has emerged
as a new bot that is posting a comment on numerous reviews with comments
like:[1]

"Patch Set 1: Code-Review-1

Not urgent to merge just before Ocata is released. And please save load to
the CI system."

"Patch Set 1: Code-Review-1

With more than 50 same topics. Such work seems to be for some marketing
slides. Why not submit the same corrections in one patch?"

"Patch Set 1: Code-Review-1

Such work seems to be for some marketing slides, not for OpenStack
technology. Why not submit the same corrections in one patch?"

Would someone in Infra or the TC please take a look into who this anonymous
user is? I believe that to post reviews, one must register and sign a CLA.
Could someone look into who may have signed this CLA?

Thanks,

-amrith 

[1]
https://review.openstack.org/#/q/reviewer:%22OpenStack+Clean+%253Copenstack.
clean%2540gmail.com%253E%22



--
Amrith Kumar
amrith.ku...@gmail.com
+1-978-563-9590
GPG: 0x5e48849a9d21a29b



smime.p7s
Description: S/MIME cryptographic signature
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] [openstack-infra][fuel-ccp-designate] Repository empty core-group

2017-02-14 Thread Peter Razumovsky 
No one from our team have no rights to merge patches to fuel-ccp-designate.
Please, help us to land patches asap, it's very important! Gerrit patch
with created project: https://review.openstack.org/#/c/425035/

2017-02-13 18:13 GMT+04:00 Peter Razumovsky :

>
> -- Forwarded message --
> From: Peter Razumovsky 
> Date: 2017-02-13 13:21 GMT+04:00
> Subject: [openstack-infra][fuel-ccp-designate] Repository empty core-group
> To: "OpenStack Development Mailing List (not for usage questions)" <
> openstack-...@lists.openstack.org>
>
>
> Hi openstack-infra!
>
> I recently added new repo named fuel-ccp-designate [1], but we can't merge
> any patch due to no one consists in fuel-ccp-designate core-group. Can you
> add me to core-group, and then I'll add other cores to it. Thanks!
>
> [1] https://github.com/openstack/fuel-ccp-designate
>
> --
> Best regards,
> Peter Razumovsky
>
>
>
> --
> Best regards,
> Peter Razumovsky
>



-- 
Best regards,
Peter Razumovsky
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Ask.o.o down

2017-02-14 Thread Joshua Hesketh 
On Tue, Feb 14, 2017 at 7:15 PM, Tom Fifield  wrote:

> On 14/02/17 16:11, Joshua Hesketh wrote:
>
>> Hey Tom,
>>
>> Where is that script being fired from (a quick grep doesn't find it), or
>> is it a tool people are using?
>>
>> If it's a tool we'd need to make sure whoever is using it gets a new
>> version to rule it out.
>>
>
> Indeed.
>
>
> It's fired from a PHP service on www.openstack.org itself, which writes
> to the Member database:
>
> https://github.com/OpenStackweb/openstack-org/blob/master/
> auc-metrics/code/services/ActiveModeratorService.php



Right. I wonder if somebody could check the logs to see if the process
times out. Sadly looking at that code it looks like any output messages
from the script will be discarded.

 - Josh




>
>
> The next step is to update the copy of the script it references:
>
> https://github.com/OpenStackweb/openstack-org/blob/master/
> auc-metrics/lib/uc-recognition/tools/get_active_moderator.py
>
> I am not sure if this is in place using git submodules or manually, but
> will figure it out and get that updated.
>
>
>
>
>  - Josh
>>
>> On Tue, Feb 14, 2017 at 7:07 PM, Tom Fifield > > wrote:
>>
>> On 14/02/17 16:06, Joshua Hesketh wrote:
>>
>> Hey,
>>
>> I've brought the service back up, but have no new clues as to why.
>>
>>
>> Cheers.
>>
>> Going to try: https://review.openstack.org/#/c/433478/
>> 
>> to see if this script is culprit.
>>
>>
>> - Josh
>>
>> On Tue, Feb 14, 2017 at 6:50 PM, Tom Fifield > 
>> >> wrote:
>>
>> On 10/02/17 22:39, Jeremy Stanley wrote:
>>
>> On 2017-02-10 16:08:51 +0800 (+0800), Tom Fifield wrote:
>> [...]
>>
>> Down again, this time with "Network is unreachable".
>>
>> [...]
>>
>> I'm not finding any obvious errors on the server nor
>> relevant
>> maintenance notices/trouble tickets from the service
>> provider to
>> explain this. I do see conspicuous gaps in network
>> traffic volume
>> and system load from ~06:45 to ~08:10 UTC according to
>> cacti:
>>
>> http://cacti.openstack.org/?tree_id=1_id=156
>> 
>> > >
>>
>> Skipping back through previous days I find some similar
>> gaps
>> starting anywhere from 06:30 to 07:00 and ending between
>> 07:00 and
>> 08:00 but they don't seem to occur every day and I'm not
>> having much
>> luck finding a pattern. It _is_ conspicuously close to
>> when
>> /etc/cron.daily scripts get fired from the crontab so
>> might coincide
>> with log rotation/service restarts? The graphs don't
>> show these gaps
>> correlating with any spikes in CPU, memory or disk
>> activity so it
>> doesn't seem to be resource starvation (at least not for
>> any common
>> resources we're tracking).
>>
>>
>> Indeed. It's down again today during the same timeslot.
>>
>> Another idea for the cron-based theory:
>>
>>
>> https://github.com/openstack/uc-recognition/blob/master/tool
>> s/get_active_moderator.py
>> > ls/get_active_moderator.py>
>>
>> > ls/get_active_moderator.py
>> > ls/get_active_moderator.py>>
>>
>> loops through the list of Ask OpenStack users via the API on
>> a cron
>> running on www.openstack.org 
>> . Not sure
>> when that cron runs, but if it's similar, this could
>> potentially be
>> a high-load generator.
>>
>>
>>
>>
>> Regards,
>>
>>
>> Tom
>>
>>
>> ___
>> OpenStack-Infra mailing list
>> OpenStack-Infra@lists.openstack.org
>> 
>> > >
>>
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstac
>> k-infra
>> > ck-infra>
>>
>>

Re: [OpenStack-Infra] Ask.o.o down

2017-02-14 Thread Tom Fifield 

On 14/02/17 16:11, Joshua Hesketh wrote:

Hey Tom,

Where is that script being fired from (a quick grep doesn't find it), or
is it a tool people are using?

If it's a tool we'd need to make sure whoever is using it gets a new
version to rule it out.


Indeed.


It's fired from a PHP service on www.openstack.org itself, which writes 
to the Member database:


https://github.com/OpenStackweb/openstack-org/blob/master/auc-metrics/code/services/ActiveModeratorService.php

The next step is to update the copy of the script it references:

https://github.com/OpenStackweb/openstack-org/blob/master/auc-metrics/lib/uc-recognition/tools/get_active_moderator.py

I am not sure if this is in place using git submodules or manually, but 
will figure it out and get that updated.






 - Josh

On Tue, Feb 14, 2017 at 7:07 PM, Tom Fifield > wrote:

On 14/02/17 16:06, Joshua Hesketh wrote:

Hey,

I've brought the service back up, but have no new clues as to why.


Cheers.

Going to try: https://review.openstack.org/#/c/433478/

to see if this script is culprit.


- Josh

On Tue, Feb 14, 2017 at 6:50 PM, Tom Fifield 
>> wrote:

On 10/02/17 22:39, Jeremy Stanley wrote:

On 2017-02-10 16:08:51 +0800 (+0800), Tom Fifield wrote:
[...]

Down again, this time with "Network is unreachable".

[...]

I'm not finding any obvious errors on the server nor
relevant
maintenance notices/trouble tickets from the service
provider to
explain this. I do see conspicuous gaps in network
traffic volume
and system load from ~06:45 to ~08:10 UTC according to
cacti:

http://cacti.openstack.org/?tree_id=1_id=156

>

Skipping back through previous days I find some similar gaps
starting anywhere from 06:30 to 07:00 and ending between
07:00 and
08:00 but they don't seem to occur every day and I'm not
having much
luck finding a pattern. It _is_ conspicuously close to when
/etc/cron.daily scripts get fired from the crontab so
might coincide
with log rotation/service restarts? The graphs don't
show these gaps
correlating with any spikes in CPU, memory or disk
activity so it
doesn't seem to be resource starvation (at least not for
any common
resources we're tracking).


Indeed. It's down again today during the same timeslot.

Another idea for the cron-based theory:



https://github.com/openstack/uc-recognition/blob/master/tools/get_active_moderator.py




>

loops through the list of Ask OpenStack users via the API on
a cron
running on www.openstack.org 
. Not sure
when that cron runs, but if it's similar, this could
potentially be
a high-load generator.




Regards,


Tom


___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org

>

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


>







___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Ask.o.o down

2017-02-14 Thread Joshua Hesketh 
Hey Tom,

Where is that script being fired from (a quick grep doesn't find it), or is
it a tool people are using?

If it's a tool we'd need to make sure whoever is using it gets a new
version to rule it out.

 - Josh

On Tue, Feb 14, 2017 at 7:07 PM, Tom Fifield  wrote:

> On 14/02/17 16:06, Joshua Hesketh wrote:
>
>> Hey,
>>
>> I've brought the service back up, but have no new clues as to why.
>>
>
> Cheers.
>
> Going to try: https://review.openstack.org/#/c/433478/
> to see if this script is culprit.
>
>
> - Josh
>>
>> On Tue, Feb 14, 2017 at 6:50 PM, Tom Fifield > > wrote:
>>
>> On 10/02/17 22:39, Jeremy Stanley wrote:
>>
>> On 2017-02-10 16:08:51 +0800 (+0800), Tom Fifield wrote:
>> [...]
>>
>> Down again, this time with "Network is unreachable".
>>
>> [...]
>>
>> I'm not finding any obvious errors on the server nor relevant
>> maintenance notices/trouble tickets from the service provider to
>> explain this. I do see conspicuous gaps in network traffic volume
>> and system load from ~06:45 to ~08:10 UTC according to cacti:
>>
>> http://cacti.openstack.org/?tree_id=1_id=156
>> 
>>
>> Skipping back through previous days I find some similar gaps
>> starting anywhere from 06:30 to 07:00 and ending between 07:00 and
>> 08:00 but they don't seem to occur every day and I'm not having
>> much
>> luck finding a pattern. It _is_ conspicuously close to when
>> /etc/cron.daily scripts get fired from the crontab so might
>> coincide
>> with log rotation/service restarts? The graphs don't show these
>> gaps
>> correlating with any spikes in CPU, memory or disk activity so it
>> doesn't seem to be resource starvation (at least not for any
>> common
>> resources we're tracking).
>>
>>
>> Indeed. It's down again today during the same timeslot.
>>
>> Another idea for the cron-based theory:
>>
>> https://github.com/openstack/uc-recognition/blob/master/tool
>> s/get_active_moderator.py
>> > ls/get_active_moderator.py>
>>
>> loops through the list of Ask OpenStack users via the API on a cron
>> running on www.openstack.org . Not sure
>> when that cron runs, but if it's similar, this could potentially be
>> a high-load generator.
>>
>>
>>
>>
>> Regards,
>>
>>
>> Tom
>>
>>
>> ___
>> OpenStack-Infra mailing list
>> OpenStack-Infra@lists.openstack.org
>> 
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>> 
>>
>>
>>
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Ask.o.o down

2017-02-14 Thread Tom Fifield 

On 14/02/17 16:06, Joshua Hesketh wrote:

Hey,

I've brought the service back up, but have no new clues as to why.


Cheers.

Going to try: https://review.openstack.org/#/c/433478/
to see if this script is culprit.



- Josh

On Tue, Feb 14, 2017 at 6:50 PM, Tom Fifield > wrote:

On 10/02/17 22:39, Jeremy Stanley wrote:

On 2017-02-10 16:08:51 +0800 (+0800), Tom Fifield wrote:
[...]

Down again, this time with "Network is unreachable".

[...]

I'm not finding any obvious errors on the server nor relevant
maintenance notices/trouble tickets from the service provider to
explain this. I do see conspicuous gaps in network traffic volume
and system load from ~06:45 to ~08:10 UTC according to cacti:

http://cacti.openstack.org/?tree_id=1_id=156


Skipping back through previous days I find some similar gaps
starting anywhere from 06:30 to 07:00 and ending between 07:00 and
08:00 but they don't seem to occur every day and I'm not having much
luck finding a pattern. It _is_ conspicuously close to when
/etc/cron.daily scripts get fired from the crontab so might coincide
with log rotation/service restarts? The graphs don't show these gaps
correlating with any spikes in CPU, memory or disk activity so it
doesn't seem to be resource starvation (at least not for any common
resources we're tracking).


Indeed. It's down again today during the same timeslot.

Another idea for the cron-based theory:


https://github.com/openstack/uc-recognition/blob/master/tools/get_active_moderator.py



loops through the list of Ask OpenStack users via the API on a cron
running on www.openstack.org . Not sure
when that cron runs, but if it's similar, this could potentially be
a high-load generator.




Regards,


Tom


___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org

http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra






___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Ask.o.o down

2017-02-14 Thread Joshua Hesketh 
Hey,

I've brought the service back up, but have no new clues as to why.

- Josh

On Tue, Feb 14, 2017 at 6:50 PM, Tom Fifield  wrote:

> On 10/02/17 22:39, Jeremy Stanley wrote:
>
>> On 2017-02-10 16:08:51 +0800 (+0800), Tom Fifield wrote:
>> [...]
>>
>>> Down again, this time with "Network is unreachable".
>>>
>> [...]
>>
>> I'm not finding any obvious errors on the server nor relevant
>> maintenance notices/trouble tickets from the service provider to
>> explain this. I do see conspicuous gaps in network traffic volume
>> and system load from ~06:45 to ~08:10 UTC according to cacti:
>>
>> http://cacti.openstack.org/?tree_id=1_id=156
>>
>> Skipping back through previous days I find some similar gaps
>> starting anywhere from 06:30 to 07:00 and ending between 07:00 and
>> 08:00 but they don't seem to occur every day and I'm not having much
>> luck finding a pattern. It _is_ conspicuously close to when
>> /etc/cron.daily scripts get fired from the crontab so might coincide
>> with log rotation/service restarts? The graphs don't show these gaps
>> correlating with any spikes in CPU, memory or disk activity so it
>> doesn't seem to be resource starvation (at least not for any common
>> resources we're tracking).
>>
>>
> Indeed. It's down again today during the same timeslot.
>
> Another idea for the cron-based theory:
>
> https://github.com/openstack/uc-recognition/blob/master/tool
> s/get_active_moderator.py
>
> loops through the list of Ask OpenStack users via the API on a cron
> running on www.openstack.org. Not sure when that cron runs, but if it's
> similar, this could potentially be a high-load generator.
>
>
>
>
> Regards,
>
>
> Tom
>
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra