subject:"Re\: \[OpenStack\-Infra\] Wiki.o.o sustaining spam attack"


On 23/03/16 13:05, Tom Fifield wrote:

On 23/03/16 11:19, Tom Fifield wrote:

On 23/03/16 00:14, Paul Belanger wrote:

On Tue, Mar 22, 2016 at 03:32:23PM +0800, Tom Fifield wrote:

Hi all,


I'm sad to say that:

* spammers are back - 100-odd pages have gone in over the weekend
https://wiki.openstack.org/wiki/Special:NewPages

* Cleanup was ineffective, with many spam pages still existing on
the wiki
(scroll through the NewPages link above)


So, just a quick update to this.  We just landed 287323[1] which
installs
SmiteSpam[2]. If you are in the bureaucrat group on wiki.o.o you'll
be able to
access it.

Currently I've started the process on cleanup the wiki.  At the
moment, I've
blocked 54[3] account and delete 430+[4] pages. I'll send a follow up
email once
I am done for the day, but moving forward it will be easier for
admins to detect
the spam and stop it faster.

[1] https://review.openstack.org/#/c/287232/
[2] https://wiki.openstack.org/wiki/Special:SmiteSpam
[3] https://wiki.openstack.org/wiki/Special:Log/block
[4] https://wiki.openstack.org/wiki/Special:Log/delete



Cheers Paul! This looks like a very handy extension. I've deleted a
couple pages already



So, *sigh*. I've been trying to use

https://wiki.openstack.org/wiki/Special:Nuke

to delete pages matching

%1%800% (there are about 5000)

and it doesn't work :(

Everything appears to be fine, but the pages don't get deleted or appear
on the deletion log.


Also, appear to have exhausted the usefulness of smitespam for our 
particular case (though I suspect it will be good for the future). It 
appears to focus on pages with lots of external links, whereas our spam 
was mainly text-based, so even though many spam pages remain they aren't 
listed at smitespam.




___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack


On 23/03/16 11:19, Tom Fifield wrote:

On 23/03/16 00:14, Paul Belanger wrote:

On Tue, Mar 22, 2016 at 03:32:23PM +0800, Tom Fifield wrote:

Hi all,


I'm sad to say that:

* spammers are back - 100-odd pages have gone in over the weekend
https://wiki.openstack.org/wiki/Special:NewPages

* Cleanup was ineffective, with many spam pages still existing on the wiki
(scroll through the NewPages link above)


So, just a quick update to this.  We just landed 287323[1] which installs
SmiteSpam[2]. If you are in the bureaucrat group on wiki.o.o you'll be able to
access it.

Currently I've started the process on cleanup the wiki.  At the moment, I've
blocked 54[3] account and delete 430+[4] pages. I'll send a follow up email once
I am done for the day, but moving forward it will be easier for admins to detect
the spam and stop it faster.

[1] https://review.openstack.org/#/c/287232/
[2] https://wiki.openstack.org/wiki/Special:SmiteSpam
[3] https://wiki.openstack.org/wiki/Special:Log/block
[4] https://wiki.openstack.org/wiki/Special:Log/delete



Cheers Paul! This looks like a very handy extension. I've deleted a
couple pages already



So, *sigh*. I've been trying to use

https://wiki.openstack.org/wiki/Special:Nuke

to delete pages matching

%1%800% (there are about 5000)

and it doesn't work :(

Everything appears to be fine, but the pages don't get deleted or appear 
on the deletion log.



___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-03-22 Thread JP Maxwell

My intention is to make a series of rapid fire changes to the config files
while tailing the log files in order to quickly suss out if the spammers
are defeating or bypassing captcha and what change would need to be
implemented to prevent this from happening. Once we actually know what is
happening and how to stop it then we would propose a permanent patch which
could be submitted through the normal processes.

J.P. Maxwell | tipit.net | fibercove.com
On Mar 22, 2016 5:39 PM, "Jeremy Stanley"  wrote:

> On 2016-03-22 08:23:08 -0500 (-0500), JP Maxwell wrote:
> > If anyone wants to approve this I am still happy to help.
> >
> > https://review.openstack.org/#/c/285641/1
>
> Can you elaborate on how you intend to help which has to be done
> first with root access to the server (rather than merely with the
> assistance of someone with root access)? The commit message on that
> change indicates you just want access to logs files, which I or
> other root sysadmins can certainly provide.
>
> We want to make sure that all modifications are reflected in
> configuration management so that it's reviewed, tracked and
> repeatable, and this is why we generally limit production server
> root access to people who also have the ability to approve
> configuration management changes for the same servers. This service
> is already in a bit of an unfortunate state because years ago we
> were less strict and in a moment of weakness allowed the MW
> deployment/migration to precede the configuration management of that
> deployment (which was subsequently never completed). We need to make
> sure its tenuous situation doesn't regress further.
>
> > I don't think you are ever going to be successful at blocking
> > accounts or IPs. You must block the creation of the spam by the
> > bots. IMHO focusing on improving the captcha or understanding the
> > bypass path around the captcha is the best short term path to
> > accomplish this.
>
> I'm pretty sure we have consensus on this already. Blocking accounts
> and manual cleanup are only viewed as a temporary workaround while
> we plan for a safe upgrade to a more recent MW (and as a
> prerequisite, more recent Ubuntu) release so that we can take
> advantage of current access control measures and similar mitigation
> solutions developed by their community in response to escalating
> advancement in defacement and valdalism on Wikipedia and elsewhere.
> --
> Jeremy Stanley
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

On 23/03/16 00:14, Paul Belanger wrote:
> On Tue, Mar 22, 2016 at 03:32:23PM +0800, Tom Fifield wrote:
>> Hi all,
>>
>>
>> I'm sad to say that:
>>
>> * spammers are back - 100-odd pages have gone in over the weekend
>> https://wiki.openstack.org/wiki/Special:NewPages
>>
>> * Cleanup was ineffective, with many spam pages still existing on the wiki
>> (scroll through the NewPages link above)
>>
> So, just a quick update to this.  We just landed 287323[1] which installs
> SmiteSpam[2]. If you are in the bureaucrat group on wiki.o.o you'll be able to
> access it.
> 
> Currently I've started the process on cleanup the wiki.  At the moment, I've
> blocked 54[3] account and delete 430+[4] pages. I'll send a follow up email 
> once
> I am done for the day, but moving forward it will be easier for admins to 
> detect
> the spam and stop it faster.
> 
> [1] https://review.openstack.org/#/c/287232/
> [2] https://wiki.openstack.org/wiki/Special:SmiteSpam
> [3] https://wiki.openstack.org/wiki/Special:Log/block
> [4] https://wiki.openstack.org/wiki/Special:Log/delete
> 

Cheers Paul! This looks like a very handy extension. I've deleted a
couple pages already

Regards,


Tom

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-03-22 Thread Jeremy Stanley

On 2016-03-22 08:23:08 -0500 (-0500), JP Maxwell wrote:
> If anyone wants to approve this I am still happy to help.
> 
> https://review.openstack.org/#/c/285641/1

Can you elaborate on how you intend to help which has to be done
first with root access to the server (rather than merely with the
assistance of someone with root access)? The commit message on that
change indicates you just want access to logs files, which I or
other root sysadmins can certainly provide.

We want to make sure that all modifications are reflected in
configuration management so that it's reviewed, tracked and
repeatable, and this is why we generally limit production server
root access to people who also have the ability to approve
configuration management changes for the same servers. This service
is already in a bit of an unfortunate state because years ago we
were less strict and in a moment of weakness allowed the MW
deployment/migration to precede the configuration management of that
deployment (which was subsequently never completed). We need to make
sure its tenuous situation doesn't regress further.

> I don't think you are ever going to be successful at blocking
> accounts or IPs. You must block the creation of the spam by the
> bots. IMHO focusing on improving the captcha or understanding the
> bypass path around the captcha is the best short term path to
> accomplish this.

I'm pretty sure we have consensus on this already. Blocking accounts
and manual cleanup are only viewed as a temporary workaround while
we plan for a safe upgrade to a more recent MW (and as a
prerequisite, more recent Ubuntu) release so that we can take
advantage of current access control measures and similar mitigation
solutions developed by their community in response to escalating
advancement in defacement and valdalism on Wikipedia and elsewhere.
-- 
Jeremy Stanley

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-03-22 Thread JP Maxwell

If anyone wants to approve this I am still happy to help.

https://review.openstack.org/#/c/285641/1

I don't think you are ever going to be successful at blocking accounts or
IPs. You must block the creation of the spam by the bots. IMHO focusing on
improving the captcha or understanding the bypass path around the captcha
is the best short term path to accomplish this.

J.P. Maxwell | tipit.net | fibercove.com
On Mar 22, 2016 8:15 AM, "Paul Belanger"  wrote:

> On Tue, Mar 22, 2016 at 03:32:23PM +0800, Tom Fifield wrote:
> > Hi all,
> >
> >
> > I'm sad to say that:
> >
> > * spammers are back - 100-odd pages have gone in over the weekend
> > https://wiki.openstack.org/wiki/Special:NewPages
> >
> > * Cleanup was ineffective, with many spam pages still existing on the
> wiki
> > (scroll through the NewPages link above)
> >
> So, we are still working through the clean up of the wiki.  Right now,
> we've
> only stopped the creation of new accounts.  Both from openid and mobile
> users.
>
> We're going to be adding SmitSpam[1] to allow admins to run some cleanup
> tools.
> But that hasn't landed yet.
>
> Until now, I am going into the wiki every few days to ban existing
> accounts that
> have already been created manually.
>
> [1] https://review.openstack.org/#/c/287232/
> >
> >
> > Regards,
> >
> >
> > Tom
> >
> >
> > On 28/02/16 01:11, JP Maxwell wrote:
> > >Elizabeth
> > >
> > >I hope you feel better.
> > >
> > >Just FYI, this is going full force in IRC right now.  I’ve bowed out as
> > >the approach I was suggesting didn’t get traction.
> > >
> > >I proposed to manually iterate on this to confirm precisely which change
> > >solves the spam problem.  Once that has been identified we can revert
> > >and come up with a proper patch.  Right now the assumption is that
> > >disabling manual accounts will solve the problem (and it might).  As a
> > >result the team is trying to solve for the consequences of not having
> > >manual accounts.  Some bots currently use manual accounts among other
> > >issues.  If the assumption is correct, these efforts will be worth it.
> > >  However, if it isn’t it will have been a great waste of energy.
> > >
> > >In any case have a good weekend everyone.  I’m off to eat some delicious
> > >central Texas BBQ!
> > >
> > >
> > >*J.P. Maxwell* | tipit.net  | fibercove.com
> > >
> > >
> > >On Sat, Feb 27, 2016 at 10:15 AM, Elizabeth K. Joseph
> > > wrote:
> > >
> > >We'll be getting together on Monday around 1700 UTC to work through
> > >this together in a debug session in #openstack-infra (I'm too sick
> > >this weekend, plus we need a time when more infra-root folks with
> > >the institutional knowledge are around).
> > >
> > >On Feb 27, 2016 05:37, "Marton Kiss"  > >> wrote:
> > >
> > >Yeah, the Settings.php was overriden by the latest puppet run.
> > >We need to wait for some infra guys to approve my patches and
> > >make it permanent:
> > >https://review.openstack.org/285669 Disable standard password
> > >based auth
> > >https://review.openstack.org/285672 Disable mobile frontend
> > >
> > >M.
> > >
> > >On Sat, Feb 27, 2016 at 2:27 PM JP Maxwell  > >> wrote:
> > >
> > >FYI. Still seeing the mobile view...
> > >
> > >J.P. Maxwell | tipit.net  | fibercove.com
> > >
> > >
> > >On Feb 27, 2016 6:53 AM, "Marton Kiss"
> > >mailto:marton.k...@gmail.com>>
> wrote:
> > >
> > >Yes, applied them manually. Let's wait a few hours, and
> > >check for new spam content / user accounts.
> > >
> > >M.
> > >JP Maxwell mailto:j...@tipit.net>>
> > >(időpont: 2016. febr. 27., Szo, 13:50) ezt írta:
> > >
> > >Cool. Are these applied? Any indication it has
> > >stopped the spam? Should we clear out these non
> > >launchpad accounts from the DB?
> > >
> > >J.P. Maxwell | tipit.net  |
> > >fibercove.com 
> > >
> > >On Feb 27, 2016 6:47 AM, "Marton Kiss"
> > > > >> wrote:
> > >
> > >And the mobile frontend will be disabled
> > >permanently with this patch:
> > >https://review.openstack.org/285672 Disable
> > >mobile frontend
> > >
> > >M.
> > >
> > >On Sat, Feb 27, 2016 at 1:39 PM Marton Kiss
> > > > >> wrote:
> > >
> > >I made some investigation, and it seems to
> > >

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-03-22 Thread Paul Belanger

On Tue, Mar 22, 2016 at 03:32:23PM +0800, Tom Fifield wrote:
> Hi all,
> 
> 
> I'm sad to say that:
> 
> * spammers are back - 100-odd pages have gone in over the weekend
> https://wiki.openstack.org/wiki/Special:NewPages
> 
> * Cleanup was ineffective, with many spam pages still existing on the wiki
> (scroll through the NewPages link above)
> 
So, just a quick update to this.  We just landed 287323[1] which installs
SmiteSpam[2]. If you are in the bureaucrat group on wiki.o.o you'll be able to
access it.

Currently I've started the process on cleanup the wiki.  At the moment, I've
blocked 54[3] account and delete 430+[4] pages. I'll send a follow up email once
I am done for the day, but moving forward it will be easier for admins to detect
the spam and stop it faster.

[1] https://review.openstack.org/#/c/287232/
[2] https://wiki.openstack.org/wiki/Special:SmiteSpam
[3] https://wiki.openstack.org/wiki/Special:Log/block
[4] https://wiki.openstack.org/wiki/Special:Log/delete

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-03-22 Thread Paul Belanger

On Tue, Mar 22, 2016 at 08:23:08AM -0500, JP Maxwell wrote:
> If anyone wants to approve this I am still happy to help.
> 
> https://review.openstack.org/#/c/285641/1
> 
Looking at the review, Jim would like to see more history of collaborting in git
first. Which is a fair requirement.

> I don't think you are ever going to be successful at blocking accounts or
> IPs. You must block the creation of the spam by the bots. IMHO focusing on
> improving the captcha or understanding the bypass path around the captcha
> is the best short term path to accomplish this.
> 
I think we're also wanting to upgrade to ubuntu trusty next and bump our
version of mediawiki too.  I understand you want to look into why captcha is
getting bypassed but I think we are in 2 camps currently, we have a security
issue (which gets resolved by patching / upgrading) or they've cracked our
captcha with humans (which means rotate our questions more often).

Something else on the plate has been the discussion to move from launchpad.net
to openstackid.org as our SSO provider, which in theory will have less spammers.

So, moving forward. We'll land 287232[1], then purge / block users created with
no passwords (these are mobile users), use SmiteSpam to clean up current
content.

Then, plan migration to ubuntu trusty sometime after summit. With latest version
of mediawiki.

[1] https://review.openstack.org/#/c/287232/

> J.P. Maxwell | tipit.net | fibercove.com
> On Mar 22, 2016 8:15 AM, "Paul Belanger"  wrote:
> 
> > On Tue, Mar 22, 2016 at 03:32:23PM +0800, Tom Fifield wrote:
> > > Hi all,
> > >
> > >
> > > I'm sad to say that:
> > >
> > > * spammers are back - 100-odd pages have gone in over the weekend
> > > https://wiki.openstack.org/wiki/Special:NewPages
> > >
> > > * Cleanup was ineffective, with many spam pages still existing on the
> > wiki
> > > (scroll through the NewPages link above)
> > >
> > So, we are still working through the clean up of the wiki.  Right now,
> > we've
> > only stopped the creation of new accounts.  Both from openid and mobile
> > users.
> >
> > We're going to be adding SmitSpam[1] to allow admins to run some cleanup
> > tools.
> > But that hasn't landed yet.
> >
> > Until now, I am going into the wiki every few days to ban existing
> > accounts that
> > have already been created manually.
> >
> > [1] https://review.openstack.org/#/c/287232/
> > >
> > >
> > > Regards,
> > >
> > >
> > > Tom
> > >
> > >
> > > On 28/02/16 01:11, JP Maxwell wrote:
> > > >Elizabeth
> > > >
> > > >I hope you feel better.
> > > >
> > > >Just FYI, this is going full force in IRC right now.  I’ve bowed out as
> > > >the approach I was suggesting didn’t get traction.
> > > >
> > > >I proposed to manually iterate on this to confirm precisely which change
> > > >solves the spam problem.  Once that has been identified we can revert
> > > >and come up with a proper patch.  Right now the assumption is that
> > > >disabling manual accounts will solve the problem (and it might).  As a
> > > >result the team is trying to solve for the consequences of not having
> > > >manual accounts.  Some bots currently use manual accounts among other
> > > >issues.  If the assumption is correct, these efforts will be worth it.
> > > >  However, if it isn’t it will have been a great waste of energy.
> > > >
> > > >In any case have a good weekend everyone.  I’m off to eat some delicious
> > > >central Texas BBQ!
> > > >
> > > >
> > > >*J.P. Maxwell* | tipit.net  | fibercove.com
> > > >
> > > >
> > > >On Sat, Feb 27, 2016 at 10:15 AM, Elizabeth K. Joseph
> > > > wrote:
> > > >
> > > >We'll be getting together on Monday around 1700 UTC to work through
> > > >this together in a debug session in #openstack-infra (I'm too sick
> > > >this weekend, plus we need a time when more infra-root folks with
> > > >the institutional knowledge are around).
> > > >
> > > >On Feb 27, 2016 05:37, "Marton Kiss"  > > >> wrote:
> > > >
> > > >Yeah, the Settings.php was overriden by the latest puppet run.
> > > >We need to wait for some infra guys to approve my patches and
> > > >make it permanent:
> > > >https://review.openstack.org/285669 Disable standard password
> > > >based auth
> > > >https://review.openstack.org/285672 Disable mobile frontend
> > > >
> > > >M.
> > > >
> > > >On Sat, Feb 27, 2016 at 2:27 PM JP Maxwell  > > >> wrote:
> > > >
> > > >FYI. Still seeing the mobile view...
> > > >
> > > >J.P. Maxwell | tipit.net  | fibercove.com
> > > >
> > > >
> > > >On Feb 27, 2016 6:53 AM, "Marton Kiss"
> > > >mailto:marton.k...@gmail.com>>
> > wrote:
> > > >
> > > >Yes, applied them manually. Let's wait a few hours, and
> > > >check for new spam content / user

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-03-22 Thread JP Maxwell

If anyone wants to approve this I am still happy to help.

https://review.openstack.org/#/c/285641/1

I don't think you are ever going to be successful at blocking accounts or
IPs. You must block the creation of the spam by the bots. IMHO focusing on
improving the captcha or understanding the bypass path around the captcha
is the best short term path to accomplish this.

J.P. Maxwell | tipit.net | fibercove.com
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-03-22 Thread Paul Belanger

On Tue, Mar 22, 2016 at 03:32:23PM +0800, Tom Fifield wrote:
> Hi all,
> 
> 
> I'm sad to say that:
> 
> * spammers are back - 100-odd pages have gone in over the weekend
> https://wiki.openstack.org/wiki/Special:NewPages
> 
> * Cleanup was ineffective, with many spam pages still existing on the wiki
> (scroll through the NewPages link above)
> 
So, we are still working through the clean up of the wiki.  Right now, we've
only stopped the creation of new accounts.  Both from openid and mobile users.

We're going to be adding SmitSpam[1] to allow admins to run some cleanup tools.
But that hasn't landed yet.

Until now, I am going into the wiki every few days to ban existing accounts that
have already been created manually.

[1] https://review.openstack.org/#/c/287232/
> 
> 
> Regards,
> 
> 
> Tom
> 
> 
> On 28/02/16 01:11, JP Maxwell wrote:
> >Elizabeth
> >
> >I hope you feel better.
> >
> >Just FYI, this is going full force in IRC right now.  I’ve bowed out as
> >the approach I was suggesting didn’t get traction.
> >
> >I proposed to manually iterate on this to confirm precisely which change
> >solves the spam problem.  Once that has been identified we can revert
> >and come up with a proper patch.  Right now the assumption is that
> >disabling manual accounts will solve the problem (and it might).  As a
> >result the team is trying to solve for the consequences of not having
> >manual accounts.  Some bots currently use manual accounts among other
> >issues.  If the assumption is correct, these efforts will be worth it.
> >  However, if it isn’t it will have been a great waste of energy.
> >
> >In any case have a good weekend everyone.  I’m off to eat some delicious
> >central Texas BBQ!
> >
> >
> >*J.P. Maxwell* | tipit.net  | fibercove.com
> >
> >
> >On Sat, Feb 27, 2016 at 10:15 AM, Elizabeth K. Joseph
> > wrote:
> >
> >We'll be getting together on Monday around 1700 UTC to work through
> >this together in a debug session in #openstack-infra (I'm too sick
> >this weekend, plus we need a time when more infra-root folks with
> >the institutional knowledge are around).
> >
> >On Feb 27, 2016 05:37, "Marton Kiss"  >> wrote:
> >
> >Yeah, the Settings.php was overriden by the latest puppet run.
> >We need to wait for some infra guys to approve my patches and
> >make it permanent:
> >https://review.openstack.org/285669 Disable standard password
> >based auth
> >https://review.openstack.org/285672 Disable mobile frontend
> >
> >M.
> >
> >On Sat, Feb 27, 2016 at 2:27 PM JP Maxwell  >> wrote:
> >
> >FYI. Still seeing the mobile view...
> >
> >J.P. Maxwell | tipit.net  | fibercove.com
> >
> >
> >On Feb 27, 2016 6:53 AM, "Marton Kiss"
> >mailto:marton.k...@gmail.com>> wrote:
> >
> >Yes, applied them manually. Let's wait a few hours, and
> >check for new spam content / user accounts.
> >
> >M.
> >JP Maxwell mailto:j...@tipit.net>>
> >(időpont: 2016. febr. 27., Szo, 13:50) ezt írta:
> >
> >Cool. Are these applied? Any indication it has
> >stopped the spam? Should we clear out these non
> >launchpad accounts from the DB?
> >
> >J.P. Maxwell | tipit.net  |
> >fibercove.com 
> >
> >On Feb 27, 2016 6:47 AM, "Marton Kiss"
> > >> wrote:
> >
> >And the mobile frontend will be disabled
> >permanently with this patch:
> >https://review.openstack.org/285672 Disable
> >mobile frontend
> >
> >M.
> >
> >On Sat, Feb 27, 2016 at 1:39 PM Marton Kiss
> > >> wrote:
> >
> >I made some investigation, and it seems to
> >be that the spam pages are created by
> >accounts registered with password accounts,
> >and the launchpad openid auth is not
> >affected at all.
> >
> >So the spam script is creating accounts like
> >this:
> >mysql> select * from user where user_name =
> >'CedricJamieson'\G;
> >*** 1. row
> >***
> >user_id: 7494
> >

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

Hi all,

I'm sad to say that:

* spammers are back - 100-odd pages have gone in over the weekend
https://wiki.openstack.org/wiki/Special:NewPages

* Cleanup was ineffective, with many spam pages still existing on the
wiki (scroll through the NewPages link above)

Regards,

Tom

On 28/02/16 01:11, JP Maxwell wrote:

Elizabeth

I hope you feel better.

Just FYI, this is going full force in IRC right now. I’ve bowed out as
the approach I was suggesting didn’t get traction.

I proposed to manually iterate on this to confirm precisely which change
solves the spam problem. Once that has been identified we can revert
and come up with a proper patch. Right now the assumption is that
disabling manual accounts will solve the problem (and it might). As a
result the team is trying to solve for the consequences of not having
manual accounts. Some bots currently use manual accounts among other
issues. If the assumption is correct, these efforts will be worth it.
However, if it isn’t it will have been a great waste of energy.

In any case have a good weekend everyone. I’m off to eat some delicious
central Texas BBQ!

*J.P. Maxwell* | tipit.net | fibercove.com

On Sat, Feb 27, 2016 at 10:15 AM, Elizabeth K. Joseph
wrote:

We'll be getting together on Monday around 1700 UTC to work through
this together in a debug session in #openstack-infra (I'm too sick
this weekend, plus we need a time when more infra-root folks with
the institutional knowledge are around).

On Feb 27, 2016 05:37, "Marton Kiss" mailto:marton.k...@gmail.com>> wrote:

Yeah, the Settings.php was overriden by the latest puppet run.
We need to wait for some infra guys to approve my patches and
make it permanent:
https://review.openstack.org/285669 Disable standard password
based auth
https://review.openstack.org/285672 Disable mobile frontend

On Sat, Feb 27, 2016 at 2:27 PM JP Maxwell mailto:j...@tipit.net>> wrote:

FYI. Still seeing the mobile view...

J.P. Maxwell | tipit.net | fibercove.com

On Feb 27, 2016 6:53 AM, "Marton Kiss"
mailto:marton.k...@gmail.com>> wrote:

Yes, applied them manually. Let's wait a few hours, and
check for new spam content / user accounts.

M.
JP Maxwell mailto:j...@tipit.net>>
(időpont: 2016. febr. 27., Szo, 13:50) ezt írta:

Cool. Are these applied? Any indication it has
stopped the spam? Should we clear out these non
launchpad accounts from the DB?

J.P. Maxwell | tipit.net |
fibercove.com

On Feb 27, 2016 6:47 AM, "Marton Kiss"
mailto:marton.k...@gmail.com>> wrote:

And the mobile frontend will be disabled
permanently with this patch:
https://review.openstack.org/285672 Disable
mobile frontend

On Sat, Feb 27, 2016 at 1:39 PM Marton Kiss
mailto:marton.k...@gmail.com>> wrote:

I made some investigation, and it seems to
be that the spam pages are created by
accounts registered with password accounts,
and the launchpad openid auth is not
affected at all.

So the spam script is creating accounts like
this:
mysql> select * from user where user_name =
'CedricJamieson'\G;
*** 1. row
***
user_id: 7494
user_name: CedricJamieson
user_real_name: Cedric Jamieson
user_password:

:pbkdf2:sha256:1:128:Mlo9tdaP+38niZrrEka7Ow==:jEVnrTclkwIpE1RzJywDlrSvkY5G3idYwOwYRkv5O0J/MSHjY+gdhtKmArQ53v6/w7o8E1wXb2QOR6HdL5TPfOI1bswS/fYXVVYjPjkEEdxqZ8q9L5p2f3N6rEYpMfT5tk+wDiy+j5aimrHrGSga44hndAHgX9/SnqUyxlutDVY=
user_newpassword:
user_newpass_time: NULL
user_email: balashkina.evdok...@mail.ru

user_touched: 20160227052454
user_token: 7c39e44e849fb0e2bfae8790d6cc1379
user_email_authenticated: NULL
user_email_token:

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

Ok, I'll be there.

M.

On Sat, Feb 27, 2016 at 5:15 PM Elizabeth K. Joseph 
wrote:

> We'll be getting together on Monday around 1700 UTC to work through this
> together in a debug session in #openstack-infra (I'm too sick this weekend,
> plus we need a time when more infra-root folks with the institutional
> knowledge are around).
> On Feb 27, 2016 05:37, "Marton Kiss"  wrote:
>
>> Yeah, the Settings.php was overriden by the latest puppet run. We need to
>> wait for some infra guys to approve my patches and make it permanent:
>> https://review.openstack.org/285669 Disable standard password based auth
>> https://review.openstack.org/285672 Disable mobile frontend
>>
>> M.
>>
>> On Sat, Feb 27, 2016 at 2:27 PM JP Maxwell  wrote:
>>
>>> FYI. Still seeing the mobile view...
>>>
>>> J.P. Maxwell | tipit.net | fibercove.com
>>> On Feb 27, 2016 6:53 AM, "Marton Kiss"  wrote:
>>>
 Yes, applied them manually. Let's wait a few hours, and check for new
 spam content / user accounts.

 M.
 JP Maxwell  (időpont: 2016. febr. 27., Szo, 13:50) ezt
 írta:

> Cool. Are these applied? Any indication it has stopped the spam?
> Should we clear out these non launchpad accounts from the DB?
>
> J.P. Maxwell | tipit.net | fibercove.com
> On Feb 27, 2016 6:47 AM, "Marton Kiss"  wrote:
>
>> And the mobile frontend will be disabled permanently with this patch:
>> https://review.openstack.org/285672 Disable mobile frontend
>>
>> M.
>>
>> On Sat, Feb 27, 2016 at 1:39 PM Marton Kiss 
>> wrote:
>>
>>> I made some investigation, and it seems to be that the spam pages
>>> are created by accounts registered with password accounts, and the
>>> launchpad openid auth is not affected at all.
>>>
>>> So the spam script is creating accounts like this:
>>> mysql> select * from user where user_name = 'CedricJamieson'\G;
>>> *** 1. row ***
>>>  user_id: 7494
>>>user_name: CedricJamieson
>>>   user_real_name: Cedric Jamieson
>>>user_password:
>>> :pbkdf2:sha256:1:128:Mlo9tdaP+38niZrrEka7Ow==:jEVnrTclkwIpE1RzJywDlrSvkY5G3idYwOwYRkv5O0J/MSHjY+gdhtKmArQ53v6/w7o8E1wXb2QOR6HdL5TPfOI1bswS/fYXVVYjPjkEEdxqZ8q9L5p2f3N6rEYpMfT5tk+wDiy+j5aimrHrGSga44hndAHgX9/SnqUyxlutDVY=
>>> user_newpassword:
>>>user_newpass_time: NULL
>>>   user_email: balashkina.evdok...@mail.ru
>>> user_touched: 20160227052454
>>>   user_token: 7c39e44e849fb0e2bfae8790d6cc1379
>>> user_email_authenticated: NULL
>>> user_email_token: be963ac3bd43e70ff2f323063c61e320
>>> user_email_token_expires: 20160305052441
>>>user_registration: 20160227052441
>>>   user_editcount: 2
>>>user_password_expires: NULL
>>>
>>> The user_password field is always filled with a value, meanwhile
>>> this field of non-infected user accounts with openid logins is empty.
>>> We have 423 total accounts with passwords:
>>> mysql> select count(*) from user where user_password != '';
>>> +--+
>>> | count(*) |
>>> +--+
>>> |  423 |
>>> +--+
>>> 1 row in set (0.00 sec)
>>>
>>> Mediawiki logs-in the newly created users without any preliminary
>>> email confirmation, right after the registration. I disabled the 
>>> standard
>>> user login page, as described here:
>>> https://www.mediawiki.org/wiki/Manual:Special_pages#Disabling_Special:UserLogin_and_Special:UserLogout_pages
>>>
>>> And I made this patch to make it permanent:
>>> https://review.openstack.org/285669 Disable standard password based
>>> auth
>>>
>>> Just for the record, the last spam user account:
>>> 7536 | EarthaChester22
>>>
>>> Marton
>>>
>>>
>>> On Sat, Feb 27, 2016 at 8:31 AM Marton Kiss 
>>> wrote:
>>>
 Hi,

 I created the following patch, infra cores must approve that:
 https://review.openstack.org/285641 Add ssh key of JP Maxwell to
 wiki.o.o

 Marton

 On Sat, Feb 27, 2016 at 6:41 AM JP Maxwell  wrote:

> Marton has SSH access and applied a patch earlier today.  It
> appears the spam continues to flow:
>
>
> https://wiki.openstack.org/wiki/40_Thoughts_Of_Using_Open_Shelves_On_A_Kitchen
>
> Marton let me know if you can look at it some more or Infra if you
> want to give me SSH I'll do so as well in the morning (public key
> attached).
>
>
>
> ssh-rsa
> B3NzaC1yc2EBIwAAAQEA2b5I7Yff9FCrtRmSjpILUePi54Vbc8zqJTbzrIAQZGFLBi3xd2MLlhV5QVgpDBC9H3lGjbdnc81D3aFd3HwHT4dvvvyedT12PR3VDEpftdW84vw3jzdtALcayOQznjbGnScwvX5SgnRhNxuX9Rkh8qNvOsjYPUafRr9azkQoomJFkdNVI4Vb5D

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-27 Thread JP Maxwell

Elizabeth
I hope you feel better.
Just FYI, this is going full force in IRC right now. I’ve bowed out as the
approach I was suggesting didn’t get traction.
I proposed to manually iterate on this to confirm precisely which change solves
the spam problem. Once that has been identified we can revert and come up with a
proper patch. Right now the assumption is that disabling manual accounts will
solve the problem (and it might). As a result the team is trying to solve for
the consequences of not having manual accounts. Some bots currently use manual
accounts among other issues. If the assumption is correct, these efforts will be
worth it. However, if it isn’t it will have been a great waste of energy.
In any case have a good weekend everyone. I’m off to eat some delicious central
Texas BBQ!

J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com 
[http://www.fibercove.com]
On Sat, Feb 27, 2016 at 10:15 AM, Elizabeth K. Joseph 
wrote:
We'll be getting together on Monday around 1700 UTC to work through this
together in a debug session in #openstack-infra (I'm too sick this weekend, plus
we need a time when more infra-root folks with the institutional knowledge are
around).

On Feb 27, 2016 05:37, "Marton Kiss" < marton.k...@gmail.com 
[marton.k...@gmail.com] > wrote:
Yeah, the Settings.php was overriden by the latest puppet run. We need to wait
for some infra guys to approve my patches and make it permanent: 
https://review.openstack.org/285669 [https://review.openstack.org/285669] 
Disable standard password based auth
https://review.openstack.org/285672 [https://review.openstack.org/285672] 
Disable mobile frontend
M.
On Sat, Feb 27, 2016 at 2:27 PM JP Maxwell < j...@tipit.net [j...@tipit.net] > 
wrote:
FYI. Still seeing the mobile view...

J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com 
[http://fibercove.com]

On Feb 27, 2016 6:53 AM, "Marton Kiss" < marton.k...@gmail.com 
[marton.k...@gmail.com] > wrote:
Yes, applied them manually. Let's wait a few hours, and check for new spam
content / user accounts.

M.
JP Maxwell < j...@tipit.net [j...@tipit.net] > (időpont: 2016. febr. 27., Szo, 
13:50) ezt írta:
Cool. Are these applied? Any indication it has stopped the spam? Should we clear
out these non launchpad accounts from the DB?

J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com 
[http://fibercove.com]

On Feb 27, 2016 6:47 AM, "Marton Kiss" < marton.k...@gmail.com 
[marton.k...@gmail.com] > wrote:
And the mobile frontend will be disabled permanently with this patch: 
https://review.openstack.org/285672 [https://review.openstack.org/285672] 
Disable mobile frontend

M.
On Sat, Feb 27, 2016 at 1:39 PM Marton Kiss < marton.k...@gmail.com 
[marton.k...@gmail.com] > wrote:
I made some investigation, and it seems to be that the spam pages are created by
accounts registered with password accounts, and the launchpad openid auth is not
affected at all.
So the spam script is creating accounts like this: mysql> select * from user 
where user_name = 'CedricJamieson'\G; *** 1. row 
*** user_id: 7494 user_name: CedricJamieson 
user_real_name: Cedric Jamieson user_password:
:pbkdf2:sha256:1:128:Mlo9tdaP+38niZrrEka7Ow==:jEVnrTclkwIpE1RzJywDlrSvkY5G3idYwOwYRkv5O0J/MSHjY+gdhtKmArQ53v6/w7o8E1wXb2QOR6HdL5TPfOI1bswS/fYXVVYjPjkEEdxqZ8q9L5p2f3N6rEYpMfT5tk+wDiy+j5aimrHrGSga44hndAHgX9/SnqUyxlutDVY=
 user_newpassword: user_newpass_time: NULL user_email: 
balashkina.evdok...@mail.ru [balashkina.evdok...@mail.ru] user_touched: 
20160227052454 user_token: 7c39e44e849fb0e2bfae8790d6cc1379 
user_email_authenticated: NULL user_email_token: 
be963ac3bd43e70ff2f323063c61e320 user_email_token_expires: 20160305052441 
user_registration: 20160227052441 user_editcount: 2 user_password_expires: NULL
The user_password field is always filled with a value, meanwhile this field of
non-infected user accounts with openid logins is empty. We have 423 total 
accounts with passwords: mysql> select count(*) from user where user_password 
!= ''; +--+ | count(*) | +--+ | 423 | +--+ 1 row in set 
(0.00 sec)
Mediawiki logs-in the newly created users without any preliminary email
confirmation, right after the registration. I disabled the standard user login
page, as described here: 
https://www.mediawiki.org/wiki/Manual:Special_pages#Disabling_Special:UserLogin_and_Special:UserLogout_pages
[https://www.mediawiki.org/wiki/Manual:Special_pages#Disabling_Special:UserLogin_and_Special:UserLogout_pages]
And I made this patch to make it permanent: https://review.openstack.org/285669 
[https://review.openstack.org/285669] Disable standard password based auth

Just for the record, the last spam user account: 7536 | EarthaChester22

Marton

On Sat, Feb 27, 2016 at 8:31 AM Marton Kiss < marton.k...@gmail.com 
[marton.k...@gmail.com] > wrote:
Hi,
I created the following patch, infra cores must approve that: 
https://review.openstack.org/285641 [https://review.openstack.org

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-27 Thread Elizabeth K. Joseph

We'll be getting together on Monday around 1700 UTC to work through this
together in a debug session in #openstack-infra (I'm too sick this weekend,
plus we need a time when more infra-root folks with the institutional
knowledge are around).
On Feb 27, 2016 05:37, "Marton Kiss"  wrote:

> Yeah, the Settings.php was overriden by the latest puppet run. We need to
> wait for some infra guys to approve my patches and make it permanent:
> https://review.openstack.org/285669 Disable standard password based auth
> https://review.openstack.org/285672 Disable mobile frontend
>
> M.
>
> On Sat, Feb 27, 2016 at 2:27 PM JP Maxwell  wrote:
>
>> FYI. Still seeing the mobile view...
>>
>> J.P. Maxwell | tipit.net | fibercove.com
>> On Feb 27, 2016 6:53 AM, "Marton Kiss"  wrote:
>>
>>> Yes, applied them manually. Let's wait a few hours, and check for new
>>> spam content / user accounts.
>>>
>>> M.
>>> JP Maxwell  (időpont: 2016. febr. 27., Szo, 13:50) ezt
>>> írta:
>>>
 Cool. Are these applied? Any indication it has stopped the spam? Should
 we clear out these non launchpad accounts from the DB?

 J.P. Maxwell | tipit.net | fibercove.com
 On Feb 27, 2016 6:47 AM, "Marton Kiss"  wrote:

> And the mobile frontend will be disabled permanently with this patch:
> https://review.openstack.org/285672 Disable mobile frontend
>
> M.
>
> On Sat, Feb 27, 2016 at 1:39 PM Marton Kiss 
> wrote:
>
>> I made some investigation, and it seems to be that the spam pages are
>> created by accounts registered with password accounts, and the launchpad
>> openid auth is not affected at all.
>>
>> So the spam script is creating accounts like this:
>> mysql> select * from user where user_name = 'CedricJamieson'\G;
>> *** 1. row ***
>>  user_id: 7494
>>user_name: CedricJamieson
>>   user_real_name: Cedric Jamieson
>>user_password:
>> :pbkdf2:sha256:1:128:Mlo9tdaP+38niZrrEka7Ow==:jEVnrTclkwIpE1RzJywDlrSvkY5G3idYwOwYRkv5O0J/MSHjY+gdhtKmArQ53v6/w7o8E1wXb2QOR6HdL5TPfOI1bswS/fYXVVYjPjkEEdxqZ8q9L5p2f3N6rEYpMfT5tk+wDiy+j5aimrHrGSga44hndAHgX9/SnqUyxlutDVY=
>> user_newpassword:
>>user_newpass_time: NULL
>>   user_email: balashkina.evdok...@mail.ru
>> user_touched: 20160227052454
>>   user_token: 7c39e44e849fb0e2bfae8790d6cc1379
>> user_email_authenticated: NULL
>> user_email_token: be963ac3bd43e70ff2f323063c61e320
>> user_email_token_expires: 20160305052441
>>user_registration: 20160227052441
>>   user_editcount: 2
>>user_password_expires: NULL
>>
>> The user_password field is always filled with a value, meanwhile this
>> field of non-infected user accounts with openid logins is empty.
>> We have 423 total accounts with passwords:
>> mysql> select count(*) from user where user_password != '';
>> +--+
>> | count(*) |
>> +--+
>> |  423 |
>> +--+
>> 1 row in set (0.00 sec)
>>
>> Mediawiki logs-in the newly created users without any preliminary
>> email confirmation, right after the registration. I disabled the standard
>> user login page, as described here:
>> https://www.mediawiki.org/wiki/Manual:Special_pages#Disabling_Special:UserLogin_and_Special:UserLogout_pages
>>
>> And I made this patch to make it permanent:
>> https://review.openstack.org/285669 Disable standard password based
>> auth
>>
>> Just for the record, the last spam user account:
>> 7536 | EarthaChester22
>>
>> Marton
>>
>>
>> On Sat, Feb 27, 2016 at 8:31 AM Marton Kiss 
>> wrote:
>>
>>> Hi,
>>>
>>> I created the following patch, infra cores must approve that:
>>> https://review.openstack.org/285641 Add ssh key of JP Maxwell to
>>> wiki.o.o
>>>
>>> Marton
>>>
>>> On Sat, Feb 27, 2016 at 6:41 AM JP Maxwell  wrote:
>>>
 Marton has SSH access and applied a patch earlier today.  It
 appears the spam continues to flow:


 https://wiki.openstack.org/wiki/40_Thoughts_Of_Using_Open_Shelves_On_A_Kitchen

 Marton let me know if you can look at it some more or Infra if you
 want to give me SSH I'll do so as well in the morning (public key
 attached).



 ssh-rsa
 B3NzaC1yc2EBIwAAAQEA2b5I7Yff9FCrtRmSjpILUePi54Vbc8zqJTbzrIAQZGFLBi3xd2MLlhV5QVgpDBC9H3lGjbdnc81D3aFd3HwHT4dvvvyedT12PR3VDEpftdW84vw3jzdtALcayOQznjbGnScwvX5SgnRhNxuX9Rkh8qNvOsjYPUafRr9azkQoomJFkdNVI4Vb5DbLhTpt18FPeOf0UuqDt/J2tHI4SjZ3kjzr7Nbwpg8xGgANPNE0+2pJbwCA8YDt4g3bzfzvVafQs5o9Gfc9tudkR9ugQG1M+EWCgu42CleOwMTd/rYEB2fgNNPsZAWqwQfdPajVuk70EBKUEQSyoA09eEZX+xJN9Q==
 jpmax...@tipit.net

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-27 Thread Paul Belanger

On phone but patch puppet-mediawiki and enable captcha for all pages. We only 
did edit and create On Feb 26, 2016 10:38 AM, Marton Kiss 
 wrote:
I see a ton of incoming post requests:

POST
/w/index.php?title=Special%3ARunJobs&tasks=jobs&maxjobs=1&sigexpiry=1456508270&signature=571cfb216f944b15d2eee1c0253d08b77003328e

M.

On Fri, Feb 26, 2016 at 6:35 PM Marton Kiss  wrote:

> Oh, I can login. So what we need?
>
> M.
>
> On Fri, Feb 26, 2016 at 6:33 PM JP Maxwell  wrote:
>
>> I think what Jimmy is referring to is what I was suggesting by removing
>> the extensions / making the question impossible to answer.  Basically a
>> series of rapid fire changes while tailing the logs and seeing what stops
>> the spam.  Once you know what worked then you can submit as an official
>> patch.  But being able to quickly try these things on a server actually
>> under attack is the fastest path toward identifying the fix.
>>
>> *J.P. Maxwell* | tipit.net | fibercove.com 
>>
>> On Fri, Feb 26, 2016 at 11:25 AM, Paul Belanger 
>> wrote:
>>
>> On Fri, Feb 26, 2016 at 11:08:18AM -0600, Jimmy McArthur wrote:
>> > Given the state of the wiki a the moment, I think taking the quickest
>> path
>> > to get it fixed would be prudent. Is there a way we can get JP root
>> access
>> > to this server, even temporarily? We get 25% of our website traffic (2
>> > million visitors) to the wiki. I realize we're all after the same
>> thing, but
>> > spammers are not going to hit the dev environment, so there's really no
>> way
>> > to tell if teh problem is fixed without actually working directly on the
>> > production machine. This should be a 30 minute fix.
>> >
>> I am still unclear what the 30min fix is. If really 30mins, then it
>> shouldn't be
>> hard to get the fix into our workflow. Could somebody please elaborate.
>>
>> If we are talking about deploying new versions of php or mediawiki
>> manually, I
>> not be in-favor of this. To me, while the attack sucks, we should be
>> working on
>> 2 fronts. Getting the help needed to mitigate the attack, then adding the
>> changes into -infra workflow in parallel.
>>
>> > I realize there is a lot of risk in giving ssh access to infra
>> machines, but
>> > I think it's worth taking a look at either putting this machine in a
>> place
>> > where a different level of admin could access it without giving away the
>> > keys to the entire OpenStack infrastructure or figuring out a way to
>> set up
>> > credentials with varying levels of access.
>> >
>> As a note, all the work I've been doing to help with the attack hasn't
>> require
>> SSH access for me to wiki.o.o. I did need infra-root help to expose our
>> configuration safely. I'd rather take some time to see what the fixes are,
>> having infra-root apply changes, then move them into puppet.
>>
>> It also has been discussed to simply disable write access to the wiki if
>> we
>> really want spamming to stop, obviously that will affect normal usage.
>>
>> > Jimmy
>> >
>> > Paul Belanger wrote:
>> > >On Fri, Feb 26, 2016 at 10:12:12AM -0600, JP Maxwell wrote:
>> > >>But if you wanted to upgrade everything, remove the mobile view
>> extension,
>> > >>test in a dev/staging environment then deploy to production fingers
>> > >>crossed, I think that would be a valid approach as well.
>> > >>
>> > >Current review up[1]. I'll launch a node tonight / tomorrow locally to
>> see how
>> > >puppet reacts. I suspect there will be some issues.
>> > >
>> > >If infra-roots are fine with this approach, we can use that box to
>> test against.
>> > >
>> > >[1] https://review.openstack.org/#/c/285405/
>> > >
>> > >>J.P. Maxwell | tipit.net | fibercove.com
>> > >>On Feb 26, 2016 10:08 AM, "JP Maxwell" wrote:
>> > >>
>> > >>>Plus one except in this case it is much easier to know if our
>> efforts are
>> > >>>working on production because the spam either stops or not.
>> > >>>
>> > >>>J.P. Maxwell | tipit.net | fibercove.com
>> > >>>On Feb 26, 2016 9:48 AM, "Paul Belanger"
>> wrote:
>> > >>>
>> > On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote:
>> > >I really think you might consider the option that there is a
>> > vulnerability
>> > >in one of the extensions. If that is the case black listing IPs
>> will be
>> > an
>> > >ongoing wild goose chase.
>> > >
>> > >I think this would be easily proven or disproven by making the
>> questy
>> > >question impossible and see if the spam continues.
>> > >
>> > We'll have to let an infra-root make that call. Since nobody would
>> be
>> > able to
>> > use the wiki. Honestly, I'd rather spend the time standing up a
>> mirror dev
>> > instance for us to work on, rather then production.
>> > 
>> > >J.P. Maxwell | tipit.net | fibercove.com
>> > >On Feb 26, 2016 9:12 AM, "Paul Belanger"
>> wrote:
>> > >
>> > >>On Thu, Feb 25, 2016 at 08:10:34PM -0800, Elizabeth K. Joseph
>> wrote:
>> > >>>On Thu, Feb 25, 2016 at 6:35 AM, Jer

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

Yeah, the Settings.php was overriden by the latest puppet run. We need to
wait for some infra guys to approve my patches and make it permanent:
https://review.openstack.org/285669 Disable standard password based auth
https://review.openstack.org/285672 Disable mobile frontend

M.

On Sat, Feb 27, 2016 at 2:27 PM JP Maxwell  wrote:

> FYI. Still seeing the mobile view...
>
> J.P. Maxwell | tipit.net | fibercove.com
> On Feb 27, 2016 6:53 AM, "Marton Kiss"  wrote:
>
>> Yes, applied them manually. Let's wait a few hours, and check for new
>> spam content / user accounts.
>>
>> M.
>> JP Maxwell  (időpont: 2016. febr. 27., Szo, 13:50) ezt
>> írta:
>>
>>> Cool. Are these applied? Any indication it has stopped the spam? Should
>>> we clear out these non launchpad accounts from the DB?
>>>
>>> J.P. Maxwell | tipit.net | fibercove.com
>>> On Feb 27, 2016 6:47 AM, "Marton Kiss"  wrote:
>>>
 And the mobile frontend will be disabled permanently with this patch:
 https://review.openstack.org/285672 Disable mobile frontend

 M.

 On Sat, Feb 27, 2016 at 1:39 PM Marton Kiss 
 wrote:

> I made some investigation, and it seems to be that the spam pages are
> created by accounts registered with password accounts, and the launchpad
> openid auth is not affected at all.
>
> So the spam script is creating accounts like this:
> mysql> select * from user where user_name = 'CedricJamieson'\G;
> *** 1. row ***
>  user_id: 7494
>user_name: CedricJamieson
>   user_real_name: Cedric Jamieson
>user_password:
> :pbkdf2:sha256:1:128:Mlo9tdaP+38niZrrEka7Ow==:jEVnrTclkwIpE1RzJywDlrSvkY5G3idYwOwYRkv5O0J/MSHjY+gdhtKmArQ53v6/w7o8E1wXb2QOR6HdL5TPfOI1bswS/fYXVVYjPjkEEdxqZ8q9L5p2f3N6rEYpMfT5tk+wDiy+j5aimrHrGSga44hndAHgX9/SnqUyxlutDVY=
> user_newpassword:
>user_newpass_time: NULL
>   user_email: balashkina.evdok...@mail.ru
> user_touched: 20160227052454
>   user_token: 7c39e44e849fb0e2bfae8790d6cc1379
> user_email_authenticated: NULL
> user_email_token: be963ac3bd43e70ff2f323063c61e320
> user_email_token_expires: 20160305052441
>user_registration: 20160227052441
>   user_editcount: 2
>user_password_expires: NULL
>
> The user_password field is always filled with a value, meanwhile this
> field of non-infected user accounts with openid logins is empty.
> We have 423 total accounts with passwords:
> mysql> select count(*) from user where user_password != '';
> +--+
> | count(*) |
> +--+
> |  423 |
> +--+
> 1 row in set (0.00 sec)
>
> Mediawiki logs-in the newly created users without any preliminary
> email confirmation, right after the registration. I disabled the standard
> user login page, as described here:
> https://www.mediawiki.org/wiki/Manual:Special_pages#Disabling_Special:UserLogin_and_Special:UserLogout_pages
>
> And I made this patch to make it permanent:
> https://review.openstack.org/285669 Disable standard password based
> auth
>
> Just for the record, the last spam user account:
> 7536 | EarthaChester22
>
> Marton
>
>
> On Sat, Feb 27, 2016 at 8:31 AM Marton Kiss 
> wrote:
>
>> Hi,
>>
>> I created the following patch, infra cores must approve that:
>> https://review.openstack.org/285641 Add ssh key of JP Maxwell to
>> wiki.o.o
>>
>> Marton
>>
>> On Sat, Feb 27, 2016 at 6:41 AM JP Maxwell  wrote:
>>
>>> Marton has SSH access and applied a patch earlier today.  It appears
>>> the spam continues to flow:
>>>
>>>
>>> https://wiki.openstack.org/wiki/40_Thoughts_Of_Using_Open_Shelves_On_A_Kitchen
>>>
>>> Marton let me know if you can look at it some more or Infra if you
>>> want to give me SSH I'll do so as well in the morning (public key
>>> attached).
>>>
>>>
>>>
>>> ssh-rsa
>>> B3NzaC1yc2EBIwAAAQEA2b5I7Yff9FCrtRmSjpILUePi54Vbc8zqJTbzrIAQZGFLBi3xd2MLlhV5QVgpDBC9H3lGjbdnc81D3aFd3HwHT4dvvvyedT12PR3VDEpftdW84vw3jzdtALcayOQznjbGnScwvX5SgnRhNxuX9Rkh8qNvOsjYPUafRr9azkQoomJFkdNVI4Vb5DbLhTpt18FPeOf0UuqDt/J2tHI4SjZ3kjzr7Nbwpg8xGgANPNE0+2pJbwCA8YDt4g3bzfzvVafQs5o9Gfc9tudkR9ugQG1M+EWCgu42CleOwMTd/rYEB2fgNNPsZAWqwQfdPajVuk70EBKUEQSyoA09eEZX+xJN9Q==
>>> jpmax...@tipit.net
>>>
>>>
>>>
>>>
>>> J.P. Maxwell / tipit.net 
>>>
>>>
>>> On Fri, Feb 26, 2016 at 12:09 PM, Jimmy McArthur <
>>> ji...@openstack.org> wrote:
>>>
 Super thankful for all the folks that have jumped in over the last
 couple of days to help with the puppetization, etc... I just feel like
 we're taking a very wrong approach here.

 Pa

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-27 Thread JP Maxwell

FYI. Still seeing the mobile view...

J.P. Maxwell | tipit.net | fibercove.com
On Feb 27, 2016 6:53 AM, "Marton Kiss"  wrote:

> Yes, applied them manually. Let's wait a few hours, and check for new spam
> content / user accounts.
>
> M.
> JP Maxwell  (időpont: 2016. febr. 27., Szo, 13:50) ezt írta:
>
>> Cool. Are these applied? Any indication it has stopped the spam? Should
>> we clear out these non launchpad accounts from the DB?
>>
>> J.P. Maxwell | tipit.net | fibercove.com
>> On Feb 27, 2016 6:47 AM, "Marton Kiss"  wrote:
>>
>>> And the mobile frontend will be disabled permanently with this patch:
>>> https://review.openstack.org/285672 Disable mobile frontend
>>>
>>> M.
>>>
>>> On Sat, Feb 27, 2016 at 1:39 PM Marton Kiss 
>>> wrote:
>>>
 I made some investigation, and it seems to be that the spam pages are
 created by accounts registered with password accounts, and the launchpad
 openid auth is not affected at all.

 So the spam script is creating accounts like this:
 mysql> select * from user where user_name = 'CedricJamieson'\G;
 *** 1. row ***
  user_id: 7494
user_name: CedricJamieson
   user_real_name: Cedric Jamieson
user_password:
 :pbkdf2:sha256:1:128:Mlo9tdaP+38niZrrEka7Ow==:jEVnrTclkwIpE1RzJywDlrSvkY5G3idYwOwYRkv5O0J/MSHjY+gdhtKmArQ53v6/w7o8E1wXb2QOR6HdL5TPfOI1bswS/fYXVVYjPjkEEdxqZ8q9L5p2f3N6rEYpMfT5tk+wDiy+j5aimrHrGSga44hndAHgX9/SnqUyxlutDVY=
 user_newpassword:
user_newpass_time: NULL
   user_email: balashkina.evdok...@mail.ru
 user_touched: 20160227052454
   user_token: 7c39e44e849fb0e2bfae8790d6cc1379
 user_email_authenticated: NULL
 user_email_token: be963ac3bd43e70ff2f323063c61e320
 user_email_token_expires: 20160305052441
user_registration: 20160227052441
   user_editcount: 2
user_password_expires: NULL

 The user_password field is always filled with a value, meanwhile this
 field of non-infected user accounts with openid logins is empty.
 We have 423 total accounts with passwords:
 mysql> select count(*) from user where user_password != '';
 +--+
 | count(*) |
 +--+
 |  423 |
 +--+
 1 row in set (0.00 sec)

 Mediawiki logs-in the newly created users without any preliminary email
 confirmation, right after the registration. I disabled the standard user
 login page, as described here:
 https://www.mediawiki.org/wiki/Manual:Special_pages#Disabling_Special:UserLogin_and_Special:UserLogout_pages

 And I made this patch to make it permanent:
 https://review.openstack.org/285669 Disable standard password based
 auth

 Just for the record, the last spam user account:
 7536 | EarthaChester22

 Marton


 On Sat, Feb 27, 2016 at 8:31 AM Marton Kiss 
 wrote:

> Hi,
>
> I created the following patch, infra cores must approve that:
> https://review.openstack.org/285641 Add ssh key of JP Maxwell to
> wiki.o.o
>
> Marton
>
> On Sat, Feb 27, 2016 at 6:41 AM JP Maxwell  wrote:
>
>> Marton has SSH access and applied a patch earlier today.  It appears
>> the spam continues to flow:
>>
>>
>> https://wiki.openstack.org/wiki/40_Thoughts_Of_Using_Open_Shelves_On_A_Kitchen
>>
>> Marton let me know if you can look at it some more or Infra if you
>> want to give me SSH I'll do so as well in the morning (public key
>> attached).
>>
>>
>>
>> ssh-rsa
>> B3NzaC1yc2EBIwAAAQEA2b5I7Yff9FCrtRmSjpILUePi54Vbc8zqJTbzrIAQZGFLBi3xd2MLlhV5QVgpDBC9H3lGjbdnc81D3aFd3HwHT4dvvvyedT12PR3VDEpftdW84vw3jzdtALcayOQznjbGnScwvX5SgnRhNxuX9Rkh8qNvOsjYPUafRr9azkQoomJFkdNVI4Vb5DbLhTpt18FPeOf0UuqDt/J2tHI4SjZ3kjzr7Nbwpg8xGgANPNE0+2pJbwCA8YDt4g3bzfzvVafQs5o9Gfc9tudkR9ugQG1M+EWCgu42CleOwMTd/rYEB2fgNNPsZAWqwQfdPajVuk70EBKUEQSyoA09eEZX+xJN9Q==
>> jpmax...@tipit.net
>>
>>
>>
>>
>> J.P. Maxwell / tipit.net 
>>
>>
>> On Fri, Feb 26, 2016 at 12:09 PM, Jimmy McArthur > > wrote:
>>
>>> Super thankful for all the folks that have jumped in over the last
>>> couple of days to help with the puppetization, etc... I just feel like
>>> we're taking a very wrong approach here.
>>>
>>> Paul Belanger wrote:
>>>
>>> Right, and I don't have an issue with that approach.  Based on the work 
>>> we did
>>> yesterday, anybody can do that via our workflow. Please submit a patch 
>>> to
>>> puppet-mediawiki[1] and ping an infra-root in #openstack-infra IRC.
>>>
>>> What I'm proposing is the workflow is really meant for software, not
>>> for web applications. It's tedious and time consuming when what's needed
>>> here is a set of

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

Yes, applied them manually. Let's wait a few hours, and check for new spam
content / user accounts.

M.
JP Maxwell  (időpont: 2016. febr. 27., Szo, 13:50) ezt írta:

> Cool. Are these applied? Any indication it has stopped the spam? Should we
> clear out these non launchpad accounts from the DB?
>
> J.P. Maxwell | tipit.net | fibercove.com
> On Feb 27, 2016 6:47 AM, "Marton Kiss"  wrote:
>
>> And the mobile frontend will be disabled permanently with this patch:
>> https://review.openstack.org/285672 Disable mobile frontend
>>
>> M.
>>
>> On Sat, Feb 27, 2016 at 1:39 PM Marton Kiss 
>> wrote:
>>
>>> I made some investigation, and it seems to be that the spam pages are
>>> created by accounts registered with password accounts, and the launchpad
>>> openid auth is not affected at all.
>>>
>>> So the spam script is creating accounts like this:
>>> mysql> select * from user where user_name = 'CedricJamieson'\G;
>>> *** 1. row ***
>>>  user_id: 7494
>>>user_name: CedricJamieson
>>>   user_real_name: Cedric Jamieson
>>>user_password:
>>> :pbkdf2:sha256:1:128:Mlo9tdaP+38niZrrEka7Ow==:jEVnrTclkwIpE1RzJywDlrSvkY5G3idYwOwYRkv5O0J/MSHjY+gdhtKmArQ53v6/w7o8E1wXb2QOR6HdL5TPfOI1bswS/fYXVVYjPjkEEdxqZ8q9L5p2f3N6rEYpMfT5tk+wDiy+j5aimrHrGSga44hndAHgX9/SnqUyxlutDVY=
>>> user_newpassword:
>>>user_newpass_time: NULL
>>>   user_email: balashkina.evdok...@mail.ru
>>> user_touched: 20160227052454
>>>   user_token: 7c39e44e849fb0e2bfae8790d6cc1379
>>> user_email_authenticated: NULL
>>> user_email_token: be963ac3bd43e70ff2f323063c61e320
>>> user_email_token_expires: 20160305052441
>>>user_registration: 20160227052441
>>>   user_editcount: 2
>>>user_password_expires: NULL
>>>
>>> The user_password field is always filled with a value, meanwhile this
>>> field of non-infected user accounts with openid logins is empty.
>>> We have 423 total accounts with passwords:
>>> mysql> select count(*) from user where user_password != '';
>>> +--+
>>> | count(*) |
>>> +--+
>>> |  423 |
>>> +--+
>>> 1 row in set (0.00 sec)
>>>
>>> Mediawiki logs-in the newly created users without any preliminary email
>>> confirmation, right after the registration. I disabled the standard user
>>> login page, as described here:
>>> https://www.mediawiki.org/wiki/Manual:Special_pages#Disabling_Special:UserLogin_and_Special:UserLogout_pages
>>>
>>> And I made this patch to make it permanent:
>>> https://review.openstack.org/285669 Disable standard password based auth
>>>
>>> Just for the record, the last spam user account:
>>> 7536 | EarthaChester22
>>>
>>> Marton
>>>
>>>
>>> On Sat, Feb 27, 2016 at 8:31 AM Marton Kiss 
>>> wrote:
>>>
 Hi,

 I created the following patch, infra cores must approve that:
 https://review.openstack.org/285641 Add ssh key of JP Maxwell to
 wiki.o.o

 Marton

 On Sat, Feb 27, 2016 at 6:41 AM JP Maxwell  wrote:

> Marton has SSH access and applied a patch earlier today.  It appears
> the spam continues to flow:
>
>
> https://wiki.openstack.org/wiki/40_Thoughts_Of_Using_Open_Shelves_On_A_Kitchen
>
> Marton let me know if you can look at it some more or Infra if you
> want to give me SSH I'll do so as well in the morning (public key
> attached).
>
>
>
> ssh-rsa
> B3NzaC1yc2EBIwAAAQEA2b5I7Yff9FCrtRmSjpILUePi54Vbc8zqJTbzrIAQZGFLBi3xd2MLlhV5QVgpDBC9H3lGjbdnc81D3aFd3HwHT4dvvvyedT12PR3VDEpftdW84vw3jzdtALcayOQznjbGnScwvX5SgnRhNxuX9Rkh8qNvOsjYPUafRr9azkQoomJFkdNVI4Vb5DbLhTpt18FPeOf0UuqDt/J2tHI4SjZ3kjzr7Nbwpg8xGgANPNE0+2pJbwCA8YDt4g3bzfzvVafQs5o9Gfc9tudkR9ugQG1M+EWCgu42CleOwMTd/rYEB2fgNNPsZAWqwQfdPajVuk70EBKUEQSyoA09eEZX+xJN9Q==
> jpmax...@tipit.net
>
>
>
>
> J.P. Maxwell / tipit.net 
>
>
> On Fri, Feb 26, 2016 at 12:09 PM, Jimmy McArthur 
> wrote:
>
>> Super thankful for all the folks that have jumped in over the last
>> couple of days to help with the puppetization, etc... I just feel like
>> we're taking a very wrong approach here.
>>
>> Paul Belanger wrote:
>>
>> Right, and I don't have an issue with that approach.  Based on the work 
>> we did
>> yesterday, anybody can do that via our workflow. Please submit a patch to
>> puppet-mediawiki[1] and ping an infra-root in #openstack-infra IRC.
>>
>> What I'm proposing is the workflow is really meant for software, not
>> for web applications. It's tedious and time consuming when what's needed
>> here is a set of tests on the server. Submitting a patch, waiting for a 
>> +1,
>> then getting on IRC to find someone with access (and time) to paste the
>> logs is a pretty time consuming process for what should be a series of
>> rapid-fire changes/fixes on the se

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-27 Thread JP Maxwell

Cool. Are these applied? Any indication it has stopped the spam? Should we
clear out these non launchpad accounts from the DB?

J.P. Maxwell | tipit.net | fibercove.com
On Feb 27, 2016 6:47 AM, "Marton Kiss"  wrote:

> And the mobile frontend will be disabled permanently with this patch:
> https://review.openstack.org/285672 Disable mobile frontend
>
> M.
>
> On Sat, Feb 27, 2016 at 1:39 PM Marton Kiss  wrote:
>
>> I made some investigation, and it seems to be that the spam pages are
>> created by accounts registered with password accounts, and the launchpad
>> openid auth is not affected at all.
>>
>> So the spam script is creating accounts like this:
>> mysql> select * from user where user_name = 'CedricJamieson'\G;
>> *** 1. row ***
>>  user_id: 7494
>>user_name: CedricJamieson
>>   user_real_name: Cedric Jamieson
>>user_password:
>> :pbkdf2:sha256:1:128:Mlo9tdaP+38niZrrEka7Ow==:jEVnrTclkwIpE1RzJywDlrSvkY5G3idYwOwYRkv5O0J/MSHjY+gdhtKmArQ53v6/w7o8E1wXb2QOR6HdL5TPfOI1bswS/fYXVVYjPjkEEdxqZ8q9L5p2f3N6rEYpMfT5tk+wDiy+j5aimrHrGSga44hndAHgX9/SnqUyxlutDVY=
>> user_newpassword:
>>user_newpass_time: NULL
>>   user_email: balashkina.evdok...@mail.ru
>> user_touched: 20160227052454
>>   user_token: 7c39e44e849fb0e2bfae8790d6cc1379
>> user_email_authenticated: NULL
>> user_email_token: be963ac3bd43e70ff2f323063c61e320
>> user_email_token_expires: 20160305052441
>>user_registration: 20160227052441
>>   user_editcount: 2
>>user_password_expires: NULL
>>
>> The user_password field is always filled with a value, meanwhile this
>> field of non-infected user accounts with openid logins is empty.
>> We have 423 total accounts with passwords:
>> mysql> select count(*) from user where user_password != '';
>> +--+
>> | count(*) |
>> +--+
>> |  423 |
>> +--+
>> 1 row in set (0.00 sec)
>>
>> Mediawiki logs-in the newly created users without any preliminary email
>> confirmation, right after the registration. I disabled the standard user
>> login page, as described here:
>> https://www.mediawiki.org/wiki/Manual:Special_pages#Disabling_Special:UserLogin_and_Special:UserLogout_pages
>>
>> And I made this patch to make it permanent:
>> https://review.openstack.org/285669 Disable standard password based auth
>>
>> Just for the record, the last spam user account:
>> 7536 | EarthaChester22
>>
>> Marton
>>
>>
>> On Sat, Feb 27, 2016 at 8:31 AM Marton Kiss 
>> wrote:
>>
>>> Hi,
>>>
>>> I created the following patch, infra cores must approve that:
>>> https://review.openstack.org/285641 Add ssh key of JP Maxwell to
>>> wiki.o.o
>>>
>>> Marton
>>>
>>> On Sat, Feb 27, 2016 at 6:41 AM JP Maxwell  wrote:
>>>
 Marton has SSH access and applied a patch earlier today.  It appears
 the spam continues to flow:


 https://wiki.openstack.org/wiki/40_Thoughts_Of_Using_Open_Shelves_On_A_Kitchen

 Marton let me know if you can look at it some more or Infra if you want
 to give me SSH I'll do so as well in the morning (public key attached).



 ssh-rsa
 B3NzaC1yc2EBIwAAAQEA2b5I7Yff9FCrtRmSjpILUePi54Vbc8zqJTbzrIAQZGFLBi3xd2MLlhV5QVgpDBC9H3lGjbdnc81D3aFd3HwHT4dvvvyedT12PR3VDEpftdW84vw3jzdtALcayOQznjbGnScwvX5SgnRhNxuX9Rkh8qNvOsjYPUafRr9azkQoomJFkdNVI4Vb5DbLhTpt18FPeOf0UuqDt/J2tHI4SjZ3kjzr7Nbwpg8xGgANPNE0+2pJbwCA8YDt4g3bzfzvVafQs5o9Gfc9tudkR9ugQG1M+EWCgu42CleOwMTd/rYEB2fgNNPsZAWqwQfdPajVuk70EBKUEQSyoA09eEZX+xJN9Q==
 jpmax...@tipit.net




 J.P. Maxwell / tipit.net 


 On Fri, Feb 26, 2016 at 12:09 PM, Jimmy McArthur 
 wrote:

> Super thankful for all the folks that have jumped in over the last
> couple of days to help with the puppetization, etc... I just feel like
> we're taking a very wrong approach here.
>
> Paul Belanger wrote:
>
> Right, and I don't have an issue with that approach.  Based on the work 
> we did
> yesterday, anybody can do that via our workflow. Please submit a patch to
> puppet-mediawiki[1] and ping an infra-root in #openstack-infra IRC.
>
> What I'm proposing is the workflow is really meant for software, not
> for web applications. It's tedious and time consuming when what's needed
> here is a set of tests on the server. Submitting a patch, waiting for a 
> +1,
> then getting on IRC to find someone with access (and time) to paste the
> logs is a pretty time consuming process for what should be a series of
> rapid-fire changes/fixes on the server. Especially when we're dealign with
> an active attack.
>
>
> We can then have somebody look at the logs.  I think it is more about 
> scheduling
> the task since more infra-root as travling back from the mid-cycle last 
> night
> and today.
>
> Right

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

And the mobile frontend will be disabled permanently with this patch:
https://review.openstack.org/285672 Disable mobile frontend

M.

On Sat, Feb 27, 2016 at 1:39 PM Marton Kiss  wrote:

> I made some investigation, and it seems to be that the spam pages are
> created by accounts registered with password accounts, and the launchpad
> openid auth is not affected at all.
>
> So the spam script is creating accounts like this:
> mysql> select * from user where user_name = 'CedricJamieson'\G;
> *** 1. row ***
>  user_id: 7494
>user_name: CedricJamieson
>   user_real_name: Cedric Jamieson
>user_password:
> :pbkdf2:sha256:1:128:Mlo9tdaP+38niZrrEka7Ow==:jEVnrTclkwIpE1RzJywDlrSvkY5G3idYwOwYRkv5O0J/MSHjY+gdhtKmArQ53v6/w7o8E1wXb2QOR6HdL5TPfOI1bswS/fYXVVYjPjkEEdxqZ8q9L5p2f3N6rEYpMfT5tk+wDiy+j5aimrHrGSga44hndAHgX9/SnqUyxlutDVY=
> user_newpassword:
>user_newpass_time: NULL
>   user_email: balashkina.evdok...@mail.ru
> user_touched: 20160227052454
>   user_token: 7c39e44e849fb0e2bfae8790d6cc1379
> user_email_authenticated: NULL
> user_email_token: be963ac3bd43e70ff2f323063c61e320
> user_email_token_expires: 20160305052441
>user_registration: 20160227052441
>   user_editcount: 2
>user_password_expires: NULL
>
> The user_password field is always filled with a value, meanwhile this
> field of non-infected user accounts with openid logins is empty.
> We have 423 total accounts with passwords:
> mysql> select count(*) from user where user_password != '';
> +--+
> | count(*) |
> +--+
> |  423 |
> +--+
> 1 row in set (0.00 sec)
>
> Mediawiki logs-in the newly created users without any preliminary email
> confirmation, right after the registration. I disabled the standard user
> login page, as described here:
> https://www.mediawiki.org/wiki/Manual:Special_pages#Disabling_Special:UserLogin_and_Special:UserLogout_pages
>
> And I made this patch to make it permanent:
> https://review.openstack.org/285669 Disable standard password based auth
>
> Just for the record, the last spam user account:
> 7536 | EarthaChester22
>
> Marton
>
>
> On Sat, Feb 27, 2016 at 8:31 AM Marton Kiss  wrote:
>
>> Hi,
>>
>> I created the following patch, infra cores must approve that:
>> https://review.openstack.org/285641 Add ssh key of JP Maxwell to wiki.o.o
>>
>> Marton
>>
>> On Sat, Feb 27, 2016 at 6:41 AM JP Maxwell  wrote:
>>
>>> Marton has SSH access and applied a patch earlier today.  It appears the
>>> spam continues to flow:
>>>
>>>
>>> https://wiki.openstack.org/wiki/40_Thoughts_Of_Using_Open_Shelves_On_A_Kitchen
>>>
>>> Marton let me know if you can look at it some more or Infra if you want
>>> to give me SSH I'll do so as well in the morning (public key attached).
>>>
>>>
>>>
>>> ssh-rsa
>>> B3NzaC1yc2EBIwAAAQEA2b5I7Yff9FCrtRmSjpILUePi54Vbc8zqJTbzrIAQZGFLBi3xd2MLlhV5QVgpDBC9H3lGjbdnc81D3aFd3HwHT4dvvvyedT12PR3VDEpftdW84vw3jzdtALcayOQznjbGnScwvX5SgnRhNxuX9Rkh8qNvOsjYPUafRr9azkQoomJFkdNVI4Vb5DbLhTpt18FPeOf0UuqDt/J2tHI4SjZ3kjzr7Nbwpg8xGgANPNE0+2pJbwCA8YDt4g3bzfzvVafQs5o9Gfc9tudkR9ugQG1M+EWCgu42CleOwMTd/rYEB2fgNNPsZAWqwQfdPajVuk70EBKUEQSyoA09eEZX+xJN9Q==
>>> jpmax...@tipit.net
>>>
>>>
>>>
>>>
>>> J.P. Maxwell / tipit.net 
>>>
>>>
>>> On Fri, Feb 26, 2016 at 12:09 PM, Jimmy McArthur 
>>> wrote:
>>>
 Super thankful for all the folks that have jumped in over the last
 couple of days to help with the puppetization, etc... I just feel like
 we're taking a very wrong approach here.

 Paul Belanger wrote:

 Right, and I don't have an issue with that approach.  Based on the work we 
 did
 yesterday, anybody can do that via our workflow. Please submit a patch to
 puppet-mediawiki[1] and ping an infra-root in #openstack-infra IRC.

 What I'm proposing is the workflow is really meant for software, not
 for web applications. It's tedious and time consuming when what's needed
 here is a set of tests on the server. Submitting a patch, waiting for a +1,
 then getting on IRC to find someone with access (and time) to paste the
 logs is a pretty time consuming process for what should be a series of
 rapid-fire changes/fixes on the server. Especially when we're dealign with
 an active attack.


 We can then have somebody look at the logs.  I think it is more about 
 scheduling
 the task since more infra-root as travling back from the mid-cycle last 
 night
 and today.

 Right, this is my point. This has been going on for 3 weeks (or more).
 Tom Fifeldt was asking for help without response. And here we are through
 another week and no closer to stemming the flow.

 I'm fully aware what I'm proposing goes against what Infra and the
 OpenStack workflow is all about, but I'd ask you all to look at this fro

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

I made some investigation, and it seems to be that the spam pages are
created by accounts registered with password accounts, and the launchpad
openid auth is not affected at all.

So the spam script is creating accounts like this:
mysql> select * from user where user_name = 'CedricJamieson'\G;
*** 1. row ***
 user_id: 7494
   user_name: CedricJamieson
  user_real_name: Cedric Jamieson
   user_password:
:pbkdf2:sha256:1:128:Mlo9tdaP+38niZrrEka7Ow==:jEVnrTclkwIpE1RzJywDlrSvkY5G3idYwOwYRkv5O0J/MSHjY+gdhtKmArQ53v6/w7o8E1wXb2QOR6HdL5TPfOI1bswS/fYXVVYjPjkEEdxqZ8q9L5p2f3N6rEYpMfT5tk+wDiy+j5aimrHrGSga44hndAHgX9/SnqUyxlutDVY=
user_newpassword:
   user_newpass_time: NULL
  user_email: balashkina.evdok...@mail.ru
user_touched: 20160227052454
  user_token: 7c39e44e849fb0e2bfae8790d6cc1379
user_email_authenticated: NULL
user_email_token: be963ac3bd43e70ff2f323063c61e320
user_email_token_expires: 20160305052441
   user_registration: 20160227052441
  user_editcount: 2
   user_password_expires: NULL

The user_password field is always filled with a value, meanwhile this field
of non-infected user accounts with openid logins is empty.
We have 423 total accounts with passwords:
mysql> select count(*) from user where user_password != '';
+--+
| count(*) |
+--+
|  423 |
+--+
1 row in set (0.00 sec)

Mediawiki logs-in the newly created users without any preliminary email
confirmation, right after the registration. I disabled the standard user
login page, as described here:
https://www.mediawiki.org/wiki/Manual:Special_pages#Disabling_Special:UserLogin_and_Special:UserLogout_pages

And I made this patch to make it permanent:
https://review.openstack.org/285669 Disable standard password based auth

Just for the record, the last spam user account:
7536 | EarthaChester22

Marton


On Sat, Feb 27, 2016 at 8:31 AM Marton Kiss  wrote:

> Hi,
>
> I created the following patch, infra cores must approve that:
> https://review.openstack.org/285641 Add ssh key of JP Maxwell to wiki.o.o
>
> Marton
>
> On Sat, Feb 27, 2016 at 6:41 AM JP Maxwell  wrote:
>
>> Marton has SSH access and applied a patch earlier today.  It appears the
>> spam continues to flow:
>>
>>
>> https://wiki.openstack.org/wiki/40_Thoughts_Of_Using_Open_Shelves_On_A_Kitchen
>>
>> Marton let me know if you can look at it some more or Infra if you want
>> to give me SSH I'll do so as well in the morning (public key attached).
>>
>>
>>
>> ssh-rsa
>> B3NzaC1yc2EBIwAAAQEA2b5I7Yff9FCrtRmSjpILUePi54Vbc8zqJTbzrIAQZGFLBi3xd2MLlhV5QVgpDBC9H3lGjbdnc81D3aFd3HwHT4dvvvyedT12PR3VDEpftdW84vw3jzdtALcayOQznjbGnScwvX5SgnRhNxuX9Rkh8qNvOsjYPUafRr9azkQoomJFkdNVI4Vb5DbLhTpt18FPeOf0UuqDt/J2tHI4SjZ3kjzr7Nbwpg8xGgANPNE0+2pJbwCA8YDt4g3bzfzvVafQs5o9Gfc9tudkR9ugQG1M+EWCgu42CleOwMTd/rYEB2fgNNPsZAWqwQfdPajVuk70EBKUEQSyoA09eEZX+xJN9Q==
>> jpmax...@tipit.net
>>
>>
>>
>>
>> J.P. Maxwell / tipit.net 
>>
>>
>> On Fri, Feb 26, 2016 at 12:09 PM, Jimmy McArthur 
>> wrote:
>>
>>> Super thankful for all the folks that have jumped in over the last
>>> couple of days to help with the puppetization, etc... I just feel like
>>> we're taking a very wrong approach here.
>>>
>>> Paul Belanger wrote:
>>>
>>> Right, and I don't have an issue with that approach.  Based on the work we 
>>> did
>>> yesterday, anybody can do that via our workflow. Please submit a patch to
>>> puppet-mediawiki[1] and ping an infra-root in #openstack-infra IRC.
>>>
>>> What I'm proposing is the workflow is really meant for software, not for
>>> web applications. It's tedious and time consuming when what's needed here
>>> is a set of tests on the server. Submitting a patch, waiting for a +1, then
>>> getting on IRC to find someone with access (and time) to paste the logs is
>>> a pretty time consuming process for what should be a series of rapid-fire
>>> changes/fixes on the server. Especially when we're dealign with an active
>>> attack.
>>>
>>>
>>> We can then have somebody look at the logs.  I think it is more about 
>>> scheduling
>>> the task since more infra-root as travling back from the mid-cycle last 
>>> night
>>> and today.
>>>
>>> Right, this is my point. This has been going on for 3 weeks (or more).
>>> Tom Fifeldt was asking for help without response. And here we are through
>>> another week and no closer to stemming the flow.
>>>
>>> I'm fully aware what I'm proposing goes against what Infra and the
>>> OpenStack workflow is all about, but I'd ask you all to look at this from a
>>> web development perspective instead of a software development perspective.
>>>
>>> Jimmy
>>>
>>>
>>> Last email from me, just on a plane.  Will follow up when I land.
>>>
>>> [1] https://git.openstack.org/cgit/openstack-infra/puppet-mediawiki
>>>
>>>  J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com
>>> [http://www.fibercove.

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

Hi,

I created the following patch, infra cores must approve that:
https://review.openstack.org/285641 Add ssh key of JP Maxwell to wiki.o.o

Marton

On Sat, Feb 27, 2016 at 6:41 AM JP Maxwell  wrote:

> Marton has SSH access and applied a patch earlier today.  It appears the
> spam continues to flow:
>
>
> https://wiki.openstack.org/wiki/40_Thoughts_Of_Using_Open_Shelves_On_A_Kitchen
>
> Marton let me know if you can look at it some more or Infra if you want to
> give me SSH I'll do so as well in the morning (public key attached).
>
>
>
> ssh-rsa
> B3NzaC1yc2EBIwAAAQEA2b5I7Yff9FCrtRmSjpILUePi54Vbc8zqJTbzrIAQZGFLBi3xd2MLlhV5QVgpDBC9H3lGjbdnc81D3aFd3HwHT4dvvvyedT12PR3VDEpftdW84vw3jzdtALcayOQznjbGnScwvX5SgnRhNxuX9Rkh8qNvOsjYPUafRr9azkQoomJFkdNVI4Vb5DbLhTpt18FPeOf0UuqDt/J2tHI4SjZ3kjzr7Nbwpg8xGgANPNE0+2pJbwCA8YDt4g3bzfzvVafQs5o9Gfc9tudkR9ugQG1M+EWCgu42CleOwMTd/rYEB2fgNNPsZAWqwQfdPajVuk70EBKUEQSyoA09eEZX+xJN9Q==
> jpmax...@tipit.net
>
>
>
>
> J.P. Maxwell / tipit.net 
>
>
> On Fri, Feb 26, 2016 at 12:09 PM, Jimmy McArthur 
> wrote:
>
>> Super thankful for all the folks that have jumped in over the last couple
>> of days to help with the puppetization, etc... I just feel like we're
>> taking a very wrong approach here.
>>
>> Paul Belanger wrote:
>>
>> Right, and I don't have an issue with that approach.  Based on the work we 
>> did
>> yesterday, anybody can do that via our workflow. Please submit a patch to
>> puppet-mediawiki[1] and ping an infra-root in #openstack-infra IRC.
>>
>> What I'm proposing is the workflow is really meant for software, not for
>> web applications. It's tedious and time consuming when what's needed here
>> is a set of tests on the server. Submitting a patch, waiting for a +1, then
>> getting on IRC to find someone with access (and time) to paste the logs is
>> a pretty time consuming process for what should be a series of rapid-fire
>> changes/fixes on the server. Especially when we're dealign with an active
>> attack.
>>
>>
>> We can then have somebody look at the logs.  I think it is more about 
>> scheduling
>> the task since more infra-root as travling back from the mid-cycle last night
>> and today.
>>
>> Right, this is my point. This has been going on for 3 weeks (or more).
>> Tom Fifeldt was asking for help without response. And here we are through
>> another week and no closer to stemming the flow.
>>
>> I'm fully aware what I'm proposing goes against what Infra and the
>> OpenStack workflow is all about, but I'd ask you all to look at this from a
>> web development perspective instead of a software development perspective.
>>
>> Jimmy
>>
>>
>> Last email from me, just on a plane.  Will follow up when I land.
>>
>> [1] https://git.openstack.org/cgit/openstack-infra/puppet-mediawiki
>>
>>  J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com
>> [http://www.fibercove.com]
>> On Fri, Feb 26, 2016 at 11:25 AM, Paul Belanger  
>> 
>> wrote:
>> On Fri, Feb 26, 2016 at 11:08:18AM -0600, Jimmy McArthur wrote:
>>
>> Given the state of the wiki a the moment, I think taking the quickest path
>> to get it fixed would be prudent. Is there a way we can get JP root access
>> to this server, even temporarily? We get 25% of our website traffic (2
>> million visitors) to the wiki. I realize we're all after the same thing,
>>
>> but
>>
>> spammers are not going to hit the dev environment, so there's really no
>>
>> way
>>
>> to tell if teh problem is fixed without actually working directly on the
>> production machine. This should be a 30 minute fix.
>>
>>
>> I am still unclear what the 30min fix is. If really 30mins, then it
>> shouldn't be
>> hard to get the fix into our workflow. Could somebody please elaborate.
>>
>> If we are talking about deploying new versions of php or mediawiki manually,
>> I
>> not be in-favor of this. To me, while the attack sucks, we should be working
>> on
>> 2 fronts. Getting the help needed to mitigate the attack, then adding the
>> changes into -infra workflow in parallel.
>>
>>
>> I realize there is a lot of risk in giving ssh access to infra machines,
>>
>> but
>>
>> I think it's worth taking a look at either putting this machine in a place
>> where a different level of admin could access it without giving away the
>> keys to the entire OpenStack infrastructure or figuring out a way to set
>>
>> up
>>
>> credentials with varying levels of access.
>>
>>
>> As a note, all the work I've been doing to help with the attack hasn't
>> require
>> SSH access for me to wiki.o.o. I did need infra-root help to expose our
>> configuration safely. I'd rather take some time to see what the fixes are,
>> having infra-root apply changes, then move them into puppet.
>>
>> It also has been discussed to simply disable write access to the wiki if we
>> really want spamming to stop, obviously that will affect normal usage.
>>
>>
>> Jimmy
>>
>> Paul Belanger wrote:
>>
>> On Fri, Feb 26, 2016 at 10:12:12AM -0600, JP Maxwell wrote:
>>
>> But if you

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

Marton has SSH access and applied a patch earlier today.  It appears the
spam continues to flow:

https://wiki.openstack.org/wiki/40_Thoughts_Of_Using_Open_Shelves_On_A_Kitchen

Marton let me know if you can look at it some more or Infra if you want to
give me SSH I'll do so as well in the morning (public key attached).



ssh-rsa
B3NzaC1yc2EBIwAAAQEA2b5I7Yff9FCrtRmSjpILUePi54Vbc8zqJTbzrIAQZGFLBi3xd2MLlhV5QVgpDBC9H3lGjbdnc81D3aFd3HwHT4dvvvyedT12PR3VDEpftdW84vw3jzdtALcayOQznjbGnScwvX5SgnRhNxuX9Rkh8qNvOsjYPUafRr9azkQoomJFkdNVI4Vb5DbLhTpt18FPeOf0UuqDt/J2tHI4SjZ3kjzr7Nbwpg8xGgANPNE0+2pJbwCA8YDt4g3bzfzvVafQs5o9Gfc9tudkR9ugQG1M+EWCgu42CleOwMTd/rYEB2fgNNPsZAWqwQfdPajVuk70EBKUEQSyoA09eEZX+xJN9Q==
jpmax...@tipit.net




J.P. Maxwell / tipit.net 


On Fri, Feb 26, 2016 at 12:09 PM, Jimmy McArthur 
wrote:

> Super thankful for all the folks that have jumped in over the last couple
> of days to help with the puppetization, etc... I just feel like we're
> taking a very wrong approach here.
>
> Paul Belanger wrote:
>
> Right, and I don't have an issue with that approach.  Based on the work we did
> yesterday, anybody can do that via our workflow. Please submit a patch to
> puppet-mediawiki[1] and ping an infra-root in #openstack-infra IRC.
>
> What I'm proposing is the workflow is really meant for software, not for
> web applications. It's tedious and time consuming when what's needed here
> is a set of tests on the server. Submitting a patch, waiting for a +1, then
> getting on IRC to find someone with access (and time) to paste the logs is
> a pretty time consuming process for what should be a series of rapid-fire
> changes/fixes on the server. Especially when we're dealign with an active
> attack.
>
>
> We can then have somebody look at the logs.  I think it is more about 
> scheduling
> the task since more infra-root as travling back from the mid-cycle last night
> and today.
>
> Right, this is my point. This has been going on for 3 weeks (or more). Tom
> Fifeldt was asking for help without response. And here we are through
> another week and no closer to stemming the flow.
>
> I'm fully aware what I'm proposing goes against what Infra and the
> OpenStack workflow is all about, but I'd ask you all to look at this from a
> web development perspective instead of a software development perspective.
>
> Jimmy
>
>
> Last email from me, just on a plane.  Will follow up when I land.
>
> [1] https://git.openstack.org/cgit/openstack-infra/puppet-mediawiki
>
>  J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com
> [http://www.fibercove.com]
> On Fri, Feb 26, 2016 at 11:25 AM, Paul Belanger  
> 
> wrote:
> On Fri, Feb 26, 2016 at 11:08:18AM -0600, Jimmy McArthur wrote:
>
> Given the state of the wiki a the moment, I think taking the quickest path
> to get it fixed would be prudent. Is there a way we can get JP root access
> to this server, even temporarily? We get 25% of our website traffic (2
> million visitors) to the wiki. I realize we're all after the same thing,
>
> but
>
> spammers are not going to hit the dev environment, so there's really no
>
> way
>
> to tell if teh problem is fixed without actually working directly on the
> production machine. This should be a 30 minute fix.
>
>
> I am still unclear what the 30min fix is. If really 30mins, then it
> shouldn't be
> hard to get the fix into our workflow. Could somebody please elaborate.
>
> If we are talking about deploying new versions of php or mediawiki manually,
> I
> not be in-favor of this. To me, while the attack sucks, we should be working
> on
> 2 fronts. Getting the help needed to mitigate the attack, then adding the
> changes into -infra workflow in parallel.
>
>
> I realize there is a lot of risk in giving ssh access to infra machines,
>
> but
>
> I think it's worth taking a look at either putting this machine in a place
> where a different level of admin could access it without giving away the
> keys to the entire OpenStack infrastructure or figuring out a way to set
>
> up
>
> credentials with varying levels of access.
>
>
> As a note, all the work I've been doing to help with the attack hasn't
> require
> SSH access for me to wiki.o.o. I did need infra-root help to expose our
> configuration safely. I'd rather take some time to see what the fixes are,
> having infra-root apply changes, then move them into puppet.
>
> It also has been discussed to simply disable write access to the wiki if we
> really want spamming to stop, obviously that will affect normal usage.
>
>
> Jimmy
>
> Paul Belanger wrote:
>
> On Fri, Feb 26, 2016 at 10:12:12AM -0600, JP Maxwell wrote:
>
> But if you wanted to upgrade everything, remove the mobile view
>
> extension,
>
> test in a dev/staging environment then deploy to production fingers
> crossed, I think that would be a valid approach as well.
>
>
> Current review up[1]. I'll launch a node tonight / tomorrow locally to
>
> see
> how
>
> puppet reacts. I suspect there will be some issues.

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-26 Thread Jimmy McArthur

Super thankful for all the folks that have jumped in over the last 
couple of days to help with the puppetization, etc... I just feel like 
we're taking a very wrong approach here.


Paul Belanger wrote:

Right, and I don't have an issue with that approach.  Based on the work we did
yesterday, anybody can do that via our workflow. Please submit a patch to
puppet-mediawiki[1] and ping an infra-root in #openstack-infra IRC.
What I'm proposing is the workflow is really meant for software, not for 
web applications. It's tedious and time consuming when what's needed 
here is a set of tests on the server. Submitting a patch, waiting for a 
+1, then getting on IRC to find someone with access (and time) to paste 
the logs is a pretty time consuming process for what should be a series 
of rapid-fire changes/fixes on the server. Especially when we're dealign 
with an active attack.


We can then have somebody look at the logs.  I think it is more about scheduling
the task since more infra-root as travling back from the mid-cycle last night
and today.
Right, this is my point. This has been going on for 3 weeks (or more). 
Tom Fifeldt was asking for help without response. And here we are 
through another week and no closer to stemming the flow.


I'm fully aware what I'm proposing goes against what Infra and the 
OpenStack workflow is all about, but I'd ask you all to look at this 
from a web development perspective instead of a software development 
perspective.


Jimmy


Last email from me, just on a plane.  Will follow up when I land.

[1] https://git.openstack.org/cgit/openstack-infra/puppet-mediawiki



J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com
[http://www.fibercove.com]
On Fri, Feb 26, 2016 at 11:25 AM, Paul Belanger
wrote:
On Fri, Feb 26, 2016 at 11:08:18AM -0600, Jimmy McArthur wrote:

Given the state of the wiki a the moment, I think taking the quickest path
to get it fixed would be prudent. Is there a way we can get JP root access
to this server, even temporarily? We get 25% of our website traffic (2
million visitors) to the wiki. I realize we're all after the same thing,

but

spammers are not going to hit the dev environment, so there's really no

way

to tell if teh problem is fixed without actually working directly on the
production machine. This should be a 30 minute fix.


I am still unclear what the 30min fix is. If really 30mins, then it
shouldn't be
hard to get the fix into our workflow. Could somebody please elaborate.

If we are talking about deploying new versions of php or mediawiki manually,
I
not be in-favor of this. To me, while the attack sucks, we should be working
on
2 fronts. Getting the help needed to mitigate the attack, then adding the
changes into -infra workflow in parallel.


I realize there is a lot of risk in giving ssh access to infra machines,

but

I think it's worth taking a look at either putting this machine in a place
where a different level of admin could access it without giving away the
keys to the entire OpenStack infrastructure or figuring out a way to set

up

credentials with varying levels of access.


As a note, all the work I've been doing to help with the attack hasn't
require
SSH access for me to wiki.o.o. I did need infra-root help to expose our
configuration safely. I'd rather take some time to see what the fixes are,
having infra-root apply changes, then move them into puppet.

It also has been discussed to simply disable write access to the wiki if we
really want spamming to stop, obviously that will affect normal usage.


Jimmy

Paul Belanger wrote:

On Fri, Feb 26, 2016 at 10:12:12AM -0600, JP Maxwell wrote:

But if you wanted to upgrade everything, remove the mobile view

extension,

test in a dev/staging environment then deploy to production fingers
crossed, I think that would be a valid approach as well.


Current review up[1]. I'll launch a node tonight / tomorrow locally to

see
how

puppet reacts. I suspect there will be some issues.

If infra-roots are fine with this approach, we can use that box to test

against.

[1] https://review.openstack.org/#/c/285405/


J.P. Maxwell | tipit.net | fibercove.com
On Feb 26, 2016 10:08 AM, "JP Maxwell"  wrote:


Plus one except in this case it is much easier to know if our efforts

are

working on production because the spam either stops or not.

J.P. Maxwell | tipit.net | fibercove.com
On Feb 26, 2016 9:48 AM, "Paul Belanger"  wrote:


On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote:

I really think you might consider the option that there is a

vulnerability

in one of the extensions. If that is the case black listing IPs will

be

an

ongoing wild goose chase.

I think this would be easily proven or disproven by making the questy
question impossible and see if the spam continues.


We'll have to let an infra-root make that call. Since nobody would be
able to
use the wiki. Honestly, I'd rather spend the time standing up a mirror

dev

instance for us to work on, rather then produc

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

A quick google indicates this may be an unrelated issue that should be fixed,
but I don’t *think* it is related to the spam.

J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com 
[http://www.fibercove.com]
On Fri, Feb 26, 2016 at 11:56 AM, Marton Kiss  wrote:
I'm going to get a dinner, but I'll be on irc after, so if I can help somehow, I
will be here. #openstack-infra mrmartin
M.
On Fri, Feb 26, 2016 at 6:51 PM Paul Belanger < pbela...@redhat.com 
[pbela...@redhat.com] > wrote:
On phone but patch puppet-mediawiki and enable captcha for all pages. We only
did edit and create

On Feb 26, 2016 10:38 AM, Marton Kiss < marton.k...@gmail.com 
[marton.k...@gmail.com] > wrote:

I see a ton of incoming post requests:
POST
/w/index.php?title=Special%3ARunJobs&tasks=jobs&maxjobs=1&sigexpiry=1456508270&signature=571cfb216f944b15d2eee1c0253d08b77003328e

M.
On Fri, Feb 26, 2016 at 6:35 PM Marton Kiss < marton.k...@gmail.com 
[marton.k...@gmail.com] > wrote:
Oh, I can login. So what we need?
M.
On Fri, Feb 26, 2016 at 6:33 PM JP Maxwell < j...@tipit.net [j...@tipit.net] > 
wrote:
I think what Jimmy is referring to is what I was suggesting by removing the
extensions / making the question impossible to answer. Basically a series of
rapid fire changes while tailing the logs and seeing what stops the spam. Once
you know what worked then you can submit as an official patch. But being able to
quickly try these things on a server actually under attack is the fastest path
toward identifying the fix.

J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com 
[http://www.fibercove.com]
On Fri, Feb 26, 2016 at 11:25 AM, Paul Belanger < pabelan...@redhat.com 
[pabelan...@redhat.com] > wrote:
On Fri, Feb 26, 2016 at 11:08:18AM -0600, Jimmy McArthur wrote:
> Given the state of the wiki a the moment, I think taking the quickest path
> to get it fixed would be prudent. Is there a way we can get JP root access
> to this server, even temporarily? We get 25% of our website traffic (2
> million visitors) to the wiki. I realize we're all after the same thing, but
> spammers are not going to hit the dev environment, so there's really no way
> to tell if teh problem is fixed without actually working directly on the
> production machine. This should be a 30 minute fix.
>
I am still unclear what the 30min fix is. If really 30mins, then it shouldn't be
hard to get the fix into our workflow. Could somebody please elaborate.

If we are talking about deploying new versions of php or mediawiki manually, I
not be in-favor of this. To me, while the attack sucks, we should be working on
2 fronts. Getting the help needed to mitigate the attack, then adding the
changes into -infra workflow in parallel.

> I realize there is a lot of risk in giving ssh access to infra machines, but
> I think it's worth taking a look at either putting this machine in a place
> where a different level of admin could access it without giving away the
> keys to the entire OpenStack infrastructure or figuring out a way to set up
> credentials with varying levels of access.
>
As a note, all the work I've been doing to help with the attack hasn't require
SSH access for me to wiki.o.o. I did need infra-root help to expose our
configuration safely. I'd rather take some time to see what the fixes are,
having infra-root apply changes, then move them into puppet.

It also has been discussed to simply disable write access to the wiki if we
really want spamming to stop, obviously that will affect normal usage.

> Jimmy
>
> Paul Belanger wrote:
> >On Fri, Feb 26, 2016 at 10:12:12AM -0600, JP Maxwell wrote:
> >>But if you wanted to upgrade everything, remove the mobile view extension,
> >>test in a dev/staging environment then deploy to production fingers
> >>crossed, I think that would be a valid approach as well.
> >>
> >Current review up[1]. I'll launch a node tonight / tomorrow locally to see
how
> >puppet reacts. I suspect there will be some issues.
> >
> >If infra-roots are fine with this approach, we can use that box to test
against.
> >
> >[1] https://review.openstack.org/#/c/285405/
[https://review.openstack.org/#/c/285405/]
> >
> >>J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com 
> >>[http://fibercove.com]
> >>On Feb 26, 2016 10:08 AM, "JP Maxwell"< j...@tipit.net [j...@tipit.net] > 
> >>wrote:
> >>
> >>>Plus one except in this case it is much easier to know if our efforts are
> >>>working on production because the spam either stops or not.
> >>>
> >>>J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com 
> >>>[http://fibercove.com]
> >>>On Feb 26, 2016 9:48 AM, "Paul Belanger"< pabelan...@redhat.com 
> >>>[pabelan...@redhat.com] > wrote:
> >>>
> On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote:
> >I really think you might consider the option that there is a
> vulnerability
> >in one of the extensions. If that is the case black listing IPs will be
> an
> >ongoing wild goose chase.
> >
> >I think this would

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

Yeah, I checked it and it is internal job runner:
https://www.mediawiki.org/wiki/Manual:Job_queue

M.

On Fri, Feb 26, 2016 at 7:00 PM JP Maxwell  wrote:

> A quick google indicates this may be an unrelated issue that should be
> fixed, but I don’t *think* it is related to the spam.
>
> *J.P. Maxwell* | tipit.net | fibercove.com 
>
> On Fri, Feb 26, 2016 at 11:56 AM, Marton Kiss 
> wrote:
>
> I'm going to get a dinner, but I'll be on irc after, so if I can help
> somehow, I will be here. #openstack-infra mrmartin
>
> M.
>
> On Fri, Feb 26, 2016 at 6:51 PM Paul Belanger  wrote:
>
>> On phone but patch puppet-mediawiki and enable captcha for all pages. We
>> only did edit and create
>> On Feb 26, 2016 10:38 AM, Marton Kiss  wrote:
>>
>> I see a ton of incoming post requests:
>>
>> POST
>> /w/index.php?title=Special%3ARunJobs&tasks=jobs&maxjobs=1&sigexpiry=1456508270&signature=571cfb216f944b15d2eee1c0253d08b77003328e
>>
>> M.
>>
>> On Fri, Feb 26, 2016 at 6:35 PM Marton Kiss 
>> wrote:
>>
>>> Oh, I can login. So what we need?
>>>
>>> M.
>>>
>>> On Fri, Feb 26, 2016 at 6:33 PM JP Maxwell  wrote:
>>>
 I think what Jimmy is referring to is what I was suggesting by removing
 the extensions / making the question impossible to answer. Basically a
 series of rapid fire changes while tailing the logs and seeing what stops
 the spam. Once you know what worked then you can submit as an official
 patch. But being able to quickly try these things on a server actually
 under attack is the fastest path toward identifying the fix.

 *J.P. Maxwell* | tipit.net | fibercove.com 

 On Fri, Feb 26, 2016 at 11:25 AM, Paul Belanger 
 wrote:

 On Fri, Feb 26, 2016 at 11:08:18AM -0600, Jimmy McArthur wrote:
 > Given the state of the wiki a the moment, I think taking the quickest
 path
 > to get it fixed would be prudent. Is there a way we can get JP root
 access
 > to this server, even temporarily? We get 25% of our website traffic (2
 > million visitors) to the wiki. I realize we're all after the same
 thing, but
 > spammers are not going to hit the dev environment, so there's really
 no way
 > to tell if teh problem is fixed without actually working directly on
 the
 > production machine. This should be a 30 minute fix.
 >
 I am still unclear what the 30min fix is. If really 30mins, then it
 shouldn't be
 hard to get the fix into our workflow. Could somebody please elaborate.

 If we are talking about deploying new versions of php or mediawiki
 manually, I
 not be in-favor of this. To me, while the attack sucks, we should be
 working on
 2 fronts. Getting the help needed to mitigate the attack, then adding
 the
 changes into -infra workflow in parallel.

 > I realize there is a lot of risk in giving ssh access to infra
 machines, but
 > I think it's worth taking a look at either putting this machine in a
 place
 > where a different level of admin could access it without giving away
 the
 > keys to the entire OpenStack infrastructure or figuring out a way to
 set up
 > credentials with varying levels of access.
 >
 As a note, all the work I've been doing to help with the attack hasn't
 require
 SSH access for me to wiki.o.o. I did need infra-root help to expose our
 configuration safely. I'd rather take some time to see what the fixes
 are,
 having infra-root apply changes, then move them into puppet.

 It also has been discussed to simply disable write access to the wiki
 if we
 really want spamming to stop, obviously that will affect normal usage.

 > Jimmy
 >
 > Paul Belanger wrote:
 > >On Fri, Feb 26, 2016 at 10:12:12AM -0600, JP Maxwell wrote:
 > >>But if you wanted to upgrade everything, remove the mobile view
 extension,
 > >>test in a dev/staging environment then deploy to production fingers
 > >>crossed, I think that would be a valid approach as well.
 > >>
 > >Current review up[1]. I'll launch a node tonight / tomorrow locally
 to see how
 > >puppet reacts. I suspect there will be some issues.
 > >
 > >If infra-roots are fine with this approach, we can use that box to
 test against.
 > >
 > >[1] https://review.openstack.org/#/c/285405/
 > >
 > >>J.P. Maxwell | tipit.net | fibercove.com
 > >>On Feb 26, 2016 10:08 AM, "JP Maxwell" wrote:
 > >>
 > >>>Plus one except in this case it is much easier to know if our
 efforts are
 > >>>working on production because the spam either stops or not.
 > >>>
 > >>>J.P. Maxwell | tipit.net | fibercove.com
 > >>>On Feb 26, 2016 9:48 AM, "Paul Belanger"
 wrote:
 > >>>
 > On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote:
 > >I really think you might consider the option

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

I'm going to get a dinner, but I'll be on irc after, so if I can help
somehow, I will be here. #openstack-infra mrmartin

M.

On Fri, Feb 26, 2016 at 6:51 PM Paul Belanger  wrote:

> On phone but patch puppet-mediawiki and enable captcha for all pages. We
> only did edit and create
> On Feb 26, 2016 10:38 AM, Marton Kiss  wrote:
>
> I see a ton of incoming post requests:
>
> POST
> /w/index.php?title=Special%3ARunJobs&tasks=jobs&maxjobs=1&sigexpiry=1456508270&signature=571cfb216f944b15d2eee1c0253d08b77003328e
>
> M.
>
> On Fri, Feb 26, 2016 at 6:35 PM Marton Kiss  wrote:
>
>> Oh, I can login. So what we need?
>>
>> M.
>>
>> On Fri, Feb 26, 2016 at 6:33 PM JP Maxwell  wrote:
>>
>>> I think what Jimmy is referring to is what I was suggesting by removing
>>> the extensions / making the question impossible to answer.  Basically a
>>> series of rapid fire changes while tailing the logs and seeing what stops
>>> the spam.  Once you know what worked then you can submit as an official
>>> patch.  But being able to quickly try these things on a server actually
>>> under attack is the fastest path toward identifying the fix.
>>>
>>> *J.P. Maxwell* | tipit.net | fibercove.com 
>>>
>>> On Fri, Feb 26, 2016 at 11:25 AM, Paul Belanger 
>>> wrote:
>>>
>>> On Fri, Feb 26, 2016 at 11:08:18AM -0600, Jimmy McArthur wrote:
>>> > Given the state of the wiki a the moment, I think taking the quickest
>>> path
>>> > to get it fixed would be prudent. Is there a way we can get JP root
>>> access
>>> > to this server, even temporarily? We get 25% of our website traffic (2
>>> > million visitors) to the wiki. I realize we're all after the same
>>> thing, but
>>> > spammers are not going to hit the dev environment, so there's really
>>> no way
>>> > to tell if teh problem is fixed without actually working directly on
>>> the
>>> > production machine. This should be a 30 minute fix.
>>> >
>>> I am still unclear what the 30min fix is. If really 30mins, then it
>>> shouldn't be
>>> hard to get the fix into our workflow. Could somebody please elaborate.
>>>
>>> If we are talking about deploying new versions of php or mediawiki
>>> manually, I
>>> not be in-favor of this. To me, while the attack sucks, we should be
>>> working on
>>> 2 fronts. Getting the help needed to mitigate the attack, then adding the
>>> changes into -infra workflow in parallel.
>>>
>>> > I realize there is a lot of risk in giving ssh access to infra
>>> machines, but
>>> > I think it's worth taking a look at either putting this machine in a
>>> place
>>> > where a different level of admin could access it without giving away
>>> the
>>> > keys to the entire OpenStack infrastructure or figuring out a way to
>>> set up
>>> > credentials with varying levels of access.
>>> >
>>> As a note, all the work I've been doing to help with the attack hasn't
>>> require
>>> SSH access for me to wiki.o.o. I did need infra-root help to expose our
>>> configuration safely. I'd rather take some time to see what the fixes
>>> are,
>>> having infra-root apply changes, then move them into puppet.
>>>
>>> It also has been discussed to simply disable write access to the wiki if
>>> we
>>> really want spamming to stop, obviously that will affect normal usage.
>>>
>>> > Jimmy
>>> >
>>> > Paul Belanger wrote:
>>> > >On Fri, Feb 26, 2016 at 10:12:12AM -0600, JP Maxwell wrote:
>>> > >>But if you wanted to upgrade everything, remove the mobile view
>>> extension,
>>> > >>test in a dev/staging environment then deploy to production fingers
>>> > >>crossed, I think that would be a valid approach as well.
>>> > >>
>>> > >Current review up[1]. I'll launch a node tonight / tomorrow locally
>>> to see how
>>> > >puppet reacts. I suspect there will be some issues.
>>> > >
>>> > >If infra-roots are fine with this approach, we can use that box to
>>> test against.
>>> > >
>>> > >[1] https://review.openstack.org/#/c/285405/
>>> > >
>>> > >>J.P. Maxwell | tipit.net | fibercove.com
>>> > >>On Feb 26, 2016 10:08 AM, "JP Maxwell" wrote:
>>> > >>
>>> > >>>Plus one except in this case it is much easier to know if our
>>> efforts are
>>> > >>>working on production because the spam either stops or not.
>>> > >>>
>>> > >>>J.P. Maxwell | tipit.net | fibercove.com
>>> > >>>On Feb 26, 2016 9:48 AM, "Paul Belanger"
>>> wrote:
>>> > >>>
>>> > On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote:
>>> > >I really think you might consider the option that there is a
>>> > vulnerability
>>> > >in one of the extensions. If that is the case black listing IPs
>>> will be
>>> > an
>>> > >ongoing wild goose chase.
>>> > >
>>> > >I think this would be easily proven or disproven by making the
>>> questy
>>> > >question impossible and see if the spam continues.
>>> > >
>>> > We'll have to let an infra-root make that call. Since nobody would
>>> be
>>> > able to
>>> > use the wiki. Honestly, I'd rather spend the time standing up a
>>> mirror dev
>>> >

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

On the wiki instance, my ssh access is working now. What I see in the logs
are the continuous POST requests.

M.

On Fri, Feb 26, 2016 at 6:42 PM JP Maxwell  wrote:

> Marton
>
> Where are you seeing the logs?
>
> Paul
>
> The point is that to comment out a line in VI and watch the logs in
> another window takes about a minute or two.  To submit a patch, get
> approval, push, ask someone to share the logs takes a lot longer and relies
> on other people.
>
> I understand the need for openness and I appreciate the process and the
> automation.  I think Jimmy’s point was if we want a quick fix….
>
> *J.P. Maxwell* | tipit.net | fibercove.com 
>
> On Fri, Feb 26, 2016 at 11:38 AM, Marton Kiss 
> wrote:
>
> I see a ton of incoming post requests:
>
> POST
> /w/index.php?title=Special%3ARunJobs&tasks=jobs&maxjobs=1&sigexpiry=1456508270&signature=571cfb216f944b15d2eee1c0253d08b77003328e
>
> M.
>
> On Fri, Feb 26, 2016 at 6:35 PM Marton Kiss  wrote:
>
>> Oh, I can login. So what we need?
>>
>> M.
>>
>> On Fri, Feb 26, 2016 at 6:33 PM JP Maxwell  wrote:
>>
>>> I think what Jimmy is referring to is what I was suggesting by removing
>>> the extensions / making the question impossible to answer. Basically a
>>> series of rapid fire changes while tailing the logs and seeing what stops
>>> the spam. Once you know what worked then you can submit as an official
>>> patch. But being able to quickly try these things on a server actually
>>> under attack is the fastest path toward identifying the fix.
>>>
>>> *J.P. Maxwell* | tipit.net | fibercove.com 
>>>
>>> On Fri, Feb 26, 2016 at 11:25 AM, Paul Belanger 
>>> wrote:
>>>
>>> On Fri, Feb 26, 2016 at 11:08:18AM -0600, Jimmy McArthur wrote:
>>> > Given the state of the wiki a the moment, I think taking the quickest
>>> path
>>> > to get it fixed would be prudent. Is there a way we can get JP root
>>> access
>>> > to this server, even temporarily? We get 25% of our website traffic (2
>>> > million visitors) to the wiki. I realize we're all after the same
>>> thing, but
>>> > spammers are not going to hit the dev environment, so there's really
>>> no way
>>> > to tell if teh problem is fixed without actually working directly on
>>> the
>>> > production machine. This should be a 30 minute fix.
>>> >
>>> I am still unclear what the 30min fix is. If really 30mins, then it
>>> shouldn't be
>>> hard to get the fix into our workflow. Could somebody please elaborate.
>>>
>>> If we are talking about deploying new versions of php or mediawiki
>>> manually, I
>>> not be in-favor of this. To me, while the attack sucks, we should be
>>> working on
>>> 2 fronts. Getting the help needed to mitigate the attack, then adding the
>>> changes into -infra workflow in parallel.
>>>
>>> > I realize there is a lot of risk in giving ssh access to infra
>>> machines, but
>>> > I think it's worth taking a look at either putting this machine in a
>>> place
>>> > where a different level of admin could access it without giving away
>>> the
>>> > keys to the entire OpenStack infrastructure or figuring out a way to
>>> set up
>>> > credentials with varying levels of access.
>>> >
>>> As a note, all the work I've been doing to help with the attack hasn't
>>> require
>>> SSH access for me to wiki.o.o. I did need infra-root help to expose our
>>> configuration safely. I'd rather take some time to see what the fixes
>>> are,
>>> having infra-root apply changes, then move them into puppet.
>>>
>>> It also has been discussed to simply disable write access to the wiki if
>>> we
>>> really want spamming to stop, obviously that will affect normal usage.
>>>
>>> > Jimmy
>>> >
>>> > Paul Belanger wrote:
>>> > >On Fri, Feb 26, 2016 at 10:12:12AM -0600, JP Maxwell wrote:
>>> > >>But if you wanted to upgrade everything, remove the mobile view
>>> extension,
>>> > >>test in a dev/staging environment then deploy to production fingers
>>> > >>crossed, I think that would be a valid approach as well.
>>> > >>
>>> > >Current review up[1]. I'll launch a node tonight / tomorrow locally
>>> to see how
>>> > >puppet reacts. I suspect there will be some issues.
>>> > >
>>> > >If infra-roots are fine with this approach, we can use that box to
>>> test against.
>>> > >
>>> > >[1] https://review.openstack.org/#/c/285405/
>>> > >
>>> > >>J.P. Maxwell | tipit.net | fibercove.com
>>> > >>On Feb 26, 2016 10:08 AM, "JP Maxwell" wrote:
>>> > >>
>>> > >>>Plus one except in this case it is much easier to know if our
>>> efforts are
>>> > >>>working on production because the spam either stops or not.
>>> > >>>
>>> > >>>J.P. Maxwell | tipit.net | fibercove.com
>>> > >>>On Feb 26, 2016 9:48 AM, "Paul Belanger"
>>> wrote:
>>> > >>>
>>> > On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote:
>>> > >I really think you might consider the option that there is a
>>> > vulnerability
>>> > >in one of the extensions. If that is the case black listing IPs
>>> will be
>>> > an
>>>

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

Marton
Where are you seeing the logs?
Paul
The point is that to comment out a line in VI and watch the logs in another
window takes about a minute or two. To submit a patch, get approval, push, ask
someone to share the logs takes a lot longer and relies on other people.
I understand the need for openness and I appreciate the process and the
automation. I think Jimmy’s point was if we want a quick fix….

J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com 
[http://www.fibercove.com]
On Fri, Feb 26, 2016 at 11:38 AM, Marton Kiss  wrote:
I see a ton of incoming post requests:
POST
/w/index.php?title=Special%3ARunJobs&tasks=jobs&maxjobs=1&sigexpiry=1456508270&signature=571cfb216f944b15d2eee1c0253d08b77003328e

M.
On Fri, Feb 26, 2016 at 6:35 PM Marton Kiss < marton.k...@gmail.com 
[marton.k...@gmail.com] > wrote:
Oh, I can login. So what we need?
M.
On Fri, Feb 26, 2016 at 6:33 PM JP Maxwell < j...@tipit.net [j...@tipit.net] > 
wrote:
I think what Jimmy is referring to is what I was suggesting by removing the
extensions / making the question impossible to answer. Basically a series of
rapid fire changes while tailing the logs and seeing what stops the spam. Once
you know what worked then you can submit as an official patch. But being able to
quickly try these things on a server actually under attack is the fastest path
toward identifying the fix.

J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com 
[http://www.fibercove.com]
On Fri, Feb 26, 2016 at 11:25 AM, Paul Belanger < pabelan...@redhat.com 
[pabelan...@redhat.com] > wrote:
On Fri, Feb 26, 2016 at 11:08:18AM -0600, Jimmy McArthur wrote:
> Given the state of the wiki a the moment, I think taking the quickest path
> to get it fixed would be prudent. Is there a way we can get JP root access
> to this server, even temporarily? We get 25% of our website traffic (2
> million visitors) to the wiki. I realize we're all after the same thing, but
> spammers are not going to hit the dev environment, so there's really no way
> to tell if teh problem is fixed without actually working directly on the
> production machine. This should be a 30 minute fix.
>
I am still unclear what the 30min fix is. If really 30mins, then it shouldn't be
hard to get the fix into our workflow. Could somebody please elaborate.

If we are talking about deploying new versions of php or mediawiki manually, I
not be in-favor of this. To me, while the attack sucks, we should be working on
2 fronts. Getting the help needed to mitigate the attack, then adding the
changes into -infra workflow in parallel.

> I realize there is a lot of risk in giving ssh access to infra machines, but
> I think it's worth taking a look at either putting this machine in a place
> where a different level of admin could access it without giving away the
> keys to the entire OpenStack infrastructure or figuring out a way to set up
> credentials with varying levels of access.
>
As a note, all the work I've been doing to help with the attack hasn't require
SSH access for me to wiki.o.o. I did need infra-root help to expose our
configuration safely. I'd rather take some time to see what the fixes are,
having infra-root apply changes, then move them into puppet.

It also has been discussed to simply disable write access to the wiki if we
really want spamming to stop, obviously that will affect normal usage.

> Jimmy
>
> Paul Belanger wrote:
> >On Fri, Feb 26, 2016 at 10:12:12AM -0600, JP Maxwell wrote:
> >>But if you wanted to upgrade everything, remove the mobile view extension,
> >>test in a dev/staging environment then deploy to production fingers
> >>crossed, I think that would be a valid approach as well.
> >>
> >Current review up[1]. I'll launch a node tonight / tomorrow locally to see
how
> >puppet reacts. I suspect there will be some issues.
> >
> >If infra-roots are fine with this approach, we can use that box to test
against.
> >
> >[1] https://review.openstack.org/#/c/285405/
[https://review.openstack.org/#/c/285405/]
> >
> >>J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com 
> >>[http://fibercove.com]
> >>On Feb 26, 2016 10:08 AM, "JP Maxwell"< j...@tipit.net [j...@tipit.net] > 
> >>wrote:
> >>
> >>>Plus one except in this case it is much easier to know if our efforts are
> >>>working on production because the spam either stops or not.
> >>>
> >>>J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com 
> >>>[http://fibercove.com]
> >>>On Feb 26, 2016 9:48 AM, "Paul Belanger"< pabelan...@redhat.com 
> >>>[pabelan...@redhat.com] > wrote:
> >>>
> On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote:
> >I really think you might consider the option that there is a
> vulnerability
> >in one of the extensions. If that is the case black listing IPs will be
> an
> >ongoing wild goose chase.
> >
> >I think this would be easily proven or disproven by making the questy
> >question impossible and see if the spam continues.
> >
> We'll have to le

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

I see a ton of incoming post requests:

POST
/w/index.php?title=Special%3ARunJobs&tasks=jobs&maxjobs=1&sigexpiry=1456508270&signature=571cfb216f944b15d2eee1c0253d08b77003328e

M.

On Fri, Feb 26, 2016 at 6:35 PM Marton Kiss  wrote:

> Oh, I can login. So what we need?
>
> M.
>
> On Fri, Feb 26, 2016 at 6:33 PM JP Maxwell  wrote:
>
>> I think what Jimmy is referring to is what I was suggesting by removing
>> the extensions / making the question impossible to answer.  Basically a
>> series of rapid fire changes while tailing the logs and seeing what stops
>> the spam.  Once you know what worked then you can submit as an official
>> patch.  But being able to quickly try these things on a server actually
>> under attack is the fastest path toward identifying the fix.
>>
>> *J.P. Maxwell* | tipit.net | fibercove.com 
>>
>> On Fri, Feb 26, 2016 at 11:25 AM, Paul Belanger 
>> wrote:
>>
>> On Fri, Feb 26, 2016 at 11:08:18AM -0600, Jimmy McArthur wrote:
>> > Given the state of the wiki a the moment, I think taking the quickest
>> path
>> > to get it fixed would be prudent. Is there a way we can get JP root
>> access
>> > to this server, even temporarily? We get 25% of our website traffic (2
>> > million visitors) to the wiki. I realize we're all after the same
>> thing, but
>> > spammers are not going to hit the dev environment, so there's really no
>> way
>> > to tell if teh problem is fixed without actually working directly on the
>> > production machine. This should be a 30 minute fix.
>> >
>> I am still unclear what the 30min fix is. If really 30mins, then it
>> shouldn't be
>> hard to get the fix into our workflow. Could somebody please elaborate.
>>
>> If we are talking about deploying new versions of php or mediawiki
>> manually, I
>> not be in-favor of this. To me, while the attack sucks, we should be
>> working on
>> 2 fronts. Getting the help needed to mitigate the attack, then adding the
>> changes into -infra workflow in parallel.
>>
>> > I realize there is a lot of risk in giving ssh access to infra
>> machines, but
>> > I think it's worth taking a look at either putting this machine in a
>> place
>> > where a different level of admin could access it without giving away the
>> > keys to the entire OpenStack infrastructure or figuring out a way to
>> set up
>> > credentials with varying levels of access.
>> >
>> As a note, all the work I've been doing to help with the attack hasn't
>> require
>> SSH access for me to wiki.o.o. I did need infra-root help to expose our
>> configuration safely. I'd rather take some time to see what the fixes are,
>> having infra-root apply changes, then move them into puppet.
>>
>> It also has been discussed to simply disable write access to the wiki if
>> we
>> really want spamming to stop, obviously that will affect normal usage.
>>
>> > Jimmy
>> >
>> > Paul Belanger wrote:
>> > >On Fri, Feb 26, 2016 at 10:12:12AM -0600, JP Maxwell wrote:
>> > >>But if you wanted to upgrade everything, remove the mobile view
>> extension,
>> > >>test in a dev/staging environment then deploy to production fingers
>> > >>crossed, I think that would be a valid approach as well.
>> > >>
>> > >Current review up[1]. I'll launch a node tonight / tomorrow locally to
>> see how
>> > >puppet reacts. I suspect there will be some issues.
>> > >
>> > >If infra-roots are fine with this approach, we can use that box to
>> test against.
>> > >
>> > >[1] https://review.openstack.org/#/c/285405/
>> > >
>> > >>J.P. Maxwell | tipit.net | fibercove.com
>> > >>On Feb 26, 2016 10:08 AM, "JP Maxwell" wrote:
>> > >>
>> > >>>Plus one except in this case it is much easier to know if our
>> efforts are
>> > >>>working on production because the spam either stops or not.
>> > >>>
>> > >>>J.P. Maxwell | tipit.net | fibercove.com
>> > >>>On Feb 26, 2016 9:48 AM, "Paul Belanger"
>> wrote:
>> > >>>
>> > On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote:
>> > >I really think you might consider the option that there is a
>> > vulnerability
>> > >in one of the extensions. If that is the case black listing IPs
>> will be
>> > an
>> > >ongoing wild goose chase.
>> > >
>> > >I think this would be easily proven or disproven by making the
>> questy
>> > >question impossible and see if the spam continues.
>> > >
>> > We'll have to let an infra-root make that call. Since nobody would
>> be
>> > able to
>> > use the wiki. Honestly, I'd rather spend the time standing up a
>> mirror dev
>> > instance for us to work on, rather then production.
>> > 
>> > >J.P. Maxwell | tipit.net | fibercove.com
>> > >On Feb 26, 2016 9:12 AM, "Paul Belanger"
>> wrote:
>> > >
>> > >>On Thu, Feb 25, 2016 at 08:10:34PM -0800, Elizabeth K. Joseph
>> wrote:
>> > >>>On Thu, Feb 25, 2016 at 6:35 AM, Jeremy Stanley<
>> fu...@yuggoth.org>
>> > >>wrote:
>> > On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:
>> > >Please be

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

On Fri, Feb 26, 2016 at 11:29:31AM -0600, JP Maxwell wrote:
> I think what Jimmy is referring to is what I was suggesting by removing the
> extensions / making the question impossible to answer. Basically a series of
> rapid fire changes while tailing the logs and seeing what stops the spam.
> Once
> you know what worked then you can submit as an official patch. But being
> able to
> quickly try these things on a server actually under attack is the fastest
> path
> toward identifying the fix.
> 
Right, and I don't have an issue with that approach.  Based on the work we did
yesterday, anybody can do that via our workflow. Please submit a patch to
puppet-mediawiki[1] and ping an infra-root in #openstack-infra IRC.

We can then have somebody look at the logs.  I think it is more about scheduling
the task since more infra-root as travling back from the mid-cycle last night
and today.

Last email from me, just on a plane.  Will follow up when I land.

[1] https://git.openstack.org/cgit/openstack-infra/puppet-mediawiki


> J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com
> [http://www.fibercove.com]
> On Fri, Feb 26, 2016 at 11:25 AM, Paul Belanger 
> wrote:
> On Fri, Feb 26, 2016 at 11:08:18AM -0600, Jimmy McArthur wrote:
> > Given the state of the wiki a the moment, I think taking the quickest path
> > to get it fixed would be prudent. Is there a way we can get JP root access
> > to this server, even temporarily? We get 25% of our website traffic (2
> > million visitors) to the wiki. I realize we're all after the same thing,
> but
> > spammers are not going to hit the dev environment, so there's really no
> way
> > to tell if teh problem is fixed without actually working directly on the
> > production machine. This should be a 30 minute fix.
> >
> I am still unclear what the 30min fix is. If really 30mins, then it
> shouldn't be
> hard to get the fix into our workflow. Could somebody please elaborate.
> 
> If we are talking about deploying new versions of php or mediawiki manually,
> I
> not be in-favor of this. To me, while the attack sucks, we should be working
> on
> 2 fronts. Getting the help needed to mitigate the attack, then adding the
> changes into -infra workflow in parallel.
> 
> > I realize there is a lot of risk in giving ssh access to infra machines,
> but
> > I think it's worth taking a look at either putting this machine in a place
> > where a different level of admin could access it without giving away the
> > keys to the entire OpenStack infrastructure or figuring out a way to set
> up
> > credentials with varying levels of access.
> >
> As a note, all the work I've been doing to help with the attack hasn't
> require
> SSH access for me to wiki.o.o. I did need infra-root help to expose our
> configuration safely. I'd rather take some time to see what the fixes are,
> having infra-root apply changes, then move them into puppet.
> 
> It also has been discussed to simply disable write access to the wiki if we
> really want spamming to stop, obviously that will affect normal usage.
> 
> > Jimmy
> >
> > Paul Belanger wrote:
> > >On Fri, Feb 26, 2016 at 10:12:12AM -0600, JP Maxwell wrote:
> > >>But if you wanted to upgrade everything, remove the mobile view
> extension,
> > >>test in a dev/staging environment then deploy to production fingers
> > >>crossed, I think that would be a valid approach as well.
> > >>
> > >Current review up[1]. I'll launch a node tonight / tomorrow locally to
> see
> how
> > >puppet reacts. I suspect there will be some issues.
> > >
> > >If infra-roots are fine with this approach, we can use that box to test
> against.
> > >
> > >[1] https://review.openstack.org/#/c/285405/
> > >
> > >>J.P. Maxwell | tipit.net | fibercove.com
> > >>On Feb 26, 2016 10:08 AM, "JP Maxwell" wrote:
> > >>
> > >>>Plus one except in this case it is much easier to know if our efforts
> are
> > >>>working on production because the spam either stops or not.
> > >>>
> > >>>J.P. Maxwell | tipit.net | fibercove.com
> > >>>On Feb 26, 2016 9:48 AM, "Paul Belanger" wrote:
> > >>>
> > On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote:
> > >I really think you might consider the option that there is a
> > vulnerability
> > >in one of the extensions. If that is the case black listing IPs will
> be
> > an
> > >ongoing wild goose chase.
> > >
> > >I think this would be easily proven or disproven by making the questy
> > >question impossible and see if the spam continues.
> > >
> > We'll have to let an infra-root make that call. Since nobody would be
> > able to
> > use the wiki. Honestly, I'd rather spend the time standing up a mirror
> dev
> > instance for us to work on, rather then production.
> > 
> > >J.P. Maxwell | tipit.net | fibercove.com
> > >On Feb 26, 2016 9:12 AM, "Paul Belanger"
> wrote:
> > >
> > >>On Thu, Feb 25, 2016 at 08:10:34PM -0800, Elizabeth K. Joseph wrote:
> > >>>On Thu, Feb 25, 2016 at 6:

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

Oh, I can login. So what we need?

M.

On Fri, Feb 26, 2016 at 6:33 PM JP Maxwell  wrote:

> I think what Jimmy is referring to is what I was suggesting by removing
> the extensions / making the question impossible to answer.  Basically a
> series of rapid fire changes while tailing the logs and seeing what stops
> the spam.  Once you know what worked then you can submit as an official
> patch.  But being able to quickly try these things on a server actually
> under attack is the fastest path toward identifying the fix.
>
> *J.P. Maxwell* | tipit.net | fibercove.com 
>
> On Fri, Feb 26, 2016 at 11:25 AM, Paul Belanger 
> wrote:
>
> On Fri, Feb 26, 2016 at 11:08:18AM -0600, Jimmy McArthur wrote:
> > Given the state of the wiki a the moment, I think taking the quickest
> path
> > to get it fixed would be prudent. Is there a way we can get JP root
> access
> > to this server, even temporarily? We get 25% of our website traffic (2
> > million visitors) to the wiki. I realize we're all after the same thing,
> but
> > spammers are not going to hit the dev environment, so there's really no
> way
> > to tell if teh problem is fixed without actually working directly on the
> > production machine. This should be a 30 minute fix.
> >
> I am still unclear what the 30min fix is. If really 30mins, then it
> shouldn't be
> hard to get the fix into our workflow. Could somebody please elaborate.
>
> If we are talking about deploying new versions of php or mediawiki
> manually, I
> not be in-favor of this. To me, while the attack sucks, we should be
> working on
> 2 fronts. Getting the help needed to mitigate the attack, then adding the
> changes into -infra workflow in parallel.
>
> > I realize there is a lot of risk in giving ssh access to infra machines,
> but
> > I think it's worth taking a look at either putting this machine in a
> place
> > where a different level of admin could access it without giving away the
> > keys to the entire OpenStack infrastructure or figuring out a way to set
> up
> > credentials with varying levels of access.
> >
> As a note, all the work I've been doing to help with the attack hasn't
> require
> SSH access for me to wiki.o.o. I did need infra-root help to expose our
> configuration safely. I'd rather take some time to see what the fixes are,
> having infra-root apply changes, then move them into puppet.
>
> It also has been discussed to simply disable write access to the wiki if we
> really want spamming to stop, obviously that will affect normal usage.
>
> > Jimmy
> >
> > Paul Belanger wrote:
> > >On Fri, Feb 26, 2016 at 10:12:12AM -0600, JP Maxwell wrote:
> > >>But if you wanted to upgrade everything, remove the mobile view
> extension,
> > >>test in a dev/staging environment then deploy to production fingers
> > >>crossed, I think that would be a valid approach as well.
> > >>
> > >Current review up[1]. I'll launch a node tonight / tomorrow locally to
> see how
> > >puppet reacts. I suspect there will be some issues.
> > >
> > >If infra-roots are fine with this approach, we can use that box to test
> against.
> > >
> > >[1] https://review.openstack.org/#/c/285405/
> > >
> > >>J.P. Maxwell | tipit.net | fibercove.com
> > >>On Feb 26, 2016 10:08 AM, "JP Maxwell" wrote:
> > >>
> > >>>Plus one except in this case it is much easier to know if our efforts
> are
> > >>>working on production because the spam either stops or not.
> > >>>
> > >>>J.P. Maxwell | tipit.net | fibercove.com
> > >>>On Feb 26, 2016 9:48 AM, "Paul Belanger"
> wrote:
> > >>>
> > On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote:
> > >I really think you might consider the option that there is a
> > vulnerability
> > >in one of the extensions. If that is the case black listing IPs
> will be
> > an
> > >ongoing wild goose chase.
> > >
> > >I think this would be easily proven or disproven by making the
> questy
> > >question impossible and see if the spam continues.
> > >
> > We'll have to let an infra-root make that call. Since nobody would be
> > able to
> > use the wiki. Honestly, I'd rather spend the time standing up a
> mirror dev
> > instance for us to work on, rather then production.
> > 
> > >J.P. Maxwell | tipit.net | fibercove.com
> > >On Feb 26, 2016 9:12 AM, "Paul Belanger"
> wrote:
> > >
> > >>On Thu, Feb 25, 2016 at 08:10:34PM -0800, Elizabeth K. Joseph
> wrote:
> > >>>On Thu, Feb 25, 2016 at 6:35 AM, Jeremy Stanley >
> > >>wrote:
> > On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:
> > >Please be aware that you can now create accounts under the
> mobile
> > >view in the wiki native user table. I just created an account
> for
> > >JpMaxMan. Not sure if this matters but wanted to make sure you
> > >were aware.
> > Oh, yes I think having a random garbage question/answer was in
> > fact
> > previously preventing account creation u

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack


I think what Jimmy is referring to is what I was suggesting by removing the
extensions / making the question impossible to answer. Basically a series 
of
rapid fire changes while tailing the logs and seeing what stops the spam. 
Once
you know what worked then you can submit as an official patch. But being 
able to
quickly try these things on a server actually under attack is the fastest 
path

toward identifying the fix.

J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com 
[http://www.fibercove.com]
On Fri, Feb 26, 2016 at 11:25 AM, Paul Belanger  
wrote:

On Fri, Feb 26, 2016 at 11:08:18AM -0600, Jimmy McArthur wrote:
> Given the state of the wiki a the moment, I think taking the quickest 
path
> to get it fixed would be prudent. Is there a way we can get JP root 
access

> to this server, even temporarily? We get 25% of our website traffic (2
> million visitors) to the wiki. I realize we're all after the same thing, 
but
> spammers are not going to hit the dev environment, so there's really no 
way

> to tell if teh problem is fixed without actually working directly on the
> production machine. This should be a 30 minute fix.
>
I am still unclear what the 30min fix is. If really 30mins, then it 
shouldn't be

hard to get the fix into our workflow. Could somebody please elaborate.

If we are talking about deploying new versions of php or mediawiki 
manually, I
not be in-favor of this. To me, while the attack sucks, we should be 
working on

2 fronts. Getting the help needed to mitigate the attack, then adding the
changes into -infra workflow in parallel.

> I realize there is a lot of risk in giving ssh access to infra machines, 
but
> I think it's worth taking a look at either putting this machine in a 
place

> where a different level of admin could access it without giving away the
> keys to the entire OpenStack infrastructure or figuring out a way to set 
up

> credentials with varying levels of access.
>
As a note, all the work I've been doing to help with the attack hasn't 
require

SSH access for me to wiki.o.o. I did need infra-root help to expose our
configuration safely. I'd rather take some time to see what the fixes are,
having infra-root apply changes, then move them into puppet.

It also has been discussed to simply disable write access to the wiki if we
really want spamming to stop, obviously that will affect normal usage.

> Jimmy
>
> Paul Belanger wrote:
> >On Fri, Feb 26, 2016 at 10:12:12AM -0600, JP Maxwell wrote:
> >>But if you wanted to upgrade everything, remove the mobile view 
extension,

> >>test in a dev/staging environment then deploy to production fingers
> >>crossed, I think that would be a valid approach as well.
> >>
> >Current review up[1]. I'll launch a node tonight / tomorrow locally to 
see

how
> >puppet reacts. I suspect there will be some issues.
> >
> >If infra-roots are fine with this approach, we can use that box to test
against.
> >
> >[1] https://review.openstack.org/#/c/285405/
> >
> >>J.P. Maxwell | tipit.net | fibercove.com
> >>On Feb 26, 2016 10:08 AM, "JP Maxwell" wrote:
> >>
> >>>Plus one except in this case it is much easier to know if our efforts 
are

> >>>working on production because the spam either stops or not.
> >>>
> >>>J.P. Maxwell | tipit.net | fibercove.com
> >>>On Feb 26, 2016 9:48 AM, "Paul Belanger" 
wrote:

> >>>
> On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote:
> >I really think you might consider the option that there is a
> vulnerability
> >in one of the extensions. If that is the case black listing IPs 
will be

> an
> >ongoing wild goose chase.
> >
> >I think this would be easily proven or disproven by making the 
questy

> >question impossible and see if the spam continues.
> >
> We'll have to let an infra-root make that call. Since nobody would 
be

> able to
> use the wiki. Honestly, I'd rather spend the time standing up a 
mirror dev

> instance for us to work on, rather then production.
> 
> >J.P. Maxwell | tipit.net | fibercove.com
> >On Feb 26, 2016 9:12 AM, "Paul Belanger" 
wrote:

> >
> >>On Thu, Feb 25, 2016 at 08:10:34PM -0800, Elizabeth K. Joseph 
wrote:
> >>>On Thu, Feb 25, 2016 at 6:35 AM, Jeremy 
Stanley

> >>wrote:
> On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:
> >Please be aware that you can now create accounts under the 
mobile
> >view in the wiki native user table. I just created an account 
for

> >JpMaxMan. Not sure if this matters but wanted to make sure you
> >were aware.
> Oh, yes I think having a random garbage question/answer was in
> fact
> previously preventing account creation under the mobile view. We
> probably need a way to disable mobile view account creation as 
it

> bypasses OpenID authentication entirely.
> >>>So that's what it was doing! We'll have to tackle the mobile view
> issue.
> >>>Otherwise,

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

On Fri, Feb 26, 2016 at 11:08:18AM -0600, Jimmy McArthur wrote:
> Given the state of the wiki a the moment, I think taking the quickest path
> to get it fixed would be prudent. Is there a way we can get JP root access
> to this server, even temporarily? We get 25% of our website traffic (2
> million visitors) to the wiki. I realize we're all after the same thing, but
> spammers are not going to hit the dev environment, so there's really no way
> to tell if teh problem is fixed without actually working directly on the
> production machine. This should be a 30 minute fix.
> 
I am still unclear what the 30min fix is. If really 30mins, then it shouldn't be
hard to get the fix into our workflow. Could somebody please elaborate.

If we are talking about deploying new versions of php or mediawiki manually, I
not be in-favor of this.  To me, while the attack sucks, we should be working on
2 fronts.  Getting the help needed to mitigate the attack, then adding the
changes into -infra workflow in parallel.

> I realize there is a lot of risk in giving ssh access to infra machines, but
> I think it's worth taking a look at either putting this machine in a place
> where a different level of admin could access it without giving away the
> keys to the entire OpenStack infrastructure or figuring out a way to set up
> credentials with varying levels of access.
> 
As a note, all the work I've been doing to help with the attack hasn't require
SSH access for me to wiki.o.o.  I did need infra-root help to expose our
configuration safely.  I'd rather take some time to see what the fixes are,
having infra-root apply changes, then move them into puppet.

It also has been discussed to simply disable write access to the wiki if we
really want spamming to stop, obviously that will affect normal usage.

> Jimmy
> 
> Paul Belanger wrote:
> >On Fri, Feb 26, 2016 at 10:12:12AM -0600, JP Maxwell wrote:
> >>But if you wanted to upgrade everything, remove the mobile view extension,
> >>test in a dev/staging environment then deploy to production fingers
> >>crossed, I think that would be a valid approach as well.
> >>
> >Current review up[1]. I'll launch a node tonight / tomorrow locally to see 
> >how
> >puppet reacts.  I suspect there will be some issues.
> >
> >If infra-roots are fine with this approach, we can use that box to test 
> >against.
> >
> >[1] https://review.openstack.org/#/c/285405/
> >
> >>J.P. Maxwell | tipit.net | fibercove.com
> >>On Feb 26, 2016 10:08 AM, "JP Maxwell"  wrote:
> >>
> >>>Plus one except in this case it is much easier to know if our efforts are
> >>>working on production because the spam either stops or not.
> >>>
> >>>J.P. Maxwell | tipit.net | fibercove.com
> >>>On Feb 26, 2016 9:48 AM, "Paul Belanger"  wrote:
> >>>
> On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote:
> >I really think you might consider the option that there is a
> vulnerability
> >in one of the extensions. If that is the case black listing IPs will be
> an
> >ongoing wild goose chase.
> >
> >I think this would be easily proven or disproven by making the questy
> >question impossible and see if the spam continues.
> >
> We'll have to let an infra-root make that call. Since nobody would be
> able to
> use the wiki. Honestly, I'd rather spend the time standing up a mirror dev
> instance for us to work on, rather then production.
> 
> >J.P. Maxwell | tipit.net | fibercove.com
> >On Feb 26, 2016 9:12 AM, "Paul Belanger"  wrote:
> >
> >>On Thu, Feb 25, 2016 at 08:10:34PM -0800, Elizabeth K. Joseph wrote:
> >>>On Thu, Feb 25, 2016 at 6:35 AM, Jeremy Stanley
> >>wrote:
> On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:
> >Please be aware that you can now create accounts under the mobile
> >view in the wiki native user table. I just created an account for
> >JpMaxMan.  Not sure if this matters but wanted to make sure you
> >were aware.
> Oh, yes I think having a random garbage question/answer was in
> fact
> previously preventing account creation under the mobile view. We
> probably need a way to disable mobile view account creation as it
> bypasses OpenID authentication entirely.
> >>>So that's what it was doing! We'll have to tackle the mobile view
> issue.
> >>>Otherwise, quick update here:
> >>>
> >>>The captcha didn't appear to help stem the spam tide. We'll want to
> >>>explore and start implementing some of the other solutions.
> >>>
> >>>I did some database poking around today and it does seem like all
> the
> >>>users do have launchpad accounts and email addresses.
> >>>
> >>So, I have a few hours before jumping on my plane and checked into
> this.
> >>We are
> >>using QuestyCaptcha which according to docs, should almost be
> impossible
> >>for
> >>spammers to by pass in an automated fashion.

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-26 Thread Jimmy McArthur

Given the state of the wiki a the moment, I think taking the quickest 
path to get it fixed would be prudent. Is there a way we can get JP root 
access to this server, even temporarily? We get 25% of our website 
traffic (2 million visitors) to the wiki. I realize we're all after the 
same thing, but spammers are not going to hit the dev environment, so 
there's really no way to tell if teh problem is fixed without actually 
working directly on the production machine. This should be a 30 minute fix.


I realize there is a lot of risk in giving ssh access to infra machines, 
but I think it's worth taking a look at either putting this machine in a 
place where a different level of admin could access it without giving 
away the keys to the entire OpenStack infrastructure or figuring out a 
way to set up credentials with varying levels of access.


Jimmy

Paul Belanger wrote:

On Fri, Feb 26, 2016 at 10:12:12AM -0600, JP Maxwell wrote:

But if you wanted to upgrade everything, remove the mobile view extension,
test in a dev/staging environment then deploy to production fingers
crossed, I think that would be a valid approach as well.


Current review up[1]. I'll launch a node tonight / tomorrow locally to see how
puppet reacts.  I suspect there will be some issues.

If infra-roots are fine with this approach, we can use that box to test against.

[1] https://review.openstack.org/#/c/285405/


J.P. Maxwell | tipit.net | fibercove.com
On Feb 26, 2016 10:08 AM, "JP Maxwell"  wrote:


Plus one except in this case it is much easier to know if our efforts are
working on production because the spam either stops or not.

J.P. Maxwell | tipit.net | fibercove.com
On Feb 26, 2016 9:48 AM, "Paul Belanger"  wrote:


On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote:

I really think you might consider the option that there is a

vulnerability

in one of the extensions. If that is the case black listing IPs will be

an

ongoing wild goose chase.

I think this would be easily proven or disproven by making the questy
question impossible and see if the spam continues.


We'll have to let an infra-root make that call. Since nobody would be
able to
use the wiki. Honestly, I'd rather spend the time standing up a mirror dev
instance for us to work on, rather then production.


J.P. Maxwell | tipit.net | fibercove.com
On Feb 26, 2016 9:12 AM, "Paul Belanger"  wrote:


On Thu, Feb 25, 2016 at 08:10:34PM -0800, Elizabeth K. Joseph wrote:

On Thu, Feb 25, 2016 at 6:35 AM, Jeremy Stanley

wrote:

On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:

Please be aware that you can now create accounts under the mobile
view in the wiki native user table. I just created an account for
JpMaxMan.  Not sure if this matters but wanted to make sure you
were aware.

Oh, yes I think having a random garbage question/answer was in

fact

previously preventing account creation under the mobile view. We
probably need a way to disable mobile view account creation as it
bypasses OpenID authentication entirely.

So that's what it was doing! We'll have to tackle the mobile view

issue.

Otherwise, quick update here:

The captcha didn't appear to help stem the spam tide. We'll want to
explore and start implementing some of the other solutions.

I did some database poking around today and it does seem like all

the

users do have launchpad accounts and email addresses.


So, I have a few hours before jumping on my plane and checked into

this.

We are
using QuestyCaptcha which according to docs, should almost be

impossible

for
spammers to by pass in an automated fashion.  So, either our captcha

is too

easy, or we didn't set it up properly.  I don't have SSH on wiki.o.o

so

others
will have to check logs.  I did test new pages and edits, and was

promoted

by
captcha.

As a next step, we might need to add additional apache2 configuration

to

blacklist IPs.  I am reading up on that now.


--
Elizabeth Krumbach Joseph || Lyz || pleia2

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra



___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

On Fri, Feb 26, 2016 at 10:12:12AM -0600, JP Maxwell wrote:
> But if you wanted to upgrade everything, remove the mobile view extension,
> test in a dev/staging environment then deploy to production fingers
> crossed, I think that would be a valid approach as well.
> 
Current review up[1]. I'll launch a node tonight / tomorrow locally to see how
puppet reacts.  I suspect there will be some issues.

If infra-roots are fine with this approach, we can use that box to test against.

[1] https://review.openstack.org/#/c/285405/

> J.P. Maxwell | tipit.net | fibercove.com
> On Feb 26, 2016 10:08 AM, "JP Maxwell"  wrote:
> 
> > Plus one except in this case it is much easier to know if our efforts are
> > working on production because the spam either stops or not.
> >
> > J.P. Maxwell | tipit.net | fibercove.com
> > On Feb 26, 2016 9:48 AM, "Paul Belanger"  wrote:
> >
> >> On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote:
> >> > I really think you might consider the option that there is a
> >> vulnerability
> >> > in one of the extensions. If that is the case black listing IPs will be
> >> an
> >> > ongoing wild goose chase.
> >> >
> >> > I think this would be easily proven or disproven by making the questy
> >> > question impossible and see if the spam continues.
> >> >
> >> We'll have to let an infra-root make that call. Since nobody would be
> >> able to
> >> use the wiki. Honestly, I'd rather spend the time standing up a mirror dev
> >> instance for us to work on, rather then production.
> >>
> >> > J.P. Maxwell | tipit.net | fibercove.com
> >> > On Feb 26, 2016 9:12 AM, "Paul Belanger"  wrote:
> >> >
> >> > > On Thu, Feb 25, 2016 at 08:10:34PM -0800, Elizabeth K. Joseph wrote:
> >> > > > On Thu, Feb 25, 2016 at 6:35 AM, Jeremy Stanley 
> >> > > wrote:
> >> > > > > On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:
> >> > > > >> Please be aware that you can now create accounts under the mobile
> >> > > > >> view in the wiki native user table. I just created an account for
> >> > > > >> JpMaxMan.  Not sure if this matters but wanted to make sure you
> >> > > > >> were aware.
> >> > > > >
> >> > > > > Oh, yes I think having a random garbage question/answer was in
> >> fact
> >> > > > > previously preventing account creation under the mobile view. We
> >> > > > > probably need a way to disable mobile view account creation as it
> >> > > > > bypasses OpenID authentication entirely.
> >> > > >
> >> > > > So that's what it was doing! We'll have to tackle the mobile view
> >> issue.
> >> > > >
> >> > > > Otherwise, quick update here:
> >> > > >
> >> > > > The captcha didn't appear to help stem the spam tide. We'll want to
> >> > > > explore and start implementing some of the other solutions.
> >> > > >
> >> > > > I did some database poking around today and it does seem like all
> >> the
> >> > > > users do have launchpad accounts and email addresses.
> >> > > >
> >> > > So, I have a few hours before jumping on my plane and checked into
> >> this.
> >> > > We are
> >> > > using QuestyCaptcha which according to docs, should almost be
> >> impossible
> >> > > for
> >> > > spammers to by pass in an automated fashion.  So, either our captcha
> >> is too
> >> > > easy, or we didn't set it up properly.  I don't have SSH on wiki.o.o
> >> so
> >> > > others
> >> > > will have to check logs.  I did test new pages and edits, and was
> >> promoted
> >> > > by
> >> > > captcha.
> >> > >
> >> > > As a next step, we might need to add additional apache2 configuration
> >> to
> >> > > blacklist IPs.  I am reading up on that now.
> >> > >
> >> > > > --
> >> > > > Elizabeth Krumbach Joseph || Lyz || pleia2
> >> > > >
> >> > > > ___
> >> > > > OpenStack-Infra mailing list
> >> > > > OpenStack-Infra@lists.openstack.org
> >> > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
> >> > >
> >> > > ___
> >> > > OpenStack-Infra mailing list
> >> > > OpenStack-Infra@lists.openstack.org
> >> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
> >> > >
> >>
> >

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

But if you wanted to upgrade everything, remove the mobile view extension,
test in a dev/staging environment then deploy to production fingers
crossed, I think that would be a valid approach as well.

J.P. Maxwell | tipit.net | fibercove.com
On Feb 26, 2016 10:08 AM, "JP Maxwell"  wrote:

> Plus one except in this case it is much easier to know if our efforts are
> working on production because the spam either stops or not.
>
> J.P. Maxwell | tipit.net | fibercove.com
> On Feb 26, 2016 9:48 AM, "Paul Belanger"  wrote:
>
>> On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote:
>> > I really think you might consider the option that there is a
>> vulnerability
>> > in one of the extensions. If that is the case black listing IPs will be
>> an
>> > ongoing wild goose chase.
>> >
>> > I think this would be easily proven or disproven by making the questy
>> > question impossible and see if the spam continues.
>> >
>> We'll have to let an infra-root make that call. Since nobody would be
>> able to
>> use the wiki. Honestly, I'd rather spend the time standing up a mirror dev
>> instance for us to work on, rather then production.
>>
>> > J.P. Maxwell | tipit.net | fibercove.com
>> > On Feb 26, 2016 9:12 AM, "Paul Belanger"  wrote:
>> >
>> > > On Thu, Feb 25, 2016 at 08:10:34PM -0800, Elizabeth K. Joseph wrote:
>> > > > On Thu, Feb 25, 2016 at 6:35 AM, Jeremy Stanley 
>> > > wrote:
>> > > > > On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:
>> > > > >> Please be aware that you can now create accounts under the mobile
>> > > > >> view in the wiki native user table. I just created an account for
>> > > > >> JpMaxMan.  Not sure if this matters but wanted to make sure you
>> > > > >> were aware.
>> > > > >
>> > > > > Oh, yes I think having a random garbage question/answer was in
>> fact
>> > > > > previously preventing account creation under the mobile view. We
>> > > > > probably need a way to disable mobile view account creation as it
>> > > > > bypasses OpenID authentication entirely.
>> > > >
>> > > > So that's what it was doing! We'll have to tackle the mobile view
>> issue.
>> > > >
>> > > > Otherwise, quick update here:
>> > > >
>> > > > The captcha didn't appear to help stem the spam tide. We'll want to
>> > > > explore and start implementing some of the other solutions.
>> > > >
>> > > > I did some database poking around today and it does seem like all
>> the
>> > > > users do have launchpad accounts and email addresses.
>> > > >
>> > > So, I have a few hours before jumping on my plane and checked into
>> this.
>> > > We are
>> > > using QuestyCaptcha which according to docs, should almost be
>> impossible
>> > > for
>> > > spammers to by pass in an automated fashion.  So, either our captcha
>> is too
>> > > easy, or we didn't set it up properly.  I don't have SSH on wiki.o.o
>> so
>> > > others
>> > > will have to check logs.  I did test new pages and edits, and was
>> promoted
>> > > by
>> > > captcha.
>> > >
>> > > As a next step, we might need to add additional apache2 configuration
>> to
>> > > blacklist IPs.  I am reading up on that now.
>> > >
>> > > > --
>> > > > Elizabeth Krumbach Joseph || Lyz || pleia2
>> > > >
>> > > > ___
>> > > > OpenStack-Infra mailing list
>> > > > OpenStack-Infra@lists.openstack.org
>> > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>> > >
>> > > ___
>> > > OpenStack-Infra mailing list
>> > > OpenStack-Infra@lists.openstack.org
>> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>> > >
>>
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

Plus one except in this case it is much easier to know if our efforts are
working on production because the spam either stops or not.

J.P. Maxwell | tipit.net | fibercove.com
On Feb 26, 2016 9:48 AM, "Paul Belanger"  wrote:

> On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote:
> > I really think you might consider the option that there is a
> vulnerability
> > in one of the extensions. If that is the case black listing IPs will be
> an
> > ongoing wild goose chase.
> >
> > I think this would be easily proven or disproven by making the questy
> > question impossible and see if the spam continues.
> >
> We'll have to let an infra-root make that call. Since nobody would be able
> to
> use the wiki. Honestly, I'd rather spend the time standing up a mirror dev
> instance for us to work on, rather then production.
>
> > J.P. Maxwell | tipit.net | fibercove.com
> > On Feb 26, 2016 9:12 AM, "Paul Belanger"  wrote:
> >
> > > On Thu, Feb 25, 2016 at 08:10:34PM -0800, Elizabeth K. Joseph wrote:
> > > > On Thu, Feb 25, 2016 at 6:35 AM, Jeremy Stanley 
> > > wrote:
> > > > > On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:
> > > > >> Please be aware that you can now create accounts under the mobile
> > > > >> view in the wiki native user table. I just created an account for
> > > > >> JpMaxMan.  Not sure if this matters but wanted to make sure you
> > > > >> were aware.
> > > > >
> > > > > Oh, yes I think having a random garbage question/answer was in fact
> > > > > previously preventing account creation under the mobile view. We
> > > > > probably need a way to disable mobile view account creation as it
> > > > > bypasses OpenID authentication entirely.
> > > >
> > > > So that's what it was doing! We'll have to tackle the mobile view
> issue.
> > > >
> > > > Otherwise, quick update here:
> > > >
> > > > The captcha didn't appear to help stem the spam tide. We'll want to
> > > > explore and start implementing some of the other solutions.
> > > >
> > > > I did some database poking around today and it does seem like all the
> > > > users do have launchpad accounts and email addresses.
> > > >
> > > So, I have a few hours before jumping on my plane and checked into
> this.
> > > We are
> > > using QuestyCaptcha which according to docs, should almost be
> impossible
> > > for
> > > spammers to by pass in an automated fashion.  So, either our captcha
> is too
> > > easy, or we didn't set it up properly.  I don't have SSH on wiki.o.o so
> > > others
> > > will have to check logs.  I did test new pages and edits, and was
> promoted
> > > by
> > > captcha.
> > >
> > > As a next step, we might need to add additional apache2 configuration
> to
> > > blacklist IPs.  I am reading up on that now.
> > >
> > > > --
> > > > Elizabeth Krumbach Joseph || Lyz || pleia2
> > > >
> > > > ___
> > > > OpenStack-Infra mailing list
> > > > OpenStack-Infra@lists.openstack.org
> > > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
> > >
> > > ___
> > > OpenStack-Infra mailing list
> > > OpenStack-Infra@lists.openstack.org
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
> > >
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

On Fri, Feb 26, 2016 at 09:18:00AM -0600, JP Maxwell wrote:
> I really think you might consider the option that there is a vulnerability
> in one of the extensions. If that is the case black listing IPs will be an
> ongoing wild goose chase.
> 
> I think this would be easily proven or disproven by making the questy
> question impossible and see if the spam continues.
> 
We'll have to let an infra-root make that call. Since nobody would be able to
use the wiki. Honestly, I'd rather spend the time standing up a mirror dev
instance for us to work on, rather then production.

> J.P. Maxwell | tipit.net | fibercove.com
> On Feb 26, 2016 9:12 AM, "Paul Belanger"  wrote:
> 
> > On Thu, Feb 25, 2016 at 08:10:34PM -0800, Elizabeth K. Joseph wrote:
> > > On Thu, Feb 25, 2016 at 6:35 AM, Jeremy Stanley 
> > wrote:
> > > > On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:
> > > >> Please be aware that you can now create accounts under the mobile
> > > >> view in the wiki native user table. I just created an account for
> > > >> JpMaxMan.  Not sure if this matters but wanted to make sure you
> > > >> were aware.
> > > >
> > > > Oh, yes I think having a random garbage question/answer was in fact
> > > > previously preventing account creation under the mobile view. We
> > > > probably need a way to disable mobile view account creation as it
> > > > bypasses OpenID authentication entirely.
> > >
> > > So that's what it was doing! We'll have to tackle the mobile view issue.
> > >
> > > Otherwise, quick update here:
> > >
> > > The captcha didn't appear to help stem the spam tide. We'll want to
> > > explore and start implementing some of the other solutions.
> > >
> > > I did some database poking around today and it does seem like all the
> > > users do have launchpad accounts and email addresses.
> > >
> > So, I have a few hours before jumping on my plane and checked into this.
> > We are
> > using QuestyCaptcha which according to docs, should almost be impossible
> > for
> > spammers to by pass in an automated fashion.  So, either our captcha is too
> > easy, or we didn't set it up properly.  I don't have SSH on wiki.o.o so
> > others
> > will have to check logs.  I did test new pages and edits, and was promoted
> > by
> > captcha.
> >
> > As a next step, we might need to add additional apache2 configuration to
> > blacklist IPs.  I am reading up on that now.
> >
> > > --
> > > Elizabeth Krumbach Joseph || Lyz || pleia2
> > >
> > > ___
> > > OpenStack-Infra mailing list
> > > OpenStack-Infra@lists.openstack.org
> > > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
> >
> > ___
> > OpenStack-Infra mailing list
> > OpenStack-Infra@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
> >

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

On Fri, Feb 26, 2016 at 10:34:56AM +, Marton Kiss wrote:
> I've deployed the mediawiki using our puppet modules to my dev machine, and
> we have more problems here:
> [image: The MediaWiki logo] MediaWiki 1.27 internal error
> 
> MediaWiki 1.27 requires at least PHP version 5.5.9, you are using PHP
> 5.3.10-1ubuntu3.21.
> Supported PHP versions
> 
> Please consider upgrading your copy of PHP
> . PHP versions less than 5.5.0 are no
> longer supported by the PHP Group and will not receive security or bugfix
> updates.
> 
> If for some reason you are unable to upgrade your PHP version, you will
> need to download  an older version
> of MediaWiki from our website. See our compatibility page
>  for details of which
> versions are compatible with prior versions of PHP.
> 
> The wiki.o.o seems to be running on precise, meanwhile the git consumed
> repo simply not supporting the PHP version provided there.
> 
So, wiki.o.o is running precise and already on my radar to upgrade. I can do the
leg work to stand up wiki-dev.o.o on trusty, if others would like to tackle the
migration plan for the database (mediawiki).  Ideally we should have the latest
stable release trusty support.

> M.
> 
> On Fri, Feb 26, 2016 at 5:19 AM JP Maxwell  wrote:
> 
> > Is it an option to put the question back to an impossible answer for even
> > a little while? I think it would be very telling if the spam continues then
> > there may be an exploit possibly tied to the launchpad SSO.  It would at
> > least give a clue where to focus.
> >
> > J.P. Maxwell | tipit.net | fibercove.com
> > On Feb 25, 2016 10:10 PM, "Elizabeth K. Joseph" 
> > wrote:
> >
> >> On Thu, Feb 25, 2016 at 6:35 AM, Jeremy Stanley 
> >> wrote:
> >> > On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:
> >> >> Please be aware that you can now create accounts under the mobile
> >> >> view in the wiki native user table. I just created an account for
> >> >> JpMaxMan.  Not sure if this matters but wanted to make sure you
> >> >> were aware.
> >> >
> >> > Oh, yes I think having a random garbage question/answer was in fact
> >> > previously preventing account creation under the mobile view. We
> >> > probably need a way to disable mobile view account creation as it
> >> > bypasses OpenID authentication entirely.
> >>
> >> So that's what it was doing! We'll have to tackle the mobile view issue.
> >>
> >> Otherwise, quick update here:
> >>
> >> The captcha didn't appear to help stem the spam tide. We'll want to
> >> explore and start implementing some of the other solutions.
> >>
> >> I did some database poking around today and it does seem like all the
> >> users do have launchpad accounts and email addresses.
> >>
> >> --
> >> Elizabeth Krumbach Joseph || Lyz || pleia2
> >>
> > ___
> > OpenStack-Infra mailing list
> > OpenStack-Infra@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
> >

> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra


___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

I really think you might consider the option that there is a vulnerability
in one of the extensions. If that is the case black listing IPs will be an
ongoing wild goose chase.

I think this would be easily proven or disproven by making the questy
question impossible and see if the spam continues.

J.P. Maxwell | tipit.net | fibercove.com
On Feb 26, 2016 9:12 AM, "Paul Belanger"  wrote:

> On Thu, Feb 25, 2016 at 08:10:34PM -0800, Elizabeth K. Joseph wrote:
> > On Thu, Feb 25, 2016 at 6:35 AM, Jeremy Stanley 
> wrote:
> > > On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:
> > >> Please be aware that you can now create accounts under the mobile
> > >> view in the wiki native user table. I just created an account for
> > >> JpMaxMan.  Not sure if this matters but wanted to make sure you
> > >> were aware.
> > >
> > > Oh, yes I think having a random garbage question/answer was in fact
> > > previously preventing account creation under the mobile view. We
> > > probably need a way to disable mobile view account creation as it
> > > bypasses OpenID authentication entirely.
> >
> > So that's what it was doing! We'll have to tackle the mobile view issue.
> >
> > Otherwise, quick update here:
> >
> > The captcha didn't appear to help stem the spam tide. We'll want to
> > explore and start implementing some of the other solutions.
> >
> > I did some database poking around today and it does seem like all the
> > users do have launchpad accounts and email addresses.
> >
> So, I have a few hours before jumping on my plane and checked into this.
> We are
> using QuestyCaptcha which according to docs, should almost be impossible
> for
> spammers to by pass in an automated fashion.  So, either our captcha is too
> easy, or we didn't set it up properly.  I don't have SSH on wiki.o.o so
> others
> will have to check logs.  I did test new pages and edits, and was promoted
> by
> captcha.
>
> As a next step, we might need to add additional apache2 configuration to
> blacklist IPs.  I am reading up on that now.
>
> > --
> > Elizabeth Krumbach Joseph || Lyz || pleia2
> >
> > ___
> > OpenStack-Infra mailing list
> > OpenStack-Infra@lists.openstack.org
> > http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

On Thu, Feb 25, 2016 at 08:10:34PM -0800, Elizabeth K. Joseph wrote:
> On Thu, Feb 25, 2016 at 6:35 AM, Jeremy Stanley  wrote:
> > On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:
> >> Please be aware that you can now create accounts under the mobile
> >> view in the wiki native user table. I just created an account for
> >> JpMaxMan.  Not sure if this matters but wanted to make sure you
> >> were aware.
> >
> > Oh, yes I think having a random garbage question/answer was in fact
> > previously preventing account creation under the mobile view. We
> > probably need a way to disable mobile view account creation as it
> > bypasses OpenID authentication entirely.
> 
> So that's what it was doing! We'll have to tackle the mobile view issue.
> 
> Otherwise, quick update here:
> 
> The captcha didn't appear to help stem the spam tide. We'll want to
> explore and start implementing some of the other solutions.
> 
> I did some database poking around today and it does seem like all the
> users do have launchpad accounts and email addresses.
> 
So, I have a few hours before jumping on my plane and checked into this.  We are
using QuestyCaptcha which according to docs, should almost be impossible for
spammers to by pass in an automated fashion.  So, either our captcha is too
easy, or we didn't set it up properly.  I don't have SSH on wiki.o.o so others
will have to check logs.  I did test new pages and edits, and was promoted by
captcha.

As a next step, we might need to add additional apache2 configuration to
blacklist IPs.  I am reading up on that now.

> -- 
> Elizabeth Krumbach Joseph || Lyz || pleia2
> 
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

Yeah, I'm waiting for my ssh access, it will arrive soon, so I can do a
proper clone of the site. Anyway it is interesting that mediawiki is
rendering a different output based on user agent.

M.

On Fri, Feb 26, 2016 at 2:41 PM JP Maxwell  wrote:

> Marton
>
> Make sure you are using the right upstream repository. They are in version
> 1.25. Check out: https://wiki.openstack.org/wiki/Special:Version
>
> Not that it shouldn't all be upgraded ;) be aware there seem to be config
> file formatting differences in the latest version vs 1.25 as well.
>
> J.P. Maxwell | tipit.net | fibercove.com
> On Feb 26, 2016 4:35 AM, "Marton Kiss"  wrote:
>
>> I've deployed the mediawiki using our puppet modules to my dev machine,
>> and we have more problems here:
>> [image: The MediaWiki logo] MediaWiki 1.27 internal error
>>
>> MediaWiki 1.27 requires at least PHP version 5.5.9, you are using PHP
>> 5.3.10-1ubuntu3.21.
>> Supported PHP versions
>>
>> Please consider upgrading your copy of PHP
>> . PHP versions less than 5.5.0 are no
>> longer supported by the PHP Group and will not receive security or bugfix
>> updates.
>>
>> If for some reason you are unable to upgrade your PHP version, you will
>> need to download  an older
>> version of MediaWiki from our website. See our compatibility page
>>  for details of which
>> versions are compatible with prior versions of PHP.
>>
>> The wiki.o.o seems to be running on precise, meanwhile the git consumed
>> repo simply not supporting the PHP version provided there.
>>
>> M.
>>
>> On Fri, Feb 26, 2016 at 5:19 AM JP Maxwell  wrote:
>>
>>> Is it an option to put the question back to an impossible answer for
>>> even a little while? I think it would be very telling if the spam continues
>>> then there may be an exploit possibly tied to the launchpad SSO.  It would
>>> at least give a clue where to focus.
>>>
>>> J.P. Maxwell | tipit.net | fibercove.com
>>> On Feb 25, 2016 10:10 PM, "Elizabeth K. Joseph" 
>>> wrote:
>>>
 On Thu, Feb 25, 2016 at 6:35 AM, Jeremy Stanley 
 wrote:
 > On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:
 >> Please be aware that you can now create accounts under the mobile
 >> view in the wiki native user table. I just created an account for
 >> JpMaxMan.  Not sure if this matters but wanted to make sure you
 >> were aware.
 >
 > Oh, yes I think having a random garbage question/answer was in fact
 > previously preventing account creation under the mobile view. We
 > probably need a way to disable mobile view account creation as it
 > bypasses OpenID authentication entirely.

 So that's what it was doing! We'll have to tackle the mobile view issue.

 Otherwise, quick update here:

 The captcha didn't appear to help stem the spam tide. We'll want to
 explore and start implementing some of the other solutions.

 I did some database poking around today and it does seem like all the
 users do have launchpad accounts and email addresses.

 --
 Elizabeth Krumbach Joseph || Lyz || pleia2

>>> ___
>>> OpenStack-Infra mailing list
>>> OpenStack-Infra@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>>>
>>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

Marton

Make sure you are using the right upstream repository. They are in version
1.25. Check out: https://wiki.openstack.org/wiki/Special:Version

Not that it shouldn't all be upgraded ;) be aware there seem to be config
file formatting differences in the latest version vs 1.25 as well.

J.P. Maxwell | tipit.net | fibercove.com
On Feb 26, 2016 4:35 AM, "Marton Kiss"  wrote:

> I've deployed the mediawiki using our puppet modules to my dev machine,
> and we have more problems here:
> [image: The MediaWiki logo] MediaWiki 1.27 internal error
>
> MediaWiki 1.27 requires at least PHP version 5.5.9, you are using PHP
> 5.3.10-1ubuntu3.21.
> Supported PHP versions
>
> Please consider upgrading your copy of PHP
> . PHP versions less than 5.5.0 are no
> longer supported by the PHP Group and will not receive security or bugfix
> updates.
>
> If for some reason you are unable to upgrade your PHP version, you will
> need to download  an older
> version of MediaWiki from our website. See our compatibility page
>  for details of which
> versions are compatible with prior versions of PHP.
>
> The wiki.o.o seems to be running on precise, meanwhile the git consumed
> repo simply not supporting the PHP version provided there.
>
> M.
>
> On Fri, Feb 26, 2016 at 5:19 AM JP Maxwell  wrote:
>
>> Is it an option to put the question back to an impossible answer for even
>> a little while? I think it would be very telling if the spam continues then
>> there may be an exploit possibly tied to the launchpad SSO.  It would at
>> least give a clue where to focus.
>>
>> J.P. Maxwell | tipit.net | fibercove.com
>> On Feb 25, 2016 10:10 PM, "Elizabeth K. Joseph" 
>> wrote:
>>
>>> On Thu, Feb 25, 2016 at 6:35 AM, Jeremy Stanley 
>>> wrote:
>>> > On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:
>>> >> Please be aware that you can now create accounts under the mobile
>>> >> view in the wiki native user table. I just created an account for
>>> >> JpMaxMan.  Not sure if this matters but wanted to make sure you
>>> >> were aware.
>>> >
>>> > Oh, yes I think having a random garbage question/answer was in fact
>>> > previously preventing account creation under the mobile view. We
>>> > probably need a way to disable mobile view account creation as it
>>> > bypasses OpenID authentication entirely.
>>>
>>> So that's what it was doing! We'll have to tackle the mobile view issue.
>>>
>>> Otherwise, quick update here:
>>>
>>> The captcha didn't appear to help stem the spam tide. We'll want to
>>> explore and start implementing some of the other solutions.
>>>
>>> I did some database poking around today and it does seem like all the
>>> users do have launchpad accounts and email addresses.
>>>
>>> --
>>> Elizabeth Krumbach Joseph || Lyz || pleia2
>>>
>> ___
>> OpenStack-Infra mailing list
>> OpenStack-Infra@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>>
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

I've deployed the mediawiki using our puppet modules to my dev machine, and
we have more problems here:
[image: The MediaWiki logo] MediaWiki 1.27 internal error

MediaWiki 1.27 requires at least PHP version 5.5.9, you are using PHP
5.3.10-1ubuntu3.21.
Supported PHP versions

Please consider upgrading your copy of PHP
. PHP versions less than 5.5.0 are no
longer supported by the PHP Group and will not receive security or bugfix
updates.

If for some reason you are unable to upgrade your PHP version, you will
need to download  an older version
of MediaWiki from our website. See our compatibility page
 for details of which
versions are compatible with prior versions of PHP.

The wiki.o.o seems to be running on precise, meanwhile the git consumed
repo simply not supporting the PHP version provided there.

M.

On Fri, Feb 26, 2016 at 5:19 AM JP Maxwell  wrote:

> Is it an option to put the question back to an impossible answer for even
> a little while? I think it would be very telling if the spam continues then
> there may be an exploit possibly tied to the launchpad SSO.  It would at
> least give a clue where to focus.
>
> J.P. Maxwell | tipit.net | fibercove.com
> On Feb 25, 2016 10:10 PM, "Elizabeth K. Joseph" 
> wrote:
>
>> On Thu, Feb 25, 2016 at 6:35 AM, Jeremy Stanley 
>> wrote:
>> > On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:
>> >> Please be aware that you can now create accounts under the mobile
>> >> view in the wiki native user table. I just created an account for
>> >> JpMaxMan.  Not sure if this matters but wanted to make sure you
>> >> were aware.
>> >
>> > Oh, yes I think having a random garbage question/answer was in fact
>> > previously preventing account creation under the mobile view. We
>> > probably need a way to disable mobile view account creation as it
>> > bypasses OpenID authentication entirely.
>>
>> So that's what it was doing! We'll have to tackle the mobile view issue.
>>
>> Otherwise, quick update here:
>>
>> The captcha didn't appear to help stem the spam tide. We'll want to
>> explore and start implementing some of the other solutions.
>>
>> I did some database poking around today and it does seem like all the
>> users do have launchpad accounts and email addresses.
>>
>> --
>> Elizabeth Krumbach Joseph || Lyz || pleia2
>>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-25 Thread JP Maxwell

Is it an option to put the question back to an impossible answer for even a
little while? I think it would be very telling if the spam continues then
there may be an exploit possibly tied to the launchpad SSO.  It would at
least give a clue where to focus.

J.P. Maxwell | tipit.net | fibercove.com
On Feb 25, 2016 10:10 PM, "Elizabeth K. Joseph" 
wrote:

> On Thu, Feb 25, 2016 at 6:35 AM, Jeremy Stanley  wrote:
> > On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:
> >> Please be aware that you can now create accounts under the mobile
> >> view in the wiki native user table. I just created an account for
> >> JpMaxMan.  Not sure if this matters but wanted to make sure you
> >> were aware.
> >
> > Oh, yes I think having a random garbage question/answer was in fact
> > previously preventing account creation under the mobile view. We
> > probably need a way to disable mobile view account creation as it
> > bypasses OpenID authentication entirely.
>
> So that's what it was doing! We'll have to tackle the mobile view issue.
>
> Otherwise, quick update here:
>
> The captcha didn't appear to help stem the spam tide. We'll want to
> explore and start implementing some of the other solutions.
>
> I did some database poking around today and it does seem like all the
> users do have launchpad accounts and email addresses.
>
> --
> Elizabeth Krumbach Joseph || Lyz || pleia2
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-25 Thread Elizabeth K. Joseph

On Thu, Feb 25, 2016 at 6:35 AM, Jeremy Stanley  wrote:
> On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:
>> Please be aware that you can now create accounts under the mobile
>> view in the wiki native user table. I just created an account for
>> JpMaxMan.  Not sure if this matters but wanted to make sure you
>> were aware.
>
> Oh, yes I think having a random garbage question/answer was in fact
> previously preventing account creation under the mobile view. We
> probably need a way to disable mobile view account creation as it
> bypasses OpenID authentication entirely.

So that's what it was doing! We'll have to tackle the mobile view issue.

Otherwise, quick update here:

The captcha didn't appear to help stem the spam tide. We'll want to
explore and start implementing some of the other solutions.

I did some database poking around today and it does seem like all the
users do have launchpad accounts and email addresses.

-- 
Elizabeth Krumbach Joseph || Lyz || pleia2

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-25 Thread Jeremy Stanley

On 2016-02-25 02:46:13 -0600 (-0600), JP Maxwell wrote:
> Please be aware that you can now create accounts under the mobile
> view in the wiki native user table. I just created an account for
> JpMaxMan.  Not sure if this matters but wanted to make sure you
> were aware.

Oh, yes I think having a random garbage question/answer was in fact
previously preventing account creation under the mobile view. We
probably need a way to disable mobile view account creation as it
bypasses OpenID authentication entirely.
-- 
Jeremy Stanley

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-25 Thread JP Maxwell

Please be aware that you can now create accounts under the mobile view in
the wiki native user table. I just created an account for JpMaxMan.  Not
sure if this matters but wanted to make sure you were aware.

J.P. Maxwell | tipit.net | fibercove.com
On Feb 24, 2016 6:16 PM, "Elizabeth K. Joseph"  wrote:

> On Wed, Feb 24, 2016 at 12:50 PM, Paul Belanger 
> wrote:
> > I've started updating our LocalSettings.pp based on we're talking about
> here.
> > We'll start with edit / create captcha then move to other pages if
> spaming
> > continues.
>
> We just landed a captcha change and I have tested that after I save a
> page, it then prompts for a question. on the next page before allowing
> you to fully save.
>
> The UI is not great for this, so people might miss where they are
> supposed to type in the answer to the question, but we should just
> keep an eye on this in channel as questions pop up until we make this
> better.
>
> --
> Elizabeth Krumbach Joseph || Lyz || pleia2
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-24 Thread Elizabeth K. Joseph

On Wed, Feb 24, 2016 at 12:50 PM, Paul Belanger  wrote:
> I've started updating our LocalSettings.pp based on we're talking about here.
> We'll start with edit / create captcha then move to other pages if spaming
> continues.

We just landed a captcha change and I have tested that after I save a
page, it then prompts for a question. on the next page before allowing
you to fully save.

The UI is not great for this, so people might miss where they are
supposed to type in the answer to the question, but we should just
keep an eye on this in channel as questions pop up until we make this
better.

-- 
Elizabeth Krumbach Joseph || Lyz || pleia2

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-24 Thread Paul Belanger

On Wed, Feb 24, 2016 at 02:33:41PM -0600, JP Maxwell wrote:
> It looks like you are using it (you can see it in the mobile login view),
> but it is not being used once you are logged in:
> 
> $wgGroupPermissions['user' ]['skipcaptcha'] = true;
> 
> I think you need to remove the above line. And add in the two below:
> $wgCaptchaTriggers['edit'] = true;
> $wgCaptchaTriggers['create'] = true;
> 
> 
> J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com
> [http://www.fibercove.com]
> On Wed, Feb 24, 2016 at 2:23 PM, Elizabeth K. Joseph 
> wrote:
> 
> > $wgCaptchaTriggers['edit'] = true;
> > $wgCaptchaTriggers['create'] = true;
> 
> 
> We're now storing our Settings.php (LocalSettings.php points at it) in
> git and people can submit patches against it in the puppet-mediawiki
> module:
> 
> 
> https://git.openstack.org/cgit/openstack-infra/puppet-mediawiki/tree/templates/Settings.php.erb
> 
> A quick glance looks like we're already loading QuestyCaptcha, but it
> seems to not be enabled/used?
> 
I've started updating our LocalSettings.pp based on we're talking about here.
We'll start with edit / create captcha then move to other pages if spaming
continues.

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-24 Thread Elizabeth K. Joseph

On Wed, Feb 24, 2016 at 12:33 PM, JP Maxwell  wrote:
> It looks like you are using it (you can see it
> in the mobile login view), but it is not being used once you are logged
> in:
>
> $wgGroupPermissions['user'
> ]['skipcaptcha'] = true;
>
> I think you need to remove the above line.  And add in the two below:
>
> $wgCaptchaTriggers['edit'] = true;
> $wgCaptchaTriggers['create'] = true;

Tried to grab you on IRC, but I can follow up here. Do you want to
submit this patch?

We'll also need to update the question (and add more?), since the one
there now is not useful or answerable by humans. We can stash the
answers in hiera like we have other secrets so it's not
programmatically subvertable by simply looking at git.

-- 
Elizabeth Krumbach Joseph || Lyz || pleia2

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-24 Thread JP Maxwell

It looks like you are using it (you can see it in the mobile login view), 
but it is not being used once you are logged in:

$wgGroupPermissions['user' ]['skipcaptcha'] = true;

I think you need to remove the above line. And add in the two below:
$wgCaptchaTriggers['edit'] = true;
$wgCaptchaTriggers['create'] = true;

J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com 
[http://www.fibercove.com]

On Wed, Feb 24, 2016 at 2:23 PM, Elizabeth K. Joseph 
wrote:

> $wgCaptchaTriggers['edit'] = true;
> $wgCaptchaTriggers['create'] = true;

We're now storing our Settings.php (LocalSettings.php points at it) in
git and people can submit patches against it in the puppet-mediawiki
module:

https://git.openstack.org/cgit/openstack-infra/puppet-mediawiki/tree/templates/Settings.php.erb

A quick glance looks like we're already loading QuestyCaptcha, but it
seems to not be enabled/used?

--
Elizabeth Krumbach Joseph || Lyz || pleia2___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-24 Thread Elizabeth K. Joseph

On Mon, Feb 22, 2016 at 11:34 PM, JP Maxwell  wrote:
> OK - so per the info here, you have to set the type of Captcha and add in
> editing and create page as triggers requiring Captcha.
>
> As an example to use QuestyCaptcha a the bottom of the LocalSettings.php
> file:
>
> https://www.mediawiki.org/wiki/Extension:ConfirmEdit#QuestyCaptcha
>
> And make sure the triggers are set:
>
> https://www.mediawiki.org/wiki/Extension:ConfirmEdit#Configuration
>
> So, for example (you might want to change the questions), but the below
> should at least stop the bleeding?
>
> require_once "$IP/extensions/ConfirmEdit/ConfirmEdit.php";
>
> // Use this line ONLY if your MediaWiki version is 1.25 or newer:
> //wfLoadExtension( 'ConfirmEdit/QuestyCaptcha' );
> // Use this line ONLY if your MediaWiki version is older than 1.25:
> require_once "$IP/extensions/ConfirmEdit/QuestyCaptcha.php";
>
> $wgCaptchaClass = 'QuestyCaptcha';
>
> // Add your questions in LocalSettings.php using this format
> $wgCaptchaQuestions[] = array( 'question' => "A question?", 'answer' => "An
> Answer");
> $wgCaptchaQuestions[] = array( 'question' => 'How much wood would a
> woodchuck chuck if a woodchuck could chuck wood?', 'answer' => 'as much wood
> as...' );
> $wgCaptchaQuestions[] = array( 'question' => "What is this wiki's name?",
> 'answer' => "$wgSitename" );
> // You can also provide several acceptable answers to a given question (the
> answers shall be in lowercase):
> $wgCaptchaQuestions[] = array( 'question' => "2 + 2 ?", 'answer' => array(
> '4', 'four' ) );
>
> $wgCaptchaTriggers['edit']  = true;
> $wgCaptchaTriggers['create']= true;


We're now storing our Settings.php (LocalSettings.php points at it) in
git and people can submit patches against it in the puppet-mediawiki
module:

https://git.openstack.org/cgit/openstack-infra/puppet-mediawiki/tree/templates/Settings.php.erb

A quick glance looks like we're already loading QuestyCaptcha, but it
seems to not be enabled/used?

-- 
Elizabeth Krumbach Joseph || Lyz || pleia2

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

Understood on the issue tracker.

As I've said before I love the automation dream.   Let me know what we can
do in the short term and long term.

Cheers!

J.P. Maxwell / tipit.net 


On Tue, Feb 23, 2016 at 11:46 AM, Elizabeth K. Joseph 
wrote:

> On Tue, Feb 23, 2016 at 9:33 AM, JP Maxwell  wrote:
> > Thanks Elizabeth - good info - that document answers the questions of
> where
> > the code lives and how updates are performed.  It would all require ssh
> > access to the server it seems, which I don’t have.
>
> Right, this is not the situation we want it to be. Once it's all
> Puppetized we'll be able to more easily accept contributions from
> folks who can't log in (like everything else we run).
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-23 Thread Elizabeth K. Joseph

On Tue, Feb 23, 2016 at 9:33 AM, JP Maxwell  wrote:
> Thanks Elizabeth - good info - that document answers the questions of where
> the code lives and how updates are performed.  It would all require ssh
> access to the server it seems, which I don’t have.

Right, this is not the situation we want it to be. Once it's all
Puppetized we'll be able to more easily accept contributions from
folks who can't log in (like everything else we run).

> I created an ether pad here:
>
> https://etherpad.openstack.org/p/wiki.openstack.org

Great, thanks!

> Though it would be great if we could use an issue tracker (I know there has
> been some discussion on this in the past).  We use chiliproject.org (though
> it is no longer supported) - the upstream parent Redmine might be an option.

Technically we are currently using Storyboard for task tracking, but
it's under development so using it can be a bit tricky (hooray for dog
fooding!).

As an aside: The task tracker is a whole different discussion with a
very long history and a couple competing solutions on the table right
now, so I'd rather not get into this again, or use yet another tool
for a one off project like this.

-- 
Elizabeth Krumbach Joseph || Lyz || pleia2

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

Thanks Elizabeth - good info - that document answers the questions of where the
code lives and how updates are performed. It would all require ssh access to the
server it seems, which I don’t have.
I created an ether pad here:
https://etherpad.openstack.org/p/wiki.openstack.org
[https://etherpad.openstack.org/p/wiki.openstack.org]
Though it would be great if we could use an issue tracker (I know there has been
some discussion on this in the past). We use chiliproject.org (though it is no
longer supported) - the upstream parent Redmine might be an option.

J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com 
[http://www.fibercove.com]
On Tue, Feb 23, 2016 at 11:14 AM, Elizabeth K. Joseph 
wrote:
On Tue, Feb 23, 2016 at 8:53 AM, JP Maxwell  wrote:
> Thanks Marton & Paul.
>
> Marton, however the infra community wants to handle the puppetization of the
> local settings file is fine with me. It is a very typical PHP app.
> Whatever is done we should have an easy path to update it.

Agreed. And we really do want to have it fully Puppetized in public so
folks can pitch in and help us with these things. The half-Puppetized
state makes this hard, as we can see from this thread.

As for where everything lives, we do have a little documentation on
it. Where the Puppet module lives, the exact placement of the
configuration we have Puppetized and some upgrade nodes here:
http://docs.openstack.org/infra/system-config/wiki.html

I've gone ahead and sent some sanitized Settings files from production
Paul's way, so that will get us started (thanks Paul!)

> The MediaWiki version should also be updated to the latest version at some
> point.
>
> So it seems like there are a few potential tasks around the wiki if we want
> to stay on top of things. I’m happy to help drive this if it would be
> helpful.

Tracking these tasks would indeed be helpful to us, thank you for
offering. You can start putting notes in an etherpad.openstack.org
pad, or try to use Storyboard (linked from the wiki documentation page
above).

--
Elizabeth Krumbach Joseph || Lyz || pleia2___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

I did setup a wiki and have a look at this briefly.   Can you confirm what
extensions you are loading?  When you setup the wiki it generates a
localsettings.php file that lists the extensions:

[image: Inline image 1]

# Enabled Extensions. Most extensions are enabled by including the base
extension file here
# but check specific extension documentation for more details
# The following extensions were automatically enabled:
wfLoadExtension( 'ConfirmEdit' );
wfLoadExtension( 'InputBox' );
wfLoadExtension( 'SpamBlacklist' );
wfLoadExtension( 'TitleBlacklist' );
wfLoadExtension( 'WikiEditor' );

I think if you have that ConfirmEdit extension you can enable captcha when
creating new pages / editing existing ones.  In addition, there do seem to
be some spam extensions that come built in.

J.P. Maxwell / tipit.net 

On Tue, Feb 23, 2016 at 12:43 AM, Tom Fifield  wrote:

> Hi all,
>
> Spam pages now outnumber content pages on our wiki.
>
> I have stopped trying to keep up with deleting them, after putting in
> many, many hours.
>
>
> Is there any chance someone can make the configuration changes I posted a
> couple weeks back, as an emergency measure?
>
>
> *  update Apache web server configuration to block all spammer IPs in
> lists available from www.stopforumspam.com . Alternate, use the mediawiki
> plugin that does the same:
> https://www.mediawiki.org/wiki/Extension:StopForumSpam
>
> * Configre the ConfirmEdit extension to set it to display a CAPTCHA when
> creating a new page:
>
> $wgCaptchaTriggers['create']= true;
>
>
>
> Regards,
>
>
> Tom
>
> On 15/02/16 22:05, Tom Fifield wrote:
>
>> Is there anyone who can help with this? There's still a ton of spam
>> going in, and manually cleaning it is a ton of effort.
>>
>> On 12/02/16 06:40, Tom Fifield wrote:
>>
>>> OK, so, MediaWiki nas a nice manual on how to combat spam (
>>> https://www.mediawiki.org/wiki/Manual:Combating_spam )
>>>
>>> Would it be possible to get these implemented:
>>>
>>> *  update Apache web server configuration to block all spammer IPs in
>>> lists available from www.stopforumspam.com . Alternate, use the
>>> mediawiki plugin that does the same:
>>> https://www.mediawiki.org/wiki/Extension:StopForumSpam
>>>
>>> * Configre the ConfirmEdit extension to set it to display a CAPTCHA when
>>> creating a new page:
>>>
>>> $wgCaptchaTriggers['create']= true;
>>>
>>>
>>> Regards,
>>>
>>>
>>> Tom
>>>
>>>
>>> On 12/02/16 11:08, Tom Fifield wrote:
>>>
 Hi,

 Since about 11th January, wiki.o.o has been under attack by spammers.

 They're creating new pages at a rate of more than 50 a day, with titles
 that hint at calling certain phone numbers for various services.

 If you look at

 https://wiki.openstack.org/w/index.php?title=Special:NewPages

 you'll see them.

 It's across more than a dozen user accounts. I'm going to attempt some
 cleaning, but we probably need to look for an extension to do something
 about this pro-actively.

 Regards,

 Tom

 ___
 OpenStack-Infra mailing list
 OpenStack-Infra@lists.openstack.org
 http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

>>> ___
>>> OpenStack-Infra mailing list
>>> OpenStack-Infra@lists.openstack.org
>>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>>>
>>
>> ___
>> OpenStack-Infra mailing list
>> OpenStack-Infra@lists.openstack.org
>> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>>
>
>
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
>
___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-23 Thread Elizabeth K. Joseph

On Tue, Feb 23, 2016 at 8:53 AM, JP Maxwell  wrote:
> Thanks Marton & Paul.
>
> Marton, however the infra community wants to handle the puppetization of the
> local settings file is fine with me.  It is a very typical PHP app.
> Whatever is done we should have an easy path to update it.

Agreed. And we really do want to have it fully Puppetized in public so
folks can pitch in and help us with these things. The half-Puppetized
state makes this hard, as we can see from this thread.

As for where everything lives, we do have a little documentation on
it. Where the Puppet module lives, the exact placement of the
configuration we have Puppetized and some upgrade nodes here:
http://docs.openstack.org/infra/system-config/wiki.html

I've gone ahead and sent some sanitized Settings files from production
Paul's way, so that will get us started (thanks Paul!)

> The MediaWiki version should also be updated to the latest version at some
> point.
>
> So it seems like there are a few potential tasks around the wiki if we want
> to stay on top of things.  I’m happy to help drive this if it would be
> helpful.

Tracking these tasks would indeed be helpful to us, thank you for
offering. You can start putting notes in an etherpad.openstack.org
pad, or try to use Storyboard (linked from the wiki documentation page
above).

-- 
Elizabeth Krumbach Joseph || Lyz || pleia2

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

Thanks Marton & Paul.
Marton, however the infra community wants to handle the puppetization of the
local settings file is fine with me. It is a very typical PHP app. Whatever is
done we should have an easy path to update it.
The MediaWiki version should also be updated to the latest version at some
point.
So it seems like there are a few potential tasks around the wiki if we want to
stay on top of things. I’m happy to help drive this if it would be helpful.
Paul, thanks, let me know if I can be of any assistance.

J.P. Maxwell | tipit.net [http://tipit.net] | fibercove.com 
[http://www.fibercove.com]
On Tue, Feb 23, 2016 at 9:00 AM, Marton Kiss  wrote:
It is using the openstack-infra's puppet-mediawiki module, and for first sight
this setting seems to be unmanaged by puppet. I not found any related entries in
system-config's wiki.pp. Would be great to ssh in, but just an infra core have
access for this instance. Maybe we could replace rlane's account with mine, he
is not maintaining the server as I know, but infra reviewers used to refuse
those changes. As I see this LocalSettings.php was generated during the
application installation process, but it could be moved somehow to puppet
modules.
I don't have this experience with mediawiki, but I think it is a typical php
app, and if it not implements an anti-pattern, we can add the config file to
puppet. But first of all we need to retrieve the existing file from an infra
member.
M.___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-23 Thread Paul Belanger

On Tue, Feb 23, 2016 at 02:43:55PM +0800, Tom Fifield wrote:
> Hi all,
> 
> Spam pages now outnumber content pages on our wiki.
> 
> I have stopped trying to keep up with deleting them, after putting in many,
> many hours.
> 
> 
> Is there any chance someone can make the configuration changes I posted a
> couple weeks back, as an emergency measure?
> 
> 
> *  update Apache web server configuration to block all spammer IPs in lists
> available from www.stopforumspam.com . Alternate, use the mediawiki plugin
> that does the same: https://www.mediawiki.org/wiki/Extension:StopForumSpam
> 
> * Configre the ConfirmEdit extension to set it to display a CAPTCHA when
> creating a new page:
> 
> $wgCaptchaTriggers['create']= true;
> 
I have some spare cycles to look at this today.

> 
> 
> Regards,
> 
> 
> Tom
> 
> On 15/02/16 22:05, Tom Fifield wrote:
> >Is there anyone who can help with this? There's still a ton of spam
> >going in, and manually cleaning it is a ton of effort.
> >
> >On 12/02/16 06:40, Tom Fifield wrote:
> >>OK, so, MediaWiki nas a nice manual on how to combat spam (
> >>https://www.mediawiki.org/wiki/Manual:Combating_spam )
> >>
> >>Would it be possible to get these implemented:
> >>
> >>*  update Apache web server configuration to block all spammer IPs in
> >>lists available from www.stopforumspam.com . Alternate, use the
> >>mediawiki plugin that does the same:
> >>https://www.mediawiki.org/wiki/Extension:StopForumSpam
> >>
> >>* Configre the ConfirmEdit extension to set it to display a CAPTCHA when
> >>creating a new page:
> >>
> >>$wgCaptchaTriggers['create']= true;
> >>
> >>
> >>Regards,
> >>
> >>
> >>Tom
> >>
> >>
> >>On 12/02/16 11:08, Tom Fifield wrote:
> >>>Hi,
> >>>
> >>>Since about 11th January, wiki.o.o has been under attack by spammers.
> >>>
> >>>They're creating new pages at a rate of more than 50 a day, with titles
> >>>that hint at calling certain phone numbers for various services.
> >>>
> >>>If you look at
> >>>
> >>>https://wiki.openstack.org/w/index.php?title=Special:NewPages
> >>>
> >>>you'll see them.
> >>>
> >>>It's across more than a dozen user accounts. I'm going to attempt some
> >>>cleaning, but we probably need to look for an extension to do something
> >>>about this pro-actively.
> >>>
> >>>
> >>>
> >>>Regards,
> >>>
> >>>
> >>>Tom
> >>>
> >>>
> >>>___
> >>>OpenStack-Infra mailing list
> >>>OpenStack-Infra@lists.openstack.org
> >>>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
> >>>
> >>
> >>___
> >>OpenStack-Infra mailing list
> >>OpenStack-Infra@lists.openstack.org
> >>http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
> >
> >___
> >OpenStack-Infra mailing list
> >OpenStack-Infra@lists.openstack.org
> >http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra
> 
> 
> ___
> OpenStack-Infra mailing list
> OpenStack-Infra@lists.openstack.org
> http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

___
OpenStack-Infra mailing list
OpenStack-Infra@lists.openstack.org
http://lists.openstack.org/cgi-bin/mailman/listinfo/openstack-infra

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-23 Thread Marton Kiss

It is using the openstack-infra's puppet-mediawiki module, and for first
sight this setting seems to be unmanaged by puppet. I not found any related
entries in system-config's wiki.pp. Would be great to ssh in, but just an
infra core have access for this instance. Maybe we could replace rlane's
account with mine, he is not maintaining the server as I know, but infra
reviewers used to refuse those changes. As I see this LocalSettings.php was
generated during the application installation process, but it could be
moved somehow to puppet modules.

I don't have this experience with mediawiki, but I think it is a typical
php app, and if it not implements an anti-pattern, we can add the config
file to puppet. But first of all we need to retrieve the existing file from
an infra member.

M.

On Tue, Feb 23, 2016 at 3:31 PM JP Maxwell  wrote:

> Thanks Marton. So is there a Git repo for the code or are you just relying
> on an upstream wiki media repository directly?  If so is this setting file
> populated by puppet or unmanaged?
>
> If the latter I would suggest we just ssh in and make the change to the
> file as the  wiki is being effectively owned by the spammers otherwise.
>
> Happy to do this or work with somebody on this...
>
> J.P. Maxwell | tipit.net | fibercove.com
> On Feb 23, 2016 3:40 AM, "Marton Kiss"  wrote:
>
>> Tom,
>>
>> I can help in infra contribution if required, but don't expect a quick
>> resolution, as the infra team is hell overloaded. This is the process:
>> - setup the same wiki in local dev env using infra puppet to make sure we
>> are not breaking anything irreversible in production
>> - create the patch
>> - deliver the patch to ci
>> - nagging infra core reviewers (hardest part)
>> - we can beg for an account to execute cleanup scripts to remove spam
>> content automagically
>>
>> Cheers,
>> Marton
>> JP Maxwell  (időpont: 2016. febr. 23., K, 8:59) ezt írta:
>>
>>> One final thought, I recall on the mobile view there is a secret word
>>> request in the account creation page:
>>>
>>>
>>> https://wiki.openstack.org/w/index.php?title=Special:UserLogin&type=signup&returnto=Main+Page&returntoquery=mobileaction%3Dtoggle_view_mobile%26welcome%3Dyes
>>>
>>> So, this is probably already setup.  It's possible you only need to add
>>> the triggers.   Though I might make the question something a human could
>>> reasonably figure out if you want people to continue to be able to edit the
>>> wiki in the meantime:
>>>
>>>
>>> $wgCaptchaTriggers['edit']  = true;
>>> $wgCaptchaTriggers['create']= true;
>>>
>>> J.P. Maxwell / tipit.net 
>>>
>>>
>>> On Tue, Feb 23, 2016 at 1:48 AM, JP Maxwell  wrote:
>>>
 Hah. Well, I'm not entirely sure how this is setup to manage code
 changes.  I looked in GitHub and just see the puppet configs.  Not sure
 where or how I could push changes into LocalSettings.php, otherwise I'd be
 happy to do it :D   Gotta catch a little rest now, but will check in on
 this in a few hours.

 J.P. Maxwell / tipit.net 


 On Tue, Feb 23, 2016 at 1:43 AM, Tom Fifield  wrote:

> Cheers, that's exactly what we need someone to do.
>
>
> On 23/02/16 15:34, JP Maxwell wrote:
>
>> OK - so per the info here, you have to set the type of Captcha and add
>> in editing and create page as triggers requiring Captcha.
>>
>> As an example to use QuestyCaptcha a the bottom of the
>> LocalSettings.php
>> file:
>>
>> https://www.mediawiki.org/wiki/Extension:ConfirmEdit#QuestyCaptcha
>>
>> And make sure the triggers are set:
>>
>> https://www.mediawiki.org/wiki/Extension:ConfirmEdit#Configuration
>>
>> So, for example (you might want to change the questions), but the
>> below
>> should at least stop the bleeding?
>>
>> require_once "$IP/extensions/ConfirmEdit/ConfirmEdit.php";
>>
>> // Use this line ONLY if your MediaWiki version is 1.25 or newer:
>> //wfLoadExtension( 'ConfirmEdit/QuestyCaptcha' );
>> // Use this line ONLY if your MediaWiki version is older than 1.25:
>> require_once "$IP/extensions/ConfirmEdit/QuestyCaptcha.php";
>>
>> $wgCaptchaClass = 'QuestyCaptcha';
>>
>> // Add your questions in LocalSettings.php using this format
>> $wgCaptchaQuestions[] = array( 'question' => "A question?", 'answer'
>> =>
>> "An Answer");
>> $wgCaptchaQuestions[] = array( 'question' => 'How much wood would a
>> woodchuck chuck if a woodchuck could chuck wood?', 'answer' => 'as
>> much
>> wood as...' );
>> $wgCaptchaQuestions[] = array( 'question' => "What is this wiki's
>> name?", 'answer' => "$wgSitename" );
>> // You can also provide several acceptable answers to a given question
>> (the answers shall be in lowercase):
>> $wgCaptchaQuestions[] = array( 'question' => "2 + 2 ?", 'answer' =>
>> array( '4', 'four' ) );
>>
>> $wgCa

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

Thanks Marton. So is there a Git repo for the code or are you just relying
on an upstream wiki media repository directly?  If so is this setting file
populated by puppet or unmanaged?

If the latter I would suggest we just ssh in and make the change to the
file as the  wiki is being effectively owned by the spammers otherwise.

Happy to do this or work with somebody on this...

J.P. Maxwell | tipit.net | fibercove.com
On Feb 23, 2016 3:40 AM, "Marton Kiss"  wrote:

> Tom,
>
> I can help in infra contribution if required, but don't expect a quick
> resolution, as the infra team is hell overloaded. This is the process:
> - setup the same wiki in local dev env using infra puppet to make sure we
> are not breaking anything irreversible in production
> - create the patch
> - deliver the patch to ci
> - nagging infra core reviewers (hardest part)
> - we can beg for an account to execute cleanup scripts to remove spam
> content automagically
>
> Cheers,
> Marton
> JP Maxwell  (időpont: 2016. febr. 23., K, 8:59) ezt írta:
>
>> One final thought, I recall on the mobile view there is a secret word
>> request in the account creation page:
>>
>>
>> https://wiki.openstack.org/w/index.php?title=Special:UserLogin&type=signup&returnto=Main+Page&returntoquery=mobileaction%3Dtoggle_view_mobile%26welcome%3Dyes
>>
>> So, this is probably already setup.  It's possible you only need to add
>> the triggers.   Though I might make the question something a human could
>> reasonably figure out if you want people to continue to be able to edit the
>> wiki in the meantime:
>>
>>
>> $wgCaptchaTriggers['edit']  = true;
>> $wgCaptchaTriggers['create']= true;
>>
>> J.P. Maxwell / tipit.net 
>>
>>
>> On Tue, Feb 23, 2016 at 1:48 AM, JP Maxwell  wrote:
>>
>>> Hah. Well, I'm not entirely sure how this is setup to manage code
>>> changes.  I looked in GitHub and just see the puppet configs.  Not sure
>>> where or how I could push changes into LocalSettings.php, otherwise I'd be
>>> happy to do it :D   Gotta catch a little rest now, but will check in on
>>> this in a few hours.
>>>
>>> J.P. Maxwell / tipit.net 
>>>
>>>
>>> On Tue, Feb 23, 2016 at 1:43 AM, Tom Fifield  wrote:
>>>
 Cheers, that's exactly what we need someone to do.


 On 23/02/16 15:34, JP Maxwell wrote:

> OK - so per the info here, you have to set the type of Captcha and add
> in editing and create page as triggers requiring Captcha.
>
> As an example to use QuestyCaptcha a the bottom of the
> LocalSettings.php
> file:
>
> https://www.mediawiki.org/wiki/Extension:ConfirmEdit#QuestyCaptcha
>
> And make sure the triggers are set:
>
> https://www.mediawiki.org/wiki/Extension:ConfirmEdit#Configuration
>
> So, for example (you might want to change the questions), but the below
> should at least stop the bleeding?
>
> require_once "$IP/extensions/ConfirmEdit/ConfirmEdit.php";
>
> // Use this line ONLY if your MediaWiki version is 1.25 or newer:
> //wfLoadExtension( 'ConfirmEdit/QuestyCaptcha' );
> // Use this line ONLY if your MediaWiki version is older than 1.25:
> require_once "$IP/extensions/ConfirmEdit/QuestyCaptcha.php";
>
> $wgCaptchaClass = 'QuestyCaptcha';
>
> // Add your questions in LocalSettings.php using this format
> $wgCaptchaQuestions[] = array( 'question' => "A question?", 'answer' =>
> "An Answer");
> $wgCaptchaQuestions[] = array( 'question' => 'How much wood would a
> woodchuck chuck if a woodchuck could chuck wood?', 'answer' => 'as much
> wood as...' );
> $wgCaptchaQuestions[] = array( 'question' => "What is this wiki's
> name?", 'answer' => "$wgSitename" );
> // You can also provide several acceptable answers to a given question
> (the answers shall be in lowercase):
> $wgCaptchaQuestions[] = array( 'question' => "2 + 2 ?", 'answer' =>
> array( '4', 'four' ) );
>
> $wgCaptchaTriggers['edit']  = true;
> $wgCaptchaTriggers['create']= true;
>
>
> J.P. Maxwell / tipit.net 
>
>
> On Tue, Feb 23, 2016 at 12:55 AM, Tom Fifield  > wrote:
>
> For wiki.o.o, I believe this is at:
>
> https://wiki.openstack.org/wiki/Special:Version
>
> On 23/02/16 14:51, JP Maxwell wrote:
>
> I did setup a wiki and have a look at this briefly.   Can you
> confirm
> what extensions you are loading?  When you setup the wiki it
> generates a
> localsettings.php file that lists the extensions:
>
>
>
> Inline image 1
>
> # Enabled Extensions. Most extensions are enabled by including
> the base
> extension file here
> # but check specific extension documentation for more details
> # T

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack

2016-02-23 Thread Marton Kiss

Tom,

I can help in infra contribution if required, but don't expect a quick
resolution, as the infra team is hell overloaded. This is the process:
- setup the same wiki in local dev env using infra puppet to make sure we
are not breaking anything irreversible in production
- create the patch
- deliver the patch to ci
- nagging infra core reviewers (hardest part)
- we can beg for an account to execute cleanup scripts to remove spam
content automagically

Cheers,
Marton
JP Maxwell  (időpont: 2016. febr. 23., K, 8:59) ezt írta:

> One final thought, I recall on the mobile view there is a secret word
> request in the account creation page:
>
>
> https://wiki.openstack.org/w/index.php?title=Special:UserLogin&type=signup&returnto=Main+Page&returntoquery=mobileaction%3Dtoggle_view_mobile%26welcome%3Dyes
>
> So, this is probably already setup.  It's possible you only need to add
> the triggers.   Though I might make the question something a human could
> reasonably figure out if you want people to continue to be able to edit the
> wiki in the meantime:
>
>
> $wgCaptchaTriggers['edit']  = true;
> $wgCaptchaTriggers['create']= true;
>
> J.P. Maxwell / tipit.net 
>
>
> On Tue, Feb 23, 2016 at 1:48 AM, JP Maxwell  wrote:
>
>> Hah. Well, I'm not entirely sure how this is setup to manage code
>> changes.  I looked in GitHub and just see the puppet configs.  Not sure
>> where or how I could push changes into LocalSettings.php, otherwise I'd be
>> happy to do it :D   Gotta catch a little rest now, but will check in on
>> this in a few hours.
>>
>> J.P. Maxwell / tipit.net 
>>
>>
>> On Tue, Feb 23, 2016 at 1:43 AM, Tom Fifield  wrote:
>>
>>> Cheers, that's exactly what we need someone to do.
>>>
>>>
>>> On 23/02/16 15:34, JP Maxwell wrote:
>>>
 OK - so per the info here, you have to set the type of Captcha and add
 in editing and create page as triggers requiring Captcha.

 As an example to use QuestyCaptcha a the bottom of the LocalSettings.php
 file:

 https://www.mediawiki.org/wiki/Extension:ConfirmEdit#QuestyCaptcha

 And make sure the triggers are set:

 https://www.mediawiki.org/wiki/Extension:ConfirmEdit#Configuration

 So, for example (you might want to change the questions), but the below
 should at least stop the bleeding?

 require_once "$IP/extensions/ConfirmEdit/ConfirmEdit.php";

 // Use this line ONLY if your MediaWiki version is 1.25 or newer:
 //wfLoadExtension( 'ConfirmEdit/QuestyCaptcha' );
 // Use this line ONLY if your MediaWiki version is older than 1.25:
 require_once "$IP/extensions/ConfirmEdit/QuestyCaptcha.php";

 $wgCaptchaClass = 'QuestyCaptcha';

 // Add your questions in LocalSettings.php using this format
 $wgCaptchaQuestions[] = array( 'question' => "A question?", 'answer' =>
 "An Answer");
 $wgCaptchaQuestions[] = array( 'question' => 'How much wood would a
 woodchuck chuck if a woodchuck could chuck wood?', 'answer' => 'as much
 wood as...' );
 $wgCaptchaQuestions[] = array( 'question' => "What is this wiki's
 name?", 'answer' => "$wgSitename" );
 // You can also provide several acceptable answers to a given question
 (the answers shall be in lowercase):
 $wgCaptchaQuestions[] = array( 'question' => "2 + 2 ?", 'answer' =>
 array( '4', 'four' ) );

 $wgCaptchaTriggers['edit']  = true;
 $wgCaptchaTriggers['create']= true;


 J.P. Maxwell / tipit.net 


 On Tue, Feb 23, 2016 at 12:55 AM, Tom Fifield >>> > wrote:

 For wiki.o.o, I believe this is at:

 https://wiki.openstack.org/wiki/Special:Version

 On 23/02/16 14:51, JP Maxwell wrote:

 I did setup a wiki and have a look at this briefly.   Can you
 confirm
 what extensions you are loading?  When you setup the wiki it
 generates a
 localsettings.php file that lists the extensions:



 Inline image 1

 # Enabled Extensions. Most extensions are enabled by including
 the base
 extension file here
 # but check specific extension documentation for more details
 # The following extensions were automatically enabled:
 wfLoadExtension( 'ConfirmEdit' );
 wfLoadExtension( 'InputBox' );
 wfLoadExtension( 'SpamBlacklist' );
 wfLoadExtension( 'TitleBlacklist' );
 wfLoadExtension( 'WikiEditor' );

 I think if you have that ConfirmEdit extension you can enable
 captcha
 when creating new pages / editing existing ones.  In addition,
 there do
 seem to be some spam extensions that come built in.



>>>
>>
> _

Re: [OpenStack-Infra] Wiki.o.o sustaining spam attack