[Wikitech-l] Evaluation of clustering solutions (continued)

[Giuseppe Lavagetto]

Hi all,

as previously announced, we've been evaluating a "clustering solution"
for use as an alternative to GridEngine for toollabs

https://lists.wikimedia.org/pipermail/wikitech-l/2015-August/082853.html

Our goal is also to find a suitable, modern, stable tool to run not
only toollabs webservices, but also - on a longer term - to find a
modern, easier, more convenient way to run our microservices in
production: a clusterized environment that will allow us to enhance
single service availalbility and also to apply easier scaling of
applications, reducing further the friction surface and the direct ops
involvement in the day-to-day setup and deployment of services.

Our evaluation of the available solutions is ongoing, and while we're
mostly done filling up an "evaluation spreadsheet"
(https://docs.google.com/spreadsheets/d/1YkVsd8Y5wBn9fvwVQmp9Sf8K9DZCqmyJ-ew-PAOb4R4/edit?usp=sharing),
we would welcome and we encourage further involvement/suggestions. You
can provide these easily on the tracking ticket for the evaluation,
https://phabricator.wikimedia.org/T106475

We received some interesting feedback already, and we look forward
incorporating more!

 We are considering two solutions - mesospheres' Marathon (which is
based on Mesos) - https://mesosphere.github.io/marathon/ and Google's
Kubernetes https://kubernetes.io.

Now let us summarize a bit our findings so far:
MESOS/MARATHON:

Pros:
- Mesos is stable and battle tested, although Marathon is
quite young and mostly used in mesosphere's commercial offering
- Supports overcommitting resources (which is important in
toollabs, probably less so in production)
- Has a nice, clean API and is fully distributed with no potential SPOFs
- Chronos is another framework that can run on mesos and is a
great distributed cron

Cons:
- Multitenancy story is non-existent, it was not designed to
be a public PaaS offering. This is an issue even in production if we
want to grant independence to single teams.
- Container support seems experimental at best.(but getting
better in newer versions)
- Adoption of Marathon seems little and the community is not
very lively.
- Discovery/scaling logic is somewhat limited

KUBERNETES

Pros:
- The design seems to be very well thought out, based off of
experiences running Google's internal Borg system (see
http://research.google.com/pubs/pub43438.html for details of Google's
Borg clustering system).
- A pretty refined security model is already implemented, so
that single users/teams could be given access to individual namespaces
and act independently
- The community is very lively, and adoption is gaining
momentum: kubernetes is the default way to deploy apps on Google
Compute Engine, it's used by Red Hat for its own cloud solution (and
they contribute patches to it), it has a clear roadmap to overcome
most of its limitations
- Container support is native and it's tecnology-agnostic,
allowing (for now) Docker and Rkt containers to be used
- The API is quite nice
- Documentation is decently complete
- Google engineers are actively supporting us in evaluating its usage
Cons:
- The master node is not highly available, although our
cluster survived a pretty serious outage in labs that froze the master
and wiped out one worker
- No overcommitting allowed, it will be possible to mimic it
with QoS (coming in the next version)
- The ability to schedule one-off jobs is offered, but there
is no distributed cron facility
- In general it's a younger project with some outstanding bugs

As you can see there are pretty big pros/cons for both these
technologies, due to the fact they are still quite "not boring" -
although one could argue that mesos and chronos at least have entered
their "boring" stage. Our spreadsheet slightly favours Kubernetes at
the moment, but that might change drastically, if we evaluate that
some limitations are absolute showstoppers for us.

In the remainder of this week and the next few ones, we will keep
stress testing both our test installations to find out "surprises" and
bugs.

Let us know what you think - or reach out to us if you want to help in
this evaluation process. We will keep you posted!

Cheers,

Giuseppe & Yuvi

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Defining and marking inactive/unmaintained code repositories

[Andre Klapper]

Hi everybody,

one requirement for the Gerrit Cleanup Day planned for September 23rd
([1], I still need to send a proper announcement, sorry for that) is to
allow marking code repositories as inactive.

This requires agreeing on criteria what's "inactive", after which time
span, and on a process that allows people to mark projects as such.

There is a proposal in 
 https://phabricator.wikimedia.org/T102920 
concentrating on what's feasible currently with our given tools. 

It welcomes more feedback. If you're interested I kindly ask you to
take a look at the Phabricator task and add your comments there.

Thanks in advance!
andre

[1] https://phabricator.wikimedia.org/T88531
-- 
Andre Klapper | Wikimedia Bugwrangler
http://blogs.gnome.org/aklapper/



___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Lightning Talks August 25

[Rachel Farrand]

Hello!

Lightning talks start in 30 min.
Here is the link the follow along on YouTube:
http://www.youtube.com/watch?v=fnqEMIxFnGE
IRC: #wikimedia-tech

Topics and presenters here:
https://www.mediawiki.org/wiki/Lightning_Talks

Rachel

On Mon, Aug 24, 2015 at 11:43 AM, Rachel Farrand 
wrote:

> Reminder - lightning talks tomorrow!
> We have four topics.
> Participation links will be send out just in advance of the talks.
> https://www.mediawiki.org/wiki/Lightning_Talks
> Thanks,
> Rachel
>
> On Wed, Aug 12, 2015 at 10:53 AM, Rachel Farrand 
> wrote:
>
>> We (Kevin Leduc, Megan Nestler and I) will be doing a second run of
>> lightning talks  on August
>> 25. The first round had about 40 people viewing living bother remotely and
>> in SF as well as another 45 views after the talks.
>> ~ More logistic details/signups found in the email below ~
>>
>> These lightning talks are meant for anyone working on wikimedia tech and
>> are not limited to WMF engineers. If you would like to do a lightning talk
>> on any project you are working on please feel free to sign up. Make sure to
>> include contact information (or email it to me) so that we (the lightning
>> talk organizers) can coordinate with you. These lightning talks should
>> be relevant to something related to our movement and will mostly be focused
>> on tech related projects however it is not mandatory that they are related
>> to tech.
>>
>> These talks will continue every month until there is no longer interest.
>>
>> A youtube stream will be sent out to this list just before the talks start
>> so you can follow along, you can ask questions on IRC, and the talks will
>> also be recorded so that you can watch them whenever you like.
>>
>> We will adapt this process as we go to make it as interesting and useful
>> to everyone as we can.
>>
>> -- Forwarded message --
>> From: Kevin Leduc 
>> Date: Tue, Aug 11, 2015 at 6:26 PM
>> Subject:
>> To: wmf...@lists.wikimedia.org, Engineering list <
>> engineer...@lists.wikimedia.org>
>> Cc: Rachel Farrand , Megan Neisler <
>> mneis...@wikimedia.org>
>>
>>
>> Hello!
>>
>>
>> The next round of Lightning Talks are in two weeks. This effort will not
>> work without your participation...
>>
>> Lightning Talks are an opportunity for teams @ WMF & in the Community to
>> showcase a Quarterly Goal acheived, significant milestone, release, or
>> anything of significance to the rest of the foundation and the movement as
>> a whole. These talks will be open to our communities.
>>
>> Each presentation will be 10 minutes or less, the formal part should be
>> not be longer than 5 minutes and the remainder can be used for questions.
>>
>> Too many questions to answer in the allotted time?  No worries, your
>> lightning talk will be a great candidate for a future Tech Talk.
>>
>> Second Round of Lightning Talks:
>>
>> When: Tuesday August 25, 1800 UTC
>> ,
>> 11am PDT
>>
>> Where: 5th Floor
>>
>> Remotees: On-Air google hangout will be provided just before the meeting
>>
>> IRC: #wikimedia-tech
>>
>> Sign up for a 10 minute slot here:
>> https://www.mediawiki.org/wiki/Lightning_Talks
>>
>> With interest, this meeting will become a monthly event.
>>
>> Thanks!
>> Kevin Leduc, Rachel Farrand, Megan Neisler
>>
>>
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Code of conduct committee

[Frances Hocutt]

On Mon, Aug 24, 2015 at 9:01 AM, Brion Vibber  wrote:

> But I strongly recommend we seek out people who have experience with
> organizing this sort of thing before we try cobbling together an
> enforcement committee on our own, just as we seek out people with domain
> expertise on technical issues that we wish to implement.
>

Agree completely. Within the WMF, my sense is that Community Advocacy has
the most domain expertise around community enforcement mechanisms, although
the dynamics in the projects' communities of course don't map perfectly to
those in our technical community, and I'd value their input.

More generally:
>
> I think it's pretty well-known within our community (that includes me, that
> includes you if you're reading this, that includes everyone who works on
> MediaWiki, MediaWiki extensions, JS gadgets and user scripts, templates and
> Lua modules for Wikipedia, Wiktionary, and other sites, etc) that we've
> seen lot of negative interactions between people: anger, put-downs, "not my
> department", "RTFM", passive-aggressive eye-rolling sarcasm, etc -- these
> are the sorts of things that poison a community and make it harder to
> attract and retain people who start out excited about helping.
>
> I believe it's important that we explicitly acknowledge this and improve
> our community norms -- and especially our enforcement systems -- with it in
> mind. Among other things, this means that listing out specific offenses has
> only limited utility; toxic behavior can easily extend itself through
> "rules-lawyering", as I think we've all seen on Wikipedia.
>

I agree that lists of specific behaviors aren't enough to produce a
genuinely welcoming community, but it's still very important to have them
in order to address the "oh, I didn't know that was inappropriate" argument
(an unfortunately common one almost no matter what the offense). As they
say, "common sense" isn't, especially in as broad a community as Wikimedia
contributors.

-Frances
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] [Engineering] Code of conduct

[Frances Hocutt]

Thanks for sharing this, Risker. If I were a casual contributor, I'm not
sure that I would attend either--not because I'd expect that Wikimedia
conferences/hackathons would necessarily be worse than other tech
conferences, but because I'd have no reason to expect them to be better.

Same with our online spaces: I was hesitant to apply to the internship that
started me contributing technically because I'd observed profoundly
unfriendly open source projects and had also been bitten on enwiki, and I
expected the technical spaces to be similar. I'm glad I did apply. Since
then, I've been able to work with some wonderful people here and I've made
contributions I'm proud of for a cause I support. I would like to be able
to point the next person with similar worries to a community-supported code
of conduct as a reassurance that the community as a whole will have their
back even if one or a few individuals treat them poorly.

-Frances

On Sat, Aug 22, 2015 at 6:04 PM, Risker  wrote:

> David, thanks for this find.
>
> THIS is why the Code of Conduct is needed.  I recognized myself in this
> blog.  I remembered avoiding any aspect of socialization at conferences I
> had to attend for work, and simply didn't even consider attending
> conferences for any other purpose.  I remembered how readily "the guys"
> assumed that any woman there was there for more than just networking and
> learning.  I remembered having my butt pinched, my breasts "accidentally
> touched", my questions ignored or laughed at. I remember how the buzz of
> background conversation is always much louder when the speaker is a woman
> than when the speaker is a man.
>
> It's changed for me. Not because there's any less of all of this going on.
> No, it's because my hair is grey and I'm now old enough to be the mom of
> half the people in the room at any male-dominated conferences I attend; and
> outside of Wikimedia events, the conferences I go to are usually full of
> conservative businesswomen, and alcohol is rarely involved.
>
> So yeah...you need a code of conduct. Because if I was even 15 years
> younger, I'd never go to a Wikimedia conference.
>
> Risker/Anne
>
> On 22 August 2015 at 20:03, David Gerard  wrote:
>
> > I saw this today, I wonder if it's relevant to the thread:
> >
> >
> >
> http://www.perpendicularangel.com/2015/08/no-i-dont-trust-your-conference-without-a-code-of-conduct/
> >
> > Of course we're talking about stuff beyond conferences, but it still
> > applies I'd think.
> >
> >
> > - d.
> >
> > ___
> > Wikitech-l mailing list
> > Wikitech-l@lists.wikimedia.org
> > https://lists.wikimedia.org/mailman/listinfo/wikitech-l
> >
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Improving CAPTCHA friendliness for humans, and increasing CAPTCHA difficulty for bots

[Platonides]

Jamison Lofthouse wrote:

The subject sounds exactly like the reCAPTCHA<
https://www.google.com/recaptcha/intro/index.html>  tagline. Not sure how
beneficial the project would be but I have seen it used. Maybe worth
looking into.
Thanks,
Negative24


I should note that the latest reCAPTCHA¹ is actually *less* friendly.
 ¹ the “please select all images of foobars” version.


I don't think it should be considered as the “best solution” (for wikis 
where it's suitable to install), nor should we repeat their errors.


A small list:

1) It still has user assumptions based on
 1a) language: the user must understand what a "foobar" is before he 
can select those², and they aren't always common use words, precisely.


 1b) cultural: will the user easily discover all the photos expected to 
be coffee if that's not a common beverage on his country?


 ² no, the sample image³ is not enough to discern what they want. At 
least with the expected easiness.


 ³ confusing UI btw, since the naive assumption would be to expect you 
also had to tick it (which is disabled).




2) confusing images: Sometimes it's not clear what is depicted in the 
photograph. Not even being a human.




3) wrong images: Sometimes there are images that are not really foobars 
(suppose they are the similar barfoos), and thus *shouldn't* be marked 
as such. But according to recaptcha they are. (and your grudgingly 
selection of the barfoo in order to pass the captcha probably means that 
Google is performing a wrong training reinforcing their idea that it is 
indeed a foobar)





In terms of difficulty for humans I would score them as:
images recaptcha > original reCaptcha > door numbers recaptcha > 
nocaptcha recaptcha


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Improving CAPTCHA friendliness for humans, and increasing CAPTCHA difficulty for bots

[Dan Garry]

On 19 August 2015 at 23:06, Matthew Flaschen 
wrote:
>
> I agree this is an important issue.  It just isn't resourced right now by
> the WMF, as far as I know.
>

Indeed. There have been numerous discussions about the captcha on this
mailing list over the past few years, but no progress because this issue
just isn't being worked on by anyone.

Dan

-- 
Dan Garry
Lead Product Manager, Discovery
Wikimedia Foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Improving CAPTCHA friendliness for humans, and increasing CAPTCHA difficulty for bots

[Brian Wolff]

On 8/25/15, Dan Garry  wrote:
> On 19 August 2015 at 23:06, Matthew Flaschen 
> wrote:
>>
>> I agree this is an important issue.  It just isn't resourced right now by
>> the WMF, as far as I know.
>>
>
> Indeed. There have been numerous discussions about the captcha on this
> mailing list over the past few years, but no progress because this issue
> just isn't being worked on by anyone.
>
> Dan
>

Well there's that, but its also because its very unclear what "we"
actually want and how we would get there.

Make captchas not suck, well a great goal, is not an action plan in itself.

--
-bawolff

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] renaming Wikimedia domains

[Amir E. Aharoni]

Hi,

Is it possible in 2015 to rename Wikimedia domains?

The usual domain name structure for a Wikimedia project is
languageCode.project.org: en.wikipedia.org, it.wikisource.org, etc.

In a few projects the language code in the domain is different from the
actual language code: 'als' is a code for Tosk Albanian, but
als.wikipedia.org is written in Alemanic; 'no' is a code for both Bokmal
Norwegian and Nynorsk Norwegian, but no.wikipedia.org is only Bokmal; and
there are several other examples.

In the past when requests to rename such domains were raised, the usual
replies were along the lines of "it's impossible" or "it's not worth the
technical effort", but I don't know the details.

Is this still correct in 2015?

I would love to get that done, because these inconsistent and non-standard
codes repeatedly cause issues in various languages applications, the
current big one being ContentTranslation.

Thanks!

--
Amir Elisha Aharoni · אָמִיר אֱלִישָׁע אַהֲרוֹנִי
http://aharoni.wordpress.com
‪“We're living in pieces,
I want to live in peace.” – T. Moore‬
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] renaming Wikimedia domains

[Kartik Mistry]

On Wed, Aug 26, 2015 at 10:50 AM, Amir E. Aharoni
 wrote:
> In a few projects the language code in the domain is different from the
> actual language code: 'als' is a code for Tosk Albanian, but
> als.wikipedia.org is written in Alemanic; 'no' is a code for both Bokmal
> Norwegian and Nynorsk Norwegian, but no.wikipedia.org is only Bokmal; and
> there are several other examples.
>
> In the past when requests to rename such domains were raised, the usual
> replies were along the lines of "it's impossible" or "it's not worth the
> technical effort", but I don't know the details.

Also see: https://phabricator.wikimedia.org/T21986 (Wikis waiting to
be renamed (tracking))

-- 
Kartik Mistry/કાર્તિક મિસ્ત્રી | IRC: kart_
{kartikm, 0x1f1f}.wordpress.com

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Tutorials / HowTos for VisualEditor

[Robert Vogel]

Hi everybody!

I want to write an extension for the VisualEditor (great work by the way) and I 
need a little help. I've browsed through various documentation [1-4](and more) 
and already learned that using RL module 
'ext.visualEditor.desktopArticleTarget.init'  and 'mw.lib.ve.addPlugin' method 
is the way to go. Unfortunately my attempts to hook into the user interface by 
binding on events provided by the components (like 'save' from 
've.ui.MWSaveDialog' or 've.init.mw.Target') have not been successful yet. 
Maybe you could give me a little push into the right direction.

Are there any HowTos or Tutorials for VisualEditor that cover topics like

1.   Adding buttons to the toolbar

2.   Accessing the data model (e.g. retrieve all set categories from the 
Parsoid DOM)

3.   Extending/replacing a single tool (e.g. the GalleryInspector)

4.   Binding on components events (e.g. the 'save' event, to analyze 
content and maybe abort saving)

Points 1 and 2 get more or less explained by the docs I've found. But 3 and 4 
are very unclear to me. Some hints would be appreciated!

[1] 
https://www.mediawiki.org/wiki/VisualEditor_gadgets#Gadget_-_Registering_VE_plugin
[2] https://en.wikipedia.org/wiki/User:%D7%A2%D7%A8%D7%9F/veReplace.js
[3] https://doc.wikimedia.org/VisualEditor/master
[4] https://www.mediawiki.org/wiki/OOjs_UI/Toolbars

Greetings,
Robert Vogel
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l