Re: [Wikitech-l] Public Event Streams (AKA RCStream replacement) question

2016-09-26 Thread Joaquin Oltra Hernandez
Why not expose the websockets as a standard websocket server so that it can
be consumed by any language/platform that has a standard websocket
implementation?

https://www.npmjs.com/package/ws

Pinning to socket.io versions or other abstractions leads to what happened
before, you can get stuck on an old version, and the protocol is specific
to the library and platforms where that library has been implemented.

By using a standard websocket server, you can provide a minimal standards
compliant service that can be consumed across other languages/clients, and
if there are services that need the socket.io features you can provide a
different service that proxies the original one but puts socket.io on top
of it.

---

For the RCStream use case server sent events (SSE) are a great use case too
(given you don't need bidirectional communication), so that would make a
lot of sense too instead of websockets (it'd probably be easier to scale).

Whatever it is I'd vote for sticking to standard implementations, either
pure websockets or http server sent events, and let shim layers that
provide other features like socket.io be implemented in other proxy servers.



On Sun, Sep 25, 2016 at 4:02 PM, Merlijn van Deen (valhallasw) <
valhall...@arctus.nl> wrote:

> Hi Andrew,
>
> On 23 September 2016 at 23:15, Andrew Otto  wrote:
>
> > We’ve been busy working on building a replacement for RCStream.  This new
> > service would expose recentchanges as a stream as usual, but also other
> > types of event streams that we can make public.
> >
>
> First of all, why does it need to be a replacement, rather than something
> that builds on existing infrastructure? Re-using the existing
> infrastructure provides a much more convenient path for consumers to
> upgrade.
>
>
> > But we’re having a bit of an existential crisis!  We had originally
> chosen
> > to implement this using an up to date socket.io server, as RCStream also
> > uses socket.io.  We’re mostly finished with this, but now we are taking
> a
> > step back and wondering if socket.io/websockets are the best technology
> to
> > use to expose stream data these days.
> >
>
> For what it's worth, I'm on the fence about socket.io. My biggest argument
> for socket.io is the fact that rcstream already uses it, but my experience
> with implementing the pywikibot consumer for rcstream is that the Python
> libraries are lacking, especially when it comes to stuff like reconnecting.
> In addition, debugging issues requires knowledge of both socket.io and the
> underlying websockets layer, which are both very different from regular
> http.
>
> From the task description, I understand that the goal is to allow easy
> resumation by passing information on the the last received message. You
> could consider not implementing streaming /at all/, and just ask clients to
> poll an http endpoint, which is much easier to implement client-side than
> anything streaming (especially when it comes to handling disconnects).
>
> So: My preference would be extending the existing rcstream framework, but
> if that's not possible, my preference would be with not streaming at all.
>
> Merlijn
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Public Event Streams (AKA RCStream replacement) question

2016-09-26 Thread Andrew Otto
Thanks for feedback so far, this is great.

> If I’m not using whatever language happens to have a library already
written, there’s no spec so I have to reverse-engineer it from an
implementation.
Brad, sorry, that is just an example on the nodejs Kasocki library.  We
will need more non language specific docs about how to interact with this
service, no matter what it might be.  In either the socket.io or
SSE/EventSource (HTTP) cases, you will need a client library, so there will
be language specific code and documentation needed.  But, the interface
will be documented in a non language specific way.  Keep on me about this.
When this thing is ‘done’, if the documentation is not what you are looking
for, yell at me and I will make it better! :)

BTW, given that halfak wrote the original proposal[1] for this project, and
that he maintains a Python abstraction for MediaWiki Events[2] based on
recent changes, I wouldn’t be surprised if he (or someone) incorporated a
Python abstraction top of EventStreams, whatever transport we end up
choosing.


Antoine, you are right that we that we don’t really have a  plan for
phasing out the older systems.  RCFeed especially, since there are so many
tools built on it.  RCStream will be similar to EventStreams / Kasocki
stuff, and we know that people at least want it to use an up to date
socket.io version, so that might be easier to phase out.  I don’t even know
who maintains RCFeed.  I’ll reach out and see if I can understand this and
make a phase out plan as a subtask of the larger Public Event Streams
project.


> First of all, why does it need to be a replacement, rather than something that
builds on existing infrastructure?
​​
We want a larger feature set than the existing infrastructure provides.
RCStream is built for only the Recent Changes events, and has no historical
addressing.  Clients should be able to reconnect and start the stream from
where they last left off, or even wherever they choose.  In a dream world,
I’d love to see this thing support timestamp based consumption for any
point in time.  That is, if you wanted to start consuming a stream of edits
starting in March 2013, you could do it.

>You could consider not implementing streaming /at all/, and just ask
clients to poll an http endpoint, which is much easier to implement
client-side than anything streaming (especially when it comes to handling
disconnects).
True, but I think this would change the way people interact with this
data.  But, maybe that is ok?  I’m not sure.  I’m not a browser developer,
so I don’t know a lot about what is easy or hard in browsers (which is why
I started this thread :) ). But, keeping the stream model intact will be
powerful.  A public resumable stream of Wikimedia events would allow folks
outside of WMF networks to build realtime stream processing tooling on top
of our data.  Folks with their own Spark or Flink or Storm clusters (in
Amazon or labs or wherever) could consume this and perform complex stream
processing (e.g. machine learning algorithms (like ORES), windowed trending
aggregations, etc.).

>Why not expose the websockets as a standard web socket server so that it
can be consumed by any language/platform that has a standard web socket
implementation?
This is a good question, and not something I had considered.  I started
with socket.io because that was what RCStream used, and it seemed to have a
lot of really nice abstractions and solved problems that I’d have to deal
with myself if I used websockets.  I had assumed that socket.io was
generally preferred to working with websockets, but maybe this is not the
case?



[1]
https://meta.wikimedia.org/wiki/Research:MediaWiki_events:_a_generalized_public_event_datasource
[2] https://github.com/mediawiki-utilities/python-mwevents


On Mon, Sep 26, 2016 at 12:28 PM, Joaquin Oltra Hernandez <
jhernan...@wikimedia.org> wrote:

> Why not expose the websockets as a standard websocket server so that it can
> be consumed by any language/platform that has a standard websocket
> implementation?
>
> https://www.npmjs.com/package/ws
>
> Pinning to socket.io versions or other abstractions leads to what happened
> before, you can get stuck on an old version, and the protocol is specific
> to the library and platforms where that library has been implemented.
>
> By using a standard websocket server, you can provide a minimal standards
> compliant service that can be consumed across other languages/clients, and
> if there are services that need the socket.io features you can provide a
> different service that proxies the original one but puts socket.io on top
> of it.
>
> ---
>
> For the RCStream use case server sent events (SSE) are a great use case too
> (given you don't need bidirectional communication), so that would make a
> lot of sense too instead of websockets (it'd probably be easier to scale).
>
> Whatever it is I'd vote for sticking to standard implementations, either
> pure websockets or http server sent events, and let s

[Wikitech-l] SQLite Mediawiki support

2016-09-26 Thread Jefsey
The personal way I am using wikimedia as an SQLite textbase I can 
easily copy/backup from machine to machine and modifiy through 
external bundled applications leads me to consider there is a need 
for a wikimedia user 100% compatible "WIKILite" integrated/maintained 
solution set.


1. has something like that been investigated in the past?
2. I would be interested by comments on the idea?
3. also about the approach that can best help users and possibly 
wikimedia dévelopment?


I note that as a networked individual/professionnal I am interested 
in multi-agent oriented interwares and would like to investigate 
"wikilite" networking capabilities (both about what networked 
architectures could bring, and aboout capacity based content 
security/protection).


Thank you for your attention.
Best
jfc
 



___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Gerrit screen size

2016-09-26 Thread Bartosz Dziewoński
Indeed, the interface is pretty terrible and pretty wide. I also have
a bunch of CSS tweaks locally (mostly to make the "Verified" column
narrower, which Bryan's CSS also does).

Another thing I would recommend is going to Settings → Preferences,
and removing a bunch of entries you don't use from "My Menu" (or
editing them to make labels shorter). It made a big difference in
making the interface fit on my screen.

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Gerrit screen size

2016-09-26 Thread Paladox
Hi, I deployed your css hack here http://gerrit-test.wmflabs.org/gerrit/ 
Please could you test it and I will then upload the change for a review and 
hopefully get it merged :)



 

On Monday, 26 September 2016, 2:54, Bryan Davis  wrote:
 

 On Sun, Sep 25, 2016 at 6:41 AM, Tim Starling  wrote:
> On 25/09/16 21:09, Bináris wrote:
>> Hi,
>>
>> I try to familiarize myself with Gerrit which is not a good example for
>> user-friendly interface.
>> I noticed a letter B in the upper right corner of the screen, and I
>> suspected it could be a portion of my login name. So I looked at it in HTML
>> source, and it was. I pushed my mouse on it and I got another half window
>> as attached.
>>
>> So did somebody perhaps wire the size of a 25" monitor into page rendering?
>> My computer is a Samsung notebook.
>
> In T38471 I complained that the old version was too wide at 1163px
> (for my dashboard on a random day). Now the new version is 1520px. I'm
> not sure if the Gerrit folks are serious or are trolling us. Perhaps
> it is a tactic to encourage UI code contributions?

Sadly I don't think that's the case as the upstream has moved on to
building yet another UI layer to replace the one that we are currently
using [0].

I hacked the heck out of the CSS to make things fit in my ~1024px
preferred browser width [1]. There is a greasemonkey script that will
apply this css [2]. Perhaps some other folks could test this css out
and see if I would be a useful change to add to the stylesheet
overrides already in use by the WMF's deployment.

[0]: https://gerrit.googlesource.com/gerrit/+/master/polygerrit-ui/
[1]: https://github.com/bd808/userscripts/blob/gh-pages/wmfgerrit.user.css
[2]: http://bd808.com/userscripts/wmfgerrit.user.js

Bryan
-- 
Bryan Davis              Wikimedia Foundation    
[[m:User:BDavis_(WMF)]]  Sr Software Engineer            Boise, ID USA
irc: bd808                                        v:415.839.6885 x6855

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

   
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Public Event Streams (AKA RCStream replacement) question

2016-09-26 Thread Brad Jorsch (Anomie)
On Sun, Sep 25, 2016 at 10:02 AM, Merlijn van Deen (valhallasw) <
valhall...@arctus.nl> wrote:

> You could consider not implementing streaming /at all/, and just ask
> clients to poll an http endpoint, which is much easier to implement
> client-side than anything streaming (especially when it comes to handling
> disconnects).
>

On the other hand, polling requires repeated TCP handshakes, repeated HTTP
headers sent and received, all that work done even when there aren't any
new events, non-real-time reception of events (i.e. you only get events
when you poll), and decision on what acceptable minimum values for the
polling interval are.

And chances are that clients that want to do polling are already doing it
with the action API. ;) Although I don't know what events are planned to be
made available from this new service to be able to say whether they're all
already available via the action API.


-- 
Brad Jorsch (Anomie)
Senior Software Engineer
Wikimedia Foundation
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Gerrit screen size

2016-09-26 Thread Paladox
What does everyone think of using this skin 
https://github.com/shellscape/OctoGerrit it's more modern, doint know if it is 
mobile friendly and large screen friendly.

 

On Monday, 26 September 2016, 15:41, Paladox  
wrote:
 

 Hi, I deployed your css hack here http://gerrit-test.wmflabs.org/gerrit/ 
Please could you test it and I will then upload the change for a review and 
hopefully get it merged :)



 

On Monday, 26 September 2016, 2:54, Bryan Davis  wrote:
 

 On Sun, Sep 25, 2016 at 6:41 AM, Tim Starling  wrote:
> On 25/09/16 21:09, Bináris wrote:
>> Hi,
>>
>> I try to familiarize myself with Gerrit which is not a good example for
>> user-friendly interface.
>> I noticed a letter B in the upper right corner of the screen, and I
>> suspected it could be a portion of my login name. So I looked at it in HTML
>> source, and it was. I pushed my mouse on it and I got another half window
>> as attached.
>>
>> So did somebody perhaps wire the size of a 25" monitor into page rendering?
>> My computer is a Samsung notebook.
>
> In T38471 I complained that the old version was too wide at 1163px
> (for my dashboard on a random day). Now the new version is 1520px. I'm
> not sure if the Gerrit folks are serious or are trolling us. Perhaps
> it is a tactic to encourage UI code contributions?

Sadly I don't think that's the case as the upstream has moved on to
building yet another UI layer to replace the one that we are currently
using [0].

I hacked the heck out of the CSS to make things fit in my ~1024px
preferred browser width [1]. There is a greasemonkey script that will
apply this css [2]. Perhaps some other folks could test this css out
and see if I would be a useful change to add to the stylesheet
overrides already in use by the WMF's deployment.

[0]: https://gerrit.googlesource.com/gerrit/+/master/polygerrit-ui/
[1]: https://github.com/bd808/userscripts/blob/gh-pages/wmfgerrit.user.css
[2]: http://bd808.com/userscripts/wmfgerrit.user.js

Bryan
-- 
Bryan Davis              Wikimedia Foundation    
[[m:User:BDavis_(WMF)]]  Sr Software Engineer            Boise, ID USA
irc: bd808                                        v:415.839.6885 x6855

___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

   

   
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Gerrit screen size

2016-09-26 Thread Bináris
2016-09-26 16:54 GMT+02:00 Paladox :

> What does everyone think of using this skin https://github.com/shellscape/
> OctoGerrit it's more modern, doint know if it is mobile friendly and
> large screen friendly.
>

"Gerrit is a good tool built on a solid framework. But Gerrit, with regard
> to user experience, is bad. Really bad. Really, really bad. Functional?
> Yes. Pretty? Good lord no."
>
Fine. :-)

"OctoGerrit was written and tested on Gerrit v2.12. If it works with other
> older versions, that's wonderful! But not something we're going to test.
> OctoGerrit is not guaranteed to work on newer versions, nor the new
> 'PolyGerrit' being developed."
>
Caution!


-- 
Bináris
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Gerrit screen size

2016-09-26 Thread Ariel Glenn WMF
(off topic) Paladox, for some reason google seriously disliked your last 2
emails, just so you know. (Big read warning banner, etc.)

Ariel

On Mon, Sep 26, 2016 at 6:01 PM, Bináris  wrote:

> 2016-09-26 16:54 GMT+02:00 Paladox :
>
> > What does everyone think of using this skin
> https://github.com/shellscape/
> > OctoGerrit it's more modern, doint know if it is mobile friendly and
> > large screen friendly.
> >
>
> "Gerrit is a good tool built on a solid framework. But Gerrit, with regard
> > to user experience, is bad. Really bad. Really, really bad. Functional?
> > Yes. Pretty? Good lord no."
> >
> Fine. :-)
>
> "OctoGerrit was written and tested on Gerrit v2.12. If it works with other
> > older versions, that's wonderful! But not something we're going to test.
> > OctoGerrit is not guaranteed to work on newer versions, nor the new
> > 'PolyGerrit' being developed."
> >
> Caution!
>
>
> --
> Bináris
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
>
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Gerrit screen size

2016-09-26 Thread Paladox
Oh, I guess google hate yahoo. 

On Monday, 26 September 2016, 16:04, Ariel Glenn WMF  
wrote:
 

 (off topic) Paladox, for some reason google seriously disliked your last 2 
emails, just so you know. (Big read warning banner, etc.)

Ariel

On Mon, Sep 26, 2016 at 6:01 PM, Bináris  wrote:

2016-09-26 16:54 GMT+02:00 Paladox :

> What does everyone think of using this skin https://github.com/shellscape/
> OctoGerrit it's more modern, doint know if it is mobile friendly and
> large screen friendly.
>

"Gerrit is a good tool built on a solid framework. But Gerrit, with regard
> to user experience, is bad. Really bad. Really, really bad. Functional?
> Yes. Pretty? Good lord no."
>
Fine. :-)

"OctoGerrit was written and tested on Gerrit v2.12. If it works with other
> older versions, that's wonderful! But not something we're going to test.
> OctoGerrit is not guaranteed to work on newer versions, nor the new
> 'PolyGerrit' being developed."
>
Caution!


--
Bináris
__ _
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/ mailman/listinfo/wikitech-l



   
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] SQLite Mediawiki support

2016-09-26 Thread Aran
I developed a MediaWikiLite system many years ago which worked
reasonably well. It was for having a wiki on a memory stick that
included the content and ability to edit it in the field without net
access. It ran on SQLite and use Nanoweb, a web-server written in PHP to
reduce dependencies further.

It's very out of date now, but may be helpful:

https://www.organicdesign.co.nz/MediaWikiLite


On 26/09/16 11:00, Jefsey wrote:
> The personal way I am using wikimedia as an SQLite textbase I can
> easily copy/backup from machine to machine and modifiy through
> external bundled applications leads me to consider there is a need for
> a wikimedia user 100% compatible "WIKILite" integrated/maintained
> solution set.
>
> 1. has something like that been investigated in the past?
> 2. I would be interested by comments on the idea?
> 3. also about the approach that can best help users and possibly
> wikimedia dévelopment?
>
> I note that as a networked individual/professionnal I am interested in
> multi-agent oriented interwares and would like to investigate
> "wikilite" networking capabilities (both about what networked
> architectures could bring, and aboout capacity based content
> security/protection).
>
> Thank you for your attention.
> Best
> jfc
>  
>
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l


___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Gerrit screen size

2016-09-26 Thread Rob Lanphier
On Sun, Sep 25, 2016 at 5:41 AM, Tim Starling 
wrote:

> On 25/09/16 21:09, Bináris wrote:
> > I try to familiarize myself with Gerrit which is not a good example for
> > user-friendly interface.
> > I noticed a letter B in the upper right corner of the screen, and I
> > suspected it could be a portion of my login name. So I looked at it in
> HTML
> > source, and it was. I pushed my mouse on it and I got another half window
> > as attached.
> >
> > So did somebody perhaps wire the size of a 25" monitor into page
> rendering?
> > My computer is a Samsung notebook.
>
> In T38471 I complained that the old version was too wide at 1163px
> (for my dashboard on a random day). Now the new version is 1520px. I'm
> not sure if the Gerrit folks are serious or are trolling us. Perhaps
> it is a tactic to encourage UI code contributions?
>

My suspicion is that the Gerrit folks (in particular, Shawn Pierce) aren't
so much trolling us as saying "stop relying on the UI of Gerrit!  That's
not the point!"  The last time I was paying close attention to this, Gerrit
upstream seems to be particularly focused on building code review features
suitable for:
1.  Incorporation into git upstream
2.  Integrated into development UIs like Eclipse

The strategy seems to be "Gerrit is a reference implementation of a code
review UI for Git".  I haven't paid close enough attention to either Gerrit
upstream or Git upstream to know if the Gerrit core contributors have made
progress in getting code review functionality added to Git core (or if
they've given up, or if I completely misunderstood their strategy).  I'm
guessing that Eclipse has pretty good Gerrit support, but I rarely play
with Eclipse, so that's just a guess based on the Eclipse Foundation's
involvement with Gerrit.

As bd808 noted, Gerrit upstream seems to be working on yet another UI,
which would make sense if their goal is to create a variety of compatible
implementations of advanced Git functionality.

Rob
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Gerrit screen size

2016-09-26 Thread Paladox
There new skin called polygerrit fixes all the issues described here. It is 
moving along greatly but it doesn't support all browsers yet, namely Internet 
Explorer due to polygerrit using es6 and internet explorer does not support 
es6. They are going to something in the q2 of next year work on internet 
explorer support, hopefully it will make it into gerrit 2.14, 2.15, and 
hopefully we will still be using gerrit then and update it and also hopefully 
polygerrit will have added all the missing features.
Polygerrit is very responsive as I tried this on my mobile (iPhone) and it 
worked.

 

On Monday, 26 September 2016, 18:39, Rob Lanphier  
wrote:
 

 On Sun, Sep 25, 2016 at 5:41 AM, Tim Starling 
wrote:

> On 25/09/16 21:09, Bináris wrote:
> > I try to familiarize myself with Gerrit which is not a good example for
> > user-friendly interface.
> > I noticed a letter B in the upper right corner of the screen, and I
> > suspected it could be a portion of my login name. So I looked at it in
> HTML
> > source, and it was. I pushed my mouse on it and I got another half window
> > as attached.
> >
> > So did somebody perhaps wire the size of a 25" monitor into page
> rendering?
> > My computer is a Samsung notebook.
>
> In T38471 I complained that the old version was too wide at 1163px
> (for my dashboard on a random day). Now the new version is 1520px. I'm
> not sure if the Gerrit folks are serious or are trolling us. Perhaps
> it is a tactic to encourage UI code contributions?
>

My suspicion is that the Gerrit folks (in particular, Shawn Pierce) aren't
so much trolling us as saying "stop relying on the UI of Gerrit!  That's
not the point!"  The last time I was paying close attention to this, Gerrit
upstream seems to be particularly focused on building code review features
suitable for:
1.  Incorporation into git upstream
2.  Integrated into development UIs like Eclipse

The strategy seems to be "Gerrit is a reference implementation of a code
review UI for Git".  I haven't paid close enough attention to either Gerrit
upstream or Git upstream to know if the Gerrit core contributors have made
progress in getting code review functionality added to Git core (or if
they've given up, or if I completely misunderstood their strategy).  I'm
guessing that Eclipse has pretty good Gerrit support, but I rarely play
with Eclipse, so that's just a guess based on the Eclipse Foundation's
involvement with Gerrit.

As bd808 noted, Gerrit upstream seems to be working on yet another UI,
which would make sense if their goal is to create a variety of compatible
implementations of advanced Git functionality.

Rob
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

   
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] Public Event Streams (AKA RCStream replacement) question

2016-09-26 Thread Gergo Tisza
On Mon, Sep 26, 2016 at 5:57 AM, Andrew Otto  wrote:

>  A public resumable stream of Wikimedia events would allow folks
> outside of WMF networks to build realtime stream processing tooling on top
> of our data.  Folks with their own Spark or Flink or Storm clusters (in
> Amazon or labs or wherever) could consume this and perform complex stream
> processing (e.g. machine learning algorithms (like ORES), windowed trending
> aggregations, etc.).
>

I recall WMDE trying something similar a year ago (via PubSubHubbub) and
getting vetoed by ops. If they are not aware yet, might be worth contacting
them and asking if the new streaming service would cover their use cases
(it was about Wikidata change invalidation on third-party wikis, I think).
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

Re: [Wikitech-l] SQLite Mediawiki support

2016-09-26 Thread Brian Wolff
Mediawiki supports sqlite. And you can pretty much use with any webserver -
so are you basically asking for an installer? Some sort of xulrunner-esque
thing so the interface doesnt look like a web browser?

--
Brian

On Monday, September 26, 2016, Jefsey  wrote:
> The personal way I am using wikimedia as an SQLite textbase I can easily
copy/backup from machine to machine and modifiy through external bundled
applications leads me to consider there is a need for a wikimedia user 100%
compatible "WIKILite" integrated/maintained solution set.
>
> 1. has something like that been investigated in the past?
> 2. I would be interested by comments on the idea?
> 3. also about the approach that can best help users and possibly
wikimedia dévelopment?
>
> I note that as a networked individual/professionnal I am interested in
multi-agent oriented interwares and would like to investigate "wikilite"
networking capabilities (both about what networked architectures could
bring, and aboout capacity based content security/protection).
>
> Thank you for your attention.
> Best
> jfc
>
>
> ___
> Wikitech-l mailing list
> Wikitech-l@lists.wikimedia.org
> https://lists.wikimedia.org/mailman/listinfo/wikitech-l
___
Wikitech-l mailing list
Wikitech-l@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/wikitech-l

[Wikitech-l] Wikimedia Developer Summit 2017 discussion

2016-09-26 Thread Rob Lanphier
Hi everyone,

Abstract
---
This mail doubles as an invitation to come to this week's ArchCom
office hour (Phab:E285), and provides an attempt to provide answers to
questions about the agenda of the Wikimedia Developer Summit
(WikiDev17).  I'm hoping we find a way to work with people who can't
travel, and noting we're also working to make remote participation
more rewarding.  This emphasizes the importance of WikiDev17 being a
better event for online attendance this year, and the primacy of
online conversations in our community's decision making.

Table of contents for the rest:
* ArchCom office hour E285
* Previous Dev Summits (2014-16)
* The dangers of prior RFC requirement
* Less spectators, more participants

ArchCom office hour

This week's ArchCom IRC office hour will be this coming Wednesday,


Wednesday, 2016-09-28, 21:00 UTC (2pm PDT, 23:00 CEST) on #wikimedia-office

This is a continuation of the many conversations we've had on this
mailing list (e,g, the "Wikimedia Developer Summit 2017: Registration
Open" thread last week) and elsewhere about the summit.

Previous Dev Summits (2014-16)
--
This is an admittedly biased version of history, based on my
involvement in the program committee that is still forming.  Quim is
chairing the program committee, and I'm one of the members, but my
understanding from Quim is that we're still waiting for some invitees
to respond.

Previous years, we had a more explicit emphasis on "architecture"
(e.g. even calling our 2014 event the "Architecture Summit"[1]).  The
ties between "architecture"<->MediaWiki<->wikitech-l are very strong.
Additionally, the 2014 and 2016 events had very explicit instructions
insisting on submission of MediaWiki RFCs[2].

The benefit of requiring submission of RFCs was that it caused many
people to write down "this is what I want to talk about".  There were
many conversations leading up to last year's summit that might not
have happened without such an explicit prompt.  Many of the
unconference sessions last year were discussions that were submitted
as RFCs, but were turned down for plenary session time.  Those
unconference sessions benefited from the prep work.

The dangers of prior RFC requirement
--
We don't intend to make the requirement so tough this year.  A
challenge we face is that many topics don't fit well into RFC form.
"RFC" and "conversation" are not interchangeable terms.  We hope that
all RFCs are indeed conversations, but certainly not all conversations
belong in RFCs.  Last year, the RFC requirement also meant that all of
the scheduled topics had Phab IDs associated with them.  Is there some
other short identifier we can use as a standard conversation
identifier?  Maybe Wikidata QIDs?  ;-)

The hope is that WikiDev17 enriches conversations that are well
underway *before* everyone shows up in San Francisco.  Complicated
conversations require a shared context, but humans have traditionally
had a difficult time building shared context without physically
putting everyone in the same room at the same time.  Developers
frequently want to cram "context building" into their discussion time,
spending 70 minutes out of a 90 minute conversation time bringing
attendees up-to-speed, so that we can have a "really good" 20 minute
conversation.

A big fear: we fail to connect WMF staff developers with larger
Wikimedia community developers.  Meetings bust up unconference style
into two camps: WMF staff led discussions where participation relies
on having the kind of knowledge that one needs "insider" access to
stay abreast of, and non-WMF staff led discussions where participants
try to solve problems that WMF staff doesn't seem interested in
solving.

A huge challenge: we can't *know* in September 2016 what conversations
will be important in January 2017, but based on our experience with
past Dev Summits, it's worth creating the opportunity for important
conversations to happen.  We have plenty of conversations that are
still ongoing, and plenty of conversations many of you all likely know
need to happen in January.  Let's start the conversations we know
about now, and *hope* that they're already done before the summit

Less spectators, more participants
--
One thing we know from all of our experience: y'all don't want to make
a point of coming to San Francisco to be talked at by someone.  As in
past years, we're working to bring up to 200 people together to have
great conversations about the collective hopes of the Wikimedia
development community.  It's happening the same week as the Wikimedia
Foundation All-Staff meeting, so the attendance will be heavily biased
toward WMF staff, but we hope this isn't just us talking to ourselves.
We're working hard to provide a framework for good conversations to
happen; not for u