Re: [Services] EventStreams launch and RCStream deprecation

[Andrew Otto]

Alright, we’ve done it!

RCStream is disabled, so any remaining socket.io service connecting to
stream.wikimedia.org/rc will fail.

Thanks all!


On Thu, Jun 22, 2017 at 1:00 PM, Andrew Otto  wrote:

> Hi all,
>
> This is just a friendly reminder that we plan to turn off the RCStream
> service after July 7th.
>
> We’re tracking as best we can the progress of porting clients over at
> https://phabricator.wikimedia.org/T156919.  But, we can only help with
> what we know about.  If you’ve got something still running on RCStream that
> hasn’t yet ported, let us know, and/or switch soon!
>
> Thanks!
> -Andrew Otto
>
>
>
> On Wed, Feb 8, 2017 at 9:28 AM, Andrew Otto  wrote:
>
>> Hi everyone!
>>
>> Wikimedia is releasing a new service today: EventStreams
>> .  This service allows
>> us to publish arbitrary streams of JSON event data to the public.
>> Initially, the only stream available will be good ol’ RecentChanges
>> .  This event stream
>> overlaps functionality already provided by irc.wikimedia.org and RCStream
>> .  However, this new
>> service has advantages over these (now deprecated) services.
>>
>>
>>1.
>>
>>We can expose more than just RecentChanges.
>>2.
>>
>>Events are delivered over streaming HTTP (chunked transfer) instead
>>of IRC or socket.io.  This requires less client side code and fewer
>>special routing cases on the server side.
>>3.
>>
>>Streams can be resumed from the past.  By using EventSource, a
>>disconnected client will automatically resume the stream from where it 
>> left
>>off, as long as it resumes within one week.  In the future, we would like
>>to allow users to specify historical timestamps from which they would like
>>to begin consuming, if this proves safe and tractable.
>>
>>
>> I did say deprecated!  Okay okay, we may never be able to fully deprecate
>> irc.wikimedia.org.  It’s used by too many (probably sentient by now)
>> bots out there.  We do plan to obsolete RCStream, and to turn it off in a
>> reasonable amount of time.  The deadline iis July 7th, 2017.  All
>> services that rely on RCStream should migrate to the HTTP based
>> EventStreams service by this date.  We are committed to assisting you in
>> this transition, so let us know how we can help.
>>
>> Unfortunately, unlike RCStream, EventStreams does not have server side
>> event filtering (e.g. by wiki) quite yet.  How and if this should be done
>> is still under discussion .
>>
>> The RecentChanges data you are used to remains the same, and is available
>> at https://stream.wikimedia.org/v2/stream/recentchange. However, we may
>> have something different for you, if you find it useful. We have been
>> internally producing new Mediawiki specific events
>> 
>> for a while now, and could expose these via EventStreams as well.
>>
>> Take a look at these events, and tell us what you think.  Would you find
>> them useful?  How would you like to subscribe to them?  Individually as
>> separate streams, or would you like to be able to compose multiple event
>> types into a single stream via an API?  These things are all possible.
>>
>> I asked for a lot of feedback in the above paragraphs.  Let’s try and
>> centralize this discussion over on the mediawiki.org EventStreams talk
>> page .   In summary,
>> the questions are:
>>
>>
>>-
>>
>>What RCStream clients do you maintain, and how can we help you
>>migrate to EventStreams?
>>
>>-
>>
>>Is server side filtering, by wiki or arbitrary event field, useful to
>>you? 
>>-
>>
>>Would you like to consume streams other than RecentChanges?
>>  (Currently
>>available events are described here
>>
>> 
>>.)
>>
>>
>>
>> Thanks!
>> - Andrew Otto
>>
>>
>>
>
___
Services mailing list
Services@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/services

Re: [Services] EventStreams launch and RCStream deprecation

[Andrew Otto]

Hi all,

This is just a friendly reminder that we plan to turn off the RCStream
service after July 7th.

We’re tracking as best we can the progress of porting clients over at
https://phabricator.wikimedia.org/T156919.  But, we can only help with what
we know about.  If you’ve got something still running on RCStream that
hasn’t yet ported, let us know, and/or switch soon!

Thanks!
-Andrew Otto



On Wed, Feb 8, 2017 at 9:28 AM, Andrew Otto  wrote:

> Hi everyone!
>
> Wikimedia is releasing a new service today: EventStreams
> .  This service allows
> us to publish arbitrary streams of JSON event data to the public.
> Initially, the only stream available will be good ol’ RecentChanges
> .  This event stream
> overlaps functionality already provided by irc.wikimedia.org and RCStream
> .  However, this new
> service has advantages over these (now deprecated) services.
>
>
>1.
>
>We can expose more than just RecentChanges.
>2.
>
>Events are delivered over streaming HTTP (chunked transfer) instead of
>IRC or socket.io.  This requires less client side code and fewer
>special routing cases on the server side.
>3.
>
>Streams can be resumed from the past.  By using EventSource, a
>disconnected client will automatically resume the stream from where it left
>off, as long as it resumes within one week.  In the future, we would like
>to allow users to specify historical timestamps from which they would like
>to begin consuming, if this proves safe and tractable.
>
>
> I did say deprecated!  Okay okay, we may never be able to fully deprecate
> irc.wikimedia.org.  It’s used by too many (probably sentient by now) bots
> out there.  We do plan to obsolete RCStream, and to turn it off in a
> reasonable amount of time.  The deadline iis July 7th, 2017.  All
> services that rely on RCStream should migrate to the HTTP based
> EventStreams service by this date.  We are committed to assisting you in
> this transition, so let us know how we can help.
>
> Unfortunately, unlike RCStream, EventStreams does not have server side
> event filtering (e.g. by wiki) quite yet.  How and if this should be done
> is still under discussion .
>
> The RecentChanges data you are used to remains the same, and is available
> at https://stream.wikimedia.org/v2/stream/recentchange. However, we may
> have something different for you, if you find it useful. We have been
> internally producing new Mediawiki specific events
> 
> for a while now, and could expose these via EventStreams as well.
>
> Take a look at these events, and tell us what you think.  Would you find
> them useful?  How would you like to subscribe to them?  Individually as
> separate streams, or would you like to be able to compose multiple event
> types into a single stream via an API?  These things are all possible.
>
> I asked for a lot of feedback in the above paragraphs.  Let’s try and
> centralize this discussion over on the mediawiki.org EventStreams talk
> page .   In summary,
> the questions are:
>
>
>-
>
>What RCStream clients do you maintain, and how can we help you migrate
>to EventStreams?
>
>-
>
>Is server side filtering, by wiki or arbitrary event field, useful to
>you? 
>-
>
>Would you like to consume streams other than RecentChanges?
>  (Currently
>available events are described here
>
> 
>.)
>
>
>
> Thanks!
> - Andrew Otto
>
>
>
___
Services mailing list
Services@lists.wikimedia.org
https://lists.wikimedia.org/mailman/listinfo/services