Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
On 08/13/2014 10:26 PM, Peter Saint-Andre wrote: > On 6/2/14, 10:48 AM, XMPP Extensions Editor wrote: >> The XMPP Extensions Editor has received a proposal for a new XEP. >> >> Title: Recipient Server Side Notifications Filtering >> >> Abstract: This specification defines a modern efficient way to deliver >> PubSub notifications. >> >> URL: http://xmpp.org/extensions/inbox/rsf.html >> >> The XMPP Council will decide in the next two weeks whether to accept >> this proposal as an official XEP. >> > > I've finally made time to read this proposal. Thank you for that. > > It seems to me that the proposal is saying that PEP isn't always ideal, > so we need a new protocol. However, I think that for high-scale > scenarios (e.g., a bot collecting geolocation information about all > users in a large company, or microblogging for users with lots of > followers), it might be better to use generic pubsub (XEP-0060) without > presence involved at all, not PEP (which as noted depends on having > bidirectional presence subscriptions in place). PEP is a simplification > of pubsub for IM systems, but not all pubsub systems are based on IM. Surely, and for those systems we have bare jids, but what if we do need some list of subscriptions like in blogging, why don't use roster in such cases? > However, a system could be architected such that it provides two ways to > subscribe to the same information (via pubsub and PEP). Or a system > could implement full pubsub on every JID, instead of using the > simplified technology we defined in the PEP spec. I'm not, however, > seeing the need for a completely new protocol. Firstly, I don't think that it's a completely different protocol, it just changes the flow to solve some problems and for end users it changes completely nothing except of more smooth experience because user expects PEP to work in conjunction with presence and it not always does so. Secondly, if anybody would come up with easier solution on one-way subs, I would love to accept it, but nobody knows, it seems, how to do that. > > Just my centigram of silver... Thanks. > > Peter > > -- With best regards, Sergey Dobrov, XMPP Developer and JRuDevels.org founder.
Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
On 6/2/14, 10:48 AM, XMPP Extensions Editor wrote: The XMPP Extensions Editor has received a proposal for a new XEP. Title: Recipient Server Side Notifications Filtering Abstract: This specification defines a modern efficient way to deliver PubSub notifications. URL: http://xmpp.org/extensions/inbox/rsf.html The XMPP Council will decide in the next two weeks whether to accept this proposal as an official XEP. I've finally made time to read this proposal. It seems to me that the proposal is saying that PEP isn't always ideal, so we need a new protocol. However, I think that for high-scale scenarios (e.g., a bot collecting geolocation information about all users in a large company, or microblogging for users with lots of followers), it might be better to use generic pubsub (XEP-0060) without presence involved at all, not PEP (which as noted depends on having bidirectional presence subscriptions in place). PEP is a simplification of pubsub for IM systems, but not all pubsub systems are based on IM. However, a system could be architected such that it provides two ways to subscribe to the same information (via pubsub and PEP). Or a system could implement full pubsub on every JID, instead of using the simplified technology we defined in the PEP spec. I'm not, however, seeing the need for a completely new protocol. Just my centigram of silver... Peter
Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
The proposal has been updated getting rid of XEP-33 inside. On 06/02/2014 11:48 PM, XMPP Extensions Editor wrote: > The XMPP Extensions Editor has received a proposal for a new XEP. > > Title: Recipient Server Side Notifications Filtering > > Abstract: This specification defines a modern efficient way to deliver PubSub > notifications. > > URL: http://xmpp.org/extensions/inbox/rsf.html > > The XMPP Council will decide in the next two weeks whether to accept this > proposal as an official XEP. > > -- With best regards, Sergey Dobrov, XMPP Developer and JRuDevels.org founder.
Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
Mon, 23 Jun 2014 15:23:40 +0200 Kim Alvefur wrote: > When does this happen? Other clients should check with disco#info and > caps if the client supports things before sending them. This happens when a server doesn't cache caps. > Presence de-duplication would be very interesting to implement. As > in, only route presence that differs from the previous one. Indeed. However, I don't know how it is in Prosody, but c2s module became a monster in ejabberd with all those protocol extensions and don't want to breathe into it.
Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
On 2014-06-23 15:03, Evgeny Khramtsov wrote: > Mon, 23 Jun 2014 12:11:35 +0100 > Dave Cridland wrote: > >> On 23 June 2014 11:44, Evgeny Khramtsov wrote: >> >>> 1) servers hold too much state >>> >> >> What state do you think they hold that they should not? > > CAPs cache for example. Prosody for one doesn't actually cache caps. Our PEP module caches a set of nodes of interest, but not the full caps. We've been considering implementing full caching of disco#info/caps, would be interesting to have the server reply to disco#info queries with cached data. And Jabber/XMPP was designed as a thin client - fat server system. So there will be lots of state on the server. >>> 2) clients receive too much data >>> >> >> By "too much data", do you mean irrelevant data or redundant data, or >> something else? > > 1) It's quite pointless to route a stanza with feature X to a client if > a client doesn't support this feature X. When does this happen? Other clients should check with disco#info and caps if the client supports things before sending them. > 2) We route shitload of presences to a client. On and on this goes. Presence de-duplication would be very interesting to implement. As in, only route presence that differs from the previous one. -- Kim "Zash" Alvefur signature.asc Description: OpenPGP digital signature
Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
On 23 June 2014 14:03, Evgeny Khramtsov wrote: > Mon, 23 Jun 2014 12:11:35 +0100 > Dave Cridland wrote: > > > On 23 June 2014 11:44, Evgeny Khramtsov wrote: > > > > > 1) servers hold too much state > > > > > > > What state do you think they hold that they should not? > > CAPs cache for example. > > Oh. I wouldn't classify that as state, so I'm glad I asked what you meant. I think keeping a Caps cache around is OK, though, not least because you can use it for servicing disco requests much faster (and without bothering local clients), but also the levels of optimizations and interop stuff one can do with one are quite high. Admittedly, many of these are most useful for local clients, but remote client caps, too, have their uses. > > > 2) clients receive too much data > > > > > > > By "too much data", do you mean irrelevant data or redundant data, or > > something else? > > 1) It's quite pointless to route a stanza with feature X to a client if > a client doesn't support this feature X. > OK. There's an implication there that you think this is happening; I've not generally seen this. Can you give concrete examples? Incidentally, if this *is* happening, then one way to solve it is with your caps cache. > 2) We route shitload of presences to a client. On and on this goes. > We do. We're a presence protocol though, so this goes with the territory. Do you think that we're routing redundant presence information to clients, or irrelevant presence information to clients? ie, do you think clients don't care about status/show updates, or do you think they're getting presence stanzas which contain the same state information as the previous received? Dave.
Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
Mon, 23 Jun 2014 12:11:35 +0100 Dave Cridland wrote: > On 23 June 2014 11:44, Evgeny Khramtsov wrote: > > > 1) servers hold too much state > > > > What state do you think they hold that they should not? CAPs cache for example. > > 2) clients receive too much data > > > > By "too much data", do you mean irrelevant data or redundant data, or > something else? 1) It's quite pointless to route a stanza with feature X to a client if a client doesn't support this feature X. 2) We route shitload of presences to a client. On and on this goes.
Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
On 23 June 2014 11:44, Evgeny Khramtsov wrote: > 1) servers hold too much state > What state do you think they hold that they should not? > 2) clients receive too much data > By "too much data", do you mean irrelevant data or redundant data, or something else?
Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
Mon, 23 Jun 2014 12:01:21 +0200 Philipp Hancke wrote: > Am 23.06.2014 09:41, schrieb Sergey Dobrov: > > So guys, what are the exact concern around the XEP-0033 > > None of us has ever seen 0033 deliver on "reducing the amount of s2s > traffic". Even the authors of 0033 agreed to this in the repeater > introduction. How did you measure that? Anyway, I don't even consider s2s traffic as a problem. From my own experience (I have seen deployments with around 0.5-1 million c2s connections online). Two major problems I see: 1) servers hold too much state 2) clients receive too much data
Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
On 06/23/2014 05:01 PM, Philipp Hancke wrote: > Am 23.06.2014 09:41, schrieb Sergey Dobrov: >> So guys, what are the exact concern around the XEP-0033 > > None of us has ever seen 0033 deliver on "reducing the amount of s2s > traffic". Even the authors of 0033 agreed to this in the repeater > introduction. But it seems to be obvious to me that one event, even with overhead on XEP-33 semantic will have less weight that multiple events, especially on big nodes, huh? > > Using 0033 unconditionally has trust issues. How do you avoid the > spammer that just puts a bunch of addresses there? You need to do heavy > checks on each use here -- which is why reusing lists may have > advantages, as you need to do those checks less (which depends on the > usecase however). How to avoid to spam each person individually? > >> and what should I use instead? I did not get your points. > > Ignore 0033 for now. > I have not heard any else concerns but I do have some -1's, so I am not sure what should I do next with the proposal? Thanks. -- With best regards, Sergey Dobrov, XMPP Developer and JRuDevels.org founder.
Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
Am 23.06.2014 09:41, schrieb Sergey Dobrov: So guys, what are the exact concern around the XEP-0033 None of us has ever seen 0033 deliver on "reducing the amount of s2s traffic". Even the authors of 0033 agreed to this in the repeater introduction. Using 0033 unconditionally has trust issues. How do you avoid the spammer that just puts a bunch of addresses there? You need to do heavy checks on each use here -- which is why reusing lists may have advantages, as you need to do those checks less (which depends on the usecase however). and what should I use instead? I did not get your points. Ignore 0033 for now.
Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
So guys, what are the exact concern around the XEP-0033 and what should I use instead? I did not get your points. On 06/02/2014 11:48 PM, XMPP Extensions Editor wrote: > The XMPP Extensions Editor has received a proposal for a new XEP. > > Title: Recipient Server Side Notifications Filtering > > Abstract: This specification defines a modern efficient way to deliver PubSub > notifications. > > URL: http://xmpp.org/extensions/inbox/rsf.html > > The XMPP Council will decide in the next two weeks whether to accept this > proposal as an official XEP. > > -- With best regards, Sergey Dobrov, XMPP Developer and JRuDevels.org founder.
Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
Wed, 18 Jun 2014 10:21:36 +0100 Kevin Smith wrote: > The recipient-side filtering seems OKish, but could lead to situations > where the publishing servers blindly send out to all possible > recipient JIDs and let the recipients deal with whether to send to the > clients or not - clearly not OK. Why not? With recipient-side filtering we have a tradeoff, so a publishing server is free to choose.
Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
Guys, could you please be more specific on what's wrong with XEP-33 because I don't really understand your concerns. I don't think that it will be better to use my own protocol to put the subscribers' JIDs into the event, right? So what are the options? Thanks. On 06/18/2014 04:21 PM, Kevin Smith wrote: > On Wed, Jun 18, 2014 at 9:20 AM, Lance Stout wrote: >> >> On Jun 2, 2014, at 9:48 AM, XMPP Extensions Editor wrote: >> >>> The XMPP Extensions Editor has received a proposal for a new XEP. >>> >>> Title: Recipient Server Side Notifications Filtering >>> >>> Abstract: This specification defines a modern efficient way to deliver >>> PubSub notifications. >>> >>> URL: http://xmpp.org/extensions/inbox/rsf.html >>> >>> The XMPP Council will decide in the next two weeks whether to accept this >>> proposal as an official XEP. >>> >> >> >> I'm a -1 based on the reasons Fippo explained concerning XEP-0033 use. >> >> >> However, the event filtering by the receiver's server via caps part of this >> proposal is useful and could stand on its own without the XEP-0033 parts. >> >> That enters into SIFT-like territory, but maybe that's a direction we need >> to experiment in so that we can actually have things that implement SIFT >> features. > > This is one I'd really like to thrash out at a summit, because I think > it's slightly more involved than is immediately obvious. > > The -33 use makes me uncomfortable, because I'm not convinced yet that > -33 actually buys us much here. I'd rather like to see figures here to > convince me. > > The recipient-side filtering seems OKish, but could lead to situations > where the publishing servers blindly send out to all possible > recipient JIDs and let the recipients deal with whether to send to the > clients or not - clearly not OK. > > It seems that this is potentially trying to answer part of a larger > 'how do I manage my pubsub subscriptions?' question, as there are > other things you want to do with your subscriptions. I think the > buddycloud folks have looked at some of this stuff with their inbox > services. I don't know if cross-polination there makes sense? > > /K > -- With best regards, Sergey Dobrov, XMPP Developer and JRuDevels.org founder.
Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
On Wed, Jun 18, 2014 at 9:20 AM, Lance Stout wrote: > > On Jun 2, 2014, at 9:48 AM, XMPP Extensions Editor wrote: > >> The XMPP Extensions Editor has received a proposal for a new XEP. >> >> Title: Recipient Server Side Notifications Filtering >> >> Abstract: This specification defines a modern efficient way to deliver >> PubSub notifications. >> >> URL: http://xmpp.org/extensions/inbox/rsf.html >> >> The XMPP Council will decide in the next two weeks whether to accept this >> proposal as an official XEP. >> > > > I'm a -1 based on the reasons Fippo explained concerning XEP-0033 use. > > > However, the event filtering by the receiver's server via caps part of this > proposal is useful and could stand on its own without the XEP-0033 parts. > > That enters into SIFT-like territory, but maybe that's a direction we need to > experiment in so that we can actually have things that implement SIFT > features. This is one I'd really like to thrash out at a summit, because I think it's slightly more involved than is immediately obvious. The -33 use makes me uncomfortable, because I'm not convinced yet that -33 actually buys us much here. I'd rather like to see figures here to convince me. The recipient-side filtering seems OKish, but could lead to situations where the publishing servers blindly send out to all possible recipient JIDs and let the recipients deal with whether to send to the clients or not - clearly not OK. It seems that this is potentially trying to answer part of a larger 'how do I manage my pubsub subscriptions?' question, as there are other things you want to do with your subscriptions. I think the buddycloud folks have looked at some of this stuff with their inbox services. I don't know if cross-polination there makes sense? /K
Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
On Jun 2, 2014, at 9:48 AM, XMPP Extensions Editor wrote: > The XMPP Extensions Editor has received a proposal for a new XEP. > > Title: Recipient Server Side Notifications Filtering > > Abstract: This specification defines a modern efficient way to deliver PubSub > notifications. > > URL: http://xmpp.org/extensions/inbox/rsf.html > > The XMPP Council will decide in the next two weeks whether to accept this > proposal as an official XEP. > I'm a -1 based on the reasons Fippo explained concerning XEP-0033 use. However, the event filtering by the receiver's server via caps part of this proposal is useful and could stand on its own without the XEP-0033 parts. That enters into SIFT-like territory, but maybe that's a direction we need to experiment in so that we can actually have things that implement SIFT features. - Lance signature.asc Description: Message signed with OpenPGP using GPGMail
Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
Hello Philipp, thanks for your feedback. On 06/17/2014 03:02 PM, Philipp Hancke wrote: > Am 02.06.2014 18:48, schrieb XMPP Extensions Editor: >> The XMPP Extensions Editor has received a proposal for a new XEP. >> >> Title: Recipient Server Side Notifications Filtering >> >> Abstract: This specification defines a modern efficient way to deliver >> PubSub notifications. >> >> URL: http://xmpp.org/extensions/inbox/rsf.html >> >> The XMPP Council will decide in the next two weeks whether to accept >> this proposal as an official XEP. > > -1 --I think this needs a decent flame war^W^W^Wmore discussion about > the usage of XEP-0033. > > >> it's impossible to be subscribed to a person's feed if the person is >> not subscribed to you because in this case his server doesn't know >> your presence information and your capabilities and hence it can't >> filter the messages appropriately, so you won't receive any >> notifications at all; > > Assuming the publisher uses filtered notifications. Which it does not > have to if the "explicit subscribe" described in XEP-0277 is used. > Note that this probably requires the publisher to support > pubsub-on-a-jid instead of just pep. That doesn't seem as a consistent way to me too because it will lead to sending the events to all the resources, even if they are not able to process them from the one side and it will lead to problems with synchronization between resources in the case when there are more then one simultaneously connected resources. > > >> servers need to know not only their users' capabilities but also the >> capabilities of all the users that it wants to communicate; > > Right. I'd note that a caps-aware/pep-enabled server already does that. Sure, the point is that they will be able to exclude capabilities of the users' of the RSF-enabled servers from their caches. > > >> clients can't decide what kind of notifications they want to receive; >> node owner decides instead, but clients might not be interested in >> some of notifications or vice versa. > > It seems to be useful for a client to instruct their server to filter > events sent to the bare JID. For this, an approach like the one > described in 3.4 seems to make sense. > Which would basically turn this proposal into a way to do pep-like > filtering on the receiver side instead (or in addition to?) the sender. > That seems generally useful IMO. Yes, that's one of the (two) main reasons to introduce the XEP. > > > >> we can't reduce amount of S2S traffic by sending one notification for >> each server, we need to send as much notifications as much users are >> subscribed on it; > > I'd note that the distribution problem (or multicast optimization) is > not limited to pubsub. Presence, pep and MUC have the same problem. > > > Back in 2008 (april) we had another proposal that went in the same > direction: http://xmpp.org/extensions/inbox/repeaters.html > (and we've had lengthy discussions about that and "smarticast" in 2006) > I agree with the authors of the repeaters proposal that "in practice > Extended Stanza Addressing has not resulted in traffic optimization". I > do not expect this to for the use of 0033 in this specification either. > stanza repeaters hasn't been moved forward. I don't know whether this > was by council decision or because the authors decided not to move it > forward... > > That said, I haven't seen much effort in that field since 2008. So it > seems this doesn't hurt the ops side much which implies there is no need > for standardized optimizations. > I have some concerns here but I won't argue here because it's not the main reason to introduce the XEP. I just want PEP to correspond the presence model tightly to be able to use it more widely, and I would like to hear if you have any reasons why I should not want that to become real :) I am ill a little now, so my apologies if some of my thoughts were not explained well enough. I will be happy to continue the discussion though. -- With best regards, Sergey Dobrov, XMPP Developer and JRuDevels.org founder.
Re: [Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
Am 02.06.2014 18:48, schrieb XMPP Extensions Editor: The XMPP Extensions Editor has received a proposal for a new XEP. Title: Recipient Server Side Notifications Filtering Abstract: This specification defines a modern efficient way to deliver PubSub notifications. URL: http://xmpp.org/extensions/inbox/rsf.html The XMPP Council will decide in the next two weeks whether to accept this proposal as an official XEP. -1 --I think this needs a decent flame war^W^W^Wmore discussion about the usage of XEP-0033. > it's impossible to be subscribed to a person's feed if the person is > not subscribed to you because in this case his server doesn't know > your presence information and your capabilities and hence it can't > filter the messages appropriately, so you won't receive any > notifications at all; Assuming the publisher uses filtered notifications. Which it does not have to if the "explicit subscribe" described in XEP-0277 is used. Note that this probably requires the publisher to support pubsub-on-a-jid instead of just pep. > servers need to know not only their users' capabilities but also the > capabilities of all the users that it wants to communicate; Right. I'd note that a caps-aware/pep-enabled server already does that. > clients can't decide what kind of notifications they want to receive; > node owner decides instead, but clients might not be interested in > some of notifications or vice versa. It seems to be useful for a client to instruct their server to filter events sent to the bare JID. For this, an approach like the one described in 3.4 seems to make sense. Which would basically turn this proposal into a way to do pep-like filtering on the receiver side instead (or in addition to?) the sender. That seems generally useful IMO. > we can't reduce amount of S2S traffic by sending one notification for > each server, we need to send as much notifications as much users are > subscribed on it; I'd note that the distribution problem (or multicast optimization) is not limited to pubsub. Presence, pep and MUC have the same problem. Back in 2008 (april) we had another proposal that went in the same direction: http://xmpp.org/extensions/inbox/repeaters.html (and we've had lengthy discussions about that and "smarticast" in 2006) I agree with the authors of the repeaters proposal that "in practice Extended Stanza Addressing has not resulted in traffic optimization". I do not expect this to for the use of 0033 in this specification either. stanza repeaters hasn't been moved forward. I don't know whether this was by council decision or because the authors decided not to move it forward... That said, I haven't seen much effort in that field since 2008. So it seems this doesn't hurt the ops side much which implies there is no need for standardized optimizations.
[Standards] Proposed XMPP Extension: Recipient Server Side Notifications Filtering
The XMPP Extensions Editor has received a proposal for a new XEP. Title: Recipient Server Side Notifications Filtering Abstract: This specification defines a modern efficient way to deliver PubSub notifications. URL: http://xmpp.org/extensions/inbox/rsf.html The XMPP Council will decide in the next two weeks whether to accept this proposal as an official XEP.
