Re: [Standards] Microblogging: XEP-0277 and beyond
On 09/07/2011 07:40 AM, Justin Karneges wrote: > On Sunday, September 04, 2011 11:47:55 PM Sergey Dobrov wrote: >> On 09/05/2011 12:17 PM, Justin Karneges wrote: >>> On Saturday, September 03, 2011 03:07:45 AM Sergey Dobrov wrote: On 09/03/2011 12:41 AM, Justin Karneges wrote: > The drawback to node metadata is it does not support update > notifications. I have the need to track this data, particularly the > conversation title. Don't understand why to change that. >>> >>> Don't understand why the title of a conversation might change? >> >> Yup. > > In my experience a title change for a commenting area is usually only done to > correct a typo. Still, applications that cache titles (think long running > apps like web sites) should have a way to keep them up to date. ok, maybe it can be useful but not a first-time needed feature. anyway, from my point of view it will be more useful to invent more flexible metadata protocol for nodes. (i mean, if node configuration changes may be delivered immediately as event, why metadata can't? but I really hate producing entities in a kind of nodes) > > Activity makes it much easier to determine what has happened to an > item. For example, if a moderator edits a comment, this is easily > described in activity terms. Simply doing complete replacement of > comment items is not terrible but it makes it harder for listeners to > figure out what is going on. > > The main point is the activity node is an append-only changelog, and > the comments node is an up-to-date compilation of that log. That's funny: some earlier you said: "My stance is there is nothing wrong with using a journal to implement pubsub, but ideally pubsub-using protocols should not be so complex that they require a journal-based implementation to participate." And now you talking about inventing the same journal but in specific (non-universal) way and in much more complex manner. Why? >>> >>> It's simple: the activity node has optional persistence, hence the >>> journal is optional. >> >> Journal is optional feature anyway. So it's more usable to make it >> universal and not force it to depend on an applied protocol, isn't it? > > The activity node is actually the most normal. It doesn't even define any > parameters. It's the raw stream of what's happening. It is. But it's not flexible (for other protocols than XEP-303). Again, I think that it's completely unnecessary complication for servers (since server should support consistency of that nodes). Pubsub services are not too ideal without that complications. If we will produce more and more entities that can not be reused we will have the sad end. So I think that it will be better to invent FLEXIBLE journal than reinvent it in each applied protocol. > > The comments node is the weird one. It's a view/compilation of the activity. > > The plan for likes is you'd publish to the activity node ("Justin likes > comment X"), but on the comment node this would be reflected as a > property of an existing comment rather than a new item. Comment items > could have "like" data stored within them, such as a count and maybe a > list of the last five people to like the comment. I think that all these actions should not be copied from centralized services since that services have more possibilities to control them. I mean, it will be very hard to check if likes are winded by some bad mans. Also it will be hard to represent some statistics of most liked items. So I think that these things should be solved in that manner how google ranking web pages: aggregators will rank items by repost counts or something. And, yes, for user like button should be replaced by "repost" or "repost with comment", I have such functionality in my microblogging service and I want to add it into the XEP-277. >>> >>> What do you mean by centralized? That the conversation is the authority >>> on all of its replies and likes? >> >> Yes, I don't understand what is the reason to have likes in such system >> if we can't count them globally and give stats. > > The problem with distributing a single conversation across servers is then > you > need a way of aggregating even in the common case. Simply viewing the > conversation properly with nested threads and like points requires an > aggregator. >This is what Salmon is for in the HTTP world: telling upstream > you've made a reply/like/etc. But it's already solved in the pubsub world: we have event delivering system. > My humble opinion is that this approach does > have some cool-factor but it is also a giant pain in the ass in terms of > practicality. > > I think that if we ever did support distributed conversations then you'd > still > want the main conversation node (root node) to be able to hold copies of all > the content (e.g. use a salmon-like process so
Re: [Standards] Microblogging: XEP-0277 and beyond
On Sunday, September 04, 2011 11:47:55 PM Sergey Dobrov wrote: > On 09/05/2011 12:17 PM, Justin Karneges wrote: > > On Saturday, September 03, 2011 03:07:45 AM Sergey Dobrov wrote: > >> On 09/03/2011 12:41 AM, Justin Karneges wrote: > >>> The drawback to node metadata is it does not support update > >>> notifications. I have the need to track this data, particularly the > >>> conversation title. > >> > >> Don't understand why to change that. > > > > Don't understand why the title of a conversation might change? > > Yup. In my experience a title change for a commenting area is usually only done to correct a typo. Still, applications that cache titles (think long running apps like web sites) should have a way to keep them up to date. > >>> Activity makes it much easier to determine what has happened to an > >>> item. For example, if a moderator edits a comment, this is easily > >>> described in activity terms. Simply doing complete replacement of > >>> comment items is not terrible but it makes it harder for listeners to > >>> figure out what is going on. > >>> > >>> The main point is the activity node is an append-only changelog, and > >>> the comments node is an up-to-date compilation of that log. > >> > >> That's funny: some earlier you said: "My stance is there is nothing > >> wrong with using a journal to implement pubsub, but ideally pubsub-using > >> protocols should not be so complex that they require a journal-based > >> implementation to participate." And now you talking about inventing the > >> same journal but in specific (non-universal) way and in much more > >> complex manner. Why? > > > > It's simple: the activity node has optional persistence, hence the > > journal is optional. > > Journal is optional feature anyway. So it's more usable to make it > universal and not force it to depend on an applied protocol, isn't it? The activity node is actually the most normal. It doesn't even define any parameters. It's the raw stream of what's happening. The comments node is the weird one. It's a view/compilation of the activity. > >>> The plan for likes is you'd publish to the activity node ("Justin likes > >>> comment X"), but on the comment node this would be reflected as a > >>> property of an existing comment rather than a new item. Comment items > >>> could have "like" data stored within them, such as a count and maybe a > >>> list of the last five people to like the comment. > >> > >> I think that all these actions should not be copied from centralized > >> services since that services have more possibilities to control them. I > >> mean, it will be very hard to check if likes are winded by some bad > >> mans. Also it will be hard to represent some statistics of most liked > >> items. So I think that these things should be solved in that manner how > >> google ranking web pages: aggregators will rank items by repost counts > >> or something. And, yes, for user like button should be replaced by > >> "repost" or "repost with comment", I have such functionality in my > >> microblogging service and I want to add it into the XEP-277. > > > > What do you mean by centralized? That the conversation is the authority > > on all of its replies and likes? > > Yes, I don't understand what is the reason to have likes in such system > if we can't count them globally and give stats. The problem with distributing a single conversation across servers is then you need a way of aggregating even in the common case. Simply viewing the conversation properly with nested threads and like points requires an aggregator. This is what Salmon is for in the HTTP world: telling upstream you've made a reply/like/etc. My humble opinion is that this approach does have some cool-factor but it is also a giant pain in the ass in terms of practicality. I think that if we ever did support distributed conversations then you'd still want the main conversation node (root node) to be able to hold copies of all the content (e.g. use a salmon-like process so that the root node remains in- the-know and is able to serve the full dataset to clients). Assuming this is acceptable, then we can always add distribution in the future. Justin
Re: [Standards] Microblogging: XEP-0277 and beyond
On 09/05/2011 12:17 PM, Justin Karneges wrote: > On Saturday, September 03, 2011 03:07:45 AM Sergey Dobrov wrote: >> On 09/03/2011 12:41 AM, Justin Karneges wrote: >>> The drawback to node metadata is it does not support update >>> notifications. I have the need to track this data, particularly the >>> conversation title. >> >> Don't understand why to change that. > > Don't understand why the title of a conversation might change? Yup. > >>> Activity makes it much easier to determine what has happened to an item. >>> For example, if a moderator edits a comment, this is easily described in >>> activity terms. Simply doing complete replacement of comment items is >>> not terrible but it makes it harder for listeners to figure out what is >>> going on. >>> >>> The main point is the activity node is an append-only changelog, and the >>> comments node is an up-to-date compilation of that log. >> >> That's funny: some earlier you said: "My stance is there is nothing >> wrong with using a journal to implement pubsub, but ideally pubsub-using >> protocols should not be so complex that they require a journal-based >> implementation to participate." And now you talking about inventing the >> same journal but in specific (non-universal) way and in much more >> complex manner. Why? > > It's simple: the activity node has optional persistence, hence the journal is > optional. Journal is optional feature anyway. So it's more usable to make it universal and not force it to depend on an applied protocol, isn't it? > >>> The plan for likes is you'd publish to the activity node ("Justin likes >>> comment X"), but on the comment node this would be reflected as a >>> property of an existing comment rather than a new item. Comment items >>> could have "like" data stored within them, such as a count and maybe a >>> list of the last five people to like the comment. >> >> I think that all these actions should not be copied from centralized >> services since that services have more possibilities to control them. I >> mean, it will be very hard to check if likes are winded by some bad >> mans. Also it will be hard to represent some statistics of most liked >> items. So I think that these things should be solved in that manner how >> google ranking web pages: aggregators will rank items by repost counts >> or something. And, yes, for user like button should be replaced by >> "repost" or "repost with comment", I have such functionality in my >> microblogging service and I want to add it into the XEP-277. > > What do you mean by centralized? That the conversation is the authority on > all of its replies and likes? Yes, I don't understand what is the reason to have likes in such system if we can't count them globally and give stats. > > Justin > -- With best regards, Sergey Dobrov, XMPP Developer and JRuDevels.org founder.
Re: [Standards] Microblogging: XEP-0277 and beyond
On Saturday, September 03, 2011 03:07:45 AM Sergey Dobrov wrote: > On 09/03/2011 12:41 AM, Justin Karneges wrote: > > The drawback to node metadata is it does not support update > > notifications. I have the need to track this data, particularly the > > conversation title. > > Don't understand why to change that. Don't understand why the title of a conversation might change? > > Activity makes it much easier to determine what has happened to an item. > > For example, if a moderator edits a comment, this is easily described in > > activity terms. Simply doing complete replacement of comment items is > > not terrible but it makes it harder for listeners to figure out what is > > going on. > > > > The main point is the activity node is an append-only changelog, and the > > comments node is an up-to-date compilation of that log. > > That's funny: some earlier you said: "My stance is there is nothing > wrong with using a journal to implement pubsub, but ideally pubsub-using > protocols should not be so complex that they require a journal-based > implementation to participate." And now you talking about inventing the > same journal but in specific (non-universal) way and in much more > complex manner. Why? It's simple: the activity node has optional persistence, hence the journal is optional. > > The plan for likes is you'd publish to the activity node ("Justin likes > > comment X"), but on the comment node this would be reflected as a > > property of an existing comment rather than a new item. Comment items > > could have "like" data stored within them, such as a count and maybe a > > list of the last five people to like the comment. > > I think that all these actions should not be copied from centralized > services since that services have more possibilities to control them. I > mean, it will be very hard to check if likes are winded by some bad > mans. Also it will be hard to represent some statistics of most liked > items. So I think that these things should be solved in that manner how > google ranking web pages: aggregators will rank items by repost counts > or something. And, yes, for user like button should be replaced by > "repost" or "repost with comment", I have such functionality in my > microblogging service and I want to add it into the XEP-277. What do you mean by centralized? That the conversation is the authority on all of its replies and likes? Justin
Re: [Standards] Microblogging: XEP-0277 and beyond
On 09/03/2011 12:41 AM, Justin Karneges wrote: On Friday, September 02, 2011 04:29:25 AM Sergey Dobrov wrote: On 09/02/2011 03:32 AM, Justin Karneges wrote: On Thursday, September 01, 2011 11:22:30 AM Sergey Dobrov wrote: Then, my questions about XEP-303: 1) Why to use three nodes? Info node can be replaced by some item with some constant id. about necessity of activity node I don't understood at all, why to have it? I didn't want to put the info item in with the comments since they are not really part of the same collection of data. The gain with a separate node is that you could subscribe to just the 'info', and not the comments, if for some reason you wanted to do that. Maybe that isn't very useful. Exactly, maybe we should extend node metadata instead of that? The drawback to node metadata is it does not support update notifications. I have the need to track this data, particularly the conversation title. Don't understand why to change that. So I think the data belongs in a node. Or, we could consider adding notification support for node metadata. Activity makes it much easier to determine what has happened to an item. For example, if a moderator edits a comment, this is easily described in activity terms. Simply doing complete replacement of comment items is not terrible but it makes it harder for listeners to figure out what is going on. The main point is the activity node is an append-only changelog, and the comments node is an up-to-date compilation of that log. That's funny: some earlier you said: "My stance is there is nothing wrong with using a journal to implement pubsub, but ideally pubsub-using protocols should not be so complex that they require a journal-based implementation to participate." And now you talking about inventing the same journal but in specific (non-universal) way and in much more complex manner. Why? Then, I don't understand how "likes" can work in such model, who will count likes? I think that this feature should be implemented in aggregators, some services which aggregates messages from one or more servers and give services of statistics, search or something else. The plan for likes is you'd publish to the activity node ("Justin likes comment X"), but on the comment node this would be reflected as a property of an existing comment rather than a new item. Comment items could have "like" data stored within them, such as a count and maybe a list of the last five people to like the comment. I think that all these actions should not be copied from centralized services since that services have more possibilities to control them. I mean, it will be very hard to check if likes are winded by some bad mans. Also it will be hard to represent some statistics of most liked items. So I think that these things should be solved in that manner how google ranking web pages: aggregators will rank items by repost counts or something. And, yes, for user like button should be replaced by "repost" or "repost with comment", I have such functionality in my microblogging service and I want to add it into the XEP-277. I want to point out again how the activity node makes it far easier to figure out what has actually happened. If you have a notification app that is tracking your comments, subscribing to the activity node and listening for "Justin likes comment X" is much more straightforward than subscribing to the comments node and making inferences. About post deletion, why to have this notification? Is it not enough to receive a regular retract event? Comments node could send a retract event. But activity node is always appending, and would send some notification such as "User N deleted comment X". Justin
Re: [Standards] Microblogging: XEP-0277 and beyond
On Friday, September 02, 2011 04:29:25 AM Sergey Dobrov wrote: > On 09/02/2011 03:32 AM, Justin Karneges wrote: > > On Thursday, September 01, 2011 11:22:30 AM Sergey Dobrov wrote: > >> Then, my questions about XEP-303: > >> > >> 1) Why to use three nodes? Info node can be replaced by some item with > >> some constant id. about necessity of activity node I don't understood at > >> all, why to have it? > > > > I didn't want to put the info item in with the comments since they are > > not really part of the same collection of data. The gain with a > > separate node is that you could subscribe to just the 'info', and not > > the comments, if for some reason you wanted to do that. Maybe that > > isn't very useful. > > Exactly, maybe we should extend node metadata instead of that? The drawback to node metadata is it does not support update notifications. I have the need to track this data, particularly the conversation title. So I think the data belongs in a node. Or, we could consider adding notification support for node metadata. > > I do believe there is a need to have separate comments and activity nodes > > though, and possibly even more in the future (I want to have an > > "occupants" node containing a userlist of people with temporary > > subscriptions -- think MUC over pubsub). So, I have already resigned to > > needing multiple nodes, and when it came time to need 'info', I decided > > a singleton made logical sense. > > > > The activity node is actually the core node of the system. Flowing > > through it would be the stream of activity occuring in the conversation. > > If you want to post a comment, like a comment, delete a comment, etc, > > it makes sense to publish them as "activity". If you want to receive > > events for these kinds of activities (e.g. to track the conversation > > while you are away, and power something like Facebook Notifications), > > then again you want to be subscribed to items of "activity". This is > > distinct from the comments node, which only holds the comments in their > > current state. For example, if a comment is posted and edited three > > times after that, this would be represented as four items in the > > activity node but still only one item (updated three times) in the > > comments node. The comments node is needed to drive application > > displays. So this is why I believe both nodes are needed. > > I still don't understand. You trying to solve problems with extra > entities when it should (in my opinion, of course) be solved by some > generic pubsub extensions, for example, it could be really usable to > store each item version, so you will be able to retrieve any edits of > the comment too. But I still consider that activity node is unnecessary > and comments node should be in a simple atom format. Activity makes it much easier to determine what has happened to an item. For example, if a moderator edits a comment, this is easily described in activity terms. Simply doing complete replacement of comment items is not terrible but it makes it harder for listeners to figure out what is going on. The main point is the activity node is an append-only changelog, and the comments node is an up-to-date compilation of that log. > Then, I don't understand how "likes" can work in such model, who will > count likes? I think that this feature should be implemented in > aggregators, some services which aggregates messages from one or more > servers and give services of statistics, search or something else. The plan for likes is you'd publish to the activity node ("Justin likes comment X"), but on the comment node this would be reflected as a property of an existing comment rather than a new item. Comment items could have "like" data stored within them, such as a count and maybe a list of the last five people to like the comment. I want to point out again how the activity node makes it far easier to figure out what has actually happened. If you have a notification app that is tracking your comments, subscribing to the activity node and listening for "Justin likes comment X" is much more straightforward than subscribing to the comments node and making inferences. > About post deletion, why to have this notification? Is it not enough to > receive a regular retract event? Comments node could send a retract event. But activity node is always appending, and would send some notification such as "User N deleted comment X". Justin
Re: [Standards] Microblogging: XEP-0277 and beyond
On 09/02/2011 03:32 AM, Justin Karneges wrote: > On Thursday, September 01, 2011 11:22:30 AM Sergey Dobrov wrote: >> Then, my questions about XEP-303: >> >> 1) Why to use three nodes? Info node can be replaced by some item with >> some constant id. about necessity of activity node I don't understood at >> all, why to have it? > > I didn't want to put the info item in with the comments since they are not > really part of the same collection of data. The gain with a separate node is > that you could subscribe to just the 'info', and not the comments, if for > some > reason you wanted to do that. Maybe that isn't very useful. Exactly, maybe we should extend node metadata instead of that? > > I do believe there is a need to have separate comments and activity nodes > though, and possibly even more in the future (I want to have an "occupants" > node containing a userlist of people with temporary subscriptions -- think > MUC > over pubsub). So, I have already resigned to needing multiple nodes, and > when > it came time to need 'info', I decided a singleton made logical sense. > > The activity node is actually the core node of the system. Flowing through > it > would be the stream of activity occuring in the conversation. If you want to > post a comment, like a comment, delete a comment, etc, it makes sense to > publish them as "activity". If you want to receive events for these kinds of > activities (e.g. to track the conversation while you are away, and power > something like Facebook Notifications), then again you want to be subscribed > to > items of "activity". This is distinct from the comments node, which only > holds the comments in their current state. For example, if a comment is > posted and edited three times after that, this would be represented as four > items in the activity node but still only one item (updated three times) in > the comments node. The comments node is needed to drive application > displays. > So this is why I believe both nodes are needed. I still don't understand. You trying to solve problems with extra entities when it should (in my opinion, of course) be solved by some generic pubsub extensions, for example, it could be really usable to store each item version, so you will be able to retrieve any edits of the comment too. But I still consider that activity node is unnecessary and comments node should be in a simple atom format. Then, I don't understand how "likes" can work in such model, who will count likes? I think that this feature should be implemented in aggregators, some services which aggregates messages from one or more servers and give services of statistics, search or something else. About post deletion, why to have this notification? Is it not enough to receive a regular retract event? > > I don't want servers to have to maintain an activity log to be compliant > though, which is why item persistence on the activity node is optional. A > simple server can receive a published activity item, relay it to subscribers, > apply it to the comments node state, and then throw it away. > >> 2) Why to use Activity Streams here? I totally misunderstanding why to >> have such complication since commenting behavior totally get into simple >> Atom format. > > AS make the most sense for the activity node. It makes less sense for the > comments node though. Perhaps the comments node could be an Atom-only > representation. > >> 3) You have filter parameter by parent id but have no any mechanism to >> represent such relations. But this mechanism is already defined in XEP-277. > > Hmm, yes. Agree it should be thr:in-reply-to. > >> 4) atom:id MUST be an URI. > > Oops, you're right. > >> 5) Again, why to url encode a node name? >> ("comments?order=-created&parent_ids=1%2C5a%2Co19g%2C") > > It was just an easy approach that would work with existing pubsub client > libraries. Perhaps a better idea would be to allow in pubsub iq > requests, then we can have an x:data form for the extra parameters. It seems so. > > Justin > -- With best regards, Sergey Dobrov, XMPP Developer and JRuDevels.org founder.
Re: [Standards] Microblogging: XEP-0277 and beyond
On Thursday, September 01, 2011 11:22:30 AM Sergey Dobrov wrote: > Then, my questions about XEP-303: > > 1) Why to use three nodes? Info node can be replaced by some item with > some constant id. about necessity of activity node I don't understood at > all, why to have it? I didn't want to put the info item in with the comments since they are not really part of the same collection of data. The gain with a separate node is that you could subscribe to just the 'info', and not the comments, if for some reason you wanted to do that. Maybe that isn't very useful. I do believe there is a need to have separate comments and activity nodes though, and possibly even more in the future (I want to have an "occupants" node containing a userlist of people with temporary subscriptions -- think MUC over pubsub). So, I have already resigned to needing multiple nodes, and when it came time to need 'info', I decided a singleton made logical sense. The activity node is actually the core node of the system. Flowing through it would be the stream of activity occuring in the conversation. If you want to post a comment, like a comment, delete a comment, etc, it makes sense to publish them as "activity". If you want to receive events for these kinds of activities (e.g. to track the conversation while you are away, and power something like Facebook Notifications), then again you want to be subscribed to items of "activity". This is distinct from the comments node, which only holds the comments in their current state. For example, if a comment is posted and edited three times after that, this would be represented as four items in the activity node but still only one item (updated three times) in the comments node. The comments node is needed to drive application displays. So this is why I believe both nodes are needed. I don't want servers to have to maintain an activity log to be compliant though, which is why item persistence on the activity node is optional. A simple server can receive a published activity item, relay it to subscribers, apply it to the comments node state, and then throw it away. > 2) Why to use Activity Streams here? I totally misunderstanding why to > have such complication since commenting behavior totally get into simple > Atom format. AS make the most sense for the activity node. It makes less sense for the comments node though. Perhaps the comments node could be an Atom-only representation. > 3) You have filter parameter by parent id but have no any mechanism to > represent such relations. But this mechanism is already defined in XEP-277. Hmm, yes. Agree it should be thr:in-reply-to. > 4) atom:id MUST be an URI. Oops, you're right. > 5) Again, why to url encode a node name? > ("comments?order=-created&parent_ids=1%2C5a%2Co19g%2C") It was just an easy approach that would work with existing pubsub client libraries. Perhaps a better idea would be to allow in pubsub iq requests, then we can have an x:data form for the extra parameters. Justin
Re: [Standards] Microblogging: XEP-0277 and beyond
On 09/02/2011 01:22 AM, Sergey Dobrov wrote: On 09/01/2011 03:21 AM, Justin Karneges wrote: On Wednesday, August 31, 2011 11:32:53 AM Sergey Dobrov wrote: IMO, the commenting approach described in XEP-277 is too simple and rather uninspired (full disclaimer: I work at a company whose sole product is commenting). Commenting is similar, but not identical to microblogging, and the two deserve separate specs. Probably the commenting section in XEP-277 should be exchanged with a pointer to XEP-303. I reviewed the XEP-303 and I'm sorry that I did not that before since it has some really significant notices, so I'll do some notes below and maybe you are right with a pointer to XEP-303. Anyway, are you saying that all of the functional requirements of XEP-303 could be extracted out into separate smaller "reusable" extensions that implementors of generic XEP-60 servers might embrace? Really ejabberd might add an extension that mirrors VCards content in Atom entries? I guess in theory that would be cool, but it seems a bit on the dreamy side. I meant that common cases should be in XEP-60 or some other XEPs that will describe some specific node behavior. For example, I don't sure that this is ok to include filters into the node name (why to url quote these?), but it can be useful to determine some queries protocol which can be inserted in the request of items, so we can use generic pubsub nodes without some functionality. Then, my questions about XEP-303: 1) Why to use three nodes? Info node can be replaced by some item with some constant id. about necessity of activity node I don't understood at all, why to have it? 2) Why to use Activity Streams here? I totally misunderstanding why to have such complication since commenting behavior totally get into simple Atom format. 3) You have filter parameter by parent id but have no any mechanism to represent such relations. But this mechanism is already defined in XEP-277. 4) atom:id MUST be an URI. 5) Again, why to url encode a node name? ("comments?order=-created&parent_ids=1%2C5a%2Co19g%2C") I forgot about one thing: I think that this is unnecessary to validate a sender on server side. That is why: I wrote a blog importer from the juick.com microblogging service to my implementation of the XEP-277 based service and when it imports comments it uses fake JIDs but the service sends publisher attribute so it can be validated who exactly sent a comment but it's also possible to make such imports or another pretty things. So I think that validation is unnecessary but publisher attribute usage must be clarified more strictly in the XEP-60.
Re: [Standards] Microblogging: XEP-0277 and beyond
On 09/01/2011 03:21 AM, Justin Karneges wrote: On Wednesday, August 31, 2011 11:32:53 AM Sergey Dobrov wrote: IMO, the commenting approach described in XEP-277 is too simple and rather uninspired (full disclaimer: I work at a company whose sole product is commenting). Commenting is similar, but not identical to microblogging, and the two deserve separate specs. Probably the commenting section in XEP-277 should be exchanged with a pointer to XEP-303. I reviewed the XEP-303 and I'm sorry that I did not that before since it has some really significant notices, so I'll do some notes below and maybe you are right with a pointer to XEP-303. Anyway, are you saying that all of the functional requirements of XEP-303 could be extracted out into separate smaller "reusable" extensions that implementors of generic XEP-60 servers might embrace? Really ejabberd might add an extension that mirrors VCards content in Atom entries? I guess in theory that would be cool, but it seems a bit on the dreamy side. I meant that common cases should be in XEP-60 or some other XEPs that will describe some specific node behavior. For example, I don't sure that this is ok to include filters into the node name (why to url quote these?), but it can be useful to determine some queries protocol which can be inserted in the request of items, so we can use generic pubsub nodes without some functionality. Then, my questions about XEP-303: 1) Why to use three nodes? Info node can be replaced by some item with some constant id. about necessity of activity node I don't understood at all, why to have it? 2) Why to use Activity Streams here? I totally misunderstanding why to have such complication since commenting behavior totally get into simple Atom format. 3) You have filter parameter by parent id but have no any mechanism to represent such relations. But this mechanism is already defined in XEP-277. 4) atom:id MUST be an URI. 5) Again, why to url encode a node name? ("comments?order=-created&parent_ids=1%2C5a%2Co19g%2C")
Re: [Standards] Microblogging: XEP-0277 and beyond
On Wednesday, August 31, 2011 11:32:53 AM Sergey Dobrov wrote: > On 08/31/2011 02:56 AM, Justin Karneges wrote: > > On Tuesday, August 30, 2011 04:16:29 AM Sergey Dobrov wrote: > > > > By dynamic node, I just mean the node can accept parameters when > > fulfilling requests. > > > > I guess you could write some reusable code that marshals the parameters > > (i.e. popping them off the end of a node name). But, which parameters > > are possible and how the parameters shall be honored is node-specific, > > so there's not really much more generic code you can write. > > Do you mean that pubsub service which serves commenting node should be > not a generic pubsub service but some specific? > > Then, I don't really understand what is necessity of XEP-303 at all. In > the XEP-277 is already defined a way how to comment on microblog posts, > why not to use the same way, why is needed another entity? > > Then, some things in XEP-303 are just corrections for XEP-60 which will > bring mess in standards and pubsub developers will not know where to > find proper specification which will lead to incompatibilities in > different pubsub implementation. > > About dynamic nodes, I think that it's completely unnecessary thing and > it's enough to define the way to sort on created or modified time in the > XEP-60 itself. But I really afraid about that inconsistency which may be > invented with such XEPs as 303 (277 too, he-he). > > And, maybe it will be useful to make possible to make some hooks on item > publishing to validate posted data but that's a subject for another thread. IMO, the commenting approach described in XEP-277 is too simple and rather uninspired (full disclaimer: I work at a company whose sole product is commenting). Commenting is similar, but not identical to microblogging, and the two deserve separate specs. Probably the commenting section in XEP-277 should be exchanged with a pointer to XEP-303. Anyway, are you saying that all of the functional requirements of XEP-303 could be extracted out into separate smaller "reusable" extensions that implementors of generic XEP-60 servers might embrace? Really ejabberd might add an extension that mirrors VCards content in Atom entries? I guess in theory that would be cool, but it seems a bit on the dreamy side. -Justin
Re: [Standards] Microblogging: XEP-0277 and beyond
On 08/31/2011 02:56 AM, Justin Karneges wrote: > On Tuesday, August 30, 2011 04:16:29 AM Sergey Dobrov wrote: > > By dynamic node, I just mean the node can accept parameters when fulfilling > requests. > > I guess you could write some reusable code that marshals the parameters (i.e. > popping them off the end of a node name). But, which parameters are possible > and how the parameters shall be honored is node-specific, so there's not > really > much more generic code you can write. Do you mean that pubsub service which serves commenting node should be not a generic pubsub service but some specific? Then, I don't really understand what is necessity of XEP-303 at all. In the XEP-277 is already defined a way how to comment on microblog posts, why not to use the same way, why is needed another entity? Then, some things in XEP-303 are just corrections for XEP-60 which will bring mess in standards and pubsub developers will not know where to find proper specification which will lead to incompatibilities in different pubsub implementation. About dynamic nodes, I think that it's completely unnecessary thing and it's enough to define the way to sort on created or modified time in the XEP-60 itself. But I really afraid about that inconsistency which may be invented with such XEPs as 303 (277 too, he-he). And, maybe it will be useful to make possible to make some hooks on item publishing to validate posted data but that's a subject for another thread. -- With best regards, Sergey Dobrov, XMPP Developer and JRuDevels.org founder.
Re: [Standards] Microblogging: XEP-0277 and beyond
On Tuesday, August 30, 2011 04:16:29 AM Sergey Dobrov wrote: > On 08/26/2011 01:08 AM, Justin Karneges wrote: > > On Thursday, August 25, 2011 07:27:16 AM Sergey Dobrov wrote: > >> On 08/25/2011 03:36 AM, Justin Karneges wrote: > >>> For example, my XEP-0303 proposes selecting ordering using special node > >>> name suffixing, such as "?order=-created" to indicate created time > >>> descending order. But this is just one way to go about it. > >> > >> I don't have enough time to dive into XEP-303 enough but I don't think > >> that this is a good idea to use "dynamic" pubsub nodes without separate > >> XEP for it. Again, I don't think that different ordering is really need > >> at this time. > > > > Maybe you don't need it, but I need it. :) The commenting protocol > > requires both created time ordering (for application display render) and > > modified time ordering (for tracking updates). But, I am open to > > discussing alternatives that don't involve dynamic node names. > > But where determined what is "dynamic" node? How servers can implement > it without specification? By dynamic node, I just mean the node can accept parameters when fulfilling requests. I guess you could write some reusable code that marshals the parameters (i.e. popping them off the end of a node name). But, which parameters are possible and how the parameters shall be honored is node-specific, so there's not really much more generic code you can write. Justin
Re: [Standards] Microblogging: XEP-0277 and beyond
On 08/26/2011 01:08 AM, Justin Karneges wrote: > On Thursday, August 25, 2011 07:27:16 AM Sergey Dobrov wrote: >> On 08/25/2011 03:36 AM, Justin Karneges wrote: >>> Yes, if tracking changes is needed then the node should have a way of >>> offering items ordered by modified time. >>> >>> Being able to query for items using different kinds of orderings would be >>> useful though, so I didn't want to stipulate that pubsub with RSM must >>> always be ordered by modified time. Rather, each node would have some >>> sort of natural ordering depending on intended purpose, and if we want >>> the client to be able to select among multiple orderings offered by a >>> node then that would be a separate extension or scheme. >> >> Yes, it's sounds ok but what with retracts again? > > At the moment, I feel this is best done by having a node-specific item format > indicating deletion. For Atom-based pubsub nodes, this could be contentless > . I don't sure that it's fine decision since blank entry is not equal to a retracted item. Anyway, it needs to be described in XEP-60. > > But I think it would also be good to see about getting into the > spec > for iq exchange. > >> And how I can >> understand if I missed something and should to re-request items from node? > > Each pubsub message event should contain two RSM ids: the id of the item > contained in the event as well as the id of the item that directly precedes > it > according to the item ordering. Whether by iq or message, the client would > remember the last known RSM id. If a message event is received and the "id > of > preceding item" in that message does not match the last id known by the > client, then the client can choose to catch up by using iq to fetch items > since the last known id. > > We could use SHIM for this: > > http://jabber.org/protocol/shim";> > 64fc10d2 > f4ac5920 > > That is seems to be ok. >>> For example, my XEP-0303 proposes selecting ordering using special node >>> name suffixing, such as "?order=-created" to indicate created time >>> descending order. But this is just one way to go about it. >> >> I don't have enough time to dive into XEP-303 enough but I don't think >> that this is a good idea to use "dynamic" pubsub nodes without separate >> XEP for it. Again, I don't think that different ordering is really need >> at this time. > > Maybe you don't need it, but I need it. :) The commenting protocol requires > both created time ordering (for application display render) and modified time > ordering (for tracking updates). But, I am open to discussing alternatives > that don't involve dynamic node names. But where determined what is "dynamic" node? How servers can implement it without specification? > > Justin > -- With best regards, Sergey Dobrov, XMPP Developer and JRuDevels.org founder.
Re: [Standards] Microblogging: XEP-0277 and beyond
On Thursday, August 25, 2011 07:27:16 AM Sergey Dobrov wrote: > On 08/25/2011 03:36 AM, Justin Karneges wrote: > > Yes, if tracking changes is needed then the node should have a way of > > offering items ordered by modified time. > > > > Being able to query for items using different kinds of orderings would be > > useful though, so I didn't want to stipulate that pubsub with RSM must > > always be ordered by modified time. Rather, each node would have some > > sort of natural ordering depending on intended purpose, and if we want > > the client to be able to select among multiple orderings offered by a > > node then that would be a separate extension or scheme. > > Yes, it's sounds ok but what with retracts again? At the moment, I feel this is best done by having a node-specific item format indicating deletion. For Atom-based pubsub nodes, this could be contentless . But I think it would also be good to see about getting into the spec for iq exchange. > And how I can > understand if I missed something and should to re-request items from node? Each pubsub message event should contain two RSM ids: the id of the item contained in the event as well as the id of the item that directly precedes it according to the item ordering. Whether by iq or message, the client would remember the last known RSM id. If a message event is received and the "id of preceding item" in that message does not match the last id known by the client, then the client can choose to catch up by using iq to fetch items since the last known id. We could use SHIM for this: http://jabber.org/protocol/shim";> 64fc10d2 f4ac5920 > > For example, my XEP-0303 proposes selecting ordering using special node > > name suffixing, such as "?order=-created" to indicate created time > > descending order. But this is just one way to go about it. > > I don't have enough time to dive into XEP-303 enough but I don't think > that this is a good idea to use "dynamic" pubsub nodes without separate > XEP for it. Again, I don't think that different ordering is really need > at this time. Maybe you don't need it, but I need it. :) The commenting protocol requires both created time ordering (for application display render) and modified time ordering (for tracking updates). But, I am open to discussing alternatives that don't involve dynamic node names. Justin
Re: [Standards] Microblogging: XEP-0277 and beyond
On 08/25/2011 03:36 AM, Justin Karneges wrote: > On Wednesday, August 24, 2011 03:34:27 AM Sergey Dobrov wrote: >> On 08/24/2011 01:38 AM, Justin Karneges wrote: >>> On Tuesday, August 23, 2011 04:34:12 AM Sergey Dobrov wrote: On 08/23/2011 09:14 AM, Justin Karneges wrote: > The basic solve for item retraction is for the server to keep deletion > markers around. For example the server could clear all content of an > item and flag it as deleted, but keep it in the same position in the > item list. This way retracted items can be reported in iq responses. That would not work because it is equal to the item update with the blank payload but how could you know that the item was updated? >>> >>> I'm still unsure what you're saying but I'll try to guess: do you mean to >>> say that you cannot determine the difference between a publish vs >>> update, unless you have a local copy of the item to compare with? If >>> so, I have found that in many situations this does not actually matter. >>> Can you share a use case? >> >> No, I tell you that even if I will be able to receive items according to >> the publish time then I can't find if some items was updated before that >> time. But if it's by modify time then yes, all seems to be ok except of >> retract. > > Yes, if tracking changes is needed then the node should have a way of > offering > items ordered by modified time. > > Being able to query for items using different kinds of orderings would be > useful though, so I didn't want to stipulate that pubsub with RSM must always > be ordered by modified time. Rather, each node would have some sort of > natural > ordering depending on intended purpose, and if we want the client to be able > to select among multiple orderings offered by a node then that would be a > separate extension or scheme. Yes, it's sounds ok but what with retracts again? And how I can understand if I missed something and should to re-request items from node? > > For example, my XEP-0303 proposes selecting ordering using special node name > suffixing, such as "?order=-created" to indicate created time descending > order. > But this is just one way to go about it. I don't have enough time to dive into XEP-303 enough but I don't think that this is a good idea to use "dynamic" pubsub nodes without separate XEP for it. Again, I don't think that different ordering is really need at this time. > > Justin > -- With best regards, Sergey Dobrov, XMPP Developer and JRuDevels.org founder.
Re: [Standards] Microblogging: XEP-0277 and beyond
On Tue, 2011-08-23 at 08:34 -0600, Joe Hildebrand wrote: > On 8/21/11 4:45 PM, "Justin Karneges" > wrote: > > > Good catch, I didn't consider clock skew. I wonder if the client should > > just > > specify a timestamp 5-10m in the past instead of having magic at the node > > servers. Then accurate time queries (made relative to timestamps of other > > items provided by the server) stay optimized. > > Seems to me that client writers are more likely to incorrectly think the > server clock is going to be "right" than the other way around, but I don't > care where the skew is added as long as it's specified which side does it, > so we don't double-skew. I'm wonder how good this would work in practice. Distributed clocks are hard. I am also not sure if you would actually need/want this in practice. Thinking aloud from here on. The use of information like User Mood and User Tune in IM clients is usually that they are just shown as icons in a roster. However, I image that most 'microblogging' applications would have a 'river of updates' UI. In this setting, I'd assume you want to limit the updates in some way (more specific types, more specific senders (not everyone on your roster), etc). Would you then still use implicit subscriptions? If not, I assume the subscription would persist over sessions, only sending out notifications based on presence. Now you need a way to exchange your client's state to each of the services that you hold subscriptions with. These RSM ids would work, but it is a cumbersome to send these out every time you come online. The only thing I could come up with is a twisted scheme were you basically do acking on received notifications. I.e. the service remembers what you got last. Hmm. -- ralphm
Re: [Standards] Microblogging: XEP-0277 and beyond
On Wed, 2011-08-24 at 17:34 +0700, Sergey Dobrov wrote: > On 08/24/2011 01:38 AM, Justin Karneges wrote: > > On Tuesday, August 23, 2011 04:34:12 AM Sergey Dobrov wrote: > >> On 08/23/2011 09:14 AM, Justin Karneges wrote: > >>> The basic solve for item retraction is for the server to keep deletion > >>> markers around. For example the server could clear all content of an > >>> item and flag it as deleted, but keep it in the same position in the > >>> item list. This way retracted items can be reported in iq responses. > >> > >> That would not work because it is equal to the item update with the > >> blank payload but how could you know that the item was updated? > > > > I'm still unsure what you're saying but I'll try to guess: do you mean to > > say > > that you cannot determine the difference between a publish vs update, > > unless > > you have a local copy of the item to compare with? If so, I have found > > that > > in many situations this does not actually matter. Can you share a use case? > No, I tell you that even if I will be able to receive items according to > the publish time then I can't find if some items was updated before that > time. But if it's by modify time then yes, all seems to be ok except of > retract. Are you saying here that you want intermediate changes for the same item (i.e. the same ItemID) that took place while the client was not online to receive the notifications? If so, the model in XEP-0060 is that each update to an item is a complete replacement for that item, so a service will not keep record of previous 'versions'. If you really want that you should probably use unique identifiers for each update. In the field of microblogging I don't see a particular use case for this. Microblog entries typically don't change once posted, and even if they would, an end-user would probably only be interested in the most recent version. For more expanded use cases in Federating Social Networks (beyond microblogs) that cover other social objects (regular blogs, stories, pictures, people's profiles) updates are more common. Let me expand on a real-world example. At Mediamatic we've built a network of sites that federate using (amongst others) XMPP PubSub. The sites are basically collections of Things that have relationships or Edges between them, with a particular predicate. Very semantic web, RDF-like. For exchanging Things, we make an Atom representation of the Thing with all of its 'outgoing' Edges, encoded as elements. (Example: http://www.mediamatic.net/atom/24879). Each Thing has its own node (xmpp:pubsub.mediamatic.net?;node=id/24879). In this setting, for each update (changed title, new relationships) we just push the whole representation to the subscribing sites (ItemID 'current'). Upon reception, the local shadow representation is entirely replaced with the version just received. We did think about sending around delta's at some point, especially for edge updates, because the pings are really fat. Up till now, we haven't pursued this because it makes the operation more fragile. If you missed an update, you have to figure what you've missed, likely by replaying a journal. Alternatively, I could think of having a more explicit subscription, where you have the publishing service filter out certain bits (unwanted edges with particular predicates) in the Atom in the notification, and only send updates for 'fields' you are interested in. We also played with the thought of having separate items for derived (meta) data for each thing. These could hold information on the amount of comments, likes, visits and various 'ratings', that you might want to show on the federated sites. We haven't pursued this mostly because lack of time. I'm also not sure about how to encode this information, as items in a node typically have their payload in the same namespace. Encoding this meta data in Atom might be somewhat of an overkill. > >> From my point of view, the best solution is node journal but I can't be > >> sure since that idea was not discussed. > > > > True. I am trying to avoid this if possible. My stance is there is > > nothing > > wrong with using a journal to implement pubsub, but ideally pubsub-using > > protocols should not be so complex that they require a journal-based > > implementation to participate. > I don't insist on journal, I just need a reliable way to retrieve node > changes and I can't think up another way. In our case, we don't retract items, but do send node deletes (sometimes with a redirected to another Thing, when there was a move between sites or two things have been consolidated into one). I suppose that wouldn't work nicely in your setting, because all of our subscriptions are explicit. We do have something akin to collections or 'generic subscriptions', which act more like an Atom Feed. There we create or update items based on the atom:id inside the payloads, and don't actually look at the ItemID. Deletes are not published in this sett
Re: [Standards] Microblogging: XEP-0277 and beyond
On Wednesday, August 24, 2011 03:34:27 AM Sergey Dobrov wrote: > On 08/24/2011 01:38 AM, Justin Karneges wrote: > > On Tuesday, August 23, 2011 04:34:12 AM Sergey Dobrov wrote: > >> On 08/23/2011 09:14 AM, Justin Karneges wrote: > >>> The basic solve for item retraction is for the server to keep deletion > >>> markers around. For example the server could clear all content of an > >>> item and flag it as deleted, but keep it in the same position in the > >>> item list. This way retracted items can be reported in iq responses. > >> > >> That would not work because it is equal to the item update with the > >> blank payload but how could you know that the item was updated? > > > > I'm still unsure what you're saying but I'll try to guess: do you mean to > > say that you cannot determine the difference between a publish vs > > update, unless you have a local copy of the item to compare with? If > > so, I have found that in many situations this does not actually matter. > > Can you share a use case? > > No, I tell you that even if I will be able to receive items according to > the publish time then I can't find if some items was updated before that > time. But if it's by modify time then yes, all seems to be ok except of > retract. Yes, if tracking changes is needed then the node should have a way of offering items ordered by modified time. Being able to query for items using different kinds of orderings would be useful though, so I didn't want to stipulate that pubsub with RSM must always be ordered by modified time. Rather, each node would have some sort of natural ordering depending on intended purpose, and if we want the client to be able to select among multiple orderings offered by a node then that would be a separate extension or scheme. For example, my XEP-0303 proposes selecting ordering using special node name suffixing, such as "?order=-created" to indicate created time descending order. But this is just one way to go about it. Justin
Re: [Standards] Microblogging: XEP-0277 and beyond
On 08/24/2011 01:38 AM, Justin Karneges wrote: > On Tuesday, August 23, 2011 04:34:12 AM Sergey Dobrov wrote: >> On 08/23/2011 09:14 AM, Justin Karneges wrote: >>> The basic solve for item retraction is for the server to keep deletion >>> markers around. For example the server could clear all content of an >>> item and flag it as deleted, but keep it in the same position in the >>> item list. This way retracted items can be reported in iq responses. >> >> That would not work because it is equal to the item update with the >> blank payload but how could you know that the item was updated? > > I'm still unsure what you're saying but I'll try to guess: do you mean to say > that you cannot determine the difference between a publish vs update, unless > you have a local copy of the item to compare with? If so, I have found that > in many situations this does not actually matter. Can you share a use case? No, I tell you that even if I will be able to receive items according to the publish time then I can't find if some items was updated before that time. But if it's by modify time then yes, all seems to be ok except of retract. > >> From my point of view, the best solution is node journal but I can't be >> sure since that idea was not discussed. > > True. I am trying to avoid this if possible. My stance is there is nothing > wrong with using a journal to implement pubsub, but ideally pubsub-using > protocols should not be so complex that they require a journal-based > implementation to participate. I don't insist on journal, I just need a reliable way to retrieve node changes and I can't think up another way. > > Justin > -- With best regards, Sergey Dobrov, XMPP Developer and JRuDevels.org founder.
Re: [Standards] Microblogging: XEP-0277 and beyond
On Tuesday, August 23, 2011 04:34:12 AM Sergey Dobrov wrote: > On 08/23/2011 09:14 AM, Justin Karneges wrote: > > The basic solve for item retraction is for the server to keep deletion > > markers around. For example the server could clear all content of an > > item and flag it as deleted, but keep it in the same position in the > > item list. This way retracted items can be reported in iq responses. > > That would not work because it is equal to the item update with the > blank payload but how could you know that the item was updated? I'm still unsure what you're saying but I'll try to guess: do you mean to say that you cannot determine the difference between a publish vs update, unless you have a local copy of the item to compare with? If so, I have found that in many situations this does not actually matter. Can you share a use case? > From my point of view, the best solution is node journal but I can't be > sure since that idea was not discussed. True. I am trying to avoid this if possible. My stance is there is nothing wrong with using a journal to implement pubsub, but ideally pubsub-using protocols should not be so complex that they require a journal-based implementation to participate. Justin
Re: [Standards] Microblogging: XEP-0277 and beyond
On 8/22/11 8:14 PM, "Justin Karneges" wrote: > The basic solve for item retraction is for the server to keep deletion markers > around. For example the server could clear all content of an item and flag it > as deleted, but keep it in the same position in the item list. This way > retracted items can be reported in iq responses. > > Unfortunately XEP-0060 never discusses sending in an iq response > where normally is, so I am not sure if this would be allowed today > without a protocol update. Alternatively, retractions could be represented > within the items themselves. For example if you were using Atom, an empty > element may suffice to indicate deletion. > > Item publish and update are fine though, right? Or what are you thinking > about > them? I was hoping for something more like atomic-subscribe-and-send-updates as a subscription option, so that the client's notification handling didn't have to change. The server could then just send normal retracts in that flow. If there was another subscription option that says "it's ok to send me multiple notifications in the same message", the server could intelligently batch the notifications together for clients that have added the extra foreach loop. -- Joe Hildebrand
Re: [Standards] Microblogging: XEP-0277 and beyond
On 8/21/11 4:45 PM, "Justin Karneges" wrote: > Good catch, I didn't consider clock skew. I wonder if the client should just > specify a timestamp 5-10m in the past instead of having magic at the node > servers. Then accurate time queries (made relative to timestamps of other > items provided by the server) stay optimized. Seems to me that client writers are more likely to incorrectly think the server clock is going to be "right" than the other way around, but I don't care where the skew is added as long as it's specified which side does it, so we don't double-skew. -- Joe Hildebrand
Re: [Standards] Microblogging: XEP-0277 and beyond
On 08/23/2011 09:14 AM, Justin Karneges wrote: > On Monday, August 22, 2011 05:12:54 AM Sergey Dobrov wrote: >> On 08/19/2011 04:55 AM, Justin Karneges wrote: >>> On Thursday, August 18, 2011 05:10:58 AM Goffi wrote: - the ability to get missed items between the last time a contact was disconnected and the time it reconnect >>> >>> I have a protocol flow for reliable pubsub that I've not yet publicized, >>> but essentially the idea is to include Result Set Management (XEP-0059) >>> UID values in pubsub event notifications. The client can then notice if >>> it missed an item and use iq-get to catch up. >>> >>> Maybe this could go in some "RSM with PubSub" XEP that sorely needs to be >>> written. >>> >>> Justin >> >> You forgot that we need to know about item retraction as well as about >> item publishing. The same thing about item update. The possibility to >> get items by date and time is good but it's not cover these problems and >> it's not necessary at this time from my point of view. > > The basic solve for item retraction is for the server to keep deletion > markers > around. For example the server could clear all content of an item and flag > it > as deleted, but keep it in the same position in the item list. This way > retracted items can be reported in iq responses. That would not work because it is equal to the item update with the blank payload but how could you know that the item was updated? > > Unfortunately XEP-0060 never discusses sending in an iq response > where normally is, so I am not sure if this would be allowed today > without a protocol update. Alternatively, retractions could be represented > within the items themselves. For example if you were using Atom, an empty > element may suffice to indicate deletion. Protocol needs many edits anyway. For example, it very useful to see a publisher of an item but XEP-60 doesn't give us good explanation how to determine if pubsub service can send such notifications. > > Item publish and update are fine though, right? Or what are you thinking > about > them? Publish is ok since we have RSM. Update has the same problem as retract. >From my point of view, the best solution is node journal but I can't be sure since that idea was not discussed. > > Justin > -- With best regards, Sergey Dobrov, XMPP Developer and JRuDevels.org founder.
Re: [Standards] Microblogging: XEP-0277 and beyond
On Monday, August 22, 2011 05:12:54 AM Sergey Dobrov wrote: > On 08/19/2011 04:55 AM, Justin Karneges wrote: > > On Thursday, August 18, 2011 05:10:58 AM Goffi wrote: > >> - the ability to get missed items between the last time a contact was > >> disconnected and the time it reconnect > > > > I have a protocol flow for reliable pubsub that I've not yet publicized, > > but essentially the idea is to include Result Set Management (XEP-0059) > > UID values in pubsub event notifications. The client can then notice if > > it missed an item and use iq-get to catch up. > > > > Maybe this could go in some "RSM with PubSub" XEP that sorely needs to be > > written. > > > > Justin > > You forgot that we need to know about item retraction as well as about > item publishing. The same thing about item update. The possibility to > get items by date and time is good but it's not cover these problems and > it's not necessary at this time from my point of view. The basic solve for item retraction is for the server to keep deletion markers around. For example the server could clear all content of an item and flag it as deleted, but keep it in the same position in the item list. This way retracted items can be reported in iq responses. Unfortunately XEP-0060 never discusses sending in an iq response where normally is, so I am not sure if this would be allowed today without a protocol update. Alternatively, retractions could be represented within the items themselves. For example if you were using Atom, an empty element may suffice to indicate deletion. Item publish and update are fine though, right? Or what are you thinking about them? Justin
Re: [Standards] Microblogging: XEP-0277 and beyond
On 08/18/2011 09:47 PM, Goffi wrote: > Le Jeudi 18 Août 2011 20:40:05, Sergey Dobrov a écrit : >> >> It's a bad idea to append the number to the namespace since it reserved >> for different revisions of XEP and not for your purpose. Again, I think >> that this should be solved by some privacy lists extension since your >> decision again restricted. > > As I said before, it just a Q&D hack for testing purpose, I know it's not a > good solution. I would rather use privacy lists or whatever if we could have > a > per-item access management with it. But you should understand that this standard can be proposed only if the basic microblogging will work well. > >> >>> If I have to poll this by myself at client side, I'll have to send X IQ >>> stanza for my X contacts, and flood the server. >> >> Client should cache data. You will flood anyway the only difference is >> who will flood: client or server. > > In all cases the client will need the data, and it's better to send one > request to server, that X request for X contacts. > Actually I think about something similar to Extended Stanza Adressing > (http://xmpp.org/extensions/xep-0033.html), but to get several pubsub nodes > at > once. XEP-33 is good but it can't be used for iq stanzas... > > >>> I ask last items presence to not have to poll individually all my >>> contacts roster. On some blue centralised services, it's common to have >>> 100+ contacts, should I poll all of them each time I log-in ? >> >> No, Pubsub node can send you an event when you came online with the >> serial for it's journal so you can decide to ask the node for new items. > > Oki, that looks like a nice solution, once again, waiting an answer from > pubsub people. > > It seems that the only way is to design a prototype for the XEP and publish it... -- With best regards, Sergey Dobrov, XMPP Developer and JRuDevels.org founder.
Re: [Standards] Microblogging: XEP-0277 and beyond
On 08/19/2011 04:55 AM, Justin Karneges wrote: > On Thursday, August 18, 2011 05:10:58 AM Goffi wrote: >> - the ability to get missed items between the last time a contact was >> disconnected and the time it reconnect > > I have a protocol flow for reliable pubsub that I've not yet publicized, but > essentially the idea is to include Result Set Management (XEP-0059) UID > values > in pubsub event notifications. The client can then notice if it missed an > item > and use iq-get to catch up. > > Maybe this could go in some "RSM with PubSub" XEP that sorely needs to be > written. > > Justin > You forgot that we need to know about item retraction as well as about item publishing. The same thing about item update. The possibility to get items by date and time is good but it's not cover these problems and it's not necessary at this time from my point of view. -- With best regards, Sergey Dobrov, XMPP Developer and JRuDevels.org founder.
Re: [Standards] Microblogging: XEP-0277 and beyond
On Saturday, August 20, 2011 12:11:05 AM Joe Hildebrand wrote: > On 8/19/11 12:12 PM, "Justin Karneges" > > wrote: > > Indeed. In my case I consider a time-based query to be inclusive, so as > > to not miss updates occuring within the same second. In the rare case > > you pick up the same item again, it's harmless. > > I'd go a little further, and suggest that the publishing server should back > up enough time to account for clock skew, and republish everything from > that earlier time. 5-10m should do it, as long as there aren't a lot of > items. Good catch, I didn't consider clock skew. I wonder if the client should just specify a timestamp 5-10m in the past instead of having magic at the node servers. Then accurate time queries (made relative to timestamps of other items provided by the server) stay optimized. Justin
Re: [Standards] Microblogging: XEP-0277 and beyond
On 8/19/11 12:12 PM, "Justin Karneges" wrote: > In the context of PEP, it would just be a matter of putting this same type of > position spec into the subscription request, and then each remote node would > reply with all items since that time (or just the one item only if it has > changed, I think PEP is always singleton?). There's no reason PEP is always a singleton. If the publisher wants singleton, she uses id='current'. > Indeed. In my case I consider a time-based query to be inclusive, so as to > not miss updates occuring within the same second. In the rare case you pick > up the same item again, it's harmless. I'd go a little further, and suggest that the publishing server should back up enough time to account for clock skew, and republish everything from that earlier time. 5-10m should do it, as long as there aren't a lot of items. -- Joe Hildebrand
Re: [Standards] Microblogging: XEP-0277 and beyond
On Friday, August 19, 2011 10:10:01 AM Joe Hildebrand wrote: > On 8/18/11 2:55 PM, "Justin Karneges" > > wrote: > > I have a protocol flow for reliable pubsub that I've not yet publicized, > > but essentially the idea is to include Result Set Management (XEP-0059) > > UID values in pubsub event notifications. The client can then notice if > > it missed an item > > and use iq-get to catch up. > > > > Maybe this could go in some "RSM with PubSub" XEP that sorely needs to be > > written. > > I like the direction you're heading with this. While we're at it, we need > something similar for PEP that uses date/timestamps on implicit subscribes, > since one implicit subscription goes to multiple publishing nodes, each of > which have different RSM ids. The hope is that you could cache between > sessions, and only receive PEP updates for things that have changed. This > would be a huge win for XEP-292 PEP-based vCard, particularly if avatars > are included. To fetch items since some timestamp, I'm encoding that in the item retrieval request using client-meaningful RSM ids. For example, you could do an iq-get for items since RSM id "time:{isodate}". This type of id would only be used in requests. (Note: I'm open to using a different approach for letting the client specify positions, but this seemed simple enough). In the context of PEP, it would just be a matter of putting this same type of position spec into the subscription request, and then each remote node would reply with all items since that time (or just the one item only if it has changed, I think PEP is always singleton?). > One thing we have going for us is that a re-publish of an existing item (by > id) replaces the previous version, so a little bit of overlap between > sessions is ok Indeed. In my case I consider a time-based query to be inclusive, so as to not miss updates occuring within the same second. In the rare case you pick up the same item again, it's harmless. Justin
Re: [Standards] Microblogging: XEP-0277 and beyond
On 8/18/11 2:55 PM, "Justin Karneges" wrote: > I have a protocol flow for reliable pubsub that I've not yet publicized, but > essentially the idea is to include Result Set Management (XEP-0059) UID values > in pubsub event notifications. The client can then notice if it missed an > item > and use iq-get to catch up. > > Maybe this could go in some "RSM with PubSub" XEP that sorely needs to be > written. I like the direction you're heading with this. While we're at it, we need something similar for PEP that uses date/timestamps on implicit subscribes, since one implicit subscription goes to multiple publishing nodes, each of which have different RSM ids. The hope is that you could cache between sessions, and only receive PEP updates for things that have changed. This would be a huge win for XEP-292 PEP-based vCard, particularly if avatars are included. One thing we have going for us is that a re-publish of an existing item (by id) replaces the previous version, so a little bit of overlap between sessions is ok If there's any way to incorporate that into your thinking so we end up with a consistent design, it would be cool. -- Joe Hildebrand
Re: [Standards] Microblogging: XEP-0277 and beyond
On Thursday, August 18, 2011 05:10:58 AM Goffi wrote: > - the ability to get missed items between the last time a contact was > disconnected and the time it reconnect I have a protocol flow for reliable pubsub that I've not yet publicized, but essentially the idea is to include Result Set Management (XEP-0059) UID values in pubsub event notifications. The client can then notice if it missed an item and use iq-get to catch up. Maybe this could go in some "RSM with PubSub" XEP that sorely needs to be written. Justin
Re: [Standards] Microblogging: XEP-0277 and beyond
It's my understanding of the XEP process that we standardise ways of doing something so that many clients and servers can interoperate. And for machine to machine services this is easy: machines do as we say and interoperate as we say. It's a predictable environment. Social is not easy and neither are user and groups of users predictable. Additionally, social services built on XMPP will have different business logic and be built into different applications in different ways. Getting this business-logic right will depend on the problem being solved. That's why I like XEP-0277; it says all one can really say about social (summary of XEP-0277: use atom for posts and post to a pub-sub like node). We all want interoperation between services. Social is different for different products: what works for one kind of use case will not work for someone else: some people think it should be roster based with groups like Google circles. Others want it decoupled from the roster. Some people want to add code to XMPP servers, others build pub-sub solutions. Some services have a single poster model, others are more like MUC with many posters. To help make this conversation go forward we need a solid description of what is being offered to the user. These are the questions we need to answer before we try to work out a spec(s): * the target user (hint: my parents won't be twiddling roster groups) * define the use case * how does it work? what happens when... (this is the social part) * clearly define the publishing and following models I am extremely doubtful that this process leads to a one-size-fits-all answer. And it would be a rather mediocre mishmash of kludges. I think a far more realistic result will be a couple of types of social services. Off the top of my head: * microblogging to pep nodes * twitter-like service * buddycloud-like service Each service (and hopefully spec) will relevant to a particular way of interacting and designed for a type of use case. It's really important that we avoid a rush to standardise that results in something mediocre and end up with the MySpace of the XMPP world: dropped after 2 years and filled with spam. XMPP gives us a huge advantage when building social services: its native federation and its built-in support for remote user-ids are awesome. Let's build awesome services for our users based off specs relevant for a particular use case. S. On 18/08/2011 16:47, Goffi wrote: > Le Jeudi 18 Août 2011 20:40:05, Sergey Dobrov a écrit : >> It's a bad idea to append the number to the namespace since it reserved >> for different revisions of XEP and not for your purpose. Again, I think >> that this should be solved by some privacy lists extension since your >> decision again restricted. > As I said before, it just a Q&D hack for testing purpose, I know it's not a > good solution. I would rather use privacy lists or whatever if we could have > a > per-item access management with it. > >>> If I have to poll this by myself at client side, I'll have to send X IQ >>> stanza for my X contacts, and flood the server. >> Client should cache data. You will flood anyway the only difference is >> who will flood: client or server. > In all cases the client will need the data, and it's better to send one > request to server, that X request for X contacts. > Actually I think about something similar to Extended Stanza Adressing > (http://xmpp.org/extensions/xep-0033.html), but to get several pubsub nodes > at > once. > > >>> I ask last items presence to not have to poll individually all my >>> contacts roster. On some blue centralised services, it's common to have >>> 100+ contacts, should I poll all of them each time I log-in ? >> No, Pubsub node can send you an event when you came online with the >> serial for it's journal so you can decide to ask the node for new items. > Oki, that looks like a nice solution, once again, waiting an answer from > pubsub people. > -- Simon Tennant mobile: +49 17 8545 0880 office: +49 89 4209 55854 office: +44 20 7043 6756 xmpp: si...@buddycloud.com build your own open and federated social network - http://open.buddycloud.com
Re: [Standards] Microblogging: XEP-0277 and beyond
Le Jeudi 18 Août 2011 20:40:05, Sergey Dobrov a écrit : > > It's a bad idea to append the number to the namespace since it reserved > for different revisions of XEP and not for your purpose. Again, I think > that this should be solved by some privacy lists extension since your > decision again restricted. As I said before, it just a Q&D hack for testing purpose, I know it's not a good solution. I would rather use privacy lists or whatever if we could have a per-item access management with it. > > > If I have to poll this by myself at client side, I'll have to send X IQ > > stanza for my X contacts, and flood the server. > > Client should cache data. You will flood anyway the only difference is > who will flood: client or server. In all cases the client will need the data, and it's better to send one request to server, that X request for X contacts. Actually I think about something similar to Extended Stanza Adressing (http://xmpp.org/extensions/xep-0033.html), but to get several pubsub nodes at once. > > I ask last items presence to not have to poll individually all my > > contacts roster. On some blue centralised services, it's common to have > > 100+ contacts, should I poll all of them each time I log-in ? > > No, Pubsub node can send you an event when you came online with the > serial for it's journal so you can decide to ask the node for new items. Oki, that looks like a nice solution, once again, waiting an answer from pubsub people.
Re: [Standards] Microblogging: XEP-0277 and beyond
On 08/18/2011 08:09 PM, Goffi wrote: > Le Jeudi 18 Août 2011 19:29:21, Sergey Dobrov a écrit : >>> first, here are my main needs for microblogging: >>> >>> - the possibility to have several nodes with different access models, >>> and for a user to subscribe to them automatically >> >> Could you give some example usecases for that? Since I don't really >> understand how you see it possible. > > Well, in my quick and dirty implementation (that I have made mainly to work > on > the frontend), I have several node: urn:xmpp:microblog:0 for the main > microblog node, open access, that everybody can read, even people without > XMPP > account from the website. > > Then I have many other nodes: urn:xmpp:microblog:1, urn:xmpp:microblog:2, ... > Each node has a different access (roster-access "familly", roster-access > "friends"), and I have an other private node (with a whitelist access for > only > the published) which keep a map between my roster-group and the node, so only > the published know that "urn:xmpp:microblog:1" is linked with "familly" > roster > group. Thanks to this map, I can publish to the good node when needed. > > My contacts have a notification because they have discovery feature for > urn:xmpp:microblog:1+notify, urn:xmpp:microblog:2+notify, ..., > urn:xmpp:microblog:big_enough_number+notify (BTW, it would be nice to have > some wildcard charactere like urn:xmpp:microblog:*+notify), and thanks to > access model, then just have notifications if they are in the good group. > > This implementation is a dirty hack, but it works at least for testing > purpose. It would be actually not that bad if I could ask the server all the > items (or the last x ones) that I could access from a given jid. > > You can try it at www.libervia.org (very old interface, a new one is on its > way), but you'll have to create several accounts and play with roster to test > it. > It's a bad idea to append the number to the namespace since it reserved for different revisions of XEP and not for your purpose. Again, I think that this should be solved by some privacy lists extension since your decision again restricted. > >> >>> - the possibility to pull all items of severals peoples (e.g. a roster >>> group) at once >> >> That impossible since these users can be served on different hosts. So >> you should to ask users separately. I don't think that this is too >> ambiguous because it will be needed only once and then you will receive >> events immediately. > > I know that of course, but I think it's server job. A server can host many of > my contacts nodes, so it can at least return the ones it is hosting, and then > ask to other servers the informations it is lacking. At client side, that > mean > just on IQ get stanza, then having the result, which can be huge so maybe > splitted in several stanza. > > If I have to poll this by myself at client side, I'll have to send X IQ > stanza > for my X contacts, and flood the server. Client should cache data. You will flood anyway the only difference is who will flood: client or server. > > >>> - push notification when a contact is connected >> >> Presence?? :)) > > Yes this point is already managed in xep-0277, I just mentionned it cause > it's > a must-have. > > >> >>> - the ability to get missed items between the last time a contact was >>> disconnected and the time it reconnect >> >> I already proposed the solution for that but nobody reacted: >> http://mail.jabber.org/pipermail/standards/2011-June/024597.html > > Yes I know, so I hope this time there will be more feedback > >>> The current XEP is based on PEP, and has IMHO many issues: >>> >>> - if you connect with +notify, you have all notifications from all you >>> items, without the ability to filter (e.g. from only on group). >> >> I think that this issue can be solved with an extension for the privacy >> lists. > > I agree > >>> - if somebody published an item 6 months ago, and nothing since, you'll >>> still have it in your notifications. OK you can filter this at client >>> side, but it would be nice to have a date-based filtering way. >> >> Why? I configure the node that it doesn't send last items when presence >> came and this is quite ok. > > I ask last items presence to not have to poll individually all my contacts > roster. On some blue centralised services, it's common to have 100+ contacts, > should I poll all of them each time I log-in ? > > No, Pubsub node can send you an event when you came online with the serial for it's journal so you can decide to ask the node for new items. >>> - maybe the most important: there is no way to know how many items you >>> have missed neither to get them. Once again, a date-based retrieving >>> way would be a nice addition >> >> Once again: http://mail.jabber.org/pipermail/standards/2011-June/024597.html > > Hope to have feedback from pubsub specialists > Me too :) > >>> - there is no way to retrieve the x last items for
Re: [Standards] Microblogging: XEP-0277 and beyond
Le Jeudi 18 Août 2011 19:29:21, Sergey Dobrov a écrit : > > first, here are my main needs for microblogging: > > > > - the possibility to have several nodes with different access models, > > and for a user to subscribe to them automatically > > Could you give some example usecases for that? Since I don't really > understand how you see it possible. Well, in my quick and dirty implementation (that I have made mainly to work on the frontend), I have several node: urn:xmpp:microblog:0 for the main microblog node, open access, that everybody can read, even people without XMPP account from the website. Then I have many other nodes: urn:xmpp:microblog:1, urn:xmpp:microblog:2, ... Each node has a different access (roster-access "familly", roster-access "friends"), and I have an other private node (with a whitelist access for only the published) which keep a map between my roster-group and the node, so only the published know that "urn:xmpp:microblog:1" is linked with "familly" roster group. Thanks to this map, I can publish to the good node when needed. My contacts have a notification because they have discovery feature for urn:xmpp:microblog:1+notify, urn:xmpp:microblog:2+notify, ..., urn:xmpp:microblog:big_enough_number+notify (BTW, it would be nice to have some wildcard charactere like urn:xmpp:microblog:*+notify), and thanks to access model, then just have notifications if they are in the good group. This implementation is a dirty hack, but it works at least for testing purpose. It would be actually not that bad if I could ask the server all the items (or the last x ones) that I could access from a given jid. You can try it at www.libervia.org (very old interface, a new one is on its way), but you'll have to create several accounts and play with roster to test it. > > > - the possibility to pull all items of severals peoples (e.g. a roster > > group) at once > > That impossible since these users can be served on different hosts. So > you should to ask users separately. I don't think that this is too > ambiguous because it will be needed only once and then you will receive > events immediately. I know that of course, but I think it's server job. A server can host many of my contacts nodes, so it can at least return the ones it is hosting, and then ask to other servers the informations it is lacking. At client side, that mean just on IQ get stanza, then having the result, which can be huge so maybe splitted in several stanza. If I have to poll this by myself at client side, I'll have to send X IQ stanza for my X contacts, and flood the server. > > - push notification when a contact is connected > > Presence?? :)) Yes this point is already managed in xep-0277, I just mentionned it cause it's a must-have. > > > - the ability to get missed items between the last time a contact was > > disconnected and the time it reconnect > > I already proposed the solution for that but nobody reacted: > http://mail.jabber.org/pipermail/standards/2011-June/024597.html Yes I know, so I hope this time there will be more feedback > > The current XEP is based on PEP, and has IMHO many issues: > > > > - if you connect with +notify, you have all notifications from all you > > items, without the ability to filter (e.g. from only on group). > > I think that this issue can be solved with an extension for the privacy > lists. I agree > > - if somebody published an item 6 months ago, and nothing since, you'll > > still have it in your notifications. OK you can filter this at client > > side, but it would be nice to have a date-based filtering way. > > Why? I configure the node that it doesn't send last items when presence > came and this is quite ok. I ask last items presence to not have to poll individually all my contacts roster. On some blue centralised services, it's common to have 100+ contacts, should I poll all of them each time I log-in ? > > - maybe the most important: there is no way to know how many items you > > have missed neither to get them. Once again, a date-based retrieving > > way would be a nice addition > > Once again: http://mail.jabber.org/pipermail/standards/2011-June/024597.html Hope to have feedback from pubsub specialists > > - there is no way to retrieve the x last items for x contacts. I'd like > > to be able to ask to a server « all the post since 1 week from all my > > contacts in roster-group "friends" ». > > You are forget about decentralization again. Again, why is this needed > if you receive the events IMMEDIATELY? Because I'm not always online. For the decentralized issue, see my answer above. > [SNIP] > > Furthermore, there is this email from Sergey Dobrov that has not any > > answer: > > http://mail.jabber.org/pipermail/standards/2011-June/024618.html . It's > > a pity 'cause there are many interesting remarks in it. For example, I > > agree that adding tags would be nice, or the need to have a way to get > > items count. > I working to
Re: [Standards] Microblogging: XEP-0277 and beyond
On 08/18/2011 07:10 PM, Goffi wrote: > G'day everybody, > > as it is my first post on @standard, I just quickly present myself: I'm the > developper of "Salut à Toi", a multi-frontend XMPP client (you can have a > presentation here ==> > http://www.goffi.org/post/2011/06/05/Salut-%C3%A0-Toi%3A- > a-multi-frontends-XMPP-client ). My client use microblogging, and I have an > implementation of XEP-0277 on it (with a dirty custom hack to have several > roster-access nodes), which was made before the replies/comments update. > > first, here are my main needs for microblogging: > > - the possibility to have several nodes with different access models, and for > a > user to subscribe to them automatically Could you give some example usecases for that? Since I don't really understand how you see it possible. > > - the possibility to pull all items of severals peoples (e.g. a roster group) > at once That impossible since these users can be served on different hosts. So you should to ask users separately. I don't think that this is too ambiguous because it will be needed only once and then you will receive events immediately. > > - push notification when a contact is connected Presence?? :)) > > - the ability to get missed items between the last time a contact was > disconnected and the time it reconnect I already proposed the solution for that but nobody reacted: http://mail.jabber.org/pipermail/standards/2011-June/024597.html > > > > The current XEP is based on PEP, and has IMHO many issues: > > - if you connect with +notify, you have all notifications from all you items, > without the ability to filter (e.g. from only on group). I think that this issue can be solved with an extension for the privacy lists. > > - if somebody published an item 6 months ago, and nothing since, you'll still > have it in your notifications. OK you can filter this at client side, but it > would be nice to have a date-based filtering way. Why? I configure the node that it doesn't send last items when presence came and this is quite ok. > > - maybe the most important: there is no way to know how many items you have > missed neither to get them. Once again, a date-based retrieving way would be > a > nice addition Once again: http://mail.jabber.org/pipermail/standards/2011-June/024597.html > > - there is no way to retrieve the x last items for x contacts. I'd like to be > able to ask to a server « all the post since 1 week from all my contacts in > roster-group "friends" ». You are forget about decentralization again. Again, why is this needed if you receive the events IMMEDIATELY? > > > > I have heard about the buddycloud draft on the subject: > https://buddycloud.org/wiki/XMPP_XEP , and I think there are good ideas > there, > like using Message Archive Management ( > http://doomsong.co.uk/extensions/render/message-archive-management.html ) for > the date-based retrieving. I think that this feature can be omitted for the first time since it can be solved more easily. > > > Furthermore, there is this email from Sergey Dobrov that has not any answer: > http://mail.jabber.org/pipermail/standards/2011-June/024618.html . It's a > pity > 'cause there are many interesting remarks in it. For example, I agree that > adding tags would be nice, or the need to have a way to get items count. I working to integrate these edits by myself and propose to accept the new version since it seems that nobody if not me. > > I think it's important to focus on microblogging, as it become more and more > common these days. I also think that roster-access need special attention, > specially in server side: it's important to have permission management easy > for the end-user. That's ok but I think that we should work on independent solution that keeps in mind our own way (decentralized, flexibility, restrictions absence). > > Cheers > Jérôme Poisson (aka Goffi) > > PS: I have also worked on features for my client that I'd like to > standardize, > like a generic card games management. > Great thing! -- With best regards, Sergey Dobrov, XMPP Developer and JRuDevels.org founder.