Re: [Standards] Unread syncing

[Michal Piotrowski]

On 14 December 2016 at 14:50, Kevin Smith  wrote:

> On 14 Dec 2016, at 11:46, Michal Piotrowski  tions.com> wrote:
>
>
>
> On 2 December 2016 at 18:18, forenjunkie  wrote:
>
>> Ah now im understanding, basically the server should give you a list of
>> contacts to query for messages.
>>
>> i would see this as a simply addition to MAM.
>>
>
> Would you see it as a new parameter to MAM query? or rather a completely
> new kind of iq to MAM service?
>
>
> What I’m currently speccing up after a few side discussions with folks
> recently is actually something that replaces the initial bind and does all
> the bits that need doing for multi-client at once. I need to find a couple
> of hours to finish writing it up so we can get it in the inbox and start
> discussion.
>
> The Archive already could know what messages you read, because of
>> chatmarkers. it would only need to hold the last read marker stanza id for
>> every contact in roster, and perform some SQL magic on query.
>>
>
> I can think of a situation where there is no roster (on XMPP server) but
> still there is archive and we want to get list of contacts with unread
> messages.
>
>
> I like Kevin's idea with the iq and list of unread messages in a result.
> Also I think it would be beneficial for clients to have not only id of the
> unread message but also the content.
> This could be optional, of course, but in some cases getting the unread
> message content would make UI update easier.
>
>
> I think the content of the messages is fairly straightforward to get from
> a MAM query immediately afterwards. I was originally thinking that you’d
> get sent the messages themselves, but I’m coming to the conclusion that it
> doesn’t buy much, and makes the protocol more complex. I guess we’ll see
> once I’ve finished speccing it whether this works or not.
>
> The client doesn't have to query the archive if it only wants to display
> part of the unread message content on the UI.
>
>
> That’s true, but is it significantly harder for the client to send that
> query, if the server has already told it exactly the IDs it needs to query
> between?
>

For me as a server dev it doesn't make much difference if the client asks
for the message or not.
>From my experience working with client devs (mostly mobile recently) I can
say that they (who I worked with) would like to have everything what's
needed in single result so that the app doesn't have to send additional
requests.
I agree this is not always possible and makes the protocol more complex.
I just wanted to share my experience regarding this.


>
>
/K
>
> ___
> Standards mailing list
> Info: https://mail.jabber.org/mailman/listinfo/standards
> Unsubscribe: standards-unsubscr...@xmpp.org
> ___
>
>
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Unread syncing

[Kevin Smith]

On 14 Dec 2016, at 11:46, Michal Piotrowski 
 wrote:
> 
> 
> On 2 December 2016 at 18:18, forenjunkie  > wrote:
> Ah now im understanding, basically the server should give you a list of 
> contacts to query for messages.
> 
> i would see this as a simply addition to MAM.
> 
> Would you see it as a new parameter to MAM query? or rather a completely new 
> kind of iq to MAM service?

What I’m currently speccing up after a few side discussions with folks recently 
is actually something that replaces the initial bind and does all the bits that 
need doing for multi-client at once. I need to find a couple of hours to finish 
writing it up so we can get it in the inbox and start discussion.

> The Archive already could know what messages you read, because of 
> chatmarkers. it would only need to hold the last read marker stanza id for 
> every contact in roster, and perform some SQL magic on query.
> 
> I can think of a situation where there is no roster (on XMPP server) but 
> still there is archive and we want to get list of contacts with unread 
> messages.
> 
> 
> I like Kevin's idea with the iq and list of unread messages in a result.
> Also I think it would be beneficial for clients to have not only id of the 
> unread message but also the content.
> This could be optional, of course, but in some cases getting the unread 
> message content would make UI update easier.

I think the content of the messages is fairly straightforward to get from a MAM 
query immediately afterwards. I was originally thinking that you’d get sent the 
messages themselves, but I’m coming to the conclusion that it doesn’t buy much, 
and makes the protocol more complex. I guess we’ll see once I’ve finished 
speccing it whether this works or not.

> The client doesn't have to query the archive if it only wants to display part 
> of the unread message content on the UI.

That’s true, but is it significantly harder for the client to send that query, 
if the server has already told it exactly the IDs it needs to query between?

/K___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Unread syncing

[Michal Piotrowski]

On 2 December 2016 at 18:18, forenjunkie  wrote:

> Ah now im understanding, basically the server should give you a list of
> contacts to query for messages.
>
> i would see this as a simply addition to MAM.
>

Would you see it as a new parameter to MAM query? or rather a completely
new kind of iq to MAM service?


> The Archive already could know what messages you read, because of
> chatmarkers. it would only need to hold the last read marker stanza id for
> every contact in roster, and perform some SQL magic on query.
>

I can think of a situation where there is no roster (on XMPP server) but
still there is archive and we want to get list of contacts with unread
messages.


I like Kevin's idea with the iq and list of unread messages in a result.
Also I think it would be beneficial for clients to have not only id of the
unread message but also the content.
This could be optional, of course, but in some cases getting the unread
message content would make UI update easier.
The client doesn't have to query the archive if it only wants to display
part of the unread message content on the UI.


>
> it would send you a list of contacts and afterwards you could start
> querying the archive for messages of these contacts.
>
>
>
> Am 02.12.2016 um 17:14 schrieb Kevin Smith:
>
>>
>> On 2 Dec 2016, at 16:03, forenjunkie  wrote:
>>>
>>> Why would you querry the whole archive?
>>>
>> Because the question isn't "do I have unread messages for contact X?"
>> It's "in what chats do I have unread messages and how many?" on login. I
>> don't see a way to solve this using the building blocks we have without
>> exhaustive querying of the archive. The case of fetching context when the
>> user later opens a chat is the easy part.
>>
>> This is why I asked with a specific example, for which I don't think we
>> currently have protocol. Can you show me what the protocol would look like
>> for the case I mentioned earlier?
>>
>> /K
>>
>> if you open a chat with a contact you querry "Give me the last X messages
>>> for contact A"
>>>
>>> And if you open the chat window with contact B you do the same for
>>> contact B.
>>>
>>> I never did write a implementation of MAM myself, but if im reading the
>>> XEP, you can filter with various attributes, you dont have to step through
>>> EVERY message.
>>>
>>> Am 02.12.2016 um 16:57 schrieb Kevin Smith:

> On 2 Dec 2016, at 15:49, forenjunkie  wrote:
> Its not written down somewhere that its up to the client, but it makes
> no sense to put a selflimiting hard rule on a archiving XEP like MAM and
> exclude certain messages per XEP rule.
>

 We have store hints, the most prominent servers respect these. And it
> would make no sense to not respect it if a client explicitly wishes for a
> message to be stored.
>
 Oh, it’d make lots of sense. Server admins can very reasonably want to
 choose how their archive is populated, rather than having remote clients do
 so.

 So to make it more clear, if i want to save a message on a current
> prosody or ejabbered MAM implemenation, i can do this as a client with a
> store hint.
>
 While that’s nice for you, Prosody and ejabberd are certainly not the
 whole world ;)

 you dont have to querry the whole archive, you just querry until you
> get a read marker, then you know everything that comes before that was
> already read so i dont have to query it.
>
 This is untrue, though. If I have two contacts, it’s quite easy for me
 to have unread messages for contact A that are hundreds or thousands of
 messages older than my messages from contact B, all of which are read. If I
 merely read the archive backwards until I found a read marker, I wouldn’t
 find the unread messages for contact A.

 /K

 Am 02.12.2016 um 16:34 schrieb Kevin Smith:
>>
>>> On 2 Dec 2016, at 15:26, forenjunkie  wrote:
>>> in a single chat conversation, it makes no sense to query unread
>>> messages.
>>>
>> I’m not sure that’s true at all, but putting that to the side for the
>> minute...
>>
>> you could just query the last 10 MAM Messages, if no readmarker comes
>>> with it, query another 10. such a implementation of MAM would be very 
>>> weird
>>> to me, but you could do it and get only unread messages with it. So this
>>> already works without problem with the MAM and 0333 XEP.
>>> And in single chat its not possible to "miss" a message in between,
>>> which you would want to query.
>>>
>> I think you’re missing the details of what I asked - how do you
>> achieve this where there are a sufficient number of messages that just
>> keeping querying until you have every message in your local archive isn’t
>> viable?
>>
>> And this is not a theory, XEP-0333 are stored by the 

Re: [Standards] Unread syncing

[forenjunkie]

Ah now im understanding, basically the server should give you a list of 
contacts to query for messages.


i would see this as a simply addition to MAM.

The Archive already could know what messages you read, because of 
chatmarkers. it would only need to hold the last read marker stanza id 
for every contact in roster, and perform some SQL magic on query.


it would send you a list of contacts and afterwards you could start 
querying the archive for messages of these contacts.



Am 02.12.2016 um 17:14 schrieb Kevin Smith:



On 2 Dec 2016, at 16:03, forenjunkie  wrote:

Why would you querry the whole archive?

Because the question isn't "do I have unread messages for contact X?" It's "in what 
chats do I have unread messages and how many?" on login. I don't see a way to solve this using 
the building blocks we have without exhaustive querying of the archive. The case of fetching 
context when the user later opens a chat is the easy part.

This is why I asked with a specific example, for which I don't think we 
currently have protocol. Can you show me what the protocol would look like for 
the case I mentioned earlier?

/K


if you open a chat with a contact you querry "Give me the last X messages for 
contact A"

And if you open the chat window with contact B you do the same for contact B.

I never did write a implementation of MAM myself, but if im reading the XEP, 
you can filter with various attributes, you dont have to step through EVERY 
message.


Am 02.12.2016 um 16:57 schrieb Kevin Smith:

On 2 Dec 2016, at 15:49, forenjunkie  wrote:
Its not written down somewhere that its up to the client, but it makes no sense 
to put a selflimiting hard rule on a archiving XEP like MAM and exclude certain 
messages per XEP rule.



We have store hints, the most prominent servers respect these. And it would 
make no sense to not respect it if a client explicitly wishes for a message to 
be stored.

Oh, it’d make lots of sense. Server admins can very reasonably want to choose 
how their archive is populated, rather than having remote clients do so.


So to make it more clear, if i want to save a message on a current prosody or 
ejabbered MAM implemenation, i can do this as a client with a store hint.

While that’s nice for you, Prosody and ejabberd are certainly not the whole 
world ;)


you dont have to querry the whole archive, you just querry until you get a read 
marker, then you know everything that comes before that was already read so i 
dont have to query it.

This is untrue, though. If I have two contacts, it’s quite easy for me to have 
unread messages for contact A that are hundreds or thousands of messages older 
than my messages from contact B, all of which are read. If I merely read the 
archive backwards until I found a read marker, I wouldn’t find the unread 
messages for contact A.

/K


Am 02.12.2016 um 16:34 schrieb Kevin Smith:

On 2 Dec 2016, at 15:26, forenjunkie  wrote:
in a single chat conversation, it makes no sense to query unread messages.

I’m not sure that’s true at all, but putting that to the side for the minute...


you could just query the last 10 MAM Messages, if no readmarker comes with it, 
query another 10. such a implementation of MAM would be very weird to me, but 
you could do it and get only unread messages with it. So this already works 
without problem with the MAM and 0333 XEP.
And in single chat its not possible to "miss" a message in between, which you 
would want to query.

I think you’re missing the details of what I asked - how do you achieve this 
where there are a sufficient number of messages that just keeping querying 
until you have every message in your local archive isn’t viable?


And this is not a theory, XEP-0333 are stored by the server if the client 
wishes that, and working implementations of MAM and 0333 together you can 
witness for example in Conversations.

It’s not up to the client whether 333 is stored by the server or not.


All i see in this proposal is adding complexity to the whole process in 
introducing another thing the server has to support.

Referring back to my previous question, can you provide an example of how to 
achieve this case with just 313 and 333 (in protocol)?

/K
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Unread syncing

[Kevin Smith]

> On 2 Dec 2016, at 16:03, forenjunkie  wrote:
> 
> Why would you querry the whole archive?

Because the question isn't "do I have unread messages for contact X?" It's "in 
what chats do I have unread messages and how many?" on login. I don't see a way 
to solve this using the building blocks we have without exhaustive querying of 
the archive. The case of fetching context when the user later opens a chat is 
the easy part. 

This is why I asked with a specific example, for which I don't think we 
currently have protocol. Can you show me what the protocol would look like for 
the case I mentioned earlier?

/K

> 
> if you open a chat with a contact you querry "Give me the last X messages for 
> contact A"
> 
> And if you open the chat window with contact B you do the same for contact B.
> 
> I never did write a implementation of MAM myself, but if im reading the XEP, 
> you can filter with various attributes, you dont have to step through EVERY 
> message.
> 
>> Am 02.12.2016 um 16:57 schrieb Kevin Smith:
>>> On 2 Dec 2016, at 15:49, forenjunkie  wrote:
>>> Its not written down somewhere that its up to the client, but it makes no 
>>> sense to put a selflimiting hard rule on a archiving XEP like MAM and 
>>> exclude certain messages per XEP rule.
>> 
>> 
>>> We have store hints, the most prominent servers respect these. And it would 
>>> make no sense to not respect it if a client explicitly wishes for a message 
>>> to be stored.
>> Oh, it’d make lots of sense. Server admins can very reasonably want to 
>> choose how their archive is populated, rather than having remote clients do 
>> so.
>> 
>>> So to make it more clear, if i want to save a message on a current prosody 
>>> or ejabbered MAM implemenation, i can do this as a client with a store hint.
>> While that’s nice for you, Prosody and ejabberd are certainly not the whole 
>> world ;)
>> 
>>> you dont have to querry the whole archive, you just querry until you get a 
>>> read marker, then you know everything that comes before that was already 
>>> read so i dont have to query it.
>> This is untrue, though. If I have two contacts, it’s quite easy for me to 
>> have unread messages for contact A that are hundreds or thousands of 
>> messages older than my messages from contact B, all of which are read. If I 
>> merely read the archive backwards until I found a read marker, I wouldn’t 
>> find the unread messages for contact A.
>> 
>> /K
>> 
>>> 
 Am 02.12.2016 um 16:34 schrieb Kevin Smith:
> On 2 Dec 2016, at 15:26, forenjunkie  wrote:
> in a single chat conversation, it makes no sense to query unread messages.
 I’m not sure that’s true at all, but putting that to the side for the 
 minute...
 
> you could just query the last 10 MAM Messages, if no readmarker comes 
> with it, query another 10. such a implementation of MAM would be very 
> weird to me, but you could do it and get only unread messages with it. So 
> this already works without problem with the MAM and 0333 XEP.
> And in single chat its not possible to "miss" a message in between, which 
> you would want to query.
 I think you’re missing the details of what I asked - how do you achieve 
 this where there are a sufficient number of messages that just keeping 
 querying until you have every message in your local archive isn’t viable?
 
> And this is not a theory, XEP-0333 are stored by the server if the client 
> wishes that, and working implementations of MAM and 0333 together you can 
> witness for example in Conversations.
 It’s not up to the client whether 333 is stored by the server or not.
 
> All i see in this proposal is adding complexity to the whole process in 
> introducing another thing the server has to support.
 Referring back to my previous question, can you provide an example of how 
 to achieve this case with just 313 and 333 (in protocol)?
 
 /K
 ___
 Standards mailing list
 Info: https://mail.jabber.org/mailman/listinfo/standards
 Unsubscribe: standards-unsubscr...@xmpp.org
 ___
>>> ___
>>> Standards mailing list
>>> Info: https://mail.jabber.org/mailman/listinfo/standards
>>> Unsubscribe: standards-unsubscr...@xmpp.org
>>> ___
>> ___
>> Standards mailing list
>> Info: https://mail.jabber.org/mailman/listinfo/standards
>> Unsubscribe: standards-unsubscr...@xmpp.org
>> ___
> 
> ___
> Standards mailing list
> Info: https://mail.jabber.org/mailman/listinfo/standards
> Unsubscribe: standards-unsubscr...@xmpp.org
> ___

Re: [Standards] Unread syncing

[forenjunkie]

Why would you querry the whole archive?

if you open a chat with a contact you querry "Give me the last X 
messages for contact A"


And if you open the chat window with contact B you do the same for 
contact B.


I never did write a implementation of MAM myself, but if im reading the 
XEP, you can filter with various attributes, you dont have to step 
through EVERY message.


Am 02.12.2016 um 16:57 schrieb Kevin Smith:

On 2 Dec 2016, at 15:49, forenjunkie  wrote:

Its not written down somewhere that its up to the client, but it makes no sense 
to put a selflimiting hard rule on a archiving XEP like MAM and exclude certain 
messages per XEP rule.




We have store hints, the most prominent servers respect these. And it would 
make no sense to not respect it if a client explicitly wishes for a message to 
be stored.

Oh, it’d make lots of sense. Server admins can very reasonably want to choose 
how their archive is populated, rather than having remote clients do so.


So to make it more clear, if i want to save a message on a current prosody or 
ejabbered MAM implemenation, i can do this as a client with a store hint.

While that’s nice for you, Prosody and ejabberd are certainly not the whole 
world ;)


you dont have to querry the whole archive, you just querry until you get a read 
marker, then you know everything that comes before that was already read so i 
dont have to query it.

This is untrue, though. If I have two contacts, it’s quite easy for me to have 
unread messages for contact A that are hundreds or thousands of messages older 
than my messages from contact B, all of which are read. If I merely read the 
archive backwards until I found a read marker, I wouldn’t find the unread 
messages for contact A.

/K



Am 02.12.2016 um 16:34 schrieb Kevin Smith:

On 2 Dec 2016, at 15:26, forenjunkie  wrote:

in a single chat conversation, it makes no sense to query unread messages.

I’m not sure that’s true at all, but putting that to the side for the minute...


you could just query the last 10 MAM Messages, if no readmarker comes with it, 
query another 10. such a implementation of MAM would be very weird to me, but 
you could do it and get only unread messages with it. So this already works 
without problem with the MAM and 0333 XEP.
And in single chat its not possible to "miss" a message in between, which you 
would want to query.

I think you’re missing the details of what I asked - how do you achieve this 
where there are a sufficient number of messages that just keeping querying 
until you have every message in your local archive isn’t viable?


And this is not a theory, XEP-0333 are stored by the server if the client 
wishes that, and working implementations of MAM and 0333 together you can 
witness for example in Conversations.

It’s not up to the client whether 333 is stored by the server or not.


All i see in this proposal is adding complexity to the whole process in 
introducing another thing the server has to support.

Referring back to my previous question, can you provide an example of how to 
achieve this case with just 313 and 333 (in protocol)?

/K
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___


___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Unread syncing

[Kevin Smith]

On 2 Dec 2016, at 15:49, forenjunkie  wrote:
> 
> Its not written down somewhere that its up to the client, but it makes no 
> sense to put a selflimiting hard rule on a archiving XEP like MAM and exclude 
> certain messages per XEP rule.



> 
> We have store hints, the most prominent servers respect these. And it would 
> make no sense to not respect it if a client explicitly wishes for a message 
> to be stored.

Oh, it’d make lots of sense. Server admins can very reasonably want to choose 
how their archive is populated, rather than having remote clients do so.

> So to make it more clear, if i want to save a message on a current prosody or 
> ejabbered MAM implemenation, i can do this as a client with a store hint.

While that’s nice for you, Prosody and ejabberd are certainly not the whole 
world ;)

> you dont have to querry the whole archive, you just querry until you get a 
> read marker, then you know everything that comes before that was already read 
> so i dont have to query it.

This is untrue, though. If I have two contacts, it’s quite easy for me to have 
unread messages for contact A that are hundreds or thousands of messages older 
than my messages from contact B, all of which are read. If I merely read the 
archive backwards until I found a read marker, I wouldn’t find the unread 
messages for contact A.

/K

> 
> 
> Am 02.12.2016 um 16:34 schrieb Kevin Smith:
>> On 2 Dec 2016, at 15:26, forenjunkie  wrote:
>>> in a single chat conversation, it makes no sense to query unread messages.
>> I’m not sure that’s true at all, but putting that to the side for the 
>> minute...
>> 
>>> you could just query the last 10 MAM Messages, if no readmarker comes with 
>>> it, query another 10. such a implementation of MAM would be very weird to 
>>> me, but you could do it and get only unread messages with it. So this 
>>> already works without problem with the MAM and 0333 XEP.
>>> And in single chat its not possible to "miss" a message in between, which 
>>> you would want to query.
>> I think you’re missing the details of what I asked - how do you achieve this 
>> where there are a sufficient number of messages that just keeping querying 
>> until you have every message in your local archive isn’t viable?
>> 
>>> And this is not a theory, XEP-0333 are stored by the server if the client 
>>> wishes that, and working implementations of MAM and 0333 together you can 
>>> witness for example in Conversations.
>> It’s not up to the client whether 333 is stored by the server or not.
>> 
>>> All i see in this proposal is adding complexity to the whole process in 
>>> introducing another thing the server has to support.
>> Referring back to my previous question, can you provide an example of how to 
>> achieve this case with just 313 and 333 (in protocol)?
>> 
>> /K
>> ___
>> Standards mailing list
>> Info: https://mail.jabber.org/mailman/listinfo/standards
>> Unsubscribe: standards-unsubscr...@xmpp.org
>> ___
> 
> ___
> Standards mailing list
> Info: https://mail.jabber.org/mailman/listinfo/standards
> Unsubscribe: standards-unsubscr...@xmpp.org
> ___

___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Unread syncing

[forenjunkie]

Its not written down somewhere that its up to the client, but it makes 
no sense to put a selflimiting hard rule on a archiving XEP like MAM and 
exclude certain messages per XEP rule.


We have store hints, the most prominent servers respect these. And it 
would make no sense to not respect it if a client explicitly wishes for 
a message to be stored.
So to make it more clear, if i want to save a message on a current 
prosody or ejabbered MAM implemenation, i can do this as a client with a 
store hint.


you dont have to querry the whole archive, you just querry until you get 
a read marker, then you know everything that comes before that was 
already read so i dont have to query it.



Am 02.12.2016 um 16:34 schrieb Kevin Smith:

On 2 Dec 2016, at 15:26, forenjunkie  wrote:

in a single chat conversation, it makes no sense to query unread messages.

I’m not sure that’s true at all, but putting that to the side for the minute...


you could just query the last 10 MAM Messages, if no readmarker comes with it, 
query another 10. such a implementation of MAM would be very weird to me, but 
you could do it and get only unread messages with it. So this already works 
without problem with the MAM and 0333 XEP.
And in single chat its not possible to "miss" a message in between, which you 
would want to query.

I think you’re missing the details of what I asked - how do you achieve this 
where there are a sufficient number of messages that just keeping querying 
until you have every message in your local archive isn’t viable?


And this is not a theory, XEP-0333 are stored by the server if the client 
wishes that, and working implementations of MAM and 0333 together you can 
witness for example in Conversations.

It’s not up to the client whether 333 is stored by the server or not.


All i see in this proposal is adding complexity to the whole process in 
introducing another thing the server has to support.

Referring back to my previous question, can you provide an example of how to 
achieve this case with just 313 and 333 (in protocol)?

/K
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___


___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Unread syncing

[Kevin Smith]

On 2 Dec 2016, at 15:26, forenjunkie  wrote:
> in a single chat conversation, it makes no sense to query unread messages.

I’m not sure that’s true at all, but putting that to the side for the minute...

> you could just query the last 10 MAM Messages, if no readmarker comes with 
> it, query another 10. such a implementation of MAM would be very weird to me, 
> but you could do it and get only unread messages with it. So this already 
> works without problem with the MAM and 0333 XEP.
> And in single chat its not possible to "miss" a message in between, which you 
> would want to query.

I think you’re missing the details of what I asked - how do you achieve this 
where there are a sufficient number of messages that just keeping querying 
until you have every message in your local archive isn’t viable?

> And this is not a theory, XEP-0333 are stored by the server if the client 
> wishes that, and working implementations of MAM and 0333 together you can 
> witness for example in Conversations.

It’s not up to the client whether 333 is stored by the server or not.

> All i see in this proposal is adding complexity to the whole process in 
> introducing another thing the server has to support.

Referring back to my previous question, can you provide an example of how to 
achieve this case with just 313 and 333 (in protocol)?

/K
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Unread syncing

[forenjunkie]

Hi,

in a single chat conversation, it makes no sense to query unread messages.

you could just query the last 10 MAM Messages, if no readmarker comes 
with it, query another 10. such a implementation of MAM would be very 
weird to me, but you could do it and get only unread messages with it. 
So this already works without problem with the MAM and 0333 XEP.
And in single chat its not possible to "miss" a message in between, 
which you would want to query.


And this is not a theory, XEP-0333 are stored by the server if the 
client wishes that, and working implementations of MAM and 0333 together 
you can witness for example in Conversations.


All i see in this proposal is adding complexity to the whole process in 
introducing another thing the server has to support.


best wishes
lovetox
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Unread syncing

[Kevin Smith]

> On 2 Dec 2016, at 14:35, Christian Schudt  wrote:
> 
> 
>> Except that they’re not chat messages, so won’t be stored, and if they were 
>> you’d be potentially up to doubling the size of your archive (I guess adding 
>> a quarter to, on average) as you fill it with read markers - unless you want 
>> to customise the MAM service to understand unread state, in which case what 
>> have you gained?
> 
> As I read this, I was suprised, that MAM would only store chat messages. But 
> it turns out MAM allows to store "normal" and "additional non-conversation 
> messages" as per § 5.1.1.

Indeed. But you seemed to be wanting to re-use MAM services without 
code-change, which isn’t likely to happen.

>> If you log on, your client does a complete synchronisation of all history 
>> from the (modified to include non-chat history for read markers) archive to 
>> local storage, and then processes the stanzas it will be able to see which 
>> contacts have unread messages and how many, yes. Having to do a full history 
>> download is clearly not tenable in the general case.
> 
> Are we talking about "read by user", or "read by resource”?

“read by user"

> If it's "read by user", a client may receive zero unread messages and 
> therefore not updating its local history, because the messages have already 
> been read by another client (of the same user of course).

The “there are no unread messages” case is the simple one, yes.

> In that case, it's desirable to do a full history download anyway (not a full 
> one, but only since the last download).

Why? I don’t think there’s a requirement that clients keep completely local 
copies of history (they may choose to do so, but they shouldn’t be required to, 
as there are plenty of other history-management strategies)

> If it would only download the unread messages, the client would miss the 
> messages, which have already been read on another device, resulting in an 
> incomplete local history presentation.

I think this is making a big assumption about how clients are going to deal 
with history (that they want to have a complete local copy).

To put this another way, quoting my earlier strawman:












how would one, using just 333 and 313 achieve the same thing? That is: after 
logging in, how does a client quickly discover which of their contacts have 
unread messages, and how many (and the id they need to query MAM for to get 
those messages, when they later want to show them)?

Assume that this is a reasonably high-volume account, that the client hasn’t 
been online for a while, and there are some contacts with unread messages that 
are buried hundreds of messages deep in the MAM archive since that JID was last 
mentioned, such that waiting for a complete download of all ‘missing’ data on 
the current network connection would take a (too large) number of minutes. Or 
just that it’s the first time this client has been used on this account, and 
there’s a lot of data in the archive.

/K
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Unread syncing

[Christian Schudt]

> Except that they’re not chat messages, so won’t be stored, and if they were 
> you’d be potentially up to doubling the size of your archive (I guess adding 
> a quarter to, on average) as you fill it with read markers - unless you want 
> to customise the MAM service to understand unread state, in which case what 
> have you gained?
 
As I read this, I was suprised, that MAM would only store chat messages. But it 
turns out MAM allows to store "normal" and "additional non-conversation 
messages" as per § 5.1.1.


> If you log on, your client does a complete synchronisation of all history 
> from the (modified to include non-chat history for read markers) archive to 
> local storage, and then processes the stanzas it will be able to see which 
> contacts have unread messages and how many, yes. Having to do a full history 
> download is clearly not tenable in the general case.

Are we talking about "read by user", or "read by resource"?

If it's "read by user", a client may receive zero unread messages and therefore 
not updating its local history, because the messages have already been read by 
another client (of the same user of course).
In that case, it's desirable to do a full history download anyway (not a full 
one, but only since the last download).
If it would only download the unread messages, the client would miss the 
messages, which have already been read on another device, resulting in an 
incomplete local history presentation.

If it's "read by resource", then you want MAM to manage which messages have 
been read by which resource?

-- Christian
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Unread syncing

[Kevin Smith]

> On 2 Dec 2016, at 13:42, Christian Schudt  wrote:
> 
>> I think you possibly don’t :)
>> 
>> This is for synchronising the ‘read’ status between all of my clients, such 
>> that a) they’re consistent and b) when a new client comes online it can 
>> quickly determine which contacts have unread messages.
> Isn't MAM supposed to address the issue of "synchronizing multiple 
> resources/clients", so that every client sees the same history of (chat) 
> messages, even if they were originally delivered to another client?

Yes.

> If that syncing works for chat messages, it should work for "read receipt" 
> messages as well, no?

Except that they’re not chat messages, so won’t be stored, and if they were 
you’d be potentially up to doubling the size of your archive (I guess adding a 
quarter to, on average) as you fill it with read markers - unless you want to 
customise the MAM service to understand unread state, in which case what have 
you gained?

> The problem could also be exanded to other use cases like "syncing message 
> corrections" between clients (XEP-0308), which would work the same: each 
> client retrieves messages from MAM and applies them in order. If there's a 
> message correction message, the client corrects the message.
> Likewise, if there are messages, which have no corresponding read marker, 
> they are unread and can be displayed as unread by that client.

If you log on, your client does a complete synchronisation of all history from 
the (modified to include non-chat history for read markers) archive to local 
storage, and then processes the stanzas it will be able to see which contacts 
have unread messages and how many, yes. Having to do a full history download is 
clearly not tenable in the general case.

What I’m proposing is something like












We don’t have a sensible way of providing that with any of our current 
protocols (or, at least, none that spring to my mind).

/K
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Unread syncing

[Christian Schudt]

Isn't MAM supposed to address the issue of "synchronizing multiple resources/clients", so that every client sees the same history of (chat) messages, even if they were originally delivered to another client?

 

If that syncing works for chat messages, it should work for "read receipt" messages as well, no?

 

The problem could also be exanded to other use cases like "syncing message corrections" between clients (XEP-0308), which would work the same: each client retrieves messages from MAM and applies them in order. If there's a message correction message, the client corrects the message.

Likewise, if there are messages, which have no corresponding read marker, they are unread and can be displayed as unread by that client.


 

I think you possibly don’t :)

This is for synchronising the ‘read’ status between all of my clients, such that a) they’re consistent and b) when a new client comes online it can quickly determine which contacts have unread messages.



___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Unread syncing

[Kevin Smith]

> On 2 Dec 2016, at 13:00, Christian Schudt  wrote:
> 
> Can't XEP-0333 used for that?
>  
> A sends a message to B with a "read request".
> B reads the message and sends a "read receipt" back to A.
>  
> The read receipt is stored in the server archive normally as any other (chat) 
> message.
>  
> Clients can query the archive in a normal way, e.g. all message from "today", 
> e.g. receive 3 chat messages and 2 read receipts and therefore know, that 
> there's one "new", unread message.
>  
> I probably don't understand your point though…

I think you possibly don’t :)

This is for synchronising the ‘read’ status between all of my clients, such 
that a) they’re consistent and b) when a new client comes online it can quickly 
determine which contacts have unread messages.

/K

>  
> -- Christian
>  
> Gesendet: Freitag, 02. Dezember 2016 um 12:44 Uhr
> Von: "Kevin Smith" 
> An: "XMPP Standards" 
> Betreff: [Standards] Unread syncing
> A question:
> 
> I’ve always assumed that we would do sync of the unread (or, rather read) 
> status of messages between contacts via private PEP and, indeed, this is the 
> approach we verbally specced at a summit a while back (which I have yet to 
> write up).
> 
> I’ve been thinking about this further, in the context of how the big picture 
> looks for a server and many clients, and I’m coming to the conclusion that 
> it’s not the best approach. Yes, it avoids need for any changes on the 
> server, but I think we’re in a world (MIX, PAM, MAM) where for a modern XMPP 
> setup, we’re going to need modern XMPP servers and so as long as we don’t 
> break existing interop with older systems that’s ok.
> 
> I think that the better solution is going to be putting this into the server, 
> tied with MAM. The basic flow then is that the server injects stable IDs (the 
> ones used by MAM) into stanzas you receive. Then, whenever it likes, a client 
> can tell the server that id X for contact Y has been read. The server checks 
> that this is a newer read marker than was there, to avoid race conditions, 
> and updates its state for that contact (this then gets propagated to other 
> clients). Then when coming online a client can query this state to find not 
> only the contacts with unread messages, but also the number of unread 
> messages. It can then, whenever it wants (e.g. immediately, or when a chat 
> window is opened) query either a chunk of history (e.g. last 20 messages) or 
> all unread messages for that contact (or those contacts). I think this makes 
> the client implementation vastly simpler, the server implementation isn’t 
> complex (and is simpler than the client implementation would need to be for 
> the other approach), it avoids nasty race conditions that need fancy handling 
> otherwise, and also lets a client show useful information (the unread count 
> per contact) with less exchanged data on startup.
> 
> Thoughts?
> 
> /K
> ___
> Standards mailing list
> Info: https://mail.jabber.org/mailman/listinfo/standards
> Unsubscribe: standards-unsubscr...@xmpp.org
> ___
> ___
> Standards mailing list
> Info: https://mail.jabber.org/mailman/listinfo/standards
> Unsubscribe: standards-unsubscr...@xmpp.org
> ___

___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Unread syncing

[Christian Schudt]

Can't XEP-0333 used for that?

 

A sends a message to B with a "read request".

B reads the message and sends a "read receipt" back to A.

 

The read receipt is stored in the server archive normally as any other (chat) message.

 

Clients can query the archive in a normal way, e.g. all message from "today", e.g. receive 3 chat messages and 2 read receipts and therefore know, that there's one "new", unread message.


 

I probably don't understand your point though...

 

-- Christian

 


Gesendet: Freitag, 02. Dezember 2016 um 12:44 Uhr
Von: "Kevin Smith" 
An: "XMPP Standards" 
Betreff: [Standards] Unread syncing

A question:

I’ve always assumed that we would do sync of the unread (or, rather read) status of messages between contacts via private PEP and, indeed, this is the approach we verbally specced at a summit a while back (which I have yet to write up).

I’ve been thinking about this further, in the context of how the big picture looks for a server and many clients, and I’m coming to the conclusion that it’s not the best approach. Yes, it avoids need for any changes on the server, but I think we’re in a world (MIX, PAM, MAM) where for a modern XMPP setup, we’re going to need modern XMPP servers and so as long as we don’t break existing interop with older systems that’s ok.

I think that the better solution is going to be putting this into the server, tied with MAM. The basic flow then is that the server injects stable IDs (the ones used by MAM) into stanzas you receive. Then, whenever it likes, a client can tell the server that id X for contact Y has been read. The server checks that this is a newer read marker than was there, to avoid race conditions, and updates its state for that contact (this then gets propagated to other clients). Then when coming online a client can query this state to find not only the contacts with unread messages, but also the number of unread messages. It can then, whenever it wants (e.g. immediately, or when a chat window is opened) query either a chunk of history (e.g. last 20 messages) or all unread messages for that contact (or those contacts). I think this makes the client implementation vastly simpler, the server implementation isn’t complex (and is simpler than the client implementation would need to be for the other approach), it avoids nasty race conditions that need fancy handling otherwise, and also lets a client show useful information (the unread count per contact) with less exchanged data on startup.

Thoughts?

/K
___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___



___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___

Re: [Standards] Unread syncing

[Kevin Smith]

> On 2 Dec 2016, at 11:44, Kevin Smith  wrote:
> 
> A question:
> 
> I’ve always assumed that we would do sync of the unread (or, rather read) 
> status of messages between contacts via private PEP and, indeed, this is the 
> approach we verbally specced at a summit a while back (which I have yet to 
> write up).

Typo: Between *clients*, not contacts. *facepalm*.

/K

> 
> I’ve been thinking about this further, in the context of how the big picture 
> looks for a server and many clients, and I’m coming to the conclusion that 
> it’s not the best approach. Yes, it avoids need for any changes on the 
> server, but I think we’re in a world (MIX, PAM, MAM) where for a modern XMPP 
> setup, we’re going to need modern XMPP servers and so as long as we don’t 
> break existing interop with older systems that’s ok.
> 
> I think that the better solution is going to be putting this into the server, 
> tied with MAM. The basic flow then is that the server injects stable IDs (the 
> ones used by MAM) into stanzas you receive. Then, whenever it likes, a client 
> can tell the server that id X for contact Y has been read. The server checks 
> that this is a newer read marker than was there, to avoid race conditions, 
> and updates its state for that contact (this then gets propagated to other 
> clients). Then when coming online a client can query this state to find not 
> only the contacts with unread messages, but also the number of unread 
> messages. It can then, whenever it wants (e.g. immediately, or when a chat 
> window is opened) query either a chunk of history (e.g. last 20 messages) or 
> all unread messages for that contact (or those contacts). I think this makes 
> the client implementation vastly simpler, the server implementation isn’t 
> complex (and is simpler than the client implementation would need to be for 
> the other approach), it avoids nasty race conditions that need fancy handling 
> otherwise, and also lets a client show useful information (the unread count 
> per contact) with less exchanged data on startup.
> 
> Thoughts?
> 
> /K
> ___
> Standards mailing list
> Info: https://mail.jabber.org/mailman/listinfo/standards
> Unsubscribe: standards-unsubscr...@xmpp.org
> ___

___
Standards mailing list
Info: https://mail.jabber.org/mailman/listinfo/standards
Unsubscribe: standards-unsubscr...@xmpp.org
___