Re: [Standards] Evaluating gitlab.com as new location for XEP Editor repositories (xeps+registar)
On Sun, Jun 21, 2020 at 10:34 AM Waqas Hussain wrote: > On Tue, Jun 16, 2020 at 1:13 PM Jonas Schäfer wrote: > >> > Alternatively, if we do still want to use Docker, why not just use >> > whatever GitHub's CI is or one of the many CI solutions that can work >> > with GitHub without setting up lots of new infrastructure, repos, >> > syncing, etc? (ie. Travis, Circle CI, Drone, etc. there are tons of them >> > and many of them are free but also designed to work with GitHub) >> >> Due to the messed up permission model of GitHub, all of them (I can’t >> test >> travis because I signed up with them a long time ago, Circle CI does, >> GitLab >> CI for GitHub does, Docker Hub does for newly added repositories; Drone >> seems >> to require infrastructure we don’t have or want to maintain on the iteam >> side) >> seem to require full write access to all repositories whichever account >> is >> used to set them up has access to or will ever have access to, public and >> private. >> >> > I'd second what Sam suggested elsewhere in the thread. If the main issue > is Github's permission model (due to us using personal human accounts for > doing CI auth), we should use Github's recommended alternatives: machine > users is what they've recommended prior to Github Actions. > > See > https://developer.github.com/v3/guides/managing-deploy-keys/#machine-users > > With my security hat on, using human accounts for CI is an anti-pattern. > You /want/ a machine CI user, even if human accounts would work perfectly. > This helps fully compartmentalize CI, limits blast radius when incidents > happen, is easier when humans eventually leave the org. > > If Github Actions work for our use-cases, that might be ideal though. It's > more managed, which I'd expect to translate to less burden on iteam, and > allow easier contribution by folks not on iteam. > > I'm on the side of keeping issues and PRs on Github, that's where the > users are. Asking every contributor to create a Gitlab account seems > unfortunate, when practically every contributor already has a Github > account. > > I do appreciate the idea of supporting account-less contributions (that > Zash called out), and the historical channel for that has been the mailing > list. So that seems covered in any case. > > Thanks, > Waqas > > Oh, and I wanted to add: thanks for working on this Jonas, and everyone else on iteam. Regardless of where we land, I'm sure we all appreciate the effort being put in. ___ Standards mailing list Info: https://mail.jabber.org/mailman/listinfo/standards Unsubscribe: standards-unsubscr...@xmpp.org ___
Re: [Standards] Evaluating gitlab.com as new location for XEP Editor repositories (xeps+registar)
On Tue, Jun 16, 2020 at 1:13 PM Jonas Schäfer wrote: > > Alternatively, if we do still want to use Docker, why not just use > > whatever GitHub's CI is or one of the many CI solutions that can work > > with GitHub without setting up lots of new infrastructure, repos, > > syncing, etc? (ie. Travis, Circle CI, Drone, etc. there are tons of them > > and many of them are free but also designed to work with GitHub) > > Due to the messed up permission model of GitHub, all of them (I can’t test > travis because I signed up with them a long time ago, Circle CI does, > GitLab > CI for GitHub does, Docker Hub does for newly added repositories; Drone > seems > to require infrastructure we don’t have or want to maintain on the iteam > side) > seem to require full write access to all repositories whichever account is > used to set them up has access to or will ever have access to, public and > private. > > I'd second what Sam suggested elsewhere in the thread. If the main issue is Github's permission model (due to us using personal human accounts for doing CI auth), we should use Github's recommended alternatives: machine users is what they've recommended prior to Github Actions. See https://developer.github.com/v3/guides/managing-deploy-keys/#machine-users With my security hat on, using human accounts for CI is an anti-pattern. You /want/ a machine CI user, even if human accounts would work perfectly. This helps fully compartmentalize CI, limits blast radius when incidents happen, is easier when humans eventually leave the org. If Github Actions work for our use-cases, that might be ideal though. It's more managed, which I'd expect to translate to less burden on iteam, and allow easier contribution by folks not on iteam. I'm on the side of keeping issues and PRs on Github, that's where the users are. Asking every contributor to create a Gitlab account seems unfortunate, when practically every contributor already has a Github account. I do appreciate the idea of supporting account-less contributions (that Zash called out), and the historical channel for that has been the mailing list. So that seems covered in any case. Thanks, Waqas ___ Standards mailing list Info: https://mail.jabber.org/mailman/listinfo/standards Unsubscribe: standards-unsubscr...@xmpp.org ___
Re: [Standards] LAST CALL: XEP-0335 (JSON Containers)
On Mon, Feb 4, 2019 at 2:19 PM Jonas Schäfer wrote: > This message constitutes notice of a Last Call for comments on > XEP-0335. > > Title: JSON Containers > Abstract: > This specification defines an element to be used for encapsulating > JSON data in XMPP. > > URL: https://xmpp.org/extensions/xep-0335.html > > This Last Call begins today and shall end at the close of business on > 2019-02-19. > > Please consider the following questions during this Last Call and send > your feedback to the standards@xmpp.org discussion list: > > 1. Is this specification needed to fill gaps in the XMPP protocol > stack or to clarify an existing protocol? > > 2. Does the specification solve the problem stated in the introduction > and requirements? > > 3. Do you plan to implement this specification in your code? If not, > why not? > > 4. Do you have any security concerns related to this specification? > > 5. Is the specification accurate and clearly written? > > Your feedback is appreciated! > ___ > Standards mailing list > Info: https://mail.jabber.org/mailman/listinfo/standards > Unsubscribe: standards-unsubscr...@xmpp.org > ___ > It would be good if the specification noted the incompatibility between JSON and XML definitions of what a character is, and certain characters MUST be escaped be escaped via JSON escaping to be valid XML. In XML 1.0: > Char::=#x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | > [#x1-#x10] > /* any Unicode character, excluding the surrogate blocks, FFFE, and . > */ In JSON (RFC 7159): > char = unescaped / > escape ( > %x22 / ; "quotation mark U+0022 > %x5C / ; \reverse solidus U+005C > %x2F / ; /solidus U+002F > %x62 / ; bbackspace U+0008 > %x66 / ; fform feed U+000C > %x6E / ; nline feed U+000A > %x72 / ; rcarriage return U+000D > %x74 / ; ttab U+0009 > %x75 4HEXDIG ) ; uU+ > escape = %x5C ; \ > quotation-mark = %x22 ; " > unescaped = %x20-21 / %x23-5B / %x5D-10 So all control characters, surrogates and similar MUST be escaped via \X or \u escapes in XML strings to allow lossless and valid passthrough of all possible JSON content. I'd mainly like this noted because a lot of implementors only think of ASCII and normal characters when implementing such things, and edge case bugs tend to sneak into the wild and cause compat issues. Thanks, Waqas ___ Standards mailing list Info: https://mail.jabber.org/mailman/listinfo/standards Unsubscribe: standards-unsubscr...@xmpp.org ___
Re: [Standards] Nonzas: What are they and do we need them?
On Thu, Apr 23, 2015 at 3:52 AM, Kim Alvefur wrote: > tors apr 23 02:35:24 2015 GMT+0200 skrev Waqas Hussain: > > On Wed, Apr 22, 2015 at 3:41 AM, Florian Schmaus > wrote: > > > > > The discussion drifted a bit into whether non-stanza top-level stream > > > elements should be used for a particular use case/XEP > > > or not. But what I really wanted to discuss is whether they could be > > > used after resource binding in general, or if this should be > disallowed. > > > That's why I asked the council members to express their opinion on this > > > in their next meeting. > > > > > > As side note: I still think it is advantageous to have a unambiguous > > > term defined for non-stanza top-level stream elements. It would clearly > > > distinguish stanzas from non-stanzas in specifications and may help to > > > avoid the case where specification authors erroneously refer to > > > non-stanzas as stanzas. See for example the EXI XEP (XEP-322) where > this > > > is done (nearly?) everywhere. Not to mention that this may cause > > > confusion when we take XEP-198: Stream Management into consideration. > > > > > > - Florian > > > > > > > > Some thoughts: In the Prosody XMPP server implementation, we classify > > top-level elements into two categories: stanzas, and non-stanzas > (nonzas!). > > We call non-stanzas 'elements'. Interestingly, stanzas sent before a bind > > are categorized with non-stanzas, given how different they are from > normal > > stanzas (several normally expected invariants don't hold for them, e.g., > no > > reliable 'from' attribute). The bind stanza is special, and is almost a > > third category (it awkwardly exists in this space between having a > username > > and not having a resource). > > > > These three categories require different sets of validation. > > > > Normally we expect non-stanzas to be purely affecting the state of a > > specific stream, and they don't have any affect beyond that. Stanzas > > typically do not affect the stream itself. The exceptions make code > > awkward, and the main (only?) one is the bind IQ (which we are forced to > > special case). > > I would like to point out that dialback elements are also in an awkward > space, being treated as stanzas when sending (being routed etc) and nonzas > when receiving. > I'll add this to your point: bind is more awkward than dialback. While dialback is awkward, it can relatively easily be isolated in the s2s and dialback plugins in Prosody. Bind awareness however needs to be in the core stanza router. In the XMPP layering violation contest, my vote goes to bind :) Inspecting stanza routers of different servers is actually quite fascinating. You get to see all the more awkward parts of XMPP core. -- Waqas Hussain
Re: [Standards] Nonzas: What are they and do we need them?
On Wed, Apr 22, 2015 at 3:41 AM, Florian Schmaus wrote: > The discussion drifted a bit into whether non-stanza top-level stream > elements should be used for a particular use case/XEP > or not. But what I really wanted to discuss is whether they could be > used after resource binding in general, or if this should be disallowed. > That's why I asked the council members to express their opinion on this > in their next meeting. > > As side note: I still think it is advantageous to have a unambiguous > term defined for non-stanza top-level stream elements. It would clearly > distinguish stanzas from non-stanzas in specifications and may help to > avoid the case where specification authors erroneously refer to > non-stanzas as stanzas. See for example the EXI XEP (XEP-322) where this > is done (nearly?) everywhere. Not to mention that this may cause > confusion when we take XEP-198: Stream Management into consideration. > > - Florian > > Some thoughts: In the Prosody XMPP server implementation, we classify top-level elements into two categories: stanzas, and non-stanzas (nonzas!). We call non-stanzas 'elements'. Interestingly, stanzas sent before a bind are categorized with non-stanzas, given how different they are from normal stanzas (several normally expected invariants don't hold for them, e.g., no reliable 'from' attribute). The bind stanza is special, and is almost a third category (it awkwardly exists in this space between having a username and not having a resource). These three categories require different sets of validation. Normally we expect non-stanzas to be purely affecting the state of a specific stream, and they don't have any affect beyond that. Stanzas typically do not affect the stream itself. The exceptions make code awkward, and the main (only?) one is the bind IQ (which we are forced to special case). This all has actual architectural implications: stream features can be consumed by XMPP proxies (BOSH/websocket components, TLS unwrapping proxies, other edge server scenarios), while non-stanzas are passed through to the XMPP core/router. The idea of XMPP reverse proxies has very interesting deployment implications, and something we have experimented with. -- Waqas Hussain
Re: [Standards] Nonzas: What are they and do we need them?
On Mon, Apr 20, 2015 at 2:45 PM, Florian Schmaus wrote: > On 20.04.2015 18:22, Christian Schudt wrote: > > > >> For me personally, the contra-Nonza arguments did not convince me. It > >> appears that nothing in the specification prevents you from using Nonzas > >> after resource binding with BOSH. XEP-206 3. only says "SHOULD contain". > >> I also don't see why they would introduce "a bunch of conceptual and > >> implementations problems". > > > > I agree with you. The contra arguments are weak. I think Prosody even > advertises XEP-0198 over BOSH, so clients would use it. And I also see no > implementation problems. > > Not sure what prosody is doing and it's slightly unrelated to the Nonza > discussion, but I want to point out that it makes no sense to use xep198 > SM over BOSH. Ignoring Prosody for a moment, why do you think 198 on BOSH is nonsensical? BOSH is a low level protocol, and the stream abstraction still exists. XEP-0198 operates over the stream abstraction. Whether it's TCP, BOSH, WebSocket or something else under the stream, 198 can continue operating. It certainly isn't (and shouldn't be) disallowed. Use-case: you can in theory start a BOSH stream, and use 198 to upgrade to a WebSocket stream later. It's an interesting idea, where you can swap the low level transport based on changing network conditions without interrupting the user's session. I'm not sure if this works in Prosody (if it doesn't, then advertising it is a bug). -- Waqas Hussain
Re: [Standards] RFC: XEP-0045 MUC should not rewrite message IDs
On Fri, Jul 25, 2014 at 10:06 AM, Georg Lukas wrote: > Hi, > > XEP-0045, section 7.4 <http://xmpp.org/extensions/xep-0045.html#message> > states: > > "Note well that for tracking purposes this service assigns a new 'id' to > each message it generates (here using a UUID as defined in RFC 4122 [18])." > Prosody's MUC service doesn't do that. Broadcasted messages have the same 'id' as the original (including a missing 'id' attribute). > I suggest changing that line to: > > "The service SHOULD reflect the message with the same 'id' that was > generated by the client. If the client did not provide an 'id', the > server SHOULD generate one 'id' and use it for all reflections of the > same message (e.g. using a UUID as defined in RFC 4122 [18])." > I object to the second SHOULD. Comments on the rest of the message follow. > (The example XML above should be changed accordingly; we might even > replace SHOULD with MUST after the major implementations have changed) > > Rationale: > > A client needs some way to track if a sent message was succesfully > processed and reflected. This is required to indicate success of > transmission, to allow for message correction (XEP-0308), and it helps > in displaying message errors in the context of the outgoing message. > > To determine if a newly received MUC message was indeed the reflection > of the last outgoing MUC message, in theory three elements can be used: > from, id, and body. > > * from: the message 'from' attribute must be the participant's >nickname, however this will also be the case if the participant >is connected with two clients sharing the same nickname, and the >other client sends a MUC message. > > * id: the message 'id' attribute can only be relied upon if the server >does not rewrite it > > * body: the message body could be used to check if an incoming message >is the reflection of the just-sent message. However, some services >rewrite the body, i.e. to replace a large message with a pastebin >link. > > Therefore, none of the three elements can be reliably used to identify > if an incoming MUC message is the reflection of the last outgoing > message. To improve the situation, I suggest changing the XEP in a way > that MUC message reflections keep the original message 'id'. > > Implications: > > This change might violate the uniqueness guarantee of message 'id's, > however as the current behavior is not a MUST, service implementations > exist that do not touch the message 'id', and the world is not falling > apart. > I'll note that there is no uniqueness guarantee in message 'id's. It's entirely up to the sender. I can happily make my client send id="waqas" for all messages forever, and that does not violate the spec. Given a lack of uniqueness guarantees, a server should either be generating IDs for all messages, or it should be generating them for none. Mixed-mode ID generation doesn't make a lot of sense. Using 'id' to match messages doesn't work well. Security issue: I can notice the 'id' on your messages, and can happily send messages with that 'id' to confuse your client. You don't own the value of the 'id'. If you are using this to sync history on reconnect, I can even temporarily take over your nick while you are disconnected, leaving your client with nothing to detect the sync marker being invalid. Given the current text of the XEP, even if this change is accepted, you may not be compatible with existing servers out there, and there's no way to reliably check for compatibility until after you've sent some messages. History sync is one area of XMPP that's terribly broken. All the popular XMPP protocols involving history (MUC, offline messages, PubSub, etc) have large gaps and race conditions, making reliable sync impossible. Heck many of the specs only work on the assumption of synchronised clocks. This is a separate email though. -- Waqas Hussain
Re: [Standards] server IP check (XEP-0279)
On Wed, Apr 16, 2014 at 7:56 PM, Peter Saint-Andre wrote: > XEP-0279 defines a simple XMPP extension that enables a client to discover > its external IP address from its server. This feature can help a client > discover if it's behind a NAT and thus trigger things like STUN lookups. > I'd like to ask the XMPP Council to issue a Last Call on XEP-0279, but I'd > also like to know if there any implementations of this extension in clients > or servers. > > Thanks! > > Peter > > -- > Peter Saint-Andre > https://stpeter.im/ > I recall implementing it for Prosody in a few minutes the day the XEP was first published: https://code.google.com/p/prosody-modules/source/browse/mod_ipcheck/mod_ipcheck.lua I'm aware of one private BOSH application that uses this. -- Waqas Hussain
Re: [Standards] XEP-0138: security considerations
On Tue, Apr 8, 2014 at 7:18 PM, Peter Saint-Andre wrote: > Before we released the security note about application-layer compression > last week [1] (which now seems to have been overshadowed by the heartbleed > bug in OpenSSL), I started to work on some updates to XEP-0138. Here is my > proposed text for the Security Considerations section: > > ### > > 7. Security Considerations > > Stream encryption via TLS (as defined in RFC 6120) and stream compression > (as defined herein) are not mutually exclusive. However, if both are used > then TLS-layer encryption MUST be negotiated before negotiation of > application-layer compression, so that the stream is secured first. > > Many of the security considerations related to TLS compression (see > Section 6 of RFC 3749) also apply to stream compression. > > Several attacks against TLS-layer and application-layer compression have > been found, including the CRIME and BEAST attacks (see > draft-sheffer-uta-tls-attacks [7]). Mitigation for the CRIME attack > involves disabling TLS-layer compression. At the time of this writing > (early 2014), there are no general mitigations against the BEAST attack. > However, the following safeguards are appropriate: > > 1. A server implementation MUST NOT turn on compression by default; > instead, it MUST enable a server administrator to turn on compression if > desired. > 2. A server implementation MUST enable a server administrator to limit > the size of stanzas it will accept from a connected client or peer server, > and also MUST set a reasonable default limit of at least 10,000 bytes. In > general, it is reasonable for the maximum stanza size to be the same as the > size of the buffer passed to zlib when storing uncompressed data. > 3. A server implementation MUST enable a server administrator to limit > the amount of bandwidth it will allow a connected client or peer server to > use in a given time period. > > The last two requirements are consistent with but stronger than > recommendations provided in Section 13.2 of RFC 6120. Failure to provide > such protections opens an implementation denial of service attacks. > > ### > > Your feedback is welcome. > > (I have cc'd Giancarlo Pellegrino, who reported the "xmppbomb" > vulnerability; please reply to all so that he can be included in the > conversation.) > > Thanks! > > Peter > > [1] http://xmpp.org/resources/security-notices/uncontrolled- > resource-consumption-with-highly-compressed-xmpp-stanzas/ > I have a few minor disagreements with this. The spec is assuming implementation details here, and is enforcing based on that (with MUST-level statements). Consider CRIME, BEAST and similar attacks: 1. The spec is about DoS attacks, which these are not 2. 'MUST NOT enable compression by default' doesn't do anything useful in particular here, unless we also add "MUST educate admin on BEAST and CRIME before allowing them to enable". In general, admins frequently go through server configs enabling things which make sense. Compression makes sense to most of them. Whether they should or shouldn't do this is a separate question, but that's what a /lot/ of new admins do. 3. We are kind of assuming that because clients can't be trusted to get this sort of thing right, servers MUST do this. Why exactly is this not a recommendation for clients as well? Consider xmppbomb and other flooding/DoS attacks 1. A server doing anything interesting (e.g., smart dynamic limits based on currently available resources) shouldn't be disallowed from using resources that are available and unused. If a server has 100GB of free RAM, no CPU usage, a client sends a 100MB gzipped payload, which expands into a 1GB stanza, and is directed to e.g., the client itself, the server should be allowed to accept it if it deems it reasonable. A better thing to do is to require mitigation of attacks, but only make suggestions on how to do it. We shouldn't require specific ways of doing it, not with a MUST. Specs shouldn't dictate implementation details. 2. To protect clients, a better limit would be on what the server sends on behalf of users, not on what the server receives, with the goal of protecting other clients (remember how conference.jabber.org's disco#items response used to be too large, and got dropped by a lot of federated servers?) 3. One interesting case where limits limited a useful stanza was the conference.jabber.org disco#items response -- Waqas Hussain
Re: [Standards] Need help with problems regarding XEP-0114.
On Mon, Apr 7, 2014 at 1:30 PM, Peter Waher wrote: > Hello > > > > I've now examined the possibility to use Jabber Components (XEP-0114) for > IoT Discovery and Provisioning, but have some problems. Since it's very > hard to find documentation on this XEP, and searching the internet for > solutions only returns pages with people having the same problems (but no > solutions), I'm turning to the XSF community to see if somebody has any > good input. > > > > First, I can mention that I can send and receive messages and iq stanzas > to and from normal JIDs and a component, as long as I have the resource of > the corresponding JID I want to communicate with. This is no problem, if > the client itself initiates communication. But what if the server component > needs to initiate communication? How does the server component obtain the > resource(s) of a corresponding bare JID? > > > > I've tried a presence probe. That only returns a not-authorized response. > The reason being that otherwise, it would be a security problem, if the > component was able to probe the presence of a JID, without being a friend. > (However, sending messages and iq stanzas to the JID, if you had the > resource, was OK, even without friendship.) > > > > The next thing I tried was to send a presence subscription to the JID from > the component. This request was not received by the client, regardless if I > tried with sending it from the component itself, or a virtual JID simulated > by the component. Searching the internet, this was the response I was > given: You needed to simulate accounts behind the component (using the same > component domain name). > > > > Is there a step that needs to be taken before sending the presence > subscription, for it to be routed correctly to the device? > > > > Or is presence subscriptions somehow disabled in server components? > > > > Or might this behave differently on different XMPP servers? (I'm using an > OpenFire XMPP server to test my implementation.) > > > > Example of communication that goes well. Here the component, given the > full JID of a client, is able to perform an IQ get & receive a response: > > > > C: type='get' id='1'>http://jabber.org/protocol/disco#info > "/> > > S: type="result" id="1"> xmlns="http://jabber.org/protocol/disco#info";> category="client" type="pc" name="Clayster"/>http://jabber.org/protocol/disco#info"/>http://jabber.org/protocol/disco#items"/> var="jabber:iq:roster"/> var="vcard-temp"/> > > > > While, on the client side, it looks as follows: > > > > S: type="get" id="1">http://jabber.org/protocol/disco#info > "/> > > C: type='result' id='1'> xmlns="http://jabber.org/protocol/disco#info";> category="client" type="pc" name="Clayster"/>http://jabber.org/protocol/disco#info"/>http://jabber.org/protocol/disco#items"/> var="jabber:iq:roster"/> var="vcard-temp"/> > > > > > > This is how the failed attempt looks like, where the component itself > makes a presence probe to retrieve the resource of a bare JID. When it > fails it tries to request a subscription on presence. > > > > C: to="unitt...@clayster.cl" type="probe"/> > > S: from="unitt...@clayster.cl"> code="401" type="auth"> xmlns="urn:ietf:params:xml:ns:xmpp-stanzas"/> > > C: to="unitt...@clayster.cl" type="subscribe"/> > > > > Nothing is received on the client side. I try the same thing, but with a > simulated account behind the component domain: > > > > C: > > S: xmlns="urn:ietf:params:xml:ns:xmpp-stanzas"/> > > C: > > > > Nothing is received by the client. > > > > Hopefully, somebody with more insight can answer the above questions, or > give other insights on how to proceed. If I can initiate communication from > a server component, it would be a very good option to use in the IoT > Discovery XEP proposal and the Provisioning XEP. > > > > Best regards, > > Peter Waher > > > Your component can send messages to the bare JID of the user, which would then be sent to the resources with highest priority (depending on server config, etc). That's the normal way to initiate a conversation with a bare JID when you don't have a full JID available. Regarding presence subscriptions, is your component doing any roster handling? It has to send a subscription request to the user, and receive the user's subscription acceptance, optionally storing this information (as the component's roster), after which it can probe the user (assuming the user accepted). The XMPP server does not manage rosters or subscriptions for XEP-0114 components. It only does so for users. The server mainly handles XMPP Core (RFC 6120) for components, while everything in XMPP IM (RFC 6121) is up to the components themselves. Prosody plug: I've switched to writing components as Prosody plugins, so I can reuse a lot of Prosody's c2s machinery, and I wrote mod_component_client to then allow Prosody to act as a XEP-0114 component to other servers. -- Waqas Hussain
Re: [Standards] XEP-0045: Multi-User Chat: Question/problem with modifying member list (e.g. in offline case)
On Tue, Mar 25, 2014 at 9:54 AM, Christian Schudt wrote: > Dear XMPP community, > > I recently had the following requirement for a Multi-User Chat implementation: > > User A (owner) and User B are in a members-only room. User A modifies the > member list according to "9.3 and 9.5 Modifying the Member List" and adds > another User C to the member list and also sends an invitation to User C. > > The room shall then inform User B about the new member / affiliation change. > But it doesn't, because User C is not available in the room yet (he might be > offline or currently doesn't wish to participate in a discussion). > > 9.3 Granting Membership (Example 123) says: > > "If the user is in the room, the service MUST then send updated presence from > this individual to all occupants, indicating the granting of membership..." > > So, why is there the "If the user is in the room" clause? Unfortunately with > that, the above requirement can't be met. > > What do you think of the possibility to send the following stanza for this > use case (with type="unavailable")? > > type='unavailable' > from='co...@chat.shakespeare.lit/thirdwitch' > to='cro...@shakespeare.lit/desktop'> > >jid='ha...@shakespeare.lit/pda' > role='participant' > nick='thirdwitch'/> > > > > (same is true for other affiliation changes, if the user, whose affiliation > has changed is not in the room). > > Kind regards, > Christian So you essentially want occupants to be informed about affiliation changes of users not in the room. MUC is inconsistent here. For affiliation changes involving admins or owners, an implementation MAY broadcast a message to occupants (see XEP-0045, examples 176 and 190). Example 190 has a typo, 'member' should be 'owner'. These messages are however limited to admin and owner affiliations, not members. And they are also optional for implementations. Your suggestion of using presence for this doesn't fit the MUC protocol very well. Instead it would be useful to update the XEP to be consistent about the messages it already sends (send for all affiliations, not select ones), while allowing implementations freedom in what they send (it should remain a MAY, and deployments should be free to limit broadcasts to e.g., moderators only). -- Waqas Hussain
Re: [Standards] eventlogging xeps
On Mon, Dec 9, 2013 at 4:04 PM, Waqas Hussain wrote: > On Mon, Dec 2, 2013 at 9:58 AM, Peter Waher wrote: >> Hello Philipp >> >> Thanks for your input. I've updated the according to your comments, and >> attach a new version. >> >> Best regards, >> Peter Waher >> > > Hey Peter, > > Something which I don't think has been discussed: Encoding. I'm > assuming syslog-to-xmpp bridging is something you wish to enable. > Syslog does recommend UTF-8, but doesn't require it. Data can be > non-UTF-8. Also, syslog messages can be truncated by bytes, which can > lead to invalid UTF-8 at the end of a message (the spec specifically > points this out). And lastly, syslog messages may contain a > byte-order-mark (BOM) at the start of messages to specify encoding. > Also, XML supports a subset of UTF-8, not all UTF-8 characters are > allowed. > > What we probably want is some form of escaping to be specified in the > spec. Perhaps as simple as: replace \ with \\ and replace any invalid > XML bytes with \XX, XX being the byte in hex. BOMs should be removed > if available, and data should be converted to UTF-8 if possible. > > One thing which looks awkward to me are the 'type' attributes in the > 'tag' element. What's a use-case where specifying the data-type would > help the receiver (particularly when this would generally be the same > in every log message sent by the entity)? And while you are using > 'xs:*', the 'xs' namespace isn't defined in the XML stream the stanza > is in. > > -- > Waqas Hussain Lance and I just had a little chat about the xs: prefix in the Prosody chatroom (here's hoping the formatting comes out nicely): Lance 4:07 waqas: is it required to have a namespace declared when using a prefixed attribute *value*? I thought those values were literally xs:*, just a hack to make unique registry values easier waqas 4:08 Lance: Required? Arguably. Look at everywhere you see xs:* being used. Schemas, SOAP, etc. It's always specified, and interpreted in that context. 4:08 We could certainly define it to always be xs:* in the spec 4:08 But that's not normal Lance 4:09 because it looks like the same pattern as 122 4:09 !xep 0122 HAL 4:09 Lance: XEP-0122: Data Forms Validation is Standards Track (Draft, 2004-09-22) See: http://xmpp.org/extensions/xep-0122.html waqas 4:10 Ah, that XEP specifically defines it as "•Start with "xs:", and be one of the "built-in" datatypes defined in XML Schema Part 2 [2]" Lance 4:10 right, with a method to add new prefixes too waqas 4:11 (which is unusual mind you, you have full freedom to use whatever suffix you want in SOAP and XML Schema) 4:11 *prefix waqas 4:11 Lance: Should I post this little chat to the list? Lance 4:11 yeah Lance 4:12 i'd say we'd make the eventlogging stuff reuse the types from 122 4:12 ah, which it does already 4:12 section 5.9 of whichever version this is that i'm looking at waqas 4:12 That makes sense
Re: [Standards] eventlogging xeps
On Mon, Dec 2, 2013 at 9:58 AM, Peter Waher wrote: > Hello Philipp > > Thanks for your input. I've updated the according to your comments, and > attach a new version. > > Best regards, > Peter Waher > Hey Peter, Something which I don't think has been discussed: Encoding. I'm assuming syslog-to-xmpp bridging is something you wish to enable. Syslog does recommend UTF-8, but doesn't require it. Data can be non-UTF-8. Also, syslog messages can be truncated by bytes, which can lead to invalid UTF-8 at the end of a message (the spec specifically points this out). And lastly, syslog messages may contain a byte-order-mark (BOM) at the start of messages to specify encoding. Also, XML supports a subset of UTF-8, not all UTF-8 characters are allowed. What we probably want is some form of escaping to be specified in the spec. Perhaps as simple as: replace \ with \\ and replace any invalid XML bytes with \XX, XX being the byte in hex. BOMs should be removed if available, and data should be converted to UTF-8 if possible. One thing which looks awkward to me are the 'type' attributes in the 'tag' element. What's a use-case where specifying the data-type would help the receiver (particularly when this would generally be the same in every log message sent by the entity)? And while you are using 'xs:*', the 'xs' namespace isn't defined in the XML stream the stanza is in. -- Waqas Hussain
Re: [Standards] Proposed XMPP Extension: JSON Containers
On Sat, Oct 12, 2013 at 6:31 PM, Dave Cridland wrote: > > On 12 Oct 2013 17:17, "Waqas Hussain" wrote: >> As a general rule, if you embed one textual language inside another, >> escaping is pretty much mandatory. If you see a way around it, you are >> probably mistaken and missing the edge cases. > > MIME. > > Do I win a prize? > Would getting to write a MIME over XMPP spec be a suitable prize? But wait, MIME has escape sequences (and also, what MIME exactly? there's so much to choose from in there...). :) If you have a structured language with delimiters, and the content inside those delimiters can contain the delimiters themselves, you need escaping (as we discovered in the entity caps preimage issue). Ways around that include length-encoding (which I'd say is very non-friendly to humans), or EOF as end delimiter (which greatly limits your structure). -- Waqas
Re: [Standards] Proposed XMPP Extension: JSON Containers
On Fri, Oct 11, 2013 at 9:22 PM, Field Tian (fitian) wrote: > Yea, cdata is a traditional way in XML. XMPP seems not prefer CDATA. I don’t > know what's the detailed reason. But I think CDATA has its advantage > sometimes. There is generally this assumption that CDATA would allow adding JSON to XML without any further escaping. This isn't true. XML 1.0 defines: CData ::= (Char* - (Char* ']]>' Char*)) So JSON containing "]]>" inside cannot be added to XML without escaping in some way. Also, XML 1.0 defines valid characters (all characters, in PCDATA, CDATA, even escaped via entities) as: Char ::= #x9 | #xA | #xD | [#x20-#xD7FF] | [#xE000-#xFFFD] | [#x1-#x10]/* any Unicode character, excluding the surrogate blocks, FFFE, and . */ While JSON (RFC 4627) defines unescaped characters as: unescaped = %x20-21 / %x23-5B / %x5D-10 Which are not the same. JSON can contain unescaped characters that are not allowed in a CDATA section. So you have to escape those anyway. As a general rule, if you embed one textual language inside another, escaping is pretty much mandatory. If you see a way around it, you are probably mistaken and missing the edge cases. -- Waqas Hussain
Re: [Standards] XEP-280 and MUC private chats
On Wed, Sep 25, 2013 at 4:47 PM, Matt Miller wrote: > > On Sep 25, 2013, at 2:24 PM, Waqas Hussain wrote: > >> On Wed, Sep 25, 2013 at 4:00 PM, Kim Alvefur wrote: >>> Hello, >>> >>> On 2013-06-09 20:40, Mathieu Pasquet wrote: >>>> I was starting to implement carbons in poezio when I came across some >>>> kind of design issue that I haven’t been able to work out. >>>> >>>> However, in the case of private MUC messages (XEP-0045, 7.5), the >>>> messages are also of type 'chat', causing them to be forwarded as normal >>>> chat messages. But the other resources are not necessarily present on >>>> that MUC, so they will receive the messages just fine, as with any >>>> direct conversation with a fulljid, but they won’t be able to reply, >>>> because I believe most MUC implementations will check the fulljid and >>>> reply with an error. >>> >>> I would like to propose that the MUC server shall stamp something like >>> on private messages >>> passing through, similar to what is already done to . >>> >>> This would allow a Carbons implementation to treat those differently, eg >>> by not copying them to Carbons-interested resources, or a client >>> receiving a carbon copied message can indicate to the user that replying >>> won't be possible. >>> >>> Aside: I also think this would be useful for message archives, to help >>> not record duplicated PMs if the user has joined a room from multiple >>> clients. >>> >> >> Ignoring the protocol for a bit, what do users want? There's a lot of >> user interest in having the same chats available across devices. I >> would assume users would want the same for chatrooms and MUC PMs… >> >> We also have mutli-party chats where a one-to-one chat is upgraded to >> a MUC [1]. Would that mean users are happily chatting across multiple >> devices, they add a person to the chat and are suddenly stuck to one >> device? >> >> I think MUC is just incompatible with carbons with no clean way to >> combine them, however a carbons-like XEP for MUC would be highly >> desirable. Coincidentally, this was a problem I was attempting to >> solve today for a service I'm working on: Multiple clients (BOSH >> clients on multiple tabs, devices, native application) which needed to >> have the same chatrooms open (It's like [1], but starting out with MUC >> from the start). I'm starting out with broadcasts on join/leave from >> the MUC service to users' resources, which may or may not be the ideal >> design. >> >> [1] http://xmpp.org/extensions/xep-0045.html#continue > > > I think one could do achieve carbons-like behavior in MUC today if the > service supported multiple joins (say, different resourceparts under the same > bare JID) to the same nickname. It looks like most (if not all) of the > protocol bits are already in XEP-0045 to allow for that. Such behavior is > underspecified, and that could be addressed by an informational XEP. > Prosody does support that. The missing pieces is that new resources need to discover which chatrooms the other resources are in, and when a resource joins or leaves a room, the others need to be notified. There is no standard mechanism for this, and leaving the up to the clients (as opposed to the MUC host) leads to various race conditions that I'm thinking through. Another missing piece was figuring out which nicks in a chatroom were owned by you. Luckily Prosody's MUC always shows you your own real JID even if you are not an admin. Multiple sessions behind a nick are still invisible (Prosody's implementation currently completely hides other nick-sessions from the client behind the same nick, for compat reasons). -- Waqas Hussain
Re: [Standards] XEP-280 and MUC private chats
On Wed, Sep 25, 2013 at 4:00 PM, Kim Alvefur wrote: > Hello, > > On 2013-06-09 20:40, Mathieu Pasquet wrote: >> I was starting to implement carbons in poezio when I came across some >> kind of design issue that I haven’t been able to work out. >> >> However, in the case of private MUC messages (XEP-0045, 7.5), the >> messages are also of type 'chat', causing them to be forwarded as normal >> chat messages. But the other resources are not necessarily present on >> that MUC, so they will receive the messages just fine, as with any >> direct conversation with a fulljid, but they won’t be able to reply, >> because I believe most MUC implementations will check the fulljid and >> reply with an error. > > I would like to propose that the MUC server shall stamp something like > on private messages > passing through, similar to what is already done to . > > This would allow a Carbons implementation to treat those differently, eg > by not copying them to Carbons-interested resources, or a client > receiving a carbon copied message can indicate to the user that replying > won't be possible. > > Aside: I also think this would be useful for message archives, to help > not record duplicated PMs if the user has joined a room from multiple > clients. > Ignoring the protocol for a bit, what do users want? There's a lot of user interest in having the same chats available across devices. I would assume users would want the same for chatrooms and MUC PMs… We also have mutli-party chats where a one-to-one chat is upgraded to a MUC [1]. Would that mean users are happily chatting across multiple devices, they add a person to the chat and are suddenly stuck to one device? I think MUC is just incompatible with carbons with no clean way to combine them, however a carbons-like XEP for MUC would be highly desirable. Coincidentally, this was a problem I was attempting to solve today for a service I'm working on: Multiple clients (BOSH clients on multiple tabs, devices, native application) which needed to have the same chatrooms open (It's like [1], but starting out with MUC from the start). I'm starting out with broadcasts on join/leave from the MUC service to users' resources, which may or may not be the ideal design. [1] http://xmpp.org/extensions/xep-0045.html#continue
Re: [Standards] Multiple elements in a HTTP request with BOSH
Nope, that would be a malformed XML document. What are you trying to do? On Fri, Dec 7, 2012 at 4:01 AM, Jaussoin Timothée wrote: > Hye everyone, > > All the question is in the title of my message. I've read that BOSH allow > to add more than one stanza per element to save bandwidth during > the session. > Am I authorized to put multiple elements in the same HTTP BOSH > request ? > > Tim >
Re: [Standards] Gateway roster management
On Mon, Feb 20, 2012 at 1:18 PM, Sergey Dobrov wrote: > On 02/20/2012 02:56 AM, Waqas Hussain wrote: >> >> Have you looked at http://jkaluza.fedorapeople.org/remote-roster.html ? >> >> Prosody and Spectrum have implementations for sections 2.3, 2.4 and >> 2.5, and it works fine. The Prosody implementation is >> mod_remote_roster, and allows trusted hosts (currently local >> components) to edit roster items for that host (mostly normal >> jabber:iq:roster queries). This lets Spectrum cleanly manage the >> user's transport contacts. >> >> http://code.google.com/p/prosody-modules/source/browse/mod_remote_roster/mod_remote_roster.lua > > Seems legit for me, thanks. Unfortunately, I can't use prosody, when > it's expected to be accepted as an experimental XEP so I can ask to > implement that to ejabberd? > Looks like there's a patch available for ejabberd-2.1.5 as well: http://spectrum.im/projects/spectrum/wiki/Remote_roster#Ejabberd-21x -- Waqas Hussain
Re: [Standards] Gateway roster management
On Fri, Feb 17, 2012 at 1:11 AM, Sergey Dobrov wrote: > Hello, > > My gateways need to use "from", "to" and "none" subscription states, not > only both but I don't see the way to make it possible gateway to pass > them from gateway to a client: XEP-144 doesn't handle subscription > states, plain subscriptions mechanism doesn't cover all of them too. > > Maybe it's reasonable to make it possible to interact with gateways with > a "jabber:iq:roster" namespace? It can be done on the client side or the > server side: > > 1. Client side: client will ask each gateway for a roster, then it will > receive all items and get updates as usually. Benefit is that client can > divide rosters from different sources easily. > 2. Server side: server can ask gateways and merge them rosters into the > general one. The benefit is in unnecessary of clients completions. > > Both these options have benefit that it will be easier to move across > gateways. For now, it's really problematic to move from the one gateway > to another one. > > -- > With best regards, > Sergey Dobrov, > XMPP Developer and JRuDevels.org founder. > Have you looked at http://jkaluza.fedorapeople.org/remote-roster.html ? Prosody and Spectrum have implementations for sections 2.3, 2.4 and 2.5, and it works fine. The Prosody implementation is mod_remote_roster, and allows trusted hosts (currently local components) to edit roster items for that host (mostly normal jabber:iq:roster queries). This lets Spectrum cleanly manage the user's transport contacts. http://code.google.com/p/prosody-modules/source/browse/mod_remote_roster/mod_remote_roster.lua -- Waqas Hussain
Re: [Standards] XEP-0045 1.25rc10 comments
On Wed, Jan 25, 2012 at 2:26 AM, Peter Saint-Andre wrote: > On 1/24/12 11:14 AM, Peter Saint-Andre wrote: > > After jabbering with Kev for a bit, here's a follow-up on the status > code 210 issue (originally raised by Waqas Hussain). > >>>>> "If the user's nickname is modified by the service as a result of >>>>> registration and the user is in the room, the service SHOULD include >>>>> status code "210" in the updated presence notification that it sends >>>>> to all users." - this is new, I think, couldn't it break things? *** >>>> >>>> In what way does that break things? >>> >>> Prior to this change, 210 could only be received on 'your own' >>> stanzas, so it's been reasonable for clients/libs to assume any time >>> it sees 210 it's receiving its own stanza (and, given that servers >>> tend to only send one status code at a time (to work around buggy >>> clients), this is probably a sensible thing to do). If clients start >>> receiving 210 from other people, I think it's entirely likely that >>> things will break. >> >> But we have a separate status code (110) for self-presence. > > First, I found the original message from Waqas: > > http://mail.jabber.org/pipermail/standards/2011-September/025183.html > > He wrote: > > ### > > 3. Service changing room nick > > I'd like some text stating that a service can change the occupant's > nick at any time, including room join. An occupant MUST listen for > status code=100 in available presence for their current nick. > > ### > > I think Waqas meant that the client needs to listen for status code > "110" (self-presence) plus "210" (nick changed) but I'm not sure. Waqas, > please confirm. > > Via IM, Kev pointed out that this should be for self-presence only. I > think he's right about that, so one of the paragraphs currently in the > working version is wrong (right after Example 76): > > If the user's nickname is modified by the service as a result of > registration and the user is in the room, the service SHOULD include > status code "210" in the updated presence notification that it sends > to all users. > > I think the other instances are correct (after Examples 23 and 50 > respectively): > > If the user has connected using a MUC client (as indicated on > joining the room by inclusino of the MUC extension), then the > service MUST include a status code of "210" in the presence > broadcast that it sends to the new occupant. > > and: > > If the service modifies the user's nickname in accordance with local > service policies, it MUST include a MUC status code of 210 in the > presence stanza sent to the user. An example follows (here the > service changes the nickname to all lowercase). > > See Example 51: > > Example 51. Occupant Changes Nickname, Modified by Service > > from='ha...@shakespeare.lit/pda' > id='nx6z2v5' > to='co...@chat.shakespeare.lit/OldHag'/> > > from='co...@chat.shakespeare.lit/oldhag' > id='D0E2B666-3373-42C9-B726-D52C40A48383' > to='ha...@shakespeare.lit/pda'> > > jid='ha...@shakespeare.lit/pda' > role='participant'/> > > > > > > So I propose that we fix the text after Example 76: > > OLD > If the user's nickname is modified by the service as a result of > registration and the user is in the room, the service SHOULD include > status code "210" in the updated presence notification that it sends > to all users. > > NEW > If the user's nickname is modified by the service as a result of > registration and the user is in the room, the service SHOULD include > status code "210" in the updated presence notification that it sends > to the user. > > Now, Kev raised another issue, which is that some clients don't properly > handle presence updates with more than one status code (e.g., they might > read only the first status code). My reply to that is: fix your client > or file a bug report. > > Peter > > -- > Peter Saint-Andre > https://stpeter.im/ > > Yes, I did mean 110, not 100. There are three cases in which a service might rewrite a nick: 1. on room join (status codes 110 and 210) 2. on user initiating a nick change (status codes 110 and 303) 3. at some point after join, without the user requesting a normal nick change (e.g., after nick registration, or service policy change) I don't think (1) and (2) really need any extra status codes than what get sent normally, while (3) can be treated exactly like (2), as if the user requested the change. It may make sense to extend 210 to (2) and (3). The question is, how should clients use these status codes? In my opinion, simply assuming your current nick is the last one with a 110 works in all cases for maintaining internal client state, while 303 and 210 could be used to generate user notifications if required about server modification of nick. -- Waqas Hussain
Re: [Standards] NEW: XEP-0309 (Service Directories)
On Wed, Jan 11, 2012 at 12:36 AM, XMPP Extensions Editor wrote: > Version 0.1 of XEP-0309 (Service Directories) has been released. > > Abstract: This specification shows how to combine and extend a number of > existing XMPP protocols for improved sharing of information about XMPP > servers. > > Changelog: Initial published version. (psa) > > Diff: N/A > > URL: http://xmpp.org/extensions/xep-0309.html > http://code.google.com/p/prosody-modules/source/browse/mod_service_directories/mod_service_directories.lua This is a work in progress implementation, is mostly untested, and will contain bugs. Testers appreciated. Both the directory and normal server side of the protocol are implemented. Missing features: - More complete pubsub support (currently only supports getting all items) - Persistence (all data is currently kept in memory) This module does not serve vcard4 queries (that would be part of mod_vcard in Prosody trunk). -- Waqas Hussain
Re: [Standards] XEP-0055: Jabber Search / How to handle searches containing both kind of forms?
On Wed, Dec 21, 2011 at 8:00 PM, Tobias Markmann wrote: > Hi, > > just reading over XEP-0055 and wondering how x-data forms are supposed > to be used in the XEP? The examples only show searches containing only > jabber:search forms or x-data data forms, not both (which would be > desirable for backwards compatibility). > Sending both kind of forms back to the client there are two sane > options for it to interpret that: > > 1. The x-data form replaces/overrides the jabber:search form fields > (i.e. ). > 2. The x-data form extends the other fields. > > The first interpretations implies sending back only the filled out > x-data form while the second would have to send back a mix of > jabber:search reply and filled out x-data form. > > How have you interpreted/implemented the XEP? What do you think about this? > > Cheers, > Tobi (Tobias poked me off list about this. This message is evidence of persistent pestering producing results. What I discovered is described here, in case someone needs it.) The spec is less explicit than it could be, and various corner cases are undefined. In such cases, it's useful to look at implementations. Here's ejabberd's: https://github.com/processone/ejabberd/blob/v3.0.0-alpha-5/src/mod_vcard.erl#L360 https://github.com/processone/ejabberd/blob/v3.0.0-alpha-5/src/mod_vcard.erl#L404 It only handles forms, and returns when it's missing. The XEP doesn't specify any way to have wildcards or patterns. In fact it avoids specifying anything regarding how the matching is done (e.g., is the search case sensitive?). The ejabberd implementation sends an explicit "(Add * to the end of field to match substring)" instruction. Another implementation is Openfire's: http://fisheye.igniterealtime.org/browse/openfire/trunk/src/plugins/search/src/java/org/jivesoftware/openfire/plugin/SearchPlugin.java?hb=true It supports both the form and non-form based methods. If a form is found, non-form data is not processed. Additionally, an empty string value should mean the form field is not used in the search. The alternative would be too annoying for users. Openfire does this. I suspect the ejabberd implementation does this too, but haven't verified. A comment in the Openfire code indicates some clients are hard-coded with specific form values (see the SearchPlugin constructor in the linked code). Another unspecified behavior is how multiple field values are searched on. Openfire uses OR. ejabberd I suspect is using AND, and doing a case insensitive comparison. I personally prefer AND. Prosody does not implement this in a stable release (we have a basic experimental plugin), but the feature is requested fairly often. I'll omit the moral of this story, and let readers decide on their own. -- Waqas Hussain
Re: [Standards] UPDATED: XEP-0300 (Use of Cryptographic Hash Functions in XMPP)
On Tue, Dec 6, 2011 at 3:32 AM, Peter Saint-Andre wrote: > On 12/5/11 3:16 PM, XMPP Extensions Editor wrote: >> Version 0.2 of XEP-0300 (Use of Cryptographic Hash Functions in XMPP) has >> been released. >> >> Abstract: This document provides recommendations for the use of >> cryptographic hash functions in XMPP protocol extensions. >> >> Changelog: Updated to reflect initial analysis of existing XMPP protocol >> extensions. (psa) >> >> Diff: http://xmpp.org/extensions/diff/api/xep/0300/diff/0.1/vs/0.2 >> >> URL: http://xmpp.org/extensions/xep-0300.html > > Folks, I started to look at XEP-0300 in relation to existing extensions. > Please review my work so far, and do your own thinking about how useful > (or not useful) XEP-0300 is. > I'm curious about the descriptive feature namespaces (urn:xmpp:hash-function-textual-names:md5)... I'm sure there is something behind not using urn:xmpp:hash:md5, or similar :) Also, the encapsulating element isn't really necessary, except for cases where only a single element is allowed (pubsub). I recall we were measuring bytes when defining entity caps in presence, which would suggest changing this protocol to more compact. A consistent approach to hashes is a good thing. Changing widely deployed protocols is a bad thing. The nature of the XEP makes it awkward to use in many protocols (as noted at the end of this message). I'm -0 on this XEP. Of the XEPs listed in XEP-0300 section 4.5, the widely deployed protocols are entity caps, vcard based avatars, and socks5 bytestreams. BOSH is widely deployed, but I don't think the hashing part is. I'd suggest leaving vCard based avatars alone. Entity caps is arguably supposed to change, due to security issues. I'm not sure about the SOCKS5 XEPs. They are quite widely deployed, and if we do change things, backwards compatibility will need to be kept. That said, changing things in these various protocols would be fairly awkward, given the existing use of attributes for hashes. e.g., it would be fairly awkward to change the BOSH 'key' and 'newkey' attribute to elements in . -- Waqas Hussain
Re: [Standards] [XEP-0234] File checking in JingleFT
ng a protocol here, we have to keep scaling in mind. The simplest and most scalable solution would be to simply make a merkel hash file available as a separate file transfer, and only transfer the root hash in the jingle negotiation. This also allows ranged transfer of only necessary parts of the hash file, and doesn't need base64 encoding. Do we have anything like Jingle URIs? Something like XEP-0231: Bits of Binary, but for Jingle. That would be useful in general, and for this specific solution. -- Waqas Hussain
Re: [Standards] request for reviews: XEP-0045 v1.25rc5
On Wed, Sep 28, 2011 at 1:44 AM, Peter Saint-Andre wrote: > On 9/27/11 7:29 AM, Waqas Hussain wrote: >> On Tue, Sep 27, 2011 at 2:36 AM, Peter Saint-Andre >> wrote: >>> On 9/19/11 11:34 PM, Waqas Hussain wrote: >>> >>>> 3. Service changing room nick >>>> >>>> I'd like some text stating that a service can change the occupant's >>>> nick at any time, including room join. An occupant MUST listen for >>>> status code=100 in available presence for their current nick. >>> >>> I understand nick changes on join, but not after that. What is the use case? >>> >> >> Implementing policies such as lowercasing all nicks (these happen on >> room join, or when user initiates a nick change). > > Right, the nick change use case makes sense. Sorry I missed that. > >> And there's also the >> case of applying these when a user has not requested a nick change >> (e.g., a user registers a nick with a room, the service may force a >> nick on them when an admin accepts the registration). > > Maybe. But why wouldn't the service enforce the nick rules on registration? > Example: User's current nick is 'psa', and sends a registration form with nick 'stpeter', then on registration acceptance server switches their nick to 'stpeter' for them. Example 2: Room owner selects the "lowercase nicks only" option in the room config. Room updates nicks of those present. What I want is text saying either this is allowed, or that this is not allowed. My vote is for allowing the server to do this. >> Software which seems to follow my definition of a subject change, and >> not your's: ejabberd, Tigase, Gajim, Miranda. And Prosody. >> >> Pidgin is special, in that it displays the in the chat log, >> and displays the above the chatlog. M-Link on jabber.org >> seems to return a error on getting with >> any siblings. I didn't manage to find anything else which didn't >> follow my definition. >> >> Also, some servers add elements to the subject message sent >> to the client, with the time of when the subject was set, and could >> add other things (crypto signatures, etc). > > Yes, that's possible. But we've defined a subject change request as > message-with-subject-but-no-body for a long time now. > > Naturally, if we were designing this all from scratch we'd probably use > an ad-hoc command for subject changes. :) > What I was implying was, most deployed software is not following the 'message-with-subject-but-no-body' rule, and is following the 'message-with-subject-is-a-subject' rule. Making the latter wrong and the former right is going to make most deployed software non-compliant. >>>> 11. Full-to-bare JID rewriting to support vCards >>>> >>>> All(?) implementations are doing it, but it's not specified anywhere. >>>> Should it be? >>> >>> Yes, it should. Proposed text would be appreciated. >>> >> >> Err... a quick attempt, probably not too good: >> >> [Section 16.4: IQ] >> >> 6. If an occupant sends an IQ get to another occupant with the child >> element , the room SHOULD route the stanza >> to the target occupant's bare real JID. The room should also rewrite >> the 'from' attribute of the IQ result response to the initial target >> occupant's full in-room JID. The room can store any state required in >> 'id' or 'from' attributes of the IQ get stanza it sends. > > Not bad. But do we really want to special-case this for "vcard-temp"? At > the very least, why not "urn:ietf:params:xml:ns:vcard-4.0"? And at most, > the same logic might apply to extensions other than vCard... I was mostly thinking of documenting historical practice. There's a case for vcard4 as well, but perhaps that could be solved along with the PEP in MUC problem. At that point we may even have the option of deprecating the whole vcard-temp thing. I'm not too concerned about documenting this, and we could leave the whole thing for later. Aside: There has also some discussion in the jdev room and elsewhere about the room itself querying and caching vCards (and other data) of occupants, and serving occupants' vCard queries from that cache. vCards account for most of the traffic on new room join. -- Waqas Hussain
Re: [Standards] request for reviews: XEP-0045 v1.25rc5
On Tue, Sep 27, 2011 at 2:13 AM, Peter Saint-Andre wrote: > On 9/24/11 1:53 PM, Waqas Hussain wrote: >> >> I note that this feature has no disco feature defined. > > MUC does not have the plethora of disco features that PubSub has. You > decide whether that's a good thing or a bad thing. > >> Given that >> no-one seems to have deployed this yet, we need a way to discover >> support. > > Do we want or need to define fine-grained disco features for XEP-0045? > And if so, why limit the new features to just this one? > Many clients enable/disable/hide UI elements based on what is allowed/supported. The argument for having these particular features discoverable and not all others is that most other non-discoverable features are available in pretty much all server implementations. These features however are not present on existing servers. Clients should therefore only be showing UI elements when the features are known to be available. Doing otherwise is going to annoy users when it fails after they've put in the effort of filling in a form and submitting it, with no previous indication that it wasn't supported. -- Waqas Hussain
Re: [Standards] request for reviews: XEP-0045 v1.25rc5
On Tue, Sep 27, 2011 at 2:36 AM, Peter Saint-Andre wrote: > On 9/19/11 11:34 PM, Waqas Hussain wrote: > >> 3. Service changing room nick >> >> I'd like some text stating that a service can change the occupant's >> nick at any time, including room join. An occupant MUST listen for >> status code=100 in available presence for their current nick. > > I understand nick changes on join, but not after that. What is the use case? > Implementing policies such as lowercasing all nicks (these happen on room join, or when user initiates a nick change). And there's also the case of applying these when a user has not requested a nick change (e.g., a user registers a nick with a room, the service may force a nick on them when an admin accepts the registration). I'd like the spec to be explicit on whether the server is allowed to do this or not. I'm in favor of allowing it. I don't like it being undefined. >> 4. Presence from existing occupant to bare room JID >> >> The MUC's behavior is currently undefined. At least for the >> type="unavailable" case (and perhaps also for the available case), the >> MUC should handle and treat it as if it was sent to the occupant's >> nick. > > I disagree. Sending presence to the Room JID is different > from sending it to the Occupant JID (e.g., the user > might be sharing presence with the room itself, if the user and the room > are subscribed to each other's presence). > Alright. What I had in mind was one popular client (I can't remember which) which sent unavailable to the bare room JID. I'm fine with it being considered a client issue, and leaving the spec as it is. I'll note that both ejabberd and Prosody do this. I haven't tested other servers. >> 5. Both and in a single message >> >> "(A message with a and a is a legitimate message, >> but it SHALL NOT be interpreted as a subject change.)" >> >> I object to this. It complicates subject handling. I believe much >> existing MUC software considers a message a subject change if it has a >> in it. How should software determine this? Assume it's a >> subject change if there's no ? What if there is not body, but >> xHTML-IM is included? What if there's no body, but some >> ? > > IMHO a subject change should be a message with *only* a child > element and no other children. > Software which seems to follow my definition of a subject change, and not your's: ejabberd, Tigase, Gajim, Miranda. And Prosody. Pidgin is special, in that it displays the in the chat log, and displays the above the chatlog. M-Link on jabber.org seems to return a error on getting with any siblings. I didn't manage to find anything else which didn't follow my definition. Also, some servers add elements to the subject message sent to the client, with the time of when the subject was set, and could add other things (crypto signatures, etc). >> And I have mixed feelings about that MUST NOT. > > We had some discussion about that years ago, and decided that Occupant > JIDs like "room@service/ " would be problematic. Do you have a use > case for those? > What are these problems? For one thing, most existing software doesn't enforce this. For another, 'invisible' isn't defined. Does this mean nicks consisting entirely of unicode whitespace characters? Does this fix all the problems? Or do we have to take this further? e.g., does the partially invisible nick "[space]stpeter[space]" have many of the problems of all-whitespace nicks? That MUST NOT just seems like an inadequate fix for a complex problem (assuming the problems are what I think they are). >> 9. In schema section 18.2 http://jabber.org/protocol/muc#user >> >> I'd like changed to > ref='item' minOccurs='0' maxOccurs='unbounded'/>, to allow one >> element for each session in a multi-session nick when including 'jid'. > > I have no deep objections to that, as long as we don't try to define > mult-session nicks in this spec. > Agreed, I'm all for making XEP-0045 smaller, and moving most optional stuff to separate specs. >> 11. Full-to-bare JID rewriting to support vCards >> >> All(?) implementations are doing it, but it's not specified anywhere. >> Should it be? > > Yes, it should. Proposed text would be appreciated. > Err... a quick attempt, probably not too good: [Section 16.4: IQ] 6. If an occupant sends an IQ get to another occupant with the child element , the room SHOULD route the stanza to the target occupant's bare real JID. The room should also rewrite the 'from' attribute of the IQ result response to the initial target occupant's full in-room JID. The room can store any state required in 'id' or 'from' attributes of the IQ get stanza it sends. -- Waqas Hussain
Re: [Standards] Suggestion for XEP-0045 : permit alias for the MUC address
On Sat, Sep 24, 2011 at 10:38 PM, Alexander Holler wrote: > Am 24.09.2011 11:57, schrieb Kevin Smith: >> >> MUC aliasing is actually one of the very few cases in XMPP where >> aliasing could work without protocol extensions. The server would know >> which alias each client has joined and can route messages from that >> JID. I don't know any server implementation that does this, but if one >> did it wouldn't require client changes or deviation from XEP-0045. > > I think confusion on the client side would start if a client connects to > both aliases without knowing they are aliases. A server would have to forbid > that with a meaningful message. > > And handling room-aliases on the server side would have many pitfalls > (e.g.the delay elements in the history). I'm not sure where else room-JIDs > are used (besides in 'from'), that would require checking the whole XEP for > appearances of room-JIDs (which currently isn't that easy) ;) > Kev is correct. MUC aliasing can work fine without any protocol or client changes. A server doesn't need to forbid letting a client enter a room in two ways. It would work fine. And I can't think of any pitfalls regarding the delay element. This is effectively one room 'object' on the server, and two 'views' on it. It wouldn't be hard to modify Prosody's implementation to do this. Most of the work required has already been done to support multi-session nicks. I think the only change required would be to make the internal data structures use fullrealjid+fullroomjid as a key where they currently use just fullrealjid. -- Waqas Hussain
Re: [Standards] request for reviews: XEP-0045 v1.25rc5
On Sat, Sep 24, 2011 at 2:08 AM, Peter Saint-Andre wrote: > On 9/20/11 6:00 PM, Evgeniy Khramtsov wrote: >> On 20.09.2011 08:46, Peter Saint-Andre wrote: >>> On 9/19/11 4:40 PM, Alexander Holler wrote: >>>> >>>> No, but maybe adding some muc-features which are making it obvious what >>>> is supported by the server is an option. I don't know if there is an >>>> implemention which supports e.g. those voice-requests as described, >>>> those I've tested seem not to have it implemented. >>> If you test more implementations and find that none of them support the >>> feature (and the developers say they have no plans to implement the >>> feature), then it might make sense to remove the feature from the spec. >>> >>> Peter >>> >> >> We have a patch and we are going to include it in the new ejabberd >> version. Please don't remove the feature from the spec. > > Cool, thanks for letting us know. I won't touch that part of the spec. :) > I note that this feature has no disco feature defined. Given that no-one seems to have deployed this yet, we need a way to discover support. I propose the features http://jabber.org/protocol/muc#request and http://jabber.org/protocol/muc#register Also, it's worth considering moving this (nick register, registration approval, voice request) out of XEP-0045, and into an XEP of its own. As far as I see, MUC implementations have up until now treated this is an optional secondary part of the main MUC spec. The new XEP could also include text about service-level nick registrations, which is what it currently implemented and deployed, and can have interesting interactions with room-level registration. I do intend to implement these in Prosody. P.S. An important thing in this is the room requesting additional information. One obvious example is captcha support. How should that flow? The room should send a captcha form IQ-set or message to the requester? -- Waqas Hussain
Re: [Standards] request for reviews: XEP-0045 v1.25rc5
On Tue, Sep 20, 2011 at 3:50 PM, Alexander Holler wrote: > Am 20.09.2011 00:46, schrieb Peter Saint-Andre: >> >> On 9/19/11 4:40 PM, Alexander Holler wrote: >>> >>> Am 19.09.2011 20:23, schrieb Peter Saint-Andre: >>>> >>>> On 9/6/11 10:38 AM, Alexander Holler wrote: >>>>> >>>>> Looking again at XEP-0045, >>>>> >>>>> I don't see a reason why a request for voice should be handled in >>>>> another way than a request for membership. ;) >>>>> >>>>> In fact I would suggest to replace both with an unified "request for >>>>> affiliation" which should work like the request for membership in 7.10 >>>>> (with an attribute 'affiliation' and maybe a xmlns something other than >>>>> 'jabber:iq:register'). >>>> >>>> Is there a strong reason to change this now, other than protocol >>>> hygiene? >>> >>> No, but maybe adding some muc-features which are making it obvious what >>> is supported by the server is an option. I don't know if there is an >>> implemention which supports e.g. those voice-requests as described, >>> those I've tested seem not to have it implemented. >> >> If you test more implementations and find that none of them support the >> feature (and the developers say they have no plans to implement the >> feature), then it might make sense to remove the feature from the spec. > > Since sending a private messages to administrators is always possible (even > without voice), I think there isn't really a need for this feature. That's not true. PMs to room admins are not always possible. For example, jab...@conference.jabber.org has PMs disabled for non-moderators, and sending to admins doesn't work. -- Waqas Hussain
Re: [Standards] request for reviews: XEP-0045 v1.25rc5
On Thu, Aug 18, 2011 at 6:43 PM, Peter Saint-Andre wrote: > I've completed a round of revisions to XEP-0045 (Multi-User Chat) in an > effort to incorporate developer feedback I've received since the last > version 3 years ago. The XMPP Council would like to vote on these revisions > before the end of September or possibly early October, so it would be great > if folks could check the diff in the next few weeks: > > http://xmpp.org/extensions/diff/api/xep/0045/diff/1.24/vs/1.25rc5 > 1. Using GC instead of "groupchat 1.0" in various places This doesn't affect me, but new readers of the document might find it confusing. 2. Room subject The current text suggests the room should send a subject if it's set. I'd like it to send in the case when it's not set. The subject will then act as a clear end marker for room history. 3. Service changing room nick I'd like some text stating that a service can change the occupant's nick at any time, including room join. An occupant MUST listen for status code=100 in available presence for their current nick. 4. Presence from existing occupant to bare room JID The MUC's behavior is currently undefined. At least for the type="unavailable" case (and perhaps also for the available case), the MUC should handle and treat it as if it was sent to the occupant's nick. 5. Both and in a single message "(A message with a and a is a legitimate message, but it SHALL NOT be interpreted as a subject change.)" I object to this. It complicates subject handling. I believe much existing MUC software considers a message a subject change if it has a in it. How should software determine this? Assume it's a subject change if there's no ? What if there is not body, but xHTML-IM is included? What if there's no body, but some ? 6. action nick and jid for kicks and bans Examples 85 and 108 have been updated from to , but the text immediately before them still says "the bare JID of the user who initiated [...]". What I'd like here is to allow the room to include a nick and/or a bare/full JID. A nick is useful for general consumption, but a service should be allowed to turn this off. And a service should be allowed to include JID in the stanzas going to occupants who are allowed to see JIDs (moderators). And I don't see any particular reason to not make them full JIDs. 7. Section 16.1 restates what RFC6122 already specifies (and calls it RFC6120 instead). And I have mixed feelings about that MUST NOT. 8. Presence subscriptions I've been wanting to add this in Prosody, but have worried about client behavior on receiving both MUC-specific, and normal presence from the same JID. I'm +1 to it though. Many users do add rooms as contacts. 9. In schema section 18.2 http://jabber.org/protocol/muc#user I'd like changed to , to allow one element for each session in a multi-session nick when including 'jid'. 10. MUC child element in presence errors "Note: If an error occurs in relation to joining a room, the service SHOULD include the MUC child element (i.e., ) in the stanza of type "error"." What is the rationale behind this SHOULD? 'id' attributes are the canonical XMPP way for matching stanzas to error replies; RFC6120#8.1.3 is quite clear that 'id' attributes can be depended on to work fine with presence. IIRC, in Prosody, we added this specifically because Gajim was having issues with nick changes (I note that you didn't specify the above quoted text for this and other error cases). But should this really be a SHOULD? 11. Full-to-bare JID rewriting to support vCards All(?) implementations are doing it, but it's not specified anywhere. Should it be? 12. History management Should it perhaps be noted that clients shouldn't depend on getting only what they asked for? I don't think all MUC implementations support history management. And 'maxchars' is particularly troublesome, as anything between the MUC history code and the client could modify the stanza. Misc: s/"non"/"none"/ "Nicknames can contain virtually any Unicode character introduces the possibility of nick spoofing" - grammar fail. s/hte/the/ All this came from skimming the diff. I'll probably have more feedback when I manage to review the entire spec in detail while looking at Prosody's code. -- Waqas Hussain
Re: [Standards] Addressing Security Concerns in XEP-0115 Entity Capabilities
On Thu, Sep 15, 2011 at 1:37 AM, Matthew A. Miller wrote: > > On Sep 14, 2011, at 13:40, Waqas Hussain wrote: > >> >> So.. which caps is included in presence? The current exploitable one? >> Then this doesn't help with preventing poisoning, does it? >> > > the caps hash would be as it is today. So, yes, a client that doesn't > understand this double-verify would still be vulnerable. > An entity which understood double verify would have the option to either be vulnerable to poisoning, or participate in IQ floods. It's this that I'm against. >> What does considering the result suspect mean? I'm hoping you don't >> mean it isn't cached. Because that would be much worse than just using >> a new XEP, which would be a perfectly smooth transition. >> > > Being suspect means it's up to the implementation and deployment. > * Some might accept (and cache) them but with a log message somewhere. Err So poisoning succeeds. And what happens with these logs? How do you find the poison needle in the haystack of legitimate messages? I hope you don't want admins to do this... > * Some might reject (and not cache) them unconditionally. And therefore early adopters are punished with IQ floods... > * Some might put in a timebomb into their implementations that will switch > from "accepted" to "rejected". I just don't like this particular idea... > >> I have assumed the whole point of all this effort to keep the old XEP >> is to get rid of the transition period, so if you are not going to >> cache exploitable caps at all, might as well define a clean new XEP. > > The point of trying to keep the old XEP is not to eliminate a transition, but > to make it less painful. I do think having a complete replacement to caps is > more painful than applying an addendum or two. As Dave said, the building is > not burning (yet). > I see. Frankly I was (and still am) confused at what the point was. Make it less painful? For whom? Less painful for developers? I disagree that adding tons of extra checks and magic disco features and forms is going to make developers happy. A nice clean separate algorithm is much more likely to do that. Less painful for users/server-admins? Your particular proposal gives them IQ floods (most of the proposals from stpeter do not have this issue). Less painful for spec writers? I don't think so. This is how I see it: stpeter's proposals: 1. dont succeed in fixing the issue, 2. don't have the flood issue, 3. don't add anything to presence, 4. complicate the spec. your proposal: 1. either doesn't fix the issue, 2. or causes floods, 3. doesn't add anything to presence, 4. complicates the spec, as both the current and new algorithms are now part of it. new XEP: 1. fixes the issue, 2. doesn't cause floods, 3. does add something to presence during the transition, 4. simplifies the spec. Oh, and I agree there is no burning building. I don't think anyone thinks there is at the moment. > > - m&m > <http://goo.gl/voEzk> > > -- Waqas Hussain
Re: [Standards] Addressing Security Concerns in XEP-0115 Entity Capabilities
On Wed, Sep 14, 2011 at 12:04 AM, Matthew A. Miller wrote: > > On Sep 13, 2011, at 12:18, Matthew A. Miller wrote: > > >> I've been thinking of something that might be a less-awful compromise. I'll >> post to this list about it soon for us all to mock and ridicule (-: >> > > So, the less-awful compromise I was talking about is this: > > We develop a XEP-0128 extension that defines how to canonicalize and hash > disco#info results, which fixes all of the known issues with caps' current > canonicalization and can be applied to any disco#info result. e.g.: > > > > ... > > > ... > > > urn:xmpp:discohash:0 > > > SHA-1 > > > EHdJI0CahGt8XjSWUz59ODb4OrY= > > > > > In terms of XEP-0115, a "concerned" implementation would first look and > validate this hash (according to the new TBD spec). If this extension is > missing it might consider the disco results suspect (and still validate the > XEP-0115 hash) or outright invalid (maybe sometime in the distant future). > If present and valid, it could either A) assume the caps hash is valid (if > just mildly concerned), or B) validate the caps hash but confident it will > not find any of the known attacks to XEP-0115 (if correctly paranoid). > > It does mean a fully-conforming implementation is canonicalizing and hashing > twice (once for the TBD spec, once for XEP-0115), but it does mean existing > implementations would still work in both directions. Plus, we could then use > this for any disco#info result, potentially applying a cryptographic > signature. > > > Thoughts? > > - m&m > <http://goo.gl/voEzk> So.. which caps is included in presence? The current exploitable one? Then this doesn't help with preventing poisoning, does it? What does considering the result suspect mean? I'm hoping you don't mean it isn't cached. Because that would be much worse than just using a new XEP, which would be a perfectly smooth transition. I have assumed the whole point of all this effort to keep the old XEP is to get rid of the transition period, so if you are not going to cache exploitable caps at all, might as well define a clean new XEP. -- Waqas Hussain
Re: [Standards] Addressing Security Concerns in XEP-0115 Entity Capabilities
t; Joe and Matt, do correct me if I've misrepresented our discussion. > > Peter > > [1] http://www.w3.org/TR/xmlschema-2/#string > [2] http://www.w3.org/TR/2008/REC-xml-20081126/#NT-Char > [3] http://xmpp.org/registrar/disco-features.html > [4] http://www.jclark.com/xml/xmlns.htm > Okay.. let's see... while that was a nice try, it still doesn't work.. First, are you mandating that every client include that disco form in their caps, even if they don't use any dataforms? If yes, ugly. If no, they are still open to attack. Second, I'm assuming you want to keep backwards compatibility, i.e., clients with the current exploitable caps still get their caps cached. If so, then I can simply bypass your protection: assume you have feature: "!" feature: "f1" feature: "f2" feature: "f3" then I can replace that all with: ! f2 f3 I suspect I may have missed something, because this was too easy. -- Waqas Hussain
Re: [Standards] Addressing Security Concerns in XEP-0115 Entity Capabilities
On Mon, Sep 5, 2011 at 5:39 PM, Dave Cridland wrote: > On Wed Aug 31 06:48:26 2011, Waqas Hussain wrote: >> >> Most protocol attacks are based on unexpected input. Attackers >> wouldn't really care whether the values they send are registered or >> usable in any way, as long as the attack succeeds. I don't think you >> are proposing all caps handling entities ship with a copy of the >> registry and fail on anything not included. > > No, but we could potentially restrict input quite heavily. Disco extensions remain a sticky point, and I dislike forbidding '/' in identity.name. > Let's say that we add (yet) another attribute to the caps element, > indicating that the "new" restrictions are in place. (These might include > the various restrictions mentioned in this thread). > > Now, when a client sees a caps marked as restricted, it can validate the > disco#info it gets. > > If a client sees two caps strings, one marked as restricted and one not, it > should assume that the caps string is intended to be restricted as proceed > accordingly. That's a fairly interesting idea. More thinking required. > I'm also wondering if it's worth considering optional (and marked) removal > of XEP-0128 in cases where it's not used. > > Of course, it may be simplest just to bite the bullet and switch hash > algorithm - or even change the 'hash' attribute name - because then it'll > get treated as a pre-1.4 caps by the vast majority of entities and > everything will happen right (or at least, no worse than it often does today > anyway). > > I'm gradually leaning toward this, because although it's *quite* violent, > the downside is not impossible. The transition should be smooth. I was initially for fixing the current protocol, and would be for it if we found a good way to do so. > BTW, anyone any idea what happens if you include more than one in a > presence, in practical terms? I suspect some would pick the first and some the last. That's the two kinds of child-element-scanning logic I've seen in clients. I don't think we can depend on any behavior here. Also, see last discussion about getting rid of order attribute in privacy lists :) >> I know Miranda and Tkabber send software version. Probably many >> others. Can we gather stats on jabber.org for this? > > M-Link cannot capture that data, except by capturing all traffic (which is > inadvisable). Sad. I'd hoped there would be a way to dump the PEP caps table. > I would re-iterate that we still have no burning building. We're getting > closer, though. I don't disagree. That's why I never really pushed for getting this resolved instantly. It's an annoyance, but not a burning building at the moment. > Dave. -- Waqas Hussain
Re: [Standards] Addressing Security Concerns in XEP-0115 Entity Capabilities
On Wed, Aug 31, 2011 at 7:10 AM, Peter Saint-Andre wrote: > On 8/30/11 5:41 PM, Waqas Hussain wrote: >>> Can we agree that attacks #1, #3, and #4 are easily overcome by proper >>> handling of inputs? >>> >> >> Those examples, yes. The attack, no. Read my response to your other >> email for a description of the general attack. Presented below are >> variations of these attacks which the new 'proper handling of inputs' >> doesn't catch. >> >> Let's assume we have entirely disallowed '<'. And we validate >> attributes to be valid. >> >> >> is the same as >> >> >> >> is the same as >> >> >> >> is the same as >> > > So, to use examples closer to real life... > > > EXAMPLE 1 > > > > is the same as: > > > > > EXAMPLE 2 > > > > is the same as: > > > > note: "http://jabber.org"; and "protocol" are not registered categories > and types at http://xmpp.org/registrar/disco-categories.html > > note: "caps" is not a registered language tag, although if you visit > http://www.iana.org/assignments/language-subtag-registry I'm sure you > can find interesting entries that are legitimate. > > > EXAMPLE 3 > > > > is the same as > > > > note: none of the types registered at > http://xmpp.org/registrar/disco-categories.html contain '/' > Most protocol attacks are based on unexpected input. Attackers wouldn't really care whether the values they send are registered or usable in any way, as long as the attack succeeds. I don't think you are proposing all caps handling entities ship with a copy of the registry and fail on anything not included. >>>> Attack #2 can be mitigated by forbidding forms without fields in >>>> XEP-0068 and XEP-0128. >>> >>> Regarding attack #2, one approach, which some folks on the formerly evil >>> former Jabber team in Denver just discussed IRL, is to simply ban >>> XEP-0128 forms from computation of the caps hash. The only legitimate >>> use of XEP-0128 that anyone has ever tried to standardize was XEP-0232, >>> but we were never able to reach consensus on that spec and it has been >>> in the Deferred state for over two years. >> >> This is a fundamental change in the algorithm. By fundamental change I >> mean entities using the older and newer versions will not be able to >> interoperate. This is a new algorithm pretending to be the old one. >> There are probably a lot of deployed clients which include disco >> extensions in their disco results. > > As far as I can see, only Coccinella supports service discovery > extensions, and then only for questionable purposes: > > > > http://coccinella.sourceforge.net/protocol/servers > > > 8234 > > > 8077 > > 194.25.44.218 > > I know Miranda and Tkabber send software version. Probably many others. Can we gather stats on jabber.org for this? > > Peter > -- Waqas Hussain
Re: [Standards] Addressing Security Concerns in XEP-0115 Entity Capabilities
On Wed, Aug 31, 2011 at 1:41 AM, Peter Saint-Andre wrote: > Replying to myself before I reply to the latest message from Waqas. :) > > On 8/30/11 12:37 PM, Peter Saint-Andre wrote: > >> So far, two of the attacks (#3 and #4) that you have described (via >> examples only) depend on violations of the XML spec and XEP-0030. >> >> Another of the attacks (#1) depends on converting the literal string >> "<" to "<", which we've said for years now is incorrect. > Err, while we may have said '<' is bad in disco response text and attribute values, you have phrased it wrong here. XML attribute node and text node values are the unescaped '<', not the escaped '<' which is just the serialization of the actual value. Past the XML parser, all code sees '<'. What you are saying is it be re-escaped at that level. > Can we agree that attacks #1, #3, and #4 are easily overcome by proper > handling of inputs? > Those examples, yes. The attack, no. Read my response to your other email for a description of the general attack. Presented below are variations of these attacks which the new 'proper handling of inputs' doesn't catch. Let's assume we have entirely disallowed '<'. And we validate attributes to be valid. is the same as is the same as is the same as >> Attack #2 can be mitigated by forbidding forms without fields in >> XEP-0068 and XEP-0128. > > Regarding attack #2, one approach, which some folks on the formerly evil > former Jabber team in Denver just discussed IRL, is to simply ban > XEP-0128 forms from computation of the caps hash. The only legitimate > use of XEP-0128 that anyone has ever tried to standardize was XEP-0232, > but we were never able to reach consensus on that spec and it has been > in the Deferred state for over two years. This is a fundamental change in the algorithm. By fundamental change I mean entities using the older and newer versions will not be able to interoperate. This is a new algorithm pretending to be the old one. There are probably a lot of deployed clients which include disco extensions in their disco results. Now I have to ask, what is the motivation for going this far in keeping the same XEP that you are willing to update the algorithm in a non-backwards compatible way? It doesn't seem to be to not force developers to write more code. All these fixes are going to lead to more code. Next thing we know, we'll be validating xml:lang is following the large BNF specified in RFC 4646. It doesn't seem to be interop, or you wouldn't suggest this change. What is it then? > Peter -- Waqas Hussain
Re: [Standards] Addressing Security Concerns in XEP-0115 Entity Capabilities
On Tue, Aug 30, 2011 at 11:37 PM, Peter Saint-Andre wrote: > On 8/30/11 10:58 AM, Waqas Hussain wrote: >> >> The problem here seems to be people focusing on the exact bytes of the >> example I happened to type out for these attacks, and not the general >> logic of the attacks themselves. > > Perhaps you should have described the general logic of the attacks > themselves, rather than just showing particular examples. > >> Those exact examples can be easily >> fixed. But there's a general issue here which can't be fixed that >> easily. The general attack acts on all namespaces of the form a/b/d/*. > > Please describe the general attack, then. > >> Note, the XML spec isn't violated. It doesn't *require* validation of >> the xml:lang attribute: >> > A simple declaration for xml:lang might take the form >> > xml:lang CDATA #IMPLIED >> - http://www.w3.org/TR/2008/REC-xml-20081126/#sec-lang-tag > > That's because XML schema post-dates the XML spec. Checking that a > language matches the xs:language datatype seems pretty straightforward, > given that the definitions in the XML spec and the schema datatype spec > both reference BCP47 (RFC 5646). > >> XEP-0030 was only violated because that example (and admittedly most >> other examples of namespaces currently in use) had '//' in them. The >> general pattern of a/b/d/* stays under attack no matter what you do or >> forbid for all namespaces which have at least three slashes and >> something between those slashes. > > Do elaborate. > > So far, two of the attacks (#3 and #4) that you have described (via > examples only) depend on violations of the XML spec and XEP-0030. > > Another of the attacks (#1) depends on converting the literal string > "<" to "<", which we've said for years now is incorrect. > > Attack #2 can be mitigated by forbidding forms without fields in > XEP-0068 and XEP-0128. > >> Now, are we just going to say 'no-one should or will ever use >> namespaces with three or more slashes with something between them'? > > Not until we understand the general attack that you haven't described. > >>>> What you are proposing had already been proposed in those threads by >>>> others, but wasn't sufficient. >>>> >>>> It's rather awkward to forbid '<', or any other valid character in >>>> identity names or disco extensions (which can be user input). >>> >>> What's to be forbidden is the four characters '&', 'l', 't', ';' -- or, >>> more specifically, coverting those literal characters into '<' (since >>> the latter character has a special meaning in XEP-0115). That string of >>> characters would not need to be forbidden in XEP-0030, only treated >>> carefully when taking disco#info strings as input to the XEP-0115 hash >>> calculation. >>> >>> If implementations perform that one check, then Attack #1 is null. >> >> So, forbid < in what could be a user-specified string (name)? Ugly... > > We're not here for aesthetics. > >>> As to Attack #2, that can be solved by saying that a data form of type >>> "result" MUST include at least one element. That is, changing >>> XEP-0004 from: >>> >>> A data form of type "form", "submit", or "result" SHOULD contain at >>> least one element >>> >>> to: >>> >>> A data form of type "form", "submit", or "result" MUST contain at >>> least one element >>> >>> Which seems sensible anyway. >>> >> >> Err.. you are fixing the exact example again, not the general attack.. >> >> >> >> >> >> >> remains equivalent to: >> >> >> >> >> feature1 >> >> >> feature3 >> >> > > Is it fair to say that your general attack depends on inclusion of > XEP-0128 forms? Or do you have an even more general attack to describe? > >> Now, the important part of my email: >> >> While fixing the exact examples I typed out is nice, the real problem >> which needs solving is this: structural information is lost in the >> current algorithm. The disco stanza itself is logically separated into >> sections: identities, features, disco extensions. Disco extensions are >> further separated into forms and forms are mad
Re: [Standards] Addressing Security Concerns in XEP-0115 Entity Capabilities
On Tue, Aug 30, 2011 at 5:49 AM, Peter Saint-Andre wrote: > On 8/26/11 4:32 PM, Waqas Hussain wrote: >> >> The proposed changes don't solve the problem. See what I wrote in the >> "Attempting to fix the algorithm" section of the email you linked. >> Particularly "As far as I can see, there is no way of fixing attacks 2 >> and 3 other than fundamentally changing the algorithm." > > As we've seen, Attacks #3 and #4 violate both XEP-0030 and the XML spec. > The problem here seems to be people focusing on the exact bytes of the example I happened to type out for these attacks, and not the general logic of the attacks themselves. Those exact examples can be easily fixed. But there's a general issue here which can't be fixed that easily. The general attack acts on all namespaces of the form a/b/d/*. Note, the XML spec isn't violated. It doesn't *require* validation of the xml:lang attribute: > A simple declaration for xml:lang might take the form > xml:lang CDATA #IMPLIED - http://www.w3.org/TR/2008/REC-xml-20081126/#sec-lang-tag XEP-0030 was only violated because that example (and admittedly most other examples of namespaces currently in use) had '//' in them. The general pattern of a/b/d/* stays under attack no matter what you do or forbid for all namespaces which have at least three slashes and something between those slashes. Now, are we just going to say 'no-one should or will ever use namespaces with three or more slashes with something between them'? >> What you are proposing had already been proposed in those threads by >> others, but wasn't sufficient. >> >> It's rather awkward to forbid '<', or any other valid character in >> identity names or disco extensions (which can be user input). > > What's to be forbidden is the four characters '&', 'l', 't', ';' -- or, > more specifically, coverting those literal characters into '<' (since > the latter character has a special meaning in XEP-0115). That string of > characters would not need to be forbidden in XEP-0030, only treated > carefully when taking disco#info strings as input to the XEP-0115 hash > calculation. > > If implementations perform that one check, then Attack #1 is null. So, forbid < in what could be a user-specified string (name)? Ugly... > As to Attack #2, that can be solved by saying that a data form of type > "result" MUST include at least one element. That is, changing > XEP-0004 from: > > A data form of type "form", "submit", or "result" SHOULD contain at > least one element > > to: > > A data form of type "form", "submit", or "result" MUST contain at > least one element > > Which seems sensible anyway. > Err.. you are fixing the exact example again, not the general attack.. remains equivalent to: feature1 feature3 Now, the important part of my email: While fixing the exact examples I typed out is nice, the real problem which needs solving is this: structural information is lost in the current algorithm. The disco stanza itself is logically separated into sections: identities, features, disco extensions. Disco extensions are further separated into forms and forms are made up of fields and fields are made up of values. The algorithm discards all this structure. It's lossy. This is the actual issue. You can patch over the simpler individual examples easily, but the algorithm is broken fundamentally and you will not be able to fix that. >> I'm >> still in favor of a clean new algorithm and XEP, > > I'd prefer to salvage XEP-0115 if we possibly can, rather than defining > a completely new algorithm and spec. > The second important part of the email: I completely understand why everyone wants to salvage XEP-0115. But we have a *security* issue here. Implementations which are insecure at the moment must stop working in the future. Trying to let them continue to work would be fine if this weren't a security issue, but is irresponsible when it is a security issue. We'll just tighten restrictions in the document and assume all is fine. All would not be fine as most implementors will remain blissfully unaware of it. New implementators will put in the minimum amount of effort necessary to make PEP work, and that will not include detailed validation. I'm willing to bet most client authors are unaware of there being a security issue at all. Other than checking the source code, no-one would really be able to trust any implementations to be secure in this respect, because the algorithm would be broken by default, with patchwork laid on top to
Re: [Standards] Remarks on XEP-0114
2011/8/16 Jehan Pagès : > Hi, > > so I have just implemented something with XEP-0114 (client side) and I > have a few questions. > > (1) In my case, the component was locale. And I imagine that's quite > the most common case. But that's definitely not an obligation > (especially as we could imagine "component service providers"). Hence > TLS encryption should be possible. > > Maybe it is, but XEP-0114, definitely tells nothing about it. It looks > like the way-to-do is connecting and directly handshake in plain text. > Maybe a logics would be nice. It would allow TLS, compression, > etc. > This could be added in the same time we pass to an SASL authentication > for component. Correct. We have been aware of these issues for some time. That's why XEP-0225: Component Connections exists. XEP-0225 is the way forward. However, there aren't many implementations. I suspect the reason is that XEP-0114 is good enough for most. Most component deployments are on the same machine as the XMPP server, and using TLS or compression in that case is rather pointless. We plan to add support for this to Prosody soon. It can enable all sorts of interesting features not currently available to components. > (2) Is there not any default port (xmpp-component) for components, > same as we have default ports for xmpp-client and xmpp-server? > Or do we consider it the same as the "secret", which is that anyway > every component is expected to use a different port, that is known > only by configuration? > Note that this point does not annoy me. After all, components are > specific configured items (and usually limited) so we can consider the > port as a "configuration-only" information. Nevertheless a text about > this point may be worth added in the XEP to better explain the logics. > Also I think a server implementation should allow for multiple > components on a single port (I did not test if they already do this. I > just say we should add the text). If your server has 20 components, > you might prefer not to have to each time you add a component, open a > new port, but simply use the same as for your other components. Some older servers in fact require that each component be on a separate port. XEP-0225 solves this too. We just need to get everyone to start using it :) > (3) The current logics seems to be that the component connects as > component.shakespeare.lit, keeps a long-live connection and from there > sends stanzas as [localpart@]component.shakespeare.lit[/resource] and > process any message sent to such an address. > That makes sense in this very common connection logics. > > Now I have a use case where I would like to be able my component to > connect as different "entities" on this component. Basically I cannot > make a long-live connection. So I just want to be able to connect > several times. > For instance the component could connect as: > xmlns='jabber:component:accept' > xmlns:stream='http://etherx.jabber.org/streams' > to='plays.shakespeare.lit/someresource'> > > Or it could be also with a localpart (basically any JID that the > component plays.shakespeare.lit has control on). > For this kind of use case, we would want routing rules. > Basically if I connect with component domain part only, I would want > to receive any stanza. But if there is a stanza sent to a given > resource, and if the component is connected with this resource > specifically, it would grab the priority to receive this stanza. On > the other hand, the component connected with a resource must not > receive any stanza if another precise resource is specified. > > And so on. Basically I propose to take into account this new kind of > component connection logics with routing rules (that we could now > discuss, or I could just propose the list of rules which I have in > mind). > That does not prevent the current usual logics: a single connection, > long-live, as the main component domain part. But that allows new > logics with multiple connections and appropriate routing. > What do you think of this? What's the use-case? And I'd like to see your proposed rules, particularly how it works when a component goes down. If it's load-balancing, the reason we don't have a standard is because there are multiple possible load balancing strategies, none of which is perfect. Prosody allows components which are "node@host" or "host/resource". I haven't verified if this works for external components, but we've used it to do a number of interesting things with internal components (e.g., MUC loaded on "r...@example.com", with "example.com" being a normal host with user accounts). > Jehan > -- Waqas Hussain
Re: [Standards] Proposed XMPP Extension: Extensible Status Conditions for Multi-User Chat
On Sat, Aug 13, 2011 at 1:37 AM, XMPP Extensions Editor wrote: > The XMPP Extensions Editor has received a proposal for a new XEP. > > Title: Extensible Status Conditions for Multi-User Chat > > Abstract: This document defines an extensible format for status conditions in > Multi-User Chat, similar to the error format used in the core of XMPP. > > URL: http://www.xmpp.org/extensions/inbox/mucstatus.html > > The XMPP Council will decide at its next meeting whether to accept this > proposal as an official XEP. > Is this intended to remain as a separate XEP, or will this be merged into XEP-0045? I hope the current status codes are going to go away. > 102 > 103 I admit to not being aware of these. How do these work? I see them in the status codes registry, but nowhere else. If they don't work, then we shouldn't have them in the new XEP. Codes 170 to 174 in MUC were specific to configuration change, but I'd like these to be sendable by the service outside of configuration change (e.g., on room join). > 100 > 172 is redundant. can be used in its place. If the client really wants to know whether this is a new configuration change, it can just detect . -- Waqas Hussain
Re: [Standards] Proposed XMPP Extension: XMPP Quickstart
On Thu, Aug 11, 2011 at 2:28 AM, XMPP Extensions Editor wrote: > The XMPP Extensions Editor has received a proposal for a new XEP. > > Title: XMPP Quickstart > > Abstract: This document defines methods for speeding the process of > connecting or reconnecting to an XMPP. > > URL: http://www.xmpp.org/extensions/inbox/quickstart.html > > The XMPP Council will decide at its next meeting whether to accept this > proposal as an official XEP. > > Example 1 doesn't take things to their logical extreme. STEP 1 can be merged with the TLS ClientHello message. STEP 4 can be merged with the server's TLS Finished message. STEP 5 can be merged with the client's TLS Finished message. STEP 9 can be merged with STEP 7. If the server handles it just right, STEP 8 and 10 could be merged into one TCP packet (in response to 7+9). If you are not negotiating TLS in the middle, you can start a stream, do PLAIN or ANONYMOUS login, request roster, set presence, join a chatroom, and send messages, all in the first TCP packet. This makes legacy SSL somewhat attractive. The same can be done with BOSH, if you skip the stream restart on SASL (which virtually all existing clients do?). Moving on... Does the client even need to pay attention to the pipelining stream feature? When connecting to any server, it can first attempt to use full pipelining. If that fails, it can simply reconnect without pipelining. It can cache the failures. It would need to do this even if the server indicates it can support pipelining, as the server may be lying/buggy and e.g., might not support merging TLS negotiation and XMPP data in the same TCP packet. Looking at Example 1 closely, the client pipelined before looking at the stream feature, so what's the stream feature for? Is the client expected to not pipeline unless it has seen the feature in a previous connection to the host? Why? Given that the failure case is harmless, and that pipelining might work on many existing servers, why wouldn't it want to use it on first connect? Would using it violate any specification? -- Waqas Hussain
Re: [Standards] XEP-0045 Multi-User Chat presence errors
On Sun, Aug 7, 2011 at 1:31 PM, Kevin Smith wrote: >> Anecdotal evidence from different service implementations seems to >> support my expectation that they are sent from the JID the original >> presence was sent to. That'd mean that either the spec is wrong or all >> implementations are. > > 100% of the servers that I tested this against (sample size: 1) are > sending the join error from the room JID as per the spec, so it's > certainly not the case that all implementations aren't compliant. > > The examples seem to be logically correct to me - > jab...@conference.jabber.org/Kev (i.e. me) is not sending you an > error, jab...@conference.jabber.org is. I disagree. Arguments follow. As an example take the error. If you try to acquire a nick which is already taken, you get a conflict error. You may or may not have an existing nick (initial join vs. nick change). Now, this does weird things to directed presence book-keeping on the user's server. The interaction between directed presence and error presence and bare and full JIDs is a gray area (something I'd like clarified in 6120bis). If there is no interaction, then that needs to be fixed IMHO (possible unavailable presence leak). If there is, then bare JID response for both initial join and nick change is going to cause issues. This also causes issues with ID mapping. I think it's reasonable for a client to listen for the exact (ID,JID) pair it sent to. In fact, a library shall do it for the client, and it shouldn't be forced to special case for just MUC errors :) XMPP errors always come from the JID the stanza was sent to, not from the entity which actually generated the error. An obvious example is remote-server-not-found. Another example is from the MUC XEP itself: errors from a room which doesn't actually exist. Are you suggesting that should that be coming from the MUC service? :) In Prosody, we have a specific pattern for sending an error reply. e.g.: reply = st.error_reply(stanza, "cancel", "conflict"). This constructs a new stanza with to and from swapped. Looking at ejabberd's MUC code, they seem to have something just like that. XMPP is so consistent in this that we have pretty much no manual error stanza construction other than that function. -- Waqas Hussain
Re: [Standards] XEP-0045 Multi-User Chat presence errors
On Sun, Aug 7, 2011 at 12:20 PM, Ralph Meijer wrote: > Hi, > > I've been reviewing the code in the branch for MUC client support in > Wokkel against XEP-0045. I noticed that the examples for error > conditions in reply to room join and status and nick change presences > are sent from the room JID, instead from the occupant JID the presence > (change) was sent to. There are many different error conditions, so I > pick one possible exchange here: > > Example 18. User Seeks to Enter a Room (Multi-User Chat): > > from='ha...@shakespeare.lit/pda' > to='co...@chat.shakespeare.lit/thirdwitch'> > > > > Example 29. Service Denies Access Because of Nick Conflict > from='co...@chat.shakespeare.lit' > to='ha...@shakespeare.lit/pda' > type='error'> > > > > > > > Anecdotal evidence from different service implementations seems to > support my expectation that they are sent from the JID the original > presence was sent to. That'd mean that either the spec is wrong or all > implementations are. > > I can see arguments for both, but I personally prefer getting a 'reply' > from the JID sent to, as it eases response checking a bit. I.e. how > implementations are doing it now. I suggest we fix the examples in > XEP-0045 to that effect. +1. How implementations are doing it is logical and consistent with other XMPP specifications. -- Waqas Hussain
Re: [Standards] Proposed XMPP Extension: XMPP Compliance Suites 2012
On Thu, Jul 21, 2011 at 2:36 AM, Dave Cridland wrote: > On Tue Jul 19 17:03:00 2011, Peter Saint-Andre wrote: >> >> On Tue, Jul 19, 2011 at 09:00:36AM +0100, Dave Cridland wrote: >> > My plan would be that implementers would host an XML description of >> > their compliance levels, which the XSF's software listings would >> > consume and render into the software list. This would mean that most >> > changes (latest version, Core/Advanced, XEPs) would be >> > implementer-driven. >> > >> > If there is interest, I'll sketch this out in more detail. >> >> Sounds intriguing. > > So the idea is that instead of saying to the XSF, "Hey, we're Isode, and we > do this server called M-Link", we'd say "Hey, we're Isode, and we have an > implementation description at http://www.isode.com/mlink.xml";. > > Now, mlink.xml would say something like: > > > Isode Ltd > Isode > M-Link > R15.0v1 > > > > > > > > > So the XSF gathers together all these in a magical way - such as having a > file somewhere which says: > > > > > > And then runs an XSLT over that periodically which spits out the HTML page, > where this stuff is rendered. Now we can have that XSLT check to see what > compliance levels are hit automatically, and add them in. > > Also added in would be a disclaimer to state that all information was > provided by the developers and the XSF takes no position on its reliability > - which is mechanically true. > > Dave. This is a nice idea. It has been discussed before, and attempts made. One similar though not equivalent project was by Kim Alvefur (https://github.com/Zash/XMPP-Features/tree/yaml), and vendors providing the data in an XML file was discussed in the Prosody room. There's just one problem: This requires that vendors provide sane data. As one example jabberd2 has XEP-0198 support listed on their home page: http://codex.xiaoka.com/wiki/jabberd2:start (the link on the page has been changed to an older version of the spec, which is better). That said, I suppose I'm being too pessimistic. The idea is good enough to deserve an implementation. -- Waqas Hussain
Re: [Standards] Proposed XMPP Extension: XMPP Compliance Suites 2012
On Wed, Jul 6, 2011 at 10:26 PM, XMPP Extensions Editor wrote: > The XMPP Extensions Editor has received a proposal for a new XEP. > > Title: XMPP Compliance Suites 2012 > > Abstract: This document defines XMPP protocol compliance levels for 2012. > > URL: http://www.xmpp.org/extensions/inbox/compliance2012.html > > The XMPP Council will decide at its next meeting whether to accept this > proposal as an official XEP. > > Some thoughts: Why is BOSH included in the list when we say "* Support can be enabled via an external component or an internal server module/plugin."? Any XMPP compliant server would pass that, so there's no point in making this an explicit item. RFC 6122 is missing. I'm assuming the XSF is using the compliance XEPs as a tool to encourage implementation. If that is correct, then: There's a case to be made for caps support for Advanced Server, as some servers do flood users with PEP without taking caps into account. What is the case for Chat State Notifications for Advanced Client? I mean it's useful, just like a hundred other XEPs, but is it useful enough to be made into a compliance requirement? Now, things which are missing, but shouldn't be: Working file transfer should be a requirement for Advanced Client. I'm not sure if audio/video support should be a compliance requirement for Advanced Client, but some would think so. And finally, I'd personally like Message Receipts being included in more clients. They make a huge difference when you are on a bad network (e.g., most mobile networks outside of central city areas across the world). -- Waqas Hussain
Re: [Standards] hash agility in file transfer
On Thu, Jun 2, 2011 at 4:00 AM, Glenn Maynard wrote: > > It should be specified explicitly that the hash is of the complete > file, even when only a range is being offered. > Note that a specific range isn't offered. Only generic support for ranges is advertised. So the hash can only be of the complete file. What HTTP does is relevant here. The "Content-MD5" HTTP header applies to the current payload being transferred, not the whole file. But given that this XEP defines an offer-accept protocol, initiated by the file sender (as opposed to a request-response protocol initiated by the file recipient), it isn't possible to verify range transfers. There is actually quite an interesting use-case: There are download managers which can download partial content from inside a large archive. All they need to do is retrieve the archive index (e.g., zip files have an index at the end), and then request relevant portion(s) of the archive with the content of interest. > -- > Glenn Maynard > -- Waqas Hussain
Re: [Standards] hash agility in file transfer
On Thu, Jun 2, 2011 at 3:26 AM, Peter Saint-Andre wrote: > Last year we had some discussions about hash agility in file transfer, > as a result of which I made an interim version of XEP-0096: > > http://xmpp.org/extensions/diff/api/xep/0096/diff/1.1/vs/1.2rc1 > > However, it seems that the XMPP Council never considered publication of > that version. Do folks on the list think we need to do anything more > than what's at that diff? > > (And yes, we might need to look at hash agility in other extensions, > too, but we started with file transfer...) > > Peter > This way of going about hash agility has the same issue as the entity caps hash's agility. You can only serve one hash, and you have no idea what hashes the recipient supports. Senders would forever be stuck with sending MD5, because doing otherwise will not be interoperable with existing recipients, unless all clients were updated. The interop failure case is quite severe, with the user possibly getting a prompt that the transferred file is corrupted, data getting auto-discarded, etc. A simple solution is to use tags instead of attributes: 552da749930852c69ae5d2141d3766b1 This lets clients which wish to use a stronger hash interop nicely with clients which only support MD5. Tags also allow specifying additional meta-data via attributes, and complex structured data which is more than just a hex string. But if we go with tags, do we even need to specify that? We can always add tags later in new namespaces, even defined in new XEPs, e.g. ... or ... -- Waqas Hussain
Re: [Standards] vCard4: IQ vs. PEP
On Thu, Apr 7, 2011 at 12:13 AM, Dave Cridland wrote: > On Wed Apr 6 19:55:17 2011, Waqas Hussain wrote: >> >> Also, PEP in MUC is not a solved problem. I do like avatars in MUC. > > I'd note that in Belgium, the consensus seemed to be to solve that problem > rather than continue to work around it. Additionally, a recall the consensus > as being to deprecate vcard4-for-avatars in favour of XEP-0084. > +1 to solving that problem. XEP-0084's way is sensible. It does have issues though, described later. Aside: I note XEP-0084 has an example with an avatar posted at a HTTP URL. XEP-0153 explicitly said "The element SHOULD NOT contain an that points to a URI for the image file." Has our stand on the data leak changed? >> All that said, I would really really love to have a generic framework >> to do for PEP and pubsub and disco and everything what we currently do >> for avatars via hash in presence. We added hashes in presence to avoid >> iq:avatar, iq:disco and iq:version floods on login. When we start >> having large data in PEP/PubSub, PEP floods are going to become a >> serious problem. >> >> One obvious solution is a 'hashes' node in PEP. You subscribe to that, >> and you get hash updates. A hash update is a set of hashes for avatar, >> versions, custom namespace, etc. You send a manual request for what >> you want to actually get, and cache the result for the future. It >> would be nice to have this also requestable via IQ (possibly a pubsub >> IQ). > > Perhaps a subscription option? > > We treat PEP's auto-subscribe from the bare jid, plus filtered > notifications, as an auto-subscribe from each full-jid, incidentally, which > makes this easier (if semantically wrong), but if these subscriptions could > (by dint of an additional disco#info feature advertised via caps) indicate > that a they only wishes to receive a hash of the last item on subscribe, > that might work. What's the hashing function? We don't have a standardized hashing function for arbitrary XML payloads in the XMPP community. Perhaps id of the last item would work. > But I suspect it places us into a canonicalization trap, and in any case it > will increase, not decrease, the number of stanzas. > > Alternatives are to send a timestamp with initial presence - possibly > managed by the server on a per contact basis to mitigate the privacy issues. What timestamp? Unless you send a per-PEP-node-per-contact timestamp, you get issues where you miss one update, and never know of it until the next change, and/or get a lot more updates than you need. (I note, timestamp here can be replaced by id or hash) > Dave. -- Waqas Hussain
Re: [Standards] vCard4: IQ vs. PEP
On Wed, Apr 6, 2011 at 10:32 PM, Peter Saint-Andre wrote: > A few minutes ago I published an Internet-Draft that specifies a vCard > KIND of "thing" (instead of "individual" etc.) for use by entities such > as XMPP servers or MUC rooms. This made me think again about whether we > want to use IQs (as in vcard-temp) or PEP (as currently in XEP-0292) as > the way to transport vCard4 data over XMPP. It seems to me that PEP > might make sense for individuals, but not for things like servers or MUC > rooms. > > One approach might be to use IQs for vCard publication and retrieval, > but a special PEP node for notifications that a vCard has changed (if > anyone cares). > > Thoughts? > > /psa > > [1] http://www.ietf.org/id/draft-saintandre-vcard-thing-00.txt > My personal preference is IQs for vCards. Rationale follows. PEP is sent to contact resources when they come online. Given avatars, vCards can be large. I don't want to be sent vCards of my 100+ contacts every time I come online just because I want to see their avatars. Avatars are cached, and updates are via hash in presence. That's lightweight, and lets me have avatars in my contact list. Also, PEP in MUC is not a solved problem. I do like avatars in MUC. PEP for the XMPP server or MUC room itself isn't trivial either. We can ignore subscriptions, and let anyone who sends presence see PEP. However, either the server would need to keep track of everyone who has sent us presence, or blindly respond to all presence with PEP. Workarounds do exist, e.g., keeping a fixed size LRU cache of presence, but it's still not a perfect solution, and would still lead to sending PEP unnecessarily. -- All that said, I would really really love to have a generic framework to do for PEP and pubsub and disco and everything what we currently do for avatars via hash in presence. We added hashes in presence to avoid iq:avatar, iq:disco and iq:version floods on login. When we start having large data in PEP/PubSub, PEP floods are going to become a serious problem. One obvious solution is a 'hashes' node in PEP. You subscribe to that, and you get hash updates. A hash update is a set of hashes for avatar, versions, custom namespace, etc. You send a manual request for what you want to actually get, and cache the result for the future. It would be nice to have this also requestable via IQ (possibly a pubsub IQ). -- That was a brain dump more than anything. Let me know if I overlooked anything obvious. -- Waqas Hussain
Re: [Standards] ACTIVE: XEP-0295 (JSON Encodings for XMPP)
On Fri, Apr 1, 2011 at 4:06 PM, XMPP Extensions Editor wrote:
> Version 1.0 of XEP-0295 (JSON Encodings for XMPP) has been released.
>
> Abstract: This specification defines an alternative JSON encoding for XMPP
> stanzas and other elements.
>
> Changelog: April Fools! (ks, mw)
>
> Diff: N/A
>
> URL: http://xmpp.org/extensions/xep-0295.html
>
>
{
"implemented": "http://blog.prosody.im/json-encodings-for-xmpp/";,
"working_demo": "http://prosody.im/chat/";,
"documentation":
"http://code.google.com/p/prosody-modules/wiki/mod_json_streams";
}
--
Waqas Hussain
Re: [Standards] Improvement of XEP-0277: Microblogging over XMPP
On Wed, Mar 30, 2011 at 11:16 PM, Vanaryon wrote: > > I think this enhancement might encourage more microblogging XEP > implementations. But there are still lots of missing features, and we should > discuss about it in the future (such as: how do the clients can simply > notify the user when an @user post is published by someone?). > > I also noticed a little error: urn%3Axmpp%3Atmp%3Amicroblog seems not to be > in use, but it is present in some XMPP URI, can you change it to the new > XMLNS? > > Thanks for your great work at the XSF, > > Valérian Saliou, the Jappix project founder. Hiya I'll just note one minor issue. You added as a child of . You can't do it like that (in the Atom namespace, which you don't own/control). Your options are: 1. Put it under a different namespace Someone some...@example.com 2. Use , which is defined in Atom Someone xmpp:some...@example.com I prefer . -- Waqas Hussain
Re: [Standards] Proposed XMPP Extension: vCard4 Over XMPP
On Fri, Feb 25, 2011 at 11:30 PM, Peter Saint-Andre wrote: > On 2/12/11 4:26 PM, XMPP Extensions Editor wrote: >> The XMPP Extensions Editor has received a proposal for a new XEP. >> >> Title: vCard4 Over XMPP >> >> Abstract: This document specifies an XMPP extension for use of the >> vCard4 XML format in XMPP systems, with the intent of obsoleting the >> vcard-temp format. >> >> URL: http://www.xmpp.org/extensions/inbox/vcard4.html >> >> The XMPP Council will decide at its next meeting whether to accept >> this proposal as an official XEP. >> > > BTW, I've fixed some errors in the examples, so the latest version is > now 0.0.4. Reload http://www.xmpp.org/extensions/inbox/vcard4.html as > necessary. > > Peter Many more errors in the examples. The prefix element's end tag is invalid. Fixing that makes the XML valid. Running a Relax NG validator on that brings up many errors. required before in As meaningful as the following is, "love" is not a valid organization type. can only contain "work" or "home". love Juliet Same for the in . and are required before and are required before is no longer allowed. Here's the example fixed: Romeo Montague Montague Romeo Sir --0203 15000808T1430-0500 male work Juliet home 123 Capulet Way Verona Italy work ro...@montague.lit -- Now some comments. This was the first time I saw the vcard4 XML schema. I was surprised at how it requires elements (even if they are empty), and requires they be in a specific order. Just for amusement, I proceeded to run the validator on the two largest examples within the vcardxml draft. As I half expected, they failed. The schema seemed long enough that I gave up on figuring out how compatible this is with the existing vcard-temp. It looks like mapping vCard4, vcard-temp, User Avatar, etc to a single backend is going to be much more work than I had anticipated. -- Waqas Hussain
Re: [Standards] need clarification about adding own jid to roster
On Wed, Feb 23, 2011 at 6:10 PM, Sebastian Reitenbach wrote: > Hi, > > as xmpp client developer (coccinella), I want to add my own jid to my roster. > The last time I worked on that, was about a year ago. There I took > RFC3921bis-08 as reference. There in chapter 2.3.3 at the end it stated: > > The server MUST return a stanza error to the client if > the value of the element's 'jid' attribute matches the bare > JID portion of the element's 'from' attribute > (i.e., a JID MUST NOT be allowed to add itself to its own roster). > > Therefore I was looking into adding some exceptions to handle the own JID in > the roster, or better explained as adding a fake jid to the roster view in the > client. I got a bit distracted with other stuff, and just recently started > again working on that. > > now I just looked again to the in the latest revision, 20, of the RFC, and I > read this now: > Interoperability Note: Some servers return a stanza > error to the client if the value of the element's 'jid' > attribute matches the bare JID of the > user's account. > > So I should understand this statement now as: > Servers should generally allow the client user to add the own JID to the > roster, this is no error anymore. Only older server implementations may not > allow adding the own jid. > > If my understanding is right, then this would make my life much more easier > since I don't need to implement exceptions for the own jid. > > cheers, > Sebastian Your client shouldn't disallow it. However, this is a relatively recent change, and your client should handle a error properly. -- Waqas Hussain
Re: [Standards] Proposed XMPP Extension: Remote Authentication
On Thu, Dec 2, 2010 at 7:58 PM, Peter Saint-Andre wrote: > On 12/2/10 7:50 AM, Waqas Hussain wrote: >> On Thu, Dec 2, 2010 at 7:54 AM, XMPP Extensions Editor >> wrote: >>> The XMPP Extensions Editor has received a proposal for a new XEP. >>> >>> Title: Remote Authentication >>> >>> Abstract: This document defines an XMPP protocol extension that enables >>> entities to use SASL for authentication with remote services (such as >>> Multi-User Chat rooms). >>> >>> URL: http://www.xmpp.org/extensions/inbox/remote-auth.html >>> >>> The XMPP Council will decide at its next meeting whether to accept this >>> proposal as an official XEP. >>> >> >> Some comments: >> >> 1. The SASL mechanisms list can be sent back in the presence error, >> avoiding a round trip. > > Yeah, I wasn't sure whether to do that or not. My email to the jdev list > on this topic included the mechanisms in the presence error. I'm still > mulling it over, but you're right that it would save a round trip. I > wonder: can that model be generalized to other extensions? (Think > pubsub, gateways, etc.) > Various fun things come to mind. e.g., my client could authenticate with an MSN transport using some X-MSN SASL mechanism, and the transport never needs to know my password. (it can probably change the password once it's authenticated, but that's besides the point :) ) >> 2. The presence after authentication need not be sent. On successful >> auth, the initially sent presence can be used, avoiding a round trip. > > Good point -- and a good reason to include the SASL mechanisms in the > presence error. > >> 3. What the authentication identity looks like is undefined. I'm not >> sure if this is good or bad. > > I think it's bad. We need to fix that up. > >> 4. The error condition is 'sasl-required'. Does this imply that normal >> MUC password auth should fail, even with a correct password? > > What do you think? > > Because legacy MUC passwords are sent in the clear, a given MUC service > might not want to accept that other method, but in practice I think they > would for quite a while. I think allow. That is, if the client has already sent us a ***, there isn't much point in ignoring that and redoing auth. The damage is done already. >> And finally, an implementation: >> http://code.google.com/p/prosody-modules/source/browse/mod_saslauth_muc/mod_saslauth_muc.lua > > Cool. :) > >> The linked implementation works with Prosody trunk, and verifies that >> the user knows the room password. This would be far more interesting >> with some per-user credentials. > > Indeed. How to establish those? > > It would also be interesting to use certificate-based authentication. > I've been very interested in certificate-based auth for clients lately. S2S poses an issue for normal EXTERNAL auth for remote clients. >> How about something new instead of `> var='muc_passwordprotected'/>' to advertise SASL support. > > Yep, we need that. How about 'urn:ietf:params:xml:ns:xmpp-sasl'? If that > is not specific enough, we could define urn:xmpp:remote-auth as a > feature (but not an XML namespace). > > Peter > +1 to 'urn:ietf:params:xml:ns:xmpp-sasl' -- Waqas Hussain
Re: [Standards] Proposed XMPP Extension: Remote Authentication
On Thu, Dec 2, 2010 at 7:54 AM, XMPP Extensions Editor wrote: > The XMPP Extensions Editor has received a proposal for a new XEP. > > Title: Remote Authentication > > Abstract: This document defines an XMPP protocol extension that enables > entities to use SASL for authentication with remote services (such as > Multi-User Chat rooms). > > URL: http://www.xmpp.org/extensions/inbox/remote-auth.html > > The XMPP Council will decide at its next meeting whether to accept this > proposal as an official XEP. > Some comments: 1. The SASL mechanisms list can be sent back in the presence error, avoiding a round trip. 2. The presence after authentication need not be sent. On successful auth, the initially sent presence can be used, avoiding a round trip. 3. What the authentication identity looks like is undefined. I'm not sure if this is good or bad. 4. The error condition is 'sasl-required'. Does this imply that normal MUC password auth should fail, even with a correct password? And finally, an implementation: http://code.google.com/p/prosody-modules/source/browse/mod_saslauth_muc/mod_saslauth_muc.lua The linked implementation works with Prosody trunk, and verifies that the user knows the room password. This would be far more interesting with some per-user credentials. -- Waqas Hussain
Re: [Standards] Message delivered time
On Thu, Oct 14, 2010 at 7:12 PM, Leon Roy wrote: > Hello, > > We're coding new XMPP clients which report to each other the time a message > is read by the far party. > > > > Would extending the element with urn:xmpp:time from XEP-0202 like > below be valid? > > > from='northumberl...@shakespeare.lit/westminster' > id='richard2-4.1.247' > to='kingrich...@royalty.england.lit/throne'> > My lord, dispatch; read o'er these articles. > > > > > > > > from='kingrich...@royalty.england.lit/throne' > id='bi29sg183b4v' > to='northumberl...@shakespeare.lit/westminster'> > > > > -06:00 > 2006-12-19T17:58:35Z > > > > > best, > -Leon I'd use XEP-0203: Delayed Delivery. from='kingrich...@royalty.england.lit/throne' id='bi29sg183b4v' to='northumberl...@shakespeare.lit/westminster'> That would be because of that 'from' attribute on the element, among other things. -- Waqas Hussain
Re: [Standards] JID matching in XEP-0045
On Wed, Oct 6, 2010 at 6:32 PM, Florian Zeitz wrote: > On 20.09.2010 13:53, Florian Zeitz wrote: >> I'll try to come up with a patch against xep-45 trying to catch all edge >> cases. If only to see how feasible this would be. >> > As promised I tried to change XEP-45 accordingly. > Taking into account my lack of experience with editing XEPs and not > being a native speaker this has the potentially to be horrible, but it's > hopefully still a good start. > > The diff is attached. > > You can find the html and PDF version at: > http://babelmonkeys.de/~florob/xep-0045.html > http://babelmonkeys.de/~florob/xep-0045.pdf > > I'd still like to hear thought on whether you think this is a good idea > (since only 2 people have responded and I talked them into it ;) ). > > Obviously I also welcome any comments/constructive criticism on the > attached patch. > > -- > Florian Zeitz > > I'm +0.5 to allowing to inherit affiliation of . However, changing affiliations from being based on bare JIDs to being based on full JIDs seems pretty significant. What's the rationale? How useful is this? This requires some thinking, and perhaps testing to determine interop issues. I'll also need to review our code to determine how significant a change this would be in Prosody's MUC implementation. This may also affect other XEPs (distributed MUC, etc). I haven't checked yet. -- Waqas Hussain
[Standards] XEP-0114: Jabber Component Protocol: Problem with escaping in the component handshake
In XEP-0114 we have: "1. Concatenate the Stream ID received from the server with the shared secret (if necessary, characters that map to predefined XML entities MUST be escaped according to the rules defined in section 4.6 of the XML specification, and any non-ASCII characters MUST be encoded according to the encoding of XML streams as specified in XMPP Core, i.e., UTF-8 as defined in RFC 3269 [6])." The MUST specified in parenthesis is unnecessary, and all implementations I looked at didn't follow this. Implementations which don't seem to follow this include: 1. Prosody 2. jabberd2 3. ejabberd 4. Tigase 5. Openfire 6. Gloox 7. Spectrum (based on Gloox) 8. Twisted (used by PyICQt, etc) 9. Verse Seeing as how noone follows that MUST, and it's useless anyway, that whole parenthesis should be removed. -- Waqas Hussain
Re: [Standards] JID matching in XEP-0045
On Mon, Sep 20, 2010 at 4:53 PM, Florian Zeitz wrote: > I disagree. The text does not explicitly talk about the JID. Removing > 'example.com' is equivalent to removing "his or her own ownership" for > 'm...@example.com'. However, regardless of the current text: Do you agree > it makes sense for 'm...@example.com' not to be able to remove > 'example.com' if that is the only JID in the owners list? I agree with that. >> As I said before, I'm not against the change (I can certainly see how >> it can be useful). I'd just like implementation behavior in all such >> cases to be explicitly specified, and text which depends on current >> behavior to be updated. My current preference for the ban list is to >> keep bare JIDs only though. >> > I'm still not convinced the current behavior of ban/outcast lists is > "bare JIDs only". Even in your implementation it can also contain domain > parts. While personally I can't see a use-case either, I don't think > matching by full JID is harmful. Also do you think other lists should be > able to contain full JIDs? >From the same section you copied the rules from: "Note: The ban list is always based on a user's bare JID." - http://xmpp.org/extensions/xep-0045.html#modifyban Also, 'outcast' is an affiliation, and "Affiliations are granted, revoked, and maintained based on the user's bare JID." - http://xmpp.org/extensions/xep-0045.html#affil -- Waqas Hussain
Re: [Standards] JID matching in XEP-0045
On Mon, Sep 20, 2010 at 6:41 AM, Florian Zeitz wrote: > On 19.09.2010 04:09, Waqas Hussain wrote: >> >> That's only partially true. Full JIDs are still not allowed. >> > Not as far as I can tell. XEP-45 explicitly says: > > When an entity is banned from a room, an implementation SHOULD match > JIDs in the following order > (these matching rules are the same as those defined for privacy > lists in RFC 3921): > > 1. (only that resource matches) > 2. (any resource matches) > 3. (only that resource matches) > 4. (the domain itself matches, as does any u...@domain or > domain/resource) > > This is BTW not implemented in Prosody, which AFAIR always throws away > the resource. > Hmm, the XEP is inconsistent here. See http://xmpp.org/extensions/xep-0045.html#ban "The ban MUST be performed based on the occupant's bare JID." and "The service MUST add that bare JID to the ban list [...]" I personally don't see much value in banning individual resources. >> This adds a ton of edge cases. I'm not very opposed to the change >> mind, but the complications it adds need to be taken into >> consideration. >> >> With the current behavior, we can make an assumption: All occupants in >> the room are either present in the affiliations' lists, or have >> affiliation='none'. >> >> With the new proposed behavior, we get new cases: >> 1. If 'example.com' is owner, and 'm...@example.com' is an occupant, >> what happens if example.com's affiliation changes? >> 2. If an owner removes affiliation of an occupant, who had affiliation >> inherited from a hostname, what happens? >> 3. If there's a single owner 'example.com'. m...@example.com joins, and >> is an owner. Can it remove example.com from the owners list, given >> that it itself is an owner, and has a different JID? Can it see itself >> in the owners list? >> >> The above list isn't exhaustive. The point is, this wouldn't be a >> simple two-line change. >> > I think that some/most of these issues are solved just by matching in > the right order. > This is possible because a person only has one affiliation at a time. > E.g. If example.com is in the owners list, you can put > evild...@example.com in the member or outcast list to prevent him from > doing anything bad. > > As for the new cases, I don't even think all of the three are new: > 1. Matching bare before domain as in privacy lists m...@example.com would > always be an occupant in this scenario > I meant does m...@example.com's affiliation (as seen by all occupants) change as well? My answer is yes. > 2. Personally I'd like to generally think of it as removing a JID from > one of the lists. As the occupant is never on any list this is > impossible. See above for a possible workaround. However giving someone > on a host that is in one of the list a affiliation of none is not > possible this way... I don't see this as a major issue. > 3. XEP-45 is already quite specific about that. Removing an owner from > the list (including yourself) is only permitted if there are still > owners left after that: > > A service MUST NOT allow an owner to revoke his or her own ownership > privileges if there are no other owners; if an owner attempts to do > this, the service MUST return a error to the owner. > However, a service SHOULD allow an owner to revoke his or her own > ownership privileges if there are other owners. > > So, such an action would be permitted, if there is another entry besides > 'example.com' in the owner list. Going strictly by that text, m...@example.com should be able to remove example.com, since it's not 'his or her' JID. The current text assumes that an owner would always be in the owners' list. > -- > Florian Zeitz > As I said before, I'm not against the change (I can certainly see how it can be useful). I'd just like implementation behavior in all such cases to be explicitly specified, and text which depends on current behavior to be updated. My current preference for the ban list is to keep bare JIDs only though. -- Waqas Hussain
Re: [Standards] BOSH "initial data"
On Sun, Sep 19, 2010 at 7:22 AM, Matthew Wild wrote: > On 19 September 2010 03:11, Waqas Hussain wrote: >> On Wed, Sep 15, 2010 at 8:44 AM, Justin Karneges >> wrote: >>> In trying to reduce the number of network round trips in our plain >>> (non-XMPP) >>> BOSH implementation, I wonder about putting data in the session creation >>> request and response exchanges. In XEP-0124, there is no mention >>> about >>> whether the body must be empty in either case, but all examples show it >>> empty. >>> However, XEP-0206 has an example of the server replying with >>> >>> in the response, so at least that much seems to be allowed, right? I want >>> to >>> know if putting data in the initial client request is allowed too. >>> >> >> It's not disallowed, which I take to mean it's allowed. >> > > I'd rather it be disallowed. It makes no sense for the client to send > anything prior to stream features. I'm certainly in favour of the > server being able to include data in the creation response, that's > just natural (and I thought it was in the XEP already - we have > discussed it before). > I disagree. It makes perfect sense for a client to not wait for stream features if it expects a feature to be supported and is willing to handle the error that would occur if the feature wasn't supported. That's the basic idea behind pipelining (in XMPP and other protocols). This search get us some useful past discussion for TCP, but the idea applies just as well to BOSH: http://jabber.markmail.org/search/?q=pipelining+-bosh There was a long thread from some time back about saving roundtrips, which I can't find it at the moment. HTTP pipelining works the same way (and as discussed on the ML, HTTP pipelining would work fine with BOSH, despite using POST requests). TLS False Start is the same idea applied to TLS: https://tools.ietf.org/html/draft-bmoeller-tls-falsestart-00 Taking it further (but not yet deployed?) is TLS Snap Start, which can reduce TLS roundtrips to zero: http://tools.ietf.org/html/draft-agl-tls-snapstart-00 Take something like Speeqe. This is what the BOSH session might look like: That's five roundtrips reduced to two. It could be reduced to one if the xmpp:restart step wasn't required (I actually don't like this useless requirement; it's pretty much a noop and reminds me of , which we thankfully got rid of). It can also be reduced to one by using HTTP pipelining. While I haven't tested, I suspect this works with Prosody, and xmpp:restart isn't required :) -- Waqas Hussain
Re: [Standards] BOSH "initial data"
On Wed, Sep 15, 2010 at 8:44 AM, Justin Karneges wrote: > In trying to reduce the number of network round trips in our plain (non-XMPP) > BOSH implementation, I wonder about putting data in the session creation > request and response exchanges. In XEP-0124, there is no mention about > whether the body must be empty in either case, but all examples show it empty. > However, XEP-0206 has an example of the server replying with > in the response, so at least that much seems to be allowed, right? I want to > know if putting data in the initial client request is allowed too. > It's not disallowed, which I take to mean it's allowed. > XEP-0124 could probably use some clarification on this, particularly if it is > allowed. > +1 > One caveat of an initial data exchange is that the client doesn't know the sid > yet, and therefore it cannot retransmit requests in the event of failure. > This means you'd only want to include initial data if loss or dups are > acceptable (such as the case with stream:features in XEP-0206). > > Thanks, > -Justin >
Re: [Standards] JID matching in XEP-0045
On Mon, Sep 13, 2010 at 5:20 AM, Florian Zeitz wrote: > On 10.09.2010 19:31, Kevin Smith wrote: >> On Thu, Aug 5, 2010 at 7:50 PM, Florian Zeitz wrote: >>> [This time from the correct account...] >>> >>> Hi, >>> as discussed in jdev@ today XEP-0045 is slightly underspecified and/or >>> different from what people expect in terms of matching JIDs. >>> >>> 1. Roles: (not discussed in jdev@) >>> "Roles are granted, revoked, and maintained based on the occupant's room >>> nickname or full JID" >>> One can join a room with the same nickname from multiple different >>> resources. What happens if different full JIDs have different roles, but >>> the same nick is unspecified (AFAICT). I'd expect this to be strictly >>> nickname based... >> >> Makes sense to me at the moment. >> >>> 2. Affiliations: (discussed in jdev@) >>> Affiliations are supposed to be handled by bare JID, however: >>> a) Outcasts are an exception from this rule and use the same matching as >>> privacy lists. That's only partially true. Full JIDs are still not allowed. >>> b) It's not clear what happens if e.g. a domain is in the list of >>> members. Does that mean only the server itself may join the groupchat, >>> or are all accounts on the server supposed to be able to join the >>> groupchat too. >>> It seems people (including me) mostly expect the JID matching rules >>> specified for privacy list to apply to all forms of affiliation. In fact >>> it seems to be implemented this way in ejabberd. >>> My personal use-case is that I'm hosting a MUC for university purposes >>> on my own server that should be members only, but all users of the >>> university's server should be members. >> >> I think server-JID means all JIDs on the server are the given >> affiliation. I don't see any other way making sense. >> > Which (just to be clear) is exactly what the usual (privacy list) > matching specifies. > So, as a sort of push: > Is there anyone who actually disagrees with this assessment? > Otherwise I'd propose this to be changed in an iteration of XEP-0045 > (also taking into account it already is the status-quo in at least > ejabberd). > This adds a ton of edge cases. I'm not very opposed to the change mind, but the complications it adds need to be taken into consideration. With the current behavior, we can make an assumption: All occupants in the room are either present in the affiliations' lists, or have affiliation='none'. With the new proposed behavior, we get new cases: 1. If 'example.com' is owner, and 'm...@example.com' is an occupant, what happens if example.com's affiliation changes? 2. If an owner removes affiliation of an occupant, who had affiliation inherited from a hostname, what happens? 3. If there's a single owner 'example.com'. m...@example.com joins, and is an owner. Can it remove example.com from the owners list, given that it itself is an owner, and has a different JID? Can it see itself in the owners list? The above list isn't exhaustive. The point is, this wouldn't be a simple two-line change. -- Waqas Hussain
Re: [Standards] MUC vcards
On Tue, Sep 7, 2010 at 8:10 AM, Evgeniy Khramtsov wrote: > We've got a feature request to make it possible to set and get vcards of > multi-user conference. There is no problem with retrieving vcard, but > setting it requires a trick: an owner of a conference can send vcard-temp > request to the conference: > > to='c...@conference.jabber.org'> > > > +1, this is the standard vCard protocol. The fact that the target is a MUC is just a detail. The target could be a host (jabber.org or conference.jabber.org) itself, or pretty much any JID. It should work as long as the sender has the privileges (is a server admin, room owner, etc). I would like the vCard XEP (section 3.2: Updating One's vCard) to explicitly allow authorized entities to update vCards of other entities. > A conference MUST check if the sender is really owner and set the vcard ;) > If the vcard contains PHOTO element, sha1 SHOULD be calculated and SHOULD be > provided in every presence sent from bare JID of the conference. Well, > something like that ;) > Sure. When would the presence from bare JID be sent though? I think it should be in response to a probe. It may make sense to not include the MUC child element in that presence stanza. This works if the user e.g., adds the room in their roster, but not otherwise. > What do you think of it? Is it possible to describe such behaviour in > XEP-0045? Or do you know easier and more correct way to set vcard for a > conference? > I'm not sure if this needs to be in XEP-0045 explicitly. It's simply the logical connection between two basic XMPP building blocks (MUC and vCards), so it should implicitly be assumed to work. -- Waqas Hussain
Re: [Standards] timestamp consistency
On Sat, Jul 17, 2010 at 3:10 AM, Peter Saint-Andre wrote: > On 7/16/10 3:58 PM, Waqas Hussain wrote: > >> I'd love that. Not having a fixed timestamp was one reason last >> activity didn't make it into XEP-0227 (IIRC it was the only data >> servers commonly stored which didn't make it into the current version >> of the XEP). The relative-to-now time it currently uses is awkward in >> various ways. Servers internally store timestamps, not 'seconds', and >> I suspect clients convert 'seconds' to timestamps on receiving last >> activity. > > Thanks for the feedback, Waqas. Now, shall we update XEP-0012 or define > a new extension in a new namespace? > > /psa > I was going to side with updating the existing XEP. Then I looked at XEP-0256: Last Activity in Presence (which is perhaps what most clients actually make use of, rather than the IQ query), and it uses XEP-0203: Delayed Delivery in addition to XEP-0012: Last Activity. Would it make sense to just use Message? -- Waqas
Re: [Standards] timestamp consistency
On Sat, Jul 17, 2010 at 2:39 AM, Peter Saint-Andre wrote: > There's an inconsistency between last activity (XEP-0012) and various > other time-related specs (XEPs 82, 202, 203). Consider: > > id='last1' > to='ro...@montague.net/orchard' > type='result'> > > > > vs. > > from='jul...@capulet.com/balcony' > to='ro...@montague.net/orchard' > id='time_1'> > > -06:00 > 2006-12-19T17:58:35Z > > > > > anon! > xa > 1 > from='jul...@capulet.com/balcony' > stamp='2002-09-10T23:41:07Z'/> > > > We see this clearly in XEP-0256: > > > away > > from='capulet.com' > stamp='2002-09-10T23:41:07Z'/> > > > I wonder if we want to add a UTC timestamp to XEP-0012... > > id='last1' > to='ro...@montague.net/orchard' > type='result'> > seconds='903' > stamp='2002-09-10T23:41:07Z'/> > > > Peter > > -- > Peter Saint-Andre > https://stpeter.im/ > > > I'd love that. Not having a fixed timestamp was one reason last activity didn't make it into XEP-0227 (IIRC it was the only data servers commonly stored which didn't make it into the current version of the XEP). The relative-to-now time it currently uses is awkward in various ways. Servers internally store timestamps, not 'seconds', and I suspect clients convert 'seconds' to timestamps on receiving last activity. -- Waqas
Re: [Standards] XEP-0184: vs. vs.
On Fri, Jun 18, 2010 at 11:20 PM, Konstantin Kozlov wrote: > No. We don't know how messages represented on the other side. It could be > chat windows, pop-ups or whatever author of the application can imagine. So, > in some implementations some (or all) chat state notifications are > meaningless or has limited usability. > Also, analyzing chat state of the other side trying to guess if the user > paid attention to your message is much less convenient than just receive > notification. My mistake. I assumed you were agreeing with Kev. You were suggesting a new element be added to message receipts? As Yann Leboulanger noted, does what you think should do. See http://xmpp.org/extensions/xep-0085.html#definitions >>> >>> As for me, running chat application is for chatting. For reading and >>> writing messages. I can't image a guy who just ignores messages from >>> contacts in his contact list, just confirming without even reading them. >>> At >>> >> >> If you can't imagine that, then you clearly aren't trying. >> > > Believe me, I really did. You mustn't get a lot of messages then. I can get swamped by messages at times. Most of them don't require/expect a response. Also, I'm not always paying attention to the screen, or to the IM client. When you have multiple items requiring focus, some get higher priority than others. That's not unreasonable. >>> >>> least neither me nor any of my friends behave that way. What is he >>> running >>> his IM application for? If he don't want to read message from the >>> specific >>> contact, he can just put it into hist ignore list, and he will never >>> receive >>> any confirmation message. But, once again, confirming reading of the >>> message >>> without reading it actually sounds strange for me. >>> >> >> Not all conversations are equal. Much of it is background chatter, or >> messages which don't actually require a response. It's quite >> convenient following such conversations in a background window, or >> through popups (toasts). > > So, what? So, I don't want to take action to mark those as read immediately, and I don't want the other side to assume that I'm not reading. I am reading them. Just don't want to alt-tab when a response doesn't need to be written. > Your response sounds like either you didn't read my previous messages at > all, or just didn't understood the idea. Yes, as noted above. does what you want. > The idea is: > 1. You don't "have to manually indicate your having read every message you > receive". As I said before, most of the clients already marking just arrived > messages as "new", to tell user, that he has unread messages. Then somehow > they decide that user read the message, the message marked as "read". That > (already existing) mechanism is supposed to be used to send that > confirmation. So, you don't have to do manually. In fact, as a recipient of > the message, you won't notice any changes at all! > I do. Clients mark as "read" on focus. That can happen unintentionally (quickly switching tabs via keyboard). This can sometimes not happen when it should (reading, but without the window/tab focused). > 2. Adding element to the XEP won't change existing behavior at all. > If either of clients do not support element, it won't notice > anything. It will just process notification and following > notification either will not be sent or will be ignored. If both of > them are supporting new XEP, then after receiving notification, > you'll get notification just after message on the other side will > be marked as "read". > Now, tell me, how this extension of the original behavior can break it, > making "awkward, error prone, and of limited utility"? Or how it can make > suffer anyone? > Yes, as noted above: does what you want. -- Waqas Hussain
Re: [Standards] XEP-0184: vs. vs.
On Fri, Jun 18, 2010 at 5:00 PM, Konstantin Kozlov wrote: > Peter Saint-Andre пишет: >> >> This is my point: it is *impossible* for you to know if I have actually >> read a message you have sent me (we discussed this in our IM chat the >> other day). You can know if the message has been delivered to my client >> (and this helps to determine if there are delivery problems along the >> XMPP communication path). You can know if the message has been presented >> in an interface that has focus. You can know if the message has been >> marked-as-read. But you can't know if I have physically looked over all >> the words of your message (or heard all the words in an interface for >> the blind or whatever) and have conceptually absorbed what you have >> written. Even if my client forces me to click some "OK" button to >> acknowledge that I have read and understood each message, I'll just >> click that each time to get rid of the annoying popup. You can't know if >> I've read the message, so why pretend? >> I fully agree with that. The XEP serves a useful purpose as implemented currently in clients. Any definition of "read" wouldn't be feasible and complete at the same time. > > Well. According to my experience, based on knowledge of my behavior and > behavior of my friends, the fact that my message is not "new" anymore on > the other side is enough to assume that user on the other side read it. See XEP-0085: Chat State Notifications. That nicely solves the problem of determining if the other side is paying attention to the conversation. > As for me, running chat application is for chatting. For reading and > writing messages. I can't image a guy who just ignores messages from > contacts in his contact list, just confirming without even reading them. At If you can't imagine that, then you clearly aren't trying. > least neither me nor any of my friends behave that way. What is he running > his IM application for? If he don't want to read message from the specific > contact, he can just put it into hist ignore list, and he will never receive > any confirmation message. But, once again, confirming reading of the message > without reading it actually sounds strange for me. Not all conversations are equal. Much of it is background chatter, or messages which don't actually require a response. It's quite convenient following such conversations in a background window, or through popups (toasts). I absolutely wouldn't want to have to manually indicate my having read every message I receive. > If there are some people, who doing so, that's the problem of those > people. Why other people should suffer because of them? Let's give to other > people such ability, 'cause in the most cases it will be really useful. > XEP-0184: Message Receipts solves a very specific problem for me: It lets me know my message got through, and wasn't swallowed by a network issue or server crash. All clients implementing the XEP do let me be sure of this. Changing this existing behavior would awkward, error prone, and of limited utility: i.e., very unwelcome. A lot of people are on wireless networks, or have otherwise turbulent connections, or have servers which crash. This isn't a rare problem at all. Your useful ability can be achieved through other means (XEP-0085: Chat State Notifications). Don't force the rest of us to suffer please. -- Waqas Hussain
[Standards] Use of to/from attributes in XEP-0077: In-Band Registration inconsistent and underspecified
The operations described in the XEP can be divided into two categories. 1. From an unauthenticated user (request registration form, register) Theses involve sending IQs to the host. What isn't specified is what the host is when the stanza is missing a 'to' attribute (as most of the examples are). The obvious host would be the one specified in the stream tag's 'to' attribute. If both 'to' attributes are missing, a stanza error should be sent. This needs to be specified in the XEP. 2. From an authenticated user (cancel account, change password) For account cancellation, this involves sending IQs with a missing 'to' attribute: "If the entity cancels its registration with its "home" server (i.e., the server at which it has maintained its XMPP account), then the entity SHOULD NOT include a 'from' or 'to' address in the remove request the server SHOULD then return a stream error and terminate all active sessions for the entity." (also, grammar fail in the quoted text) A missing 'to' attribute implies the stanzas are being sent to the user's account (user's bare JID). The IQ responses in the XEP examples however are coming from the host, not the account (from='shakespeare.lit'). This needs to be fixed. For password change, the XEP examples do include a 'to' attribute for the local host (to='shakespeare.lit'), while the server's response is missing a 'from' attribute in some examples, and not in others. This adds to the inconsistency. The examples throughout the XEP need to be made consistent. The client behavior for account cancellation (SHOULD NOT inclde 'from' or 'to' attributes) should apply to password change as well. And it should be clarified that the server should in fact handle jabber:iq:register stanzas to both the user's bare JID, and to the host itself. -- Waqas Hussain
Re: [Standards] UPDATED: XEP-0109 (Out-of-Office Messages)
On Wed, Jun 9, 2010 at 5:51 AM, XMPP Extensions Editor wrote: > Version 0.3 of XEP-0109 (Out-of-Office Messages) has been released. > > Abstract: This document defines an XMPP protocol extension for communicating > out-of-office status. > > Changelog: [See revision history] (rdp) > > Diff: http://xmpp.org/extensions/diff/api/xep/0109/diff/0.2/vs/0.3 > > URL: http://xmpp.org/extensions/xep-0109.html > > The examples show the publish and retraction requests going to 'example.com'. They should really go to 'u...@example.com', or the 'to' attribute shouldn't be specified. PEP is associated with the user's account (n...@host), not the server (host). Also, in example 2, the request should come from the full JID, not the bare JID of the user.
[Standards] Comments on SIFT
end all _allowed_ messages to the
newly available resource
4. for each sent message, DELETE FROM offline_messages WHERE JID ==
${account_jid} and MESSAGEID == $(unique_message_id)
This could be optimized somewhat, but would still be relatively complex.
I've added this here to get comments from other server developers. How
significant is this overhead in your opinion?
3. Automatic IQ responses.
Currently SIFT allows blocking IQs based on payload. The server
auto-reponds with an error. It would be interesting if the server
could be made to reply with an IQ result preset by the client.
Maybe something along these lines:
[...] service discovery reply payload here [...]
The above example has some issues (think service discovery nodes), but
the approach is worth considering regardless. This fits perfectly for
version replies, etc.
4. mod_sift for Prosody
Our implementation is a work in progress, but it does the basics.
Hopefully we'll have some implementation experience soon. Now if only
those client developers hurry up.
Docs: http://code.google.com/p/prosody-modules/wiki/mod_sift
Source:
http://code.google.com/p/prosody-modules/source/browse/mod_sift/mod_sift.lua
--
Waqas Hussain
Re: [Standards] Algorithms and XMPP
On Sun, Feb 28, 2010 at 1:08 AM, Stephan Maka wrote: > Waqas Hussain wrote: >> I propose that we start a small project to act as an aggregator for existing >> open source implementations which could be used as references. Once we have >> that going, an implementation selected for its readability could become the >> (official?) reference implementation. > > I like this idea, as long as reference implementations are published > under a liberal (non-viral) license. > > Still, I want to argue against it because I think the standard specs are > way more important. > I meant them merely as correct examples. And yes, I agree completely about a liberal license. >> 1. It would save people writing new implementations hours and hours of >> guesswork > > Guesswork means the standards miss detail. > They do. Or they are extremely verbose. Or they have bugs. Or they are ignored in favor of shorter tutorials elsewhere which frequently have bugs themselves. >> 2. It would make new implementations more interoperable, reducing the chance >> of mistakes > > Not all implementers will fall back on the reference implementation, > leading to probably less interoperability experience. > The likelihood of two different implementations inter-operating would in my opinion increase. >> 3. It would make existing implementations more visible, improving the chance >> of mistakes being found and reported, and implementations being reused > > Yes, given that feedback works. > Indeed, but I think in XSFs case it would. >> 4. For experimental XEPs this would give direct evidence of how simple or >> complex an algorithm is, what the edge cases are, and if it could be >> simplified without losing its important characteristics > > Good point. > Thanks. I just noticed that algorithms within XEPs have had (sometimes serious) issues which came to light when implementations were being written. >> In fact I wouldn’t mind it being required that any XEP moving beyond >> Experimental have implementations available for the algorithms it defines, >> under a permissive license. >> >> I’m hoping to not be the only one who sees this as a problem we should >> solve. What does everyone else think? > > Alternative proposal: link to all (open) implementations out there, not > neccessarily reference implementations. That could improve interop as > well. As a bonus, hint on the used license. > > The funny thing is, that's my primary proposal. I added text about a formal reference implementation (which I define as a short example piece of code included or linked directly from the spec) as an after thought, but it ended up dominating the rest of the discussion :-) -- Waqas Hussain
Re: [Standards] [jdev] Algorithms and XMPP
On Mon, Feb 22, 2010 at 1:26 PM, Simon Josefsson wrote: > Waqas Hussain writes: > >> There are a number of algorithms an XMPP developer needs to deal with, >> either directly or through a library. Some of these are defined in XEPs, >> while some are external specifications which we work with. >> >> These include: >> >> * DIGEST-MD5 >> * SCRAM >> * Entity capabilities hashing >> * JID escaping >> >> Over the years, I’ve seen people trying to implement these through trial and >> error, and frequently getting them done only partially correctly. After >> helping people fix their DIGEST-MD5 implementations at least a dozen times, >> I think we have a problem. >> >> I propose that we start a small project to act as an aggregator for existing >> open source implementations which could be used as references. Once we have >> that going, an implementation selected for its readability could become the >> (official?) reference implementation. > > I believe maintaining pointers to existing implementations (in various > languages), and publishing interop details between those > implementations, would help more than selecting one implementation as a > "reference" implementation. > This would help a lot. That's basically what I end up doing when tracking down compatibility issues between Prosody and other software (which happens much more frequently than I'd like). > In my experience, selecting one reference implementation have a tendency > to lead to software mono-culture, which eventually may lead to less > interop, in particular between existing deployments and newly written > software from the specification. > > So I'm strongly in favor of helping XMPP implementers find good security > and i18n libraries to use (gsasl and libidn! :)) but I wouldn't support > focusing on just one implementation. > I partially agree. My goal wasn't an implementation which everyone would reuse. It was an example which everyone could look at when they wrote their own implementation. Specifications tend to be difficult to read (thankfully those generated by the XSF are exceptions). Even something like pseudocode or a step by step tutorial would be better than nothing. I originally intended to do just that myself, i.e., write a couple tutorials describing steps in simple enough language that you'll have a hard time getting them wrong, and with warnings against potential pitfalls, but decided to post here to see what others thought. The most frequently used DIGEST-MD5 guide seems to be this: http://web.archive.org/web/20050224191820/http://cataclysm.cx/wip/digest-md5-crash.html I wouldn't be surprised to know that a large number of implementations follow it rather than the RFCs. That's because a large number of implementations tend to have the incorrect behavior it describes... -- Waqas Hussain
[Standards] Algorithms and XMPP
There are a number of algorithms an XMPP developer needs to deal with, either directly or through a library. Some of these are defined in XEPs, while some are external specifications which we work with. These include: * DIGEST-MD5 * SCRAM * Entity capabilities hashing * JID escaping Over the years, I’ve seen people trying to implement these through trial and error, and frequently getting them done only partially correctly. After helping people fix their DIGEST-MD5 implementations at least a dozen times, I think we have a problem. I propose that we start a small project to act as an aggregator for existing open source implementations which could be used as references. Once we have that going, an implementation selected for its readability could become the (official?) reference implementation. What this would achieve: 1. It would save people writing new implementations hours and hours of guesswork 2. It would make new implementations more interoperable, reducing the chance of mistakes 3. It would make existing implementations more visible, improving the chance of mistakes being found and reported, and implementations being reused 4. For experimental XEPs this would give direct evidence of how simple or complex an algorithm is, what the edge cases are, and if it could be simplified without losing its important characteristics In fact I wouldn’t mind it being required that any XEP moving beyond Experimental have implementations available for the algorithms it defines, under a permissive license. I’m hoping to not be the only one who sees this as a problem we should solve. What does everyone else think? -- Waqas Hussain
[Standards] Some issues in XEP-0106: JID Escaping
Guus der Kinderen<http://jabber.markmail.org/message/kihf36azo2kvmczf?q=guus> pointed out some problems in XEP-0106 in the JDev chatroom. Section 1. of the XEP lists nine code points which are forbidden in a node identifier. The character '\' is not in that list, and is used as the escape character in the algorithms described in the XEP. Section 4.3 states: "In order to maintain as much backward compatibility as possible, partial > escape sequences and escape sequences corresponding to characters not on the > list of disallowed characters MUST be ignored." The sequence '\5c' (corresponding to the character '\') as input to the escaping algorithm needs to be escaped. But it isn't a partial escape sequence, and it doesn't correspond to a disallowed character, so the above text dictates that it not be escaped. This breaks the algorithm. A possible correction is to refer to the mapping in tables 1 or 2, instead of the list of disallowed characters. In section 5.1, table 3, the last four examples 9, 10, 11 and 12 do not follow section 4.3 exceptions. The '\' characters in those examples MUST not be escaped, according to section 4.3. Example 12 in that list would be correct with the above suggested correction to section 4.3. -- Waqas Hussain
Re: [Standards] length of disco attributes
On Wed, Sep 23, 2009 at 8:23 PM, Fabio Forno wrote: > AFAIK, since features must be sorted, the only thing you can replace > with an identity is the first feature with the last identity. If we > insert a dummy feature or identity between them the problem could be > avoided (besides possible implementation problems, but a dummy feature > preceding all the others doesn't seem dangerous) > That hack slightly reduces the attack surface, but does not eliminate it. If any of the element's attributes have a '/' in them, they are open to attack despite the dummy feature. The thread on the security ML talks about several related attacks, of which this one is probably the most minor. On Wed, Sep 23, 2009 at 8:28 PM, Peter Saint-Andre wrote: > On 9/23/09 8:05 AM, Waqas Hussain wrote: > > > Sure, but I see no point in implementations actually _failing_ on > > receiving them. If my code works correctly with valid implementations, > > and my code can also work with some broken implementations, I don't see > > much reason to add extra validation code just to stop working with > > broken implementations (unless Prosody is running in strict mode of > > course ;) ). > > This is a MUST on generation of service discovery identities in an IQ > result. There is no error to return on receipt of an IQ result. Yes, of course. I was referring to caching the verification string. It seems safe enough to cache even if the category and type are empty. -- Waqas Hussain
Re: [Standards] length of disco attributes
On Wed, Sep 23, 2009 at 6:45 PM, Peter Saint-Andre wrote: > Primarily, zero-length categories and types are useless in service > discovery. So I think that we need to change the disco spec itself > anyway. I am *not* saying that this modification would fix all security > problems in XEP-0115. That's fine then. > > In its current form, the hashing function always succeeds for any given > > non-null input. This is desirable because it simplifies implementations, > > and is exactly the same as popular hashing functions (MD5, SHA, etc). > > Specifying minimum lengths is fine, but is there a reason for receiving > > implementations to actually enforce these limits? > > Because zero-length categories and types are useless. Sure, but I see no point in implementations actually _failing_ on receiving them. If my code works correctly with valid implementations, and my code can also work with some broken implementations, I don't see much reason to add extra validation code just to stop working with broken implementations (unless Prosody is running in strict mode of course ;) ). > > The caps algorithm in XEP-0115 actually talks about missing 'type' > > attributes. This ought to be fixed. > > That's a spec bug in XEP-0115, because 'type' is a MUST in XEP-0030. > > Peter > -- Waqas Hussain
Re: [Standards] length of disco attributes
On Wed, Sep 23, 2009 at 3:04 AM, Peter Saint-Andre wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > XEP-0030 allows the 'category' and 'type' attributes to have any length, > including zero. This opens the door to certain attacks in entity > capabilities (see the recent discussion on the secur...@xmpp.org list) > and in any case I think it is not a good idea (is there any meaning to a > zero-length category or type?). Also, we need to harmonize the 'jid' > attribute in disco with rfc3920bis. I propose the following: > > 1. 'category' shall have a minimum length of 1 > > 2. 'type' shall have a minimum length of 1 > > 3. 'jid' shall be a length between 1 and 3071 (see 3920bis) > > Peter > > - -- > Peter Saint-Andre > https://stpeter.im/ > > > -BEGIN PGP SIGNATURE- > Version: GnuPG v1.4.8 (Darwin) > Comment: Using GnuPG with Mozilla - http://enigmail.mozdev.org/ > > iEYEARECAAYFAkq5SdMACgkQNL8k5A2w/vx+nQCgsIQ5LAYHoUQ14dtCrf6mbVG/ > shcAnj+i73sM80zRVUIrD5MkPJeiR6yG > =FAvy > -END PGP SIGNATURE- > Quoting from one of my messages on the security list: can still be replaced by which can be replaced by Therefore, the security benefit of requiring minimum lengths is questionable. In its current form, the hashing function always succeeds for any given non-null input. This is desirable because it simplifies implementations, and is exactly the same as popular hashing functions (MD5, SHA, etc). Specifying minimum lengths is fine, but is there a reason for receiving implementations to actually enforce these limits? The caps algorithm in XEP-0115 actually talks about missing 'type' attributes. This ought to be fixed. -- Waqas Hussain
Re: [Standards] LAST CALL: XEP-0227 (Portable Import/Export Format for XMPP-IM Servers)
On Thu, Sep 10, 2009 at 7:07 AM, Peter Saint-Andre wrote: > -BEGIN PGP SIGNED MESSAGE- > Hash: SHA1 > > On 8/5/09 3:22 PM, XMPP Extensions Editor wrote: > > This message constitutes notice of a Last Call for comments on > > XEP-0227 (Portable Import/Export Format for XMPP-IM Servers). > > Given the feedback received during this Last Call, it is clear that the > XEP will need to be revised, followed by a second Last Call to be issued > by the XMPP Council. I have communicated via IM with Magnus Henoch > (author of XEP-0227) and he does not have time to work on the necessary > revisions right now. Would someone else like to volunteer to help out? > Your co-authorship could lead to eternal fame. :) > > Peter > I wouldn't mind helping out with this. Summarizing the list and off-list discussion, the changes which need to be made are: 1. Inclusion of incoming presence subscriptions. ... 2. Inclusion of persistent PEP nodes. ... ... 3. Privacy lists. ... 4. The problem of XIncludes in user data. One solution would be to forbid processing of XIncludes which are more than three levels deep in the document. Text about this needs to be added to the XIncludes section, and the Security Considerations section. -- Waqas Hussain
Re: [Standards] LAST CALL: XEP-0244 (IO Data)
On Wed, Sep 9, 2009 at 10:58 PM, Peter Saint-Andre wrote:
> -BEGIN PGP SIGNED MESSAGE-
> Hash: SHA1
>
> On 9/6/09 12:28 PM, Waqas Hussain wrote:
>
> > Now with the short answers out of the way, I do have some concerns.
> >
> > The specification does three things: One, it defines a way to do ad-hoc
> > commands asynchronously (possibly across user sessions). Two, it defines
> > a new payload format for ad-hoc commands. Three, it defines a schema
> > discovery format. I think at least the asynchronous ad-hoc command part
> > deserves to be separate from the other two.
> >
> > What I would prefer:
> >
> > 1. An Asynchronous Ad-Hoc Commands XEP.
>
> I agree that it makes sense to split the asynchronicity stuff out of
> XEP-0244 into a separate spec -- perhaps a new XEP but I think
> preferably a revised version of XEP-0050 so that everything about the
> core of ad-hoc commands is in one specification.
A revised XEP-0050 sounds fine.
> 2. The IO Data custom payload/schema format in an IO Data XEP, but as
> > payload for ad-hoc commands, not just asynchronous ad-hoc commands.
>
> IMHO this can remain in XEP-0244.
Agreed.
> > 3. Other formats in the future (SOAP, whatever). I would love a WSDL
> > discovery XEP.
>
> Wouldn't those be separate specs?
Yes, I did mean separate specs. I should have been clearer.
> Some problems with the XEP:
> >
> > 1. The XEP basically allows a service to expose a set of global
> > functions. There is no possibility for function namespaces. This is
> > similar to the SOAP over XMPP XEP, where you can have only one service
> > end point per-JID (which I consider a non-minor deficiency).
>
> It's not yet clear to me (from this message or your reply to Egon) what
> changes you want to make. Could you perhaps suggest some text?
On further consideration I can see that the node values can consist of any
UTF-8 characters. So I can have nodes 'physics/calculate_distance' and
'chemistry/balance_equation' on the same JID (the point being the hierarchy
denoted by the '/'). I'm not sure if a guideline about this in the XEP for
IO Data consumers, and possibly a preferred hierarchy indicator ('/' or '.')
would be useful. This is just a convention though, to help in mapping
existing APIs directly to IO Data.
> 2. The schemata discovery protocol requires at least two IQ requests per
> > exposed function. For a service exposing a large number of functions,
> > this makes it impractical to use for generating marshalling code at
> > runtime. And there’s no allowance for caching the schemas (i.e., you
> > have to load all of them every time, since there is no assurance that it
> > didn’t change since you last checked). For something like Prosody, where
> > we might be exposing hundreds of functions, which can appear and
> > disappear when modules (including third-party modules) are
> > loaded/unloaded, this makes for some nastiness.
>
> As far I understand your concern, this is more a matter of versioning
> the payload namespaces than exactly a problem for XEP-0244, but perhaps
> I'm missing something.
>
My first concern here is scalability. The schemata protocol does not scale
well. As an analogy, think of the roster. Think what it would be like if the
roster protocol required that after retrieving the roster, you must send
separate IQs to discover meta-data (subscription, nick, etc) and disco#info
data for each individual roster item (2 IQ requests per roster item). How
well would that scale for large rosters (without roster versioning and caps
hashes)?
Continuing with the roster analogy, Egon Willighagen basically says that the
roster and the item meta-data and disco#info data should all be static for a
given service. Now, this doesn't make any sense for dynamic web services.
But let's assume the XEP doesn't cater to those. But even static web
services need to evolve. Any professional web service with a static contract
that you are likely to find is versioned (e.g., see Google's various AJAX
APIs).
And no, this isn't a problem of versioning of payload namespaces. The
payload is not the API. It is data on which the API works. Continuing with
the roster analogy, the payload namespace version is like caps has of the
disco#info data of a single roster item. But I'm referring to versioning the
roster itself. (Since the disco#info and item meta-data is local to the IO
Data service, the roster's version could simply encompass changes in all of
these, eliminating full roster retrieval, and two IQs per roster item to
refresh item data).
Now, I'm not stuck on this. The schemata protocol still serves its intended
purpose of API discovery. It's just that the lack of scalability, and the
(in my opinion, unreasonable) requirement of a never changing API contract
limits its usefulness.
Thanks for your feedback.
>
> Peter
>
> - --
> Peter Saint-Andre
> https://stpeter.im/
--
Waqas Hussain
Re: [Standards] LAST CALL: XEP-0244 (IO Data)
On Mon, Sep 7, 2009 at 12:12 AM, Egon Willighagen <
egon.willigha...@gmail.com> wrote:
> Dear Wasaq,
>
> On Sun, Sep 6, 2009 at 8:28 PM, Waqas Hussain wrote:
> > Some problems with the XEP:
> >
> > 1. The XEP basically allows a service to expose a set of global
> functions.
> > There is no possibility for function namespaces. This is similar to the
> SOAP
> > over XMPP XEP, where you can have only one service end point per-JID
> (which
> > I consider a non-minor deficiency).
>
> Can you elaborate on this? If you run a service discovery on
> ws1.bmc.uu.se you will see various JIDs, each with one and often more
> functions...
I mean the basic concept of namespaces (the concept of a class is similar,
if you only consider static members). Basically, instead of GlobalFunc1,
GlobalFunc2, and so on, SomeNamespace.Func1, SomeNamespace.Func2, and
possibly SomeOtherNamespace.Func1 (same function name, possibly with a
different meaning). I'm using the '.' operator here, but others ('::', '->',
'/', etc) are used in other languages and contexts to show membership.
Why this is useful should be obvious. The reason is the same as the reason
behind XML namespaces, and Java packages, and C++ namespaces. Even VB6 has
modules. Your example of various components running under a host implies
that you are only thinking of servers exposing services, and using
components to group them. But why really shouldn't a component, or even a
single client resource expose multiple services? The protocol might as well
be generic.
Note that pretty much any modern ORB system has support for these. Even
CORBA exposes an object hierarchy, not a single object per-host. See
http://en.wikipedia.org/wiki/Object_request_broker - I'm personally familiar
with .NET Remoting, WCF, DCOM, RMI, and various RPC mechanisms (SOAP, REST,
etc), etc. Exposing hierarchies of objects, and not just a single object
per-host looks like a pretty standard feature.
Note that if you simply don't want to support that in IO Data, then please
clarify the reasons, because that's a pretty useful feature.
> 2. The schemata discovery protocol requires at least two IQ requests per
> > exposed function. For a service exposing a large number of functions,
> this
> > makes it impractical to use for generating marshalling code at runtime.
>
> I do not understand how you see the number of needed IQ requests to
> make generation of marshalling code at runtime difficult. Could you
> please explain?
Think of a dynamically generated UI. A UI generated from the collection of
schemata exposed by an IO Data service. All that is being generated on the
fly. Now assume the service has a hundred functions. Do you suppose the 200+
IQ requests required to generate said UI might slow things down a bit?
Unless of course IO Data clients are supposed to be statically generated. If
that is the case, then it needs to be explicitly stated.
> And
> > there’s no allowance for caching the schemas (i.e., you have to load all
> of
> > them every time, since there is no assurance that it didn’t change since
> you
> > last checked). For something like Prosody, where we might be exposing
> > hundreds of functions, which can appear and disappear when modules
> > (including third-party modules) are loaded/unloaded, this makes for some
> > nastiness.
>
> As far as I know there is no standard for versioning on XML schemata,
> other than with the schema itself. Indeed, the current proposal does
> not propose a mechanism for doing this,
> and defines the schemata itself is 'contract' for the service interaction.
>
That is true, but various parts of XMPP do have support for versioning.
Roster versioning and entity caps in presence being examples. In my opinion,
something which requires multiple IQ requests to refresh is more in need of
versioning that either of those.
I do understand that if you wish the change that contract regularly
> *and *you wish the services to keep the same JID, that this places
> stress on the clients. However, I am not sure that the XEP should
> provide best practices for dealing with service providers with such
> wishes.
>
So.. basically once you have defined a contract, it shouldn't ever change?
API versioning is a pretty basic and quite useful concept. All APIs get
updated. Some more frequently than others. Pick up any modern API, and you
do see some versioning support.
If IO Data is to be limited to static services, which never change, that
should really be explicitly stated. Otherwise it needs a better versioning
system than re-querying a hundred nodes.
If we could get to, or reuse, a different XEP for specifying detailed
> versioning information for schemata, this would have been useful
> indeed. But in th
Re: [Standards] LAST CALL: XEP-0244 (IO Data)
Some short answers: > 1. Is this specification needed to fill gaps in the XMPP protocol stack or to clarify an existing protocol? Yes. > 2. Does the specification solve the problem stated in the introduction and requirements? Yes. > 3. Do you plan to implement this specification in your code? If not, why not? Yes. > 4. Do you have any security concerns related to this specification? No. > 5. Is the specification accurate and clearly written? Yes. Now with the short answers out of the way, I do have some concerns. The specification does three things: One, it defines a way to do ad-hoc commands asynchronously (possibly across user sessions). Two, it defines a new payload format for ad-hoc commands. Three, it defines a schema discovery format. I think at least the asynchronous ad-hoc command part deserves to be separate from the other two. What I would prefer: 1. An Asynchronous Ad-Hoc Commands XEP. 2. The IO Data custom payload/schema format in an IO Data XEP, but as payload for ad-hoc commands, not just asynchronous ad-hoc commands. 3. Other formats in the future (SOAP, whatever). I would love a WSDL discovery XEP. Some problems with the XEP: 1. The XEP basically allows a service to expose a set of global functions. There is no possibility for function namespaces. This is similar to the SOAP over XMPP XEP, where you can have only one service end point per-JID (which I consider a non-minor deficiency). 2. The schemata discovery protocol requires at least two IQ requests per exposed function. For a service exposing a large number of functions, this makes it impractical to use for generating marshalling code at runtime. And there’s no allowance for caching the schemas (i.e., you have to load all of them every time, since there is no assurance that it didn’t change since you last checked). For something like Prosody, where we might be exposing hundreds of functions, which can appear and disappear when modules (including third-party modules) are loaded/unloaded, this makes for some nastiness. -- Waqas Hussain
Re: [Standards] Call for Experience: XEP-0203 (Delayed Delivery)
> 1. Who has implemented XEP-0203? Please note that the protocol must be > implemented in at least two separate codebases (and preferably more). First used in Prosody in November 2008 (before Prosody 0.1 was released). > 3. Is the text of XEP-0203 clear and unambiguous? Are more examples > needed? Is the conformance language (MAY/SHOULD/MUST) appropriate? Have > developers found the text confusing at all? Please describe any > suggestions you have for improving the text. Some clarification about ordered delivery might be nice. Should implementations assume that delayed messages from an entity would always be ordered by timestamp? A specific case I'm thinking of is history sync for distributed MUC rooms (of course, that has additional quirks and concerns). -- Waqas Hussain
Re: [Standards] Call for Experience: XEP-0202 (Entity Time)
> 1. Who has implemented XEP-0202? Please note that the protocol must be > implemented in at least two separate codebases (and preferably more). Prosody has had a mod_time since November 2008 (before Prosody 0.1 was released).
Re: [Standards] LAST CALL: XEP-0227 (Portable Import/Export Format for XMPP-IM Servers)
> 1. Is this specification needed to fill gaps in the XMPP protocol stack or to > clarify an existing protocol? Yes. > 2. Does the specification solve the problem stated in the introduction and > requirements? A missing piece of data is incoming subscriptions, for which there is no roster entry. I suggest adding a section for in the XEP. It might be appropriate for the XEP to explicitly acknowledge that implementations may extend the format by including custom elements and attributes (qualified by appropriate namespaces). Off the top of my head, some extensions I would like to use (listed here for my reference, if no one else's): - Privacy lists, i.e., under . - Last activity information. Possibly a under . - Other user meta-data. For example, marking a user as an admin: under , or maybe . - PEP data under . - PubSub and MUC data (out of scope of the XEP, yes, but no reason not to export and import it). - Server and host configuration data under and (yes, implementation specific, but still useful). - Replacing the 'password' attribute on the element with something else when using hashed passwords, certificates, etc. About the XInclude support, I like how the examples use namespace prefixes on attributes. I think the SOAP over XMPP XEP is the only other one with examples using those. > 3. Do you plan to implement this specification in your code? If not, why not? Yes, I plan to write an exporter and importer for Prosody. It would be interesting to have a Prosody storage backend which uses the XEP-0227 format directly. It would work fine for small installations. > 4. Do you have any security concerns related to this specification? Need to be careful about XInclude, but nothing else in particular. > 5. Is the specification accurate and clearly written? Yes. > Your feedback is appreciated! Good. -- Waqas Hussain
Re: [Standards] XEP-0054: vcard-temp
On Wed, Jul 29, 2009 at 11:14 PM, Daniel Willmann < dan...@totalueberwachung.de> wrote: > Hi, > > I just noticed a discrepancy between handling of the ext. address field > in vcards. > > The XEP specifies that the value should be saved in tags and > this is what jabberd2 seems to be expecting. > Both Psi and Gajim though seem to be using . This doesn't > bother ejabberd since it just saves the xml, but jabberd2 (at least > with the mysql storage) stores the data in individual fields and so > ignores . > > Now I just want to make sure that EXTADD is indeed correct an I'll file > the appropriate bugs at the Psi and Gajim bugtracker. > > Regards, > Daniel Willmann > EXTADD is correct. EXTADR is not. Neither the vcard-temp XEP, nor draft-dawson-vcard-xml-dtd-01 have any mention of EXTADR. -- Waqas Hussain
Re: [Standards] SIFT
On Fri, May 15, 2009 at 8:15 AM, Peter Saint-Andre wrote: >> The namespace seems to have retained its roots instead of using sift. > > Not sure what you mean here. > s/urn:xmpp:dc:0/urn:xmpp:sift:0/ under http://xmpp.org/extensions/inbox/sift.html#registrar-ns
Re: [Standards] UPDATED: XEP-0237 (Roster Versioning)
2009/5/13 Jiří Zárevúcký : > If it is a hash of a complete roster (as Peter has told) then the > server would have to decode this hash, figure out exactly what version > that was, create a difference, figure out the version for every > change, and send every change with the appropriate full roster > checksum again. I'm not that much into cryptography, and it depends on > the kind of hashing used, but if that possible, someone implementing > this would at least be completely out of his/her mind. > I don't think so. Using hashes is quite practical. It wouldn't be much more complicated than using sequence numbers in fact, and not inefficient either. Depending on the implementation, the server itself can treat the ver value as opaque, except for when it is computing a new one (which is a relatively rare operation). Think about it more, and feel free to talk to me if you can't see how. >> The XEP is short and clear. The 'ver' attribute is an opaque string >> for clients, and client programmers shouldn't care what it's value is. >> I don't see a problem here. >> > > XEP is short and clear and nobody will ever have any problem reading > it. Examples are way more confusing because of some theoretical idiot > that's implementing example and not the XEP. Do we really want to make > obscure examples because of people that are probably incapable of > implementing it at all? > I for one don't see the examples as obscure at all. -- Waqas Hussain
Re: [Standards] UPDATED: XEP-0237 (Roster Versioning)
2009/5/13 Jiří Zárevúcký :
> 2009/5/12 Peter Saint-Andre :
>> -BEGIN PGP SIGNED MESSAGE-
>> Hash: SHA1
>>
>> On 5/9/09 12:52 PM, Jiří Zárevúcký wrote:
>>> I have read the text again after a while and the following text got my
>>> attention:
>>>
>>>> If a series of roster modifications result in a roster item that does not
>>>> differ from the
>>>> version cached by the client (e.g., a modification to the item's 'name'
>>>> attribute and
>>>> then a modification back to the original value), the server SHOULD NOT
>>>> consider the
>>>> item to have been modified for purposes of roster versioning and therefore
>>>> SHOULD NOT
>>>> push the item to the client in an interim roster push.
>>>
>>> That part is not very useful, since servers will hardly keep track of
>>> every previous roster version. I don't even want to think about the
>>> CPU overhead that will kill the cluster every 30 minutes (that's a
>>> hyperbole of course). > The simplest "full featured" implementation
>>> consists of integer version numbers and "mver" field for every roster
>>> item.
>>
>> This is purely an implementation decision and I think we need to remove
>> the conformance language here (i.e., no MUST, SHOULD, SHOULD NOT, MAY).
>> If the server chooses the "hash complete roster" approach then it won't
>> send the push. If the server chooses the "integer version numbers"
>> approach then it will send the push. End of story.
>>
>>> Server will then send all the items whose mver is greater than
>>> client's ver. Any throughout checking would hardly ever have any
>>> benefit.
>>
>> What is mver?
>>
>
> The same relation as with time and mtime on files. It's the version of
> last change.
>
>>> -
>>>
>>> About the opaque vs. integer examples dilemma. I think if someone
>>> really doesn't read the text, his implementation won't work at all
>>> anyway. The examples are kinda confusing this way. If you really want
>>> some opaque strings there, use "A", "B", "C" or something
>>> like that, so the sequentiality is obvious.
>>
>> A sequence number is not really opaque, is it? The examples currently
>> use the "hash complete roster" approach. I would prefer to err on the
>> side of not misleading clients about sequentiality ("the examples have
>> A and B so I suppose the next 'ver' will be C" and then the
>> client breaks when that's not the case).
>>
>> Peter
>>
>
> Ok, so in order to not confuse stupid programmers with a specific
> implementation, we will confuse them with one entirely impossible. I
> get your thinking *THUMBS UP*
>
Programmers for whom ver='qAxdnWNcA+lYf7CoN5wpBsvVVno=' would be
entirely impossible... I think those exact programmers are the reason
for not using ver='1' :-)
The XEP is short and clear. The 'ver' attribute is an opaque string
for clients, and client programmers shouldn't care what it's value is.
I don't see a problem here.
--
Waqas Hussain
Re: [Standards] Privacy lists and the order of items
On Tue, May 12, 2009 at 4:40 AM, Justin Karneges wrote: > On Monday 11 May 2009 16:28:00 Waqas Hussain wrote: >> On Mon, May 11, 2009 at 7:10 PM, Peter Saint-Andre > wrote: >> > Can you trust the order of items? >> >> Err, explain to me why you wouldn't. Order of nodes (except attributes >> on an element) is significant in XML. > > I've heard that some XMPP codebases out there may (stupidly?) store a stanza's > element hierarchy in a "hash" tree or other, which may not maintain element > order. So the XML is parsed properly, and stanzas are processed in order, > but elements within a stanza may not be processed in order because the > ordering is lost during the dom->hash transition. > Making life difficult for everyone else because of (stupid?) codebases which may be out there... Said codebases are not based on any standard API (DOM, SAX, etc) for one, and wouldn't work correctly for many of the existing XEPs anyway. > I want to say at least one such codebase was using Perl, where it's > commonplace to use these hash trees for everything. Not that I'm saying it's > right... > So.. since the codebase you are referring to uses hashtables, and therefore can't handle two children with the same name (or whatever hash key they are using), XMPP would never allow that to happen? (Several specs do in fact allow it to happen, some might even require it.) And if we are saving codebases, there's XML and UTF-8 and stringprepping! I'm sure we would save a LOT of codebases out there, if we dropped the valid XML requirement, etc. Look at HTML 5. They have added polite handling guidelines for "badly-formed code" to the specification itself, so we might as well too... This may appear snarky, but I'm mostly somewhere between bemused and amused. > -Justin > -- Waqas Hussain
Re: [Standards] Privacy lists and the order of items
On Mon, May 11, 2009 at 7:10 PM, Peter Saint-Andre wrote: > On 5/11/09 1:59 AM, Remko Tronçon wrote: >>> Also, I'm wondering why the order attribute is used on privacy lists' >>> items, instead of using the implicit order of the items. >> >> I always wondered that myself. I assume it's historical baggage. A >> pity though, because it makes things needlessly complicated to >> implement on both client and server side. > > Can you trust the order of items? > Err, explain to me why you wouldn't. Order of nodes (except attributes on an element) is significant in XML. An XML vocabulary may not make use of it, but I don't see why you wouldn't want to in this case. I don't know of a single XML-traversal API that wouldn't let you iterate over the items in document-order (indeed, in pretty much all, that's either the only, or the most convenient traversal method). If the decision to use the order attribute was only due to lack of trust of the intrinsic item order, then the answer to (a) in my original post is yes, and maybe to (b) too. Right? By the way, if you can't trust the order of your XML, the various XMPP specs' dependency on in-order processing would be affected. Sending multiple stanzas in a single TCP/BOSH packet would start having consequences. > Peter > > - -- > Peter Saint-Andre > https://stpeter.im/ > -- Waqas Hussain
[Standards] Privacy lists and the order of items
If a client adds/edits a privacy list a) may a server reorder the list's item elements (sorted by the order attribute for example)? That is, the client saves but on retrieval gets b) may a server change the value of the order attributes (making them consecutive for example)? That is, the client saves but on retrieval gets In all of the above, the lists are equivalent, but the XML is not. Also, I'm wondering why the order attribute is used on privacy lists' items, instead of using the implicit order of the items. Am I missing something, or is this historical baggage? I don't see anything gained by explicitly specifying order. Part of some discarded list editing protocol maybe? -- Waqas Hussain
Re: [Standards] Presence distribution
What follows might just be nonsense, but I'm still speaking out. For the trees, you understand. On Mon, Apr 6, 2009 at 5:19 PM, Dave Cridland wrote: [snip] > You want to replace 2-4 with: > > 2') Home server collates roster by remote domain and emits one presence > stanza per remote domain which has a subscribed contacts known to be > available. This is, I'll accept, likely to be close to the energy cost of 2, > although due to the fluctuating nature of the final clause that collation > has to be done each time, so it'll be a little above. Cost E2' is, therefore > > E2. > O(n) + O(d) - I have a feeling this can be reduced to O(n). > Keep a list of domains (affectively a cache) to send to along with the roster object (possibly even storing it on disk along with the roster), and E2' becomes less than E2. > 3') encode/transmit/decode 1 stanza per remote domain. This is certainly an > energy/cpu/cost saving compared to the 3 above, no argument from me here. > Cost E3' is < E3. O(d) (One stanza per domain. Still linear, of course). > > 4') Lookup sender against all rosters in the system, and detirmine which of > the resulting potential recipients is online and available to the sender. It > seems reasonable that it would be possible to limit the lookup to only > contacts who are online and available - assuming we ignore privacy lists - > but you're still adding a lookup and the associating lookup mechanics (like > a hash table or something) which must be maintained in-memory. I strongly > suspect that this much, much more costly to use, build, and maintain than 4 > above, hence E4' >> E4. I believe this to be possible to implement as > O(log(y)), but I also suspect it's overwhemlingly more likely that it's O(y) > (as derived from a reverse roster lookup O(log(y)) combined with a > resource-broadcast lookup O(y)). > I don't think E4' >> E4. I think E4' < E4. "you're still adding a lookup" you say. No, this replaces y hashtable lookups with one lookup. Think about it. While yes, a hashtable would indeed need to be maintained, the resource usage wouldn't be significant. The additional memory cost would be an order of magnitude less than what in-memory rosters already use. The hashtable update/remove cost would only apply when local resources become unavailable/available, which tends to happen only once for most c2s sessions. Presence broadcast on the other hand tends to happen more than once (at least that's what it looks like for my contacts). Simply stated, writes would be (much?) rarer than reads, and read cost would be reduced by a reverse hashtable (think of database indexes). > This give E' as O(n) + O(d) + O(y), a linear compexity algorithm. Poof goes > your argument above about linear complexity versus constant amortized time. > The Big-O notation is about complexity, not cost. While two different algorithms may both have the same complexity, that does not imply that the resource cost is the same, or even close. Several optimizations reduce cost, but not complexity (making certain boolean checks unnecessary, for example). And since the topic is trees, the resources being consumed by the intermediate nodes (routers, etc), should be added to the equation. > I can't see (E3' - E3) < ((E4' - E4) + (E2' - E2)) as being at all likely, > so I continue to maintain that this is a false optimization. (I also > continue to maintain that this is an arse-clenchingly more complex solution > to the problem of getting presence from one contact to another, but I hope > nobody's arguing against me, there). > > Now, as always, I encourage you to change my mind with suitably backed > factual evidence. > I suspect the CPU cost would be lower with this proposal. Saved bandwidth was more appealing to me than saved CPU time, but the missing unsubscribed issue makes this optimization invalid, so no point in attempting tests. > Oh, and I would add that this does, of course, change dramatically if we > introduce a non-meshed routing architecture between servers - since that > reassociates responsibility anyway, it's not a problem there. (FWIW, I would > note that this already occurs between client and home server). > > Dave. > -- > Dave Cridland - mailto:d...@cridland.net - xmpp:d...@dave.cridland.net > - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/ > - http://dave.cridland.net/ > Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade > -- Waqas Hussain
Re: [Standards] Inconsistent Subscriptions in XMPP
On Thu, Mar 5, 2009 at 5:37 PM, Pedro Melo wrote: > Hi, > > On Feb 28, 2009, at 10:18 AM, Waqas Hussain wrote: >> >> On Thu, Feb 26, 2009 at 9:32 AM, Pedro Melo wrote: >>> >>> On Feb 25, 2009, at 7:36 PM, Pavel Simerda wrote: >>>> >>>> On Tue, 24 Feb 2009 15:54:38 + >>>> Pedro Melo wrote: >>>> >>>>> Hi, >>>>> >>>>> On Feb 24, 2009, at 12:49 AM, Pavel Simerda wrote: >>>>> >>>>>> There are several cases when subscription databases in XMPP are >>>>>> inconsistent. >>>>>> >>>>>> You may view subscription information as a global distributed >>>>>> database. >>>>>> Subscription state between two JIDs, for example a...@a and b...@b are >>>>>> stored >>>>>> in two places at the same time. Servers A and B maintain their own >>>>>> copies of subscription state. >>>>> >>>>> [] >>>>> >>>>>> What with the roster items that are inconsistent? >>>>>> >>>>>> * Mark as inconsistent, let the client present it to the user to >>>>>> take action. >>>>>> >>>>>> * Auto-repair and thus maintain consistency >>>>>> >>>>>> Looking forward to all feedback. >>>>> >>>>> When you send out a include the local >>>>> "view" of the subscription state. >>>> >>>> Btw presence probe seems too weak... as it doesn't reveal full >>>> subscription state. >>> >>> that's what I'm saying: include the full subscription state in the >>> presence >>> probe so that the other side can detect mis-matches. >>> >>> Best regards, >>> >>> >> >> I'm considering doing this in Prosody: >> > to="y...@yourhost.com"> > > Don't know if you should also include the ask-out/ask-in flag. Also original > request . > > I would, but I admit not having done some serious thinking on the > implications. It *seems* ok. > > >> What action is appropriate is open for debate. What should the >> resulting state be? The lowest common permissions (possibly sending >> unsubscribe[d] to the contact or changing the user's subscription for >> contact)? The highest common permissions (possibly sending a >> subscrive[d] to the contact and changing the user's subscription for >> the contact)? > > Highest common permissions, maybe. I've started a table below for some > cases, and I have some doubts. Should we upgrade the Receiver subscription > to a better state? For example: if you have a To subscription to me and I > have a None to you, should I be upgraded to a From? Not sure yet. It can be > used for presence spam. A more careful approach would send a unsubscribe. > > And we need to look this in the other direction. If the Receiver is 'To' or > 'Both' and the other side is 'None' or 'To', should we send a 'subscribed'? > It makes sense for the Receiver, but from the POV of the Sender, you are > modifying my own subscription status, upgrading it. > > I wrote the following table of what I think are the most safe transactions: > none of the subscriptions are "upgraded" in any way. > > (note: for each "Recv resets to 'X'" it is implied that a roster push would > be sent to all connected resources) > > Sender: none > Recv: to > > => Recv resets to 'none' > > > Sender: none > Recv: from > > => Recv resets to 'none' > > > Sender: none > Recv: Both > > => Recv resets to 'none' > > > Sender: To > Recv: none > > => Recv sends 'unsubscribe' > > > Sender: To > Recv: To > > => Recv resets to 'none', sends 'unsubscribe' > > > Sender: To > Recv: Both > > => Recv resets to 'From' > > > Sender: From > Recv: none > > => Recv sends 'unsubscribed' > > > Sender: From > Recv: From > > => Recv resets to 'none', sends 'unsubscribed' > > > Sender: From > Recv: Both > > => Recv resets to 'To' > > > Sender: Both > Recv: none > > => Recv sends 'unsubscribe' and 'unsubscribed' > > > Sender: From > Recv: From > > => Recv resets to &#
Re: [Standards] Inconsistent Subscriptions in XMPP
On Thu, Feb 26, 2009 at 9:32 AM, Pedro Melo wrote: > > On Feb 25, 2009, at 7:36 PM, Pavel Simerda wrote: > >> On Tue, 24 Feb 2009 15:54:38 + >> Pedro Melo wrote: >> >>> Hi, >>> >>> On Feb 24, 2009, at 12:49 AM, Pavel Simerda wrote: >>> >>>> There are several cases when subscription databases in XMPP are >>>> inconsistent. >>>> >>>> You may view subscription information as a global distributed >>>> database. >>>> Subscription state between two JIDs, for example a...@a and b...@b are >>>> stored >>>> in two places at the same time. Servers A and B maintain their own >>>> copies of subscription state. >>> >>> [] >>> >>>> What with the roster items that are inconsistent? >>>> >>>> * Mark as inconsistent, let the client present it to the user to >>>> take action. >>>> >>>> * Auto-repair and thus maintain consistency >>>> >>>> Looking forward to all feedback. >>> >>> When you send out a include the local >>> "view" of the subscription state. >> >> Btw presence probe seems too weak... as it doesn't reveal full >> subscription state. > > that's what I'm saying: include the full subscription state in the presence > probe so that the other side can detect mis-matches. > > Best regards, > > I'm considering doing this in Prosody: It wouldn't break anything. I don't see any privacy issues. And it would give the receiving server a chance to detect any inconsistency. If there is an inconsistency, the receiving server can take an appropriate action. What action is appropriate is open for debate. What should the resulting state be? The lowest common permissions (possibly sending unsubscribe[d] to the contact or changing the user's subscription for contact)? The highest common permissions (possibly sending a subscrive[d] to the contact and changing the user's subscription for the contact)? >From an IM user's point of view, trying to settle on the highest common permissions seems more appropriate (and less confusing). -- Waqas Hussain
Re: [Standards] roster views
On Thu, Feb 26, 2009 at 8:35 AM, Peter Saint-Andre wrote: > Matthew Wild wrote: >> On Thu, Feb 26, 2009 at 2:57 AM, Peter Saint-Andre >> wrote: >>> Matthew Wild wrote: >>>> On Thu, Feb 26, 2009 at 2:33 AM, Peter Saint-Andre >>>> wrote: >>>>> [...] >>>>> We had rough consensus that the server would not change its processing >>>>> of your outbound presence, i.e., it would send your presence to your >>>>> entire contact list, not only contacts in the group(s) you specify via >>>>> roster views. >>>>> >>>> I was of the impression that it would also apply to outgoing >>>> presences, and the filtered roster would essentially become your >>>> roster for that session. I don't know what others think though. >>> I could go either way. If roster views result in filtering of your >>> outbound presence then they are essentially a replacement for (some of) >>> what's now in privacy lists. I like that idea a lot because I don't like >>> privacy lists. :) >>> >> >> As a user I like that idea a lot, because it cuts out privacy lists. >> As a developer I like it a lot because it cuts out privacy lists :) > > Bingo! > >> Roster groups/tags are pretty flexible. This combined with simple >> blocking of users (and a way to enable/disable receiving messages from >> people not on your roster) is all that is required in most cases I >> believe, particularly for end-users. > > Right: > > http://xmpp.org/extensions/xep-0191.html > >>>>> If people think this would be useful, I'd be happy to write a small spec >>>>> about it. Right now I don't think this belongs in rfc3921bis but I could >>>>> be persuaded to change my mind about that (e.g., it might make sense to >>>>> have both roster versioning and roster views in the same core spec). >>>>> >>>> I think I already said somewhere that I believe this should be in >>>> core, versioning should be a XEP. Requiring all implementations to >>>> support versioning just feels wrong, and I tend to like smaller specs. >>>> However I understand if others don't feel the same way. >>> I think that either both versioning and views belong in rfc3921bis or >>> neither does. The syntax of what's currently in XEP-0237 (using an >>> attribute to indicate the version number) makes it difficult to split it >>> out into a XEP, but I suppose that could be overcome. >>> >> >> That's funny, it wasn't like that last week... ;-) > > Yeah, then we decided to make it roster-specific as it was in the > beginning, using an attribute. But we could change this: > > > > > > to something like this: > > > > > > > >> I'm not too fussed, it's going to be somewhere at the end of the day, >> doesn't really affect me where it ends up. > > Perhaps you're right that we'd better leave rosters alone and define > both versioning and views as extensions. I'll give that some further > thought... > > Peter > > -- > Peter Saint-Andre > https://stpeter.im/ > > In their current form, the only thing roster views do is save bandwidth in rare cases, which wouldn't be too useful for most IM users (most people don't have 1000+ contacts, and are generally not trying to save that much bandwidth, even on mobile devices). They don't fit the partial roster activation they are being considered for, and are not a replacement for privacy lists. I have often wanted to appear offline to a select group of people, but that doesn't mean I don't want to see their presence. Partial roster activation is a much much more attractive feature for me than partial roster retrieval. I would probably never consider using partial roster retrieval for my jabber account. Anyway, some things to consider for roster views: 1. More than one activated group at a time needs to work 2. They may be more useful if they worked based on filters on all data associated with a contact, e.g., groups, subscription, hosts 3. IM users would obviously want to change the activated groups more than once during a session, so that needs to be covered Fabio Forno's email covers 1 and 2. As for the RFC vs XEP argument, my personal opinion is to keep the RFC short, and have these in an XEP, but I don't feel too strongly about that. -- Waqas Hussain
Re: [Standards] XEP-0045 (Multi-User chat): Use case not covered
On Fri, Feb 27, 2009 at 7:06 PM, Mickael Remond wrote: > > Hello, > > The following use case does not seem covered if you want to use non > anonymous or semi anonymous rooms (to get / display user data for > example): > > 1. Enter a persistent room and say something bad. > 2. Leave the room. > 3. Someone enter the room and get the archived (contextual message). > As identity (JID) is disclosed only on presence packet and the user is > not available, the identity of the sender has been lost and cannot be used. > > A possible option would be to specify that server should broadcast > identity on presence for unavailable contacts that are still referenced > in discussion history. > > Any comments ? > > -- > Mickaël Rémond > http://www.process-one.net/ The use case is covered: "The 'from' attribute SHOULD be the full JID of the original sender in non-anonymous rooms, but MUST NOT be in semi-anonymous rooms (where the 'from' attribute SHOULD be set to the JID of the room itself)." - http://xmpp.org/extensions/xep-0045.html#enter-history So, the information is available. Most clients don't display it though. Sending unavailable presence for unavailable participants with messages in history may cause some clients to display leave messages including JIDs, but that's a hack. -- Waqas Hussain
Re: [Standards] groupchat and error message routing
On Thu, Feb 12, 2009 at 3:57 PM, Dave Cridland wrote: > On Wed Feb 11 18:27:03 2009, Jiří Zárevúcký wrote: >> >> I'm not entirely sure, but I think that nobody is ever supposed to >> send an error message to a bare JID. > > Not true - PEP nodes send from a bare jid, if these bounce they end up back > at a connected resource, much to the client's bewilderment, especially if > the server is reusing the client's id for publishing. > > However, Waqas is saying that errors sent to non-existent resources are also > sent to any conveniently connected client, too. > >> Errors are sent in a response to >> an invalid stanza, which always originates from a resource. As for the >> "groupchat", I would suggest taking a look at the relevant XEP. >> >> > The "relevant XEP" is RFC 3921. > > FWIW, I think groupchat to non-existent resources or bare jids should be > bounced, and errors to bare jids MAY be handled by the server, and MUST NOT > be sent to any resource except the original addressee. That's exactly what I had in mind. And note that that's similar to how offline storage works when there are no resources. As for the current behavior, I can't think of any cases where it would actually be useful. And it can cause problems: If we have two resources (same bare JID) in the same MUC room, and one disconnects just as the MUC server is sending a groupchat message. The second would receive and display duplicate messages. If the unavailable presence of the first one to the MUC server is lost, the second would continue to display duplicate messages, for as long as the first one isn't kicked by a presence error. > > Dave. > -- > Dave Cridland - mailto:d...@cridland.net - xmpp:d...@dave.cridland.net > - acap://acap.dave.cridland.net/byowner/user/dwd/bookmarks/ > - http://dave.cridland.net/ > Infotrope Polymer - ACAP, IMAP, ESMTP, and Lemonade > -- Waqas Hussain
