Hi, After a while I've decided to finally give it a try to the url shortener thing.
The main concept we discussed last time is pretty simple: the long url should be stored in a separate post inside one's own torrent. This long url is then referenced by a shortened url format we have yet to decide, but basically it must include just the username and post number k (optimizations are possible, see below). My concern however is about the recoverability of such information. We don't want the url resolver to take ages to complete. The first obvious way to get the long url is fetching it from DHT. While it may work for a while after the post is produced, the DHT's post copy will eventually expire following our network rules. The second easy way (which is actually faster) is the case we are following the user. In this case, the original post has probably already been downloaded as a local copy and fetch is immediate. The real problem is when DHT expires and we are not following user. In this case we must start a new torrent to obtain just a single piece. This is in kind of conflict with bittorrent anti-leech mechanisms that foster cooperation. A new peer connection starts "choked" so it may not perform piece requests until "unchoked". Only a few connections are unchoked for a given time until they contribute something back. Here i'm worried about the possibly long waiting time the client will stay on the queue until unchoked. This can make the resolver to take a lot of time to complete. At the same time i cannot simply provide an exception to be abused by bad clients. So what i'm currently experiencing is adding a "peek" extension to our bittorrent protocol fork inspired in hashcash. This is a simple proof-of-work (originally intended for anti-spam) the client must perform in order to be allowed to make the request. PoW difficulty starts very small (about ~10ms on i7) but is increased if twister detects an abusive behavior. In my tests it seems to be working fine. Of course we will only be sure when it starts getting deployed to all peers the network. Until then, url shortener will work but possibly a bit slow. Another interesting discussion will be about the actual shortened url format: Something like "xxxx://user:k" is the most obvious candidate but we can do better... as username is limited on allowed (lowercase) characters and k is an integer, we may encode it to a binary format and then base64 it. Another possibility is using user registration number instead. Suggestions are welcomed! regards, Miguel -- You received this message because you are subscribed to the Google Groups "twister-dev" group. To unsubscribe from this group and stop receiving emails from it, send an email to [email protected]. To post to this group, send email to [email protected]. For more options, visit https://groups.google.com/d/optout.
