[Bitcoin-development] Network propagation speeds
Since this came up again during the discussion of the Cornell paper I thought I'd dig up my measurement code from the Information Propagation paper and automate it as much as possible. The result is the Network Propagation page on bitcoinstats.com (http://bitcoinstats.com/network/propagation/). It takes a daily snapshot of the situation, then calculates the time until blocks and transactions reach a certain percentile of the nodes in the network. There is also a detailed page showing the density function describing at what times nodes learn about the existence of a block/transaction (for example yesterdays distribution: http://bitcoinstats.com/network/propagation/2013/11/23). I intend to add more information and plots over time, but I wanted to push this out quickly as there were some people asking for it. Hope this helps getting the blockchain fork rate down :-) Regards, Chris -- Christian Decker -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Network propagation speeds
This is great, thanks for doing it. Tip sent your way. Graphs of how propagation data change over time would also be helpful (as well as raw data so we can calculate overhead per kilobyte and so on). I know there are only two days worth of data, but for future, it'd be good. I think the next part of figuring out why there's such huge disparity is instrumenting bitcoind to find out where the time goes when relaying a block. On Sun, Nov 24, 2013 at 5:26 PM, Gregory Maxwell gmaxw...@gmail.com wrote: On Sun, Nov 24, 2013 at 8:20 AM, Christian Decker decker.christ...@gmail.com wrote: Since this came up again during the discussion of the Cornell paper I thought I'd dig up my measurement code from the Information Propagation paper and automate it as much as possible. Could you publish the block ids and timestamp sets for each block? It would be useful in correlating propagation information against block characteristics. -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Network propagation speeds
Sure thing, I'm looking for a good way to publish these measurements, but I haven't found a good option yet. They are rather large in size, so I'd rather not serve them along with the website as it hasn't got the capacity. Any suggestions? If the demand is not huge I could provide them on a per user basis. -- Christian Decker On Sun, Nov 24, 2013 at 5:26 PM, Gregory Maxwell gmaxw...@gmail.com wrote: On Sun, Nov 24, 2013 at 8:20 AM, Christian Decker decker.christ...@gmail.com wrote: Since this came up again during the discussion of the Cornell paper I thought I'd dig up my measurement code from the Information Propagation paper and automate it as much as possible. Could you publish the block ids and timestamp sets for each block? It would be useful in correlating propagation information against block characteristics. -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Network propagation speeds
On Sun, Nov 24, 2013 at 05:20:22PM +0100, Christian Decker wrote: Since this came up again during the discussion of the Cornell paper I thought I'd dig up my measurement code from the Information Propagation paper and automate it as much as possible. The result is the Network Propagation page on bitcoinstats.com (http://bitcoinstats.com/network/propagation/). It takes a daily snapshot of the situation, then calculates the time until blocks and transactions reach a certain percentile of the nodes in the network. There is also a detailed page showing the density function describing at what times nodes learn about the existence of a block/transaction (for example yesterdays distribution: http://bitcoinstats.com/network/propagation/2013/11/23). I intend to add more information and plots over time, but I wanted to push this out quickly as there were some people asking for it. Hope this helps getting the blockchain fork rate down :-) Do you have the resources to save the raw log data? You'll also need to save transaction timestamp data - whether or not a given node has a transaction already matters re: propagation. Of course given pool centralization the moment pools start peering directly with each other all these stats might not mean all that much. Note that the number that's important isn't seconds, rather rather seconds/actual block interval as long as hashing power is growing. Unfortunately actually determining that is tricky - block interval is inherently noisy so you'll want to use a fairly agressively smoothed average. So here's a rough calculation: right now blocks are happening roughly %15 faster than they would at equilibrium, and blockchain.info reports about 2 orphans a day. 2/166=1.2% orphan rate. Now with a simplistic model where it takes exactly t seconds for a block to propagate to 100% of the hashing power, and until then 0% has it, you'd get: orphan rate = t / actual block interval - t = rate * interval Or 6.2 seconds with our orphan rate data. Now whether or not blockchain.info succesfully captures all orphans I don't know, but given you're reporting 4.5 to 9.4 seconds for 50th and 75th percentile respectively that number 6.2s seems ballpark reasonable - remember that hashing power is definitely not distributed evenly among the nodes you are sampling from. Which is another point... it may be the case that your propagation data doesn't actually give any insight into real-world orphan rates because the distribution of hashing power is concentrated into pools. -- 'peter'[:-1]@petertodd.org 00064bb57c6681a117371f06c4efe26917d9179a56cc20cff9f2 signature.asc Description: Digital signature -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
