Re: [Bitcoin-development] Stealth Addresses
I think we have a winner in reusable address. Yes an existing address can be reused and will superficially appear to work, it just won't work well. Calling them reusable addresses helps reinforce the idea in peoples mind that the other kind shouldn't be reused. On Thu, Jan 16, 2014 at 11:14 AM, Drak d...@zikula.org wrote: On 16 January 2014 00:05, Jeremy Spilman jer...@taplink.co wrote: Might I propose reusable address. I think that describes it best to any non-programmer, and even more so encourages wallets to present options as 'one time use' vs 'reusable'. The problem is all addresses are reusable and to an average user, addresses are already reusable so there is little to distinguish the address format. It might be better to call it a public address in common terminology. Drak -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Tor / SPV
Yes correct, using hidden services just as a kind of more complicated, out of process/sandboxable SSL. would the overall transactions/second the Bitcoin network could handle go down? If all nodes talked to each other all the time over Tor, probably yes because Bitcoin is quite sensitive to latency. But what I'm proposing here is less ambitious. It's just about protecting (parts of) end-user-to-network communication, which is a much less risky sort of change. P2P nodes would still talk to each other in the clear. SSL for everything is still an idea I like, but it's true that increasing bitcoind attack surface area is not something to take lightly. Considering that the clearnet sybil protection also relies on scaling up the resource requirements for an attacker, why not require hidden service addresses following a certain pattern, like a fixed prefix? I'm sure we can come up with all kinds of neat anti-sybil techniques, but IMHO they are separate projects. I'm trying to find an upgrade that's small enough to be easily switched on by default for lots of users, today, that is low risk for the network overall. Later on we can add elaborations. The SPV node could connect to the IP using Tor. It would preserve the privacy of the SPV node - hard to see it's running Bitcoin. It also reduces the ability of an attacker to MITM because the routing varies with each exit node. Right so the key question is, to what extent does Tor open you up to MITM attacks? I don't have a good feel for this. I read about exit nodes routinely doing very naughty things, but I don't know how widespread that is. Probably you're right that with random selection of exits you're not excessively likely to get MITMd. How does Tor itself manage anti-sybil? I know they have the directory consensus and they measure nodes to ensure they're delivering the resources they claim to have. Punting anti-sybil up to the Tor people and letting them worry about it is quite an attractive idea. -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Stealth Addresses
Do any people who aren't computer programmers or physicists ever use the term static? I liked routing address. On Wed, Jan 15, 2014 at 9:44 PM, Jeff Garzik jgar...@bitpay.com wrote: static address seems like a reasonable attempt at describing intended use/direction. On Wed, Jan 15, 2014 at 3:38 PM, Gregory Maxwell gmaxw...@gmail.comwrote: On Wed, Jan 15, 2014 at 12:22 PM, Ben Davenport bendavenp...@gmail.com wrote: But may I suggest we consider changing the name stealth address to something more neutral? ACK. Regardless of the 'political' overtones, I think stealth is a little cringe-worthy. Private address would be fine if not for confusion with private-keys. Static address is perhaps the best in my view. (also helps improve awareness that normal addresses are intended to be more one-use-ness) -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Jeff Garzik Bitcoin core developer and open source evangelist BitPay, Inc. https://bitpay.com/ -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Tor / SPV
May need to modify the network address format to include the ability to differentiate IPv6 clearnet vs. Tor addresses sipa already implemented some clever hack where the 80-bit Tor keys are mapped to a subregion of reserved IPv6 space, giving magical IPv6 hidden service addresses. So addr packets can and do already contain onion addresses. but then you remove the implication that a node has to give both public and private IPs to a peer. If it's part of a batch of addrs, it could be my own hidden service ID, but it could also be one that I learned from someone else and is now propagating, for anyone to bootstrap with Tor hidden service peers if they'd like. Hmm. So you mean that we pick a set of peers we believe to not be sybils of each other, but they might give us hidden services run by other people? I need to think about that. If they're getting the hidden services just from addr announcements themselves, then you just punt the issue up a layer - what stops me generating 1 hidden service keys that all map to my same malicious node, announcing them, and then waiting for the traffic to arrive? If clearnet nodes inform of their own hidden service IDs, that issue is avoided. My goal here is not necessarily to hide P2P nodes - we still need lots of clearnet P2P nodes for the forseeable future no matter what. Rather we're just using hidden services as a way to get authentication and encryption. Actually the 6-hop hidden service circuits are overkill for this application, a 3-hop circuit would work just as well for most nodes that aren't Tor-exclusive. -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Stealth Payments - Sample Code / Proof of Concept
Cool! On Mon, Jan 13, 2014 at 10:18 AM, Jeremy Spilman jer...@taplink.co wrote: I spent 1BTC on TestNet to a stealth address... TxID: df092896c1347b303da299bc84c92bef1946f455dbdc80ffdb01a18ea4ed8b4c ... but can you redeem it? Code which generated this transaction is here: https://gist.github.com/jspilman/8396495 That's rather interesting code. Is this using a private C# bitcoin implementation? I wonder if the 0BTC OP_RETURN transactions should be hidden from the Transaction List? Yes, of course. The transaction list should just say something like Payment received from Jeremy, 0.1 BTC Maybe the simple way to punt on this is to just show 'Merchant' in the address column if it is available and an address is not. I am surprised it's not already the case! Though merchant is perhaps a bit biased as a name, internally it perhaps should just be called Recipient. There's no requirement for you to be a merchant to create payment protocol requests. I can probably make the necessary changes to IsMine, but I don't know where we should keep 'd2'/'Q2' unencrypted so it's available for doing the necessary tests, but has no chance of ever be used as a stand-alone private key? The wallet format would need extending. I'd feel a lot more comfortable if the protocol was reviewed by a professional cryptographer though. I think think Gregory already brought up an issue to do with people able to detect such payments by testing if decrypted values are points on the curve, or something like that. -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Stealth Addresses
On further reflection, I'm not sure I understand this use case of the payment protocol. Since a PaymentRequest currently contains the Outputs that specify the addresses to send to, reusing a PaymentRequest like this without using stealth addresses implies address reuse. Yes indeed .. which is why we're talking about extending the protocol (in a future version! the first version isn't even out yet!). -- CenturyLink Cloud: The Leader in Enterprise Cloud Services. Learn Why More Businesses Are Choosing CenturyLink Cloud For Critical Workloads, Development Environments Everything In Between. Get a Quote or Start a Free Trial Today. http://pubads.g.doubleclick.net/gampad/clk?id=119420431iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Stealth Addresses
You can always just extend the payment protocol with the new fields as well, vs making very long addresses. If this technique can be made to work well, it would have applicability in both fixed textual address context, and for a fixed/upload-once payment protocol file. That has the advantage of backwards compatibility as well - the new addresses would not be clickable or acceptable by old wallets, but with the payment protocol you can always craft a bitcoin URI that contains a regular current style address, and a link to a fixed payment protocol file (uploaded to a pastebin type site), and modern wallets would ignore the address and use the ECDH based system instead. On Sun, Jan 12, 2014 at 11:33 AM, Jeremy Spilman jer...@taplink.co wrote: Oh, sorry, I forgot to mention it in my first write-up but you can easily make stealth addresses include a second pubkey for the purpose of the communication that either isn't used in the scriptPubKey at all, or is part of a n-of-m multisig. (n=2) Interestingly that also means you can give a third-party that key and out-source the effort of scanning the blockchain for you. Great point. Even if it's not a 3rd party, I think it's really important to be able to scan for transactions with a key which can't actually spend the funds. The first approach is just one-pass ECDH. I think you're saying the second approach is two rounds of ECDH but re-using the same e/P (usually referred to as r/R in ECIES). I think this is safe, unlike reusing an ephemeral key for signing operations. Payee: Publish Q, Q2 [d, d2 are privkeys, Q, Q2 are pubkeys] Payer: 1) Generate ephemeral key: e / P [e is privkey, P is pubkey] 2) S = e * Q [first shared secret] 3) S2 = e * Q2[second shared secret, reusing 'e'] 4) Q' = Q + H(S) [pay-to stealth address] 5) Q2' = Q2 + H(S2) [stealth 'marker'] Watch: 1) Look for TxOut with OP_RETURN P 2) Q2' = Q2 + H(d2 * P) 3) Check for Q2' elsewhere in the Tx S/MIME for example, allows reuse of the ephemeral keypair. When reusing an ephemeral keypair where A reuses (x, X) to encrypt different messages to more than one user, A should verify the static public keys to prevent small-subgroup attacks.[1][2] Let's say you pay-to Q' and then Q2' value has to be somewhere else in the transaction. You could put it next to the shared P in OP_RETURN. OP_RETURN P Q2' would be 66 bytes. But then Mallory could generate transactions with the right Q2' but with his own pubkey in Step 2 instead of Q. So your scanner would detect a payment, but you wouldn't be able to spend it, and Mallory could. That's a good argument for putting Q2' in a 2-of-2 multisig so that pulling this trick would at least make the transaction unspendable for both parties, which may be good enough deterrent, but you're still going to want to check it against your 'd' before fulfilling a large order. Your online watch process could queue the matching transactions, which you could move to your offline machine, decrypt your key, and verify the transactions are spendable. Now, you would need to get two pubkeys to the payer, throw in a prefix to help standardize it, and end up with addresses that could look like (for example): xSTLxsn59oaupR6ZKt2yddQ4dp5hcFinnTWkvsDiXtHgAEDg5ajNVzTY8MMQsmqnEn3ZMKxQzrfC3pDimfTWMkiUb7x3jX3J26JSX tSTLcpwzupCFL3maVZkSiB9ib3BXsCAfkhMLgckQEyoj8Tk83ANzofeuDdbX6bSPqNRfACLLFYK8EwVo1jdjxNDFNDWxhnQiAy4ba Those addresses are 74 bytes: PrefixCompressedPubKey1CompressedPubKey2Checksum xSTL Prefix = 0xC0CB9270 tSTL Prefix = 0xB2E27D50 NOTE: I do NOT have the corresponding privkeys for these four pubkeys! Those just happened to be the first matching prefixes I found for 74 byte addresses. I could try to find ones which start with a specific byte if that's somehow better, like 0x04 to match BIP32. Unfortunately, I don't think you can just derive a second public key from the first to keep the address shorter, and still keep the first private key secure, even if the second private key is stolen. You only get equivalent security as BIP32 public derivation, where you can't lose a child private key. Do we also want xSTL (or whatever user friendly string) prefixes for single pubkey (41 byte) stealth addresses? I'll wait a couple days for feedback, then I'll try to implement the following prototypes: 1) Pay to STL addresses 2) Watcher process to detect and queue STL payments for a given d2/Q2 3) Offline verifier to take output from Watcher and verify spendable given encrypted d/d2 Obviously extending QT directly for #1 would be ideal, I may even be able to do that since supporting a new address type should be fairly contained. But if not I'll punt to writing a node.js or python script which connects to bitcoind via RPC. Thanks, Jeremy
Re: [Bitcoin-development] Dedicated server for bitcoin.org, your thoughts?
That seems overly complicated, there's no need for the Bitcoin protocol to be involved. Deterministic builds with threshold signed updates are a problem the entire crypto community is now interested in solving - any solution should be generic. Really all you need is an update engine that allows a CHECKMULTISIG type approach. When the update engine is not under our control, i.e. on Android, Shoup style RSA threshold signatures can potentially work (though I must admit, I have never found time to play with the implementation I have for that algorithm). On Tue, Dec 31, 2013 at 9:25 PM, Jeremy Spilman jer...@taplink.co wrote: I didn't know about the dedicated server meltdown, it wasn't any of my infra. Anyway, my previous offer still stands. One less 'security theater' approach would be if we could provide forward-validation of updates using the blockchain. It's always going to be up to the user the first time they install the wallet to verify the provenance of the binaries/source. From that point forward, we could make it easier if the wallet could detect updates and prove they were valid. This could be as simple as hard-coding a public key into the client and checking a signature on the new binaries. But it could also be more interesting... For example, a well known address on the blockchain corresponds to multi-sig with keys controlled by developers (or whatever key policy the release team wants to impose). A spend from that address announces a new release, and includes the expected hash of the file. You would probably need some way to handle the different release targets. A more rigorous approach could identify all the various releases in terms of a BIP32 xpubkey whose branches would correspond to the different release trains and platform builds. Spends from a node announce the release and the expected hash. This provides zero benefit if the wallet software is already compromised, but I think this would allow trusted automatic update notification, and a trusted way to deliver the expected hashes. It also might resolve some of the consternation around when a release is truly released, if that's really a problem. I'm not sure how far along the slope you would want to go; 1) announcing updates in the UI, 2) providing a button the user could click to verify a binary matches its expected hash, 3) click to download and verify the upgrade matches the expected hash, 4) click to upgrade Formalizing the release process around a set of privkeys (or split shares of keys) may raise its own set of questions. For the download itself, I've heard the advocates of announcing availability on the blockchain leading to a BitTorrent magnet link, but I also understand objections to adding an entire BitTorrent stack into a wallet. On Tue, 31 Dec 2013 06:23:55 -0800, Mike Hearn m...@plan99.net wrote: The site was actually moved onto a dedicated server temporarily and it melted down under the load. I wouldn't call that no progress. Oh, it did? When was that? I must have missed this excitement :) Any idea how much load it had? Perhaps I wasn't clear on the point I was making Drak's threat model is not improved in the slightest by SSL. It would be improved by increasing the use of signature checking, e.g. by making it easier. Well, that depends. If you watch Applebaums talk he is pushing TLS pretty hard, and saying that based on the access to the source docs some of their MITM attacks can't beat TLS. It appears that they have the capability to do bulk MITM and rewrite of downloads as Drak says but *not* when TLS is present, that would force more targeted attacks. So to me that implies that TLS does raise the bar and is worth doing. However if we can't find a server that won't melt under the load, then that'd be an issue. We could consider hosting downloads on AppEngine or something else that can handle both high load and TLS. -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Dedicated server for bitcoin.org, your thoughts?
Oh, it did? When was that? I must have missed this excitement :) I would be very interested to learn more about this. It seems the steady state load on the site is not very high: https://github.com/bitcoin/bitcoin.org/pull/287 (Saivann ran Google Analytics on the site for a little while to get traffic figures). Peak of 10 visitors per second, assume a 10x blowup on resources, that's only ~100 reqs/sec steady state, that shouldn't strain any kind of reasonable server. So perhaps the specs of the dedicated server were not what you might imagine. Perhaps we should move the site over to Jeremy's hosting? It shouldn't be very expensive to serve outside of major press cycles. Once that is done, perhaps we can find/blag some SSL-protected file hosting. -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Dedicated server for bitcoin.org, your thoughts?
Given that hardly anyone checks the signatures, it's fair to say downloads aren't protected by anything at the moment. SSL for downloads can only raise the bar, never lower it, and if the NSA want to kick off the process of revoking some of the big CA's then I'm game (assuming anyone detects it of course) :) Anyway, nobody is dragging feet, the problem is right now we get what is effectively a huge free subsidy from github and SourceForge for site hosting. The cost is no SSL. So getting SSL would require that we pay for it ourselves, but the primary method we have for funding public goods/infrastructure (the Foundation) which is the subject of various conspiracy theories. Jeremy has made a generous offer further up the thread, the issue being I guess none of us know how much traffic we actually get :( I remember suggesting that we whack Google Analytics or some other statistics package on when the new website design was done and that was rejected for similar reasons (organisations are bad). So we are in a position where we get a subsidy of large but unknown size from various existing US corporations, but moving to different ones is controversial, hence no progress :) On Tue, Dec 31, 2013 at 1:48 PM, Gregory Maxwell gmaxw...@gmail.com wrote: On Tue, Dec 31, 2013 at 5:39 AM, Drak d...@zikula.org wrote: The NSA has the ability, right now to change every download of bitcoin-qt, on the fly and the only cure is encryption. Please cut it out with the snake oil pedaling. This is really over the top. You're invoking the NSA as the threat here? Okay. The NSA can trivially compromise an HTTPS download site: even ignoring the CA insecurity, and government run CAs certificate authorities issue CA certs to random governments and corporations for dataloss prevention purposes. Not to mention unparalleled access to exploits. The downloads are protected by something far stronger than SSL already, which might even have a chance against the NSA. Actual signatures of the downloads with offline keys. I'm all pro-SSL and all that, but you are— piece by piece— really convincing me that it produces an entirely false sense of security which is entirely unjustified. -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Access to Mempool
I was reading there are some commands to access a peer's mempool state. The purpose being to allow miners to recover faster after a reboot, I think? The mempool command allows nodes to request the contents of a peers memory pool, yes. It is currently used by SPV clients to find transactions that were broadcast before they were started up (but not yet confirmed). Reading peer mempool definitely allows recovering faster after a reboot. So does persisting mempool in a database locally. 0.9 has code to save the mempool to disk. But what can you learn about a node from its mempool? Basically, are there distinguishing features in the mempool, or could there be? Er, you mean, distinguishing features beyond the nodes IP address? The contents of the mempool may vary depending on when the node was started and what it saw at what times. I guess it's distinguishing in a way, but not in any important way. Nodes are not intended to be completely indistinguishable, just indistinguishable enough that it doesn't matter which you connect to. Are there transactions you can receive which go into your own mempool but which you don't forward? I don't think so, unless there are quirks to do with sendrawtransaction RPCs or strangely crafted wallet spends. Normally if a tx is in the mempool it will be relayed. By the way, are there recommended places to go to compare features implemented by different wallet software? I don't know of any such place, but I'm sure people have compiled tables somewhere. -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Fees / prio to be confirmed within ....
(nb: Gavin is on vacation at the moment, I post this now just to give food for thought over the holidays). I patched my bitcoind to use a modified version of Gavin's fee estimation framework. Here is what it's currently estimating. This shows number of samples taken for fee-paying transactions and free transactions (where confirmation requires priority), along with the median fee paid and then the 10th percentile fee paid, and median priority for the free transactions (no 10th percentile there). -1 means missing data, of course. Max 1000 samples. 2013-12-28 18:34:42 estimates: for confirming within 0 blocks based on 1000/383 samples, fee=38759.7/16155.1 (10%) per kilobyte, prio=8.41811e+08 2013-12-28 18:34:42 estimates: for confirming within 1 blocks based on 1000/202 samples, fee=26738/16155.1 (10%) per kilobyte, prio=2.91473e+08 2013-12-28 18:34:42 estimates: for confirming within 2 blocks based on 298/115 samples, fee=22831.1/10341.3 (10%) per kilobyte, prio=3.18117e+08 2013-12-28 18:34:42 estimates: for confirming within 3 blocks based on 115/88 samples, fee=22831.1/8810.57 (10%) per kilobyte, prio=2.53442e+08 2013-12-28 18:34:42 estimates: for confirming within 4 blocks based on 17/26 samples, fee=14992.5/2759.38 (10%) per kilobyte, prio=2.99917e+08 2013-12-28 18:34:42 estimates: for confirming within 5 blocks based on 1/12 samples, fee=7468.26/7468.26 (10%) per kilobyte, prio=4.12797e+08 2013-12-28 18:34:42 estimates: for confirming within 6 blocks based on 1/9 samples, fee=8071.03/8071.03 (10%) per kilobyte, prio=1.32007e+08 2013-12-28 18:34:42 estimates: for confirming within 7 blocks based on 5/22 samples, fee=3018.41/1.91939 (10%) per kilobyte, prio=9.60733e+07 2013-12-28 18:34:42 estimates: for confirming within 8 blocks based on 0/9 samples, fee=-1/-1 (10%) per kilobyte, prio=1.22123e+08 2013-12-28 18:34:42 estimates: for confirming within 9 blocks based on 0/8 samples, fee=-1/-1 (10%) per kilobyte, prio=6.42686e+07 2013-12-28 18:34:42 estimates: for confirming within 10 blocks based on 0/3 samples, fee=-1/-1 (10%) per kilobyte, prio=6.72846e+06 2013-12-28 18:34:42 estimates: for confirming within 11 blocks based on 0/9 samples, fee=-1/-1 (10%) per kilobyte, prio=5.42872e+08 2013-12-28 18:34:42 estimates: for confirming within 12 blocks based on 0/1 samples, fee=-1/-1 (10%) per kilobyte, prio=1.13419e+07 2013-12-28 18:34:42 estimates: for confirming within 13 blocks based on 0/3 samples, fee=-1/-1 (10%) per kilobyte, prio=4.57343e+08 2013-12-28 18:34:42 estimates: for confirming within 18 blocks based on 0/2 samples, fee=-1/-1 (10%) per kilobyte, prio=5.51321e+08 2013-12-28 18:34:42 estimates: for confirming within 20 blocks based on 0/3 samples, fee=-1/-1 (10%) per kilobyte, prio=4.41654e+08 2013-12-28 18:34:42 estimates: for confirming within 22 blocks based on 0/4 samples, fee=-1/-1 (10%) per kilobyte, prio=4.04413e+08 2013-12-28 18:34:42 estimates: for confirming within 23 blocks based on 0/4 samples, fee=-1/-1 (10%) per kilobyte, prio=5.02467e+08 2013-12-28 18:34:42 estimates: for confirming within 24 blocks based on 0/1 samples, fee=-1/-1 (10%) per kilobyte, prio=2.76975e+08 2013-12-28 18:34:42 estimates: for confirming within 25 blocks based on 0/1 samples, fee=-1/-1 (10%) per kilobyte, prio=2.90481e+08 2013-12-28 18:34:42 estimates: for confirming within 27 blocks based on 0/3 samples, fee=-1/-1 (10%) per kilobyte, prio=3.49409e+08 2013-12-28 18:34:42 estimates: for confirming within 28 blocks based on 0/1 samples, fee=-1/-1 (10%) per kilobyte, prio=1.35682e+09 2013-12-28 18:34:42 estimates: for confirming within 31 blocks based on 0/2 samples, fee=-1/-1 (10%) per kilobyte, prio=2.17966e+08 2013-12-28 18:34:42 estimates: for confirming within 36 blocks based on 1/0 samples, fee=8103.73/8103.73 (10%) per kilobyte, prio=-1 2013-12-28 18:34:42 estimates: for confirming within 47 blocks based on 1/0 samples, fee=608.273/608.273 (10%) per kilobyte, prio=-1 2013-12-28 18:34:42 estimates: for confirming within 48 blocks based on 1/0 samples, fee=11415.5/11415.5 (10%) per kilobyte, prio=-1 2013-12-28 18:34:42 estimates: for confirming within 51 blocks based on 0/1 samples, fee=-1/-1 (10%) per kilobyte, prio=1.01951e+09 2013-12-28 18:34:42 estimates: for confirming within 55 blocks based on 1/0 samples, fee=3891.05/3891.05 (10%) per kilobyte, prio=-1 -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Testnet block explorer
For a long time the only block explorer for testnet has been the original blockexplorer.com, which is unfortunately often broken / behind / slow and not really maintained any more. There is now a new one, here: https://www.biteasy.com/testnet/blocks There's also a REST/JSON API for it. Please note one curiosity of this block explorer is that the coinbase tx doesn't necessarily come first in the listing (it's sorted by time received, see). Other interesting thing to note: this site is built using bitcoinj. The author can be contacted on IRC sometimes using the nick damethos. -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Peer Discovery and Overlay
Thanks Warren! That's great. It's also a prerequisite for chain pruning, so it's not only about decentralisation but also scalability. Looking forward to reviewing and merging that. On Tue, Dec 24, 2013 at 6:11 PM, Warren Togami Jr. wtog...@gmail.comwrote: I was concerned about this issue so we sponsored BlueMatt to implement an address database for bitcoinj. In the future it won't be entirely reliant on what DNS tells it. Warren On Tue, Dec 24, 2013 at 6:02 AM, Peter Todd p...@petertodd.org wrote: As for node addresses being a service, that's what the DNS seeds are! bitcoinj clients, for instance, depend very heavily on those seeds and can be easily compromised in a variety of ways by them. -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Dual elliptic curve algorithms
It is irrelevant. -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Fees UI warning
On Mon, Dec 16, 2013 at 12:31 PM, Pieter Wuille pieter.wui...@gmail.comwrote: Will that also mean no longer reusing (change) addresses? Jim seems to be planning some parallel development to what I'm doing, but HD wallets and stopping address re-use is the current feature I'm working on for bitcoinj. Only code review and merging takes higher priority at the moment. So I think we might be able to stop re-using addresses at least on devices with sufficient memory some time in Q1 -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Fees UI warning
On Mon, Dec 16, 2013 at 11:13 AM, Drak d...@zikula.org wrote: It just occurs to me this kind of sad story could be averted if wallets implemented a confirmation box if the fee amount seems crazy - for example, if it's 10x what the default fee should be, or if it's greater than x% of the sending amount. Most good wallets have UI's designed to be safe. Unfortunately this guy was using brainwallet.org which is by no means a good wallet in that sense (it's not really even a wallet app at all) I think most of us have expressed displeasure at the existence of this site before, and I once even asked the guy to stop running it, but he refused. It's an extremely sharp tool which makes it easy to cut yourself, except it doesn't look dangerous, it looks like ordinary software designed for ordinary people. I don't know how to solve this. Badly designed software that looks appealing will always be a danger. -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Merge avoidance and P2P connection encryption
Why would there be an iteration count? The payer would handle that, wouldn't they? I'm thinking about a use case I hope will become common next year - pastebin style hosting sites for payment requests. Like, if I as a regular end user wish to use the payment protocol, I could just upload a (possibly signed) payment request to: payr.com/a62gahZ or whatever, and then payr.com can take care of incrementing the iteration count on each download of my file. That's why it's useful for it to be unsigned. If the use case is: I give the Foundation a here's where to pay my salary PaymentRequest, maybe with several Outputs each having a different xpubkey, then it seems to me the Foundation's wallet software should take care of iterating. Absolutely. The two use cases can both be supported. You could give iteration ranges, for instance, if you want to specify expiry in terms of number of payments rather than time. -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Merge avoidance and P2P connection encryption
Or alternatively, the user-signed payment request without iteration count is enclosed within a payr.com-signed envelope that contains the iteration count. But how does that show up in the user interface? I don't know how you would explain what the signature means or implies, or what you do if the signature is broken/missing. The only thing that a maliciously modified iteration count can do is cause money to be sent to an address that's beyond the recipients gap limit, meaning they won't receive it (unless they reconfigure their software and rescan). But you can't steal money that way. -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Merge avoidance and P2P connection encryption
I wrote an article intended for a broad/non-developer audience on a few Bitcoin privacy topics: - P2P connection encryption - Address re-use/payment protocol - CoinJoin and merge avoidance I don't think there's anything much new here for people who were involved with the BIP70 design discussions, but it may prove a useful resource when talking about privacy features in the payment protocol. Specifically the ability to request multiple outputs and submit multiple transactions that satisfy them. The article elaborates on how to use that feature to achieve some useful privacy outcomes. I also analyze what using SSL for P2P connections would buy us and what it wouldn't. https://medium.com/p/7f95a386692f -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Merge avoidance and P2P connection encryption
I think the right way to integrate BIP32 and BIP70 would be to specify
output scripts as normal for backwards compatibility, and then allow each
output to have an additional xpubkey and iteration count field. The
iteration counts could be unsigned.
Unfortunately to add data that isn't signed requires a backwards
incompatible change to the protocol :( There isn't currently any area that
isn't covered by the signature. We would have to add one, and then have a
matching array of iteration counts for each xpubkey that was specified in
the output.
I wonder if we should make a last minute change to BIP70 before wallets
have shipped and merchant support starts, something like
message PaymentRequest {
optional byte unsigned_data = 6;
}
that would be deleted like the signature is before reserialization.
On Thu, Dec 12, 2013 at 9:28 AM, Paul Rabahy prab...@gmail.com wrote:
First off, nice article. Very clear and informative.
I don't know if this is the best place to post this, but it seems related
to me.
As more wallets implement BIP32, I believe that bitcoin wallets should
begin to encourage people to use
https://github.com/bitcoin/bips/blob/master/bip-0032.mediawiki#recurrent-business-to-business-transactions-mi0style
address instead of traditional addresses. In the end, this would
improve privacy because users never need to merge coin if they had one of
these super addresses.
In addition, super addresses would fit nicely into BIP70. Right now, the
PaymentDetails message allows the merchant to provide multiple outputs. If
instead the PaymentDetails provide 1 traditional output (for reverse
compatibility) and 1 super address, the payment could be broken into as
many pieces as is needed to match unspent outputs already in the customers
wallet. Finally, the refund_to address in Payment could also be upgraded to
a super address to enhance privacy there.
I am not sure if there is a large memory requirement for super
addresses, but to me, it seems that a lot of these privacy enhancing
possibilities will be simple to implement once BIP32 is widely deployed.
On Thu, Dec 12, 2013 at 11:03 AM, Mike Hearn m...@plan99.net wrote:
I wrote an article intended for a broad/non-developer audience on a few
Bitcoin privacy topics:
- P2P connection encryption
- Address re-use/payment protocol
- CoinJoin and merge avoidance
I don't think there's anything much new here for people who were involved
with the BIP70 design discussions, but it may prove a useful resource when
talking about privacy features in the payment protocol. Specifically the
ability to request multiple outputs and submit multiple transactions that
satisfy them. The article elaborates on how to use that feature to achieve
some useful privacy outcomes.
I also analyze what using SSL for P2P connections would buy us and what
it wouldn't.
https://medium.com/p/7f95a386692f
--
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics
Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
--
Rapidly troubleshoot problems before they affect your business. Most IT
organizations don't have a clear picture of how application performance
affects their revenue. With AppDynamics, you get 100% visibility into your
Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro!
http://pubads.g.doubleclick.net/gampad/clk?id=84349831iu=/4140/ostg.clktrk___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Floating fees and SPV clients
On Wed, Dec 4, 2013 at 2:06 PM, Peter Todd p...@petertodd.org wrote: replace-by-fee is no less speculative than your original proposals; you're also trying to convince people that things should work differently re: fees The original proposal I started this thread with hasn't even received comments - presumably it's uncontroversial. The other discussions are about how to handle fees in requests that use the payment protocol, which isn't currently used anywhere so doing things differently isn't possible. On the other hand you have been talking about a fundamental change to the behaviour of how all Bitcoin nodes operate, which is off topic for this thread. If you have something specific to say about how floating fees should be managed by SPV wallets or how fees should be negotiated when the payment protocol is in use, this thread is appropriate. Otherwise please take it elsewhere. -- Sponsored by Intel(R) XDK Develop, test and display web and hybrid apps with a single code base. Download it for free now! http://pubads.g.doubleclick.net/gampad/clk?id=111408631iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Floating fees and SPV clients
On Tue, Dec 3, 2013 at 2:40 AM, Gavin Andresen gavinandre...@gmail.comwrote: optional uint64 allowfeetag number=1000 Let's just use a normal/low tag number. The extensions mechanism is great for people who want to extend the protocol outside the core development process. It'd be weird if nobody ever used the low numbers again though. Tag numbers are varint encoded so using smaller ones does have a minor efficiency benefit, it's not just aesthetics :) Allow up to allowfee satoshis to be deducted from the amount paid to be used to pay Bitcoin network transaction fees. A wallet implementation must not reduce the amount paid for fees more than allowfee, and transaction fees must be equal to or greater than the amount reduced. Hmmm. Why allow? Should it not be called min_fee instead? Wallets would have to attach at least that much in fees, right? Also, why describe it as reducing the amount paid? Which output would be reduced in value? Why not just have it be added to the total value displayed to the user and the outputs are left alone/not reduced. We also want to allow users to pay MORE in fees, if they need to (fragmented wallet, maybe, or big CoinJoin transaction) or decide to. I like the idea but it seems this gets us back to the original problem - senders don't care about confirmations, ever, not even if they make an annoying set of transactions. The protocol allows users to submit transactions directly to receivers, I guess, if the receiver does not like the transactions they get they could potentially reject the payment. But I'd hope that's really rare. PS: I think there was also consensus that the BIP72 request=... should be shortened to just r=... (save 6 chars in QR codes). Unless somebody objects, I'll change the BIP and the reference implementation code to make it so... Sweet, thanks! -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Floating fees and SPV clients
On Tue, Dec 3, 2013 at 11:36 AM, Drak d...@zikula.org wrote: I dont like the idea of putting the min fee in the hands of the receiver. Seems like that will work against the best interests of senders in the long run. Senders have no interest in ever attaching any kind of fee, which is one reason we explored child-pays-for-parent for a while. It's not the sender who cares about double spending risk. Left to their own devices, all senders would always attach no fee at all (or rather: whatever the min was to get the transaction relayed to the merchant). However, receivers do want a fee attached, and ideally we would do this without redundant transactions. Hence, receivers asking senders to attach a fee and effectively folding it into the price that is paid. That is, if you go into a restaurant and the menu says Burger: 10mBTC then when you come to pay, what you see on your phone screen is 10mBTC. The fact that actually the shop with receiver 9.9mBTC and the tx fee is 0.1mBTC is hidden in the user interface - creating a situation like many others, where receivers eat a transaction cost. For instance in Europe sales taxes are included in the price, not attached separately later. There's no need to trust the vendor. If a vendor asks for a ridiculously high tx fee, it will just surface as uncompetitively priced goods/services. Buyers will go elsewhere. Why not try a different path of calculating the min fee like difficulty retarget. You can analyse the last 2016 blocks to find the average fee accepted per kb (which would include transactions that were included without fees) and then write that into the block as a soft recommendation that wallets could use in the UI. This way the price can vary up and down according to what people were willing to spend on fees and miners willing to accept. That's what fee estimation does, essentially, minus the encoding into blocks. Once you start getting miners telling people what fees are directly you run into cases where they might try to lie about their behaviour or otherwise influence the average. Querying all nodes avoids that problem. -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Floating fees and SPV clients
On Tue, Dec 3, 2013 at 12:41 PM, Gavin Andresen gavinandre...@gmail.comwrote: If users want to pay with a huge transaction then it seems to me the user should cover that cost. Allowing users to pay merchants with 100K transactions full of dust and expecting them to eat the cost seems like a great way to enable bleed-the-merchant-dry attacks. A merchant can always refuse the payment and refund it if that's a practical problem. I doubt it would be though. If a user is trying to buy something from the merchant, they will want it to work, and it'll be up to the developers of the wallet they're using to ensure it never does anything obnoxious or unacceptable that would result in people hating to receive money from that app. RE: hiding or showing fees: I pointed out to Peter that there doesn't have to be One True Answer. Let wallets experiment with either hiding or exposing fees, and may the best user experience win. Sure. I think there will be experimentation in this regard. -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Floating fees and SPV clients
On Tue, Dec 3, 2013 at 12:57 PM, Taylor Gerring taylor.gerr...@gmail.comwrote: Why should there be two classes of transactions? Where does paying a local business at a farmer’s stand lie in that realm? Transactions should work the same regardless of who is on the receiving end. Lots and lots of people are psychologically trained to expect that they pay the sticker price for things. Yes in recent times some places have started to show additional fees for using credit cards, but only as a way to try and push people onto cheaper forms of payment, not because customers love surcharges. It's for that reason that many merchants don't do this, even when they could - I pay for things with Maestro Debit all the time and I don't think I've ever seen a surcharge. That system obviously has costs, but they're included. This is just a basic cultural thing - when I buy something from a shop, the social expectation is that the seller should be grateful for receiving my money. The customer is always right. When I send money to a friend, the social expectation is different. If my friend said, hey Mike, could you send me that 10 bucks you owe me from last weekend and what he receives is less than 10 bucks, he would probably feel annoyed - if I owe him 10 bucks then I owe him 10 bucks and it's my job the cover the fees. That's why PayPal makes sender pay fees in that case. Maybe we need new terminology for this. *Interior fees* for included in the price/receiver pays and *exterior fees* for excluded from the price/sender pays? Fees are only confusing because existing clients do a terrible job of presenting the information to the user. In Hive Wallet, I’m of the opinion that we should inform the user in an intuitive way to let them make an informed decision. Have you thought through the UI for that in detail? How exactly are you going to explain the fee structure? Let the user pick the number of blocks they need to wait for? What's a block? Why should I care? Why shouldn't I just set the slider all the way to the other end and pay no fees at all? Is the merchant going to refuse to take my payment? Gavin just said that's not possible with Bitcoin-Qt. I'm thinking for bitcoinj I might go in a slightly different direction and not broadcast payments submitted via the payment protocol (and definitely not have one wire tx pay multiple payment requests simultaneously, at least not for consumer wallets). -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Floating fees and SPV clients
Lately I was pondering how to make floating fees and SPV wallets work well together. I propose the following plan: 1) 0.9 ships with something dead simple, like a command to query what a node estimates and then clients just take the average, or cross-check a centralised estimate against the P2P network. It's fast to implement and simple, but not very secure or decentralised. However it will allow the feature to launch on some kind of reasonable timeframe. 2) We bump the protocol version and the tx message now gets an optional protobuf buffer stuck on the end. The first thing put in this protobuf is a list of the values of the inputs. Using this data, the fee paid by a transaction can be calculated. In step 2 the data is unauthenticated. 3) Some SPV wallets already set themselves up so that they sync with the network in the background, e.g. the Android wallet syncs at least every 24 hours. This should become more common, using scheduler capabilities built into most operating systems. When the wallet syncs with the network, it sets a deliberately very noisy Bloom filter on its peers and waits around for 30-60 seconds or so. The wallet observes some of the broadcasts taking place and records the hashes and associated fees that were paid to disk. Next time it syncs, it includes the observed hashes into the Bloom filter used to download the chain, and thus learns how quickly they confirmed. It can calculate its own fee estimate from that. 4) Finally, when we next hard fork, we make v2 transactions include the output value in the signature, same as the output script (this proposal has been on the forums for a while now). That allows the fee data added in step 2 to be cross-checked against the signatures on the inputs, thus authenticating it. I think this is a small and easy set of steps that would make it quite hard to attack - malicious nodes could make it appear that some transactions never confirmed thus seeming to force the price up, but it's easy to simply exclude transactions which never confirm at all from the calculations. Plus of course you can cross-check nodes against each other to try and catch nodes that are failing to match transactions properly. One obvious concern is what to do if nodes don't converge on very similar estimates. Wallets will always want to pay the lowest fee possible, so that means they'll always be riding the very edge of what's acceptable, opening up tx propagation to random flaky failures if fee estimates change whilst a transaction is in progress, or if some nodes don't calculate the same estimates as others. If a wallet gets a reject message for a tx that has a fee that are by its own estimates acceptable, what should it do? What if only some nodes report that and others don't? -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Floating fees and SPV clients
Both can be combined into adapting the current generic messages (This payment should become spendable shortly for incoming and This payment has not been transmitted yet for outgoing transactions). What would the new messages say? We need to get away from the notion of senders attaching fees anyway. This is the wrong way around because it’s the recipient who cares about double spending risk, not the sender. That’s why merchants keep running into issues with people attaching zero fees. Of course they attach zero fees. They know they aren’t going to double spend. It’s the merchant who cares about getting the security against that. The UI for sending money should end up dead simple - no mention of fees anywhere, IMO. The UI for receiving money could be a bit more complicated but even then - I think if ordinary people using smartphone wallets are having to think about how quickly they want their transaction to confirm and adjust fees, etc on the receiving side then we’re getting dangerously close to the usability failure zone. Unfortunately we lack the protocol pieces to get the right UI here :( Someone needs to sit down and figure out what the UI *should* look like, in the ideal world, and then work backwards to figure out what needs to be done to get us there. For outgoing transactions, if it is very clear that they're never going to be confirmed, I'd like to see a Revoke button. Disagree. There should never be any cases in which a transaction doesn’t confirm. Period. I know there have been bugs with bitcoinj that could cause this in the past, but they were bugs and they got fixed/will get fixed. Settlement failure is just unacceptable and building a UI around the possibility will just encourage people to think of it as normal, when it should not be so. smime.p7s Description: S/MIME cryptographic signature -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Floating fees and SPV clients
This payment did not become spendable since xxx minutes. Check with the sender if s/he paid the Bitcoin network fee. Check if your internet connection is working properly. (incoming) That seems reasonable. The other message should be implementable today, I think? If numBroadcastPeers 0 post 0.10.3 then you know the tx made it out to the internet. Unfortunately if nodes start to diverge a lot in terms of what they will accept, then “transmitted” is no longer a clean binary yes/no thing. Guess we’ll have to jump that hurdle when we come to it. Guess you're right. But as you said, we're not there yet. The payment protocol at least would need some notion of fee, or possibly (better?) the ability for a recipient to specify some inputs as well as some outputs. Originally I think we were hoping for child-pays-for-parent. I guess that needs someone to sit down and focus on it for a while, assuming we still think that’s a good idea. smime.p7s Description: S/MIME cryptographic signature -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Floating fees and SPV clients
Bitcoin is and always will be limited in capacity - transactions may not confirm in a reasonable about of time because of high-demand and/or DoS attacks. I agree in the general case, but I was talking about the mobile wallet case specifically (i.e. people who are sending money between themselves or making small purchases of physical things). I think Bitcoin should be able to scale to handle these sorts of ordinary every-day transactions. Where I’d expect to see transactions falling off the edge is in more specialised cases like very small single micropayments, or “optional” internal transactions like mixing/re/defragmentation of wallets that don’t correspond to an actual payment. Those sorts of transactions would I guess be the first to go when faced with a sudden capacity crunch, but they wouldn’t show up in a mobile wallet UI anyway. re: merchants paying tx fees, child-pays-for-parent is inefficient I know the existing code is, but is that fundamentally the case or just how the code has been written? I haven’t looked at this issue much but I know you’ve worked on it, so I’m curious to learn about why it’s inefficient and whether there are any fixes possible. smime.p7s Description: S/MIME cryptographic signature -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Network propagation speeds
Hey Christian, Could you sort the snapshots by date? At the moment they're kind of in a random order. Sometimes I wish we had real-time stats too but this is a great start. On Mon, Nov 25, 2013 at 8:27 PM, Christian Decker decker.christ...@gmail.com wrote: Thanks Mike for the Tip :-) I will definitely extend the calculations to include a size-normalized version. As for transaction propagations, being much smaller the measurements tend to be much noisier, but given enough samples we might be able to reconstruct some of the system parameters. Good idea to attempt to correlate propagation speed and number of inputs/outputs, might be interesting to see whether processing at the nodes has an influence. Regards, Chris -- Christian Decker On Mon, Nov 25, 2013 at 9:51 AM, Michael Gronager grona...@ceptacle.com wrote: Hi Christian, Cool - thanks for posting - agree, that it would be nice to normalize the results with block size - so divide by size and: 1. see if there is a correlation (we all presume there still is) 2. plot the delay graph as e.g. normalized to the averaged blocksize or lets define a standard block size of 200kb or what ever so we can compare the plot btw days. Also, does the correlation of propagation times hold for transaction sizes as well (would be ice to find the logical t0 and the constant - I guess the interesting measure is not kb but signatures, so number of inputs - some correlation with size though). Best, Michael On 24/11/13, 17:37 , Christian Decker wrote: Sure thing, I'm looking for a good way to publish these measurements, but I haven't found a good option yet. They are rather large in size, so I'd rather not serve them along with the website as it hasn't got the capacity. Any suggestions? If the demand is not huge I could provide them on a per user basis. -- Christian Decker On Sun, Nov 24, 2013 at 5:26 PM, Gregory Maxwell gmaxw...@gmail.com wrote: On Sun, Nov 24, 2013 at 8:20 AM, Christian Decker decker.christ...@gmail.com wrote: Since this came up again during the discussion of the Cornell paper I thought I'd dig up my measurement code from the Information Propagation paper and automate it as much as possible. Could you publish the block ids and timestamp sets for each block? It would be useful in correlating propagation information against block characteristics. -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Rapidly troubleshoot problems before they affect your business. Most IT organizations don't have a clear picture of how application performance affects their revenue. With AppDynamics, you get 100% visibility into your Java,.NET, PHP application. Start your 15-day FREE TRIAL of AppDynamics Pro! http://pubads.g.doubleclick.net/gampad/clk?id=84349351iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Network propagation speeds
This is great, thanks for doing it. Tip sent your way. Graphs of how propagation data change over time would also be helpful (as well as raw data so we can calculate overhead per kilobyte and so on). I know there are only two days worth of data, but for future, it'd be good. I think the next part of figuring out why there's such huge disparity is instrumenting bitcoind to find out where the time goes when relaying a block. On Sun, Nov 24, 2013 at 5:26 PM, Gregory Maxwell gmaxw...@gmail.com wrote: On Sun, Nov 24, 2013 at 8:20 AM, Christian Decker decker.christ...@gmail.com wrote: Since this came up again during the discussion of the Cornell paper I thought I'd dig up my measurement code from the Information Propagation paper and automate it as much as possible. Could you publish the block ids and timestamp sets for each block? It would be useful in correlating propagation information against block characteristics. -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Who or what is /Satoshi:0.8.99/Gangnam Style:2.1/ ?
Thanks. By the way, your bitnodes site is excellent. Thanks for doing that. If you're in the mood for extending it, it'd be great to gather and chart data on block and tx propagation times. Do you think the recent explosion in running nodes is real, or due to some kind of custom experimental thing? On Thu, Nov 21, 2013 at 2:55 PM, Addy Yeow ayeo...@gmail.com wrote: Try https://bitcointalk.org/index.php?action=profile;u=19897? On Fri, Nov 22, 2013 at 12:48 AM, Mike Hearn m...@plan99.net wrote: I added some additional logging to my node and ran it for a few days. There's a pull req open for my extra logging, it is quite trivial. Here's what it looks like: 2013-11-21 13:41:04 AcceptToMemoryPool: 5.9.24.81:7834/Satoshi:0.8.99/Gangnam Style:2.1/ : accepted 2d1bbcc2bf64dfcb57a2f0180b2607a48a34de4422c446929b26b190083bbfe7 (poolsz 2087) 2013-11-21 13:41:05 AcceptToMemoryPool: 198.12.127.2:29057/Satoshi:0.8.99/Gangnam Style:2.1/ : accepted 28bb94978bdaa224faeafa95d03a0c4f5743396d6f592469c5ac2b64184ac716 (poolsz 2088) 2013-11-21 13:41:06 ERROR: AcceptToMemoryPool : nonstandard transaction: dust 2013-11-21 13:41:06 42323d9553e4c592d27765dc3ef9152c186cb7d67b08d783d72974a56085032d from 82.68.68.254:39232 /Satoshi:0.8.1/ was not accepted into the memory pool: dust 2013-11-21 13:41:06 AcceptToMemoryPool: 198.12.127.2:29057/Satoshi:0.8.99/Gangnam Style:2.1/ : accepted 2fdb19e5e87d518b7b6bb7371d547a5f60c2bb056ba4522190460f0bc41b51fb (poolsz 2089) 2013-11-21 13:41:08 AcceptToMemoryPool: 5.9.24.81:7834/Satoshi:0.8.99/Gangnam Style:2.1/ : accepted 52c8ed6a48f89d48b1152b67ac0b718a7aadb5f9a0c70c18b9b2fed058ca3323 (poolsz 2090) 2013-11-21 13:41:08 AcceptToMemoryPool: 198.12.127.2:29057/Satoshi:0.8.99/Gangnam Style:2.1/ : accepted 980bbdbd4a6b365fa6f13fb5247eb6cb1e54847e490c3b7c3026d1548fb9efc6 (poolsz 2091) 2013-11-21 13:41:08 AcceptToMemoryPool: 64.120.253.194:60896/Satoshi:0.8.99/Gangnam Style:2.0/ : accepted 03f79c611bbdc1afa7afa67eb0bbd4d8bc86a730a7066622e2709ae506e61e0f (poolsz 2092) 2013-11-21 13:41:10 AcceptToMemoryPool: 5.9.24.81:7834/Satoshi:0.8.99/Gangnam Style:2.1/ : accepted af8096ad637af1ca022a5146e07cf1fc6bfbec877935f9e114b279fcfe26c06d (poolsz 2093) 2013-11-21 13:41:10 AcceptToMemoryPool: 5.9.24.81:7834/Satoshi:0.8.99/Gangnam Style:2.1/ : accepted 751c2415d058d45ca602fdf1b6490edb6e57fc718e914d628c11b17e25aac834 (poolsz 2094) Despite that I have 87 connections from regular nodes, virtually all transactions seen by my node are being announced by this modified software, which appears to run on several different machines. I am wondering if anyone out there knows/owns these nodes and if they are relaying transactions without checking their validity. That seems the most likely reason for how they are always able to win the race to be the first to announce to my node. -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Who or what is /Satoshi:0.8.99/Gangnam Style:2.1/ ?
I added some additional logging to my node and ran it for a few days. There's a pull req open for my extra logging, it is quite trivial. Here's what it looks like: 2013-11-21 13:41:04 AcceptToMemoryPool: 5.9.24.81:7834/Satoshi:0.8.99/Gangnam Style:2.1/ : accepted 2d1bbcc2bf64dfcb57a2f0180b2607a48a34de4422c446929b26b190083bbfe7 (poolsz 2087) 2013-11-21 13:41:05 AcceptToMemoryPool: 198.12.127.2:29057/Satoshi:0.8.99/Gangnam Style:2.1/ : accepted 28bb94978bdaa224faeafa95d03a0c4f5743396d6f592469c5ac2b64184ac716 (poolsz 2088) 2013-11-21 13:41:06 ERROR: AcceptToMemoryPool : nonstandard transaction: dust 2013-11-21 13:41:06 42323d9553e4c592d27765dc3ef9152c186cb7d67b08d783d72974a56085032d from 82.68.68.254:39232 /Satoshi:0.8.1/ was not accepted into the memory pool: dust 2013-11-21 13:41:06 AcceptToMemoryPool: 198.12.127.2:29057/Satoshi:0.8.99/Gangnam Style:2.1/ : accepted 2fdb19e5e87d518b7b6bb7371d547a5f60c2bb056ba4522190460f0bc41b51fb (poolsz 2089) 2013-11-21 13:41:08 AcceptToMemoryPool: 5.9.24.81:7834/Satoshi:0.8.99/Gangnam Style:2.1/ : accepted 52c8ed6a48f89d48b1152b67ac0b718a7aadb5f9a0c70c18b9b2fed058ca3323 (poolsz 2090) 2013-11-21 13:41:08 AcceptToMemoryPool: 198.12.127.2:29057/Satoshi:0.8.99/Gangnam Style:2.1/ : accepted 980bbdbd4a6b365fa6f13fb5247eb6cb1e54847e490c3b7c3026d1548fb9efc6 (poolsz 2091) 2013-11-21 13:41:08 AcceptToMemoryPool: 64.120.253.194:60896/Satoshi:0.8.99/Gangnam Style:2.0/ : accepted 03f79c611bbdc1afa7afa67eb0bbd4d8bc86a730a7066622e2709ae506e61e0f (poolsz 2092) 2013-11-21 13:41:10 AcceptToMemoryPool: 5.9.24.81:7834/Satoshi:0.8.99/Gangnam Style:2.1/ : accepted af8096ad637af1ca022a5146e07cf1fc6bfbec877935f9e114b279fcfe26c06d (poolsz 2093) 2013-11-21 13:41:10 AcceptToMemoryPool: 5.9.24.81:7834/Satoshi:0.8.99/Gangnam Style:2.1/ : accepted 751c2415d058d45ca602fdf1b6490edb6e57fc718e914d628c11b17e25aac834 (poolsz 2094) Despite that I have 87 connections from regular nodes, virtually all transactions seen by my node are being announced by this modified software, which appears to run on several different machines. I am wondering if anyone out there knows/owns these nodes and if they are relaying transactions without checking their validity. That seems the most likely reason for how they are always able to win the race to be the first to announce to my node. -- Shape the Mobile Experience: Free Subscription Software experts and developers: Be at the forefront of tech innovation. Intel(R) Software Adrenaline delivers strategic insight and game-changing conversations that shape the rapidly evolving mobile landscape. Sign up now. http://pubads.g.doubleclick.net/gampad/clk?id=63431311iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Testnet under attack?
I don't use testnet much anymore, partly because it sometimes kind of breaks like this. It's a public resource and people sometimes abuse it. You can create your own local network with -regtest and that lets you mint new blocks instantly. It's a much simpler way to do testing and app development. On Fri, Nov 15, 2013 at 8:34 PM, Mike Belshe m...@belshe.com wrote: It appears that someone is minting new blocks literally every couple of seconds on the testnet chain right now. You can see it on both blockexplorer: http://blockexplorer.com/testnet and also btclook: http://testnet.btclook.com/ Is this something we should worry about? thanks, Mike -- DreamFactory - Open Source REST JSON Services for HTML5 Native Apps OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access Free app hosting. Or install the open source package on any LAMP server. Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! http://pubads.g.doubleclick.net/gampad/clk?id=63469471iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- DreamFactory - Open Source REST JSON Services for HTML5 Native Apps OAuth, Users, Roles, SQL, NoSQL, BLOB Storage and External API Access Free app hosting. Or install the open source package on any LAMP server. Sign up and see examples for AngularJS, jQuery, Sencha Touch and Native! http://pubads.g.doubleclick.net/gampad/clk?id=63469471iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] [ANN] High-speed Bitcoin Relay Network
I took a brief look at the code - it's looking very reasonable. You can
replace any construct like
try {
Thread.sleep(1000);
} catch (InterruptedException e) {
throw new RuntimeException(e);
}
which is quite verbose, just with
Uninterruptibles.sleepUninterruptably(1000, TimeUnit.MILLISECONDS); (and of
course static imports help too)
I think for this concept to take off, you'd need a website and to recruit
someone to help you market it. Pool operators won't reach out to you.
I still find it perhaps more elegant to just boost the connectivity of the
existing network with bitcoind changes, but this can help for now.
On Thu, Nov 7, 2013 at 12:35 AM, Matt Corallo bitcoin-l...@bluematt.mewrote:
No, the transactions relayed are piped through a bitcoind first (ie
fully verified by a bitcoind). For blocks, for which the timing needs to
be tighter, bitcoinj does SPV-validation. Though it is possible to
create a block which passes SPV validation but causes a DoS score, doing
so would cost a miner a full block's worth of profits, which they are
fairly unlikely to do. In any case, if it every becomes a problem, its
not hard to adapt addnode to allow higher DoS scores for individual nodes.
Matt
On 11/06/13 07:25, Tier Nolan wrote:
On Wed, Nov 6, 2013 at 5:50 AM, Matt Corallo bitcoin-l...@bluematt.me
mailto:bitcoin-l...@bluematt.me wrote:
Relay node details:
* The relay nodes do some data verification to prevent DoS, but in
order to keep relay fast, they do not fully verify the data they are
relaying, thus YOU SHOULD NEVER mine a block building on top of a
relayed block without fully checking it with your own bitcoin
validator
(as you would any other block relayed from the P2P network).
Wouldn't this cause disconnects due to misbehavior?
A standard node connecting to a relay node would receive
blocks/transactions that are not valid in some way and then disconnect.
Have you looked though the official client to find what things are
considered signs that a peer is hostile? I assume things like double
spending checks count as misbehavior and can't be quickly checked by a
relay node.
Maybe another bit could be assigned in the services field as relay.
This means that the node doesn't do any checking.
Connects to relay nodes could be command line/config file only. Peers
wouldn't connect to them.
--
November Webinars for C, C++, Fortran Developers
Accelerate application performance with scalable programming models.
Explore
techniques for threading, error checking, porting, and tuning. Get the most
from the latest Intel processors and coprocessors. See abstracts and
register
http://pubads.g.doubleclick.net/gampad/clk?id=60136231iu=/4140/ostg.clktrk
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
--
November Webinars for C, C++, Fortran Developers
Accelerate application performance with scalable programming models. Explore
techniques for threading, error checking, porting, and tuning. Get the most
from the latest Intel processors and coprocessors. See abstracts and register
http://pubads.g.doubleclick.net/gampad/clk?id=60136231iu=/4140/ostg.clktrk___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] On the optimal block size and why transaction fees are 8 times too low (or transactions 8 times too big)
I think trying to help miners figure out the propagation/fees tradeoff at the moment is a non-starter until we understand it better ourselves. A server that tracks and records block propagation times, how many fees per passed up per block, orphan stats per size bucket etc would be tremendously helpful. On Thu, Nov 7, 2013 at 4:19 PM, Pieter Wuille pieter.wui...@gmail.comwrote: Correcting myself: On Thu, Nov 7, 2013 at 4:00 PM, Pieter Wuille pieter.wui...@gmail.com wrote: I believe that C. Decker's paper used measurements for propagation delays for blocks 18-19, which happened between may and juli 2012. The latest bitcoind/bitcoin-qt release at the time was 0.6.3. They did use data from blocks 2-21, september-november 2012. That was still before the 0.8 release, however. -- Pieter -- November Webinars for C, C++, Fortran Developers Accelerate application performance with scalable programming models. Explore techniques for threading, error checking, porting, and tuning. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60136231iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- November Webinars for C, C++, Fortran Developers Accelerate application performance with scalable programming models. Explore techniques for threading, error checking, porting, and tuning. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60136231iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] we can all relax now
Once the ASIC race calms down because everyone has one, has more or less optimal power supplies, process improvements aren't easily reachable anymore etc then I'd expect people to dissipate from the large pools because eliminating their fees will become the next lowest hanging fruit to squeeze out extra profit. There's no particular reason we need only a handful of pools that control a major fraction of the hashpower. If we end up with a few hundred pools or lots of miners on p2pool, then a lot of these theoretical attacks become not very relevant (I don't think ID sacrifices will be so common or large as to justify a pile of custom mining code+strategies at any point ...) On Thu, Nov 7, 2013 at 2:24 PM, Peter Todd p...@petertodd.org wrote: On Thu, Nov 07, 2013 at 02:56:56PM +1000, Gavin Andresen wrote: P.S: If any large pools want to try this stuff out, give me a shout. You have my PGP key - confidentiality assured. If I find out one of the large pools decides to run this 'experiment' on the main network, I will make it my mission to tell people to switch to a more responsible pool. I hope they listen. A few months ago ASICMiner could have made use of that attack if my memories of their peak hashing power were correct. They certainely could have used the selfish miner version, (we need better name for that) although development costs would eat into profits. GHash.IO, 22%, says they're a private Bitfury ASIC mining pool - dunno what they mean by that, but they're involved with CEX.IO who has physical control of a bunch of hashing power so I guess that means their model is like ASICMiners. They're a bit short of 30%, but maybe some behind-the-scenes deals would fix that, and/or lowering the barrier with reactive block publishing. (a better name) And if you think you can get away with driving up EVERYBODY's orphan rate without anybody noticing, you should think again. ...and remember, if you only do the attack a little bit, you still can earn more profit, and only drive up the orphan rate a little bit. So who knows, maybe the orphans are real, or maybe they're an attack? ASICMiner was involved with a bunch of orphans a while back... You know what this calls for? A witchhunt! BURN THE LARGE POOLS! P.P.S: If you're mining on a pool with more than, like, 1% hashing power, do the math on varience... Seriously, stop it and go mine on a smaller pool, or better yet, p2pool. That I agree with. Glad to hear. -- 'peter'[:-1]@petertodd.org 0007bd936f19e33bc8b8f9bb1f4c013b863ef60a7f5a6a5d2112 -- November Webinars for C, C++, Fortran Developers Accelerate application performance with scalable programming models. Explore techniques for threading, error checking, porting, and tuning. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60136231iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- November Webinars for C, C++, Fortran Developers Accelerate application performance with scalable programming models. Explore techniques for threading, error checking, porting, and tuning. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60136231iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] [ANN] High-speed Bitcoin Relay Network
Very cool, thanks Matt. I was actually thinking this morning, maybe we should require all nodes to go through the inv/getdata dance. Otherwise it's possible to improve your chances at racing a block by mining a block, waiting to see a block inv from another node, then blasting out your block while other nodes are still waiting on their getdatas. On Wed, Nov 6, 2013 at 6:50 AM, Matt Corallo bitcoin-l...@bluematt.mewrote: Recently, there has been a reasonable amount of discussion about the continued fragility of the public Bitcoin network on IRC and elsewhere (1). To this extent, I'm organizing a system of peering between nodes in the network by creating a system of high-speed relay nodes for miners and merchants/exchanges. This system will a) act as a fallback in the case that the public Bitcoin network encounters issues and b) decrease block propagation times between miners. It is NOT designed to in any way replace or decrease the need for the public Bitcoin P2P network. It is NOT any kind of attempt at centralization, and I still encourage interested parties to establish their own private peering agreements with large miners as needed. Currently the network consists of one specially-designed relay node, but I hope to bring more online in the coming days. This network is open to everyone via a few public relay nodes, but also will have nodes which are made available only to large miners and merchants/exchanges to mitigate the ability of malicious parties to DoS the network. To peer with the public relay nodes, simply select the closest region out of us-west (West Coast US), us-east (East Coast US), eu (Western Europe), au (Australia), or jpy (Japan) and add public.REGION.relay.mattcorallo.com to your addnode list. Note that since all of the relay nodes will relay between each other, you gain no latency advantage by peering with more than the closest node to you (and currently all the regions map to one node, so there they're redundant anyway). For each relay node, you can connect to either port 8334 or 8335. Connecting on port 8334 will relay only blocks, and port 8335 will relay both blocks and transactions. The relay nodes will request any transactions which appear in your invs no matter which port you connect to. Relay node details: * The relay nodes do some data verification to prevent DoS, but in order to keep relay fast, they do not fully verify the data they are relaying, thus YOU SHOULD NEVER mine a block building on top of a relayed block without fully checking it with your own bitcoin validator (as you would any other block relayed from the P2P network). * The relay nodes do not follow the standard inv-getdata-tx/block flow, but instead relay transactions/blocks immediately after they have done their cursory verification. They do keep some track of whether or not your nodes claim to have seen the transactions/blocks before relaying, but you may see transactions/blocks being sent which you already have and have not requested, if this is a problem for you due to bandwith issues, you should reconsider your bandwith constraints and/or are peering with too many nodes. * The relay nodes will all relay among themselves very quickly, so there is no advantage to peering with as many relay nodes as you can find, in fact, the increased incoming bandwidth during block relay spikes may result in higher latency for your nodes. * The relay nodes are NOT designed to ensure that you never miss data, and may fail to relay some transactions. Additionally, because the relay nodes do not respond to standard getdata requests, if you miss a relay and then reconnect, that data will not be sent again by the relay nodes. The relay nodes are NOT a replacement for having peers on the standard P2P network, they are only there to augment the existing P2P network. If you are a merchant/exchange/large miner/other important node operator and wish to gain access to additional domain names which map to relay nodes with fewer peers, please fill out the form at https://docs.google.com/forms/d/1UL82QdcXXEhZwSHJAK04Sk_cWg4zLOu8a216nO7Mt8c/viewform You can find the source for the relay nodes at https://github.com/TheBlueMatt/RelayNode If you have any comments/concerns/suggestions, please do not hesitate to email bitcoin-peer...@mattcorallo.com Thanks, Matt (1) There has been extended discussion on #bitcoin-wizards as well as #bitcoin-dev of the very small number of active, listening nodes. Additionally, because many of those nodes are versions prior to 0.8.4, it seems very likely that maliciously creating network splits or at least drastically reducing the number of peers for most nodes would not be particularly challenging in the current network. Also, http://www.tik.ee.ethz.ch/file/49318d3f56c1d525aabf7fda78b23fc0/P2P2013_041.pdf noted that they were able to single-handledly decrease the network-wide orphan rate by around 50% by improving
[Bitcoin-development] Auto-generated miner backbone
W.R.T. this paper and the oft-discussed miner backbone, http://arxiv.org/pdf/1311.0243v1.pdf I'm wondering about an alternative protocol change that perhaps has less subtle implications than their suggested change. Rather than address the problem by assuming the network is full of sybil nodes and changing the rules for selecting the chain to build on, how about if we wrote code to automatically build a miner backbone by having IP addresses of nodes embedded into coinbases, then having any bitcoind that is creating work automatically connect to IPs that appeared in enough recent blocks? This would have the effect of automatically linking all the major pools together, with no administration overhead. For bonus points, the IPs could be IPv6 and then the trick we use to pack hidden services into IPv6 address space would allow nodes to be reached via Tor. This might be useful in the case of pools that don't to reveal the location of their bitcoin node[s], like for anti-DoS reasons. It feels like this should be achievable with a few days of solid coding and a couple of new command line flags, and the impact is much easier to reason about than a fundamental rule change like the one proposed by the paper. -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Auto-generated miner backbone
On Mon, Nov 4, 2013 at 12:53 PM, Peter Todd p...@petertodd.org wrote: I proposed this as a means of giving a mechanism for wallets to get non-sybilled peers as well. Ah yes, good point. Doing so encourages pools to only bother connecting to other pools, which is a strong centralizing force. They could already create such a setup, but we don't observe it in practice. On a technical level, the coinbase is limited in size, and people use it for other purposes, so lets define a standard Given that IP address data is inherently transient, perhaps a better solution is to define a short hash in the coinbase that commits to extra data that is relayed along with block data (e.g. appended to the block message). It can then be stored temporarily in the block db and erased after some time, like a few months. It would therefore not really be a part of the chain, but could be extended as we see fit with any other semi-transient data required. A new getextra message would let nodes query for it. The hash can be short because it doesn't have to survive brute forcing attacks longer than the expected validity period of the transient data anyway. 80 bits would probably be overkill. -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Auto-generated miner backbone
On Mon, Nov 4, 2013 at 3:26 PM, Peter Todd p...@petertodd.org wrote: The attacker now only needs to connect to every identified miner with especially fast nodes. With judicious use of DoS attacks and low latency . So you're back to a complicated sybil attack. I don't follow your thought process here - I didn't say anything about numerical advantage. The attack outlined in the paper *requires* you to be able to race the rest of the network and win some non-trivial fraction of the time. If you can't do that then all it means is that when you try to release a private block to compete with the other found block, you're quite likely to lose and you sacrifice the block rewards by doing so. The correct, and rational, approach for a miner is to always mine to extend the block that the majority of hashing power is trying to extend. There's no stable way to know that. The whole purpose of the block chain to establish the majority. I think your near-miss headers solution is circular/unstable for that reason, it's essentially a recursive solution. Mining strategy is now to mine to extend the first block you see, on the assumption that the earlier one probably propagated to a large portion of the total hashing power. But as you receive near-blocks that are under the PoW target, use them to estimate the hashing power on each fork, and if it looks like you are not on the majority side, switch. But you can't reliably estimate that. You can't even reliably estimate the speed of the overall network especially not on a short term basis like a block interval. -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Committing to extra block data/a better merge-mine standard
I like the UUID-as-path idea. That resolves the problem of how to share the alt-chain merkle tree quite nicely. On Mon, Nov 4, 2013 at 7:16 PM, Peter Todd p...@petertodd.org wrote: No sense in compromising - you need a whole merkle path to prove the extra data is valid so you might as well make this a full 256 bits; The Merkle branch doesn't get stored indefinitely though, whereas the coinbase hash does. The data stored in the coinbase [output] can always just be the 256-bit root hash truncated to less. I doubt the additional bytes make much difference really, so the additional complexity may not be worth it. But it wouldn't be an issue to do. -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Committing to extra block data/a better merge-mine standard
Yes, sure. I was talking about the case of transiently relayed data, like IP addresses. On Mon, Nov 4, 2013 at 8:53 PM, Mark Friedenbach m...@monetize.io wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA1 On 11/4/13 11:38 AM, Mike Hearn wrote: The Merkle branch doesn't get stored indefinitely though, whereas the coinbase hash does. The data stored in the coinbase [output] can always just be the 256-bit root hash truncated to less. I doubt the additional bytes make much difference really, so the additional complexity may not be worth it. But it wouldn't be an issue to do. The bits make a difference if you are merged mining. You can use the birthday attack to construct two data trees whose hash match the (truncated) value, each containing separate aux block headers. This allows you to double-count the bitcoin PoW for more than one aux block on the same chain, potentially facilitating aux chain attacks. If you want 128 bits of security for merged mined aux chains, you need 256 bits of hash in the coinbase. -BEGIN PGP SIGNATURE- Version: GnuPG/MacGPG2 v2.0.19 (Darwin) Comment: GPGTools - http://gpgtools.org Comment: Using GnuPG with Thunderbird - http://www.enigmail.net/ iQIcBAEBAgAGBQJSd/shAAoJEAdzVfsmodw48a0P/RaCOctBDvhU0THnsUw6nRBm A8oH3Kpio4ZltU4oIT0tznZbUOG2j2xVrmATqXDYOZQ6FuGihjmkKJ9jHgl57pb5 0qDdCBiEuWtLIh2+Awrb3Y0s8czyCQP9/1CJyzdEFmI8rSwCaqJMa6B2Ny6Xz6+8 eiK45YdXCPgdTAb56FKOi9WzOe0g1aOO5KiUOci22xRkXvh4qPYrt2F0LIgjZTdC koyXU6dcKON9H8Cecu+ag7jJ5A9ZDj7oIq5rflEyolh2V4ie0tGQ50rFGg/ii6iQ Tz9AWwigsHEkuinBTuN5041Xb8nAgHLvA60RQ41lWUHJxfAvDE+wN6NqgHmMVaRo NHqlZcCuEl1jn7HW81XQTpgarrXHk1G7b2vK10pB/lUxUNIstZvCSjcp8QdtmC9v tIhC2czSnsQaE6kIBuHxDNZxOlZ8DxBYCAgXSkycwznwzGhFPP0xB1lV9HfaP5+i aikmx5SQmqBXQQKsxmIacoykrfu5x+O2TB/bq8JhJ1ak2jG9LVFyQqjorABVAgA7 pLEN6EomWht5qstaLVfHYpNsLMf6WA7UzRG08HKItUeDPtG7bDx8vBx5TvIUjT44 A0i09bOt8ZIgp+lJ8lFLWiPLChViAoy7fqKy2vrdsZerOF3l4LUQeQO/xnfZc+dG AEG+7iCBOMxJSVoJ5bP6 =nydG -END PGP SIGNATURE- -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Message Signing based authentication
Guys, identity systems for the web are off-topic for this list. Other than the anonymous passports/SINs/fidelity bond ideas, Bitcoin doesn't have any relevance to it. On Sat, Nov 2, 2013 at 2:19 PM, Hannu Kotipalo hannu.kotip...@iki.fiwrote: Maybe this is a bit off-topic, but the *real* answer to the question why-is-nobody-using-ssl-client-certificates is that it would force www pages to be encrypted and would make it a lot more difficult for NSA to log www-trafic. No, it wouldn't. You can log a user in using SSL and then redirect the user back to an encrypted page, using cookies for the rest of the session. Please don't clutter up this list with conspiracy theories. The brutal reality is that identity is a hard problem. -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Feedback requested: reject p2p message
But if you are getting soft-forked recent versions of the reference implementation WILL alert you; see this code in main.cpp: Perhaps I'm confused about how we're using the term soft fork. My understanding is that this is where a new upgrade is designed to look valid to old nodes, and if you don't upgrade you rely on the miner majority to get you back on track. For instance, P2SH was done this way - old nodes that didn't upgrade during that transition believed all spends of P2SH outputs were valid, even those spending someone elses coins. In this case, the code you cite won't do anything because your client will never reject a block during a soft-forking upgrade, even if it does something that's supposed to be invalid or nonsensical. If a new block version changes the serialization format or script language or SIGHASH rules such that old clients reject the block, then they will end up on a hard fork and the alerting code will trigger, which is correct and as it should be. -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Feedback requested: reject p2p message
On Wed, Oct 30, 2013 at 10:05 AM, Mark Friedenbach m...@monetize.io wrote: If I understand the code correctly, it's not about rejecting blocks. I was referring to the fork alerts that Matt did. They also alert you if there's a missed upgrade. -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Feedback requested: reject p2p message
For tx reject, should there be a code for unknown version? That is, tx.nVersion bestKnownVersion == reject? In that case 0x40 would become non-standard transaction type. I think unknown transaction type is a bit vague. Or do we want new tx messages to always be backwards compatible? 0x42 and 0x43 seems a bit similar to me. The sender knows what fee was paid (presumably). If free transactions and fee-paying transactions end up having a unified ranking applied, then distinguishing between them in the reject message won't make much sense. For block 0x11 again shall there be a separate code for block is from the future? We don't want to lose the nVersion field to people just using it for nonsense, so does it make sense to reject blocks that claim to be v2 or v3? On Tue, Oct 29, 2013 at 6:37 AM, Gavin Andresen gavinandre...@gmail.comwrote: Thanks for the feedback, everybody, gist updated: https://gist.github.com/gavinandresen/7079034 Categories are: 0x01-0x0f Protocol syntax errors0x10-0x1f Protocol semantic errors0x40-0x4fServer policy rule https://gist.github.com/gavinandresen/7079034#rejection-codes-common-to-all-message-types RE: why not a varint: because we're never ever going to run out of reject codes. Eight are defined right now, if we ever defined eight more I'd be surprised. RE: why not use HTTP codes directly: because we'd be fitting round pegs into square holes. -- -- Gavin Andresen -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Feedback requested: reject p2p message
Yes, exactly. That's the point. As you well know I think the whole soft-fork mechanism is wrong and should not be used. If the rules change, your node is *supposed* to end up on a chain fork and trigger an alert to you, that's pretty much the whole purpose of Bitcoin's design. Undermining that security model is problematic. On Tue, Oct 29, 2013 at 12:38 PM, Peter Todd p...@petertodd.org wrote: -BEGIN PGP SIGNED MESSAGE- Hash: SHA256 Peter Todd p...@petertodd.org wrote: On Tue, Oct 29, 2013 at 10:52:31AM +0100, Mike Hearn wrote: For block 0x11 again shall there be a separate code for block is from the future? We don't want to lose the nVersion field to people just using it for nonsense, so does it make sense to reject blocks that claim to be v2 or v3? That would prevent us from using nVersion as a soft-forking mechanism. Actually, that statement didn't go far enough: rejecting blocks with nVersions that you don't expect is a hard fork. -BEGIN PGP SIGNATURE- Version: APG v1.0.9 iQFQBAEBCAA6BQJSb544MxxQZXRlciBUb2RkIChsb3cgc2VjdXJpdHkga2V5KSA8 cGV0ZUBwZXRlcnRvZGQub3JnPgAKCRAZnIM7qOfwhfuGCADHB+5WZ3oSRCCYgId+ 5c4rxZHjjmXXIVOlXySjoRQ20JUnGbkUqN057VlutYbWaGV7OqR0oQyzh0LGpMdL BU9hg8XoHbyIvA0WhCfEJvFzkwseN8Ac77UxtV3leBpBkSzjqlMS9QBGU6L5rw2U uo8Sd7bQaqkadOPode3MMWDtmmqAZaj2dN02w/8C1rRna3SrbYRVYbaVAuN9yREO 99DOGEM2V7ni+eo4sQoxP2jf8vmNzy1EuQH8v1OloPgcpxl/GkLVXzQh4ZfO1ApE UVKBo93oT34Tce9LwZy+k8XpeCvBRJ/+QwsbAAgdVYKr8KmRcAW4oR2KN7Y0jjq4 44xU =OaON -END PGP SIGNATURE- -- Android is increasing in popularity, but the open development platform that developers love is also attractive to malware creators. Download this white paper to learn more about secure code signing practices that can help keep Android apps secure. http://pubads.g.doubleclick.net/gampad/clk?id=65839951iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Payment protocol for onion URLs.
On Mon, Oct 28, 2013 at 1:14 PM, Adam Back a...@cypherspace.org wrote: Maybe I voice this opinion a bit late in the cycle, but A bit late is one way to put it. All these topics and more were discussed to death a year ago when the payment protocol was first being designed. Bluntly, I think we're all sick of it. You are welcome to PGP sign your payment requests if you want to. If not, then please see my FAQ for discussion: https://bitcointalk.org/index.php?topic=300809.msg3225143#msg3225143 tl;dr - the right way to tackle governments getting bogus certs issued is certificate transparency. All other suggestions tend to boil down to here's some handwaving that doesn't actually solve the problem. By the way, the evidence from the Snowden case rather reinforces the strength of the CA system. Did we see stories about bulk usage of fake certificates? No. What we read is that the increased usage of SSL was a major game-changer for intelligence agencies. They solve SSL by compiling databases of private keys they obtain in various ways. True to form when the FBI wanted access to LavaBit, they tried to obtain his private keys rather than just push a convenient give me a fake cert button, and when it became known that Lavabit had to hand over their key, GoDaddy revoked their certificate. Industry policies forced their hand and those policies don't have a get-out clause for the FBI. It's without a doubt that there are government-issued fake certs floating about, somewhere, just due to the scale of hacking that's been taking place. However, demanding perfection in a system that handles security for over a billion people and tens of millions of operators is unreasonable. All we can ask for is that it it's being improved, which through initiatives like cert transparency, it is. Please, let's call time on these discussions. They long ago ceased to have any value. -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Feedback requested: reject p2p message
Yeah, something like HTTP would work well. I'm really looking forward to this. Currently bitcoinj gets a small but steady stream of bug reports of the form my transaction did not propagate. It's flaky because the library picks one peer to send the transaction to, and then watches it propagate across the network. But if that selected peer refuses the tx for whatever reason, that propagation never comes, and there's currently no timeout to make it retry with a different node. The transactions as created usually look fine, so it's not clear to me why some nodes would accept it others wouldn't given the absence of double spends, and there's no way to debug and find out :( On Sat, Oct 26, 2013 at 6:32 AM, kjj bitcoin-de...@jerviss.org wrote: The HTTP status code system seems to work well enough, and seems to give the best of both worlds. A 3 digit numeric code that is machine-readable, and a freeform text note for humans. The clever part about that system was in realizing that the numeric codes didn't need to account for every possible error. They just need to give the other node the most useful information, like try that again later, I'm having a temporary problem vs. That is just plain wrong and it will still be wrong next time too, so don't bother to retry. We can leave it to the humans to puzzle out the meaning of 403: values of txid gives rise to dom! Gavin wrote: On Oct 26, 2013, at 11:01 AM, Jean-Paul Kogelman jeanpaulkogel...@me.com wrote: Would it make sense to use either fixed length strings or maybe even enums? No. Enums or fixed length strings just make it harder to extend, for no benefit (bandwidth of 'reject' messages doesn't matter, they will be rare and are not relayed). -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Feedback requested: reject p2p message
These nodes are much more likely to just be broken than malicious, but without any way to diagnose why they are dropping a transaction it's hard to find out what's really going on. Anyway, yes, I need to spend time adding timeouts and all kinds of other things, although of course if the transactions are being rejected due to a change in network rules that won't help either - if the nodes you're connected to are silently eating your transaction, there's no sane UI that can result from that without more explicit error handling. On Sun, Oct 27, 2013 at 3:39 PM, Luke-Jr l...@dashjr.org wrote: On Sunday, October 27, 2013 2:32:57 PM Mike Hearn wrote: Currently bitcoinj gets a small but steady stream of bug reports of the form my transaction did not propagate. It's flaky because the library picks one peer to send the transaction to, and then watches it propagate across the network. But if that selected peer refuses the tx for whatever reason, that propagation never comes, and there's currently no timeout to make it retry with a different node. Sounds like the real bug is BitcoinJ relies on good/servant behaviour from other nodes. Don't assume your random node isn't hostile. Handling a peer that doesn't relay your transaction for any reason (including if they lie to you about having done so) should be expected behaviour. Luke -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Making fee estimation better
On Thu, Oct 24, 2013 at 4:30 PM, Peter Todd p...@petertodd.org wrote: Quick thought on how to make blockchain-based fee estimates work better in the context of out-of-band mining contracts: have miners advertise in their coinbase's what fees were actually paid, as opposed to appear to have been paid. This is interesting, but I suppose some miners may have business models that can't be easily summed up as a fee - like all-you-can-eat deals with certain providers, or preference to certain kinds of transactions etc. For the concern that estimation might force fees down too far if miners include private transactions, I thought the estimates were calculated only on broadcast transactions, so transactions that just appear in a block won't ever influence the estimate? -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135991iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] A critique of bitcoin open source community
I was hoping to see something interesting and useful, but all I saw was absurd ranting. Example quote: It is not known where bitcoin contributors are based. Gavin Andersson, a major contributor, is a well-known South African anarchist/crypto-libertarian. Most contributors hide their identities. I don't know who this guy is or why anyone should care what he thinks, but I doubt any of us have time for someone who can't even be bothered spelling Gavin's name correctly, thinks he is South African or would describe him as an anarchist. Open source development can be intimidating and brutal at times, it's probably one factor that causes the massive gender skew. But many pages have been written on that topic, here is probably not the right place to thrash it out for the umpteenth time. -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60135031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Code review
Git makes it easy to fork peoples work off and create long series of commits that achieve some useful goal. That's great for many things. Unfortunately, code review is not one of those things. I'd like to make a small request - when submitting large, complex pieces of work for review, please either submit it as one giant squashed change, or be an absolute fascist about keeping commits logically clean and separated. It really sucks to review things in sequence and then discover that some code you spent some time thinking about or puzzling out got deleted/rewritten/changed in a later commit. It also can make it harder to review things when later code uses new APIs or behaviour changes introduced in earlier commits - you have to either keep it all in your head, do lots of tab switching, or do a squash yourself (in which case every reviewer would have to manually do that). On a related note, github seems to have lost the plot with regards to code review - they are spending their time adding 3D renderers to their diff viewer but not making basic improvements other tools had for years. So, I'd like to suggest the idea of using Review Board: http://www.reviewboard.org/ It's an open source, dedicated code review tool used by lots of big name companies for their internal work. It has git[hub] integration and a lot of very neat features, like the ability to attach screenshots to reviews. Also more basic ones, like side by side diffs. Branches can be and often are submitted to the system as single reviews. The company behind it (disclosure - written and run by a long time friend of mine) offers hosting plans, but we could also host it on a Foundation server instead. -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60134791iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Code review
There is more to a git branch than just the overall difference. Every single log message and diff is individually valuable. When the log messages don't accurately describe the contents of the diff, it's just misinformation and noise. Everyone starts out by wanting a neat collection of easy to understand and review commits, but in practice it's extremely hard to always get it. I know how to make squashed commits, thanks. I've done LOTS of code review in my life. I'm making a point here as one of the few people who goes through large pull requests and reviews them line by line. It's hard, partly because github sucks, and partly because reviewing lots of small commits sucks. There's nothing that makes a single large commit harder to review. It's the same amount of code or strictly less, given the tendency for later commits to change earlier ones. You can easily search the entire change whilst reviewing. There are lots of things that make it easier. FWIW inside Google the code review process is one-commit-one-review. -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60134791iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Code review
On Fri, Oct 4, 2013 at 1:53 PM, Peter Todd p...@petertodd.org wrote: When I'm reviewing multiple commit pull-requests and want to see every change made, I always either click on the Files Changed tab on github, which collapses every commit into a single diff, or do the equivalent with git log. Why doesn't that work for you? The files changed tab definitely works better for reading. In the past comments I put there have disappeared, but I think that can also be true of comments put on the individual commit reviews (which is another issue with github, but it's unrelated to how the commits are presented). So I have lost trust in doing reviews that way. It does make things easier to read though. One advantage of using github is that they're an independent third party; we should think carefully about the risks of furthering the impression that Bitcoin development is a closed process by moving the code review it to a server that we control with explicit review groups. I guess anyone would be able to sign up and comment. -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60134791iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Identity protocol observation
Interesting observation, thanks. I'd think any competent implementation of such an identity scheme would not involve end users directly handling randomized nonsense words, however. I always imagined a sacrifice as being a file that you make with a GUI tool and load into a browser extension. On Thu, Oct 3, 2013 at 3:35 PM, Daniel Lidstrom lidstro...@gmail.comwrote: A couple more thoughts on this: 1) Both c and k can be kept if c is pronounced 'ch', giving ~10.9 bits per phoneme. 2) An extra phoneme (4 encode 43 bits total) gives room to put extra information into the name, e.g. the first 5 bits could be input as the key to a PRP that permutes the last 38 back to a standard encoding of a tx location. This would give the user 32 random names per sacrifice to choose from, and 38 bits to encode its location in the blockchain, which is enough for pretty large blocks. Sample 4 phoneme names: ~milmoz-vyrnyx ~mypnoz-fojzas ~sawfex-bovlec ~fidhut-guvgis ~bobfej-jessuk ~furcos-diwhuw ~wokryx-wilrox ~bygbyl-caggos ~vewcyv-jyjsal ~daxsaf-cywkul They're not that bad IMHO, especially if you get to pick a decent one from a bunch. On Thu, Oct 3, 2013 at 3:35 AM, Daniel Lidstrom lidstro...@gmail.comwrote: The location of a tx in the blockchain can be encoded in n=log2(h)+log2(t) bits, where h is the block height, and t is the number of transactions in the block. Currently h~250,000 and t~500, so n~27. A CVC phoneme encodes ~10.7 bits *, so a transaction today can be located in the blockchain with 3 of these, e.g. reb-mizvig. This is reasonably short, readable and memorable. The identity protocol Jeff Garzik is working on will link a public key fingerprint to a miner sacrifice transaction. This tx could in turn be uniquely described with a short name as above. Associating this name with the public key becomes secure once the tx is sufficiently buried in the blockchain. In the identity protocol, lightweight clients check the validity of a sacrifice tx by checking that its merkle path is valid. But this path encodes, via the ordering of the hashes at each level, the location of the transaction in the block, so the lightweight client can verify the sacrifice tx's short name using only the information he already has. Some more random names: vec-halhic wom-vizpyd guv-zussof jog-copwug seg-rizges jyg-somgod pax-synjem zyg-zuxdyj gid-mutdyj rel-hyrdaj Sources of inspiration: urbit.org https://en.bitcoin.it/wiki/Identity_protocol_v1 * This is somewhat restricted: I disallowed q for obvious reasons and k because it conflicts with c, and c looks much softer and less like Klingon. H is allowed for the first consonant, but not the second, and x is allowed for the last one, but not the first one. Y is a vowel, but not a consonant. Maybe these weren't quite the right choices. Paint away! -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60134791iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60134791iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Identity protocol observation
1) Generate sacrifice proof file using an app 2) Load file into browser 3) Surf Where are the names in that design? I'm not sure where NameCoin comes into this. The point of a sacrifice is it's an anonymous identity, there's no point attaching a name to it. BTW I keep phone numbers in an address book ;) On Thu, Oct 3, 2013 at 5:16 PM, Daniel Lidstrom lidstro...@gmail.comwrote: Fair enough, though people still manage okay with phone numbers. And a decentralized naming system seems to come at great cost - with namecoin you need the whole blockchain to resolve names without trust. Strip out a bell and whistle - meaningfulness and transferability of names - and you get a simple, rudimentary (spam killing!) system that scales on any device. I'll only argue that it seems to be Good Enough *for the types of people who might care about decentralized names*. Probably a very small set :) On Thu, Oct 3, 2013 at 8:00 AM, Mike Hearn m...@plan99.net wrote: Interesting observation, thanks. I'd think any competent implementation of such an identity scheme would not involve end users directly handling randomized nonsense words, however. I always imagined a sacrifice as being a file that you make with a GUI tool and load into a browser extension. On Thu, Oct 3, 2013 at 3:35 PM, Daniel Lidstrom lidstro...@gmail.comwrote: A couple more thoughts on this: 1) Both c and k can be kept if c is pronounced 'ch', giving ~10.9 bits per phoneme. 2) An extra phoneme (4 encode 43 bits total) gives room to put extra information into the name, e.g. the first 5 bits could be input as the key to a PRP that permutes the last 38 back to a standard encoding of a tx location. This would give the user 32 random names per sacrifice to choose from, and 38 bits to encode its location in the blockchain, which is enough for pretty large blocks. Sample 4 phoneme names: ~milmoz-vyrnyx ~mypnoz-fojzas ~sawfex-bovlec ~fidhut-guvgis ~bobfej-jessuk ~furcos-diwhuw ~wokryx-wilrox ~bygbyl-caggos ~vewcyv-jyjsal ~daxsaf-cywkul They're not that bad IMHO, especially if you get to pick a decent one from a bunch. On Thu, Oct 3, 2013 at 3:35 AM, Daniel Lidstrom lidstro...@gmail.comwrote: The location of a tx in the blockchain can be encoded in n=log2(h)+log2(t) bits, where h is the block height, and t is the number of transactions in the block. Currently h~250,000 and t~500, so n~27. A CVC phoneme encodes ~10.7 bits *, so a transaction today can be located in the blockchain with 3 of these, e.g. reb-mizvig. This is reasonably short, readable and memorable. The identity protocol Jeff Garzik is working on will link a public key fingerprint to a miner sacrifice transaction. This tx could in turn be uniquely described with a short name as above. Associating this name with the public key becomes secure once the tx is sufficiently buried in the blockchain. In the identity protocol, lightweight clients check the validity of a sacrifice tx by checking that its merkle path is valid. But this path encodes, via the ordering of the hashes at each level, the location of the transaction in the block, so the lightweight client can verify the sacrifice tx's short name using only the information he already has. Some more random names: vec-halhic wom-vizpyd guv-zussof jog-copwug seg-rizges jyg-somgod pax-synjem zyg-zuxdyj gid-mutdyj rel-hyrdaj Sources of inspiration: urbit.org https://en.bitcoin.it/wiki/Identity_protocol_v1 * This is somewhat restricted: I disallowed q for obvious reasons and k because it conflicts with c, and c looks much softer and less like Klingon. H is allowed for the first consonant, but not the second, and x is allowed for the last one, but not the first one. Y is a vowel, but not a consonant. Maybe these weren't quite the right choices. Paint away! -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60134791iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60134791iu=/4140/ostg.clktrk___ Bitcoin-development mailing list
Re: [Bitcoin-development] Payment Protocol: BIP 70, 71, 72
We could also say that if protocol part (https://) is missing, it's implied automatically. So just: bitcoin:1abc?r=bob.com/r/aZgR I think that's about as small as possible without re-using the pubkey as a token in the url. On Wed, Sep 25, 2013 at 1:35 AM, Gavin Andresen gavinandre...@gmail.comwrote: On Tue, Sep 24, 2013 at 11:52 PM, Mike Hearn m...@plan99.net wrote: BTW, on the make qrcodes more scannable front -- is it too late to change BIP 72 so the new param is just r instead of request? Every byte helps when it comes to qrcodes ... Not too late, assuming there are no objections. Smaller QR codes is a very good reason to change it. -- -- Gavin Andresen -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60133471iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Payment Protocol: BIP 70, 71, 72
Low light shouldn't be an issue for QRcodes generated by phones. They have backlit screens that should always be bright enough. I can see how it might be an issue for printed codes. If your phone has no Bitcoin app installed then being redirected to an invoice page is pretty useless, you still won't be able to pay the bill no matter what (where do you get the money from?). If they are just raw HTTP URLs then it means the effect of scanning a QRcode with a standalone scanner app is different to scanning it inside the wallet, which is unlike all other uses of QRcodes I know of. So I'm not really convinced by that UX yet. Perhaps we can thrash it out in Amsterdam. Right now I'm thinking QRcodes should always contain bitcoin URIs. On Wed, Sep 25, 2013 at 4:31 PM, Jeff Garzik jgar...@bitpay.com wrote: BitPay experimented with QR codes in low light, restaurant and other conditions. QR codes become difficult to use even at 100 chars. On the merchant side, we prefer a short URL that speaks payment protocol if visited via bitcoin client, but will gracefully work if scanned by a phone with zero bitcoin support -- you will simply be redirected to a BitPay invoice page for a normal browser. On Wed, Sep 25, 2013 at 7:59 AM, Andreas Schildbach andr...@schildbach.de wrote: On 09/25/2013 01:45 PM, Mike Hearn wrote: OK, it might fit if you don't use any of the features the protocol provides :) Now you're dver-dramaticing (-: I'm just skipping one feature which I think is useless for QR codes scanned in person. You can try it here: Thanks. A typical request would be around 60 bytes, which should produce an URL with around 100 chars. That should be fine for scanning, but I will experiment. If you're thinking about governments and so on subverting CA's, then there is a plan for handling that (outside the Bitcoin world) called certificate transparency which is being implemented now. Good to hear. Let's see if it gets momentum. Now when you are getting a QR code from the web, it's already being served over HTTPS. So if you're up against an attacker who can break a CA in order to steal your money, then you already lose, the QRcode itself as MITMd. Sure. I was talking about QR codes scanned in person. In the Bluetooth case we might have to keep the address around and use it to do ECDHE or something like that. Yeah, will look at that as soon as we're implementing the payment protocol fully. -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60133471iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Jeff Garzik Senior Software Engineer and open source evangelist BitPay, Inc. https://bitpay.com/ -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60133471iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- October Webinars: Code for Performance Free Intel webinars can help you accelerate application performance. Explore tips for MPI, OpenMP, advanced profiling, and more. Get the most from the latest Intel processors and coprocessors. See abstracts and register http://pubads.g.doubleclick.net/gampad/clk?id=60133471iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Faster databases than LevelDB
LevelDB is fast - very fast if you give it enough CPU time and disk seeks. But it's not the last word in performance. HyperLevelDB is a forked LevelDB with some changes, mostly, finer grained locking and changes to how compaction works: http://hyperdex.org/performance/leveldb/ However, it comes with a caveat - one of the changes they made is to take away write throttling if compaction falls behind, the app itself is expected to do that. Sophia is a competitor to LevelDB. The website claims that in benchmarks it completely smokes LevelDB. I have not explored how it does this or tried to replicate their benchmarks myself: http://sphia.org/index.html http://sphia.org/benchmarks.html It's written in C and BSD licensed. As an example of the kind of speedup they claim to be capable of, they say LevelDB could do 167,476 random reads per second on their SSD based machine. Sophia could do 438,084 reads/sec. Random reads are of course the most interesting for us because that's what UTXO lookups involve. They also compare against HyperLevelDB, where the differences are much less pronounced and actually HyperLevelDB appears to be able to do random writes faster than Sophia. -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Faster databases than LevelDB
Nobody has written code to use a better format, migrate old wallets, etc. On Tue, Sep 17, 2013 at 1:41 PM, Jorge Timón jti...@monetize.io wrote: Only slightly related to this... What's the reason why BerkleyDB is maintained for the wallet? I think it would be a good thing to get rid of the libdb4.8++-dev dependency that makes bitcoind harder to compile on debian and ubuntu. Unless, of course, there's a reason I am missing... On 9/17/13, Mike Hearn m...@plan99.net wrote: LevelDB is fast - very fast if you give it enough CPU time and disk seeks. But it's not the last word in performance. HyperLevelDB is a forked LevelDB with some changes, mostly, finer grained locking and changes to how compaction works: http://hyperdex.org/performance/leveldb/ However, it comes with a caveat - one of the changes they made is to take away write throttling if compaction falls behind, the app itself is expected to do that. Sophia is a competitor to LevelDB. The website claims that in benchmarks it completely smokes LevelDB. I have not explored how it does this or tried to replicate their benchmarks myself: http://sphia.org/index.html http://sphia.org/benchmarks.html It's written in C and BSD licensed. As an example of the kind of speedup they claim to be capable of, they say LevelDB could do 167,476 random reads per second on their SSD based machine. Sophia could do 438,084 reads/sec. Random reads are of course the most interesting for us because that's what UTXO lookups involve. They also compare against HyperLevelDB, where the differences are much less pronounced and actually HyperLevelDB appears to be able to do random writes faster than Sophia. -- Jorge Timón http://freico.in/ -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Simple contacts exchange (was: Social network integration (brainstorm))
The payment protocol doesn't *require* signed certificates, it just gives the option of using them. However if you don't have some kind of cryptographic proof of identity, what stops me putting your name and face into my payment requests and claiming to be you? -- LIMITED TIME SALE - Full Year of Microsoft Training For Just $49.99! 1,500+ hours of tutorials including VisualStudio 2012, Windows 8, SharePoint 2013, SQL 2012, MVC 4, more. BEST VALUE: New Multi-Library Power Pack includes Mobile, Cloud, Java, and UX Design. Lowest price ever! Ends 9/20/13. http://pubads.g.doubleclick.net/gampad/clk?id=58041151iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] An app store and non-network transaction fees
Hey Wendell, Interesting idea you have there! On Wed, Sep 4, 2013 at 9:47 PM, Wendell w...@grabhive.com wrote: Obviously there are a couple of brain-dead approaches: We could track what users do in the app, and send the business a bill (with blockchain proof, of course). It might be simpler to not think of it as an app store, but rather see it as a set of affiliate schemes. To get placed into the apps section you can say that the business must have an affiliate scheme in place (i.e. open to more than just you) and then you use the normal mechanisms of affiliate codes and so on. The apps don't have to be offline. They can (and probably should) be online, so the businesses can retain control of their features and brand. If you refer a lot of users to that business, you get the referral bonuses. Affiliate schemes are a common way for open source projects to monetize - e.g. Firefox development is largely paid for by search engine referrals. It's compatible with the ideals of openness because their income relies directly on their traffic, and there are several competing search engines the projects can play off against each other to get the best prices. Also, users expect search engine integration these days, so they'd be sending search traffic regardless. The main downside, of course, is it distorts technical judgement. You can get projects pushing certain businesses heavily not because it's technically the best thing for users, but because their income depends on it. One alternative funding model you could explore is allowing users to bid on assurance contracts for feature development. This means you think up a bunch of improvements you could make, then allow users to pledge Kickstarter-style towards their development. The upside is it allows the community to direct development, and users feel directly involved and not exploited. The downside is, no recurring income you can use to support yourself whilst engaged in other endeavours. 2) Although our BitcoinKit.framework supports both bitcoind and bitcoinj, we will be using bitcoinj with an integrated VM for the time being. The main draw is SPV, although to be honest we would prefer to support the network more via something like Peter Todd's partial UTXO sets idea (hint hint, anyone?). Bear in mind that regardless of how much *you* want to support the network, it's ultimately *your users* resources that will actually get spent. That's why I'm a bit skeptical of any schemes that rely on random end users donating lots of cpu time or bandwidth to the network. If they want to do it, partial UTXO sets and other interesting ideas are worthwhile, but I guess most users won't. I think Bitcoin will over time be more and more like Tor where relays are run on dedicated servers by people who have some modicum of knowledge and community involvement. -- Learn the latest--Visual Studio 2012, SharePoint 2013, SQL 2012, more! Discover the easy way to master current and previous Microsoft technologies and advance your career. Get an incredible 1,500+ hours of step-by-step tutorial videos with LearnDevNow. Subscribe today and save! http://pubads.g.doubleclick.net/gampad/clk?id=58041391iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Payment Protocol: BIP 70, 71, 72
I think the confidence of the tx is not really the users concern anyway. They wrote it so they know it's valid. If the merchant disagrees for some reason then the user can find out, out of band when the goods/services are not delivered. On Tue, Aug 20, 2013 at 1:19 AM, Gavin Andresen gavinandre...@gmail.comwrote: On Tue, Aug 20, 2013 at 8:15 AM, Andreas Petersson andr...@petersson.atwrote: I was just reviewing the integration work to integrate the Payment Protocol into our products. Is there any notion of a standardized invoice serialisation? If i pay for two Burgers and one Club Mate, how would my Bitcoin Wallet be able to know that? No. There are XML-based (shudder) standards for electronic invoicing that include all sorts of bells and whistles; the PaymentDetails message could easily encapsulate one of them in an 'invoice' field extension. Or we could reinvent the wheel and come up with our own, but I'd rather use an existing standard (or maybe a subset of an existing standard). I didn't want to wade into that swamp for the 1.0 version of the payment protocol. Right now, i would simply put that into memo and come up with my own serialisation mechanism. Two Burgers, one Club Mate seems pretty user-friendly. Second, is there a way to communicate acceptance levels of TX (unconfirmed, 1 conf, 6 conf) maybe using several PaymentACK? No, because the Payment-PaymentACK communication round-trip is done in one, non-persistent http request-response round-trip. I don't think we want to allow merchants to push messages to the wallet (wouldn't take long for merchants to use the opportunity to push annoying advertising at me, I think), and I don't think we want wallets to poll the merchant. Although maybe a payment protocol version 2.0 feature could be a PaymentACK extension that says ask me how the transaction is going at THIS URL in THIS many minutes. -- -- Gavin Andresen -- Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Introducing Performance Central, a new site from SourceForge and AppDynamics. Performance Central is your source for news, insights, analysis and resources for efficient Application Performance Management. Visit us today! http://pubads.g.doubleclick.net/gampad/clk?id=48897511iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Gavin's post-0.9 TODO list...
On Mon, Aug 19, 2013 at 5:09 AM, John Dillon john.dillon...@googlemail.comwrote: Here's another question for you Mike: So does bitcoinj have any protections against peers flooding you with useless garbage? It'd be easy to rack up a user's data bill for instance by just creating junk unconfirmed transactions matching the bloom filter. Unconfirmed transactions that are received show up as unspendable and in most wallets they have a little graphic that changes as more peers announce the tx. So if a peer sent non-existent transactions then they'd allow show up as seen by only one peer, which would look different to how normal broadcast transactions show up. Whether users really notice this graphic or understand what it means is debatable, of course, but all Bitcoin wallets have that problem. I've yet to see any that would successfully communicate the notion of confidence to new, untrained users. That's why the default is to not let you spend unconfirmed transactions, unless they were created by yourself (you're allowed to spend change). bitcoinj does not attempt to handle DoS attacks by malicious remote peers today, because such an attack has never been observed, has no obvious profit motive and as you don't get to choose which nodes the wallets connect to it'd be difficult to pull off. Unless you control the users internet connection of course, but that's a well known caveat which is documented on the website. -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] NODE_BLOOM BIP
The original Bloom filtering spec did not make this feature optional for the same reason gzip isn't an optional part of the PNG specification. I see no reason to revisit that. It's definitely not the case that making every possible feature optional is smart design, often it's the opposite. If in future there are nodes that for some reason can't technically support this feature, then there'd be a stronger rationale for something like this. However no such nodes exist, nor are they likely to in future given that it's a simple feature to implement. For these reason I oppose this BIP. On Sun, Aug 18, 2013 at 4:59 AM, Peter Todd p...@petertodd.org wrote: My draft is as follows. Gregory Maxwell: Can you assign a BIP # for this? The next number, 38, is on the wiki as Passphrase-protected private key by Mike Caldwell, although it isn't in the list so I don't know if that is official or not. BIP: ? Title: NODE_BLOOM service bit Author: Peter Todd p...@petertodd.org Type: Standards Track (draft) Created: 17-08-2013 Abstract This BIP extends BIP 37, Connection Bloom filtering, by defining a service bit to allow peers to advertise that they support bloom filters explicitly. Motivation == BIP 37 did not specify a service bit for the bloom filter service, thus implicitly assuming that all nodes that serve peers data support it. There are however cases where a node may want to provide data, such as mempool transactions and blocks, but do not want to or have not implemented bloom filtering. Additionally it is good practice for nodes to be given options as to the granularity of the services they are providing the public - a full-node operator may be able to donate only a small amount of bandwidth and may want those efforts to be used by other full-node operators. Specification = The following protocol bit is added: NODE_BLOOM = (1 1) In addition the protocol version is increased from 70001 to 70002 in the reference implementation. Nodes that support bloom filters should set that protocol bit. Otherwise it should remain unset. NODE_BLOOM is distinct from NODE_NETWORK, and it is legal to advertise NODE_BLOOM but not NODE_NETWORK. If a node does not support bloom filters but receives a filterload, filteradd, or filterclear message from a peer the node should disconnect that peer immediately. While outside the scope of this BIP it is suggested that DNS seeds and other peer discovery mechanisms support the ability to specify the services required; current implementations simply check only that NODE_NETWORK is set. Design rational === A service bit was chosen as applying a bloom filter is a service. The increase in protocol version is for backwards compatibility. Nodes that require the bloom filter service can set NODE_BLOOM for peers advertising a protocol version 70002, allowing the rest of the implementation to be unchanged. Nodes with implementations that do not know of the NODE_BLOOM bit will be disconnected immediately as though the connection failed for some reason, and thus will not have incoming bandwidth wasted by that peer and can easily connect to another peer. Supporting NODE_BLOOM but not NODE_NETWORK allows for situations where a node may have data that its peers may be interested in, but is not a full node and thus does not have block data in general. For instance an SPV node that receives a full, unfiltered, block from a peer may want to let its SPV peers know about the existence of that block and provide them that data if requested. Those peers in turn may only be interested in knowing about the block if it matches a specific bloom filter. Note how in this example DoS attacks are made prohibitively expensive by the work required to create a valid block header. Reference Implementation https://github.com/bitcoin/bitcoin/pull/2900 Copyright = This document is placed in the public domain. -- 'peter'[:-1]@petertodd.org -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in
Re: [Bitcoin-development] BIP 32.5
I filed a bug in the bitcoinj tracker for this a few days ago referencing rfc 6967, but that RFC is very complicated and I'm not sure it's really necessary to go that far. H(sighash||key) is easy to implement and I feel I understand it better. In our case it wouldn't have helped anyway - if anything it would just delayed discovery of the underlying weakness. The same RNG is typically used to generate both keys and signatures today. However in future it may be the case that people put more effort into generating a really random key because they only have to do it once, and then the signing RNG would be different. Your concern about hardware devices leaking private key bits via a side channel is also well made, although I think you have to find some way to establish trust in these devices anyway as sniffing all their IO traffic and analysing it is really hard (plus it inverts the threat model - if you trust your computer and not your hardware wallet, why do you have a hardware wallet?) The other advantage is that deterministic keys and signatures together mean two instances of the same wallet generate identical transactions given an identical sequence of commands. This could help keep wallets in sync. For example we had a few users who got confused because they had cloned their Android wallets across devices (NOT SUPPORTED!) and then one device updated first, did key rotation, and then the other device showed a transaction that sent all their money to a new address it knew nothing about. If they didn't realise the other device had updated this looked identical to theft! I don't think fractional BIP numbers are the way to go :) but a new BIP that standardised a way to select K would, if reviewed, be something I'd implement. On Fri, Aug 16, 2013 at 4:26 AM, Gregory Maxwell gmaxw...@gmail.com wrote: I am wondering if we shouldn't have a BIP32 addendum which makes the following signing related recommendations: (1) Recommend a specific deterministic DSA derandomization procedure (a deterministic way to generate the DSA nonce), presumably one based on HMAC-SHA512 (since BIP32 uses that construct) or SHA256 in the style of RFC 6979. DSA systems being compromised due to poor randomness at runtime is not new. It effected other systems before it effected Bitcoin systems, it's not a new problem and it's not going away. It's difficult to tell if an implementation is correct or not. Use of a fully deterministic signature would allow for complete test vectors in signing and complete confidence that there is no random number related weakness in a signing implementation. In particular, with relevance to our ecosystem a maliciously modified difficult to audit hardware wallet could be leaking its keys material via its signatures. Even without producing insecure K values it could use the choice of K to leak a couple bits of an encrypted root key with every signature, and allow the malicious party to recover the keys by simply observing the network. Making the signatures deterministic would make this kind of misbehavior practically discoverable. We wouldn't be alone in making this change, in general industry is moving in this direction because it has become clear that DSA is a hazard otherwise. The primary arguments in most spaces against derandomizing DSA are FIPS conformance (irrelevant for us) and reasonable concerns about the risks of using a (less) reviewed cryptographic construct. With widespread motion towards derandomized DSA this latter concern is less of an issue. Libcrypt has also implemented derandomized DSA in git. The ed25519 signature system of DJB, et. al. also uses a similar derandomization. An alternative is implementing a still random construct where K is some H(message||key||random) which should remain secure even where the randomness is poor, but this loses the advantage of being able to externally verify that an implementation is not leaking information. OpenSSL development has implemented a form of this recently. See also: http://tools.ietf.org/rfc/rfc6979.txt (2) Recommends a procedure for using only even S values in signatures, eliminating this source of mutability in transactions. This can be accomplished via post-processing of existing signatures, but since it requires bignum math it is usually preferable to implement it along with signing. I believe someday this will become a network requirement for Bitcoin, but regardless it makes sense to implement it as a best practice sooner rather than later. Thoughts? -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk
Re: [Bitcoin-development] Gavin's post-0.9 TODO list...
Cool. Maybe it's time for another development update on the foundation blog? On Fri, Aug 16, 2013 at 3:00 AM, Gavin Andresen gavinandre...@gmail.comwrote: Mike asked what non-0.9 code I'm working on; the three things on the top of my list are: 1) Smarter fee handling on the client side, instead of hard-coded fees. I was busy today generating scatter-plots and histograms of transaction fees versus priorities to get some insight into what miner policies look like right now. 2) First double-spend relaying and alerting, to better support low-value in-person transactions. Related: *Have *a *Snack*, Pay with *Bitcoins*http://www.tik.ee.ethz.ch/file/848064fa2e80f88a57aef43d7d5956c6/P2P2013_093.pdf 3) Work on 2-3 whitepapers on why we need to increase or remove the 1MB block size limit, how we can do it safely, and go through all of the arguments that have been made against it and explain why they're wrong. -- -- Gavin Andresen -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Gavin's post-0.9 TODO list...
The only other thing I'd like to see there is the start of a new anti-DoS framework. I think once the outline is in place other people will be able to fill it in appropriately. But the current framework has to be left behind. If I had to choose one thing to evict to make time for that, it'd be the whitepapers. At the moment we still have plenty of headroom in block sizes, even post April. It can probably be safely delayed for a while. On Fri, Aug 16, 2013 at 2:11 PM, Mike Hearn m...@plan99.net wrote: Cool. Maybe it's time for another development update on the foundation blog? On Fri, Aug 16, 2013 at 3:00 AM, Gavin Andresen gavinandre...@gmail.comwrote: Mike asked what non-0.9 code I'm working on; the three things on the top of my list are: 1) Smarter fee handling on the client side, instead of hard-coded fees. I was busy today generating scatter-plots and histograms of transaction fees versus priorities to get some insight into what miner policies look like right now. 2) First double-spend relaying and alerting, to better support low-value in-person transactions. Related: *Have *a *Snack*, Pay with *Bitcoins*http://www.tik.ee.ethz.ch/file/848064fa2e80f88a57aef43d7d5956c6/P2P2013_093.pdf 3) Work on 2-3 whitepapers on why we need to increase or remove the 1MB block size limit, how we can do it safely, and go through all of the arguments that have been made against it and explain why they're wrong. -- -- Gavin Andresen -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Gavin's post-0.9 TODO list...
A ban-subnet RPC would be a reasonable addition, but obviously DoS attackers that are IP or bandwidth constrained are really just script kiddies. Also anything that involves every node operator doing manual intervention rather works against decentralisation and having a big network. That's why I keep pushing for automated heuristic driven prioritisation. On Fri, Aug 16, 2013 at 3:41 PM, Warren Togami Jr. wtog...@gmail.comwrote: https://togami.com/~warren/archive/2013/example-bitcoind-dos-mitigation-via-iptables.txt *Anti-DoS Low Hanging Fruit: source IP or subnet connection limits* If you disallow the same IP and/or subnet from establishing too many TCP connections with your node, it becomes more expensive for attackers to use a single host exhaust a target node's resources. This iptables firewall based example has almost zero drawbacks, but it is too complicated for most people to deploy. Yes, there is a small chance that you will block legitimate connections, but there are plenty of other nodes for random connections to choose from. Configurable per source IP and source subnet limits with sane defaults enforced by bitcoind itself would be a big improvement over the current situation where one host address can consume limited resources of many target nodes. This doesn't remove the risk of a network-wide connection exhaustion attack by a determined attacker, but it at least makes multiple types of attacks a lot more expensive. This also doesn't do much against the io vulnerability, which would require major redesigns to prevent in Bitcoin. https://github.com/litecoin-project/litecoin/commit/db4d8e21d99551bef4c807aa1534a074e4b7964d *Want to safely delay the block size limit increase for another year or two?* This patch alone enables that. On Fri, Aug 16, 2013 at 2:24 AM, Mike Hearn m...@plan99.net wrote: The only other thing I'd like to see there is the start of a new anti-DoS framework. I think once the outline is in place other people will be able to fill it in appropriately. But the current framework has to be left behind. If I had to choose one thing to evict to make time for that, it'd be the whitepapers. At the moment we still have plenty of headroom in block sizes, even post April. It can probably be safely delayed for a while. On Fri, Aug 16, 2013 at 2:11 PM, Mike Hearn m...@plan99.net wrote: Cool. Maybe it's time for another development update on the foundation blog? On Fri, Aug 16, 2013 at 3:00 AM, Gavin Andresen gavinandre...@gmail.com wrote: Mike asked what non-0.9 code I'm working on; the three things on the top of my list are: 1) Smarter fee handling on the client side, instead of hard-coded fees. I was busy today generating scatter-plots and histograms of transaction fees versus priorities to get some insight into what miner policies look like right now. 2) First double-spend relaying and alerting, to better support low-value in-person transactions. Related: *Have *a *Snack*, Pay with *Bitcoins*http://www.tik.ee.ethz.ch/file/848064fa2e80f88a57aef43d7d5956c6/P2P2013_093.pdf 3) Work on 2-3 whitepapers on why we need to increase or remove the 1MB block size limit, how we can do it safely, and go through all of the arguments that have been made against it and explain why they're wrong. -- -- Gavin Andresen -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140
Re: [Bitcoin-development] Gavin's post-0.9 TODO list...
That change was made in response to user complaints. Heck we get complaints about battery life and bandwidth impact even with Bloom filtering. We can't just randomly start using peoples bandwidth for relaying blocks, especially as I guess most SPV nodes are behind NAT. If Gavin is right and the future is dominated by mobiles and tablets, then it will require a change of thinking in how P2P networks work. I think there are plenty of people with private servers who would be willing to run nodes though. I'm not too worried about this. On Fri, Aug 16, 2013 at 4:27 PM, Warren Togami Jr. wtog...@gmail.comwrote: bitcoinj-0.10 release notes: - We now require Bloom-capable (0.8+) peers by default and will disconnect from older nodes. This avoids accidental bandwidth saturation on mobile devices. Given the user-security concern that Peter brings up, reconsideration of this new default behavior in SPV clients may be warranted. On Fri, Aug 16, 2013 at 4:15 AM, Peter Todd p...@petertodd.org wrote: On Fri, Aug 16, 2013 at 10:01:16AM -0400, Peter Todd wrote: Doing this also makes it more difficult to sybil the network - for instance right now you can create SPV honeypots that allow incoming connections only from SPV nodes, thus attracting a disproportionate % of the total SPV population given a relatively small number of nodes. You can then use that to harm SPV nodes by, for instance, making a % of transactions be dropped deterministicly, either by the bloom matching code, or when sent. Users unlucky enough to be surrounded by sybil nodes will have their transactions mysteriously fail to arrive in their wallets, or have their transactions mysteriously never confirm. Given how few full nodes there are, it probably won't take very many honeypots to pull off this attack, especially if you combine it with a simultaneous max connections or bloom io attack to degrade the capacity of honest nodes. Oh, here's an even better way to do the tx drop attack: when you drop a transaction, make a fake one that pays the same scriptPubKeys with the same amount, and send it to the SPV peer instead. They'll see the transaction go through and show up in their wallet, but it'll look like it got stuck and never confirmed. They'll soon wind up with a wallet full of useless transactions, effectively locking them out of their money. Here's another question for you Mike: So does bitcoinj have any protections against peers flooding you with useless garbage? It'd be easy to rack up a user's data bill for instance by just creating junk unconfirmed transactions matching the bloom filter. -- 'peter'[:-1]@petertodd.org 0018dcf5bcc3f018a05517ba1c479b432ba422015d4506496e55 -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Gavin's post-0.9 TODO list...
On Fri, Aug 16, 2013 at 4:59 PM, Peter Todd p...@petertodd.org wrote: UPNP seems to work well for the reference client. What's the situation there on Android? Not sure - it could be investigated. I think UPNP is an entirely userspace-implementable protocol, so in theory it could be done by a userspace library (even libminiupnp - java is not a requirement on android) I leave my phone plugged in and connected via wifi for most of the day; lots of people do that. I suspect you mean I think lots of people do that. I'm not so sure. We could potentially run an experiment in the Android app to measure how many users are in a position to contribute back, but just because you have wifi doesn't mean you can reconfigure it using UPnP. That helps a lot in home networks, but at the office it doesn't help. I'm wary of a ton of work being put in to achieve not very much here. Satoshi's original vision was always that millions of users were supported by 100,000 or so nodes. I don't think that's unreasonable over the long term. Besides, prioritisation isn't very hard. Nodes can just hand clients a signed timestamp which they remember. When re-connecting, the signed timestamp is handed back to the node and it gives priority to those with old timestamps. No state is required on the node side. Signing and checking can be passed onto the general ECDSA thread pool that works its way through pending signature operations, they'd be prioritised lower than checking blocks/broadcasts. -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Version 0.9 goals
Sounds awesome! Pieter told me at lunch that headers first cut sync time to 45 minutes for him, which is another amazing improvement from the master of optimisations. Pieter, Matt and I also agreed that for maximum impact we should really try to ship payment protocol support in at least two clients simultaneously and ideally with a big merchant signed up too - to send a powerful message that we really mean it. Someone volunteered last week to do it for bitcoinj and if he doesn't pull through, I have some old code from EOY 2012 that I could update to the latest spec and ship at least some basic support. I'd hope that we can get Bitcoin Wallet or MultiBit updates out once bcj has support pretty fast. Also, Jeff said that BitPay want to be a leader in support for the protocol. So let's try and co-ordinate release dates so we can make a bit of a splash and grab the ecosystems attention. On Thu, Aug 15, 2013 at 2:29 AM, Gavin Andresen gavinandre...@gmail.comwrote: I plan on spending about half my time on code review and helping get pull requests tested, and the other half of my time working on code that probably won't make it into the 0.9 release. Sounds brilliant. It'll be nice to see the pull request queue drain. Any ideas what the non-0.9 code will be? Fee rework? DoS work? -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Version 0.9 goals
On Thu, Aug 15, 2013 at 10:22 AM, slush sl...@centrum.cz wrote: We're planning to support payment protocol in Trezor as well, if it counts. I think it's a missing piece in absolute security of hardware wallets. Yup, that's always been the plan :-) Any idea how much work it is, and would it be a v1 feature of the Trezor or added later via firmware update? -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Version 0.9 goals
Our plan is to add support for that into v1 firmware, but it also depends on readiness of surrounding infrastructure; mainly if there'll be support for payment protocol in multibit in the time of v1 release (I suppose that the Multibit will be the first wallet compatible with Trezor AND supporting payment protocol). Yeah, OK. Let's see how much progress Gary makes. Supporting HD wallets is the trickiest part and I don't know how much time I will have - the Android RNG issue and getting bcj 0.10 released have sucked up a lot of my time lately and I need to refocus on other things for a bit. But between the guy who volunteered to do payment protocol, and Gary doing TrezorJ, and Matija already having done the core algorithms, I'm hoping the only parts I'll have to do are integrating the HD code with the core wallet code. Possibly if we're running out of time I can do a real basic HD wallet implementation that only iterates a key once and doesn't generate new keys for each transaction, as that's really the trickiest part (because of the need for lookahead/behind and memory bloat on phones). -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] bitcoinj 0.10
Hello, I'm pleased to announce version 0.10 of bitcoinj, a Java library for writing Bitcoin applications. BitcoinJ has been used to create everything from end-user wallet apps to network crawlers to SatoshiDice. To learn how to obtain bitcoinj 0.10, please see the following page: https://code.google.com/p/bitcoinj/wiki/UsingMaven The v0.10 release is signed by Andreas Schildbach's GPG key. The git hash of the release is 777e6781d789. This paragraph is signed by the same Bitcoin key as with previous releases (check their release announcements to establish continuity). Signature: H9Nl7FPnmrUOmjhUZ0+xB4YW3q5F5gIkGdvllsDWmWYvOkNQHAE9jZE0I/qE1VfLPeMV+Rzo7geTB43uDSFSMek= *New in this release* - An implementation of *micropayment channels* was added. There have been many bugfixes and improvements since the first announcement. This feature allows you to set up a 1:1 payment relationship with a remote server and after a short setup process send very tiny payments, very rapidly. It's suitable for metered billing applications. An article, Working with micropayments explains how to use it. This work was a joint effort between Matt and myself. - A simple sublibrary has been added that provides async IO based client/server classes that transmit length prefixed protocol buffers. - Thanks to Matija Mazi, some classes have been added that implement *the BIP 32 deterministic wallet algorithm*. Note that these classes are not yet used elsewhere in the system and full deterministic wallet support is therefore not available, however, a low level API is available for experimentation. That API is very likely to change in future releases so don't get too attached to it. - Thanks to Gary Rowe, we have integrated *a new Maven plugin* that checks the SHA1 hashes of downloaded dependencies against a hard-coded list. This means that even if an upstream Maven repository or developer were to be compromised, library dependencies could not be switched out for corrupted versions without someone noticing. For 0.10 the dependency hashes were just initialised based on what was already downloaded. In future, reproducible builds of upstream dependencies and auditing of changes would provide better security. You can and should use Gary's pluginhttps://github.com/gary-rowe/BitcoinjEnforcerRules in your own projects to defend against a possible compromise of the bitcoinj repository. - *Callback handling* has been much improved. Each event listener can have an Executor specified which takes responsibility for running the callback. If you don't specify one they run by default on a single background thread, the user thread, instead of the origin framework threads. This means your callbacks no longer need to be thread safe as they're always run serially. You can also change the default executor if you would like to control the thread on which callbacks run, for example to marshal them into your GUI toolkit thread automatically. This fixes some of the most painful parts of the pre-0.10 API, for instance that transaction confidence listeners were not allowed to re-enter the library. - *Exception handling* has also improved. You can assign a global Thread.UncaughtExceptionHandler which receives any exceptions thrown on the user thread (i.e. by your own event listeners), as well as any internal exceptions thrown by network threads (like inability to parse a message sent by a remote peer). Because your listeners now run on a separate thread by default, you can no longer accidentally cause internal data corruption or prevent other callbacks from running by leaking exceptions out of your callbacks; a subtle knife-edge in the previous API. - Support for *automatic wallet key rotation* has been added. - We now require Bloom-capable (0.8+) peers by default and will disconnect from older nodes. This avoids accidental bandwidth saturation on mobile devices. - The wallet now accepts timelocked transactions if it created them itself. - The wallet can be told to empty itself out, in which case the fee will be subtracted from the total amount instead of added. This simplifies the common case of wanting to send your entire balance whilst still including a fee. - Some JNI peers for event listeners were added. Auto-generated JNI bindings are experimental and not yet merged in to the mainline codebase: for now they are available as part of a separate project on github. This work allows you to access the bitcoinj API using relatively natural looking C++ code and an embedded JVM. - You can now register custom PeerFilterProvider implementors to add things to Bloom filters that aren't necessarily in wallets. - We have begun adding nullity annotations to the API. Combined with a strong static analysis engine like FindBugs or the IntelliJ Inspector, you can find cases where you
[Bitcoin-development] Idea for new payment protocol PKI
This is just me making notes for myself, I'm not seriously suggesting this be implemented any time soon. Mozilla Persona is an infrastructure for web based single sign on. It works by having email providers sign temporary certificates for their users, whose browsers then sign server-provided challenges to prove their email address. Because an SSO system is a classic chicken/egg setup, they run various fallback services that allow anyone with an email address to take part. They also integrate with the Google/Yahoo SSO systems as well. The intention being that they do this until Persona becomes big enough to matter, and then they can remove the centralised struts and the system becomes transparently decentralised. In other words, they seem to do a lot of things right. Of course you can already sign payments using an X.509 cert issued to an email address with v1 of the payment protocol, so technically no new PKI is needed. But the benefit of leveraging Persona would be convenience - you can get yourself a Persona cert and use it to sign in to websites with a single click, and the user experience is smart and professional. CAs in contrast are designed for web site admins really so the experience of getting a cert for an email address is rather variable and more heavyweight. Unfortunately Persona does not use X.509. It uses a custom thing based on JSON. However, under the hood it's just assertions signed by RSA keys, so an implementation is likely to be quite easy. From the users perspective, their wallet app would embed a browser and drive it as if it were signing into a website, but stop after the user is signed into Persona and a user cert has been provisioned. It can then sign payment requests automatically. For many users, it'd be just one click, which is pretty neat. -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] SPV client in pure JavaScript?
Oh, I forgot to make it clear - Chrome apps/extensions can make raw TCP socket connections: http://blog.chromium.org/2012/11/introducing-tcp-listen-new-api-for.html You would do it as a packaged app: http://developer.chrome.com/apps/about_apps.html because then they're a lot more similar to native apps (they get their own windows, run offline, etc). But these aren't standard APIs. They're all Chrome extensions. I doubt HTML5 will support USB access anytime soon, for instance, but packaged apps do. On Fri, Aug 9, 2013 at 2:10 PM, Mike Hearn m...@plan99.net wrote: Code that runs inside NativeClient has the same access level as JavaScript does. It's just a way to do things faster. Distribution as a Chrome app via the Chrome store is a fine approach, as long as people understand it's just an app platform like any other. It has pros and cons that must be weighed up. For instance, Chrome for mobile doesn't really do apps, at least not at the moment. Also, you're still limited by what APIs Chrome exposes, which are a strict subset of what a real OS provides. On Fri, Aug 9, 2013 at 2:05 PM, Wendell w...@grabhive.com wrote: Right, I'm not suggesting that we have this wallet in a web app, but rather precisely what you are talking about: using special browser features, and bundling it. I am fundamentally monoculture-opposed, but given Chrome's present installed base, that initial target makes sense to me, provided that we could have a one-click installation (as per normal, via the Chrome Store). Chrome also has this Native Client plug-in: I know next to nothing about it, and this goes off the rails of the Subject, but perhaps an SPV implementation there could be a solution to the concerns you expressed? -wendell grabhive.com | twitter.com/grabhive | gpg: 6C0C9411 On Aug 9, 2013, at 1:48 PM, Mike Hearn wrote: JavaScript is turing complete so of course it can be done. The real question you're asking is, can it be done in a web app? I think the answer is I think no because web apps aren't allowed to make raw TCP socket connections. Now there may be a way around that by using browser-specific things like extensions or installable apps which give your code greater access permissions. This approach means you essentially use Chrome as your app platform instead of a JVM, the assumption presumably being that more users have Chrome than a JVM. The flip side is that users who don't would probably balk at the idea of installing an entire browser in order to run a wallet app, whereas a JVM can be bundled and the resulting app acts like any other. I don't know of a convenient way to statically link Chrome into a regular-looking application. I personally wouldn't find such a design compelling. Whilst Java isn't exactly a great language, JavaScript is significantly worse in virtually all aspects. I don't understand why anyone would want to use JavaScript outside the browser - you get less safety, less performance, fewer features, less mature tools and so on. If the end result is an installable app like any other, all you did is cripple yourself vs the competition that's using languages/platforms designed for it. -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Optional wallet-linkable address format (Re-request)
will always go to the next address in j=1, no matter which chains are used to provide inputs. - Add code to figure out lookaheads for each alternate chain. Not just each chain, but looking ahead a couple chains, too. Luckily, the lookahead doesn't have to be very big for chains j=1 - Add an interface to display and choose the different chains in your wallet, and export the pubkeychaincode in some soon-to-be-standardized format. - Add code and interface to receive and track alternate j-chains from other clients/users, and maintain those. Should we try associating incoming and outgoing chains? What happens if they do it wrong? Meh... Just as one final swipe at this idea, you can see that I gotta do quite a bit of work to support the multi-chain idea, and adds a little extra burden on the user to maintain the organization of the wallet. This would all be totally unnecessary with a simple alternate encoding. Granted, I think the multi-chain idea is good, and one that I will probably implement anyway, but it seems like overkill in terms of developer complexity, and interface complexity to achieve something much simpler. Developers of much simpler/lightweight clients would probably find this prohibitive. On another note: I thought we weren't encouraging automatic payments without requesting from the other party...? It makes me uneasy, but it sounds like group thought has converged on that being acceptable. I bring it up, because there are situations where it makes sense, but it sounds unsafe for general users. Alice will give Bob his own chain for sending Alice money, then a year later Bob will send money automatically to Alice not realizing that the wallet was lost, retired or compromised. It's not that Bob can't ask for a new address, it's that if the interface says Send Money to Alice, that looks legit enough that Bob may not feel it necessary to check with Alice first. That's more of an interface issue though. We can add a warning to check with the recipient that they still have access to wallet 3cQ398x, etc. But I just know someone is going to lose money anyway... -Alan On 06/20/2013 03:32 AM, Mike Hearn wrote: Agree with Jeremy and once the payment protocol work is further along I'd like to see us define an extension that lets you send payment requests containing public keys+chain codes, so further payments can be made push-style with no recipient interaction (e.g. for repeated billing). How apps choose to arrange their chains internally seems like an area for experimentation. I definitely want to implement HD wallets in bitcoinj to allow this and if that means not using the same tree structure as in the BIP then so be it. On Thu, Jun 20, 2013 at 5:54 AM, Jeremy Spilman jer...@taplink.co wrote: BIP 32 already specifies how to use the first three tree levels: M/i/j/k, i~wallet, j~Internal/External, k~address. The first level is actually type-1 derived, and thus we cannot create an arbitrary number of them without pre-computing them from the offline wallet. So it's not free to create new wallets unless we redefine how the levels work. Initially I was thinking that you would share the public key and chain code from [m/i'/0] so that you can receive payments at [m/i'/0/k], for a unique value of 'i' for each receive chain. For the case of generating new receive chains from a *watch-only* wallet, as you say, the options are to either keep a cache of PubKey/ChainCode for unused [m/i'] or simply increment 'j' past 1 for an existing [m/i'/j] -- the concept of 'internal/'external' and change addresses at Depth=2 don't make sense for handing out receive chains to lots of people anyway, and certainly BIP32 doesn't *require* 0 = j = 1. So I think incrementing 'j' is the way to go here... The default layout of BIP32 does NOT mean that implementations should not check for transactions with j 1. That would be a useless constraint and obviously self-limiting. It might be helpful to add to the 'Compatibility' section some minimum expectations about how a wallet should be 'probed' when imported. If you don't feel completely free to monotonically increment 'j' to your hearts content to achieve major usability benefits, then I say BIP32 could use some clarifying. BTW - the spec calls for addition not multiplication now, so we should call it the 'Addend' not the 'Multiplier' :-) Do these extra wallet chains behave as different wallets, or sub-wallets? They could, but they certainly don't need to! A single-wallet implementation treats this merely as an address-generation algorithm, and does not expose any hierarchy to the user interface. The user just “magically” gets the ability to send multiple payments to their contacts without immediately sacrificing their privacy (http://www.wired.com/wiredenterprise/2013/06/bitcoin_retai/). Everything goes into the same ledger, balance, coin pool, etc. Most
Re: [Bitcoin-development] Optional wallet-linkable address format (Re-request)
It's BIP specified and implemented in Bitcoin-Qt so now is the time to start :) I'm hoping that most wallets can announce support near simultaneously On Fri, Aug 9, 2013 at 10:12 PM, Alan Reiner etothe...@gmail.com wrote: That's fine. I just want to make sure it's considered for inclusion at some point, because I really hope to leverage the identity mechanism I just described, and it's much easier if it's part of a standard instead of convincing others to go around the standard with us. I have not spent much time looking at the payment protocol itself. I didn't feel like I'd have much to contribute (besides requesting a feature I know isn't there). I was planning to wait until it was complete before fully grokking and implementing it in Armory. On 08/09/2013 03:58 PM, Mike Hearn wrote: Payment protocol is locked down for v1 already. But did you read it? It doesn't use addresses anywhere. Payments are specified in terms of a list of outputs which can contain any script. Of course it could be a pay-to-address script, but pay-to-address uses more bytes in the chain and there isn't any typeability benefit. The multiplication trick for deterministic keys is a nice one and worth doing, but it has to be a v2 feature by this point. It's more important to get v1 widely implemented and deployed first. On Fri, Aug 9, 2013 at 7:57 PM, Alan Reiner etothe...@gmail.com wrote: Guys, I'd like to reiterate my previous request to support this alternate address serialization in the payment protocol. We got caught up in the specifics of one use case, but didn't acknowledge that it's still a valid address representation that will provide value to those who wish to use it and can be safely ignored by others. Current address format: binary_to_base58( idbyte + hash160(pubkey) + checksum) Alternate format: binary_to_base58( idbyte + parentpubkey + multiplier + checksum) The receiving party will multiply the pubkey by the multiplier, and then hash it to get the 20-byte address to send to. The idea is that you use your BIP 32 parent public key, and then you generate whatever child you want, and only send them the multiplier used (not the chaincode). This preserves privacy, but if the recipient has your parent public key already, they can identify that address being linked to you, but cannot determine any other addresses in your wallet. This form has no drawbacks to the existing address format except for being longer and requiring an extra EC multiplication by the person sending to that address. But the advantage is that it optionally allows the sender to provide more information than currently contained in the 25-byte hash160 form. The discussion about this got side-tracked with the use case I presented, but I believe there are plenty of other uses for this. The particular use case I had in mind was that certain services could be setup (pre-arranged), say between wallet software and a business/exchange. The exchange would like to be able to reliably send addresses to the user for deposit, without risk of MITM, or even if their own public server is compromised. The author of wallet software pre-verifies the public key portion of the service, and either hardcodes it into the software, or hardcodes their own public key into the software and makes the service's signed public key available through query server (allowing the software author to offline-sign replacement keys, or add keys for new service providers, as needed). When the user's software receives a payment address, the software can verify it belongs to that service. You can't use dedicated chain technique, because it would either have to be exchanged with the user on first transaction which half defeats the purpose, or they give them the full public key and chaincode which allows the user to see *all *addresses ever used by the service. Neither one is a reasonable solution. This use case doesn't necessarily scale, but it doesn't have to. It simply allows service providers to skip the SSL and go right to public key exchange/verification for a few of the important services they provide access to, and will provide better security than relying on SSL/PKI. This would simply be one, coexisting option for providing payment details in the absence (or in addition to) SSL/PKI infrastructure. I'm sure there's other use cases, but it seems simple enough and non-disruptive enough that it could be supported easily for no other reason than to support that use case (which I intend to implement in Armory to help verify high-volume services). -Alan On 06/26/2013 11:29 AM, Alan Reiner wrote: Although I'd still prefer my original request, I get much of what I want from your guys' recommendation. It complicates the wallet design, because it requires tracking and associating a matrix of addresses for each wallet, instead of a single linear list. But if this is what
Re: [Bitcoin-development] Payment Protocol: BIP 70, 71, 72
I'd like to hear from other wallet implementors. Do you have a notion of 'locked inputs' ? The tricky bit in constructing a transaction but not broadcasting it right away is the inputs must be locked, so they're not accidentally double-spent. bitcoinj separates the concept of committing a tx to the wallet from broadcasting it. However by default transactions that weren't seen in the chain yet will be announced when a new peer is connected to. It'd take extra code to suppress that, and it's unclear to me why that's useful. I agree with Pieter that it should be the merchants responsibility to get the tx out there, but having the client do the broadcast as well can't really hurt (except perhaps some privacy impact). -- Get 100% visibility into Java/.NET code with AppDynamics Lite! It's a free troubleshooting tool designed for production. Get down to code-level detail for bottlenecks, with 2% overhead. Download for free and get started troubleshooting in minutes. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Preparing for the Cryptopocalypse
They have poor space/bandwidth usage properties, which is one reason why Bitcoin doesn't use them today, but as far as I know the same is so for all post-QC schemes. I believe post-QC schemes based on Regev's LWE assumption are getting competitive with more traditional schemes. A paper from 2010 says they were able to get to around the same as large RSA key sizes (2048 bits), which is much worse than ECC but not entirely infeasible. Especially given that barring some breakthrough, by the time QC is a real problem we'll have gigabit wifi and 32 core devices with a terabyte of storage embedded in our hands :) -- Get your SQL database under version control now! Version control is standard for application code, but databases havent caught up. So what steps can you take to put your SQL databases under version control? Why should you start doing it? Read more to find out. http://pubads.g.doubleclick.net/gampad/clk?id=48897031iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] [bitcoin-list] BitMail - p2p Email 0.1. beta
Support for a TPM is a rather tricky thing. By itself the TPM is independent of any CPU. However, it's also not very useful (though for Pond's use case, it works). The TPM gets much more useful when it's integrated with features on the motherboard, BIOS, CPU, northbridge, IOMMU etc. Then you have a full blown TCG-compliant TC environment, which is useful for many things. Actually it was never very useful for DRM - that was only one theoretical possibility that was never implemented and even if it had been, TC is to DRM much as cryptography is to DRM. So the FUD was just that: fear, uncertainty and doubt which probably crippled a highly useful cryptographic security tool for good. One of the more shameful periods of the tech industries history, if you ask me. On Wed, Jul 31, 2013 at 5:39 AM, Blibbet blib...@gmail.com wrote: On 7/30/13 3:58 PM, grarpamp wrote: [...] And if AMD even has this stuff. [...] Yes, AMD does have TPM. Sorry, not sure which models support it. http://www.amd.com/us/products/embedded/das/Pages/security.aspx http://www.amd.com/us/products/desktop/platforms/Pages/desktop-platforms.aspx -- Get your SQL database under version control now! Version control is standard for application code, but databases havent caught up. So what steps can you take to put your SQL databases under version control? Why should you start doing it? Read more to find out. http://pubads.g.doubleclick.net/gampad/clk?id=49501711iu=/4140/ostg.clktrk ___ bitcoin-list mailing list bitcoin-l...@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-list -- Get your SQL database under version control now! Version control is standard for application code, but databases havent caught up. So what steps can you take to put your SQL databases under version control? Why should you start doing it? Read more to find out. http://pubads.g.doubleclick.net/gampad/clk?id=49501711iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] [bitcoin-list] BitMail - p2p Email 0.1. beta
Sorry, I just noticed that this thread was CCd to the announce list not the development list (why is it open access?) It's offtopic anyway. Let's continue this discussion in private if anyone wants to. On Wed, Jul 31, 2013 at 5:53 PM, Mike Hearn m...@plan99.net wrote: The reason why TPM functionality was so much hated upon is because it was pushed by a software/hardware monopoly, not just for DRM but for locking down the system in general. Regardless of what some people might have imagined or extrapolated at the time, the actual published specifications and technologies were nothing like that. There has never been a TC/TPM mode that would have generally locked systems down or even been useful for DRM (that'd have required a trusted hardware path which was never specced nor implemented). Locking a system down against tampering or for DRM does not require flexible open specifications with multiple competing implementations. It requires you to do an Xbox 360. -- Get your SQL database under version control now! Version control is standard for application code, but databases havent caught up. So what steps can you take to put your SQL databases under version control? Why should you start doing it? Read more to find out. http://pubads.g.doubleclick.net/gampad/clk?id=49501711iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BitMail - p2p Email 0.1. beta
The TPM is a piece of secure* hardware that provides various cryptographic services to the host system. It is important to understand that it is not a crypto accelerator. It is a place to store keys and small pieces of data (like hashes, counters) where it's difficult for someone to extract them even if they have physical access. The TPM is designed to support trusted computing, a rather splendid set of extensions to the x86 architecture that let you do remote attestation, software sealing and other things. Or at least it would be splendid if it had been really finished off and pushed to completion by the designers. Unfortunately due to various political issues it exists in a quasi-finished, semi-broken state which only experts can use. Without a doubt you have never run any software in a TC environment. As part of that role, the TPM provides some permanent storage in the form of NVRAM. Because the TPM is designed to be as cheap as possible, it has a limited number of write cycles. Normally you're meant to store Intel TXT launch control policies and sealed keys there, but Pond uses it in a different way by storing keys there that it encrypts local data with. By erasing the key in the TPM chips memory area, the data on disk is effectively destroyed too. This is useful because modern disks are often SSD drives, or physical metal disks that use log structured file systems. Because flash memory has a limited number of write cycles per cell, internally SSDs have firmware that remap writes from logical addresses to different physical addresses, the goal is to avoid wearing down the drive and extend its useful life. Normally it doesn't matter, but if you want to delete data such that it's really really gone, it obviously poses a problem. Using TPM NVRAM solves it, albiet, at a high usability cost. *note: actual tamper resistance of real-world TPM chips is not something that seems to have been studied much On Tue, Jul 30, 2013 at 1:27 PM, Wendell w...@grabhive.com wrote: Can you explain this process for those of us not too familiar with TPM chips? -wendell grabhive.com | twitter.com/grabhive | gpg: 6C0C9411 On Jul 30, 2013, at 10:40 AM, Mike Hearn wrote: As a testament to the seriousness with which Pond takes forward security, it can use the NVRAM in a TPM chip to reliably destroy keys for data that an SSD device might have otherwise made un-erasable. -- Get your SQL database under version control now! Version control is standard for application code, but databases havent caught up. So what steps can you take to put your SQL databases under version control? Why should you start doing it? Read more to find out. http://pubads.g.doubleclick.net/gampad/clk?id=49501711iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Tor and Bitcoin
Various ideas are possible: * Use the Tor SOCKS proxy in such a way that it creates a guaranteed independent circuit to a different exit node each time you connect. This gets you back to the slightly stronger clearnet heuristic of if I saw a bunch of peers announce my tx, then it's probably valid. I don't know if this is possible. * Have a set of hard-coded long term stable hidden peers, that are run by known community members who are not going to collaborate to defraud people. Of course if they're run by people who are well known that rather defeats the point of them being hidden, but you benefit from the fact that the .onion names double as authentication tokens. * Talk the Tor protocol directly and have the app explicitly pick its own diverse set of exit nodes, one per p2p connection. This is likely to be complicated. Last time I looked Tor doesn't provide any kind of library or API. I agree that it's a kind of theoretical attack right now, but then again, I'm not aware of any countries that block Bitcoin either. The thing with Thailand seems like it might be the result of some confusion over who exactly can make laws in that country. I'd be more concerned about Argentina, but we're a long way from ISPs searching for people to arrest by looking for port 8333. Supporting SOCKS (really: blocking sockets) would be a good thing anyway. Using blocking sockets also means we'd get SSL support, so if at some point Bitcoin nodes start supporting SSL we'd be able to use it more easily. -- Get your SQL database under version control now! Version control is standard for application code, but databases havent caught up. So what steps can you take to put your SQL databases under version control? Why should you start doing it? Read more to find out. http://pubads.g.doubleclick.net/gampad/clk?id=49501711iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] [bitcoin-list] BitMail - p2p Email 0.1. beta
TPMs have come as standard with nearly all computers (except Macs, doh) for a long time. They certainly don't cost $100. More like a few dollars at most. That's why they're so slow. On Tue, Jul 30, 2013 at 10:43 PM, grarpamp grarp...@gmail.com wrote: On Tue, Jul 30, 2013 at 8:12 AM, Mike Hearn m...@plan99.net wrote: The TPM is a piece of secure* hardware I've seen some motherboards with a TPM module header but none came with it installed. I think the modules themselves might be $50-$100 range. They might come with some API docs. Some of you might have links to ones you've used... As part of that role, the TPM provides some permanent storage in the form of NVRAM. Because the TPM is designed to be as cheap as possible, it has a limited number of write cycles. Normally you're meant to store Intel TXT launch control policies and sealed keys there the goal is to avoid wearing down the drive and extend its useful life. Normally it doesn't matter, but if you want to delete data such that it's really really gone, it obviously poses a problem. Using TPM NVRAM solves it, albiet, at a high usability cost. If said TPM storage has a 'limited [but unfixed number of write cycles', that sounds unreliable. It would seem to me that both reliable and 'really gone' are achievable on platters (or lesser, with ssd) provided the disk was also encrypted. Nuke that key and it's reliably gone. -- Get your SQL database under version control now! Version control is standard for application code, but databases havent caught up. So what steps can you take to put your SQL databases under version control? Why should you start doing it? Read more to find out. http://pubads.g.doubleclick.net/gampad/clk?id=49501711iu=/4140/ostg.clktrk ___ bitcoin-list mailing list bitcoin-l...@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-list -- Get your SQL database under version control now! Version control is standard for application code, but databases havent caught up. So what steps can you take to put your SQL databases under version control? Why should you start doing it? Read more to find out. http://pubads.g.doubleclick.net/gampad/clk?id=49501711iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Introducing BitcoinKit.framework
As an FYI, I've sent Wendell and co some example code for how to use CPPJVM to use bitcoinj from native code. A rather rough Hello World app looks like this: https://github.com/mikehearn/cppjvm/blob/master/mytest/bcj-hello-world.cpp So, fairly C++ like. Further discussion of this should take place on the bitcoinj mailing list. -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] [RFC] Proposal: Base58 encoded HD Wallet master seed with optional encryption
This isn't usable for SPV wallets unless it has a birthday in it. Otherwise you either need to scan the entire chain (slow) or find a fully indexed copy of the block chain (expensive, more centralised). Just add a UNIX time as an extra 4 bytes, or if you want to save a few characters then use a uint16 that represents days since birth of this specification. -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Introducing BitcoinKit.framework
Actually bitcoinj typically doesn't download all the headers (just from the last checkpoint) and it throws away headers that are very old. By now there's quite a lot of difference in how they manage things and I guess it will diverge from bitcoind even more in future. For instance we're going to start only storing relevant outputs in the wallet and doing other things to try and save memory. Some people managed to get themselves wallets that don't actually fit in ram :( On 21 Jul 2013 17:55, Pieter Wuille pieter.wui...@gmail.com wrote: On Tue, Jul 16, 2013 at 12:59:56PM +0200, Mike Hearn wrote: I think that's a great approach. Here is the patch Satoshi sent me back in 2010. All the code has changed since but it can be a source of inspiration. From Satoshi: *The simplified payment verification in the paper imagined you would receive transactions directly, as with sending to IP address which nobody uses, or a node would index all transactions by public key and you could download them like downloading mail from a mail server. I'm currently working on headers-first sync, which I believe is generally very useful (it fixes tons of edge-cases block synchronization currently experiences), but it's also a first step towards SPV mode. So headers-first sync means you first synchronize just the headers, and then, when you already know (or have strong evidence for a guess on) the best chain, start requesting blocks along that best chain - potentially in parallel from different peers. SPV mode is basically headers-first sync, but never do the full block sync step, and replace it with a bloom/birthday/...-based fetching of blocks interesting to the associated wallets. In SPV you'll also need to disable the mempool though, and there will be more small changes, but I think the separate headers-sync phase will be most of the work. -- Pieter -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] SPV bitcoind? (was: Introducing BitcoinKit.framework)
The 90 minutes is not - the blockchain has grown quite a lot since last year, and as for the 3.5 speed, I havn't tested it since Pieter's ultraprune - libcoin also has something similar to ultraprune, done directly in the sqlite database backend, but I should run a head to head again - could be fun. I would assume, though, that the result would be similar timings. ultraprune made a huge difference. I think it's very likely that this claim is no longer true. Bitcoin got a lot more optimised since you first did libcoin. -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] SPV bitcoind? (was: Introducing BitcoinKit.framework)
SPV clients behaving normally are highly abusive: they use up maximum node resources with minimum cost to themselves. This must be a new use of the word abuse I haven't come across before :) At any rate, some of these assumptions are incorrect. Botnets of compromised web servers are quite common, and asymmetry in node resources is obviously biased against the kinds of devices people increasingly have (phones, tablets) where extremely limited memory bandwidth is common and apps routinely have just 16 or 32mb of memory to do everything including the GUI. A good anti-DoS strategy looks much the same as a good load shedding strategy. There's little reason to treat them separately. Perhaps instead of talking about DoS we should instead talk about what happens if Bitcoin suddenly gets too popular. Now there are suddenly lots of good users all wanting to use the network, and not enough nodes to support them all. What do we do? Some rules seem obvious - try to prioritise existing users over new users, old coins over new coins (dPriority already does this) etc. If you run out of TCP sockets prefer to disconnect recent connections (probably new users) to long lived connections (probably high powered backbone peers). If you run out of disk seeks prefer processing new blocks to serving old parts of the chain, etc. -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] SPV bitcoind? (was: Introducing BitcoinKit.framework)
Partial UTXO sets is a neat idea. Unfortunately my intuition is that many SPV wallets only remain open for 1 minute at a time because the user wants to see they received money, or to send it. It'd be neat to get some telemetry from the Android wallet for this - I will ask Andreas to let users opt in to usage statistics. So for anti-DoS I think smart prioritisation heuristics are the way to go again. Perhaps by letting clients have an identity that they provide to a node when it's load shedding. Clients that have been seen before, have a track record of not being abusive etc get priority and new clients that were never seen before get dropped. Coming up with a way to do that whilst preserving privacy sounds like an interesting cryptographic challenge. On Wed, Jul 17, 2013 at 12:58 PM, Peter Todd p...@petertodd.org wrote: On Tue, Jul 16, 2013 at 04:16:23PM +0200, Wendell wrote: Hello everyone, In the previous thread, I expressed interest in seeing an SPV bitcoind, further stating that I would fund such work. Mike Hearn followed up with some of Satoshi's old code for this, which is now quite broken. The offer and interest on my side still stand, as more diversity in SPV options seems like the right way to go. Time-permitting, I would really appreciate feedback from knowledgable parties about the possible approaches to an SPV bitcoind. We at Hive ideally want to see something that could one be merge into master, rather than a fork. Keep in mind that SPV mode is newer than many realize: bloom filters are a 0.8 feature, itself released only last Febuary. As John Dillon posted earlier this week in Protecting Bitcoin against network-wide DoS attack the Bitcoin codebase will have to implement much better anti-DoS attack defences soon, and in a decentralized system there aren't any options other than requiring peers to either do work (useful or not) or sacrifice something of value. SPV peers can't do useful work, leaving only sacrifice - to what extent and how much is unknown. In addition SPV nodes have serious privacy issues because their peers know that any transaction sent to them by the SPV node is guaranteed to be from the node rather than relayed; bloom filters are only really helpful with payment protocols that don't exist yet and don't apply to merchants. Then you have MITM problems, vulnerability to fake blocks etc. It'll be awhile before we know how serious these issues are in practice, and we're likely to find new issues we didn't think of too. In any case Bitcoin is far better off if we make it easy to run a full node, donating whatever resources you can. Fortunately there's a whole continuum between SPV and full nodes. The way you do this is by maintaining partial UTXO sets. The trick is that if you have verified every block in some range i to j, every time you see a txout created by a transaction, and not subsequently spent, you can be sure that at height j the txout existed. If height j is the current block, you can be sure the txout exists provided that the chain itself is valid. Any transaction that only spends txouts in this partial set is a transaction you can fully verify and safely relay; for other transactions you just don't know and have to wait until you see them in a block. So what's useful about that? Basically it means your node starts with the same security level, and usefulness to the network, as a SPV node. But over time you keep downloading blocks as they are created, and with whatever bandwidth you have left (out of some user-configurable allocation) you download additional blocks going further and further back in time. Gradually your UTXO set becomes more complete, and over time you can verify a higher and higher % of all valid transactions. Eventually your node becomes a full node, but in the meantime it was still useful for the user, and still contributed to the network by relaying blocks and an increasingly large subset of all transactions. (optionally you can store a subset of the chain history too for other nodes to bootstrap from) You've also got better security because you *are* validating blocks, starting off incompletely, and increasingly completely until your finally validating fully. Privacy is improved, for both you and others, by mixing your transactions with others and adding to the overall anonymity set. In the future we'll have miners commit a hash of the UTXO set, and that gives us even more options to, for instance, have relayed transactions include proof that their inputs were valid, allowing all nodes to relay them safely. As for specifics, you need to maintain a UTXO set, and in addition a set of spent txouts (the STXO set) for which you haven't seen the transaction that created the txout. As download newer blocks you update the UTXO set; as you download older blocks you update the UTXO set and STXO set. Nodes now advertise this new variable to their peers: nOldestBlock
Re: [Bitcoin-development] SPV bitcoind? (was: Introducing BitcoinKit.framework)
On Wed, Jul 17, 2013 at 2:37 PM, Tamas Blummer ta...@bitsofproof.comwrote: A majority coalition of miner (pool operator) might even decide to change block reward rules if the rest of the network only verifies POW. Which is why it's still vital that any important node in the economy uses full validation. A majority miner coalition could change the block reward and award themselves money which SPV clients would accept, however, the moment somebody tried to cash that money out via an exchange, or use it to purchase something from an online shop, or just see if it propagated across the P2P network effectively, they'd notice something had gone wrong. Of course it'd be in the news long before this happened SPV is really meant for nodes that go away and come back a lot, i.e. end user wallets. If you're a merchant it'd be dumb to run one unless you're on such a tight budget that your server resembles a powerful tablet. -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] SPV bitcoind?
Yeah, what I meant is, it'd be useful to know the average amount of time that the app was holding connections open for. On Wed, Jul 17, 2013 at 4:32 PM, Andreas Schildbach andr...@schildbach.dewrote: Android apps do whatever they are programmed to do. They become active when the user installs and inactive when they are uninstalled. Inbetween, they are not limited in runtime. That said, the current programming is that when receiving a block, it stays connected for at least ~2 more minutes. This generally allows the chain to catch up while at the same time avoiding endless battery drain because something gets stuck. Upon sending or receiving of a transaction, it stays connected for at least ~8 more minutes, because it is likely the wallet will see more activity. Additionally, on the send and request coins screens and the network monitor it stays connected for as long as the screen is on and the app in the foreground (= resumed state). On 07/17/2013 02:29 PM, Mike Hearn wrote: Partial UTXO sets is a neat idea. Unfortunately my intuition is that many SPV wallets only remain open for 1 minute at a time because the user wants to see they received money, or to send it. It'd be neat to get some telemetry from the Android wallet for this - I will ask Andreas to let users opt in to usage statistics. -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Introducing BitcoinKit.framework
Clear. Your right. C++ would give us more flexibility (other platforms) and also android compatibility (through NDK). I'm a bit confused I'm afraid. bitcoinj already runs SPV wallets on Android on top of Dalvik. In fact that's what it's designed for. The NDK is not necessary to work with Bitcoin at any point. That's a great idea. Let me look into the quality of j2c's output. There's an example of what it looks like here: https://code.google.com/a/eclipselabs.org/p/j2c/wiki/Examples If you're serious about playing with j2c let me know. It's an amazing piece of work BUT it was written for fun, and as such isn't really documented at all. It took me a little while to figure out how to make it work properly. I'm now fixing bugs in it and making various improvements along with filling out the native stubs (a.k.a. portability layer). If you want to catch up to where I'm at, I can send you some notes because otherwise you might waste a lot of time on blind alleys. The main things be aware of so far are: - Lots of explicit null pointer checks are generated. The reason is that the output is meant to be entirely portable, so Jacek doesn't want to rely on platform specific stuff like signals or SEH. Simplest solution is just to disable npc() generation entirely because normal C++ libraries just segfault if a null pointer gets in the wrong place, they don't throw exceptions. Losing the Java behaviour would not be a downgrade for people used to C++. - Array accesses don't seem to be properly bounds-checked. That's a part of the Java security model - bitcoinj is written on the assumption that buffer and heap overflows aren't possible because they're caught by the runtime. If those checks go missing then it'd likely become possible to hack your program by exploiting buffer overflows. So that needs to be fixed. - Generated code doesn't use the STL of course, it can't because the Java library has more features than the STL. However as the way j2c works is you transpile your code alongside a copy of the (open source) Java class library, you can go in and modify the generated code for java::lang::String or java::util::List and so on to add helper methods for converting to various other forms. On Linux you'd have implicit c'tors to go back and forth between std::string, on MacOS X you'd have conversions for NSString, you could add code for QStrings or raw C strings too. Once the code has been generated you can extend or patch it to make the API more convenient. - Obviously, the resulting code requires the Boehm GC because there are no explicit delete calls anywhere. This is a safety feature though, it avoids use-after-free and double-free bugs that can create security holes. - The code generator doesn't do dependency tracing, so you end up with generated code that isn't used anywhere. It's up to the linker to do a dead code elimination pass. Otherwise the resulting binaries can be huge. -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Introducing BitcoinKit.framework
Let's re-add the list as this is a topic of general interest. On Tue, Jul 16, 2013 at 11:36 AM, Jonas Schnelli jonas.schne...@include7.ch wrote: What do you think about extending the BitcoinKit.framework (based on bitcoind) so that the kit could handle the very basic SPV concept (getHeaders aka fast catchup, wallet-birthday, bloom filters)? Maybe it would be possible to only port some of the bitcoinj work to c++ (and use it for extending BitcoinKit or bitcoind)? Maybe then it could also be a starting point for someone who likes to create a SPV mode for bitcoind? Making bitcoind/Bitcoin-Qt support SPV mode was the original plan some years ago, Satoshi even sent me some code he wrote that did the first parts, but it was incomplete. At the time, I decided to do a separate implementation for a few different reasons. One is that my understanding of his code wasn't so good back then and I lacked confidence to change it. Especially as there were no unit tests back then (and still aren't any for most of it), making invasive changes to the core validation code was and is highly risky. A separate code base seemed to reduce the risk a lot. Another reason is that Satoshi encouraged me to write a simple re-implementation that people could learn from. And I wanted a documented, object oriented API that people could use to build a variety of apps. Yet another reason was bitcoind is security critical code that scrapes complex data structures from untrusted sources on the internet, and it's written in an unmanaged language. Ordinarily this would be a recipe for disaster as a single overflow or memory management error could lead to hacking and theft on a massive scale. It's like taking a chainsaw and using it to carve an ice sculpture. Satoshi, incredibly, pulled it off, mostly by using advanced C++ features that made his code hard to read for many people and by being very, very careful. I was not convinced I could do such a good job and was worried about accidentally introducing vulnerabilities. A final reason is that it was clear that the bitcoind codebase would need serious changes for mobiles, beyond that required for ordinary SPV support. For example, Satoshi's code assumes it has access to block headers via a std::map and that assumption is made in a lot of places. On Android phones, you can't fit all the block headers in RAM. bitcoinj uses a circular ring buffer of the last N thousand headers for this reason. It's quite different to how bitcoind works. All that said, it was a ton of work and it's still unclear that it was the right call. Anyway, your situation is a little different. Firstly you don't care about mobiles, your app is intended for desktops. So the changes required are less invasive. Also, there are more unit tests and more people with a good understanding of the code these days, so perhaps the risk of introducing bugs is lower. And these days we have some nice APIs for building apps so that need is already met. If you wanted to implement SPV mode in bitcoind, Gavin or I could send you Satoshi's old patch although of course it is no longer usable. It would indicate the basic cut lines though. -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] SPV bitcoind? (was: Introducing BitcoinKit.framework)
You'd want to create and get merged patches in the following order: 1) Be able to store just block headers in the blk.dat files instead of full block contents. At this point you are still *downloading* full blocks, but they are not being stored. The contents are still sent to the wallet for extracting relevant transactions though (see SyncWithWallets). You also need to disable listening and addr announcements to the P2P network at this point. You need to be able to re-org and do all the usual things without storing block contents. You also need to short-circuit the leveldbs so they aren't created or used. All that needs to be unit tested. You need to also rewrite the mempool logic so it throws out irrelevant transactions. The RPC interface needs to adjust itself so you can't try to start mining, query the utxo set, etc. At this point you have an SPV node, albeit one that still downloads the entire block chain. However total disk storage used will be much lower. Getting this written and reviewed is a big chunk of work but is the hardest part. Once it's done you can breath easy. 2) Next step, use getheaders to catch up with the chain until the min(wallet birthdays) is reached. You can see in Satoshi's patch where he adds support for receiving headers messages. Because key times are recorded as dates and you don't know the dates of blocks in advance, you need to download headers until you see one that goes past the key birthday minus some slack period, then throw out the headers you downloaded and switch to downloading full blocks again from that point onwards. 3) Next step, implement client side support for Bloom filtering. Switch from downloading full blocks to filteredblocks, verify the Merkle branches then apply them to the wallet. Watch out for accidental re-orderings of transactions here from block order (e.g. if you accidentally insert them into a std::map or other unordered collection it can lead to bugs). Come up with some way to decide on a FP rate. Probably you want a fairly high FP rate for desktop wallets. 4) Next step (optional), implement monitoring of broadcast propagation for transactions that are received. SPV clients cannot verify unconfirmed transactions so you can either just give up entirely and accept any old garbage, or assume a non-MITMd internet connection and use network propagation as a rough proxy for likely to be valid and mined upon. 4) Optimize! How much you need to optimize really depends on a lot of things. I found that to be competitive with Electrum/blockchain.info I had to do a ton of optimizations including very aggressive checkpointing so new users don't have to download more than a month or twos worth of headers, as downloading all the headers was becoming a bottleneck. You'd need to download about 16mb+ of data at the moment to grab all the headers and on a weakass mobile phone with a weak Dalvik VM and 3G internet this was way too much. I also had to spend some time profiling to ensure we weren't accidentally thrashing the UI due to too-fast updates, we weren't bottlenecking on updating last seen block data in the wallet, we weren't accidentally de/reserializing messages redundantly etc. After about 3-4 evenings of non-stop profiling and optimising I ended up with a relatively flat profile whilst doing initial catchup and chain sync. On a desktop I bet you can get away with much less optimisation because your CPUs, network and disk tend to be much stronger. On Tue, Jul 16, 2013 at 4:16 PM, Wendell w...@grabhive.com wrote: Hello everyone, In the previous thread, I expressed interest in seeing an SPV bitcoind, further stating that I would fund such work. Mike Hearn followed up with some of Satoshi's old code for this, which is now quite broken. The offer and interest on my side still stand, as more diversity in SPV options seems like the right way to go. Time-permitting, I would really appreciate feedback from knowledgable parties about the possible approaches to an SPV bitcoind. We at Hive ideally want to see something that could one be merge into master, rather than a fork. -wendell grabhive.com | twitter.com/grabhive -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks and diagnose root cause in seconds. Start your free trial of AppDynamics Pro today! http://pubads.g.doubleclick.net/gampad/clk?id=48808831iu=/4140/ostg.clktrk ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- See everything from the browser to the database with AppDynamics Get end-to-end visibility with application monitoring from AppDynamics Isolate bottlenecks
