Re: [Bitcoin-development] Building a node crawler to map network
Actually Steve, take a look at the bitcoinj mailing list today. Somebody has already built this and has it running. It's accumulating data at the moment, they'll announce it more widely soon. But I think there's no need to duplicate work. -- Special Offer -- Download ArcSight Logger for FREE! Finally, a world-class log management solution at an even better price-free! And you'll get a free Love Thy Logs t-shirt when you download Logger. Secure your free ArcSight Logger TODAY! http://p.sf.net/sfu/arcsisghtdev2dev___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Request review: drop misbehaving peers
If I think you're trying to DoS me, why would I be nice to you? The issue is, what if I'm not trying to DoS you, but something went wrong? think response messages would just give an attacker another potential attack vector, and it is clear from the debug.log what triggers a ban. Only clear to the node owner. Not the sender, who may in a better position to debug. It's pretty common for protocols to return useful errors even in DoS conditions. Eg, http servers will often return 503 Service Unavailable in overload conditions. Google actually sends a redirect telling you why you got blocked. -- Doing More with Less: The Next Generation Virtual Desktop What are the key obstacles that have prevented many mid-market businesses from deploying virtual desktops? How do next-generation virtual desktops provide companies an easier-to-deploy, easier-to-manage and more affordable virtual desktop model.http://www.accelacomm.com/jaw/sfnl/114/51426474/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Transaction Delivery and Storage
I imagine a lot of the things on the contracts page will be implemented by specialized software that interacts with the Bitcoin network directly. Transactions would then be moved around, for example, by having clients do HTTP POSTs of protocol buffers to servers that are listening and know how to interpret the received messages. -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity and more. Splunk takes this data and makes sense of it. Business sense. IT sense. Common sense. http://p.sf.net/sfu/splunk-d2dcopy1___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Determine input addresses of a transaction
Interesting suggestion! So if I understand correctly, greensig would be the signature generated from signing the transaction with the key of a green address? Sure. Or just a key. It wouldn't have to be an actual key used in the block chain. Sounds good - I guess I never thought in this direction, as I always assumed doing anything 'non-standard' with the scripting language would create a number of knock-on problems. It won't break the IsStandard checks, if that's what you mean. You can put any data you like into a scriptSig. In practice only data is useful, there's no purpose in having an actual script there (or at least, I wasn't able to find one yet). 1) Get something working reasonable fast to detect current green address style transactions. It's fine if it is a little bit of a hack, as long as it's safe, since I don't expect it to be merged with mainline anyway at this point. You could easily change the bitcoin code to detect such transactions - just look for scriptSigs that have 3 items instead of two, where the 3rd item is the right size to be a signature. Criticism ranging from 'unnecessary, as 0-confirmation transactions are fairly safe today' to 'encourages too much centralization and therefore evil' Heh, if that's a reference to my feedback, I definitely wouldn't describe such a feature as evil, that's rather strong :-) -- The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning@Cisco Self-Assessment and learn about Cisco certifications, training, and career opportunities. http://p.sf.net/sfu/cisco-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Detecting OP_EVAL scriptPubKeys that are to you
scriptPubKeys that use OP_EVAL contain a hash of a script. If I understand correctly, that means to detect a transaction in a block that is relevant to your wallet, that means you need to pre-calculate every possible hash that might appear. For the case of a single payment, that's not a problem. It means for each key you now have to check for: - raw key - key hash - hash of script that contains key hash - hash of script that contains raw key which isn't so bad. What is the complexity like when multi-signing comes into the picture? I *think* it's not an issue for the use cases currently envisioned, but being unable to see into a script could complicate things later. Specifically: for a wallet protection service, you have to make sure the WPS keys are matched 1:1 with your own private keys. You must never mix them up otherwise you have to check the block chain for the cross-product. Deterministic wallets are one way to achieve that without compromising privacy. For escrow contracts, using OP_EVAL means you cannot detect them unless the sender has told you the pubkey they are going to use, because otherwise you can't recreate the hashed script. Escrow protocols require some out of band communication anyway in order to set up the escrow key, so this isn't inherently a problem. Are there any use cases where you will want to recognize transactions to you, where you can't predict the full script contents? -- The demand for IT networking professionals continues to grow, and the demand for specialized networking skills is growing even more rapidly. Take a complimentary Learning@Cisco Self-Assessment and learn about Cisco certifications, training, and career opportunities. http://p.sf.net/sfu/cisco-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Lock protocol version numbers
BitCoinJ already sets the subver field to its name and version. -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] multisig, op_eval and lock_time/sequence...
BIPs are either standards track (affects everyone, represents consensus), informational (ie basically just summarizing the authors viewpoints on things) or process. My point is you can't have a credible standards track BIP until something has been implemented end to end. I don't think it's a good plan to design these things in isolation. You'll end up with bizarre user experiences because of technical decisions taken months earlier that are now hard to reverse. A working end to end implementation gives you the confidence to say, yes, this is how it should work, because here's the demo and you can see it works very well and the code is clean. If your BIP is informational then no problems, but I don't think there's much point in informational BIPs to be honest - it's easier to just write an email or forum post summarizing your views on things. If you find it a useful framework to write your thoughts in that's OK, but don't expect implementors to follow what's written there just because it's a BIP. It carries no more weight than any other document would. -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] multisig, op_eval and lock_time/sequence...
Sure, of course, as long as it's clearly labelled as just your thoughts, no issues. For dispute mediation the way I'd start is playing around with some UI design stuff and a toy protocol underneath. Once the process is smooth from the users POV (no seeing binary blobs disguised as text) then it should become clearer what steps the protocol needs and what order they need to come in. Specific feedback on this format - as far as I can tell the format represents a subset of the regular bitcoin transaction format? Couldn't you just serialize a Bitcoin CTransaction structure with the txins containing the output scripts? -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] [RFC] BIP 14 - Protocol Version and User Agent
Looks pretty reasonable to me. If Gavin changes the mainline client to use this format I'll change BitcoinJ as well. It'll need a bit of API work so clients are sure to set it up properly. On Thu, Nov 10, 2011 at 10:16 PM, Amir Taaki zgen...@yahoo.com wrote: Hi, https://en.bitcoin.it/wiki/BIP_0014 Thanks to Gavin Andresen for proof reading and suggesting clarifications. Thanks to Patrick Strateman for suggesting the hierarchical format and pointing out some flaws of browser user-agents to me. The timeline is written in the past tense since BIPs are meant to be readable in the future for explaining why we took certain decisions with bitcoin. A nice cache for future historians when bitcoin is ubiquitous ;) The next version 0.6 should be the protocol version which becomes peeled off from the current client. There are still some changes migrating into the protocol that need to be finished. -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1 ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- RSA(R) Conference 2012 Save $700 by Nov 18 Register now http://p.sf.net/sfu/rsa-sfdev2dev1___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] [ANNOUNCE] BitCoinJ 0.3
Perhaps a bit off-topic for this list, maybe there should be a software/services announcements list? Anyway ... I'm happy to announce version 0.3 of the leading Java implementation of the Bitcoin protocol. BitCoinJ is a widely used library that forms the foundation of projects as diverse as the Android Bitcoin Wallethttps://market.android.com/details?id=de.schildbach.wallethl=en, the p2p network status graphs http://bitcoinstatus.rowit.co.uk/, MultiBithttp://multibit.org/ , PoolServerJ http://poolserverj.org/ and more. You can get it either from our Maven repositoryhttp://code.google.com/p/bitcoinj/wiki/UsingMavenor the downloads section http://code.google.com/p/bitcoinj/downloads/list of the website. New in this release: - Many bugfixes, robustness and test suite improvements. - Major optimizations to reduce parsing overhead, most protocol messages are now parsed on demand. - A new PeerGroup API that handles the management of multiple peer connections. - Switched to using Maven for the build process, removed the bundled Bouncy Castle as a result. You can now depend on BitCoinJ using Maven if you don't need any special patches. - A bunch of new APIs to make writing Bitcoin apps easier. This release would not have been possible without the major contributions from: *Steve Coughlan*, who contributed many parsing improvements and optimizations *Miron Cuperman*, who did significant work on the PeerGroup API *Andreas Schildbach*, developer of the Android wallet, who as always reported many bugs and useful suggestions for improvement *Gary Rowe and Jonny Hegheim*, who set up the continuous build and Maven infrastructure What's next? The next release will focus on more of the same, that is, fixing bugs and filling out missing features so projects using the library don't feel any need to patch their local copy of the library. By popular request we'll be switching from Subversion to git. We'll also introduce a stable wallet format that isn't dependent on Java serialization, and timestamp key creation to resolve some issues with clients that ship block chain copies. And finally of course, whatever is contributed by the community. thanks /mike -- Google Switzerland GmbH -- All the data continuously generated in your IT infrastructure contains a definitive record of customers, application performance, security threats, fraudulent activity, and more. Splunk takes this data and makes sense of it. IT sense. And common sense. http://p.sf.net/sfu/splunk-novd2d___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Version bytes 2.0
Why does anyone care what an address looks like? If the user is seeing an address, that's a usability fail right there. It's common today because AFAIK nobody finished off the URL handling support in the main client for browser integration. It'd be a much better use of time to finish off that integration and make it easy for people to create links containing a bitcoin: URL (like with copy/paste of text/html content). -- Systems Optimization Self Assessment Improve efficiency and utilization of IT resources. Drive out cost and improve service delivery. Take 5 minutes to use this Systems Optimization Self Assessment. http://www.accelacomm.com/jaw/sdnl/114/51450054/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Fwd: [BIP 15] Aliases
I was in brmlab and wanted to pay 1 BTC for a Club Mate. They had on the wall a picture of their QR code and a bitcoin address. I don't own a mobile phone so the QR code is useless. Fixed addresses like that are a temporary thing during Bitcoins maturation period. They lead to merchants exposing data they probably don't realize they're exposing, like their income, which is basically unacceptable for any payment system. There's no point trying to optimize a case where: 1) You are in the minority (no phone?) 2) The perfect experience leaks private data in such a way that would be deemed a gross security breach by any serious payment processor. OK, some thoughts on the general proposal, from the POV of what it'd take for a large deployment, like for every Gmail or every Facebook user. In terms of ease of implementation it is ordered HTTPS/HTTP then DNS trailing by a large margin. Big sites, even small sites, typically have high-speed load balancing and demuxing already implemented for HTTP[S] and it's usually easy to add new endpoints. The same is *not* true of DNS, and whilst coding up a custom DNS server is possible it's definitely a worse fit. FirstBits seems out of the question for the same privacy reasons as given above. No banking system worth its salt would let everyone look up other peoples income. The simplest approach would be to request a full public key with an HTTPS request like foo@domain - https://domain/_bitcoin/getnewkey?user=foolabel=Payment%20from%20Bob If you then want to turn the resulting public key into an address before creating a transaction you can obviously do that. BTW the BIP is pretty hard to read. Your spec for the HTTPS proposal is a big pile of source code. I think it's the same as above, but it's hard to tell without more effort. -- Systems Optimization Self Assessment Improve efficiency and utilization of IT resources. Drive out cost and improve service delivery. Take 5 minutes to use this Systems Optimization Self Assessment. http://www.accelacomm.com/jaw/sdnl/114/51450054/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Version bytes 2.0
Base58 was chosen not for human readability but to make it easy to copy/paste. It was also chosen for hand-writeability, weirdly enough. That's why it excludes some confusible characters. But Satoshi didn't really understand how people would end up using Bitcoin, he originally imagined most transactions being done directly between pairs of IP addresses. Copy/paste of HTML content is currently not possible. You *can* already dragdrop the bitcoin: link to the client. Bluematt has a pull request to automatically handle bitcoin: URLs when clicked in the browser. That's cool. I hope Matts change gets merged soon. Then the issue becomes how do people find out about this capability? Expecting people to learn how to hand-craft Bitcoin links won't work. But all modern operating systems support copy/paste and drag/drop of rich content. Qt probably makes it easy to expose an UI like this: *Pay me*[Copy to clipboard] Clicking the link in the UI would pop up an alert saying something like You can drag this link to an email, chat window or editing program. Dragging it/pushing the copy button would just set the drag/clipboard data as a bit of text/html content. So then you can just copy/paste into an email or HTML editor. It wouldn't work for forums that use bbCode, though I guess there's no particular reason the forum software can't turn a href into [url=] automatically. -- Systems Optimization Self Assessment Improve efficiency and utilization of IT resources. Drive out cost and improve service delivery. Take 5 minutes to use this Systems Optimization Self Assessment. http://www.accelacomm.com/jaw/sdnl/114/51450054/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] [ANNOUNCE] BitCoinJ 0.4
I'm pleased to announce the release of BitCoinJ 0.4, the leading Java implementation of the Bitcoin protocol. BitCoinJ implements simplified payment verification, a lightweight mode in which no central server or authority is needed but the resource requirements are still low enough to be usable on smartphones. This version of the library is used in the new releases of Android Wallet and MultiBit. New in this release http://code.google.com/p/bitcoinj/wiki/ReleaseNotes - Ability to use getheaders to quickly catch up new users to the head of the chain. This is a big performance win. - ECKeys no longer require the private part, allowing for watching wallets that cannot spend, but still gather and track the transactions associated with the public keys. - A new API that implements transaction confidences. Get a quick summary or detailed information about how much confidence you can have that a given transaction won't be reversed. - A new DerbyBlockStore that stores block headers and related data in the Apache Derby relational database. - Protocol buffers are now a supported serialization format for the wallet. This means BitCoinJ based protobuf wallets can be read and manipulated by any language/platform with a protobufs implementation, which is most of them. There are extension points in the format to allow third parties to add new features. - Various new event listeners that help you learn when the state of the wallet or transactions change. - Support for post February 20th version handshakes (most library users already got this fix via backports) - All event listeners are now allowed to remove themselves during their own execution. - New APIs that allow you to create offline transactions and then broadcast them at a later point. Pending relevant transactions are recorded and announced to all newly connected nodes, ensuring a transaction won't get lost if there was flaky network connectivity at the time of creation. Pending transactions are supported much better in this release than in previous releases. - Wallet now can now take an invalid transaction and complete it by adding sufficient inputs and a change output. This enables the creation of multi-sends, as well as making experimentation with contracts easier. - Support for BIP 14: apps can now set their own user agent which will be put in the subVer field along with the library version. - Updated DNS seeds list. - A new WalletTool program for command line usage, and a ToyWallet app showing how to set everything up. - Support parsing and checking of alert messages. - New articles explaining how to use the library: - Working with transactionshttp://code.google.com/p/bitcoinj/wiki/WorkingWithTransactions - Working with the wallethttp://code.google.com/p/bitcoinj/wiki/WorkingWithTheWallet - The usual assortment of bugfixes, new APIs, robustness and test suite improvements. Thanks to everyone who contributed to this release, in particular Andreas Schildbach, Miron Cuperman, Roman Maneleil, Chris Rico and Vasile Rotaru. In the next release cycle, I'll be focusing on the following areas: 1. Real support for transaction fee calculations (most users apply a custom patch for this today) 2. A better block chain API 3. Have the library manage save points for the wallet itself 4. Further chain download time optimizations 5. More support for moving apps onto work done as a confidence measurement Of course contributors are welcome to work on whatever they want. thanks -- Virtualization Cloud Management Using Capacity Planning Cloud computing makes use of virtualization - but cloud computing also focuses on allowing computing to be delivered as a service. http://www.accelacomm.com/jaw/sfnl/114/51521223/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Adding a pong message
TCP keep-alives aren't reliably implemented. I've got reports that sometimes we struggle to keep connection to the network on mobile, eg, because we roam into an area with poor connectivity but not poor enough for the network stack to drop access entirely. Being able to quickly check if the connection is really there with some kind of bounded, app layer deadline is probably useful and besides, it's cheap. -- Keep Your Developer Skills Current with LearnDevNow! The most comprehensive online learning library for Microsoft developers is just $99.99! Visual Studio, SharePoint, SQL - plus HTML5, CSS3, MVC3, Metro Style Apps, more. Free future releases when you subscribe now! http://p.sf.net/sfu/learndevnow-d2d ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] BIP 31
Jeff asked for a BIP for the pong message, so here it is: https://en.bitcoin.it/wiki/BIP_0031 -- Better than sec? Nothing is better than sec when it comes to monitoring Big Data applications. Try Boundary one-second resolution app monitoring today. Free. http://p.sf.net/sfu/Boundary-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoin TX fill-or-kill deterministic behavior
It sounds OK as long as you exclude nLockTimed transactions.
That said, if you broadcast a transaction that does not meet the fee
rules, you should be able to notice that it wasn't accepted by your
peers immediately. Today it's painful because the protocol isn't very
chatty - in bitcoinj I plan to do this by announcing to half the
connected peers and waiting to see if the transaction comes back on
the other half. Getting a response from a peer that the TX was dropped
for reasons {x,y,z} is a better design but needs another protocol
change.
So having transactions expire would address the case where somebody
broadcasts a transaction that successfully propagates across the
network, but then isn't actually accepted by miners for some reason.
For instance due to a change in the default fee schedules. That risk
can be mitigated somewhat by being careful about such changes (timed
phase ins set multiple months out so people have time to upgrade,
alerts announcing it, etc).
I'm not sure we should be encouraging users to attach fees to
transactions though. Even if you can replace a transaction after a
couple of days, the user experience of trying to get the fee right
is atrocious. I don't think any sensible merchant will actually be
willing to put their customers through this nonsense. If somebody
broadcasts a transaction that successfully propagates across a big
chunk of the network but then gets stuck due to lacking sufficient
fees, the best fix is for the merchant to broadcast another
transaction that spends the first and increases the fees on it that
way. After this happens a few times, if I was a merchant I'd be
tempted to just ask buyers to submit the TX to me directly and I'll
handle keeping up with what miners currently charge and attaching
fees. I don't want my customers to have to think about this and have
trades spuriously fail when they forget.
That design requires a minor change to how fees are calculated inside
the memory pool, to include fees on un-included dependencies. But that
seems fairly uncontroversial to me. It's best for users, merchants and
miners to not leave chains of transactions in limbo when together
their fees add up to the minimum required amount.
--
For Developers, A Lot Can Happen In A Second.
Boundary is the first to Know...and Tell You.
Monitor Your Applications in Ultra-Fine Resolution. Try it FREE!
http://p.sf.net/sfu/Boundary-d2dvs2
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] new bitcoin.org clients page
We're debating the descriptions on the thread. I provided rewritten descriptions that try and keep with the theme per client goal, whilst being less technical. I think it's unclear how best to run this page. It's clear we need one though. If everyone can just submit whatever they like then we'll end up with 4 or 5 pick me! pick me! type descriptions, which avoids a lot of arguing but doesn't really help our users make a decision. If we have a Benign Dictator model we might end up with descriptions that are wrong or don't highlight the strengths / weaknesses of each client properly. So although it's messy I think the right path is probably the middle one - have some descriptions that try to be neutral, then improve them based on feedback from users and developers. They need to be flexible and evolve over time as the clients evolve too. At some point every client will support deterministic wallets so easy backups won't be worth mentioning any more, but there'll be new distinguishing features. And we all need to try and be honest about our own work. Here is the current content. Like I said, the descriptions are *not* set in stone at all. Bitcoin-Qt http://bitcoin.org/ The original software written by Satoshi Nakamoto, the project's founder. If you aren't sure which program to pick, this is a good bet. This application is a peer-to-peer client that builds the backbone of the Bitcoin network. It is suited for enthusiasts, merchants, miners, developers and people who want to help support the project. People who run Bitcoin-Qt are first class network citizens and have the highest levels of security, privacy and stability. However, it can be very resource intensive and you should be willing to leave it running in the background so other computers can connect to yours. If your computer is low powered or you aren't willing to tolerate a 24-hour+ initial start time, you should consider other clients. Cutting edge features tend to be implemented in other clients first. Website: bitcoin.org Platforms: MultiBit http://multibit.org/ MultiBit's primary focus is being fast and easy to use, even for people with no technical knowledge. It has a YouTube channel to help you learn the software, and includes helpful features such as an exchange rate ticker. MultiBit supports many languages such as German, Spanish and Greek. MultiBit synchronizes with the network much faster than Bitcoin-Qt and should be ready for you to use within a few minutes. This is a good choice for non technical users who want an easy to use experience, especially if you use a Mac. Website: multibit.org Platforms: Armory http://bitcoinarmory.com/ Armory focuses on advanced wallet management features, such as the ability to construct transactions whilst disconnected from the internet. It operates in conjunction with a Bitcoin-Qt install. It requires a large amount of RAM to operate and if you use Windows, it requires a 64 bit version. It is a good choice for tech-savvy enthusiasts or merchants who want to try out cutting edge ideas in the Bitcoin world. Armory was partly funded by a community donation drive which raised over $4000. Website: bitcoinarmory.com Platforms: Electrum http://ecdsa.org/electrum Electrum's focus is speed, with low resource usage and making wallet backups easy. It operates in conjunction with remote servers that handle the most complicated parts of the Bitcoin system, which is why it's fast. However, by running this client you don't contribute your computer's resources to the core network, and the remote servers that help give it good performance have the ability to see all your transactions and tie them together. Whilst you need provide no personal information to use Electrum (as is true for all Bitcoin clients), this means the privacy level is lower than for other clients. Merchants are recommended to use other p2p clients. Electrum is not quite user friendly yet - currently it is more suited for tech-saavy individuals. Website: ecdsa.org/electrum Platforms: -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] new bitcoin.org clients page
What computer is the initial start time 24-hours+ now? On normal systems initial sync-up now takes a couple hours. OK, I haven't tried a full block chain sync for a while. If it's only a couple of hours that's great. Let's change that. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] new bitcoin.org clients page
Bitcoin-qt is translated into a pretty broad set of languages (now— I cant tell you how many of them are _good_). Listing language just under multibit makes it sound like a distinguishing characteristic. Fair enough then, let's take that out. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP 33 - Stratized Nodes
Thanks for getting this started. With regards to the specific proposal, I don't believe it's the best option and still plan to eventually implement the original design outlined more than a year ago in this thread: https://bitcointalk.org/index.php?topic=7972.msg116285#msg116285 Namely that you use a new protocol command to set a Bloom filter on a connection. Only transactions matching that filter will appear in relayed inventory. Blocks that are requested will arrive as a header plus transaction/merkle branch pairs. Clients are expected to maintain and track the block chain as per usual, but instead of downloading the whole chain and then dropping the irrelevant transactions, that filtering is done server side. By strengthening or weakening the Bloom filters you can choose your preferred point on the privacy/bandwidth-usage spectrum. It is a fairly simple change to the Satoshi and BitcoinJ codebases but still allows clients to gain confidence in their balance by examining the chain, and this is true even in the presence of a hijacked internet connection (you can't trust pending transactions that way, but you can still trust confirmed transactions). The filters would be applied to each data block in each script rather than having a specific knowledge of addresses. In this way you can select for things like multisig outputs or outputs which don't use addresses / pubkeys to authenticate. I could write a BIP for this alternative protocol if somebody else wants to implement it. I was going to wait until I had time to do both BIP and implementation, but I think some simple optimizations to BitcoinJ can keep its performance good enough for the short term. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] [ANNOUNCE] BitCoinJ 0.5 released
I'm pleased to announce the release of BitCoinJ 0.5, the library that powers Android Wallet, SatoshiDice, Bitcoin Status, the server side part of BCCAPI and much more. This release focusses on bug fixes, making the build more standard and completing the transition to the protobuf wallet format. It also includes the first preview of the native API, allowing you to access bitcoinj from C++/Objective-C++ using a straightforward, intuitive mapping from the Java API. Much easier than JNI and no JVM is required, just the libgcj support library. Examples of a native Cocoa app for OS X and a command line hello world app are included. Because it's not fully finished/documented yet, this work is available on a branch rather than in the main release. We now have a Google+ page where we'll post announcements and developer tips/ideas: https://plus.google.com/102614914114364947458 New in this release: - Address.getParameters() and Address.getParametersFromAddress() let you figure out for what network the address is for (test, production, etc). BitcoinURI no longer requires a NetworkParameters for the same reason. - Updated to latest bouncy castle version, remove the need for the Android artifact by using the SpongyCastle build - Receives pending transactions much faster than before - Update to the testnet2 rules - Wallets now store the current chain head - wallet-tool can now create and broadcast transactions from the command line - Wallets will now be auto-migrated to protobuf format if they were previously serialized Java objects - Now uses the standard Maven directory layout - Many important bugfixes I'd like to thank Jim Burton, Miron Cuperman, Andreas Schildbach and Gary Young for their contributions to this release. You can get it from the download page on www.bitcoinj.org -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bootstrapping full nodes post-pruning
Actual BDB files are absolutely not deterministic. Nor is the raw blockchain itself currently, because blocks aren't always added in the same order (plus they get orphans in them) That's true. Though if you prune up to the last checkpoint, orphans before that point can be safely thrown away. I wonder if swapping out bdb for LevelDB might make sense at some point. I'm not sure how deterministic that is either though :) -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Near-term scalability
I had to hit the sack last night as it was 2am CET, but I'd like to sum up the discussion we had on IRC about scalability and SatoshiDice in particular. I think we all agreed on the following: - Having senders/buyers pay no fees is psychologically desirable even though we all understand that eventually, somebody, somewhere will be paying fees to use Bitcoin - In the ideal world Bitcoin would scale perfectly and there would be no need for there to be some winners and some losers when it comes to confirmation time. There was discussion of some one-off changes to address the current situation, namely de-ranking transactions that re-use addresses. Gavin and myself were not keen on this idea, primarily because it just avoids the real problem and Bitcoin already has a good way to prioritize transactions via the fees mechanism itself. The real issue is that SatoshiDice does indeed pay fees and generates a lot of transactions, pushing more traditional traffic out due to artificial throttles. The following set of proposals were discussed: (1) Change the mining code to group transactions together with their mempool dependencies and then calculate all fees as a group. A tx with a fee of 1 BTC that depends on 5 txns with zero fees would result in all 6 transactions being considered to have a fee of 1BTC and therefore become prioritized for inclusion. This allows a transition to receiver pays model for fees. There are many advantages. One is that it actually makes sense ... it's always the receiver who wants confirmations because it's the receiver that fears double spends. Senders never do. What's more, whilst Bitcoin is designed to operate on a zero-trust model in the real world trust often exists and it can be used to optimize by passing groups of transactions around with their dependencies, until that group passes a trust boundary and gets broadcast with a send-to-self tx to add fees. Another advantage is it simplifies usage for end users who primarily buy rather than sell, because it avoids the need to guess at fees, one of the most problematic parts of Bitcoins design now. The disadvantages are that it can result in extra transactions that exist only for adding fees, and it requires a more modern payment protocol than the direct-IP protocol Satoshi designed. It would help address the current situation by avoiding angry users who want to buy things, but don't know what fee to set and so their transactions get stuck. (2) SatoshiDice should use the same fee algorithms as Bitcoin-Qt to avoid paying excessive fees and queue-jumping. Guess that's on my plate. (3) Scalability improvements seem like a no brainer to everyone, it's just a case of how complicated they are. (4) Making the block size limit float is better than picking a new arbitrary threshold. On the forums Matt stated that block chain pruning was a no-go because it makes bitcoin more centralized. I think we've thrashed this one out sufficiently well by now that there should be a united opinion on it. There are technical ways to implement it such that there is no change of trust requirements. All the other issues (finding archival nodes, etc) can be again addressed with sufficient programming. For the case of huge blocks slowing down end user syncing and wasting their resources, SPV clients like MultiBit and Android Wallet already exist and will get better with time. If Jeff implements the bloom filtering p2p commands I'll make bitcoinj use them and that'll knock out excessive bandwidth usage and parse overheads from end users who are on these clients. At some point Bitcoin-Qt can have a dual mode, but who knows when that'll get implemented. Does that all sound reasonable? -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] New P2P commands for diagnostics, SPV clients
Need to specify the format of how these arrive. It means that when a new block is found instead of inv-getdata-block we'd see something like inv-getdata-merkleblock where a merkleblock structure is a header + list of transactions + list of merkle branches linking them to the root. Thinking about it some more and re-reading the Scalability wiki page, I remembered that a nice bandwidth optimization to the protocol is to distribute blocks as header+list of tx hashes. If a node has already seen that tx before (eg, it's in the mempool) there is no need to send it again. With the new command to download the contents of the mempool on startup, this means that blocks could potentially propagate across the network faster as download time is taken out of the equation, and indeed, with the signature cache the hard work of verifying is already done. So this could also help reduce orphan blocks and spurious chain splits. Are you planning on implementing any of this Jeff? I think we have the opportunity to kill a few birds with one or two stones. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] New P2P commands for diagnostics, SPV clients
Yes, the format is something that must be hashed out (no pun intended). Need input from potential users about what information they might need. Matts point that a branch-per-transaction may duplicate data is well made, that said, I suspect a format that tries to fix this would be much more complicated. How about see this project as a three part change? First step - add the mempool command and make nodes sync up their mempools on startup. Second step - if protocol version = X, the block message consists of a header + num transactions + vectorhash instead of the full transactions themselves. On receiving such a block, we go look to see which transactions we're missing from the mempool and request them with getdata. Each time we receive a tx message we check to see if it was one we were missing from a block. Once all transactions in the block message are in memory, we go ahead and assemble the block, then verify as per normal. This should speed up block propagation. Miners have an incentive to upgrade because it should reduce wasted work. Third step - new message, getmerkletx takes a vectorhash and returns a merkletx message: merkle branch missing the root + transaction data itself for each requested transaction. The filtering commands are added, so the block message now only lists transaction hashes that match the filter which can then be requested with getmerkletx. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] New P2P commands for diagnostics, SPV clients
The bottleneck for the android Bitcoin Wallet app is rapidly becoming bandwidth and parse time. On Fri, Jun 15, 2012 at 8:42 PM, Amir Taaki zgen...@yahoo.com wrote: Why though? The bottleneck is not network traffic but disk space usage/blockchain validation time. - Original Message - From: Mike Hearn m...@plan99.net To: Jeff Garzik jgar...@exmulti.com Cc: Bitcoin Development bitcoin-development@lists.sourceforge.net Sent: Friday, June 15, 2012 3:43 PM Subject: Re: [Bitcoin-development] New P2P commands for diagnostics, SPV clients Yes, the format is something that must be hashed out (no pun intended). Need input from potential users about what information they might need. Matts point that a branch-per-transaction may duplicate data is well made, that said, I suspect a format that tries to fix this would be much more complicated. How about see this project as a three part change? First step - add the mempool command and make nodes sync up their mempools on startup. Second step - if protocol version = X, the block message consists of a header + num transactions + vectorhash instead of the full transactions themselves. On receiving such a block, we go look to see which transactions we're missing from the mempool and request them with getdata. Each time we receive a tx message we check to see if it was one we were missing from a block. Once all transactions in the block message are in memory, we go ahead and assemble the block, then verify as per normal. This should speed up block propagation. Miners have an incentive to upgrade because it should reduce wasted work. Third step - new message, getmerkletx takes a vectorhash and returns a merkletx message: merkle branch missing the root + transaction data itself for each requested transaction. The filtering commands are added, so the block message now only lists transaction hashes that match the filter which can then be requested with getmerkletx. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] After compressed pubkeys: hybrid pubkeys
* 0x04 [32-byte X coord] [32-byte Y coord]: uncompressed format * 0x06 [32-byte X coord] [32-byte Y coord]: hybrid format for even Y coords * 0x07 [32-byte X coord] [32-byte Y coord]: hybrid format for odd Y coords So what's the actual difference in format? Is there any at all, or it's just the first number that's different? -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] LevelDB benchmarking
I switched the transaction database to use the Google LevelDB library, which is a refactored out part of BigTable. Here are my results. All tests are done on this hard disk: http://wdc.custhelp.com/app/answers/detail/a_id/1409/~/specifications-for-the-500-gb-caviar-blue-and-caviar-se16-serial-ata-drive which has an average 8.9msec seek time. It is a 6 core Ubuntu machine. I used -loadblock on a chain with with 185127 blocks in it, so it has lots of SatoshiDice traffic. 8.9 ms (average) seek time Regular BDB as we have today: real96m6.836s user49m55.220s sys 2m29.850s Throughput usually 4-5MB/sec according to iotop, pauses of 8-10 seconds for “Flushing wallet ...”. 611mb of blkindex.dat BDB without sig checking Throughput, 12-17mb/sec real42m51.508s user11m52.700s sys 2m36.590s Disabling EC verification halves running time. LevelDB no customized options (I ran the wrong time command here, hence the different format) 3184.73user 181.02system 51:20.81elapsed 109%CPU (0avgtext+0avgdata 1220096maxresident)k 1104inputs+125851776outputs (293569major+37436202minor)pagefaults 0swaps So, 50 minutes. Throughput often in range of 20-30mb/sec. 397MB of data files. LevelDB w/ 10 bit per key bloom filter real50m52.740s user53m38.870s sys 3m4.990s 424mb of data files No change. LevelDB w/ 10 bit per key bloom filter + 30mb cache (up from 8mb by default) real50m53.054s user53m26.910s sys 3m10.720s No change. The reason is, signature checking is the bottleneck not IO. LevelDB w/10 bit per key bloom filter, 30mb cache, no sigs real12m58.998s user11m42.330s sys 2m5.670s 12 minutes vs 42 minutes for BDB on the same benchmark. Conclusion: LevelDB is a clear win, taking a sync in the absence of network delays from 95 minutes to 50, at which point signature checking becomes the bottleneck. It is nearly 4x as fast when signature checks are not done (ie, when receiving a block containing only mempool transactions you already verified). -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] LevelDB benchmarking
+list On Mon, Jun 18, 2012 at 9:07 PM, Gregory Maxwell gmaxw...@gmail.com wrote: In addition to the ECDSA caching, ECDSA can can easily be run on multiple cores for basically a linear speedup.. so even with the checking in place once ECDSA was using multiple threads we'd be back to the DB being the bottleneck for this kind of case. Maybe ... looking again I think I may be wrong about being IO bound in the last benchmark. The core running the main Bitcoin thread is still pegged and the LevelDB background thread is only spending around 20% of its time in iowait. An oprofile shows most of the time being spent inside a std::map. OK, to make progress on this work I need a few decisions (Gavin?) 1) Shall we do it? 2) LevelDB is obscure, new and has a very minimalist build system. It supports make but not make install, for example, and is unlikely to be packaged. It's also not very large. I suggest we just check the source into the main Bitcoin tree and link it statically rather than complicate the build. 3) As the DB format would change and a slow migration period necessary, any other tweaks to db format we could make at the same time? Right now the key/values are the same as before, though using satoshi serialization for everything is a bit odd. We'd need UI for migration as well. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Enforcing inflation rules for SPV clients
I've been having a discussion with d'aniel from the forums about how to handle the possibility of a majority-miner conspiracy to raise inflation, if most economic actors use SPV clients. Because of how blocks are formatted you cannot check the coinbase of a transaction without knowing the fees in the block, and the fees can only be calculated if you have all the input transactions for every transaction in that block. Because the attack scenario is an attempted takeover of the economy by miners, attempting to put hints into the blocks won't work - we have to assume the hardest chain is in fact wrong according to the rules signed up to by the Bitcoin user. The most obvious goal for a cartel of miners is to change the inflation formula, either for purely selfish reasons (they want more money than can be obtained by fees) or due to coercion by governments/central banks who still subscribe to the inflation is good idea. Whilst good nodes (still on the old ruleset) won't relay blocks that violate the rules no matter how hard they are, in a situation where an SPV client DOES hear about the bad best chain, it would switch to it automatically. And who knows how the network might look in future - perhaps most nodes would end up run by miners, or other entities that upgrade to the new ruleset for other reasons. d'aniel made a good proposal - having good nodes broadcast announcements when they detect a rule that breaks the rules, along with a proof that it did so. Checking the proof might be very expensive, but it would only have to be done for split points, limiting the potential for DoS. If a node announces that it has a weaker chain and that the split point is a rule-breaker, the SPV client would download the headers for the side chain to verify the split, then download all the transactions in the split block along with all their inputs, and the merkle branches linking the inputs to the associated block headers. In this way the fee can be calculated, the inflation formula applied and the coinbase value checked. If the block is indeed found to be a rule-breaker, it'd be blacklisted and chains from that point forward ignored. Miners may decide to allow themselves to create money with non-index-zero transactions to work around this. In that case the good node can announce that a given tx in the rule-breaker block is invalid. The SPV node would then challenge nodes announcing the longer chain to provide the inputs for the bad tx all the way back to a pre-split coinbase. Doing these checks would be rather time consuming with huge blocks, but it's a last resort only. In the absence of bugs, the mere presence of the mechanism should ensure it never has to be used. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Enforcing inflation rules for SPV clients
Link? It was a private conversation for some reason. I also proposed this on this list (see the response in the tree datastructures thread) along with more elaboration on IRC. Ah OK. I wasn't paying much attention to those threads. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] LevelDB benchmarking
I've added some more commits: https://github.com/mikehearn/bitcoin/commits/leveldb It's still not ready for a pull req but is a lot closer: 1) Auto-migration is there but not well tested enough (I only tested with empty wallets). 2) Migration progress UI is there so you have something to watch for the few minutes it takes. Script execution is disabled during migration 3) LevelDB source is checked in to the main tree, bitcoin-qt.pro updated to use it 4) LevelDB is conditionally compiled so if there's some unexpected issue or regression on some platform it can be switched back to BDB Still to go: 1) More testing, eg, with actual wallets :-) 2) Update the non-Qt makefiles 3) On Windows it's currently de-activated due to some missing files from leveldb + I didn't test it If you want to help out, some testing and makefile work would be useful. I may not get a chance to work on this again until next week. On Wed, Jun 20, 2012 at 2:41 PM, Mike Hearn m...@plan99.net wrote: Two days ago on #bitcoin-dev: 21:01:19 sipa what was CTxDB::ReadOwnerTxes ever used for? 21:01:31 sipa maybe it predates the wallet logic (read: it's not used anywhere in the code, and apparently wasn't ever, even in 0.1.5) Great, in that case Stefan is right and I'll delete that code when I next work on the patch. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Random order for clients page
I strongly agree, but this is *why* I suggested moving it to the wiki. I recently had to choose an XMPP client and I looked on xmpp.org - after a frustrating experience with their listing [1] Probably because their listing is even more useless than any of the proposals that were presented here. Thank goodness it didn't end up like that. Their table doesn't even attempt to list features or differentiating aspects of each client. I think the XMPP guys have pretty much given up on directly marketing the system to end users. - more up-to-date (anyone can update them) Fortunately reasonable clients don't appear/disappear/change that often. - more in touch with users: I think by users you mean, geeks who understand wiki syntax. Because that's what it'll end up trending towards. I don't believe a wiki would reflect the needs of your average person. It's still better to have these arguments here and try to find a user-focussed consensus than hope one will converge from a wiki. If you want to see the result of internal politics, the current client page is a good example. We couldn't agree on the columns for a feature matrix, so now we just have walls of text. Inability to agree on columns isn't why the page looks like that. I know because I'm the one who argued for the current design. It looks like that because feature matrices aren't especially helpful for newbies to make a decision, especially when the features in question were often things like how they handled the block chain or which protocol standards they support, ie, things only of interest to developers. It's much easier to communicate the differences to people with a short piece of text, and maybe if there is no obvious way to explain why you'd want to use a given client, that's a good sign it's not worth listing there. Otherwise you end up like xmpp.org. Some of the options that are de-facto the most popular with users like BlockChain.info or just using your MtGox account are not mentioned at all. It's true that bitcoin.org needs to be conservative. That said, I'd like there to be sections for them too, actually. I agree that risk isn't purely about how it's implemented and that whilst we might like to push particular ideologies around protocols or code licensing, that isn't especially relevant to end users who have different priorities. Track record counts for a lot as well. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Accepting broken QRcodes
Hi bitcoin-development, blockchain.info generates non-BIP-compliant URIs in its QRcodes, as does its iPhone app. They are of the form bitcoin://address not bitcoin:address. I asked Ben to fix this (social networks don't parse QRcodes after all), but after explaining that social networks don't parse URLs without :// in them, he stopped responding to my emails. So I've gone ahead and added support for reading these types of URLs to bitcoinj, in the interests of just works interoperability. This mail is just a heads up in case anyone else wants to do the same thing. Hopefully at some point, Ben will stop generating such QRcodes and we can remove these hacks and get back to BIP compliance. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Reconsidering block version number use
That'd be 7 bytes of nonce in the block header, which is 72,057,594,037,927,936 ~ 72 petahashes = 72,000 terahashes So: the changes for version 2 blocks would be has height in the coinbase, and has a 1-byte version number with a 3-byte extranonce. I don't understand why more nonce bits are necessary. Is it really impossible for a multi-core CPU to keep up with the merkle root re-calculation and keep an ASIC miner fed, or is this working around a performance bottleneck somewhere else? -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Reconsidering block version number use
My point is that stuffing nonces into whatever spaces we can find to eke out a bit more scalability in pools seems like a very short term fix with potentially very long term consequences. Although it may sound harsh, if your pool is struggling to keep up with calculating merkle roots (which is cheap!) then it's time to either upgrade your pool or for some of those users to migrate to p2pool and handle creation of work themselves. Trying to squash more nonce bits out of fields that were never meant for that seems like a bad precedent with no real motivation beyond making running centralized pools a bit cheaper. What I'm interested in is, can a powerful server-class machine really not keep up with work generation for things like the BitForce SC devices? How many devices would you need to exhaust the ability to generate work for them? You'll need powerful machines just to run a node at all sooner or later. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Scalability issues
The Satoshi client uses a pure reentrant mutexes model As you presumably already know, the reference client doesn't attempt to parallelise most operations at all. Chain download is entirely single threaded. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Bitcoin script opcode counts
OP_CODESEPARATOR 14 OP_DEPTH 182 I'm interested to see what scripts were using OP_DEPTH and OP_CODESEPARATOR, as the latter appears to be useless to my eyes. Could you give some tx ids which use unusual opcodes? -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Signing release binaries
MacOS X 10.8 makes application signing borderline mandatory, in that you cannot run unsigned apps unless you tweak your settings via the control panel. You must sign with a certificate issued by Apple via their identified developer program. Windows allows but does not require signing. However, anti-virus systems tend to use signers with good reputation as a whitelisting signal. Signing Bitcoin releases makes sense because it may lead to, at minimum, higher performance if AV engines ignore file reads/writes by Bitcoin. And it can also shield us from false positives. You only need to see the mess that the mining tools world has become to understand why this is important. As I don't take part in the release process, I can't help out with this directly, but I believe it's important and would be willing to throw some money in towards buying the signing certs for both platforms. I guess Gavin would be the final signer. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP: Custom Services
I think it's pretty reasonable, although people will want to use node flags to get into the addr broadcasts anyway. That said, I suspect (based on previous discussions) that there would be quite some pushback against putting extra functionality into the core Bitcoin network. Most likely people will re-use the code with different peer discovery seeds and bootstrap similar but unrelated P2P networks for doing new applications. For instance, what if we want to do the language translation app I've talked about a few times before? You need a way to floodfill broadcast invalid transactions to interested parties. The pubsub mechanism in the Bitcoin protocol was an interesting way to do that, but I think it got removed. To broadcast to interested nodes now, you'd have to find them via addr broadcasts and then connect directly. And if you're going to do that, you may as well just form an entirely independent network. More elaboration of the use cases might therefore be useful. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] BIP 35: add mempool message
I think MSG_TX is fine. Simply sending an inv to the other node at startup would work, but it's better to request it explicitly as it will let the connecting peer configure a bloom filter before requesting mempool contents. It's already too heavy for mobile clients to download the entire mempool contents at startup so I probably wouldn't implement/activate support for this on the bitcoinj side until bloom filtering is done, and then this BIP would have to be updated to reflect that the response from mempool is filtered. On Thu, Aug 16, 2012 at 11:05 PM, Jeff Garzik jgar...@exmulti.com wrote: On MSG_MEMTX: The current version has a much higher Just Works value. On empty inv: It is generally better to do something unconditionally, than have a response generated only under certain conditions. And Alan is correct to note that unknown messages are ignored (intentionally, for expansion). However, unconditionally returning a response has little to do with feature probing/discovery. It is simply a clear, deterministic indication that processing is complete, for each invocation. -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/ ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Atomic coin swapping?
Why Signing the input scripts as well would obviously make it impossible to construct a transaction? As it states in the source code, signatures cannot sign themselves. If scriptSigs were included in the data that is being signed, the act of inserting the newly calculated signature for one input would break the signatures for all the others. -- How fast is your code? 3 out of 4 devs don\\\'t know how their code performs in production. Find out how slow your code is with AppDynamics Lite. http://ad.doubleclick.net/clk;262219672;13503038;z? http://info.appdynamics.com/FreeJavaPerformanceDownload.html___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Large backlog of transactions building up?
Sounds good— my only concern is that nodes will repeat their own transactions but not the unconfirmed parents. Nodes repeat wallet transactions and any previous transactions that are not yet included in the chain (see CWalletTx::RelayWalletTransaction). So I don't think it's an issue. (ok, bitcoinj clients don't do that, they just announce their own transactions) -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://ad.doubleclick.net/clk;258768047;13503038;j? http://info.appdynamics.com/FreeJavaPerformanceDownload.html ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] bitcoinj 0.6 now available
I'm pleased to announce the release of version 0.6 of bitcoinj, the leading Java implementation of Bitcoin. You can download the source from Google Code, or use the release-0.6 branch from git. Our Nexus repository will be updated soon. This release focuses on improved compliance with the protocol, improved and more scalable network handling, a more flexible send API, other misc API improvements and of course, a large pile of bug fixes. You should upgrade your software to use bitcoinj 0.6 as soon as possible, if only to benefit from the fixes - the API changes are minimal so it should be easy. In more detail: - Thanks to Jim Burton, the wallet now stores the depth and work done for all transactions, and coinbase transactions are now processed correctly. The ability to handle pubkey-only outputs was added, so these are now spendable. Migration from 0.5 wallets that don't store this is supported, but only for depth, by using WalletProtobufSerializer.setChainHeight(). - Made some more APIs documented and public. - Improved block chain download handling. - Added compatibility with the broken URIs generated by blockchain.info, meaning that the iPhone app and Android apps can now read each others QRcodes. - Wallets can now auto-save themselves, taking the hassle of managing wallet persistence away from your app. See the javadocs for Wallet.autoSaveToFile() for information on this. - The network layer was rewritten on top of Netty to be more robust, more scalable and to remove flakyness in the unit tests. Thanks to Miron Cuperman for this work. - Thanks to Matt Corallo the ping/pong protocol is now supported. Also various protocol conformance issues and other misc bugs were resolved. - WalletTool now has a RAW_DUMP option that prints the raw protocol buffer form as text. - You can now explicitly set fees on a created transaction using the fee member of SendRequest. Please note that the correct fees for a transaction are still not auto-calculated or minimized. This will come in a future release. - Many bug fixes. API changes: - TransactionConfidence.OVERRIDDEN_BY_DOUBLE_SPEND is now called DEAD - PeerGroup.broadcastTransaction now returns a Guava ListenableFuture (which is a subclass of Future, so it's compatible). The future completes when the transaction has been heard back from the network, instead of just being written out. - Wallet.sendCoins() now returns a SendResult that contains both the transaction, and the future returned by PeerGroup.broadcastTransaction(), so it will no longer block. As a result sendCoinsAsync() has been removed. - Various send methods on Wallet now take a SendRequest object that lets you customize the created transactions. The methods that let you explicitly set the change address are removed, you should set the changeAddress member of the SendRequest instead. -- Live Security Virtual Conference Exclusive live event will cover all the ways today's security and threat landscape has changed and how IT managers can respond. Discussions will include endpoint security, mobile security and the latest in malware threats. http://www.accelacomm.com/jaw/sfrnl04242012/114/50122263/___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Payment protocol thoughts
I've been thinking about the requirements for a payment protocol
lately. It seems we have consensus that we need one of these. Pieter
has a gist on the topic here: https://gist.github.com/1237788
IMHO we'll want to move away from send X BTC to address Y and more
towards upload to me transactions that send X BTC to outputs
A,BF,G. In this context by output I mean the obvious
interpretation of script+value.
The reasons are privacy and security. Privacy: you don't want people
to find out if you have an unexpectedly large balance (from the block
chain). Example worst case scenarios
- in a hypothetical Bitcoin-using country, as most people receive
income exactly once a month from their employer, you could potentially
find out other peoples salaries. Drama ensues.
- business partners could find out their counterparty has an
unexpectedly good financial position, messing up negotiations
Security:
- some of the outputs could be in a hot wallet, others in cold
wallets, helping you maintain correct balances between them in real
time
- the outputs can be multi-sig scripts
Wallets would then craft a series of transactions to try and somewhat
balance the size of inputs vs outputs. Because they are separate
transactions and all the keys are fresh, there's no way to link them
together into a single payment, especially not if they're broadcast in
random order with some jitter.
The upside of this is better privacy. The downside is obviously more
transactions and therefore more overhead. In theory the sum of tx
outputs would end up converging to a reasonable coin size for the
recipients, eg, businesses might be happy to receive a lot of money in
a single output, individuals less, children or very poor people maybe
much less.
Let's call payment requests invoices. Here is a brainstorm on other
features that may be desirable. I'm not suggesting they're all in v1,
just that we think about them a bit to ensure we don't paint ourselves
into a corner.
- Optional list of {signature, certificate} pairs. SSL certs can be
embedded into the payment request file itself so they can be checked
instantly for wallets that want to show a verified identity, but you
can also provide other certificates issued outside the regular SSL CA
system. For example maybe MtGox issues you a trusted vendor
certificate. Maybe the better business bureau issues you a cert, etc.
- Optional expiry time (from sipas gist) so outputs that were never
sent to can be recycled
- Upload target (URI), where to send the created transactions
- Optional message and branding image/icon that a wallet can display
to make the transaction history a bit prettier
- Opaque token that the wallet is supposed to copy into the payment.
The merchant can use to link invoice with payment. It's technically
redundant, the output set would identify the invoice too, as could a
token in the upload target URL, but it may be simpler for some
merchant implementations
- Ability to specify payment amount[s] in terms of other currencies.
If the amount is specified statically it can just be recorded in the
wallet for informational purposes. If there's a URL provided also, it
is an endpoint where quotes can be obtained. This allows merchants to
make long-lived invoices which are protected against FX volatility.
Downside: complicates wallets. Upsides: invoices can be kept around
for longer.
- Web/human-usable URL for the order so users can, eg, send messages
to the merchant specific to an order, post a review of the merchant,
etc
- Support for setting up 2-of-3 dispute mediation. Invoices should be
able to name a list of acceptable mediators and the wallet software
can intersect this with a list of mediators acceptable to the user, to
find one that works best. The whole mechanism by which merchants and
users agree on mediators isn't designed yet but we can at least think
about it now. It may be there's a simple design everyone agrees on
already.
- Support for requesting recurring payments. Eg, I should be able to
provide N sets of M outputs, one set for each payment with a payment
schedule. Wallets can then ensure they run at the appropriate times to
keep up the subscription.
- Suggested tip/service charge. Wallet would give a simple UI to
adjust this up/down by X percentage points
- Request to sign inputs with SIGHASH_ANYONECANPAY, allowing the
payment to be a pledge for an assurance contract
- Ability to specify minimum confirmation level of coins that will be
spent. Some merchants may be OK with you immediately re-spending
unconfirmed coins. Other merchants will care more and might want you
to take on the burden of getting your transactions into the chain.
On the payment upload side:
- An optional signature under a stable user key that lets users
optionally link their payments across merchants. In this way a user
can build cross-market reputation which may help them in future, by
relaxing confirmation requirements or reducing the chance of being
asked to enter dispute
Re: [Bitcoin-development] Payment protocol thoughts
I think it's worth pondering the different things we may want in future, even if that future is quite far out, just to ensure we have a robust design that won't box us in later. Brainstorming feature ideas now doesn't commit anyone to implementing them, but it may help improve the final v1 design. + Bitcoin addresses by themselves are insecure against man-in-the-middle attacks. A simple way to solve this problem is just use the SSL identity of the server that is taking part in the protocol, but it's not much harder to embed a signature + cert chain into the invoice itself. And once you're doing that, allowing several different sigs/cert chains is pretty easy. It means you keep the design open to cases where SSL may not be appropriate. Eg, you could create invoices signed by your web-of-trust identity, or some non-SSL Bitcoin specific verification system. None of those things have to actually be implemented, but by considering them now we can make the protocol more future prooof. + After sending payment I should have a receipt that proves I followed the payee's instructions, so if the payee says they never received the funds I can prove that it wasn't my fault. A signed invoice + the blockchain transactions does this, BUT with a major caveat: if you have not set up dispute mediation, there is nobody to prove faultlessness to. So I'm not sure this would be very useful. Supporting real dispute mediation seems more practical, but also more work. + Protocol for gathering signatures from multiple devices (extension/variation of the basic payment protocol, I think). This would be nice, I think invoices could be wrapped by another protocol that handles it. I'm not sure it needs to be a part of the core payment protocol. There are lots of different ways to implement this and I'm not sure there's agreement on what it should look like - somebody needs to build a proprietary implementation first. -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Electrum security model concerns
+gary Electrum also has a daemon for merchants. Well, I suggest taking it up with Thomas directly. A thread here won't do much. Considering the dislike of Java that exist reflexively in much of the non-java community and the greater ease of deployment and the integration of type-2 split key management I'm hoping that MultiBit Merchant will provide something similar based on bcj, ie, you don't have to actually be a Java developer to use it, it can just talk to your app via POSTs and GETs. WRT deterministic wallets, yes, right now that's indeed a competitive advantage of Electrum. So much code to write, so little time. Generally for thin clients— a lying server can make clients think they've received confirmed payments they haven't, and unless the client is constructed to be a bit less thin a lying server can lie about input values and cause thin clients to spend large values to fees. Yes indeed. This also gives [hacked] server operators a way to steal money from users without private keys, they can get clients to create some very high fee transactions and then provide them directly to a miner who promises to cut them in (or they can mine themselves, of course). Beyond that the protocol between the clients and servers is unauthenticated cleartext JSON in TCP. I thought it used SSL. Maybe I'm thinking of BCCAPI which is a similar approach. that the payment is unconfirmed. There is a pro mode, that shows 'processing' for unconfirmed transactions I think communicating transaction confidence to users is something of an open UI design problem right now. I agree that hiding it entirely seems suboptimal, but in reality explaining what the risks are for a given number confirmations is difficult. Given the lack of actually reported double-spends against unconfirmed transactions, I can understand this choice, even if I wouldn't recommend it. My only question is will they know this because we as a community and the authors of the thin clients provided clear explanations and appropriate caution Well, I pushed for English-text explanations of clients on bitcoin.org rather than a feature matrix, for this kind of reason :) Unfortunately the current texts are too small to really give a detailed explanation of the security models involved. It may be worth adding one-liners that link to a page explaining different security models (full, SPV, superlight). One thing I'm really hoping we can find and get agreement on is somebody clueful and trustworthy to work on the bitcoin.org website. Bitcoin, the project, needs a stronger voice than it currently has, partly to speak about such issues. For instance, an FAQ that isn't on the wiki would be good. And a simple Welcome to Bitcoin flow on the bitcoin.org website that guides people to appropriate clients, teaches them the security basics, etc, would be excellent. -- Don't let slow site performance ruin your business. Deploy New Relic APM Deploy New Relic app performance management and know exactly what is happening inside your Ruby, Python, PHP, Java, and .NET app Try New Relic at no cost today and get our sweet Data Nerd shirt too! http://p.sf.net/sfu/newrelic-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Public key and signature malleability
No objections. On Sun, Oct 21, 2012 at 7:05 PM, Gavin Andresen gavinandre...@gmail.com wrote: Any objections from other transaction-validating implementations? I strongly support more precisely defining the transaction validity rules by changing the reference implementation. -- -- Gavin Andresen -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] Draft BIP for Bloom filtering
I've written a draft BIP describing the bloom filtering protocol extension developed by myself and Matt. https://en.bitcoin.it/wiki/BIP_0037 (yes I know there's some kind of process around getting allocated a number - it seems overkill for this). Please read it and let me know if there are any missing details or things which sound wrong. Design-wise, it occurred to me as I wrote the BIP that the method of delaying reception of invs is a bit ad-hoc. It may be better to have a bloom filter be sent in the version message itself. On the other hand, having a flag to delay invs means that the filter can be calculated in parallel to bringing up the network connections. Whilst actually making a Bloom filter is fast, with deterministic wallets you may need to do a lot of calculations to find the keys to scan for. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Draft BIP for Bloom filtering
Some questions: * why limit the number of matching transactions to 255? Copy/paste error in the does :( * what does each hash and key in the output script mean exactly? what about the output script in its entirety? It's an informal way to say data elements. If you insert a key then it matches both single and multi sig outputs regardless of location. * is sharing parts of the merkle branches not worth it? We think probably not. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Draft BIP for Bloom filtering
What is the worst-case for an attacker interested in trying to get you to saturate your upstream bandwidth or use lots of memory? Set a bloom filter that matches everything, and then start requesting old blocks in the chain? It would be slightly worse than shipping a full block but not seriously so. If you just want to saturate bandwidth or disk IOPS you could probably just request random blocks over and over again. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Draft BIP for Bloom filtering
Presumably these components will just get implemented a few times in some carefully constructed library code, so I don't see an implementation complexity argument here— except the fact that it isn't what Matt has implemented so far. Well, yes, that is basically the implementation complexity argument :) Engineering time isn't free. I don't feel I understand the effort required to do some kind of partial tree encoding. Having a kind of custom compression whereby branches are represented as varint indexes into a dictionary, I can feel how much work that involves and maybe I can make time over the next few weeks to implement it. Has anyone got example code for representing partial Merkle trees? Also, it's not mentioned in the page— but the hash function used is not cryptographically strong, so what prevents a complexity (well, bandwidth in this case) attack? someone could start using txids and txouts that collide with the maximum number of other existing txouts in order to waste bandwidth for people. Is this avenue of attack not a concern? If you just want to waste bandwidth of nodes you can connect to nodes and repeatedly download blocks, or fill the network with fake nodes that spam random generated transactions to whoever connects. I don't see how to avoid that so it seems odd to worry about a much more complicated attack. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Draft BIP for Bloom filtering
Because I can potentially waste bandwidth of all nodes forever (well as long as users are still scanning blocks with my transactions in them) with O(1) work. And this gets you what? Users who have active wallets will have their bandwidth wasted for as long as you keep up the attack. Once you stop active wallets won't be rescanning that part of the chain and new users won't be scanning it either, as they skip blocks before their earliest key time using getheaders. So basically you can waste the bandwidth of active users for a while, by spamming transactions. This is not a new attack. Anyway, it's trivial to DoS the entire Bitcoin network today. It hasn't ever happened. Maybe one day it will, but the only rationale people can come up with for such an attack beyond random griefing is governments, and complexity attacks are really not their style. Much easier to just pass a law. I'm not saying DoS should be ignored, but I do feel there are limits to how far down that rabbithole it's worth going. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_sfd2d_oct ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] IRC meeting agenda, 18:00 UTC Thursday
I won't be able to make it this time. My feeling is IRC is a good place to
bounce ideas around when time and people happen to be available, but having
meetings there will inevitably lead to decision making that's better done
in a slower manner via email.
Comments:
BIP process: are we happy with how it is working? What can we do to improve
it?
Needing some kind of process to allocate a number is over the top. I
skipped this for the bloom filtering BIP. We should take off the part of
the {{BIP}} template that says don't just pick a number and add a bip -
that's exactly what people should do. I'm not sure there's any need for an
editing role either.
Is it time to feature-freeze 0.8
I'd like more time to get the bloom filtering work in. It'll be easier to
promote the 0.8 release if we can sell it as important
scalability/performance improvement for the network, upgrade to help
Bitcoin keep growing, as whilst there's no real auto update or organized
people who religiously update promotion is very important. I think
ultraprune + bloom filtering is the two major scalability improvements we
have right now.
--
Everyone hates slow websites. So do we.
Make your web apps faster with AppDynamics
Download AppDynamics Lite for free today:
http://p.sf.net/sfu/appdyn_d2d_nov___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Electrum security model concerns
BTW have you checked the code? I took a quick look and didn't see things I was expecting to see. In particular I couldn't find any code that manages wallet state in the presence of re-orgs. It appears to check that transactions appeared in the block chain, but if there's a chain switch it's not clear to me the wallet will be in the right state. I saw a message from Thomas on his thread saying something like can't spend coins bug happens when there's a re-org and the server gives you the wrong histories, to fix it reset your wallet and switch to a new server which to me rather implies there's no re-org handling at all. If Electrum does end up doing all SPV work correctly, how is it different to MultiBit? Just the deterministic wallet seeding? On Fri, Nov 16, 2012 at 4:59 PM, Mike Hearn m...@plan99.net wrote: Great to hear that. -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Payment Protocol Proposal: Invoices/Payments/Receipts
On Tue, Nov 27, 2012 at 9:43 AM, Michael Gronager grona...@ceptacle.com wrote: * What if the SignedReceipt is not received AND the transactions IS posted on the p2p. I think this is a problem with confusing terminology rather then the spec itself. The original formulation had a receipt being something generated purely by the buyer. The signed Invoice message + the Bitcoin transactions paying to the outputs + the merkle branches showing acceptance by the network *is* the receipt. The SignedReceipt message is useful in the sense that it shows confirmation by the merchant, but if you don't get one, you can still prove you paid the invoice. So from this perspective perhaps SignedReceipt should be renamed to Acceptance or something like that, and then the spec should call out that a signed invoice plus accepted Bitcoin transactions is mathematically a proof of purchase. -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Payment Protocol Proposal: Invoices/Payments/Receipts
That hash would then be reported via some secure channel outside of bitcoin's domain. OK, I see. I guess that could be a reasonable fallback for the case where you have a secure channel. I don't understand what the relevance of multi-factor is to invoices? Yes, exactly. It's about paying who you think you're paying (when you confirm on a second uncompromised device). -- Monitor your physical, virtual and cloud infrastructure from a single web console. Get in-depth insight into apps, servers, databases, vmware, SAP, cloud infrastructure, etc. Download 30-day Free Trial. Pricing starts from $795 for 25 servers or applications! http://p.sf.net/sfu/zoho_dev2dev_nov ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Payment Protocol Proposal: Invoices/Payments/Receipts
I'd still like to understand the rationale for having the merchant broadcast the transaction There are several reasons for this: 1) P2P network sockets are a limited resource and bringing up connections to the network, whilst somewhat fast today, is not guaranteed to be fast in future. Passing transactions to the merchant for broadcast reduces the load on the P2P nodes because lots of thin clients aren't any longer connecting and disconnecting when sending. They only need to talk to the network when the user has received money. 2) Some users may not have network connectivity at all. For example, this happens quite often whilst traveling at Bitcoin conferences ;) The solution, which Andreas and I prototyped in Berlin together, is for the buyer to communicate only with the seller which can be done over Bluetooth or WiFi Direct or some other mobile radio protocol. Again, send only, but for the common case where you load up your wallet before setting out and then buy things, it works OK. 4) A longer term reason - in time, people may choose to not broadcast transactions at all in some cases. I think how network speed will be funded post-inflation is still an open question. Assuming the simplest arrangement where users pay fees, getting transactions into the chain has a cost. In cases where you trust the sender to not double spend on you, you may keep a fee-less transaction around in your pocket. Then when it's your turn to pay, you use some unconfirmed transactions to do so. People pass around longer and longer chains of un-broadcast transactions until a payment crosses a trust boundary, at which point the receiver adds on their own transaction that spends back to himself but with a fee, and broadcasts them all together as a unit. In this way only people who genuinely need to fear double spends pay for security. -- Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Has anyone compiled under MacOS 10.8?
I found that the problem is the version of the Qt SDK I used didn't like the new MacOS version. Re-installing Qt fixed it. On Mon, Nov 26, 2012 at 4:05 PM, Mike Hearn m...@plan99.net wrote: It appears that something about Boost doesn't play nicely with the default build instructions (possibly the switch to clang++?). I will dig in eventually but for now, if anyone has a recipe that fixes things, let me know. -- Keep yourself connected to Go Parallel: VERIFY Test and improve your parallel project with help from experts and peers. http://goparallel.sourceforge.net ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Chain dust mitigation: Demurrage based Chain Vacuuming
The main source for these 1 Satoshi payouts is Sahtoshi Dice. Because people are making 1 satoshi bets, or is this part of their messaging system? Pieter is right, getting consensus behind your proposal is too hard and it's not likely to ever happen (I wouldn't support it, for one). Outputs that never get spent are simply using disk space, the working set is really defined by the coins that are moving. Disk space is cheap. So this problem doesn't feel that urgent to me. Now if people were routinely spending those 1 satoshi outputs, it'd be less great as it'd increase the working set size. I suspect some of these coins can be cleared over time by adjusting wallets to consolidate outputs into the change outputs when a transaction that has spare space before reaching the next size/fee level takes place. -- Keep yourself connected to Go Parallel: BUILD Helping you discover the best ways to construct your parallel projects. http://goparallel.sourceforge.net ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Chain dust mitigation: Demurrage based Chain Vacuuming
It's part of their messaging system. Every losing play results in a new 1e-8 output being created. Every losing play? That's ... not excellent. Well, this why the payment protocol spec has a way for merchants to reply to customers with text instead of outputs. -- Keep yourself connected to Go Parallel: BUILD Helping you discover the best ways to construct your parallel projects. http://goparallel.sourceforge.net ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Chain dust mitigation: Demurrage based Chain Vacuuming
Perhaps it could be improved by cleaning up dust from any address by default (not just ones already included in the tx), with the option for the user to disable that behavior. After all, anonymity was never a core feature of the network It's cool that Armory already does this. I never had time to implement good coin selection for bitcoinj :( Just a couple of points: as this is primarily a side effect of SatoshiDice, and a successful payment protocol will stop them doing it, code put in place to do temporary cleanup now probably won't seriously affect peoples privacy over the long term. Most people aren't going to end up with lots of tiny outputs. Second thing, it's best to carefully separate anonymity from privacy. Privacy is supposed to be a feature of the system (it says so in Satoshis paper) because people demand it. If I loan a tenner to my friend and he is able to find out what I earned last month, then that trade was neither anonymous nor private. In this case I want privacy but anonymity isn't useful. Mixing up anonymity with privacy is not only a public relations problem, but can lead to confusion from users when they, eg, try and buy Bitcoins from an exchange and are asked to provide ID proofs. -- Keep yourself connected to Go Parallel: BUILD Helping you discover the best ways to construct your parallel projects. http://goparallel.sourceforge.net ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Payment Protocol Proposal: Invoices/Payments/Receipts
So, if a bitcoin client is getting Invoice messages via email or from
a web server, the version will be specified as part of the MIME type;
for example:
Content-Type: application/x-bitcoin-invoice; version=1
The version= syntax is part of the MIME standard.
I think that's OK. However, you should only be getting the version you
expect because when you request an invoice, your client should be
telling the merchant what protocol version you implement.
Does it make sense to have this spec not include the details of
bootstrapping? It's not complicated - we extend the URI spec in a
backwards compatible way:
bitcoin:1AbCdEfG?value=10.0label=Pay%20for%20Fooinvoice=https://merchant.com/inv/aB425az
When a compatible client sees the invoice param, it ignores the rest
of the URI and downloads the URL
https://merchant.com/inv/aB425az?ver=1.0
A server on merchant.com sees that the client expects a version 1.0
invoice and vends it. If ver=2.0 or whatever, it knows it can use 2.0
features. If extensions are supported, add new query params.
We should define a simple mechanism for extending the protocol now, so
people who want to make proprietary extensions don't conflict. The
simplest is to just say, if you want to add new fields to an Invoice
message, please update a wiki page with the tag numbers you're going
to use, and start from number X. Protobufs have a simple way to
formalize this in the language:
https://developers.google.com/protocol-buffers/docs/proto#extensions
message Invoice {
extensions 1000 to max;
}
The point of this is to allow you to define new parts of the messages
in separate .proto files. It's only a minor convenience but it means
if you want to use, say, two extensions that weren't yet folded into
the main spec, you can more easily do so without having to do a manual
merge of the message definitions together.
For instance, if you wanted to extend the protocol to support
specification of recurring billing, you could make a file called
recurring-invoices.proto containing:
message Recurrences {
required uint32 every_seconds = 1;
optional uint32 start_time = 2;
}
extend Invoice {
optional Recurrences recurrences = 1005;
}
then you update the wiki page to claim tag number 1005 and apps can
easily use your new features. If/when the feature gets standardized
via a BIP, the core .proto definition can be extended to include these
messages and the extensions can go away.
--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Roadmap to getting users onto SPV clients
It sounds to me that you're insisting that you're asking people who oppose degrading our recommendations to commit to a costly rushed development timeline. I think this is a false choice. Hardly. I don't have any particular timeline in mind. But I disagree we have forever. New ideas have a certain time window to take off and become credible. If they never overcome their problems in that time window, eventually people just give up and move on. Does anyone take desktop Linux seriously anymore? No. The year of desktop Linux is a joke. People took it seriously in 2001 but despite great progress since, the excitement and attention has gone. There were steady improvements over the last 10 years but nobody is creating desktop Linux startups anymore - Bitcoin shouldn't go the same way. It's unclear we need to have every man and his dog run a full node. Tor is a successful P2P network where the number of users vastly outstrips the number of nodes, and exit nodes in particular are a scarce resource run by people who know what they're doing and commit to it. The Tor guys could have said every node should be an exit if possible, but that would have been a short term optimization at the cost of long term stability, and anyway doesn't seem to have been necessary so far. Even with no incentives, they were able to obtain the resources they need. So why should Bitcoin be different? If there are a million users supported by 50,000 full nodes, that wouldn't sound unhealthy to me. We can easily send a clear and consistent this is important, please help message without complicated auto-upgrade/downgrade schemes that risk annoying users. -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Roadmap to getting users onto SPV clients
I was under the impression that connectedness was the real metric of concern I think the real thing we need full nodes for is sockets where by socket I mean resources needed to serve another node. Last year we actually ran out of sockets and it took forever for new nodes to connect because so many existing nodes were full. We don't want to be in that situation again. So we need full nodes, nobody disputes that. The question is, if you have a node on your average desktop machine that gets switched off at night, has a stupid virus scanner that insists on checking every database write, has users who go from a bit of light word processing to watching HD video and expect no stutters or slowdowns - how valuable is such a node, really? Also has to be weighed against the risk of eventual user frustration when they discover Bitcoin is slowing their computer down and go around telling their friends how much it sucks. Ultraprune+LevelDB+other optimizations are great. They aren't game changers for two reasons: 1) Eventually network traffic should increase to use up the additional performance unlocked by optimizations 2) Users demand instant on not just at first start, but any time they open their wallet. I don't think it ever makes sense for a regular end user to have their wallet integrated with a full node because it means if you get an email saying oh hey I sent you the money and you start your wallet so you can see it/spend it, you still have to wait a while until it catches up from whenever it was last quit. I've done this a bunch of times and it really sucks to wait. The only time it makes sense to have a wallet integrated with a full node is if that node never shuts down, ie, it's a merchant node. If a casual user has to be using an SPV wallet all the time no matter what, then it's not a big leap to simply have both an SPV client and a full node running in parallel for users who want to support the network. And how do we recruit such users? Well I've got nothing against light wallets noticing that the system seems to have high uptime, external connectivity etc and putting a notice on the screen asking users to take part. For Windows users you could have a one-click install that sets up a background service (I think .NET OneClick makes this possible), so getting a full node is totally easy and transparent. Going back to the Tor analogy, whilst I agree with Gregorys arguments that they aren't quite the same, the Tor guys have wanted to automatically opt users in to being relays for a while. But the technical complexity of doing it well is really high. It's still on their wishlist even though Tor is quite old. A good first base to reach is simply having accurate recommendations. If users start complaining that they were asked to run a full node but when they did, performance suffered unacceptably, then we know we need better heuristics before automatically opting users in. -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Payment Protocol Proposal: Invoices/Payments/Receipts
Escrow/multisig is complicated enough to wait for another day. But certainly having a payment protocol is an important step towards it On 6 Dec 2012 07:32, Andreas Petersson andr...@petersson.at wrote: During/before the Payment Request there should be a method to exchange the public keys to be able to generate a common multisig address. Should this be handled in a different protocol, or be included in this spec? Or is there a method for the customer to verify that the specified BIP16 Output contains his address and the one from an escrow service? -- Andreas -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Payment Protocol Proposal: Invoices/Payments/Receipts
Re: the newest spec. Rather than make the signature over the
concatenation of, why not just make it a signature over the
serialized protobuf minus the signature field (as I did in my demo
code). Otherwise it seems like we'd need more code than really
necessary. We can state explicitly tags must be ordered if you want,
even though all implementations should do that already.
Thumbnail sketch: escrow service or participant sends around an
EscrowProposal, gets EscrowProposalACK's with public keys to use, then sends
all participants an EscrowEstablished message with the final multisig script
or address.
Yeah, that sounds reasonable. Not that we should really design it now,
but let's use the term mediation rather than escrow, which has
connotations of depositing funds with the escrow service.
I think it's best to see the existing payment messages as structures
that'll get filled out with more features over time. So rather than
have a separate EscrowProposal message, you would integrate it with
payment requests. Older clients that don't understand mediation would
just ignore the extra data they don't recognize.
message PaymentRequest {
// One per mediator acceptable to the seller.
repeated MediationProposal mediation_data = 10;
}
message MediationProposal {
required SignedMediatorIdentity identity = 2;
// Opaque bytes that the mediator can be asked to turn into a human
readable description
// of how disputes will be mediated. The merchant sets this to
describe whatever policy it
// is willing to go along with, so policies may be arbitrarily complicated.
required bytes policy = 3;
}
message SignedMediatorIdentity {
required MediatorIdentity identity = 1;
// If the identity data is signed ...
optional string pki_type = 2;
optional bytes pki_data = 3;
optional bytes pki_signature = 4;
}
message MediatorIdentity {
// Name of the mediator to be displayed to the user.
required string friendly_name = 1;
// PNG image that can be used to represent the mediator to the user.
optional bytes logo = 2;
// Some text shown to the user under the name explaining the
mediators policies, why they should be chosen, etc.
optional string blurb = 3;
// An HTTP URL where a mediator can be reached to do things like
prove ownership of pubkeys, initiate the protocols, etc.
required string contact_url = 4;
}
... etc
So the user experience would be that when a payment request is received:
- older clients ignore the mediation_data field and do a direct
payment as normal
- newer clients ask the user to pick a mediator (if they want to) and
if mediation is requested, the PaymentRequest is then discarded and
the next step of the mediation protocol begins.
The old request has to be discarded because the outputs would have
been written on the assumption of no mediation being in use (for
backwards compatibility).
Anyway, though I'm awfully guilty, let's not get off track. Just that
this is how I imagined new payment features being done - as new
extensions to the payment protocol, which would be a living document
amended by BIPs.
--
LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial
Remotely access PCs and mobile devices and provide instant support
Improve your efficiency, and focus on delivering more value-add services
Discover what IT Professionals Know. Rescue delivers
http://p.sf.net/sfu/logmein_12329d2d
___
Bitcoin-development mailing list
Bitcoin-development@lists.sourceforge.net
https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Payment Protocol Proposal: Invoices/Payments/Receipts
OK. I want to keep the signature field required, though, so how about: signature: digital signature over a protocol buffer serialized variation of the SignedPaymentRequest message where signature is a zero-byte array and fields are serialized in numerical order (all current protocol buffer implementations serialize fields in numerical order), using the public key in pki_data. Looks good to me. -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Payment Protocol Proposal: Invoices/Payments/Receipts
Yet more comments (I guess at some point we need to stick a fork in it - or at least move on to implementing a prototype version). Maybe don't require the payment URI to be HTTPS. If you want to pay a Tor hidden service then HTTPS just adds unnecessary complexity. Just recommend to merchants that they use an encrypted connection and leave it at that. Though it's not strictly necessary, it'd be nice to have defined behavior for if you want to pay more than the requested amount, for a tip. Perhaps rather than zero value outputs will be ignored say, if some outputs have value and others don't, the user will be given the option of overpaying and the extra money will be split evenly between the zero valued outputs. That way a waitress can have the phone add a zero-valued output to her own wallet and that would prompt the wallet software to display some convenient UI for adding on 10% or whatever. receiptURI - receipt_url ? technically it has to be resolvable so uri isn't quite right. Display the proposed Outputs in as human-friendly a form as possible ??? Surely you'd just display the total amount requested? I don't think it ever makes sense to try and display outputs to the user directly. Re: the UI TODO - agreed but let's take it out of the BIP and maybe make it an alternative document. Or just replace it with a recommendation that the user interface should be designed to ensure users understand the difference between an unsigned and signed payment request, for best practices see here serialized_paymentrequest - serialized_payment_request? Otherwise languages that use CamelCase will look odd setSerializedPaymentrequest() The question of root CAs still needs resolution. I stick with my recommendation to support all CAs that browsers support. Obviously, it's better for a merchant to obtain an EV cert than a domain verification cert - the UI can reflect the higher level of verification. I doubt there's a need to specify a max number of certs in a chain. But if you want to, go for something high, like 256. There's no point in trying to put DoS mitigations into something like this, for the same reason HTML doesn't impose a maximum page size. It's in the message builders interest to ensure it gets read by all users. Crashing their clients doesn't achieve anything as long as the crash isn't exploitable. On Fri, Dec 7, 2012 at 11:45 AM, Mike Hearn m...@plan99.net wrote: OK. I want to keep the signature field required, though, so how about: signature: digital signature over a protocol buffer serialized variation of the SignedPaymentRequest message where signature is a zero-byte array and fields are serialized in numerical order (all current protocol buffer implementations serialize fields in numerical order), using the public key in pki_data. Looks good to me. -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Payment Protocol Proposal: Invoices/Payments/Receipts
yeah... I had similar thoughts on what to do if some Outputs specify an amount and others don't. I'm still waffling on whether or not I like allowing repeated Outputs; a single Output would make the spec a fair bit simpler Yes, but at the cost of privacy. Generators of payment requests always have the option of just adding a single output and being done with it. But in future they'll probably want to keep their income in unlinkable chunks of a size that's up to them, and multi-outputs are needed for this (the idea being, the users wallet tries to keep a close-as-possible match between the requested outputs and their own). OK, let's punt on tipping for now. If a bitcoin client accepts unsigned payment requests (a couple of people have asked if that would be possible so I think that is desired), then it doesn't have the payer's identity-- all it has is the Outputs that will be paid. I see. If I were to implement a wallet I'd just display nothing (except the size of the request). Showing an address doesn't really help the user in any way. I still like the idea of only including the root CAs who have jumped through the hoops needed to get the allowed to issue EV certs blessing. The hoops only actually apply for EV certs though, they aren't required to do that verification for DV certs. The main reason to use the browser root CAs is that merchants are guaranteed to be able to re-use their existing certs. Otherwise they might have to buy new ones, which would be annoying. -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Payment Protocol Proposal: Invoices/Payments/Receipts
Thanks for the thoughts. For those who don't know, Stephen works for BitPay. My first observation is that the proposal is too heavily oriented around a merchant/customer interaction. The term merchant is just being used to mean the entity requesting the payment. I'm hopeful that in future mobile wallets will be creating these messages where today they'd make URLs and QRcodes. So it's definitely intended for person to person payments too. you may find yourself in a situation needing to parse a protobuf message in a web browser Nothing stops you converting them into whatever form you want on the server side. If you don't care about the signature checking then it's no problem to use a server. If you do then you'd need to ship all the code for verifying signatures that to the client anyway, at which point a small protobuf parser is hardly a deal killer. ...like what about the casual user that wants to create a payment request to send to their friend over email They can send an unsigned payment request. Note that if you mail it as an attachment from a competent, up to date email provider then the attachment isn't really unsigned. The whole thing is covered by the emails DKIM signature which is applied transparently by the ESP. If the signature fails to verify then the mail client can show that or treat the mail differently (as Gmail does). This is easy to use for the end user - they don't have to think about cryptography or PKI. As long as their email account is secure then they can send signed mails asserting to their identity. Refund addresses...this is not going to be as useful as people might think...most refunds that bitpay needs to process happen days or even months after the initial purchase Useful feedback, thanks. Still, there may be other types of merchants for whom it's useful, and many users won't change their wallet. It certainly simplifies things if you can present the refund address and give a one-click option to use it. If the user wants to use a different address, then they can go onto the slow/complicated path. This current spec deliberately punts on the topic of identifying end users. It's a difficult problem. Whereas many merchants have SSL certs, most end users don't have published keys in any useful form. By far the easiest way for 99% of people to generate a signed message is to send email that's signed by DKIM (from gmail, hotmail, yahoo, other providers etc). Then it's all transparent and behind the scenes. Their identity is their email address. So for BitPays scenario, you could require an email to be sent by the end user containing new instructions. Your MTA can show you whether the mail is correctly DKIM signed or not when deciding whether to follow the instructions. But, if the recipient never cashes the check, the sender could pull those funds back after a certain period of time. Yeah, I like the term check/cheque for that concept of a reversible payment. I like the use of merchant_data...this means that you no longer will need a unique bitcoin address for every invoice. It's still a good idea to use one for privacy reasons. The merchant data is there so you can stuff whatever state you want into it. So it's like cookies. You don't have to keep state on the server side. Just encrypt/sign it, put it in the invoice, and when you get a payment message back there's no need to do database lookups or anything, you can just do some crypto and know who is submitting it. Generally speaking, I'm not a fan of embedding things like that What's wrong with it? Isn't your proposal more complex? I don't see why it's better than just embedding it. The Receipt should be signed...it could be used as proof of payment by wallets. There's no Receipt message, a SignedPaymentRequest + transactions that pay to the requested outputs are together the proof of payment. Re a new URI scheme. Interesting idea, thanks for the suggestion. It seemed like it'd be easier for merchants to integrate if a single linked worked for all wallet implementations/versions. But I guess we could do both schemes, even. -- LogMeIn Rescue: Anywhere, Anytime Remote support for IT. Free Trial Remotely access PCs and mobile devices and provide instant support Improve your efficiency, and focus on delivering more value-add services Discover what IT Professionals Know. Rescue delivers http://p.sf.net/sfu/logmein_12329d2d ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Has anyone compiled under MacOS 10.8?
The problem mysteriously resurfaced. The magic incantation this time is to add: unix:*-g++*: QMAKE_CXXFLAGS += -fpermissive to the top. I suspect this may be related to how I am using qmake. To make it spit out a real makefile instead of an xcode project, I have to run it like gmake -spec macx-g++, which perhaps ends up making gcc stricter than it's supposed to be. On Thu, Nov 29, 2012 at 8:34 PM, Mike Hearn m...@plan99.net wrote: I found that the problem is the version of the Qt SDK I used didn't like the new MacOS version. Re-installing Qt fixed it. On Mon, Nov 26, 2012 at 4:05 PM, Mike Hearn m...@plan99.net wrote: It appears that something about Boost doesn't play nicely with the default build instructions (possibly the switch to clang++?). I will dig in eventually but for now, if anyone has a recipe that fixes things, let me know. -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnmore_122712 ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Draft BIP for Bloom filtering
Here's a quick update on where we're up to. Thanks to Matts excellent work, I was able to test his bitcoinj and bitcoin-qt work together today. There are a few minor tweaks needed, but I feel like we're maybe a week away from having all the code in a mergeable state. Here is the remaining work: - There are a couple of bugfixes needed on the bitcoinj side: the fallback to downloading full blocks is problematic and needs to be deleted, there's an API change we want - Adjust the default FP rate requested by BCJ to be 0.0001, this is appropriate for the latest blocks in the chain and yields 0-5 false positives per block - Introduce a new part to the filter protocol that allows clients to control auto-expansion. This turned out to be very volatile, we saw jumps from 0-3 FPs per block to 500 in the space of 1 block, perhaps if a SatoshiDice transaction got into the filter. A simple yes/no flag can suffice for now, but a better solution would be for the client to submit templates for output scripts that would trigger auto-adding the matched outpoint - autoexpansion is only needed in the case where the input script doesn't contain any predictable data. For pay-to-address and P2SH it does, so expansion doesn't help. Matt said he'd hopefully try to look at this soon. With auto-expansion disabled, the FP rate adjusted and a bugfix on the bcj side I was able to sync a wallet using a bloom filtered chain. Although it's tight, I think this work should go into 0.8 - it'll be much more compelling to advertise it this way, we can say Upgrade to 0.8 and help network performance for everyone. And in the case that we discover a showstopper problem, we just don't deploy the code that uses the new messages into clients. -- Master Visual Studio, SharePoint, SQL, ASP.NET, C# 2012, HTML5, CSS, MVC, Windows 8 Apps, JavaScript and much more. Keep your skills current with LearnDevNow - 3,200 step-by-step video tutorials by Microsoft MVPs and experts. ON SALE this month only -- learn more at: http://p.sf.net/sfu/learnmore_122712 ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Draft BIP for Bloom filtering
Oh, one last stat - syncing the entire chain with a wallet containing two keys and a 0.0001 FP rate (one or two FPs every 5 blocks or so) resulted in a download of about 46mb of data. On Fri, Jan 11, 2013 at 3:11 PM, Mike Hearn m...@plan99.net wrote: I did some very rough initial performance tests. Syncing from a local peer gives me about 50 blocks per second in the later parts of the chain (post SD), which is about a 10-20x speedup over what I could do before. This is on a MacBook Pro. But at those points it's clearly bottlenecked by bitcoind which has saturated its CPU core. This makes sense - the filtering is much more server than client intensive because every transaction in every block has to be loaded and checked. I think filtering can be fairly well parallelized on the server side. So the current 10-20x speedup could potentially be larger if the server becomes more efficient at scanning and filtering blocks. It's still a very nice win for now, especially bandwidth wise. And if Matt makes the mempool command filtered it solves a common usability problem as well. Once we get this code in, merged and rolled out I think what we need for bloom v2 is clear: - Multi-thread the filtering process in bitcoind so transactions can be checked in parallel. A 4-core server would then get 4x faster at filtering blocks and assuming it's not too busy doing other stuff we could maybe sync at more like 200 blocks per second, which is cool ... more than a days worth of history for each second of syncing. - Make the client smarter so the FP rate is adapted during the sync process. An FP rate that makes sense post-SD results in no false positives pre-SD, more or less. - Make the client shard its wallet keys over multiple peers, for better privacy. - Make the client suck down filtered blocks in parallel from multiple peers, for better speed. As it seems the bottleneck for chain sync is now CPU time, the latter point may be the most important from a practical perspective. On Fri, Jan 11, 2013 at 6:02 AM, Jeff Garzik jgar...@exmulti.com wrote: On Thu, Jan 10, 2013 at 10:59 PM, Matt Corallo bitcoin-l...@bluematt.me wrote: Ive been missing lately, when is 0.8 targeted for freeze? 0.8rc1 will probably happen when the core ultraprune/leveldb stuff is stable. -- Jeff Garzik exMULTI, Inc. jgar...@exmulti.com -- Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and much more. Get web development skills now with LearnDevNow - 350+ hours of step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122812 ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Draft BIP for Bloom filtering
Matts latest code has been tested by Andreas and seems to work correctly. He had to extend the client a bit to refresh the filter every 25k blocks because even with the extra flag, eventually the filter degrades into uselessness, but it did still improve the situation quite a bit. Because it's unit tested, been reviewed by me several times, has an interoperable implementation that has also been tested by Andreas in a build of his smartphone app, I'm going to ACK the current code and request that it be merged in to 0.8. What do you say Gavin? The next step after that would be profiling. It's a big performance improvement for SPV clients already, but not as much as I anticipated. I suspect there's a simple bottleneck or missed optimization somewhere. But that can obviously come post-0.8 -- Master Java SE, Java EE, Eclipse, Spring, Hibernate, JavaScript, jQuery and much more. Keep your Java skills current with LearnJavaNow - 200+ hours of step-by-step video tutorials by Java experts. SALE $49.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122612 ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Draft BIP for Bloom filtering
I'm thinking we should actually make the change we talked about before and have the filtered block sent before the transaction data. For one, it's not intuitive (API wise) that you'd get a callback saying new pending tx immediately before another callback saying tx was confirmed, but that's what the current setup makes most natural. To fix it we'd have to notice that a tx message wasn't requested by us, buffer it, and wait for the corresponding filteredblock message. It seems cleaner to receive a filteredblock and then for any tx that matches it, attach it to the FilteredBlock object and wait until it is full up, then pass it to the wallet code all at once. Another issue is that to risk analyze unconfirmed transactions you really have to download all dependencies. That has to be triggered by seeing an unconfirmed transaction. It's dumb to start this process for a tx that is actually in the chain, so you need to have some notion of whether it came from a filtered block anyway. I only realized this today. I think when we discussed this before, the justification for having it work the current way was that it was simpler to integrate with the SPV client code if it was done this way around. But I don't think it's really simpler. There are enough odd side effects of doing it this way, that I feel it'd be better to tweak the protocol now whilst we have the chance. On Wed, Jan 16, 2013 at 4:00 PM, Matt Corallo bitcoin-l...@bluematt.me wrote: Actually, there is one more minor algorithmic change I would like to make to the way the hash function is computed really quick before it gets merged, I'll have that finished up by the end of today. Matt On Wed, 2013-01-16 at 11:43 +0100, Mike Hearn wrote: Matts latest code has been tested by Andreas and seems to work correctly. He had to extend the client a bit to refresh the filter every 25k blocks because even with the extra flag, eventually the filter degrades into uselessness, but it did still improve the situation quite a bit. Because it's unit tested, been reviewed by me several times, has an interoperable implementation that has also been tested by Andreas in a build of his smartphone app, I'm going to ACK the current code and request that it be merged in to 0.8. What do you say Gavin? The next step after that would be profiling. It's a big performance improvement for SPV clients already, but not as much as I anticipated. I suspect there's a simple bottleneck or missed optimization somewhere. But that can obviously come post-0.8 -- Master HTML5, CSS3, ASP.NET, MVC, AJAX, Knockout.js, Web API and much more. Get web development skills now with LearnDevNow - 350+ hours of step-by-step video tutorials by Microsoft MVPs and experts. SALE $99.99 this month only -- learn more at: http://p.sf.net/sfu/learnmore_122812 ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Draft BIP for Bloom filtering
Sorry, to clarify, these are test builds available here: https://code.google.com/p/bitcoin-wallet/downloads/detail?name=bitcoin-wallet-2.39_bitcoinj0.7.apkcan=2q= It's not on the Play store yet. It probably makes sense to release after some more testing and after Bitcoin 0.8 comes out, as otherwise there's a risk that 0.7 snapshot nodes will get overloaded. On Wed, Jan 30, 2013 at 12:09 PM, Mike Hearn m...@plan99.net wrote: Andreas has uploaded Android builds that use the new bloom filtering and peer selection code (also, dependency analysis of transactions). The performance gain is very cool. The app feels dramatically faster to start up and sync. Because the app syncs on charge when I opened it around lunchtime it had only 7 hours of data to sync (42 blocks) and it brought up 6 peer connections, found a 0.7.99 node and synced all in 2 seconds. That was on wifi. The next lowest hanging perf fruit is almost certainly to optimize disk accesses. Flash on Android devices seems to be much slower than laptop flash storage, and current bitcoinj is very inefficient in how it writes (one write per block header!). This matters a lot when doing fast catchup for first time users. The BIP is now a little bit stale, but only slightly. On Wed, Jan 16, 2013 at 4:00 PM, Matt Corallo bitcoin-l...@bluematt.mewrote: Actually, there is one more minor algorithmic change I would like to make to the way the hash function is computed really quick before it gets merged, I'll have that finished up by the end of today. Matt On Wed, 2013-01-16 at 11:43 +0100, Mike Hearn wrote: Matts latest code has been tested by Andreas and seems to work correctly. He had to extend the client a bit to refresh the filter every 25k blocks because even with the extra flag, eventually the filter degrades into uselessness, but it did still improve the situation quite a bit. Because it's unit tested, been reviewed by me several times, has an interoperable implementation that has also been tested by Andreas in a build of his smartphone app, I'm going to ACK the current code and request that it be merged in to 0.8. What do you say Gavin? The next step after that would be profiling. It's a big performance improvement for SPV clients already, but not as much as I anticipated. I suspect there's a simple bottleneck or missed optimization somewhere. But that can obviously come post-0.8 -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_jan___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Draft BIP for Bloom filtering
Can somebody please unlock the BIP wiki page? I don't know why it was locked but it's stale. On Wed, Jan 30, 2013 at 12:13 PM, Mike Hearn m...@plan99.net wrote: Sorry, to clarify, these are test builds available here: https://code.google.com/p/bitcoin-wallet/downloads/detail?name=bitcoin-wallet-2.39_bitcoinj0.7.apkcan=2q= It's not on the Play store yet. It probably makes sense to release after some more testing and after Bitcoin 0.8 comes out, as otherwise there's a risk that 0.7 snapshot nodes will get overloaded. On Wed, Jan 30, 2013 at 12:09 PM, Mike Hearn m...@plan99.net wrote: Andreas has uploaded Android builds that use the new bloom filtering and peer selection code (also, dependency analysis of transactions). The performance gain is very cool. The app feels dramatically faster to start up and sync. Because the app syncs on charge when I opened it around lunchtime it had only 7 hours of data to sync (42 blocks) and it brought up 6 peer connections, found a 0.7.99 node and synced all in 2 seconds. That was on wifi. The next lowest hanging perf fruit is almost certainly to optimize disk accesses. Flash on Android devices seems to be much slower than laptop flash storage, and current bitcoinj is very inefficient in how it writes (one write per block header!). This matters a lot when doing fast catchup for first time users. The BIP is now a little bit stale, but only slightly. On Wed, Jan 16, 2013 at 4:00 PM, Matt Corallo bitcoin-l...@bluematt.mewrote: Actually, there is one more minor algorithmic change I would like to make to the way the hash function is computed really quick before it gets merged, I'll have that finished up by the end of today. Matt On Wed, 2013-01-16 at 11:43 +0100, Mike Hearn wrote: Matts latest code has been tested by Andreas and seems to work correctly. He had to extend the client a bit to refresh the filter every 25k blocks because even with the extra flag, eventually the filter degrades into uselessness, but it did still improve the situation quite a bit. Because it's unit tested, been reviewed by me several times, has an interoperable implementation that has also been tested by Andreas in a build of his smartphone app, I'm going to ACK the current code and request that it be merged in to 0.8. What do you say Gavin? The next step after that would be profiling. It's a big performance improvement for SPV clients already, but not as much as I anticipated. I suspect there's a simple bottleneck or missed optimization somewhere. But that can obviously come post-0.8 -- Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] RFC: empty scriptPubKeys and OP_RETURN for marking unspendable txouts
So what exactly was the OP_RETURN bug anyway? I know it has something to do with not executing the scriptSig and scriptPubKey separately (https://bitcointalk.org/index.php?topic=58579.msg691432#msg691432) but commit 7f7f07 that you reference isn't in the tree, nor is 0.3.5 tagged. It was fixed by Satoshi long ago, back when we used CVS I think. The problem was how scripts were executed. They were concatenated together and then run as a single unit. The now obsolete OP_CODESEPARATOR was put between them to control what was hashed and what wasn't. The obvious problem with that arrangement being that scriptSig ran first (it has to, to push the signatures onto the stack), so nothing stopped you setting a scriptSig to OP_RETURN and making the script evaluate to true, always. A pretty amazing oversight given the thought and care that went into Bitcoin generally, and its robustness since then. The fix was to move to the current system whereby the two scripts are executed independently but sharing a stack, and it's only the return value of the scriptPubKey that matters. The scripting system always struck me as a rather late addition to the design. Satoshi admitted as much when he said that he added it after encountering an explosion of special cases as he designed various types of contracts. The fact that there's an obvious bug in CHECKMULTISIG is more evidence of this part being a general rush job, along with Satoshis willingness to disable much of its functionality later with the IsStandard checks. Also the design of CHECKSIG is an obvious retrofit, it would have made far more sense to decompose it, and we never found a use case for 99% of the opcodes despite having successfully designed (redesigned?) all the contract types he ever mentioned. -- Free Next-Gen Firewall Hardware Offer Buy your Sophos next-gen firewall before the end March 2013 and get the hardware for free! Learn more. http://p.sf.net/sfu/sophos-d2d-feb___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] bitcoinj 0.7 released
I'm pleased to announce the release of version 0.7 of the bitcoinj Java library for working with Bitcoin. Bitcoinj forms the foundation of MultiBit, Bitcoin Wallet for Android, SatoshiDice and more. To get bitcoinj 0.7, check out our source from git and then run *git reset --hard a9bd8631b904*. This will place you on the 0.7 release in a secure manner. This paragraph was written on Tuesday 19th February 2013 and is signed with the following key, which will be used in all release announcements in future: 16vSNFP5Acsa6RBbjEA7QYCCRDRGXRFH4m. Signature for the last paragraph: IMvY1FsQobjU2t83ztQL3CTA+V+7WWKBFwMC+UWKCOMyTKA+73iSsFnCHdbFjAOEFMQH/NvJMTgGeVCSV/F9hfs= If you want to, you can check that the original announcement mail sent to bitco...@googlegroups.com is correctly signed with the google.com DKIM key, to establish a full chain of trust. *Release notes* - Thanks to Matt Corallo, we now support a* fully verifying mode* in addition to simplified verification. This is a tremendous amount of work that wouldn't have happened without Matt! Right now, we strongly discourage anyone from using it for mining (which is not supported out of the box anyway). Use it in a production environment only if you know what you're doing and are willing to risk losing money. If you do use it, let us know so we can contact you when problems are discovered. Read the documentation carefully before you begin. - Also thanks to Matt, *Bloom filtering* is now implemented and activated by default. When bitcoinj connects to a peer that supports Bloom filtering, only transactions relevant to the wallet will be downloaded which makes bandwidth usage scale with the size of your wallet, not global system activity. A configurable false positive ratio allows you to trade off bandwidth vs privacy. App developers don't need to do anything to take advantage of this, it is enabled automatically. - PeerGroup now pings its peers and calculates moving averages of the ping times. Ping time, versions and block heights are taken into account when selecting the peer to download the chain from. - You can now customize which outputs the wallet uses to create spends. The new default coin selector object allows you to spend unconfirmed change as long as it's been seen propagating across the network, addressing a common end-user pain point in wallet apps. - Optimized networking code for faster startup. - A new PeerMonitor example app shows how to put properties of connected peers into a GUI. - The Wallet is now decoupled from the BlockChain using the new BlockChainListener interface. This will simplify the development of some apps that want to process transactions but not maintain an actual wallet. - The dependencies of broadcast transactions are now downloaded and risk analyzed. At the moment they are only being checked for having a timelock. In future we may also analyze tree depth. The goal is to make certain kinds of protocol abuse harder. Wallets will reject timelocked transactions by default, this can be overridden via a property. - You can now create timelocked transactions with WalletTool?http://code.google.com/p/bitcoinj/w/edit/WalletTool if you want to. - Compressed public keys are now used by default. - Support testnet3 - Support bitcoin-qt compatible message signing and verification. - ECDSA key recovery is now implemented and allows you to obtain the public key from an extended signature. If the signature is not extended then there are multiple key possibilities returned. - Many bugfixes and minor improvements API changes: - ECKey.sign() now takes a Sha256Hash as an argument and returns an ECDSASignature object in response. To get DER encoded signatures, use the encodeToDER() method of ECDSASignature. - ECKey.publicKeyFromPrivate now takes an additional compressed parameter. - PeerGroup.start()/PeerGroup.shutDown() now run asynchronously and return futures you can use to wait for them. You cannot restart a PeerGroup once it has been shut down any more. *Credits* * * Thanks to Matt Corallo (a.k.a. BlueMatt) for his huge contributions to this release. As always, thanks to Andreas Schildbach for his thorough testing, ideas and high volume of quality bug reports. Also thanks to Jim Burton for the same reasons. Finally thanks to Ben (piuk) of blockchain.info for funding the ECDSA key recovery feature. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Draft BIP for Bloom filtering
I paid the new anti-spam deposit and updated the BIP 37 page to the latest version of the protocol, then marked it as accepted. High fives all round, but especially to Matt for doing the heavy lifting on this feature. On Wed, Feb 6, 2013 at 5:45 PM, Gregory Maxwell gmaxw...@gmail.com wrote: On Wed, Feb 6, 2013 at 8:33 AM, Mike Hearn m...@plan99.net wrote: Can somebody please unlock the BIP wiki page? I don't know why it was locked but it's stale. I asked for permissions to unlock it but haven't heard back— will prod. -- Everyone hates slow websites. So do we. Make your web apps faster with AppDynamics Download AppDynamics Lite for free today: http://p.sf.net/sfu/appdyn_d2d_feb___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Large-blocks and censorship
To summarize your post - it's another go at arguing for strongly limited block sizes, this time on the grounds that large blocks make it easier for $AUTHORITY to censor transactions? Is that right? -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Large-blocks and censorship
As an aside, there's a paper coming out in perhaps a few months that describes a new way to provide Chaum-style privacy integrated with Bitcoin, but without the use of blinding and without any need for banks. It's quite smart, I was reviewing the paper this week. Unfortunately the technique is too slow and too complicated to actually integrate, but you'd probably get a kick out of it. It's based on zero knowledge proofs. You can talk to Ian Miers if you like, perhaps he'll send you a copy for review. Back on topic. This idea is not new. I proposed the idea of regulating miners to freeze certain outputs two years ago: https://bitcointalk.org/index.php?action=printpage;topic=5979.0 I concluded that it was not a real risk because both mining and transactions can be done anonymously. Your argument rests on the assumption that you can't mine large blocks anonymously because Tor doesn't scale. Even if we go along with the idea that Tor is the only way to escape regulation (it's not), you should maybe take up its inability to move data sufficiently fast with the developers. Given that they routinely push two gigabits/second today, with an entirely volunteer run network, I think they'll be surprised to learn that their project is doomed to never be usable by miners. -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Blocking uneconomical UTXO creation
Why does demurrage even still come up? The base rules of Bitcoin will not be changing in such a fundamental way. With regards to trying to minimize the size of the UTXO set, this again feels like a solution in search of a problem. Even with SD abusing micropayments as messages, it's only a few hundred megabytes today. That fits in RAM, let alone disk. If one day people do get concerned about the working set size, miners can independently set their own policies for what they confirm, for instance maybe they just bump the priority of any transaction that has fewer outputs than inputs. An IsStandard() rule now that tries to ban micropayments will just risk hurting interesting applications for no real benefit. It's like trying to anticipate and fix problems we might face in 2020. There are lots of less invasive changes for improving scalability, like making transaction validation multi-threaded in every case, transmitting merkle blocks instead of full blocks, moving blocking disk IO off the main loop so nodes don't go unresponsive when somebody downloads the chain from them, and finishing the payment protocol work so there's less incentive to replicate the SD transactions as messages design. -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Blocking uneconomical UTXO creation
This would be bloating UTXO data at a speed of 52 GB/year. That's a very big memory leak. And this is just the unspendable outputs. Firstly, the UTXO set is a LevelDB, it's not stored in memory. Outputs that never get spent are not in the working set by definition, after a while they just end up in the bottom levels and hardly ever get accessed. If need be we can always help LevelDB out a bit by moving outputs that we suspect are unlikely to get spent into a separate database, but I doubt it's needed. Secondly, if an output can be proven unspendable it can be pruned immediately. We already reached consensus on adding some standard template using OP_RETURN that results in insta-pruning. So people who want to create unspendable outputs can do so with the only side-effect being long term chain storage. It would be effectively free to pruning nodes. So the issue is not really with unspendable outputs but with low-value spendable outputs. Wallets with lots of tiny outputs end up generating large transactions that take a long time to verify, in situations where the network redlines those transactions would end up at the bottom of the priority queue and might take longer to confirm. So wallet apps already have incentives to try and find a good balance in output sizes and defragment themselves if their average output gets too low in value, eg, by send-to-self transactions at night. -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Warning: many 0.7 nodes break on large number of tx/block; fork risk
Just so we're all on the same page, can someone confirm my understanding - are any of the following statements untrue? BDB ran out of locks. However, only on some 0.7 nodes. Others, perhaps nodes using different flags, managed it. We have processed 1mb sized blocks on the testnet. Therefore it isn't presently clear why that particular block caused lock exhaustion when other larger blocks have not. The reason for increasing the soft limit is still present (we have run out of space). Therefore transactions are likely to start stacking up in the memory pool again very shortly, as they did last week. There are no bounds on the memory pool size. If too many transactions enter the pool then nodes will start to die with OOM failures. Therefore it is possible that we have a very limited amount of time until nodes start dying en-masse. Even if nodes do not die, users have no way to find out what the current highest fees/bids for block space are, nor any way to change the fee on sent transactions. Therefore Bitcoin will shortly start to break for the majority of users who don't have a deep understanding of the system. If all the above statements are true, we appear to be painted into a corner - can't roll forward and can't roll back, with very limited time to come up with a solution. I see only a small number of alternatives: 1) Start aggressively trying to block or down-prioritize SatoshiDice transactions at the network level, to buy time and try to avoid mempool exhaustion. I don't know a good way to do this, although it appears that virtually all their traffic is actually coming via blockchain.infos My Wallet service. During their last outage block sizes seemed to drop to around 50kb. Alternatively, ask SD to temporarily suspend their service (this seems like a long shot). 2) Perform a crash hard fork as soon as possible, probably with no changes in it except a new block size limit. Question - try to lift the 1mb limit at the same time, or not? On Tue, Mar 12, 2013 at 2:01 AM, Pieter Wuille pieter.wui...@gmail.com wrote: Hello again, block 015c50b165fcdd33556f8b44800c5298943ac70b112df480c023 (height=225430) seems indeed to have cause pre-0.8 and 0.8 nodes to fork (at least mostly). Both chains are being mined on - the 0.8 one growing faster. After some emergency discussion on #bitcoin-dev, it seems best to try to get the majority mining power back on the old chain, that is, the one which 0.7 accepts (with 01c108384350f74090433e7fcf79a606b8e797f065b130575932 at height 225430). That is the only chain every client out there will accept. BTC Guild is switching to 0.7, so majority should abandon the 0.8 chain soon. Short advice: if you're a miner, please revert to 0.7 until we at least understand exactly what causes this. If you're a merchant, and are on 0.8, stop processing transactions until both sides have switched to the same chain again. We'll see how to proceed afterwards. -- Pieter On Tue, Mar 12, 2013 at 1:18 AM, Pieter Wuille pieter.wui...@gmail.com wrote: Hello everyone, Í've just seen many reports of 0.7 nodes getting stuck around block 225430, due to running out of lock entries in the BDB database. 0.8 nodes do not seem to have a problem. In any case, if you do not have this block: 2013-03-12 00:00:10 SetBestChain: new best=015aab28064a4c521d6a5325ff6e251e8ca2edfdfe6cb5bf832c height=225439 work=853779625563004076992 tx=14269257 date=2013-03-11 23:49:08 you're likely stuck. Check debug.log and db.log (look for 'Lock table is out of available lock entries'). If this is a widespread problem, it is an emergency. We risk having (several) forked chains with smaller blocks, which are accepted by 0.7 nodes. Can people contact pool operators to see which fork they are on? Blockexplorer and blockchain.info seem to be stuck as well. Immediate solution is upgrading to 0.8, or manually setting the number of lock objects higher in your database. I'll follow up with more concrete instructions. If you're unsure, please stop processing transactions. -- Pieter -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the
Re: [Bitcoin-development] Warning: many 0.7 nodes break on large number of tx/block; fork risk
However, most nodes are not running in such a loop today. Probably almost no nodes are. I suppose you could consider mass node death to be more benign than a hard fork, but both are pretty damn serious and warrant immediate action. Otherwise we're going to see the number of nodes drop sharply over the coming days as unattended nodes die and then don't get restarted. -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Warning: many 0.7 nodes break on large number of tx/block; fork risk
We just saw a hard-fork happen because we ran into previously unknown scaling issues with the current codebase. Technically, it with the previous codebase ;) -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Warning: many 0.7 nodes break on large number of tx/block; fork risk
I'm not even sure I'd say the upgrade went wrong. The problem if anything is the upgrade didn't happen fast enough. If we had run out of block space a few months from now, or if miners/merchants/exchanges had upgraded faster, it'd have made more sense to just roll forward and tolerate the loss of the older clients. This really reinforces the importance of keeping nodes up to date. On Tue, Mar 12, 2013 at 12:44 PM, Pieter Wuille pieter.wui...@gmail.com wrote: On Tue, Mar 12, 2013 at 11:13:09AM +0100, Michael Gronager wrote: Yes, 0.7 (yes 0.7!) was not sufficiently tested it had an undocumented and unknown criteria for block rejection, hence the upgrade went wrong. We're using 0.7 as a short moniker for all clients, but this was a limitation that all BDB-based bitcoins ever had. The bug is simply a limit in the number of lock objects that was reached. It's ironic that 0.8 was supposed to solve all problems we had due to BDB (except the wallet...), but now it seems it's still coming back to haunt us. I really hated telling miners to go back to 0.7, given all efforts to make 0.8 signficantly more tolerable... More space in the block is needed indeed, but the real problem you are describing is actually not missing space in the block, but proper handling of mem-pool transactions. They should be pruned on two criteria: 1. if they gets to old 24hr 2. if the client is running out of space, then the oldest should probably be pruned clients are anyway keeping, and re-relaying, their own transactions and hence it would mean only little, and only little for clients. Dropping free / old transaction is a much a better behavior than dying... Even a scheme where the client dropped all or random mempool txes would be a tolerable way of handling things (dropping all is similar to a restart, except for no user intervention). Right now, mempools are relatively small in memory usage, but with small block sizes, it indeed risks going up. In 0.8, conflicting (=double spending) transactions in the chain cause clearing the mempool of conflicts, so at least the mempool is bounded by the size of the UTXO subset being spent. Dropping transactions from the memory pool when they run out of space seems a correct solution. I'm less convinced about a deterministic time-based rule, as that creates a double spending incentive at that time, and a counter incentive to spam the network with your risking-to-be-cleared transaction as well. Regarding the block space, we've seen the pct% of one single block chain space consumer grow simultaneously with the introduction of larger blocks, so I'm not actually convinced there is right now a big need for larger blocks (note: right now). The competition for block chain space is mostly an issue for client software which doesn't deal correctly with non-confirming transactions, and misleading users. It's mostly a usability problem now, but increasing block sizes isn't guaranteed to fix that; it may just make more space for spam. However, the presence of this bug, and the fact that a full solution is available (0.8), probably helps achieving consensus fixing it (=a hardfork) is needed, and we should take advantage of that. But please, let's not rush things... -- Piter -- Symantec Endpoint Protection 12 positioned as A LEADER in The Forrester Wave(TM): Endpoint Security, Q1 2013 and remains a good choice in the endpoint security space. For insight on selecting the right partner to tackle endpoint security challenges, access the full report. http://p.sf.net/sfu/symantec-dev2dev ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] bitcoin pull requests
My general hope/vague plan for bitcoinj based wallets is to get them all on to automatic updates with threshold signatures. Combined with regular audits of the initial downloads for new users, that should give a pretty safe result that is immune to a developer going rogue. On Wed, Apr 3, 2013 at 7:12 PM, grarpamp grarp...@gmail.com wrote: Users will have available multisig addresses which require transactions to be signed off by a wallet HSM. (E.g. a keyfob Hardware is a good thing. But only if you do the crypto in the hardware and trust the hardware and its attack models ;) For instance, the fingerprint readers you see everywhere... many of them just present the raw fingerprint scan to the host (and host software), instead of hashing the fingerprint internally and using that as primitive in crypto exchanges with the host. They cheaped out and/or didn't think. So oops, there went both your security (host replay) and your personal privacy (biometrics), outside of your control. All with no protection against physical fingerprint lifting. This doesn't remove the need to improve repository integrity. ... but repository integrity is a general problem that is applicable to many things (after all, what does it matter if you can't compromise Bitcoin if you can compromise boost, openssl, or gcc?) Yes, that case would matter zero to the end product. However having a strong repo permits better auditing of the BTC codebase. That's a good thing, and eliminates the need to talk chicken and egg. It's probably best that Bitcoin specalists stay focused on Bitcoin security measures, and other people interested in repository security come and help out improving it. An obvious area of improvement might be oddity detection and alerting: It's weird that I can rewrite history on github, so long as I do it quickly, without anyone noticing. If no one is verifying the repo, sure, even entire repos could be swapped out for seemingly identical ones. Many repos do not have any strong internal verification structures at all, and they run on filesystems that accept bitrot. Take a look at some OS's... OpenBSD and FreeBSD, supposedly the more secure ones out there... both use legacy repos on FFS. Seems rather ironic in the lol department. Thankfully some people out there are finally getting a clue on these issues, making and learning the tools, converting and migrating things, working on top down signed build and distribution chain, etc... so maybe in ten years the opensource world will be much farther ahead. Or at least have a strong audit trail. -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] bitcoin pull requests
By the way, I have a download of the Bitcoin-Qt client and signature verification running in a cron job. On Thu, Apr 4, 2013 at 10:11 AM, Mike Hearn m...@plan99.net wrote: My general hope/vague plan for bitcoinj based wallets is to get them all on to automatic updates with threshold signatures. Combined with regular audits of the initial downloads for new users, that should give a pretty safe result that is immune to a developer going rogue. On Wed, Apr 3, 2013 at 7:12 PM, grarpamp grarp...@gmail.com wrote: Users will have available multisig addresses which require transactions to be signed off by a wallet HSM. (E.g. a keyfob Hardware is a good thing. But only if you do the crypto in the hardware and trust the hardware and its attack models ;) For instance, the fingerprint readers you see everywhere... many of them just present the raw fingerprint scan to the host (and host software), instead of hashing the fingerprint internally and using that as primitive in crypto exchanges with the host. They cheaped out and/or didn't think. So oops, there went both your security (host replay) and your personal privacy (biometrics), outside of your control. All with no protection against physical fingerprint lifting. This doesn't remove the need to improve repository integrity. ... but repository integrity is a general problem that is applicable to many things (after all, what does it matter if you can't compromise Bitcoin if you can compromise boost, openssl, or gcc?) Yes, that case would matter zero to the end product. However having a strong repo permits better auditing of the BTC codebase. That's a good thing, and eliminates the need to talk chicken and egg. It's probably best that Bitcoin specalists stay focused on Bitcoin security measures, and other people interested in repository security come and help out improving it. An obvious area of improvement might be oddity detection and alerting: It's weird that I can rewrite history on github, so long as I do it quickly, without anyone noticing. If no one is verifying the repo, sure, even entire repos could be swapped out for seemingly identical ones. Many repos do not have any strong internal verification structures at all, and they run on filesystems that accept bitrot. Take a look at some OS's... OpenBSD and FreeBSD, supposedly the more secure ones out there... both use legacy repos on FFS. Seems rather ironic in the lol department. Thankfully some people out there are finally getting a clue on these issues, making and learning the tools, converting and migrating things, working on top down signed build and distribution chain, etc... so maybe in ten years the opensource world will be much farther ahead. Or at least have a strong audit trail. -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] A mining pool at 46%
51% isn't a magic number - it's possible to do double spends against confirmed transactions before that. If Michael wanted to do so, with the current setup he could, and that's obviously rather different to how Satoshi envisioned mining working. However, you're somewhat right in the sense that it's a self-defeating attack. If the pool owner went bad, he could pull it off once, but the act of doing so would leave a permanent record and many of the people mining on his pool would leave. As he doesn't own the actual mining hardware, he then wouldn't be able to do it again. There are also other mining protocols that allow people to pool together, without p2pool and without the pool operator being able to centrally pick which transactions go into the block. However I'm not sure they're widely deployed at the moment. It'd be better if people didn't cluster around big mining pools, but I think p2pool still has a lot of problems dealing with FPGA/ASIC hardware and it hasn't been growing for a long time. On Fri, Apr 5, 2013 at 11:30 AM, Melvin Carvalho melvincarva...@gmail.comwrote: There was some chat on IRC about a mining pool reaching 46% http://blockchain.info/pools What's the risk of a 51% attack. I suggested that the pool itself is decentralized so you could not launch one On IRC people were saying that the pool owner gets to choose what goes in the block Surely with random non colliding nonces, it would be almost impossible to coordinate a 51% even by the owner Someone came back and said that creating random numbers on a GPU is hard. But what about just creating ONE random number and incrementing from there ... It would be great to know if this is a threat or a non issue -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] Integration testing for BitCoin
In bitcoinj we desperately need integration tests to exercise the wallet code, and I think if it was done well the tests would be applicable to bitcoind as well. There have been a series of bugs in bitcoinj that boiled down to the unit tests were not realistic enough, either because they stopped simulating too early or they weren't combining multiple different things together in the same ways as happens on the real network. Sometimes timing was an issue too. Examples of what I mean - ensure that re-orgs are handled correctly and update the wallet properly in every case, etc. Something else that would be really useful, a standalone tool that stress-tests the system. If we had a tool that randomly generated chains of transactions we might have caught the bdb lock limit bug earlier. You could write such a tool using bitcoinj easily, or the raw transaction APIs on bitcoind. On Fri, Apr 5, 2013 at 8:29 PM, Adam Ritter arit...@gmail.com wrote: Thanks guys, it sounds great. Testing the JSON-RPC is/was not the main goal, just an interface for testing. I didn't know that the bitcoinj implementation is getting close to a full implementation..it sounds interesting, as it's much easier to understand and work with. I'll look at the test cases. Thanks very much, Adam On Fri, Apr 5, 2013 at 12:42 PM, Gregory Maxwell gmaxw...@gmail.com wrote: On Fri, Apr 5, 2013 at 10:24 AM, Adam Ritter arit...@gmail.com wrote: Hey guys, I just bought some BitCoins after being lazy to do it for the last few years, but also looked at the client code and the messages that are going on this mailing list. I saw that there are quite some unit tests, but I didn't find integration test for BitCoin, and I believe that it's quite important for the future of BitCoin (making the current code more stable, testing attack scenarios, refactoring and extending code). [...] Tests that simulate multiple bitcoin users and can verify that the whole network of bitcoin clients work together to achieve the goals of Bitcoin. Also maybe [System testing](http://en.wikipedia.org/wiki/System_testing) would be a better name for the tests, but I'm not sure. I prefer to call them system tests. We use a system called blocktester that Matt Corallo wrote, https://code.google.com/r/bluemattme-bitcoinj/source/browse/core/src/test/java/com/google/bitcoin/core/FullBlockTestGenerator.java?name=fullverifr=874c5904b12d1fcec5b556429cf208f63cd4e1bc It's based on BitcoinJ and works by simulating a peer against a slightly instrumented copy of Bitcoin(d/-qt) (modified to avoid computationally expensive mining). The tests simulates many complicated network scenarios and tests the boundaries of many (hopefully all) the particular rules of the blockchain validation protocol. We can use these tests to compare different versions of the reference software to each other and to bitcoinj (or other full node implementations) as well as comparing them to our abstract understanding of what we believe the rules of the protocol to be. These tests are run as part of the automated tests on every proposed patch to the reference software. Via a robot called pulltester which comments on github requests and produces logs like this: http://jenkins.bluematt.me/pull-tester/92a129980fb9b506da6c7f876aa8adb405c88e17/ . Pulltester also performs automatic code coverage measurements. Additionally, we run a public secondary test bitcoin network called 'testnet', which can be accessed by anyone by starting the reference software with testnet=1. Testnet operates the same as the production network except it allows mining low difficulty blocks to prevent it going for long times without blocks, and some of the protective relaying rules against non standard transaction types are disabled. Most of this testing work has been centered around validating the blockchain behavior because thats what has serious systemic risk. Measuring the json rpc behavior is strictly less interesting, though interesting too. -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal
Re: [Bitcoin-development] Who is creating non-DER signatures?
It'd help to know how the signatures are invalid. On Sun, Apr 7, 2013 at 5:34 PM, Pieter Wuille pieter.wui...@gmail.comwrote: (cross-post from bitcointalk.org) Hello all, as some may know, Bitcoin uses DER-encoded signatures in its transactions. However, OpenSSL (which is used to verify them) accepts more than just the strict DER specification (it allows negative numbers, extra zero padding, extra bytes at the end, and perhaps more). As we don't like the de-facto specification of the Bitcoin block validity rules to depend on OpenSSL, we're trying to introduce a rule to make such non-standard signatures invalid. Obviously, that can't be done as long as any significant amount of clients on the network is creating these. I've monitored all transactions the past weeks (1.4M transactions), and it seems 9641 of them contain at least one non-standard signature. See https://bitcointalk.org/index.php?topic=169620.0 for a list of the top addresses that had coins used as inputs in such transactions. If you recognize any of these addresses, or have an idea of who owns them or what software they are using, please let me know. Thanks! -- Pieter -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html ___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development -- Minimize network downtime and maximize team effectiveness. Reduce network management and security costs.Learn how to hire the most talented Cisco Certified professionals. Visit the Employer Resources Portal http://www.cisco.com/web/learning/employer_resources/index.html___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] On-going data spam
OK, as the start of that conversation is now on the list, I might as well post the other thoughts we had. Or at least that I had :) It's tempting to see this kind of abuse through the lens of fees, because we only have a few hammers and so everything looks like a kind of nail. The problem is the moment you try to define abuse economically you end up excluding legitimate and beneficial uses as well. Maybe Peters patch for uneconomical outputs is different because of how it works. But mostly it's true. In this case, fees would never work - Peter said the guy who uploaded Wikileaks paid something like $500 to do it. I guess by now it's more like $600-$700. It's hard for regular end users to compete with that kind of wild-eyed dedication to the cause. The root problem here is people believe the block chain is a data structure that will live forever and be served by everyone for free, in perpetuity, and is thus the perfect place for uncensorable stuff. That's a reasonable assumption given how Bitcoin works today. But there's no reason it will be true in the long run (I know this can be an unpopular viewpoint). Firstly, legal issues - I think it's very unlikely any sane court would care about illegal stuff in the block chain given you need special tools to extract it (mens rea). Besides, I guess most end users will end up on SPV clients as they mature. So these users already don't have a copy of the entire block chain. I don't worry too much about this. Secondly, the need to host blocks forever. In future, many (most?) full nodes will be pruning, and won't actually store old blocks at all. They'll just have the utxo database, some undo blocks and some number of old blocks for serving, probably whatever fits in the amount of disk space the user is willing to allocate. But very old blocks will have been deleted. This leads to the question of what incentives people have to not prune. The obvious incentive is money - charge for access to older parts of the chain. The fewer people that host it, the more you can charge. In the worst case scenario where, you know, only 10 different organizations store a copy of the chain, it might mean that bootstrapping a new node in a trust-less manner is expensive. But I really doubt it'd ever get so few. Serving large static datasets just isn't that expensive. Also, you don't actually need to replay from the genesis block to bring up a new code, you can copy the UTXO database from somewhere else. By comparing the databases of lots of different nodes together, the chances of you being in a matrix-like sybil world can be reduced to beyond reasonable doubt. Maybe nodes would charge for copies of their database too, but ideally there are lots of nodes and so the charge for that should be so close to zero as makes no odds - you can trivially undercut someone by buying access to the dataset and then reselling it for a bit less, so the price should converge on the actual cost of providing the service. Which will be very cheap. There was one last thought I had, which is that if there's a shorter team need to discourage this kind of thing we can use a network/bandwith related hack by changing the protocol. Nodes can serve up blocks encrypted under a random key. You only get the key when you finish the download. A blacklist can apply to Bloom filtering such that transactions which are known to be abusive require you to fully download the block rather than select the transactions with a filter. This means that people can still access the data in the chain, but the older it gets the slower and more bandwidth intensive it becomes. Stuffing Wikileaks into the chain sounds good when a 20 line Python script can extract it instantly. If someone who wants the files has to download gigabytes of padding around it first, suddenly hosting it on a Tor hidden service becomes more attractive. -- Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
Re: [Bitcoin-development] On-going data spam
However, there should be some metrics and heuristics that take care of this problem. Notably the dev consensus (sans you, Mike :)) seems to be that uneconomical outputs should be made non-standard. I think that patch is ok as it doesn't really have any fixed concept of what is uneconomical. But I haven't thought about it much. As Gavin says, there's an obvious backwards compatibility problem there. It should probably wait until the payment protocol work is done, so the major user of micropayments-as-messages can migrate off them. -- Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
[Bitcoin-development] bitcoinj 0.8
I'm happy to announce the release of bitcoinj 0.8, a Java library for writing Bitcoin applications. Both simplified and full verification are supported. BitcoinJ has been used to create everything from end-user wallet apps to network crawlers to SatoshiDice. To get bitcoinj 0.8, check out our source from git and then run *git fetch --all; git checkout **cbbb1a2bf4d1*. This will place you on the 0.8 release in a secure manner. This message was written on Tuesday 9th April 2013 and is signed with the following key, which will be used in all release announcements in future: 16vSNFP5Acsa6RBbjEA7QYCCRDRGXRFH4m. Signature for previous paragraph: H8itldUGHHt8jXmFwRX/gASXrhG1a/k0VG0vwFMjQCAWDpxgA17ODfSPFNgAOPDnPmT1gLLUlHsEqwXHBoj+JMU= You can also verify the google.com DKIM signature on the official announcementhttps://groups.google.com/forum/?fromgroups=#!topic/bitcoinj-announce/IB7dlc_g9sU . I'm especially happy about this release because for the first time, we have an SPV implementation that is competitive performance-wise with more centralised solutions that rely on custom servers. Wallets based on bitcoinj 0.8 complete first time setup for new users in only a few seconds, eliminating the last source of significant delays. Every operation except key import now completes more or less immediately. *New in this release* - Thanks to Jim Burton, encryption of private keys in the wallet is now supported. Keys are encrypted using an AES key derived using scrypt. - A new SPVBlockStore provides dramatically better performance and bounded disk usage by storing block headers in an mmapped ring buffer. This makes syncing headers for new chains/wallets network limited instead of disk io limited. - A new tool is provided to create lists of block header checkpoints that can then be used to initialize a new block store. This allows most headers to not be downloaded when initializing a new chain/wallet, making first-run of new wallets much faster. - Bloom-filtering capable nodes are now queried for transactions at startup, meaning you can receive payments that weren't confirmed yet even if your wallet wasn't running at the time. - Many static analysis warnings have been cleared. - All event listeners except transaction confidence listeners now run unlocked and core objects have been converted to use cycle detecting locks. Multiple lock inversions were fixed. - DNS seeds are now supported for testnet. - PeerEventListener now lets you catch and process exceptions thrown during peer message processing. This is useful for reporting crashes that don't take out your entire app, but just result in disconnection of a peer. - Matt Corallo's bitcoind comparison tool was merged in. It runs a large set of regression tests that compares the behaviour of bitcoinj in full verification mode against bitcoind. - The vast bulk of the changes in this release are bug fixes, optimizations and minor API improvements. They are too numerous to list here, please refer to the commit logs for details. *API changes:* - Event listeners were previously locked before being called, and the object being listened to was also locked. This is no longer true - your event listeners must be thread safe and the objects that triggered the event may be changing in parallel. - IrcDiscovery is now deprecated, as LFnet has gone offline and DNS seeding can be used for both test and production networks. The code is still there in case you want to use IRC bootstrapping for a private experimental network. - BoundedOverheadBlockStore is now deprecated. It was replaced by SPVBlockStore. The file format has changed, so BOBS will stick around for a while so users can be upgraded. - The Derby based block store has been deleted. It only supported SPV mode and wasn't used much. - The static NetworkParameters methods now vend singleton objects. - WalletEventListener.onCoinsSent is no longer run when a transaction sends to self but the balance doesn't change. *Known issues:* - Transaction confidence listeners are still run with the wallet lock held, which means it's possible to trigger unexpected lock inversions by doing certain things inside them. Also, confidence listeners sometimes run in places where the wallet code is not fully re-entrant, meaning that modifying the wallet whilst inside a confidence listener may cause problems. A simple fix is to run your listener code in a separate thread. A future release will fix this by ensuring that listeners only ever run at the end of wallet mutating operations and with the wallet unlocked. Core objects will also switch to using non-reentrant locks so unexpected reentrancy deadlocks early and reliably. - If multiple peers disconnect simultaneously it's possible for the system to deadlock due to Netty allowing uncontrolled reentrancy when sending outbound
[Bitcoin-development] Anti DoS for tx replacement
This was previously discussed on the forums a bunch of times, but in particular here: https://bitcointalk.org/index.php?topic=91732.0 BTW, I don't think all this has to be solved to re-activate replacement on testnet. It's useful for people to be able to develop apps that use this feature, indeed, it helps build the case for re-activating it on the main network after the necessary work is done. Otherwise there'll inevitably be people who say why re-activate something even though we think it's safe when there are no use cases for it. Letting people develop and deploy interesting prototypes in parallel solves that catch-22. --- Refresher: since the first release Bitcoin has had the ability to replace transactions that sit in the memory pool if the transaction is non-final, the inputs are the same and the replacement is newer than the replacee. Being non-final means not having reached the nLockTime threshold, and having at least one input with a sequence number UINT_MAX. Around the time of the bugs in various opcodes being found, Satoshi disabled the feature because nothing was using it - it was something he'd planned for the future, it had no utility in the Bitcoin of 2010. The purpose of tx replacement is to implement high frequency trading, according to material Satoshi sent me when I asked him what it was all for (I wanted to know why sequence numbers were a property of inputs not the transaction). It's very important to understand that this does *NOT* mean high-frequency from the networks perspective. In normal operation, tx replacement is not actually intended to be used at all. Sort of like double-spending protection, it's a code path that's only meant to be triggered when one or the other party is maliciously trying to roll back a negotiated contract. And when a party is trying to do that, you don't need lots of replacements. A single replacement is enough. To see why this is the case please review the micropayment channel protocol here: https://en.bitcoin.it/wiki/Contracts#Example_7:_Rapidly-adjusted_.28micro.29payments_to_a_pre-determined_party This isn't the only use of contractual HFT in Bitcoin, it's a deliberately simplified and stripped down example (eg, that only uses two parties). The example Satoshi gave me was more abstract and actually had N parties in it - it left me puzzled for a while and struggling to see practical application. The billing for a metered resource use case is easier to understand. Now the obvious problem is that even though the feature is only intended to be used occasionally or never, nothing in the existing code stops you using it as fast as possible and exhausting nodes CPU time and bandwidth. What's more, solving this is not as easy as it looks. Most proposed solutions will not work: 1) Requiring higher fees for each replacement means that a channel/contract has to be torn down and rebuilt much, much faster than before because otherwise the amount of money lost to fees quickly becomes the entire size of the channel (or you can't update it very often). Remember, you'd have to increase the fee for each replacement regardless of whether it's presented to the network or not. As the whole point of the setup is to avoid putting lots of transactions on the network, anything that pushes you back towards doing that undermines the entire utility of the system. 2) Refusing to update the transaction after certain thresholds are reached, having cooldown periods, etc also won't work because the replacement mechanism is there to protect each counter-party in the HFT contract. Simply converting a DoS on the network to a DoS on the participants means one malicious party can break the mechanism that protects all the others by broadcasting the initial set of updates all at once and deliberately tripping the thresholds. OK, let's take a step back. What is the purpose of abusing this feature? It's to mount a denial of service attack - either against the entire Bitcoin network, or against the other participants in the contract. But someone, somewhere has to be denied service, otherwise the attack is pointless. We can exploit this fact by realising that typically anti-DoS is a prioritisation problem. It doesn't usually matter if you serve some abusive traffic if all legitimate traffic gets served first because it removes the denial of service from the attack, and usually there are lots of ways to attack someone with methods that don't work - real world experience indicates that people don't pointlessly mount attacks over and over again if there's nothing to be gained by doing so. So we can do the following - multi-thread verification of transactions that are trying to enter the memory pool, and order them such that high priority transactions are verified first, low priority next, and then replacements of transactions sorted by age of last replacement. Same thing for relaying - faced with getdatas, service the new transactions first, replacements with whatever is left
Re: [Bitcoin-development] Anti DoS for tx replacement
Or are you talking about some sort of new decentralized high frequency trading system that is self-matching and self-clearing? (I'd be very interested in hearing more if this is the case). I'm using the term high frequency trading because Satoshi did. Like the way he used the word contract it is perhaps a bit misleading, but we lack anything better to describe this new concept. Today HFT typically means companies that submits tons of micro-trades to centralised asset exchanges to try and exploit statistically expected correlations. HFT using tx replacement has nothing to do this with - it is instead a way that N parties can negotiate amongst themselves as fast as they can compute and verify signatures. Here is how Satoshi explained it to me, in his words: An unrecorded open transaction can keep being replaced until nLockTime. It may contain payments by multiple parties. Each input owner signs their input. For a new version to be written, each must sign a higher sequence number (see IsNewerThan). By signing, an input owner says I agree to put my money in, if everyone puts their money in and the outputs are this. There are other options in SignatureHash such as SIGHASH_SINGLE which means I agree, as long as this one output (i.e. mine) is what I want, I don't care what you do with the other outputs.. If that's written with a high nSequenceNumber, the party can bow out of the negotiation except for that one stipulation, or sign SIGHASH_NONE and bow out completely. The parties could create a pre-agreed default option by creating a higher nSequenceNumber tx using OP_CHECKMULTISIG that requires a subset of parties to sign to complete the signature. The parties hold this tx in reserve and if need be, pass it around until it has enough signatures. One use of nLockTime is high frequency trades between a set of parties. They can keep updating a tx by unanimous agreement. The party giving money would be the first to sign the next version. If one party stops agreeing to changes, then the last state will be recorded at nLockTime. If desired, a default transaction can be prepared after each version so n-1 parties can push an unresponsive party out. Intermediate transactions do not need to be broadcast. Only the final outcome gets recorded by the network. Just before nLockTime, the parties and a few witness nodes broadcast the highest sequence tx they saw. -- Precog is a next-generation analytics platform capable of advanced analytics on semi-structured data. The platform includes APIs for building apps and a phenomenal toolset for data science. Developers can use our toolset for easy data analysis visualization. Get a free account! http://www2.precog.com/precogplatform/slashdotnewsletter___ Bitcoin-development mailing list Bitcoin-development@lists.sourceforge.net https://lists.sourceforge.net/lists/listinfo/bitcoin-development
