Re: [bitcoin-dev] BIP Proposal: Wallet Labels Export Format

[NVK via bitcoin-dev]

Hello,

Thanks for this proposal.

I was trying to avoid adding more opinions / bike-shdding to the discussion and 
didn’t want to particularly pick at any of the threads.

But, I think I’d like to at least voice at how important having a human 
readable format for this is. CSV is indeed a format with many shortcomings, but 
so are most cross applications open formats that are human readable. I go 
through this every month for business and personal.

If contention is too high for CSV as cross application for import/export then 
maybe the route of two file formats maybe awkward but necessary. JSON maybe 
used as the choice for bitcoin clients for label syncing and CSV as the export 
for other purposes. I believe CSV is importable by most accounting software, 
old and new. JSON is not.

In regards to encryption, AES on 7z is a great, wide os native support.

Best,

NVK

> On Aug 24, 2022, at 05:46, Craig Raw via bitcoin-dev 
>  wrote:
> 
> Hi all,
> 
> I would like to propose a BIP that specifies a format for the export and 
> import of labels from a wallet. While transferring access to funds across 
> wallet applications has been made simple through standards such as BIP39, 
> wallet labels remain siloed and difficult to extract despite their value, 
> particularly in a privacy context.
> 
> The proposed format is a simple two column CSV file, with the reference to a 
> transaction, address, input or output in the first column, and the label in 
> the second column. CSV was chosen for its wide accessibility, especially to 
> users without specific technical expertise. Similarly, the CSV file may be 
> compressed using the ZIP format, and optionally encrypted using AES.
> 
> The full text of the BIP can be found at 
> https://github.com/craigraw/bips/blob/master/bip-wallet-labels.mediawiki and 
> also copied below.
> 
> Feedback is appreciated.
> 
> Thanks,
> Craig Raw
> 
> ---
> 
> 
>   BIP: wallet-labels
>   Layer: Applications
>   Title: Wallet Labels Export Format
>   Author: Craig Raw 
>   Comments-Summary: No comments yet.
>   Comments-URI: 
> https://github.com/bitcoin/bips/wiki/Comments:BIP-wallet-labels
>   Status: Draft
>   Type: Informational
>   Created: 2022-08-23
>   License: BSD-2-Clause
> 
> 
> ==Abstract==
> 
> This document specifies a format for the export of labels that may be 
> attached to the transactions, addresses, input and outputs in a wallet.
> 
> ==Copyright==
> 
> This BIP is licensed under the BSD 2-clause license.
> 
> ==Motivation==
> 
> The export and import of funds across different Bitcoin wallet applications 
> is well defined through standards such as BIP39, BIP32, BIP44 etc.
> These standards are well supported and allow users to move easily between 
> different wallets.
> There is, however, no defined standard to transfer any labels the user may 
> have applied to the transactions, addresses, inputs or outputs in their 
> wallet.
> The UTXO model that Bitcoin uses makes these labels particularly valuable as 
> they may indicate the source of funds, whether received externally or as a 
> result of change from a prior transaction.
> In both cases, care must be taken when spending to avoid undesirable leaks of 
> private information.
> Labels provide valuable guidance in this regard, and have even become 
> mandatory when spending in several Bitcoin wallets.
> Allowing users to export their labels in a standardized way ensures that they 
> do not experience lock-in to a particular wallet application.
> In addition, by using common formats, this BIP seeks to make manual or bulk 
> management of labels accessible to users without specific technical expertise.
> 
> ==Specification==
> 
> In order to make the import and export of labels as widely accessible as 
> possible, this BIP uses the comma separated values (CSV) format, which is 
> widely supported by consumer, business, and scientific applications.
> Although the technical specification of CSV in RFC4180 is not always 
> followed, the application of the format in this BIP is simple enough that 
> compatibility should not present a problem.
> Moreover, the simplicity and forgiving nature of CSV (over for example JSON) 
> lends itself well to bulk label editing using spreadsheet and text editing 
> tools. 
> 
> A CSV export of labels from a wallet must be a UTF-8 encoded text file, 
> containing one record per line, with records containing two fields delimited 
> by a comma.
> The fields may be quoted, but this is unnecessary, as the first comma in the 
> line will always be the delimiter.
> The first line in the file is a header, and should be ignored on import.
> Thereafter, each line represents a record that refers to a label applied in 
> the wallet.
> The order in which these records appear is not defined.
> 
> The first field in the record contains a reference to the transaction, 
> address, input or output in the wallet.
> This is specified as one of the following:
> * Transaction ID (txid)
> * Address
> * Input

Re: [bitcoin-dev] BIP Proposal: Wallet Labels Export Format

[Christopher Allen via bitcoin-dev]

On Mon, Aug 29, 2022 at 9:12 AM Ali Sherief via bitcoin-dev <
bitcoin-dev@lists.linuxfoundation.org> wrote:

> I am aware that business processes are mostly CSV file oriented


I disagree that business processes are mostly CSV.  Amateur processes
maybe, but professional accounting, no. Trying to do my business accounting
with CSV files from various exchanges is PITA.


> so you can make a statement akin to BIP174 in the Goal 2 BIP, that expects
> the medium of exchange to be in files ending in .csv. I wouldn't mind if
> you require .csv file extension in a BIP for Goal 2. But such a statement
> is not appropriate in the Goal 1 BIP which is only concerned with the
> wallet label format itself.


I too would like to see some separation of layers here, as there are other
possible output formats. Maybe expanding on another use case for this data
would help.

I've been working with @nochiel  on export
to a Plain-Text
Accounting  friendly format,
initially the beancount
python app : (our prototype is
current at /beancounter.py but it is being refactored into new repo).

Basically what the final tool will do is: given a descriptor, get any
transactions for that descriptor from a random explora via Tor (initially
ours and Blockstream's), and then get price information from a random
Spotbit price server via Tor (initially just ours, but seeking more hosts),
and export a beancount compatible file.

```

python app.py beancount
"wpkh(tpubD9hudZxy8Uj3453QrsEbr8KiyXTYC5ExHjJ5sNDVW7yKJ8wc7acKQcpdbvZX6dFerHK6MfVvs78VvGfotjN28yC4ij6nr4uSVhX2qorUV8V/0/*)"
Outputs: spotbit.beancount

2008-10-31 commodity BTC
  name: "Bitcoin"
  asset-class: "cryptocurrency"

2018-04-02 open Assets:BTC BTC
2018-04-02 open Liabilities:Cash:USDT USDT

2018-04-02 * "tb1qcrekknrspx28t9vl53ltsag5gqgqdj066ydf75" "Transaction
hash: 2a2f7f24761fa54cb6e559efea5678415d9cbbabc42e6a4e2ce463ee3c446230"
Assets:BTC 1. BTC { 6935.16 USDT }
Liabilities:Cash:USDT - 6935.16 USDT

2018-04-02 * "tb1q45whzx3emntntnpzjdx3gzj6z5cgxakkg7s3sa" "Transaction
hash: 387123efcaa707759a4af8159cb1309fae86b793d26b5fd8bba42637852dde89"
Assets:BTC - 0.36300616 BTC @  6935.16 USDT
Liabilities:Cash:USDT 2517.51 USDT

2018-04-02 * "tb1qgv5484m83e2mzz3n8tf4snvnwj5qgqgampnhvv" "Transaction
hash: 387123efcaa707759a4af8159cb1309fae86b793d26b5fd8bba42637852dde89"
Assets:BTC - 0.63699243 BTC @  6935.16 USDT
Liabilities:Cash:USDT 4417.64 USDT

2018-04-02 * "tb1q45whzx3emntntnpzjdx3gzj6z5cgxakkg7s3sa" "Transaction
hash: 387123efcaa707759a4af8159cb1309fae86b793d26b5fd8bba42637852dde89"
Assets:BTC 0.36300616 BTC { 6935.16 USDT }
Liabilities:Cash:USDT - 2517.51 USDT
```

I can then use the beancount cli app (or it's fava webapp) to easily add
other details to this file to do my bitcoin accounting (and any other
accounting I need). In particular, as beancount support lots, it solves a
problem for me with US taxes which is unrealized capital gain (I get 1 BTC
from donor at $20K, the price goes up to $30K and I pay it to an engineer,
my BTC balance is 0 but my unrealized capital gain for US tax purposes  is
$10K).

More ideally, if there were additional details that I could merge in from
my wallet export, such as payer and payee, notes, etc. it would make my
accounting much easier.

Thus I'd like to see an easier and interoperable way to merge these details
(my account details from an Esplora and price details from Spotbit), with
what my different wallets may (or may not) have available.

I hope that this might inspire some ideas from the people working on this
wallet export format.

-- Christopher Allen
___
bitcoin-dev mailing list
bitcoin-dev@lists.linuxfoundation.org
https://lists.linuxfoundation.org/mailman/listinfo/bitcoin-dev

Re: [bitcoin-dev] BIP Proposal: Wallet Labels Export Format

[Ali Sherief via bitcoin-dev]

> I am attempting to achieve two goals with this proposal, primarily for the
> benefit of wallet users:
>
> Goal #1. Transfer labels between different wallet implementations
> Goal #2. Manage labels in applications outside of Bitcoin wallets (such as
> Excel)
>
> Much of the feedback so far has indicated the tension between these two
> goals - it may be that it is too difficult to achieve both, in which case
> Goal #1 is the most important. That said, I think further exploration is
> still necessary before abandoning Goal #2, because removing it would
> significantly reduce the value of this proposal and mean users need to rely
> on application-specific workarounds.
In my opinion, it would be best if these two goals were split into two separate 
BIPs where the BIP for Goal 2 requires Goal 1's BIP, gut Goal 1's BIP is 
independent. This is because wallet software and business spreadsheet processes 
have different and in some cases divergent needs.

A BIP shouldn't try to address too many things at once, that's why technologies 
like Segwit and Taproot were split into four or five BIPs each.

>  > Don't mandate the file extension... There is no way to enforce this on a
> BIP level.
> I'm not quite sure what you mean here - for example BIP174, which is widely
> used, states "Binary PSBT files should use the .psbt file extension." Also,
> this contradicts Goal #2 - Excel and Numbers register as handlers for .csv,
> and so make it clear that the file is editable outside of a wallet.
BIP174's assignment is a specification but not a hard requirement, becase if 
you have a file whose extension implies one type, but its MIME type (obtained 
from inspecting the file contents) indicates another type, then the extension 
should be disregarded by the parser.

I am aware that business processes are mostly CSV file oriented so you can make 
a statement akin to BIP174 in the Goal 2 BIP, that expects the medium of 
exchange to be in files ending in .csv. I wouldn't mind if you require .csv 
file extension in a BIP for Goal 2. But such a statement is not appropriate in 
the Goal 1 BIP which is only concerned with the wallet label format itself.

> > ZIP does not have good performance or compression ratio
> Indeed, but it is very widely available. That said, gzip is supported
> widely too these days. Unfortunately, gzip does not offer encryption (see
> next answer).
>
> > ZIP is an archiving format, that happens to have its own compression
> format.
> I agree this is not ideal. My main reason for choosing ZIP was that it
> supports encryption. It seems to me that without considering encryption, an
> application must create label export files that allow privacy-sensitive
> wallet information to be readable in plain text. Being able to transfer
> labels without risking privacy is IMO valuable. I considered other
> encryption formats such as PGP, but they are much more niche and so again
> contradict Goal #2.
Both of these look like parts of the spec that should be in the Goal 2 BIP. 
Because Goal 1, which is only concerned with wallet label importing, does not 
need to interact with compression or encryption.

I don't mind if you make Goal 2 BIP utilize ZIP compression with optional 
encryption, it's just that specifying this in the same place in the Goal 1 BIP 
stuff forces wallets to check for that stuff too to be compliant. It's 
important to make compliance as easy as possible.

Regardless, I still believe that making the xpub the ZIP password is a bad 
design, because some wallets that are made from a random list of private keys 
do not have xpubs at all. If the purpose of a password is to make label sharing 
between two parties secure, then why not simply let them agree on a password 
for their own use?

> > I don't see the benefit of encrypting addresses and labels together...
> additionally, the password you propose is insecure - anybody with access to
> the wallet can unlock it
> I'm not sure I understand your question, but both wallet addresses and
> wallet labels contain privacy-sensitive information that should be
> protected. Wrt to the password, there is actually a more fundamental
> problem with using the wallet xpub - there is no equivalent for multisig
> wallets. For this reason I'll remove that requirement in future iterations.
Let me explain.

Before you partitioned the BIP into two goals, I was under the impression that 
wallets would have to read an encrypted export file, which seemed very overkill 
to me (for one, all wallets would now need to bundle a ZIP or AES dependency 
module with their program).

But now I see why a password and encryption would be desireable for Goal 2 BIP 
applications. Like I said though, Goal 1 BIP applications (i.e. wallets) do not 
need any of that.

> > Why the need for input and output formats? There is no difference between
> them on the wallet level, because they are always identified with a txid
> and output index.
> The input refers to the txid and the input index (in t

Re: [bitcoin-dev] BIP Proposal: Wallet Labels Export Format

[Craig Raw via bitcoin-dev]

Thanks for your feedback @Ali.

I am attempting to achieve two goals with this proposal, primarily for the
benefit of wallet users:

Goal #1. Transfer labels between different wallet implementations
Goal #2. Manage labels in applications outside of Bitcoin wallets (such as
Excel)

Much of the feedback so far has indicated the tension between these two
goals - it may be that it is too difficult to achieve both, in which case
Goal #1 is the most important. That said, I think further exploration is
still necessary before abandoning Goal #2, because removing it would
significantly reduce the value of this proposal and mean users need to rely
on application-specific workarounds.

> it is important that a version byte is defined
If Goal #2 is to be achieved it's difficult to mandate this, particularly
if one requires bit flags to be set. Should an importing wallet fail to
import if the version byte is not present, even if all the data is
otherwise correct? Although it is difficult to know in advance how a format
may be extended, it is certainly possible to extend this format with
additional types where the nature of hashes serve as unique identifiers
(more on this below).

 > Don't mandate the file extension... There is no way to enforce this on a
BIP level.
I'm not quite sure what you mean here - for example BIP174, which is widely
used, states "Binary PSBT files should use the .psbt file extension." Also,
this contradicts Goal #2 - Excel and Numbers register as handlers for .csv,
and so make it clear that the file is editable outside of a wallet.

> ZIP does not have good performance or compression ratio
Indeed, but it is very widely available. That said, gzip is supported
widely too these days. Unfortunately, gzip does not offer encryption (see
next answer).

> ZIP is an archiving format, that happens to have its own compression
format.
I agree this is not ideal. My main reason for choosing ZIP was that it
supports encryption. It seems to me that without considering encryption, an
application must create label export files that allow privacy-sensitive
wallet information to be readable in plain text. Being able to transfer
labels without risking privacy is IMO valuable. I considered other
encryption formats such as PGP, but they are much more niche and so again
contradict Goal #2.

> I don't see the benefit of encrypting addresses and labels together...
additionally, the password you propose is insecure - anybody with access to
the wallet can unlock it
I'm not sure I understand your question, but both wallet addresses and
wallet labels contain privacy-sensitive information that should be
protected. Wrt to the password, there is actually a more fundamental
problem with using the wallet xpub - there is no equivalent for multisig
wallets. For this reason I'll remove that requirement in future iterations.

> Why the need for input and output formats? There is no difference between
them on the wallet level, because they are always identified with a txid
and output index.
The input refers to the txid and the input index (in the set of vin), so
the difference is the context in which they are displayed. A wallet will
not necessarily store the spent outputs for a funding transaction
containing a UTXO coming into the wallet, but it will contain references to
the inputs as part of that transaction.

> Another important point is that practically nobody labels inputs or
outputs
To the contrary, UTXOs are very frequently labelled, as they link and
reveal information when spent. Inputs are much less frequently labelled,
but there is no particular reason to exclude them.

> there is a net benefit for the addresses to be exported in ascending order
Indeed, but it makes achieving Goal #2 much more difficult for marginal
benefit.

> It's better to mandate that they should always be double-quoted, since
only wallets will generate label exports anyway.
Rather I think it's better to mandate RFC4180 is followed, as per
recommendations in other feedback.

> The importing code is too naive... it should utilize a dedicate item type
field that unambiguously identifies the item
It's unclear to me what you mean here. As I've indicated it is currently
possible to disambiguate between addresses/transactions/etc without the
need for a 3rd column, but in any case the hash functions used ensure that
labels will not be associated incorrectly. Even in the unlikely event of
some future address type being indistinguishable from a txid, it will
simply not match any txids in the wallet.

Craig



On Wed, Aug 24, 2022 at 9:10 PM Ali Sherief  wrote:

> Hi Craig,
>
> This a really good proposal. I studied your BIP and I have some feedback
> on some parts of it.
>
> > The first line in the file is a header, and should be ignored on import.
>
> From past experience and lessons, most notably BIP39, it is important that
> a version byte is defined somewhere in case someone wants to extend it in
> the future, currently there is no version byte which someone