Skip to content

Latest commit

 

History

4,033 Commits

Folders and files

NameName
Last commit message
Last commit date
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

A Python library for 'bitcoin cryptography'

PyPI version GitHub release development status license downloads supported Python versions implementation wheel

pre-commit.ci status lint workflow status test workflow status docs workflow status documentation build vendored-vectors workflow status mutation workflow status fuzz workflow status integration-bitcoind workflow status zkp-oracle workflow status integration-hwi workflow status deps-latest workflow status pypi-install workflow status py-arm-authority workflow status os-macos workflow status os-ubuntu workflow status os-windows workflow status links workflow status codeql workflow status

OpenSSF Scorecard OpenSSF Best Practices


btclib is a Python type annotated library for teaching, learning and using bitcoin, focused on elliptic curve cryptography and bitcoin's blockchain. It started as a teaching tool for Ferdinando Ametrano's Bitcoin and Blockchain Technology course, it is used in production today (still marked as beta because it is often refactored for improved clarity — CONTRIBUTING.md's Breaking a caller is not an argument says what that promises a caller and what it does not).

The test suite covers virtually the whole code base, a floor the build enforces, and it answers to vectors their authors publish: the BIPs' and the SLIPs' own, Bitcoin Core's script, transaction, sighash and key-encoding files, HWI's, trezor's for BIP39 and SLIP39, and Appendix A.2 of RFC 6979. tests/_data/README.md pins each vendored file to the upstream commit it was copied from, and says whether the two still match — including the few vectors that are btclib's own, having no upstream.

The library is not limited to secp256k1, and for that curve it delegates to btclib-secp256k1, FFI bindings to Bitcoin Core's optimized C library libsecp256k1, wherever a call's own guard admits them. They are the recommended install and what pip install "btclib[secp256k1]" asks for, needing one of their wheels or a C toolchain; without them, or with the delegation turned off in a process that has them, btclib still answers, on the Python arithmetic, tens of times more slowly and not in constant time — SECURITY.md publishes both. That Python arithmetic serves every other curve anyway, and the suite validates it against the bindings: libsecp256k1 says what the right answer is, being what bitcoin consensus relies on.

Included features are:

  • modulo algebra functions (gcd, inverse, legendre symbol, square root)
  • octets / integer / point / var_int / var_bytes helper functions
  • elliptic curve class
    • fast algebra implemented using Jacobian coordinates
    • double scalar multiplication (Straus's algorithm, also known as Shamir's trick)
    • multi scalar multiplication (Bos-coster's algorithm)
    • point symmetry solution: odd/even, low/high, and quadratic residue
    • elliptic curves: SEC 1 v1 and v2, NIST, Brainpool, and low cardinality test curves
  • ECDSA signature with (transaction) DER encoding
  • ECDSA signature with (message) compact encoding: standard p2pkh and BIP137/Electrum extensions to p2wpkh and p2wpkh-p2sh
  • BIP322 signed messages, where the address is a script to satisfy rather than a key to recover: the simple, full and proof-of-funds variants, verified by the script engine, so multisig, taproot and time locks sign as well as p2pkh does
  • RFC 6979 for deterministic signature schemes
  • EC Schnorr signature (according to BIP340 bitcoin standardization)
    • batch validation
    • threshold signature (see test-suite)
    • MuSig2 multi-signature: key aggregation with plain and x-only tweaking, nonce aggregation, partial signatures and their aggregation, one primitive per round of the protocol
  • Borromean ring signature
  • Sign-to-contract commitment
  • Diffie-Hellman, and the x-only ECDH on the BIP324 ElligatorSwift encoding of a public key
  • BIP374 discrete logarithm equality proofs: 64 bytes proving that an ECDH shared secret was computed from the key that signed, without revealing that key, over an arbitrary generator and an optional message
  • ECIES in the BIE1 layout, the block cipher supplied by the caller
  • Pedersen commitment
  • Base58 encoding/decoding
  • p2pkh/p2sh addresses and WIFs
  • Bech32 encoding/decoding
  • p2wpkh/p2wsh native segwit addresses and their legacy p2sh-wrapped versions
  • BIP32 hierarchical deterministic key chains
  • BIP39 mnemonic for generating deterministic keys, in the wordlists of the reference implementation, with the language read off the words
  • Electrum standard for mnemonic, reading the same wordlists as BIP39 except for Portuguese, which is Electrum's own list
  • SLIP39 Shamir backup: a master secret split into mnemonic shares, of which a threshold number recovers it
  • BIP44 address from an extended key and a m/purpose'/coin_type'/account'/change/address_index path, the purpose selecting the encoding: 44 p2pkh, 49 p2wpkh-p2sh, 84 p2wpkh (BIP84), 86 p2tr (BIP86)
  • SLIP132 key versions (xprv, yprv, zprv, Yprv, Zprv, tprv, uprv, vprv, and Uprv) with corresponding mapping to p2pkh/p2sh, p2wpkh-p2sh, p2wpkh, p2wsh-p2sh, p2wsh and p2tr addresses
  • BIP85 deterministic entropy: one root key behind many wallets, a hardened path saying which, and each application taking what it needs of the 512 bits it reaches — a BIP39 mnemonic, the Bitcoin Core hdseed WIF, an xprv, raw bytes, a base64 or base85 password, dice rolls, and the SHAKE256 stream an RSA key generator reads
  • BIP352 silent payments: one reusable bech32m address, and a different taproot output for every payment to it — the sender's outputs, the receiver's scan, the labels that give one wallet many published addresses, and the tweak data a light client scans from
  • Script encoding/decoding
  • nulldata, p2pk, p2ms, p2pkh, p2sh, p2wpkh, p2wsh and p2tr ScriptPubKeys
  • a script engine: a transaction verified against the consensus rules, legacy, segwit and tapscript, with Bitcoin Core's own vectors behind it
  • BIP380 output descriptors: the checksum, the parser, the scripts a descriptor names, and the spend
  • BIP379 miniscript, read, written and spent, inside wsh() and as a tr() leaf: the expression compiled to a script, a script read back into the expression it is, the type system that says an expression is well formed, the bounds a spend of it is analysed by, and the non-malleable witness that satisfies it
  • OutPoint, TxIn, TxOut, and TX data classes
  • legacy, segwit_v0 and taproot transaction hash signatures
  • BlockHeader and Block data classes
  • merkle proofs verified against a header's merkle root
  • proof-of-work arithmetic: compact targets, retargeting, work, hash rate
  • BIP174 partially signed bitcoin transactions (PSBT): PsbtIn, PsbtOut, and Psbt data classes, with the taproot fields of BIP371 and the MuSig2 ones of BIP373
  • PsbtView, the same psbt read a map at a time out of a stream, for a signer with less memory than the psbt takes: the maps it is asked for, the transaction being built, the outputs being spent and both sig_hashes
  • BIP370 PSBT version 2, the unsigned transaction computed from the fields rather than carried as one: the lock time its inputs require, the identifier that ignores their sequences, the modifiable flags a Constructor must obey, and conversion either way
  • BIP375 silent payments in a PSBT: the six fields that carry an ECDH share, its BIP374 proof and the address being paid, the output script that may not exist yet, and the identifier that reads the address in its place — with both roles the BIP adds, the Signer that writes the shares and derives the scripts and the Transaction Extractor that recomputes every one of them before the transaction goes out
  • BIP21 bitcoin: payment URIs
  • fee rates carrying their unit (sat/kvB, sat/vB, and the BTC/kvB Bitcoin Core quotes one in), the fee a virtual size owes at one, what a child owes for the unconfirmed ancestors it is mined with, and the dust threshold of any output type, computed as Bitcoin Core computes it rather than tabulated
  • wallets, several sources of addresses behind one vocabulary: an extended key at a BIP44 account or a set of individual keys, which also answer the private key that signs for an address — what sign(address, msg) needs — an output descriptor per chain, and a script template with multisig quorums in it, for the pre-descriptor wallets no descriptor states — and, for the ones that turn out to have a descriptor after all, the ranged descriptor lifted out of the script itself, confirmed against the addresses it derives. Each answers address(branch, index), script_pub_key(branch, index) and position_of(script_pub_key), the last being "is this output mine", compared whole and never on a key origin's fingerprint
  • an external signer behind one contract, with Bitcoin Core's HWI behind it for a hardware wallet
  • a chain backend behind one interface — a transaction by id, the output an outpoint names, the chain tip — over a full node's JSON-RPC or a block explorer's HTTP api

Secrets, and where constant time ends

btclib is used to teach and to prototype as much as to build, and the two uses want different things of it. What follows is the boundary between them, before a private key is handed to any of the above.

A Python object carrying secret material cannot be reliably zeroized: it stays in the process memory until garbage collection, and the interpreter may have copied it meanwhile. The constant-time properties are libsecp256k1's, and they hold on the C side of the call — not before it, and not after.

Not every operation crosses that call, and what decides is one predicate — a process-wide dispatch switch, secp256k1 as the curve, and sha256 or no hash function at all — with whatever further conditions the call site ands onto it. Those conditions differ from one function to the next, and SECURITY.md states each of them, for dsa.sign, ssa.sign and silent_payments.output_keys alike. Whatever that conjunction declines runs the Python arithmetic, which the suite validates against the bindings but which is not constant-time. A process that has the bindings turns that switch off with curves.set_libsecp256k1_serving(serving=False), or with BTCLIB_NO_LIBSECP256K1 in the environment, and every operation here is then the Python arithmetic. So a caller whose threat model includes timing should stay on the delegated paths, or keep the key out of the process altogether: btclib.hwi drives a hardware wallet through HWI, behind the same PsbtSigner contract a software signer answers. silent_payments.scan_outputs, BIP352's light-client scan, is Python-only regardless: it accepts the shared secret already reduced, the shape a light client has and the bindings have no entry point for. scan_transaction_outputs, its full-node sibling, is not: where the bindings serve secp256k1 it reaches them with b_scan, the recipient's scan private key, the same as output_keys above — a caller holding the transaction itself gets the delegated path a light client cannot reach.

Crossing that call is not the same as constant time. Where that conjunction delegates a mult of a point that is not the generator, its work follows the scalar, so a secret multiplied by a point you supplied — the shared point of a key agreement, among others — carries no timing guarantee on the delegated path either; SECURITY.md has the accounting, and which call a multiplication takes is part of it.

What that path does about it is in the names, and it is worth knowing before calling one. A function whose duration follows the value it is given ends in _var, and the plain name beside it is the one a secret may be handed: mod_inv draws a random blinding factor where mod_inv_var is the bare extended Euclid, and mult makes the same additions for every scalar where double_mult_var does not. It is libsecp256k1's own convention, and forgetting to choose gives the safer call rather than the faster one.

The suffix is not a safety label, and no name here promises constant time. It says which of two spellings to reach for, and each one was measured rather than assumed — including the ones that kept a plain name, which CONTRIBUTING lists with the figure that earned it.

SECURITY's "Limitations, not vulnerabilities" states each condition exactly — which arguments delegate, which do not, and what the Python path does hide — and is the canonical text; this section is the pointer to it.


Module layout

Each pair of modules below is one idea split in two, and each split runs one way only:

the codec / the arithmetic the bitcoin semantics on top
btclib.curvesCurve, mult btclib.eccdsa, ssa, bms
btclib.base58 — the encoding btclib.b58 — WIF, p2pkh, p2sh
btclib.bech32 — the encoding btclib.b32 — p2wpkh, p2wsh, p2tr

The right column imports the left one; the left never imports the right.

So from btclib.ecc import dsa for a signature, from btclib.curves import mult for a point multiplication, btclib.b58 for an address, btclib.base58 for the encoding on its own. Each of these modules says the same in its own docstring.

The rest, roughly bottom-up. alias holds the types the public API accepts, much of it taking anything convertible rather than one type, and exceptions the errors it raises. The scalar and the curve point in their octet spellings are curves' own, read by curves.scalar_from_prv_key and curves.point_from_pub_key; what is left to to_prv_key and to_pub_key is the network and compression a record carries and a key does not. A spelling that carries either belongs to the module that defines it, so a WIF is b58's, an extended key is bip32's, and a Casascius minikey is minikey's, read-only. bip32 and mnemonic derive keys. script, tx, block and psbt build and validate what goes on the chain, and script.engine runs a transaction against the consensus rules. p2p is the wire format peers speak — the message envelope, its framing, the message start each network begins with, and the payloads a connection opens with — and it opens no socket: fetch is the one package that goes and asks, and neither imports the other.

Above them, bip44 composes bip32, script.taproot and both address encodings into an address from an extended key and a derivation path, and descriptors reads the BIP380 grammar and hands back the scripts a descriptor names -- with descriptors.miniscript reading BIP379's language, which is a script written as a tree of fragments, and satisfying one. psbt_signer is the contract an external signer answers; hwi is that contract over Bitcoin Core's HWI.

Nothing in the library imports bip322, bip85, bip38, slip132, wallet, hwi, p2p or fetch: they are the top of the stack, and fetch is the only one that goes out to the network. bip38 is a password-protected private key, Base58Check with scrypt and the caller's own AES-256, over b58 and curves alone. bolt11 is BOLT11's own codec -- bech32 with m=1 explicit, ecc.dsa for the signature and its recovery -- and bip21 is this tier's one importer, composing it for the typed lightning= parameter the way it already composes b32, b58 and network below it. wallet remembers which addresses it has handed out — over bip44, over descriptors or over a script template of its own — and its key wallets sign for one with ecc.bms. bip322 is the other message signing, and it is at the top rather than beside ecc.bms because it needs everything below it: a script, a transaction, a psbt and the engine that runs them. bip85 derives the entropy behind another wallet's seed from one root key, and is up here because a BIP39 sentence and a WIF are two of the formats it hands back.

The rpc client fetch speaks through is not in that stack: it is bitcoin-core-rpc, a package of its own that btclib depends on — zero dependencies of its own, standard library only, and usable by anyone who wants a node client and no bitcoin library. btclib.fetch turns its answers into Tx and TxOut, and where the backend is a node — over JSON-RPC or over -rest — checks the chain it reports against the network those are labelled for.

The dependency stops at src/btclib/fetch/ and at src/btclib/p2p/magic.py, which is where the p2p message start is — that package's table, not a second copy of it. bitcoin-core-rpc declares its own FetchError, and declares zero dependencies of its own, importing nothing of btclib's; btclib.fetch.fetcher.client_errors re-raises it as btclib.exceptions' own, with the status and the code carried across: an except FetchError written against btclib catches what a fetcher raises. No module loads urllib.request on its way to anything else: importing it is what reaching the client or its transport costs, and src/btclib/fetch/ is the only place that does. btclib.p2p's message start reaches this same package's chain vocabulary instead, which depends on nothing beyond the standard library, so a caller who parses messages pays nothing for a client it never uses. Constructing a client opens no socket; the first call does.


To install, or upgrade:

python -m pip install --upgrade btclib

In a virtual environment:

python -m venv venv_btclib
source venv_btclib/bin/activate
python -m pip install --upgrade btclib

On Windows the second line is venv_btclib\Scripts\activate in CMD and PowerShell, source venv_btclib/Scripts/activate in Git bash.

CONTRIBUTING is for development, REVIEWING for what a pull request is answered against, SECURITY for reporting a vulnerability.


The btclib organization and its projects are actively supported by DGI and CheckSig.