IPLD for Bitcoin
JavaScript Bitcoin data multiformats codecs and utilities for IPLD
About
This codec is intended to be used with multiformats. It provides decode and encode functionality for the Bitcoin native format to and from IPLD.
The following IPLD codecs are available; they each support encode()
and decode()
functionality compatible with the multiformats BlockCodec
type.
Codecs
bitcoin-block
bitcoin-block
/ 0xb0
is the Bitcoin block header, commonly identified by "Bitcoin block identifiers" (hashes with leading zeros).
import * as bitcoinBlock from '@ipld/bitcoin/block'
bitcoin-tx
bitcoin-tx
/ 0xb1
are Bitcoin transactions and nodes in a binary merkle tree, the tip of which is referenced by the Bitcoin block header.
import * as bitcoinTx from '@ipld/bitcoin/tx'
bitcoin-witness-commitment
bitcoin-witness-commitment
/ 0xb2
is the Bitcoin witness commitment that is used to reference transactions with intact witness data (a complication introduced by SegWit).
import * as bitcoinWitnessCommitment from '@ipld/bitcoin/witness-commitment'
Hasher
The following multihash is available, compatible with the multiformats MultihashHasher
type.
dbl-sha2-256
dbl-sha2-256
/ 0x56
is a double SHA2-256 hash: SHA2-256(SHA2-256(bytes))
, used natively across all Bitcoin blocks, forming block identifiers, transaction identifiers and hashes and binary merkle tree nodes.
import * as dblSha2256 from '@ipld/bitcoin/dbl-sha2-256'
Utilities
In addition to the multiformats codecs and hasher, utilities are also provided to convert between Bitcoin hash identifiers and CIDs and to convert to and from full Bitcoin raw block data to a full collection of IPLD blocks. Additional conversion functionality for bitcoin raw data and the bitcoin-cli
JSON format is provided by the bitcoin-block library.
See the API section below for details on the additional utility functions.
Example
This example reads Bitcoin IPLD blocks from a CAR file; assuming that CAR contains a complete (enough) graph representing a Bitcoin block (whose identifier is supplied as the second argument) and its transactions, it navigates to the first transaction (the Coinbase) and prints the scriptSig
as UTF-8. This is often used to store arbitrary messages and other "graffiti".
Running this example on the Genesis block (CAR provided in this project: example-genesis.car) produces the following output:
$ node example.js ./example-genesis.car 000000000019d6689c085ae165831e934ff763ae46a2a6c172b3f1b60a8ce26f
��EThe Times 03/Jan/2009 Chancellor on brink of second bailout for banks
import fs from 'fs'
import * as bitcoinBlock from '@ipld/bitcoin/block'
import * as bitcoinTx from '@ipld/bitcoin/tx'
import { CarReader } from '@ipld/car'
// Assumes a CAR with at least one full Bitcoin block represented as IPLD blocks
// and a "blockId" which is the commonly used Bitcoin block identifier (32-byte
// digest in hexadecimal, with leading zeros).
async function run (pathToCar, blockId) {
const reader = await CarReader.fromIterable(fs.createReadStream(pathToCar))
const headerCid = bitcoinBlock.blockHashToCID(blockId)
const header = bitcoinBlock.decode((await reader.get(headerCid)).bytes)
// navigate the transaction binary merkle tree to the first transaction, the coinbase,
// which will be at the leftmost side of the tree.
let txCid = header.tx
let tx
while (true) {
tx = bitcoinTx.decode((await reader.get(txCid)).bytes)
if (!Array.isArray(tx)) { // is not an inner merkle tree node
break
}
txCid = tx[0] // leftmost side of the tx binary merkle
}
// convert the scriptSig to UTF-8 and cross our fingers that there's something
// interesting in there
console.log(Buffer.from(tx.vin[0].coinbase, 'hex').toString('utf8'))
}
run(process.argv[2], process.argv[3]).catch((err) => {
console.error(err.stack)
process.exit(1)
})
Usage
In the API docs below, the names denote the export locations, such that they may be obtained by the following:
// The whole bundle (note this object also includes additional properties that
// can be used to access all the others)
import * as Bitcoin from '@ipld/bitcoin'
// the `bitcoin-block` / `0xb0` codec
import * as BitcoinBlock from '@ipld/bitcoin/block'
// the `bitcoin-tx` / `0xb1` codec
import * as BitcoinTransaction from '@ipld/bitcoin/tx'
// the `bitcoin-witness-commitment` / `0xb2` codec
import * as BitcoinWitnessCommitment from '@ipld/bitcoin/witness-commitment'
// the `dbl-sha2-256` / `0x56` multihasher
import * as DblSha2256 from '@ipld/bitcoin/dbl-sha2-256'
API
Contents
Bitcoin.deserializeFullBitcoinBytes()(bytes)
Bitcoin.serializeFullBitcoinBytes()(obj)
Bitcoin.cidToHash()(cid)
Bitcoin.encodeAll()
Bitcoin.assemble()
BitcoinBlock.encode()
BitcoinBlock.decode()
BitcoinBlock.name
BitcoinBlock.code
BitcoinBlock.blockHashToCID()
BitcoinTransaction.encode()
BitcoinTransaction.encodeNoWitness()
BitcoinTransaction.encodeAll()
BitcoinTransaction.encodeAllNoWitness()
BitcoinTransaction.decode()
BitcoinTransaction.name
BitcoinTransaction.name
BitcoinTransaction.txHashToCID()
BitcoinWitnessCommitment.encode()
BitcoinWitnessCommitment.decode()
BitcoinWitnessCommitment.name
BitcoinWitnessCommitment.code
DblSha2256.name
DblSha2256.code
DblSha2256.encode()
DblSha2256.digest()
Bitcoin.deserializeFullBitcoinBytes()(bytes)
-
bytes
(Uint8Array)
: a binary form of a Bitcoin block graph -
Returns:
BlockPorcelain
: an object representation of the full Bitcoin block graph
Instantiate a full object form from a full Bitcoin block graph binary representation. This binary form is typically extracted from a Bitcoin network node, such as with the Bitcoin Core bitcoin-cli
getblock <identifier> 0
command (which outputs hexadecimal form and therefore needs to be decoded prior to handing to this function). This full binary form can also be obtained from the utility assemble
function which can construct the full graph form of a Bitcoin block from the full IPLD block graph.
The object returned, if passed through JSON.stringify()
should be identical to the JSON form provided by the Bitcoin Core bitcoin-cli
getblock <identifier> 2
command (minus some chain-context elements that are not possible to derive without the full blockchain).
Bitcoin.serializeFullBitcoinBytes()(obj)
-
obj
(BlockPorcelain)
: a full JavaScript object form of a Bitcoin block graph -
Returns:
Uint8Array
: a binary form of the Bitcoin block graph
Encode a full object form of a Bitcoin block graph into its binary
equivalent. This is the inverse of
Bitcoin.deserializeFullBitcoinBytes()
and should produce the exact
binary representation of a Bitcoin block graph given the complete input.
The object form must include both the header and full transaction (including witness data) data for it to be properly serialized.
As of writing, the witness merkle nonce is not currently present in the JSON
output from Bitcoin Core's bitcoin-cli
. See
https://github.com/bitcoin/bitcoin/pull/18826 for more information. Without
this nonce, the exact binary form cannot be fully generated.
Bitcoin.cidToHash()(cid)
-
cid
(CID|string)
: a CID -
Returns:
string
: a hexadecimal big-endian representation of the identifier.
Convert a CID to a Bitcoin block or transaction identifier. This process is
the reverse of blockHashToCID()
and txHashToCID()
and involves extracting
and decoding the multihash from the CID, reversing the bytes and presenting
it as a big-endian hexadecimal string.
Works for both block identifiers and transaction identifiers.
Bitcoin.encodeAll()
-
block
(BlockPorcelain)
-
Returns:
IterableIterator<{cid: CID, bytes: Uint8Array}>
Encodes a full Bitcoin block, as presented in BlockPorcelain
form (which is
available as JSON output from the bitcoin-cli
tool—see the bitcoin-block
npm package for more information) into its constituent IPLD blocks. This
includes the header, the transaction merkle intermediate nodes, the
transactions and SegWit forms of the transaction merkle and nodes if present
along with the witness commitment block if required.
Bitcoin.assemble()
-
loader
(IPLDLoader)
: an IPLD block loader function that takes a CID argument and returns aUint8Array
containing the binary block data for that CID -
blockCid
(CID)
: a CID of typebitcoin-block
pointing to the Bitcoin block header for the block to be assembled -
Returns:
Promise<{deserialized:BlockPorcelain, bytes:Uint8Array}>
: an object containing two properties,deserialized
andbytes
wheredeserialized
contains a full JavaScript instantiation of the Bitcoin block graph andbytes
contains aUint8Array
with the binary representation of the graph.
Given a CID for a bitcoin-block
Bitcoin block header and an IPLD block
loader that can retrieve Bitcoin IPLD blocks by CID, re-assemble a full
Bitcoin block graph into both object and binary forms. This is the inverse
of the Bitcoin.encodeAll()
function in that it puts the
BitcoinPorcelain
back together. A JSON form of this output should match
the output provided by bitcoin-cli
(with some possible minor differences).
The loader should be able to return the binary form for bitcoin-block
,
bitcoin-tx
and bitcoin-witness-commitment
CIDs.
BitcoinBlock.encode()
-
node
(BitcoinHeader)
-
Returns:
ByteView<BitcoinHeader>
bitcoin-block
/ 0xb0
codec: Encodes an IPLD node representing a
Bitcoin header object into byte form.
BitcoinBlock.decode()
-
data
(ByteView<BitcoinHeader>)
-
Returns:
BitcoinHeader
bitcoin-block
/ 0xb0
codec: Decodes a bytes form of a Bitcoin header
into an IPLD node representation.
BitcoinBlock.name
bitcoin-block
/ 0xb0
codec: the codec name
BitcoinBlock.code
bitcoin-block
/ 0xb0
codec: the codec code
BitcoinBlock.blockHashToCID()
-
blockHash
(string)
: a string form of a block hash -
Returns:
CID
: a CID object representing this block identifier.
Convert a Bitcoin block identifier (hash) to a CID. The identifier should be in big-endian form, i.e. with leading zeros.
The process of converting to a CID involves reversing the hash (to little-endian form), encoding as a dbl-sha2-256
multihash and encoding as a bitcoin-block
multicodec. This process is reversable, see cidToHash
.
BitcoinTransaction.encode()
-
node
(BitcoinTransaction|BitcoinTransactionMerkleNode)
-
Returns:
ByteView<(BitcoinTransaction|BitcoinTransactionMerkleNode)>
bitcoin-tx
/ 0xb1
codec: Encodes an IPLD node representing a
Bitcoin transaction object into byte form.
Note that a bitcoin-tx
IPLD node can either be a full transaction with or
without SegWit data, or an intermediate transaction Merkle tree node; in
which case it is simply an array of two CIDs.
BitcoinTransaction.encodeNoWitness()
-
node
(BitcoinTransaction)
-
Returns:
ByteView<BitcoinTransaction>
Same as BitcoinTransaction.encode()
but will explictly exclude any
witness (SegWit) data from the output. This is necessary for encoding SegWit
blocks since transactions must be stored both with and without witness data
to correctly represent the full content addressed structure.
BitcoinTransaction.encodeAll()
-
obj
(BlockPorcelain)
-
Returns:
Encodes all transactions in a complete BlockPorcelain
(see the
bitcoin-block
npm package for details on this type) representation of an
entire Bitcoin transaction; including intermediate Merkle tree nodes.
Intermediate Merkle tree nodes won't have the transaction
property on the
output as they aren't full transactions and their bytes
will have a length
of 64.
BitcoinTransaction.encodeAllNoWitness()
-
obj
(BlockPorcelain)
-
Returns:
Same as BitcoinTransaction.encodeAll()
but only encodes non-SegWit
transaction data, that is, transactions without witness data and no secondary
SegWit transactions Merkle tree.
BitcoinTransaction.decode()
-
data
(ByteView<(BitcoinTransaction|BitcoinTransactionMerkleNode)>)
-
Returns:
BitcoinTransaction|BitcoinTransactionMerkleNode
bitcoin-block
/ 0xb0
codec: Decodes a bytes form of a Bitcoin
transaction into an IPLD node representation.
Note that a bitcoin-tx
IPLD node can either be a full transaction with or
without SegWit data, or an intermediate transaction Merkle tree node; in
which case it is simply an array of two CIDs. As byte form, an intermediate
Merkle tree node is a fixed 64-bytes.
BitcoinTransaction.name
bitcoin-tx
/ 0xb1
codec: the codec name
BitcoinTransaction.name
bitcoin-tx
/ 0xb1
codec: the codec name
BitcoinTransaction.txHashToCID()
-
txHash
(string)
: a string form of a transaction hash -
Returns:
CID
: A CID (multiformats.CID
) object representing this transaction identifier.
Convert a Bitcoin transaction identifier (hash) to a CID. The identifier should be in big-endian form as typically understood by Bitcoin applications.
The process of converting to a CID involves reversing the hash (to little-endian form), encoding as a dbl-sha2-256
multihash and encoding as a bitcoin-tx
multicodec. This process is reversable, see cidToHash
.
BitcoinWitnessCommitment.encode()
-
node
(BitcoinWitnessCommitment)
-
Returns:
ByteView<BitcoinWitnessCommitment>
bitcoin-witness-commitment
/ 0xb2
codec: Encodes an IPLD node
representing a Bitcoin witness commitment object into byte form.
The object is expected to be in the form
{witnessMerkleRoot:CID, nonce:Uint8Array}
where the witnessMerkleRoot
may be null.
BitcoinWitnessCommitment.decode()
-
data
(ByteView<BitcoinWitnessCommitment>)
-
Returns:
BitcoinWitnessCommitment
bitcoin-witness-commitment
/ 0xb2
codec: Decodes a bytes form of a
Bitcoin witness commitment into an IPLD node representation.
.
The returned object will be in the form
{witnessMerkleRoot:CID, nonce:Uint8Array}
where the witnessMerkleRoot
may be null.
BitcoinWitnessCommitment.name
bitcoin-witness-commitment
/ 0xb2
codec: the codec name
BitcoinWitnessCommitment.code
bitcoin-witness-commitment
/ 0xb2
codec: the codec code
DblSha2256.name
dbl-sha2-256
/ 0x56
multihash: the multihash name
DblSha2256.code
dbl-sha2-256
/ 0x56
multihash: the multihash code
DblSha2256.encode()
-
bytes
(Uint8Array)
: a Uint8Array -
Returns:
Uint8Array
: a 32-byte digest
dbl-sha2-256
/ 0x56
multihash: Encode bytes using the multihash
algorithm, creating raw 32-byte digest without multihash prefix.
DblSha2256.digest()
-
input
(Uint8Array)
-
Returns:
dbl-sha2-256
/ 0x56
multihash: Encode bytes using the multihash
algorithm, creating multihash Digest
(i.e. with multihash prefix).
License
Licensed under either of
- Apache 2.0, (LICENSE-APACHE / http://www.apache.org/licenses/LICENSE-2.0)
- MIT (LICENSE-MIT / http://opensource.org/licenses/MIT)
Contribution
Unless you explicitly state otherwise, any contribution intentionally submitted for inclusion in the work by you, as defined in the Apache-2.0 license, shall be dual licensed as above, without any additional terms or conditions.