@web3-storage/blob-index

An index for slices that may be sharded across multiple blobs.

Install

npm install @web3-storage/blob-index

Usage

Create:

import { ShardedDAGIndex } from '@web3-storage/blob-index'

// Create a brand new index
const index = ShardedDAGIndex.create(rootCID)

// Add index data for slices within a shard
index.setSlice(shardMultihash, sliceMultihash, [offset, length])
// ...

// Create CAR archive
const result = index.archive()

console.log(result.ok) // a Uint8Array

Read:

import { ShardedDAGIndex } from '@web3-storage/blob-index'
import { base58btc } from 'multiformats/bases/base58'

const index = ShardedDAGIndex.extract(car)

console.log(index.content)

for (const [shard, slices] of index.shards.entries()) {
  console.log(`Shard ${base58btc.encode(shard.bytes)}`)
  console.log('  Slices:')
  for (const [slice, [offset, length]] of slices.entries()) {
    console.log(`    ${base58btc.encode(slice.bytes)} @ ${offset} -> ${offset + length}`)
  }
}

// Output:
// Shard zQmQKw6B745GGL3eeTcEE5kAoLAJgkBQydJPC5fWv5HA68A
//  Slices:
//    zQmeHPRNRDxHU5YMPewcBCbPYxzA3jBcadAZQwpQXm3jFFt @ 96 -> 128
// ...

Contributing

Feel free to join in. All welcome. Please open an issue!

License

Dual-licensed under MIT + Apache 2.0