ssb-split-publish

Side-step the 8kb content limit for messages by splitting your content across multiple messages!

Usage

const SplitPublish = require('ssb-split-publish')

const splitPublish = new SplitPublish(ssb, splitter, { afterPublish })

function splitter (content) {
  const first = {
    type: 'post',
    text: content.text.slice(0, 4000), // janky!
    root: content.root || null
  }
  if (content.branch) first.branch = content.branch

  const second = {
    type: 'post',
    text: content.text.slice(4000), // janky!
    root: content.root,
    branch: ['TODO']
  }

  return [first, second]
}
function afterPublish (firstMsg, secondChunk) {
  secondChunk.root = firstMsg.value.content.root || firstMsg.key
  secondChunk.branch = [firstMsg.key]
  return secondChunk
}

const content = {
  type: 'post',
  text: 'You know what really grings my gears? {....}', // TOO LONG
  root: null
}

splitPublish(content, (err, msgs) => {
  console.log(msgs)
  // => [
  //   msg1,
  //   msg2...
  // ]
})

API

SplitPublish(ssb, splitter, opts) => splitPublish

where

• ssb is a scuttlebutt instance
• splitter function of signature (content) => [firstChunk, secondChunk]
  • each "chunk" is expected to be content which can be pubished
  • you are responsible for e.g. recps, tangles etc
  • firstChunk should be chunk most likely to fail (see algorithm)
• opts Object a collection of optional features
  • opts.afterPublish function of signature (firstMsg, secondChunk) => secondChunkMutated
    • firstMsg is a message which comes from firstChunk being published (decrypted if possible)

splitPublish(content, cb)

where

• content Object is of form { type, ... }
• cb function is a callback which calls back with (err, [msg])
  • note it calls back with an Array of messages (if success)

Algorithm

Split-publish first tries to publish content as-is. If this fails with the "too large" error, then it invokes the splitter to split your content into two parts.

If then publishes the first part. If this fails, publishing fails completely. If it succeds, then it moves on to publishing the second part. * If that second part is too large, the process recurses. On completing we call back with the Array of messages published

NOTES:

• When you split, you should return the chunk most likely to fail first
  • if publishing that fails, nothing will be published
  • if you split something tiny off for publishing first, you may later hit a fail and have only part of your content published
• There's a hook after the first chunk is published, but before the second chunk is published (see *), where you have access to the published first chunk and can use that to mutate your second chunk before that's published.