ddbjson

💫 JSON from/to DynamoDB JSON on CLI

Use ddbjson to convert from DynamoDB JSON format to regular JSON and vice versa (marshall/unmarshall) on any terminal:

• 🔥 Works on all platforms!
• 📄 Convert a file from given file path!
• ✏️ Convert a JSON string!
• ⛓ Read from stdin: pipe in JSON from another command!
• 🍰 Read and convert only a subset of the JSON!
• 🤝 The output can be piped or redirected to other commands!
• 🧰 Integrate it into your workflow, when using AWS DynamoDB CLI!

💡 See usage below! 👇

🔩 Installation

yarn global add ddbjson

npm i -g ddbjson

🎳 Requirements

• NodeJS 12+

🔑 Usage

ddbjson <command> [options] <json>

Commands:
  ddbjson unmarshall, u    Converts a DynamoDB JSON format to regular JSON
  ddbjson marshall, m      Converts a regular JSON to a DynamoDB JSON format

Options:
  -g <path>                Parse only a subset of given JSON by passing the path to a property

Usage examples

• 🔶 Unmarshall: convert from DynamoDB JSON to regular JSON
  • 🔸 Unmarshall from a JSON file
  • 🔸 Unmarshall from a JSON string
  • 🔸 Unmarshall from stdin
• 🔷 Marshall: convert from regular JSON to DynamoDB JSON
  • 🔹 Marshall from a JSON file
  • 🔹 Marshall from a JSON string
  • 🔹 Marshall from stdin
• 🍰 Parse only a subset of JSON using -g

🔶 Unmarshall: convert from DynamoDB JSON to regular JSON

⌨️ Command: unmarshall or u

🔸 Unmarshall from a JSON file

# food.json
{
  "fruits": {
    "L": [
      { "S": "apple" },
      { "S": "kiwi" }
    ]
  }
}

# read from the file:
$ ddbjson u food.json

# {"fruits":["apple","kiwi"]}

🔸 Unmarshall from a JSON string

$ ddbjson u '{"fruits":{"L":[{"S":"apple"},{"S":"kiwi"}]}}'

# {"fruits":["apple","kiwi"]}

🔸 Unmarshall from stdin

• Pipe in JSON from other commands.
• Pipe in from AWS CLI DynamoDB commands (use -g argument – understand why)
• See more usage ideas below!

$ aws dynamodb get-item --table-name food --key '{"type":{"S":"fruit"}}' | ddbjson u -g "Item" -

# {"fruits":["apple","kiwi"]}

🔷 Marshall: convert from regular JSON to DynamoDB JSON

⌨️ Command: marshall or m

🔹 Marshall from a JSON file

# food.json
{
  "fruits": [
    "apple",
    "kiwi"
  ]
}

# read from the file:
$ ddbjson m food.json

# {"fruits":{"L":[{"S":"apple"},{"S":"kiwi"}]}}

🔹 Marshall from a JSON string

$ ddbjson m '{"fruits":["apple","kiwi"]}'

# {"fruits":{"L":[{"S":"apple"},{"S":"kiwi"}]}}

🔹 Marshall from stdin

• Pipe in JSON from other commands.
• See more usage ideas below!

$ curl https://food.com/api | ddbjson m -

# {"fruits":{"L":[{"S":"apple"},{"S":"kiwi"}]}}

🍰 Parse only a subset of JSON using -g

• Use dot notation to pass in a property path.
• Useful when reading from AWS CLI DynamoDB commands (read more).
• Get properties of objects, array item by index or * for all items.

# food.json
{
  "fruits": [
    {
      "name": "apple",
      "color": "red",
      "benefits": ["fiber","vitamin b","vitamin e"]
    },
    {
      "name": "kiwi",
      "color": "green",
      "benefits": ["potassium","vitamin e","vitamin c"]
    }
  ]
}

Convert only an item in the array

$ ddbjson m -g "fruits.0.benefits" food.json

# [{"S":"fiber"},{"S":"vitamin b"},{"S":"vitamin e"}]

Convert all items in the array

$ ddbjson m -g "fruits.*.name" food.json

# [{"S":"apple"},{"S":"kiwi"}]

Note: -g "fruits.*" and -g "fruits" are the same because they both return all items in the array.

⚠️ When reading from AWS CLI DynamoDB commands

Given this food table:

type	foodItems
fruit	[{"S":"apple"},{"S":"kiwi"}]

⚡️ aws dynamodb get-item

AWS CLI wraps the table output in the Item property:

$ aws dynamodb get-item --table-name food --key '{"type":{"S":"fruit"}}' 

# {
#   "Item": {
#     "foodItems": {
#       "L": [
#         {
#           "S": "apple"
#         },
#         ...

Use -g "Item" when unmarshalling output from aws dynamodb get-item:

$ aws dynamodb get-item --table-name food --key '{"type":{"S":"fruit"}}'  | ddbjson u -g "Item" -

# {"foodItems":["apple","kiwi"],"type":"fruit"}

⚡️ aws dynamodb scan

AWS CLI wraps the table output in the "Items" property:

$ aws dynamodb scan --table-name food

{
  "Items": [
    {
      "type": { "S": "fruit" },
      "foodItems": {
        "L": [
            { "S": "apple" },
            { "S": "kiwi" }
            ...

Use -g "Items" to unmarshall output from aws dynamodb scan.

$ aws dynamodb scan --table-name food | ddbjson u -g "Items" -

# [{"foodItems":["apple","kiwi"],"type":"fruit"},...]

💡 More usage ideas

🌷 Format output using jq

Use jq to format the JSON output:

$ ddbjson u marshalled-food.json | jq

# {
#   "fruits": [
#     "apple",
#     "kiwi"
#   ]
# }

Redirect output to a file:

$ ddbjson u marshalled-food.json | jq > unmarshalled-food.json
$ cat unmarshalled-food.json

# {
#   "fruits": [
#     "apple",
#     "kiwi"
#   ]
# }

When combined with AWS CLI output this is really powerful:

$ aws dynamodb get-item --table-name food --key `ddbjson m '{"type":"fruit"}'` | ddbjson u -g 'Item' - | jq > result.json
$ cat result.json

# {
#   "foodItems": [
#     "apple",
#     "kiwi"
#   ],
#   "type": "fruit"
# }

⛓ Use as part of other commands

For example, marshall the Partition key in the AWS CLI command:

$ aws dynamodb get-item --table-name food --key `ddbjson m '{"type":"fruit"}'`

🎬 Use heredoc

Read from stdin when combined with heredocs:

$ cat << EOF | ddbjson m -               
{               
  "fruits": [
    "apple",
    "kiwi"
  ]
}
EOF

# {"fruits":{"L":[{"S":"apple"},{"S":"kiwi"}]}}

✏️ Convert regular JSON file when writing to DynamoDB

# meat.json
{
  "type": "meat",
  "foodItems": [
    "beef",
    "pork"
  ]
}

Read from regular JSON file to write to DynamoDB:

$ aws dynamodb put-item --table-name food --item `ddbjson m meat.json`