Shortcode Tokenizer and AST
Tokenizes a string containing shortcodes (re-popularized by WordPress) and outputs it as an AST that can be used for further parsing.
If you are only looking for simple transformations from a shortcode to a string I suggest that you give these libs a try:
- https://github.com/mendezcode/shortcode-parser
- https://www.npmjs.com/package/meta-shortcodes (handles nested shortcodes)
If you need more control this is the lib for you.
Install
npm install shortcode-tokenizer
Usage
const input = `<h1>Cool Shop</h1>[row] [col width=6 class="featured"] [product-list list="featured" /] [/col] [col width=6 class="featured"] <div>Ad: Buy more, Buy often!</div> [/col][/row]` const tokenizer = inputtokenizer
The AST outputted is at root level an array of two nodes, a text node and a code-node:
type: 'TEXT' body: "<h1>Cool Shop</h1>" pos: 0 type: 'OPEN' name: 'row' pos: 19 body: '[row]' isClosed: true params: {} children: ... a whitespace TEXT token ... type: 'OPEN' name: 'col' pos: 27 body: '[col width=6 class="featured"]' isClosed: true params: width: 6 class: 'featured' children: ... and so on ...
(Note: the same data-structure is used to represent tokens from the lexing and nodes in the AST, see CLOSE, below)
There are 5 token types:
- TEXT: plain text.
body
contains the content. - OPEN: an open token, e.g.
[row]
. - SELF_CLOSING: a self-closing token
[post id=1/]
. - CLOSE: a close token, e.g.
[/row]
. You will only see the left-over of these tokens in the AST as OPEN tokens that have theirisClosed
value set to true. - ERROR: when in non-strict mode offending tokens are converted to ERROR tokens and which behaves like TEXT nodes.
API
options.strict
Property By default strict-mode is enabled and syntax errors will be thrown. Setting strict-mode to false will convert all errors into ERROR nodes.
options.skipWhiteSpace
Property By default skipWhiteSpace is off. When turned on all whitespace is trimmed and if there is nothing left the TEXT node is skippid.
input()
constructor and You can pass input in the constructor you can set it later using input()
.
Examples:
let t = '[code][/code]'t tinput'[code][/code]'t t
tokens()
Returns all tokens.
ast([input])
Returns an AST created from the input.
buildTemplate([Token token], [object|string|null params=null], [bool deep=false])
Builds template on given token. You can overwrite params in root param by params
and also in all nested setting deep="true"
param.
Changelog
See CHANGELOG.md