Marktwin parser
This library implements the Marktwin (mkt) parser used by Eternal-Twin. Marktwin is the format used by Eternal-Twin for its messages. It is inspired by Markdown and the Twinoid forum code. See the Marktwin library for more details.
The parser is implemented using Rust and compiled to Wasm. It is compatible with both Node and webpack.
All inputs produce valid parse trees, parsing cannot fail.
Design
The parser uses a pipeline to gradually transform the input into a tree.
- Input: A string of valid unicode codepoints. It is up to the consumer to provide handle the encoding and provide this string.
- Context-free token stream: The lexer tokenizes the string only using the current index.
- Parse-event stream: Contains potentially remapped tokens end start/end events for nodes.
- Concrete Syntax Tree: Tree representation of the parse-event stream
- Abstract Syntax Tree: An adapter on top of the CST to provide easier access to the data.
Fuzzing
The Rust implementation supports fuzzing:
# Make sure that you have `cargo-fuzz`
cargo install cargo-fuzz
# Fuzz the `lexer`
cargo fuzz run lexer
# Fuzz the `parser`
cargo fuzz run parser