bulkai

BulkAI: Revolutionize Your Markdown Processing with the Power of OpenAI

Are you tired of manually processing large sets of markdown files, or struggling to find the time to enhance your content with the latest AI technology? BulkAI is here to change the game. With just a few simple commands, you can transform how you manage and optimize your markdown and text files, all while harnessing the incredible power of OpenAI's GPT-4. Whether you're looking to rewrite entire websites, translate books, or simply batch-process content with custom formatting, BulkAI makes it easier than ever.

Don't wait—experience the difference BulkAI can make in your workflow today!

Usage

No installation required! You can run BulkAI directly using npx:

npx bulkai -i input-dir -o output-dir [options]

Or like:

npx bulkai -p emma,about.md,./semantics.md,./goals.md,---,article -s rewrite,shock,res -i ./input -o ./output

Environment Variables

You can configure BulkAI using environment variables:

• OPENAI_API_KEY: Your OpenAI API key (required)
• DEFAULT_MODEL: The default OpenAI model to use (default: "gpt-4o-mini")

Example:

export OPENAI_API_KEY="your-api-key"
export DEFAULT_MODEL="gpt-4"
npx bulkai -p emma,rewrite -i ./input -o ./output

Example Command

npx bulkai -p prefix.md -s suffix.md -i ./input -o ./output -f -H -e .md,.txt

Command-Line Options

• -p, --prefix-file <path>: Path to the file containing the prefix to add to each file (e.g., prefix.md).
• -s, --suffix-file <path>: Path to the file containing the suffix to add to each file (e.g., suffix.md).
• -i, --input-dir <path>: Path to the input directory containing files to process.
• -o, --output-dir <path>: Path to the output directory where processed files will be saved.
• -f, --force: Force overwrite of existing files in the output directory.
• -H, --hugo: Enable Hugo front matter processing.
• -e, --extensions <extensions>: Comma-separated list of file extensions to process (default: .md,.txt).
• -x, --excluded <parts>: Comma-separated list of path parts to exclude from processing (e.g., /.,_private).

Introduction

BulkAI is a powerful Node.js CLI tool designed to automate the processing of markdown and text files using OpenAI's GPT-4. Ideal for developers, content creators, and technical writers, BulkAI allows you to efficiently batch-process your .md and .txt files, adding custom prefixes and suffixes, and even handling Hugo front matter with ease. Whether you're preparing large sets of documents, rewriting entire websites, or translating books, BulkAI simplifies the workflow and maximizes productivity.

Features

🔄 Batch Processing of Markdown and Text Files

BulkAI enables the bulk processing of .md and .txt files, making it perfect for large-scale content management tasks. Simply point the tool to your input directory, and it will recursively scan and process all specified file types.

🎨 Custom Prefix and Suffix Support

Easily add custom prefixes and suffixes to your content before sending it to the OpenAI API. This feature is particularly useful for standardizing content, adding disclaimers, or appending signatures. For example, you can use prefix.md and suffix.md files to define the content you want to prepend or append to your documents.

You can specify multiple presets and files by comma-separating them:

npx bulkai -p emma,rewrite,./tmp/file.md -s article,hugo -i ./input -o ./output

The tool will:

1. First look for presets in the presets directory (e.g., emma.md, rewrite.md, hugo.md)
2. Then try to find files at the specified paths (e.g., ./tmp/file.md)
3. Combine all found content in the order specified

Available presets:

• emma: Emma's writing style and personality
• rewrite: Content rewriting instructions
• hugo: Hugo front matter processing
• article: Article formatting
• ---: Front matter separator
• raw: Raw content processing
• res: Response formatting
• shock: Shock content style

🔤 Template Variables

BulkAI supports template variables in your presets and files. Variables are replaced with actual values before processing. The syntax uses double curly braces: {{VARIABLE_NAME}}.

Available variables:

• {{DATE}} - Current date in YYYY-MM-DD format
• {{DATETIME}} - Current date and time in ISO format
• {{VARIABLE_NAME}} - Environment variable (looks for T_VARIABLE_NAME)

Example usage:

# Set environment variable
export T_SITE_NAME="My Awesome Site"

# Use in a preset or file
# Content: "Welcome to {{SITE_NAME}} - Generated on {{DATE}}"
# Will become: "Welcome to My Awesome Site - Generated on 2024-04-19"

Template variables are replaced in:

1. Preset files
2. User-provided files
3. Input content
4. Final output content

📚 Multiple File Generation Mode

BulkAI can generate multiple files from a single input using the -m or --multiple flag. In this mode, the tool:

1. Takes input from the -p parameter (presets or files)
2. Sends the combined content to OpenAI
3. Expects a JSON response with file paths as keys and content as values
4. Creates all specified files in the output directory

Example usage:

npx bulkai -p emma,./book.md -o ./output -m

The OpenAI response should be a JSON object in this format:

{
    "chapter1.md": "Content for chapter 1...",
    "chapter2.md": "Content for chapter 2...",
    "chapter3.md": "Content for chapter 3..."
}

The tool will:

1. Parse the JSON response
2. Create each file in the specified output directory
3. Handle markdown code blocks and structured data automatically
4. Convert structured data (like instruction/plot/notes) into markdown format

This mode is perfect for:

• Generating book chapters
• Creating multiple related documents
• Building structured content hierarchies
• Generating documentation sets

🚀 OpenAI GPT-4 Integration

Leverage the power of OpenAI's GPT-4 to enhance your content. BulkAI sends your processed content to the OpenAI API, where it can be improved, summarized, or expanded upon, based on your needs. This capability is powerful enough to rewrite entire websites or even translate entire books into different languages, making BulkAI a versatile tool for various content processing tasks.

🛠️ Hugo Front Matter Handling

With the --hugo option, BulkAI automatically detects and handles Hugo front matter in your markdown files. If the AI response contains Hugo front matter (denoted by ---), the tool will strip any content before the first occurrence, ensuring clean, ready-to-use output for your Hugo site.

📝 Flexible File Extension Support

BulkAI allows you to specify which file extensions to process using the --extensions option. By default, it processes .md and .txt files, but you can customize this to fit your specific needs.

💾 Force Overwrite Option

The --force option ensures that processed files are always written to the output directory, even if they already exist. This is useful when you need to re-process files without manually deleting old versions.

🔍 SEO-Friendly Content Processing

Designed with SEO in mind, BulkAI helps you optimize your markdown content for better search engine visibility. By integrating GPT-4's advanced language model, you can generate more engaging, keyword-rich content that resonates with both readers and search engines.

Benefits

• Boost Productivity: Automate repetitive content enhancement tasks, saving time and reducing manual effort.
• Enhance Content Quality: Use OpenAI's GPT-4 to improve the readability, engagement, and SEO performance of your markdown files.
• Flexible and Customizable: Tailor the processing to your specific workflow needs, from file handling to content customization.

License

BulkAI is licensed under the MIT License.