repo-crawler-for-gpt

Inspire by gpt-crawler

This package will crawl source code and doc files from the Github repo and generate an output file that can be shoved into GPT for building a repo assistant. You can do all kind of things using this assistant:

• Write README document for the whole project.
• Find potential bugs.
• Analyze and learn the architecture.
• ...more

Here are some GPTs created to analyze code:

• react-virtuoso to analyze https://github.com/petyosi/react-virtuoso
• gpt-crawler to analyze https://github.com/BuilderIO/gpt-crawler

Github repo GPT Crawler

Crawl a Github repo to generate code files to create a custom GPT

• Example
• Get started
  • Install
  • Start to crawl
  • Upload your data to OpenAI
    • Create a custom GPT
    • Create a custom assistant
• Contributing

Example

Here is a custom GPT that was made to help answer questions about the repo gpt-crawler

Note that you may need a paid ChatGPT plan to access this feature

Get started

Direct Usage

npx repo-crawler-for-gpt

Install

npm i repo-crawler-for-gpt

Start to crawl

E.g. to crawl the source code of gpt-crawler to a custom GPT

const {crawlerGithubForGPT } = require("repo-crawler-for-gpt");

crawlerGithubForGPT({
  githubRepoUrl: 'https://github.com/BuilderIO/gpt-crawler',
  branch: 'main',
  // or
  tag: 'v1.0.0'
})

Upload your data to OpenAI

The crawler will generate a file called output.json at the root of this project. Upload that to OpenAI to create your custom assistant or custom GPT.

Create a custom GPT

Use this option for UI access to your generated knowledge that you can easily share with others

Note: you may need a paid ChatGPT plan to create and use custom GPTs right now

1. Go to https://chat.openai.com/
2. Click your name in the bottom left corner
3. Choose "My GPTs" in the menu
4. Choose "Create a GPT"
5. Choose "Configure"
6. Under "Knowledge" choose "Upload a file" and upload the file you generated
7. Use the prompt or your own prompt to create a assistant

specialized GPT for comprehensive analysis of GitHub projects through JSON files, interpreting 'html' as code and 'url' as file titles. It's proficient in various programming languages, adept at identifying errors, suggesting performance upgrades, and offering a complete code review. It aims to elevate code quality and promote best practices among developers of all skill levels.

Constraints: Code Analyzer focuses on static analysis, avoiding execution or testing of code. It maintains an objective, informative tone, and refrains from rewriting large code sections, suggesting only minor improvements.

Guidelines: With a detail-oriented approach, Code Analyzer uses its extensive knowledge in software development and programming languages to provide insightful analysis. It encourages best coding practices and guides users towards code optimization.

Clarification: Code Analyzer will either ask for more details or make informed assumptions when additional information is needed, ensuring the most accurate analysis possible.

Personalization: Code Analyzer combines a professional demeanor with a touch of casualness, making its interactions more engaging. Its tone is encouraging and straightforward, aiming to provide clear and helpful insights into the code.

Create a custom assistant

Use this option for API access to your generated knowledge that you can integrate into your product.

1. Go to https://platform.openai.com/assistants
2. Click "+ Create"
3. Choose "upload" and upload the file you generated