Search results

sitemap-ts

Sitemap generator for TypeScript projects

• Sitemap
• Typescript
• Generator
• Robots
• Sitemap.xml
• Robots.txt

jbaubree

• 1.8.0 • 9 months ago • 0 dependents • MITpublished version 1.8.0, 9 months ago0 dependents licensed under $MIT

microfrontier

A web crawler frontier implementation in TypeScript backed by Redis. MicroFrontier is a scalable and distributed frontier implemented through Redis Queues.

• Web Crawler
• Frontier
• Search Engine
• URL
• Redis
• Queue
• Spider
• Robots.txt

adileo

• 1.0.2 • 3 years ago • 0 dependents • ISCpublished version 1.0.2, 3 years ago0 dependents licensed under $ISC

sitemap-ts-advanced

Sitemap generator for TypeScript projects

• Sitemap
• Typescript
• Generator
• Robots
• Sitemap.xml
• Robots.txt

manuelmeister

• 0.2.0 • 3 years ago • 0 dependents • MITpublished version 0.2.0, 3 years ago0 dependents licensed under $MIT

norobots

Robots.txt rules

• Robots.txt
• node.js
• Web
• crawler
• bot

wdfang

• 0.1.0 • 9 years ago • 0 dependents • MITpublished version 0.1.0, 9 years ago0 dependents licensed under $MIT