Generate 18 AI readiness files (llms.txt, ai.txt, RAG indexes, schema) for any website.
Install
KitScout test notes
Not tested by KitScout yet. When our harness runs it, pros, cons and the tasks we tried appear here, with the date and evidence.
Similar tools
- webclaw 2,440 ★
Web scraping, search, crawling, structured extraction and research for AI agents. - WebReaper 152 ★
AI-native web scraper: scrape, crawl and map any site to clean markdown over stdio. MIT-licensed. - Xtracticle 17 ★
Read public X (Twitter) posts, threads and X Articles as clean Markdown. Read-only, no auth. - ai.smithery/222wcnm-bilistalkermcp 11 ★
Track Bilibili creators and get the latest updates on videos, dynamics, and articles. Fetch user p… - Octocrawl 6 ★
Pages, crawls and URL batches as clean Markdown and JSON for AI agents, with an Evidence Record.