📄

Robotstxt-validator

Technical SEO

✓ Verified

👁️ 2 views

Fast library to check robots.txt compliance with Google standards

Visit Tool →

💰 Pricing

Open-source

Billing: one-time

💻 Platform

cli

👤 Target User

expert

Company: agency,enterprise

📜 License

Apache

🔌 Integrations

python,cpp

🔄 Last Update

8/15/2025

⭐ 1,800

Stars

👥 5,000

Users

⭐ 4.4

Rating

🏷️ Tags

robots-txt crawler-compliance google-standards technical-seo

Similar Tools & Alternatives

View All (236) →

📦

Puppeteer

Node library to control headless Chrome for complex SPA SEO rendering tests

🛠️

Hoppscotch

Open-source API development ecosystem used by SEOs to test and debug headless CMS endpoints and schema APIs

🕸️

Scrapy

Powerful open-source web crawling framework for Python developers to build custom SEO scrapers

📦

Meilisearch

Lightning fast open-source search engine API to improve site-search SEO and UX

🚀

Astro

The web framework for content-driven websites with built-in SEO optimizations and API routes

🏗️

Workbox

JavaScript libraries for adding offline support and power to Progressive Web Apps

← Back to list