Robots Radar tekijä René Dhemant

Live robots.txt analyser. Instantly see if Googlebot, Bingbot or CCBot can crawl the current page — animated colour-coded icon in your address bar on every navigation. No clicking needed.

0 (0 arviota)

Lataa Firefox ja hanki laajennus

Lataa tiedosto

Tietoja tästä laajennuksesta

Robots Radar analyses your site's /robots.txt in real time and shows a colour-coded icon directly inside your address bar — no toolbar button, no clicking, instant feedback on every page load.

What the icon means

🟢 Green ✓ — the current path is allowed for crawling
🔴 Red ✗ — the current path is disallowed
🟢 Teal ✓ — no robots.txt found, all paths open
📡 Animated radar sweep — shown while fetching (looks great)
⚫ Grey ? — robots.txt could not be fetched

Specification-compliant parsing

Robots Radar implements Google's exact robots.txt specification (RFC 9309):

Longest path specificity — more specific rules win
Allow beats Disallow on equal path length
Multiple groups for the same agent are merged
Agent normalisation: Googlebot/2.1 and googlebot* both match correctly
Relaxed blank-line handling (matches Googlebot's documented behaviour)
Correct wildcard matching: * and $ anchors

Bot priority order

Rules are evaluated in this order, falling back to the next if no explicit group is found:

Googlebot → Bingbot → CCBot → Google-Extended → * wildcard

Features

Animated radar sweep icon during every page scan
Dark mode and Light mode with a one-click toggle — preference remembered
Per-domain caching (1 hour TTL) — no redundant fetches on the same domain
Works in private / incognito windows
Click the address bar icon for a full breakdown:
- Which bot group was matched (explicit vs wildcard fallback)
- The single winning rule with type badge (Allow / Disallow)
- Specificity bar showing how precise the match was
- Syntax-highlighted robots.txt source viewer
Refresh button to bust the cache and re-fetch live

Who is this for?

SEO professionals, web developers, site owners, and anyone curious about how search engines see their pages. Robots Radar turns robots.txt from a text file you forget to check into a live dashboard that follows you as you browse.

Privacy

Robots Radar only contacts the domain you are currently visiting, solely to fetch its public /robots.txt file. No data is collected, stored remotely, or shared with anyone. There are no analytics, no tracking, no third-party requests of any kind.

Open source

The full source code is available on GitHub. All code is plain, readable JavaScript — no bundlers, no minification, no obfuscation.

Kehittäjän kommentit

Robots Radar is a robots.txt analyser that shows a colour-coded icon in the Firefox address bar (page_action) on every page load.

How it Works

background.js fetches /robots.txt via the background script (bypasses CORS) and caches the result per domain for 1 hour.
The parser implements Google's exact robots.txt spec (RFC 9309): longest-path wins, Allow beats Disallow on equal length, group merging, correct wildcard (* and $) matching, relaxed blank-line handling.
Bot evaluation order: Googlebot → Bingbot → CCBot → Google-Extended → *
Results are drawn as a colour-coded OffscreenCanvas icon set via browser.pageAction.setIcon(). An animated radar sweep plays during fetch.

Permissions Justification

activeTab: read the current tab URL to evaluate the path
tabs: listen for tab switches so the icon updates when switching tabs
webNavigation: trigger analysis on every navigation without user interaction
<all_urls>: fetch /robots.txt from the visited domain only

Privacy

No user data is collected or transmitted. The only outbound request is a GET to the current domain's /robots.txt — a public resource. No analytics, no remote storage, no third-party requests.

Arvio 0 0 arvostelijan toimesta

Robots Radar tekijä René Dhemant

Vaaditut käyttöoikeudet:

Tiedonkeruu: