RobotsRule Lab por Lieshe

Test pasted robots.txt rules against manually entered paths and URLs.

Acerca desta extensão

Test pasted robots.txt rules against paths or URLs entered in the Firefox toolbar popup. Use it to check how a user-agent would be treated by the rules before publishing a robots file or when reviewing crawler access notes.