RobotsRule Lab Lieshe מאת

Test pasted robots.txt rules against paths or URLs entered in the Firefox toolbar popup. Use it to check how a user-agent would be treated by the rules before publishing a robots file or when reviewing crawler access notes.