foxlens — see a page with a vision model автор Pooria

foxlens takes a screenshot of the current tab and asks a vision model about it. Use it when a page does not show its content in the DOM, for example a button drawn on a canvas or a button that is only an image.

Who it is for: people who build or try browser agents, and people who want a vision model to describe a page or find an element on it.

What it does

• "Describe this tab" takes a screenshot and asks the model what the page shows.
• "Find" asks the model where an element is, for example "the blue Subscribe button". foxlens maps the answer back to a real element and draws an outline around it.
• An optional numbered grid helps small models point at the right place.
• Each result says which model answered and whether the screenshot left this device.

What leaves your computer, and when

• The default model server is Ollama on this computer (127.0.0.1). The screenshot and your question go only to that server.
• Another server (off by default): if you enter a server address that is not on this computer, foxlens sends the screenshot and your question to it only after you tick "Send screenshots to this server" and Firefox's data consent for website content says yes.
• An Ollama model whose name ends in "-cloud" runs on Ollama's servers. foxlens treats it as a cloud model and refuses it unless you tick the same box.
• Firefox's on-device model (optional): the screenshot stays in Firefox. Firefox downloads the model from Mozilla's model hub the first time.
• foxlens has no server and sends nothing to its authors.

Permissions

• Access to all websites and scripting: take a screenshot of the tab you choose and find the element on that page.
• Storage: keep your server address, model name and the last result in your browser.
• Trial ML (optional, asked from a click): use Firefox's on-device image-to-text model.

foxlens is an early version. The source code is MIT licensed.