Hi there! Some permissions are a very strange thing in SSS and they lead to doubt from users (understandably!). :) The main culprits are the permissions for downloads and web navigation, which are both related to very simple things that have almost nothing to do with downloads or web navigation.
The first one, from your question, is needed simply for saving/reading your SSS preferences to/from a file on your computer (at your request by clicking a button in the options page). This is in the domain of "downloads" and needs this permission.
The web navigation permission (you didn't ask but it's still important to make clear) was needed to fix a problem with the popup not working in pages with nested frames. Unfortunately the only way I could fix that problem requires the permission to detect when a page/frame finishes loading. This link has more information: https://github.com/CanisLupus/swift-selection-search/issues/69
I hope that makes it clear. :) Rest assured that SSS does not otherwise read or modify your downloads, does not care about what sites you use, and does not send your data anywhere (except perhaps in the future sending your preferences to Firefox Sync, but still only if you wish to do that).