Programtipset: Ta hand om spam med SpamSieve

av Mikael Winterkvist | sep 22, 2023 | Tips & Råd | 0 Kommentarer

SpamSieve är enkelt att installera. De flesta e-postprogram har direktstöd för SpamSieve. Programmet uppdateras automatiskt om du ställer in det. Då uppdateras filter och regler i programmet. Du kan komplettera reglerna genom att lära SpamSieve genom att skilja mellan spam och e-post som inte är spam.

SpamSieve is a robust spam filter for major email clients that uses powerful Bayesian spam filtering.

SpamSieve understands what your spam looks like in order to block it all, but also learns what your legitimate messages look like to avoid confusion. SpamSieve only marks spam in your email client so you never lose any mail.

The following is a partial list of SpamSieve’s features, so that you can quickly see what it can do and how it’s unique:

• Powerful Bayesian spam filtering results in high accuracy and almost no false positives. It adapts to the mail that you receive to get even better
  More…

What’s new in version 3.0

Updated on Sep 22 2023

Highlights

• SpamSieve 3 is compatible with macOS 10.13 (High Sierra) through macOS 14 (Sonoma) and is recommended for all customers using those versions of macOS. SpamSieve 3 is required for customers using Apple Mail with Sonoma. We recommend updating SpamSieve before updating to Sonoma.
• When you launch SpamSieve 3, it will offer to automatically migrate all your old training data and accuracy statistics (so that you don’t need to retrain it), or you can choose to start fresh. Your SpamSieve 2 data is left untouched, so it’s possible to revert to the old version if necessary. The new version of the Apple Mail plug-in also works with SpamSieve 2. Training done with version 2 after updating to version 3 will stay with version 2 (and vice-versa).
• The Apple Mail integration has been completely rewritten for compatibility with macOS 14 Sonoma. On Sonoma, SpamSieve now uses a Mail extension rather than a Mail plug-in (as Sonoma no longer supports plug-ins). The extension needs to be enabled in Mail’s Settings ‣ Extensions window. It is no longer necessary to install any files or to give SpamSieve Full Disk Access. You will now train SpamSieve from the SpamSieve menu bar icon at the top of the screen or from SpamSieve’s Dock icon, rather than from the Message menu in Mail.
• If you had previously used SpamSieve with Apple Mail, the first time SpamSieve launches on Sonoma it will explain how to make the transition to using the Mail extension. The new setup is also described in the Setting Up Apple Mail section of the manual. With earlier versions of macOS, you can continue using the Mail plug-in, as before. With macOS 13 Ventura and macOS 12 Monterey, you can choose whether to use the Mail extension (which is simpler to set up) or the Mail plug-in (which is more customizable). You can click Uninstall Plug-In… (which will also offer to delete the old Mail rule) in Settings ‣ Apple Mail ‣ Setup if you plan to switch all your Macs to using the extension.
• The Microsoft Outlook support has been rewritten to be simpler and easier to set up. It is no longer necessary to install any script files. The Outlook Filter Mailboxes helper app is no longer needed. If you don’t store large amounts of Mail in your inbox, it’s no longer necessary to create any rules in Outlook.
• If you had previously used SpamSieve with Outlook, you can continue to use the existing scripts and Outlook Filter Mailboxes, or you can remove the Outlook Filter Mailboxes app and switch to the new setup by following either the Setting Up Outlook or Setting Up Outlook (Large Inboxes) instructions.
• SpamSieve’s e-mail parser has been rewritten in Swift for better accuracy, stability, and speed. SpamSieve is better able to understand messages that use newer e-mail formats, as well as malformed messages that don’t follow the specification. Where previous versions might get confused and give up (ignoring parts of the message) or even crash, the new parser engine is more resilient. Being able to read and understand more of the e-mail message increases the filtering accuracy, since it’s able to see spammy content that was invisible to SpamSieve 2, either because of bugs in the e-mail sending software or because spammers were deliberately trying to evade filters.
• The corpus, blocklist, and allowlist have been rewritten to use a new database format. This greatly reduces SpamSieve’s launch time and memory use, while enabling lots of new features.
• The text-based log file and history database have been replaced by a new log database and Log window, which is easier to browse and search. The log includes lots more information about each message to help you understand what SpamSieve did and why, and what you might want to do in response. It should be your first stop when troubleshooting.
• The Settings window has been redesigned, consolidating options and commands that were previously in other menus, scripts, or esoteric preferences. Many of the labels have been reworded for clarity, and there are now links to open relevant sections of the help for more detailed explanations.
• There is now a Show Dock icon setting to more easily hide SpamSieve’s Dock icon. With the Dock icon hidden, you can access all of SpamSieve’s windows and commands from the new SpamSieve menu bar icon. If both the Dock icon and menu bar icon are hidden, you can still access the settings by opening SpamSieve from your Applications folder when it’s already running.

Apple Mail

• Filtering is now controlled from within SpamSieve’s Settings window rather than from Mail’s settings. You can now choose to Enable spam filtering using Mail extension (for the newer, simpler setup) or Enable spam filtering using Mail plug-in (which works the same way as in SpamSieve 2). You can also select Disable spam filtering in Apple Mail to temporarily turn off filtering (without having to uninstall anything).
• The Settings window has new buttons to Install Plug-In or Uninstall Plug-In…. The latter will also uninstall any SpamSieve rules. To stop using the Mail extension, you can just select Disable spam filtering in Apple Mail, which removes SpamSieve’s launch agent. There’s nothing further to uninstall, aside from the app itself.
• There’s a new Filter Messages menu command to manually apply SpamSieve to the selected messages. This has a keyboard shortcut of Command-Control-L. Mail’s Apply Rulescommand has a keyboard shortcut of Command-Option-L, and it still works to apply the Mail rules, but it no longer applies SpamSieve when using the Mail extension (because there is no more SpamSieve rule).
• We have long recommended setting Mail to move the messages SpamSieve catches to the All Junk mailbox rather than to the Spam mailbox. This is still the case when using the Mail plug-in, and with the Mail extension the Spam mailbox is no longer an option. There remains a SpamSieve setting to control where spam messages move when you train them. If you had been using Spam, you may want to change this to Junk so that all your spam messages end up in the same place.
• SpamSieve normally moves spam messages to the Junk mailbox. When using the Mail extension, there’s a new Move it to the Trash if the spam score is at least option. This separates the more spammy messages (in Trash), which are less likely to be mistakes, from the less spammy ones (in Junk), so that you can concentrate your efforts on reviewing those. For example, if you set the score to 99, when you train a message as spam, its sender will go on SpamSieve’s blocklist, and future messages from the same name or address will go directly to the Trash.
• There’s a new Mark it as read setting to mark new messages as read if SpamSieve thinks they’re spam, e.g. so that you aren’t distracted by the unread count in Mail’s sidebar.
• The Color the background according to how spammy it is setting is now available in the Settings window (instead of via the Change Settings command). There’s now a legend so that you can see which spam levels the different colors correspond to.
• There’s a new Filter spam messages in other mailboxes option. This makes it possible for SpamSieve to filter messages that other rules (either in Mail or on your mail server) moved out of the inbox. This is especially important when using the Mail extension, because Mail only sends new messages to the extension if they remain in the inbox after processing your Mail rules. With the Mail plug-in, this was not a concern because the SpamSieve rule was at the top of the list, so it was applied first. However, even with the plug-in, Mail does not apply rules to messages that were moved out of the inbox by a server rule or by another device before they got to your Mac. With this option, it is now possible for SpamSieve to filter all new messages, no matter where they are.
• Filtering messages in other mailboxes is not immediate: SpamSieve needs to periodically check whether any new messages have arrived. If you receive lots of new messages and spend a lot of time in Mail, you may see a new message before SpamSieve does. It may not be clear whether a spam message is visible because SpamSieve made a mistake and thought it was good or simply because SpamSieve hadn’t had a chance to examine it yet. The new Add green flag to unread good messages option lets you choose to have SpamSieve mark messages that it has checked with a green flag. Messages with no flag are pending processing; they are not mistakes that need to be trained as spam.
• In rare cases, a Mail bug may prevent new messages from being sent to the SpamSieve extension for analysis. You can enable green flags for the inbox using the Including in the inbox option, and if this and the Log window show that SpamSieve is not seeing the inbox messages, you can add the inboxes to the Select Mailboxes to Filter… sheet to have SpamSieve filter them without relying on the Mail extension.
• You can now control from the Settings window whether messages that you train as spam are moved to the Junk, to the Trash mailbox, or to a custom mailbox; and whether they should be marked as read. You can also control whether messages trained as good are moved back to the inbox or marked as unread.
• SpamSieve now shows an error message and help link in the settings if it’s missing Automation access to control Apple Mail or Full Disk Access to use the Apple Mail plug-in.
• The new Setting the Junk Mailbox in Apple Mail section to the manual consolidates and updates several previous sections.
• The Apple Mail – Rescue Good Messages and Apple Mail – Remote Training scripts are still supported. They do not need to be updated to continue working with SpamSieve 3. They work best when SpamSieve is set to move trained messages to the Junk mailbox.

Outlook

• Filtering is now controlled by SpamSieve itself rather than using the Outlook Filter Mailboxes helper app. In SpamSieve’s Settings window, you can choose to Enable spam filtering for Inbox. This is a simpler setup that’s available for users who don’t keep lots of old messages in the inbox. You can also choose to Enable spam filtering for InboxSpamSieve, which uses Outlook rules to temporarily store messages in the InboxSpamSieve folder until they have been processed, as with previous versions of SpamSieve. You can also select Disable spam filtering in Outlook to temporarily turn off filtering or if you want to uninstall—there are no more extra files that need to be manually deleted.
• You can now train SpamSieve from the SpamSieve menu bar icon at the top of the screen or from SpamSieve’s Dock icon, rather than from the system script menu. The training commands now have keyboard shortcuts.
• There’s a new Filter Messages menu command to manually apply SpamSieve to the selected messages.
• You can now set in the Settings window how often SpamSieve checks for new messages to filter. There’s also a Filter Now button that shows whether filtering is currently in progress and lets you tell SpamSieve to filter right away.
• The Settings window now has a Mark less spammy messages as “Uncertain Junk” checkbox. There are also checkboxes to control whether messages trained as spam (or good) are moved to the Junk E-mail folder (or to the inbox) or marked as read (or as unread).
• SpamSieve still supports Outlook 2011. This works the same way as in SpamSieve 2, except that the scripts can now be installed directly from the Settings window, and there are links to the setup instructions right there.
• SpamSieve now shows an error message and help link in the settings if it’s missing Automation access to control Apple Mail or Outlook.

Other Settings

• The Filters and Training settings have been consolidated. There are now buttons to open the various SpamSieve windows directly from the Settings window, along with links to open the relevant help sections. The old Use Habeas Safelist, “ADV” messages are spam, and Encoded HTML mail is spam filters have been retired. The old Allow good/spam duplicates in corpus checkboxes were not very useful and have been removed.
• The Update Address Book “Me” Card command has been replaced with an Edit Addresses button, as it no longer worked due to privacy restrictions in macOS. This is useful in conjunction with the Catch spam sent from my own addresses setting (which used to be called Exclude my addresses).
• Notification sounds are now shown without filename extensions and are remembered even if extension hidden flag is changed.
• SpamSieve now supports Notification Center, with options for notifying for Good messages sent from Contacts, Other good messages, The number of new good messages, and Uncertain spam messages. There’s now a slider for controlling which messages are considered uncertain. The old support for Growl notifications has been retired.
• There’s a new Other Clients tab of the Settings, which offers installation buttons and links to the setup instructions for the other mail apps that SpamSieve supports.

Bayesian Filtering

• SpamSieve is better able to understand Emoji and Asian languages, so that runs of these characters are properly tokenized into words even when they are not separated by whitespace. This greatly improves the filtering accuracy for messages containing such characters.
• SpamSieve also has a better understanding of Unicode normalization and of certain special characters, which improves recognition of words that had been seen before, but in a slightly different form.
• Made various other changes to improve SpamSieve’s filtering accuracy.

Rules Filtering

• Normally, when you train a message as spam, SpamSieve will disable any matching allowlistrules; and when you train a message as good, SpamSieve will disable any matching blocklistrules. This prevents such “incorrect” rules from causing filtering mistakes for future messages. SpamSieve now lets you lock certain rules, which prevents them from being automatically disabled in this manner. This is useful when you are really sure that a rule does what you intend, for example if you want messages matching a certain rule to always be classified as good, even if some spam messages also match that rule.
• SpamSieve is much faster at matching messages against mailing list rules.
• Regex rules work better with Unicode text.
• Added logging to help detect slow regexes that are slowing down rule filtering.
• If you have a rule that has matched a lot of messages, SpamSieve no longer stops counting at 65,535 hits.

Corpus Window

• The Words tab of the Corpus window works much the same as before, but it’s much faster to open and to change the sorted column. A large corpus can take a while to reload after an update, and this now happens in the background, rather than blocking the user interface and filtering. The text is larger and easier to read.
• There are new tabs in the Corpus window where you can view the list of Good Messages and Spam Messages that SpamSieve has been trained with. In most cases, the full message data is not available for messages trained with SpamSieve 2, so in that case only placeholders will be shown. For messages trained with SpamSieve 3, you can view metadata about the message, as well as when and how it was trained. You can also view the message’s rendered content and its raw source and MIME structure.
• Messages can now be trained from the Corpus window. Normally, it’s better to train from within your mail client, but sometimes you can’t find a message in the mail client or it’s already been deleted, due to a client or server bug or another device moving the message to a local mailbox. Having the message available in the Corpus window lets you properly update SpamSieve’s training in such cases. Previously, the only way to fix an incorrect training if you no longer had the message was to reset the corpus. Also, you can view the lists of spam and good messages to make sure that SpamSieve has been trained properly, i.e. that there were no mistakes that you forgot to correct.
• You can flag messages of note to come back to them later.
• You can open SpamSieve’s copy of a message into your e-mail client to reply to it there or to copy it back to the mail server.
• You can drag and drop messages out of the corpus window to export them for backup or for reporting to customer support.
• Searches now support wildcards (* and ?), and you can also search for messages by their subject, address, or SpamSieve identifier.
• The Corpus window now supports secondary sorting.

Rules Windows

• The Blocklist and Allowlist windows work much the same as before, but they’re much faster to open, loading now happens in the background, and the text is larger and easier to read.
• The “whitelist” has been renamed to “allowlist,” as recommended by the IETF and Apple.
• There are new columns to show when a rule was last Modified and when it last Matched a message. The Hits column has been replaced with separate columns to show the number ofCorrect and Incorrect times it was used to classify a message. This can help you identify rules that are not working as expected.
• Secondary sorting is now supported. For example, to find allowlist rules that you might no longer need, you could sort by Date Created and then by Correct, to see all the rules with 0 hits ordered by how old they are.
• Editing a rule now takes place in a separate sheet, where there’s more room to see the different options and to edit the text. You can now see live whether the regex that you entered is valid. The sheet also shows information about when and how the rule was created and how it has been used.
• There’s a space to enter comments for each rule, e.g. to help you remember why you created a rule or how a regex was intended to work. You can search for rules by comment.
• You can also flag a rule that you’re observing or tweaking or that you’ve put on probation to easily find it again later.
• Rules with invalid regular expressions are shown in red. Some rules created with SpamSieve 2 may need to be updated, as the regex engine has changed from PCRE to ICU.
• Changes that you make to rules are now undoable.
• Searches now support wildcards (* and ?).

Log

• The new Log window shows all the message trainings and predictions, as before. They are now color-coded so that you can see SpamSieve’s accuracy: green means that it was correct, orange indicates a false negative, and red indicates a false positive.
• You can now view a message’s rendered content and its raw source and MIME structure, rather than just its metadata. There’s a new setting to control how long SpamSieve should store the full message data for old log entries, to prevent it from using too much storage.
• Log entries show more information about each message. You can see which mail client (and in some cases which account and mailbox) a message came from. You can see whether a server junk filter agreed with SpamSieve about whether or not a message is spam. It also shows more information about what SpamSieve did with the message and why. If a log message is for a training, and you’ve correcting SpamSieve, the training log entry will show you information right there about why SpamSieve made that mistake. You can also search the log for the message’s identifier to see all the log entries related to that message. The log will also try to alert you to unusual situations, e.g. if you trained a message that was not actually a SpamSieve mistake, or if the exact same message had been repeatedly classified or trained.
• The log also shows new types of entries. You can see changes that you or SpamSieve have made to rules. You can see errors reported by the mail client when SpamSieve asks it to do something—information that was previously only available in the system log.
• SpamSieve 2 recorded less extensive log and history information, but SpamSieve 3 imports it and updates it to the new format where possible. If you had enabled the experimental message backup feature in SpamSieve 2, to protect against the Mail data loss bug, any backed up messages will now be available in the Log window. Messages from the old Save false negatives to disk feature are also imported.
• As with the corpus, you can open messages from the log in Apple Mail and other mail clients or export them via drag and drop.
• If SpamSieve thinks that processing a particular message caused a crash, it will be shown in red in the log. You can drag the log entry into an e-mail to report the problematic data to us so that we can try to prevent the crash in the future.
• You can also see whether a particular message took an abnormally long time to process and report such problematic messages.
• The contents of the log are now localized.
• Log entries can now display context-specific links to help pages. For example, there is a particular link that’s only relevant when you are using Apple Mail and the Mail extension and you’ve trained a message in the inbox that SpamSieve had not seen before as spam.
• If you’ve enabled notifications for new messages, e.g. to see which messages SpamSieve is uncertain about, clicking the notification will find that message in the log.
• The log database is now stored as shards, so it works much better with backups (old shards rarely change and so won’t need to be recopied) and is more resilient (if one file becomes damaged, you don’t have to throw away the entire log).
• Message data is no longer stored in plaintext, to avoid problems with anti-virus software deleting data out from under SpamSieve, even though it was inert.
• Diagnostic reports now include a fixed number of recent log entries, whereas previously there could be too much or too little data depending on when the log file had last rolled over.

Statistics Window

• The accuracy statistics now ignore duplicate classifications of the same message.
• Calculating the corpus and rules statistics no longer blocks the user interface.
• Show Statistics Since now uses a date picker rather than a free-form text field in a separate sheet.

General

• SpamSieve has a new Dock icon. The top-left of the icon shows an envelope when SpamSieve is idle, a downward pointing arrow when SpamSieve is classifying a message, a check mark when it’s training a message as good, and an x mark when it’s training a message as spam.
• You can now open the Settings window from the Dock menu.
• The SpamSieve menu bar icon changes color while an operation initiated by the menu or a hotkey is in progress (blue when filtering messages, green when training as good, and brown when training as spam).
• AppleScript support for the corpus and rules has been rewritten to be faster and use less memory. You can now use considering case and ignoring case blocks when looking up words and rules.
• The Setting Up Airmail instructions now have screenshots.
• Each help page now has a link to a Google translation so you can view it in a different language.
• Removed support for PowerMail 4.x; versions 5.x and 6.x are still supported.
• Updated the Dutch, French, German, Japanese, and Spanish localizations.
• Removed the Danish, Italian, Korean, Russian, Swedish, and Vietnamese localizations, which are out-of-date. If you’d like to help update the localizations (or add a new one), please contact spamsieve@c-command.com.