There are two sets of functionality that people refer to as anti spam. The first is the content filtering rules in R6 which I have (very briefly) described here

The second (and most powerful) is the DNSRBL functionality which we have used since PR1 and which works very well indeed. The trick is to use the right block lists.
We use:
- taiwan.blackholes.us; (a small number of rejections but all spam; we get no real email from Taiwan)
- dun.dnsrbl.net; (a partial list of IPs known to be allocated to dial up or dynamic hosts - should never be used for real email but are often used by spammers for direct-to-MX spam runs)
- relays.ordb.org; ( a good list of open relays)
- relays.visi.com; (another good list of open relays)
- list.dsbl.org; (yet another, heir to ORBZ; also includes open SOCKS proxies)
- opm.blitzed.org; (a list of open proxies)
- sbl.spamhaus.org; (a list of known spamhausen)
- bl.spamcop.net (don't know how to describe this, but it is hugely powerful - IPs get listed when spam starts to flow, regardless of whether they are open relays, proxies or anything else - they then get unlisted shortly after the spam stops. You do get some collateral kills with this but in our experience not too many; usually when a spammer is using a dialup and the mail is relayed via a smart host operated by the provider of that dialup which is also the source of legitimate mail - and yes, the spamcop DNSRBL is free - all of these lists are free)
- We also have a local list of domains from which we will not accept mail (including our own domains; by definition any third party that attempts to deliver mail to me "from" my own domain is lying - rule #1 in action)
- and a local list of IPs from which we accept no mail (sources of spam that remain unlisted by any blocklist for mysterious reasons).
I counted the ratio of mail rejected at source to mail delivered in one recent week and found that we rejected 40% of all inbound Internet mail. During that week, I had precisely no (zero, zip, zilch) complaints about legitimate mail being blocked.
There's a lot of spam out there people. These new functions in the Domino server turned up just in the nick of time...