When a user agent (usually a human using a web browser) surfs from one page to the next, the server serves pages the user agent requests. As part of every request, user agents usually supply the referring URL – the page you’re coming from.

Some wikis and blogs use this to automatically link back to pages. See AutomaticLinkBack and BidirectionalLink.

This can be abused by spammers: They use a piece of software that visits your site and provides faked referring URLs: Following these URLs will land you on pages talking about naked women, small penises and mortages. This “referrer spam” is one small part of WikiSpam.

Ways out: Reduce your costs, increase their costs.

  1. Don’t link to referrers, use some other technology. – Useless advice, we can do better than that. The cheapest way out.
  2. Check for existing links: Before adding a referring URL to you list of links to link back to, verify that the other site is in fact linking to you. Watch out for delays: Consider using very short timeouts for this check. If spammers have to link to all the sites they spam, this will cost the space and verifications will cost them bandwidth.
  3. Expire existing link backs, thus only link back to sites whose links are actually being followed. If the spammers use bots to keep links alive, this will cost them CPU cycles and bandwidth.
  4. Use blacklists to filter out spammers. This is rather ineffective against dedicated spammers, but helps weed out false positives due to search engines, for example.

EditNearLinks: BidirectionalLink AutomaticLinkBack