Yes I agree with Jason for quality ranking, linking to relevant websites is very important however, there is no disadvantage if you link to an unrelated websites.
Why would anyone want to link to unrelated websites when it would be of no use? Not sure about its effects from SE's perspective.But logically it should not have any adverse effect on one's website.
Most important is that you shouldn't loose the theme of your website. For e.g. If you are a service provider, links that mention more in relation to drugs and explicit content doesn't work and is definitely considered as spam.
__________________ Redundant Dedicated Server Hosting Solutions Only at BODHost
24x7 Toll-Free ph. : +1. 866-662-0909
Email : sales@bodhost.com | MSN : sales@bodhost.com
Yes.... it definitely increases spam and it can harm your site PR if its linked to irrelevant sites. I always link to sites in the same niche. 1 relevant link is worth more than 10 irrelevant links. Decrease spam by drawing relevant sites to your niche, the traffic conversion will be better too.