The Importance of Non-Reciprocal Links
When conducting Search Engine Optimization it is important that your links come from a variety of sources and that these links be non-reciprocal (meaning your site isn't linking back to that site as well).
Think of your site as starting off with 0 links going to it and each link is worth 1 point. (This is just a metaphor, the exact algorithm Google uses to determine search ranking is too complex to explain and is always being updated.)
#1. For every site linking in to yours you get a point, but if you link back to the site linking to you that point becomes useless.
#2. If a website is more popular its worth extra points. See the PageRank Chart to the lower right for details. A PageRank 0 site is worth very little (almost nothing), but the higher the PageRank the site is the links coming from the linking site are much more valuable. ie. A PageRank 3 link is worth approximately fifteen PageRank 1 links (or 75 PageRank 0 links).
#3. The types of links are important. If they are larger fonts, bolded or headers they are worth extra.
#4. The type of page matters. A blog, educational or government website is worth more.
#5. Where the link is on the page matters. Higher up on the page, inside the content is best.
#6. Local / keyword accurate websites are best for linking in from. ie. If you're selling real estate in Toronto you want links from websites talking about Toronto real estate.
#7. The number of links matters. If a site linking to you has a tonne of links on it the value of those links are diluted.
However lets pretend for a moment that you link out to other websites, or worse, link to the websites which are linking to you (sometimes known as "link exchanges")... well then all your incoming links will lose their value.
Even three-way link exchanges are bad. They're borderline Black Hat SEO. Any outgoing link is going to hurt your site so the only sites you should be linking to should be sites that are actually important to your business (ie. the company Twitter page or company Facebook profile).
The end result is that you want lots of quality links going to your website and very few outgoing links (or no outgoing links if possible).
There is however a solution if you want to have outgoing links which don't damage your SEO potential. Use rel="nofollow" in the link coding. Using rel="nofollow" you can link to friends / colleagues websites without fear of crippling your site.
This way you can have your SEO cake and eat it too.