Yahoo! tries to edit out many of the "site in a box"-type affiliate sites as well as any content that does not add value to your search experience. Yahoo! does not hate affiliate websites, they hate unoriginal, replicated content, as it degrades the quality of their search results if a searcher sees the same things over and over again.
It does not hurt you to have affiliate links on your website as long as you also have some original content. It is also worth noting that some search engines may also have custom algorithms designed to detect and demote thin affiliate sites.