I see this happening all of the time with company sites.  Not only do they have no handling of "404 page not found", if they do handle it there is no tracking of what page the visitor was trying to reach.So let's think about this.  Someone has come to your site through a bad link and here's the results.

1. They get peeved by the bad link and abandon your site.  You just lost money.
2. You even don't know they came and went.
3. You don't know where the bad link is out on the web.  So you can't fix it.

Now here are the solutions:

1. Oh Snap!  Create a custom 404 page according to some best practices out there.  Each time I do this I look up great examples and borrow their ideas.
   • Smashing Magazine
   • CSS Tricks
2. Track hits to your 404 page.  This will show you which pages are not being found as well as show you where the links are coming from.  Have these reports sent to you on a daily basis.  Else you lose more traffic.
   • Google Analytics has a simple addition to the JavaScript tracking code on the 404 page.
   • Omniture SiteCatalyst: in the code of your 404 page, leave s.pageName="" blank and set s.pageType="errorPage".  This will create a custom entry into your "Pages Not Found" report in SiteCatalyst.
3. Once you know which pages are being requested but not found, create a 301 redirect to the closest matching page you have.  Don't give them options if you know where they should be going.  If you're not sure where they should go (you are the webmaster, aren't you?), then send them to a tables of contents page.  Not the sitemap.html or sitemap.xml page, but rather a page with a few targeted pages that will help them decided.
4. Contact the owner of the inbound links and fix the problem.  Sometimes it's as simple as claiming your business on a review page, other times it's going to be harder because they're in someone's blog comments.  At least you'll have the 301 to back you up.

Don't be blind to this visitor loss.  Everyone has bad inbound links.  Fix yours or lose money.