DEFCON 17: Screen Scraper Tricks: Extracting Data from Difficult Websites
iSpeech
Speaker: Michael Schrenk
Screen scrapers and data mining bots often encounter problems when extracting data from modern websites. Obstacles like AJAX discourage many bot writers from completing screen scraping projects. The good news is that you can overcome most challenges if you learn a few tricks.
This session describes the (sometimes mind numbing) roadblocks that can come between you and your ability to apply a screen scraper to a website. You'll discover simple techniques for extracting data from websites that freely employ DHTML, AJAX, complex cookie management as well as other techniques. Additionally, you will also learn how "agencies" create large scale CAPTCHA solutions.
All the tools discussed in this talk are available for free, offer complete customization and run on multiple platforms.
For more information visit: http://bit.ly/defcon17_information
To download the video visit: http://bit.ly/defcon17_videos
2011-01-15 22:13:57
source
Gloss