ScreenScraping - Classic ASP Code I Can't Bear to Throw Away
Having written code for as long as I have, there's quite a bit of "old" garbage that is completely useless today. Believe it or not, I still get search-engine hits on my original site (lippeatt.com) for older "classic" asp code examples, so I hate to throw anything away.
One gem that brings back fond memories is some old screen scraping utilities. The first time I ScreenScraped, it was using curl to gather the current weather and news from CompuServe and add it as "posts" to my News/Weather/Sports BBS. ScreenScraping at 1200bps -- old school baby! :) Back before server-side RSS feed parsers were common, I created a way to "search" a group of RSS feeds all at once. Yeah, Google Reader put an end to that need. :)
Regardless, here's some of the code.
' * ================================================================================================== ' ' GETSCRAPEDPAGE ' Returns the HTML from a provided URL ' Function GetScrapedPage(sQString) ' ---- Retrieve Info: Set xobj = CreateObject("SOFTWING.ASPtear") xobj.FollowRedirects = True xobj.ConnectionTimeout = 999 On Error Resume Next ' URL, action, payload, username, password strRetval = xobj.Retrieve(sQString, Request_GET, "", "", "") Set xobj = Nothing If strRetVal = "" Then strRetVal = "Error processing page: The information from the exteral location " & _ "is not available." End If GetScrapedPage = strRetval End Function