ScreenScraping – Classic ASP Code I Can’t Bear to Throw Away

ScreenScraping – Classic ASP Code I Can’t Bear to Throw Away

Having written code for as long as I have, there’s quite a bit of “old” garbage that is completely useless today. Believe it or not, I still get search-engine hits on my original site (lippeatt.com) for older “classic” asp code examples, so I hate to throw anything away.

One gem that brings back fond memories is some old screen scraping utilities. The first time I ScreenScraped, it was using curl to gather the current weather and news from CompuServe and add it as “posts” to my News/Weather/Sports BBS. ScreenScraping at 1200bps — old school baby! 🙂 Back before server-side RSS feed parsers were common, I created a way to “search” a group of RSS feeds all at once. Yeah, Google Reader put an end to that need. 🙂

Regardless, here’s some of the code.

' * ==================================================================================================
'
' GETSCRAPEDPAGE
' Returns the HTML from a provided URL
' 
Function GetScrapedPage(sQString)

	' ---- Retrieve Info:
	Set xobj = CreateObject("SOFTWING.ASPtear")
	xobj.FollowRedirects = True
	xobj.ConnectionTimeout = 999
	On Error Resume Next
	' URL, action, payload, username, password
	strRetval = xobj.Retrieve(sQString, Request_GET, "", "", "")

	Set xobj = Nothing
	
	If strRetVal = "" Then
		strRetVal = "Error processing page: The information from the exteral location " & _
			"is not available."

	End If
        
        GetScrapedPage = strRetval

End Function
Parsing HTML Text – Removing Unneeded Tags (More Classic ASP)

Parsing HTML Text – Removing Unneeded Tags (More Classic ASP)

More classic ASP code that may help someone trying to repair a legacy system.

This little function was a nifty way to remove unneeded HTML tags from a string. Used this to clean up “screenscrapes” of reports generated in HTML before parsing the content.

' * ==============

Function RipTags(sHTML,sTagType)
	Dim iStart, sFront, sBack
	Do While inStr(1,sHTML,"<" & sTagType,1) > 0
		iStart = instr(1,sHTML,"<" & sTagType,1)
		If iStart > 0 Then
			sFront = left(sHTML,iStart-1)
			iStart = instr(iStart,sHTML,">",1)
			sBack = right(sHTML,len(sHTML)-iStart)
			sHTML = sFront & sBack
		End If
		iStart = instr(1,sHTML,"",1)
		If iStart > 0 Then 
			sFront = left(sHTML,iStart-1)
			iStart = instr(iStart,sHTML,">",1)
			sBack = right(sHTML,len(sHTML)-iStart)
			sHTML = sFront & sBack
		End If
	Loop
	RipTags = sHTML
End Function