These days every company conducting business on the Internet must be vigilant to stave off the endless onslaught of viruses and other malware. Attackers who use software programs to make unauthorized intrusions are eager to steal website content and gain access to the content and databases behind each website. Then these thieves use the stolen data for their own purposes or sell it for a lofty price.
We have seen a number of solutions entering this space. The latest solution to protect websites from these assailants is ScrapeDefender, which launched its cloud-based anti-scraping and monitoring service. This service, also called ScrapeDefender, uses anti-intrusion algorithms and patented technology to analyze network activity. The service immediately informs clients about suspicious scraping activity on the customer websites.
Screen scraping is a common practice in many industries, including the travel industry, where competitive pricing information is sought and has led to battles between the airlines and travel websites.
This practice has many legitimate uses and there are many tools to help developers gather and curate information, such as that as Connotate. According to an article written by Rami Essaid, CEO of Distil Networks:
“In 2009, Facebook won one of the first copyright suits against a web screen scraper. This laid the groundwork for numerous lawsuits that tied any web scraping with a direct copyright violation and very clear monetary damages. The most recent case was AP v Meltwater , where the courts stripped away what is referred to as fair use on the Internet. Previously, for academic, personal, or information aggregation people could rely on fair use and use web scrapers. In the Meltwater case, the court gutted the fair use clause that companies had used to defend web scraping. The court determined that even small percentages, sometimes as little as 4.5 percent of the content, are significant enough to not fall under fair use.”
Robert Kane, ScrapeDefender’s CEO, told securitycurrent that ScrapeDefender is the “only solution on the market” that scans monitors and protects websites against suspicious scraping activity. “Before ScrapeDefender, there was no one comprehensive solution for companies on the Internet to thwart intruders and monitor their websites for bots that are continuously trying to plunder websites,” Kane added. Despite his claims, there are many traditional defenses against website abuse such as Imperva’s Web Application Firewall, various defenses against automated logins such as nuCaptcha, and although it does not market itself as anti-screen scraping, Shape Security.
Although Kane declined to identify their beta testers, he said ScrapeDefender worked with companies in the airlines industry, consumer electronics and financial markets. The service is available now as a managed service by ScrapeDefender, or the customer can take control of the service.
The chief executive estimates that losses from web scraping and content theft are close to $5 billion annually. According to a recent industry study by Incapsula, malicious non-human-based bot traffic now represents more than 50 percent of all website visits.
“This nasty business of screen-scraping and unauthorized use of corporate data behind websites is growing at an alarming rate,” Kane said.
While it is not always nasty, defending against theft and re-use of valuable data can be invaluable to many website operators.
Gail Bronson is an accomplished technology journalist and security start-up entrepreneur. She was the Founding Managing Editor of Bloomberg and the Founding Editor of Forbes Science & Technology section and she held stints at other publications including U.S. News & World Report and Internet Week.