How to build a Scalable Crawler on the cloud, that can mine thousands of data points, costing less than a Tim Hortons box of Timbits a month.

Motivation and some context As someone who just moved from Rio de Janeiro (Brazil) all the way to Vancouver (Canada), the first thing that hits you right in the face (aside from the beautiful scenery and the Tim Hortons) are the rental prices. As some of you may know, Vancouver is currently constantly ranked among…Read more How to build a Scalable Crawler on the cloud, that can mine thousands of data points, costing less than a Tim Hortons box of Timbits a month.

Advertisements

Living in the Cloud

What is the cloud exactly? The first thing you should understand about the cloud is that it is not a physical thing. The cloud is a network of servers, and each server has a different function. Some servers use computing power to run applications or "deliver a service." Other servers in the network are responsible…Read more Living in the Cloud