You are here: Home > Articles > Using Amazon S3, EC2, SQS, Lucene, and Ruby for Web Spidering
Using Amazon S3, EC2, SQS, Lucene, and Ruby for Web Spidering
by Robert Dempsey on February 4, 2008
My last article for the Amazon Web Services Developer Connection was a how-to on creating a web spider using Ruby, Rails, Amazon S3, Amazon EC2, Amazon SQS, and then making it all searchable using Lucene. If you are wondering how to make that happen, check out the article: Using Amazon S3, EC2, SQS, Lucene, and Ruby for Web Spidering.
Other Posts That Might Interest You
- Persistent storage coming to Amazon EC2
- Building a Product Catalog with Amazon SimpleDB
- Keep AWS Simple Storage Service (S3) costs low using Ruby on Rails
Tagged as:
amazon ec2,
amazon s3,
amazon sqs,
amazon web services,
Ruby,
ruby on rails,
spidering
