ADS specializes in using Ruby on Rails to build advanced, scalable, database-backed web sites for organizations of all sizes. Find out more at our website.

Sep: 28; 2007

Use what you have got and you won’t need what you have not

By: Robert Dempsey | Tags:

That phrase is something I heard on a cartoon when I was a kid. It is still relevant today, and something that Brion Vibber can relate to. Last night at the Orlando Java Users Group it was standing room only to hear the Wikipedia CTO talk.

Wikipedia is a incorporated, 501(c)(3) not for profit organization, that runs one of the most popular websites on the Internet, with an average of 32,000 hits per second. They run this incredible site with 300 hundred servers, 3 data centers, a technical staff of four (that’s right, four), and the open source LAMP stack. At the core of Wikipedia is MediaWiki, an open source package you can use for your projects. Add to that Lucene, a powerful Java based search engine (that we Rails folks can utilize too), a number of caching packages including Squid, APC and memcache, MySQL replication, geographic DNS and a whole lot of technical expertise and voila, Wikipedia.

Frankly, I was blown away by the sheer scale of it all, and their total utilization of every machine under their control, all with a $1 million per year budget. Nothing sits idle. Web servers are also memcache servers, and storage is distributed throughout the farm. A few tricks that I picked up were:

Applying schema updates to the slave databases first, and then swap out the masters
Divide database servers into groups
Split data along functional boundaries (they use language)

It was a great time. A big thanks goes out to Mike Levin for brining Brion over for an excellent talk.

Share this post