Using Google AppEngine for a Little Micro-Scalability
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Tue, 22 Apr. 2008
Over the years I've accumulated quite a rag tag collection of personal systems scattered wide across a galaxy of different servers. For the past month I've been on a quest to rationalize this conglomeration by moving everything to a managed service of one kind or another. The goal: lift a load of worry from my mind. I like to do my own stuff my self so I learn something and have control. Contr
readmore
View Item
The Search for the Source of Data - How SimpleDB Differs from a RDBMS
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Mon, 21 Apr. 2008
All your life you've used a relational database. At the tender age of five you banged out your first SQL query to track your allowance. Your RDBMS allegiance was just assumed, like your politics or religion would have been assumed 100 years ago. They now say--you know them--that relations won't scale and we have to do things differently. New databases like SimpleDB and readmore
View Item
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Mon, 21 Apr. 2008
All your life you've used a relational database. At the tender age of five you banged out your first SQL query to track your allowance. Your RDBMS allegiance was just assumed, like your politics or religion would have been assumed 100 years ago. They now say--you know them--that relations won't scale and we have to do things differently. New databases like SimpleDB and readmore
View Item
Scaling Mania at MySQL Conference 2008
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Fri, 18 Apr. 2008
The 2008 MySQL Conference & Expo has now closed, but what is still open for viewing is all the MySQL scaling knowledge that was shared. readmore
View Item
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Fri, 18 Apr. 2008
The 2008 MySQL Conference & Expo has now closed, but what is still open for viewing is all the MySQL scaling knowledge that was shared. readmore
View Item
HSCALE - Handling 200 Million Transactions Per Month Using Transparent Partitioning With MySQL Proxy
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Fri, 11 Apr. 2008
After writing Skype Plans for PostgreSQL to Scale to 1 Billion Users, which shows how Skype smartly uses a proxy architecture for scaling, I'm now seeing readmore
View Item
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Fri, 11 Apr. 2008
After writing Skype Plans for PostgreSQL to Scale to 1 Billion Users, which shows how Skype smartly uses a proxy architecture for scaling, I'm now seeing readmore
View Item
Google AppEngine - A First Look
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Tue, 08 Apr. 2008
I haven't developed an AppEngine application yet, I'm just taking a look around their documentation and seeing what stands out for me. It's not the much speculated super cluster VM. AppEngine is solidly grounded in code and structure. It reminds me a little of the guy who ran a readmore
View Item
Related
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Tue, 08 Apr. 2008
I haven't developed an AppEngine application yet, I'm just taking a look around their documentation and seeing what stands out for me. It's not the much speculated super cluster VM. AppEngine is solidly grounded in code and structure. It reminds me a little of the guy who ran a readmore
View Item
Related
Skype Plans for PostgreSQL to Scale to 1 Billion Users
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Sun, 06 Apr. 2008
Skype uses PostgreSQL as their backend database. PostgreSQL doesn't get enough run in the database world so I was excited to see how PostgreSQL is used "as the main DB for most of [Skype's] business needs." Their approach is to use a traditional stored procedure interface for accessing data and on top of that layer proxy servers which hash SQL requests to a set of database servers that actually readmore
View Item
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Sun, 06 Apr. 2008
Skype uses PostgreSQL as their backend database. PostgreSQL doesn't get enough run in the database world so I was excited to see how PostgreSQL is used "as the main DB for most of [Skype's] business needs." Their approach is to use a traditional stored procedure interface for accessing data and on top of that layer proxy servers which hash SQL requests to a set of database servers that actually readmore
View Item
Rumors of Signs and Portents Concerning Freeish Google Cloud
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Tue, 01 Apr. 2008
First Dave Winer hallucinates a pig on the mean streets of Walnut Creek that told him Google's long foretold cloud offering will be free for bloggers of "modest needs." GigaOM then says a free cloud service is how Google could eat Amazon's bacon for lunch readmore
View Item
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Tue, 01 Apr. 2008
First Dave Winer hallucinates a pig on the mean streets of Walnut Creek that told him Google's long foretold cloud offering will be free for bloggers of "modest needs." GigaOM then says a free cloud service is how Google could eat Amazon's bacon for lunch readmore
View Item
Heroku - Simultaneously Develop and Deploy Automatically Scalable Rails Applications in the Cloud
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Sun, 30 Mar. 2008
Adam Wiggins of Heroku presented at the lollapalooza that was theCloud Computing Demo Night. The idea behind Heroku is that you upload a Rails application into Heroku and it automatically deploys into readmore
View Item
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Sun, 30 Mar. 2008
Adam Wiggins of Heroku presented at the lollapalooza that was theCloud Computing Demo Night. The idea behind Heroku is that you upload a Rails application into Heroku and it automatically deploys into readmore
View Item
20 New Rules for Faster Web Pages
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Sat, 29 Mar. 2008
Update: Nice explanation in The importance of bandwidth versus latency of how long latencies cause cascading delays in resource loading. Doloto tries to optimize how resources are loaded. Twenty new rules have been added to th readmore
View Item
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Sat, 29 Mar. 2008
Update: Nice explanation in The importance of bandwidth versus latency of how long latencies cause cascading delays in resource loading. Doloto tries to optimize how resources are loaded. Twenty new rules have been added to th readmore
View Item
Amazon Announces Static IP Addresses and Multiple Datacenter Operation
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Thu, 27 Mar. 2008
Amazon is fixing two of their major problems: no static IP addresses and single datacenter operation. By adding these two new features developers can finally build a no apology system on Amazon. Before you always had to throw in an apology or two. No, we don't have low failover times because of the silly DNS games and unexceptionable DNS update and propagation times and no, we don't operate in readmore
View Item
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Thu, 27 Mar. 2008
Amazon is fixing two of their major problems: no static IP addresses and single datacenter operation. By adding these two new features developers can finally build a no apology system on Amazon. Before you always had to throw in an apology or two. No, we don't have low failover times because of the silly DNS games and unexceptionable DNS update and propagation times and no, we don't operate in readmore
View Item
Paper: On Designing and Deploying Internet-Scale Services
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Tue, 25 Mar. 2008
Greg Linden links to a heavily lesson ladened LISA 2007 paper titled On Designing and Deploying Internet-Scale Services by James Hamilton of the Windows Live Services Platform group. I know people crave nitty-gritty details, but this isn't a how to configure a web server article. It hitches you to a rock readmore
View Item
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Tue, 25 Mar. 2008
Greg Linden links to a heavily lesson ladened LISA 2007 paper titled On Designing and Deploying Internet-Scale Services by James Hamilton of the Windows Live Services Platform group. I know people crave nitty-gritty details, but this isn't a how to configure a web server article. It hitches you to a rock readmore
View Item
Paper: Consistent Hashing and Random Trees: Distributed Caching Protocols for Relieving Hot Spots on the World Wide Web
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Tue, 18 Mar. 2008
Consistent hashing is one of those ideas that really puts the science in computer science and reminds us why all those really smart people spend years slaving over algorithms. Consistent hashing is "a scheme that provides hash table functionality in a way that the addition or removal of one slot does not significantly change the mapping of keys to slots" and was originally a way of distributin readmore
View Item
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Tue, 18 Mar. 2008
Consistent hashing is one of those ideas that really puts the science in computer science and reminds us why all those really smart people spend years slaving over algorithms. Consistent hashing is "a scheme that provides hash table functionality in a way that the addition or removal of one slot does not significantly change the mapping of keys to slots" and was originally a way of distributin readmore
View Item
Product: GlusterFS
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Sun, 16 Mar. 2008
Adapted from their website: GlusterFS is a clustered file-system capable of scaling to several peta-bytes. It aggregates various storage bricks over Infiniband RDMA or TCP/IP interconnect into one large parallel network file system. Storage bricks can be made of any commodity hardware such as x86-64 server with SATA-II readmore
View Item
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Sun, 16 Mar. 2008
Adapted from their website: GlusterFS is a clustered file-system capable of scaling to several peta-bytes. It aggregates various storage bricks over Infiniband RDMA or TCP/IP interconnect into one large parallel network file system. Storage bricks can be made of any commodity hardware such as x86-64 server with SATA-II readmore
View Item
YouTube Architecture
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Thu, 13 Mar. 2008
Update: YouTube: The Platform. YouTube adds a new rich set of APIs in order to become your video platform leader--all for free. Upload, edit, watch, search, and comment on video from your own site without visiting YouTube. Compose your site internally from APIs because you'll need to expose them later anyway. readmore
View Item
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Thu, 13 Mar. 2008
Update: YouTube: The Platform. YouTube adds a new rich set of APIs in order to become your video platform leader--all for free. Upload, edit, watch, search, and comment on video from your own site without visiting YouTube. Compose your site internally from APIs because you'll need to expose them later anyway. readmore
View Item
Product: nginx
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Tue, 04 Mar. 2008
Update 4: Nginx better than Pound for load balancing. Pound spikes at 80% CPU, Nginx uses 3% and is easier to understand and better documented. Update 3: igvita.com combines two cool tools together for better performance in readmore
View Item
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Tue, 04 Mar. 2008
Update 4: Nginx better than Pound for load balancing. Pound spikes at 80% CPU, Nginx uses 3% and is easier to understand and better documented. Update 3: igvita.com combines two cool tools together for better performance in readmore
View Item
Friends for Sale Architecture - A 300 Million Page View/Month Facebook RoR App
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Sun, 17 Feb. 2008
In a short three months Friends for Sale (think Hot-or-Not with a market economy) grew to become a top 10 Facebook application handling 200 gorgeous requests per second and a stunning 300 million page views a month. They did all this using readmore
View Item
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Sun, 17 Feb. 2008
In a short three months Friends for Sale (think Hot-or-Not with a market economy) grew to become a top 10 Facebook application handling 200 gorgeous requests per second and a stunning 300 million page views a month. They did all this using readmore
View Item
How Rackspace Now Uses MapReduce and Hadoop to Query Terabytes of Data
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Wed, 30 Jan. 2008
How do you query hundreds of gigabytes of new data each day streaming in from over 600 hyperactive servers? If you think this sounds like the perfect battle ground for a head-to-head skirmish in the great MapReduce Versus readmore
View Item
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Wed, 30 Jan. 2008
How do you query hundreds of gigabytes of new data each day streaming in from over 600 hyperactive servers? If you think this sounds like the perfect battle ground for a head-to-head skirmish in the great MapReduce Versus readmore
View Item
On-Demand Infinitely Scalable Database Seed the Amazon EC2 Cloud
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Sun, 02 Sep. 2007
Amazon's EC2 sounds good, but how do you make use of all that throbbing CPU power? A few companies are stepping readmore
View Item
Todd Hoff via High Scalability - Building bigger, faster, more reliable websites. on Sun, 02 Sep. 2007
Amazon's EC2 sounds good, but how do you make use of all that throbbing CPU power? A few companies are stepping readmore
View Item
Popular This Week