Comments on: Scaling out 37 Signal-style applications is convenient http://almaer.com/blog/scaling-out-37-signal-style-applications-is-convenient blogging about life, the universe, and everything tech Sat, 08 Sep 2012 07:06:53 -0700 http://wordpress.org/?v=2.8.4 hourly 1 By: Danyel Lawson http://almaer.com/blog/scaling-out-37-signal-style-applications-is-convenient/comment-page-1#comment-32156 Danyel Lawson Sun, 21 May 2006 13:55:58 +0000 http://almaer.com/blog2/scaling-out-37-signal-style-applications-is-convenient#comment-32156 If you really want to scale and deal with concurrency issues you have to give up on the notion that read and write locks are anything but a hack. When you work out the read write logic in your code and run your code through some scenario testing. You will find that most of you locks are unneccessary. What is needed is transaction begin and commit statements around writes to the database to avoid partially stale data being read. If your writes to multiple tables are atomic your data can be distributed across multiple servers easily without preventing reads. Stale reads are only considered stale until the new data is written. There is never a need to lock a row and definitely not a whole table. Any real database is designed to serve read requests concurrently with writes. Work though the logic and you may be surprised by how most of your reads have almost nothing to do with your writes. The dependent database reads are already synchronized in you current code by their order of occurence after the datbase writes. If you really want to scale and deal with concurrency issues you have to give up on the notion that read and write locks are anything but a hack. When you work out the read write logic in your code and run your code through some scenario testing. You will find that most of you locks are unneccessary. What is needed is transaction begin and commit statements around writes to the database to avoid partially stale data being read. If your writes to multiple tables are atomic your data can be distributed across multiple servers easily without preventing reads. Stale reads are only considered stale until the new data is written. There is never a need to lock a row and definitely not a whole table. Any real database is designed to serve read requests concurrently with writes. Work though the logic and you may be surprised by how most of your reads have almost nothing to do with your writes. The dependent database reads are already synchronized in you current code by their order of occurence after the datbase writes.

]]> By: analogueboy http://almaer.com/blog/scaling-out-37-signal-style-applications-is-convenient/comment-page-1#comment-32155 analogueboy Tue, 09 May 2006 21:22:01 +0000 http://almaer.com/blog2/scaling-out-37-signal-style-applications-is-convenient#comment-32155 s/out/our/g s/out/our/g

]]> By: Todd Huss http://almaer.com/blog/scaling-out-37-signal-style-applications-is-convenient/comment-page-1#comment-32154 Todd Huss Tue, 09 May 2006 06:50:12 +0000 http://almaer.com/blog2/scaling-out-37-signal-style-applications-is-convenient#comment-32154 We have this very problem at our company! The scalability of our application is currently limited by our read-write MySQL master. We do caching and have 2 dedicated read-write slaves. Still, on a couple of very high traffic occasions we've had the write operations go high enough that our 10 web/application servers started waiting on the read-write database, then the db connections started backing up waiting for the lock to free, we hit max connections, and down we went. We're still using MyISAM (which does table level locking) and I'm hopeful that when we make the switch to InnoDB that load testing will show that InnoDB's row level locking is much more performant for write operations while still being nearly as performant for reads. As an aside, on the Java front we discovered that the MySQL JDBC driver offers a nice solution to easily load balance read operations across the slaves using their com.mysql.jdbc.ReplicationDriver: http://gabrito.com/post/load-balancing-across-mysql-servers-using-jdbc We have this very problem at our company! The scalability of our application is currently limited by our read-write MySQL master. We do caching and have 2 dedicated read-write slaves. Still, on a couple of very high traffic occasions we’ve had the write operations go high enough that our 10 web/application servers started waiting on the read-write database, then the db connections started backing up waiting for the lock to free, we hit max connections, and down we went.

We’re still using MyISAM (which does table level locking) and I’m hopeful that when we make the switch to InnoDB that load testing will show that InnoDB’s row level locking is much more performant for write operations while still being nearly as performant for reads.

As an aside, on the Java front we discovered that the MySQL JDBC driver offers a nice solution to easily load balance read operations across the slaves using their com.mysql.jdbc.ReplicationDriver: http://gabrito.com/post/load-balancing-across-mysql-servers-using-jdbc

]]> By: Patrick Lightbody http://almaer.com/blog/scaling-out-37-signal-style-applications-is-convenient/comment-page-1#comment-32153 Patrick Lightbody Tue, 09 May 2006 02:04:02 +0000 http://almaer.com/blog2/scaling-out-37-signal-style-applications-is-convenient#comment-32153 Thank god someone finally pointed this out. "[Language] does/does not scale" is the dumbest comment ever. Applications do or do not scale, not the languages. We are doing the exact approach you mentioned with HostedQA (http://www.hostedqa.com). Because our customers get a sub-domain, scaling is cake! Thank god someone finally pointed this out. “[Language] does/does not scale” is the dumbest comment ever. Applications do or do not scale, not the languages.

We are doing the exact approach you mentioned with HostedQA (http://www.hostedqa.com). Because our customers get a sub-domain, scaling is cake!

]]>