Part 1 of our series set-up our 'test' application and looked at boosting performance of the application by buffer MySQL with memcached. Our test application is simple and requires only 3 basic operations per transaction 2 reads and 1 write. Using memcached combined with MySQL we ended up nearly getting a 10X performance boost from the application. Now we are going to look at what we could achieve if we did not have to write to the database at all. So let's look at what happens if we push everything including writes into memcached.
Wow that's shockingly fast isn't it! I guess being completely in memory helps for this app. What is very interesting is accessing 100% of the data in memcached gives very similar numbers to accessing 100% of the data in memory in the DB ( part 1 benchmarked a 4GB bp as being able to handle 7K TPS)... something is not 100% right here. It stands to reason that memcached should be faster for this application then the DB. Its just doing two gets via key and 1 set. So why the similar numbers?
Well glad you asked. It's the API. The api in this case was Cache::Memcached, by switching to using Cache::Memcached::Fast look what happens:
That is a nice jump in performance!
Using Memcached::Fast was kind of a mixed bag when looking at the benchmarks for mixing MySQL and Memcached in my tests:
In this case I think the Fast api was slower when working with MySQL with a 256m BP because the slower returns from memcached acted as a bottleneck to thin the demands on MySQL to write data, smoothing out the work load. When we eliminate this bottleneck with the Fast api, MySQL gets hammered. This type of thing happens a lot. For example an application is CPU bound, so you add more processing power, but then you hit disks harder and now your disk bound.
A couple of good things to remember here: #1 resolving 1 bottleneck can open another bottleneck that is much worse. #2 is to understand that not all API's are created equal. Additionally the configuration and setup that works well on one system may not work well on another. Because of this people often leave lots of performance on the table. Don't just trust that your current API or config is optimal, test and make sure it fits your application.
So adding Memcached on top of MySQL for our test application can significantly boost performance. But you notice that if we were running 100% in memcached and could cut out MySQL we could get 2.5x more performance over a mixed solution and 100X over just stock MySQL. As the number of writes against the database increase this gap will increase. So let's ditch the database! But wait! you need the DB for persistence, right?
It depends. A database may not be the best fit for every application. There are several “NOSQL” solutions out in the open source space that can give you some of the ease of a Memcached but with persistence most people use their database for. Each application is different and understanding the application's requirements is key to picking an appropriate solution. I am going to look at several database alternatives over the next few months. I need to start somewhere, so I decided to start with Tokyo Tyrant and Cabinet. So stop in next time for part 3 of this series where we will focus on running the same tests against Tokyo Tyrant.
Wow that's shockingly fast isn't it! I guess being completely in memory helps for this app. What is very interesting is accessing 100% of the data in memcached gives very similar numbers to accessing 100% of the data in memory in the DB... something is not 100% right here. It stands to reason that memcached should be faster for this application then the DB, two gets via key and 1 set. So why the similar numbers?
Well glad you asked. It's the API. The api in this case was Cache::Memcached, by switching to using Cache::Memcached::Fast look what happens:
That is a nice jump in performance!
Using Memcached::Fast was kind of a mixed bag when looking at the benchmarks for mixing MySQL and Memcached in my tests:
In this case I think Fast was slower when working with MySQL with a 256m BP because the slower returns from memcached acted as a bottleneck to thin the demands on MySQL to write data, smoothing out the work load. When we eliminate this bottleneck with the Fast api, MySQL gets hammered.
A couple of good things to remember here: #1 resolving 1 bottleneck can open another bottleneck that is much worse. #2 is to understand that not all API's are created equal. Additionally the configuration and setup that works well on one system may not work well on another. Because of this people often leave lots of performance on the table. Don't just trust that your current API or config is optimal, test and make sure it fits your application.
So adding Memcached on top of MySQL for our test application can significantly boost performance. But you notice that if we were running 100% in memcached and could cut out MySQL we could get 2.5x more performance. As the number of writes against the database increase this gap will increase. So let's ditch the database! But wait! you need the DB for persistence, right?
It depends. A database may not be the best fit for every application. There are several “NOSQL” solutions out in the open source space that can give you some of the ease of a Memcached but with persistence most people use their database for. Each application is different and understanding the application's requirements is key to picking an appropriate solution. I am going to look at several database alternatives over the next few months. I need to start somewhere, so I decided to start with Tokyo Tyrant and Cabinet.
Entry posted by matt |
No comment
Add to: 
PlanetMySQL Voting:
Vote UP /
Vote DOWN"
Posts
No comments:
Post a Comment