How-to Dump Keys from Memcache
来源:互联网 发布:php数组90度旋转 编辑:程序博客网 时间:2024/06/06 01:12
How-to Dump Keys from Memcache
You spent already 50GB on the memcache cluster, but you still see many evictions and the cache hit ratio doesn't look good since a few days. The developers swear that they didn't change the caching recently, they checked the code twice and have found no problem.
What now? How to get some insight into the black box of memcached? One way would be to add logging to the application to see and count what is being read and written and then to guess from this about the cache efficiency. For to debug what's happening we need to set how the cache keys are used by the application.
An Easier Way
Memcache itself provides a means to peek into its content. The memcache protocol providescommands to peek into the data that is organized by slabs (categories of data of a given size range). There are some significant limitations though:
- You can only dump keys per slab class (keys with roughly the same content size)
- You can only dump one page per slab class (1MB of data)
- This is an unofficial feature that might be removed anytime.
The second limitation is propably the hardest because 1MB of several gigabytes is almost nothing. Still it can be useful to watch how you use a subset of your keys. But this might depend on your use case.
If you don't care about the technical details just skip to the tools section to learn about what tools allow you to easily dump everything. Alternatively follow the following guide and try the commandsusing telnet against your memcached setup.
How it Works
First you need to know how memcache organizes its memory. If you start memcache with option "-vv" you see the slab classes it creates. For example
$ memcached -vvslab class 1: chunk size 96 perslab 10922slab class 2: chunk size 120 perslab 8738slab class 3: chunk size 152 perslab 6898slab class 4: chunk size 192 perslab 5461[...]
In the configuration printed above memcache will keep fit 6898 pieces of data between 121 and 152 byte in a single slab of 1MB size (6898*152). All slabs are sized as 1MB per default. Use the following command to print all currently existing slabs:
stats slabs
If you've added a single key to an empty memcached 1.4.13 with
set mykey 0 60 11STORED
you'll now see the following result for the "stats slabs" command:
stats slabsSTAT 1:chunk_size 96STAT 1:chunks_per_page 10922STAT 1:total_pages 1STAT 1:total_chunks 10922STAT 1:used_chunks 1STAT 1:free_chunks 0STAT 1:free_chunks_end 10921STAT 1:mem_requested 71STAT 1:get_hits 0STAT 1:cmd_set 2STAT 1:delete_hits 0STAT 1:incr_hits 0STAT 1:decr_hits 0STAT 1:cas_hits 0STAT 1:cas_badval 0STAT 1:touch_hits 0STAT active_slabs 1STAT total_malloced 1048512END
The example shows that we have only one active slab type #1. Our key being just one byte large fits into this as the smallest possible chunk size. The slab statistics show that currently on one page of the slab class exists and that only one chunk is used.
Most importantly it shows a counter for each write operation (set, incr, decr, cas, touch) and one for gets. Using those you can determine a hit ratio!
You can also fetch another set of infos using "stats items" with interesting counters concerning evictions and out of memory counters.
stats itemsSTAT items:1:number 1STAT items:1:age 4STAT items:1:evicted 0STAT items:1:evicted_nonzero 0STAT items:1:evicted_time 0STAT items:1:outofmemory 0STAT items:1:tailrepairs 0STAT items:1:reclaimed 0STAT items:1:expired_unfetched 0STAT items:1:evicted_unfetched 0END
What We Can Guess Already...
Given the statistics infos per slabs class we can already guess a lot of thing about the application behaviour:
- How is the cache ratio for different content sizes?
- How good is the caching of large HTML chunks?
- How much memory do we spend on different content sizes?
- How much do we spend on simple numeric counters?
- How much do we spend on our session data?
- How much do we spend on large HTML chunks?
- How many large objects can we cache at all?
Of course to answer the questions you need to know about the cache objects of your application.
Now: How to Dump Keys?
Keys can be dumped per slabs class using the "stats cachedump" command.
stats cachedump <slab class> <number of items to dump>
To dump our single key in class #1 run
stats cachedump 1 1000ITEM mykey [1 b; 1350677968 s]END
The "cachedump" returns one item per line. The first number in the braces gives the size in bytes, the second the timestamp of the creation. Given the key name you can now also dump its value using
get mykeyVALUE mykey 0 11END
This is it: iterate over all slabs classes you want, extract the key names and if need dump there contents.
Dumping Tools
There are different dumping tools sometimes just scripts out there that help you with printing memcache keys:
PHPsimple scriptPrints key names.Perlsimple scriptPrints keys and valuesRubysimple scriptPrints key names.PerlmemdumpTool in CPAN module Memcached-libmemcachedPHPmemcache.phpMemcache Monitoring GUI that also allows dumping keyslibmemcachedpeepDoes freeze your memcached process!!!
Be careful when using this in production. Still using it you can workaround the 1MB limitation and really dumpall keys.
osd@OSD-144:~$ printf "stats \r\n" | nc localhost 11211
STAT pid 1123
STAT uptime 74699
STAT time 1427430453
STAT version 1.4.14 (Ubuntu)
END
来源:
http://lzone.de/dump%20memcache%20keys
http://blog.goyiyo.com/archives/862
http://www.ttlsa.com/memcache/memcache-list-all-keys/
- How-to Dump Keys from Memcache
- how to view bitmaps from memory dump
- How to analyze dump
- how to dump oracle block
- How to analyze Thread Dump
- How to Use Google API Keys
- Updating to KMS from MAK: Product Keys
- How to map android keys to your usb/bluetooth keyboard
- How to Export/import database dump correctly
- How to use Oracle Dump Function
- How to Create Dump File for Applications
- How to dump Oracle Data Block?
- iPhone How-to:如何递归dump UIView
- How to read crash dump of Android
- how to analysis the crash dump
- how to dump undo header and block
- How to class-dump iPad apps?
- How to dump redo log entry?
- Fragment详解
- Search 2D matrix II
- hdu 1222 Wolf and Rabbit(递归)
- 用composer进行更新操作
- JavaScript随机产生文字实例
- How-to Dump Keys from Memcache
- 服务器安全部署文档
- C/C++中函数参数传递详解
- leetcode--Unique Binary Search Trees
- Employees Earning More Than Their Managers
- php 把“今天”作为查询条件
- 关于composer.json的格式问题
- 程序员技术练级攻略
- Qt控制台不能调试Cannot connect creator comm.....