Setting MySQL INNODB Compression KEY_BLOCK_SIZE
来源:互联网 发布:linux export 编辑:程序博客网 时间:2024/05/17 17:16
http://serverfault.com/questions/358444/setting-mysql-innodb-compression-key-block-size
Setting MySQL INNODB Compression KEY_BLOCK_SIZE
I recently started using the Barracuda InnoDB/MySQL table format which allows compression.
I compressed one of my tables by running:
alter table pricing row_format=compressed, key_block_size=8;
After I ran this I viewed the compression statistics (I had cleared them right before the ALTER TABLE):
mysql> select * from INFORMATION_SCHEMA.INNODB_CMP;+-----------+--------------+-----------------+---------------+----------------+-----------------+| page_size | compress_ops | compress_ops_ok | compress_time | uncompress_ops | uncompress_time |+-----------+--------------+-----------------+---------------+----------------+-----------------+| 1024 | 0 | 0 | 0 | 0 | 0 || 2048 | 0 | 0 | 0 | 0 | 0 || 4096 | 0 | 0 | 0 | 0 | 0 || 8192 | 7029231 | 6352315 | 1437 | 339708 | 41 || 16384 | 0 | 0 | 0 | 0 | 0 |+-----------+--------------+-----------------+---------------+----------------+-----------------+5 rows in set (0.00 sec)mysql> select * from INFORMATION_SCHEMA.INNODB_CMPMEM;+-----------+------------+------------+----------------+-----------------+| page_size | pages_used | pages_free | relocation_ops | relocation_time |+-----------+------------+------------+----------------+-----------------+| 128 | 11214 | 0 | 8434571 | 2 || 256 | 0 | 37 | 0 | 0 || 512 | 0 | 34 | 0 | 0 || 1024 | 0 | 2 | 0 | 0 || 2048 | 0 | 141 | 0 | 0 || 4096 | 0 | 298 | 96657 | 0 || 8192 | 15133 | 0 | 4121178 | 5 || 16384 | 0 | 0 | 0 | 0 |+-----------+------------+------------+----------------+-----------------+8 rows in set (0.00 sec)
If I divide compress_ops_ok by compress_ops, that's 6352315/7029231 = 90.4%. My understanding is that basically 90.4% of the pages compressed from 16 KB to 8KB, and the remainder were not able to compress down by 2x.
I have read that these pages that fail compression hurt performance, but the over 90% that compressed successfully should improve performance quite a bit (by lowering I/O ops). Is there a rule of thumb of what percentage of pages should compress for this to be considered OK? My other option would probably be to just disable compression.
My net goal is to reduce the number of I/O operations, and I don't want to enable compression if this is going to be counterproductive.
2 Answers
Even after running the compression, you still may not get the performance you are looking for. Why ?
InnoDB has the Buffer Pool to load data pages and index pages read to fulfill queries. When reading a table and its indexes for the first time, the compressed page must be uncompressed. In fact, you may have twice as much data in the buffer pool as a result of this.
Note how this is the case from the MySQL Documentation
Compression and the InnoDB Buffer Pool
In a compressed InnoDB table, every compressed page (whether 1K, 2K, 4K or 8K) corresponds to an uncompressed page of 16K bytes. To access the data in a page, InnoDB reads the compressed page from disk if it is not already in the buffer pool, then uncompresses the page to its original 16K byte form. This section describes how InnoDB manages the buffer pool with respect to pages of compressed tables.
To minimize I/O and to reduce the need to uncompress a page, at times the buffer pool contains both the compressed and uncompressed form of a database page. To make room for other required database pages, InnoDB may “evict” from the buffer pool an uncompressed page, while leaving the compressed page in memory. Or, if a page has not been accessed in a while, the compressed form of the page may be written to disk, to free space for other data. Thus, at any given time, the buffer pool may contain both the compressed and uncompressed forms of the page, or only the compressed form of the page, or neither.
InnoDB keeps track of which pages to keep in memory and which to evict using a least-recently-used (LRU) list, so that “hot” or frequently accessed data tends to stay in memory. When compressed tables are accessed, InnoDB uses an adaptive LRU algorithm to achieve an appropriate balance of compressed and uncompressed pages in memory. This adaptive algorithm is sensitive to whether the system is running in an I/O-bound or CPU-bound manner. The goal is to avoid spending too much processing time uncompressing pages when the CPU is busy, and to avoid doing excess I/O when the CPU has spare cycles that can be used for uncompressing compressed pages (that may already be in memory). When the system is I/O-bound, the algorithm prefers to evict the uncompressed copy of a page rather than both copies, to make more room for other disk pages to become memory resident. When the system is CPU-bound, InnoDB prefers to evict both the compressed and uncompressed page, so that more memory can be used for “hot” pages and reducing the need to uncompress data in memory only in compressed form.
If this duplication of data content is going on in the Buffer Pool, you need to increaseinnodb_buffer_pool_size by a small linear factor of the new compression rate. Here is how:
SCENARIO
- You have a DB Server with a 8G Buffer Pool
- You ran compression with
key_block_size=8
8
is50.00%
of16
50.00%
of8G
is4G
- raise
innodb_buffer_pool_size
to12G
(8G
+4G
)
- You ran compression with
key_block_size=4
4
is25.00%
of16
25.00%
of8G
is2G
- raise
innodb_buffer_pool_size
to10G
(8G
+2G
)
- You ran compression with
key_block_size=2
2
is12.50%
of16
12.50%
of8G
is1G
- raise
innodb_buffer_pool_size
to9G
(8G
+1G
)
- You ran compression with
key_block_size=1
1
is06.25%
of16
06.25%
of8G
is0.5G
(512M
)- raise
innodb_buffer_pool_size
to8704M
(8G
(8192M
) +512M
)
MORAL OF THE STORY : The InnoDB Buffer Pool just needs additional breathing room when handling compressed data and index pages.
- Setting MySQL INNODB Compression KEY_BLOCK_SIZE
- innodb data compression
- 14.7 InnoDB Table Compression
- MySQL5.7 InnoDB Page Compression
- Tomcat6(X64) [SetAllPropertiesRule]{Server/Service/Connector} Setting property 'compression' 解决方法
- mysql innodb
- mysql-Innodb
- MySQL-InnoDB
- MySQL Blob Compression performance benefits
- mysql 5.7 Transparent PageIO Compression
- Compression
- phpmyadmin MySQL invalid setting
- 【Setting】MySQL环境搭建
- Mysql InnoDB介绍
- Mysql InnoDB介绍
- Mysql InnoDb简介
- mysql InnoDB的count(*)
- mysql innodb 性能优化
- 运行adb devices的时候出现adb server version (31) doesn't match this client (39);
- BMP转YUV
- js学习记录10
- Android笔记——Button点击事件的几种写法
- Java常用加密算法工具类
- Setting MySQL INNODB Compression KEY_BLOCK_SIZE
- C-socket编程-多进程版并发服务器
- vue仿163musicPC端
- spring源码学习之:xml配置文件标签自定义
- Android-taskAffinity
- 如何在需要登录的界面中使用(需要登录的界面)
- lucene
- 栈相关知识
- Linux 目录结构