MongoDB: Index Size and Memory & Possible performance impact Options
来源:互联网 发布:网络大电影营销推广 编辑:程序博客网 时间:2024/05/23 15:39
这个是google group mongodb user小组的一则讨论,提问者首先提出如下的问题:
on our main mongo box, we have 4G memory (wohooo). Our data size is
1G and index size is 5.6G. This box is 1 node of a 3 node replica
set. We have 1 collection and a few indexes on that collection.
This one collection stores a lot of writes from our website.
We are mostly concerned that our write speed does not degrade (right
now, we're ~ 3-4ms/write). (near) worst case scenario is that the
write speed would degrade dramatically (while mongo is paging).
What things should we be keeping an eye on? any particular metric to
comb for? would writes (including index updates) be efficient even
if, in our case, we see index size greater than ram?
Thanks.
下面是一个叫gates的哥们给的答复,comments很不错,推荐认真看一下!
Here's the general advice on scaling:
- *Replica Sets* is used for scaling reads and for providing
redundancy
- *Sharding* is used for scaling writes
---
It looks like you're already past that.
Metric #2 is to check IO Utilization. (see iostat)
---
You want to keep IO utilization % under 100%. By default, MongoDB is
flushing every 60 seconds, which may result in IO spikes every minute
or so. If you're seeing low utilization with spikes every minute, then
you may want to set --syncdelay when you run mongodb. If you can set
it lower
Metric #3: watch for paging.
---
You can often do this with a tool like top. We also have mongostat
which should give you an idea of how often data is being "paged in".
The Problems:
This is really hard to guarantee when you've already overflowed the
RAM and it's going to grow.
If you only have one index, then maybe you'll get lucky. MongoDB is
pretty good about "writing to the end" of the index. So old index data
would just flow out of memory and rarely if ever get re-used.
However, I suspect that you have multiple indexes here. If this is the
case, then it's hard to guarantee that we won't be going to disk to
fetch those indexes.
The other problem here is queries. At some point you'll want to query
this data. When you do, it's quite likely that you're going to force
the whole index back into memory and this is going to slow down the
whole system.
Possible Solutions:
#1: Add more RAM -> will delay the problem
#2: Less indexes? -> if possible
#3: Break out data
It sounds like you have lots of transactional data.
Is it possible to break out data by hour or by day or by week?
This will make queries a little more difficult, but it's going to
minimize the amount of indexes you need in memory at any given time.
If you have more details on what data you're storing and how you're
indexing, we may be able to provide additional guidance.
- Gates
- MongoDB: Index Size and Memory & Possible performance impact Options
- Impact of memory allocators on MySQL performance
- MySQL performance: Impact of memory allocators (Part 2)
- MySQL performance: Impact of memory allocators (Part 2)
- JDeveloper Memory And Performance
- Understanding B+tree Indexes and how they Impact Performance
- performance impact of printf
- MongoDB Performance for more data than memory
- Linux System and Performance Monitoring(Memory篇)
- AMD Radeon HD5870 memory capacity and performance
- Linux System and Performance Monitoring(Memory篇)
- Performance tune case - Suboptimal index and missing index
- CPU Overcommitment and Its Impact on SQL Server Performance on VMware
- arm:typical memory size and access time
- Improve Performance and Reduce Memory with PVRTC Textures and Cocos2d
- SQL Server Index Data Structure and Performance Tuning
- [转] Linux System and Performance Monitoring(Memory篇)
- Find CPU Speed and Memory Size on Solaris
- 【orange】特权级变换总结
- Power designer 15生成sql server 2005/2008列注释
- 【Android游戏开发二十二】(图文详解)游戏中灵活实现动画播放!简述J2me的游戏类库与Android游戏开发!
- 3月3日 阶段记录
- Linux下如何删除cluster的卷组
- MongoDB: Index Size and Memory & Possible performance impact Options
- 跟踪球 鼠标旋转3D物体
- 打包下载:基于Flash Player 11(Molehill)和Alternatova3D的3D赛车游戏
- flash player 11(Molehill)公开测试,附上有用资料链接
- dynamic programming
- 给VS2010自动设置模板,加头注释
- Flex异步令牌AsyncToken的用法介绍
- 深入挖掘Molehill API的一些特征
- 新浪发布财报微博用户数破亿