MonetDB用户指南4——(资源)内存使用Memory footprint

原文：http://www.monetdb.org/Documentation/MemoryClaim

 

MonetDB的要求，需要在任何给定时间点，积极融入到地址空间---当然放在存储设备中的所有数据，（例如，磁盘系统）。在32位系统，地址空间最多32位（4 GB）；在实践中，它实际上是有限的3 GB或甚至2 GB的大多数系统。在64位系统中，地址空间理论上可以是64位，但在实践中往往是“有限公司”48位左右，不使任何差异…

MonetDB excessively uses main memory for processing, but does not require that all data fit in the available physical memory. To handle a dataset that exceeds the available physical memory, MonetDB does not (only) rely on the available swap space, but (also) uses memory-mapped files to exploit disk storage beyond the swap space as virtual memory.

 

MonetDB过度使用主内存中进行处理，而不需要所有的数据适合在可用的物理内存。处理超过可用的物理内存的数据集，MonetDB不（仅）依靠现有的交换空间，但是（也）使用内存映射文件利用磁盘存储在交换空间作为虚拟内存。

For example, while bulk-loading data (preferably using a COPY INTO statement from a (possibly compressed) CSV file), MonetDB needs to have all columns of the table that is currently being loaded "active", i.e., accessible in the address space. However, during loading, parts of the data are continuously written to the persistent files on disk, i.e., the whole table does not have to fit into main memory. E.g., loading a 100 GB table works fine on a system with 8 GB RAM and 16 GB swap -- provided there is sufficient free disk space.

 

例如，在批量加载数据（最好使用复制到语句从一个（可能是压缩）的CSV文件），MonetDB都需要所有的表中的列，目前正在加载“主动”，即，在可访问的地址空间。然而，在加载过程中，部分的数据是连续地写入磁盘，即持续性文件，整个表没有适合到主存储器。例如，加载一个100 GB的表工作的很好，在同一个系统的8 GB的RAM和16 GB的交换——只要有足够的可用磁盘空间。

During query processing, MonetDB requires for each single MAL operation during the query execution that all its inputs, its outputs, and possible temporary data structures fit in the address space. MonetDB automatically resorts to virtual memory and memory-mapped files for large intermediate results. Also (large) persistent tables are accessed using memory mapping.

 

在查询处理过程中，需要对每个MonetDB误操作查询执行所有的输入，其输出的过程中，和可能的临时数据结构适合在地址空间。MonetDB自动度假村的虚拟内存和内存映射文件大的中间结果。也（大）持续使用内存映射访问表。

While running, you might see your mserver5 process' virtual size grow well beyond the available physical memory and possibly also well beyond your swap space. In principle, this is not a problem at all. Most of this virtual size is due to the fact that large base tables (or intermediate results) that reside as files on disk are memory-mapped into the address space. Those parts of the data that are currently not accessed do not consume any physical memory (except possibly for caching purposes).

 

运行时，你可能会看到你的mserver5过程虚拟大小的增长远远超出了可用的物理内存和交换空间也可能超出你的。在原则上，这根本不是问题。大部分的虚拟的大小是由于大的基表（或中间结果），存在磁盘上的文件是内存地址空间映射到。这是目前无法访问数据的那些部分不消耗任何物理内存（可能除了缓存的目的）。

However, if individual columns of your table(s) and/or individual columns of intermediate results exceed the size of the available physical memory, the performance of MonetDB might (will) decrease due to increased I/O requirements.

 

然而，如果你的表的单个列（S）和/或单个列的中间结果超过可用的物理内存的大小，MonetDB的性能可能（会）由于增加I/O需求减少。