内存转换和分段 Memory Translation and Segmentation

这是那位外国友人的另外一片关于内存分段和转换的文章

原文地址：http://duartes.org/gustavo/blog/post/memory-translation-and-segmentation

这次的翻译，挑选重点，作者的一些跟技术和主题没关系的话语被我视作浮云没有翻译：）

Memory Translation and Segmentation 内存转换和分段

This post is the first in a series about memory and protection in Intel-compatible (x86) computers, going further down the path of how kernels work. As in the boot series, I’ll link to Linux kernel sources but give Windows examples as well (sorry, I’m ignorant about the BSDs and the Mac, but most of the discussion applies). Let me know what I screw up.

这次的文章是接着之前关于系统启动的一系列文章之后，描述intel兼容系列的计算机如何处理内存。

In the chipsets that power Intel motherboards, memory is accessed by the CPU via the front side bus, which connects it to the northbridge chip. The memory addresses exchanged in the front side bus are physical memory addresses, raw numbers from zero to the top of the available physical memory. These numbers are mapped to physical RAM sticks by the northbridge. Physical addresses are concrete and final – no translation, no paging, no privilege checks – you put them on the bus and that’s that. Within the CPU, however, programs use logical memory addresses, which must be translated into physical addresses before memory access can take place. Conceptually address translation looks like this:

支持intel的主板上的芯片组，CPU通过前段总线来访问内存，而内存是通过北桥芯片与CPU连接的（但是，现在流行使用SMP，则倾向于让CPU直接连接memory或者通过memory control来连接内存，从而绕过北桥）。在前段总线交换的内存地址都是物理内存地址，从0到最大可用物理内存。这些数字通过北桥来映射到物理内存单元。物理内存是不支持翻译，不支持分页，没有权限检查的连接到总线上的物理存在。而对于CPU中运行的程序，它使用逻辑内存地址，它必须被翻译成物理地址，然后才能进行真正的内存访问。理论上的地址翻译像下图所示：

Memory address translation in x86 CPUs with paging enabled

This is not a physical diagram, only a depiction of the address translation process, specifically for when the CPU has paging enabled. If you turn off paging, the output from the segmentation unit is already a physical address; in 16-bit real mode that is always the case. Translation starts when the CPU executes an instruction that refers to a memory address. The first step is translating that logic address into a linear address. But why go through this step instead of having software use linear (or physical) addresses directly? For roughly the same reason humans have an appendix whose primary function is getting infected. It’s a wrinkle of evolution. To really make sense of x86 segmentation we need to go back to 1978.

上图不是一个物理框图，只是一个描述地址转换过程，特别是CPU启用了分页机制的示意图。如果禁止了分页机制，那么从分段单元导出的就是物理地址了；这其实就是16位实模式时发生的例子。地址翻译发生在CPU执行一条包含内存地址的指令时。第一步是把指令中的逻辑地址转换为线性地址。为什么需要这步，而不是在软件端直接使用线性（或物理）地址呢？这就需要追溯到1978年x86分段机制的开始。

The original 8086 had 16-bit registers and its instructions used mostly 8-bit or 16-bit operands. This allowed code to work with 2¹⁶ bytes, or 64K of memory, yet Intel engineers were keen on letting the CPU use more memory without expanding the size of registers and instructions. So they introduced segment registers as a means to tell the CPU which 64K chunk of memory a program’s instructions were going to work on. It was a reasonable solution: first you load a segment register, effectively saying “here, I want to work on the memory chunk starting at X”; afterwards, 16-bit memory addresses used by your code are interpreted as offsets into your chunk, or segment. There were four segment registers: one for the stack (ss), one for program code (cs), and two for data (ds, es). Most programs were small enough back then to fit their whole stack, code, and data each in a 64K segment, so segmentation was often transparent.

最初的8086拥有16位的寄存器，它的指令都似乎8位或者16位的操作数。这就使得代码可以访问2¹⁶ 字节，即64k的内存。然而当时，intel的工程师热心于在不扩大寄存器和指令大小的前提下增加CPU的寻址空间。所以，它们引入了分段寄存器，通过它来告诉CPU哪一块64k的内存是一个程序的指令所要访问的。它是一个合理的解决方案：首先你需要读取分段寄存器，即告诉CPU我需要工作在以X开始的内存块。然后，代码中你所指定的16位地址就被翻译为在你指定的分段中的偏移处。总共有4个分段寄存器：ss,cs,ds,es。

Nowadays segmentation is still present and is always enabled in x86 processors. Each instruction that touches memory implicitly uses a segment register. For example, a jump instruction uses the code segment register (cs) whereas a stack push instruction uses the stack segment register (ss). In most cases you can explicitly override the segment register used by an instruction. Segment registers store 16-bit segment selectors; they can be loaded directly with instructions like MOV. The sole exception is cs, which can only be changed by instructions that affect the flow of execution, like CALL or JMP. Though segmentation is always on, it works differently in real mode versus protected mode.

一直到现在，分段机制一直都存在于x86的体系结构中，并且总是启用的。每一条指令都隐含的需要使用分段寄存器。例如，跳转指令使用cs而压栈指令使用ss。大多数情况下，你可以显示的指定一个指令所要使用的分段寄存器。分段寄存器中保存有一个16位的段选择子；它们直接被像MOV一类的指令直接读入。唯一的例外是cs，它只能被会影响流程控制的如CALL和JMP指令所更改。因为分段机制一直是启用的，所以在实模式和保护模式下它的运作方式也是有区别的。

In real mode, such as during early boot, the segment selector is a 16-bit number specifying the physical memory address for the start of a segment. This number must somehow be scaled, otherwise it would also be limited to 64K, defeating the purpose of segmentation. For example, the CPU could use the segment selector as the 16 most significant bits of the physical memory address (by shifting it 16 bits to the left, which is equivalent to multiplying by 2¹⁶). This simple rule would enable segments to address 4 gigs of memory in 64K chunks, but it would increase chip packaging costs by requiring more physical address pins in the processor. So Intel made the decision to multiply the segment selector by only 2⁴ (or 16), which in a single stroke confined memory to about 1MB and unduly complicated translation. Here’s an example showing a jump instruction where cs contains 0×1000:

实模式下，比如启动过程的最开始阶段，段选择子十一个16位的数字，指向分段开始的物理地址。这个数值需要被扩大，否则它也只能受限于64K。例如，CPU可以使用这个选择子，把它当成是某一个物理地址的前16位（即把它左移16位）。这个简单的规则，就使得分段能够寻址以64k为单位的4G空间，但是这会导致CPU需要更多的物理地址引脚从而增加芯片的分页代价。所以，intel决定只是把段选择子左移4位（即相当于扩大16倍），这就导致此时只能依赖于一些复杂的转换才能访问到1MB空间。下面的例子展示了一个跳转指令的转换：

 
Real mode segmentation

Real mode segment starts range from 0 all the way to 0xFFFF0 (16 bytes short of 1 MB) in 16-byte increments. To these values you add a 16-bit offset (the logical address) between 0 and 0xFFFF. It follows that there are multiple segment/offset combinations pointing to the same memory location, and physical addresses fall above 1MB if your segment is high enough (see the infamous A20 line). Also, when writing C code in real mode a far pointer is a pointer that contains both the segment selector and the logical address, which allows it to address 1MB of memory. Far indeed. As programs started getting bigger and outgrowing 64K segments, segmentation and its strange ways complicated development for the x86 platform. This may all sound quaintly odd now but it has driven programmers into the wretched depths of madness.

实模式下，段的大小从0一直到0xffff0(1MB还少16字节)。用16位的在0到0xFFFF的偏移（逻辑地址）加上段地址就是物理地址了。结果就是如果你的段最够大，段基址加上偏移指向的物理地址就会超过1MB。同样，当你用C语言在实模式下写一个far point，该指针包含了段选择子和逻辑地址，就允许你访问1MB的内存。随着程序越来越大，超出了对64K段的限制，则引入了诡异的分段机制。

In 32-bit protected mode, a segment selector is no longer a raw number, but instead it contains an index into a table of segment descriptors. The table is simply an array containing 8-byte records, where each record describes one segment and looks thus:

32位保护模式下，一个段选择子就不再是一个单纯的数字，而是一个指向段描述表的索引。这个表包含简单的8字节的记录数组，每个记录描述了一个段：

Segment descriptor

There are three types of segments: code, data, and system. For brevity, only the common features in the descriptor are shown here. The base address is a 32-bit linear address pointing to the beginning of the segment, while the limit specifies how big the segment is. Adding the base address to a logical memory address yields a linear address. DPL is the descriptor privilege level; it is a number from 0 (most privileged, kernel mode) to 3 (least privileged, user mode) that controls access to the segment.

有三种类型的段：code, data, system。 简单起见，上图只包括了一些常用的描述。base address是一个32位的闲心地址，指向段的开始处，而limit用来限制段的大小。逻辑地址加上base address就转换为一个线性地址。DPL是描述权限级别的；从0（内核模式）到3（用户模式），它限制了如何访问合适的段。

These segment descriptors are stored in two tables: the Global Descriptor Table (GDT) and theLocal Descriptor Table (LDT). Each CPU (or core) in a computer contains a register called gdtrwhich stores the linear memory address of the first byte in the GDT. To choose a segment, you must load a segment register with a segment selector in the following format:

这些段描述保存在两个表中：全局描述符表（GDT）和局部描述符表（LDT）。每个CPU（或者核）都包含一个叫做gdtr的寄存器，它包含GDT中的第一个字节的线性地址，你必须使用下面的段选择子的格式来读入一个段寄存器：

 
Segment Selector

The TI bit is 0 for the GDT and 1 for the LDT, while the index specifies the desired segment selector within the table. We’ll deal with RPL, Requested Privilege Level, later on. Now, come to think of it, when the CPU is in 32-bit mode registers and instructions can address the entire linear address space anyway, so there’s really no need to give them a push with a base address or other shenanigan. So why not set the base address to zero and let logical addresses coincide with linear addresses? Intel docs call this “flat model” and it’s exactly what modern x86 kernels do (they use the basic flat model, specifically). Basic flat model is equivalent to disabling segmentation when it comes to translating memory addresses. So in all its glory, here’s the jump example running in 32-bit protected mode, with real-world values for a Linux user-mode app:

TI位0表示GDT,1表示LDT，而index指向了表中的哪一个选择子。让我们考虑下面这个例子，CPU运行在32位保护模式下，它的指令能够访问所有的线性地址，所以这里不许要用到base address或者其它的转换。所以，为什么不只是简单的把base address设置为0，然后让逻辑地址直接就等于线性地址呢？intel的文档中把这种情况称之为flat mode，并且这就是现代的x86内核的做法（它们使用基本的flat model）。它意味着在转换内存地址时禁止分段。所以，下面的运行在32位保护模式下的跳转指令的例子，描述了一个真实世界中linux用户模式的应用程序：

Protected Mode Segmentation

The contents of a segment descriptor are cached once they are accessed, so there’s no need to actually read the GDT in subsequent accesses, which would kill performance. Each segment register has a hidden part to store the cached descriptor that corresponds to its segment selector. For more details, including more info on the LDT, see chapter 3 of the Intel System Programming Guide Volume 3a. Volumes 2a and 2b, which cover every x86 instruction, also shed light on the various types of x86 addressing operands – 16-bit, 16-bit with segment selector (which can be used by far pointers), 32-bit, etc.

以上的原文没什么实际意义，就是告诉你可以在 Intel System Programming Guide Volume 3a. Volumes 2a and 2b中找到足够的指令信息。

In Linux, only 3 segment descriptors are used during boot. They are defined with the GDT_ENTRYmacro and stored in the boot_gdt array. Two of the segments are flat, addressing the entire 32-bit space: a code segment loaded into cs and a data segment loaded into the other segment registers. The third segment is a system segment called the Task State Segment. After boot, each CPU has its own copy of the GDT. They are all nearly identical, but a few entries change depending on the running process. You can see the layout of the Linux GDT in segment.h and its instantiation is here. There are four primary GDT entries: two flat ones for code and data in kernel mode, and another two for user mode. When looking at the Linux GDT, notice the holes inserted on purpose to align data with CPU cache lines – an artifact of the von Neumann bottleneck that has become a plague. Finally, the classic “Segmentation fault” Unix error message is not due to x86-style segments, but rather invalid memory addresses normally detected by the paging unit – alas, topic for an upcoming post.

Linux中，启动过程中只用到了3个段描述符。他们是GDT_ENTRY宏所定义的，并存储在boot_gdt数组中。其中的两个是flat，可以访问32位的空间：cs,ds。第三个段是system 段，被称作任务状态段。启动后，每个CPU有各自的GDT。大部分情况它们都相同，只有一些域在执行不同的程序时会有区别。你可以在segment.h和x86下的common.h中找到Linux GDT的定义。有四个主要的GDT域：2个flat的kernel模式下的cs,ds，和两外两个为用户模式服务的cs,ds。当你看Linux GDT的时候，注意其中为了维持cache lines所插入的预留位。最后，典型的“Segmentation fault”跟x86体系的分段机制没有关系，而是因为分页单元检测到了非法的内存访问。

Intel deftly worked around their original segmentation kludge, offering a flexible way for us to choose whether to segment or go flat. Since coinciding logical and linear addresses are simpler to handle, they became standard, such that 64-bit mode now enforces a flat linear address space. But even in flat mode segments are still crucial for x86 protection, the mechanism that defends the kernel from user-mode processes and every process from each other. It’s a dog eat dog world out there! In the next post, we’ll take a peek at protection levels and how segments implement them.

Intel为我们提供了灵巧的实现，可以让我们选择是否支持分段还是使用flat模式。因为把逻辑和线性地址看成是一样的更容易处理，在64位模式下采用flat作为标准。但是，即使是在flat模式下，分段机制依然有着它关键的作用。它被用来明确内核和用户模式进程之间的区别。具体请看下回的分解。