2014-12-1 xen上的vuart

来源：互联网发布：淘宝店怎么找人刷销量编辑：程序博客网时间：2024/04/29 13:18

在ARM上调试xen、linux的时候printk和early_printk都是非常重要的方式。即使有DSTREAM，还是一定会用到屏幕或串口输出。但ARM平台上不像X86那样有bios里的基本的VGA支持，LCD一般都启动的比较靠后了。所以通过串口输出很重要。

在用xen里作为dom0跑linux的时候，会遇到linux串口的输出不见了。原因是xen起来后自己要用，就对dom0屏蔽了串口。Linux要想通过串口输出，可以指定console=hvc0。

Linux里的early_printk是在内核刚加载的时候就基本上可以用了，一旦early_printk可以用，至少也可以松口气，说明后面可以调了。最怕的就是一上来屏幕一黑什么都没有。Early_printk是的配置是和平台相关的，不同的板子都不一样。首先要linux里面支持，通过指定串口控制芯片的MMIO基址，用直接写寄存器的方式输出。当然正常串口的驱动最后也会是这样，early_printk比较直接吧，因为linux刚加载的时候驱动框架什么的都还没有起来，比如在内核自解压缩的时候。

以xen 4.3为例，平台是arndale5250-K，芯片是三星的Exynos5250，两个Cortex A15。

xen\xen\drivers\char\exynos4210-uart.c

/* TODO: Parse UARTconfig from the command line */static int __init exynos4210_uart_init(structdt_device_node *dev,                                       const void *data){    const char *config =data;    struct exynos4210_uart *uart;    int res;    u64 addr,size;     if ( strcmp(config,"") )    {        early_printk("WARNING: UART configuration is notsupported\n");    }     uart =&exynos4210_com;     /*uart->clock_hz  = 0x16e3600; */    uart->baud      =BAUD_AUTO;    uart->data_bits = 8;    uart->parity    =PARITY_NONE;    uart->stop_bits = 1;     res = dt_device_get_address(dev,0, &addr, &size);    if ( res )    {        early_printk("exynos4210: Unable to retrieve the base"                     "address of the UART\n");       returnres;    }     uart->regs =ioremap_nocache(addr,size);    if ( !uart->regs )    {        early_printk("exynos4210: Unable to map the UART memory\n");        return-ENOMEM;    }    res = dt_device_get_irq(dev,0, &uart->irq);    if ( res )    {        early_printk("exynos4210: Unable to retrieve the IRQ\n");        return res;    }     uart->vuart.base_addr=addr;    uart->vuart.size =size;    uart->vuart.data_off=UTXH;    uart->vuart.status_off=UTRSTAT;    uart->vuart.status =UTRSTAT_TXE |UTRSTAT_TXFE;     /* Register withgeneric serial driver. */    serial_register_uart(SERHND_DTUART, &exynos4210_uart_driver,uart);     dt_device_set_used_by(dev,DOMID_XEN);     return 0;}

/* TODO: Parse UARTconfig from the command line */

static int __init exynos4210_uart_init(structdt_device_node *dev,

const void *data)

{

const char *config =data;

struct exynos4210_uart *uart;

int res;

u64 addr,size;

if ( strcmp(config,"") )

{

early_printk("WARNING: UART configuration is notsupported\n");

}

uart =&exynos4210_com;

/*uart->clock_hz = 0x16e3600; */

uart->baud =BAUD_AUTO;

uart->data_bits = 8;

uart->parity =PARITY_NONE;

uart->stop_bits = 1;

res = dt_device_get_address(dev,0, &addr, &size);

if ( res )

{

early_printk("exynos4210: Unable to retrieve the base"

"address of the UART\n");

returnres;

}

uart->regs =ioremap_nocache(addr,size);

if ( !uart->regs )

{

early_printk("exynos4210: Unable to map the UART memory\n");

return-ENOMEM;

}

res = dt_device_get_irq(dev,0, &uart->irq);

if ( res )

{

early_printk("exynos4210: Unable to retrieve the IRQ\n");

return res;

}

uart->vuart.base_addr=addr;

uart->vuart.size =size;

uart->vuart.data_off=UTXH;

uart->vuart.status_off=UTRSTAT;

uart->vuart.status =UTRSTAT_TXE |UTRSTAT_TXFE;

/* Register withgeneric serial driver. */

serial_register_uart(SERHND_DTUART, &exynos4210_uart_driver,uart);

dt_device_set_used_by(dev,DOMID_XEN);

return 0;

}

Xen里平台相关的串口驱动的Init函数。

res = dt_device_get_address(dev,0, &addr, &size);

串口设备的MMIO基址和大小通过FDT文件获得。这里得到的是物理地址。

uart->regs = ioremap_nocache(addr,size);

把串口的物理地址映射到xen的地址空间。在xen环境下就可以通过uart->regs操作串口了。

uart->vuart.base_addr=addr;

uart->vuart.size =size;

uart->vuart.data_off=UTXH;

uart->vuart.status_off=UTRSTAT;

uart->vuart.status =UTRSTAT_TXE|UTRSTAT_TXFE;

这里的vuart就是xen在模拟串口设备时要用到的参数。

/* Register withgeneric serial driver. */

serial_register_uart(SERHND_DTUART, &exynos4210_uart_driver,uart);

这句是xen注册串口驱动的。下面仔细看看它。

xen\xen\drivers\char\serial.c

void __initserial_register_uart(intidx,structuart_driver*driver,

void *uart)

{

/* StoreUART-specific info. */

com[idx].driver =driver;

com[idx].uart =uart;

}

函数本身非常简单，不过在我看到这个函数之前，一直没搞懂com是怎样用的，一会看vuart的代码。

xen\xen\drivers\char\serial.c

static struct serial_portcom[SERHND_IDX+ 1] = {

[0 ... SERHND_IDX]= {

.rx_lock= SPIN_LOCK_UNLOCKED,

.tx_lock= SPIN_LOCK_UNLOCKED

}

};

xen\xen\include\xen\serial.h

/* 'Serial handles'are composed from the following fields. */

#define SERHND_IDX (3<<0)/* COM1, COM2, DBGP, DTUART? */

# define SERHND_COM1 (0<<0)

# define SERHND_COM2 (1<<0)

# define SERHND_DBGP (2<<0)

# define SERHND_DTUART (0<<0)/* Steal SERHND_COM1 value */

#define SERHND_HI (1<<2)/* Mux/demux each transferredchar by MSB. */

#define SERHND_LO (1<<3)/* Ditto, except that the MSB iscleared. */

#define SERHND_COOKED (1<<4)/* Newline/carriage-returntranslation? */

Xen一共有3个串口，SERHND_DTUART就看做com1吧，还不知道为什么不直接用com1表示。

现在知道vuart里的数据，如base和size这些是从哪来的了。确实是平台相关的，是具体的串口驱动提供的。不过因为用了FDT，所以最终base是在FDT里描述的那个。

回过头去看我最开始看的vuart.c

xen\xen\arch\arm\vuart.c

* xen/arch/arm/vuart.c

* Virtual UART Emulator.

* This emulator uses the information fromdtuart. This is not intended to be

* a full emulation of an UART device. Ratherit is intended to provide a

* sufficient veneer of one that early code(such as Linux's boot time

* decompressor) which hardcodes outputdirectly to such a device are able to

* make progress.

* The minimal register set to emulate an UARTare:

* -Single byte transmit register

* -Single status register

* /!\ This device is not intended to beenumerable or exposed to the OS

* (e.g. via Device Tree).

* Julien Grall <julien.grall@linaro.org>

* Ian Campbell <ian.campbell@citrix.com>

* This program is free software; you canredistribute it and/or modify

* it under the terms of the GNU General PublicLicense as published by

* the Free Software Foundation; either version2 of the License, or

* (at your option) any later version.

* This program is distributed in the hope thatit will be useful,

* but WITHOUT ANY WARRANTY; without even theimplied warranty of

* MERCHANTABILITY or FITNESS FOR A PARTICULARPURPOSE. See the

* GNU General Public License for more details.

int domain_vuart_init(structdomain *d)

{

ASSERT(!d->domain_id);

d->arch.vuart.info =serial_vuart_info(SERHND_DTUART);

if ( !d->arch.vuart.info )

return0;

spin_lock_init(&d->arch.vuart.lock);

d->arch.vuart.idx = 0;

d->arch.vuart.buf =xzalloc_array(char,VUART_BUF_SIZE);

if ( !d->arch.vuart.buf )

return-ENOMEM;

return 0;

}

arch_domain_create()调用domain_vuart_init()

xen\xen\arch\arm\domain.c

* Virtual UART is only used by linux earlyprintk and decompress code.

* Only use it for dom0 because the linuxkernel may not support

* multi-platform.

if ( (d->domain_id== 0) && (rc =domain_vuart_init(d)))

goto fail;

domain_vuart_init初始化xen虚拟的串口，其实最重要的是把虚拟串口信息的放到domain里。

xen\xen\drivers\char\serial.c

const struct vuart_info*serial_vuart_info(intidx)

{

if ( (idx >= 0) && (idx<ARRAY_SIZE(com))&&

com[idx].driver&&com[idx].driver->vuart_info)

return com[idx].driver->vuart_info(&com[idx]);

return NULL;

}

这回再看就很清楚了，com里保存的是注册过的串口驱动。com[idx].driver->vuart_info(&com[idx])也很简单。

xen\xen\drivers\char\exynos4210-uart.c

static struct uart_driver__read_mostly exynos4210_uart_driver= {

.init_preirq = exynos4210_uart_init_preirq,

.init_postirq= exynos4210_uart_init_postirq,

.endboot = NULL,

.suspend = exynos4210_uart_suspend,

.resume = exynos4210_uart_resume,

.tx_ready = exynos4210_uart_tx_ready,

.putc = exynos4210_uart_putc,

.getc = exynos4210_uart_getc,

.irq = exynos4210_uart_irq,

.dt_irq_get = exynos4210_uart_dt_irq,

.vuart_info = exynos4210_vuart_info,

};

static const struct vuart_info *exynos4210_vuart_info(structserial_port*port)

{

struct exynos4210_uart *uart=port->uart;

return&uart->vuart;

}

顺便提下

xen\xen\include\xen\serial.h

struct serial_port {

/* Uart-driverparameters. */

struct uart_driver *driver;

void *uart;

enum serial_port_statestate;

/* Transmit databuffer (interrupt-driven uart). */

char *txbuf;

unsigned int txbufp,txbufc;

bool_t tx_quench;

int tx_log_everything;

/* Forcesynchronous transmit. */

int sync;

/* Receivercallback functions (asynchronous receivers). */

serial_rx_fn rx_lo,rx_hi, rx;

/* Receive databuffer (polling receivers). */

char rxbuf[serial_rxbufsz];

unsigned int rxbufp,rxbufc;

/* Serial I/O isconcurrency-safe. */

spinlock_t rx_lock,tx_lock;

};

Com的结构，注册串口驱动时，就填driver和uart，driver里是各种操作的函数指针，uart里是这个com的参数。

好了，就算是xen里已经初始化了串口驱动，domain里已经有了vuart的信息，作为dom0的linux怎样把early_printk的内容输出到串口里呢？在linux里要不要配置early_printk和板子相关的参数呢？参数又是什么？

当然要在linux里配early_printk的参数啊，否则early_printk都没办法用。要是用printk，也只能通过console=hvc0了。奇怪我怎么会问这个问题。参数，就是配置时的串口MMIO的物理地址。这样就是说当linux里调用early_printk时，early_printk会写物理地址。所以接下来的问题就是xen有没有把这段物理地址映射到dom0中？Linux在写这段物理地址的时候会发生什么？

这时候到时想到了在前面的函数exynos4210_uart_init里见到过这一句

dt_device_set_used_by(dev,DOMID_XEN);

意思是挺明显，说这个设备是让xen用的。但里面的实现无非是

xen\xen\include\xen\device_tree.h

static inline void dt_device_set_used_by(structdt_device_node *device,

domid_tused_by)

{

/* TODO: childrenmust inherit to the used_by thing */

device->used_by =used_by;

}

Used_by只是一个标志位。搜了下用到它的地方。

xen\xen\arch\arm\domain_build.c

static int handle_node(struct domain *d, structkernel_info *kinfo,

conststructdt_device_node*node)

{

static conststructdt_device_matchskip_matches[]__initconst =

{

DT_MATCH_COMPATIBLE("xen,xen"),

DT_MATCH_COMPATIBLE("xen,multiboot-module"),

DT_MATCH_COMPATIBLE("arm,psci"),

DT_MATCH_PATH("/cpus"),

DT_MATCH_TYPE("memory"),

{ /* sentinel*/ },

};

staticconststructdt_device_matchgic_matches[]__initconst =

{

DT_MATCH_GIC,

{ /* sentinel*/ },

};

static conststructdt_device_matchtimer_matches[]__initconst =

{

DT_MATCH_TIMER,

{ /* sentinel*/ },

};

const structdt_device_node*child;

int res;

const char *name;

const char *path;

path = dt_node_full_name(node);

DPRINT("handle %s\n",path);

/* Skip thesesnodes and the sub-nodes */

if ( dt_match_node(skip_matches,node) )

{

DPRINT(" Skip it(matched)\n");

return0;

}

if ( platform_device_is_blacklisted(node) )

{

DPRINT(" Skip it(blacklisted)\n");

return0;

}

/* Replace thesenodes with our own. Note that the original may be

* used_by DOMID_XEN so this check comesfirst. */

if ( dt_match_node(gic_matches,node) )

return make_gic_node(d,kinfo->fdt,node);

if ( dt_match_node(timer_matches,node) )

return make_timer_node(d,kinfo->fdt,node);

/* Skip nodesused by Xen */

if ( dt_device_used_by(node)==DOMID_XEN )

{

DPRINT(" Skip it(used by Xen)\n");

return0;

}

* Some device doesn't need to be mapped inXen:

* -Memory: the guest will see a different view of memory. It will

* be allocated later.

* -Disabled device: Linux is able to cope with status="disabled"

* property. Therefore these device doesn't need to be mapped. This

* solution can be use later for pass through.

if ( !dt_device_type_is_equal(node,"memory") &&

dt_device_is_available(node) )

{

res= map_device(d,node);

if ( res )

returnres;

}

* The property "name" is used tohave a different name on older FDT

* version. We want to keep the nameretrieved during the tree

* structure creation, that is store in thenode path.

name = strrchr(path,'/');

name = name ?name + 1:path;

res = fdt_begin_node(kinfo->fdt,name);

if ( res )

return res;

res = write_properties(d,kinfo,node);

if ( res )

return res;

for ( child =node->child;child !=NULL;child=child->sibling)

{

res= handle_node(d,kinfo,child);

if ( res )

returnres;

}

if ( node ==dt_host)

{

res= make_hypervisor_node(d,kinfo->fdt,node);

if ( res )

returnres;

res= make_psci_node(kinfo->fdt,node);

if ( res )

returnres;

res= make_cpus_node(d,kinfo->fdt,node);

if ( res )

returnres;

res= make_memory_node(d,kinfo->fdt,node,kinfo);

if ( res )

returnres;

}

res = fdt_end_node(kinfo->fdt);

return res;

}

再看这个函数的调用过程

construct_dom0()->prepare_dtb()->handle_node()

可以看出除了一些特殊的设备，gic、timer什么的，还有在黑名单里的设备不会被映射。其余的设备，如果是被标记上used_by DOMID_XEN，就不会调用map_device()，因此不会被映射到dom0里。

xen\xen\arch\arm\domain_build.c

/* Map the device inthe domain */

static int map_device(struct domain *d, conststructdt_device_node*dev)

{

unsigned intnirq;

unsigned intnaddr;

unsigned inti;

int res;

struct dt_irqirq;

struct dt_raw_irqrirq;

u64 addr,size;

nirq = dt_number_of_irq(dev);

naddr = dt_number_of_address(dev);

DPRINT("%s nirq = %d naddr = %u\n",dt_node_full_name(dev),nirq,naddr);

/* Map IRQs */

for ( i = 0;i <nirq;i++ )

{

res= dt_device_get_raw_irq(dev,i, &rirq);

if ( res )

{

printk(XENLOG_ERR"Unableto retrieve irq %u for %s\n",

i,dt_node_full_name(dev));

returnres;

}

* Don't map IRQ that have no physicalmeaning

* ie: IRQ whose controller is not theGIC

if ( rirq.controller!=dt_interrupt_controller )

{

DPRINT("irq %u not connected to primary controller."

"Connectedto %s\n", i, dt_node_full_name(rirq.controller));

continue;

}

res= dt_irq_translate(&rirq, &irq);

if ( res )

{

printk(XENLOG_ERR"Unableto translate irq %u for %s\n",

i,dt_node_full_name(dev));

returnres;

}

DPRINT("irq %u = %u type = 0x%x\n",i,irq.irq,irq.type);

/* Don'tcheck return because the IRQ can be use by multiple device */

gic_route_irq_to_guest(d, &irq,dt_node_name(dev));

}

/* Map theaddress ranges */

for ( i = 0;i <naddr;i++ )

{

res= dt_device_get_address(dev,i, &addr, &size);

if ( res )

{

printk(XENLOG_ERR"Unableto retrieve address %u for %s\n",

i,dt_node_full_name(dev));

returnres;

}

DPRINT("addr %u = 0x%"PRIx64" - 0x%"PRIx64"\n",

i,addr,addr+size - 1);

res= map_mmio_regions(d,addr &PAGE_MASK,

PAGE_ALIGN(addr+size) - 1,

addr & PAGE_MASK);

if ( res )

{

printk(XENLOG_ERR"Unableto map 0x%"PRIx64

"- 0x%"PRIx64" in dom0\n",

addr& PAGE_MASK, PAGE_ALIGN(addr +size) -1);

returnres;

}

return 0;

}

Map_device里对MMIO会调用map_mmio_regions()。

这样看来，当串口驱动注册时，标明了com1是used_by DOMID_XEN，也就是说这个串口在dom0中是不存在的，它的MMIO物理地址在dom0的地址空间里是没有被xen映射出来的。并且在dom0 linux看到的FDT中，也不存在描述串口的这个node。

如果不标记used_by DOMID_XEN的话，应该linux直接就可以输出到串口，相当于xen和linux共用这个设备。我想串口这个设备比较简单，这样用也不会有什么问题。是不是所说的passthrough就是把某一设备直接映射到一个domain里，其它domian和xen都不可见呢？这个以后再看。

当linux里的early_printk写物理地址时，一定会引发写异常。这个异常就会由xen处理了。

接下来就看xen怎样处理这个异常。

xen\xen\arch\arm\vuart.c

static int vuart_mmio_check(struct vcpu *v, paddr_taddr)

{

const structvuart_info*info =v->domain->arch.vuart.info;

return (domain_has_vuart(v->domain) &&addr>=info->base_addr&&

addr<= (info->base_addr+info->size));

}

static int vuart_mmio_read(struct vcpu *v, mmio_info_t*info)

{

struct domain *d =v->domain;

struct hsr_dabtdabt =info->dabt;

struct cpu_user_regs *regs=guest_cpu_user_regs();

register_t*r =select_user_reg(regs,dabt.reg);

paddr_t offset =info->gpa -d->arch.vuart.info->base_addr;

/* By defaultzeroed the register */

*r = 0;

if ( offset ==d->arch.vuart.info->status_off)

/* Allholding registers empty, ready to send etc */

*r =d->arch.vuart.info->status;

return 1;

}

static int vuart_mmio_write(struct vcpu *v, mmio_info_t*info)

{

struct domain *d =v->domain;

struct hsr_dabtdabt =info->dabt;

struct cpu_user_regs *regs=guest_cpu_user_regs();

register_t*r =select_user_reg(regs,dabt.reg);

paddr_t offset =info->gpa -d->arch.vuart.info->base_addr;

if ( offset ==d->arch.vuart.info->data_off)

/* ignore anystatus bits */

vuart_print_char(v, *r &0xFF);

return 1;

}

const struct mmio_handlervuart_mmio_handler = {

.check_handler= vuart_mmio_check,

.read_handler = vuart_mmio_read,

.write_handler= vuart_mmio_write,

};

xen\xen\arch\arm\io.c

static const struct mmio_handler *constmmio_handlers[] =

{

&vgic_distr_mmio_handler,

&vuart_mmio_handler,

};

#define MMIO_HANDLER_NRARRAY_SIZE(mmio_handlers)

int handle_mmio(mmio_info_t*info)

{

struct vcpu *v =current;

int i;

for ( i = 0;i <MMIO_HANDLER_NR;i++)

if ( mmio_handlers[i]->check_handler(v,info->gpa))

returninfo->dabt.write ?

mmio_handlers[i]->write_handler(v,info) :

mmio_handlers[i]->read_handler(v,info);

return0;

}

Xen注册的mmio_handler只有两个，一个是vgic一个是vuart，不知道其它的设备是怎样模拟的，比如IDE什么的。这个问题以后在看。

从代码中可以看到通过check_handler判断异常的mmio是不是在自己的处理范围。如果是就分别用write_handler和read_handler做处理，也就是模拟。我想通过mmio和中断这两种操作的模拟，就能模拟所有硬件了吧，vuart只用到了mmio。

最后验证下谁在调用handle_mmio()。

do_trap_hypervisor()->do_trap_data_abort_guest()->handle_mmio()

xen\xen\arch\arm\arm32\entry.S

#defineDEFINE_TRAP_ENTRY(trap) \

ALIGN; \

trap_##trap: \

SAVE_ALL; \

cpsie i; /* local_irq_enable */ \

adr lr, return_from_trap; \

mov r0, sp; \

mov r11, sp; \

bic sp, #7; /* Align the stack pointer(noop on guest trap) */ \

b do_trap_##trap

#defineDEFINE_TRAP_ENTRY_NOIRQ(trap) \

ALIGN; \

trap_##trap: \

SAVE_ALL; \

adr lr, return_from_trap; \

mov r0, sp; \

mov r11, sp; \

bic sp, #7; /* Align the stack pointer(noop on guest trap) */ \

b do_trap_##trap

.align 5

GLOBAL(hyp_traps_vector)

.word 0 /* 0x00 - Reset */

b trap_undefined_instruction /* 0x04 - Undefined Instruction */

b trap_supervisor_call /* 0x08 - Supervisor Call */

b trap_prefetch_abort /* 0x0c - Prefetch Abort */

b trap_data_abort /* 0x10 - Data Abort */

b trap_hypervisor /* 0x14 - Hypervisor */

b trap_irq /* 0x18 - IRQ */

b trap_fiq /* 0x1c - FIQ */

DEFINE_TRAP_ENTRY(undefined_instruction)

DEFINE_TRAP_ENTRY(supervisor_call)

DEFINE_TRAP_ENTRY(prefetch_abort)

DEFINE_TRAP_ENTRY(data_abort)

DEFINE_TRAP_ENTRY(hypervisor)

DEFINE_TRAP_ENTRY_NOIRQ(irq)

DEFINE_TRAP_ENTRY_NOIRQ(fiq)

最终在hyp_traps_vector里的trap_hypervisor调用 do_trap_hypervisor()。

总结下，对于之前的疑问，hypervisor为什么会截获到vuart的mmio物理内存读写操作，是因为xen没有映射对应的物理内存给dom0，dom0在fdt里也没有看到这个device的描述，所以也就不会建立起va到pa的映射，当然early_printk是直接用物理内存访问的，所以在读写的时候，是读写了一段不存在的物理内存，导致hyperviosr截获这个事件。

这也就是为什么我在linux开early_printk的时候，串口输出的消息前都加了个”DOM0:”。

0 0