xenbus
来源:互联网 发布:淘宝钱夫人和王总 编辑:程序博客网 时间:2024/05/22 15:02
【转】XenBus的结构
xenbus <--> xenstore
xenbus和xenstore通信的代码主要在pvops/drivers/xen/xenbus/目录下的xenbus_xs.c xenbus_comms.c
xenbus驱动在初始化时,会调用xs_init函数,xs_init调用xs_init_comms先初始化xenbus和xenstore通信的两个ring。最后通过kthread_run调用xenwatch_thread,xenbus_thread两个内核线程。通过ps可以看到[xenwatch] [xenbus]。
[xenwatch]线程会调用wait_event_interruptible来监控watches_events链表,所有需要watch的xenstore node都在这个watches_events里面。对应的wait_queue是watch_events_waitq。OK, 下面遍历开始,对watch_events里每一个list_head的结构,通过list_entry找到包含他的xs_stored_msg,调用xs_stored_msg.u.watch.handle->callback函数。
[xenbus]线程循环调用process_msg,前面我们知道,xenbus和xenstore通过两个ring通信,如果ring上有xenstore返回给xenbus的rsp,这时break出循环开始处理rsp。首先kmalloc一块xs_stored_msg的内存,先读取msg的head,然后msg的body。如何msg是一个watch event,把event挂到watch_events中,然后wake_upwatch_events_waitq(这玩意儿像个sem),如果是xenstore的reply,就挂到xs_state.reply_list中,并wake_upreply_waitq。
xenbus_xs.c中,xs_talkv是xenbus和xenstore通信的核心函数,xs_talkv调用xb_write把请求发到xenstore,调用read_reply接受xenstore的rsp。
xenbus_directory, xenbus_read,xenbus_write, xenbus_exists, xenbus_rm, xenbus_mkdir等一系列xenbus_xxxx函数都是基于上述方法
xenbus_comms.c中,主要都是底层的通信实现
xenbus <--> frontend, backend
xenbus与前后端设备打交道也是其重要功能之一。无论前端还是后端设备,在xenbus看来都是一个xenbus_device结构。
xenbus_client.c
xenbus_grant_ring:前(后)端给后(前)端设备某个ring的access
__xenbus_switch_state:改变前(后)端设备的state
xenbus_alloc_evtchn:为前(后)端和后(前)端设备通信建立event channel。其中本地domain是DOMID_SELF,remote_domain是dev->otherend_id,最后调HYPERVISOR_event_channel_op
xenbus_bind_evtchn:为本地event channel绑定一个remote port。我们知道event channel一般有两个port,传进来bind_interdomain.remote_dom = dev->otherend_id,和bind_interdomain.remote_port。通过调用HYPERVISOR_event_channel_op(EVTCHNOP_bind_interdomain),得到本地的port为bind_interdomain.local_port
xenbus_map_ring_valloc:授权dev->otherend_id的domain来访问一个page。该page是通过xen_alloc_vm_area(PAGE_SIZE)生成出来的。
xenbus_map_ring:授权dev->otherend_id的domain来访问vaddr所在的内存
xenbus_probe.c
xenbus_read_otherend_details:前(后)端调用xenbus_gather收集后(前)端信息。xenbus_gather最终调用xenbus_read,通过xenstore读出。
xenbus_otherend_changed:调用xenbus_read_driver_state,读取另一端设备状态
xenbus_dev_probe:传入的struct device结构__dev为本地端设备,通过to_xenbus_device,to_xenbus_driver得到struct xenbus_device *dev, struct xenbus_driver *drv结构。接着调用talk_to_otherend得到另一端设备信息,最后调用drv->probe。其实核心还是drv->probe,用来发现dev设备是不是在xenbus上(??这个我不太确定)
xenbus_dev_remove:核心也是调用drv->remove函数,其中drv是传入的_dev对应的xenbus_driver。之后把dev对应的xenbus state置为XenbusStateClosed
xenbus_dev_shutdown:调用xenbus_switch_state,修改dev的xenbus状态
get_device, put_device:dev对应的引用计数
xenbus_probe_node:基于nodename,new一片内存并生成一个xenbus_device结构体。最后调用device_register注册该设备。
xenbus_probe_device_type:探测bus下设备的类型
xenbus_init
xenbus_init是xenbus的初始化函数(废话,看名字就知道啦)。首先判断下是不是xen_initial_domain,如果在dom0里执行,要做一系列初始化的工作,包括
申请一个page的空间,用于和xenstore通信,并且把地址写到start_info那个共享page中
调用HYPERVISOR_event_channel_op,创建一个event channel,并把event channel其中一个port写到start_info里面。这样xenstore就可以去bind这个port
如果在domU里执行,继续判断是否是hvm,无论是否hvm,最终都为了获取到xenbus和xenstore通信的三个数据结构:共享ring的页空间,event channel的port,并基于该空间生成xenstore_domain_interface结构。
最后调用xs_init,初始化和xenstore通信的工作
我把xenstore的API简单的封装了一下,方便用户空间的C++使用。主要功能:
1. 把char换成了string,让c++去处理那什么gc。xenstore源代码里面的各种"call free() after use!!!"注释看得我很无语。
2. 提供一个共享的xs_handle对象,一般的读写操作不需要自己定义xs_handle和xs_transaction,方便使用。
3. 简单的全局工具函数,拼合路径名、读写键值更加方便,不需要自己去维护buffer
4. 多线程处理监视事件。
下面是几个比较困惑的地方。
1 想让map_path一次能够读入多个参数,实现map_path("a","b", "c")="a/b/c"这样的效果,最好输入参数还能泛型。现在的做法:(-std=c++0x)
- string map_path_n(vector<string> vec)
- {
- string ret;
- for (int i=0; i<vec.size()-1; i++)
- ret+=vec[i]+'/';
- ret+=vec[vec.size()-1];
- return ret;
- }
- map_path_n({pwd, ls[j], "cancel"});
- map_path_n({to_s(1), to_s(2.65), string(char_str)}); // 结果:1/2.65/char_str
至于其他的做法,我首先尝试了一下C语言的stdarg.h的那种声明里用"..."的方式。结果发现这样根本没法知道传入了几个参数。如果参数都是已知大小的倒还好说,但是字符串……
然后尝试使用vector,在调用的时候使用map_path({"a", "b", "c"})这样子。结果编译提示:
- could not convert ‘。。。’ from ‘<花括号里的初始化列表>’ to ‘std::vector<std::basic_string<char> >
然后试了一下map_path((const vector<string>){"a", "b", "c"}); 结果编译通过了。但是这样也太麻烦了吧!
最后在编译选项里面加上了-std=c++0x,然后这样的代码就可以编译通过了。
期待更好的办法。
2 模板推断,重载的问题
- template <typename T> static bool write_key(xs_handle *xh, int xt, string path, T value)
- {...}
- template <typename T> static T read_key(xs_handle *xh, int xt, string path)
- {...}
- template <typename T> static bool write_key(string path, T value)
- { return write_key(xh_shared, xt_shared, path, value); }
- template <typename T> static T read_key(string path)
- { return read_key<T>(xh_shared, xt_shared, path); }
第一个函数write_key,参数表里有模板,在调用4个参数的write_key时,不需要告诉模板类型,编译器可以自己推断。
第二个函数read_key,返回值类型是模板,调用3个参数的read_key时必须加上<T>,否则出错:
- xsutil.hpp:144:48: 错误:对‘xsutil::XSHelper::read_key(xs_handle*&, int&, std::string&)’的调用没有匹配的函数
- xsutil.hpp:144:48: 附注:备选是:
- <em><span style="color:#ff0000;">xsutil.hpp:117:35: 附注:template<class T> static T xsutil::XSHelper::read_key(xs_handle*, int, std::string)
- xsutil.hpp:117:35: 附注: template argument deduction/substitution failed:
- xsutil.hpp:144:48: 附注: couldn't deduce template parameter ‘T’</span></em>
- xsutil.hpp:143:35: 附注:template<class T> static T xsutil::XSHelper::read_key(std::string)
- xsutil.hpp:143:35: 附注: template argument deduction/substitution failed:
- xsutil.hpp:144:48: 附注: cannot convert ‘xsutil::XSHelper::xh_shared’ (type ‘xs_handle*’) to type ‘std::string {aka std::basic_string<char>}’
不解。
3 小技巧:判断线程是否存在(linux)
- inline bool is_on_watch()
- {
- return (th && pthread_kill(th, 0)==0); //send empty signal to check existence
- }
4 如何控制线程的退出。
5
6
7
代码结构:
前面是全局的字符串处理函数:to|from_s, map_path[_n], map_guest, get_feature。后面是一个类Class XSHelper,前半部分静态函数,可以方便地使用共享的连接(xh_shared, xt_shared)操作xenstore;后半部分主要是多线程监视的部分。
- #ifndef _XSUTIL_HPP
- #define _XSUTIL_HPP
- #include "string"
- #include "sstream"
- #include "iostream"
- #include "vector"
- #include "assert.h"
- #include "pthread.h"
- #include "semaphore.h"
- #include "errno.h"
- #include "unistd.h"
- #include "signal.h"
- #include "stdarg.h"
- extern "C"
- {
- //#define USE_PTHREAD
- #include "xs.h" //libxenstore
- #include "xs_lib.h"
- }
- using namespace std;
- namespace xsutil
- {
- //string process func
- template <typename T> const string to_s(T x)
- {
- stringstream ss; ss.clear();
- ss << x;
- return ss.str();
- }
- template <typename T> T from_s(string s)
- {
- T ret;
- stringstream ss; ss.clear();
- ss << s; ss >> ret;
- return ret;
- }
- const char* from_s(string s){return s.c_str();}
- string map_path_n(vector<string> vec)
- {
- string ret;
- for (int i=0; i<vec.size()-1; i++)
- ret+=vec[i]+'/';
- ret+=vec[vec.size()-1];
- return ret;
- }
- string map_path(string dir, string key)
- {
- string ret= dir + '/' + key;
- //cout << "map_path -> " << ret << endl;
- return ret;
- }
- const string map_guest(int domid, string path)
- {
- stringstream ss; ss.clear();
- ss << "/local/domain/" << domid << "/" << path;
- //cout << "map_guest -> " << ss.str() << endl;
- return ss.str();
- }
- template <typename T>
- size_t get_feature(T x)
- {
- const int p=113, size=sizeof(T);
- size_t ret=0, i;
- uint8_t *ptr=(void *)&x;
- for (i=0; i<size; i++)
- ret+=(size_t)ptr[i]*p;
- return ret;
- }
- //used to extract (>_<) xs_handle internal members.
- typedef struct list_head {
- struct list_head *next, *prev;
- }list_head_struct;
- typedef struct
- {
- int fd;
- pthread_t read_thr;
- int read_thr_exists;
- struct list_head watch_list;
- pthread_mutex_t watch_mutex;
- pthread_cond_t watch_condvar;
- int watch_pipe[2];
- struct list_head reply_list;
- pthread_mutex_t reply_mutex;
- pthread_cond_t reply_condvar;
- pthread_mutex_t request_mutex;
- }my_xs_handle;
- #if __GNUC__ > 3
- #define offsetof(a,b) __builtin_offsetof(a,b)
- #else
- #define offsetof(a,b) ((unsigned long)&(((a *)0)->b))
- #endif
- class XSHelper
- {
- //static (shared) functions and members
- protected:
- static xs_handle *xh_shared;
- static int xt_shared;
- public:
- static bool is_open;
- static void open_shared(){ assert(xh_shared=xs_open(0)); is_open=true; }
- static void close_shared(){ xs_close(xh_shared); is_open=false; }
- static void start_shared(){ xt_shared=xs_transaction_start(xh_shared); }
- static void end_shared(){ xs_transaction_end(xh_shared, xt_shared, 0); }
- //read/write key, recursive rm
- template <typename T> static bool write_key(xs_handle *xh, int xt, string path, T value)
- { //note: use sstream "<<" to process type T.
- return xs_write(xh, xt, path.c_str(), to_s(value).c_str(), to_s(value).length());
- }
- template <typename T> static T read_key(xs_handle *xh, int xt, string path)
- {
- T ret;
- stringstream ss; ss.clear();
- int len=0;
- char *buf = xs_read(xh, xt, path.c_str(), &len);
- ss << buf; ss >> ret;
- free(buf);
- return ret;
- }
- static void rmdir(xs_handle *xh, int xt, string dir)
- {
- int n;
- char **ls = xs_directory(xh, xt, dir.c_str(), &n);
- if (ls)
- {
- for (int i=0; i<n; i++)
- rmdir(map_path(dir, ls[i]));
- xs_rm(xh, xt, dir.c_str());
- free(ls);
- }
- }
- //simpler overrided version, use xh_shared and xt_shared
- template <typename T> static bool write_key(string path, T value)
- { return write_key(xh_shared, xt_shared, path, value); }
- template <typename T> static T read_key(string path)
- { return read_key<T>(xh_shared, xt_shared, path); }
- static void rmdir(string dir)
- { return rmdir(xh_shared, xt_shared, dir); }
- static int mkdir(string dir)
- { return xs_mkdir(xh_shared, xt_shared, dir.c_str()); }
- //extra functions
- static vector<string> directory(string dir)
- {
- vector<string> ret; ret.clear();
- int num=0, i;
- //cout << "list-directory -> " << dir << endl;
- char **ls=xs_directory(xh_shared, xt_shared, dir.c_str(), &num);
- for (i=0; i<num; i++)
- ret.push_back(ls[i]);
- free(ls);
- return ret;
- }
- static bool permission(int domid, string path, bool b_read, bool b_write)
- {
- int value=b_read | ((int)b_write) << 1;
- xs_permissions xp={domid, value};
- return xs_set_permissions(xh_shared, xt_shared, path.c_str(), &xp, 1);
- }
- //local members and functions
- protected:
- pthread_t th;
- void (*callback)(XSHelper *pthis, const char *path); //arg = the char* path which xs returns
- bool flag;
- public:
- xs_handle *xh; //used by threads
- int xt; //unused, but callback func may use this
- string path;
- public:
- XSHelper(): xh(0), xt(-1), th(0), flag(false){}
- ~XSHelper()
- {
- if (is_on_watch())
- {
- cout << "dtor -> ";
- unwatch();
- }
- }
- inline bool is_on_watch()
- {
- return (th && pthread_kill(th, 0)==0); //send empty signal to check existence
- }
- inline pthread_t get_thread(){ return th; } //used when join
- void watch(string path_str, void (*watch_callback)(XSHelper *, const char *))
- {
- cout << "watch -> add watch on path " << path_str << " callback func " << hex << (void *)watch_callback << endl;
- path=path_str;
- callback=watch_callback;
- xh=xs_open(0);
- assert(xs_watch(xh, path.c_str(), map_path("watch", path).c_str()));
- pthread_create(&th, 0, XSHelper::thread_func, (void *)this); //thread does xs_read_watch only.
- }
- void finish() {flag=true;} //used by callback func to make working thread exit normally
- void unwatch() //cancel working thread. cleanup done by main thread
- {
- cout << "unwatch -> ";
- if (is_on_watch())
- {
- pthread_cancel(th);
- cout << "**stopping work thread. waiting here...";
- pthread_join(th, 0);
- cout << "work thread already stopped." << endl;
- thread_cleanup(this);
- }
- else cout << "--work thread already stopped." << endl;
- }
- static void *thread_func(void *arg_this) //work thread functions
- {
- XSHelper *pthis = (XSHelper *)arg_this;
- int num, i;
- sigset_t mask;
- //set up environment
- cout << "thread_func -> begin watch thread path= " << pthis->path << endl;
- sigemptyset(&mask);
- sigaddset(&mask, SIGINT);
- assert(0 == pthread_sigmask(SIG_BLOCK, &mask, 0)); //mask SIGINT
- pthis->flag=false;
- while (!pthis->flag)
- {
- char **result = 0;
- result = xs_read_watch(pthis->xh, &num); //has cancellation point (read) inside.
- if (result)
- {
- for (i=0; i<num; i+=2)
- {
- cout << "thread_func -> watch event path= " << result[i] << " token= " << result[i+1] << endl;
- if (map_path("watch", pthis->path) == string(result[i+1]))
- {
- //cout << "thread_func -> call callback func -> ";
- pthis->callback(pthis, result[i]);
- }
- }
- //cout << "thread_func -> free result" << endl;
- free(result);
- }
- }
- thread_cleanup(arg_this);
- cout << "thread_func -> work thread exit normally." << endl;
- pthread_exit(0);
- }
- #define check_xh_lock(x) do {\
- pthread_mutex_t *pm = (pthread_mutex_t *)(((void *)pthis->xh) + offsetof(my_xs_handle, x)); \
- if (pthread_mutex_trylock(pm) == EBUSY){ \
- cout << "thread_cleanup -> " #x " is already locked!" << endl; \
- if (0 != pthread_mutex_unlock(pm)) \
- cout << "thread_cleanup -> error unlocking!" << endl; \
- else cout << "thread_cleanup -> unlocking " #x << endl; \
- } else assert(pthread_mutex_unlock(pm)==0); \
- } while (0)
- static void thread_cleanup(void *arg_this)
- {
- XSHelper *pthis=(XSHelper *)arg_this;
- cout << "thread_cleanup -> path= " << pthis->path << " thread=" << hex << pthis->th << endl;
- /* The thread is stopped by pthread_cancel
- * however, the xs_read_watch API does not release the mutex lock xh->watch_mutex when cancelled (**possibly a bug?**)
- * but following APIs still try to get the lock -> deadlock here
- * so we extract the watch_mutex from xs_handle struct to unlock it here.
- */
- if (pthis->xh)
- {
- check_xh_lock(watch_mutex);
- check_xh_lock(request_mutex);
- check_xh_lock(reply_mutex);
- cout << "----- unwatch -----" << endl;
- xs_unwatch(pthis->xh, pthis->path.c_str(), map_path("watch", pthis->path).c_str());
- check_xh_lock(watch_mutex);
- check_xh_lock(request_mutex);
- check_xh_lock(reply_mutex);
- cout << "----- close -----" << endl;
- xs_close(pthis->xh);
- pthis->xh=0;
- }
- cout.flush();
- pthis->flag=true;
- }
- };
- //static member declaration
- xs_handle *XSHelper::xh_shared;
- int XSHelper::xt_shared;
- bool XSHelper::is_open;
- }; //namespace
- #endif
- Xenbus
- xenbus
- xenbus
- XenBus and XenStore
- Xenstore和Xenbus
- Xenstore and Xenbus
- xenbus and xenstore
- XenStore 和 Xenbus
- A brief introduction to XenBus from the perspective of the driver
- 根据kplx中的三种情况进行分类,group by 的用法
- XenStore: 使用,结构和原理
- 错误页面配置和java
- 学会jQuery 不用买书!
- eclipse窗口布局重置
- xenbus
- 草图检索和识别[开源]
- 数据库笔试题整理的
- 保护ASP脚本源代码
- LeetCode ZigZag Conversion
- xenbus
- JDK1.5新特性之自动装箱与自动拆箱
- leetcode Combination Sum &Combination Sum II
- XML解析