socket programming in kernel

mostly developers programming socket related function and modules in user space.

but sometimes one may want to do it in kernel layer.

expecially for the open sourced linux kernel.

<1>introduction.

Google's Adnroid projects do have tried this kind once!

for AOSP, it originally implemented PPTP/L2TP protocols partially in the linux kernel, and there is just one kernel socket usage.

source code lies as kernel/drivers/net/ppp/pppopns.c.

in fact, it is an protocol implementation, but the core running logic do base on an socket live in the kernel space.

to make it simple, just let it know that all packets sent from user space are transfered to this module bypppopns_xmit(), and its only work is to put the packets into a queue and wait work queue to handle it, in the work queue it call the socket created in the kernel and sent it out at last.

<2>key structures and flow

its socket impl for the user space was simple, just register the protocol to PPPOX famlily.

static struct proto_ops pppopns_proto_ops = {    .family = PF_PPPOX,    .owner = THIS_MODULE,    .release = pppopns_release,    .bind = sock_no_bind,    .connect = pppopns_connect,    .socketpair = sock_no_socketpair,    .accept = sock_no_accept,    .getname = sock_no_getname,    .poll = sock_no_poll,    .ioctl = pppox_ioctl,    .listen = sock_no_listen,    .shutdown = sock_no_shutdown,    .setsockopt = sock_no_setsockopt,    .getsockopt = sock_no_getsockopt,    .sendmsg = sock_no_sendmsg,    .recvmsg = sock_no_recvmsg,    .mmap = sock_no_mmap,};static int pppopns_create(struct net *net, struct socket *sock){    struct sock *sk;    sk = sk_alloc(net, PF_PPPOX, GFP_KERNEL, &pppopns_proto);    if (!sk)        return -ENOMEM;    sock_init_data(sock, sk);    sock->state = SS_UNCONNECTED;    sock->ops = &pppopns_proto_ops;    sk->sk_protocol = PX_PROTO_OPNS;    sk->sk_state = PPPOX_NONE;    return 0;}

while socket creating, it just makes pppopns_proto_ops to act as the real service,and the protocol only have connect/release, all the packets are in fact from PPP channel after connected.

the key connection is the connect function:

//it is used for PPP channel to sent packetsstatic struct ppp_channel_ops pppopns_channel_ops = {    .start_xmit = pppopns_xmit,};static int pppopns_connect(struct socket *sock, struct sockaddr *useraddr,    int addrlen, int flags){    struct sock *sk = sock->sk;    struct pppox_sock *po = pppox_sk(sk);    struct sockaddr_pppopns *addr = (struct sockaddr_pppopns *)useraddr;...// there is one connected tcp socket fd transfered in    sock_tcp = sockfd_lookup(addr->tcp_socket, &error);...//get the addr from the transfered tcp socket    error = kernel_getpeername(sock_tcp, (struct sockaddr *)&ss, &addrlen);...//create GRE raw socket in the kernel    error = sock_create(ss.ss_family, SOCK_RAW, IPPROTO_GRE, &sock_raw);...//connect to the peer by tcp socket's addr    error = kernel_connect(sock_raw, (struct sockaddr *)&ss, addrlen, 0);...// associate the created kernel socket with the target ppp channel chain    po->chan.private = sk_raw;    po->chan.ops = &pppopns_channel_ops;...//register this channel chain to ppp proto    error = ppp_register_channel(&po->chan); 

after this process, we got an socket created in the kernel and make it ready to receive packets from PPP.

let's see how PPP packets xmited to our socket

static DECLARE_WORK(delivery_work, pppopns_xmit_core);static int pppopns_xmit(struct ppp_channel *chan, struct sk_buff *skb){    struct sock *sk_raw = (struct sock *)chan->private;...    skb_set_owner_w(skb, sk_raw);    skb_queue_tail(&delivery_queue, skb);    schedule_work(&delivery_work); 

just update skb and queue it, waiting for work task.

and the work task is just the real socket operations!

//send packets by kernel socketstatic struct sk_buff_head delivery_queue;static void pppopns_xmit_core(struct work_struct *delivery_work){ mm_segment_t old_fs = get_fs(); struct sk_buff *skb; set_fs(KERNEL_DS);//dequeue packetswhile ((skb = skb_dequeue(&delivery_queue))) { struct sock *sk_raw = skb->sk;//construct msghdr from packets datastruct kvec iov = {.iov_base = skb->data, .iov_len = skb->len}; struct msghdr msg = { .msg_iov = (struct iovec *)&iov, .msg_iovlen = 1, .msg_flags = MSG_NOSIGNAL | MSG_DONTWAIT, };//send packets by kernel socketint ret = sk_raw->sk_prot->sendmsg(NULL, sk_raw, &msg, skb->len); kfree_skb(skb); } set_fs(old_fs);}

dequeuing packets from queue and sending by sk->sk_prot->sendmsg, that's the real kernel socket does!

notice that set_fs/get_fs !? that's one key point of sending packets from kernel socket.

normally most memory address can be accessed in the user space is 0xc0000000, but as these pakcets lay in the kernel memory space but the calling API has user space label, it needs to extend it to the max kernel address limitations to avoid problem by "set_fs(KERNEL_DS)".

after usage these address config need to be restored, so there is one "get_fs" at the front and one "set_fs" in the end.

<3>summary

so in fact there is no any mistery for using socket in the kernel space.

just call sock_create/connect/send/recv methods or relatedsk->sk_prot->xxx method after socket created.

the logic just the same as that used in user space.

and do remember for APIs that have user space address lable adding fs_get/set accordingly.

here is one another impl from github: tcp socket server in kernel.

in fact, there is impl for pptp from linux kernel mainline: kernel/drivers/net/ppp/pptp.c

which does not depend any work queue and just sending packets like an true protocol handles, even without any send/recv method implimentaions either(as packets from PPP).

<4> reference

https://code.google.com/p/android-source-browsing/source/browse/drivers/net/pppopns.c?spec=svn.kernel--samsung.3cc95e30e5cd54c8e0d968993c1b9f4dcae5d544&repo=kernel--samsung&name=android-samsung-2.6.35-gingerbread&r=3cc95e30e5cd54c8e0d968993c1b9f4dcae5d544

https://android.googlesource.com/kernel/samsung/+/3cc95e30e5cd54c8e0d968993c1b9f4dcae5d544/drivers/net/pppopns.c

http://blog.chinaunix.net/uid-20564848-id-74231.html

http://laokaddk.blog.51cto.com/368606/927201/

https://gist.github.com/llj098/752417