[知其然不知其所以然-14] cpu hotplug引出的cgroup故障

How to reprduce:

root@turing:/sys/devices/system/cpu/cpu2# pwd
/sys/devices/system/cpu/cpu2
root@turing:/sys/devices/system/cpu/cpu2# echo 0 > online

root@turing:/sys/devices/system/cpu/cpu2# echo 1 > online

root@turing:/sys/devices/system/cpu/cpu2# taskset -c 2 ls

since the taskset will invoke sched_setaffinity to set cpumask for task,

after investigation, the problem is caused by incorrect value of task's subsys - cpuset

's  effective_cpus mask, it is still 0,1,3 after re-pluggin cpu2. and the value of effective_cpus

is updated in workqueue of cpuset_hotplug_workfn, and the effective_cpus will be updated

by cpu_active_mask dynamically:

cpumask_copy(&new_cpus, cpu_active_mask);

//if not equal, update effective_cpus

cpus_updated = !cpumask_equal(top_cpuset.effective_cpus, &new_cpus);

cpumask_copy(top_cpuset.effective_cpus, &new_cpus);

// then update the tree recusively

cpuset_for_each_descendant_pre(cs, pos_css, &top_cpuset) {

cpuset_hotplug_update_tasks(cs);

}

thus the child cpuset effective_mask will be updated according to top_cpuset:

cpumask_and(&new_cpus, cs->cpus_allowed, parent_cs(cs)->effective_cpus);

cpumask_copy(cs->cpus_allowed, new_cpus);
cpumask_copy(cs->effective_cpus, new_cpus);

finnaly all the tasks inside this cpuset will be updated :

while ((task = css_task_iter_next(&it)))
set_cpus_allowed_ptr(task, cs->effective_cpus);

Then, who will trigger the workqueue ? the answer is cpu online/offline,

and the online callback is cpuset_cpu_active, while the offline callback is

cpuset_cpu_inactive, which are registered in sched_init_smp:

hotcpu_notifier(cpuset_cpu_active, CPU_PRI_CPUSET_ACTIVE);
hotcpu_notifier(cpuset_cpu_inactive, CPU_PRI_CPUSET_INACTIVE);

As we mentioned before , the problem is that after re-plugging, the 

the effective_cpus is not updated to 0,1,2,3, which should be  updated

according to cpu_active_mask  by cpuset_cpu_active.

and how cpu_active_mask  is updated?  it is updated by set_cpu_online

and set_cpu_active.