[知其然不知其所以然-30] How to work around when system can not be woken up

So previously we found problems that, the platform react very slow to user input, either

from terminal or from remote. One of my colleagues has found that, it is because this platform

can not be woken up from mwait, more precisely, 

Monitored cached line may not wake up from mwait on certainGoldmont based CPUs. This patch will avoid calling current_set_polling_and_test() and thereby not set the TIF_ flag. The result is that we'll always send IPIs for wakeups.

So the comment above is very short, but contains quite a lot of informations.

Let's see the patch:

diff --git a/arch/x86/include/asm/cpufeatures.h b/arch/x86/include/asm/cpufeatures.hindex 4a41348..a5ba0ff 100644--- a/arch/x86/include/asm/cpufeatures.h+++ b/arch/x86/include/asm/cpufeatures.h@@ -303,6 +303,7 @@ #define X86_BUG_SYSRET_SS_ATTRS        X86_BUG(8) /* SYSRET doesn't fix up SS attrs */ #define X86_BUG_NULL_SEG       X86_BUG(9) /* Nulling a selector preserves the base */ #define X86_BUG_SWAPGS_FENCE   X86_BUG(10) /* SWAPGS without input dep on GS */+#define X86_BUG_MONITOR                X86_BUG(11) /* IPI required to wake up remote cpu */   #ifdef CONFIG_X86_32diff --git a/arch/x86/include/asm/mwait.h b/arch/x86/include/asm/mwait.hindex 0deeb2d..f37f2d8 100644--- a/arch/x86/include/asm/mwait.h+++ b/arch/x86/include/asm/mwait.h@@ -97,7 +97,7 @@ static inline void __sti_mwait(unsigned long eax, unsigned long ecx)  */ static inline void mwait_idle_with_hints(unsigned long eax, unsigned long ecx) {-       if (!current_set_polling_and_test()) {+       if (static_cpu_has_bug(X86_BUG_MONITOR) || !current_set_polling_and_test()) {                if (static_cpu_has_bug(X86_BUG_CLFLUSH_MONITOR)) {                        mb();                        clflush((void *)¤t_thread_info()->flags);diff --git a/arch/x86/kernel/cpu/intel.c b/arch/x86/kernel/cpu/intel.cindex 6e2ffbe..a79e8af 100644--- a/arch/x86/kernel/cpu/intel.c+++ b/arch/x86/kernel/cpu/intel.c@@ -509,6 +509,11 @@ static void init_intel(struct cpuinfo_x86 *c)            (c->x86_model == 29 || c->x86_model == 46 || c->x86_model == 47))                set_cpu_bug(c, X86_BUG_CLFLUSH_MONITOR); +       if (c->x86 == 6 && boot_cpu_has(X86_FEATURE_MWAIT) &&+               ((c->x86_model == 0x5f) || (c->x86_model == 0x5c))) {+               set_cpu_bug(c, X86_BUG_MONITOR);+               pr_warn("BIOS update needed to avoid MONITOR BUG!!, INTERNAL WORKAROUND!!");+       }

there are mainly two methods to wake up a idle task on one cpu,

suppose we have CPU1 running, and CPU2 in idle, task A is migrated

from CPU1 to CPU2, and we need to wake CPU2 up:

1.  if CPU2 is at mwait(idle.flag), then CPU1 set the idle.flag with TIF_POLLING_NRFLAG,

    thus CPU2 wakes up from mwait, and switch to task A

2. if CPU2 is not in mwait, thus CPU1 has to send a resched ipi to CPU2, thus CPU2 been woken

up by ipi, and then switch to task A at ret_irq:

void wake_up_if_idle(int cpu){        struct rq *rq = cpu_rq(cpu);        unsigned long flags;        rcu_read_lock();        if (!is_idle_task(rcu_dereference(rq->curr)))                goto out;        if (set_nr_if_polling(rq->idle)) {                trace_sched_wake_idle_without_ipi(cpu);        } else {                raw_spin_lock_irqsave(&rq->lock, flags);                if (is_idle_task(rq->curr))                        smp_send_reschedule(cpu);                /* Else cpu is not in idle, do nothing here */                raw_spin_unlock_irqrestore(&rq->lock, flags);        }out:        rcu_read_unlock();}

static void ttwu_queue_remote(struct task_struct *p, int cpu, int wake_flags){        struct rq *rq = cpu_rq(cpu);        p->sched_remote_wakeup = !!(wake_flags & WF_MIGRATED);        if (llist_add(&p->wake_entry, &cpu_rq(cpu)->wake_list)) {                if (!set_nr_if_polling(rq->idle))                        smp_send_reschedule(cpu);                else                        trace_sched_wake_idle_without_ipi(cpu);        }}

#define TIF_POLLING_NRFLAG      21      /* idle is polling for TIF_NEED_RESCHED */

So for our case,  we want the cpu to be in mwait, but we also want the CPU1 to send ipi to wake ip CPU2,

thus we do not set polling mode for CPU2, although it is in mwait, thus CPU1 will find out CPU2 has no

polling mode set, thus to send ipi to wake CPU2 up.

So the problem above is solved very smartly.