i387: move AMD K7/K8 fpu fxsave/fxrstor workaround from save to restore

The AMD K7/K8 CPUs don't save/restore FDP/FIP/FOP unless an exception is
pending. In order to not leak FIP state from one process to another, we
need to do a floating point load after the fxsave of the old process,
and before the fxrstor of the new FPU state. That resets the state to
the (uninteresting) kernel load, rather than some potentially sensitive
user information.

We used to do this directly after the FPU state save, but that is
actually very inconvenient, since it

(a) corrupts what is potentially perfectly good FPU state that we might
want to lazy avoid restoring later and

(b) on x86-64 it resulted in a very annoying ordering constraint, where
"__unlazy_fpu()" in the task switch needs to be delayed until after
the DS segment has been reloaded just to get the new DS value.

Coupling it to the fxrstor instead of the fxsave automatically avoids
both of these issues, and also ensures that we only do it when actually
necessary (the FP state after a save may never actually get used). It's
simply a much more natural place for the leaked state cleanup.

Signed-off-by: Linus Torvalds <torvalds@linux-foundation.org>

Linus Torvalds 14 years ago 4903062b b3b0870e

+16 -22

3 changed files

expand all

arch

x86

include

asm

i387.h

kernel

process_64.c

traps.c

-19

arch/x86/include/asm/i387.h

··· 211 211 212 212 #endif /* CONFIG_X86_64 */ 213 213 214 - /* We need a safe address that is cheap to find and that is already 215 - in L1 during context switch. The best choices are unfortunately 216 - different for UP and SMP */ 217 - #ifdef CONFIG_SMP 218 - #define safe_address (__per_cpu_offset[0]) 219 - #else 220 - #define safe_address (__get_cpu_var(kernel_cpustat).cpustat[CPUTIME_USER]) 221 - #endif 222 - 223 214 /* 224 215 * These must be called with preempt disabled 225 216 */ ··· 234 243 235 244 if (unlikely(fpu->state->fxsave.swd & X87_FSW_ES)) 236 245 asm volatile("fnclex"); 237 - 238 - /* AMD K7/K8 CPUs don't save/restore FDP/FIP/FOP unless an exception 239 - is pending. Clear the x87 state here by setting it to fixed 240 - values. safe_address is a random variable that should be in L1 */ 241 - alternative_input( 242 - ASM_NOP8 ASM_NOP2, 243 - "emms\n\t" /* clear stack tags */ 244 - "fildl %P[addr]", /* set F?P to defined value */ 245 - X86_FEATURE_FXSAVE_LEAK, 246 - [addr] "m" (safe_address)); 247 246 } 248 247 249 248 static inline void __save_init_fpu(struct task_struct *tsk)

+2 -3

arch/x86/kernel/process_64.c

··· 387 387 struct tss_struct *tss = &per_cpu(init_tss, cpu); 388 388 unsigned fsindex, gsindex; 389 389 390 + __unlazy_fpu(prev_p); 391 + 390 392 /* 391 393 * Reload esp0, LDT and the page table pointer: 392 394 */ ··· 416 414 savesegment(gs, gsindex); 417 415 418 416 load_TLS(next, cpu); 419 - 420 - /* Must be after DS reload */ 421 - __unlazy_fpu(prev_p); 422 417 423 418 /* 424 419 * Leave lazy mode, flushing any hypercalls made here.

+14

arch/x86/kernel/traps.c

··· 585 585 struct thread_info *thread = current_thread_info(); 586 586 struct task_struct *tsk = thread->task; 587 587 588 + /* We need a safe address that is cheap to find and that is already 589 + in L1. We just brought in "thread->task", so use that */ 590 + #define safe_address (thread->task) 591 + 588 592 if (!tsk_used_math(tsk)) { 589 593 local_irq_enable(); 590 594 /* ··· 605 601 } 606 602 607 603 __thread_fpu_begin(thread); 604 + 605 + /* AMD K7/K8 CPUs don't save/restore FDP/FIP/FOP unless an exception 606 + is pending. Clear the x87 state here by setting it to fixed 607 + values. safe_address is a random variable that should be in L1 */ 608 + alternative_input( 609 + ASM_NOP8 ASM_NOP2, 610 + "emms\n\t" /* clear stack tags */ 611 + "fildl %P[addr]", /* set F?P to defined value */ 612 + X86_FEATURE_FXSAVE_LEAK, 613 + [addr] "m" (safe_address)); 608 614 609 615 /* 610 616 * Paranoid restore. send a SIGSEGV if we fail to restore the state.

Configure Feed

Configure Feed