tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

rseq: Reset slice extension when scheduled

When a time slice extension was granted in the need_resched() check on exit
to user space, the task can still be scheduled out in one of the other
pending work items. When it gets scheduled back in, and need_resched() is
not set, then the stale grant would be preserved, which is just wrong.

RSEQ already keeps track of that and sets TIF_RSEQ, which invokes the
critical section and ID update mechanisms.

Utilize them and clear the user space slice control member of struct rseq
unconditionally within the existing user access sections. That's just an
unconditional store more in that path.

Signed-off-by: Thomas Gleixner <tglx@linutronix.de>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Reviewed-by: Mathieu Desnoyers <mathieu.desnoyers@efficios.com>
Link: https://patch.msgid.link/20251215155709.131081527@linutronix.de

authored by

Thomas Gleixner and committed by
Peter Zijlstra 7ee58f98 0ac3b5c3

+28 -2
+28 -2
include/linux/rseq_entry.h
··· 102 102 return __rseq_arm_slice_extension_timer(); 103 103 } 104 104 105 + static __always_inline void rseq_slice_clear_grant(struct task_struct *t) 106 + { 107 + if (IS_ENABLED(CONFIG_RSEQ_STATS) && t->rseq.slice.state.granted) 108 + rseq_stat_inc(rseq_stats.s_revoked); 109 + t->rseq.slice.state.granted = false; 110 + } 111 + 105 112 #else /* CONFIG_RSEQ_SLICE_EXTENSION */ 106 113 static inline bool rseq_slice_extension_enabled(void) { return false; } 107 114 static inline bool rseq_arm_slice_extension_timer(void) { return false; } 115 + static inline void rseq_slice_clear_grant(struct task_struct *t) { } 108 116 #endif /* !CONFIG_RSEQ_SLICE_EXTENSION */ 109 117 110 118 bool rseq_debug_update_user_cs(struct task_struct *t, struct pt_regs *regs, unsigned long csaddr); ··· 399 391 unsafe_put_user(ids->mm_cid, &rseq->mm_cid, efault); 400 392 if (csaddr) 401 393 unsafe_get_user(*csaddr, &rseq->rseq_cs, efault); 394 + 395 + /* Open coded, so it's in the same user access region */ 396 + if (rseq_slice_extension_enabled()) { 397 + /* Unconditionally clear it, no point in conditionals */ 398 + unsafe_put_user(0U, &rseq->slice_ctrl.all, efault); 399 + } 402 400 } 403 401 402 + rseq_slice_clear_grant(t); 404 403 /* Cache the new values */ 405 404 t->rseq.ids.cpu_cid = ids->cpu_cid; 406 405 rseq_stat_inc(rseq_stats.ids); ··· 503 488 */ 504 489 u64 csaddr; 505 490 506 - if (unlikely(get_user_inline(csaddr, &rseq->rseq_cs))) 507 - return false; 491 + scoped_user_rw_access(rseq, efault) { 492 + unsafe_get_user(csaddr, &rseq->rseq_cs, efault); 493 + 494 + /* Open coded, so it's in the same user access region */ 495 + if (rseq_slice_extension_enabled()) { 496 + /* Unconditionally clear it, no point in conditionals */ 497 + unsafe_put_user(0U, &rseq->slice_ctrl.all, efault); 498 + } 499 + } 500 + 501 + rseq_slice_clear_grant(t); 508 502 509 503 if (static_branch_unlikely(&rseq_debug_enabled) || unlikely(csaddr)) { 510 504 if (unlikely(!rseq_update_user_cs(t, regs, csaddr))) ··· 529 505 u32 node_id = cpu_to_node(ids.cpu_id); 530 506 531 507 return rseq_update_usr(t, regs, &ids, node_id); 508 + efault: 509 + return false; 532 510 } 533 511 534 512 static __always_inline bool __rseq_exit_to_user_mode_restart(struct pt_regs *regs)