tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

arm_mpam: resctrl: Add CDP emulation

Intel RDT's CDP feature allows the cache to use a different control value
depending on whether the accesses was for instruction fetch or a data
access. MPAM's equivalent feature is the other way up: the CPU assigns a
different partid label to traffic depending on whether it was instruction
fetch or a data access, which causes the cache to use a different control
value based solely on the partid.

MPAM can emulate CDP, with the side effect that the alternative partid is
seen by all MSC, it can't be enabled per-MSC.

Add the resctrl hooks to turn this on or off. Add the helpers that match a
closid against a task, which need to be aware that the value written to
hardware is not the same as the one resctrl is using.

Update the 'arm64_mpam_global_default' variable the arch code uses during
context switch to know when the per-cpu value should be used instead. Also,
update these per-cpu values and sync the resulting mpam partid/pmg
configuration to hardware.

resctrl can enable CDP for L2 caches, L3 caches or both. When it is enabled
by one and not the other MPAM globally enabled CDP but hides the effect
on the other cache resource. This hiding is possible as CPOR is the only
supported cache control and that uses a resource bitmap; two partids with
the same bitmap act as one.

Awkwardly, the MB controls don't implement CDP and CDP can't be hidden as
the memory bandwidth control is a maximum per partid which can't be
modelled with more partids. If the total maximum is used for both the data
and instruction partids then then the maximum may be exceeded and if it is
split in two then the one using more bandwidth will hit a lower
limit. Hence, hide the MB controls completely if CDP is enabled for any
resource.

Tested-by: Gavin Shan <gshan@redhat.com>
Tested-by: Shaopeng Tan <tan.shaopeng@jp.fujitsu.com>
Tested-by: Peter Newman <peternewman@google.com>
Tested-by: Zeng Heng <zengheng4@huawei.com>
Tested-by: Punit Agrawal <punit.agrawal@oss.qualcomm.com>
Tested-by: Jesse Chick <jessechick@os.amperecomputing.com>
Cc: Dave Martin <Dave.Martin@arm.com>
Cc: Amit Singh Tomar <amitsinght@marvell.com>
Reviewed-by: Zeng Heng <zengheng4@huawei.com>
Reviewed-by: Shaopeng Tan <tan.shaopeng@jp.fujitsu.com>
Reviewed-by: Jonathan Cameron <jonathan.cameron@huawei.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Co-developed-by: Ben Horgan <ben.horgan@arm.com>
Signed-off-by: Ben Horgan <ben.horgan@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>

James Morse 6789fb99 9d2e1a99

+126
+1
arch/arm64/include/asm/mpam.h
··· 4 4 #ifndef __ASM__MPAM_H 5 5 #define __ASM__MPAM_H 6 6 7 + #include <linux/arm_mpam.h> 7 8 #include <linux/bitfield.h> 8 9 #include <linux/jump_label.h> 9 10 #include <linux/percpu.h>
+1
drivers/resctrl/mpam_internal.h
··· 342 342 struct mpam_resctrl_res { 343 343 struct mpam_class *class; 344 344 struct rdt_resource resctrl_res; 345 + bool cdp_enabled; 345 346 }; 346 347 347 348 static inline int mpam_alloc_csu_mon(struct mpam_class *class)
+122
drivers/resctrl/mpam_resctrl.c
··· 35 35 /* The lock for modifying resctrl's domain lists from cpuhp callbacks. */ 36 36 static DEFINE_MUTEX(domain_list_lock); 37 37 38 + /* 39 + * MPAM emulates CDP by setting different PARTID in the I/D fields of MPAM0_EL1. 40 + * This applies globally to all traffic the CPU generates. 41 + */ 38 42 static bool cdp_enabled; 39 43 40 44 bool resctrl_arch_alloc_capable(void) ··· 52 48 } 53 49 54 50 return false; 51 + } 52 + 53 + bool resctrl_arch_get_cdp_enabled(enum resctrl_res_level rid) 54 + { 55 + return mpam_resctrl_controls[rid].cdp_enabled; 56 + } 57 + 58 + /** 59 + * resctrl_reset_task_closids() - Reset the PARTID/PMG values for all tasks. 60 + * 61 + * At boot, all existing tasks use partid zero for D and I. 62 + * To enable/disable CDP emulation, all these tasks need relabelling. 63 + */ 64 + static void resctrl_reset_task_closids(void) 65 + { 66 + struct task_struct *p, *t; 67 + 68 + read_lock(&tasklist_lock); 69 + for_each_process_thread(p, t) { 70 + resctrl_arch_set_closid_rmid(t, RESCTRL_RESERVED_CLOSID, 71 + RESCTRL_RESERVED_RMID); 72 + } 73 + read_unlock(&tasklist_lock); 74 + } 75 + 76 + int resctrl_arch_set_cdp_enabled(enum resctrl_res_level rid, bool enable) 77 + { 78 + u32 partid_i = RESCTRL_RESERVED_CLOSID, partid_d = RESCTRL_RESERVED_CLOSID; 79 + int cpu; 80 + 81 + /* 82 + * resctrl_arch_set_cdp_enabled() is only called with enable set to 83 + * false on error and unmount. 84 + */ 85 + cdp_enabled = enable; 86 + mpam_resctrl_controls[rid].cdp_enabled = enable; 87 + 88 + /* The mbw_max feature can't hide cdp as it's a per-partid maximum. */ 89 + if (cdp_enabled && !mpam_resctrl_controls[RDT_RESOURCE_MBA].cdp_enabled) 90 + mpam_resctrl_controls[RDT_RESOURCE_MBA].resctrl_res.alloc_capable = false; 91 + 92 + if (mpam_resctrl_controls[RDT_RESOURCE_MBA].cdp_enabled && 93 + mpam_resctrl_controls[RDT_RESOURCE_MBA].class) 94 + mpam_resctrl_controls[RDT_RESOURCE_MBA].resctrl_res.alloc_capable = true; 95 + 96 + if (enable) { 97 + if (mpam_partid_max < 1) 98 + return -EINVAL; 99 + 100 + partid_d = resctrl_get_config_index(RESCTRL_RESERVED_CLOSID, CDP_DATA); 101 + partid_i = resctrl_get_config_index(RESCTRL_RESERVED_CLOSID, CDP_CODE); 102 + } 103 + 104 + mpam_set_task_partid_pmg(current, partid_d, partid_i, 0, 0); 105 + WRITE_ONCE(arm64_mpam_global_default, mpam_get_regval(current)); 106 + 107 + resctrl_reset_task_closids(); 108 + 109 + for_each_possible_cpu(cpu) 110 + mpam_set_cpu_defaults(cpu, partid_d, partid_i, 0, 0); 111 + on_each_cpu(resctrl_arch_sync_cpu_closid_rmid, NULL, 1); 112 + 113 + return 0; 114 + } 115 + 116 + static bool mpam_resctrl_hide_cdp(enum resctrl_res_level rid) 117 + { 118 + return cdp_enabled && !resctrl_arch_get_cdp_enabled(rid); 55 119 } 56 120 57 121 /* ··· 185 113 186 114 mpam_set_task_partid_pmg(tsk, partid_d, partid_i, rmid, rmid); 187 115 } 116 + } 117 + 118 + bool resctrl_arch_match_closid(struct task_struct *tsk, u32 closid) 119 + { 120 + u64 regval = mpam_get_regval(tsk); 121 + u32 tsk_closid = FIELD_GET(MPAM0_EL1_PARTID_D, regval); 122 + 123 + if (cdp_enabled) 124 + tsk_closid >>= 1; 125 + 126 + return tsk_closid == closid; 127 + } 128 + 129 + /* The task's pmg is not unique, the partid must be considered too */ 130 + bool resctrl_arch_match_rmid(struct task_struct *tsk, u32 closid, u32 rmid) 131 + { 132 + u64 regval = mpam_get_regval(tsk); 133 + u32 tsk_closid = FIELD_GET(MPAM0_EL1_PARTID_D, regval); 134 + u32 tsk_rmid = FIELD_GET(MPAM0_EL1_PMG_D, regval); 135 + 136 + if (cdp_enabled) 137 + tsk_closid >>= 1; 138 + 139 + return (tsk_closid == closid) && (tsk_rmid == rmid); 188 140 } 189 141 190 142 struct rdt_resource *resctrl_arch_get_resource(enum resctrl_res_level l) ··· 343 247 dom = container_of(d, struct mpam_resctrl_dom, resctrl_ctrl_dom); 344 248 cprops = &res->class->props; 345 249 250 + /* 251 + * When CDP is enabled, but the resource doesn't support it, 252 + * the control is cloned across both partids. 253 + * Pick one at random to read: 254 + */ 255 + if (mpam_resctrl_hide_cdp(r->rid)) 256 + type = CDP_DATA; 257 + 346 258 partid = resctrl_get_config_index(closid, type); 347 259 cfg = &dom->ctrl_comp->cfg[partid]; 348 260 ··· 378 274 int resctrl_arch_update_one(struct rdt_resource *r, struct rdt_ctrl_domain *d, 379 275 u32 closid, enum resctrl_conf_type t, u32 cfg_val) 380 276 { 277 + int err; 381 278 u32 partid; 382 279 struct mpam_config cfg; 383 280 struct mpam_props *cprops; ··· 395 290 res = container_of(r, struct mpam_resctrl_res, resctrl_res); 396 291 dom = container_of(d, struct mpam_resctrl_dom, resctrl_ctrl_dom); 397 292 cprops = &res->class->props; 293 + 294 + if (mpam_resctrl_hide_cdp(r->rid)) 295 + t = CDP_DATA; 398 296 399 297 partid = resctrl_get_config_index(closid, t); 400 298 if (!r->alloc_capable || partid >= resctrl_arch_get_num_closid(r)) { ··· 419 311 break; 420 312 default: 421 313 return -EINVAL; 314 + } 315 + 316 + /* 317 + * When CDP is enabled, but the resource doesn't support it, we need to 318 + * apply the same configuration to the other partid. 319 + */ 320 + if (mpam_resctrl_hide_cdp(r->rid)) { 321 + partid = resctrl_get_config_index(closid, CDP_CODE); 322 + err = mpam_apply_config(dom->ctrl_comp, partid, &cfg); 323 + if (err) 324 + return err; 325 + 326 + partid = resctrl_get_config_index(closid, CDP_DATA); 327 + return mpam_apply_config(dom->ctrl_comp, partid, &cfg); 422 328 } 423 329 424 330 return mpam_apply_config(dom->ctrl_comp, partid, &cfg);
+2
include/linux/arm_mpam.h
··· 56 56 void resctrl_arch_set_closid_rmid(struct task_struct *tsk, u32 closid, u32 rmid); 57 57 void resctrl_arch_set_cpu_default_closid_rmid(int cpu, u32 closid, u32 rmid); 58 58 void resctrl_arch_sched_in(struct task_struct *tsk); 59 + bool resctrl_arch_match_closid(struct task_struct *tsk, u32 closid); 60 + bool resctrl_arch_match_rmid(struct task_struct *tsk, u32 closid, u32 rmid); 59 61 60 62 /** 61 63 * mpam_register_requestor() - Register a requestor with the MPAM driver