arm_mpam: Add workaround for T241-MPAM-4

In the T241 implementation of memory-bandwidth partitioning, in the absence
of contention for bandwidth, the minimum bandwidth setting can affect the
amount of achieved bandwidth. Specifically, the achieved bandwidth in the
absence of contention can settle to any value between the values of
MPAMCFG_MBW_MIN and MPAMCFG_MBW_MAX. Also, if MPAMCFG_MBW_MIN is set
zero (below 0.78125%), once a core enters a throttled state, it will never
leave that state.

The first issue is not a concern if the MPAM software allows to program
MPAMCFG_MBW_MIN through the sysfs interface. This patch ensures program
MBW_MIN=1 (0.78125%) whenever MPAMCFG_MBW_MIN=0 is programmed.

In the scenario where the resctrl doesn't support the MBW_MIN interface via
sysfs, to achieve bandwidth closer to MBW_MAX in the absence of contention,
software should configure a relatively narrow gap between MBW_MIN and
MBW_MAX. The recommendation is to use a 5% gap to mitigate the problem.

Clear the feature MBW_MIN feature from the class to ensure we don't
accidentally change behaviour when resctrl adds support for a MBW_MIN
interface.

Tested-by: Gavin Shan <gshan@redhat.com>
Tested-by: Shaopeng Tan <tan.shaopeng@jp.fujitsu.com>
Tested-by: Jesse Chick <jessechick@os.amperecomputing.com>
Reviewed-by: Zeng Heng <zengheng4@huawei.com>
Reviewed-by: Shaopeng Tan <tan.shaopeng@jp.fujitsu.com>
Reviewed-by: Fenghua Yu <fenghuay@nvidia.com>
Reviewed-by: Gavin Shan <gshan@redhat.com>
Signed-off-by: Shanker Donthineni <sdonthineni@nvidia.com>
Signed-off-by: Ben Horgan <ben.horgan@arm.com>
Signed-off-by: James Morse <james.morse@arm.com>

authored by

Shanker Donthineni and committed by

James Morse 1 month ago a7efe23e 70e81fbe

+55 -3

3 changed files

expand all

Documentation

arch

arm64

silicon-errata.rst

drivers

resctrl

mpam_devices.c

mpam_internal.h

Documentation/arch/arm64/silicon-errata.rst

··· 249 249 +----------------+-----------------+-----------------+-----------------------------+ 250 250 | NVIDIA | T241 MPAM | T241-MPAM-1 | N/A | 251 251 +----------------+-----------------+-----------------+-----------------------------+ 252 + | NVIDIA | T241 MPAM | T241-MPAM-4 | N/A | 253 + +----------------+-----------------+-----------------+-----------------------------+ 252 254 +----------------+-----------------+-----------------+-----------------------------+ 253 255 | Freescale/NXP | LS2080A/LS1043A | A-008585 | FSL_ERRATUM_A008585 | 254 256 +----------------+-----------------+-----------------+-----------------------------+

+52 -3

drivers/resctrl/mpam_devices.c

··· 679 679 .iidr_mask = MPAM_IIDR_MATCH_ONE, 680 680 .workaround = T241_SCRUB_SHADOW_REGS, 681 681 }, 682 + { 683 + /* NVIDIA t241 erratum T241-MPAM-4 */ 684 + .iidr = MPAM_IIDR_NVIDIA_T241, 685 + .iidr_mask = MPAM_IIDR_MATCH_ONE, 686 + .workaround = T241_FORCE_MBW_MIN_TO_ONE, 687 + }, 682 688 { NULL } /* Sentinel */ 683 689 }; 684 690 ··· 1470 1464 mpam_apply_t241_erratum(ris, partid); 1471 1465 } 1472 1466 1467 + static u16 mpam_wa_t241_force_mbw_min_to_one(struct mpam_props *props) 1468 + { 1469 + u16 max_hw_value, min_hw_granule, res0_bits; 1470 + 1471 + res0_bits = 16 - props->bwa_wd; 1472 + max_hw_value = ((1 << props->bwa_wd) - 1) << res0_bits; 1473 + min_hw_granule = ~max_hw_value; 1474 + 1475 + return min_hw_granule + 1; 1476 + } 1477 + 1478 + static u16 mpam_wa_t241_calc_min_from_max(struct mpam_props *props, 1479 + struct mpam_config *cfg) 1480 + { 1481 + u16 val = 0; 1482 + u16 max; 1483 + u16 delta = ((5 * MPAMCFG_MBW_MAX_MAX) / 100) - 1; 1484 + 1485 + if (mpam_has_feature(mpam_feat_mbw_max, cfg)) { 1486 + max = cfg->mbw_max; 1487 + } else { 1488 + /* Resetting. Hence, use the ris specific default. */ 1489 + max = GENMASK(15, 16 - props->bwa_wd); 1490 + } 1491 + 1492 + if (max > delta) 1493 + val = max - delta; 1494 + 1495 + return val; 1496 + } 1497 + 1473 1498 /* Called via IPI. Call while holding an SRCU reference */ 1474 1499 static void mpam_reprogram_ris_partid(struct mpam_msc_ris *ris, u16 partid, 1475 1500 struct mpam_config *cfg) ··· 1541 1504 mpam_write_partsel_reg(msc, MBW_PBM, cfg->mbw_pbm); 1542 1505 } 1543 1506 1544 - if (mpam_has_feature(mpam_feat_mbw_min, rprops) && 1545 - mpam_has_feature(mpam_feat_mbw_min, cfg)) 1546 - mpam_write_partsel_reg(msc, MBW_MIN, 0); 1507 + if (mpam_has_feature(mpam_feat_mbw_min, rprops)) { 1508 + u16 val = 0; 1509 + 1510 + if (mpam_has_quirk(T241_FORCE_MBW_MIN_TO_ONE, msc)) { 1511 + u16 min = mpam_wa_t241_force_mbw_min_to_one(rprops); 1512 + 1513 + val = mpam_wa_t241_calc_min_from_max(rprops, cfg); 1514 + val = max(val, min); 1515 + } 1516 + 1517 + mpam_write_partsel_reg(msc, MBW_MIN, val); 1518 + } 1547 1519 1548 1520 if (mpam_has_feature(mpam_feat_mbw_max, rprops)) { 1549 1521 if (mpam_has_feature(mpam_feat_mbw_max, cfg)) ··· 2334 2288 2335 2289 list_for_each_entry(vmsc, &comp->vmsc, comp_list) 2336 2290 __class_props_mismatch(class, vmsc); 2291 + 2292 + if (mpam_has_quirk(T241_FORCE_MBW_MIN_TO_ONE, class)) 2293 + mpam_clear_feature(mpam_feat_mbw_min, &class->props); 2337 2294 } 2338 2295 2339 2296 /*

drivers/resctrl/mpam_internal.h

··· 224 224 /* Workaround bits for msc->quirks */ 225 225 enum mpam_device_quirks { 226 226 T241_SCRUB_SHADOW_REGS, 227 + T241_FORCE_MBW_MIN_TO_ONE, 227 228 MPAM_QUIRK_LAST 228 229 }; 229 230

Configure Feed

Configure Feed