perf/x86/intel: Avoid unnecessary PEBS_ENABLE MSR access in PMI

The perf PMI handler, intel_pmu_handle_irq(), currently does
unnecessary MSR accesses for PEBS_ENABLE MSR in
__intel_pmu_enable/disable_all() when PEBS is enabled.

When entering the handler, global ctrl is explicitly disabled. All
counters do not count anymore. It doesn't matter if PEBS is enabled
or not in a PMI handler.
Furthermore, for most cases, the cpuc->pebs_enabled is not changed in
PMI. The PEBS status doesn't change. The PEBS_ENABLE MSR doesn't need to
be changed either when exiting the handler.

PMI throttle may change the PEBS status during PMI handler. The
x86_pmu_stop() ends up in intel_pmu_pebs_disable() which can update
cpuc->pebs_enabled. But the MSR_IA32_PEBS_ENABLE is not updated
at the same time. Because the cpuc->enabled has been forced to 0.
The patch explicitly update the MSR_IA32_PEBS_ENABLE for this case.

Use ftrace to measure the duration of intel_pmu_handle_irq() on BDX.
#perf record -e cycles:P -- ./tchain_edit

The average duration of intel_pmu_handle_irq():

Without the patch 1.144 us
With the patch 1.025 us

Signed-off-by: Kan Liang <kan.liang@linux.intel.com>
Signed-off-by: Peter Zijlstra (Intel) <peterz@infradead.org>
Signed-off-by: Ingo Molnar <mingo@kernel.org>
Link: https://lkml.kernel.org/r/20200121181338.3234-1-kan.liang@linux.intel.com

authored by

Kan Liang and committed by

Ingo Molnar 6 years ago 6c1c07b3 f861854e

+22 -3

1 changed file

expand all

arch

x86

events

intel

core.c

+22 -3

arch/x86/events/intel/core.c

··· 1945 1945 * intel_bts events don't coexist with intel PMU's BTS events because of 1946 1946 * x86_add_exclusive(x86_lbr_exclusive_lbr); there's no need to keep them 1947 1947 * disabled around intel PMU's event batching etc, only inside the PMI handler. 1948 + * 1949 + * Avoid PEBS_ENABLE MSR access in PMIs. 1950 + * The GLOBAL_CTRL has been disabled. All the counters do not count anymore. 1951 + * It doesn't matter if the PEBS is enabled or not. 1952 + * Usually, the PEBS status are not changed in PMIs. It's unnecessary to 1953 + * access PEBS_ENABLE MSR in disable_all()/enable_all(). 1954 + * However, there are some cases which may change PEBS status, e.g. PMI 1955 + * throttle. The PEBS_ENABLE should be updated where the status changes. 1948 1956 */ 1949 1957 static void __intel_pmu_disable_all(void) 1950 1958 { ··· 1962 1954 1963 1955 if (test_bit(INTEL_PMC_IDX_FIXED_BTS, cpuc->active_mask)) 1964 1956 intel_pmu_disable_bts(); 1965 - 1966 - intel_pmu_pebs_disable_all(); 1967 1957 } 1968 1958 1969 1959 static void intel_pmu_disable_all(void) 1970 1960 { 1971 1961 __intel_pmu_disable_all(); 1962 + intel_pmu_pebs_disable_all(); 1972 1963 intel_pmu_lbr_disable_all(); 1973 1964 } 1974 1965 ··· 1975 1968 { 1976 1969 struct cpu_hw_events *cpuc = this_cpu_ptr(&cpu_hw_events); 1977 1970 1978 - intel_pmu_pebs_enable_all(); 1979 1971 intel_pmu_lbr_enable_all(pmi); 1980 1972 wrmsrl(MSR_CORE_PERF_GLOBAL_CTRL, 1981 1973 x86_pmu.intel_ctrl & ~cpuc->intel_ctrl_guest_mask); ··· 1992 1986 1993 1987 static void intel_pmu_enable_all(int added) 1994 1988 { 1989 + intel_pmu_pebs_enable_all(); 1995 1990 __intel_pmu_enable_all(added, false); 1996 1991 } 1997 1992 ··· 2381 2374 * PEBS overflow sets bit 62 in the global status register 2382 2375 */ 2383 2376 if (__test_and_clear_bit(62, (unsigned long *)&status)) { 2377 + u64 pebs_enabled = cpuc->pebs_enabled; 2378 + 2384 2379 handled++; 2385 2380 x86_pmu.drain_pebs(regs); 2386 2381 status &= x86_pmu.intel_ctrl | GLOBAL_STATUS_TRACE_TOPAPMI; 2382 + 2383 + /* 2384 + * PMI throttle may be triggered, which stops the PEBS event. 2385 + * Although cpuc->pebs_enabled is updated accordingly, the 2386 + * MSR_IA32_PEBS_ENABLE is not updated. Because the 2387 + * cpuc->enabled has been forced to 0 in PMI. 2388 + * Update the MSR if pebs_enabled is changed. 2389 + */ 2390 + if (pebs_enabled != cpuc->pebs_enabled) 2391 + wrmsrl(MSR_IA32_PEBS_ENABLE, cpuc->pebs_enabled); 2387 2392 } 2388 2393 2389 2394 /*

Configure Feed

Configure Feed