Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

selftests/resctrl: Ensure measurements skip initialization of default benchmark

The CMT, MBA, and MBM tests rely on the resctrl_val() wrapper to
start and run a benchmark while providing test specific flows
via callbacks to do test specific configuration and measurements.

At a high level, the resctrl_val() flow is:
a) Start by fork()ing a child process that installs a signal
handler for SIGUSR1 that, on receipt of SIGUSR1, will
start running a benchmark.
b) Assign the child process created in (a) to the resctrl
control and monitoring group that dictates the memory and
cache allocations with which the process can run and will
contain all resctrl monitoring data of that process.
c) Once parent and child are considered "ready" (determined via
a message over a pipe) the parent signals the child (via
SIGUSR1) to start the benchmark, waits one second for the
benchmark to run, and then starts collecting monitoring data
for the tests, potentially also changing allocation
configuration depending on the various test callbacks.

A problem with the above flow is the "black box" view of the
benchmark that is combined with an arbitrarily chosen
"wait one second" before measurements start. No matter what
the benchmark does, it is given one second to initialize before
measurements start.

The default benchmark "fill_buf" consists of two parts,
first it prepares a buffer (allocate, initialize, then flush), then it
reads from the buffer (in unpredictable ways) until terminated.
Depending on the system and the size of the buffer, the first "prepare"
part may not be complete by the time the one second delay expires. Test
measurements may thus start before the work needing to be measured runs.

Split the default benchmark into its "prepare" and "runtime" parts and
simplify the resctrl_val() wrapper while doing so. This same split
cannot be done for the user provided benchmark (without a user
interface change), so the current behavior is maintained for user
provided benchmark.

Assign the test itself to the control and monitoring group and run the
"prepare" part of the benchmark in this context, ensuring it runs with
required cache and memory bandwidth allocations. With the benchmark
preparation complete it is only needed to fork() the "runtime" part
of the benchmark (or entire user provided benchmark).

Keep the "wait one second" delay before measurements start. For the
default "fill_buf" benchmark this time now covers only the "runtime"
portion that needs to be measured. For the user provided benchmark this
delay maintains current behavior.

Signed-off-by: Reinette Chatre <reinette.chatre@intel.com>
Reviewed-by: Ilpo Järvinen <ilpo.jarvinen@linux.intel.com>
Signed-off-by: Shuah Khan <skhan@linuxfoundation.org>

authored by

Reinette Chatre and committed by
Shuah Khan
3cb3f0b8 e958c21e

+52 -163
-15
tools/testing/selftests/resctrl/fill_buf.c
··· 129 129 130 130 return buf; 131 131 } 132 - 133 - int run_fill_buf(size_t buf_size, bool memflush) 134 - { 135 - unsigned char *buf; 136 - 137 - buf = alloc_buffer(buf_size, memflush); 138 - if (!buf) 139 - return -1; 140 - 141 - fill_cache_read(buf, buf_size, false); 142 - 143 - free(buf); 144 - 145 - return 0; 146 - }
-1
tools/testing/selftests/resctrl/resctrl.h
··· 169 169 unsigned char *alloc_buffer(size_t buf_size, bool memflush); 170 170 void mem_flush(unsigned char *buf, size_t buf_size); 171 171 void fill_cache_read(unsigned char *buf, size_t buf_size, bool once); 172 - int run_fill_buf(size_t buf_size, bool memflush); 173 172 int initialize_read_mem_bw_imc(void); 174 173 int measure_read_mem_bw(const struct user_params *uparams, 175 174 struct resctrl_val_param *param, pid_t bm_pid);
+52 -147
tools/testing/selftests/resctrl/resctrl_val.c
··· 373 373 return 0; 374 374 } 375 375 376 - static pid_t bm_pid, ppid; 376 + static pid_t bm_pid; 377 377 378 378 void ctrlc_handler(int signum, siginfo_t *info, void *ptr) 379 379 { ··· 429 429 sigaction(SIGHUP, &sigact, NULL)) { 430 430 ksft_perror("sigaction"); 431 431 } 432 - } 433 - 434 - static void parent_exit(pid_t ppid) 435 - { 436 - kill(ppid, SIGKILL); 437 - umount_resctrlfs(); 438 - exit(EXIT_FAILURE); 439 432 } 440 433 441 434 /* ··· 528 535 return ret; 529 536 } 530 537 531 - struct benchmark_info { 532 - const struct user_params *uparams; 533 - struct resctrl_val_param *param; 534 - }; 535 - 536 - /* 537 - * run_benchmark - Run a specified benchmark or fill_buf (default benchmark) 538 - * in specified signal. Direct benchmark stdio to /dev/null. 539 - * @signum: signal number 540 - * @info: signal info 541 - * @ucontext: user context in signal handling 542 - */ 543 - static void run_benchmark(int signum, siginfo_t *info, void *ucontext) 544 - { 545 - struct benchmark_info *benchmark_info = info->si_ptr; 546 - const struct user_params *uparams = benchmark_info->uparams; 547 - struct resctrl_val_param *param = benchmark_info->param; 548 - FILE *fp; 549 - int ret; 550 - 551 - /* 552 - * Direct stdio of child to /dev/null, so that only parent writes to 553 - * stdio (console) 554 - */ 555 - fp = freopen("/dev/null", "w", stdout); 556 - if (!fp) { 557 - ksft_perror("Unable to direct benchmark status to /dev/null"); 558 - parent_exit(ppid); 559 - } 560 - 561 - if (param->fill_buf) { 562 - if (run_fill_buf(param->fill_buf->buf_size, 563 - param->fill_buf->memflush)) 564 - fprintf(stderr, "Error in running fill buffer\n"); 565 - } else if (uparams->benchmark_cmd[0]) { 566 - /* Execute specified benchmark */ 567 - ret = execvp(uparams->benchmark_cmd[0], (char **)uparams->benchmark_cmd); 568 - if (ret) 569 - ksft_perror("execvp"); 570 - } 571 - 572 - fclose(stdout); 573 - ksft_print_msg("Unable to run specified benchmark\n"); 574 - parent_exit(ppid); 575 - } 576 - 577 538 /* 578 539 * resctrl_val: execute benchmark and measure memory bandwidth on 579 540 * the benchmark ··· 541 594 const struct user_params *uparams, 542 595 struct resctrl_val_param *param) 543 596 { 544 - struct benchmark_info benchmark_info; 545 - struct sigaction sigact; 546 - int ret = 0, pipefd[2]; 547 - char pipe_message = 0; 548 - union sigval value; 597 + unsigned char *buf = NULL; 598 + cpu_set_t old_affinity; 549 599 int domain_id; 600 + int ret = 0; 601 + pid_t ppid; 550 602 551 603 if (strcmp(param->filename, "") == 0) 552 604 sprintf(param->filename, "stdio"); ··· 556 610 return ret; 557 611 } 558 612 559 - benchmark_info.uparams = uparams; 560 - benchmark_info.param = param; 561 - 562 - /* 563 - * If benchmark wasn't successfully started by child, then child should 564 - * kill parent, so save parent's pid 565 - */ 566 613 ppid = getpid(); 567 614 568 - if (pipe(pipefd)) { 569 - ksft_perror("Unable to create pipe"); 570 - 571 - return -1; 572 - } 573 - 574 - /* 575 - * Fork to start benchmark, save child's pid so that it can be killed 576 - * when needed 577 - */ 578 - fflush(stdout); 579 - bm_pid = fork(); 580 - if (bm_pid == -1) { 581 - ksft_perror("Unable to fork"); 582 - 583 - return -1; 584 - } 585 - 586 - if (bm_pid == 0) { 587 - /* 588 - * Mask all signals except SIGUSR1, parent uses SIGUSR1 to 589 - * start benchmark 590 - */ 591 - sigfillset(&sigact.sa_mask); 592 - sigdelset(&sigact.sa_mask, SIGUSR1); 593 - 594 - sigact.sa_sigaction = run_benchmark; 595 - sigact.sa_flags = SA_SIGINFO; 596 - 597 - /* Register for "SIGUSR1" signal from parent */ 598 - if (sigaction(SIGUSR1, &sigact, NULL)) { 599 - ksft_perror("Can't register child for signal"); 600 - parent_exit(ppid); 601 - } 602 - 603 - /* Tell parent that child is ready */ 604 - close(pipefd[0]); 605 - pipe_message = 1; 606 - if (write(pipefd[1], &pipe_message, sizeof(pipe_message)) < 607 - sizeof(pipe_message)) { 608 - ksft_perror("Failed signaling parent process"); 609 - close(pipefd[1]); 610 - return -1; 611 - } 612 - close(pipefd[1]); 613 - 614 - /* Suspend child until delivery of "SIGUSR1" from parent */ 615 - sigsuspend(&sigact.sa_mask); 616 - 617 - ksft_perror("Child is done"); 618 - parent_exit(ppid); 619 - } 620 - 621 - ksft_print_msg("Benchmark PID: %d\n", (int)bm_pid); 622 - 623 - value.sival_ptr = (void *)&benchmark_info; 624 - 625 - /* Taskset benchmark to specified cpu */ 626 - ret = taskset_benchmark(bm_pid, uparams->cpu, NULL); 615 + /* Taskset test to specified CPU. */ 616 + ret = taskset_benchmark(ppid, uparams->cpu, &old_affinity); 627 617 if (ret) 628 - goto out; 618 + return ret; 629 619 630 - /* Write benchmark to specified control&monitoring grp in resctrl FS */ 631 - ret = write_bm_pid_to_resctrl(bm_pid, param->ctrlgrp, param->mongrp); 620 + /* Write test to specified control & monitoring group in resctrl FS. */ 621 + ret = write_bm_pid_to_resctrl(ppid, param->ctrlgrp, param->mongrp); 632 622 if (ret) 633 - goto out; 623 + goto reset_affinity; 634 624 635 625 if (param->init) { 636 626 ret = param->init(param, domain_id); 637 627 if (ret) 638 - goto out; 628 + goto reset_affinity; 639 629 } 640 630 641 - /* Parent waits for child to be ready. */ 642 - close(pipefd[1]); 643 - while (pipe_message != 1) { 644 - if (read(pipefd[0], &pipe_message, sizeof(pipe_message)) < 645 - sizeof(pipe_message)) { 646 - ksft_perror("Failed reading message from child process"); 647 - close(pipefd[0]); 648 - goto out; 631 + /* 632 + * If not running user provided benchmark, run the default 633 + * "fill_buf". First phase of "fill_buf" is to prepare the 634 + * buffer that the benchmark will operate on. No measurements 635 + * are needed during this phase and prepared memory will be 636 + * passed to next part of benchmark via copy-on-write thus 637 + * no impact on the benchmark that relies on reading from 638 + * memory only. 639 + */ 640 + if (param->fill_buf) { 641 + buf = alloc_buffer(param->fill_buf->buf_size, 642 + param->fill_buf->memflush); 643 + if (!buf) { 644 + ret = -ENOMEM; 645 + goto reset_affinity; 649 646 } 650 647 } 651 - close(pipefd[0]); 652 648 653 - /* Signal child to start benchmark */ 654 - if (sigqueue(bm_pid, SIGUSR1, value) == -1) { 655 - ksft_perror("sigqueue SIGUSR1 to child"); 656 - ret = -1; 657 - goto out; 649 + fflush(stdout); 650 + bm_pid = fork(); 651 + if (bm_pid == -1) { 652 + ret = -errno; 653 + ksft_perror("Unable to fork"); 654 + goto free_buf; 658 655 } 659 656 660 - /* Give benchmark enough time to fully run */ 657 + /* 658 + * What needs to be measured runs in separate process until 659 + * terminated. 660 + */ 661 + if (bm_pid == 0) { 662 + if (param->fill_buf) 663 + fill_cache_read(buf, param->fill_buf->buf_size, false); 664 + else if (uparams->benchmark_cmd[0]) 665 + execvp(uparams->benchmark_cmd[0], (char **)uparams->benchmark_cmd); 666 + exit(EXIT_SUCCESS); 667 + } 668 + 669 + ksft_print_msg("Benchmark PID: %d\n", (int)bm_pid); 670 + 671 + /* Give benchmark enough time to fully run. */ 661 672 sleep(1); 662 673 663 674 /* Test runs until the callback setup() tells the test to stop. */ ··· 632 729 break; 633 730 } 634 731 635 - out: 636 732 kill(bm_pid, SIGKILL); 637 - 733 + free_buf: 734 + free(buf); 735 + reset_affinity: 736 + taskset_restore(ppid, &old_affinity); 638 737 return ret; 639 738 }