tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

crypto: iaa - Optimize rebalance_wq_table()

The function opencodes for_each_cpu() by using a plain for-loop. The
loop calls cpumask_weight() inside the conditional section. Because
cpumask_weight() is O(1), the overall complexity of the function is
O(node * node_cpus^2). Also, cpumask_nth() internally calls hweight(),
which, if not hardware accelerated, is slower than cpumask_next() in
for_each_cpu().

If switched to the dedicated for_each_cpu(), the rebalance_wq_table()
can drop calling cpumask_weight(), together with some housekeeping code.
This makes the overall complexity O(node * node_cpus), or simply speaking
O(nr_cpu_ids).

While there, fix opencoded for_each_possible_cpu() too.

Signed-off-by: Yury Norov <yury.norov@gmail.com>
Signed-off-by: Herbert Xu <herbert@gondor.apana.org.au>

authored by

Yury Norov and committed by
Herbert Xu 714ca27e 33cd9343

+14 -21
+14 -21
drivers/crypto/intel/iaa/iaa_crypto_main.c
··· 894 894 static void rebalance_wq_table(void) 895 895 { 896 896 const struct cpumask *node_cpus; 897 - int node, cpu, iaa = -1; 897 + int node_cpu, node, cpu, iaa = 0; 898 898 899 899 if (nr_iaa == 0) 900 900 return; ··· 905 905 clear_wq_table(); 906 906 907 907 if (nr_iaa == 1) { 908 - for (cpu = 0; cpu < nr_cpus; cpu++) { 909 - if (WARN_ON(wq_table_add_wqs(0, cpu))) { 910 - pr_debug("could not add any wqs for iaa 0 to cpu %d!\n", cpu); 911 - return; 912 - } 908 + for_each_possible_cpu(cpu) { 909 + if (WARN_ON(wq_table_add_wqs(0, cpu))) 910 + goto err; 913 911 } 914 912 915 913 return; 916 914 } 917 915 918 916 for_each_node_with_cpus(node) { 917 + cpu = 0; 919 918 node_cpus = cpumask_of_node(node); 920 919 921 - for (cpu = 0; cpu < cpumask_weight(node_cpus); cpu++) { 922 - int node_cpu = cpumask_nth(cpu, node_cpus); 923 - 924 - if (WARN_ON(node_cpu >= nr_cpu_ids)) { 925 - pr_debug("node_cpu %d doesn't exist!\n", node_cpu); 926 - return; 927 - } 928 - 929 - if ((cpu % cpus_per_iaa) == 0) 930 - iaa++; 931 - 932 - if (WARN_ON(wq_table_add_wqs(iaa, node_cpu))) { 933 - pr_debug("could not add any wqs for iaa %d to cpu %d!\n", iaa, cpu); 934 - return; 935 - } 920 + for_each_cpu(node_cpu, node_cpus) { 921 + iaa = cpu / cpus_per_iaa; 922 + if (WARN_ON(wq_table_add_wqs(iaa, node_cpu))) 923 + goto err; 924 + cpu++; 936 925 } 937 926 } 927 + 928 + return; 929 + err: 930 + pr_debug("could not add any wqs for iaa %d to cpu %d!\n", iaa, cpu); 938 931 } 939 932 940 933 static inline int check_completion(struct device *dev,