tjh.dev / kernel
Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

iommupt: Optimize the gather processing for DMA-FQ mode

In PT_FEAT_FLUSH_RANGE mode the gather was accumulated but never flushed
and then the accumulated range was discarded by the dma-iommu code in
DMA-FQ mode. This is basically optimal.

However for PT_FEAT_FLUSH_RANGE_NO_GAPS the page table would push flushes
that are redundant with the flush all generated by the DMA-FQ mode.

Disable all range accumulation in the gather, and iommu_pt triggered
flushing when in iommu_iotlb_gather_queued() indicates it is in DMA-FQ
mode.

Reported-by: Robin Murphy <robin.murphy@arm.com>
Closes: https://lore.kernel.org/r/794b6121-b66b-4819-b291-9761ed21cd83@arm.com
Signed-off-by: Jason Gunthorpe <jgg@nvidia.com>
Reviewed-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Samiullah Khawaja <skhawaja@google.com>
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>

authored by

Jason Gunthorpe and committed by
Joerg Roedel 199036ae 7cd0c655

+19 -8
+19 -8
drivers/iommu/generic_pt/iommu_pt.h
··· 51 51 iommu_pages_stop_incoherent_list(free_list, 52 52 iommu_table->iommu_device); 53 53 54 - if (pt_feature(common, PT_FEAT_FLUSH_RANGE_NO_GAPS) && 55 - iommu_iotlb_gather_is_disjoint(iotlb_gather, iova, len)) { 56 - iommu_iotlb_sync(&iommu_table->domain, iotlb_gather); 57 - /* 58 - * Note that the sync frees the gather's free list, so we must 59 - * not have any pages on that list that are covered by iova/len 60 - */ 54 + /* 55 + * If running in DMA-FQ mode then the unmap will be followed by an IOTLB 56 + * flush all so we need to optimize by never flushing the IOTLB here. 57 + * 58 + * For NO_GAPS the user gets to pick if flushing all or doing micro 59 + * flushes is better for their work load by choosing DMA vs DMA-FQ 60 + * operation. Drivers should also see shadow_on_flush. 61 + */ 62 + if (!iommu_iotlb_gather_queued(iotlb_gather)) { 63 + if (pt_feature(common, PT_FEAT_FLUSH_RANGE_NO_GAPS) && 64 + iommu_iotlb_gather_is_disjoint(iotlb_gather, iova, len)) { 65 + iommu_iotlb_sync(&iommu_table->domain, iotlb_gather); 66 + /* 67 + * Note that the sync frees the gather's free list, so 68 + * we must not have any pages on that list that are 69 + * covered by iova/len 70 + */ 71 + } 72 + iommu_iotlb_gather_add_range(iotlb_gather, iova, len); 61 73 } 62 74 63 - iommu_iotlb_gather_add_range(iotlb_gather, iova, len); 64 75 iommu_pages_list_splice(free_list, &iotlb_gather->freelist); 65 76 } 66 77