Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

iommu/vt-d: Clear Present bit before tearing down PASID entry

The Intel VT-d Scalable Mode PASID table entry consists of 512 bits (64
bytes). When tearing down an entry, the current implementation zeros the
entire 64-byte structure immediately using multiple 64-bit writes.

Since the IOMMU hardware may fetch these 64 bytes using multiple
internal transactions (e.g., four 128-bit bursts), updating or zeroing
the entire entry while it is active (P=1) risks a "torn" read. If a
hardware fetch occurs simultaneously with the CPU zeroing the entry, the
hardware could observe an inconsistent state, leading to unpredictable
behavior or spurious faults.

Follow the "Guidance to Software for Invalidations" in the VT-d spec
(Section 6.5.3.3) by implementing the recommended ownership handshake:

1. Clear only the 'Present' (P) bit of the PASID entry.
2. Use a dma_wmb() to ensure the cleared bit is visible to hardware
before proceeding.
3. Execute the required invalidation sequence (PASID cache, IOTLB, and
Device-TLB flush) to ensure the hardware has released all cached
references.
4. Only after the flushes are complete, zero out the remaining fields
of the PASID entry.

Also, add a dma_wmb() in pasid_set_present() to ensure that all other
fields of the PASID entry are visible to the hardware before the Present
bit is set.

Fixes: 0bbeb01a4faf ("iommu/vt-d: Manage scalalble mode PASID tables")
Signed-off-by: Lu Baolu <baolu.lu@linux.intel.com>
Reviewed-by: Dmytro Maluka <dmaluka@chromium.org>
Reviewed-by: Samiullah Khawaja <skhawaja@google.com>
Reviewed-by: Kevin Tian <kevin.tian@intel.com>
Link: https://lore.kernel.org/r/20260120061816.2132558-2-baolu.lu@linux.intel.com
Signed-off-by: Joerg Roedel <joerg.roedel@amd.com>

authored by

Lu Baolu and committed by
Joerg Roedel
75ed0005 04b1b069

+19 -1
+5 -1
drivers/iommu/intel/pasid.c
··· 273 273 274 274 did = pasid_get_domain_id(pte); 275 275 pgtt = pasid_pte_get_pgtt(pte); 276 - intel_pasid_clear_entry(dev, pasid, fault_ignore); 276 + pasid_clear_present(pte); 277 277 spin_unlock(&iommu->lock); 278 278 279 279 if (!ecap_coherent(iommu->ecap)) ··· 287 287 iommu->flush.flush_iotlb(iommu, did, 0, 0, DMA_TLB_DSI_FLUSH); 288 288 289 289 devtlb_invalidation_with_pasid(iommu, dev, pasid); 290 + intel_pasid_clear_entry(dev, pasid, fault_ignore); 291 + if (!ecap_coherent(iommu->ecap)) 292 + clflush_cache_range(pte, sizeof(*pte)); 293 + 290 294 if (!fault_ignore) 291 295 intel_iommu_drain_pasid_prq(dev, pasid); 292 296 }
+14
drivers/iommu/intel/pasid.h
··· 234 234 */ 235 235 static inline void pasid_set_present(struct pasid_entry *pe) 236 236 { 237 + dma_wmb(); 237 238 pasid_set_bits(&pe->val[0], 1 << 0, 1); 239 + } 240 + 241 + /* 242 + * Clear the Present (P) bit (bit 0) of a scalable-mode PASID table entry. 243 + * This initiates the transition of the entry's ownership from hardware 244 + * to software. The caller is responsible for fulfilling the invalidation 245 + * handshake recommended by the VT-d spec, Section 6.5.3.3 (Guidance to 246 + * Software for Invalidations). 247 + */ 248 + static inline void pasid_clear_present(struct pasid_entry *pe) 249 + { 250 + pasid_set_bits(&pe->val[0], 1 << 0, 0); 251 + dma_wmb(); 238 252 } 239 253 240 254 /*