drm/xe/vf: Redo LRC creation while in VF fixups

If the xe module within a VM was creating a new LRC during save/
restore, this LRC will be invalid. The fixups procedure may not
be able to reach it, as there will be a race to add the new LRC
reference to an exec queue.

Even if the new LRC which was being created during VM migration is
added to EQ in time for fixups, said LRC may still remain damaged.
In a small percentage of specially crafted test cases, the resulting
LRC was still damaged and caused GPU hang.

Any LRC which could be created in such a situation, have to be
re-created.

Due to VM having arbitrarily set amount of CPU cores, it is possible
to limit the amount to 1. In such case, there is a possibility that
kernel will switch CPU contexts in a way which allows to miss
VF migration recovery running in parallel (by simply not switching
to the LRC creation thread during recovery). Therefore checking
if the migration is in progress just after LRC creation, is not
enough to ensure detection.

Free the incorrectly created LRC, and trigger a re-run of the
creation, but only after waiting for default LRC to get fixups.
Use additional atomic value increased after fixups, to ensure any VF
migration that avoided detection by just checking for recovery in
progress, will be caught.

v2: Merge marker and wait for default LRC, reducing amount of calls
within xe_init_eq(). Alter the LRC creation loop to remove a race
with post-migration fixups worker.
v3: Kerneldoc fixes. Rename fixups_complete_count.

Signed-off-by: Tomasz Lis <tomasz.lis@intel.com>
Reviewed-by: Matthew Brost <matthew.brost@intel.com>
Signed-off-by: Michal Wajdeczko <michal.wajdeczko@intel.com>
Link: https://patch.msgid.link/20260226212701.2937065-5-tomasz.lis@intel.com

authored by

Tomasz Lis and committed by

Michal Wajdeczko 3 months ago c692ae39 f3fb5f1e

+60 -14

4 changed files

expand all

drivers

gpu

drm

xe_exec_queue.c

xe_gt_sriov_vf.c

xe_gt_sriov_vf.h

xe_gt_sriov_vf_types.h

+20 -9

drivers/gpu/drm/xe/xe_exec_queue.c

··· 368 368 * from the moment vCPU resumes execution. 369 369 */ 370 370 for (i = 0; i < q->width; ++i) { 371 - struct xe_lrc *lrc; 371 + struct xe_lrc *__lrc = NULL; 372 + int marker; 372 373 373 - xe_gt_sriov_vf_wait_valid_ggtt(q->gt); 374 - lrc = xe_lrc_create(q->hwe, q->vm, q->replay_state, 375 - xe_lrc_ring_size(), q->msix_vec, flags); 376 - if (IS_ERR(lrc)) { 377 - err = PTR_ERR(lrc); 378 - goto err_lrc; 379 - } 374 + do { 375 + struct xe_lrc *lrc; 380 376 381 - xe_exec_queue_set_lrc(q, lrc, i); 377 + marker = xe_gt_sriov_vf_wait_valid_ggtt(q->gt); 378 + 379 + lrc = xe_lrc_create(q->hwe, q->vm, q->replay_state, 380 + xe_lrc_ring_size(), q->msix_vec, flags); 381 + if (IS_ERR(lrc)) { 382 + err = PTR_ERR(lrc); 383 + goto err_lrc; 384 + } 385 + 386 + xe_exec_queue_set_lrc(q, lrc, i); 387 + 388 + if (__lrc) 389 + xe_lrc_put(__lrc); 390 + __lrc = lrc; 391 + 392 + } while (marker != xe_vf_migration_fixups_complete_count(q->gt)); 382 393 } 383 394 384 395 return 0;

+36 -4

drivers/gpu/drm/xe/xe_gt_sriov_vf.c

··· 1277 1277 if (err) 1278 1278 return err; 1279 1279 1280 + atomic_inc(&gt->sriov.vf.migration.fixups_complete_count); 1281 + 1280 1282 return 0; 1281 1283 } 1282 1284 ··· 1518 1516 } 1519 1517 1520 1518 /** 1521 - * xe_gt_sriov_vf_wait_valid_ggtt() - wait for valid GGTT nodes and address refs 1522 - * @gt: the &xe_gt 1519 + * xe_vf_migration_fixups_complete_count() - Get count of VF fixups completions. 1520 + * @gt: the &xe_gt instance which contains affected Global GTT 1521 + * 1522 + * Return: number of times VF fixups were completed since driver 1523 + * probe, or 0 if migration is not available, or -1 if fixups are 1524 + * pending or being applied right now. 1523 1525 */ 1524 - void xe_gt_sriov_vf_wait_valid_ggtt(struct xe_gt *gt) 1526 + int xe_vf_migration_fixups_complete_count(struct xe_gt *gt) 1527 + { 1528 + if (!IS_SRIOV_VF(gt_to_xe(gt)) || 1529 + !xe_sriov_vf_migration_supported(gt_to_xe(gt))) 1530 + return 0; 1531 + 1532 + /* should never match fixups_complete_count value */ 1533 + if (!vf_valid_ggtt(gt)) 1534 + return -1; 1535 + 1536 + return atomic_read(&gt->sriov.vf.migration.fixups_complete_count); 1537 + } 1538 + 1539 + /** 1540 + * xe_gt_sriov_vf_wait_valid_ggtt() - wait for valid GGTT nodes and address refs 1541 + * @gt: the &xe_gt instance which contains affected Global GTT 1542 + * 1543 + * Return: number of times VF fixups were completed since driver 1544 + * probe, or 0 if migration is not available. 1545 + */ 1546 + int xe_gt_sriov_vf_wait_valid_ggtt(struct xe_gt *gt) 1525 1547 { 1526 1548 int ret; 1527 1549 1550 + /* 1551 + * this condition needs to be identical to one in 1552 + * xe_vf_migration_fixups_complete_count() 1553 + */ 1528 1554 if (!IS_SRIOV_VF(gt_to_xe(gt)) || 1529 1555 !xe_sriov_vf_migration_supported(gt_to_xe(gt))) 1530 - return; 1556 + return 0; 1531 1557 1532 1558 ret = wait_event_interruptible_timeout(gt->sriov.vf.migration.wq, 1533 1559 vf_valid_ggtt(gt), 1534 1560 HZ * 5); 1535 1561 xe_gt_WARN_ON(gt, !ret); 1562 + 1563 + return atomic_read(&gt->sriov.vf.migration.fixups_complete_count); 1536 1564 }

+2 -1

drivers/gpu/drm/xe/xe_gt_sriov_vf.h

··· 39 39 void xe_gt_sriov_vf_print_runtime(struct xe_gt *gt, struct drm_printer *p); 40 40 void xe_gt_sriov_vf_print_version(struct xe_gt *gt, struct drm_printer *p); 41 41 42 - void xe_gt_sriov_vf_wait_valid_ggtt(struct xe_gt *gt); 42 + int xe_gt_sriov_vf_wait_valid_ggtt(struct xe_gt *gt); 43 + int xe_vf_migration_fixups_complete_count(struct xe_gt *gt); 43 44 44 45 #endif

drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h

··· 54 54 wait_queue_head_t wq; 55 55 /** @scratch: Scratch memory for VF recovery */ 56 56 void *scratch; 57 + /** @fixups_complete_count: Counts completed fixups stages */ 58 + atomic_t fixups_complete_count; 57 59 /** @debug: Debug hooks for delaying migration */ 58 60 struct { 59 61 /**

Configure Feed

Configure Feed