+110

Documentation/ABI/testing/sysfs-driver-intel-xe-hwmon

reviewed

··· 109 109 110 110 Only supported for particular Intel Xe graphics platforms. 111 111 112 112 + What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/temp2_crit 113 113 + Date: January 2026 114 114 + KernelVersion: 7.0 115 115 + Contact: intel-xe@lists.freedesktop.org 116 116 + Description: RO. Package critical temperature in millidegree Celsius. 117 117 + 118 118 + Only supported for particular Intel Xe graphics platforms. 119 119 + 120 120 + What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/temp2_emergency 121 121 + Date: January 2026 122 122 + KernelVersion: 7.0 123 123 + Contact: intel-xe@lists.freedesktop.org 124 124 + Description: RO. Package shutdown temperature in millidegree Celsius. 125 125 + 126 126 + Only supported for particular Intel Xe graphics platforms. 127 127 + 112 128 What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/temp2_input 113 129 Date: March 2025 114 130 KernelVersion: 6.15 ··· 133 117 134 118 Only supported for particular Intel Xe graphics platforms. 135 119 120 120 + What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/temp2_max 121 121 + Date: January 2026 122 122 + KernelVersion: 7.0 123 123 + Contact: intel-xe@lists.freedesktop.org 124 124 + Description: RO. Package maximum temperature limit in millidegree Celsius. 125 125 + 126 126 + Only supported for particular Intel Xe graphics platforms. 127 127 + 128 128 + What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/temp3_crit 129 129 + Date: January 2026 130 130 + KernelVersion: 7.0 131 131 + Contact: intel-xe@lists.freedesktop.org 132 132 + Description: RO. VRAM critical temperature in millidegree Celsius. 133 133 + 134 134 + Only supported for particular Intel Xe graphics platforms. 135 135 + 136 136 + What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/temp3_emergency 137 137 + Date: January 2026 138 138 + KernelVersion: 7.0 139 139 + Contact: intel-xe@lists.freedesktop.org 140 140 + Description: RO. VRAM shutdown temperature in millidegree Celsius. 141 141 + 142 142 + Only supported for particular Intel Xe graphics platforms. 143 143 + 136 144 What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/temp3_input 137 145 Date: March 2025 138 146 KernelVersion: 6.15 139 147 Contact: intel-xe@lists.freedesktop.org 140 148 Description: RO. VRAM temperature in millidegree Celsius. 149 149 + 150 150 + Only supported for particular Intel Xe graphics platforms. 151 151 + 152 152 + What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/temp4_crit 153 153 + Date: January 2026 154 154 + KernelVersion: 7.0 155 155 + Contact: intel-xe@lists.freedesktop.org 156 156 + Description: RO. Memory controller critical temperature in millidegree Celsius. 157 157 + 158 158 + Only supported for particular Intel Xe graphics platforms. 159 159 + 160 160 + What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/temp4_emergency 161 161 + Date: January 2026 162 162 + KernelVersion: 7.0 163 163 + Contact: intel-xe@lists.freedesktop.org 164 164 + Description: RO. Memory controller shutdown temperature in millidegree Celsius. 165 165 + 166 166 + Only supported for particular Intel Xe graphics platforms. 167 167 + 168 168 + What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/temp4_input 169 169 + Date: January 2026 170 170 + KernelVersion: 7.0 171 171 + Contact: intel-xe@lists.freedesktop.org 172 172 + Description: RO. Memory controller average temperature in millidegree Celsius. 173 173 + 174 174 + Only supported for particular Intel Xe graphics platforms. 175 175 + 176 176 + What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/temp5_crit 177 177 + Date: January 2026 178 178 + KernelVersion: 7.0 179 179 + Contact: intel-xe@lists.freedesktop.org 180 180 + Description: RO. GPU PCIe critical temperature in millidegree Celsius. 181 181 + 182 182 + Only supported for particular Intel Xe graphics platforms. 183 183 + 184 184 + What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/temp5_emergency 185 185 + Date: January 2026 186 186 + KernelVersion: 7.0 187 187 + Contact: intel-xe@lists.freedesktop.org 188 188 + Description: RO. GPU PCIe shutdown temperature in millidegree Celsius. 189 189 + 190 190 + Only supported for particular Intel Xe graphics platforms. 191 191 + 192 192 + What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/temp5_input 193 193 + Date: January 2026 194 194 + KernelVersion: 7.0 195 195 + Contact: intel-xe@lists.freedesktop.org 196 196 + Description: RO. GPU PCIe temperature in millidegree Celsius. 197 197 + 198 198 + What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/temp[6-21]_crit 199 199 + Date: January 2026 200 200 + KernelVersion: 7.0 201 201 + Contact: intel-xe@lists.freedesktop.org 202 202 + Description: RO. VRAM channel critical temperature in millidegree Celsius. 203 203 + 204 204 + Only supported for particular Intel Xe graphics platforms. 205 205 + 206 206 + What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/temp[6-21]_emergency 207 207 + Date: January 2026 208 208 + KernelVersion: 7.0 209 209 + Contact: intel-xe@lists.freedesktop.org 210 210 + Description: RO. VRAM channel shutdown temperature in millidegree Celsius. 211 211 + 212 212 + Only supported for particular Intel Xe graphics platforms. 213 213 + 214 214 + What: /sys/bus/pci/drivers/xe/.../hwmon/hwmon<i>/temp[6-21]_input 215 215 + Date: January 2026 216 216 + KernelVersion: 7.0 217 217 + Contact: intel-xe@lists.freedesktop.org 218 218 + Description: RO. VRAM channel temperature in millidegree Celsius. 141 219 142 220 Only supported for particular Intel Xe graphics platforms. 143 221

+17 -5

drivers/gpu/drm/drm_pagemap.c

reviewed

··· 480 480 .start = start, 481 481 .end = end, 482 482 .pgmap_owner = pagemap->owner, 483 483 - .flags = MIGRATE_VMA_SELECT_SYSTEM | MIGRATE_VMA_SELECT_DEVICE_COHERENT | 484 484 - MIGRATE_VMA_SELECT_DEVICE_PRIVATE, 483 483 + /* 484 484 + * FIXME: MIGRATE_VMA_SELECT_DEVICE_PRIVATE intermittently 485 485 + * causes 'xe_exec_system_allocator --r *race*no*' to trigger aa 486 486 + * engine reset and a hard hang due to getting stuck on a folio 487 487 + * lock. This should work and needs to be root-caused. The only 488 488 + * downside of not selecting MIGRATE_VMA_SELECT_DEVICE_PRIVATE 489 489 + * is that device-to-device migrations won’t work; instead, 490 490 + * memory will bounce through system memory. This path should be 491 491 + * rare and only occur when the madvise attributes of memory are 492 492 + * changed or atomics are being used. 493 493 + */ 494 494 + .flags = MIGRATE_VMA_SELECT_SYSTEM | MIGRATE_VMA_SELECT_DEVICE_COHERENT, 485 495 }; 486 496 unsigned long i, npages = npages_in_range(start, end); 487 497 unsigned long own_pages = 0, migrated_pages = 0; ··· 592 582 593 583 err = ops->populate_devmem_pfn(devmem_allocation, npages, migrate.dst); 594 584 if (err) 595 595 - goto err_finalize; 585 585 + goto err_aborted_migration; 596 586 597 587 own_pages = 0; 598 588 ··· 631 621 err = drm_pagemap_migrate_range(devmem_allocation, migrate.src, migrate.dst, 632 622 pages, pagemap_addr, &last, &cur, 633 623 mdetails); 634 634 - if (err) 624 624 + if (err) { 625 625 + npages = i + 1; 635 626 goto err_finalize; 627 627 + } 636 628 } 637 629 cur.start = npages; 638 630 cur.ops = NULL; /* Force migration */ ··· 658 646 err_aborted_migration: 659 647 migrate_vma_pages(&migrate); 660 648 661 661 - for (i = 0; i < npages;) { 649 649 + for (i = 0; !err && i < npages;) { 662 650 struct page *page = migrate_pfn_to_page(migrate.src[i]); 663 651 unsigned long nr_pages = page ? NR_PAGES(folio_order(page_folio(page))) : 1; 664 652

+2 -1

drivers/gpu/drm/xe/Makefile

reviewed

··· 370 370 $(patsubst %.h,%.hdrtest, $(shell cd $(src) && find * -name '*.h' $(hdrtest_find_args))) 371 371 372 372 quiet_cmd_hdrtest = HDRTEST $(patsubst %.hdrtest,%.h,$@) 373 373 - cmd_hdrtest = $(CC) -DHDRTEST $(filter-out $(CFLAGS_GCOV), $(c_flags)) -S -o /dev/null -x c /dev/null -include $<; touch $@ 373 373 + cmd_hdrtest = $(CC) $(filter-out $(CFLAGS_GCOV), $(c_flags)) -S -o /dev/null -x c /dev/null -include $< -include $<; \ 374 374 + $(srctree)/scripts/kernel-doc -none -Werror $<; touch $@ 374 375 375 376 $(obj)/%.hdrtest: $(src)/%.h FORCE 376 377 $(call if_changed_dep,hdrtest)

+2 -3

drivers/gpu/drm/xe/abi/guc_lfd_abi.h

reviewed

··· 148 148 } __packed; 149 149 150 150 /** 151 151 - * struct guc_logfile_header - Header of GuC Log Streaming-LFD-File Format. 151 151 + * struct guc_lfd_file_header - Header of GuC Log Streaming-LFD-File Format. 152 152 * This structure encapsulates the layout of the guc-log-file format 153 153 */ 154 154 struct guc_lfd_file_header { ··· 163 163 #define GUC_LFD_FILE_HEADER_VERSION_MASK_MAJOR GENMASK(31, 16) 164 164 #define GUC_LFD_FILE_HEADER_VERSION_MASK_MINOR GENMASK(15, 0) 165 165 166 166 - /** @stream: A stream of one or more guc_lfd_data LFD blocks 167 167 - */ 166 166 + /** @stream: A stream of one or more guc_lfd_data LFD blocks */ 168 167 u32 stream[]; 169 168 } __packed; 170 169

+2 -2

drivers/gpu/drm/xe/compat-i915-headers/i915_vma.h

reviewed

··· 8 8 9 9 #include <uapi/drm/i915_drm.h> 10 10 11 11 - #include "xe_ggtt_types.h" 11 11 + #include "xe_ggtt.h" 12 12 13 13 #include <linux/refcount.h> 14 14 ··· 30 30 31 31 static inline u32 i915_ggtt_offset(const struct i915_vma *vma) 32 32 { 33 33 - return vma->node->base.start; 33 33 + return xe_ggtt_node_addr(vma->node); 34 34 } 35 35 36 36 #endif

+52 -52

drivers/gpu/drm/xe/display/xe_fb_pin.c

reviewed

··· 171 171 } 172 172 173 173 static void 174 174 - write_ggtt_rotated(struct xe_bo *bo, struct xe_ggtt *ggtt, u32 *ggtt_ofs, u32 bo_ofs, 174 174 + write_ggtt_rotated(struct xe_ggtt *ggtt, u32 *ggtt_ofs, 175 175 + u64 pte_flags, 176 176 + xe_ggtt_set_pte_fn write_pte, 177 177 + struct xe_bo *bo, u32 bo_ofs, 175 178 u32 width, u32 height, u32 src_stride, u32 dst_stride) 176 179 { 177 177 - struct xe_device *xe = xe_bo_device(bo); 178 180 u32 column, row; 179 179 - u64 pte = ggtt->pt_ops->pte_encode_flags(bo, xe->pat.idx[XE_CACHE_NONE]); 180 181 181 182 for (column = 0; column < width; column++) { 182 183 u32 src_idx = src_stride * (height - 1) + column + bo_ofs; ··· 185 184 for (row = 0; row < height; row++) { 186 185 u64 addr = xe_bo_addr(bo, src_idx * XE_PAGE_SIZE, XE_PAGE_SIZE); 187 186 188 188 - ggtt->pt_ops->ggtt_set_pte(ggtt, *ggtt_ofs, pte | addr); 187 187 + write_pte(ggtt, *ggtt_ofs, pte_flags | addr); 189 188 *ggtt_ofs += XE_PAGE_SIZE; 190 189 src_idx -= src_stride; 191 190 } ··· 193 192 /* The DE ignores the PTEs for the padding tiles */ 194 193 *ggtt_ofs += (dst_stride - height) * XE_PAGE_SIZE; 195 194 } 195 195 + } 196 196 + 197 197 + struct fb_rotate_args { 198 198 + const struct i915_gtt_view *view; 199 199 + struct xe_bo *bo; 200 200 + }; 201 201 + 202 202 + static void write_ggtt_rotated_node(struct xe_ggtt *ggtt, struct xe_ggtt_node *node, 203 203 + u64 pte_flags, xe_ggtt_set_pte_fn write_pte, void *data) 204 204 + { 205 205 + struct fb_rotate_args *args = data; 206 206 + struct xe_bo *bo = args->bo; 207 207 + const struct intel_rotation_info *rot_info = &args->view->rotated; 208 208 + u32 ggtt_ofs = xe_ggtt_node_addr(node); 209 209 + 210 210 + for (u32 i = 0; i < ARRAY_SIZE(rot_info->plane); i++) 211 211 + write_ggtt_rotated(ggtt, &ggtt_ofs, pte_flags, write_pte, 212 212 + bo, rot_info->plane[i].offset, 213 213 + rot_info->plane[i].width, 214 214 + rot_info->plane[i].height, 215 215 + rot_info->plane[i].src_stride, 216 216 + rot_info->plane[i].dst_stride); 196 217 } 197 218 198 219 static int __xe_pin_fb_vma_ggtt(const struct intel_framebuffer *fb, ··· 227 204 struct xe_device *xe = to_xe_device(fb->base.dev); 228 205 struct xe_tile *tile0 = xe_device_get_root_tile(xe); 229 206 struct xe_ggtt *ggtt = tile0->mem.ggtt; 207 207 + u64 pte, size; 230 208 u32 align; 231 231 - int ret; 209 209 + int ret = 0; 232 210 233 211 /* TODO: Consider sharing framebuffer mapping? 234 212 * embed i915_vma inside intel_framebuffer 235 213 */ 236 214 guard(xe_pm_runtime_noresume)(xe); 237 237 - ACQUIRE(mutex_intr, lock)(&ggtt->lock); 238 238 - ret = ACQUIRE_ERR(mutex_intr, &lock); 239 239 - if (ret) 240 240 - return ret; 241 215 242 216 align = XE_PAGE_SIZE; 243 243 - if (xe_bo_is_vram(bo) && ggtt->flags & XE_GGTT_FLAGS_64K) 244 244 - align = max_t(u32, align, SZ_64K); 217 217 + if (xe_bo_is_vram(bo) && xe->info.vram_flags & XE_VRAM_FLAGS_NEED64K) 218 218 + align = max(align, SZ_64K); 245 219 220 220 + /* Fast case, preallocated GGTT view? */ 246 221 if (bo->ggtt_node[tile0->id] && view->type == I915_GTT_VIEW_NORMAL) { 247 222 vma->node = bo->ggtt_node[tile0->id]; 248 248 - } else if (view->type == I915_GTT_VIEW_NORMAL) { 249 249 - vma->node = xe_ggtt_node_init(ggtt); 250 250 - if (IS_ERR(vma->node)) 251 251 - return PTR_ERR(vma->node); 252 252 - 253 253 - ret = xe_ggtt_node_insert_locked(vma->node, xe_bo_size(bo), align, 0); 254 254 - if (ret) { 255 255 - xe_ggtt_node_fini(vma->node); 256 256 - return ret; 257 257 - } 258 258 - 259 259 - xe_ggtt_map_bo(ggtt, vma->node, bo, xe->pat.idx[XE_CACHE_NONE]); 260 260 - } else { 261 261 - u32 i, ggtt_ofs; 262 262 - const struct intel_rotation_info *rot_info = &view->rotated; 263 263 - 264 264 - /* display seems to use tiles instead of bytes here, so convert it back.. */ 265 265 - u32 size = intel_rotation_info_size(rot_info) * XE_PAGE_SIZE; 266 266 - 267 267 - vma->node = xe_ggtt_node_init(ggtt); 268 268 - if (IS_ERR(vma->node)) { 269 269 - ret = PTR_ERR(vma->node); 270 270 - return ret; 271 271 - } 272 272 - 273 273 - ret = xe_ggtt_node_insert_locked(vma->node, size, align, 0); 274 274 - if (ret) { 275 275 - xe_ggtt_node_fini(vma->node); 276 276 - return ret; 277 277 - } 278 278 - 279 279 - ggtt_ofs = vma->node->base.start; 280 280 - 281 281 - for (i = 0; i < ARRAY_SIZE(rot_info->plane); i++) 282 282 - write_ggtt_rotated(bo, ggtt, &ggtt_ofs, 283 283 - rot_info->plane[i].offset, 284 284 - rot_info->plane[i].width, 285 285 - rot_info->plane[i].height, 286 286 - rot_info->plane[i].src_stride, 287 287 - rot_info->plane[i].dst_stride); 223 223 + return 0; 288 224 } 225 225 + 226 226 + /* TODO: Consider sharing framebuffer mapping? 227 227 + * embed i915_vma inside intel_framebuffer 228 228 + */ 229 229 + if (view->type == I915_GTT_VIEW_NORMAL) 230 230 + size = xe_bo_size(bo); 231 231 + else 232 232 + /* display uses tiles instead of bytes here, so convert it back.. */ 233 233 + size = intel_rotation_info_size(&view->rotated) * XE_PAGE_SIZE; 234 234 + 235 235 + pte = xe_ggtt_encode_pte_flags(ggtt, bo, xe->pat.idx[XE_CACHE_NONE]); 236 236 + vma->node = xe_ggtt_node_insert_transform(ggtt, bo, pte, 237 237 + ALIGN(size, align), align, 238 238 + view->type == I915_GTT_VIEW_NORMAL ? 239 239 + NULL : write_ggtt_rotated_node, 240 240 + &(struct fb_rotate_args){view, bo}); 241 241 + if (IS_ERR(vma->node)) 242 242 + ret = PTR_ERR(vma->node); 289 243 290 244 return ret; 291 245 } ··· 353 353 if (vma->dpt) 354 354 xe_bo_unpin_map_no_vm(vma->dpt); 355 355 else if (!xe_ggtt_node_allocated(vma->bo->ggtt_node[tile_id]) || 356 356 - vma->bo->ggtt_node[tile_id]->base.start != vma->node->base.start) 356 356 + vma->bo->ggtt_node[tile_id] != vma->node) 357 357 xe_ggtt_node_remove(vma->node, false); 358 358 359 359 ttm_bo_reserve(&vma->bo->ttm, false, false, NULL);

+1 -1

drivers/gpu/drm/xe/display/xe_hdcp_gsc.c

reviewed

··· 39 39 struct xe_gt *gt = tile->media_gt; 40 40 struct xe_gsc *gsc = &gt->uc.gsc; 41 41 42 42 - if (!gsc || !xe_uc_fw_is_enabled(&gsc->fw)) { 42 42 + if (!gsc || !xe_uc_fw_is_available(&gsc->fw)) { 43 43 drm_dbg_kms(&xe->drm, 44 44 "GSC Components not ready for HDCP2.x\n"); 45 45 return false;

+1 -1

drivers/gpu/drm/xe/display/xe_stolen.c

reviewed

··· 78 78 79 79 static u64 xe_stolen_node_size(const struct intel_stolen_node *node) 80 80 { 81 81 - return node->bo->ttm.base.size; 81 81 + return xe_bo_size(node->bo); 82 82 } 83 83 84 84 static struct intel_stolen_node *xe_stolen_node_alloc(struct drm_device *drm)

+6

drivers/gpu/drm/xe/regs/xe_gt_regs.h

reviewed

··· 89 89 #define UNIFIED_COMPRESSION_FORMAT REG_GENMASK(3, 0) 90 90 91 91 #define XE2_GAMREQSTRM_CTRL XE_REG_MCR(0x4194) 92 92 + #define EN_CMP_1WCOH REG_BIT(15) 92 93 #define CG_DIS_CNTLBUS REG_BIT(6) 93 94 94 95 #define CCS_AUX_INV XE_REG(0x4208) ··· 101 100 #define AUX_INV REG_BIT(0) 102 101 103 102 #define XE2_LMEM_CFG XE_REG(0x48b0) 103 103 + 104 104 + #define XE2_GAMWALK_CTRL 0x47e4 105 105 + #define XE2_GAMWALK_CTRL_MEDIA XE_REG(XE2_GAMWALK_CTRL + MEDIA_GT_GSI_OFFSET) 106 106 + #define XE2_GAMWALK_CTRL_3D XE_REG_MCR(XE2_GAMWALK_CTRL) 107 107 + #define EN_CMP_1WCOH_GW REG_BIT(14) 104 108 105 109 #define XEHP_FLAT_CCS_BASE_ADDR XE_REG_MCR(0x4910) 106 110 #define XEHP_FLAT_CCS_PTR REG_GENMASK(31, 8)

+6 -4

drivers/gpu/drm/xe/regs/xe_mert_regs.h

reviewed

··· 11 11 #define MERT_LMEM_CFG XE_REG(0x1448b0) 12 12 13 13 #define MERT_TLB_CT_INTR_ERR_ID_PORT XE_REG(0x145190) 14 14 - #define MERT_TLB_CT_VFID_MASK REG_GENMASK(16, 9) 15 15 - #define MERT_TLB_CT_ERROR_MASK REG_GENMASK(5, 0) 16 16 - #define MERT_TLB_CT_LMTT_FAULT 0x05 14 14 + #define CATERR_VFID REG_GENMASK(16, 9) 15 15 + #define CATERR_CODES REG_GENMASK(5, 0) 16 16 + #define CATERR_NO_ERROR 0x00 17 17 + #define CATERR_UNMAPPED_GGTT 0x01 18 18 + #define CATERR_LMTT_FAULT 0x05 17 19 18 20 #define MERT_TLB_INV_DESC_A XE_REG(0x14cf7c) 19 21 #define MERT_TLB_INV_DESC_A_VALID REG_BIT(0) 20 22 21 21 - #endif /* _XE_MERT_REGS_H_ */ 23 23 + #endif

+3

drivers/gpu/drm/xe/regs/xe_pcode_regs.h

reviewed

··· 21 21 #define BMG_FAN_1_SPEED XE_REG(0x138140) 22 22 #define BMG_FAN_2_SPEED XE_REG(0x138170) 23 23 #define BMG_FAN_3_SPEED XE_REG(0x1381a0) 24 24 + #define BMG_VRAM_TEMPERATURE_N(n) XE_REG(0x138260 + ((n) * (sizeof(u32)))) 24 25 #define BMG_VRAM_TEMPERATURE XE_REG(0x1382c0) 26 26 + #define TEMP_MASK_VRAM_N REG_GENMASK(30, 8) 27 27 + #define TEMP_SIGN_MASK REG_BIT(31) 25 28 #define BMG_PACKAGE_TEMPERATURE XE_REG(0x138434) 26 29 27 30 #endif /* _XE_PCODE_REGS_H_ */

+1

drivers/gpu/drm/xe/tests/xe_bo.c

reviewed

··· 18 18 #include "tests/xe_test.h" 19 19 20 20 #include "xe_bo_evict.h" 21 21 + #include "xe_gt.h" 21 22 #include "xe_pci.h" 22 23 #include "xe_pm.h" 23 24

+1 -1

drivers/gpu/drm/xe/tests/xe_guc_buf_kunit.c

reviewed

··· 67 67 68 68 KUNIT_ASSERT_EQ(test, 0, 69 69 xe_ggtt_init_kunit(ggtt, DUT_GGTT_START, 70 70 - DUT_GGTT_START + DUT_GGTT_SIZE)); 70 70 + DUT_GGTT_SIZE)); 71 71 72 72 kunit_activate_static_stub(test, xe_managed_bo_create_pin_map, 73 73 replacement_xe_managed_bo_create_pin_map);

+1 -2

drivers/gpu/drm/xe/xe_bb.c

reviewed

··· 7 7 8 8 #include "instructions/xe_mi_commands.h" 9 9 #include "xe_assert.h" 10 10 - #include "xe_device.h" 10 10 + #include "xe_device_types.h" 11 11 #include "xe_exec_queue_types.h" 12 12 #include "xe_gt.h" 13 13 - #include "xe_hw_fence.h" 14 13 #include "xe_sa.h" 15 14 #include "xe_sched_job.h" 16 15 #include "xe_vm_types.h"

+19 -14

drivers/gpu/drm/xe/xe_bo.c

reviewed

··· 26 26 #include "xe_dma_buf.h" 27 27 #include "xe_drm_client.h" 28 28 #include "xe_ggtt.h" 29 29 - #include "xe_gt.h" 30 29 #include "xe_map.h" 31 30 #include "xe_migrate.h" 31 31 + #include "xe_pat.h" 32 32 #include "xe_pm.h" 33 33 #include "xe_preempt_fence.h" 34 34 #include "xe_pxp.h" ··· 1054 1054 unsigned long *scanned) 1055 1055 { 1056 1056 struct xe_device *xe = ttm_to_xe_device(bo->bdev); 1057 1057 + struct ttm_tt *tt = bo->ttm; 1057 1058 long lret; 1058 1059 1059 1060 /* Fake move to system, without copying data. */ ··· 1079 1078 .writeback = false, 1080 1079 .allow_move = false}); 1081 1080 1082 1082 - if (lret > 0) 1081 1081 + if (lret > 0) { 1083 1082 xe_ttm_tt_account_subtract(xe, bo->ttm); 1083 1083 + update_global_total_pages(bo->bdev, -(long)tt->num_pages); 1084 1084 + } 1084 1085 1085 1086 return lret; 1086 1087 } ··· 1168 1165 if (needs_rpm) 1169 1166 xe_pm_runtime_put(xe); 1170 1167 1171 1171 - if (lret > 0) 1168 1168 + if (lret > 0) { 1172 1169 xe_ttm_tt_account_subtract(xe, tt); 1170 1170 + update_global_total_pages(bo->bdev, -(long)tt->num_pages); 1171 1171 + } 1173 1172 1174 1173 out_unref: 1175 1174 xe_bo_put(xe_bo); ··· 1714 1709 xe_assert(xe, list_empty(&ttm_bo->base.gpuva.list)); 1715 1710 1716 1711 for_each_tile(tile, xe, id) 1717 1717 - if (bo->ggtt_node[id] && bo->ggtt_node[id]->base.size) 1712 1712 + if (bo->ggtt_node[id]) 1718 1713 xe_ggtt_remove_bo(tile->mem.ggtt, bo); 1719 1714 1720 1715 #ifdef CONFIG_PROC_FS ··· 3522 3517 if (IS_DGFX(xe) && (bo->flags & XE_BO_FLAG_SYSTEM)) 3523 3518 return false; 3524 3519 3520 3520 + /* Check if userspace explicitly requested no compression */ 3521 3521 + if (bo->flags & XE_BO_FLAG_NO_COMPRESSION) 3522 3522 + return false; 3523 3523 + 3525 3524 /* 3526 3526 - * Compression implies coh_none, therefore we know for sure that WB 3527 3527 - * memory can't currently use compression, which is likely one of the 3528 3528 - * common cases. 3529 3529 - * Additionally, userspace may explicitly request no compression via the 3530 3530 - * DRM_XE_GEM_CREATE_FLAG_NO_COMPRESSION flag, which should also disable 3531 3531 - * CCS usage. 3525 3525 + * For WB (Write-Back) CPU caching mode, check if the device 3526 3526 + * supports WB compression with coherency. 3532 3527 */ 3533 3533 - if (bo->cpu_caching == DRM_XE_GEM_CPU_CACHING_WB || 3534 3534 - bo->flags & XE_BO_FLAG_NO_COMPRESSION) 3528 3528 + if (bo->cpu_caching == DRM_XE_GEM_CPU_CACHING_WB && 3529 3529 + xe->pat.idx[XE_CACHE_WB_COMPRESSION] == XE_PAT_INVALID_IDX) 3535 3530 return false; 3536 3531 3537 3532 return true; ··· 3608 3603 might_lock(&bo->client->bos_lock); 3609 3604 #endif 3610 3605 for_each_tile(tile, xe_bo_device(bo), id) 3611 3611 - if (bo->ggtt_node[id] && bo->ggtt_node[id]->ggtt) 3612 3612 - xe_ggtt_might_lock(bo->ggtt_node[id]->ggtt); 3606 3606 + if (bo->ggtt_node[id]) 3607 3607 + xe_ggtt_might_lock(tile->mem.ggtt); 3613 3608 drm_gem_object_put(&bo->ttm.base); 3614 3609 } 3615 3610 }

+5 -3

drivers/gpu/drm/xe/xe_bo.h

reviewed

··· 9 9 #include <drm/ttm/ttm_tt.h> 10 10 11 11 #include "xe_bo_types.h" 12 12 + #include "xe_ggtt.h" 12 13 #include "xe_macros.h" 13 14 #include "xe_validation.h" 14 15 #include "xe_vm_types.h" ··· 253 252 __xe_bo_ggtt_addr(struct xe_bo *bo, u8 tile_id) 254 253 { 255 254 struct xe_ggtt_node *ggtt_node = bo->ggtt_node[tile_id]; 255 255 + u64 offset; 256 256 257 257 if (XE_WARN_ON(!ggtt_node)) 258 258 return 0; 259 259 260 260 - XE_WARN_ON(ggtt_node->base.size > xe_bo_size(bo)); 261 261 - XE_WARN_ON(ggtt_node->base.start + ggtt_node->base.size > (1ull << 32)); 262 262 - return ggtt_node->base.start; 260 260 + offset = xe_ggtt_node_addr(ggtt_node); 261 261 + XE_WARN_ON(offset + xe_bo_size(bo) > (1ull << 32)); 262 262 + return offset; 263 263 } 264 264 265 265 static inline u32

+60 -15

drivers/gpu/drm/xe/xe_debugfs.c

reviewed

··· 254 254 return simple_read_from_buffer(ubuf, size, pos, buf, len); 255 255 } 256 256 257 257 + static int __wedged_mode_set_reset_policy(struct xe_gt *gt, enum xe_wedged_mode mode) 258 258 + { 259 259 + bool enable_engine_reset; 260 260 + int ret; 261 261 + 262 262 + enable_engine_reset = (mode != XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET); 263 263 + ret = xe_guc_ads_scheduler_policy_toggle_reset(&gt->uc.guc.ads, 264 264 + enable_engine_reset); 265 265 + if (ret) 266 266 + xe_gt_err(gt, "Failed to update GuC ADS scheduler policy (%pe)\n", ERR_PTR(ret)); 267 267 + 268 268 + return ret; 269 269 + } 270 270 + 271 271 + static int wedged_mode_set_reset_policy(struct xe_device *xe, enum xe_wedged_mode mode) 272 272 + { 273 273 + struct xe_gt *gt; 274 274 + int ret; 275 275 + u8 id; 276 276 + 277 277 + guard(xe_pm_runtime)(xe); 278 278 + for_each_gt(gt, xe, id) { 279 279 + ret = __wedged_mode_set_reset_policy(gt, mode); 280 280 + if (ret) { 281 281 + if (id > 0) { 282 282 + xe->wedged.inconsistent_reset = true; 283 283 + drm_err(&xe->drm, "Inconsistent reset policy state between GTs\n"); 284 284 + } 285 285 + return ret; 286 286 + } 287 287 + } 288 288 + 289 289 + xe->wedged.inconsistent_reset = false; 290 290 + 291 291 + return 0; 292 292 + } 293 293 + 294 294 + static bool wedged_mode_needs_policy_update(struct xe_device *xe, enum xe_wedged_mode mode) 295 295 + { 296 296 + if (xe->wedged.inconsistent_reset) 297 297 + return true; 298 298 + 299 299 + if (xe->wedged.mode == mode) 300 300 + return false; 301 301 + 302 302 + if (xe->wedged.mode == XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET || 303 303 + mode == XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET) 304 304 + return true; 305 305 + 306 306 + return false; 307 307 + } 308 308 + 257 309 static ssize_t wedged_mode_set(struct file *f, const char __user *ubuf, 258 310 size_t size, loff_t *pos) 259 311 { 260 312 struct xe_device *xe = file_inode(f)->i_private; 261 261 - struct xe_gt *gt; 262 313 u32 wedged_mode; 263 314 ssize_t ret; 264 264 - u8 id; 265 315 266 316 ret = kstrtouint_from_user(ubuf, size, 0, &wedged_mode); 267 317 if (ret) 268 318 return ret; 269 319 270 270 - if (wedged_mode > 2) 271 271 - return -EINVAL; 320 320 + ret = xe_device_validate_wedged_mode(xe, wedged_mode); 321 321 + if (ret) 322 322 + return ret; 272 323 273 273 - if (xe->wedged.mode == wedged_mode) 274 274 - return size; 324 324 + if (wedged_mode_needs_policy_update(xe, wedged_mode)) { 325 325 + ret = wedged_mode_set_reset_policy(xe, wedged_mode); 326 326 + if (ret) 327 327 + return ret; 328 328 + } 275 329 276 330 xe->wedged.mode = wedged_mode; 277 277 - 278 278 - guard(xe_pm_runtime)(xe); 279 279 - for_each_gt(gt, xe, id) { 280 280 - ret = xe_guc_ads_scheduler_policy_toggle_reset(&gt->uc.guc.ads); 281 281 - if (ret) { 282 282 - xe_gt_err(gt, "Failed to update GuC ADS scheduler policy. GuC may still cause engine reset even with wedged_mode=2\n"); 283 283 - return -EIO; 284 284 - } 285 285 - } 286 331 287 332 return size; 288 333 }

+1 -2

drivers/gpu/drm/xe/xe_devcoredump.c

reviewed

··· 15 15 #include "xe_device.h" 16 16 #include "xe_exec_queue.h" 17 17 #include "xe_force_wake.h" 18 18 - #include "xe_gt.h" 19 18 #include "xe_gt_printk.h" 19 19 + #include "xe_gt_types.h" 20 20 #include "xe_guc_capture.h" 21 21 #include "xe_guc_ct.h" 22 22 #include "xe_guc_log.h" 23 23 #include "xe_guc_submit.h" 24 24 #include "xe_hw_engine.h" 25 25 - #include "xe_module.h" 26 25 #include "xe_pm.h" 27 26 #include "xe_sched_job.h" 28 27 #include "xe_vm.h"

+71 -70

drivers/gpu/drm/xe/xe_device.c

reviewed

··· 8 8 #include <linux/aperture.h> 9 9 #include <linux/delay.h> 10 10 #include <linux/fault-inject.h> 11 11 - #include <linux/iopoll.h> 12 11 #include <linux/units.h> 13 12 14 13 #include <drm/drm_atomic_helper.h> ··· 35 36 #include "xe_exec_queue.h" 36 37 #include "xe_force_wake.h" 37 38 #include "xe_ggtt.h" 38 38 - #include "xe_gsc_proxy.h" 39 39 #include "xe_gt.h" 40 40 #include "xe_gt_mcr.h" 41 41 #include "xe_gt_printk.h" ··· 178 180 xa_for_each(&xef->exec_queue.xa, idx, q) { 179 181 if (q->vm && q->hwe->hw_engine_group) 180 182 xe_hw_engine_group_del_exec_queue(q->hwe->hw_engine_group, q); 181 181 - 182 182 - if (xe_exec_queue_is_multi_queue_primary(q)) 183 183 - xe_exec_queue_group_kill_put(q->multi_queue.group); 184 184 - else 185 185 - xe_exec_queue_kill(q); 186 186 - 183 183 + xe_exec_queue_kill(q); 187 184 xe_exec_queue_put(q); 188 185 } 189 186 xa_for_each(&xef->vm.xa, idx, vm) ··· 652 659 return err; 653 660 } 654 661 655 655 - static int lmem_initializing(struct xe_device *xe) 662 662 + static void assert_lmem_ready(struct xe_device *xe) 656 663 { 657 657 - if (xe_mmio_read32(xe_root_tile_mmio(xe), GU_CNTL) & LMEM_INIT) 658 658 - return 0; 664 664 + if (!IS_DGFX(xe) || IS_SRIOV_VF(xe)) 665 665 + return; 659 666 660 660 - if (signal_pending(current)) 661 661 - return -EINTR; 662 662 - 663 663 - return 1; 667 667 + xe_assert(xe, xe_mmio_read32(xe_root_tile_mmio(xe), GU_CNTL) & 668 668 + LMEM_INIT); 664 669 } 665 665 - 666 666 - static int wait_for_lmem_ready(struct xe_device *xe) 667 667 - { 668 668 - const unsigned long TIMEOUT_SEC = 60; 669 669 - unsigned long prev_jiffies; 670 670 - int initializing; 671 671 - 672 672 - if (!IS_DGFX(xe)) 673 673 - return 0; 674 674 - 675 675 - if (IS_SRIOV_VF(xe)) 676 676 - return 0; 677 677 - 678 678 - if (!lmem_initializing(xe)) 679 679 - return 0; 680 680 - 681 681 - drm_dbg(&xe->drm, "Waiting for lmem initialization\n"); 682 682 - prev_jiffies = jiffies; 683 683 - 684 684 - /* 685 685 - * The boot firmware initializes local memory and 686 686 - * assesses its health. If memory training fails, 687 687 - * the punit will have been instructed to keep the GT powered 688 688 - * down.we won't be able to communicate with it 689 689 - * 690 690 - * If the status check is done before punit updates the register, 691 691 - * it can lead to the system being unusable. 692 692 - * use a timeout and defer the probe to prevent this. 693 693 - */ 694 694 - poll_timeout_us(initializing = lmem_initializing(xe), 695 695 - initializing <= 0, 696 696 - 20 * USEC_PER_MSEC, TIMEOUT_SEC * USEC_PER_SEC, true); 697 697 - if (initializing < 0) 698 698 - return initializing; 699 699 - 700 700 - if (initializing) { 701 701 - drm_dbg(&xe->drm, "lmem not initialized by firmware\n"); 702 702 - return -EPROBE_DEFER; 703 703 - } 704 704 - 705 705 - drm_dbg(&xe->drm, "lmem ready after %ums", 706 706 - jiffies_to_msecs(jiffies - prev_jiffies)); 707 707 - 708 708 - return 0; 709 709 - } 710 710 - ALLOW_ERROR_INJECTION(wait_for_lmem_ready, ERRNO); /* See xe_pci_probe() */ 711 670 712 671 static void vf_update_device_info(struct xe_device *xe) 713 672 { ··· 714 769 if (IS_SRIOV_VF(xe)) 715 770 vf_update_device_info(xe); 716 771 772 772 + /* 773 773 + * Check for pcode uncore_init status to confirm if the SoC 774 774 + * initialization is complete. Until done, any MMIO or lmem access from 775 775 + * the driver will be blocked 776 776 + */ 717 777 err = xe_pcode_probe_early(xe); 718 778 if (err || xe_survivability_mode_is_requested(xe)) { 719 779 int save_err = err; ··· 735 785 return save_err; 736 786 } 737 787 738 738 - err = wait_for_lmem_ready(xe); 739 739 - if (err) 740 740 - return err; 788 788 + /* 789 789 + * Make sure the lmem is initialized and ready to use. xe_pcode_ready() 790 790 + * is flagged after full initialization is complete. Assert if lmem is 791 791 + * not initialized. 792 792 + */ 793 793 + assert_lmem_ready(xe); 741 794 742 742 - xe->wedged.mode = xe_modparam.wedged_mode; 795 795 + xe->wedged.mode = xe_device_validate_wedged_mode(xe, xe_modparam.wedged_mode) ? 796 796 + XE_WEDGED_MODE_DEFAULT : xe_modparam.wedged_mode; 797 797 + drm_dbg(&xe->drm, "wedged_mode: setting mode (%u) %s\n", 798 798 + xe->wedged.mode, xe_wedged_mode_to_string(xe->wedged.mode)); 743 799 744 800 err = xe_device_vram_alloc(xe); 745 801 if (err) ··· 1228 1272 * DOC: Xe Device Wedging 1229 1273 * 1230 1274 * Xe driver uses drm device wedged uevent as documented in Documentation/gpu/drm-uapi.rst. 1231 1231 - * When device is in wedged state, every IOCTL will be blocked and GT cannot be 1232 1232 - * used. Certain critical errors like gt reset failure, firmware failures can cause 1233 1233 - * the device to be wedged. The default recovery method for a wedged state 1234 1234 - * is rebind/bus-reset. 1275 1275 + * When device is in wedged state, every IOCTL will be blocked and GT cannot 1276 1276 + * be used. The conditions under which the driver declares the device wedged 1277 1277 + * depend on the wedged mode configuration (see &enum xe_wedged_mode). The 1278 1278 + * default recovery method for a wedged state is rebind/bus-reset. 1235 1279 * 1236 1280 * Another recovery method is vendor-specific. Below are the cases that send 1237 1281 * ``WEDGED=vendor-specific`` recovery method in drm device wedged uevent. ··· 1296 1340 struct xe_gt *gt; 1297 1341 u8 id; 1298 1342 1299 1299 - if (xe->wedged.mode == 0) { 1343 1343 + if (xe->wedged.mode == XE_WEDGED_MODE_NEVER) { 1300 1344 drm_dbg(&xe->drm, "Wedged mode is forcibly disabled\n"); 1301 1345 return; 1302 1346 } ··· 1328 1372 1329 1373 /* Notify userspace of wedged device */ 1330 1374 drm_dev_wedged_event(&xe->drm, xe->wedged.method, NULL); 1375 1375 + } 1376 1376 + } 1377 1377 + 1378 1378 + /** 1379 1379 + * xe_device_validate_wedged_mode - Check if given mode is supported 1380 1380 + * @xe: the &xe_device 1381 1381 + * @mode: requested mode to validate 1382 1382 + * 1383 1383 + * Check whether the provided wedged mode is supported. 1384 1384 + * 1385 1385 + * Return: 0 if mode is supported, error code otherwise. 1386 1386 + */ 1387 1387 + int xe_device_validate_wedged_mode(struct xe_device *xe, unsigned int mode) 1388 1388 + { 1389 1389 + if (mode > XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET) { 1390 1390 + drm_dbg(&xe->drm, "wedged_mode: invalid value (%u)\n", mode); 1391 1391 + return -EINVAL; 1392 1392 + } else if (mode == XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET && (IS_SRIOV_VF(xe) || 1393 1393 + (IS_SRIOV_PF(xe) && !IS_ENABLED(CONFIG_DRM_XE_DEBUG)))) { 1394 1394 + drm_dbg(&xe->drm, "wedged_mode: (%u) %s mode is not supported for %s\n", 1395 1395 + mode, xe_wedged_mode_to_string(mode), 1396 1396 + xe_sriov_mode_to_string(xe_device_sriov_mode(xe))); 1397 1397 + return -EPERM; 1398 1398 + } 1399 1399 + 1400 1400 + return 0; 1401 1401 + } 1402 1402 + 1403 1403 + /** 1404 1404 + * xe_wedged_mode_to_string - Convert enum value to string. 1405 1405 + * @mode: the &xe_wedged_mode to convert 1406 1406 + * 1407 1407 + * Returns: wedged mode as a user friendly string. 1408 1408 + */ 1409 1409 + const char *xe_wedged_mode_to_string(enum xe_wedged_mode mode) 1410 1410 + { 1411 1411 + switch (mode) { 1412 1412 + case XE_WEDGED_MODE_NEVER: 1413 1413 + return "never"; 1414 1414 + case XE_WEDGED_MODE_UPON_CRITICAL_ERROR: 1415 1415 + return "upon-critical-error"; 1416 1416 + case XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET: 1417 1417 + return "upon-any-hang-no-reset"; 1418 1418 + default: 1419 1419 + return "<invalid>"; 1331 1420 } 1332 1421 }

+2

drivers/gpu/drm/xe/xe_device.h

reviewed

··· 194 194 195 195 void xe_device_set_wedged_method(struct xe_device *xe, unsigned long method); 196 196 void xe_device_declare_wedged(struct xe_device *xe); 197 197 + int xe_device_validate_wedged_mode(struct xe_device *xe, unsigned int mode); 198 198 + const char *xe_wedged_mode_to_string(enum xe_wedged_mode mode); 197 199 198 200 struct xe_file *xe_file_get(struct xe_file *xef); 199 201 void xe_file_put(struct xe_file *xef);

+24 -1

drivers/gpu/drm/xe/xe_device_types.h

reviewed

··· 46 46 struct xe_pxp; 47 47 struct xe_vram_region; 48 48 49 49 + /** 50 50 + * enum xe_wedged_mode - possible wedged modes 51 51 + * @XE_WEDGED_MODE_NEVER: Device will never be declared wedged. 52 52 + * @XE_WEDGED_MODE_UPON_CRITICAL_ERROR: Device will be declared wedged only 53 53 + * when critical error occurs like GT reset failure or firmware failure. 54 54 + * This is the default mode. 55 55 + * @XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET: Device will be declared wedged on 56 56 + * any hang. In this mode, engine resets are disabled to avoid automatic 57 57 + * recovery attempts. This mode is primarily intended for debugging hangs. 58 58 + */ 59 59 + enum xe_wedged_mode { 60 60 + XE_WEDGED_MODE_NEVER = 0, 61 61 + XE_WEDGED_MODE_UPON_CRITICAL_ERROR = 1, 62 62 + XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET = 2, 63 63 + }; 64 64 + 65 65 + #define XE_WEDGED_MODE_DEFAULT XE_WEDGED_MODE_UPON_CRITICAL_ERROR 66 66 + #define XE_WEDGED_MODE_DEFAULT_STR "upon-critical-error" 67 67 + 49 68 #define XE_BO_INVALID_OFFSET LONG_MAX 50 69 51 70 #define GRAPHICS_VER(xe) ((xe)->info.graphics_verx100 / 100) ··· 341 322 * pcode mailbox commands. 342 323 */ 343 324 u8 has_mbx_power_limits:1; 325 325 + /** @info.has_mbx_thermal_info: Device supports thermal mailbox commands */ 326 326 + u8 has_mbx_thermal_info:1; 344 327 /** @info.has_mem_copy_instr: Device supports MEM_COPY instruction */ 345 328 u8 has_mem_copy_instr:1; 346 329 /** @info.has_mert: Device has standalone MERT */ ··· 647 626 /** @wedged.flag: Xe device faced a critical error and is now blocked. */ 648 627 atomic_t flag; 649 628 /** @wedged.mode: Mode controlled by kernel parameter and debugfs */ 650 650 - int mode; 629 629 + enum xe_wedged_mode mode; 651 630 /** @wedged.method: Recovery method to be sent in the drm device wedged uevent */ 652 631 unsigned long method; 632 632 + /** @wedged.inconsistent_reset: Inconsistent reset policy state between GTs */ 633 633 + bool inconsistent_reset; 653 634 } wedged; 654 635 655 636 /** @bo_device: Struct to control async free of BOs */

-1

drivers/gpu/drm/xe/xe_exec.c

reviewed

··· 11 11 #include <uapi/drm/xe_drm.h> 12 12 #include <linux/delay.h> 13 13 14 14 - #include "xe_bo.h" 15 14 #include "xe_device.h" 16 15 #include "xe_exec_queue.h" 17 16 #include "xe_hw_engine_group.h"

+4 -58

drivers/gpu/drm/xe/xe_exec_queue.c

reviewed

··· 21 21 #include "xe_gt_sriov_vf.h" 22 22 #include "xe_hw_engine_class_sysfs.h" 23 23 #include "xe_hw_engine_group.h" 24 24 - #include "xe_hw_fence.h" 25 24 #include "xe_irq.h" 26 25 #include "xe_lrc.h" 27 26 #include "xe_macros.h" 28 27 #include "xe_migrate.h" 29 28 #include "xe_pm.h" 30 30 - #include "xe_ring_ops_types.h" 31 29 #include "xe_trace.h" 32 30 #include "xe_vm.h" 33 31 #include "xe_pxp.h" ··· 82 84 * group is destroyed. The secondary queues hold a reference to the primary 83 85 * queue thus preventing the group from being destroyed when user destroys 84 86 * the primary queue. Once the primary queue is destroyed, secondary queues 85 85 - * can't be added to the queue group, but they can continue to submit the 86 86 - * jobs if the DRM_XE_MULTI_GROUP_KEEP_ACTIVE flag is set during the multi 87 87 - * queue group creation. 87 87 + * can't be added to the queue group and new job submissions on existing 88 88 + * secondary queues are not allowed. 88 89 * 89 90 * The queues of a multi queue group can set their priority within the group 90 91 * through the DRM_XE_EXEC_QUEUE_SET_PROPERTY_MULTI_QUEUE_PRIORITY property. ··· 464 467 } 465 468 ALLOW_ERROR_INJECTION(xe_exec_queue_create_bind, ERRNO); 466 469 467 467 - static void xe_exec_queue_group_kill(struct kref *ref) 468 468 - { 469 469 - struct xe_exec_queue_group *group = container_of(ref, struct xe_exec_queue_group, 470 470 - kill_refcount); 471 471 - xe_exec_queue_kill(group->primary); 472 472 - } 473 473 - 474 474 - static inline void xe_exec_queue_group_kill_get(struct xe_exec_queue_group *group) 475 475 - { 476 476 - kref_get(&group->kill_refcount); 477 477 - } 478 478 - 479 479 - void xe_exec_queue_group_kill_put(struct xe_exec_queue_group *group) 480 480 - { 481 481 - if (!group) 482 482 - return; 483 483 - 484 484 - kref_put(&group->kill_refcount, xe_exec_queue_group_kill); 485 485 - } 486 486 - 487 470 void xe_exec_queue_destroy(struct kref *ref) 488 471 { 489 472 struct xe_exec_queue *q = container_of(ref, struct xe_exec_queue, refcount); ··· 693 716 group->primary = q; 694 717 group->cgp_bo = bo; 695 718 INIT_LIST_HEAD(&group->list); 696 696 - kref_init(&group->kill_refcount); 697 719 xa_init_flags(&group->xa, XA_FLAGS_ALLOC1); 698 720 mutex_init(&group->list_lock); 699 721 q->multi_queue.group = group; ··· 768 792 769 793 q->multi_queue.pos = pos; 770 794 771 771 - if (group->primary->multi_queue.keep_active) { 772 772 - xe_exec_queue_group_kill_get(group); 773 773 - q->multi_queue.keep_active = true; 774 774 - } 775 775 - 776 795 return 0; 777 796 } 778 797 ··· 781 810 lrc = xa_erase(&group->xa, q->multi_queue.pos); 782 811 xe_assert(xe, lrc); 783 812 xe_lrc_put(lrc); 784 784 - 785 785 - if (q->multi_queue.keep_active) { 786 786 - xe_exec_queue_group_kill_put(group); 787 787 - q->multi_queue.keep_active = false; 788 788 - } 789 813 } 790 814 791 815 static int exec_queue_set_multi_group(struct xe_device *xe, struct xe_exec_queue *q, ··· 802 836 return -EINVAL; 803 837 804 838 if (value & DRM_XE_MULTI_GROUP_CREATE) { 805 805 - if (XE_IOCTL_DBG(xe, value & ~(DRM_XE_MULTI_GROUP_CREATE | 806 806 - DRM_XE_MULTI_GROUP_KEEP_ACTIVE))) 807 807 - return -EINVAL; 808 808 - 809 809 - /* 810 810 - * KEEP_ACTIVE is not supported in preempt fence mode as in that mode, 811 811 - * VM_DESTROY ioctl expects all exec queues of that VM are already killed. 812 812 - */ 813 813 - if (XE_IOCTL_DBG(xe, (value & DRM_XE_MULTI_GROUP_KEEP_ACTIVE) && 814 814 - xe_vm_in_preempt_fence_mode(q->vm))) 839 839 + if (XE_IOCTL_DBG(xe, value & ~DRM_XE_MULTI_GROUP_CREATE)) 815 840 return -EINVAL; 816 841 817 842 q->multi_queue.valid = true; 818 843 q->multi_queue.is_primary = true; 819 844 q->multi_queue.pos = 0; 820 820 - if (value & DRM_XE_MULTI_GROUP_KEEP_ACTIVE) 821 821 - q->multi_queue.keep_active = true; 822 822 - 823 845 return 0; 824 846 } 825 847 ··· 1373 1419 1374 1420 q->ops->kill(q); 1375 1421 xe_vm_remove_compute_exec_queue(q->vm, q); 1376 1376 - 1377 1377 - if (!xe_exec_queue_is_multi_queue_primary(q) && q->multi_queue.keep_active) { 1378 1378 - xe_exec_queue_group_kill_put(q->multi_queue.group); 1379 1379 - q->multi_queue.keep_active = false; 1380 1380 - } 1381 1422 } 1382 1423 1383 1424 int xe_exec_queue_destroy_ioctl(struct drm_device *dev, void *data, ··· 1399 1450 if (q->vm && q->hwe->hw_engine_group) 1400 1451 xe_hw_engine_group_del_exec_queue(q->hwe->hw_engine_group, q); 1401 1452 1402 1402 - if (xe_exec_queue_is_multi_queue_primary(q)) 1403 1403 - xe_exec_queue_group_kill_put(q->multi_queue.group); 1404 1404 - else 1405 1405 - xe_exec_queue_kill(q); 1453 1453 + xe_exec_queue_kill(q); 1406 1454 1407 1455 trace_xe_exec_queue_close(q); 1408 1456 xe_exec_queue_put(q);

-2

drivers/gpu/drm/xe/xe_exec_queue.h

reviewed

··· 113 113 return xe_exec_queue_is_multi_queue(q) ? q->multi_queue.group->primary : q; 114 114 } 115 115 116 116 - void xe_exec_queue_group_kill_put(struct xe_exec_queue_group *group); 117 117 - 118 116 bool xe_exec_queue_is_lr(struct xe_exec_queue *q); 119 117 120 118 bool xe_exec_queue_is_idle(struct xe_exec_queue *q);

-4

drivers/gpu/drm/xe/xe_exec_queue_types.h

reviewed

··· 62 62 struct list_head list; 63 63 /** @list_lock: Secondary queue list lock */ 64 64 struct mutex list_lock; 65 65 - /** @kill_refcount: ref count to kill primary queue */ 66 66 - struct kref kill_refcount; 67 65 /** @sync_pending: CGP_SYNC_DONE g2h response pending */ 68 66 bool sync_pending; 69 67 /** @banned: Group banned */ ··· 161 163 u8 valid:1; 162 164 /** @multi_queue.is_primary: Is primary queue (Q0) of the group */ 163 165 u8 is_primary:1; 164 164 - /** @multi_queue.keep_active: Keep the group active after primary is destroyed */ 165 165 - u8 keep_active:1; 166 166 } multi_queue; 167 167 168 168 /** @sched_props: scheduling properties */

+1 -2

drivers/gpu/drm/xe/xe_execlist.c

reviewed

··· 15 15 #include "xe_bo.h" 16 16 #include "xe_device.h" 17 17 #include "xe_exec_queue.h" 18 18 - #include "xe_gt.h" 19 19 - #include "xe_hw_fence.h" 18 18 + #include "xe_gt_types.h" 20 19 #include "xe_irq.h" 21 20 #include "xe_lrc.h" 22 21 #include "xe_macros.h"

+161 -58

drivers/gpu/drm/xe/xe_ggtt.c

reviewed

··· 20 20 #include "regs/xe_regs.h" 21 21 #include "xe_assert.h" 22 22 #include "xe_bo.h" 23 23 - #include "xe_device.h" 24 24 - #include "xe_gt.h" 25 23 #include "xe_gt_printk.h" 24 24 + #include "xe_gt_types.h" 26 25 #include "xe_map.h" 27 26 #include "xe_mmio.h" 28 27 #include "xe_pm.h" ··· 65 66 * above the WOPCM max size. Starting the GGTT allocations above the WOPCM max 66 67 * give us the correct placement for free. 67 68 */ 69 69 + 70 70 + /** 71 71 + * struct xe_ggtt_node - A node in GGTT. 72 72 + * 73 73 + * This struct needs to be initialized (only-once) with xe_ggtt_node_init() before any node 74 74 + * insertion, reservation, or 'ballooning'. 75 75 + * It will, then, be finalized by either xe_ggtt_node_remove() or xe_ggtt_node_deballoon(). 76 76 + */ 77 77 + struct xe_ggtt_node { 78 78 + /** @ggtt: Back pointer to xe_ggtt where this region will be inserted at */ 79 79 + struct xe_ggtt *ggtt; 80 80 + /** @base: A drm_mm_node */ 81 81 + struct drm_mm_node base; 82 82 + /** @delayed_removal_work: The work struct for the delayed removal */ 83 83 + struct work_struct delayed_removal_work; 84 84 + /** @invalidate_on_remove: If it needs invalidation upon removal */ 85 85 + bool invalidate_on_remove; 86 86 + }; 68 87 69 88 static u64 xelp_ggtt_pte_flags(struct xe_bo *bo, u16 pat_index) 70 89 { ··· 154 137 } 155 138 } 156 139 140 140 + /** 141 141 + * xe_ggtt_start - Get starting offset of GGTT. 142 142 + * @ggtt: &xe_ggtt 143 143 + * 144 144 + * Returns: Starting offset for this &xe_ggtt. 145 145 + */ 146 146 + u64 xe_ggtt_start(struct xe_ggtt *ggtt) 147 147 + { 148 148 + return ggtt->start; 149 149 + } 150 150 + 151 151 + /** 152 152 + * xe_ggtt_size - Get size of GGTT. 153 153 + * @ggtt: &xe_ggtt 154 154 + * 155 155 + * Returns: Total usable size of this &xe_ggtt. 156 156 + */ 157 157 + u64 xe_ggtt_size(struct xe_ggtt *ggtt) 158 158 + { 159 159 + return ggtt->size; 160 160 + } 161 161 + 157 162 static void xe_ggtt_set_pte(struct xe_ggtt *ggtt, u64 addr, u64 pte) 158 163 { 159 164 xe_tile_assert(ggtt->tile, !(addr & XE_PTE_MASK)); 160 160 - xe_tile_assert(ggtt->tile, addr < ggtt->size); 165 165 + xe_tile_assert(ggtt->tile, addr < ggtt->start + ggtt->size); 161 166 162 167 writeq(pte, &ggtt->gsm[addr >> XE_PTE_SHIFT]); 163 168 } ··· 295 256 .ggtt_get_pte = xe_ggtt_get_pte, 296 257 }; 297 258 298 298 - static void __xe_ggtt_init_early(struct xe_ggtt *ggtt, u32 reserved) 259 259 + static void __xe_ggtt_init_early(struct xe_ggtt *ggtt, u64 start, u64 size) 299 260 { 300 300 - drm_mm_init(&ggtt->mm, reserved, 301 301 - ggtt->size - reserved); 261 261 + ggtt->start = start; 262 262 + ggtt->size = size; 263 263 + drm_mm_init(&ggtt->mm, start, size); 302 264 } 303 265 304 304 - int xe_ggtt_init_kunit(struct xe_ggtt *ggtt, u32 reserved, u32 size) 266 266 + int xe_ggtt_init_kunit(struct xe_ggtt *ggtt, u32 start, u32 size) 305 267 { 306 306 - ggtt->size = size; 307 307 - __xe_ggtt_init_early(ggtt, reserved); 268 268 + __xe_ggtt_init_early(ggtt, start, size); 308 269 return 0; 309 270 } 310 271 EXPORT_SYMBOL_IF_KUNIT(xe_ggtt_init_kunit); ··· 332 293 struct xe_device *xe = tile_to_xe(ggtt->tile); 333 294 struct pci_dev *pdev = to_pci_dev(xe->drm.dev); 334 295 unsigned int gsm_size; 296 296 + u64 ggtt_start, wopcm = xe_wopcm_size(xe), ggtt_size; 335 297 int err; 336 298 337 337 - if (IS_SRIOV_VF(xe) || GRAPHICS_VERx100(xe) >= 1250) 338 338 - gsm_size = SZ_8M; /* GGTT is expected to be 4GiB */ 339 339 - else 340 340 - gsm_size = probe_gsm_size(pdev); 341 341 - 342 342 - if (gsm_size == 0) { 343 343 - xe_tile_err(ggtt->tile, "Hardware reported no preallocated GSM\n"); 344 344 - return -ENOMEM; 299 299 + if (!IS_SRIOV_VF(xe)) { 300 300 + if (GRAPHICS_VERx100(xe) >= 1250) 301 301 + gsm_size = SZ_8M; /* GGTT is expected to be 4GiB */ 302 302 + else 303 303 + gsm_size = probe_gsm_size(pdev); 304 304 + if (gsm_size == 0) { 305 305 + xe_tile_err(ggtt->tile, "Hardware reported no preallocated GSM\n"); 306 306 + return -ENOMEM; 307 307 + } 308 308 + ggtt_start = wopcm; 309 309 + ggtt_size = (gsm_size / 8) * (u64)XE_PAGE_SIZE - ggtt_start; 310 310 + } else { 311 311 + /* GGTT is expected to be 4GiB */ 312 312 + ggtt_start = wopcm; 313 313 + ggtt_size = SZ_4G - ggtt_start; 345 314 } 346 315 347 316 ggtt->gsm = ggtt->tile->mmio.regs + SZ_8M; 348 348 - ggtt->size = (gsm_size / 8) * (u64) XE_PAGE_SIZE; 349 349 - 350 317 if (IS_DGFX(xe) && xe->info.vram_flags & XE_VRAM_FLAGS_NEED64K) 351 318 ggtt->flags |= XE_GGTT_FLAGS_64K; 352 319 353 353 - if (ggtt->size > GUC_GGTT_TOP) 354 354 - ggtt->size = GUC_GGTT_TOP; 320 320 + if (ggtt_size + ggtt_start > GUC_GGTT_TOP) 321 321 + ggtt_size = GUC_GGTT_TOP - ggtt_start; 355 322 356 323 if (GRAPHICS_VERx100(xe) >= 1270) 357 324 ggtt->pt_ops = ··· 367 322 else 368 323 ggtt->pt_ops = &xelp_pt_ops; 369 324 370 370 - ggtt->wq = alloc_workqueue("xe-ggtt-wq", 0, WQ_MEM_RECLAIM); 325 325 + ggtt->wq = alloc_workqueue("xe-ggtt-wq", WQ_MEM_RECLAIM, 0); 371 326 if (!ggtt->wq) 372 327 return -ENOMEM; 373 328 374 374 - __xe_ggtt_init_early(ggtt, xe_wopcm_size(xe)); 329 329 + __xe_ggtt_init_early(ggtt, ggtt_start, ggtt_size); 375 330 376 331 err = drmm_add_action_or_reset(&xe->drm, ggtt_fini_early, ggtt); 377 332 if (err) ··· 608 563 static void xe_ggtt_assert_fit(struct xe_ggtt *ggtt, u64 start, u64 size) 609 564 { 610 565 struct xe_tile *tile = ggtt->tile; 611 611 - struct xe_device *xe = tile_to_xe(tile); 612 612 - u64 __maybe_unused wopcm = xe_wopcm_size(xe); 613 566 614 614 - xe_tile_assert(tile, start >= wopcm); 615 615 - xe_tile_assert(tile, start + size < ggtt->size - wopcm); 567 567 + xe_tile_assert(tile, start >= ggtt->start); 568 568 + xe_tile_assert(tile, start + size <= ggtt->start + ggtt->size); 616 569 } 617 570 618 571 /** ··· 653 610 } 654 611 } 655 612 656 656 - /** 657 657 - * xe_ggtt_node_insert_locked - Locked version to insert a &xe_ggtt_node into the GGTT 658 658 - * @node: the &xe_ggtt_node to be inserted 659 659 - * @size: size of the node 660 660 - * @align: alignment constrain of the node 661 661 - * @mm_flags: flags to control the node behavior 662 662 - * 663 663 - * It cannot be called without first having called xe_ggtt_init() once. 664 664 - * To be used in cases where ggtt->lock is already taken. 665 665 - * 666 666 - * Return: 0 on success or a negative error code on failure. 667 667 - */ 668 668 - int xe_ggtt_node_insert_locked(struct xe_ggtt_node *node, 669 669 - u32 size, u32 align, u32 mm_flags) 613 613 + static int xe_ggtt_node_insert_locked(struct xe_ggtt_node *node, 614 614 + u32 size, u32 align, u32 mm_flags) 670 615 { 671 616 return drm_mm_insert_node_generic(&node->ggtt->mm, &node->base, size, align, 0, 672 617 mm_flags); ··· 692 661 * This function will allocate the struct %xe_ggtt_node and return its pointer. 693 662 * This struct will then be freed after the node removal upon xe_ggtt_node_remove() 694 663 * or xe_ggtt_node_remove_balloon_locked(). 695 695 - * Having %xe_ggtt_node struct allocated doesn't mean that the node is already allocated 696 696 - * in GGTT. Only the xe_ggtt_node_insert(), xe_ggtt_node_insert_locked(), 697 697 - * xe_ggtt_node_insert_balloon_locked() will ensure the node is inserted or reserved in GGTT. 664 664 + * 665 665 + * Having %xe_ggtt_node struct allocated doesn't mean that the node is already 666 666 + * allocated in GGTT. Only xe_ggtt_node_insert(), allocation through 667 667 + * xe_ggtt_node_insert_transform(), or xe_ggtt_node_insert_balloon_locked() will ensure the node is inserted or reserved 668 668 + * in GGTT. 698 669 * 699 670 * Return: A pointer to %xe_ggtt_node struct on success. An ERR_PTR otherwise. 700 671 **/ ··· 759 726 * @ggtt: the &xe_ggtt where node will be mapped 760 727 * @node: the &xe_ggtt_node where this BO is mapped 761 728 * @bo: the &xe_bo to be mapped 762 762 - * @pat_index: Which pat_index to use. 729 729 + * @pte: The pte flags to append. 763 730 */ 764 764 - void xe_ggtt_map_bo(struct xe_ggtt *ggtt, struct xe_ggtt_node *node, 765 765 - struct xe_bo *bo, u16 pat_index) 731 731 + static void xe_ggtt_map_bo(struct xe_ggtt *ggtt, struct xe_ggtt_node *node, 732 732 + struct xe_bo *bo, u64 pte) 766 733 { 767 767 - 768 768 - u64 start, pte, end; 734 734 + u64 start, end; 769 735 struct xe_res_cursor cur; 770 736 771 737 if (XE_WARN_ON(!node)) ··· 773 741 start = node->base.start; 774 742 end = start + xe_bo_size(bo); 775 743 776 776 - pte = ggtt->pt_ops->pte_encode_flags(bo, pat_index); 777 744 if (!xe_bo_is_vram(bo) && !xe_bo_is_stolen(bo)) { 778 745 xe_assert(xe_bo_device(bo), bo->ttm.ttm); 779 746 ··· 802 771 { 803 772 u16 cache_mode = bo->flags & XE_BO_FLAG_NEEDS_UC ? XE_CACHE_NONE : XE_CACHE_WB; 804 773 u16 pat_index = tile_to_xe(ggtt->tile)->pat.idx[cache_mode]; 774 774 + u64 pte; 805 775 806 776 mutex_lock(&ggtt->lock); 807 807 - xe_ggtt_map_bo(ggtt, bo->ggtt_node[ggtt->tile->id], bo, pat_index); 777 777 + pte = ggtt->pt_ops->pte_encode_flags(bo, pat_index); 778 778 + xe_ggtt_map_bo(ggtt, bo->ggtt_node[ggtt->tile->id], bo, pte); 808 779 mutex_unlock(&ggtt->lock); 780 780 + } 781 781 + 782 782 + /** 783 783 + * xe_ggtt_node_insert_transform - Insert a newly allocated &xe_ggtt_node into the GGTT 784 784 + * @ggtt: the &xe_ggtt where the node will inserted/reserved. 785 785 + * @bo: The bo to be transformed 786 786 + * @pte_flags: The extra GGTT flags to add to mapping. 787 787 + * @size: size of the node 788 788 + * @align: required alignment for node 789 789 + * @transform: transformation function that will populate the GGTT node, or NULL for linear mapping. 790 790 + * @arg: Extra argument to pass to the transformation function. 791 791 + * 792 792 + * This function allows inserting a GGTT node with a custom transformation function. 793 793 + * This is useful for display to allow inserting rotated framebuffers to GGTT. 794 794 + * 795 795 + * Return: A pointer to %xe_ggtt_node struct on success. An ERR_PTR otherwise. 796 796 + */ 797 797 + struct xe_ggtt_node *xe_ggtt_node_insert_transform(struct xe_ggtt *ggtt, 798 798 + struct xe_bo *bo, u64 pte_flags, 799 799 + u64 size, u32 align, 800 800 + xe_ggtt_transform_cb transform, void *arg) 801 801 + { 802 802 + struct xe_ggtt_node *node; 803 803 + int ret; 804 804 + 805 805 + node = xe_ggtt_node_init(ggtt); 806 806 + if (IS_ERR(node)) 807 807 + return ERR_CAST(node); 808 808 + 809 809 + if (mutex_lock_interruptible(&ggtt->lock) < 0) { 810 810 + ret = -ERESTARTSYS; 811 811 + goto err; 812 812 + } 813 813 + 814 814 + ret = xe_ggtt_node_insert_locked(node, size, align, 0); 815 815 + if (ret) 816 816 + goto err_unlock; 817 817 + 818 818 + if (transform) 819 819 + transform(ggtt, node, pte_flags, ggtt->pt_ops->ggtt_set_pte, arg); 820 820 + else 821 821 + xe_ggtt_map_bo(ggtt, node, bo, pte_flags); 822 822 + 823 823 + mutex_unlock(&ggtt->lock); 824 824 + return node; 825 825 + 826 826 + err_unlock: 827 827 + mutex_unlock(&ggtt->lock); 828 828 + err: 829 829 + xe_ggtt_node_fini(node); 830 830 + return ERR_PTR(ret); 809 831 } 810 832 811 833 static int __xe_ggtt_insert_bo_at(struct xe_ggtt *ggtt, struct xe_bo *bo, ··· 899 815 } else { 900 816 u16 cache_mode = bo->flags & XE_BO_FLAG_NEEDS_UC ? XE_CACHE_NONE : XE_CACHE_WB; 901 817 u16 pat_index = tile_to_xe(ggtt->tile)->pat.idx[cache_mode]; 818 818 + u64 pte = ggtt->pt_ops->pte_encode_flags(bo, pat_index); 902 819 903 903 - xe_ggtt_map_bo(ggtt, bo->ggtt_node[tile_id], bo, pat_index); 820 820 + xe_ggtt_map_bo(ggtt, bo->ggtt_node[tile_id], bo, pte); 904 821 } 905 822 mutex_unlock(&ggtt->lock); 906 823 ··· 975 890 { 976 891 const struct drm_mm *mm = &ggtt->mm; 977 892 const struct drm_mm_node *entry; 978 978 - u64 hole_min_start = xe_wopcm_size(tile_to_xe(ggtt->tile)); 979 893 u64 hole_start, hole_end, hole_size; 980 894 u64 max_hole = 0; 981 895 982 896 mutex_lock(&ggtt->lock); 983 983 - 984 897 drm_mm_for_each_hole(entry, mm, hole_start, hole_end) { 985 985 - hole_start = max(hole_start, hole_min_start); 898 898 + hole_start = max(hole_start, ggtt->start); 986 899 hole_start = ALIGN(hole_start, alignment); 987 900 hole_end = ALIGN_DOWN(hole_end, alignment); 988 901 if (hole_start >= hole_end) ··· 1152 1069 { 1153 1070 const struct drm_mm *mm = &ggtt->mm; 1154 1071 const struct drm_mm_node *entry; 1155 1155 - u64 hole_min_start = xe_wopcm_size(tile_to_xe(ggtt->tile)); 1156 1072 u64 hole_start, hole_end, hole_size; 1157 1073 u64 total = 0; 1158 1074 char buf[10]; 1159 1075 1160 1076 mutex_lock(&ggtt->lock); 1161 1161 - 1162 1077 drm_mm_for_each_hole(entry, mm, hole_start, hole_end) { 1163 1163 - hole_start = max(hole_start, hole_min_start); 1078 1078 + hole_start = max(hole_start, ggtt->start); 1164 1079 hole_start = ALIGN(hole_start, alignment); 1165 1080 hole_end = ALIGN_DOWN(hole_end, alignment); 1166 1081 if (hole_start >= hole_end) ··· 1201 1120 u64 xe_ggtt_read_pte(struct xe_ggtt *ggtt, u64 offset) 1202 1121 { 1203 1122 return ioread64(ggtt->gsm + (offset / XE_PAGE_SIZE)); 1123 1123 + } 1124 1124 + 1125 1125 + /** 1126 1126 + * xe_ggtt_node_addr - Get @node offset in GGTT. 1127 1127 + * @node: &xe_ggtt_node 1128 1128 + * 1129 1129 + * Get the GGTT offset for allocated node. 1130 1130 + */ 1131 1131 + u64 xe_ggtt_node_addr(const struct xe_ggtt_node *node) 1132 1132 + { 1133 1133 + return node->base.start; 1134 1134 + } 1135 1135 + 1136 1136 + /** 1137 1137 + * xe_ggtt_node_size - Get @node allocation size. 1138 1138 + * @node: &xe_ggtt_node 1139 1139 + * 1140 1140 + * Get the allocated node's size. 1141 1141 + */ 1142 1142 + u64 xe_ggtt_node_size(const struct xe_ggtt_node *node) 1143 1143 + { 1144 1144 + return node->base.size; 1204 1145 }

+10 -4

drivers/gpu/drm/xe/xe_ggtt.h

reviewed

··· 23 23 u64 start, u64 size); 24 24 void xe_ggtt_node_remove_balloon_locked(struct xe_ggtt_node *node); 25 25 void xe_ggtt_shift_nodes_locked(struct xe_ggtt *ggtt, s64 shift); 26 26 + u64 xe_ggtt_start(struct xe_ggtt *ggtt); 27 27 + u64 xe_ggtt_size(struct xe_ggtt *ggtt); 26 28 27 29 int xe_ggtt_node_insert(struct xe_ggtt_node *node, u32 size, u32 align); 28 28 - int xe_ggtt_node_insert_locked(struct xe_ggtt_node *node, 29 29 - u32 size, u32 align, u32 mm_flags); 30 30 + struct xe_ggtt_node * 31 31 + xe_ggtt_node_insert_transform(struct xe_ggtt *ggtt, 32 32 + struct xe_bo *bo, u64 pte, 33 33 + u64 size, u32 align, 34 34 + xe_ggtt_transform_cb transform, void *arg); 30 35 void xe_ggtt_node_remove(struct xe_ggtt_node *node, bool invalidate); 31 36 bool xe_ggtt_node_allocated(const struct xe_ggtt_node *node); 32 37 size_t xe_ggtt_node_pt_size(const struct xe_ggtt_node *node); 33 33 - void xe_ggtt_map_bo(struct xe_ggtt *ggtt, struct xe_ggtt_node *node, 34 34 - struct xe_bo *bo, u16 pat_index); 35 38 void xe_ggtt_map_bo_unlocked(struct xe_ggtt *ggtt, struct xe_bo *bo); 36 39 int xe_ggtt_insert_bo(struct xe_ggtt *ggtt, struct xe_bo *bo, struct drm_exec *exec); 37 40 int xe_ggtt_insert_bo_at(struct xe_ggtt *ggtt, struct xe_bo *bo, ··· 60 57 61 58 u64 xe_ggtt_encode_pte_flags(struct xe_ggtt *ggtt, struct xe_bo *bo, u16 pat_index); 62 59 u64 xe_ggtt_read_pte(struct xe_ggtt *ggtt, u64 offset); 60 60 + 61 61 + u64 xe_ggtt_node_addr(const struct xe_ggtt_node *node); 62 62 + u64 xe_ggtt_node_size(const struct xe_ggtt_node *node); 63 63 64 64 #endif

+12 -20

drivers/gpu/drm/xe/xe_ggtt_types.h

reviewed

··· 11 11 #include "xe_pt_types.h" 12 12 13 13 struct xe_bo; 14 14 + struct xe_ggtt_node; 14 15 struct xe_gt; 15 16 16 17 /** ··· 23 22 struct xe_ggtt { 24 23 /** @tile: Back pointer to tile where this GGTT belongs */ 25 24 struct xe_tile *tile; 26 26 - /** @size: Total size of this GGTT */ 25 25 + /** @start: Start offset of GGTT */ 26 26 + u64 start; 27 27 + /** @size: Total usable size of this GGTT */ 27 28 u64 size; 28 29 29 30 #define XE_GGTT_FLAGS_64K BIT(0) ··· 54 51 struct workqueue_struct *wq; 55 52 }; 56 53 57 57 - /** 58 58 - * struct xe_ggtt_node - A node in GGTT. 59 59 - * 60 60 - * This struct needs to be initialized (only-once) with xe_ggtt_node_init() before any node 61 61 - * insertion, reservation, or 'ballooning'. 62 62 - * It will, then, be finalized by either xe_ggtt_node_remove() or xe_ggtt_node_deballoon(). 63 63 - */ 64 64 - struct xe_ggtt_node { 65 65 - /** @ggtt: Back pointer to xe_ggtt where this region will be inserted at */ 66 66 - struct xe_ggtt *ggtt; 67 67 - /** @base: A drm_mm_node */ 68 68 - struct drm_mm_node base; 69 69 - /** @delayed_removal_work: The work struct for the delayed removal */ 70 70 - struct work_struct delayed_removal_work; 71 71 - /** @invalidate_on_remove: If it needs invalidation upon removal */ 72 72 - bool invalidate_on_remove; 73 73 - }; 74 74 - 54 54 + typedef void (*xe_ggtt_set_pte_fn)(struct xe_ggtt *ggtt, u64 addr, u64 pte); 55 55 + typedef void (*xe_ggtt_transform_cb)(struct xe_ggtt *ggtt, 56 56 + struct xe_ggtt_node *node, 57 57 + u64 pte_flags, 58 58 + xe_ggtt_set_pte_fn set_pte, void *arg); 75 59 /** 76 60 * struct xe_ggtt_pt_ops - GGTT Page table operations 77 61 * Which can vary from platform to platform. ··· 66 76 struct xe_ggtt_pt_ops { 67 77 /** @pte_encode_flags: Encode PTE flags for a given BO */ 68 78 u64 (*pte_encode_flags)(struct xe_bo *bo, u16 pat_index); 79 79 + 69 80 /** @ggtt_set_pte: Directly write into GGTT's PTE */ 70 70 - void (*ggtt_set_pte)(struct xe_ggtt *ggtt, u64 addr, u64 pte); 81 81 + xe_ggtt_set_pte_fn ggtt_set_pte; 82 82 + 71 83 /** @ggtt_get_pte: Directly read from GGTT's PTE */ 72 84 u64 (*ggtt_get_pte)(struct xe_ggtt *ggtt, u64 addr); 73 85 };

+5 -4

drivers/gpu/drm/xe/xe_gpu_scheduler.c

reviewed

··· 7 7 8 8 static void xe_sched_process_msg_queue(struct xe_gpu_scheduler *sched) 9 9 { 10 10 - if (!READ_ONCE(sched->base.pause_submit)) 10 10 + if (!drm_sched_is_stopped(&sched->base)) 11 11 queue_work(sched->base.submit_wq, &sched->work_process_msg); 12 12 } 13 13 ··· 43 43 container_of(w, struct xe_gpu_scheduler, work_process_msg); 44 44 struct xe_sched_msg *msg; 45 45 46 46 - if (READ_ONCE(sched->base.pause_submit)) 46 46 + if (drm_sched_is_stopped(&sched->base)) 47 47 return; 48 48 49 49 msg = xe_sched_get_msg(sched); ··· 77 77 }; 78 78 79 79 sched->ops = xe_ops; 80 80 + spin_lock_init(&sched->msg_lock); 80 81 INIT_LIST_HEAD(&sched->msgs); 81 82 INIT_WORK(&sched->work_process_msg, xe_sched_process_msg_work); 82 83 ··· 118 117 void xe_sched_add_msg_locked(struct xe_gpu_scheduler *sched, 119 118 struct xe_sched_msg *msg) 120 119 { 121 121 - lockdep_assert_held(&sched->base.job_list_lock); 120 120 + lockdep_assert_held(&sched->msg_lock); 122 121 123 122 list_add_tail(&msg->link, &sched->msgs); 124 123 xe_sched_process_msg_queue(sched); ··· 132 131 void xe_sched_add_msg_head(struct xe_gpu_scheduler *sched, 133 132 struct xe_sched_msg *msg) 134 133 { 135 135 - lockdep_assert_held(&sched->base.job_list_lock); 134 134 + lockdep_assert_held(&sched->msg_lock); 136 135 137 136 list_add(&msg->link, &sched->msgs); 138 137 xe_sched_process_msg_queue(sched);

+9 -28

drivers/gpu/drm/xe/xe_gpu_scheduler.h

reviewed

··· 33 33 34 34 static inline void xe_sched_msg_lock(struct xe_gpu_scheduler *sched) 35 35 { 36 36 - spin_lock(&sched->base.job_list_lock); 36 36 + spin_lock(&sched->msg_lock); 37 37 } 38 38 39 39 static inline void xe_sched_msg_unlock(struct xe_gpu_scheduler *sched) 40 40 { 41 41 - spin_unlock(&sched->base.job_list_lock); 41 41 + spin_unlock(&sched->msg_lock); 42 42 } 43 43 44 44 static inline void xe_sched_stop(struct xe_gpu_scheduler *sched) ··· 56 56 struct drm_sched_job *s_job; 57 57 bool restore_replay = false; 58 58 59 59 - list_for_each_entry(s_job, &sched->base.pending_list, list) { 60 60 - struct drm_sched_fence *s_fence = s_job->s_fence; 61 61 - struct dma_fence *hw_fence = s_fence->parent; 62 62 - 59 59 + drm_sched_for_each_pending_job(s_job, &sched->base, NULL) { 63 60 restore_replay |= to_xe_sched_job(s_job)->restore_replay; 64 64 - if (restore_replay || (hw_fence && !dma_fence_is_signaled(hw_fence))) 61 61 + if (restore_replay || !drm_sched_job_is_signaled(s_job)) 65 62 sched->base.ops->run_job(s_job); 66 63 } 67 64 } ··· 67 70 xe_sched_invalidate_job(struct xe_sched_job *job, int threshold) 68 71 { 69 72 return drm_sched_invalidate_job(&job->drm, threshold); 70 70 - } 71 71 - 72 72 - static inline void xe_sched_add_pending_job(struct xe_gpu_scheduler *sched, 73 73 - struct xe_sched_job *job) 74 74 - { 75 75 - spin_lock(&sched->base.job_list_lock); 76 76 - list_add(&job->drm.list, &sched->base.pending_list); 77 77 - spin_unlock(&sched->base.job_list_lock); 78 73 } 79 74 80 75 /** ··· 78 89 static inline 79 90 struct xe_sched_job *xe_sched_first_pending_job(struct xe_gpu_scheduler *sched) 80 91 { 81 81 - struct xe_sched_job *job, *r_job = NULL; 92 92 + struct drm_sched_job *job; 82 93 83 83 - spin_lock(&sched->base.job_list_lock); 84 84 - list_for_each_entry(job, &sched->base.pending_list, drm.list) { 85 85 - struct drm_sched_fence *s_fence = job->drm.s_fence; 86 86 - struct dma_fence *hw_fence = s_fence->parent; 94 94 + drm_sched_for_each_pending_job(job, &sched->base, NULL) 95 95 + if (!drm_sched_job_is_signaled(job)) 96 96 + return to_xe_sched_job(job); 87 97 88 88 - if (hw_fence && !dma_fence_is_signaled(hw_fence)) { 89 89 - r_job = job; 90 90 - break; 91 91 - } 92 92 - } 93 93 - spin_unlock(&sched->base.job_list_lock); 94 94 - 95 95 - return r_job; 98 98 + return NULL; 96 99 } 97 100 98 101 static inline int

+2

drivers/gpu/drm/xe/xe_gpu_scheduler_types.h

reviewed

··· 47 47 const struct xe_sched_backend_ops *ops; 48 48 /** @msgs: list of messages to be processed in @work_process_msg */ 49 49 struct list_head msgs; 50 50 + /** @msg_lock: Message lock */ 51 51 + spinlock_t msg_lock; 50 52 /** @work_process_msg: processes messages */ 51 53 struct work_struct work_process_msg; 52 54 };

+7 -6

drivers/gpu/drm/xe/xe_gsc.c

reviewed

··· 414 414 } 415 415 416 416 /* 417 417 - * Some platforms can have GuC but not GSC. That would cause 418 418 - * xe_uc_fw_init(gsc) to return a "not supported" failure code and abort 419 419 - * all firmware loading. So check for GSC being enabled before 420 420 - * propagating the failure back up. That way the higher level will keep 421 421 - * going and load GuC as appropriate. 417 417 + * Starting from BMG the GSC is no longer needed for MC6 entry, so the 418 418 + * only missing features if the FW is lacking would be the content 419 419 + * protection ones. This is acceptable, so we allow the driver load to 420 420 + * continue if the GSC FW is missing. 422 421 */ 423 422 ret = xe_uc_fw_init(&gsc->fw); 424 423 if (!xe_uc_fw_is_enabled(&gsc->fw)) 424 424 + return 0; 425 425 + else if (gt_to_xe(gt)->info.platform >= XE_BATTLEMAGE && !xe_uc_fw_is_available(&gsc->fw)) 425 426 return 0; 426 427 else if (ret) 427 428 goto out; ··· 615 614 616 615 drm_printf(p, "\tfound security version %u\n", gsc->security_version); 617 616 618 618 - if (!xe_uc_fw_is_enabled(&gsc->fw)) 617 617 + if (!xe_uc_fw_is_available(&gsc->fw)) 619 618 return; 620 619 621 620 CLASS(xe_force_wake, fw_ref)(gt_to_fw(gt), XE_FW_GSC);

+2 -3

drivers/gpu/drm/xe/xe_gsc_debugfs.c

reviewed

··· 7 7 8 8 #include <drm/drm_debugfs.h> 9 9 #include <drm/drm_managed.h> 10 10 + #include <drm/drm_print.h> 10 11 11 11 - #include "xe_device.h" 12 12 - #include "xe_gt.h" 12 12 + #include "xe_gt_types.h" 13 13 #include "xe_gsc.h" 14 14 - #include "xe_macros.h" 15 14 #include "xe_pm.h" 16 15 17 16 static struct xe_gt *

+1 -1

drivers/gpu/drm/xe/xe_gsc_proxy.c

reviewed

··· 18 18 #include "xe_force_wake.h" 19 19 #include "xe_gsc.h" 20 20 #include "xe_gsc_submit.h" 21 21 - #include "xe_gt.h" 22 21 #include "xe_gt_printk.h" 22 22 + #include "xe_gt_types.h" 23 23 #include "xe_map.h" 24 24 #include "xe_mmio.h" 25 25 #include "xe_pm.h"

-2

drivers/gpu/drm/xe/xe_gsc_submit.c

reviewed

··· 11 11 #include "xe_assert.h" 12 12 #include "xe_bb.h" 13 13 #include "xe_exec_queue.h" 14 14 - #include "xe_gt_printk.h" 15 14 #include "xe_gt_types.h" 16 15 #include "xe_map.h" 17 16 #include "xe_sched_job.h" 18 17 #include "instructions/xe_gsc_commands.h" 19 19 - #include "regs/xe_gsc_regs.h" 20 18 21 19 #define GSC_HDR_SIZE (sizeof(struct intel_gsc_mtl_header)) /* shorthand define */ 22 20

+32 -2

drivers/gpu/drm/xe/xe_gt.c

reviewed

··· 13 13 #include <generated/xe_wa_oob.h> 14 14 15 15 #include "instructions/xe_alu_commands.h" 16 16 - #include "instructions/xe_gfxpipe_commands.h" 17 16 #include "instructions/xe_mi_commands.h" 18 17 #include "regs/xe_engine_regs.h" 19 18 #include "regs/xe_gt_regs.h" 20 19 #include "xe_assert.h" 21 20 #include "xe_bb.h" 22 22 - #include "xe_bo.h" 23 21 #include "xe_device.h" 24 22 #include "xe_eu_stall.h" 25 23 #include "xe_exec_queue.h" ··· 136 138 reg = xe_gt_mcr_unicast_read_any(gt, XE2_GAMREQSTRM_CTRL); 137 139 reg &= ~CG_DIS_CNTLBUS; 138 140 xe_gt_mcr_multicast_write(gt, XE2_GAMREQSTRM_CTRL, reg); 141 141 + } 142 142 + 143 143 + static void xe_gt_enable_comp_1wcoh(struct xe_gt *gt) 144 144 + { 145 145 + struct xe_device *xe = gt_to_xe(gt); 146 146 + unsigned int fw_ref; 147 147 + u32 reg; 148 148 + 149 149 + if (IS_SRIOV_VF(xe)) 150 150 + return; 151 151 + 152 152 + if (GRAPHICS_VER(xe) >= 30 && xe->info.has_flat_ccs) { 153 153 + fw_ref = xe_force_wake_get(gt_to_fw(gt), XE_FW_GT); 154 154 + if (!fw_ref) 155 155 + return; 156 156 + 157 157 + reg = xe_gt_mcr_unicast_read_any(gt, XE2_GAMREQSTRM_CTRL); 158 158 + reg |= EN_CMP_1WCOH; 159 159 + xe_gt_mcr_multicast_write(gt, XE2_GAMREQSTRM_CTRL, reg); 160 160 + 161 161 + if (xe_gt_is_media_type(gt)) { 162 162 + xe_mmio_rmw32(&gt->mmio, XE2_GAMWALK_CTRL_MEDIA, 0, EN_CMP_1WCOH_GW); 163 163 + } else { 164 164 + reg = xe_gt_mcr_unicast_read_any(gt, XE2_GAMWALK_CTRL_3D); 165 165 + reg |= EN_CMP_1WCOH_GW; 166 166 + xe_gt_mcr_multicast_write(gt, XE2_GAMWALK_CTRL_3D, reg); 167 167 + } 168 168 + 169 169 + xe_force_wake_put(gt_to_fw(gt), fw_ref); 170 170 + } 139 171 } 140 172 141 173 static void gt_reset_worker(struct work_struct *w); ··· 494 466 xe_gt_topology_init(gt); 495 467 xe_gt_mcr_init(gt); 496 468 xe_gt_enable_host_l2_vram(gt); 469 469 + xe_gt_enable_comp_1wcoh(gt); 497 470 498 471 if (xe_gt_is_main_type(gt)) { 499 472 err = xe_ggtt_init(gt_to_tile(gt)->mem.ggtt); ··· 774 745 xe_pat_init(gt); 775 746 776 747 xe_gt_enable_host_l2_vram(gt); 748 748 + xe_gt_enable_comp_1wcoh(gt); 777 749 778 750 xe_gt_mcr_set_implicit_defaults(gt); 779 751 xe_reg_sr_apply_mmio(&gt->reg_sr, gt);

+1 -5

drivers/gpu/drm/xe/xe_gt_clock.c

reviewed

··· 8 8 #include "xe_gt_clock.h" 9 9 10 10 #include "regs/xe_gt_regs.h" 11 11 - #include "regs/xe_regs.h" 12 12 - #include "xe_assert.h" 13 13 - #include "xe_device.h" 14 14 - #include "xe_gt.h" 11 11 + #include "xe_gt_types.h" 15 12 #include "xe_gt_printk.h" 16 16 - #include "xe_macros.h" 17 13 #include "xe_mmio.h" 18 14 19 15 #define f19_2_mhz 19200000

-1

drivers/gpu/drm/xe/xe_gt_debugfs.c

reviewed

··· 22 22 #include "xe_guc_hwconfig.h" 23 23 #include "xe_hw_engine.h" 24 24 #include "xe_lrc.h" 25 25 - #include "xe_macros.h" 26 25 #include "xe_mocs.h" 27 26 #include "xe_pat.h" 28 27 #include "xe_pm.h"

-1

drivers/gpu/drm/xe/xe_gt_idle.c

reviewed

··· 13 13 #include "xe_gt_sysfs.h" 14 14 #include "xe_guc_pc.h" 15 15 #include "regs/xe_gt_regs.h" 16 16 - #include "xe_macros.h" 17 16 #include "xe_mmio.h" 18 17 #include "xe_pm.h" 19 18 #include "xe_sriov.h"

-1

drivers/gpu/drm/xe/xe_gt_mcr.c

reviewed

··· 7 7 8 8 #include "regs/xe_gt_regs.h" 9 9 #include "xe_assert.h" 10 10 - #include "xe_gt.h" 11 10 #include "xe_gt_printk.h" 12 11 #include "xe_gt_topology.h" 13 12 #include "xe_gt_types.h"

+10 -11

drivers/gpu/drm/xe/xe_gt_sriov_pf_config.c

reviewed

··· 24 24 #include "xe_guc_buf.h" 25 25 #include "xe_guc_ct.h" 26 26 #include "xe_guc_db_mgr.h" 27 27 - #include "xe_guc_fwif.h" 28 27 #include "xe_guc_id_mgr.h" 29 28 #include "xe_guc_klv_helpers.h" 30 29 #include "xe_guc_klv_thresholds_set.h" 31 30 #include "xe_guc_submit.h" 32 31 #include "xe_lmtt.h" 33 33 - #include "xe_map.h" 34 32 #include "xe_migrate.h" 35 33 #include "xe_sriov.h" 36 34 #include "xe_ttm_vram_mgr.h" ··· 282 284 if (!xe_ggtt_node_allocated(node)) 283 285 return 0; 284 286 285 285 - return encode_ggtt(cfg, node->base.start, node->base.size, details); 287 287 + return encode_ggtt(cfg, xe_ggtt_node_addr(node), xe_ggtt_node_size(node), details); 286 288 } 287 289 288 290 static u32 encode_config_sched(struct xe_gt *gt, u32 *cfg, u32 n, ··· 391 393 xe_gt_assert(gt, num_dwords <= max_cfg_dwords); 392 394 393 395 if (vfid == PFID) { 394 394 - u64 ggtt_start = xe_wopcm_size(gt_to_xe(gt)); 395 395 - u64 ggtt_size = gt_to_tile(gt)->mem.ggtt->size - ggtt_start; 396 396 + u64 ggtt_start = xe_ggtt_start(gt_to_tile(gt)->mem.ggtt); 397 397 + u64 ggtt_size = xe_ggtt_size(gt_to_tile(gt)->mem.ggtt); 396 398 397 399 /* plain PF config data will never include a real GGTT region */ 398 400 xe_gt_assert(gt, !encode_config_ggtt(cfg + num_dwords, config, true)); ··· 543 545 544 546 xe_ggtt_assign(node, vfid); 545 547 xe_gt_sriov_dbg_verbose(gt, "VF%u assigned GGTT %llx-%llx\n", 546 546 - vfid, node->base.start, node->base.start + node->base.size - 1); 548 548 + vfid, xe_ggtt_node_addr(node), xe_ggtt_node_addr(node) + size - 1); 547 549 548 548 - err = pf_distribute_config_ggtt(gt->tile, vfid, node->base.start, node->base.size); 550 550 + err = pf_distribute_config_ggtt(gt->tile, vfid, xe_ggtt_node_addr(node), size); 549 551 if (unlikely(err)) 550 552 goto err; 551 553 ··· 562 564 struct xe_ggtt_node *node = config->ggtt_region; 563 565 564 566 xe_gt_assert(gt, xe_gt_is_main_type(gt)); 565 565 - return xe_ggtt_node_allocated(node) ? node->base.size : 0; 567 567 + return xe_ggtt_node_allocated(node) ? xe_ggtt_node_size(node) : 0; 566 568 } 567 569 568 570 /** ··· 3038 3040 if (!xe_ggtt_node_allocated(config->ggtt_region)) 3039 3041 continue; 3040 3042 3041 3041 - string_get_size(config->ggtt_region->base.size, 1, STRING_UNITS_2, 3043 3043 + string_get_size(xe_ggtt_node_size(config->ggtt_region), 1, STRING_UNITS_2, 3042 3044 buf, sizeof(buf)); 3043 3045 drm_printf(p, "VF%u:\t%#0llx-%#llx\t(%s)\n", 3044 3044 - n, config->ggtt_region->base.start, 3045 3045 - config->ggtt_region->base.start + config->ggtt_region->base.size - 1, 3046 3046 + n, xe_ggtt_node_addr(config->ggtt_region), 3047 3047 + xe_ggtt_node_addr(config->ggtt_region) + 3048 3048 + xe_ggtt_node_size(config->ggtt_region) - 1, 3046 3049 buf); 3047 3050 } 3048 3051

-1

drivers/gpu/drm/xe/xe_gt_sriov_pf_debugfs.c

reviewed

··· 8 8 #include <drm/drm_print.h> 9 9 #include <drm/drm_debugfs.h> 10 10 11 11 - #include "xe_bo.h" 12 11 #include "xe_debugfs.h" 13 12 #include "xe_device.h" 14 13 #include "xe_gt.h"

+1 -1

drivers/gpu/drm/xe/xe_gt_sriov_pf_migration.c

reviewed

··· 14 14 #include "xe_gt_sriov_pf.h" 15 15 #include "xe_gt_sriov_pf_config.h" 16 16 #include "xe_gt_sriov_pf_control.h" 17 17 - #include "xe_gt_sriov_pf_helpers.h" 18 17 #include "xe_gt_sriov_pf_migration.h" 19 18 #include "xe_gt_sriov_printk.h" 20 19 #include "xe_guc.h" ··· 24 25 #include "xe_sriov.h" 25 26 #include "xe_sriov_packet.h" 26 27 #include "xe_sriov_packet_types.h" 28 28 + #include "xe_sriov_pf_helpers.h" 27 29 #include "xe_sriov_pf_migration.h" 28 30 29 31 #define XE_GT_SRIOV_PF_MIGRATION_RING_SIZE 5

-1

drivers/gpu/drm/xe/xe_gt_sriov_pf_policy.c

reviewed

··· 7 7 8 8 #include "abi/guc_actions_sriov_abi.h" 9 9 10 10 - #include "xe_bo.h" 11 10 #include "xe_gt.h" 12 11 #include "xe_gt_sriov_pf_helpers.h" 13 12 #include "xe_gt_sriov_pf_policy.h"

+2 -3

drivers/gpu/drm/xe/xe_gt_sriov_pf_service.c

reviewed

··· 5 5 6 6 #include <drm/drm_managed.h> 7 7 8 8 - #include "abi/guc_actions_sriov_abi.h" 9 8 #include "abi/guc_relay_actions_abi.h" 10 9 11 10 #include "regs/xe_gt_regs.h" 12 11 #include "regs/xe_guc_regs.h" 13 13 - #include "regs/xe_regs.h" 14 12 13 13 + #include "xe_assert.h" 15 14 #include "xe_mmio.h" 16 15 #include "xe_gt_sriov_printk.h" 17 17 - #include "xe_gt_sriov_pf_helpers.h" 18 16 #include "xe_gt_sriov_pf_service.h" 19 17 #include "xe_gt_sriov_pf_service_types.h" 20 18 #include "xe_guc_ct.h" 21 19 #include "xe_guc_hxg_helpers.h" 20 20 + #include "xe_sriov.h" 22 21 #include "xe_sriov_pf_service.h" 23 22 24 23 static const struct xe_reg tgl_runtime_regs[] = {

-1

drivers/gpu/drm/xe/xe_gt_sriov_vf.c

reviewed

··· 15 15 #include "abi/guc_klvs_abi.h" 16 16 #include "abi/guc_relay_actions_abi.h" 17 17 #include "regs/xe_gt_regs.h" 18 18 - #include "regs/xe_gtt_defs.h" 19 18 20 19 #include "xe_assert.h" 21 20 #include "xe_device.h"

+2 -2

drivers/gpu/drm/xe/xe_gt_sriov_vf_types.h

reviewed

··· 43 43 }; 44 44 45 45 /** 46 46 - * xe_gt_sriov_vf_migration - VF migration data. 46 46 + * struct xe_gt_sriov_vf_migration - VF migration data. 47 47 */ 48 48 struct xe_gt_sriov_vf_migration { 49 49 - /** @migration: VF migration recovery worker */ 49 49 + /** @worker: VF migration recovery worker */ 50 50 struct work_struct worker; 51 51 /** @lock: Protects recovery_queued, teardown */ 52 52 spinlock_t lock;

+6 -1

drivers/gpu/drm/xe/xe_gt_stats.c

reviewed

··· 7 7 8 8 #include <drm/drm_print.h> 9 9 10 10 - #include "xe_gt.h" 11 10 #include "xe_gt_stats.h" 11 11 + #include "xe_gt_types.h" 12 12 13 13 /** 14 14 * xe_gt_stats_incr - Increments the specified stats counter ··· 76 76 "hw_engine_group_suspend_lr_queue_us"), 77 77 DEF_STAT_STR(HW_ENGINE_GROUP_WAIT_DMA_QUEUE_US, 78 78 "hw_engine_group_wait_dma_queue_us"), 79 79 + DEF_STAT_STR(PRL_4K_ENTRY_COUNT, "prl_4k_entry_count"), 80 80 + DEF_STAT_STR(PRL_64K_ENTRY_COUNT, "prl_64k_entry_count"), 81 81 + DEF_STAT_STR(PRL_2M_ENTRY_COUNT, "prl_2m_entry_count"), 82 82 + DEF_STAT_STR(PRL_ISSUED_COUNT, "prl_issued_count"), 83 83 + DEF_STAT_STR(PRL_ABORTED_COUNT, "prl_aborted_count"), 79 84 }; 80 85 81 86 /**

+5

drivers/gpu/drm/xe/xe_gt_stats_types.h

reviewed

··· 49 49 XE_GT_STATS_ID_HW_ENGINE_GROUP_WAIT_DMA_QUEUE_COUNT, 50 50 XE_GT_STATS_ID_HW_ENGINE_GROUP_SUSPEND_LR_QUEUE_US, 51 51 XE_GT_STATS_ID_HW_ENGINE_GROUP_WAIT_DMA_QUEUE_US, 52 52 + XE_GT_STATS_ID_PRL_4K_ENTRY_COUNT, 53 53 + XE_GT_STATS_ID_PRL_64K_ENTRY_COUNT, 54 54 + XE_GT_STATS_ID_PRL_2M_ENTRY_COUNT, 55 55 + XE_GT_STATS_ID_PRL_ISSUED_COUNT, 56 56 + XE_GT_STATS_ID_PRL_ABORTED_COUNT, 52 57 /* must be the last entry */ 53 58 __XE_GT_STATS_NUM_IDS, 54 59 };

+1 -1

drivers/gpu/drm/xe/xe_gt_sysfs.c

reviewed

··· 10 10 11 11 #include <drm/drm_managed.h> 12 12 13 13 - #include "xe_gt.h" 13 13 + #include "xe_gt_types.h" 14 14 15 15 static void xe_gt_sysfs_kobj_release(struct kobject *kobj) 16 16 {

+1 -1

drivers/gpu/drm/xe/xe_gt_throttle.c

reviewed

··· 6 6 #include <drm/drm_managed.h> 7 7 8 8 #include <regs/xe_gt_regs.h> 9 9 - #include "xe_device.h" 9 9 + #include "xe_device_types.h" 10 10 #include "xe_gt.h" 11 11 #include "xe_gt_sysfs.h" 12 12 #include "xe_gt_throttle.h"

+9 -9

drivers/gpu/drm/xe/xe_guc_ads.c

reviewed

··· 28 28 #include "xe_lrc.h" 29 29 #include "xe_map.h" 30 30 #include "xe_mmio.h" 31 31 - #include "xe_platform_types.h" 32 32 - #include "xe_uc_fw.h" 33 31 #include "xe_wa.h" 34 32 35 33 /* Slack of a few additional entries per engine */ ··· 449 451 ads_blob_write(ads, policies.max_num_work_items, 450 452 GLOBAL_POLICY_MAX_NUM_WI); 451 453 452 452 - if (xe->wedged.mode == 2) 454 454 + if (xe->wedged.mode == XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET) 453 455 global_flags |= GLOBAL_POLICY_DISABLE_ENGINE_RESET; 454 456 455 457 ads_blob_write(ads, policies.global_flags, global_flags); ··· 981 983 /** 982 984 * xe_guc_ads_scheduler_policy_toggle_reset - Toggle reset policy 983 985 * @ads: Additional data structures object 986 986 + * @enable_engine_reset: true to enable engine resets, false otherwise 984 987 * 985 985 - * This function update the GuC's engine reset policy based on wedged.mode. 988 988 + * This function update the GuC's engine reset policy. 986 989 * 987 990 * Return: 0 on success, and negative error code otherwise. 988 991 */ 989 989 - int xe_guc_ads_scheduler_policy_toggle_reset(struct xe_guc_ads *ads) 992 992 + int xe_guc_ads_scheduler_policy_toggle_reset(struct xe_guc_ads *ads, 993 993 + bool enable_engine_reset) 990 994 { 991 995 struct guc_policies *policies; 992 996 struct xe_guc *guc = ads_to_guc(ads); 993 993 - struct xe_device *xe = ads_to_xe(ads); 994 997 CLASS(xe_guc_buf, buf)(&guc->buf, sizeof(*policies)); 995 998 996 999 if (!xe_guc_buf_is_valid(buf)) ··· 1003 1004 policies->dpc_promote_time = ads_blob_read(ads, policies.dpc_promote_time); 1004 1005 policies->max_num_work_items = ads_blob_read(ads, policies.max_num_work_items); 1005 1006 policies->is_valid = 1; 1006 1006 - if (xe->wedged.mode == 2) 1007 1007 - policies->global_flags |= GLOBAL_POLICY_DISABLE_ENGINE_RESET; 1008 1008 - else 1007 1007 + 1008 1008 + if (enable_engine_reset) 1009 1009 policies->global_flags &= ~GLOBAL_POLICY_DISABLE_ENGINE_RESET; 1010 1010 + else 1011 1011 + policies->global_flags |= GLOBAL_POLICY_DISABLE_ENGINE_RESET; 1010 1012 1011 1013 return guc_ads_action_update_policies(ads, xe_guc_buf_flush(buf)); 1012 1014 }

+4 -1

drivers/gpu/drm/xe/xe_guc_ads.h

reviewed

··· 6 6 #ifndef _XE_GUC_ADS_H_ 7 7 #define _XE_GUC_ADS_H_ 8 8 9 9 + #include <linux/types.h> 10 10 + 9 11 struct xe_guc_ads; 10 12 11 13 int xe_guc_ads_init(struct xe_guc_ads *ads); ··· 15 13 void xe_guc_ads_populate(struct xe_guc_ads *ads); 16 14 void xe_guc_ads_populate_minimal(struct xe_guc_ads *ads); 17 15 void xe_guc_ads_populate_post_load(struct xe_guc_ads *ads); 18 18 - int xe_guc_ads_scheduler_policy_toggle_reset(struct xe_guc_ads *ads); 16 16 + int xe_guc_ads_scheduler_policy_toggle_reset(struct xe_guc_ads *ads, 17 17 + bool enable_engine_reset); 19 18 20 19 #endif

-1

drivers/gpu/drm/xe/xe_guc_buf.c

reviewed

··· 6 6 #include <linux/cleanup.h> 7 7 #include <drm/drm_managed.h> 8 8 9 9 - #include "xe_assert.h" 10 9 #include "xe_bo.h" 11 10 #include "xe_gt_printk.h" 12 11 #include "xe_guc.h"

+9 -5

drivers/gpu/drm/xe/xe_guc_capture.c

reviewed

··· 13 13 #include "abi/guc_log_abi.h" 14 14 #include "regs/xe_engine_regs.h" 15 15 #include "regs/xe_gt_regs.h" 16 16 - #include "regs/xe_guc_regs.h" 17 17 - #include "regs/xe_regs.h" 18 16 19 19 - #include "xe_bo.h" 17 17 + #include "xe_bo_types.h" 20 18 #include "xe_device.h" 21 19 #include "xe_exec_queue_types.h" 22 20 #include "xe_gt.h" 23 21 #include "xe_gt_mcr.h" 24 22 #include "xe_gt_printk.h" 25 23 #include "xe_guc.h" 26 26 - #include "xe_guc_ads.h" 27 24 #include "xe_guc_capture.h" 28 25 #include "xe_guc_capture_types.h" 29 26 #include "xe_guc_ct.h" ··· 1886 1889 return NULL; 1887 1890 1888 1891 xe = gt_to_xe(q->gt); 1889 1889 - if (xe->wedged.mode >= 2 || !xe_device_uc_enabled(xe) || IS_SRIOV_VF(xe)) 1892 1892 + 1893 1893 + if (xe->wedged.mode == XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET) 1894 1894 + return NULL; 1895 1895 + 1896 1896 + if (!xe_device_uc_enabled(xe)) 1897 1897 + return NULL; 1898 1898 + 1899 1899 + if (IS_SRIOV_VF(xe)) 1890 1900 return NULL; 1891 1901 1892 1902 ss = &xe->devcoredump.snapshot;

+2 -3

drivers/gpu/drm/xe/xe_guc_debugfs.c

reviewed

··· 8 8 #include <drm/drm_debugfs.h> 9 9 #include <drm/drm_managed.h> 10 10 11 11 - #include "xe_device.h" 12 12 - #include "xe_gt.h" 11 11 + #include "xe_device_types.h" 12 12 + #include "xe_gt_types.h" 13 13 #include "xe_guc.h" 14 14 #include "xe_guc_ct.h" 15 15 #include "xe_guc_log.h" 16 16 #include "xe_guc_pc.h" 17 17 - #include "xe_macros.h" 18 17 #include "xe_pm.h" 19 18 20 19 /*

-2

drivers/gpu/drm/xe/xe_guc_exec_queue_types.h

reviewed

··· 33 33 */ 34 34 #define MAX_STATIC_MSG_TYPE 3 35 35 struct xe_sched_msg static_msgs[MAX_STATIC_MSG_TYPE]; 36 36 - /** @lr_tdr: long running TDR worker */ 37 37 - struct work_struct lr_tdr; 38 36 /** @destroy_async: do final destroy async from this worker */ 39 37 struct work_struct destroy_async; 40 38 /** @resume_time: time of last resume */

+2 -2

drivers/gpu/drm/xe/xe_guc_hwconfig.c

reviewed

··· 10 10 11 11 #include "abi/guc_actions_abi.h" 12 12 #include "xe_bo.h" 13 13 - #include "xe_device.h" 14 14 - #include "xe_gt.h" 13 13 + #include "xe_device_types.h" 14 14 + #include "xe_gt_types.h" 15 15 #include "xe_guc.h" 16 16 #include "xe_map.h" 17 17

+1 -1

drivers/gpu/drm/xe/xe_guc_log.c

reviewed

··· 15 15 #include "xe_bo.h" 16 16 #include "xe_devcoredump.h" 17 17 #include "xe_force_wake.h" 18 18 - #include "xe_gt.h" 19 18 #include "xe_gt_printk.h" 19 19 + #include "xe_gt_types.h" 20 20 #include "xe_map.h" 21 21 #include "xe_mmio.h" 22 22 #include "xe_module.h"

+1 -2

drivers/gpu/drm/xe/xe_guc_relay.c

reviewed

··· 17 17 #include "abi/guc_relay_communication_abi.h" 18 18 19 19 #include "xe_assert.h" 20 20 - #include "xe_device.h" 21 21 - #include "xe_gt.h" 20 20 + #include "xe_device_types.h" 22 21 #include "xe_gt_sriov_printk.h" 23 22 #include "xe_gt_sriov_pf_service.h" 24 23 #include "xe_guc.h"

+61 -316

drivers/gpu/drm/xe/xe_guc_submit.c

reviewed

··· 17 17 #include "abi/guc_actions_abi.h" 18 18 #include "abi/guc_actions_slpc_abi.h" 19 19 #include "abi/guc_klvs_abi.h" 20 20 - #include "regs/xe_lrc_layout.h" 21 20 #include "xe_assert.h" 22 21 #include "xe_bo.h" 23 22 #include "xe_devcoredump.h" ··· 35 36 #include "xe_guc_klv_helpers.h" 36 37 #include "xe_guc_submit_types.h" 37 38 #include "xe_hw_engine.h" 38 38 - #include "xe_hw_fence.h" 39 39 #include "xe_lrc.h" 40 40 #include "xe_macros.h" 41 41 #include "xe_map.h" ··· 69 71 #define EXEC_QUEUE_STATE_KILLED (1 << 7) 70 72 #define EXEC_QUEUE_STATE_WEDGED (1 << 8) 71 73 #define EXEC_QUEUE_STATE_BANNED (1 << 9) 72 72 - #define EXEC_QUEUE_STATE_CHECK_TIMEOUT (1 << 10) 73 73 - #define EXEC_QUEUE_STATE_EXTRA_REF (1 << 11) 74 74 - #define EXEC_QUEUE_STATE_PENDING_RESUME (1 << 12) 75 75 - #define EXEC_QUEUE_STATE_PENDING_TDR_EXIT (1 << 13) 76 76 - #define EXEC_QUEUE_STATE_IDLE_SKIP_SUSPEND (1 << 14) 74 74 + #define EXEC_QUEUE_STATE_PENDING_RESUME (1 << 10) 75 75 + #define EXEC_QUEUE_STATE_IDLE_SKIP_SUSPEND (1 << 11) 77 76 78 77 static bool exec_queue_registered(struct xe_exec_queue *q) 79 78 { ··· 202 207 atomic_or(EXEC_QUEUE_STATE_WEDGED, &q->guc->state); 203 208 } 204 209 205 205 - static bool exec_queue_check_timeout(struct xe_exec_queue *q) 206 206 - { 207 207 - return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_CHECK_TIMEOUT; 208 208 - } 209 209 - 210 210 - static void set_exec_queue_check_timeout(struct xe_exec_queue *q) 211 211 - { 212 212 - atomic_or(EXEC_QUEUE_STATE_CHECK_TIMEOUT, &q->guc->state); 213 213 - } 214 214 - 215 215 - static void clear_exec_queue_check_timeout(struct xe_exec_queue *q) 216 216 - { 217 217 - atomic_and(~EXEC_QUEUE_STATE_CHECK_TIMEOUT, &q->guc->state); 218 218 - } 219 219 - 220 220 - static bool exec_queue_extra_ref(struct xe_exec_queue *q) 221 221 - { 222 222 - return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_EXTRA_REF; 223 223 - } 224 224 - 225 225 - static void set_exec_queue_extra_ref(struct xe_exec_queue *q) 226 226 - { 227 227 - atomic_or(EXEC_QUEUE_STATE_EXTRA_REF, &q->guc->state); 228 228 - } 229 229 - 230 230 - static void clear_exec_queue_extra_ref(struct xe_exec_queue *q) 231 231 - { 232 232 - atomic_and(~EXEC_QUEUE_STATE_EXTRA_REF, &q->guc->state); 233 233 - } 234 234 - 235 210 static bool exec_queue_pending_resume(struct xe_exec_queue *q) 236 211 { 237 212 return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_PENDING_RESUME; ··· 215 250 static void clear_exec_queue_pending_resume(struct xe_exec_queue *q) 216 251 { 217 252 atomic_and(~EXEC_QUEUE_STATE_PENDING_RESUME, &q->guc->state); 218 218 - } 219 219 - 220 220 - static bool exec_queue_pending_tdr_exit(struct xe_exec_queue *q) 221 221 - { 222 222 - return atomic_read(&q->guc->state) & EXEC_QUEUE_STATE_PENDING_TDR_EXIT; 223 223 - } 224 224 - 225 225 - static void set_exec_queue_pending_tdr_exit(struct xe_exec_queue *q) 226 226 - { 227 227 - atomic_or(EXEC_QUEUE_STATE_PENDING_TDR_EXIT, &q->guc->state); 228 228 - } 229 229 - 230 230 - static void clear_exec_queue_pending_tdr_exit(struct xe_exec_queue *q) 231 231 - { 232 232 - atomic_and(~EXEC_QUEUE_STATE_PENDING_TDR_EXIT, &q->guc->state); 233 253 } 234 254 235 255 static bool exec_queue_idle_skip_suspend(struct xe_exec_queue *q) ··· 553 603 /** to wakeup xe_wait_user_fence ioctl if exec queue is reset */ 554 604 wake_up_all(&xe->ufence_wq); 555 605 556 556 - if (xe_exec_queue_is_lr(q)) 557 557 - queue_work(guc_to_gt(guc)->ordered_wq, &q->guc->lr_tdr); 558 558 - else 559 559 - xe_sched_tdr_queue_imm(&q->guc->sched); 606 606 + xe_sched_tdr_queue_imm(&q->guc->sched); 560 607 } 561 608 562 609 static void xe_guc_exec_queue_group_trigger_cleanup(struct xe_exec_queue *q) ··· 586 639 WRITE_ONCE(group->banned, true); 587 640 588 641 set_exec_queue_reset(primary); 589 589 - if (!exec_queue_banned(primary) && !exec_queue_check_timeout(primary)) 642 642 + if (!exec_queue_banned(primary)) 590 643 xe_guc_exec_queue_trigger_cleanup(primary); 591 644 592 645 mutex_lock(&group->list_lock); 593 646 list_for_each_entry(eq, &group->list, multi_queue.link) { 594 647 set_exec_queue_reset(eq); 595 595 - if (!exec_queue_banned(eq) && !exec_queue_check_timeout(eq)) 648 648 + if (!exec_queue_banned(eq)) 596 649 xe_guc_exec_queue_trigger_cleanup(eq); 597 650 } 598 651 mutex_unlock(&group->list_lock); 599 652 } else { 600 653 set_exec_queue_reset(q); 601 601 - if (!exec_queue_banned(q) && !exec_queue_check_timeout(q)) 654 654 + if (!exec_queue_banned(q)) 602 655 xe_guc_exec_queue_trigger_cleanup(q); 603 656 } 604 657 } ··· 941 994 parallel_write(xe, map, wq_desc.wq_status, WQ_STATUS_ACTIVE); 942 995 } 943 996 944 944 - /* 945 945 - * We must keep a reference for LR engines if engine is registered with 946 946 - * the GuC as jobs signal immediately and can't destroy an engine if the 947 947 - * GuC has a reference to it. 948 948 - */ 949 949 - if (xe_exec_queue_is_lr(q)) 950 950 - xe_exec_queue_get(q); 951 951 - 952 997 set_exec_queue_registered(q); 953 998 trace_xe_exec_queue_register(q); 954 999 if (xe_exec_queue_is_multi_queue_primary(q)) ··· 1141 1202 struct xe_sched_job *job = to_xe_sched_job(drm_job); 1142 1203 struct xe_exec_queue *q = job->q; 1143 1204 struct xe_guc *guc = exec_queue_to_guc(q); 1144 1144 - bool lr = xe_exec_queue_is_lr(q), killed_or_banned_or_wedged = 1205 1205 + bool killed_or_banned_or_wedged = 1145 1206 exec_queue_killed_or_banned_or_wedged(q); 1146 1207 1147 1208 xe_gt_assert(guc_to_gt(guc), !(exec_queue_destroyed(q) || exec_queue_pending_disable(q)) || ··· 1171 1232 } 1172 1233 1173 1234 run_job_out: 1174 1174 - /* 1175 1175 - * We don't care about job-fence ordering in LR VMs because these fences 1176 1176 - * are never exported; they are used solely to keep jobs on the pending 1177 1177 - * list. Once a queue enters an error state, there's no need to track 1178 1178 - * them. 1179 1179 - */ 1180 1180 - if (killed_or_banned_or_wedged && lr) 1181 1181 - xe_sched_job_set_error(job, -ECANCELED); 1182 1235 1183 1236 return job->fence; 1184 1237 } ··· 1222 1291 xe_gt_warn(q->gt, "Pending enable/disable failed to respond\n"); 1223 1292 xe_sched_submission_start(sched); 1224 1293 xe_gt_reset_async(q->gt); 1225 1225 - if (!xe_exec_queue_is_lr(q)) 1226 1226 - xe_sched_tdr_queue_imm(sched); 1294 1294 + xe_sched_tdr_queue_imm(sched); 1227 1295 return; 1228 1296 } 1229 1297 ··· 1269 1339 err = devm_add_action_or_reset(guc_to_xe(guc)->drm.dev, 1270 1340 guc_submit_wedged_fini, guc); 1271 1341 if (err) { 1272 1272 - xe_gt_err(gt, "Failed to register clean-up on wedged.mode=2; " 1273 1273 - "Although device is wedged.\n"); 1342 1342 + xe_gt_err(gt, "Failed to register clean-up in wedged.mode=%s; " 1343 1343 + "Although device is wedged.\n", 1344 1344 + xe_wedged_mode_to_string(XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET)); 1274 1345 return; 1275 1346 } 1276 1347 ··· 1286 1355 { 1287 1356 struct xe_device *xe = guc_to_xe(guc); 1288 1357 1289 1289 - if (xe->wedged.mode != 2) 1358 1358 + if (xe->wedged.mode != XE_WEDGED_MODE_UPON_ANY_HANG_NO_RESET) 1290 1359 return false; 1291 1360 1292 1361 if (xe_device_wedged(xe)) ··· 1295 1364 xe_device_declare_wedged(xe); 1296 1365 1297 1366 return true; 1298 1298 - } 1299 1299 - 1300 1300 - static void xe_guc_exec_queue_lr_cleanup(struct work_struct *w) 1301 1301 - { 1302 1302 - struct xe_guc_exec_queue *ge = 1303 1303 - container_of(w, struct xe_guc_exec_queue, lr_tdr); 1304 1304 - struct xe_exec_queue *q = ge->q; 1305 1305 - struct xe_guc *guc = exec_queue_to_guc(q); 1306 1306 - struct xe_gpu_scheduler *sched = &ge->sched; 1307 1307 - struct xe_sched_job *job; 1308 1308 - bool wedged = false; 1309 1309 - 1310 1310 - xe_gt_assert(guc_to_gt(guc), xe_exec_queue_is_lr(q)); 1311 1311 - 1312 1312 - if (vf_recovery(guc)) 1313 1313 - return; 1314 1314 - 1315 1315 - trace_xe_exec_queue_lr_cleanup(q); 1316 1316 - 1317 1317 - if (!exec_queue_killed(q)) 1318 1318 - wedged = guc_submit_hint_wedged(exec_queue_to_guc(q)); 1319 1319 - 1320 1320 - /* Kill the run_job / process_msg entry points */ 1321 1321 - xe_sched_submission_stop(sched); 1322 1322 - 1323 1323 - /* 1324 1324 - * Engine state now mostly stable, disable scheduling / deregister if 1325 1325 - * needed. This cleanup routine might be called multiple times, where 1326 1326 - * the actual async engine deregister drops the final engine ref. 1327 1327 - * Calling disable_scheduling_deregister will mark the engine as 1328 1328 - * destroyed and fire off the CT requests to disable scheduling / 1329 1329 - * deregister, which we only want to do once. We also don't want to mark 1330 1330 - * the engine as pending_disable again as this may race with the 1331 1331 - * xe_guc_deregister_done_handler() which treats it as an unexpected 1332 1332 - * state. 1333 1333 - */ 1334 1334 - if (!wedged && exec_queue_registered(q) && !exec_queue_destroyed(q)) { 1335 1335 - struct xe_guc *guc = exec_queue_to_guc(q); 1336 1336 - int ret; 1337 1337 - 1338 1338 - set_exec_queue_banned(q); 1339 1339 - disable_scheduling_deregister(guc, q); 1340 1340 - 1341 1341 - /* 1342 1342 - * Must wait for scheduling to be disabled before signalling 1343 1343 - * any fences, if GT broken the GT reset code should signal us. 1344 1344 - */ 1345 1345 - ret = wait_event_timeout(guc->ct.wq, 1346 1346 - !exec_queue_pending_disable(q) || 1347 1347 - xe_guc_read_stopped(guc) || 1348 1348 - vf_recovery(guc), HZ * 5); 1349 1349 - if (vf_recovery(guc)) 1350 1350 - return; 1351 1351 - 1352 1352 - if (!ret) { 1353 1353 - xe_gt_warn(q->gt, "Schedule disable failed to respond, guc_id=%d\n", 1354 1354 - q->guc->id); 1355 1355 - xe_devcoredump(q, NULL, "Schedule disable failed to respond, guc_id=%d\n", 1356 1356 - q->guc->id); 1357 1357 - xe_sched_submission_start(sched); 1358 1358 - xe_gt_reset_async(q->gt); 1359 1359 - return; 1360 1360 - } 1361 1361 - } 1362 1362 - 1363 1363 - if (!exec_queue_killed(q) && !xe_lrc_ring_is_idle(q->lrc[0])) 1364 1364 - xe_devcoredump(q, NULL, "LR job cleanup, guc_id=%d", q->guc->id); 1365 1365 - 1366 1366 - xe_hw_fence_irq_stop(q->fence_irq); 1367 1367 - 1368 1368 - xe_sched_submission_start(sched); 1369 1369 - 1370 1370 - spin_lock(&sched->base.job_list_lock); 1371 1371 - list_for_each_entry(job, &sched->base.pending_list, drm.list) 1372 1372 - xe_sched_job_set_error(job, -ECANCELED); 1373 1373 - spin_unlock(&sched->base.job_list_lock); 1374 1374 - 1375 1375 - xe_hw_fence_irq_start(q->fence_irq); 1376 1367 } 1377 1368 1378 1369 #define ADJUST_FIVE_PERCENT(__t) mul_u64_u32_div(__t, 105, 100) ··· 1315 1462 return xe_sched_invalidate_job(job, 2); 1316 1463 } 1317 1464 1318 1318 - ctx_timestamp = lower_32_bits(xe_lrc_ctx_timestamp(q->lrc[0])); 1465 1465 + ctx_timestamp = lower_32_bits(xe_lrc_timestamp(q->lrc[0])); 1466 1466 + if (ctx_timestamp == job->sample_timestamp) { 1467 1467 + xe_gt_warn(gt, "Check job timeout: seqno=%u, lrc_seqno=%u, guc_id=%d, timestamp stuck", 1468 1468 + xe_sched_job_seqno(job), xe_sched_job_lrc_seqno(job), 1469 1469 + q->guc->id); 1470 1470 + 1471 1471 + return xe_sched_invalidate_job(job, 0); 1472 1472 + } 1473 1473 + 1474 1474 + job->sample_timestamp = ctx_timestamp; 1319 1475 ctx_job_timestamp = xe_lrc_ctx_job_timestamp(q->lrc[0]); 1320 1476 1321 1477 /* ··· 1378 1516 xe_gt_warn(guc_to_gt(guc), "Schedule enable failed to respond"); 1379 1517 set_exec_queue_banned(q); 1380 1518 xe_gt_reset_async(q->gt); 1381 1381 - if (!xe_exec_queue_is_lr(q)) 1382 1382 - xe_sched_tdr_queue_imm(&q->guc->sched); 1519 1519 + xe_sched_tdr_queue_imm(&q->guc->sched); 1383 1520 } 1384 1521 } 1385 1522 ··· 1404 1543 G2H_LEN_DW_SCHED_CONTEXT_MODE_SET, 1); 1405 1544 } 1406 1545 1407 1407 - static void __deregister_exec_queue(struct xe_guc *guc, struct xe_exec_queue *q) 1408 1408 - { 1409 1409 - u32 action[] = { 1410 1410 - XE_GUC_ACTION_DEREGISTER_CONTEXT, 1411 1411 - q->guc->id, 1412 1412 - }; 1413 1413 - 1414 1414 - xe_gt_assert(guc_to_gt(guc), !exec_queue_destroyed(q)); 1415 1415 - xe_gt_assert(guc_to_gt(guc), exec_queue_registered(q)); 1416 1416 - xe_gt_assert(guc_to_gt(guc), !exec_queue_pending_enable(q)); 1417 1417 - xe_gt_assert(guc_to_gt(guc), !exec_queue_pending_disable(q)); 1418 1418 - 1419 1419 - set_exec_queue_destroyed(q); 1420 1420 - trace_xe_exec_queue_deregister(q); 1421 1421 - 1422 1422 - if (xe_exec_queue_is_multi_queue_secondary(q)) 1423 1423 - handle_deregister_done(guc, q); 1424 1424 - else 1425 1425 - xe_guc_ct_send(&guc->ct, action, ARRAY_SIZE(action), 1426 1426 - G2H_LEN_DW_DEREGISTER_CONTEXT, 1); 1427 1427 - } 1428 1428 - 1429 1546 static enum drm_gpu_sched_stat 1430 1547 guc_exec_queue_timedout_job(struct drm_sched_job *drm_job) 1431 1548 { 1432 1549 struct xe_sched_job *job = to_xe_sched_job(drm_job); 1433 1433 - struct xe_sched_job *tmp_job; 1550 1550 + struct drm_sched_job *tmp_job; 1434 1551 struct xe_exec_queue *q = job->q; 1435 1552 struct xe_gpu_scheduler *sched = &q->guc->sched; 1436 1553 struct xe_guc *guc = exec_queue_to_guc(q); ··· 1416 1577 struct xe_device *xe = guc_to_xe(guc); 1417 1578 int err = -ETIME; 1418 1579 pid_t pid = -1; 1419 1419 - int i = 0; 1420 1580 bool wedged = false, skip_timeout_check; 1421 1581 1422 1422 - xe_gt_assert(guc_to_gt(guc), !xe_exec_queue_is_lr(q)); 1582 1582 + xe_gt_assert(guc_to_gt(guc), !exec_queue_destroyed(q)); 1423 1583 1424 1584 /* 1425 1585 * TDR has fired before free job worker. Common if exec queue ··· 1435 1597 1436 1598 /* Must check all state after stopping scheduler */ 1437 1599 skip_timeout_check = exec_queue_reset(q) || 1438 1438 - exec_queue_killed_or_banned_or_wedged(q) || 1439 1439 - exec_queue_destroyed(q); 1600 1600 + exec_queue_killed_or_banned_or_wedged(q); 1440 1601 1441 1602 /* Skip timeout check if multi-queue group is banned */ 1442 1603 if (xe_exec_queue_is_multi_queue(q) && 1443 1604 READ_ONCE(q->multi_queue.group->banned)) 1444 1605 skip_timeout_check = true; 1606 1606 + 1607 1607 + /* LR jobs can only get here if queue has been killed or hit an error */ 1608 1608 + if (xe_exec_queue_is_lr(q)) 1609 1609 + xe_gt_assert(guc_to_gt(guc), skip_timeout_check); 1445 1610 1446 1611 /* 1447 1612 * FIXME: In multi-queue scenario, the TDR must ensure that the whole ··· 1469 1628 } 1470 1629 1471 1630 /* 1472 1472 - * XXX: Sampling timeout doesn't work in wedged mode as we have to 1473 1473 - * modify scheduling state to read timestamp. We could read the 1474 1474 - * timestamp from a register to accumulate current running time but this 1475 1475 - * doesn't work for SRIOV. For now assuming timeouts in wedged mode are 1476 1476 - * genuine timeouts. 1631 1631 + * Check if job is actually timed out, if so restart job execution and TDR 1477 1632 */ 1633 1633 + if (!skip_timeout_check && !check_timeout(q, job)) 1634 1634 + goto rearm; 1635 1635 + 1478 1636 if (!exec_queue_killed(q)) 1479 1637 wedged = guc_submit_hint_wedged(exec_queue_to_guc(q)); 1480 1638 1481 1481 - /* Engine state now stable, disable scheduling to check timestamp */ 1482 1482 - if (!wedged && exec_queue_registered(q)) { 1639 1639 + set_exec_queue_banned(q); 1640 1640 + 1641 1641 + /* Kick job / queue off hardware */ 1642 1642 + if (!wedged && (exec_queue_enabled(q) || exec_queue_pending_disable(q))) { 1483 1643 int ret; 1484 1644 1485 1645 if (exec_queue_reset(q)) 1486 1646 err = -EIO; 1487 1647 1488 1488 - if (!exec_queue_destroyed(q)) { 1648 1648 + if (xe_uc_fw_is_running(&guc->fw)) { 1489 1649 /* 1490 1650 * Wait for any pending G2H to flush out before 1491 1651 * modifying state ··· 1501 1659 if (!ret || xe_guc_read_stopped(guc)) 1502 1660 goto trigger_reset; 1503 1661 1504 1504 - /* 1505 1505 - * Flag communicates to G2H handler that schedule 1506 1506 - * disable originated from a timeout check. The G2H then 1507 1507 - * avoid triggering cleanup or deregistering the exec 1508 1508 - * queue. 1509 1509 - */ 1510 1510 - set_exec_queue_check_timeout(q); 1511 1662 disable_scheduling(q, skip_timeout_check); 1512 1663 } 1513 1664 ··· 1514 1679 */ 1515 1680 smp_rmb(); 1516 1681 ret = wait_event_timeout(guc->ct.wq, 1682 1682 + !xe_uc_fw_is_running(&guc->fw) || 1517 1683 !exec_queue_pending_disable(q) || 1518 1684 xe_guc_read_stopped(guc) || 1519 1685 vf_recovery(guc), HZ * 5); ··· 1529 1693 xe_devcoredump(q, job, 1530 1694 "Schedule disable failed to respond, guc_id=%d, ret=%d, guc_read=%d", 1531 1695 q->guc->id, ret, xe_guc_read_stopped(guc)); 1532 1532 - set_exec_queue_extra_ref(q); 1533 1533 - xe_exec_queue_get(q); /* GT reset owns this */ 1534 1534 - set_exec_queue_banned(q); 1535 1696 xe_gt_reset_async(q->gt); 1536 1697 xe_sched_tdr_queue_imm(sched); 1537 1698 goto rearm; 1538 1699 } 1539 1539 - } 1540 1540 - 1541 1541 - /* 1542 1542 - * Check if job is actually timed out, if so restart job execution and TDR 1543 1543 - */ 1544 1544 - if (!wedged && !skip_timeout_check && !check_timeout(q, job) && 1545 1545 - !exec_queue_reset(q) && exec_queue_registered(q)) { 1546 1546 - clear_exec_queue_check_timeout(q); 1547 1547 - goto sched_enable; 1548 1700 } 1549 1701 1550 1702 if (q->vm && q->vm->xef) { ··· 1565 1741 if (!wedged && (q->flags & EXEC_QUEUE_FLAG_KERNEL || 1566 1742 (q->flags & EXEC_QUEUE_FLAG_VM && !exec_queue_killed(q)))) { 1567 1743 if (!xe_sched_invalidate_job(job, 2)) { 1568 1568 - clear_exec_queue_check_timeout(q); 1569 1744 xe_gt_reset_async(q->gt); 1570 1745 goto rearm; 1571 1746 } 1572 1747 } 1573 1748 1574 1574 - /* Finish cleaning up exec queue via deregister */ 1575 1575 - set_exec_queue_banned(q); 1576 1576 - if (!wedged && exec_queue_registered(q) && !exec_queue_destroyed(q)) { 1577 1577 - set_exec_queue_extra_ref(q); 1578 1578 - xe_exec_queue_get(q); 1579 1579 - __deregister_exec_queue(guc, q); 1580 1580 - } 1749 1749 + /* Mark all outstanding jobs as bad, thus completing them */ 1750 1750 + xe_sched_job_set_error(job, err); 1751 1751 + drm_sched_for_each_pending_job(tmp_job, &sched->base, NULL) 1752 1752 + xe_sched_job_set_error(to_xe_sched_job(tmp_job), -ECANCELED); 1581 1753 1582 1582 - /* Stop fence signaling */ 1583 1583 - xe_hw_fence_irq_stop(q->fence_irq); 1584 1584 - 1585 1585 - /* 1586 1586 - * Fence state now stable, stop / start scheduler which cleans up any 1587 1587 - * fences that are complete 1588 1588 - */ 1589 1589 - xe_sched_add_pending_job(sched, job); 1590 1754 xe_sched_submission_start(sched); 1591 1755 1592 1756 if (xe_exec_queue_is_multi_queue(q)) ··· 1582 1770 else 1583 1771 xe_guc_exec_queue_trigger_cleanup(q); 1584 1772 1585 1585 - /* Mark all outstanding jobs as bad, thus completing them */ 1586 1586 - spin_lock(&sched->base.job_list_lock); 1587 1587 - list_for_each_entry(tmp_job, &sched->base.pending_list, drm.list) 1588 1588 - xe_sched_job_set_error(tmp_job, !i++ ? err : -ECANCELED); 1589 1589 - spin_unlock(&sched->base.job_list_lock); 1773 1773 + /* 1774 1774 + * We want the job added back to the pending list so it gets freed; this 1775 1775 + * is what DRM_GPU_SCHED_STAT_NO_HANG does. 1776 1776 + */ 1777 1777 + return DRM_GPU_SCHED_STAT_NO_HANG; 1590 1778 1591 1591 - /* Start fence signaling */ 1592 1592 - xe_hw_fence_irq_start(q->fence_irq); 1593 1593 - 1594 1594 - return DRM_GPU_SCHED_STAT_RESET; 1595 1595 - 1596 1596 - sched_enable: 1597 1597 - set_exec_queue_pending_tdr_exit(q); 1598 1598 - enable_scheduling(q); 1599 1779 rearm: 1600 1780 /* 1601 1781 * XXX: Ideally want to adjust timeout based on current execution time ··· 1633 1829 mutex_unlock(&group->list_lock); 1634 1830 } 1635 1831 1636 1636 - if (xe_exec_queue_is_lr(q)) 1637 1637 - cancel_work_sync(&ge->lr_tdr); 1638 1832 /* Confirm no work left behind accessing device structures */ 1639 1833 cancel_delayed_work_sync(&ge->sched.base.work_tdr); 1640 1834 ··· 1936 2134 if (err) 1937 2135 goto err_sched; 1938 2136 1939 1939 - if (xe_exec_queue_is_lr(q)) 1940 1940 - INIT_WORK(&q->guc->lr_tdr, xe_guc_exec_queue_lr_cleanup); 1941 1941 - 1942 2137 mutex_lock(&guc->submission_state.lock); 1943 2138 1944 2139 err = alloc_guc_id(guc, q); ··· 2231 2432 2232 2433 /* Clean up lost G2H + reset engine state */ 2233 2434 if (exec_queue_registered(q)) { 2234 2234 - if (exec_queue_extra_ref(q) || xe_exec_queue_is_lr(q)) 2235 2235 - xe_exec_queue_put(q); 2236 2236 - else if (exec_queue_destroyed(q)) 2435 2435 + if (exec_queue_destroyed(q)) 2237 2436 __guc_exec_queue_destroy(guc, q); 2238 2437 } 2239 2438 if (q->guc->suspend_pending) { ··· 2261 2464 trace_xe_sched_job_ban(job); 2262 2465 ban = true; 2263 2466 } 2264 2264 - } else if (xe_exec_queue_is_lr(q) && 2265 2265 - !xe_lrc_ring_is_idle(q->lrc[0])) { 2266 2266 - ban = true; 2267 2467 } 2268 2468 2269 2469 if (ban) { ··· 2340 2546 q->guc->id); 2341 2547 } 2342 2548 2343 2343 - if (pending_enable && !pending_resume && 2344 2344 - !exec_queue_pending_tdr_exit(q)) { 2549 2549 + if (pending_enable && !pending_resume) { 2345 2550 clear_exec_queue_registered(q); 2346 2346 - if (xe_exec_queue_is_lr(q)) 2347 2347 - xe_exec_queue_put(q); 2348 2551 xe_gt_dbg(guc_to_gt(guc), "Replay REGISTER - guc_id=%d", 2349 2552 q->guc->id); 2350 2553 } ··· 2349 2558 if (pending_enable) { 2350 2559 clear_exec_queue_enabled(q); 2351 2560 clear_exec_queue_pending_resume(q); 2352 2352 - clear_exec_queue_pending_tdr_exit(q); 2353 2561 clear_exec_queue_pending_enable(q); 2354 2562 xe_gt_dbg(guc_to_gt(guc), "Replay ENABLE - guc_id=%d", 2355 2563 q->guc->id); ··· 2356 2566 2357 2567 if (exec_queue_destroyed(q) && exec_queue_registered(q)) { 2358 2568 clear_exec_queue_destroyed(q); 2359 2359 - if (exec_queue_extra_ref(q)) 2360 2360 - xe_exec_queue_put(q); 2361 2361 - else 2362 2362 - q->guc->needs_cleanup = true; 2363 2363 - clear_exec_queue_extra_ref(q); 2569 2569 + q->guc->needs_cleanup = true; 2364 2570 xe_gt_dbg(guc_to_gt(guc), "Replay CLEANUP - guc_id=%d", 2365 2571 q->guc->id); 2366 2572 } ··· 2374 2588 if (!pending_enable) 2375 2589 set_exec_queue_enabled(q); 2376 2590 clear_exec_queue_pending_disable(q); 2377 2377 - clear_exec_queue_check_timeout(q); 2378 2591 xe_gt_dbg(guc_to_gt(guc), "Replay DISABLE - guc_id=%d", 2379 2592 q->guc->id); 2380 2593 } ··· 2408 2623 2409 2624 /* Stop scheduling + flush any DRM scheduler operations */ 2410 2625 xe_sched_submission_stop(sched); 2411 2411 - if (xe_exec_queue_is_lr(q)) 2412 2412 - cancel_work_sync(&q->guc->lr_tdr); 2413 2413 - else 2414 2414 - cancel_delayed_work_sync(&sched->base.work_tdr); 2626 2626 + cancel_delayed_work_sync(&sched->base.work_tdr); 2415 2627 2416 2628 guc_exec_queue_revert_pending_state_change(guc, q); 2417 2629 ··· 2535 2753 struct xe_exec_queue *q) 2536 2754 { 2537 2755 struct xe_gpu_scheduler *sched = &q->guc->sched; 2538 2538 - struct xe_sched_job *job = NULL, *__job; 2756 2756 + struct xe_sched_job *job = NULL; 2757 2757 + struct drm_sched_job *s_job; 2539 2758 bool restore_replay = false; 2540 2759 2541 2541 - list_for_each_entry(__job, &sched->base.pending_list, drm.list) { 2542 2542 - job = __job; 2760 2760 + drm_sched_for_each_pending_job(s_job, &sched->base, NULL) { 2761 2761 + job = to_xe_sched_job(s_job); 2543 2762 restore_replay |= job->restore_replay; 2544 2763 if (restore_replay) { 2545 2764 xe_gt_dbg(guc_to_gt(guc), "Replay JOB - guc_id=%d, seqno=%d", ··· 2664 2881 * created after resfix done. 2665 2882 */ 2666 2883 if (q->guc->id != index || 2667 2667 - !READ_ONCE(q->guc->sched.base.pause_submit)) 2884 2884 + !drm_sched_is_stopped(&q->guc->sched.base)) 2668 2885 continue; 2669 2886 2670 2887 guc_exec_queue_unpause(guc, q); ··· 2750 2967 2751 2968 q->guc->resume_time = ktime_get(); 2752 2969 clear_exec_queue_pending_resume(q); 2753 2753 - clear_exec_queue_pending_tdr_exit(q); 2754 2970 clear_exec_queue_pending_enable(q); 2755 2971 smp_wmb(); 2756 2972 wake_up_all(&guc->ct.wq); 2757 2973 } else { 2758 2758 - bool check_timeout = exec_queue_check_timeout(q); 2759 2759 - 2760 2974 xe_gt_assert(guc_to_gt(guc), runnable_state == 0); 2761 2975 xe_gt_assert(guc_to_gt(guc), exec_queue_pending_disable(q)); 2762 2976 ··· 2761 2981 suspend_fence_signal(q); 2762 2982 clear_exec_queue_pending_disable(q); 2763 2983 } else { 2764 2764 - if (exec_queue_banned(q) || check_timeout) { 2984 2984 + if (exec_queue_banned(q)) { 2765 2985 smp_wmb(); 2766 2986 wake_up_all(&guc->ct.wq); 2767 2987 } 2768 2768 - if (!check_timeout && exec_queue_destroyed(q)) { 2988 2988 + if (exec_queue_destroyed(q)) { 2769 2989 /* 2770 2990 * Make sure to clear the pending_disable only 2771 2991 * after sampling the destroyed state. We want ··· 2828 3048 trace_xe_exec_queue_deregister_done(q); 2829 3049 2830 3050 clear_exec_queue_registered(q); 2831 2831 - 2832 2832 - if (exec_queue_extra_ref(q) || xe_exec_queue_is_lr(q)) 2833 2833 - xe_exec_queue_put(q); 2834 2834 - else 2835 2835 - __guc_exec_queue_destroy(guc, q); 3051 3051 + __guc_exec_queue_destroy(guc, q); 2836 3052 } 2837 3053 2838 3054 int xe_guc_deregister_done_handler(struct xe_guc *guc, u32 *msg, u32 len) ··· 3162 3386 snapshot->multi_queue.primary = xe_exec_queue_multi_queue_primary(q)->guc->id; 3163 3387 snapshot->multi_queue.pos = q->multi_queue.pos; 3164 3388 } 3165 3165 - spin_lock(&sched->base.job_list_lock); 3166 3166 - snapshot->pending_list_size = list_count_nodes(&sched->base.pending_list); 3167 3167 - snapshot->pending_list = kmalloc_array(snapshot->pending_list_size, 3168 3168 - sizeof(struct pending_list_snapshot), 3169 3169 - GFP_ATOMIC); 3170 3170 - 3171 3171 - if (snapshot->pending_list) { 3172 3172 - struct xe_sched_job *job_iter; 3173 3173 - 3174 3174 - i = 0; 3175 3175 - list_for_each_entry(job_iter, &sched->base.pending_list, drm.list) { 3176 3176 - snapshot->pending_list[i].seqno = 3177 3177 - xe_sched_job_seqno(job_iter); 3178 3178 - snapshot->pending_list[i].fence = 3179 3179 - dma_fence_is_signaled(job_iter->fence) ? 1 : 0; 3180 3180 - snapshot->pending_list[i].finished = 3181 3181 - dma_fence_is_signaled(&job_iter->drm.s_fence->finished) 3182 3182 - ? 1 : 0; 3183 3183 - i++; 3184 3184 - } 3185 3185 - } 3186 3186 - 3187 3187 - spin_unlock(&sched->base.job_list_lock); 3188 3389 3189 3390 return snapshot; 3190 3391 } ··· 3225 3472 drm_printf(p, "\tMulti queue primary GuC ID: %d\n", snapshot->multi_queue.primary); 3226 3473 drm_printf(p, "\tMulti queue position: %d\n", snapshot->multi_queue.pos); 3227 3474 } 3228 3228 - 3229 3229 - for (i = 0; snapshot->pending_list && i < snapshot->pending_list_size; 3230 3230 - i++) 3231 3231 - drm_printf(p, "\tJob: seqno=%d, fence=%d, finished=%d\n", 3232 3232 - snapshot->pending_list[i].seqno, 3233 3233 - snapshot->pending_list[i].fence, 3234 3234 - snapshot->pending_list[i].finished); 3235 3475 } 3236 3476 3237 3477 /** ··· 3247 3501 xe_lrc_snapshot_free(snapshot->lrc[i]); 3248 3502 kfree(snapshot->lrc); 3249 3503 } 3250 3250 - kfree(snapshot->pending_list); 3251 3504 kfree(snapshot); 3252 3505 } 3253 3506

-11

drivers/gpu/drm/xe/xe_guc_submit_types.h

reviewed

··· 61 61 u32 wq[WQ_SIZE / sizeof(u32)]; 62 62 }; 63 63 64 64 - struct pending_list_snapshot { 65 65 - u32 seqno; 66 66 - bool fence; 67 67 - bool finished; 68 68 - }; 69 69 - 70 64 /** 71 65 * struct xe_guc_submit_exec_queue_snapshot - Snapshot for devcoredump 72 66 */ ··· 141 147 /** @valid: The exec queue is part of a multi queue group */ 142 148 bool valid; 143 149 } multi_queue; 144 144 - 145 145 - /** @pending_list_size: Size of the pending list snapshot array */ 146 146 - int pending_list_size; 147 147 - /** @pending_list: snapshot of the pending list info */ 148 148 - struct pending_list_snapshot *pending_list; 149 150 }; 150 151 151 152 #endif

+3

drivers/gpu/drm/xe/xe_guc_tlb_inval.c

reviewed

··· 97 97 static int send_page_reclaim(struct xe_guc *guc, u32 seqno, 98 98 u64 gpu_addr) 99 99 { 100 100 + struct xe_gt *gt = guc_to_gt(guc); 100 101 u32 action[] = { 101 102 XE_GUC_ACTION_PAGE_RECLAMATION, 102 103 seqno, 103 104 lower_32_bits(gpu_addr), 104 105 upper_32_bits(gpu_addr), 105 106 }; 107 107 + 108 108 + xe_gt_stats_incr(gt, XE_GT_STATS_ID_PRL_ISSUED_COUNT, 1); 106 109 107 110 return xe_guc_ct_send(&guc->ct, action, ARRAY_SIZE(action), 108 111 G2H_LEN_DW_PAGE_RECLAMATION, 1);

-1

drivers/gpu/drm/xe/xe_heci_gsc.c

reviewed

··· 11 11 #include <drm/drm_print.h> 12 12 13 13 #include "xe_device_types.h" 14 14 - #include "xe_drv.h" 15 14 #include "xe_heci_gsc.h" 16 15 #include "regs/xe_gsc_regs.h" 17 16 #include "xe_platform_types.h"

-1

drivers/gpu/drm/xe/xe_huc.c

reviewed

··· 12 12 #include "abi/gsc_pxp_commands_abi.h" 13 13 #include "regs/xe_gsc_regs.h" 14 14 #include "regs/xe_guc_regs.h" 15 15 - #include "xe_assert.h" 16 15 #include "xe_bo.h" 17 16 #include "xe_device.h" 18 17 #include "xe_force_wake.h"

+2 -3

drivers/gpu/drm/xe/xe_huc_debugfs.c

reviewed

··· 7 7 8 8 #include <drm/drm_debugfs.h> 9 9 #include <drm/drm_managed.h> 10 10 + #include <drm/drm_print.h> 10 11 11 11 - #include "xe_device.h" 12 12 - #include "xe_gt.h" 12 12 + #include "xe_gt_types.h" 13 13 #include "xe_huc.h" 14 14 - #include "xe_macros.h" 15 14 #include "xe_pm.h" 16 15 17 16 static struct xe_gt *

-1

drivers/gpu/drm/xe/xe_hw_engine.c

reviewed

··· 33 33 #include "xe_hw_fence.h" 34 34 #include "xe_irq.h" 35 35 #include "xe_lrc.h" 36 36 - #include "xe_macros.h" 37 36 #include "xe_mmio.h" 38 37 #include "xe_reg_sr.h" 39 38 #include "xe_reg_whitelist.h"

+1 -1

drivers/gpu/drm/xe/xe_hw_engine_class_sysfs.c

reviewed

··· 7 7 #include <linux/kobject.h> 8 8 #include <linux/sysfs.h> 9 9 10 10 - #include "xe_device.h" 10 10 + #include "xe_device_types.h" 11 11 #include "xe_gt.h" 12 12 #include "xe_hw_engine_class_sysfs.h" 13 13 #include "xe_pm.h"

+1 -1

drivers/gpu/drm/xe/xe_hw_engine_group.c

reviewed

··· 6 6 #include <drm/drm_managed.h> 7 7 8 8 #include "xe_assert.h" 9 9 - #include "xe_device.h" 9 9 + #include "xe_device_types.h" 10 10 #include "xe_exec_queue.h" 11 11 #include "xe_gt.h" 12 12 #include "xe_gt_stats.h"

+1 -19

drivers/gpu/drm/xe/xe_hw_fence.c

reviewed

··· 8 8 #include <linux/device.h> 9 9 #include <linux/slab.h> 10 10 11 11 - #include "xe_bo.h" 12 12 - #include "xe_device.h" 13 13 - #include "xe_gt.h" 11 11 + #include "xe_device_types.h" 14 12 #include "xe_hw_engine.h" 15 13 #include "xe_macros.h" 16 14 #include "xe_map.h" ··· 103 105 104 106 void xe_hw_fence_irq_run(struct xe_hw_fence_irq *irq) 105 107 { 106 106 - irq_work_queue(&irq->work); 107 107 - } 108 108 - 109 109 - void xe_hw_fence_irq_stop(struct xe_hw_fence_irq *irq) 110 110 - { 111 111 - spin_lock_irq(&irq->lock); 112 112 - irq->enabled = false; 113 113 - spin_unlock_irq(&irq->lock); 114 114 - } 115 115 - 116 116 - void xe_hw_fence_irq_start(struct xe_hw_fence_irq *irq) 117 117 - { 118 118 - spin_lock_irq(&irq->lock); 119 119 - irq->enabled = true; 120 120 - spin_unlock_irq(&irq->lock); 121 121 - 122 108 irq_work_queue(&irq->work); 123 109 } 124 110

-2

drivers/gpu/drm/xe/xe_hw_fence.h

reviewed

··· 17 17 void xe_hw_fence_irq_init(struct xe_hw_fence_irq *irq); 18 18 void xe_hw_fence_irq_finish(struct xe_hw_fence_irq *irq); 19 19 void xe_hw_fence_irq_run(struct xe_hw_fence_irq *irq); 20 20 - void xe_hw_fence_irq_stop(struct xe_hw_fence_irq *irq); 21 21 - void xe_hw_fence_irq_start(struct xe_hw_fence_irq *irq); 22 20 23 21 void xe_hw_fence_ctx_init(struct xe_hw_fence_ctx *ctx, struct xe_gt *gt, 24 22 struct xe_hw_fence_irq *irq, const char *name);

+271 -8

drivers/gpu/drm/xe/xe_hwmon.c

reviewed

··· 39 39 REG_READ64, 40 40 }; 41 41 42 42 + #define MAX_VRAM_CHANNELS (16) 43 43 + 42 44 enum xe_hwmon_channel { 43 45 CHANNEL_CARD, 44 46 CHANNEL_PKG, 45 47 CHANNEL_VRAM, 48 48 + CHANNEL_MCTRL, 49 49 + CHANNEL_PCIE, 50 50 + CHANNEL_VRAM_N, 51 51 + CHANNEL_VRAM_N_MAX = CHANNEL_VRAM_N + MAX_VRAM_CHANNELS, 46 52 CHANNEL_MAX, 47 53 }; 48 54 ··· 57 51 FAN_2, 58 52 FAN_3, 59 53 FAN_MAX, 54 54 + }; 55 55 + 56 56 + enum xe_temp_limit { 57 57 + TEMP_LIMIT_PKG_SHUTDOWN, 58 58 + TEMP_LIMIT_PKG_CRIT, 59 59 + TEMP_LIMIT_MEM_SHUTDOWN, 60 60 + TEMP_LIMIT_PKG_MAX, 61 61 + TEMP_LIMIT_MEM_CRIT, 62 62 + TEMP_LIMIT_MAX 60 63 }; 61 64 62 65 /* Attribute index for powerX_xxx_interval sysfs entries */ ··· 106 91 */ 107 92 #define PL_WRITE_MBX_TIMEOUT_MS (1) 108 93 94 94 + /* Index of memory controller in READ_THERMAL_DATA output */ 95 95 + #define TEMP_INDEX_MCTRL 2 96 96 + 97 97 + /* Maximum characters in hwmon label name */ 98 98 + #define MAX_LABEL_SIZE 16 99 99 + 109 100 /** 110 101 * struct xe_hwmon_energy_info - to accumulate energy 111 102 */ ··· 130 109 u32 reg_val_prev; 131 110 /** @time_prev: previous timestamp */ 132 111 u64 time_prev; 112 112 + }; 113 113 + 114 114 + /** 115 115 + * struct xe_hwmon_thermal_info - to store temperature data 116 116 + */ 117 117 + struct xe_hwmon_thermal_info { 118 118 + union { 119 119 + /** @limit: temperatures limits */ 120 120 + u8 limit[TEMP_LIMIT_MAX]; 121 121 + /** @data: temperature limits in dwords */ 122 122 + u32 data[DIV_ROUND_UP(TEMP_LIMIT_MAX, sizeof(u32))]; 123 123 + }; 124 124 + /** @count: no of temperature sensors available for the platform */ 125 125 + u8 count; 126 126 + /** @value: signed value from each sensor */ 127 127 + s8 value[U8_MAX]; 128 128 + /** @vram_label: vram label names */ 129 129 + char vram_label[MAX_VRAM_CHANNELS][MAX_LABEL_SIZE]; 133 130 }; 134 131 135 132 /** ··· 176 137 u32 pl1_on_boot[CHANNEL_MAX]; 177 138 /** @pl2_on_boot: power limit PL2 on boot */ 178 139 u32 pl2_on_boot[CHANNEL_MAX]; 179 179 - 140 140 + /** @temp: Temperature info */ 141 141 + struct xe_hwmon_thermal_info temp; 180 142 }; 181 143 182 144 static int xe_hwmon_pcode_read_power_limit(const struct xe_hwmon *hwmon, u32 attr, int channel, ··· 264 224 return BMG_PACKAGE_TEMPERATURE; 265 225 else if (channel == CHANNEL_VRAM) 266 226 return BMG_VRAM_TEMPERATURE; 227 227 + else if (in_range(channel, CHANNEL_VRAM_N, CHANNEL_VRAM_N_MAX)) 228 228 + return BMG_VRAM_TEMPERATURE_N(channel - CHANNEL_VRAM_N); 267 229 } else if (xe->info.platform == XE_DG2) { 268 230 if (channel == CHANNEL_PKG) 269 231 return PCU_CR_PACKAGE_TEMPERATURE; ··· 719 677 }; 720 678 721 679 static const struct hwmon_channel_info * const hwmon_info[] = { 722 722 - HWMON_CHANNEL_INFO(temp, HWMON_T_LABEL, HWMON_T_INPUT | HWMON_T_LABEL, 723 723 - HWMON_T_INPUT | HWMON_T_LABEL), 680 680 + HWMON_CHANNEL_INFO(temp, 681 681 + HWMON_T_LABEL, 682 682 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL | 683 683 + HWMON_T_MAX, 684 684 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 685 685 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 686 686 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 687 687 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 688 688 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 689 689 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 690 690 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 691 691 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 692 692 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 693 693 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 694 694 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 695 695 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 696 696 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 697 697 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 698 698 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 699 699 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 700 700 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 701 701 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL, 702 702 + HWMON_T_CRIT | HWMON_T_EMERGENCY | HWMON_T_INPUT | HWMON_T_LABEL), 724 703 HWMON_CHANNEL_INFO(power, HWMON_P_MAX | HWMON_P_RATED_MAX | HWMON_P_LABEL | HWMON_P_CRIT | 725 704 HWMON_P_CAP, 726 705 HWMON_P_MAX | HWMON_P_RATED_MAX | HWMON_P_LABEL | HWMON_P_CAP), ··· 751 688 HWMON_CHANNEL_INFO(fan, HWMON_F_INPUT, HWMON_F_INPUT, HWMON_F_INPUT), 752 689 NULL 753 690 }; 691 691 + 692 692 + static int xe_hwmon_pcode_read_thermal_info(struct xe_hwmon *hwmon) 693 693 + { 694 694 + struct xe_tile *root_tile = xe_device_get_root_tile(hwmon->xe); 695 695 + u32 config = 0; 696 696 + int ret; 697 697 + 698 698 + ret = xe_pcode_read(root_tile, PCODE_MBOX(PCODE_THERMAL_INFO, READ_THERMAL_LIMITS, 0), 699 699 + &hwmon->temp.data[0], &hwmon->temp.data[1]); 700 700 + if (ret) 701 701 + return ret; 702 702 + 703 703 + drm_dbg(&hwmon->xe->drm, "thermal info read val 0x%x val1 0x%x\n", 704 704 + hwmon->temp.data[0], hwmon->temp.data[1]); 705 705 + 706 706 + ret = xe_pcode_read(root_tile, PCODE_MBOX(PCODE_THERMAL_INFO, READ_THERMAL_CONFIG, 0), 707 707 + &config, NULL); 708 708 + if (ret) 709 709 + return ret; 710 710 + 711 711 + drm_dbg(&hwmon->xe->drm, "thermal config count 0x%x\n", config); 712 712 + hwmon->temp.count = REG_FIELD_GET(TEMP_MASK, config); 713 713 + 714 714 + return ret; 715 715 + } 716 716 + 717 717 + static int get_mc_temp(struct xe_hwmon *hwmon, long *val) 718 718 + { 719 719 + struct xe_tile *root_tile = xe_device_get_root_tile(hwmon->xe); 720 720 + u32 *dword = (u32 *)hwmon->temp.value; 721 721 + s32 average = 0; 722 722 + int ret, i; 723 723 + 724 724 + for (i = 0; i < DIV_ROUND_UP(TEMP_LIMIT_MAX, sizeof(u32)); i++) { 725 725 + ret = xe_pcode_read(root_tile, PCODE_MBOX(PCODE_THERMAL_INFO, READ_THERMAL_DATA, i), 726 726 + (dword + i), NULL); 727 727 + if (ret) 728 728 + return ret; 729 729 + drm_dbg(&hwmon->xe->drm, "thermal data for group %d val 0x%x\n", i, dword[i]); 730 730 + } 731 731 + 732 732 + for (i = TEMP_INDEX_MCTRL; i < hwmon->temp.count - 1; i++) 733 733 + average += hwmon->temp.value[i]; 734 734 + 735 735 + average /= (hwmon->temp.count - TEMP_INDEX_MCTRL - 1); 736 736 + *val = average * MILLIDEGREE_PER_DEGREE; 737 737 + return 0; 738 738 + } 739 739 + 740 740 + static int get_pcie_temp(struct xe_hwmon *hwmon, long *val) 741 741 + { 742 742 + struct xe_tile *root_tile = xe_device_get_root_tile(hwmon->xe); 743 743 + u32 data = 0; 744 744 + int ret; 745 745 + 746 746 + ret = xe_pcode_read(root_tile, PCODE_MBOX(PCODE_THERMAL_INFO, READ_THERMAL_DATA, 747 747 + PCIE_SENSOR_GROUP_ID), &data, NULL); 748 748 + if (ret) 749 749 + return ret; 750 750 + 751 751 + /* Sensor offset is different for G21 */ 752 752 + if (hwmon->xe->info.subplatform != XE_SUBPLATFORM_BATTLEMAGE_G21) 753 753 + data = REG_FIELD_GET(PCIE_SENSOR_MASK, data); 754 754 + 755 755 + data = REG_FIELD_GET(TEMP_MASK, data); 756 756 + *val = (s8)data * MILLIDEGREE_PER_DEGREE; 757 757 + 758 758 + return 0; 759 759 + } 754 760 755 761 /* I1 is exposed as power_crit or as curr_crit depending on bit 31 */ 756 762 static int xe_hwmon_pcode_read_i1(const struct xe_hwmon *hwmon, u32 *uval) ··· 915 783 *value = DIV_ROUND_CLOSEST(REG_FIELD_GET(VOLTAGE_MASK, reg_val) * 2500, SF_VOLTAGE); 916 784 } 917 785 786 786 + static inline bool is_vram_ch_available(struct xe_hwmon *hwmon, int channel) 787 787 + { 788 788 + struct xe_mmio *mmio = xe_root_tile_mmio(hwmon->xe); 789 789 + int vram_id = channel - CHANNEL_VRAM_N; 790 790 + struct xe_reg vram_reg; 791 791 + 792 792 + vram_reg = xe_hwmon_get_reg(hwmon, REG_TEMP, channel); 793 793 + if (!xe_reg_is_valid(vram_reg) || !xe_mmio_read32(mmio, vram_reg)) 794 794 + return false; 795 795 + 796 796 + /* Create label only for available vram channel */ 797 797 + sprintf(hwmon->temp.vram_label[vram_id], "vram_ch_%d", vram_id); 798 798 + return true; 799 799 + } 800 800 + 918 801 static umode_t 919 802 xe_hwmon_temp_is_visible(struct xe_hwmon *hwmon, u32 attr, int channel) 920 803 { 921 804 switch (attr) { 805 805 + case hwmon_temp_emergency: 806 806 + switch (channel) { 807 807 + case CHANNEL_PKG: 808 808 + return hwmon->temp.limit[TEMP_LIMIT_PKG_SHUTDOWN] ? 0444 : 0; 809 809 + case CHANNEL_VRAM: 810 810 + return hwmon->temp.limit[TEMP_LIMIT_MEM_SHUTDOWN] ? 0444 : 0; 811 811 + case CHANNEL_MCTRL: 812 812 + case CHANNEL_PCIE: 813 813 + return hwmon->temp.count ? 0444 : 0; 814 814 + case CHANNEL_VRAM_N...CHANNEL_VRAM_N_MAX: 815 815 + return (is_vram_ch_available(hwmon, channel) && 816 816 + hwmon->temp.limit[TEMP_LIMIT_MEM_SHUTDOWN]) ? 0444 : 0; 817 817 + default: 818 818 + return 0; 819 819 + } 820 820 + case hwmon_temp_crit: 821 821 + switch (channel) { 822 822 + case CHANNEL_PKG: 823 823 + return hwmon->temp.limit[TEMP_LIMIT_PKG_CRIT] ? 0444 : 0; 824 824 + case CHANNEL_VRAM: 825 825 + return hwmon->temp.limit[TEMP_LIMIT_MEM_CRIT] ? 0444 : 0; 826 826 + case CHANNEL_MCTRL: 827 827 + case CHANNEL_PCIE: 828 828 + return hwmon->temp.count ? 0444 : 0; 829 829 + case CHANNEL_VRAM_N...CHANNEL_VRAM_N_MAX: 830 830 + return (is_vram_ch_available(hwmon, channel) && 831 831 + hwmon->temp.limit[TEMP_LIMIT_MEM_CRIT]) ? 0444 : 0; 832 832 + default: 833 833 + return 0; 834 834 + } 835 835 + case hwmon_temp_max: 836 836 + switch (channel) { 837 837 + case CHANNEL_PKG: 838 838 + return hwmon->temp.limit[TEMP_LIMIT_PKG_MAX] ? 0444 : 0; 839 839 + default: 840 840 + return 0; 841 841 + } 922 842 case hwmon_temp_input: 923 843 case hwmon_temp_label: 924 924 - return xe_reg_is_valid(xe_hwmon_get_reg(hwmon, REG_TEMP, channel)) ? 0444 : 0; 844 844 + switch (channel) { 845 845 + case CHANNEL_PKG: 846 846 + case CHANNEL_VRAM: 847 847 + return xe_reg_is_valid(xe_hwmon_get_reg(hwmon, REG_TEMP, 848 848 + channel)) ? 0444 : 0; 849 849 + case CHANNEL_MCTRL: 850 850 + case CHANNEL_PCIE: 851 851 + return hwmon->temp.count ? 0444 : 0; 852 852 + case CHANNEL_VRAM_N...CHANNEL_VRAM_N_MAX: 853 853 + return is_vram_ch_available(hwmon, channel) ? 0444 : 0; 854 854 + default: 855 855 + return 0; 856 856 + } 925 857 default: 926 858 return 0; 927 859 } ··· 999 803 1000 804 switch (attr) { 1001 805 case hwmon_temp_input: 1002 1002 - reg_val = xe_mmio_read32(mmio, xe_hwmon_get_reg(hwmon, REG_TEMP, channel)); 806 806 + switch (channel) { 807 807 + case CHANNEL_PKG: 808 808 + case CHANNEL_VRAM: 809 809 + reg_val = xe_mmio_read32(mmio, xe_hwmon_get_reg(hwmon, REG_TEMP, channel)); 1003 810 1004 1004 - /* HW register value is in degrees Celsius, convert to millidegrees. */ 1005 1005 - *val = REG_FIELD_GET(TEMP_MASK, reg_val) * MILLIDEGREE_PER_DEGREE; 1006 1006 - return 0; 811 811 + /* HW register value is in degrees Celsius, convert to millidegrees. */ 812 812 + *val = REG_FIELD_GET(TEMP_MASK, reg_val) * MILLIDEGREE_PER_DEGREE; 813 813 + return 0; 814 814 + case CHANNEL_MCTRL: 815 815 + return get_mc_temp(hwmon, val); 816 816 + case CHANNEL_PCIE: 817 817 + return get_pcie_temp(hwmon, val); 818 818 + case CHANNEL_VRAM_N...CHANNEL_VRAM_N_MAX: 819 819 + reg_val = xe_mmio_read32(mmio, xe_hwmon_get_reg(hwmon, REG_TEMP, channel)); 820 820 + /* 821 821 + * This temperature format is 24 bit [31:8] signed integer and 8 bit 822 822 + * [7:0] fraction. 823 823 + */ 824 824 + *val = (s32)(REG_FIELD_GET(TEMP_MASK_VRAM_N, reg_val)) * 825 825 + (REG_FIELD_GET(TEMP_SIGN_MASK, reg_val) ? -1 : 1) * 826 826 + MILLIDEGREE_PER_DEGREE; 827 827 + return 0; 828 828 + default: 829 829 + return -EOPNOTSUPP; 830 830 + } 831 831 + case hwmon_temp_emergency: 832 832 + switch (channel) { 833 833 + case CHANNEL_PKG: 834 834 + case CHANNEL_MCTRL: 835 835 + case CHANNEL_PCIE: 836 836 + *val = hwmon->temp.limit[TEMP_LIMIT_PKG_SHUTDOWN] * MILLIDEGREE_PER_DEGREE; 837 837 + return 0; 838 838 + case CHANNEL_VRAM: 839 839 + case CHANNEL_VRAM_N...CHANNEL_VRAM_N_MAX: 840 840 + *val = hwmon->temp.limit[TEMP_LIMIT_MEM_SHUTDOWN] * MILLIDEGREE_PER_DEGREE; 841 841 + return 0; 842 842 + default: 843 843 + return -EOPNOTSUPP; 844 844 + } 845 845 + case hwmon_temp_crit: 846 846 + switch (channel) { 847 847 + case CHANNEL_PKG: 848 848 + case CHANNEL_MCTRL: 849 849 + case CHANNEL_PCIE: 850 850 + *val = hwmon->temp.limit[TEMP_LIMIT_PKG_CRIT] * MILLIDEGREE_PER_DEGREE; 851 851 + return 0; 852 852 + case CHANNEL_VRAM: 853 853 + case CHANNEL_VRAM_N...CHANNEL_VRAM_N_MAX: 854 854 + *val = hwmon->temp.limit[TEMP_LIMIT_MEM_CRIT] * MILLIDEGREE_PER_DEGREE; 855 855 + return 0; 856 856 + default: 857 857 + return -EOPNOTSUPP; 858 858 + } 859 859 + case hwmon_temp_max: 860 860 + switch (channel) { 861 861 + case CHANNEL_PKG: 862 862 + *val = hwmon->temp.limit[TEMP_LIMIT_PKG_MAX] * MILLIDEGREE_PER_DEGREE; 863 863 + return 0; 864 864 + default: 865 865 + return -EOPNOTSUPP; 866 866 + } 1007 867 default: 1008 868 return -EOPNOTSUPP; 1009 869 } ··· 1415 1163 enum hwmon_sensor_types type, 1416 1164 u32 attr, int channel, const char **str) 1417 1165 { 1166 1166 + struct xe_hwmon *hwmon = dev_get_drvdata(dev); 1167 1167 + 1418 1168 switch (type) { 1419 1169 case hwmon_temp: 1420 1170 if (channel == CHANNEL_PKG) 1421 1171 *str = "pkg"; 1422 1172 else if (channel == CHANNEL_VRAM) 1423 1173 *str = "vram"; 1174 1174 + else if (channel == CHANNEL_MCTRL) 1175 1175 + *str = "mctrl"; 1176 1176 + else if (channel == CHANNEL_PCIE) 1177 1177 + *str = "pcie"; 1178 1178 + else if (in_range(channel, CHANNEL_VRAM_N, CHANNEL_VRAM_N_MAX)) 1179 1179 + *str = hwmon->temp.vram_label[channel - CHANNEL_VRAM_N]; 1424 1180 return 0; 1425 1181 case hwmon_power: 1426 1182 case hwmon_energy: ··· 1523 1263 for (channel = 0; channel < FAN_MAX; channel++) 1524 1264 if (xe_hwmon_is_visible(hwmon, hwmon_fan, hwmon_fan_input, channel)) 1525 1265 xe_hwmon_fan_input_read(hwmon, channel, &fan_speed); 1266 1266 + 1267 1267 + if (hwmon->xe->info.has_mbx_thermal_info && xe_hwmon_pcode_read_thermal_info(hwmon)) 1268 1268 + drm_warn(&hwmon->xe->drm, "Thermal mailbox not supported by card firmware\n"); 1526 1269 } 1527 1270 1528 1271 int xe_hwmon_register(struct xe_device *xe)

+8 -5

drivers/gpu/drm/xe/xe_i2c.c

reviewed

··· 5 5 * Copyright (C) 2025 Intel Corporation. 6 6 */ 7 7 8 8 + #include <drm/drm_print.h> 8 9 #include <linux/array_size.h> 9 10 #include <linux/container_of.h> 10 11 #include <linux/device.h> ··· 27 26 #include "regs/xe_i2c_regs.h" 28 27 #include "regs/xe_irq_regs.h" 29 28 30 30 - #include "xe_device.h" 31 29 #include "xe_device_types.h" 32 30 #include "xe_i2c.h" 33 31 #include "xe_mmio.h" 34 34 - #include "xe_platform_types.h" 32 32 + #include "xe_sriov.h" 33 33 + #include "xe_survivability_mode.h" 35 34 36 35 /** 37 36 * DOC: Xe I2C devices ··· 214 213 .map = xe_i2c_irq_map, 215 214 }; 216 215 217 217 - static int xe_i2c_create_irq(struct xe_i2c *i2c) 216 216 + static int xe_i2c_create_irq(struct xe_device *xe) 218 217 { 218 218 + struct xe_i2c *i2c = xe->i2c; 219 219 struct irq_domain *domain; 220 220 221 221 - if (!(i2c->ep.capabilities & XE_I2C_EP_CAP_IRQ)) 221 221 + if (!(i2c->ep.capabilities & XE_I2C_EP_CAP_IRQ) || 222 222 + xe_survivability_mode_is_boot_enabled(xe)) 222 223 return 0; 223 224 224 225 domain = irq_domain_create_linear(dev_fwnode(i2c->drm_dev), 1, &xe_i2c_irq_ops, NULL); ··· 354 351 if (ret) 355 352 return ret; 356 353 357 357 - ret = xe_i2c_create_irq(i2c); 354 354 + ret = xe_i2c_create_irq(xe); 358 355 if (ret) 359 356 goto err_unregister_notifier; 360 357

-1

drivers/gpu/drm/xe/xe_irq.c

reviewed

··· 10 10 #include <drm/drm_managed.h> 11 11 12 12 #include "display/xe_display.h" 13 13 - #include "regs/xe_guc_regs.h" 14 13 #include "regs/xe_irq_regs.h" 15 14 #include "xe_device.h" 16 15 #include "xe_drv.h"

+3 -1

drivers/gpu/drm/xe/xe_late_bind_fw_types.h

reviewed

··· 15 15 #define XE_LB_MAX_PAYLOAD_SIZE SZ_4K 16 16 17 17 /** 18 18 - * xe_late_bind_fw_id - enum to determine late binding fw index 18 18 + * enum xe_late_bind_fw_id - enum to determine late binding fw index 19 19 */ 20 20 enum xe_late_bind_fw_id { 21 21 + /** @XE_LB_FW_FAN_CONTROL: Fan control */ 21 22 XE_LB_FW_FAN_CONTROL = 0, 23 23 + /** @XE_LB_FW_MAX_ID: Number of IDs */ 22 24 XE_LB_FW_MAX_ID 23 25 }; 24 26

+1 -1

drivers/gpu/drm/xe/xe_lmtt.c

reviewed

··· 289 289 ERR_PTR(err)); 290 290 291 291 if (xe_device_has_mert(xe) && xe_tile_is_root(tile)) { 292 292 - err = xe_mert_invalidate_lmtt(tile); 292 292 + err = xe_mert_invalidate_lmtt(xe); 293 293 if (err) 294 294 xe_tile_sriov_err(tile, "MERT LMTT invalidation failed (%pe)", 295 295 ERR_PTR(err));

+31 -14

drivers/gpu/drm/xe/xe_lrc.c

reviewed

··· 857 857 * 858 858 * Returns: ctx timestamp value 859 859 */ 860 860 - u64 xe_lrc_ctx_timestamp(struct xe_lrc *lrc) 860 860 + static u64 xe_lrc_ctx_timestamp(struct xe_lrc *lrc) 861 861 { 862 862 struct xe_device *xe = lrc_to_xe(lrc); 863 863 struct iosys_map map; ··· 1067 1067 size_t max_len) 1068 1068 { 1069 1069 u32 *cmd = batch; 1070 1070 + 1071 1071 + if (IS_SRIOV_VF(gt_to_xe(lrc->gt))) 1072 1072 + return 0; 1070 1073 1071 1074 if (xe_gt_WARN_ON(lrc->gt, max_len < 12)) 1072 1075 return -ENOSPC; ··· 2409 2406 } 2410 2407 2411 2408 /** 2412 2412 - * xe_lrc_update_timestamp() - Update ctx timestamp 2409 2409 + * xe_lrc_timestamp() - Current ctx timestamp 2413 2410 * @lrc: Pointer to the lrc. 2414 2414 - * @old_ts: Old timestamp value 2415 2411 * 2416 2416 - * Populate @old_ts current saved ctx timestamp, read new ctx timestamp and 2417 2417 - * update saved value. With support for active contexts, the calculation may be 2418 2418 - * slightly racy, so follow a read-again logic to ensure that the context is 2419 2419 - * still active before returning the right timestamp. 2412 2412 + * Return latest ctx timestamp. With support for active contexts, the 2413 2413 + * calculation may bb slightly racy, so follow a read-again logic to ensure that 2414 2414 + * the context is still active before returning the right timestamp. 2420 2415 * 2421 2416 * Returns: New ctx timestamp value 2422 2417 */ 2423 2423 - u64 xe_lrc_update_timestamp(struct xe_lrc *lrc, u64 *old_ts) 2418 2418 + u64 xe_lrc_timestamp(struct xe_lrc *lrc) 2424 2419 { 2425 2425 - u64 lrc_ts, reg_ts; 2420 2420 + u64 lrc_ts, reg_ts, new_ts; 2426 2421 u32 engine_id; 2427 2427 - 2428 2428 - *old_ts = lrc->ctx_timestamp; 2429 2422 2430 2423 lrc_ts = xe_lrc_ctx_timestamp(lrc); 2431 2424 /* CTX_TIMESTAMP mmio read is invalid on VF, so return the LRC value */ 2432 2425 if (IS_SRIOV_VF(lrc_to_xe(lrc))) { 2433 2433 - lrc->ctx_timestamp = lrc_ts; 2426 2426 + new_ts = lrc_ts; 2434 2427 goto done; 2435 2428 } 2436 2429 2437 2430 if (lrc_ts == CONTEXT_ACTIVE) { 2438 2431 engine_id = xe_lrc_engine_id(lrc); 2439 2432 if (!get_ctx_timestamp(lrc, engine_id, &reg_ts)) 2440 2440 - lrc->ctx_timestamp = reg_ts; 2433 2433 + new_ts = reg_ts; 2441 2434 2442 2435 /* read lrc again to ensure context is still active */ 2443 2436 lrc_ts = xe_lrc_ctx_timestamp(lrc); ··· 2444 2445 * be a separate if condition. 2445 2446 */ 2446 2447 if (lrc_ts != CONTEXT_ACTIVE) 2447 2447 - lrc->ctx_timestamp = lrc_ts; 2448 2448 + new_ts = lrc_ts; 2448 2449 2449 2450 done: 2451 2451 + return new_ts; 2452 2452 + } 2453 2453 + 2454 2454 + /** 2455 2455 + * xe_lrc_update_timestamp() - Update ctx timestamp 2456 2456 + * @lrc: Pointer to the lrc. 2457 2457 + * @old_ts: Old timestamp value 2458 2458 + * 2459 2459 + * Populate @old_ts current saved ctx timestamp, read new ctx timestamp and 2460 2460 + * update saved value. 2461 2461 + * 2462 2462 + * Returns: New ctx timestamp value 2463 2463 + */ 2464 2464 + u64 xe_lrc_update_timestamp(struct xe_lrc *lrc, u64 *old_ts) 2465 2465 + { 2466 2466 + *old_ts = lrc->ctx_timestamp; 2467 2467 + lrc->ctx_timestamp = xe_lrc_timestamp(lrc); 2468 2468 + 2450 2469 trace_xe_lrc_update_timestamp(lrc, *old_ts); 2451 2470 2452 2471 return lrc->ctx_timestamp;

+2 -1

drivers/gpu/drm/xe/xe_lrc.h

reviewed

··· 145 145 146 146 u32 xe_lrc_ctx_timestamp_ggtt_addr(struct xe_lrc *lrc); 147 147 u32 xe_lrc_ctx_timestamp_udw_ggtt_addr(struct xe_lrc *lrc); 148 148 - u64 xe_lrc_ctx_timestamp(struct xe_lrc *lrc); 149 148 u32 xe_lrc_ctx_job_timestamp_ggtt_addr(struct xe_lrc *lrc); 150 149 u32 xe_lrc_ctx_job_timestamp(struct xe_lrc *lrc); 151 150 int xe_lrc_setup_wa_bb_with_scratch(struct xe_lrc *lrc, struct xe_hw_engine *hwe, ··· 163 164 * Returns the current LRC timestamp 164 165 */ 165 166 u64 xe_lrc_update_timestamp(struct xe_lrc *lrc, u64 *old_ts); 167 167 + 168 168 + u64 xe_lrc_timestamp(struct xe_lrc *lrc); 166 169 167 170 #endif

-2

drivers/gpu/drm/xe/xe_memirq.c

reviewed

··· 7 7 8 8 #include "regs/xe_guc_regs.h" 9 9 #include "regs/xe_irq_regs.h" 10 10 - #include "regs/xe_regs.h" 11 10 12 11 #include "xe_assert.h" 13 12 #include "xe_bo.h" ··· 15 16 #include "xe_gt.h" 16 17 #include "xe_guc.h" 17 18 #include "xe_hw_engine.h" 18 18 - #include "xe_map.h" 19 19 #include "xe_memirq.h" 20 20 #include "xe_tile_printk.h" 21 21

+56 -20

drivers/gpu/drm/xe/xe_mert.c

reviewed

··· 9 9 #include "xe_device.h" 10 10 #include "xe_mert.h" 11 11 #include "xe_mmio.h" 12 12 + #include "xe_sriov_printk.h" 12 13 #include "xe_tile.h" 13 14 14 15 /** 15 15 - * xe_mert_invalidate_lmtt - Invalidate MERT LMTT 16 16 - * @tile: the &xe_tile 16 16 + * xe_mert_init_early() - Initialize MERT data 17 17 + * @xe: the &xe_device with MERT to init 18 18 + */ 19 19 + void xe_mert_init_early(struct xe_device *xe) 20 20 + { 21 21 + struct xe_tile *tile = xe_device_get_root_tile(xe); 22 22 + struct xe_mert *mert = &tile->mert; 23 23 + 24 24 + spin_lock_init(&mert->lock); 25 25 + init_completion(&mert->tlb_inv_done); 26 26 + } 27 27 + 28 28 + /** 29 29 + * xe_mert_invalidate_lmtt() - Invalidate MERT LMTT 30 30 + * @xe: the &xe_device with MERT 17 31 * 18 32 * Trigger invalidation of the MERT LMTT and wait for completion. 19 33 * 20 34 * Return: 0 on success or -ETIMEDOUT in case of a timeout. 21 35 */ 22 22 - int xe_mert_invalidate_lmtt(struct xe_tile *tile) 36 36 + int xe_mert_invalidate_lmtt(struct xe_device *xe) 23 37 { 24 24 - struct xe_device *xe = tile_to_xe(tile); 38 38 + struct xe_tile *tile = xe_device_get_root_tile(xe); 25 39 struct xe_mert *mert = &tile->mert; 26 40 const long timeout = HZ / 4; 27 41 unsigned long flags; 28 42 29 43 xe_assert(xe, xe_device_has_mert(xe)); 30 30 - xe_assert(xe, xe_tile_is_root(tile)); 31 44 32 45 spin_lock_irqsave(&mert->lock, flags); 33 46 if (!mert->tlb_inv_triggered) { ··· 56 43 return 0; 57 44 } 58 45 46 46 + static void mert_handle_cat_error(struct xe_device *xe) 47 47 + { 48 48 + struct xe_tile *tile = xe_device_get_root_tile(xe); 49 49 + u32 reg_val, vfid, code; 50 50 + 51 51 + reg_val = xe_mmio_read32(&tile->mmio, MERT_TLB_CT_INTR_ERR_ID_PORT); 52 52 + if (!reg_val) 53 53 + return; 54 54 + xe_mmio_write32(&tile->mmio, MERT_TLB_CT_INTR_ERR_ID_PORT, 0); 55 55 + 56 56 + vfid = FIELD_GET(CATERR_VFID, reg_val); 57 57 + code = FIELD_GET(CATERR_CODES, reg_val); 58 58 + 59 59 + switch (code) { 60 60 + case CATERR_NO_ERROR: 61 61 + break; 62 62 + case CATERR_UNMAPPED_GGTT: 63 63 + xe_sriov_err(xe, "MERT: CAT_ERR: Access to an unmapped GGTT!\n"); 64 64 + xe_device_declare_wedged(xe); 65 65 + break; 66 66 + case CATERR_LMTT_FAULT: 67 67 + xe_sriov_dbg(xe, "MERT: CAT_ERR: VF%u LMTT fault!\n", vfid); 68 68 + /* XXX: track/report malicious VF activity */ 69 69 + break; 70 70 + default: 71 71 + xe_sriov_err(xe, "MERT: Unexpected CAT_ERR code=%#x!\n", code); 72 72 + xe_device_declare_wedged(xe); 73 73 + break; 74 74 + } 75 75 + } 76 76 + 59 77 /** 60 78 * xe_mert_irq_handler - Handler for MERT interrupts 61 79 * @xe: the &xe_device ··· 97 53 void xe_mert_irq_handler(struct xe_device *xe, u32 master_ctl) 98 54 { 99 55 struct xe_tile *tile = xe_device_get_root_tile(xe); 56 56 + struct xe_mert *mert = &tile->mert; 100 57 unsigned long flags; 101 58 u32 reg_val; 102 102 - u8 err; 103 59 104 60 if (!(master_ctl & SOC_H2DMEMINT_IRQ)) 105 61 return; 106 62 107 107 - reg_val = xe_mmio_read32(&tile->mmio, MERT_TLB_CT_INTR_ERR_ID_PORT); 108 108 - xe_mmio_write32(&tile->mmio, MERT_TLB_CT_INTR_ERR_ID_PORT, 0); 63 63 + mert_handle_cat_error(xe); 109 64 110 110 - err = FIELD_GET(MERT_TLB_CT_ERROR_MASK, reg_val); 111 111 - if (err == MERT_TLB_CT_LMTT_FAULT) 112 112 - drm_dbg(&xe->drm, "MERT catastrophic error: LMTT fault (VF%u)\n", 113 113 - FIELD_GET(MERT_TLB_CT_VFID_MASK, reg_val)); 114 114 - else if (err) 115 115 - drm_dbg(&xe->drm, "MERT catastrophic error: Unexpected fault (0x%x)\n", err); 116 116 - 117 117 - spin_lock_irqsave(&tile->mert.lock, flags); 118 118 - if (tile->mert.tlb_inv_triggered) { 65 65 + spin_lock_irqsave(&mert->lock, flags); 66 66 + if (mert->tlb_inv_triggered) { 119 67 reg_val = xe_mmio_read32(&tile->mmio, MERT_TLB_INV_DESC_A); 120 68 if (!(reg_val & MERT_TLB_INV_DESC_A_VALID)) { 121 121 - tile->mert.tlb_inv_triggered = false; 122 122 - complete_all(&tile->mert.tlb_inv_done); 69 69 + mert->tlb_inv_triggered = false; 70 70 + complete_all(&mert->tlb_inv_done); 123 71 } 124 72 } 125 125 - spin_unlock_irqrestore(&tile->mert.lock, flags); 73 73 + spin_unlock_irqrestore(&mert->lock, flags); 126 74 }

+9 -6

drivers/gpu/drm/xe/xe_mert.h

reviewed

··· 3 3 * Copyright(c) 2025, Intel Corporation. All rights reserved. 4 4 */ 5 5 6 6 - #ifndef __XE_MERT_H__ 7 7 - #define __XE_MERT_H__ 6 6 + #ifndef _XE_MERT_H_ 7 7 + #define _XE_MERT_H_ 8 8 9 9 #include <linux/completion.h> 10 10 #include <linux/spinlock.h> 11 11 #include <linux/types.h> 12 12 13 13 struct xe_device; 14 14 - struct xe_tile; 15 14 15 15 + /** 16 16 + * struct xe_mert - MERT related data 17 17 + */ 16 18 struct xe_mert { 17 19 /** @lock: protects the TLB invalidation status */ 18 20 spinlock_t lock; 19 21 /** @tlb_inv_triggered: indicates if TLB invalidation was triggered */ 20 22 bool tlb_inv_triggered; 21 21 - /** @mert.tlb_inv_done: completion of TLB invalidation */ 23 23 + /** @tlb_inv_done: completion of TLB invalidation */ 22 24 struct completion tlb_inv_done; 23 25 }; 24 26 25 27 #ifdef CONFIG_PCI_IOV 26 26 - int xe_mert_invalidate_lmtt(struct xe_tile *tile); 28 28 + void xe_mert_init_early(struct xe_device *xe); 29 29 + int xe_mert_invalidate_lmtt(struct xe_device *xe); 27 30 void xe_mert_irq_handler(struct xe_device *xe, u32 master_ctl); 28 31 #else 29 32 static inline void xe_mert_irq_handler(struct xe_device *xe, u32 master_ctl) { } 30 33 #endif 31 34 32 32 - #endif /* __XE_MERT_H__ */ 35 35 + #endif

-4

drivers/gpu/drm/xe/xe_mmio.c

reviewed

··· 14 14 #include <drm/drm_print.h> 15 15 16 16 #include "regs/xe_bars.h" 17 17 - #include "regs/xe_regs.h" 18 17 #include "xe_device.h" 19 19 - #include "xe_gt.h" 20 20 - #include "xe_gt_printk.h" 21 18 #include "xe_gt_sriov_vf.h" 22 22 - #include "xe_macros.h" 23 19 #include "xe_sriov.h" 24 20 #include "xe_trace.h" 25 21 #include "xe_wa.h"

-2

drivers/gpu/drm/xe/xe_mocs.c

reviewed

··· 6 6 #include "xe_mocs.h" 7 7 8 8 #include "regs/xe_gt_regs.h" 9 9 - #include "xe_bo.h" 10 9 #include "xe_device.h" 11 10 #include "xe_exec_queue.h" 12 11 #include "xe_force_wake.h" ··· 16 17 #include "xe_platform_types.h" 17 18 #include "xe_pm.h" 18 19 #include "xe_sriov.h" 19 19 - #include "xe_step_types.h" 20 20 21 21 #if IS_ENABLED(CONFIG_DRM_XE_DEBUG) 22 22 #define mocs_dbg xe_gt_dbg

+6 -4

drivers/gpu/drm/xe/xe_module.c

reviewed

··· 10 10 11 11 #include <drm/drm_module.h> 12 12 13 13 + #include "xe_device_types.h" 13 14 #include "xe_drv.h" 14 15 #include "xe_configfs.h" 15 16 #include "xe_hw_fence.h" ··· 30 29 #define DEFAULT_FORCE_PROBE CONFIG_DRM_XE_FORCE_PROBE 31 30 #define DEFAULT_MAX_VFS ~0 32 31 #define DEFAULT_MAX_VFS_STR "unlimited" 33 33 - #define DEFAULT_WEDGED_MODE 1 32 32 + #define DEFAULT_WEDGED_MODE XE_WEDGED_MODE_DEFAULT 33 33 + #define DEFAULT_WEDGED_MODE_STR XE_WEDGED_MODE_DEFAULT_STR 34 34 #define DEFAULT_SVM_NOTIFIER_SIZE 512 35 35 36 36 struct xe_modparam xe_modparam = { ··· 90 88 "[default=" DEFAULT_MAX_VFS_STR "])"); 91 89 #endif 92 90 93 93 - module_param_named_unsafe(wedged_mode, xe_modparam.wedged_mode, int, 0600); 91 91 + module_param_named_unsafe(wedged_mode, xe_modparam.wedged_mode, uint, 0600); 94 92 MODULE_PARM_DESC(wedged_mode, 95 95 - "Module's default policy for the wedged mode (0=never, 1=upon-critical-errors, 2=upon-any-hang " 96 96 - "[default=" __stringify(DEFAULT_WEDGED_MODE) "])"); 93 93 + "Module's default policy for the wedged mode (0=never, 1=upon-critical-error, 2=upon-any-hang-no-reset " 94 94 + "[default=" DEFAULT_WEDGED_MODE_STR "])"); 97 95 98 96 static int xe_check_nomodeset(void) 99 97 {

+1 -1

drivers/gpu/drm/xe/xe_module.h

reviewed

··· 21 21 #ifdef CONFIG_PCI_IOV 22 22 unsigned int max_vfs; 23 23 #endif 24 24 - int wedged_mode; 24 24 + unsigned int wedged_mode; 25 25 u32 svm_notifier_size; 26 26 }; 27 27

-1

drivers/gpu/drm/xe/xe_nvm.c

reviewed

··· 6 6 #include <linux/intel_dg_nvm_aux.h> 7 7 #include <linux/pci.h> 8 8 9 9 - #include "xe_device.h" 10 9 #include "xe_device_types.h" 11 10 #include "xe_mmio.h" 12 11 #include "xe_nvm.h"

+1 -5

drivers/gpu/drm/xe/xe_page_reclaim.c

reviewed

··· 10 10 11 11 #include "xe_page_reclaim.h" 12 12 13 13 - #include "regs/xe_gt_regs.h" 14 14 - #include "xe_assert.h" 13 13 + #include "xe_gt_stats.h" 15 14 #include "xe_macros.h" 16 16 - #include "xe_mmio.h" 17 15 #include "xe_pat.h" 18 16 #include "xe_sa.h" 19 17 #include "xe_tlb_inval_types.h" 20 20 - #include "xe_vm.h" 21 18 22 19 /** 23 20 * xe_page_reclaim_skip() - Decide whether PRL should be skipped for a VMA ··· 105 108 */ 106 109 void xe_page_reclaim_list_init(struct xe_page_reclaim_list *prl) 107 110 { 108 108 - // xe_page_reclaim_list_invalidate(prl); 109 111 prl->entries = NULL; 110 112 prl->num_entries = 0; 111 113 }

+20

drivers/gpu/drm/xe/xe_page_reclaim.h

reviewed

··· 19 19 struct xe_tlb_inval; 20 20 struct xe_tlb_inval_fence; 21 21 struct xe_tile; 22 22 + struct xe_gt; 22 23 struct xe_vma; 23 24 24 25 struct xe_guc_page_reclaim_entry { ··· 76 75 struct xe_page_reclaim_list *prl, 77 76 struct xe_tlb_inval_fence *fence); 78 77 void xe_page_reclaim_list_invalidate(struct xe_page_reclaim_list *prl); 78 78 + 79 79 + /** 80 80 + * xe_page_reclaim_list_abort() - Invalidate a PRL and log an abort reason 81 81 + * @gt: GT owning the page reclaim request 82 82 + * @prl: Page reclaim list to invalidate 83 83 + * @fmt: format string for the log message with args 84 84 + * 85 85 + * Abort page reclaim process by invalidating PRL and doing any relevant logging. 86 86 + */ 87 87 + #define xe_page_reclaim_list_abort(gt, prl, fmt, ...) \ 88 88 + do { \ 89 89 + struct xe_gt *__gt = (gt); \ 90 90 + struct xe_page_reclaim_list *__prl = (prl); \ 91 91 + \ 92 92 + xe_page_reclaim_list_invalidate(__prl); \ 93 93 + xe_gt_stats_incr(__gt, XE_GT_STATS_ID_PRL_ABORTED_COUNT, 1); \ 94 94 + vm_dbg(&gt_to_xe(__gt)->drm, "PRL aborted: " fmt, ##__VA_ARGS__); \ 95 95 + } while (0) 96 96 + 79 97 void xe_page_reclaim_list_init(struct xe_page_reclaim_list *prl); 80 98 int xe_page_reclaim_list_alloc_entries(struct xe_page_reclaim_list *prl); 81 99 /**

+46 -6

drivers/gpu/drm/xe/xe_pat.c

reviewed

··· 132 132 * in the table. 133 133 * 134 134 * Note: There is an implicit assumption in the driver that compression and 135 135 - * coh_1way+ are mutually exclusive. If this is ever not true then userptr 136 136 - * and imported dma-buf from external device will have uncleared ccs state. See 137 137 - * also xe_bo_needs_ccs_pages(). 135 135 + * coh_1way+ are mutually exclusive for platforms prior to Xe3. Starting 136 136 + * with Xe3, compression can be combined with coherency. If using compression 137 137 + * with coherency, userptr and imported dma-buf from external device will 138 138 + * have uncleared ccs state. See also xe_bo_needs_ccs_pages(). 138 139 */ 139 140 #define XE2_PAT(no_promote, comp_en, l3clos, l3_policy, l4_policy, __coh_mode) \ 140 141 { \ ··· 145 144 REG_FIELD_PREP(XE2_L3_POLICY, l3_policy) | \ 146 145 REG_FIELD_PREP(XE2_L4_POLICY, l4_policy) | \ 147 146 REG_FIELD_PREP(XE2_COH_MODE, __coh_mode), \ 148 148 - .coh_mode = (BUILD_BUG_ON_ZERO(__coh_mode && comp_en) || __coh_mode) ? \ 149 149 - XE_COH_AT_LEAST_1WAY : XE_COH_NONE, \ 147 147 + .coh_mode = __coh_mode ? XE_COH_AT_LEAST_1WAY : XE_COH_NONE, \ 150 148 .valid = 1 \ 151 149 } 152 150 ··· 181 181 [31] = XE2_PAT( 0, 0, 3, 0, 3, 3 ), 182 182 }; 183 183 184 184 + static const struct xe_pat_table_entry xe3_lpg_pat_table[] = { 185 185 + [ 0] = XE2_PAT( 0, 0, 0, 0, 3, 0 ), 186 186 + [ 1] = XE2_PAT( 0, 0, 0, 0, 3, 2 ), 187 187 + [ 2] = XE2_PAT( 0, 0, 0, 0, 3, 3 ), 188 188 + [ 3] = XE2_PAT( 0, 0, 0, 3, 3, 0 ), 189 189 + [ 4] = XE2_PAT( 0, 0, 0, 3, 0, 2 ), 190 190 + [ 5] = XE2_PAT( 0, 0, 0, 3, 3, 2 ), 191 191 + [ 6] = XE2_PAT( 1, 0, 0, 1, 3, 0 ), 192 192 + [ 7] = XE2_PAT( 0, 0, 0, 3, 0, 3 ), 193 193 + [ 8] = XE2_PAT( 0, 0, 0, 3, 0, 0 ), 194 194 + [ 9] = XE2_PAT( 0, 1, 0, 0, 3, 0 ), 195 195 + [10] = XE2_PAT( 0, 1, 0, 3, 0, 0 ), 196 196 + [11] = XE2_PAT( 1, 1, 0, 1, 3, 0 ), 197 197 + [12] = XE2_PAT( 0, 1, 0, 3, 3, 0 ), 198 198 + [13] = XE2_PAT( 0, 0, 0, 0, 0, 0 ), 199 199 + [14] = XE2_PAT( 0, 1, 0, 0, 0, 0 ), 200 200 + [15] = XE2_PAT( 1, 1, 0, 1, 1, 0 ), 201 201 + [16] = XE2_PAT( 0, 1, 0, 0, 3, 2 ), 202 202 + /* 17..19 are reserved; leave set to all 0's */ 203 203 + [20] = XE2_PAT( 0, 0, 1, 0, 3, 0 ), 204 204 + [21] = XE2_PAT( 0, 1, 1, 0, 3, 0 ), 205 205 + [22] = XE2_PAT( 0, 0, 1, 0, 3, 2 ), 206 206 + [23] = XE2_PAT( 0, 0, 1, 0, 3, 3 ), 207 207 + [24] = XE2_PAT( 0, 0, 2, 0, 3, 0 ), 208 208 + [25] = XE2_PAT( 0, 1, 2, 0, 3, 0 ), 209 209 + [26] = XE2_PAT( 0, 0, 2, 0, 3, 2 ), 210 210 + [27] = XE2_PAT( 0, 0, 2, 0, 3, 3 ), 211 211 + [28] = XE2_PAT( 0, 0, 3, 0, 3, 0 ), 212 212 + [29] = XE2_PAT( 0, 1, 3, 0, 3, 0 ), 213 213 + [30] = XE2_PAT( 0, 0, 3, 0, 3, 2 ), 214 214 + [31] = XE2_PAT( 0, 0, 3, 0, 3, 3 ), 215 215 + }; 184 216 /* Special PAT values programmed outside the main table */ 185 217 static const struct xe_pat_table_entry xe2_pat_ats = XE2_PAT( 0, 0, 0, 0, 3, 3 ); 186 218 static const struct xe_pat_table_entry xe2_pat_pta = XE2_PAT( 0, 0, 0, 0, 3, 0 ); ··· 522 490 523 491 void xe_pat_init_early(struct xe_device *xe) 524 492 { 493 493 + xe->pat.idx[XE_CACHE_WB_COMPRESSION] = XE_PAT_INVALID_IDX; 525 494 if (GRAPHICS_VERx100(xe) == 3511) { 526 495 xe->pat.ops = &xe3p_xpc_pat_ops; 527 496 xe->pat.table = xe3p_xpc_pat_table; ··· 534 501 xe->pat.idx[XE_CACHE_WB] = 2; 535 502 } else if (GRAPHICS_VER(xe) == 30 || GRAPHICS_VER(xe) == 20) { 536 503 xe->pat.ops = &xe2_pat_ops; 537 537 - xe->pat.table = xe2_pat_table; 504 504 + if (GRAPHICS_VER(xe) == 30) { 505 505 + xe->pat.table = xe3_lpg_pat_table; 506 506 + xe->pat.idx[XE_CACHE_WB_COMPRESSION] = 16; 507 507 + } else { 508 508 + xe->pat.table = xe2_pat_table; 509 509 + } 538 510 xe->pat.pat_ats = &xe2_pat_ats; 539 511 if (IS_DGFX(xe)) 540 512 xe->pat.pat_pta = &xe2_pat_pta; ··· 696 658 if (GRAPHICS_VER(xe) >= 20) { 697 659 drm_printf(p, "IDX[XE_CACHE_NONE_COMPRESSION] = %d\n", 698 660 xe->pat.idx[XE_CACHE_NONE_COMPRESSION]); 661 661 + drm_printf(p, "IDX[XE_CACHE_WB_COMPRESSION] = %d\n", 662 662 + xe->pat.idx[XE_CACHE_WB_COMPRESSION]); 699 663 } 700 664 701 665 return 0;

+2

drivers/gpu/drm/xe/xe_pat.h

reviewed

··· 12 12 struct xe_device; 13 13 struct xe_gt; 14 14 15 15 + #define XE_PAT_INVALID_IDX U16_MAX 16 16 + 15 17 /** 16 18 * struct xe_pat_table_entry - The pat_index encoding and other meta information. 17 19 */

+4 -1

drivers/gpu/drm/xe/xe_pci.c

reviewed

··· 24 24 #include "xe_gt.h" 25 25 #include "xe_gt_sriov_vf.h" 26 26 #include "xe_guc.h" 27 27 - #include "xe_macros.h" 28 27 #include "xe_mmio.h" 29 28 #include "xe_module.h" 30 29 #include "xe_pci_rebar.h" ··· 365 366 .has_fan_control = true, 366 367 .has_flat_ccs = 1, 367 368 .has_mbx_power_limits = true, 369 369 + .has_mbx_thermal_info = true, 368 370 .has_gsc_nvm = 1, 369 371 .has_heci_cscfi = 1, 370 372 .has_i2c = true, ··· 392 392 .has_sriov = true, 393 393 .has_mem_copy_instr = true, 394 394 .has_pre_prod_wa = 1, 395 395 + .has_pxp = true, 395 396 .max_gt_per_tile = 2, 396 397 .needs_scratch = true, 397 398 .needs_shared_vf_gt_wq = true, ··· 422 421 .has_gsc_nvm = 1, 423 422 .has_i2c = true, 424 423 .has_mbx_power_limits = true, 424 424 + .has_mbx_thermal_info = true, 425 425 .has_mert = true, 426 426 .has_pre_prod_wa = 1, 427 427 .has_soc_remapper_sysctrl = true, ··· 688 686 /* runtime fusing may force flat_ccs to disabled later */ 689 687 xe->info.has_flat_ccs = desc->has_flat_ccs; 690 688 xe->info.has_mbx_power_limits = desc->has_mbx_power_limits; 689 689 + xe->info.has_mbx_thermal_info = desc->has_mbx_thermal_info; 691 690 xe->info.has_gsc_nvm = desc->has_gsc_nvm; 692 691 xe->info.has_heci_gscfi = desc->has_heci_gscfi; 693 692 xe->info.has_heci_cscfi = desc->has_heci_cscfi;

+1

drivers/gpu/drm/xe/xe_pci_types.h

reviewed

··· 48 48 u8 has_late_bind:1; 49 49 u8 has_llc:1; 50 50 u8 has_mbx_power_limits:1; 51 51 + u8 has_mbx_thermal_info:1; 51 52 u8 has_mem_copy_instr:1; 52 53 u8 has_mert:1; 53 54 u8 has_pre_prod_wa:1;

+7

drivers/gpu/drm/xe/xe_pcode_api.h

reviewed

··· 50 50 #define READ_PL_FROM_FW 0x1 51 51 #define READ_PL_FROM_PCODE 0x0 52 52 53 53 + #define PCODE_THERMAL_INFO 0x25 54 54 + #define READ_THERMAL_LIMITS 0x0 55 55 + #define READ_THERMAL_CONFIG 0x1 56 56 + #define READ_THERMAL_DATA 0x2 57 57 + #define PCIE_SENSOR_GROUP_ID 0x2 58 58 + #define PCIE_SENSOR_MASK REG_GENMASK(31, 16) 59 59 + 53 60 #define PCODE_LATE_BINDING 0x5C 54 61 #define GET_CAPABILITY_STATUS 0x0 55 62 #define V1_FAN_SUPPORTED REG_BIT(0)

+22 -4

drivers/gpu/drm/xe/xe_pm.c

reviewed

··· 260 260 261 261 xe_irq_resume(xe); 262 262 263 263 - for_each_gt(gt, xe, id) 264 264 - xe_gt_resume(gt); 263 263 + for_each_gt(gt, xe, id) { 264 264 + err = xe_gt_resume(gt); 265 265 + if (err) 266 266 + break; 267 267 + } 265 268 269 269 + /* 270 270 + * Try to bring up display before bailing from GT resume failure, 271 271 + * so we don't leave the user clueless with a blank screen. 272 272 + */ 266 273 xe_display_pm_resume(xe); 274 274 + if (err) 275 275 + goto err; 267 276 268 277 err = xe_bo_restore_late(xe); 269 278 if (err) ··· 665 656 666 657 xe_irq_resume(xe); 667 658 668 668 - for_each_gt(gt, xe, id) 669 669 - xe->d3cold.allowed ? xe_gt_resume(gt) : xe_gt_runtime_resume(gt); 659 659 + for_each_gt(gt, xe, id) { 660 660 + err = xe->d3cold.allowed ? xe_gt_resume(gt) : xe_gt_runtime_resume(gt); 661 661 + if (err) 662 662 + break; 663 663 + } 670 664 665 665 + /* 666 666 + * Try to bring up display before bailing from GT resume failure, 667 667 + * so we don't leave the user clueless with a blank screen. 668 668 + */ 671 669 xe_display_pm_runtime_resume(xe); 670 670 + if (err) 671 671 + goto out; 672 672 673 673 if (xe->d3cold.allowed) { 674 674 err = xe_bo_restore_late(xe);

+1 -1

drivers/gpu/drm/xe/xe_psmi.c

reviewed

··· 6 6 #include <linux/debugfs.h> 7 7 8 8 #include "xe_bo.h" 9 9 - #include "xe_device.h" 9 9 + #include "xe_device_types.h" 10 10 #include "xe_configfs.h" 11 11 #include "xe_psmi.h" 12 12

+58 -26

drivers/gpu/drm/xe/xe_pt.c

reviewed

··· 11 11 #include "xe_drm_client.h" 12 12 #include "xe_exec_queue.h" 13 13 #include "xe_gt.h" 14 14 + #include "xe_gt_stats.h" 14 15 #include "xe_migrate.h" 15 16 #include "xe_page_reclaim.h" 16 17 #include "xe_pt_types.h" ··· 1577 1576 return false; 1578 1577 } 1579 1578 1580 1580 - /* Huge 2MB leaf lives directly in a level-1 table and has no children */ 1581 1581 - static bool is_2m_pte(struct xe_pt *pte) 1582 1582 - { 1583 1583 - return pte->level == 1 && !pte->base.children; 1584 1584 - } 1585 1585 - 1586 1579 /* page_size = 2^(reclamation_size + XE_PTE_SHIFT) */ 1587 1580 #define COMPUTE_RECLAIM_ADDRESS_MASK(page_size) \ 1588 1581 ({ \ ··· 1588 1593 struct xe_page_reclaim_list *prl, 1589 1594 u64 pte, struct xe_pt *xe_child) 1590 1595 { 1596 1596 + struct xe_gt *gt = tile->primary_gt; 1591 1597 struct xe_guc_page_reclaim_entry *reclaim_entries = prl->entries; 1592 1592 - u64 phys_page = (pte & XE_PTE_ADDR_MASK) >> XE_PTE_SHIFT; 1598 1598 + u64 phys_addr = pte & XE_PTE_ADDR_MASK; 1599 1599 + u64 phys_page = phys_addr >> XE_PTE_SHIFT; 1593 1600 int num_entries = prl->num_entries; 1594 1601 u32 reclamation_size; 1595 1602 ··· 1609 1612 * Only 4K, 64K (level 0), and 2M pages are supported by hardware for page reclaim 1610 1613 */ 1611 1614 if (xe_child->level == 0 && !(pte & XE_PTE_PS64)) { 1615 1615 + xe_gt_stats_incr(gt, XE_GT_STATS_ID_PRL_4K_ENTRY_COUNT, 1); 1612 1616 reclamation_size = COMPUTE_RECLAIM_ADDRESS_MASK(SZ_4K); /* reclamation_size = 0 */ 1617 1617 + xe_tile_assert(tile, phys_addr % SZ_4K == 0); 1613 1618 } else if (xe_child->level == 0) { 1619 1619 + xe_gt_stats_incr(gt, XE_GT_STATS_ID_PRL_64K_ENTRY_COUNT, 1); 1614 1620 reclamation_size = COMPUTE_RECLAIM_ADDRESS_MASK(SZ_64K); /* reclamation_size = 4 */ 1615 1615 - } else if (is_2m_pte(xe_child)) { 1621 1621 + xe_tile_assert(tile, phys_addr % SZ_64K == 0); 1622 1622 + } else if (xe_child->level == 1 && pte & XE_PDE_PS_2M) { 1623 1623 + xe_gt_stats_incr(gt, XE_GT_STATS_ID_PRL_2M_ENTRY_COUNT, 1); 1616 1624 reclamation_size = COMPUTE_RECLAIM_ADDRESS_MASK(SZ_2M); /* reclamation_size = 9 */ 1625 1625 + xe_tile_assert(tile, phys_addr % SZ_2M == 0); 1617 1626 } else { 1618 1618 - xe_page_reclaim_list_invalidate(prl); 1619 1619 - vm_dbg(&tile_to_xe(tile)->drm, 1620 1620 - "PRL invalidate: unsupported PTE level=%u pte=%#llx\n", 1621 1621 - xe_child->level, pte); 1627 1627 + xe_page_reclaim_list_abort(tile->primary_gt, prl, 1628 1628 + "unsupported PTE level=%u pte=%#llx", 1629 1629 + xe_child->level, pte); 1622 1630 return -EINVAL; 1623 1631 } 1624 1632 ··· 1650 1648 struct xe_pt_stage_unbind_walk *xe_walk = 1651 1649 container_of(walk, typeof(*xe_walk), base); 1652 1650 struct xe_device *xe = tile_to_xe(xe_walk->tile); 1651 1651 + pgoff_t first = xe_pt_offset(addr, xe_child->level, walk); 1652 1652 + bool killed; 1653 1653 1654 1654 XE_WARN_ON(!*child); 1655 1655 XE_WARN_ON(!level); 1656 1656 /* Check for leaf node */ 1657 1657 if (xe_walk->prl && xe_page_reclaim_list_valid(xe_walk->prl) && 1658 1658 - !xe_child->base.children) { 1658 1658 + (!xe_child->base.children || !xe_child->base.children[first])) { 1659 1659 struct iosys_map *leaf_map = &xe_child->bo->vmap; 1660 1660 - pgoff_t first = xe_pt_offset(addr, 0, walk); 1661 1661 - pgoff_t count = xe_pt_num_entries(addr, next, 0, walk); 1660 1660 + pgoff_t count = xe_pt_num_entries(addr, next, xe_child->level, walk); 1662 1661 1663 1662 for (pgoff_t i = 0; i < count; i++) { 1664 1663 u64 pte = xe_map_rd(xe, leaf_map, (first + i) * sizeof(u64), u64); 1665 1664 int ret; 1665 1665 + 1666 1666 + /* 1667 1667 + * In rare scenarios, pte may not be written yet due to racy conditions. 1668 1668 + * In such cases, invalidate the PRL and fallback to full PPC invalidation. 1669 1669 + */ 1670 1670 + if (!pte) { 1671 1671 + xe_page_reclaim_list_abort(xe_walk->tile->primary_gt, xe_walk->prl, 1672 1672 + "found zero pte at addr=%#llx", addr); 1673 1673 + break; 1674 1674 + } 1675 1675 + 1676 1676 + /* Ensure it is a defined page */ 1677 1677 + xe_tile_assert(xe_walk->tile, 1678 1678 + xe_child->level == 0 || 1679 1679 + (pte & (XE_PTE_PS64 | XE_PDE_PS_2M | XE_PDPE_PS_1G))); 1680 1680 + 1681 1681 + /* An entry should be added for 64KB but contigious 4K have XE_PTE_PS64 */ 1682 1682 + if (pte & XE_PTE_PS64) 1683 1683 + i += 15; /* Skip other 15 consecutive 4K pages in the 64K page */ 1666 1684 1667 1685 /* Account for NULL terminated entry on end (-1) */ 1668 1686 if (xe_walk->prl->num_entries < XE_PAGE_RECLAIM_MAX_ENTRIES - 1) { ··· 1692 1670 break; 1693 1671 } else { 1694 1672 /* overflow, mark as invalid */ 1695 1695 - xe_page_reclaim_list_invalidate(xe_walk->prl); 1696 1696 - vm_dbg(&xe->drm, 1697 1697 - "PRL invalidate: overflow while adding pte=%#llx", 1698 1698 - pte); 1673 1673 + xe_page_reclaim_list_abort(xe_walk->tile->primary_gt, xe_walk->prl, 1674 1674 + "overflow while adding pte=%#llx", 1675 1675 + pte); 1699 1676 break; 1700 1677 } 1701 1678 } 1702 1679 } 1703 1680 1704 1704 - /* If aborting page walk early, invalidate PRL since PTE may be dropped from this abort */ 1705 1705 - if (xe_pt_check_kill(addr, next, level - 1, xe_child, action, walk) && 1706 1706 - xe_walk->prl && level > 1 && xe_child->base.children && xe_child->num_live != 0) { 1707 1707 - xe_page_reclaim_list_invalidate(xe_walk->prl); 1708 1708 - vm_dbg(&xe->drm, 1709 1709 - "PRL invalidate: kill at level=%u addr=%#llx next=%#llx num_live=%u\n", 1710 1710 - level, addr, next, xe_child->num_live); 1681 1681 + killed = xe_pt_check_kill(addr, next, level - 1, xe_child, action, walk); 1682 1682 + 1683 1683 + /* 1684 1684 + * Verify PRL is active and if entry is not a leaf pte (base.children conditions), 1685 1685 + * there is a potential need to invalidate the PRL if any PTE (num_live) are dropped. 1686 1686 + */ 1687 1687 + if (xe_walk->prl && level > 1 && xe_child->num_live && 1688 1688 + xe_child->base.children && xe_child->base.children[first]) { 1689 1689 + bool covered = xe_pt_covers(addr, next, xe_child->level, &xe_walk->base); 1690 1690 + 1691 1691 + /* 1692 1692 + * If aborting page walk early (kill) or page walk completes the full range 1693 1693 + * we need to invalidate the PRL. 1694 1694 + */ 1695 1695 + if (killed || covered) 1696 1696 + xe_page_reclaim_list_abort(xe_walk->tile->primary_gt, xe_walk->prl, 1697 1697 + "kill at level=%u addr=%#llx next=%#llx num_live=%u", 1698 1698 + level, addr, next, xe_child->num_live); 1711 1699 } 1712 1700 1713 1701 return 0;

+1

drivers/gpu/drm/xe/xe_pt_types.h

reviewed

··· 20 20 XE_CACHE_WT, 21 21 XE_CACHE_WB, 22 22 XE_CACHE_NONE_COMPRESSION, /*UC + COH_NONE + COMPRESSION */ 23 23 + XE_CACHE_WB_COMPRESSION, 23 24 __XE_CACHE_LEVEL_COUNT, 24 25 }; 25 26

-1

drivers/gpu/drm/xe/xe_pxp.c

reviewed

··· 15 15 #include "xe_force_wake.h" 16 16 #include "xe_guc_submit.h" 17 17 #include "xe_gsc_proxy.h" 18 18 - #include "xe_gt.h" 19 18 #include "xe_gt_types.h" 20 19 #include "xe_huc.h" 21 20 #include "xe_mmio.h"

+1 -1

drivers/gpu/drm/xe/xe_pxp_debugfs.c

reviewed

··· 11 11 #include <drm/drm_managed.h> 12 12 #include <drm/drm_print.h> 13 13 14 14 - #include "xe_device.h" 14 14 + #include "xe_device_types.h" 15 15 #include "xe_pxp.h" 16 16 #include "xe_pxp_types.h" 17 17 #include "regs/xe_irq_regs.h"

+1 -4

drivers/gpu/drm/xe/xe_reg_sr.c

reviewed

··· 13 13 #include <drm/drm_managed.h> 14 14 #include <drm/drm_print.h> 15 15 16 16 - #include "regs/xe_engine_regs.h" 17 17 - #include "regs/xe_gt_regs.h" 18 16 #include "xe_device.h" 19 17 #include "xe_device_types.h" 20 18 #include "xe_force_wake.h" 21 21 - #include "xe_gt.h" 22 19 #include "xe_gt_mcr.h" 23 20 #include "xe_gt_printk.h" 21 21 + #include "xe_gt_types.h" 24 22 #include "xe_hw_engine_types.h" 25 25 - #include "xe_macros.h" 26 23 #include "xe_mmio.h" 27 24 #include "xe_rtp_types.h" 28 25

-1

drivers/gpu/drm/xe/xe_reg_whitelist.c

reviewed

··· 8 8 #include "regs/xe_engine_regs.h" 9 9 #include "regs/xe_gt_regs.h" 10 10 #include "regs/xe_oa_regs.h" 11 11 - #include "regs/xe_regs.h" 12 11 #include "xe_device.h" 13 12 #include "xe_gt_types.h" 14 13 #include "xe_gt_printk.h"

+21 -8

drivers/gpu/drm/xe/xe_ring_ops.c

reviewed

··· 11 11 #include "instructions/xe_mi_commands.h" 12 12 #include "regs/xe_engine_regs.h" 13 13 #include "regs/xe_gt_regs.h" 14 14 - #include "regs/xe_lrc_layout.h" 15 14 #include "xe_exec_queue.h" 16 16 - #include "xe_gt.h" 15 15 + #include "xe_gt_types.h" 17 16 #include "xe_lrc.h" 18 18 - #include "xe_macros.h" 19 17 #include "xe_sched_job.h" 20 18 #include "xe_sriov.h" 21 19 #include "xe_vm_types.h" ··· 233 235 return 0; 234 236 } 235 237 236 236 - static int emit_copy_timestamp(struct xe_lrc *lrc, u32 *dw, int i) 238 238 + static int emit_copy_timestamp(struct xe_device *xe, struct xe_lrc *lrc, 239 239 + u32 *dw, int i) 237 240 { 238 241 dw[i++] = MI_STORE_REGISTER_MEM | MI_SRM_USE_GGTT | MI_SRM_ADD_CS_OFFSET; 239 242 dw[i++] = RING_CTX_TIMESTAMP(0).addr; 240 243 dw[i++] = xe_lrc_ctx_job_timestamp_ggtt_addr(lrc); 241 244 dw[i++] = 0; 245 245 + 246 246 + /* 247 247 + * Ensure CTX timestamp >= Job timestamp during VF sampling to avoid 248 248 + * arithmetic wraparound in TDR. 249 249 + */ 250 250 + if (IS_SRIOV_VF(xe)) { 251 251 + dw[i++] = MI_STORE_REGISTER_MEM | MI_SRM_USE_GGTT | 252 252 + MI_SRM_ADD_CS_OFFSET; 253 253 + dw[i++] = RING_CTX_TIMESTAMP(0).addr; 254 254 + dw[i++] = xe_lrc_ctx_timestamp_ggtt_addr(lrc); 255 255 + dw[i++] = 0; 256 256 + } 242 257 243 258 return i; 244 259 } ··· 266 255 267 256 *head = lrc->ring.tail; 268 257 269 269 - i = emit_copy_timestamp(lrc, dw, i); 258 258 + i = emit_copy_timestamp(gt_to_xe(gt), lrc, dw, i); 270 259 271 260 if (job->ring_ops_flush_tlb) { 272 261 dw[i++] = preparser_disable(true); ··· 321 310 322 311 *head = lrc->ring.tail; 323 312 324 324 - i = emit_copy_timestamp(lrc, dw, i); 313 313 + i = emit_copy_timestamp(xe, lrc, dw, i); 325 314 326 315 dw[i++] = preparser_disable(true); 327 316 ··· 375 364 376 365 *head = lrc->ring.tail; 377 366 378 378 - i = emit_copy_timestamp(lrc, dw, i); 367 367 + i = emit_copy_timestamp(xe, lrc, dw, i); 379 368 380 369 dw[i++] = preparser_disable(true); 381 370 if (lacks_render) ··· 417 406 struct xe_lrc *lrc, u32 *head, 418 407 u32 seqno) 419 408 { 409 409 + struct xe_gt *gt = job->q->gt; 410 410 + struct xe_device *xe = gt_to_xe(gt); 420 411 u32 saddr = xe_lrc_start_seqno_ggtt_addr(lrc); 421 412 u32 dw[MAX_JOB_SIZE_DW], i = 0; 422 413 423 414 *head = lrc->ring.tail; 424 415 425 425 - i = emit_copy_timestamp(lrc, dw, i); 416 416 + i = emit_copy_timestamp(xe, lrc, dw, i); 426 417 427 418 i = emit_store_imm_ggtt(saddr, seqno, dw, i); 428 419

-1

drivers/gpu/drm/xe/xe_rtp.c

reviewed

··· 12 12 #include "xe_configfs.h" 13 13 #include "xe_gt.h" 14 14 #include "xe_gt_topology.h" 15 15 - #include "xe_macros.h" 16 15 #include "xe_reg_sr.h" 17 16 #include "xe_sriov.h" 18 17

+1 -1

drivers/gpu/drm/xe/xe_sa.c

reviewed

··· 10 10 #include <drm/drm_managed.h> 11 11 12 12 #include "xe_bo.h" 13 13 - #include "xe_device.h" 13 13 + #include "xe_device_types.h" 14 14 #include "xe_map.h" 15 15 16 16 static void xe_sa_bo_manager_fini(struct drm_device *drm, void *arg)

+2 -1

drivers/gpu/drm/xe/xe_sched_job.c

reviewed

··· 11 11 12 12 #include "xe_device.h" 13 13 #include "xe_exec_queue.h" 14 14 - #include "xe_gt.h" 14 14 + #include "xe_gt_types.h" 15 15 #include "xe_hw_engine_types.h" 16 16 #include "xe_hw_fence.h" 17 17 #include "xe_lrc.h" ··· 110 110 return ERR_PTR(-ENOMEM); 111 111 112 112 job->q = q; 113 113 + job->sample_timestamp = U64_MAX; 113 114 kref_init(&job->refcount); 114 115 xe_exec_queue_get(job->q); 115 116

+2

drivers/gpu/drm/xe/xe_sched_job_types.h

reviewed

··· 59 59 u32 lrc_seqno; 60 60 /** @migrate_flush_flags: Additional flush flags for migration jobs */ 61 61 u32 migrate_flush_flags; 62 62 + /** @sample_timestamp: Sampling of job timestamp in TDR */ 63 63 + u64 sample_timestamp; 62 64 /** @ring_ops_flush_tlb: The ring ops need to flush TLB before payload. */ 63 65 bool ring_ops_flush_tlb; 64 66 /** @ggtt: mapped in ggtt. */

-1

drivers/gpu/drm/xe/xe_sriov_packet.c

reviewed

··· 6 6 #include "xe_bo.h" 7 7 #include "xe_device.h" 8 8 #include "xe_guc_klv_helpers.h" 9 9 - #include "xe_printk.h" 10 9 #include "xe_sriov_packet.h" 11 10 #include "xe_sriov_packet_types.h" 12 11 #include "xe_sriov_pf_helpers.h"

+1 -3

drivers/gpu/drm/xe/xe_sriov_pf.c

reviewed

··· 90 90 */ 91 91 int xe_sriov_pf_init_early(struct xe_device *xe) 92 92 { 93 93 - struct xe_mert *mert = &xe_device_get_root_tile(xe)->mert; 94 93 int err; 95 94 96 95 xe_assert(xe, IS_SRIOV_PF(xe)); ··· 111 112 112 113 xe_sriov_pf_service_init(xe); 113 114 114 114 - spin_lock_init(&mert->lock); 115 115 - init_completion(&mert->tlb_inv_done); 115 115 + xe_mert_init_early(xe); 116 116 117 117 return 0; 118 118 }

-1

drivers/gpu/drm/xe/xe_sriov_pf_debugfs.c

reviewed

··· 16 16 #include "xe_sriov_pf_migration.h" 17 17 #include "xe_sriov_pf_provision.h" 18 18 #include "xe_sriov_pf_service.h" 19 19 - #include "xe_sriov_printk.h" 20 19 #include "xe_tile_sriov_pf_debugfs.h" 21 20 22 21 /*

-1

drivers/gpu/drm/xe/xe_sriov_vf.c

reviewed

··· 6 6 #include <drm/drm_debugfs.h> 7 7 #include <drm/drm_managed.h> 8 8 9 9 - #include "xe_gt.h" 10 9 #include "xe_gt_sriov_vf.h" 11 10 #include "xe_guc.h" 12 11 #include "xe_sriov_printk.h"

+2 -1

drivers/gpu/drm/xe/xe_step.c

reviewed

··· 5 5 6 6 #include "xe_step.h" 7 7 8 8 + #include <drm/drm_print.h> 8 9 #include <kunit/visibility.h> 9 10 #include <linux/bitfield.h> 10 11 11 11 - #include "xe_device.h" 12 12 + #include "xe_device_types.h" 12 13 #include "xe_platform_types.h" 13 14 14 15 /*

+1 -2

drivers/gpu/drm/xe/xe_survivability_mode.c

reviewed

··· 12 12 13 13 #include "xe_configfs.h" 14 14 #include "xe_device.h" 15 15 - #include "xe_gt.h" 16 15 #include "xe_heci_gsc.h" 17 16 #include "xe_i2c.h" 18 17 #include "xe_mmio.h" ··· 320 321 if (ret) 321 322 return ret; 322 323 323 323 - /* Make sure xe_heci_gsc_init() knows about survivability mode */ 324 324 + /* Make sure xe_heci_gsc_init() and xe_i2c_probe() are aware of survivability */ 324 325 survivability->mode = true; 325 326 326 327 xe_heci_gsc_init(xe);

+2

drivers/gpu/drm/xe/xe_svm.c

reviewed

··· 988 988 ret = (range->tile_present & ~range->tile_invalidated & tile_mask) == tile_mask; 989 989 if (dpagemap) 990 990 ret = ret && xe_svm_range_has_pagemap_locked(range, dpagemap); 991 991 + else 992 992 + ret = ret && !range->base.pages.dpagemap; 991 993 992 994 xe_svm_notifier_unlock(vm); 993 995

+1 -2

drivers/gpu/drm/xe/xe_tile.c

reviewed

··· 9 9 #include <drm/drm_pagemap_util.h> 10 10 11 11 #include "xe_bo.h" 12 12 - #include "xe_device.h" 12 12 + #include "xe_device_types.h" 13 13 #include "xe_ggtt.h" 14 14 - #include "xe_gt.h" 15 14 #include "xe_memirq.h" 16 15 #include "xe_migrate.h" 17 16 #include "xe_pcode.h"

+9 -8

drivers/gpu/drm/xe/xe_tlb_inval.c

reviewed

··· 5 5 6 6 #include <drm/drm_managed.h> 7 7 8 8 - #include "abi/guc_actions_abi.h" 9 9 - #include "xe_device.h" 8 8 + #include "xe_device_types.h" 10 9 #include "xe_force_wake.h" 11 11 - #include "xe_gt.h" 12 12 - #include "xe_gt_printk.h" 13 10 #include "xe_gt_stats.h" 14 14 - #include "xe_guc.h" 11 11 + #include "xe_gt_types.h" 15 12 #include "xe_guc_ct.h" 16 13 #include "xe_guc_tlb_inval.h" 17 14 #include "xe_mmio.h" ··· 91 94 xe_tlb_inval_fence_signal(fence); 92 95 } 93 96 if (!list_empty(&tlb_inval->pending_fences)) 94 94 - queue_delayed_work(system_wq, &tlb_inval->fence_tdr, 97 97 + queue_delayed_work(tlb_inval->timeout_wq, &tlb_inval->fence_tdr, 95 98 timeout_delay); 96 99 spin_unlock_irq(&tlb_inval->pending_lock); 97 100 } ··· 142 145 WQ_MEM_RECLAIM); 143 146 if (IS_ERR(tlb_inval->job_wq)) 144 147 return PTR_ERR(tlb_inval->job_wq); 148 148 + 149 149 + tlb_inval->timeout_wq = gt->ordered_wq; 150 150 + if (IS_ERR(tlb_inval->timeout_wq)) 151 151 + return PTR_ERR(tlb_inval->timeout_wq); 145 152 146 153 /* XXX: Blindly setting up backend to GuC */ 147 154 xe_guc_tlb_inval_init_early(&gt->uc.guc, tlb_inval); ··· 241 240 list_add_tail(&fence->link, &tlb_inval->pending_fences); 242 241 243 242 if (list_is_singular(&tlb_inval->pending_fences)) 244 244 - queue_delayed_work(system_wq, &tlb_inval->fence_tdr, 243 243 + queue_delayed_work(tlb_inval->timeout_wq, &tlb_inval->fence_tdr, 245 244 tlb_inval->ops->timeout_delay(tlb_inval)); 246 245 spin_unlock_irq(&tlb_inval->pending_lock); 247 246 ··· 400 399 } 401 400 402 401 if (!list_empty(&tlb_inval->pending_fences)) 403 403 - mod_delayed_work(system_wq, 402 402 + mod_delayed_work(tlb_inval->timeout_wq, 404 403 &tlb_inval->fence_tdr, 405 404 tlb_inval->ops->timeout_delay(tlb_inval)); 406 405 else

+2

drivers/gpu/drm/xe/xe_tlb_inval_types.h

reviewed

··· 109 109 struct workqueue_struct *job_wq; 110 110 /** @tlb_inval.lock: protects TLB invalidation fences */ 111 111 spinlock_t lock; 112 112 + /** @timeout_wq: schedules TLB invalidation fence timeouts */ 113 113 + struct workqueue_struct *timeout_wq; 112 114 }; 113 115 114 116 /**

-5

drivers/gpu/drm/xe/xe_trace.h

reviewed

··· 228 228 TP_ARGS(q) 229 229 ); 230 230 231 231 - DEFINE_EVENT(xe_exec_queue, xe_exec_queue_lr_cleanup, 232 232 - TP_PROTO(struct xe_exec_queue *q), 233 233 - TP_ARGS(q) 234 234 - ); 235 235 - 236 231 DECLARE_EVENT_CLASS(xe_sched_job, 237 232 TP_PROTO(struct xe_sched_job *job), 238 233 TP_ARGS(job),

-1

drivers/gpu/drm/xe/xe_ttm_stolen_mgr.c

reviewed

··· 17 17 #include "regs/xe_regs.h" 18 18 #include "xe_bo.h" 19 19 #include "xe_device.h" 20 20 - #include "xe_gt.h" 21 20 #include "xe_gt_printk.h" 22 21 #include "xe_mmio.h" 23 22 #include "xe_res_cursor.h"

-1

drivers/gpu/drm/xe/xe_ttm_sys_mgr.c

reviewed

··· 13 13 #include <drm/ttm/ttm_tt.h> 14 14 15 15 #include "xe_bo.h" 16 16 - #include "xe_gt.h" 17 16 18 17 struct xe_ttm_sys_node { 19 18 struct ttm_buffer_object *tbo;

-1

drivers/gpu/drm/xe/xe_ttm_vram_mgr.c

reviewed

··· 12 12 13 13 #include "xe_bo.h" 14 14 #include "xe_device.h" 15 15 - #include "xe_gt.h" 16 15 #include "xe_res_cursor.h" 17 16 #include "xe_ttm_vram_mgr.h" 18 17 #include "xe_vram_types.h"

-2

drivers/gpu/drm/xe/xe_uc.c

reviewed

··· 8 8 #include "xe_assert.h" 9 9 #include "xe_device.h" 10 10 #include "xe_gsc.h" 11 11 - #include "xe_gsc_proxy.h" 12 11 #include "xe_gt.h" 13 12 #include "xe_gt_printk.h" 14 13 #include "xe_gt_sriov_vf.h" ··· 16 17 #include "xe_guc_engine_activity.h" 17 18 #include "xe_huc.h" 18 19 #include "xe_sriov.h" 19 19 - #include "xe_uc_fw.h" 20 20 #include "xe_wopcm.h" 21 21 22 22 static struct xe_gt *

+1 -1

drivers/gpu/drm/xe/xe_uc_debugfs.c

reviewed

··· 7 7 8 8 #include <drm/drm_debugfs.h> 9 9 10 10 - #include "xe_gt.h" 11 10 #include "xe_gsc_debugfs.h" 12 11 #include "xe_guc_debugfs.h" 13 12 #include "xe_huc_debugfs.h" 14 13 #include "xe_macros.h" 15 14 #include "xe_uc_debugfs.h" 15 15 + #include "xe_uc_types.h" 16 16 17 17 void xe_uc_debugfs_register(struct xe_uc *uc, struct dentry *parent) 18 18 {

+10 -4

drivers/gpu/drm/xe/xe_uc_fw.c

reviewed

··· 14 14 #include "xe_device_types.h" 15 15 #include "xe_force_wake.h" 16 16 #include "xe_gsc.h" 17 17 - #include "xe_gt.h" 18 17 #include "xe_gt_printk.h" 19 18 #include "xe_gt_sriov_vf.h" 19 19 + #include "xe_gt_types.h" 20 20 #include "xe_guc.h" 21 21 #include "xe_map.h" 22 22 #include "xe_mmio.h" ··· 115 115 #define XE_GT_TYPE_ANY XE_GT_TYPE_UNINITIALIZED 116 116 117 117 #define XE_GUC_FIRMWARE_DEFS(fw_def, mmp_ver, major_ver) \ 118 118 + fw_def(NOVALAKE_S, GT_TYPE_ANY, mmp_ver(xe, guc, nvl, 70, 55, 4)) \ 118 119 fw_def(PANTHERLAKE, GT_TYPE_ANY, major_ver(xe, guc, ptl, 70, 54, 0)) \ 119 120 fw_def(BATTLEMAGE, GT_TYPE_ANY, major_ver(xe, guc, bmg, 70, 54, 0)) \ 120 121 fw_def(LUNARLAKE, GT_TYPE_ANY, major_ver(xe, guc, lnl, 70, 53, 0)) \ ··· 141 140 142 141 /* for the GSC FW we match the compatibility version and not the release one */ 143 142 #define XE_GSC_FIRMWARE_DEFS(fw_def, major_ver) \ 143 143 + fw_def(PANTHERLAKE, GT_TYPE_ANY, major_ver(xe, gsc, ptl, 105, 1, 0)) \ 144 144 fw_def(LUNARLAKE, GT_TYPE_ANY, major_ver(xe, gsc, lnl, 104, 1, 0)) \ 145 145 fw_def(METEORLAKE, GT_TYPE_ANY, major_ver(i915, gsc, mtl, 102, 1, 0)) 146 146 ··· 740 738 return 0; 741 739 } 742 740 743 743 - err = request_firmware(&fw, uc_fw->path, dev); 741 741 + err = firmware_request_nowarn(&fw, uc_fw->path, dev); 744 742 if (err) 745 743 goto fail; 746 744 ··· 769 767 XE_UC_FIRMWARE_MISSING : 770 768 XE_UC_FIRMWARE_ERROR); 771 769 772 772 - xe_gt_notice(gt, "%s firmware %s: fetch failed with error %pe\n", 773 773 - xe_uc_fw_type_repr(uc_fw->type), uc_fw->path, ERR_PTR(err)); 770 770 + if (err == -ENOENT) 771 771 + xe_gt_info(gt, "%s firmware %s not found\n", 772 772 + xe_uc_fw_type_repr(uc_fw->type), uc_fw->path); 773 773 + else 774 774 + xe_gt_notice(gt, "%s firmware %s: fetch failed with error %pe\n", 775 775 + xe_uc_fw_type_repr(uc_fw->type), uc_fw->path, ERR_PTR(err)); 774 776 xe_gt_info(gt, "%s firmware(s) can be downloaded from %s\n", 775 777 xe_uc_fw_type_repr(uc_fw->type), XE_UC_FIRMWARE_URL); 776 778

-1

drivers/gpu/drm/xe/xe_validation.c

reviewed

··· 2 2 /* 3 3 * Copyright © 2024 Intel Corporation 4 4 */ 5 5 - #include "xe_bo.h" 6 5 #include <drm/drm_exec.h> 7 6 #include <drm/drm_gem.h> 8 7 #include <drm/drm_gpuvm.h>

+20 -8

drivers/gpu/drm/xe/xe_vm.c

reviewed

··· 33 33 #include "xe_preempt_fence.h" 34 34 #include "xe_pt.h" 35 35 #include "xe_pxp.h" 36 36 - #include "xe_res_cursor.h" 37 36 #include "xe_sriov_vf.h" 38 37 #include "xe_svm.h" 39 38 #include "xe_sync.h" ··· 2208 2209 (ULL)xe_vma_start(vma), (ULL)xe_vma_size(vma)); 2209 2210 break; 2210 2211 default: 2211 2211 - drm_warn(&xe->drm, "NOT POSSIBLE"); 2212 2212 + drm_warn(&xe->drm, "NOT POSSIBLE\n"); 2212 2213 } 2213 2214 } 2214 2215 #else ··· 2311 2312 xe_bo_unlock(bo); 2312 2313 break; 2313 2314 default: 2314 2314 - drm_warn(&vm->xe->drm, "NOT POSSIBLE"); 2315 2315 + drm_warn(&vm->xe->drm, "NOT POSSIBLE\n"); 2315 2316 ops = ERR_PTR(-EINVAL); 2316 2317 } 2317 2318 if (IS_ERR(ops)) ··· 2583 2584 op->flags |= XE_VMA_OP_COMMITTED; 2584 2585 break; 2585 2586 default: 2586 2586 - drm_warn(&vm->xe->drm, "NOT POSSIBLE"); 2587 2587 + drm_warn(&vm->xe->drm, "NOT POSSIBLE\n"); 2587 2588 } 2588 2589 2589 2590 return err; ··· 2782 2783 2783 2784 break; 2784 2785 default: 2785 2785 - drm_warn(&vm->xe->drm, "NOT POSSIBLE"); 2786 2786 + drm_warn(&vm->xe->drm, "NOT POSSIBLE\n"); 2786 2787 } 2787 2788 2788 2789 err = xe_vma_op_commit(vm, op); ··· 2844 2845 /* Nothing to do */ 2845 2846 break; 2846 2847 default: 2847 2847 - drm_warn(&vm->xe->drm, "NOT POSSIBLE"); 2848 2848 + drm_warn(&vm->xe->drm, "NOT POSSIBLE\n"); 2848 2849 } 2849 2850 } 2850 2851 ··· 3028 3029 break; 3029 3030 } 3030 3031 default: 3031 3031 - drm_warn(&vm->xe->drm, "NOT POSSIBLE"); 3032 3032 + drm_warn(&vm->xe->drm, "NOT POSSIBLE\n"); 3032 3033 } 3033 3034 3034 3035 return err; ··· 3267 3268 vma_add_ufence(gpuva_to_vma(op->base.prefetch.va), ufence); 3268 3269 break; 3269 3270 default: 3270 3270 - drm_warn(&vm->xe->drm, "NOT POSSIBLE"); 3271 3271 + drm_warn(&vm->xe->drm, "NOT POSSIBLE\n"); 3271 3272 } 3272 3273 } 3273 3274 ··· 3404 3405 DRM_XE_VM_BIND_FLAG_CPU_ADDR_MIRROR; 3405 3406 u16 pat_index = (*bind_ops)[i].pat_index; 3406 3407 u16 coh_mode; 3408 3408 + bool comp_en; 3407 3409 3408 3410 if (XE_IOCTL_DBG(xe, is_cpu_addr_mirror && 3409 3411 (!xe_vm_in_fault_mode(vm) || ··· 3421 3421 pat_index = array_index_nospec(pat_index, xe->pat.n_entries); 3422 3422 (*bind_ops)[i].pat_index = pat_index; 3423 3423 coh_mode = xe_pat_index_get_coh_mode(xe, pat_index); 3424 3424 + comp_en = xe_pat_index_get_comp_en(xe, pat_index); 3424 3425 if (XE_IOCTL_DBG(xe, !coh_mode)) { /* hw reserved */ 3425 3426 err = -EINVAL; 3426 3427 goto free_bind_ops; ··· 3451 3450 XE_IOCTL_DBG(xe, obj && 3452 3451 op == DRM_XE_VM_BIND_OP_MAP_USERPTR) || 3453 3452 XE_IOCTL_DBG(xe, coh_mode == XE_COH_NONE && 3453 3453 + op == DRM_XE_VM_BIND_OP_MAP_USERPTR) || 3454 3454 + XE_IOCTL_DBG(xe, comp_en && 3454 3455 op == DRM_XE_VM_BIND_OP_MAP_USERPTR) || 3455 3456 XE_IOCTL_DBG(xe, op == DRM_XE_VM_BIND_OP_MAP_USERPTR && 3456 3457 !IS_ENABLED(CONFIG_DRM_GPUSVM)) || ··· 3532 3529 u16 pat_index, u32 op, u32 bind_flags) 3533 3530 { 3534 3531 u16 coh_mode; 3532 3532 + bool comp_en; 3535 3533 3536 3534 if (XE_IOCTL_DBG(xe, (bo->flags & XE_BO_FLAG_NO_COMPRESSION) && 3537 3535 xe_pat_index_get_comp_en(xe, pat_index))) ··· 3577 3573 */ 3578 3574 return -EINVAL; 3579 3575 } 3576 3576 + 3577 3577 + /* 3578 3578 + * Ensures that imported buffer objects (dma-bufs) are not mapped 3579 3579 + * with a PAT index that enables compression. 3580 3580 + */ 3581 3581 + comp_en = xe_pat_index_get_comp_en(xe, pat_index); 3582 3582 + if (XE_IOCTL_DBG(xe, bo->ttm.base.import_attach && comp_en)) 3583 3583 + return -EINVAL; 3580 3584 3581 3585 /* If a BO is protected it can only be mapped if the key is still valid */ 3582 3586 if ((bind_flags & DRM_XE_VM_BIND_FLAG_CHECK_PXP) && xe_bo_is_protected(bo) &&

+1 -1

drivers/gpu/drm/xe/xe_vm.h

reviewed

··· 382 382 } 383 383 384 384 /** 385 385 - * xe_vm_set_validation_exec() - Accessor to read the drm_exec object 385 385 + * xe_vm_validation_exec() - Accessor to read the drm_exec object 386 386 * @vm: The vm we want to register a drm_exec object with. 387 387 * 388 388 * Return: The drm_exec object used to lock the vm's resv. The value

+1 -3

drivers/gpu/drm/xe/xe_vram.c

reviewed

··· 13 13 #include "regs/xe_gt_regs.h" 14 14 #include "regs/xe_regs.h" 15 15 #include "xe_assert.h" 16 16 - #include "xe_bo.h" 17 16 #include "xe_device.h" 18 17 #include "xe_force_wake.h" 19 18 #include "xe_gt_mcr.h" 20 19 #include "xe_mmio.h" 21 21 - #include "xe_module.h" 22 20 #include "xe_sriov.h" 23 21 #include "xe_tile_sriov_vf.h" 24 22 #include "xe_ttm_vram_mgr.h" ··· 153 155 *tile_offset = 0; 154 156 } else { 155 157 reg = xe_mmio_read32(&tile->mmio, SG_TILE_ADDR_RANGE(tile->id)); 156 156 - *tile_size = (u64)REG_FIELD_GET(GENMASK(14, 8), reg) * SZ_1G; 158 158 + *tile_size = (u64)REG_FIELD_GET(GENMASK(17, 8), reg) * SZ_1G; 157 159 *tile_offset = (u64)REG_FIELD_GET(GENMASK(7, 1), reg) * SZ_1G; 158 160 } 159 161

-1

drivers/gpu/drm/xe/xe_vram_freq.c

reviewed

··· 5 5 #include <linux/sysfs.h> 6 6 #include <drm/drm_managed.h> 7 7 8 8 - #include "xe_gt_types.h" 9 8 #include "xe_pcode.h" 10 9 #include "xe_pcode_api.h" 11 10 #include "xe_tile.h"

-1

drivers/gpu/drm/xe/xe_vsec.c

reviewed

··· 12 12 13 13 #include "xe_device.h" 14 14 #include "xe_device_types.h" 15 15 - #include "xe_drv.h" 16 15 #include "xe_mmio.h" 17 16 #include "xe_platform_types.h" 18 17 #include "xe_pm.h"

+1 -1

drivers/gpu/drm/xe/xe_wa.c

reviewed

··· 19 19 #include "regs/xe_regs.h" 20 20 #include "xe_device_types.h" 21 21 #include "xe_force_wake.h" 22 22 - #include "xe_gt.h" 22 22 + #include "xe_gt_types.h" 23 23 #include "xe_hw_engine_types.h" 24 24 #include "xe_mmio.h" 25 25 #include "xe_platform_types.h"

-1

drivers/gpu/drm/xe/xe_wait_user_fence.c

reviewed

··· 11 11 #include <uapi/drm/xe_drm.h> 12 12 13 13 #include "xe_device.h" 14 14 - #include "xe_gt.h" 15 14 #include "xe_macros.h" 16 15 #include "xe_exec_queue.h" 17 16

+1 -1

drivers/gpu/drm/xe/xe_wopcm.c

reviewed

··· 10 10 #include "regs/xe_guc_regs.h" 11 11 #include "xe_device.h" 12 12 #include "xe_force_wake.h" 13 13 - #include "xe_gt.h" 13 13 + #include "xe_gt_types.h" 14 14 #include "xe_mmio.h" 15 15 #include "xe_uc_fw.h" 16 16

-4

include/uapi/drm/xe_drm.h

reviewed

··· 1280 1280 * then a new multi-queue group is created with this queue as the primary queue 1281 1281 * (Q0). Otherwise, the queue gets added to the multi-queue group whose primary 1282 1282 * queue's exec_queue_id is specified in the lower 32 bits of the 'value' field. 1283 1283 - * If the extension's 'value' field has %DRM_XE_MULTI_GROUP_KEEP_ACTIVE flag 1284 1284 - * set, then the multi-queue group is kept active after the primary queue is 1285 1285 - * destroyed. 1286 1283 * All the other non-relevant bits of extension's 'value' field while adding the 1287 1284 * primary or the secondary queues of the group must be set to 0. 1288 1285 * - %DRM_XE_EXEC_QUEUE_SET_PROPERTY_MULTI_QUEUE_PRIORITY - Set the queue ··· 1328 1331 #define DRM_XE_EXEC_QUEUE_SET_HANG_REPLAY_STATE 3 1329 1332 #define DRM_XE_EXEC_QUEUE_SET_PROPERTY_MULTI_GROUP 4 1330 1333 #define DRM_XE_MULTI_GROUP_CREATE (1ull << 63) 1331 1331 - #define DRM_XE_MULTI_GROUP_KEEP_ACTIVE (1ull << 62) 1332 1334 #define DRM_XE_EXEC_QUEUE_SET_PROPERTY_MULTI_QUEUE_PRIORITY 5 1333 1335 /** @extensions: Pointer to the first extension struct, if any */ 1334 1336 __u64 extensions;