The open source OpenXR runtime
1# Frame Pacing/Timing {#frame-pacing}
2
3<!--
4Copyright 2021, Collabora, Ltd. and the Monado contributors
5SPDX-License-Identifier: BSL-1.0
6-->
7
8A "brief" overview of the various time-points that a frame goes through, from
9when the application gets go ahead to render the frame to when pixels are turned
10into photons. This is only a single frame, where all of the timings are achieved and
11the application is single threaded. The HMD also only turns on the display
12during the vblank period, meaning the pixel to photon transformation is delayed
13from scanout starting to the vblank period (like for the Index).
14
15* `xrWaitFrame` returns to the application, referred to as **wake_up**.
16* The app does a logic step to move the simulation to the next predicted
17 display time.
18* `xrBeginFrame` is called by the application, referred to as **begin**.
19* The app renders the current views using the GPU.
20* `xrEndFrame` is called by the application submitting the views.
21* The compositor wakes up to do its distorting rendering and any warping,
22 checking if the applications rendering has finished. When the compositor
23 submits the work to the GPU, referred to as **submit**.
24* The compositor queues its swapbuffer to the display engine.
25* Scanout starts, the kernel checked if the compositors rendering was completed
26 in time. We refer to this time as **present**, this seems to be common.
27* During the vblank period the display lights up turning the pixels into
28 photons. We refer to this time as **display**, same as in OpenXR.
29
30The names for timepoints are chosen to align with the naming in
31[`VK_GOOGLE_display_timing`][], reading that extension can provide further
32information.
33
34## Main compositor perspective
35
36* @ref xrt_comp_wait_frame - It is within this function that the frame timing is
37 predicted, see @ref u_pa_predict and @ref u_pc_predict. The compositor will
38 then wait to **wake_up** time and then return from this function.
39* @ref xrt_comp_begin_frame - The app or IPC server calls this function when it
40 is done with CPU work and ready to do GPU work.
41* @ref xrt_comp_discard_frame - The frame is discarded.
42* @ref xrt_comp_layer_begin - Called during transfers of layers.
43* @ref xrt_comp_layer_projection - This and other layer functions are
44 called to list the layers the compositor should render.
45* @ref xrt_comp_layer_commit - The compositor starts to render the frame,
46 trying to finish at the **present** time.
47
48[`VK_GOOGLE_display_timing`]: https://www.khronos.org/registry/vulkan/specs/1.2-extensions/man/html/VK_GOOGLE_display_timing.html