Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
1.. SPDX-License-Identifier: GPL-2.0
2
3===========================================
4Userspace block device driver (ublk driver)
5===========================================
6
7Overview
8========
9
10ublk is a generic framework for implementing block device logic from userspace.
11The motivation behind it is that moving virtual block drivers into userspace,
12such as loop, nbd and similar can be very helpful. It can help to implement
13new virtual block device such as ublk-qcow2 (there are several attempts of
14implementing qcow2 driver in kernel).
15
16Userspace block devices are attractive because:
17
18- They can be written many programming languages.
19- They can use libraries that are not available in the kernel.
20- They can be debugged with tools familiar to application developers.
21- Crashes do not kernel panic the machine.
22- Bugs are likely to have a lower security impact than bugs in kernel
23 code.
24- They can be installed and updated independently of the kernel.
25- They can be used to simulate block device easily with user specified
26 parameters/setting for test/debug purpose
27
28ublk block device (``/dev/ublkb*``) is added by ublk driver. Any IO request
29on the device will be forwarded to ublk userspace program. For convenience,
30in this document, ``ublk server`` refers to generic ublk userspace
31program. ``ublksrv`` [#userspace]_ is one of such implementation. It
32provides ``libublksrv`` [#userspace_lib]_ library for developing specific
33user block device conveniently, while also generic type block device is
34included, such as loop and null. Richard W.M. Jones wrote userspace nbd device
35``nbdublk`` [#userspace_nbdublk]_ based on ``libublksrv`` [#userspace_lib]_.
36
37After the IO is handled by userspace, the result is committed back to the
38driver, thus completing the request cycle. This way, any specific IO handling
39logic is totally done by userspace, such as loop's IO handling, NBD's IO
40communication, or qcow2's IO mapping.
41
42``/dev/ublkb*`` is driven by blk-mq request-based driver. Each request is
43assigned by one queue wide unique tag. ublk server assigns unique tag to each
44IO too, which is 1:1 mapped with IO of ``/dev/ublkb*``.
45
46Both the IO request forward and IO handling result committing are done via
47``io_uring`` passthrough command; that is why ublk is also one io_uring based
48block driver. It has been observed that using io_uring passthrough command can
49give better IOPS than block IO; which is why ublk is one of high performance
50implementation of userspace block device: not only IO request communication is
51done by io_uring, but also the preferred IO handling in ublk server is io_uring
52based approach too.
53
54ublk provides control interface to set/get ublk block device parameters.
55The interface is extendable and kabi compatible: basically any ublk request
56queue's parameter or ublk generic feature parameters can be set/get via the
57interface. Thus, ublk is generic userspace block device framework.
58For example, it is easy to setup a ublk device with specified block
59parameters from userspace.
60
61Using ublk
62==========
63
64ublk requires userspace ublk server to handle real block device logic.
65
66Below is example of using ``ublksrv`` to provide ublk-based loop device.
67
68- add a device::
69
70 ublk add -t loop -f ublk-loop.img
71
72- format with xfs, then use it::
73
74 mkfs.xfs /dev/ublkb0
75 mount /dev/ublkb0 /mnt
76 # do anything. all IOs are handled by io_uring
77 ...
78 umount /mnt
79
80- list the devices with their info::
81
82 ublk list
83
84- delete the device::
85
86 ublk del -a
87 ublk del -n $ublk_dev_id
88
89See usage details in README of ``ublksrv`` [#userspace_readme]_.
90
91Design
92======
93
94Control plane
95-------------
96
97ublk driver provides global misc device node (``/dev/ublk-control``) for
98managing and controlling ublk devices with help of several control commands:
99
100- ``UBLK_CMD_ADD_DEV``
101
102 Add a ublk char device (``/dev/ublkc*``) which is talked with ublk server
103 WRT IO command communication. Basic device info is sent together with this
104 command. It sets UAPI structure of ``ublksrv_ctrl_dev_info``,
105 such as ``nr_hw_queues``, ``queue_depth``, and max IO request buffer size,
106 for which the info is negotiated with the driver and sent back to the server.
107 When this command is completed, the basic device info is immutable.
108
109- ``UBLK_CMD_SET_PARAMS`` / ``UBLK_CMD_GET_PARAMS``
110
111 Set or get parameters of the device, which can be either generic feature
112 related, or request queue limit related, but can't be IO logic specific,
113 because the driver does not handle any IO logic. This command has to be
114 sent before sending ``UBLK_CMD_START_DEV``.
115
116- ``UBLK_CMD_START_DEV``
117
118 After the server prepares userspace resources (such as creating I/O handler
119 threads & io_uring for handling ublk IO), this command is sent to the
120 driver for allocating & exposing ``/dev/ublkb*``. Parameters set via
121 ``UBLK_CMD_SET_PARAMS`` are applied for creating the device.
122
123- ``UBLK_CMD_STOP_DEV``
124
125 Halt IO on ``/dev/ublkb*`` and remove the device. When this command returns,
126 ublk server will release resources (such as destroying I/O handler threads &
127 io_uring).
128
129- ``UBLK_CMD_DEL_DEV``
130
131 Remove ``/dev/ublkc*``. When this command returns, the allocated ublk device
132 number can be reused.
133
134- ``UBLK_CMD_GET_QUEUE_AFFINITY``
135
136 When ``/dev/ublkc`` is added, the driver creates block layer tagset, so
137 that each queue's affinity info is available. The server sends
138 ``UBLK_CMD_GET_QUEUE_AFFINITY`` to retrieve queue affinity info. It can
139 set up the per-queue context efficiently, such as bind affine CPUs with IO
140 pthread and try to allocate buffers in IO thread context.
141
142- ``UBLK_CMD_GET_DEV_INFO``
143
144 For retrieving device info via ``ublksrv_ctrl_dev_info``. It is the server's
145 responsibility to save IO target specific info in userspace.
146
147- ``UBLK_CMD_GET_DEV_INFO2``
148 Same purpose with ``UBLK_CMD_GET_DEV_INFO``, but ublk server has to
149 provide path of the char device of ``/dev/ublkc*`` for kernel to run
150 permission check, and this command is added for supporting unprivileged
151 ublk device, and introduced with ``UBLK_F_UNPRIVILEGED_DEV`` together.
152 Only the user owning the requested device can retrieve the device info.
153
154 How to deal with userspace/kernel compatibility:
155
156 1) if kernel is capable of handling ``UBLK_F_UNPRIVILEGED_DEV``
157
158 If ublk server supports ``UBLK_F_UNPRIVILEGED_DEV``:
159
160 ublk server should send ``UBLK_CMD_GET_DEV_INFO2``, given anytime
161 unprivileged application needs to query devices the current user owns,
162 when the application has no idea if ``UBLK_F_UNPRIVILEGED_DEV`` is set
163 given the capability info is stateless, and application should always
164 retrieve it via ``UBLK_CMD_GET_DEV_INFO2``
165
166 If ublk server doesn't support ``UBLK_F_UNPRIVILEGED_DEV``:
167
168 ``UBLK_CMD_GET_DEV_INFO`` is always sent to kernel, and the feature of
169 UBLK_F_UNPRIVILEGED_DEV isn't available for user
170
171 2) if kernel isn't capable of handling ``UBLK_F_UNPRIVILEGED_DEV``
172
173 If ublk server supports ``UBLK_F_UNPRIVILEGED_DEV``:
174
175 ``UBLK_CMD_GET_DEV_INFO2`` is tried first, and will be failed, then
176 ``UBLK_CMD_GET_DEV_INFO`` needs to be retried given
177 ``UBLK_F_UNPRIVILEGED_DEV`` can't be set
178
179 If ublk server doesn't support ``UBLK_F_UNPRIVILEGED_DEV``:
180
181 ``UBLK_CMD_GET_DEV_INFO`` is always sent to kernel, and the feature of
182 ``UBLK_F_UNPRIVILEGED_DEV`` isn't available for user
183
184- ``UBLK_CMD_START_USER_RECOVERY``
185
186 This command is valid if ``UBLK_F_USER_RECOVERY`` feature is enabled. This
187 command is accepted after the old process has exited, ublk device is quiesced
188 and ``/dev/ublkc*`` is released. User should send this command before he starts
189 a new process which re-opens ``/dev/ublkc*``. When this command returns, the
190 ublk device is ready for the new process.
191
192- ``UBLK_CMD_END_USER_RECOVERY``
193
194 This command is valid if ``UBLK_F_USER_RECOVERY`` feature is enabled. This
195 command is accepted after ublk device is quiesced and a new process has
196 opened ``/dev/ublkc*`` and get all ublk queues be ready. When this command
197 returns, ublk device is unquiesced and new I/O requests are passed to the
198 new process.
199
200- user recovery feature description
201
202 Three new features are added for user recovery: ``UBLK_F_USER_RECOVERY``,
203 ``UBLK_F_USER_RECOVERY_REISSUE``, and ``UBLK_F_USER_RECOVERY_FAIL_IO``. To
204 enable recovery of ublk devices after the ublk server exits, the ublk server
205 should specify the ``UBLK_F_USER_RECOVERY`` flag when creating the device. The
206 ublk server may additionally specify at most one of
207 ``UBLK_F_USER_RECOVERY_REISSUE`` and ``UBLK_F_USER_RECOVERY_FAIL_IO`` to
208 modify how I/O is handled while the ublk server is dying/dead (this is called
209 the ``nosrv`` case in the driver code).
210
211 With just ``UBLK_F_USER_RECOVERY`` set, after the ublk server exits,
212 ublk does not delete ``/dev/ublkb*`` during the whole
213 recovery stage and ublk device ID is kept. It is ublk server's
214 responsibility to recover the device context by its own knowledge.
215 Requests which have not been issued to userspace are requeued. Requests
216 which have been issued to userspace are aborted.
217
218 With ``UBLK_F_USER_RECOVERY_REISSUE`` additionally set, after the ublk server
219 exits, contrary to ``UBLK_F_USER_RECOVERY``,
220 requests which have been issued to userspace are requeued and will be
221 re-issued to the new process after handling ``UBLK_CMD_END_USER_RECOVERY``.
222 ``UBLK_F_USER_RECOVERY_REISSUE`` is designed for backends who tolerate
223 double-write since the driver may issue the same I/O request twice. It
224 might be useful to a read-only FS or a VM backend.
225
226 With ``UBLK_F_USER_RECOVERY_FAIL_IO`` additionally set, after the ublk server
227 exits, requests which have issued to userspace are failed, as are any
228 subsequently issued requests. Applications continuously issuing I/O against
229 devices with this flag set will see a stream of I/O errors until a new ublk
230 server recovers the device.
231
232Unprivileged ublk device is supported by passing ``UBLK_F_UNPRIVILEGED_DEV``.
233Once the flag is set, all control commands can be sent by unprivileged
234user. Except for command of ``UBLK_CMD_ADD_DEV``, permission check on
235the specified char device(``/dev/ublkc*``) is done for all other control
236commands by ublk driver, for doing that, path of the char device has to
237be provided in these commands' payload from ublk server. With this way,
238ublk device becomes container-ware, and device created in one container
239can be controlled/accessed just inside this container.
240
241Data plane
242----------
243
244The ublk server should create dedicated threads for handling I/O. Each
245thread should have its own io_uring through which it is notified of new
246I/O, and through which it can complete I/O. These dedicated threads
247should focus on IO handling and shouldn't handle any control &
248management tasks.
249
250The's IO is assigned by a unique tag, which is 1:1 mapping with IO
251request of ``/dev/ublkb*``.
252
253UAPI structure of ``ublksrv_io_desc`` is defined for describing each IO from
254the driver. A fixed mmapped area (array) on ``/dev/ublkc*`` is provided for
255exporting IO info to the server; such as IO offset, length, OP/flags and
256buffer address. Each ``ublksrv_io_desc`` instance can be indexed via queue id
257and IO tag directly.
258
259The following IO commands are communicated via io_uring passthrough command,
260and each command is only for forwarding the IO and committing the result
261with specified IO tag in the command data:
262
263Traditional Per-I/O Commands
264~~~~~~~~~~~~~~~~~~~~~~~~~~~~
265
266- ``UBLK_U_IO_FETCH_REQ``
267
268 Sent from the server I/O pthread for fetching future incoming I/O requests
269 destined to ``/dev/ublkb*``. This command is sent only once from the server
270 IO pthread for ublk driver to setup IO forward environment.
271
272 Once a thread issues this command against a given (qid,tag) pair, the thread
273 registers itself as that I/O's daemon. In the future, only that I/O's daemon
274 is allowed to issue commands against the I/O. If any other thread attempts
275 to issue a command against a (qid,tag) pair for which the thread is not the
276 daemon, the command will fail. Daemons can be reset only be going through
277 recovery.
278
279 The ability for every (qid,tag) pair to have its own independent daemon task
280 is indicated by the ``UBLK_F_PER_IO_DAEMON`` feature. If this feature is not
281 supported by the driver, daemons must be per-queue instead - i.e. all I/Os
282 associated to a single qid must be handled by the same task.
283
284- ``UBLK_U_IO_COMMIT_AND_FETCH_REQ``
285
286 When an IO request is destined to ``/dev/ublkb*``, the driver stores
287 the IO's ``ublksrv_io_desc`` to the specified mapped area; then the
288 previous received IO command of this IO tag (either ``UBLK_IO_FETCH_REQ``
289 or ``UBLK_IO_COMMIT_AND_FETCH_REQ)`` is completed, so the server gets
290 the IO notification via io_uring.
291
292 After the server handles the IO, its result is committed back to the
293 driver by sending ``UBLK_IO_COMMIT_AND_FETCH_REQ`` back. Once ublkdrv
294 received this command, it parses the result and complete the request to
295 ``/dev/ublkb*``. In the meantime setup environment for fetching future
296 requests with the same IO tag. That is, ``UBLK_IO_COMMIT_AND_FETCH_REQ``
297 is reused for both fetching request and committing back IO result.
298
299- ``UBLK_U_IO_NEED_GET_DATA``
300
301 With ``UBLK_F_NEED_GET_DATA`` enabled, the WRITE request will be firstly
302 issued to ublk server without data copy. Then, IO backend of ublk server
303 receives the request and it can allocate data buffer and embed its addr
304 inside this new io command. After the kernel driver gets the command,
305 data copy is done from request pages to this backend's buffer. Finally,
306 backend receives the request again with data to be written and it can
307 truly handle the request.
308
309 ``UBLK_IO_NEED_GET_DATA`` adds one additional round-trip and one
310 io_uring_enter() syscall. Any user thinks that it may lower performance
311 should not enable UBLK_F_NEED_GET_DATA. ublk server pre-allocates IO
312 buffer for each IO by default. Any new project should try to use this
313 buffer to communicate with ublk driver. However, existing project may
314 break or not able to consume the new buffer interface; that's why this
315 command is added for backwards compatibility so that existing projects
316 can still consume existing buffers.
317
318- data copy between ublk server IO buffer and ublk block IO request
319
320 The driver needs to copy the block IO request pages into the server buffer
321 (pages) first for WRITE before notifying the server of the coming IO, so
322 that the server can handle WRITE request.
323
324 When the server handles READ request and sends
325 ``UBLK_IO_COMMIT_AND_FETCH_REQ`` to the server, ublkdrv needs to copy
326 the server buffer (pages) read to the IO request pages.
327
328Batch I/O Commands (UBLK_F_BATCH_IO)
329~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
330
331The ``UBLK_F_BATCH_IO`` feature provides an alternative high-performance
332I/O handling model that replaces the traditional per-I/O commands with
333per-queue batch commands. This significantly reduces communication overhead
334and enables better load balancing across multiple server tasks.
335
336Key differences from traditional mode:
337
338- **Per-queue vs Per-I/O**: Commands operate on queues rather than individual I/Os
339- **Batch processing**: Multiple I/Os are handled in single operations
340- **Multishot commands**: Use io_uring multishot for reduced submission overhead
341- **Flexible task assignment**: Any task can handle any I/O (no per-I/O daemons)
342- **Better load balancing**: Tasks can adjust their workload dynamically
343
344Batch I/O Commands:
345
346- ``UBLK_U_IO_PREP_IO_CMDS``
347
348 Prepares multiple I/O commands in batch. The server provides a buffer
349 containing multiple I/O descriptors that will be processed together.
350 This reduces the number of individual command submissions required.
351
352- ``UBLK_U_IO_COMMIT_IO_CMDS``
353
354 Commits results for multiple I/O operations in batch, and prepares the
355 I/O descriptors to accept new requests. The server provides a buffer
356 containing the results of multiple completed I/Os, allowing efficient
357 bulk completion of requests.
358
359- ``UBLK_U_IO_FETCH_IO_CMDS``
360
361 **Multishot command** for fetching I/O commands in batch. This is the key
362 command that enables high-performance batch processing:
363
364 * Uses io_uring multishot capability for reduced submission overhead
365 * Single command can fetch multiple I/O requests over time
366 * Buffer size determines maximum batch size per operation
367 * Multiple fetch commands can be submitted for load balancing
368 * Only one fetch command is active at any time per queue
369 * Supports dynamic load balancing across multiple server tasks
370
371 It is one typical multishot io_uring request with provided buffer, and it
372 won't be completed until any failure is triggered.
373
374 Each task can submit ``UBLK_U_IO_FETCH_IO_CMDS`` with different buffer
375 sizes to control how much work it handles. This enables sophisticated
376 load balancing strategies in multi-threaded servers.
377
378Migration: Applications using traditional commands (``UBLK_U_IO_FETCH_REQ``,
379``UBLK_U_IO_COMMIT_AND_FETCH_REQ``) cannot use batch mode simultaneously.
380
381Zero copy
382---------
383
384ublk zero copy relies on io_uring's fixed kernel buffer, which provides
385two APIs: `io_buffer_register_bvec()` and `io_buffer_unregister_bvec`.
386
387ublk adds IO command of `UBLK_IO_REGISTER_IO_BUF` to call
388`io_buffer_register_bvec()` for ublk server to register client request
389buffer into io_uring buffer table, then ublk server can submit io_uring
390IOs with the registered buffer index. IO command of `UBLK_IO_UNREGISTER_IO_BUF`
391calls `io_buffer_unregister_bvec()` to unregister the buffer, which is
392guaranteed to be live between calling `io_buffer_register_bvec()` and
393`io_buffer_unregister_bvec()`. Any io_uring operation which supports this
394kind of kernel buffer will grab one reference of the buffer until the
395operation is completed.
396
397ublk server implementing zero copy or user copy has to be CAP_SYS_ADMIN and
398be trusted, because it is ublk server's responsibility to make sure IO buffer
399filled with data for handling read command, and ublk server has to return
400correct result to ublk driver when handling READ command, and the result
401has to match with how many bytes filled to the IO buffer. Otherwise,
402uninitialized kernel IO buffer will be exposed to client application.
403
404ublk server needs to align the parameter of `struct ublk_param_dma_align`
405with backend for zero copy to work correctly.
406
407For reaching best IO performance, ublk server should align its segment
408parameter of `struct ublk_param_segment` with backend for avoiding
409unnecessary IO split, which usually hurts io_uring performance.
410
411Auto Buffer Registration
412------------------------
413
414The ``UBLK_F_AUTO_BUF_REG`` feature automatically handles buffer registration
415and unregistration for I/O requests, which simplifies the buffer management
416process and reduces overhead in the ublk server implementation.
417
418This is another feature flag for using zero copy, and it is compatible with
419``UBLK_F_SUPPORT_ZERO_COPY``.
420
421Feature Overview
422~~~~~~~~~~~~~~~~
423
424This feature automatically registers request buffers to the io_uring context
425before delivering I/O commands to the ublk server and unregisters them when
426completing I/O commands. This eliminates the need for manual buffer
427registration/unregistration via ``UBLK_IO_REGISTER_IO_BUF`` and
428``UBLK_IO_UNREGISTER_IO_BUF`` commands, then IO handling in ublk server
429can avoid dependency on the two uring_cmd operations.
430
431IOs can't be issued concurrently to io_uring if there is any dependency
432among these IOs. So this way not only simplifies ublk server implementation,
433but also makes concurrent IO handling becomes possible by removing the
434dependency on buffer registration & unregistration commands.
435
436Usage Requirements
437~~~~~~~~~~~~~~~~~~
438
4391. The ublk server must create a sparse buffer table on the same ``io_ring_ctx``
440 used for ``UBLK_IO_FETCH_REQ`` and ``UBLK_IO_COMMIT_AND_FETCH_REQ``. If
441 uring_cmd is issued on a different ``io_ring_ctx``, manual buffer
442 unregistration is required.
443
4442. Buffer registration data must be passed via uring_cmd's ``sqe->addr`` with the
445 following structure::
446
447 struct ublk_auto_buf_reg {
448 __u16 index; /* Buffer index for registration */
449 __u8 flags; /* Registration flags */
450 __u8 reserved0; /* Reserved for future use */
451 __u32 reserved1; /* Reserved for future use */
452 };
453
454 ublk_auto_buf_reg_to_sqe_addr() is for converting the above structure into
455 ``sqe->addr``.
456
4573. All reserved fields in ``ublk_auto_buf_reg`` must be zeroed.
458
4594. Optional flags can be passed via ``ublk_auto_buf_reg.flags``.
460
461Fallback Behavior
462~~~~~~~~~~~~~~~~~
463
464If auto buffer registration fails:
465
4661. When ``UBLK_AUTO_BUF_REG_FALLBACK`` is enabled:
467
468 - The uring_cmd is completed
469 - ``UBLK_IO_F_NEED_REG_BUF`` is set in ``ublksrv_io_desc.op_flags``
470 - The ublk server must manually deal with the failure, such as, register
471 the buffer manually, or using user copy feature for retrieving the data
472 for handling ublk IO
473
4742. If fallback is not enabled:
475
476 - The ublk I/O request fails silently
477 - The uring_cmd won't be completed
478
479Limitations
480~~~~~~~~~~~
481
482- Requires same ``io_ring_ctx`` for all operations
483- May require manual buffer management in fallback cases
484- io_ring_ctx buffer table has a max size of 16K, which may not be enough
485 in case that too many ublk devices are handled by this single io_ring_ctx
486 and each one has very large queue depth
487
488Shared Memory Zero Copy (UBLK_F_SHMEM_ZC)
489------------------------------------------
490
491The ``UBLK_F_SHMEM_ZC`` feature provides an alternative zero-copy path
492that works by sharing physical memory pages between the client application
493and the ublk server. Unlike the io_uring fixed buffer approach above,
494shared memory zero copy does not require io_uring buffer registration
495per I/O — instead, it relies on the kernel matching physical pages
496at I/O time. This allows the ublk server to access the shared
497buffer directly, which is unlikely for the io_uring fixed buffer
498approach.
499
500Motivation
501~~~~~~~~~~
502
503Shared memory zero copy takes a different approach: if the client
504application and the ublk server both map the same physical memory, there is
505nothing to copy. The kernel detects the shared pages automatically and
506tells the server where the data already lives.
507
508``UBLK_F_SHMEM_ZC`` can be thought of as a supplement for optimized client
509applications — when the client is willing to allocate I/O buffers from
510shared memory, the entire data path becomes zero-copy.
511
512Use Cases
513~~~~~~~~~
514
515This feature is useful when the client application can be configured to
516use a specific shared memory region for its I/O buffers:
517
518- **Custom storage clients** that allocate I/O buffers from shared memory
519 (memfd, hugetlbfs) and issue direct I/O to the ublk device
520- **Database engines** that use pre-allocated buffer pools with O_DIRECT
521
522How It Works
523~~~~~~~~~~~~
524
5251. The ublk server and client both ``mmap()`` the same file (memfd or
526 hugetlbfs) with ``MAP_SHARED``. This gives both processes access to the
527 same physical pages.
528
5292. The ublk server registers its mapping with the kernel::
530
531 struct ublk_shmem_buf_reg buf = { .addr = mmap_va, .len = size };
532 ublk_ctrl_cmd(UBLK_U_CMD_REG_BUF, .addr = &buf);
533
534 The kernel pins the pages and builds a PFN lookup tree.
535
5363. When the client issues direct I/O (``O_DIRECT``) to ``/dev/ublkb*``,
537 the kernel checks whether the I/O buffer pages match any registered
538 pages by comparing PFNs.
539
5404. On a match, the kernel sets ``UBLK_IO_F_SHMEM_ZC`` in the I/O
541 descriptor and encodes the buffer index and offset in ``addr``::
542
543 if (iod->op_flags & UBLK_IO_F_SHMEM_ZC) {
544 /* Data is already in our shared mapping — zero copy */
545 index = ublk_shmem_zc_index(iod->addr);
546 offset = ublk_shmem_zc_offset(iod->addr);
547 buf = shmem_table[index].mmap_base + offset;
548 }
549
5505. If pages do not match (e.g., the client used a non-shared buffer),
551 the I/O falls back to the normal copy path silently.
552
553The shared memory can be set up via two methods:
554
555- **Socket-based**: the client sends a memfd to the ublk server via
556 ``SCM_RIGHTS`` on a unix socket. The server mmaps and registers it.
557- **Hugetlbfs-based**: both processes ``mmap(MAP_SHARED)`` the same
558 hugetlbfs file. No IPC needed — same file gives same physical pages.
559
560Advantages
561~~~~~~~~~~
562
563- **Simple**: no per-I/O buffer registration or unregistration commands.
564 Once the shared buffer is registered, all matching I/O is zero-copy
565 automatically.
566- **Direct buffer access**: the ublk server can read and write the shared
567 buffer directly via its own mmap, without going through io_uring fixed
568 buffer operations. This is more friendly for server implementations.
569- **Fast**: PFN matching is a single maple tree lookup per bvec. No
570 io_uring command round-trips for buffer management.
571- **Compatible**: non-matching I/O silently falls back to the copy path.
572 The device works normally for any client, with zero-copy as an
573 optimization when shared memory is available.
574
575Limitations
576~~~~~~~~~~~
577
578- **Requires client cooperation**: the client must allocate its I/O
579 buffers from the shared memory region. This requires a custom or
580 configured client — standard applications using their own buffers
581 will not benefit.
582- **Direct I/O only**: buffered I/O (without ``O_DIRECT``) goes through
583 the page cache, which allocates its own pages. These kernel-allocated
584 pages will never match the registered shared buffer. Only ``O_DIRECT``
585 puts the client's buffer pages directly into the block I/O.
586- **Contiguous data only**: each I/O request's data must be contiguous
587 within a single registered buffer. Scatter/gather I/O that spans
588 multiple non-adjacent registered buffers cannot use the zero-copy path.
589
590Control Commands
591~~~~~~~~~~~~~~~~
592
593- ``UBLK_U_CMD_REG_BUF``
594
595 Register a shared memory buffer. ``ctrl_cmd.addr`` points to a
596 ``struct ublk_shmem_buf_reg`` containing the buffer virtual address and size.
597 Returns the assigned buffer index (>= 0) on success. The kernel pins
598 pages and builds the PFN lookup tree. Queue freeze is handled
599 internally.
600
601- ``UBLK_U_CMD_UNREG_BUF``
602
603 Unregister a previously registered buffer. ``ctrl_cmd.data[0]`` is the
604 buffer index. Unpins pages and removes PFN entries from the lookup
605 tree.
606
607References
608==========
609
610.. [#userspace] https://github.com/ming1/ubdsrv
611
612.. [#userspace_lib] https://github.com/ming1/ubdsrv/tree/master/lib
613
614.. [#userspace_nbdublk] https://gitlab.com/rwmjones/libnbd/-/tree/nbdublk
615
616.. [#userspace_readme] https://github.com/ming1/ubdsrv/blob/master/README