Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

Merge tag 'rust-6.20-7.0' of git://git.kernel.org/pub/scm/linux/kernel/git/ojeda/linux

Pull rust updates from Miguel Ojeda:
"Toolchain and infrastructure:

- Add '__rust_helper' annotation to the C helpers

This is needed to inline these helpers into Rust code

- Remove imports available via the prelude, treewide

This was possible thanks to a new lint in Klint that Gary has
implemented -- more Klint-related changes, including initial
upstream support, are coming

- Deduplicate pin-init flags

'kernel' crate:

- Add support for calling a function exactly once with the new
'do_once_lite!' macro (and 'OnceLite' type)

Based on this, add 'pr_*_once!' macros to print only once

- Add 'impl_flags!' macro for defining common bitflags operations:

impl_flags!(
/// Represents multiple permissions.
#[derive(Debug, Clone, Default, Copy, PartialEq, Eq)]
pub struct Permissions(u32);

/// Represents a single permission.
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Permission {
/// Read permission.
Read = 1 << 0,

/// Write permission.
Write = 1 << 1,

/// Execute permission.
Execute = 1 << 2,
}
);

let mut f: Permissions = Permission::Read | Permission::Write;
assert!(f.contains(Permission::Read));
assert!(!f.contains(Permission::Execute));

f |= Permission::Execute;
assert!(f.contains(Permission::Execute));

let f2: Permissions = Permission::Write | Permission::Execute;
assert!((f ^ f2).contains(Permission::Read));
assert!(!(f ^ f2).contains(Permission::Write));

- 'bug' module: support 'CONFIG_DEBUG_BUGVERBOSE_DETAILED' in the
'warn_on!' macro in order to show the evaluated condition alongside
the file path:

------------[ cut here ]------------
WARNING: [val == 1] linux/samples/rust/rust_minimal.rs:27 at ...
Modules linked in: rust_minimal(+)

- Add safety module with 'unsafe_precondition_assert!' macro,
currently a wrapper for 'debug_assert!', intended to mark the
validation of safety preconditions where possible:

/// # Safety
///
/// The caller must ensure that `index` is less than `N`.
unsafe fn set_unchecked(&mut self, index: usize, value: T) {
unsafe_precondition_assert!(
index < N,
"set_unchecked() requires index ({index}) < N ({N})"
);

...
}

- Add instructions to 'build_assert!' documentation requesting to
always inline functions when used with function arguments

- 'ptr' module: replace 'build_assert!' with a 'const' one

- 'rbtree' module: reduce unsafe blocks on pointer derefs

- 'transmute' module: implement 'FromBytes' and 'AsBytes' for
inhabited ZSTs, and use it in Nova

- More treewide replacements of 'c_str!' with C string literals

'macros' crate:

- Rewrite most procedural macros ('module!', 'concat_idents!',
'#[export]', '#[vtable]', '#[kunit_tests]') to use the 'syn'
parsing library which we introduced last cycle, with better
diagnostics

This also allows to support '#[cfg]' properly in the '#[vtable]'
macro, to support arbitrary types in 'module!' macro (not just an
identifier) and to remove several custom parsing helpers we had

- Use 'quote!' from the recently vendored 'quote' library and remove
our custom one

The vendored one also allows us to avoid quoting '"' and '{}'
inside the template anymore and editors can now highlight it. In
addition, it improves robustness as it eliminates the need for
string quoting and escaping

- Use 'pin_init::zeroed()' to simplify KUnit code

'pin-init' crate:

- Rewrite all procedural macros ('[pin_]init!', '#[pin_data]',
'#[pinned_drop]', 'derive([Maybe]Zeroable)') to use the 'syn'
parsing library which we introduced last cycle, with better
diagnostics

- Implement 'InPlaceWrite' for '&'static mut MaybeUninit<T>'. This
enables users to use external allocation mechanisms such as
'static_cell'

- Support tuple structs in 'derive([Maybe]Zeroable)'

- Support attributes on fields in '[pin_]init!' (such as
'#[cfg(...)]')

- Add a '#[default_error(<type>)]' attribute to '[pin_]init!' to
override the default error (when no '? Error' is specified)

- Support packed structs in '[pin_]init!' with
'#[disable_initialized_field_access]'

- Remove 'try_[pin_]init!' in favor of merging their feature with
'[pin_]init!'. Update the kernel's own 'try_[pin_]init!' macros to
use the 'default_error' attribute

- Correct 'T: Sized' bounds to 'T: ?Sized' in the generated
'PinnedDrop' check by '#[pin_data]'

Documentation:

- Conclude the Rust experiment

MAINTAINERS:

- Add "RUST [RUST-ANALYZER]" entry for the rust-analyzer support.
Tamir and Jesung will take care of it. They have both been active
around it for a while. The new tree will flow through the Rust one

- Add Gary as maintainer for "RUST [PIN-INIT]"

- Update Boqun and Tamir emails to their kernel.org accounts

And a few other cleanups and improvements"

* tag 'rust-6.20-7.0' of git://git.kernel.org/pub/scm/linux/kernel/git/ojeda/linux: (59 commits)
rust: safety: introduce `unsafe_precondition_assert!` macro
rust: add `impl_flags!` macro for defining common bitflag operations
rust: print: Add pr_*_once macros
rust: bug: Support DEBUG_BUGVERBOSE_DETAILED option
rust: print: Add support for calling a function exactly once
rust: kbuild: deduplicate pin-init flags
gpu: nova-core: remove imports available via prelude
rust: clk: replace `kernel::c_str!` with C-Strings
MAINTAINERS: Update my email address to @kernel.org
rust: macros: support `#[cfg]` properly in `#[vtable]` macro.
rust: kunit: use `pin_init::zeroed` instead of custom null value
rust: macros: rearrange `#[doc(hidden)]` in `module!` macro
rust: macros: allow arbitrary types to be used in `module!` macro
rust: macros: convert `#[kunit_tests]` macro to use `syn`
rust: macros: convert `concat_idents!` to use `syn`
rust: macros: convert `#[export]` to use `syn`
rust: macros: use `quote!` for `module!` macro
rust: macros: use `syn` to parse `module!` macro
rust: macros: convert `#[vtable]` macro to use `syn`
rust: macros: use `quote!` from vendored crate
...

+2758 -3428
+2
.mailmap
··· 152 152 Björn Steinbrink <B.Steinbrink@gmx.de> 153 153 Björn Töpel <bjorn@kernel.org> <bjorn.topel@gmail.com> 154 154 Björn Töpel <bjorn@kernel.org> <bjorn.topel@intel.com> 155 + Boqun Feng <boqun@kernel.org> <boqun.feng@gmail.com> 155 156 Boris Brezillon <bbrezillon@kernel.org> <b.brezillon.dev@gmail.com> 156 157 Boris Brezillon <bbrezillon@kernel.org> <b.brezillon@overkiz.com> 157 158 Boris Brezillon <bbrezillon@kernel.org> <boris.brezillon@bootlin.com> ··· 801 800 Sven Peter <sven@kernel.org> <sven@svenpeter.dev> 802 801 Szymon Wilczek <swilczek.lx@gmail.com> <szymonwilczek@gmx.com> 803 802 Takashi YOSHII <takashi.yoshii.zj@renesas.com> 803 + Tamir Duberstein <tamird@kernel.org> <tamird@gmail.com> 804 804 Tamizh Chelvam Raja <quic_tamizhr@quicinc.com> <tamizhr@codeaurora.org> 805 805 Taniya Das <quic_tdas@quicinc.com> <tdas@codeaurora.org> 806 806 Tanzir Hasan <tanzhasanwork@gmail.com> <tanzirh@google.com>
+1 -1
Documentation/process/programming-language.rst
··· 34 34 Rust 35 35 ---- 36 36 37 - The kernel has experimental support for the Rust programming language 37 + The kernel has support for the Rust programming language 38 38 [rust-language]_ under ``CONFIG_RUST``. It is compiled with ``rustc`` [rustc]_ 39 39 under ``--edition=2021`` [rust-editions]_. Editions are a way to introduce 40 40 small changes to the language that are not backwards compatible.
-18
Documentation/rust/index.rst
··· 7 7 in the kernel, please read the quick-start.rst guide. 8 8 9 9 10 - The Rust experiment 11 - ------------------- 12 - 13 - The Rust support was merged in v6.1 into mainline in order to help in 14 - determining whether Rust as a language was suitable for the kernel, i.e. worth 15 - the tradeoffs. 16 - 17 - Currently, the Rust support is primarily intended for kernel developers and 18 - maintainers interested in the Rust support, so that they can start working on 19 - abstractions and drivers, as well as helping the development of infrastructure 20 - and tools. 21 - 22 - If you are an end user, please note that there are currently no in-tree 23 - drivers/modules suitable or intended for production use, and that the Rust 24 - support is still in development/experimental, especially for certain kernel 25 - configurations. 26 - 27 - 28 10 Code documentation 29 11 ------------------ 30 12
+18 -9
MAINTAINERS
··· 4113 4113 ATOMIC INFRASTRUCTURE 4114 4114 M: Will Deacon <will@kernel.org> 4115 4115 M: Peter Zijlstra <peterz@infradead.org> 4116 - M: Boqun Feng <boqun.feng@gmail.com> 4116 + M: Boqun Feng <boqun@kernel.org> 4117 4117 R: Mark Rutland <mark.rutland@arm.com> 4118 4118 R: Gary Guo <gary@garyguo.net> 4119 4119 L: linux-kernel@vger.kernel.org ··· 4506 4506 4507 4507 BLOCK LAYER DEVICE DRIVER API [RUST] 4508 4508 M: Andreas Hindborg <a.hindborg@kernel.org> 4509 - R: Boqun Feng <boqun.feng@gmail.com> 4509 + R: Boqun Feng <boqun@kernel.org> 4510 4510 L: linux-block@vger.kernel.org 4511 4511 L: rust-for-linux@vger.kernel.org 4512 4512 S: Supported ··· 11281 11281 11282 11282 DELAY, SLEEP, TIMEKEEPING, TIMERS [RUST] 11283 11283 M: Andreas Hindborg <a.hindborg@kernel.org> 11284 - R: Boqun Feng <boqun.feng@gmail.com> 11284 + R: Boqun Feng <boqun@kernel.org> 11285 11285 R: FUJITA Tomonori <fujita.tomonori@gmail.com> 11286 11286 R: Frederic Weisbecker <frederic@kernel.org> 11287 11287 R: Lyude Paul <lyude@redhat.com> ··· 14582 14582 M: Andrea Parri <parri.andrea@gmail.com> 14583 14583 M: Will Deacon <will@kernel.org> 14584 14584 M: Peter Zijlstra <peterz@infradead.org> 14585 - M: Boqun Feng <boqun.feng@gmail.com> 14585 + M: Boqun Feng <boqun@kernel.org> 14586 14586 M: Nicholas Piggin <npiggin@gmail.com> 14587 14587 M: David Howells <dhowells@redhat.com> 14588 14588 M: Jade Alglave <j.alglave@ucl.ac.uk> ··· 14741 14741 M: Peter Zijlstra <peterz@infradead.org> 14742 14742 M: Ingo Molnar <mingo@redhat.com> 14743 14743 M: Will Deacon <will@kernel.org> 14744 - M: Boqun Feng <boqun.feng@gmail.com> (LOCKDEP & RUST) 14744 + M: Boqun Feng <boqun@kernel.org> (LOCKDEP & RUST) 14745 14745 R: Waiman Long <longman@redhat.com> 14746 14746 L: linux-kernel@vger.kernel.org 14747 14747 S: Maintained ··· 21948 21948 M: Neeraj Upadhyay <neeraj.upadhyay@kernel.org> (kernel/rcu/tasks.h) 21949 21949 M: Joel Fernandes <joelagnelf@nvidia.com> 21950 21950 M: Josh Triplett <josh@joshtriplett.org> 21951 - M: Boqun Feng <boqun.feng@gmail.com> 21951 + M: Boqun Feng <boqun@kernel.org> 21952 21952 M: Uladzislau Rezki <urezki@gmail.com> 21953 21953 R: Steven Rostedt <rostedt@goodmis.org> 21954 21954 R: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> ··· 22399 22399 M: Mathieu Desnoyers <mathieu.desnoyers@efficios.com> 22400 22400 M: Peter Zijlstra <peterz@infradead.org> 22401 22401 M: "Paul E. McKenney" <paulmck@kernel.org> 22402 - M: Boqun Feng <boqun.feng@gmail.com> 22402 + M: Boqun Feng <boqun@kernel.org> 22403 22403 L: linux-kernel@vger.kernel.org 22404 22404 S: Supported 22405 22405 F: include/trace/events/rseq.h ··· 22922 22922 22923 22923 RUST 22924 22924 M: Miguel Ojeda <ojeda@kernel.org> 22925 - R: Boqun Feng <boqun.feng@gmail.com> 22925 + R: Boqun Feng <boqun@kernel.org> 22926 22926 R: Gary Guo <gary@garyguo.net> 22927 22927 R: Björn Roy Baron <bjorn3_gh@protonmail.com> 22928 22928 R: Benno Lossin <lossin@kernel.org> ··· 22968 22968 22969 22969 RUST [PIN-INIT] 22970 22970 M: Benno Lossin <lossin@kernel.org> 22971 + M: Gary Guo <gary@garyguo.net> 22971 22972 L: rust-for-linux@vger.kernel.org 22972 22973 S: Maintained 22973 22974 W: https://rust-for-linux.com/pin-init ··· 22979 22978 F: rust/kernel/init.rs 22980 22979 F: rust/pin-init/ 22981 22980 K: \bpin-init\b|pin_init\b|PinInit 22981 + 22982 + RUST [RUST-ANALYZER] 22983 + M: Tamir Duberstein <tamird@kernel.org> 22984 + R: Jesung Yang <y.j3ms.n@gmail.com> 22985 + L: rust-for-linux@vger.kernel.org 22986 + S: Maintained 22987 + T: git https://github.com/Rust-for-Linux/linux.git rust-analyzer-next 22988 + F: scripts/generate_rust_analyzer.py 22982 22989 22983 22990 RXRPC SOCKETS (AF_RXRPC) 22984 22991 M: David Howells <dhowells@redhat.com> ··· 28394 28385 F: tools/testing/radix-tree 28395 28386 28396 28387 XARRAY API [RUST] 28397 - M: Tamir Duberstein <tamird@gmail.com> 28388 + M: Tamir Duberstein <tamird@kernel.org> 28398 28389 M: Andreas Hindborg <a.hindborg@kernel.org> 28399 28390 L: rust-for-linux@vger.kernel.org 28400 28391 S: Supported
-1
drivers/gpu/nova-core/firmware/fwsec.rs
··· 12 12 13 13 use core::{ 14 14 marker::PhantomData, 15 - mem::size_of, 16 15 ops::Deref, // 17 16 }; 18 17
+5 -7
drivers/gpu/nova-core/firmware/gsp.rs
··· 1 1 // SPDX-License-Identifier: GPL-2.0 2 2 3 - use core::mem::size_of_val; 4 - 5 3 use kernel::{ 6 4 device, 7 5 dma::{ ··· 32 34 /// that scheme before nova-core becomes stable, which means this module will eventually be 33 35 /// removed. 34 36 mod elf { 35 - use core::mem::size_of; 36 - 37 - use kernel::bindings; 38 - use kernel::str::CStr; 39 - use kernel::transmute::FromBytes; 37 + use kernel::{ 38 + bindings, 39 + prelude::*, 40 + transmute::FromBytes, // 41 + }; 40 42 41 43 /// Newtype to provide a [`FromBytes`] implementation. 42 44 #[repr(transparent)]
-2
drivers/gpu/nova-core/firmware/riscv.rs
··· 3 3 //! Support for firmware binaries designed to run on a RISC-V core. Such firmwares files have a 4 4 //! dedicated header. 5 5 6 - use core::mem::size_of; 7 - 8 6 use kernel::{ 9 7 device, 10 8 firmware::Firmware,
+3 -3
drivers/gpu/nova-core/gsp/commands.rs
··· 142 142 } 143 143 144 144 /// Message type for GSP initialization done notification. 145 - struct GspInitDone {} 145 + struct GspInitDone; 146 146 147 147 // SAFETY: `GspInitDone` is a zero-sized type with no bytes, therefore it 148 148 // trivially has no uninitialized bytes. ··· 151 151 impl MessageFromGsp for GspInitDone { 152 152 const FUNCTION: MsgFunction = MsgFunction::GspInitDone; 153 153 type InitError = Infallible; 154 - type Message = GspInitDone; 154 + type Message = (); 155 155 156 156 fn read( 157 157 _msg: &Self::Message, 158 158 _sbuffer: &mut SBufferIter<array::IntoIter<&[u8], 2>>, 159 159 ) -> Result<Self, Self::InitError> { 160 - Ok(GspInitDone {}) 160 + Ok(GspInitDone) 161 161 } 162 162 } 163 163
+1 -7
drivers/gpu/nova-core/gsp/sequencer.rs
··· 2 2 3 3 //! GSP Sequencer implementation for Pre-hopper GSP boot sequence. 4 4 5 - use core::{ 6 - array, 7 - mem::{ 8 - size_of, 9 - size_of_val, // 10 - }, 11 - }; 5 + use core::array; 12 6 13 7 use kernel::{ 14 8 device,
+1 -4
drivers/gpu/nova-core/sbuffer.rs
··· 2 2 3 3 use core::ops::Deref; 4 4 5 - use kernel::{ 6 - alloc::KVec, 7 - prelude::*, // 8 - }; 5 + use kernel::prelude::*; 9 6 10 7 /// A buffer abstraction for discontiguous byte slices. 11 8 ///
+29 -12
rust/Makefile
··· 117 117 --extern quote \ 118 118 $(call cfgs-to-flags,$(syn-cfgs)) 119 119 120 + pin_init_internal-cfgs := \ 121 + kernel 122 + 123 + pin_init_internal-flags := \ 124 + --extern proc_macro2 \ 125 + --extern quote \ 126 + --extern syn \ 127 + $(call cfgs-to-flags,$(pin_init_internal-cfgs)) 128 + 129 + pin_init-cfgs := \ 130 + kernel 131 + 132 + pin_init-flags := \ 133 + --extern pin_init_internal \ 134 + --extern macros \ 135 + $(call cfgs-to-flags,$(pin_init-cfgs)) 136 + 120 137 # `rustdoc` did not save the target modifiers, thus workaround for 121 138 # the time being (https://github.com/rust-lang/rust/issues/144521). 122 139 rustdoc_modifiers_workaround := $(if $(call rustc-min-version,108800),-Cunsafe-allow-abi-mismatch=fixed-x18) ··· 228 211 +$(call if_changed,rustdoc) 229 212 230 213 rustdoc-pin_init_internal: private rustdoc_host = yes 231 - rustdoc-pin_init_internal: private rustc_target_flags = --cfg kernel \ 214 + rustdoc-pin_init_internal: private rustc_target_flags = $(pin_init_internal-flags) \ 232 215 --extern proc_macro --crate-type proc-macro 233 216 rustdoc-pin_init_internal: $(src)/pin-init/internal/src/lib.rs \ 234 - rustdoc-clean FORCE 217 + rustdoc-clean rustdoc-proc_macro2 rustdoc-quote rustdoc-syn FORCE 235 218 +$(call if_changed,rustdoc) 236 219 237 220 rustdoc-pin_init: private rustdoc_host = yes 238 - rustdoc-pin_init: private rustc_target_flags = --extern pin_init_internal \ 239 - --extern macros --extern alloc --cfg kernel --cfg feature=\"alloc\" 221 + rustdoc-pin_init: private rustc_target_flags = $(pin_init-flags) \ 222 + --extern alloc --cfg feature=\"alloc\" 240 223 rustdoc-pin_init: $(src)/pin-init/src/lib.rs rustdoc-pin_init_internal \ 241 224 rustdoc-macros FORCE 242 225 +$(call if_changed,rustdoc) ··· 289 272 rusttestlib-proc_macro2 rusttestlib-quote rusttestlib-syn FORCE 290 273 +$(call if_changed,rustc_test_library) 291 274 292 - rusttestlib-pin_init_internal: private rustc_target_flags = --cfg kernel \ 275 + rusttestlib-pin_init_internal: private rustc_target_flags = $(pin_init_internal-flags) \ 293 276 --extern proc_macro 294 277 rusttestlib-pin_init_internal: private rustc_test_library_proc = yes 295 - rusttestlib-pin_init_internal: $(src)/pin-init/internal/src/lib.rs FORCE 278 + rusttestlib-pin_init_internal: $(src)/pin-init/internal/src/lib.rs \ 279 + rusttestlib-proc_macro2 rusttestlib-quote rusttestlib-syn FORCE 296 280 +$(call if_changed,rustc_test_library) 297 281 298 - rusttestlib-pin_init: private rustc_target_flags = --extern pin_init_internal \ 299 - --extern macros --cfg kernel 282 + rusttestlib-pin_init: private rustc_target_flags = $(pin_init-flags) 300 283 rusttestlib-pin_init: $(src)/pin-init/src/lib.rs rusttestlib-macros \ 301 284 rusttestlib-pin_init_internal $(obj)/$(libpin_init_internal_name) FORCE 302 285 +$(call if_changed,rustc_test_library) ··· 565 548 $(obj)/libquote.rlib $(obj)/libsyn.rlib FORCE 566 549 +$(call if_changed_dep,rustc_procmacro) 567 550 568 - $(obj)/$(libpin_init_internal_name): private rustc_target_flags = --cfg kernel 569 - $(obj)/$(libpin_init_internal_name): $(src)/pin-init/internal/src/lib.rs FORCE 551 + $(obj)/$(libpin_init_internal_name): private rustc_target_flags = $(pin_init_internal-flags) 552 + $(obj)/$(libpin_init_internal_name): $(src)/pin-init/internal/src/lib.rs \ 553 + $(obj)/libproc_macro2.rlib $(obj)/libquote.rlib $(obj)/libsyn.rlib FORCE 570 554 +$(call if_changed_dep,rustc_procmacro) 571 555 572 556 quiet_cmd_rustc_library = $(if $(skip_clippy),RUSTC,$(RUSTC_OR_CLIPPY_QUIET)) L $@ ··· 661 643 +$(call if_changed_rule,rustc_library) 662 644 663 645 $(obj)/pin_init.o: private skip_gendwarfksyms = 1 664 - $(obj)/pin_init.o: private rustc_target_flags = --extern pin_init_internal \ 665 - --extern macros --cfg kernel 646 + $(obj)/pin_init.o: private rustc_target_flags = $(pin_init-flags) 666 647 $(obj)/pin_init.o: $(src)/pin-init/src/lib.rs $(obj)/compiler_builtins.o \ 667 648 $(obj)/$(libpin_init_internal_name) $(obj)/$(libmacros_name) FORCE 668 649 +$(call if_changed_rule,rustc_library)
+2 -2
rust/helpers/bug.c
··· 2 2 3 3 #include <linux/bug.h> 4 4 5 - __noreturn void rust_helper_BUG(void) 5 + __rust_helper __noreturn void rust_helper_BUG(void) 6 6 { 7 7 BUG(); 8 8 } 9 9 10 - bool rust_helper_WARN_ON(bool cond) 10 + __rust_helper bool rust_helper_WARN_ON(bool cond) 11 11 { 12 12 return WARN_ON(cond); 13 13 }
+1 -1
rust/helpers/build_bug.c
··· 2 2 3 3 #include <linux/errname.h> 4 4 5 - const char *rust_helper_errname(int err) 5 + __rust_helper const char *rust_helper_errname(int err) 6 6 { 7 7 return errname(err); 8 8 }
+3 -3
rust/helpers/err.c
··· 2 2 3 3 #include <linux/err.h> 4 4 5 - __force void *rust_helper_ERR_PTR(long err) 5 + __rust_helper __force void *rust_helper_ERR_PTR(long err) 6 6 { 7 7 return ERR_PTR(err); 8 8 } 9 9 10 - bool rust_helper_IS_ERR(__force const void *ptr) 10 + __rust_helper bool rust_helper_IS_ERR(__force const void *ptr) 11 11 { 12 12 return IS_ERR(ptr); 13 13 } 14 14 15 - long rust_helper_PTR_ERR(__force const void *ptr) 15 + __rust_helper long rust_helper_PTR_ERR(__force const void *ptr) 16 16 { 17 17 return PTR_ERR(ptr); 18 18 }
+2 -1
rust/helpers/maple_tree.c
··· 2 2 3 3 #include <linux/maple_tree.h> 4 4 5 - void rust_helper_mt_init_flags(struct maple_tree *mt, unsigned int flags) 5 + __rust_helper void rust_helper_mt_init_flags(struct maple_tree *mt, 6 + unsigned int flags) 6 7 { 7 8 mt_init_flags(mt, flags); 8 9 }
+10 -10
rust/helpers/mm.c
··· 3 3 #include <linux/mm.h> 4 4 #include <linux/sched/mm.h> 5 5 6 - void rust_helper_mmgrab(struct mm_struct *mm) 6 + __rust_helper void rust_helper_mmgrab(struct mm_struct *mm) 7 7 { 8 8 mmgrab(mm); 9 9 } 10 10 11 - void rust_helper_mmdrop(struct mm_struct *mm) 11 + __rust_helper void rust_helper_mmdrop(struct mm_struct *mm) 12 12 { 13 13 mmdrop(mm); 14 14 } 15 15 16 - void rust_helper_mmget(struct mm_struct *mm) 16 + __rust_helper void rust_helper_mmget(struct mm_struct *mm) 17 17 { 18 18 mmget(mm); 19 19 } 20 20 21 - bool rust_helper_mmget_not_zero(struct mm_struct *mm) 21 + __rust_helper bool rust_helper_mmget_not_zero(struct mm_struct *mm) 22 22 { 23 23 return mmget_not_zero(mm); 24 24 } 25 25 26 - void rust_helper_mmap_read_lock(struct mm_struct *mm) 26 + __rust_helper void rust_helper_mmap_read_lock(struct mm_struct *mm) 27 27 { 28 28 mmap_read_lock(mm); 29 29 } 30 30 31 - bool rust_helper_mmap_read_trylock(struct mm_struct *mm) 31 + __rust_helper bool rust_helper_mmap_read_trylock(struct mm_struct *mm) 32 32 { 33 33 return mmap_read_trylock(mm); 34 34 } 35 35 36 - void rust_helper_mmap_read_unlock(struct mm_struct *mm) 36 + __rust_helper void rust_helper_mmap_read_unlock(struct mm_struct *mm) 37 37 { 38 38 mmap_read_unlock(mm); 39 39 } 40 40 41 - struct vm_area_struct *rust_helper_vma_lookup(struct mm_struct *mm, 42 - unsigned long addr) 41 + __rust_helper struct vm_area_struct * 42 + rust_helper_vma_lookup(struct mm_struct *mm, unsigned long addr) 43 43 { 44 44 return vma_lookup(mm, addr); 45 45 } 46 46 47 - void rust_helper_vma_end_read(struct vm_area_struct *vma) 47 + __rust_helper void rust_helper_vma_end_read(struct vm_area_struct *vma) 48 48 { 49 49 vma_end_read(vma); 50 50 }
+1 -1
rust/helpers/of.c
··· 2 2 3 3 #include <linux/of.h> 4 4 5 - bool rust_helper_is_of_node(const struct fwnode_handle *fwnode) 5 + __rust_helper bool rust_helper_is_of_node(const struct fwnode_handle *fwnode) 6 6 { 7 7 return is_of_node(fwnode); 8 8 }
+5 -4
rust/helpers/page.c
··· 4 4 #include <linux/highmem.h> 5 5 #include <linux/mm.h> 6 6 7 - struct page *rust_helper_alloc_pages(gfp_t gfp_mask, unsigned int order) 7 + __rust_helper struct page *rust_helper_alloc_pages(gfp_t gfp_mask, 8 + unsigned int order) 8 9 { 9 10 return alloc_pages(gfp_mask, order); 10 11 } 11 12 12 - void *rust_helper_kmap_local_page(struct page *page) 13 + __rust_helper void *rust_helper_kmap_local_page(struct page *page) 13 14 { 14 15 return kmap_local_page(page); 15 16 } 16 17 17 - void rust_helper_kunmap_local(const void *addr) 18 + __rust_helper void rust_helper_kunmap_local(const void *addr) 18 19 { 19 20 kunmap_local(addr); 20 21 } 21 22 22 23 #ifndef NODE_NOT_IN_PAGE_FLAGS 23 - int rust_helper_page_to_nid(const struct page *page) 24 + __rust_helper int rust_helper_page_to_nid(const struct page *page) 24 25 { 25 26 return page_to_nid(page); 26 27 }
+5 -4
rust/helpers/rbtree.c
··· 2 2 3 3 #include <linux/rbtree.h> 4 4 5 - void rust_helper_rb_link_node(struct rb_node *node, struct rb_node *parent, 6 - struct rb_node **rb_link) 5 + __rust_helper void rust_helper_rb_link_node(struct rb_node *node, 6 + struct rb_node *parent, 7 + struct rb_node **rb_link) 7 8 { 8 9 rb_link_node(node, parent, rb_link); 9 10 } 10 11 11 - struct rb_node *rust_helper_rb_first(const struct rb_root *root) 12 + __rust_helper struct rb_node *rust_helper_rb_first(const struct rb_root *root) 12 13 { 13 14 return rb_first(root); 14 15 } 15 16 16 - struct rb_node *rust_helper_rb_last(const struct rb_root *root) 17 + __rust_helper struct rb_node *rust_helper_rb_last(const struct rb_root *root) 17 18 { 18 19 return rb_last(root); 19 20 }
+2 -2
rust/helpers/slab.c
··· 2 2 3 3 #include <linux/slab.h> 4 4 5 - void * __must_check __realloc_size(2) 5 + __rust_helper void *__must_check __realloc_size(2) 6 6 rust_helper_krealloc_node_align(const void *objp, size_t new_size, unsigned long align, 7 7 gfp_t flags, int node) 8 8 { 9 9 return krealloc_node_align(objp, new_size, align, flags, node); 10 10 } 11 11 12 - void * __must_check __realloc_size(2) 12 + __rust_helper void *__must_check __realloc_size(2) 13 13 rust_helper_kvrealloc_node_align(const void *p, size_t size, unsigned long align, 14 14 gfp_t flags, int node) 15 15 {
+6 -4
rust/helpers/uaccess.c
··· 2 2 3 3 #include <linux/uaccess.h> 4 4 5 - unsigned long rust_helper_copy_from_user(void *to, const void __user *from, 6 - unsigned long n) 5 + __rust_helper unsigned long 6 + rust_helper_copy_from_user(void *to, const void __user *from, unsigned long n) 7 7 { 8 8 return copy_from_user(to, from, n); 9 9 } 10 10 11 - unsigned long rust_helper_copy_to_user(void __user *to, const void *from, 12 - unsigned long n) 11 + __rust_helper unsigned long 12 + rust_helper_copy_to_user(void __user *to, const void *from, unsigned long n) 13 13 { 14 14 return copy_to_user(to, from, n); 15 15 } 16 16 17 17 #ifdef INLINE_COPY_FROM_USER 18 + __rust_helper 18 19 unsigned long rust_helper__copy_from_user(void *to, const void __user *from, unsigned long n) 19 20 { 20 21 return _inline_copy_from_user(to, from, n); 21 22 } 22 23 24 + __rust_helper 23 25 unsigned long rust_helper__copy_to_user(void __user *to, const void *from, unsigned long n) 24 26 { 25 27 return _inline_copy_to_user(to, from, n);
+1 -1
rust/helpers/vmalloc.c
··· 2 2 3 3 #include <linux/vmalloc.h> 4 4 5 - void * __must_check __realloc_size(2) 5 + __rust_helper void *__must_check __realloc_size(2) 6 6 rust_helper_vrealloc_node_align(const void *p, size_t size, unsigned long align, 7 7 gfp_t flags, int node) 8 8 {
+5 -3
rust/helpers/workqueue.c
··· 2 2 3 3 #include <linux/workqueue.h> 4 4 5 - void rust_helper_init_work_with_key(struct work_struct *work, work_func_t func, 6 - bool onstack, const char *name, 7 - struct lock_class_key *key) 5 + __rust_helper void rust_helper_init_work_with_key(struct work_struct *work, 6 + work_func_t func, 7 + bool onstack, 8 + const char *name, 9 + struct lock_class_key *key) 8 10 { 9 11 __init_work(work, onstack); 10 12 work->data = (atomic_long_t)WORK_DATA_INIT();
+5 -5
rust/helpers/xarray.c
··· 2 2 3 3 #include <linux/xarray.h> 4 4 5 - int rust_helper_xa_err(void *entry) 5 + __rust_helper int rust_helper_xa_err(void *entry) 6 6 { 7 7 return xa_err(entry); 8 8 } 9 9 10 - void rust_helper_xa_init_flags(struct xarray *xa, gfp_t flags) 10 + __rust_helper void rust_helper_xa_init_flags(struct xarray *xa, gfp_t flags) 11 11 { 12 12 return xa_init_flags(xa, flags); 13 13 } 14 14 15 - int rust_helper_xa_trylock(struct xarray *xa) 15 + __rust_helper int rust_helper_xa_trylock(struct xarray *xa) 16 16 { 17 17 return xa_trylock(xa); 18 18 } 19 19 20 - void rust_helper_xa_lock(struct xarray *xa) 20 + __rust_helper void rust_helper_xa_lock(struct xarray *xa) 21 21 { 22 22 return xa_lock(xa); 23 23 } 24 24 25 - void rust_helper_xa_unlock(struct xarray *xa) 25 + __rust_helper void rust_helper_xa_unlock(struct xarray *xa) 26 26 { 27 27 return xa_unlock(xa); 28 28 }
+13 -7
rust/kernel/bug.rs
··· 11 11 #[cfg(all(CONFIG_BUG, not(CONFIG_UML), not(CONFIG_LOONGARCH), not(CONFIG_ARM)))] 12 12 #[cfg(CONFIG_DEBUG_BUGVERBOSE)] 13 13 macro_rules! warn_flags { 14 - ($flags:expr) => { 14 + ($file:expr, $flags:expr) => { 15 15 const FLAGS: u32 = $crate::bindings::BUGFLAG_WARNING | $flags; 16 - const _FILE: &[u8] = file!().as_bytes(); 16 + const _FILE: &[u8] = $file.as_bytes(); 17 17 // Plus one for null-terminator. 18 18 static FILE: [u8; _FILE.len() + 1] = { 19 19 let mut bytes = [0; _FILE.len() + 1]; ··· 50 50 #[cfg(all(CONFIG_BUG, not(CONFIG_UML), not(CONFIG_LOONGARCH), not(CONFIG_ARM)))] 51 51 #[cfg(not(CONFIG_DEBUG_BUGVERBOSE))] 52 52 macro_rules! warn_flags { 53 - ($flags:expr) => { 53 + ($file:expr, $flags:expr) => { 54 54 const FLAGS: u32 = $crate::bindings::BUGFLAG_WARNING | $flags; 55 55 56 56 // SAFETY: ··· 75 75 #[doc(hidden)] 76 76 #[cfg(all(CONFIG_BUG, CONFIG_UML))] 77 77 macro_rules! warn_flags { 78 - ($flags:expr) => { 78 + ($file:expr, $flags:expr) => { 79 79 // SAFETY: It is always safe to call `warn_slowpath_fmt()` 80 80 // with a valid null-terminated string. 81 81 unsafe { ··· 93 93 #[doc(hidden)] 94 94 #[cfg(all(CONFIG_BUG, any(CONFIG_LOONGARCH, CONFIG_ARM)))] 95 95 macro_rules! warn_flags { 96 - ($flags:expr) => { 96 + ($file:expr, $flags:expr) => { 97 97 // SAFETY: It is always safe to call `WARN_ON()`. 98 98 unsafe { $crate::bindings::WARN_ON(true) } 99 99 }; ··· 103 103 #[doc(hidden)] 104 104 #[cfg(not(CONFIG_BUG))] 105 105 macro_rules! warn_flags { 106 - ($flags:expr) => {}; 106 + ($file:expr, $flags:expr) => {}; 107 107 } 108 108 109 109 #[doc(hidden)] ··· 116 116 macro_rules! warn_on { 117 117 ($cond:expr) => {{ 118 118 let cond = $cond; 119 + 120 + #[cfg(CONFIG_DEBUG_BUGVERBOSE_DETAILED)] 121 + const _COND_STR: &str = concat!("[", stringify!($cond), "] ", file!()); 122 + #[cfg(not(CONFIG_DEBUG_BUGVERBOSE_DETAILED))] 123 + const _COND_STR: &str = file!(); 124 + 119 125 if cond { 120 126 const WARN_ON_FLAGS: u32 = $crate::bug::bugflag_taint($crate::bindings::TAINT_WARN); 121 127 122 - $crate::warn_flags!(WARN_ON_FLAGS); 128 + $crate::warn_flags!(_COND_STR, WARN_ON_FLAGS); 123 129 } 124 130 cond 125 131 }};
+6 -1
rust/kernel/build_assert.rs
··· 61 61 /// build_assert!(N > 1); // Build-time check 62 62 /// assert!(N > 1); // Run-time check 63 63 /// } 64 + /// ``` 64 65 /// 65 - /// #[inline] 66 + /// When a condition depends on a function argument, the function must be annotated with 67 + /// `#[inline(always)]`. Without this attribute, the compiler may choose to not inline the 68 + /// function, preventing it from optimizing out the error path. 69 + /// ``` 70 + /// #[inline(always)] 66 71 /// fn bar(n: usize) { 67 72 /// // `static_assert!(n > 1);` is not allowed 68 73 /// build_assert!(n > 1); // Build-time check
+3 -5
rust/kernel/clk.rs
··· 94 94 /// # Invariants 95 95 /// 96 96 /// A [`Clk`] instance holds either a pointer to a valid [`struct clk`] created by the C 97 - /// portion of the kernel or a NULL pointer. 97 + /// portion of the kernel or a `NULL` pointer. 98 98 /// 99 99 /// Instances of this type are reference-counted. Calling [`Clk::get`] ensures that the 100 100 /// allocation remains valid for the lifetime of the [`Clk`]. ··· 104 104 /// The following example demonstrates how to obtain and configure a clock for a device. 105 105 /// 106 106 /// ``` 107 - /// use kernel::c_str; 108 107 /// use kernel::clk::{Clk, Hertz}; 109 108 /// use kernel::device::Device; 110 109 /// use kernel::error::Result; 111 110 /// 112 111 /// fn configure_clk(dev: &Device) -> Result { 113 - /// let clk = Clk::get(dev, Some(c_str!("apb_clk")))?; 112 + /// let clk = Clk::get(dev, Some(c"apb_clk"))?; 114 113 /// 115 114 /// clk.prepare_enable()?; 116 115 /// ··· 271 272 /// device. The code functions correctly whether or not the clock is available. 272 273 /// 273 274 /// ``` 274 - /// use kernel::c_str; 275 275 /// use kernel::clk::{OptionalClk, Hertz}; 276 276 /// use kernel::device::Device; 277 277 /// use kernel::error::Result; 278 278 /// 279 279 /// fn configure_clk(dev: &Device) -> Result { 280 - /// let clk = OptionalClk::get(dev, Some(c_str!("apb_clk")))?; 280 + /// let clk = OptionalClk::get(dev, Some(c"apb_clk"))?; 281 281 /// 282 282 /// clk.prepare_enable()?; 283 283 ///
+1 -1
rust/kernel/debugfs/entry.rs
··· 148 148 /// # Guarantees 149 149 /// 150 150 /// Due to the type invariant, the value returned from this function will always be an error 151 - /// code, NULL, or a live DebugFS directory. If it is live, it will remain live at least as 151 + /// code, `NULL`, or a live DebugFS directory. If it is live, it will remain live at least as 152 152 /// long as this entry lives. 153 153 pub(crate) fn as_ptr(&self) -> *mut bindings::dentry { 154 154 self.entry
+4 -4
rust/kernel/i2c.rs
··· 262 262 /// # Example 263 263 /// 264 264 ///``` 265 - /// # use kernel::{acpi, bindings, c_str, device::Core, i2c, of}; 265 + /// # use kernel::{acpi, bindings, device::Core, i2c, of}; 266 266 /// 267 267 /// struct MyDriver; 268 268 /// ··· 271 271 /// MODULE_ACPI_TABLE, 272 272 /// <MyDriver as i2c::Driver>::IdInfo, 273 273 /// [ 274 - /// (acpi::DeviceId::new(c_str!("LNUXBEEF")), ()) 274 + /// (acpi::DeviceId::new(c"LNUXBEEF"), ()) 275 275 /// ] 276 276 /// ); 277 277 /// ··· 280 280 /// MODULE_I2C_TABLE, 281 281 /// <MyDriver as i2c::Driver>::IdInfo, 282 282 /// [ 283 - /// (i2c::DeviceId::new(c_str!("rust_driver_i2c")), ()) 283 + /// (i2c::DeviceId::new(c"rust_driver_i2c"), ()) 284 284 /// ] 285 285 /// ); 286 286 /// ··· 289 289 /// MODULE_OF_TABLE, 290 290 /// <MyDriver as i2c::Driver>::IdInfo, 291 291 /// [ 292 - /// (of::DeviceId::new(c_str!("test,device")), ()) 292 + /// (of::DeviceId::new(c"test,device"), ()) 293 293 /// ] 294 294 /// ); 295 295 ///
+272
rust/kernel/impl_flags.rs
··· 1 + // SPDX-License-Identifier: GPL-2.0 2 + 3 + //! Bitflag type generator. 4 + 5 + /// Common helper for declaring bitflag and bitmask types. 6 + /// 7 + /// This macro takes as input: 8 + /// - A struct declaration representing a bitmask type 9 + /// (e.g., `pub struct Permissions(u32)`). 10 + /// - An enumeration declaration representing individual bit flags 11 + /// (e.g., `pub enum Permission { ... }`). 12 + /// 13 + /// And generates: 14 + /// - The struct and enum types with appropriate `#[repr]` attributes. 15 + /// - Implementations of common bitflag operators 16 + /// ([`::core::ops::BitOr`], [`::core::ops::BitAnd`], etc.). 17 + /// - Utility methods such as `.contains()` to check flags. 18 + /// 19 + /// # Examples 20 + /// 21 + /// ``` 22 + /// use kernel::impl_flags; 23 + /// 24 + /// impl_flags!( 25 + /// /// Represents multiple permissions. 26 + /// #[derive(Debug, Clone, Default, Copy, PartialEq, Eq)] 27 + /// pub struct Permissions(u32); 28 + /// 29 + /// /// Represents a single permission. 30 + /// #[derive(Debug, Clone, Copy, PartialEq, Eq)] 31 + /// pub enum Permission { 32 + /// /// Read permission. 33 + /// Read = 1 << 0, 34 + /// 35 + /// /// Write permission. 36 + /// Write = 1 << 1, 37 + /// 38 + /// /// Execute permission. 39 + /// Execute = 1 << 2, 40 + /// } 41 + /// ); 42 + /// 43 + /// // Combine multiple permissions using the bitwise OR (`|`) operator. 44 + /// let mut read_write: Permissions = Permission::Read | Permission::Write; 45 + /// assert!(read_write.contains(Permission::Read)); 46 + /// assert!(read_write.contains(Permission::Write)); 47 + /// assert!(!read_write.contains(Permission::Execute)); 48 + /// assert!(read_write.contains_any(Permission::Read | Permission::Execute)); 49 + /// assert!(read_write.contains_all(Permission::Read | Permission::Write)); 50 + /// 51 + /// // Using the bitwise OR assignment (`|=`) operator. 52 + /// read_write |= Permission::Execute; 53 + /// assert!(read_write.contains(Permission::Execute)); 54 + /// 55 + /// // Masking a permission with the bitwise AND (`&`) operator. 56 + /// let read_only: Permissions = read_write & Permission::Read; 57 + /// assert!(read_only.contains(Permission::Read)); 58 + /// assert!(!read_only.contains(Permission::Write)); 59 + /// 60 + /// // Toggling permissions with the bitwise XOR (`^`) operator. 61 + /// let toggled: Permissions = read_only ^ Permission::Read; 62 + /// assert!(!toggled.contains(Permission::Read)); 63 + /// 64 + /// // Inverting permissions with the bitwise NOT (`!`) operator. 65 + /// let negated = !read_only; 66 + /// assert!(negated.contains(Permission::Write)); 67 + /// assert!(!negated.contains(Permission::Read)); 68 + /// ``` 69 + #[macro_export] 70 + macro_rules! impl_flags { 71 + ( 72 + $(#[$outer_flags:meta])* 73 + $vis_flags:vis struct $flags:ident($ty:ty); 74 + 75 + $(#[$outer_flag:meta])* 76 + $vis_flag:vis enum $flag:ident { 77 + $( 78 + $(#[$inner_flag:meta])* 79 + $name:ident = $value:expr 80 + ),+ $( , )? 81 + } 82 + ) => { 83 + $(#[$outer_flags])* 84 + #[repr(transparent)] 85 + $vis_flags struct $flags($ty); 86 + 87 + $(#[$outer_flag])* 88 + #[repr($ty)] 89 + $vis_flag enum $flag { 90 + $( 91 + $(#[$inner_flag])* 92 + $name = $value 93 + ),+ 94 + } 95 + 96 + impl ::core::convert::From<$flag> for $flags { 97 + #[inline] 98 + fn from(value: $flag) -> Self { 99 + Self(value as $ty) 100 + } 101 + } 102 + 103 + impl ::core::convert::From<$flags> for $ty { 104 + #[inline] 105 + fn from(value: $flags) -> Self { 106 + value.0 107 + } 108 + } 109 + 110 + impl ::core::ops::BitOr for $flags { 111 + type Output = Self; 112 + #[inline] 113 + fn bitor(self, rhs: Self) -> Self::Output { 114 + Self(self.0 | rhs.0) 115 + } 116 + } 117 + 118 + impl ::core::ops::BitOrAssign for $flags { 119 + #[inline] 120 + fn bitor_assign(&mut self, rhs: Self) { 121 + *self = *self | rhs; 122 + } 123 + } 124 + 125 + impl ::core::ops::BitOr<$flag> for $flags { 126 + type Output = Self; 127 + #[inline] 128 + fn bitor(self, rhs: $flag) -> Self::Output { 129 + self | Self::from(rhs) 130 + } 131 + } 132 + 133 + impl ::core::ops::BitOrAssign<$flag> for $flags { 134 + #[inline] 135 + fn bitor_assign(&mut self, rhs: $flag) { 136 + *self = *self | rhs; 137 + } 138 + } 139 + 140 + impl ::core::ops::BitAnd for $flags { 141 + type Output = Self; 142 + #[inline] 143 + fn bitand(self, rhs: Self) -> Self::Output { 144 + Self(self.0 & rhs.0) 145 + } 146 + } 147 + 148 + impl ::core::ops::BitAndAssign for $flags { 149 + #[inline] 150 + fn bitand_assign(&mut self, rhs: Self) { 151 + *self = *self & rhs; 152 + } 153 + } 154 + 155 + impl ::core::ops::BitAnd<$flag> for $flags { 156 + type Output = Self; 157 + #[inline] 158 + fn bitand(self, rhs: $flag) -> Self::Output { 159 + self & Self::from(rhs) 160 + } 161 + } 162 + 163 + impl ::core::ops::BitAndAssign<$flag> for $flags { 164 + #[inline] 165 + fn bitand_assign(&mut self, rhs: $flag) { 166 + *self = *self & rhs; 167 + } 168 + } 169 + 170 + impl ::core::ops::BitXor for $flags { 171 + type Output = Self; 172 + #[inline] 173 + fn bitxor(self, rhs: Self) -> Self::Output { 174 + Self((self.0 ^ rhs.0) & Self::all_bits()) 175 + } 176 + } 177 + 178 + impl ::core::ops::BitXorAssign for $flags { 179 + #[inline] 180 + fn bitxor_assign(&mut self, rhs: Self) { 181 + *self = *self ^ rhs; 182 + } 183 + } 184 + 185 + impl ::core::ops::BitXor<$flag> for $flags { 186 + type Output = Self; 187 + #[inline] 188 + fn bitxor(self, rhs: $flag) -> Self::Output { 189 + self ^ Self::from(rhs) 190 + } 191 + } 192 + 193 + impl ::core::ops::BitXorAssign<$flag> for $flags { 194 + #[inline] 195 + fn bitxor_assign(&mut self, rhs: $flag) { 196 + *self = *self ^ rhs; 197 + } 198 + } 199 + 200 + impl ::core::ops::Not for $flags { 201 + type Output = Self; 202 + #[inline] 203 + fn not(self) -> Self::Output { 204 + Self((!self.0) & Self::all_bits()) 205 + } 206 + } 207 + 208 + impl ::core::ops::BitOr for $flag { 209 + type Output = $flags; 210 + #[inline] 211 + fn bitor(self, rhs: Self) -> Self::Output { 212 + $flags(self as $ty | rhs as $ty) 213 + } 214 + } 215 + 216 + impl ::core::ops::BitAnd for $flag { 217 + type Output = $flags; 218 + #[inline] 219 + fn bitand(self, rhs: Self) -> Self::Output { 220 + $flags(self as $ty & rhs as $ty) 221 + } 222 + } 223 + 224 + impl ::core::ops::BitXor for $flag { 225 + type Output = $flags; 226 + #[inline] 227 + fn bitxor(self, rhs: Self) -> Self::Output { 228 + $flags((self as $ty ^ rhs as $ty) & $flags::all_bits()) 229 + } 230 + } 231 + 232 + impl ::core::ops::Not for $flag { 233 + type Output = $flags; 234 + #[inline] 235 + fn not(self) -> Self::Output { 236 + $flags((!(self as $ty)) & $flags::all_bits()) 237 + } 238 + } 239 + 240 + impl $flags { 241 + /// Returns an empty instance where no flags are set. 242 + #[inline] 243 + pub const fn empty() -> Self { 244 + Self(0) 245 + } 246 + 247 + /// Returns a mask containing all valid flag bits. 248 + #[inline] 249 + pub const fn all_bits() -> $ty { 250 + 0 $( | $value )+ 251 + } 252 + 253 + /// Checks if a specific flag is set. 254 + #[inline] 255 + pub fn contains(self, flag: $flag) -> bool { 256 + (self.0 & flag as $ty) == flag as $ty 257 + } 258 + 259 + /// Checks if at least one of the provided flags is set. 260 + #[inline] 261 + pub fn contains_any(self, flags: $flags) -> bool { 262 + (self.0 & flags.0) != 0 263 + } 264 + 265 + /// Checks if all of the provided flags are set. 266 + #[inline] 267 + pub fn contains_all(self, flags: $flags) -> bool { 268 + (self.0 & flags.0) == flags.0 269 + } 270 + } 271 + }; 272 + }
+12 -28
rust/kernel/init.rs
··· 219 219 /// [`Error`]: crate::error::Error 220 220 #[macro_export] 221 221 macro_rules! try_init { 222 - ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? { 223 - $($fields:tt)* 224 - }) => { 225 - ::pin_init::try_init!($(&$this in)? $t $(::<$($generics),*>)? { 226 - $($fields)* 227 - }? $crate::error::Error) 228 - }; 229 - ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? { 230 - $($fields:tt)* 231 - }? $err:ty) => { 232 - ::pin_init::try_init!($(&$this in)? $t $(::<$($generics),*>)? { 233 - $($fields)* 234 - }? $err) 235 - }; 222 + ($($args:tt)*) => { 223 + ::pin_init::init!( 224 + #[default_error($crate::error::Error)] 225 + $($args)* 226 + ) 227 + } 236 228 } 237 229 238 230 /// Construct an in-place, fallible pinned initializer for `struct`s. ··· 271 279 /// [`Error`]: crate::error::Error 272 280 #[macro_export] 273 281 macro_rules! try_pin_init { 274 - ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? { 275 - $($fields:tt)* 276 - }) => { 277 - ::pin_init::try_pin_init!($(&$this in)? $t $(::<$($generics),*>)? { 278 - $($fields)* 279 - }? $crate::error::Error) 280 - }; 281 - ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? { 282 - $($fields:tt)* 283 - }? $err:ty) => { 284 - ::pin_init::try_pin_init!($(&$this in)? $t $(::<$($generics),*>)? { 285 - $($fields)* 286 - }? $err) 287 - }; 282 + ($($args:tt)*) => { 283 + ::pin_init::pin_init!( 284 + #[default_error($crate::error::Error)] 285 + $($args)* 286 + ) 287 + } 288 288 }
+6 -33
rust/kernel/kunit.rs
··· 9 9 use crate::fmt; 10 10 use crate::prelude::*; 11 11 12 - #[cfg(CONFIG_PRINTK)] 13 - use crate::c_str; 14 - 15 12 /// Prints a KUnit error-level message. 16 13 /// 17 14 /// Public but hidden since it should only be used from KUnit generated code. ··· 19 22 #[cfg(CONFIG_PRINTK)] 20 23 unsafe { 21 24 bindings::_printk( 22 - c_str!("\x013%pA").as_char_ptr(), 25 + c"\x013%pA".as_char_ptr(), 23 26 core::ptr::from_ref(&args).cast::<c_void>(), 24 27 ); 25 28 } ··· 35 38 #[cfg(CONFIG_PRINTK)] 36 39 unsafe { 37 40 bindings::_printk( 38 - c_str!("\x016%pA").as_char_ptr(), 41 + c"\x016%pA".as_char_ptr(), 39 42 core::ptr::from_ref(&args).cast::<c_void>(), 40 43 ); 41 44 } ··· 57 60 break 'out; 58 61 } 59 62 60 - static FILE: &'static $crate::str::CStr = $crate::c_str!($file); 63 + static FILE: &'static $crate::str::CStr = $file; 61 64 static LINE: i32 = ::core::line!() as i32 - $diff; 62 65 static CONDITION: &'static $crate::str::CStr = $crate::c_str!(stringify!($condition)); 63 66 ··· 189 192 } 190 193 191 194 /// Represents an individual test case. 192 - /// 193 - /// The [`kunit_unsafe_test_suite!`] macro expects a NULL-terminated list of valid test cases. 194 - /// Use [`kunit_case_null`] to generate such a delimiter. 195 195 #[doc(hidden)] 196 196 pub const fn kunit_case( 197 197 name: &'static kernel::str::CStr, ··· 209 215 } 210 216 } 211 217 212 - /// Represents the NULL test case delimiter. 213 - /// 214 - /// The [`kunit_unsafe_test_suite!`] macro expects a NULL-terminated list of test cases. This 215 - /// function returns such a delimiter. 216 - #[doc(hidden)] 217 - pub const fn kunit_case_null() -> kernel::bindings::kunit_case { 218 - kernel::bindings::kunit_case { 219 - run_case: None, 220 - name: core::ptr::null_mut(), 221 - generate_params: None, 222 - attr: kernel::bindings::kunit_attributes { 223 - speed: kernel::bindings::kunit_speed_KUNIT_SPEED_NORMAL, 224 - }, 225 - status: kernel::bindings::kunit_status_KUNIT_SUCCESS, 226 - module_name: core::ptr::null_mut(), 227 - log: core::ptr::null_mut(), 228 - param_init: None, 229 - param_exit: None, 230 - } 231 - } 232 - 233 218 /// Registers a KUnit test suite. 234 219 /// 235 220 /// # Safety 236 221 /// 237 - /// `test_cases` must be a NULL terminated array of valid test cases, 222 + /// `test_cases` must be a `NULL` terminated array of valid test cases, 238 223 /// whose lifetime is at least that of the test suite (i.e., static). 239 224 /// 240 225 /// # Examples ··· 226 253 /// } 227 254 /// 228 255 /// static mut KUNIT_TEST_CASES: [kernel::bindings::kunit_case; 2] = [ 229 - /// kernel::kunit::kunit_case(kernel::c_str!("name"), test_fn), 230 - /// kernel::kunit::kunit_case_null(), 256 + /// kernel::kunit::kunit_case(c"name", test_fn), 257 + /// pin_init::zeroed(), 231 258 /// ]; 232 259 /// kernel::kunit_unsafe_test_suite!(suite_name, KUNIT_TEST_CASES); 233 260 /// ```
+3
rust/kernel/lib.rs
··· 100 100 #[cfg(CONFIG_I2C = "y")] 101 101 pub mod i2c; 102 102 pub mod id_pool; 103 + #[doc(hidden)] 104 + pub mod impl_flags; 103 105 pub mod init; 104 106 pub mod io; 105 107 pub mod ioctl; ··· 135 133 pub mod rbtree; 136 134 pub mod regulator; 137 135 pub mod revocable; 136 + pub mod safety; 138 137 pub mod scatterlist; 139 138 pub mod security; 140 139 pub mod seq_file;
+153
rust/kernel/print.rs
··· 11 11 fmt, 12 12 prelude::*, 13 13 str::RawFormatter, 14 + sync::atomic::{ 15 + Atomic, 16 + AtomicType, 17 + Relaxed, // 18 + }, 14 19 }; 15 20 16 21 // Called from `vsprintf` with format specifier `%pA`. ··· 426 421 macro_rules! pr_cont ( 427 422 ($($arg:tt)*) => ( 428 423 $crate::print_macro!($crate::print::format_strings::CONT, true, $($arg)*) 424 + ) 425 + ); 426 + 427 + /// A lightweight `call_once` primitive. 428 + /// 429 + /// This structure provides the Rust equivalent of the kernel's `DO_ONCE_LITE` macro. 430 + /// While it would be possible to implement the feature entirely as a Rust macro, 431 + /// the functionality that can be implemented as regular functions has been 432 + /// extracted and implemented as the `OnceLite` struct for better code maintainability. 433 + pub struct OnceLite(Atomic<State>); 434 + 435 + #[derive(Clone, Copy, PartialEq, Eq)] 436 + #[repr(i32)] 437 + enum State { 438 + Incomplete = 0, 439 + Complete = 1, 440 + } 441 + 442 + // SAFETY: `State` and `i32` has the same size and alignment, and it's round-trip 443 + // transmutable to `i32`. 444 + unsafe impl AtomicType for State { 445 + type Repr = i32; 446 + } 447 + 448 + impl OnceLite { 449 + /// Creates a new [`OnceLite`] in the incomplete state. 450 + #[inline(always)] 451 + #[allow(clippy::new_without_default)] 452 + pub const fn new() -> Self { 453 + OnceLite(Atomic::new(State::Incomplete)) 454 + } 455 + 456 + /// Calls the provided function exactly once. 457 + /// 458 + /// There is no other synchronization between two `call_once()`s 459 + /// except that only one will execute `f`, in other words, callers 460 + /// should not use a failed `call_once()` as a proof that another 461 + /// `call_once()` has already finished and the effect is observable 462 + /// to this thread. 463 + pub fn call_once<F>(&self, f: F) -> bool 464 + where 465 + F: FnOnce(), 466 + { 467 + // Avoid expensive cmpxchg if already completed. 468 + // ORDERING: `Relaxed` is used here since no synchronization is required. 469 + let old = self.0.load(Relaxed); 470 + if old == State::Complete { 471 + return false; 472 + } 473 + 474 + // ORDERING: `Relaxed` is used here since no synchronization is required. 475 + let old = self.0.xchg(State::Complete, Relaxed); 476 + if old == State::Complete { 477 + return false; 478 + } 479 + 480 + f(); 481 + true 482 + } 483 + } 484 + 485 + /// Run the given function exactly once. 486 + /// 487 + /// This is equivalent to the kernel's `DO_ONCE_LITE` macro. 488 + /// 489 + /// # Examples 490 + /// 491 + /// ``` 492 + /// kernel::do_once_lite! { 493 + /// kernel::pr_info!("This will be printed only once\n"); 494 + /// }; 495 + /// ``` 496 + #[macro_export] 497 + macro_rules! do_once_lite { 498 + { $($e:tt)* } => {{ 499 + #[link_section = ".data..once"] 500 + static ONCE: $crate::print::OnceLite = $crate::print::OnceLite::new(); 501 + ONCE.call_once(|| { $($e)* }); 502 + }}; 503 + } 504 + 505 + /// Prints an emergency-level message (level 0) only once. 506 + /// 507 + /// Equivalent to the kernel's `pr_emerg_once` macro. 508 + #[macro_export] 509 + macro_rules! pr_emerg_once ( 510 + ($($arg:tt)*) => ( 511 + $crate::do_once_lite! { $crate::pr_emerg!($($arg)*) } 512 + ) 513 + ); 514 + 515 + /// Prints an alert-level message (level 1) only once. 516 + /// 517 + /// Equivalent to the kernel's `pr_alert_once` macro. 518 + #[macro_export] 519 + macro_rules! pr_alert_once ( 520 + ($($arg:tt)*) => ( 521 + $crate::do_once_lite! { $crate::pr_alert!($($arg)*) } 522 + ) 523 + ); 524 + 525 + /// Prints a critical-level message (level 2) only once. 526 + /// 527 + /// Equivalent to the kernel's `pr_crit_once` macro. 528 + #[macro_export] 529 + macro_rules! pr_crit_once ( 530 + ($($arg:tt)*) => ( 531 + $crate::do_once_lite! { $crate::pr_crit!($($arg)*) } 532 + ) 533 + ); 534 + 535 + /// Prints an error-level message (level 3) only once. 536 + /// 537 + /// Equivalent to the kernel's `pr_err_once` macro. 538 + #[macro_export] 539 + macro_rules! pr_err_once ( 540 + ($($arg:tt)*) => ( 541 + $crate::do_once_lite! { $crate::pr_err!($($arg)*) } 542 + ) 543 + ); 544 + 545 + /// Prints a warning-level message (level 4) only once. 546 + /// 547 + /// Equivalent to the kernel's `pr_warn_once` macro. 548 + #[macro_export] 549 + macro_rules! pr_warn_once ( 550 + ($($arg:tt)*) => ( 551 + $crate::do_once_lite! { $crate::pr_warn!($($arg)*) } 552 + ) 553 + ); 554 + 555 + /// Prints a notice-level message (level 5) only once. 556 + /// 557 + /// Equivalent to the kernel's `pr_notice_once` macro. 558 + #[macro_export] 559 + macro_rules! pr_notice_once ( 560 + ($($arg:tt)*) => ( 561 + $crate::do_once_lite! { $crate::pr_notice!($($arg)*) } 562 + ) 563 + ); 564 + 565 + /// Prints an info-level message (level 6) only once. 566 + /// 567 + /// Equivalent to the kernel's `pr_info_once` macro. 568 + #[macro_export] 569 + macro_rules! pr_info_once ( 570 + ($($arg:tt)*) => ( 571 + $crate::do_once_lite! { $crate::pr_info!($($arg)*) } 429 572 ) 430 573 );
+6 -6
rust/kernel/ptr.rs
··· 5 5 use core::mem::align_of; 6 6 use core::num::NonZero; 7 7 8 - use crate::build_assert; 9 - 10 8 /// Type representing an alignment, which is always a power of two. 11 9 /// 12 10 /// It is used to validate that a given value is a valid alignment, and to perform masking and ··· 38 40 /// ``` 39 41 #[inline(always)] 40 42 pub const fn new<const ALIGN: usize>() -> Self { 41 - build_assert!( 42 - ALIGN.is_power_of_two(), 43 - "Provided alignment is not a power of two." 44 - ); 43 + const { 44 + assert!( 45 + ALIGN.is_power_of_two(), 46 + "Provided alignment is not a power of two." 47 + ); 48 + } 45 49 46 50 // INVARIANT: `align` is a power of two. 47 51 // SAFETY: `align` is a power of two, and thus non-zero.
+15 -12
rust/kernel/rbtree.rs
··· 414 414 // SAFETY: By the type invariant of `Self`, all non-null `rb_node` pointers stored in `self` 415 415 // point to the links field of `Node<K, V>` objects. 416 416 let this = unsafe { container_of!(node, Node<K, V>, links) }; 417 + 417 418 // SAFETY: `this` is a non-null node so it is valid by the type invariants. 418 - node = match key.cmp(unsafe { &(*this).key }) { 419 - // SAFETY: `node` is a non-null node so it is valid by the type invariants. 420 - Ordering::Less => unsafe { (*node).rb_left }, 421 - // SAFETY: `node` is a non-null node so it is valid by the type invariants. 422 - Ordering::Greater => unsafe { (*node).rb_right }, 423 - // SAFETY: `node` is a non-null node so it is valid by the type invariants. 424 - Ordering::Equal => return Some(unsafe { &(*this).value }), 419 + let this_ref = unsafe { &*this }; 420 + 421 + // SAFETY: `node` is a non-null node so it is valid by the type invariants. 422 + let node_ref = unsafe { &*node }; 423 + 424 + node = match key.cmp(&this_ref.key) { 425 + Ordering::Less => node_ref.rb_left, 426 + Ordering::Greater => node_ref.rb_right, 427 + Ordering::Equal => return Some(&this_ref.value), 425 428 } 426 429 } 427 430 None ··· 501 498 let this = unsafe { container_of!(node, Node<K, V>, links) }; 502 499 // SAFETY: `this` is a non-null node so it is valid by the type invariants. 503 500 let this_key = unsafe { &(*this).key }; 501 + 504 502 // SAFETY: `node` is a non-null node so it is valid by the type invariants. 505 - let left_child = unsafe { (*node).rb_left }; 506 - // SAFETY: `node` is a non-null node so it is valid by the type invariants. 507 - let right_child = unsafe { (*node).rb_right }; 503 + let node_ref = unsafe { &*node }; 504 + 508 505 match key.cmp(this_key) { 509 506 Ordering::Equal => { 510 507 // SAFETY: `this` is a non-null node so it is valid by the type invariants. ··· 512 509 break; 513 510 } 514 511 Ordering::Greater => { 515 - node = right_child; 512 + node = node_ref.rb_right; 516 513 } 517 514 Ordering::Less => { 518 515 let is_better_match = match best_key { ··· 524 521 // SAFETY: `this` is a non-null node so it is valid by the type invariants. 525 522 best_links = Some(unsafe { NonNull::new_unchecked(&mut (*this).links) }); 526 523 } 527 - node = left_child; 524 + node = node_ref.rb_left; 528 525 } 529 526 }; 530 527 }
+53
rust/kernel/safety.rs
··· 1 + // SPDX-License-Identifier: GPL-2.0 2 + 3 + //! Safety related APIs. 4 + 5 + /// Checks that a precondition of an unsafe function is followed. 6 + /// 7 + /// The check is enabled at runtime if debug assertions (`CONFIG_RUST_DEBUG_ASSERTIONS`) 8 + /// are enabled. Otherwise, this macro is a no-op. 9 + /// 10 + /// # Examples 11 + /// 12 + /// ```no_run 13 + /// use kernel::unsafe_precondition_assert; 14 + /// 15 + /// struct RawBuffer<T: Copy, const N: usize> { 16 + /// data: [T; N], 17 + /// } 18 + /// 19 + /// impl<T: Copy, const N: usize> RawBuffer<T, N> { 20 + /// /// # Safety 21 + /// /// 22 + /// /// The caller must ensure that `index` is less than `N`. 23 + /// unsafe fn set_unchecked(&mut self, index: usize, value: T) { 24 + /// unsafe_precondition_assert!( 25 + /// index < N, 26 + /// "RawBuffer::set_unchecked() requires index ({index}) < N ({N})" 27 + /// ); 28 + /// 29 + /// // SAFETY: By the safety requirements of this function, `index` is valid. 30 + /// unsafe { 31 + /// *self.data.get_unchecked_mut(index) = value; 32 + /// } 33 + /// } 34 + /// } 35 + /// ``` 36 + /// 37 + /// # Panics 38 + /// 39 + /// Panics if the expression is evaluated to [`false`] at runtime. 40 + #[macro_export] 41 + macro_rules! unsafe_precondition_assert { 42 + ($cond:expr $(,)?) => { 43 + $crate::unsafe_precondition_assert!(@inner $cond, ::core::stringify!($cond)) 44 + }; 45 + 46 + ($cond:expr, $($arg:tt)+) => { 47 + $crate::unsafe_precondition_assert!(@inner $cond, $crate::prelude::fmt!($($arg)+)) 48 + }; 49 + 50 + (@inner $cond:expr, $msg:expr) => { 51 + ::core::debug_assert!($cond, "unsafe precondition violated: {}", $msg) 52 + }; 53 + }
+8
rust/kernel/transmute.rs
··· 170 170 } 171 171 172 172 impl_frombytes! { 173 + // SAFETY: Inhabited ZSTs only have one possible bit pattern, and these two have no invariant. 174 + (), 175 + {<T>} core::marker::PhantomData<T>, 176 + 173 177 // SAFETY: All bit patterns are acceptable values of the types below. 174 178 u8, u16, u32, u64, usize, 175 179 i8, i16, i32, i64, isize, ··· 234 230 } 235 231 236 232 impl_asbytes! { 233 + // SAFETY: Inhabited ZSTs only have one possible bit pattern, and these two have no invariant. 234 + (), 235 + {<T>} core::marker::PhantomData<T>, 236 + 237 237 // SAFETY: Instances of the following types have no uninitialized portions. 238 238 u8, u16, u32, u64, usize, 239 239 i8, i16, i32, i64, isize,
+26 -13
rust/macros/concat_idents.rs
··· 1 1 // SPDX-License-Identifier: GPL-2.0 2 2 3 - use proc_macro::{token_stream, Ident, TokenStream, TokenTree}; 3 + use proc_macro2::{ 4 + Ident, 5 + TokenStream, 6 + TokenTree, // 7 + }; 8 + use syn::{ 9 + parse::{ 10 + Parse, 11 + ParseStream, // 12 + }, 13 + Result, 14 + Token, // 15 + }; 4 16 5 - use crate::helpers::expect_punct; 17 + pub(crate) struct Input { 18 + a: Ident, 19 + _comma: Token![,], 20 + b: Ident, 21 + } 6 22 7 - fn expect_ident(it: &mut token_stream::IntoIter) -> Ident { 8 - if let Some(TokenTree::Ident(ident)) = it.next() { 9 - ident 10 - } else { 11 - panic!("Expected Ident") 23 + impl Parse for Input { 24 + fn parse(input: ParseStream<'_>) -> Result<Self> { 25 + Ok(Self { 26 + a: input.parse()?, 27 + _comma: input.parse()?, 28 + b: input.parse()?, 29 + }) 12 30 } 13 31 } 14 32 15 - pub(crate) fn concat_idents(ts: TokenStream) -> TokenStream { 16 - let mut it = ts.into_iter(); 17 - let a = expect_ident(&mut it); 18 - assert_eq!(expect_punct(&mut it), ','); 19 - let b = expect_ident(&mut it); 20 - assert!(it.next().is_none(), "only two idents can be concatenated"); 33 + pub(crate) fn concat_idents(Input { a, b, .. }: Input) -> TokenStream { 21 34 let res = Ident::new(&format!("{a}{b}"), b.span()); 22 35 TokenStream::from_iter([TokenTree::Ident(res)]) 23 36 }
+11 -15
rust/macros/export.rs
··· 1 1 // SPDX-License-Identifier: GPL-2.0 2 2 3 - use crate::helpers::function_name; 4 - use proc_macro::TokenStream; 3 + use proc_macro2::TokenStream; 4 + use quote::quote; 5 5 6 6 /// Please see [`crate::export`] for documentation. 7 - pub(crate) fn export(_attr: TokenStream, ts: TokenStream) -> TokenStream { 8 - let Some(name) = function_name(ts.clone()) else { 9 - return "::core::compile_error!(\"The #[export] attribute must be used on a function.\");" 10 - .parse::<TokenStream>() 11 - .unwrap(); 12 - }; 7 + pub(crate) fn export(f: syn::ItemFn) -> TokenStream { 8 + let name = &f.sig.ident; 13 9 14 - // This verifies that the function has the same signature as the declaration generated by 15 - // bindgen. It makes use of the fact that all branches of an if/else must have the same type. 16 - let signature_check = quote!( 10 + quote! { 11 + // This verifies that the function has the same signature as the declaration generated by 12 + // bindgen. It makes use of the fact that all branches of an if/else must have the same 13 + // type. 17 14 const _: () = { 18 15 if true { 19 16 ::kernel::bindings::#name ··· 18 21 #name 19 22 }; 20 23 }; 21 - ); 22 24 23 - let no_mangle = quote!(#[no_mangle]); 24 - 25 - TokenStream::from_iter([signature_check, no_mangle, ts]) 25 + #[no_mangle] 26 + #f 27 + } 26 28 }
+3 -1
rust/macros/fmt.rs
··· 1 1 // SPDX-License-Identifier: GPL-2.0 2 2 3 - use proc_macro::{Ident, TokenStream, TokenTree}; 4 3 use std::collections::BTreeSet; 4 + 5 + use proc_macro2::{Ident, TokenStream, TokenTree}; 6 + use quote::quote_spanned; 5 7 6 8 /// Please see [`crate::fmt`] for documentation. 7 9 pub(crate) fn fmt(input: TokenStream) -> TokenStream {
+29 -98
rust/macros/helpers.rs
··· 1 1 // SPDX-License-Identifier: GPL-2.0 2 2 3 - use proc_macro::{token_stream, Group, Ident, TokenStream, TokenTree}; 3 + use proc_macro2::TokenStream; 4 + use quote::ToTokens; 5 + use syn::{ 6 + parse::{ 7 + Parse, 8 + ParseStream, // 9 + }, 10 + Attribute, 11 + Error, 12 + LitStr, 13 + Result, // 14 + }; 4 15 5 - pub(crate) fn try_ident(it: &mut token_stream::IntoIter) -> Option<String> { 6 - if let Some(TokenTree::Ident(ident)) = it.next() { 7 - Some(ident.to_string()) 8 - } else { 9 - None 10 - } 11 - } 16 + /// A string literal that is required to have ASCII value only. 17 + pub(crate) struct AsciiLitStr(LitStr); 12 18 13 - pub(crate) fn try_sign(it: &mut token_stream::IntoIter) -> Option<char> { 14 - let peek = it.clone().next(); 15 - match peek { 16 - Some(TokenTree::Punct(punct)) if punct.as_char() == '-' => { 17 - let _ = it.next(); 18 - Some(punct.as_char()) 19 + impl Parse for AsciiLitStr { 20 + fn parse(input: ParseStream<'_>) -> Result<Self> { 21 + let s: LitStr = input.parse()?; 22 + if !s.value().is_ascii() { 23 + return Err(Error::new_spanned(s, "expected ASCII-only string literal")); 19 24 } 20 - _ => None, 25 + Ok(Self(s)) 21 26 } 22 27 } 23 28 24 - pub(crate) fn try_literal(it: &mut token_stream::IntoIter) -> Option<String> { 25 - if let Some(TokenTree::Literal(literal)) = it.next() { 26 - Some(literal.to_string()) 27 - } else { 28 - None 29 + impl ToTokens for AsciiLitStr { 30 + fn to_tokens(&self, ts: &mut TokenStream) { 31 + self.0.to_tokens(ts); 29 32 } 30 33 } 31 34 32 - pub(crate) fn try_string(it: &mut token_stream::IntoIter) -> Option<String> { 33 - try_literal(it).and_then(|string| { 34 - if string.starts_with('\"') && string.ends_with('\"') { 35 - let content = &string[1..string.len() - 1]; 36 - if content.contains('\\') { 37 - panic!("Escape sequences in string literals not yet handled"); 38 - } 39 - Some(content.to_string()) 40 - } else if string.starts_with("r\"") { 41 - panic!("Raw string literals are not yet handled"); 42 - } else { 43 - None 44 - } 45 - }) 46 - } 47 - 48 - pub(crate) fn expect_ident(it: &mut token_stream::IntoIter) -> String { 49 - try_ident(it).expect("Expected Ident") 50 - } 51 - 52 - pub(crate) fn expect_punct(it: &mut token_stream::IntoIter) -> char { 53 - if let TokenTree::Punct(punct) = it.next().expect("Reached end of token stream for Punct") { 54 - punct.as_char() 55 - } else { 56 - panic!("Expected Punct"); 35 + impl AsciiLitStr { 36 + pub(crate) fn value(&self) -> String { 37 + self.0.value() 57 38 } 58 - } 59 - 60 - pub(crate) fn expect_string(it: &mut token_stream::IntoIter) -> String { 61 - try_string(it).expect("Expected string") 62 - } 63 - 64 - pub(crate) fn expect_string_ascii(it: &mut token_stream::IntoIter) -> String { 65 - let string = try_string(it).expect("Expected string"); 66 - assert!(string.is_ascii(), "Expected ASCII string"); 67 - string 68 - } 69 - 70 - pub(crate) fn expect_group(it: &mut token_stream::IntoIter) -> Group { 71 - if let TokenTree::Group(group) = it.next().expect("Reached end of token stream for Group") { 72 - group 73 - } else { 74 - panic!("Expected Group"); 75 - } 76 - } 77 - 78 - pub(crate) fn expect_end(it: &mut token_stream::IntoIter) { 79 - if it.next().is_some() { 80 - panic!("Expected end"); 81 - } 82 - } 83 - 84 - /// Given a function declaration, finds the name of the function. 85 - pub(crate) fn function_name(input: TokenStream) -> Option<Ident> { 86 - let mut input = input.into_iter(); 87 - while let Some(token) = input.next() { 88 - match token { 89 - TokenTree::Ident(i) if i.to_string() == "fn" => { 90 - if let Some(TokenTree::Ident(i)) = input.next() { 91 - return Some(i); 92 - } 93 - return None; 94 - } 95 - _ => continue, 96 - } 97 - } 98 - None 99 39 } 100 40 101 41 pub(crate) fn file() -> String { ··· 55 115 } 56 116 } 57 117 58 - /// Parse a token stream of the form `expected_name: "value",` and return the 59 - /// string in the position of "value". 60 - /// 61 - /// # Panics 62 - /// 63 - /// - On parse error. 64 - pub(crate) fn expect_string_field(it: &mut token_stream::IntoIter, expected_name: &str) -> String { 65 - assert_eq!(expect_ident(it), expected_name); 66 - assert_eq!(expect_punct(it), ':'); 67 - let string = expect_string(it); 68 - assert_eq!(expect_punct(it), ','); 69 - string 118 + /// Obtain all `#[cfg]` attributes. 119 + pub(crate) fn gather_cfg_attrs(attr: &[Attribute]) -> impl Iterator<Item = &Attribute> + '_ { 120 + attr.iter().filter(|a| a.path().is_ident("cfg")) 70 121 }
+119 -154
rust/macros/kunit.rs
··· 4 4 //! 5 5 //! Copyright (c) 2023 José Expósito <jose.exposito89@gmail.com> 6 6 7 - use proc_macro::{Delimiter, Group, TokenStream, TokenTree}; 8 - use std::collections::HashMap; 9 - use std::fmt::Write; 7 + use std::ffi::CString; 10 8 11 - pub(crate) fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream { 12 - let attr = attr.to_string(); 9 + use proc_macro2::TokenStream; 10 + use quote::{ 11 + format_ident, 12 + quote, 13 + ToTokens, // 14 + }; 15 + use syn::{ 16 + parse_quote, 17 + Error, 18 + Ident, 19 + Item, 20 + ItemMod, 21 + LitCStr, 22 + Result, // 23 + }; 13 24 14 - if attr.is_empty() { 15 - panic!("Missing test name in `#[kunit_tests(test_name)]` macro") 25 + pub(crate) fn kunit_tests(test_suite: Ident, mut module: ItemMod) -> Result<TokenStream> { 26 + if test_suite.to_string().len() > 255 { 27 + return Err(Error::new_spanned( 28 + test_suite, 29 + "test suite names cannot exceed the maximum length of 255 bytes", 30 + )); 16 31 } 17 32 18 - if attr.len() > 255 { 19 - panic!("The test suite name `{attr}` exceeds the maximum length of 255 bytes") 20 - } 21 - 22 - let mut tokens: Vec<_> = ts.into_iter().collect(); 23 - 24 - // Scan for the `mod` keyword. 25 - tokens 26 - .iter() 27 - .find_map(|token| match token { 28 - TokenTree::Ident(ident) => match ident.to_string().as_str() { 29 - "mod" => Some(true), 30 - _ => None, 31 - }, 32 - _ => None, 33 - }) 34 - .expect("`#[kunit_tests(test_name)]` attribute should only be applied to modules"); 35 - 36 - // Retrieve the main body. The main body should be the last token tree. 37 - let body = match tokens.pop() { 38 - Some(TokenTree::Group(group)) if group.delimiter() == Delimiter::Brace => group, 39 - _ => panic!("Cannot locate main body of module"), 33 + // We cannot handle modules that defer to another file (e.g. `mod foo;`). 34 + let Some((module_brace, module_items)) = module.content.take() else { 35 + Err(Error::new_spanned( 36 + module, 37 + "`#[kunit_tests(test_name)]` attribute should only be applied to inline modules", 38 + ))? 40 39 }; 41 40 42 - // Get the functions set as tests. Search for `[test]` -> `fn`. 43 - let mut body_it = body.stream().into_iter(); 44 - let mut tests = Vec::new(); 45 - let mut attributes: HashMap<String, TokenStream> = HashMap::new(); 46 - while let Some(token) = body_it.next() { 47 - match token { 48 - TokenTree::Punct(ref p) if p.as_char() == '#' => match body_it.next() { 49 - Some(TokenTree::Group(g)) if g.delimiter() == Delimiter::Bracket => { 50 - if let Some(TokenTree::Ident(name)) = g.stream().into_iter().next() { 51 - // Collect attributes because we need to find which are tests. We also 52 - // need to copy `cfg` attributes so tests can be conditionally enabled. 53 - attributes 54 - .entry(name.to_string()) 55 - .or_default() 56 - .extend([token, TokenTree::Group(g)]); 57 - } 58 - continue; 59 - } 60 - _ => (), 61 - }, 62 - TokenTree::Ident(i) if i.to_string() == "fn" && attributes.contains_key("test") => { 63 - if let Some(TokenTree::Ident(test_name)) = body_it.next() { 64 - tests.push((test_name, attributes.remove("cfg").unwrap_or_default())) 65 - } 66 - } 41 + // Make the entire module gated behind `CONFIG_KUNIT`. 42 + module 43 + .attrs 44 + .insert(0, parse_quote!(#[cfg(CONFIG_KUNIT="y")])); 67 45 68 - _ => (), 69 - } 70 - attributes.clear(); 71 - } 72 - 73 - // Add `#[cfg(CONFIG_KUNIT="y")]` before the module declaration. 74 - let config_kunit = "#[cfg(CONFIG_KUNIT=\"y\")]".to_owned().parse().unwrap(); 75 - tokens.insert( 76 - 0, 77 - TokenTree::Group(Group::new(Delimiter::None, config_kunit)), 78 - ); 46 + let mut processed_items = Vec::new(); 47 + let mut test_cases = Vec::new(); 79 48 80 49 // Generate the test KUnit test suite and a test case for each `#[test]`. 50 + // 81 51 // The code generated for the following test module: 82 52 // 83 53 // ``` ··· 72 102 // unsafe extern "C" fn kunit_rust_wrapper_bar(_test: *mut ::kernel::bindings::kunit) { bar(); } 73 103 // 74 104 // static mut TEST_CASES: [::kernel::bindings::kunit_case; 3] = [ 75 - // ::kernel::kunit::kunit_case(::kernel::c_str!("foo"), kunit_rust_wrapper_foo), 76 - // ::kernel::kunit::kunit_case(::kernel::c_str!("bar"), kunit_rust_wrapper_bar), 77 - // ::kernel::kunit::kunit_case_null(), 105 + // ::kernel::kunit::kunit_case(c"foo", kunit_rust_wrapper_foo), 106 + // ::kernel::kunit::kunit_case(c"bar", kunit_rust_wrapper_bar), 107 + // ::pin_init::zeroed(), 78 108 // ]; 79 109 // 80 110 // ::kernel::kunit_unsafe_test_suite!(kunit_test_suit_name, TEST_CASES); 81 111 // ``` 82 - let mut kunit_macros = "".to_owned(); 83 - let mut test_cases = "".to_owned(); 84 - let mut assert_macros = "".to_owned(); 85 - let path = crate::helpers::file(); 86 - let num_tests = tests.len(); 87 - for (test, cfg_attr) in tests { 88 - let kunit_wrapper_fn_name = format!("kunit_rust_wrapper_{test}"); 89 - // Append any `cfg` attributes the user might have written on their tests so we don't 90 - // attempt to call them when they are `cfg`'d out. An extra `use` is used here to reduce 91 - // the length of the assert message. 92 - let kunit_wrapper = format!( 93 - r#"unsafe extern "C" fn {kunit_wrapper_fn_name}(_test: *mut ::kernel::bindings::kunit) 94 - {{ 112 + // 113 + // Non-function items (e.g. imports) are preserved. 114 + for item in module_items { 115 + let Item::Fn(mut f) = item else { 116 + processed_items.push(item); 117 + continue; 118 + }; 119 + 120 + // TODO: Replace below with `extract_if` when MSRV is bumped above 1.85. 121 + let before_len = f.attrs.len(); 122 + f.attrs.retain(|attr| !attr.path().is_ident("test")); 123 + if f.attrs.len() == before_len { 124 + processed_items.push(Item::Fn(f)); 125 + continue; 126 + } 127 + 128 + let test = f.sig.ident.clone(); 129 + 130 + // Retrieve `#[cfg]` applied on the function which needs to be present on derived items too. 131 + let cfg_attrs: Vec<_> = f 132 + .attrs 133 + .iter() 134 + .filter(|attr| attr.path().is_ident("cfg")) 135 + .cloned() 136 + .collect(); 137 + 138 + // Before the test, override usual `assert!` and `assert_eq!` macros with ones that call 139 + // KUnit instead. 140 + let test_str = test.to_string(); 141 + let path = CString::new(crate::helpers::file()).expect("file path cannot contain NUL"); 142 + processed_items.push(parse_quote! { 143 + #[allow(unused)] 144 + macro_rules! assert { 145 + ($cond:expr $(,)?) => {{ 146 + kernel::kunit_assert!(#test_str, #path, 0, $cond); 147 + }} 148 + } 149 + }); 150 + processed_items.push(parse_quote! { 151 + #[allow(unused)] 152 + macro_rules! assert_eq { 153 + ($left:expr, $right:expr $(,)?) => {{ 154 + kernel::kunit_assert_eq!(#test_str, #path, 0, $left, $right); 155 + }} 156 + } 157 + }); 158 + 159 + // Add back the test item. 160 + processed_items.push(Item::Fn(f)); 161 + 162 + let kunit_wrapper_fn_name = format_ident!("kunit_rust_wrapper_{test}"); 163 + let test_cstr = LitCStr::new( 164 + &CString::new(test_str.as_str()).expect("identifier cannot contain NUL"), 165 + test.span(), 166 + ); 167 + processed_items.push(parse_quote! { 168 + unsafe extern "C" fn #kunit_wrapper_fn_name(_test: *mut ::kernel::bindings::kunit) { 95 169 (*_test).status = ::kernel::bindings::kunit_status_KUNIT_SKIPPED; 96 - {cfg_attr} {{ 170 + 171 + // Append any `cfg` attributes the user might have written on their tests so we 172 + // don't attempt to call them when they are `cfg`'d out. An extra `use` is used 173 + // here to reduce the length of the assert message. 174 + #(#cfg_attrs)* 175 + { 97 176 (*_test).status = ::kernel::bindings::kunit_status_KUNIT_SUCCESS; 98 177 use ::kernel::kunit::is_test_result_ok; 99 - assert!(is_test_result_ok({test}())); 100 - }} 101 - }}"#, 102 - ); 103 - writeln!(kunit_macros, "{kunit_wrapper}").unwrap(); 104 - writeln!( 105 - test_cases, 106 - " ::kernel::kunit::kunit_case(::kernel::c_str!(\"{test}\"), {kunit_wrapper_fn_name})," 107 - ) 108 - .unwrap(); 109 - writeln!( 110 - assert_macros, 111 - r#" 112 - /// Overrides the usual [`assert!`] macro with one that calls KUnit instead. 113 - #[allow(unused)] 114 - macro_rules! assert {{ 115 - ($cond:expr $(,)?) => {{{{ 116 - kernel::kunit_assert!("{test}", "{path}", 0, $cond); 117 - }}}} 118 - }} 119 - 120 - /// Overrides the usual [`assert_eq!`] macro with one that calls KUnit instead. 121 - #[allow(unused)] 122 - macro_rules! assert_eq {{ 123 - ($left:expr, $right:expr $(,)?) => {{{{ 124 - kernel::kunit_assert_eq!("{test}", "{path}", 0, $left, $right); 125 - }}}} 126 - }} 127 - "# 128 - ) 129 - .unwrap(); 130 - } 131 - 132 - writeln!(kunit_macros).unwrap(); 133 - writeln!( 134 - kunit_macros, 135 - "static mut TEST_CASES: [::kernel::bindings::kunit_case; {}] = [\n{test_cases} ::kernel::kunit::kunit_case_null(),\n];", 136 - num_tests + 1 137 - ) 138 - .unwrap(); 139 - 140 - writeln!( 141 - kunit_macros, 142 - "::kernel::kunit_unsafe_test_suite!({attr}, TEST_CASES);" 143 - ) 144 - .unwrap(); 145 - 146 - // Remove the `#[test]` macros. 147 - // We do this at a token level, in order to preserve span information. 148 - let mut new_body = vec![]; 149 - let mut body_it = body.stream().into_iter(); 150 - 151 - while let Some(token) = body_it.next() { 152 - match token { 153 - TokenTree::Punct(ref c) if c.as_char() == '#' => match body_it.next() { 154 - Some(TokenTree::Group(group)) if group.to_string() == "[test]" => (), 155 - Some(next) => { 156 - new_body.extend([token, next]); 178 + assert!(is_test_result_ok(#test())); 157 179 } 158 - _ => { 159 - new_body.push(token); 160 - } 161 - }, 162 - _ => { 163 - new_body.push(token); 164 180 } 165 - } 181 + }); 182 + 183 + test_cases.push(quote!( 184 + ::kernel::kunit::kunit_case(#test_cstr, #kunit_wrapper_fn_name) 185 + )); 166 186 } 167 187 168 - let mut final_body = TokenStream::new(); 169 - final_body.extend::<TokenStream>(assert_macros.parse().unwrap()); 170 - final_body.extend(new_body); 171 - final_body.extend::<TokenStream>(kunit_macros.parse().unwrap()); 188 + let num_tests_plus_1 = test_cases.len() + 1; 189 + processed_items.push(parse_quote! { 190 + static mut TEST_CASES: [::kernel::bindings::kunit_case; #num_tests_plus_1] = [ 191 + #(#test_cases,)* 192 + ::pin_init::zeroed(), 193 + ]; 194 + }); 195 + processed_items.push(parse_quote! { 196 + ::kernel::kunit_unsafe_test_suite!(#test_suite, TEST_CASES); 197 + }); 172 198 173 - tokens.push(TokenTree::Group(Group::new(Delimiter::Brace, final_body))); 174 - 175 - tokens.into_iter().collect() 199 + module.content = Some((module_brace, processed_items)); 200 + Ok(module.to_token_stream()) 176 201 }
+27 -16
rust/macros/lib.rs
··· 11 11 // to avoid depending on the full `proc_macro_span` on Rust >= 1.88.0. 12 12 #![cfg_attr(not(CONFIG_RUSTC_HAS_SPAN_FILE), feature(proc_macro_span))] 13 13 14 - #[macro_use] 15 - mod quote; 16 14 mod concat_idents; 17 15 mod export; 18 16 mod fmt; ··· 21 23 mod vtable; 22 24 23 25 use proc_macro::TokenStream; 26 + 27 + use syn::parse_macro_input; 24 28 25 29 /// Declares a kernel module. 26 30 /// ··· 131 131 /// - `firmware`: array of ASCII string literals of the firmware files of 132 132 /// the kernel module. 133 133 #[proc_macro] 134 - pub fn module(ts: TokenStream) -> TokenStream { 135 - module::module(ts) 134 + pub fn module(input: TokenStream) -> TokenStream { 135 + module::module(parse_macro_input!(input)) 136 + .unwrap_or_else(|e| e.into_compile_error()) 137 + .into() 136 138 } 137 139 138 140 /// Declares or implements a vtable trait. ··· 156 154 /// case the default implementation will never be executed. The reason for this 157 155 /// is that the functions will be called through function pointers installed in 158 156 /// C side vtables. When an optional method is not implemented on a `#[vtable]` 159 - /// trait, a NULL entry is installed in the vtable. Thus the default 157 + /// trait, a `NULL` entry is installed in the vtable. Thus the default 160 158 /// implementation is never called. Since these traits are not designed to be 161 159 /// used on the Rust side, it should not be possible to call the default 162 160 /// implementation. This is done to ensure that we call the vtable methods ··· 208 206 /// 209 207 /// [`kernel::error::VTABLE_DEFAULT_ERROR`]: ../kernel/error/constant.VTABLE_DEFAULT_ERROR.html 210 208 #[proc_macro_attribute] 211 - pub fn vtable(attr: TokenStream, ts: TokenStream) -> TokenStream { 212 - vtable::vtable(attr, ts) 209 + pub fn vtable(attr: TokenStream, input: TokenStream) -> TokenStream { 210 + parse_macro_input!(attr as syn::parse::Nothing); 211 + vtable::vtable(parse_macro_input!(input)) 212 + .unwrap_or_else(|e| e.into_compile_error()) 213 + .into() 213 214 } 214 215 215 216 /// Export a function so that C code can call it via a header file. ··· 234 229 /// This macro is *not* the same as the C macros `EXPORT_SYMBOL_*`. All Rust symbols are currently 235 230 /// automatically exported with `EXPORT_SYMBOL_GPL`. 236 231 #[proc_macro_attribute] 237 - pub fn export(attr: TokenStream, ts: TokenStream) -> TokenStream { 238 - export::export(attr, ts) 232 + pub fn export(attr: TokenStream, input: TokenStream) -> TokenStream { 233 + parse_macro_input!(attr as syn::parse::Nothing); 234 + export::export(parse_macro_input!(input)).into() 239 235 } 240 236 241 237 /// Like [`core::format_args!`], but automatically wraps arguments in [`kernel::fmt::Adapter`]. ··· 254 248 /// [`pr_info!`]: ../kernel/macro.pr_info.html 255 249 #[proc_macro] 256 250 pub fn fmt(input: TokenStream) -> TokenStream { 257 - fmt::fmt(input) 251 + fmt::fmt(input.into()).into() 258 252 } 259 253 260 254 /// Concatenate two identifiers. ··· 311 305 /// assert_eq!(BR_OK, binder_driver_return_protocol_BR_OK); 312 306 /// ``` 313 307 #[proc_macro] 314 - pub fn concat_idents(ts: TokenStream) -> TokenStream { 315 - concat_idents::concat_idents(ts) 308 + pub fn concat_idents(input: TokenStream) -> TokenStream { 309 + concat_idents::concat_idents(parse_macro_input!(input)).into() 316 310 } 317 311 318 312 /// Paste identifiers together. ··· 450 444 /// [`paste`]: https://docs.rs/paste/ 451 445 #[proc_macro] 452 446 pub fn paste(input: TokenStream) -> TokenStream { 453 - let mut tokens = input.into_iter().collect(); 447 + let mut tokens = proc_macro2::TokenStream::from(input).into_iter().collect(); 454 448 paste::expand(&mut tokens); 455 - tokens.into_iter().collect() 449 + tokens 450 + .into_iter() 451 + .collect::<proc_macro2::TokenStream>() 452 + .into() 456 453 } 457 454 458 455 /// Registers a KUnit test suite and its test cases using a user-space like syntax. ··· 481 472 /// } 482 473 /// ``` 483 474 #[proc_macro_attribute] 484 - pub fn kunit_tests(attr: TokenStream, ts: TokenStream) -> TokenStream { 485 - kunit::kunit_tests(attr, ts) 475 + pub fn kunit_tests(attr: TokenStream, input: TokenStream) -> TokenStream { 476 + kunit::kunit_tests(parse_macro_input!(attr), parse_macro_input!(input)) 477 + .unwrap_or_else(|e| e.into_compile_error()) 478 + .into() 486 479 }
+490 -377
rust/macros/module.rs
··· 1 1 // SPDX-License-Identifier: GPL-2.0 2 2 3 + use std::ffi::CString; 4 + 5 + use proc_macro2::{ 6 + Literal, 7 + TokenStream, // 8 + }; 9 + use quote::{ 10 + format_ident, 11 + quote, // 12 + }; 13 + use syn::{ 14 + braced, 15 + bracketed, 16 + ext::IdentExt, 17 + parse::{ 18 + Parse, 19 + ParseStream, // 20 + }, 21 + parse_quote, 22 + punctuated::Punctuated, 23 + Error, 24 + Expr, 25 + Ident, 26 + LitStr, 27 + Path, 28 + Result, 29 + Token, 30 + Type, // 31 + }; 32 + 3 33 use crate::helpers::*; 4 - use proc_macro::{token_stream, Delimiter, Literal, TokenStream, TokenTree}; 5 - use std::fmt::Write; 6 - 7 - fn expect_string_array(it: &mut token_stream::IntoIter) -> Vec<String> { 8 - let group = expect_group(it); 9 - assert_eq!(group.delimiter(), Delimiter::Bracket); 10 - let mut values = Vec::new(); 11 - let mut it = group.stream().into_iter(); 12 - 13 - while let Some(val) = try_string(&mut it) { 14 - assert!(val.is_ascii(), "Expected ASCII string"); 15 - values.push(val); 16 - match it.next() { 17 - Some(TokenTree::Punct(punct)) => assert_eq!(punct.as_char(), ','), 18 - None => break, 19 - _ => panic!("Expected ',' or end of array"), 20 - } 21 - } 22 - values 23 - } 24 34 25 35 struct ModInfoBuilder<'a> { 26 36 module: &'a str, 27 37 counter: usize, 28 - buffer: String, 29 - param_buffer: String, 38 + ts: TokenStream, 39 + param_ts: TokenStream, 30 40 } 31 41 32 42 impl<'a> ModInfoBuilder<'a> { ··· 44 34 ModInfoBuilder { 45 35 module, 46 36 counter: 0, 47 - buffer: String::new(), 48 - param_buffer: String::new(), 37 + ts: TokenStream::new(), 38 + param_ts: TokenStream::new(), 49 39 } 50 40 } 51 41 ··· 62 52 // Loadable modules' modinfo strings go as-is. 63 53 format!("{field}={content}\0") 64 54 }; 65 - 66 - let buffer = if param { 67 - &mut self.param_buffer 55 + let length = string.len(); 56 + let string = Literal::byte_string(string.as_bytes()); 57 + let cfg = if builtin { 58 + quote!(#[cfg(not(MODULE))]) 68 59 } else { 69 - &mut self.buffer 60 + quote!(#[cfg(MODULE)]) 70 61 }; 71 62 72 - write!( 73 - buffer, 74 - " 75 - {cfg} 76 - #[doc(hidden)] 77 - #[cfg_attr(not(target_os = \"macos\"), link_section = \".modinfo\")] 78 - #[used(compiler)] 79 - pub static __{module}_{counter}: [u8; {length}] = *{string}; 80 - ", 81 - cfg = if builtin { 82 - "#[cfg(not(MODULE))]" 83 - } else { 84 - "#[cfg(MODULE)]" 85 - }, 63 + let counter = format_ident!( 64 + "__{module}_{counter}", 86 65 module = self.module.to_uppercase(), 87 - counter = self.counter, 88 - length = string.len(), 89 - string = Literal::byte_string(string.as_bytes()), 90 - ) 91 - .unwrap(); 66 + counter = self.counter 67 + ); 68 + let item = quote! { 69 + #cfg 70 + #[cfg_attr(not(target_os = "macos"), link_section = ".modinfo")] 71 + #[used(compiler)] 72 + pub static #counter: [u8; #length] = *#string; 73 + }; 74 + 75 + if param { 76 + self.param_ts.extend(item); 77 + } else { 78 + self.ts.extend(item); 79 + } 92 80 93 81 self.counter += 1; 94 82 } ··· 119 111 }; 120 112 121 113 for param in params { 122 - let ops = param_ops_path(&param.ptype); 114 + let param_name_str = param.name.to_string(); 115 + let param_type_str = param.ptype.to_string(); 116 + 117 + let ops = param_ops_path(&param_type_str); 123 118 124 119 // Note: The spelling of these fields is dictated by the user space 125 120 // tool `modinfo`. 126 - self.emit_param("parmtype", &param.name, &param.ptype); 127 - self.emit_param("parm", &param.name, &param.description); 121 + self.emit_param("parmtype", &param_name_str, &param_type_str); 122 + self.emit_param("parm", &param_name_str, &param.description.value()); 128 123 129 - write!( 130 - self.param_buffer, 131 - " 132 - pub(crate) static {param_name}: 133 - ::kernel::module_param::ModuleParamAccess<{param_type}> = 134 - ::kernel::module_param::ModuleParamAccess::new({param_default}); 124 + let static_name = format_ident!("__{}_{}_struct", self.module, param.name); 125 + let param_name_cstr = 126 + CString::new(param_name_str).expect("name contains NUL-terminator"); 127 + let param_name_cstr_with_module = 128 + CString::new(format!("{}.{}", self.module, param.name)) 129 + .expect("name contains NUL-terminator"); 135 130 136 - const _: () = {{ 137 - #[link_section = \"__param\"] 138 - #[used] 139 - static __{module_name}_{param_name}_struct: 131 + let param_name = &param.name; 132 + let param_type = &param.ptype; 133 + let param_default = &param.default; 134 + 135 + self.param_ts.extend(quote! { 136 + #[allow(non_upper_case_globals)] 137 + pub(crate) static #param_name: 138 + ::kernel::module_param::ModuleParamAccess<#param_type> = 139 + ::kernel::module_param::ModuleParamAccess::new(#param_default); 140 + 141 + const _: () = { 142 + #[allow(non_upper_case_globals)] 143 + #[link_section = "__param"] 144 + #[used(compiler)] 145 + static #static_name: 140 146 ::kernel::module_param::KernelParam = 141 147 ::kernel::module_param::KernelParam::new( 142 - ::kernel::bindings::kernel_param {{ 143 - name: if ::core::cfg!(MODULE) {{ 144 - ::kernel::c_str!(\"{param_name}\").to_bytes_with_nul() 145 - }} else {{ 146 - ::kernel::c_str!(\"{module_name}.{param_name}\") 147 - .to_bytes_with_nul() 148 - }}.as_ptr(), 148 + ::kernel::bindings::kernel_param { 149 + name: kernel::str::as_char_ptr_in_const_context( 150 + if ::core::cfg!(MODULE) { 151 + #param_name_cstr 152 + } else { 153 + #param_name_cstr_with_module 154 + } 155 + ), 149 156 // SAFETY: `__this_module` is constructed by the kernel at load 150 157 // time and will not be freed until the module is unloaded. 151 158 #[cfg(MODULE)] 152 - mod_: unsafe {{ 159 + mod_: unsafe { 153 160 core::ptr::from_ref(&::kernel::bindings::__this_module) 154 161 .cast_mut() 155 - }}, 162 + }, 156 163 #[cfg(not(MODULE))] 157 164 mod_: ::core::ptr::null_mut(), 158 - ops: core::ptr::from_ref(&{ops}), 165 + ops: core::ptr::from_ref(&#ops), 159 166 perm: 0, // Will not appear in sysfs 160 167 level: -1, 161 168 flags: 0, 162 - __bindgen_anon_1: ::kernel::bindings::kernel_param__bindgen_ty_1 {{ 163 - arg: {param_name}.as_void_ptr() 164 - }}, 165 - }} 169 + __bindgen_anon_1: ::kernel::bindings::kernel_param__bindgen_ty_1 { 170 + arg: #param_name.as_void_ptr() 171 + }, 172 + } 166 173 ); 167 - }}; 168 - ", 169 - module_name = info.name, 170 - param_type = param.ptype, 171 - param_default = param.default, 172 - param_name = param.name, 173 - ops = ops, 174 - ) 175 - .unwrap(); 174 + }; 175 + }); 176 176 } 177 177 } 178 178 } 179 179 180 - fn param_ops_path(param_type: &str) -> &'static str { 180 + fn param_ops_path(param_type: &str) -> Path { 181 181 match param_type { 182 - "i8" => "::kernel::module_param::PARAM_OPS_I8", 183 - "u8" => "::kernel::module_param::PARAM_OPS_U8", 184 - "i16" => "::kernel::module_param::PARAM_OPS_I16", 185 - "u16" => "::kernel::module_param::PARAM_OPS_U16", 186 - "i32" => "::kernel::module_param::PARAM_OPS_I32", 187 - "u32" => "::kernel::module_param::PARAM_OPS_U32", 188 - "i64" => "::kernel::module_param::PARAM_OPS_I64", 189 - "u64" => "::kernel::module_param::PARAM_OPS_U64", 190 - "isize" => "::kernel::module_param::PARAM_OPS_ISIZE", 191 - "usize" => "::kernel::module_param::PARAM_OPS_USIZE", 182 + "i8" => parse_quote!(::kernel::module_param::PARAM_OPS_I8), 183 + "u8" => parse_quote!(::kernel::module_param::PARAM_OPS_U8), 184 + "i16" => parse_quote!(::kernel::module_param::PARAM_OPS_I16), 185 + "u16" => parse_quote!(::kernel::module_param::PARAM_OPS_U16), 186 + "i32" => parse_quote!(::kernel::module_param::PARAM_OPS_I32), 187 + "u32" => parse_quote!(::kernel::module_param::PARAM_OPS_U32), 188 + "i64" => parse_quote!(::kernel::module_param::PARAM_OPS_I64), 189 + "u64" => parse_quote!(::kernel::module_param::PARAM_OPS_U64), 190 + "isize" => parse_quote!(::kernel::module_param::PARAM_OPS_ISIZE), 191 + "usize" => parse_quote!(::kernel::module_param::PARAM_OPS_USIZE), 192 192 t => panic!("Unsupported parameter type {}", t), 193 193 } 194 194 } 195 195 196 - fn expect_param_default(param_it: &mut token_stream::IntoIter) -> String { 197 - assert_eq!(expect_ident(param_it), "default"); 198 - assert_eq!(expect_punct(param_it), ':'); 199 - let sign = try_sign(param_it); 200 - let default = try_literal(param_it).expect("Expected default param value"); 201 - assert_eq!(expect_punct(param_it), ','); 202 - let mut value = sign.map(String::from).unwrap_or_default(); 203 - value.push_str(&default); 204 - value 205 - } 196 + /// Parse fields that are required to use a specific order. 197 + /// 198 + /// As fields must follow a specific order, we *could* just parse fields one by one by peeking. 199 + /// However the error message generated when implementing that way is not very friendly. 200 + /// 201 + /// So instead we parse fields in an arbitrary order, but only enforce the ordering after parsing, 202 + /// and if the wrong order is used, the proper order is communicated to the user with error message. 203 + /// 204 + /// Usage looks like this: 205 + /// ```ignore 206 + /// parse_ordered_fields! { 207 + /// from input; 208 + /// 209 + /// // This will extract "foo: <field>" into a variable named "foo". 210 + /// // The variable will have type `Option<_>`. 211 + /// foo => <expression that parses the field>, 212 + /// 213 + /// // If you need the variable name to be different than the key name. 214 + /// // This extracts "baz: <field>" into a variable named "bar". 215 + /// // You might want this if "baz" is a keyword. 216 + /// baz as bar => <expression that parse the field>, 217 + /// 218 + /// // You can mark a key as required, and the variable will no longer be `Option`. 219 + /// // foobar will be of type `Expr` instead of `Option<Expr>`. 220 + /// foobar [required] => input.parse::<Expr>()?, 221 + /// } 222 + /// ``` 223 + macro_rules! parse_ordered_fields { 224 + (@gen 225 + [$input:expr] 226 + [$([$name:ident; $key:ident; $parser:expr])*] 227 + [$([$req_name:ident; $req_key:ident])*] 228 + ) => { 229 + $(let mut $name = None;)* 206 230 207 - #[derive(Debug, Default)] 208 - struct ModuleInfo { 209 - type_: String, 210 - license: String, 211 - name: String, 212 - authors: Option<Vec<String>>, 213 - description: Option<String>, 214 - alias: Option<Vec<String>>, 215 - firmware: Option<Vec<String>>, 216 - imports_ns: Option<Vec<String>>, 217 - params: Option<Vec<Parameter>>, 218 - } 231 + const EXPECTED_KEYS: &[&str] = &[$(stringify!($key),)*]; 232 + const REQUIRED_KEYS: &[&str] = &[$(stringify!($req_key),)*]; 219 233 220 - #[derive(Debug)] 221 - struct Parameter { 222 - name: String, 223 - ptype: String, 224 - default: String, 225 - description: String, 226 - } 227 - 228 - fn expect_params(it: &mut token_stream::IntoIter) -> Vec<Parameter> { 229 - let params = expect_group(it); 230 - assert_eq!(params.delimiter(), Delimiter::Brace); 231 - let mut it = params.stream().into_iter(); 232 - let mut parsed = Vec::new(); 233 - 234 - loop { 235 - let param_name = match it.next() { 236 - Some(TokenTree::Ident(ident)) => ident.to_string(), 237 - Some(_) => panic!("Expected Ident or end"), 238 - None => break, 239 - }; 240 - 241 - assert_eq!(expect_punct(&mut it), ':'); 242 - let param_type = expect_ident(&mut it); 243 - let group = expect_group(&mut it); 244 - assert_eq!(group.delimiter(), Delimiter::Brace); 245 - assert_eq!(expect_punct(&mut it), ','); 246 - 247 - let mut param_it = group.stream().into_iter(); 248 - let param_default = expect_param_default(&mut param_it); 249 - let param_description = expect_string_field(&mut param_it, "description"); 250 - expect_end(&mut param_it); 251 - 252 - parsed.push(Parameter { 253 - name: param_name, 254 - ptype: param_type, 255 - default: param_default, 256 - description: param_description, 257 - }) 258 - } 259 - 260 - parsed 261 - } 262 - 263 - impl ModuleInfo { 264 - fn parse(it: &mut token_stream::IntoIter) -> Self { 265 - let mut info = ModuleInfo::default(); 266 - 267 - const EXPECTED_KEYS: &[&str] = &[ 268 - "type", 269 - "name", 270 - "authors", 271 - "description", 272 - "license", 273 - "alias", 274 - "firmware", 275 - "imports_ns", 276 - "params", 277 - ]; 278 - const REQUIRED_KEYS: &[&str] = &["type", "name", "license"]; 234 + let span = $input.span(); 279 235 let mut seen_keys = Vec::new(); 280 236 281 - loop { 282 - let key = match it.next() { 283 - Some(TokenTree::Ident(ident)) => ident.to_string(), 284 - Some(_) => panic!("Expected Ident or end"), 285 - None => break, 286 - }; 237 + while !$input.is_empty() { 238 + let key = $input.call(Ident::parse_any)?; 287 239 288 240 if seen_keys.contains(&key) { 289 - panic!("Duplicated key \"{key}\". Keys can only be specified once."); 241 + Err(Error::new_spanned( 242 + &key, 243 + format!(r#"duplicated key "{key}". Keys can only be specified once."#), 244 + ))? 290 245 } 291 246 292 - assert_eq!(expect_punct(it), ':'); 247 + $input.parse::<Token![:]>()?; 293 248 294 - match key.as_str() { 295 - "type" => info.type_ = expect_ident(it), 296 - "name" => info.name = expect_string_ascii(it), 297 - "authors" => info.authors = Some(expect_string_array(it)), 298 - "description" => info.description = Some(expect_string(it)), 299 - "license" => info.license = expect_string_ascii(it), 300 - "alias" => info.alias = Some(expect_string_array(it)), 301 - "firmware" => info.firmware = Some(expect_string_array(it)), 302 - "imports_ns" => info.imports_ns = Some(expect_string_array(it)), 303 - "params" => info.params = Some(expect_params(it)), 304 - _ => panic!("Unknown key \"{key}\". Valid keys are: {EXPECTED_KEYS:?}."), 249 + match &*key.to_string() { 250 + $( 251 + stringify!($key) => $name = Some($parser), 252 + )* 253 + _ => { 254 + Err(Error::new_spanned( 255 + &key, 256 + format!(r#"unknown key "{key}". Valid keys are: {EXPECTED_KEYS:?}."#), 257 + ))? 258 + } 305 259 } 306 260 307 - assert_eq!(expect_punct(it), ','); 308 - 261 + $input.parse::<Token![,]>()?; 309 262 seen_keys.push(key); 310 263 } 311 264 312 - expect_end(it); 313 - 314 265 for key in REQUIRED_KEYS { 315 266 if !seen_keys.iter().any(|e| e == key) { 316 - panic!("Missing required key \"{key}\"."); 267 + Err(Error::new(span, format!(r#"missing required key "{key}""#)))? 317 268 } 318 269 } 319 270 ··· 284 317 } 285 318 286 319 if seen_keys != ordered_keys { 287 - panic!("Keys are not ordered as expected. Order them like: {ordered_keys:?}."); 320 + Err(Error::new( 321 + span, 322 + format!(r#"keys are not ordered as expected. Order them like: {ordered_keys:?}."#), 323 + ))? 288 324 } 289 325 290 - info 326 + $(let $req_name = $req_name.expect("required field");)* 327 + }; 328 + 329 + // Handle required fields. 330 + (@gen 331 + [$input:expr] [$($tok:tt)*] [$($req:tt)*] 332 + $key:ident as $name:ident [required] => $parser:expr, 333 + $($rest:tt)* 334 + ) => { 335 + parse_ordered_fields!( 336 + @gen [$input] [$($tok)* [$name; $key; $parser]] [$($req)* [$name; $key]] $($rest)* 337 + ) 338 + }; 339 + (@gen 340 + [$input:expr] [$($tok:tt)*] [$($req:tt)*] 341 + $name:ident [required] => $parser:expr, 342 + $($rest:tt)* 343 + ) => { 344 + parse_ordered_fields!( 345 + @gen [$input] [$($tok)* [$name; $name; $parser]] [$($req)* [$name; $name]] $($rest)* 346 + ) 347 + }; 348 + 349 + // Handle optional fields. 350 + (@gen 351 + [$input:expr] [$($tok:tt)*] [$($req:tt)*] 352 + $key:ident as $name:ident => $parser:expr, 353 + $($rest:tt)* 354 + ) => { 355 + parse_ordered_fields!( 356 + @gen [$input] [$($tok)* [$name; $key; $parser]] [$($req)*] $($rest)* 357 + ) 358 + }; 359 + (@gen 360 + [$input:expr] [$($tok:tt)*] [$($req:tt)*] 361 + $name:ident => $parser:expr, 362 + $($rest:tt)* 363 + ) => { 364 + parse_ordered_fields!( 365 + @gen [$input] [$($tok)* [$name; $name; $parser]] [$($req)*] $($rest)* 366 + ) 367 + }; 368 + 369 + (from $input:expr; $($tok:tt)*) => { 370 + parse_ordered_fields!(@gen [$input] [] [] $($tok)*) 291 371 } 292 372 } 293 373 294 - pub(crate) fn module(ts: TokenStream) -> TokenStream { 295 - let mut it = ts.into_iter(); 374 + struct Parameter { 375 + name: Ident, 376 + ptype: Ident, 377 + default: Expr, 378 + description: LitStr, 379 + } 296 380 297 - let info = ModuleInfo::parse(&mut it); 381 + impl Parse for Parameter { 382 + fn parse(input: ParseStream<'_>) -> Result<Self> { 383 + let name = input.parse()?; 384 + input.parse::<Token![:]>()?; 385 + let ptype = input.parse()?; 386 + 387 + let fields; 388 + braced!(fields in input); 389 + 390 + parse_ordered_fields! { 391 + from fields; 392 + default [required] => fields.parse()?, 393 + description [required] => fields.parse()?, 394 + } 395 + 396 + Ok(Self { 397 + name, 398 + ptype, 399 + default, 400 + description, 401 + }) 402 + } 403 + } 404 + 405 + pub(crate) struct ModuleInfo { 406 + type_: Type, 407 + license: AsciiLitStr, 408 + name: AsciiLitStr, 409 + authors: Option<Punctuated<AsciiLitStr, Token![,]>>, 410 + description: Option<LitStr>, 411 + alias: Option<Punctuated<AsciiLitStr, Token![,]>>, 412 + firmware: Option<Punctuated<AsciiLitStr, Token![,]>>, 413 + imports_ns: Option<Punctuated<AsciiLitStr, Token![,]>>, 414 + params: Option<Punctuated<Parameter, Token![,]>>, 415 + } 416 + 417 + impl Parse for ModuleInfo { 418 + fn parse(input: ParseStream<'_>) -> Result<Self> { 419 + parse_ordered_fields!( 420 + from input; 421 + type as type_ [required] => input.parse()?, 422 + name [required] => input.parse()?, 423 + authors => { 424 + let list; 425 + bracketed!(list in input); 426 + Punctuated::parse_terminated(&list)? 427 + }, 428 + description => input.parse()?, 429 + license [required] => input.parse()?, 430 + alias => { 431 + let list; 432 + bracketed!(list in input); 433 + Punctuated::parse_terminated(&list)? 434 + }, 435 + firmware => { 436 + let list; 437 + bracketed!(list in input); 438 + Punctuated::parse_terminated(&list)? 439 + }, 440 + imports_ns => { 441 + let list; 442 + bracketed!(list in input); 443 + Punctuated::parse_terminated(&list)? 444 + }, 445 + params => { 446 + let list; 447 + braced!(list in input); 448 + Punctuated::parse_terminated(&list)? 449 + }, 450 + ); 451 + 452 + Ok(ModuleInfo { 453 + type_, 454 + license, 455 + name, 456 + authors, 457 + description, 458 + alias, 459 + firmware, 460 + imports_ns, 461 + params, 462 + }) 463 + } 464 + } 465 + 466 + pub(crate) fn module(info: ModuleInfo) -> Result<TokenStream> { 467 + let ModuleInfo { 468 + type_, 469 + license, 470 + name, 471 + authors, 472 + description, 473 + alias, 474 + firmware, 475 + imports_ns, 476 + params: _, 477 + } = &info; 298 478 299 479 // Rust does not allow hyphens in identifiers, use underscore instead. 300 - let ident = info.name.replace('-', "_"); 480 + let ident = name.value().replace('-', "_"); 301 481 let mut modinfo = ModInfoBuilder::new(ident.as_ref()); 302 - if let Some(authors) = &info.authors { 482 + if let Some(authors) = authors { 303 483 for author in authors { 304 - modinfo.emit("author", author); 484 + modinfo.emit("author", &author.value()); 305 485 } 306 486 } 307 - if let Some(description) = &info.description { 308 - modinfo.emit("description", description); 487 + if let Some(description) = description { 488 + modinfo.emit("description", &description.value()); 309 489 } 310 - modinfo.emit("license", &info.license); 311 - if let Some(aliases) = &info.alias { 490 + modinfo.emit("license", &license.value()); 491 + if let Some(aliases) = alias { 312 492 for alias in aliases { 313 - modinfo.emit("alias", alias); 493 + modinfo.emit("alias", &alias.value()); 314 494 } 315 495 } 316 - if let Some(firmware) = &info.firmware { 496 + if let Some(firmware) = firmware { 317 497 for fw in firmware { 318 - modinfo.emit("firmware", fw); 498 + modinfo.emit("firmware", &fw.value()); 319 499 } 320 500 } 321 - if let Some(imports) = &info.imports_ns { 501 + if let Some(imports) = imports_ns { 322 502 for ns in imports { 323 - modinfo.emit("import_ns", ns); 503 + modinfo.emit("import_ns", &ns.value()); 324 504 } 325 505 } 326 506 ··· 478 364 479 365 modinfo.emit_params(&info); 480 366 481 - format!( 482 - " 483 - /// The module name. 484 - /// 485 - /// Used by the printing macros, e.g. [`info!`]. 486 - const __LOG_PREFIX: &[u8] = b\"{name}\\0\"; 367 + let modinfo_ts = modinfo.ts; 368 + let params_ts = modinfo.param_ts; 487 369 488 - // SAFETY: `__this_module` is constructed by the kernel at load time and will not be 489 - // freed until the module is unloaded. 490 - #[cfg(MODULE)] 491 - static THIS_MODULE: ::kernel::ThisModule = unsafe {{ 492 - extern \"C\" {{ 493 - static __this_module: ::kernel::types::Opaque<::kernel::bindings::module>; 494 - }} 370 + let ident_init = format_ident!("__{ident}_init"); 371 + let ident_exit = format_ident!("__{ident}_exit"); 372 + let ident_initcall = format_ident!("__{ident}_initcall"); 373 + let initcall_section = ".initcall6.init"; 495 374 496 - ::kernel::ThisModule::from_ptr(__this_module.get()) 497 - }}; 498 - #[cfg(not(MODULE))] 499 - static THIS_MODULE: ::kernel::ThisModule = unsafe {{ 500 - ::kernel::ThisModule::from_ptr(::core::ptr::null_mut()) 501 - }}; 375 + let global_asm = format!( 376 + r#".section "{initcall_section}", "a" 377 + __{ident}_initcall: 378 + .long __{ident}_init - . 379 + .previous 380 + "# 381 + ); 502 382 503 - /// The `LocalModule` type is the type of the module created by `module!`, 504 - /// `module_pci_driver!`, `module_platform_driver!`, etc. 505 - type LocalModule = {type_}; 383 + let name_cstr = CString::new(name.value()).expect("name contains NUL-terminator"); 506 384 507 - impl ::kernel::ModuleMetadata for {type_} {{ 508 - const NAME: &'static ::kernel::str::CStr = c\"{name}\"; 509 - }} 385 + Ok(quote! { 386 + /// The module name. 387 + /// 388 + /// Used by the printing macros, e.g. [`info!`]. 389 + const __LOG_PREFIX: &[u8] = #name_cstr.to_bytes_with_nul(); 510 390 511 - // Double nested modules, since then nobody can access the public items inside. 512 - mod __module_init {{ 513 - mod __module_init {{ 514 - use super::super::{type_}; 515 - use pin_init::PinInit; 391 + // SAFETY: `__this_module` is constructed by the kernel at load time and will not be 392 + // freed until the module is unloaded. 393 + #[cfg(MODULE)] 394 + static THIS_MODULE: ::kernel::ThisModule = unsafe { 395 + extern "C" { 396 + static __this_module: ::kernel::types::Opaque<::kernel::bindings::module>; 397 + }; 516 398 517 - /// The \"Rust loadable module\" mark. 518 - // 519 - // This may be best done another way later on, e.g. as a new modinfo 520 - // key or a new section. For the moment, keep it simple. 521 - #[cfg(MODULE)] 522 - #[doc(hidden)] 523 - #[used(compiler)] 524 - static __IS_RUST_MODULE: () = (); 399 + ::kernel::ThisModule::from_ptr(__this_module.get()) 400 + }; 525 401 526 - static mut __MOD: ::core::mem::MaybeUninit<{type_}> = 527 - ::core::mem::MaybeUninit::uninit(); 402 + #[cfg(not(MODULE))] 403 + static THIS_MODULE: ::kernel::ThisModule = unsafe { 404 + ::kernel::ThisModule::from_ptr(::core::ptr::null_mut()) 405 + }; 528 406 529 - // Loadable modules need to export the `{{init,cleanup}}_module` identifiers. 530 - /// # Safety 531 - /// 532 - /// This function must not be called after module initialization, because it may be 533 - /// freed after that completes. 534 - #[cfg(MODULE)] 535 - #[doc(hidden)] 536 - #[no_mangle] 537 - #[link_section = \".init.text\"] 538 - pub unsafe extern \"C\" fn init_module() -> ::kernel::ffi::c_int {{ 539 - // SAFETY: This function is inaccessible to the outside due to the double 540 - // module wrapping it. It is called exactly once by the C side via its 541 - // unique name. 542 - unsafe {{ __init() }} 543 - }} 407 + /// The `LocalModule` type is the type of the module created by `module!`, 408 + /// `module_pci_driver!`, `module_platform_driver!`, etc. 409 + type LocalModule = #type_; 544 410 545 - #[cfg(MODULE)] 546 - #[doc(hidden)] 547 - #[used(compiler)] 548 - #[link_section = \".init.data\"] 549 - static __UNIQUE_ID___addressable_init_module: unsafe extern \"C\" fn() -> i32 = init_module; 411 + impl ::kernel::ModuleMetadata for #type_ { 412 + const NAME: &'static ::kernel::str::CStr = #name_cstr; 413 + } 550 414 551 - #[cfg(MODULE)] 552 - #[doc(hidden)] 553 - #[no_mangle] 554 - #[link_section = \".exit.text\"] 555 - pub extern \"C\" fn cleanup_module() {{ 556 - // SAFETY: 557 - // - This function is inaccessible to the outside due to the double 558 - // module wrapping it. It is called exactly once by the C side via its 559 - // unique name, 560 - // - furthermore it is only called after `init_module` has returned `0` 561 - // (which delegates to `__init`). 562 - unsafe {{ __exit() }} 563 - }} 415 + // Double nested modules, since then nobody can access the public items inside. 416 + #[doc(hidden)] 417 + mod __module_init { 418 + mod __module_init { 419 + use pin_init::PinInit; 564 420 565 - #[cfg(MODULE)] 566 - #[doc(hidden)] 567 - #[used(compiler)] 568 - #[link_section = \".exit.data\"] 569 - static __UNIQUE_ID___addressable_cleanup_module: extern \"C\" fn() = cleanup_module; 421 + /// The "Rust loadable module" mark. 422 + // 423 + // This may be best done another way later on, e.g. as a new modinfo 424 + // key or a new section. For the moment, keep it simple. 425 + #[cfg(MODULE)] 426 + #[used(compiler)] 427 + static __IS_RUST_MODULE: () = (); 570 428 571 - // Built-in modules are initialized through an initcall pointer 572 - // and the identifiers need to be unique. 573 - #[cfg(not(MODULE))] 574 - #[cfg(not(CONFIG_HAVE_ARCH_PREL32_RELOCATIONS))] 575 - #[doc(hidden)] 576 - #[link_section = \"{initcall_section}\"] 577 - #[used(compiler)] 578 - pub static __{ident}_initcall: extern \"C\" fn() -> 579 - ::kernel::ffi::c_int = __{ident}_init; 429 + static mut __MOD: ::core::mem::MaybeUninit<super::super::LocalModule> = 430 + ::core::mem::MaybeUninit::uninit(); 580 431 581 - #[cfg(not(MODULE))] 582 - #[cfg(CONFIG_HAVE_ARCH_PREL32_RELOCATIONS)] 583 - ::core::arch::global_asm!( 584 - r#\".section \"{initcall_section}\", \"a\" 585 - __{ident}_initcall: 586 - .long __{ident}_init - . 587 - .previous 588 - \"# 432 + // Loadable modules need to export the `{init,cleanup}_module` identifiers. 433 + /// # Safety 434 + /// 435 + /// This function must not be called after module initialization, because it may be 436 + /// freed after that completes. 437 + #[cfg(MODULE)] 438 + #[no_mangle] 439 + #[link_section = ".init.text"] 440 + pub unsafe extern "C" fn init_module() -> ::kernel::ffi::c_int { 441 + // SAFETY: This function is inaccessible to the outside due to the double 442 + // module wrapping it. It is called exactly once by the C side via its 443 + // unique name. 444 + unsafe { __init() } 445 + } 446 + 447 + #[cfg(MODULE)] 448 + #[used(compiler)] 449 + #[link_section = ".init.data"] 450 + static __UNIQUE_ID___addressable_init_module: unsafe extern "C" fn() -> i32 = 451 + init_module; 452 + 453 + #[cfg(MODULE)] 454 + #[no_mangle] 455 + #[link_section = ".exit.text"] 456 + pub extern "C" fn cleanup_module() { 457 + // SAFETY: 458 + // - This function is inaccessible to the outside due to the double 459 + // module wrapping it. It is called exactly once by the C side via its 460 + // unique name, 461 + // - furthermore it is only called after `init_module` has returned `0` 462 + // (which delegates to `__init`). 463 + unsafe { __exit() } 464 + } 465 + 466 + #[cfg(MODULE)] 467 + #[used(compiler)] 468 + #[link_section = ".exit.data"] 469 + static __UNIQUE_ID___addressable_cleanup_module: extern "C" fn() = cleanup_module; 470 + 471 + // Built-in modules are initialized through an initcall pointer 472 + // and the identifiers need to be unique. 473 + #[cfg(not(MODULE))] 474 + #[cfg(not(CONFIG_HAVE_ARCH_PREL32_RELOCATIONS))] 475 + #[link_section = #initcall_section] 476 + #[used(compiler)] 477 + pub static #ident_initcall: extern "C" fn() -> 478 + ::kernel::ffi::c_int = #ident_init; 479 + 480 + #[cfg(not(MODULE))] 481 + #[cfg(CONFIG_HAVE_ARCH_PREL32_RELOCATIONS)] 482 + ::core::arch::global_asm!(#global_asm); 483 + 484 + #[cfg(not(MODULE))] 485 + #[no_mangle] 486 + pub extern "C" fn #ident_init() -> ::kernel::ffi::c_int { 487 + // SAFETY: This function is inaccessible to the outside due to the double 488 + // module wrapping it. It is called exactly once by the C side via its 489 + // placement above in the initcall section. 490 + unsafe { __init() } 491 + } 492 + 493 + #[cfg(not(MODULE))] 494 + #[no_mangle] 495 + pub extern "C" fn #ident_exit() { 496 + // SAFETY: 497 + // - This function is inaccessible to the outside due to the double 498 + // module wrapping it. It is called exactly once by the C side via its 499 + // unique name, 500 + // - furthermore it is only called after `#ident_init` has 501 + // returned `0` (which delegates to `__init`). 502 + unsafe { __exit() } 503 + } 504 + 505 + /// # Safety 506 + /// 507 + /// This function must only be called once. 508 + unsafe fn __init() -> ::kernel::ffi::c_int { 509 + let initer = <super::super::LocalModule as ::kernel::InPlaceModule>::init( 510 + &super::super::THIS_MODULE 589 511 ); 512 + // SAFETY: No data race, since `__MOD` can only be accessed by this module 513 + // and there only `__init` and `__exit` access it. These functions are only 514 + // called once and `__exit` cannot be called before or during `__init`. 515 + match unsafe { initer.__pinned_init(__MOD.as_mut_ptr()) } { 516 + Ok(m) => 0, 517 + Err(e) => e.to_errno(), 518 + } 519 + } 590 520 591 - #[cfg(not(MODULE))] 592 - #[doc(hidden)] 593 - #[no_mangle] 594 - pub extern \"C\" fn __{ident}_init() -> ::kernel::ffi::c_int {{ 595 - // SAFETY: This function is inaccessible to the outside due to the double 596 - // module wrapping it. It is called exactly once by the C side via its 597 - // placement above in the initcall section. 598 - unsafe {{ __init() }} 599 - }} 521 + /// # Safety 522 + /// 523 + /// This function must 524 + /// - only be called once, 525 + /// - be called after `__init` has been called and returned `0`. 526 + unsafe fn __exit() { 527 + // SAFETY: No data race, since `__MOD` can only be accessed by this module 528 + // and there only `__init` and `__exit` access it. These functions are only 529 + // called once and `__init` was already called. 530 + unsafe { 531 + // Invokes `drop()` on `__MOD`, which should be used for cleanup. 532 + __MOD.assume_init_drop(); 533 + } 534 + } 600 535 601 - #[cfg(not(MODULE))] 602 - #[doc(hidden)] 603 - #[no_mangle] 604 - pub extern \"C\" fn __{ident}_exit() {{ 605 - // SAFETY: 606 - // - This function is inaccessible to the outside due to the double 607 - // module wrapping it. It is called exactly once by the C side via its 608 - // unique name, 609 - // - furthermore it is only called after `__{ident}_init` has 610 - // returned `0` (which delegates to `__init`). 611 - unsafe {{ __exit() }} 612 - }} 536 + #modinfo_ts 537 + } 538 + } 613 539 614 - /// # Safety 615 - /// 616 - /// This function must only be called once. 617 - unsafe fn __init() -> ::kernel::ffi::c_int {{ 618 - let initer = 619 - <{type_} as ::kernel::InPlaceModule>::init(&super::super::THIS_MODULE); 620 - // SAFETY: No data race, since `__MOD` can only be accessed by this module 621 - // and there only `__init` and `__exit` access it. These functions are only 622 - // called once and `__exit` cannot be called before or during `__init`. 623 - match unsafe {{ initer.__pinned_init(__MOD.as_mut_ptr()) }} {{ 624 - Ok(m) => 0, 625 - Err(e) => e.to_errno(), 626 - }} 627 - }} 628 - 629 - /// # Safety 630 - /// 631 - /// This function must 632 - /// - only be called once, 633 - /// - be called after `__init` has been called and returned `0`. 634 - unsafe fn __exit() {{ 635 - // SAFETY: No data race, since `__MOD` can only be accessed by this module 636 - // and there only `__init` and `__exit` access it. These functions are only 637 - // called once and `__init` was already called. 638 - unsafe {{ 639 - // Invokes `drop()` on `__MOD`, which should be used for cleanup. 640 - __MOD.assume_init_drop(); 641 - }} 642 - }} 643 - {modinfo} 644 - }} 645 - }} 646 - mod module_parameters {{ 647 - {params} 648 - }} 649 - ", 650 - type_ = info.type_, 651 - name = info.name, 652 - ident = ident, 653 - modinfo = modinfo.buffer, 654 - params = modinfo.param_buffer, 655 - initcall_section = ".initcall6.init" 656 - ) 657 - .parse() 658 - .expect("Error parsing formatted string into token stream.") 540 + mod module_parameters { 541 + #params_ts 542 + } 543 + }) 659 544 }
+1 -1
rust/macros/paste.rs
··· 1 1 // SPDX-License-Identifier: GPL-2.0 2 2 3 - use proc_macro::{Delimiter, Group, Ident, Spacing, Span, TokenTree}; 3 + use proc_macro2::{Delimiter, Group, Ident, Spacing, Span, TokenTree}; 4 4 5 5 fn concat_helper(tokens: &[TokenTree]) -> Vec<(String, Span)> { 6 6 let mut tokens = tokens.iter();
-182
rust/macros/quote.rs
··· 1 - // SPDX-License-Identifier: Apache-2.0 OR MIT 2 - 3 - use proc_macro::{TokenStream, TokenTree}; 4 - 5 - pub(crate) trait ToTokens { 6 - fn to_tokens(&self, tokens: &mut TokenStream); 7 - } 8 - 9 - impl<T: ToTokens> ToTokens for Option<T> { 10 - fn to_tokens(&self, tokens: &mut TokenStream) { 11 - if let Some(v) = self { 12 - v.to_tokens(tokens); 13 - } 14 - } 15 - } 16 - 17 - impl ToTokens for proc_macro::Group { 18 - fn to_tokens(&self, tokens: &mut TokenStream) { 19 - tokens.extend([TokenTree::from(self.clone())]); 20 - } 21 - } 22 - 23 - impl ToTokens for proc_macro::Ident { 24 - fn to_tokens(&self, tokens: &mut TokenStream) { 25 - tokens.extend([TokenTree::from(self.clone())]); 26 - } 27 - } 28 - 29 - impl ToTokens for TokenTree { 30 - fn to_tokens(&self, tokens: &mut TokenStream) { 31 - tokens.extend([self.clone()]); 32 - } 33 - } 34 - 35 - impl ToTokens for TokenStream { 36 - fn to_tokens(&self, tokens: &mut TokenStream) { 37 - tokens.extend(self.clone()); 38 - } 39 - } 40 - 41 - /// Converts tokens into [`proc_macro::TokenStream`] and performs variable interpolations with 42 - /// the given span. 43 - /// 44 - /// This is a similar to the 45 - /// [`quote_spanned!`](https://docs.rs/quote/latest/quote/macro.quote_spanned.html) macro from the 46 - /// `quote` crate but provides only just enough functionality needed by the current `macros` crate. 47 - macro_rules! quote_spanned { 48 - ($span:expr => $($tt:tt)*) => {{ 49 - let mut tokens = ::proc_macro::TokenStream::new(); 50 - { 51 - #[allow(unused_variables)] 52 - let span = $span; 53 - quote_spanned!(@proc tokens span $($tt)*); 54 - } 55 - tokens 56 - }}; 57 - (@proc $v:ident $span:ident) => {}; 58 - (@proc $v:ident $span:ident #$id:ident $($tt:tt)*) => { 59 - $crate::quote::ToTokens::to_tokens(&$id, &mut $v); 60 - quote_spanned!(@proc $v $span $($tt)*); 61 - }; 62 - (@proc $v:ident $span:ident #(#$id:ident)* $($tt:tt)*) => { 63 - for token in $id { 64 - $crate::quote::ToTokens::to_tokens(&token, &mut $v); 65 - } 66 - quote_spanned!(@proc $v $span $($tt)*); 67 - }; 68 - (@proc $v:ident $span:ident ( $($inner:tt)* ) $($tt:tt)*) => { 69 - #[allow(unused_mut)] 70 - let mut tokens = ::proc_macro::TokenStream::new(); 71 - quote_spanned!(@proc tokens $span $($inner)*); 72 - $v.extend([::proc_macro::TokenTree::Group(::proc_macro::Group::new( 73 - ::proc_macro::Delimiter::Parenthesis, 74 - tokens, 75 - ))]); 76 - quote_spanned!(@proc $v $span $($tt)*); 77 - }; 78 - (@proc $v:ident $span:ident [ $($inner:tt)* ] $($tt:tt)*) => { 79 - let mut tokens = ::proc_macro::TokenStream::new(); 80 - quote_spanned!(@proc tokens $span $($inner)*); 81 - $v.extend([::proc_macro::TokenTree::Group(::proc_macro::Group::new( 82 - ::proc_macro::Delimiter::Bracket, 83 - tokens, 84 - ))]); 85 - quote_spanned!(@proc $v $span $($tt)*); 86 - }; 87 - (@proc $v:ident $span:ident { $($inner:tt)* } $($tt:tt)*) => { 88 - let mut tokens = ::proc_macro::TokenStream::new(); 89 - quote_spanned!(@proc tokens $span $($inner)*); 90 - $v.extend([::proc_macro::TokenTree::Group(::proc_macro::Group::new( 91 - ::proc_macro::Delimiter::Brace, 92 - tokens, 93 - ))]); 94 - quote_spanned!(@proc $v $span $($tt)*); 95 - }; 96 - (@proc $v:ident $span:ident :: $($tt:tt)*) => { 97 - $v.extend([::proc_macro::Spacing::Joint, ::proc_macro::Spacing::Alone].map(|spacing| { 98 - ::proc_macro::TokenTree::Punct(::proc_macro::Punct::new(':', spacing)) 99 - })); 100 - quote_spanned!(@proc $v $span $($tt)*); 101 - }; 102 - (@proc $v:ident $span:ident : $($tt:tt)*) => { 103 - $v.extend([::proc_macro::TokenTree::Punct( 104 - ::proc_macro::Punct::new(':', ::proc_macro::Spacing::Alone), 105 - )]); 106 - quote_spanned!(@proc $v $span $($tt)*); 107 - }; 108 - (@proc $v:ident $span:ident , $($tt:tt)*) => { 109 - $v.extend([::proc_macro::TokenTree::Punct( 110 - ::proc_macro::Punct::new(',', ::proc_macro::Spacing::Alone), 111 - )]); 112 - quote_spanned!(@proc $v $span $($tt)*); 113 - }; 114 - (@proc $v:ident $span:ident @ $($tt:tt)*) => { 115 - $v.extend([::proc_macro::TokenTree::Punct( 116 - ::proc_macro::Punct::new('@', ::proc_macro::Spacing::Alone), 117 - )]); 118 - quote_spanned!(@proc $v $span $($tt)*); 119 - }; 120 - (@proc $v:ident $span:ident ! $($tt:tt)*) => { 121 - $v.extend([::proc_macro::TokenTree::Punct( 122 - ::proc_macro::Punct::new('!', ::proc_macro::Spacing::Alone), 123 - )]); 124 - quote_spanned!(@proc $v $span $($tt)*); 125 - }; 126 - (@proc $v:ident $span:ident ; $($tt:tt)*) => { 127 - $v.extend([::proc_macro::TokenTree::Punct( 128 - ::proc_macro::Punct::new(';', ::proc_macro::Spacing::Alone), 129 - )]); 130 - quote_spanned!(@proc $v $span $($tt)*); 131 - }; 132 - (@proc $v:ident $span:ident + $($tt:tt)*) => { 133 - $v.extend([::proc_macro::TokenTree::Punct( 134 - ::proc_macro::Punct::new('+', ::proc_macro::Spacing::Alone), 135 - )]); 136 - quote_spanned!(@proc $v $span $($tt)*); 137 - }; 138 - (@proc $v:ident $span:ident = $($tt:tt)*) => { 139 - $v.extend([::proc_macro::TokenTree::Punct( 140 - ::proc_macro::Punct::new('=', ::proc_macro::Spacing::Alone), 141 - )]); 142 - quote_spanned!(@proc $v $span $($tt)*); 143 - }; 144 - (@proc $v:ident $span:ident # $($tt:tt)*) => { 145 - $v.extend([::proc_macro::TokenTree::Punct( 146 - ::proc_macro::Punct::new('#', ::proc_macro::Spacing::Alone), 147 - )]); 148 - quote_spanned!(@proc $v $span $($tt)*); 149 - }; 150 - (@proc $v:ident $span:ident & $($tt:tt)*) => { 151 - $v.extend([::proc_macro::TokenTree::Punct( 152 - ::proc_macro::Punct::new('&', ::proc_macro::Spacing::Alone), 153 - )]); 154 - quote_spanned!(@proc $v $span $($tt)*); 155 - }; 156 - (@proc $v:ident $span:ident _ $($tt:tt)*) => { 157 - $v.extend([::proc_macro::TokenTree::Ident( 158 - ::proc_macro::Ident::new("_", $span), 159 - )]); 160 - quote_spanned!(@proc $v $span $($tt)*); 161 - }; 162 - (@proc $v:ident $span:ident $id:ident $($tt:tt)*) => { 163 - $v.extend([::proc_macro::TokenTree::Ident( 164 - ::proc_macro::Ident::new(stringify!($id), $span), 165 - )]); 166 - quote_spanned!(@proc $v $span $($tt)*); 167 - }; 168 - } 169 - 170 - /// Converts tokens into [`proc_macro::TokenStream`] and performs variable interpolations with 171 - /// mixed site span ([`Span::mixed_site()`]). 172 - /// 173 - /// This is a similar to the [`quote!`](https://docs.rs/quote/latest/quote/macro.quote.html) macro 174 - /// from the `quote` crate but provides only just enough functionality needed by the current 175 - /// `macros` crate. 176 - /// 177 - /// [`Span::mixed_site()`]: https://doc.rust-lang.org/proc_macro/struct.Span.html#method.mixed_site 178 - macro_rules! quote { 179 - ($($tt:tt)*) => { 180 - quote_spanned!(::proc_macro::Span::mixed_site() => $($tt)*) 181 - } 182 - }
+92 -83
rust/macros/vtable.rs
··· 1 1 // SPDX-License-Identifier: GPL-2.0 2 2 3 - use proc_macro::{Delimiter, Group, TokenStream, TokenTree}; 4 - use std::collections::HashSet; 5 - use std::fmt::Write; 3 + use std::{ 4 + collections::HashSet, 5 + iter::Extend, // 6 + }; 6 7 7 - pub(crate) fn vtable(_attr: TokenStream, ts: TokenStream) -> TokenStream { 8 - let mut tokens: Vec<_> = ts.into_iter().collect(); 8 + use proc_macro2::{ 9 + Ident, 10 + TokenStream, // 11 + }; 12 + use quote::ToTokens; 13 + use syn::{ 14 + parse_quote, 15 + Error, 16 + ImplItem, 17 + Item, 18 + ItemImpl, 19 + ItemTrait, 20 + Result, 21 + TraitItem, // 22 + }; 9 23 10 - // Scan for the `trait` or `impl` keyword. 11 - let is_trait = tokens 12 - .iter() 13 - .find_map(|token| match token { 14 - TokenTree::Ident(ident) => match ident.to_string().as_str() { 15 - "trait" => Some(true), 16 - "impl" => Some(false), 17 - _ => None, 18 - }, 19 - _ => None, 20 - }) 21 - .expect("#[vtable] attribute should only be applied to trait or impl block"); 24 + fn handle_trait(mut item: ItemTrait) -> Result<ItemTrait> { 25 + let mut gen_items = Vec::new(); 22 26 23 - // Retrieve the main body. The main body should be the last token tree. 24 - let body = match tokens.pop() { 25 - Some(TokenTree::Group(group)) if group.delimiter() == Delimiter::Brace => group, 26 - _ => panic!("cannot locate main body of trait or impl block"), 27 - }; 27 + gen_items.push(parse_quote! { 28 + /// A marker to prevent implementors from forgetting to use [`#[vtable]`](vtable) 29 + /// attribute when implementing this trait. 30 + const USE_VTABLE_ATTR: (); 31 + }); 28 32 29 - let mut body_it = body.stream().into_iter(); 30 - let mut functions = Vec::new(); 31 - let mut consts = HashSet::new(); 32 - while let Some(token) = body_it.next() { 33 - match token { 34 - TokenTree::Ident(ident) if ident.to_string() == "fn" => { 35 - let fn_name = match body_it.next() { 36 - Some(TokenTree::Ident(ident)) => ident.to_string(), 37 - // Possibly we've encountered a fn pointer type instead. 38 - _ => continue, 39 - }; 40 - functions.push(fn_name); 41 - } 42 - TokenTree::Ident(ident) if ident.to_string() == "const" => { 43 - let const_name = match body_it.next() { 44 - Some(TokenTree::Ident(ident)) => ident.to_string(), 45 - // Possibly we've encountered an inline const block instead. 46 - _ => continue, 47 - }; 48 - consts.insert(const_name); 49 - } 50 - _ => (), 51 - } 52 - } 33 + for item in &item.items { 34 + if let TraitItem::Fn(fn_item) = item { 35 + let name = &fn_item.sig.ident; 36 + let gen_const_name = Ident::new( 37 + &format!("HAS_{}", name.to_string().to_uppercase()), 38 + name.span(), 39 + ); 53 40 54 - let mut const_items; 55 - if is_trait { 56 - const_items = " 57 - /// A marker to prevent implementors from forgetting to use [`#[vtable]`](vtable) 58 - /// attribute when implementing this trait. 59 - const USE_VTABLE_ATTR: (); 60 - " 61 - .to_owned(); 62 - 63 - for f in functions { 64 - let gen_const_name = format!("HAS_{}", f.to_uppercase()); 65 - // Skip if it's declared already -- this allows user override. 66 - if consts.contains(&gen_const_name) { 67 - continue; 68 - } 69 41 // We don't know on the implementation-site whether a method is required or provided 70 42 // so we have to generate a const for all methods. 71 - write!( 72 - const_items, 73 - "/// Indicates if the `{f}` method is overridden by the implementor. 74 - const {gen_const_name}: bool = false;", 75 - ) 76 - .unwrap(); 77 - consts.insert(gen_const_name); 78 - } 79 - } else { 80 - const_items = "const USE_VTABLE_ATTR: () = ();".to_owned(); 81 - 82 - for f in functions { 83 - let gen_const_name = format!("HAS_{}", f.to_uppercase()); 84 - if consts.contains(&gen_const_name) { 85 - continue; 86 - } 87 - write!(const_items, "const {gen_const_name}: bool = true;").unwrap(); 43 + let cfg_attrs = crate::helpers::gather_cfg_attrs(&fn_item.attrs); 44 + let comment = 45 + format!("Indicates if the `{name}` method is overridden by the implementor."); 46 + gen_items.push(parse_quote! { 47 + #(#cfg_attrs)* 48 + #[doc = #comment] 49 + const #gen_const_name: bool = false; 50 + }); 88 51 } 89 52 } 90 53 91 - let new_body = vec![const_items.parse().unwrap(), body.stream()] 92 - .into_iter() 93 - .collect(); 94 - tokens.push(TokenTree::Group(Group::new(Delimiter::Brace, new_body))); 95 - tokens.into_iter().collect() 54 + item.items.extend(gen_items); 55 + Ok(item) 56 + } 57 + 58 + fn handle_impl(mut item: ItemImpl) -> Result<ItemImpl> { 59 + let mut gen_items = Vec::new(); 60 + let mut defined_consts = HashSet::new(); 61 + 62 + // Iterate over all user-defined constants to gather any possible explicit overrides. 63 + for item in &item.items { 64 + if let ImplItem::Const(const_item) = item { 65 + defined_consts.insert(const_item.ident.clone()); 66 + } 67 + } 68 + 69 + gen_items.push(parse_quote! { 70 + const USE_VTABLE_ATTR: () = (); 71 + }); 72 + 73 + for item in &item.items { 74 + if let ImplItem::Fn(fn_item) = item { 75 + let name = &fn_item.sig.ident; 76 + let gen_const_name = Ident::new( 77 + &format!("HAS_{}", name.to_string().to_uppercase()), 78 + name.span(), 79 + ); 80 + // Skip if it's declared already -- this allows user override. 81 + if defined_consts.contains(&gen_const_name) { 82 + continue; 83 + } 84 + let cfg_attrs = crate::helpers::gather_cfg_attrs(&fn_item.attrs); 85 + gen_items.push(parse_quote! { 86 + #(#cfg_attrs)* 87 + const #gen_const_name: bool = true; 88 + }); 89 + } 90 + } 91 + 92 + item.items.extend(gen_items); 93 + Ok(item) 94 + } 95 + 96 + pub(crate) fn vtable(input: Item) -> Result<TokenStream> { 97 + match input { 98 + Item::Trait(item) => Ok(handle_trait(item)?.into_token_stream()), 99 + Item::Impl(item) => Ok(handle_impl(item)?.into_token_stream()), 100 + _ => Err(Error::new_spanned( 101 + input, 102 + "`#[vtable]` attribute should only be applied to trait or impl block", 103 + ))?, 104 + } 96 105 }
+1 -1
rust/pin-init/README.md
··· 135 135 136 136 impl DriverData { 137 137 fn new() -> impl PinInit<Self, Error> { 138 - try_pin_init!(Self { 138 + pin_init!(Self { 139 139 status <- CMutex::new(0), 140 140 buffer: Box::init(pin_init::init_zeroed())?, 141 141 }? Error)
+9 -10
rust/pin-init/examples/linked_list.rs
··· 6 6 7 7 use core::{ 8 8 cell::Cell, 9 - convert::Infallible, 10 9 marker::PhantomPinned, 11 10 pin::Pin, 12 11 ptr::{self, NonNull}, ··· 30 31 31 32 impl ListHead { 32 33 #[inline] 33 - pub fn new() -> impl PinInit<Self, Infallible> { 34 - try_pin_init!(&this in Self { 34 + pub fn new() -> impl PinInit<Self> { 35 + pin_init!(&this in Self { 35 36 next: unsafe { Link::new_unchecked(this) }, 36 37 prev: unsafe { Link::new_unchecked(this) }, 37 38 pin: PhantomPinned, 38 - }? Infallible) 39 + }) 39 40 } 40 41 41 42 #[inline] 42 43 #[allow(dead_code)] 43 - pub fn insert_next(list: &ListHead) -> impl PinInit<Self, Infallible> + '_ { 44 - try_pin_init!(&this in Self { 44 + pub fn insert_next(list: &ListHead) -> impl PinInit<Self> + '_ { 45 + pin_init!(&this in Self { 45 46 prev: list.next.prev().replace(unsafe { Link::new_unchecked(this)}), 46 47 next: list.next.replace(unsafe { Link::new_unchecked(this)}), 47 48 pin: PhantomPinned, 48 - }? Infallible) 49 + }) 49 50 } 50 51 51 52 #[inline] 52 - pub fn insert_prev(list: &ListHead) -> impl PinInit<Self, Infallible> + '_ { 53 - try_pin_init!(&this in Self { 53 + pub fn insert_prev(list: &ListHead) -> impl PinInit<Self> + '_ { 54 + pin_init!(&this in Self { 54 55 next: list.prev.next().replace(unsafe { Link::new_unchecked(this)}), 55 56 prev: list.prev.replace(unsafe { Link::new_unchecked(this)}), 56 57 pin: PhantomPinned, 57 - }? Infallible) 58 + }) 58 59 } 59 60 60 61 #[inline]
+5 -5
rust/pin-init/examples/pthread_mutex.rs
··· 98 98 // SAFETY: mutex has been initialized 99 99 unsafe { pin_init_from_closure(init) } 100 100 } 101 - try_pin_init!(Self { 102 - data: UnsafeCell::new(data), 103 - raw <- init_raw(), 104 - pin: PhantomPinned, 105 - }? Error) 101 + pin_init!(Self { 102 + data: UnsafeCell::new(data), 103 + raw <- init_raw(), 104 + pin: PhantomPinned, 105 + }? Error) 106 106 } 107 107 108 108 #[allow(dead_code)]
+30
rust/pin-init/internal/src/diagnostics.rs
··· 1 + // SPDX-License-Identifier: Apache-2.0 OR MIT 2 + 3 + use std::fmt::Display; 4 + 5 + use proc_macro2::TokenStream; 6 + use syn::{spanned::Spanned, Error}; 7 + 8 + pub(crate) struct DiagCtxt(TokenStream); 9 + pub(crate) struct ErrorGuaranteed(()); 10 + 11 + impl DiagCtxt { 12 + pub(crate) fn error(&mut self, span: impl Spanned, msg: impl Display) -> ErrorGuaranteed { 13 + let error = Error::new(span.span(), msg); 14 + self.0.extend(error.into_compile_error()); 15 + ErrorGuaranteed(()) 16 + } 17 + 18 + pub(crate) fn with( 19 + fun: impl FnOnce(&mut DiagCtxt) -> Result<TokenStream, ErrorGuaranteed>, 20 + ) -> TokenStream { 21 + let mut dcx = Self(TokenStream::new()); 22 + match fun(&mut dcx) { 23 + Ok(mut stream) => { 24 + stream.extend(dcx.0); 25 + stream 26 + } 27 + Err(ErrorGuaranteed(())) => dcx.0, 28 + } 29 + } 30 + }
-152
rust/pin-init/internal/src/helpers.rs
··· 1 - // SPDX-License-Identifier: Apache-2.0 OR MIT 2 - 3 - #[cfg(not(kernel))] 4 - use proc_macro2 as proc_macro; 5 - 6 - use proc_macro::{TokenStream, TokenTree}; 7 - 8 - /// Parsed generics. 9 - /// 10 - /// See the field documentation for an explanation what each of the fields represents. 11 - /// 12 - /// # Examples 13 - /// 14 - /// ```rust,ignore 15 - /// # let input = todo!(); 16 - /// let (Generics { decl_generics, impl_generics, ty_generics }, rest) = parse_generics(input); 17 - /// quote! { 18 - /// struct Foo<$($decl_generics)*> { 19 - /// // ... 20 - /// } 21 - /// 22 - /// impl<$impl_generics> Foo<$ty_generics> { 23 - /// fn foo() { 24 - /// // ... 25 - /// } 26 - /// } 27 - /// } 28 - /// ``` 29 - pub(crate) struct Generics { 30 - /// The generics with bounds and default values (e.g. `T: Clone, const N: usize = 0`). 31 - /// 32 - /// Use this on type definitions e.g. `struct Foo<$decl_generics> ...` (or `union`/`enum`). 33 - pub(crate) decl_generics: Vec<TokenTree>, 34 - /// The generics with bounds (e.g. `T: Clone, const N: usize`). 35 - /// 36 - /// Use this on `impl` blocks e.g. `impl<$impl_generics> Trait for ...`. 37 - pub(crate) impl_generics: Vec<TokenTree>, 38 - /// The generics without bounds and without default values (e.g. `T, N`). 39 - /// 40 - /// Use this when you use the type that is declared with these generics e.g. 41 - /// `Foo<$ty_generics>`. 42 - pub(crate) ty_generics: Vec<TokenTree>, 43 - } 44 - 45 - /// Parses the given `TokenStream` into `Generics` and the rest. 46 - /// 47 - /// The generics are not present in the rest, but a where clause might remain. 48 - pub(crate) fn parse_generics(input: TokenStream) -> (Generics, Vec<TokenTree>) { 49 - // The generics with bounds and default values. 50 - let mut decl_generics = vec![]; 51 - // `impl_generics`, the declared generics with their bounds. 52 - let mut impl_generics = vec![]; 53 - // Only the names of the generics, without any bounds. 54 - let mut ty_generics = vec![]; 55 - // Tokens not related to the generics e.g. the `where` token and definition. 56 - let mut rest = vec![]; 57 - // The current level of `<`. 58 - let mut nesting = 0; 59 - let mut toks = input.into_iter(); 60 - // If we are at the beginning of a generic parameter. 61 - let mut at_start = true; 62 - let mut skip_until_comma = false; 63 - while let Some(tt) = toks.next() { 64 - if nesting == 1 && matches!(&tt, TokenTree::Punct(p) if p.as_char() == '>') { 65 - // Found the end of the generics. 66 - break; 67 - } else if nesting >= 1 { 68 - decl_generics.push(tt.clone()); 69 - } 70 - match tt.clone() { 71 - TokenTree::Punct(p) if p.as_char() == '<' => { 72 - if nesting >= 1 && !skip_until_comma { 73 - // This is inside of the generics and part of some bound. 74 - impl_generics.push(tt); 75 - } 76 - nesting += 1; 77 - } 78 - TokenTree::Punct(p) if p.as_char() == '>' => { 79 - // This is a parsing error, so we just end it here. 80 - if nesting == 0 { 81 - break; 82 - } else { 83 - nesting -= 1; 84 - if nesting >= 1 && !skip_until_comma { 85 - // We are still inside of the generics and part of some bound. 86 - impl_generics.push(tt); 87 - } 88 - } 89 - } 90 - TokenTree::Punct(p) if skip_until_comma && p.as_char() == ',' => { 91 - if nesting == 1 { 92 - impl_generics.push(tt.clone()); 93 - impl_generics.push(tt); 94 - skip_until_comma = false; 95 - } 96 - } 97 - _ if !skip_until_comma => { 98 - match nesting { 99 - // If we haven't entered the generics yet, we still want to keep these tokens. 100 - 0 => rest.push(tt), 101 - 1 => { 102 - // Here depending on the token, it might be a generic variable name. 103 - match tt.clone() { 104 - TokenTree::Ident(i) if at_start && i.to_string() == "const" => { 105 - let Some(name) = toks.next() else { 106 - // Parsing error. 107 - break; 108 - }; 109 - impl_generics.push(tt); 110 - impl_generics.push(name.clone()); 111 - ty_generics.push(name.clone()); 112 - decl_generics.push(name); 113 - at_start = false; 114 - } 115 - TokenTree::Ident(_) if at_start => { 116 - impl_generics.push(tt.clone()); 117 - ty_generics.push(tt); 118 - at_start = false; 119 - } 120 - TokenTree::Punct(p) if p.as_char() == ',' => { 121 - impl_generics.push(tt.clone()); 122 - ty_generics.push(tt); 123 - at_start = true; 124 - } 125 - // Lifetimes begin with `'`. 126 - TokenTree::Punct(p) if p.as_char() == '\'' && at_start => { 127 - impl_generics.push(tt.clone()); 128 - ty_generics.push(tt); 129 - } 130 - // Generics can have default values, we skip these. 131 - TokenTree::Punct(p) if p.as_char() == '=' => { 132 - skip_until_comma = true; 133 - } 134 - _ => impl_generics.push(tt), 135 - } 136 - } 137 - _ => impl_generics.push(tt), 138 - } 139 - } 140 - _ => {} 141 - } 142 - } 143 - rest.extend(toks); 144 - ( 145 - Generics { 146 - impl_generics, 147 - decl_generics, 148 - ty_generics, 149 - }, 150 - rest, 151 - ) 152 - }
+548
rust/pin-init/internal/src/init.rs
··· 1 + // SPDX-License-Identifier: Apache-2.0 OR MIT 2 + 3 + use proc_macro2::{Span, TokenStream}; 4 + use quote::{format_ident, quote, quote_spanned}; 5 + use syn::{ 6 + braced, 7 + parse::{End, Parse}, 8 + parse_quote, 9 + punctuated::Punctuated, 10 + spanned::Spanned, 11 + token, Attribute, Block, Expr, ExprCall, ExprPath, Ident, Path, Token, Type, 12 + }; 13 + 14 + use crate::diagnostics::{DiagCtxt, ErrorGuaranteed}; 15 + 16 + pub(crate) struct Initializer { 17 + attrs: Vec<InitializerAttribute>, 18 + this: Option<This>, 19 + path: Path, 20 + brace_token: token::Brace, 21 + fields: Punctuated<InitializerField, Token![,]>, 22 + rest: Option<(Token![..], Expr)>, 23 + error: Option<(Token![?], Type)>, 24 + } 25 + 26 + struct This { 27 + _and_token: Token![&], 28 + ident: Ident, 29 + _in_token: Token![in], 30 + } 31 + 32 + struct InitializerField { 33 + attrs: Vec<Attribute>, 34 + kind: InitializerKind, 35 + } 36 + 37 + enum InitializerKind { 38 + Value { 39 + ident: Ident, 40 + value: Option<(Token![:], Expr)>, 41 + }, 42 + Init { 43 + ident: Ident, 44 + _left_arrow_token: Token![<-], 45 + value: Expr, 46 + }, 47 + Code { 48 + _underscore_token: Token![_], 49 + _colon_token: Token![:], 50 + block: Block, 51 + }, 52 + } 53 + 54 + impl InitializerKind { 55 + fn ident(&self) -> Option<&Ident> { 56 + match self { 57 + Self::Value { ident, .. } | Self::Init { ident, .. } => Some(ident), 58 + Self::Code { .. } => None, 59 + } 60 + } 61 + } 62 + 63 + enum InitializerAttribute { 64 + DefaultError(DefaultErrorAttribute), 65 + DisableInitializedFieldAccess, 66 + } 67 + 68 + struct DefaultErrorAttribute { 69 + ty: Box<Type>, 70 + } 71 + 72 + pub(crate) fn expand( 73 + Initializer { 74 + attrs, 75 + this, 76 + path, 77 + brace_token, 78 + fields, 79 + rest, 80 + error, 81 + }: Initializer, 82 + default_error: Option<&'static str>, 83 + pinned: bool, 84 + dcx: &mut DiagCtxt, 85 + ) -> Result<TokenStream, ErrorGuaranteed> { 86 + let error = error.map_or_else( 87 + || { 88 + if let Some(default_error) = attrs.iter().fold(None, |acc, attr| { 89 + if let InitializerAttribute::DefaultError(DefaultErrorAttribute { ty }) = attr { 90 + Some(ty.clone()) 91 + } else { 92 + acc 93 + } 94 + }) { 95 + default_error 96 + } else if let Some(default_error) = default_error { 97 + syn::parse_str(default_error).unwrap() 98 + } else { 99 + dcx.error(brace_token.span.close(), "expected `? <type>` after `}`"); 100 + parse_quote!(::core::convert::Infallible) 101 + } 102 + }, 103 + |(_, err)| Box::new(err), 104 + ); 105 + let slot = format_ident!("slot"); 106 + let (has_data_trait, data_trait, get_data, init_from_closure) = if pinned { 107 + ( 108 + format_ident!("HasPinData"), 109 + format_ident!("PinData"), 110 + format_ident!("__pin_data"), 111 + format_ident!("pin_init_from_closure"), 112 + ) 113 + } else { 114 + ( 115 + format_ident!("HasInitData"), 116 + format_ident!("InitData"), 117 + format_ident!("__init_data"), 118 + format_ident!("init_from_closure"), 119 + ) 120 + }; 121 + let init_kind = get_init_kind(rest, dcx); 122 + let zeroable_check = match init_kind { 123 + InitKind::Normal => quote!(), 124 + InitKind::Zeroing => quote! { 125 + // The user specified `..Zeroable::zeroed()` at the end of the list of fields. 126 + // Therefore we check if the struct implements `Zeroable` and then zero the memory. 127 + // This allows us to also remove the check that all fields are present (since we 128 + // already set the memory to zero and that is a valid bit pattern). 129 + fn assert_zeroable<T: ?::core::marker::Sized>(_: *mut T) 130 + where T: ::pin_init::Zeroable 131 + {} 132 + // Ensure that the struct is indeed `Zeroable`. 133 + assert_zeroable(#slot); 134 + // SAFETY: The type implements `Zeroable` by the check above. 135 + unsafe { ::core::ptr::write_bytes(#slot, 0, 1) }; 136 + }, 137 + }; 138 + let this = match this { 139 + None => quote!(), 140 + Some(This { ident, .. }) => quote! { 141 + // Create the `this` so it can be referenced by the user inside of the 142 + // expressions creating the individual fields. 143 + let #ident = unsafe { ::core::ptr::NonNull::new_unchecked(slot) }; 144 + }, 145 + }; 146 + // `mixed_site` ensures that the data is not accessible to the user-controlled code. 147 + let data = Ident::new("__data", Span::mixed_site()); 148 + let init_fields = init_fields( 149 + &fields, 150 + pinned, 151 + !attrs 152 + .iter() 153 + .any(|attr| matches!(attr, InitializerAttribute::DisableInitializedFieldAccess)), 154 + &data, 155 + &slot, 156 + ); 157 + let field_check = make_field_check(&fields, init_kind, &path); 158 + Ok(quote! {{ 159 + // We do not want to allow arbitrary returns, so we declare this type as the `Ok` return 160 + // type and shadow it later when we insert the arbitrary user code. That way there will be 161 + // no possibility of returning without `unsafe`. 162 + struct __InitOk; 163 + 164 + // Get the data about fields from the supplied type. 165 + // SAFETY: TODO 166 + let #data = unsafe { 167 + use ::pin_init::__internal::#has_data_trait; 168 + // Can't use `<#path as #has_data_trait>::#get_data`, since the user is able to omit 169 + // generics (which need to be present with that syntax). 170 + #path::#get_data() 171 + }; 172 + // Ensure that `#data` really is of type `#data` and help with type inference: 173 + let init = ::pin_init::__internal::#data_trait::make_closure::<_, __InitOk, #error>( 174 + #data, 175 + move |slot| { 176 + { 177 + // Shadow the structure so it cannot be used to return early. 178 + struct __InitOk; 179 + #zeroable_check 180 + #this 181 + #init_fields 182 + #field_check 183 + } 184 + Ok(__InitOk) 185 + } 186 + ); 187 + let init = move |slot| -> ::core::result::Result<(), #error> { 188 + init(slot).map(|__InitOk| ()) 189 + }; 190 + // SAFETY: TODO 191 + let init = unsafe { ::pin_init::#init_from_closure::<_, #error>(init) }; 192 + init 193 + }}) 194 + } 195 + 196 + enum InitKind { 197 + Normal, 198 + Zeroing, 199 + } 200 + 201 + fn get_init_kind(rest: Option<(Token![..], Expr)>, dcx: &mut DiagCtxt) -> InitKind { 202 + let Some((dotdot, expr)) = rest else { 203 + return InitKind::Normal; 204 + }; 205 + match &expr { 206 + Expr::Call(ExprCall { func, args, .. }) if args.is_empty() => match &**func { 207 + Expr::Path(ExprPath { 208 + attrs, 209 + qself: None, 210 + path: 211 + Path { 212 + leading_colon: None, 213 + segments, 214 + }, 215 + }) if attrs.is_empty() 216 + && segments.len() == 2 217 + && segments[0].ident == "Zeroable" 218 + && segments[0].arguments.is_none() 219 + && segments[1].ident == "init_zeroed" 220 + && segments[1].arguments.is_none() => 221 + { 222 + return InitKind::Zeroing; 223 + } 224 + _ => {} 225 + }, 226 + _ => {} 227 + } 228 + dcx.error( 229 + dotdot.span().join(expr.span()).unwrap_or(expr.span()), 230 + "expected nothing or `..Zeroable::init_zeroed()`.", 231 + ); 232 + InitKind::Normal 233 + } 234 + 235 + /// Generate the code that initializes the fields of the struct using the initializers in `field`. 236 + fn init_fields( 237 + fields: &Punctuated<InitializerField, Token![,]>, 238 + pinned: bool, 239 + generate_initialized_accessors: bool, 240 + data: &Ident, 241 + slot: &Ident, 242 + ) -> TokenStream { 243 + let mut guards = vec![]; 244 + let mut guard_attrs = vec![]; 245 + let mut res = TokenStream::new(); 246 + for InitializerField { attrs, kind } in fields { 247 + let cfgs = { 248 + let mut cfgs = attrs.clone(); 249 + cfgs.retain(|attr| attr.path().is_ident("cfg")); 250 + cfgs 251 + }; 252 + let init = match kind { 253 + InitializerKind::Value { ident, value } => { 254 + let mut value_ident = ident.clone(); 255 + let value_prep = value.as_ref().map(|value| &value.1).map(|value| { 256 + // Setting the span of `value_ident` to `value`'s span improves error messages 257 + // when the type of `value` is wrong. 258 + value_ident.set_span(value.span()); 259 + quote!(let #value_ident = #value;) 260 + }); 261 + // Again span for better diagnostics 262 + let write = quote_spanned!(ident.span()=> ::core::ptr::write); 263 + let accessor = if pinned { 264 + let project_ident = format_ident!("__project_{ident}"); 265 + quote! { 266 + // SAFETY: TODO 267 + unsafe { #data.#project_ident(&mut (*#slot).#ident) } 268 + } 269 + } else { 270 + quote! { 271 + // SAFETY: TODO 272 + unsafe { &mut (*#slot).#ident } 273 + } 274 + }; 275 + let accessor = generate_initialized_accessors.then(|| { 276 + quote! { 277 + #(#cfgs)* 278 + #[allow(unused_variables)] 279 + let #ident = #accessor; 280 + } 281 + }); 282 + quote! { 283 + #(#attrs)* 284 + { 285 + #value_prep 286 + // SAFETY: TODO 287 + unsafe { #write(::core::ptr::addr_of_mut!((*#slot).#ident), #value_ident) }; 288 + } 289 + #accessor 290 + } 291 + } 292 + InitializerKind::Init { ident, value, .. } => { 293 + // Again span for better diagnostics 294 + let init = format_ident!("init", span = value.span()); 295 + let (value_init, accessor) = if pinned { 296 + let project_ident = format_ident!("__project_{ident}"); 297 + ( 298 + quote! { 299 + // SAFETY: 300 + // - `slot` is valid, because we are inside of an initializer closure, we 301 + // return when an error/panic occurs. 302 + // - We also use `#data` to require the correct trait (`Init` or `PinInit`) 303 + // for `#ident`. 304 + unsafe { #data.#ident(::core::ptr::addr_of_mut!((*#slot).#ident), #init)? }; 305 + }, 306 + quote! { 307 + // SAFETY: TODO 308 + unsafe { #data.#project_ident(&mut (*#slot).#ident) } 309 + }, 310 + ) 311 + } else { 312 + ( 313 + quote! { 314 + // SAFETY: `slot` is valid, because we are inside of an initializer 315 + // closure, we return when an error/panic occurs. 316 + unsafe { 317 + ::pin_init::Init::__init( 318 + #init, 319 + ::core::ptr::addr_of_mut!((*#slot).#ident), 320 + )? 321 + }; 322 + }, 323 + quote! { 324 + // SAFETY: TODO 325 + unsafe { &mut (*#slot).#ident } 326 + }, 327 + ) 328 + }; 329 + let accessor = generate_initialized_accessors.then(|| { 330 + quote! { 331 + #(#cfgs)* 332 + #[allow(unused_variables)] 333 + let #ident = #accessor; 334 + } 335 + }); 336 + quote! { 337 + #(#attrs)* 338 + { 339 + let #init = #value; 340 + #value_init 341 + } 342 + #accessor 343 + } 344 + } 345 + InitializerKind::Code { block: value, .. } => quote! { 346 + #(#attrs)* 347 + #[allow(unused_braces)] 348 + #value 349 + }, 350 + }; 351 + res.extend(init); 352 + if let Some(ident) = kind.ident() { 353 + // `mixed_site` ensures that the guard is not accessible to the user-controlled code. 354 + let guard = format_ident!("__{ident}_guard", span = Span::mixed_site()); 355 + res.extend(quote! { 356 + #(#cfgs)* 357 + // Create the drop guard: 358 + // 359 + // We rely on macro hygiene to make it impossible for users to access this local 360 + // variable. 361 + // SAFETY: We forget the guard later when initialization has succeeded. 362 + let #guard = unsafe { 363 + ::pin_init::__internal::DropGuard::new( 364 + ::core::ptr::addr_of_mut!((*slot).#ident) 365 + ) 366 + }; 367 + }); 368 + guards.push(guard); 369 + guard_attrs.push(cfgs); 370 + } 371 + } 372 + quote! { 373 + #res 374 + // If execution reaches this point, all fields have been initialized. Therefore we can now 375 + // dismiss the guards by forgetting them. 376 + #( 377 + #(#guard_attrs)* 378 + ::core::mem::forget(#guards); 379 + )* 380 + } 381 + } 382 + 383 + /// Generate the check for ensuring that every field has been initialized. 384 + fn make_field_check( 385 + fields: &Punctuated<InitializerField, Token![,]>, 386 + init_kind: InitKind, 387 + path: &Path, 388 + ) -> TokenStream { 389 + let field_attrs = fields 390 + .iter() 391 + .filter_map(|f| f.kind.ident().map(|_| &f.attrs)); 392 + let field_name = fields.iter().filter_map(|f| f.kind.ident()); 393 + match init_kind { 394 + InitKind::Normal => quote! { 395 + // We use unreachable code to ensure that all fields have been mentioned exactly once, 396 + // this struct initializer will still be type-checked and complain with a very natural 397 + // error message if a field is forgotten/mentioned more than once. 398 + #[allow(unreachable_code, clippy::diverging_sub_expression)] 399 + // SAFETY: this code is never executed. 400 + let _ = || unsafe { 401 + ::core::ptr::write(slot, #path { 402 + #( 403 + #(#field_attrs)* 404 + #field_name: ::core::panic!(), 405 + )* 406 + }) 407 + }; 408 + }, 409 + InitKind::Zeroing => quote! { 410 + // We use unreachable code to ensure that all fields have been mentioned at most once. 411 + // Since the user specified `..Zeroable::zeroed()` at the end, all missing fields will 412 + // be zeroed. This struct initializer will still be type-checked and complain with a 413 + // very natural error message if a field is mentioned more than once, or doesn't exist. 414 + #[allow(unreachable_code, clippy::diverging_sub_expression, unused_assignments)] 415 + // SAFETY: this code is never executed. 416 + let _ = || unsafe { 417 + ::core::ptr::write(slot, #path { 418 + #( 419 + #(#field_attrs)* 420 + #field_name: ::core::panic!(), 421 + )* 422 + ..::core::mem::zeroed() 423 + }) 424 + }; 425 + }, 426 + } 427 + } 428 + 429 + impl Parse for Initializer { 430 + fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> { 431 + let attrs = input.call(Attribute::parse_outer)?; 432 + let this = input.peek(Token![&]).then(|| input.parse()).transpose()?; 433 + let path = input.parse()?; 434 + let content; 435 + let brace_token = braced!(content in input); 436 + let mut fields = Punctuated::new(); 437 + loop { 438 + let lh = content.lookahead1(); 439 + if lh.peek(End) || lh.peek(Token![..]) { 440 + break; 441 + } else if lh.peek(Ident) || lh.peek(Token![_]) || lh.peek(Token![#]) { 442 + fields.push_value(content.parse()?); 443 + let lh = content.lookahead1(); 444 + if lh.peek(End) { 445 + break; 446 + } else if lh.peek(Token![,]) { 447 + fields.push_punct(content.parse()?); 448 + } else { 449 + return Err(lh.error()); 450 + } 451 + } else { 452 + return Err(lh.error()); 453 + } 454 + } 455 + let rest = content 456 + .peek(Token![..]) 457 + .then(|| Ok::<_, syn::Error>((content.parse()?, content.parse()?))) 458 + .transpose()?; 459 + let error = input 460 + .peek(Token![?]) 461 + .then(|| Ok::<_, syn::Error>((input.parse()?, input.parse()?))) 462 + .transpose()?; 463 + let attrs = attrs 464 + .into_iter() 465 + .map(|a| { 466 + if a.path().is_ident("default_error") { 467 + a.parse_args::<DefaultErrorAttribute>() 468 + .map(InitializerAttribute::DefaultError) 469 + } else if a.path().is_ident("disable_initialized_field_access") { 470 + a.meta 471 + .require_path_only() 472 + .map(|_| InitializerAttribute::DisableInitializedFieldAccess) 473 + } else { 474 + Err(syn::Error::new_spanned(a, "unknown initializer attribute")) 475 + } 476 + }) 477 + .collect::<Result<Vec<_>, _>>()?; 478 + Ok(Self { 479 + attrs, 480 + this, 481 + path, 482 + brace_token, 483 + fields, 484 + rest, 485 + error, 486 + }) 487 + } 488 + } 489 + 490 + impl Parse for DefaultErrorAttribute { 491 + fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> { 492 + Ok(Self { ty: input.parse()? }) 493 + } 494 + } 495 + 496 + impl Parse for This { 497 + fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> { 498 + Ok(Self { 499 + _and_token: input.parse()?, 500 + ident: input.parse()?, 501 + _in_token: input.parse()?, 502 + }) 503 + } 504 + } 505 + 506 + impl Parse for InitializerField { 507 + fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> { 508 + let attrs = input.call(Attribute::parse_outer)?; 509 + Ok(Self { 510 + attrs, 511 + kind: input.parse()?, 512 + }) 513 + } 514 + } 515 + 516 + impl Parse for InitializerKind { 517 + fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> { 518 + let lh = input.lookahead1(); 519 + if lh.peek(Token![_]) { 520 + Ok(Self::Code { 521 + _underscore_token: input.parse()?, 522 + _colon_token: input.parse()?, 523 + block: input.parse()?, 524 + }) 525 + } else if lh.peek(Ident) { 526 + let ident = input.parse()?; 527 + let lh = input.lookahead1(); 528 + if lh.peek(Token![<-]) { 529 + Ok(Self::Init { 530 + ident, 531 + _left_arrow_token: input.parse()?, 532 + value: input.parse()?, 533 + }) 534 + } else if lh.peek(Token![:]) { 535 + Ok(Self::Value { 536 + ident, 537 + value: Some((input.parse()?, input.parse()?)), 538 + }) 539 + } else if lh.peek(Token![,]) || lh.peek(End) { 540 + Ok(Self::Value { ident, value: None }) 541 + } else { 542 + Err(lh.error()) 543 + } 544 + } else { 545 + Err(lh.error()) 546 + } 547 + } 548 + }
+27 -21
rust/pin-init/internal/src/lib.rs
··· 7 7 //! `pin-init` proc macros. 8 8 9 9 #![cfg_attr(not(RUSTC_LINT_REASONS_IS_STABLE), feature(lint_reasons))] 10 - // Allow `.into()` to convert 11 - // - `proc_macro2::TokenStream` into `proc_macro::TokenStream` in the user-space version. 12 - // - `proc_macro::TokenStream` into `proc_macro::TokenStream` in the kernel version. 13 - // Clippy warns on this conversion, but it's required by the user-space version. 14 - // 15 - // Remove once we have `proc_macro2` in the kernel. 16 - #![allow(clippy::useless_conversion)] 17 10 // Documentation is done in the pin-init crate instead. 18 11 #![allow(missing_docs)] 19 12 20 13 use proc_macro::TokenStream; 14 + use syn::parse_macro_input; 21 15 22 - #[cfg(kernel)] 23 - #[path = "../../../macros/quote.rs"] 24 - #[macro_use] 25 - #[cfg_attr(not(kernel), rustfmt::skip)] 26 - mod quote; 27 - #[cfg(not(kernel))] 28 - #[macro_use] 29 - extern crate quote; 16 + use crate::diagnostics::DiagCtxt; 30 17 31 - mod helpers; 18 + mod diagnostics; 19 + mod init; 32 20 mod pin_data; 33 21 mod pinned_drop; 34 22 mod zeroable; 35 23 36 24 #[proc_macro_attribute] 37 - pub fn pin_data(inner: TokenStream, item: TokenStream) -> TokenStream { 38 - pin_data::pin_data(inner.into(), item.into()).into() 25 + pub fn pin_data(args: TokenStream, input: TokenStream) -> TokenStream { 26 + let args = parse_macro_input!(args); 27 + let input = parse_macro_input!(input); 28 + DiagCtxt::with(|dcx| pin_data::pin_data(args, input, dcx)).into() 39 29 } 40 30 41 31 #[proc_macro_attribute] 42 32 pub fn pinned_drop(args: TokenStream, input: TokenStream) -> TokenStream { 43 - pinned_drop::pinned_drop(args.into(), input.into()).into() 33 + let args = parse_macro_input!(args); 34 + let input = parse_macro_input!(input); 35 + DiagCtxt::with(|dcx| pinned_drop::pinned_drop(args, input, dcx)).into() 44 36 } 45 37 46 38 #[proc_macro_derive(Zeroable)] 47 39 pub fn derive_zeroable(input: TokenStream) -> TokenStream { 48 - zeroable::derive(input.into()).into() 40 + let input = parse_macro_input!(input); 41 + DiagCtxt::with(|dcx| zeroable::derive(input, dcx)).into() 49 42 } 50 43 51 44 #[proc_macro_derive(MaybeZeroable)] 52 45 pub fn maybe_derive_zeroable(input: TokenStream) -> TokenStream { 53 - zeroable::maybe_derive(input.into()).into() 46 + let input = parse_macro_input!(input); 47 + DiagCtxt::with(|dcx| zeroable::maybe_derive(input, dcx)).into() 48 + } 49 + #[proc_macro] 50 + pub fn init(input: TokenStream) -> TokenStream { 51 + let input = parse_macro_input!(input); 52 + DiagCtxt::with(|dcx| init::expand(input, Some("::core::convert::Infallible"), false, dcx)) 53 + .into() 54 + } 55 + 56 + #[proc_macro] 57 + pub fn pin_init(input: TokenStream) -> TokenStream { 58 + let input = parse_macro_input!(input); 59 + DiagCtxt::with(|dcx| init::expand(input, Some("::core::convert::Infallible"), true, dcx)).into() 54 60 }
+504 -123
rust/pin-init/internal/src/pin_data.rs
··· 1 1 // SPDX-License-Identifier: Apache-2.0 OR MIT 2 2 3 - #[cfg(not(kernel))] 4 - use proc_macro2 as proc_macro; 3 + use proc_macro2::TokenStream; 4 + use quote::{format_ident, quote}; 5 + use syn::{ 6 + parse::{End, Nothing, Parse}, 7 + parse_quote, parse_quote_spanned, 8 + spanned::Spanned, 9 + visit_mut::VisitMut, 10 + Field, Generics, Ident, Item, PathSegment, Type, TypePath, Visibility, WhereClause, 11 + }; 5 12 6 - use crate::helpers::{parse_generics, Generics}; 7 - use proc_macro::{Group, Punct, Spacing, TokenStream, TokenTree}; 13 + use crate::diagnostics::{DiagCtxt, ErrorGuaranteed}; 8 14 9 - pub(crate) fn pin_data(args: TokenStream, input: TokenStream) -> TokenStream { 10 - // This proc-macro only does some pre-parsing and then delegates the actual parsing to 11 - // `pin_init::__pin_data!`. 12 - 13 - let ( 14 - Generics { 15 - impl_generics, 16 - decl_generics, 17 - ty_generics, 18 - }, 19 - rest, 20 - ) = parse_generics(input); 21 - // The struct definition might contain the `Self` type. Since `__pin_data!` will define a new 22 - // type with the same generics and bounds, this poses a problem, since `Self` will refer to the 23 - // new type as opposed to this struct definition. Therefore we have to replace `Self` with the 24 - // concrete name. 25 - 26 - // Errors that occur when replacing `Self` with `struct_name`. 27 - let mut errs = TokenStream::new(); 28 - // The name of the struct with ty_generics. 29 - let struct_name = rest 30 - .iter() 31 - .skip_while(|tt| !matches!(tt, TokenTree::Ident(i) if i.to_string() == "struct")) 32 - .nth(1) 33 - .and_then(|tt| match tt { 34 - TokenTree::Ident(_) => { 35 - let tt = tt.clone(); 36 - let mut res = vec![tt]; 37 - if !ty_generics.is_empty() { 38 - // We add this, so it is maximally compatible with e.g. `Self::CONST` which 39 - // will be replaced by `StructName::<$generics>::CONST`. 40 - res.push(TokenTree::Punct(Punct::new(':', Spacing::Joint))); 41 - res.push(TokenTree::Punct(Punct::new(':', Spacing::Alone))); 42 - res.push(TokenTree::Punct(Punct::new('<', Spacing::Alone))); 43 - res.extend(ty_generics.iter().cloned()); 44 - res.push(TokenTree::Punct(Punct::new('>', Spacing::Alone))); 45 - } 46 - Some(res) 47 - } 48 - _ => None, 49 - }) 50 - .unwrap_or_else(|| { 51 - // If we did not find the name of the struct then we will use `Self` as the replacement 52 - // and add a compile error to ensure it does not compile. 53 - errs.extend( 54 - "::core::compile_error!(\"Could not locate type name.\");" 55 - .parse::<TokenStream>() 56 - .unwrap(), 57 - ); 58 - "Self".parse::<TokenStream>().unwrap().into_iter().collect() 59 - }); 60 - let impl_generics = impl_generics 61 - .into_iter() 62 - .flat_map(|tt| replace_self_and_deny_type_defs(&struct_name, tt, &mut errs)) 63 - .collect::<Vec<_>>(); 64 - let mut rest = rest 65 - .into_iter() 66 - .flat_map(|tt| { 67 - // We ignore top level `struct` tokens, since they would emit a compile error. 68 - if matches!(&tt, TokenTree::Ident(i) if i.to_string() == "struct") { 69 - vec![tt] 70 - } else { 71 - replace_self_and_deny_type_defs(&struct_name, tt, &mut errs) 72 - } 73 - }) 74 - .collect::<Vec<_>>(); 75 - // This should be the body of the struct `{...}`. 76 - let last = rest.pop(); 77 - let mut quoted = quote!(::pin_init::__pin_data! { 78 - parse_input: 79 - @args(#args), 80 - @sig(#(#rest)*), 81 - @impl_generics(#(#impl_generics)*), 82 - @ty_generics(#(#ty_generics)*), 83 - @decl_generics(#(#decl_generics)*), 84 - @body(#last), 85 - }); 86 - quoted.extend(errs); 87 - quoted 15 + pub(crate) mod kw { 16 + syn::custom_keyword!(PinnedDrop); 88 17 } 89 18 90 - /// Replaces `Self` with `struct_name` and errors on `enum`, `trait`, `struct` `union` and `impl` 91 - /// keywords. 92 - /// 93 - /// The error is appended to `errs` to allow normal parsing to continue. 94 - fn replace_self_and_deny_type_defs( 95 - struct_name: &Vec<TokenTree>, 96 - tt: TokenTree, 97 - errs: &mut TokenStream, 98 - ) -> Vec<TokenTree> { 99 - match tt { 100 - TokenTree::Ident(ref i) 101 - if i.to_string() == "enum" 102 - || i.to_string() == "trait" 103 - || i.to_string() == "struct" 104 - || i.to_string() == "union" 105 - || i.to_string() == "impl" => 106 - { 107 - errs.extend( 108 - format!( 109 - "::core::compile_error!(\"Cannot use `{i}` inside of struct definition with \ 110 - `#[pin_data]`.\");" 111 - ) 112 - .parse::<TokenStream>() 113 - .unwrap() 114 - .into_iter() 115 - .map(|mut tok| { 116 - tok.set_span(tt.span()); 117 - tok 118 - }), 119 - ); 120 - vec![tt] 19 + pub(crate) enum Args { 20 + Nothing(Nothing), 21 + #[allow(dead_code)] 22 + PinnedDrop(kw::PinnedDrop), 23 + } 24 + 25 + impl Parse for Args { 26 + fn parse(input: syn::parse::ParseStream<'_>) -> syn::Result<Self> { 27 + let lh = input.lookahead1(); 28 + if lh.peek(End) { 29 + input.parse().map(Self::Nothing) 30 + } else if lh.peek(kw::PinnedDrop) { 31 + input.parse().map(Self::PinnedDrop) 32 + } else { 33 + Err(lh.error()) 121 34 } 122 - TokenTree::Ident(i) if i.to_string() == "Self" => struct_name.clone(), 123 - TokenTree::Literal(_) | TokenTree::Punct(_) | TokenTree::Ident(_) => vec![tt], 124 - TokenTree::Group(g) => vec![TokenTree::Group(Group::new( 125 - g.delimiter(), 126 - g.stream() 127 - .into_iter() 128 - .flat_map(|tt| replace_self_and_deny_type_defs(struct_name, tt, errs)) 129 - .collect(), 130 - ))], 131 35 } 36 + } 37 + 38 + pub(crate) fn pin_data( 39 + args: Args, 40 + input: Item, 41 + dcx: &mut DiagCtxt, 42 + ) -> Result<TokenStream, ErrorGuaranteed> { 43 + let mut struct_ = match input { 44 + Item::Struct(struct_) => struct_, 45 + Item::Enum(enum_) => { 46 + return Err(dcx.error( 47 + enum_.enum_token, 48 + "`#[pin_data]` only supports structs for now", 49 + )); 50 + } 51 + Item::Union(union) => { 52 + return Err(dcx.error( 53 + union.union_token, 54 + "`#[pin_data]` only supports structs for now", 55 + )); 56 + } 57 + rest => { 58 + return Err(dcx.error( 59 + rest, 60 + "`#[pin_data]` can only be applied to struct, enum and union definitions", 61 + )); 62 + } 63 + }; 64 + 65 + // The generics might contain the `Self` type. Since this macro will define a new type with the 66 + // same generics and bounds, this poses a problem: `Self` will refer to the new type as opposed 67 + // to this struct definition. Therefore we have to replace `Self` with the concrete name. 68 + let mut replacer = { 69 + let name = &struct_.ident; 70 + let (_, ty_generics, _) = struct_.generics.split_for_impl(); 71 + SelfReplacer(parse_quote!(#name #ty_generics)) 72 + }; 73 + replacer.visit_generics_mut(&mut struct_.generics); 74 + replacer.visit_fields_mut(&mut struct_.fields); 75 + 76 + let fields: Vec<(bool, &Field)> = struct_ 77 + .fields 78 + .iter_mut() 79 + .map(|field| { 80 + let len = field.attrs.len(); 81 + field.attrs.retain(|a| !a.path().is_ident("pin")); 82 + (len != field.attrs.len(), &*field) 83 + }) 84 + .collect(); 85 + 86 + for (pinned, field) in &fields { 87 + if !pinned && is_phantom_pinned(&field.ty) { 88 + dcx.error( 89 + field, 90 + format!( 91 + "The field `{}` of type `PhantomPinned` only has an effect \ 92 + if it has the `#[pin]` attribute", 93 + field.ident.as_ref().unwrap(), 94 + ), 95 + ); 96 + } 97 + } 98 + 99 + let unpin_impl = generate_unpin_impl(&struct_.ident, &struct_.generics, &fields); 100 + let drop_impl = generate_drop_impl(&struct_.ident, &struct_.generics, args); 101 + let projections = 102 + generate_projections(&struct_.vis, &struct_.ident, &struct_.generics, &fields); 103 + let the_pin_data = 104 + generate_the_pin_data(&struct_.vis, &struct_.ident, &struct_.generics, &fields); 105 + 106 + Ok(quote! { 107 + #struct_ 108 + #projections 109 + // We put the rest into this const item, because it then will not be accessible to anything 110 + // outside. 111 + const _: () = { 112 + #the_pin_data 113 + #unpin_impl 114 + #drop_impl 115 + }; 116 + }) 117 + } 118 + 119 + fn is_phantom_pinned(ty: &Type) -> bool { 120 + match ty { 121 + Type::Path(TypePath { qself: None, path }) => { 122 + // Cannot possibly refer to `PhantomPinned` (except alias, but that's on the user). 123 + if path.segments.len() > 3 { 124 + return false; 125 + } 126 + // If there is a `::`, then the path needs to be `::core::marker::PhantomPinned` or 127 + // `::std::marker::PhantomPinned`. 128 + if path.leading_colon.is_some() && path.segments.len() != 3 { 129 + return false; 130 + } 131 + let expected: Vec<&[&str]> = vec![&["PhantomPinned"], &["marker"], &["core", "std"]]; 132 + for (actual, expected) in path.segments.iter().rev().zip(expected) { 133 + if !actual.arguments.is_empty() || expected.iter().all(|e| actual.ident != e) { 134 + return false; 135 + } 136 + } 137 + true 138 + } 139 + _ => false, 140 + } 141 + } 142 + 143 + fn generate_unpin_impl( 144 + ident: &Ident, 145 + generics: &Generics, 146 + fields: &[(bool, &Field)], 147 + ) -> TokenStream { 148 + let (_, ty_generics, _) = generics.split_for_impl(); 149 + let mut generics_with_pin_lt = generics.clone(); 150 + generics_with_pin_lt.params.insert(0, parse_quote!('__pin)); 151 + generics_with_pin_lt.make_where_clause(); 152 + let ( 153 + impl_generics_with_pin_lt, 154 + ty_generics_with_pin_lt, 155 + Some(WhereClause { 156 + where_token, 157 + predicates, 158 + }), 159 + ) = generics_with_pin_lt.split_for_impl() 160 + else { 161 + unreachable!() 162 + }; 163 + let pinned_fields = fields.iter().filter_map(|(b, f)| b.then_some(f)); 164 + quote! { 165 + // This struct will be used for the unpin analysis. It is needed, because only structurally 166 + // pinned fields are relevant whether the struct should implement `Unpin`. 167 + #[allow(dead_code)] // The fields below are never used. 168 + struct __Unpin #generics_with_pin_lt 169 + #where_token 170 + #predicates 171 + { 172 + __phantom_pin: ::core::marker::PhantomData<fn(&'__pin ()) -> &'__pin ()>, 173 + __phantom: ::core::marker::PhantomData< 174 + fn(#ident #ty_generics) -> #ident #ty_generics 175 + >, 176 + #(#pinned_fields),* 177 + } 178 + 179 + #[doc(hidden)] 180 + impl #impl_generics_with_pin_lt ::core::marker::Unpin for #ident #ty_generics 181 + #where_token 182 + __Unpin #ty_generics_with_pin_lt: ::core::marker::Unpin, 183 + #predicates 184 + {} 185 + } 186 + } 187 + 188 + fn generate_drop_impl(ident: &Ident, generics: &Generics, args: Args) -> TokenStream { 189 + let (impl_generics, ty_generics, whr) = generics.split_for_impl(); 190 + let has_pinned_drop = matches!(args, Args::PinnedDrop(_)); 191 + // We need to disallow normal `Drop` implementation, the exact behavior depends on whether 192 + // `PinnedDrop` was specified in `args`. 193 + if has_pinned_drop { 194 + // When `PinnedDrop` was specified we just implement `Drop` and delegate. 195 + quote! { 196 + impl #impl_generics ::core::ops::Drop for #ident #ty_generics 197 + #whr 198 + { 199 + fn drop(&mut self) { 200 + // SAFETY: Since this is a destructor, `self` will not move after this function 201 + // terminates, since it is inaccessible. 202 + let pinned = unsafe { ::core::pin::Pin::new_unchecked(self) }; 203 + // SAFETY: Since this is a drop function, we can create this token to call the 204 + // pinned destructor of this type. 205 + let token = unsafe { ::pin_init::__internal::OnlyCallFromDrop::new() }; 206 + ::pin_init::PinnedDrop::drop(pinned, token); 207 + } 208 + } 209 + } 210 + } else { 211 + // When no `PinnedDrop` was specified, then we have to prevent implementing drop. 212 + quote! { 213 + // We prevent this by creating a trait that will be implemented for all types implementing 214 + // `Drop`. Additionally we will implement this trait for the struct leading to a conflict, 215 + // if it also implements `Drop` 216 + trait MustNotImplDrop {} 217 + #[expect(drop_bounds)] 218 + impl<T: ::core::ops::Drop + ?::core::marker::Sized> MustNotImplDrop for T {} 219 + impl #impl_generics MustNotImplDrop for #ident #ty_generics 220 + #whr 221 + {} 222 + // We also take care to prevent users from writing a useless `PinnedDrop` implementation. 223 + // They might implement `PinnedDrop` correctly for the struct, but forget to give 224 + // `PinnedDrop` as the parameter to `#[pin_data]`. 225 + #[expect(non_camel_case_types)] 226 + trait UselessPinnedDropImpl_you_need_to_specify_PinnedDrop {} 227 + impl<T: ::pin_init::PinnedDrop + ?::core::marker::Sized> 228 + UselessPinnedDropImpl_you_need_to_specify_PinnedDrop for T {} 229 + impl #impl_generics 230 + UselessPinnedDropImpl_you_need_to_specify_PinnedDrop for #ident #ty_generics 231 + #whr 232 + {} 233 + } 234 + } 235 + } 236 + 237 + fn generate_projections( 238 + vis: &Visibility, 239 + ident: &Ident, 240 + generics: &Generics, 241 + fields: &[(bool, &Field)], 242 + ) -> TokenStream { 243 + let (impl_generics, ty_generics, _) = generics.split_for_impl(); 244 + let mut generics_with_pin_lt = generics.clone(); 245 + generics_with_pin_lt.params.insert(0, parse_quote!('__pin)); 246 + let (_, ty_generics_with_pin_lt, whr) = generics_with_pin_lt.split_for_impl(); 247 + let projection = format_ident!("{ident}Projection"); 248 + let this = format_ident!("this"); 249 + 250 + let (fields_decl, fields_proj) = collect_tuple(fields.iter().map( 251 + |( 252 + pinned, 253 + Field { 254 + vis, 255 + ident, 256 + ty, 257 + attrs, 258 + .. 259 + }, 260 + )| { 261 + let mut attrs = attrs.clone(); 262 + attrs.retain(|a| !a.path().is_ident("pin")); 263 + let mut no_doc_attrs = attrs.clone(); 264 + no_doc_attrs.retain(|a| !a.path().is_ident("doc")); 265 + let ident = ident 266 + .as_ref() 267 + .expect("only structs with named fields are supported"); 268 + if *pinned { 269 + ( 270 + quote!( 271 + #(#attrs)* 272 + #vis #ident: ::core::pin::Pin<&'__pin mut #ty>, 273 + ), 274 + quote!( 275 + #(#no_doc_attrs)* 276 + // SAFETY: this field is structurally pinned. 277 + #ident: unsafe { ::core::pin::Pin::new_unchecked(&mut #this.#ident) }, 278 + ), 279 + ) 280 + } else { 281 + ( 282 + quote!( 283 + #(#attrs)* 284 + #vis #ident: &'__pin mut #ty, 285 + ), 286 + quote!( 287 + #(#no_doc_attrs)* 288 + #ident: &mut #this.#ident, 289 + ), 290 + ) 291 + } 292 + }, 293 + )); 294 + let structurally_pinned_fields_docs = fields 295 + .iter() 296 + .filter_map(|(pinned, field)| pinned.then_some(field)) 297 + .map(|Field { ident, .. }| format!(" - `{}`", ident.as_ref().unwrap())); 298 + let not_structurally_pinned_fields_docs = fields 299 + .iter() 300 + .filter_map(|(pinned, field)| (!pinned).then_some(field)) 301 + .map(|Field { ident, .. }| format!(" - `{}`", ident.as_ref().unwrap())); 302 + let docs = format!(" Pin-projections of [`{ident}`]"); 303 + quote! { 304 + #[doc = #docs] 305 + #[allow(dead_code)] 306 + #[doc(hidden)] 307 + #vis struct #projection #generics_with_pin_lt { 308 + #(#fields_decl)* 309 + ___pin_phantom_data: ::core::marker::PhantomData<&'__pin mut ()>, 310 + } 311 + 312 + impl #impl_generics #ident #ty_generics 313 + #whr 314 + { 315 + /// Pin-projects all fields of `Self`. 316 + /// 317 + /// These fields are structurally pinned: 318 + #(#[doc = #structurally_pinned_fields_docs])* 319 + /// 320 + /// These fields are **not** structurally pinned: 321 + #(#[doc = #not_structurally_pinned_fields_docs])* 322 + #[inline] 323 + #vis fn project<'__pin>( 324 + self: ::core::pin::Pin<&'__pin mut Self>, 325 + ) -> #projection #ty_generics_with_pin_lt { 326 + // SAFETY: we only give access to `&mut` for fields not structurally pinned. 327 + let #this = unsafe { ::core::pin::Pin::get_unchecked_mut(self) }; 328 + #projection { 329 + #(#fields_proj)* 330 + ___pin_phantom_data: ::core::marker::PhantomData, 331 + } 332 + } 333 + } 334 + } 335 + } 336 + 337 + fn generate_the_pin_data( 338 + vis: &Visibility, 339 + ident: &Ident, 340 + generics: &Generics, 341 + fields: &[(bool, &Field)], 342 + ) -> TokenStream { 343 + let (impl_generics, ty_generics, whr) = generics.split_for_impl(); 344 + 345 + // For every field, we create an initializing projection function according to its projection 346 + // type. If a field is structurally pinned, then it must be initialized via `PinInit`, if it is 347 + // not structurally pinned, then it can be initialized via `Init`. 348 + // 349 + // The functions are `unsafe` to prevent accidentally calling them. 350 + fn handle_field( 351 + Field { 352 + vis, 353 + ident, 354 + ty, 355 + attrs, 356 + .. 357 + }: &Field, 358 + struct_ident: &Ident, 359 + pinned: bool, 360 + ) -> TokenStream { 361 + let mut attrs = attrs.clone(); 362 + attrs.retain(|a| !a.path().is_ident("pin")); 363 + let ident = ident 364 + .as_ref() 365 + .expect("only structs with named fields are supported"); 366 + let project_ident = format_ident!("__project_{ident}"); 367 + let (init_ty, init_fn, project_ty, project_body, pin_safety) = if pinned { 368 + ( 369 + quote!(PinInit), 370 + quote!(__pinned_init), 371 + quote!(::core::pin::Pin<&'__slot mut #ty>), 372 + // SAFETY: this field is structurally pinned. 373 + quote!(unsafe { ::core::pin::Pin::new_unchecked(slot) }), 374 + quote!( 375 + /// - `slot` will not move until it is dropped, i.e. it will be pinned. 376 + ), 377 + ) 378 + } else { 379 + ( 380 + quote!(Init), 381 + quote!(__init), 382 + quote!(&'__slot mut #ty), 383 + quote!(slot), 384 + quote!(), 385 + ) 386 + }; 387 + let slot_safety = format!( 388 + " `slot` points at the field `{ident}` inside of `{struct_ident}`, which is pinned.", 389 + ); 390 + quote! { 391 + /// # Safety 392 + /// 393 + /// - `slot` is a valid pointer to uninitialized memory. 394 + /// - the caller does not touch `slot` when `Err` is returned, they are only permitted 395 + /// to deallocate. 396 + #pin_safety 397 + #(#attrs)* 398 + #vis unsafe fn #ident<E>( 399 + self, 400 + slot: *mut #ty, 401 + init: impl ::pin_init::#init_ty<#ty, E>, 402 + ) -> ::core::result::Result<(), E> { 403 + // SAFETY: this function has the same safety requirements as the __init function 404 + // called below. 405 + unsafe { ::pin_init::#init_ty::#init_fn(init, slot) } 406 + } 407 + 408 + /// # Safety 409 + /// 410 + #[doc = #slot_safety] 411 + #(#attrs)* 412 + #vis unsafe fn #project_ident<'__slot>( 413 + self, 414 + slot: &'__slot mut #ty, 415 + ) -> #project_ty { 416 + #project_body 417 + } 418 + } 419 + } 420 + 421 + let field_accessors = fields 422 + .iter() 423 + .map(|(pinned, field)| handle_field(field, ident, *pinned)) 424 + .collect::<TokenStream>(); 425 + quote! { 426 + // We declare this struct which will host all of the projection function for our type. It 427 + // will be invariant over all generic parameters which are inherited from the struct. 428 + #[doc(hidden)] 429 + #vis struct __ThePinData #generics 430 + #whr 431 + { 432 + __phantom: ::core::marker::PhantomData< 433 + fn(#ident #ty_generics) -> #ident #ty_generics 434 + >, 435 + } 436 + 437 + impl #impl_generics ::core::clone::Clone for __ThePinData #ty_generics 438 + #whr 439 + { 440 + fn clone(&self) -> Self { *self } 441 + } 442 + 443 + impl #impl_generics ::core::marker::Copy for __ThePinData #ty_generics 444 + #whr 445 + {} 446 + 447 + #[allow(dead_code)] // Some functions might never be used and private. 448 + #[expect(clippy::missing_safety_doc)] 449 + impl #impl_generics __ThePinData #ty_generics 450 + #whr 451 + { 452 + #field_accessors 453 + } 454 + 455 + // SAFETY: We have added the correct projection functions above to `__ThePinData` and 456 + // we also use the least restrictive generics possible. 457 + unsafe impl #impl_generics ::pin_init::__internal::HasPinData for #ident #ty_generics 458 + #whr 459 + { 460 + type PinData = __ThePinData #ty_generics; 461 + 462 + unsafe fn __pin_data() -> Self::PinData { 463 + __ThePinData { __phantom: ::core::marker::PhantomData } 464 + } 465 + } 466 + 467 + // SAFETY: TODO 468 + unsafe impl #impl_generics ::pin_init::__internal::PinData for __ThePinData #ty_generics 469 + #whr 470 + { 471 + type Datee = #ident #ty_generics; 472 + } 473 + } 474 + } 475 + 476 + struct SelfReplacer(PathSegment); 477 + 478 + impl VisitMut for SelfReplacer { 479 + fn visit_path_mut(&mut self, i: &mut syn::Path) { 480 + if i.is_ident("Self") { 481 + let span = i.span(); 482 + let seg = &self.0; 483 + *i = parse_quote_spanned!(span=> #seg); 484 + } else { 485 + syn::visit_mut::visit_path_mut(self, i); 486 + } 487 + } 488 + 489 + fn visit_path_segment_mut(&mut self, seg: &mut PathSegment) { 490 + if seg.ident == "Self" { 491 + let span = seg.span(); 492 + let this = &self.0; 493 + *seg = parse_quote_spanned!(span=> #this); 494 + } else { 495 + syn::visit_mut::visit_path_segment_mut(self, seg); 496 + } 497 + } 498 + 499 + fn visit_item_mut(&mut self, _: &mut Item) { 500 + // Do not descend into items, since items reset/change what `Self` refers to. 501 + } 502 + } 503 + 504 + // replace with `.collect()` once MSRV is above 1.79 505 + fn collect_tuple<A, B>(iter: impl Iterator<Item = (A, B)>) -> (Vec<A>, Vec<B>) { 506 + let mut res_a = vec![]; 507 + let mut res_b = vec![]; 508 + for (a, b) in iter { 509 + res_a.push(a); 510 + res_b.push(b); 511 + } 512 + (res_a, res_b) 132 513 }
+49 -39
rust/pin-init/internal/src/pinned_drop.rs
··· 1 1 // SPDX-License-Identifier: Apache-2.0 OR MIT 2 2 3 - #[cfg(not(kernel))] 4 - use proc_macro2 as proc_macro; 3 + use proc_macro2::TokenStream; 4 + use quote::quote; 5 + use syn::{parse::Nothing, parse_quote, spanned::Spanned, ImplItem, ItemImpl, Token}; 5 6 6 - use proc_macro::{TokenStream, TokenTree}; 7 + use crate::diagnostics::{DiagCtxt, ErrorGuaranteed}; 7 8 8 - pub(crate) fn pinned_drop(_args: TokenStream, input: TokenStream) -> TokenStream { 9 - let mut toks = input.into_iter().collect::<Vec<_>>(); 10 - assert!(!toks.is_empty()); 11 - // Ensure that we have an `impl` item. 12 - assert!(matches!(&toks[0], TokenTree::Ident(i) if i.to_string() == "impl")); 13 - // Ensure that we are implementing `PinnedDrop`. 14 - let mut nesting: usize = 0; 15 - let mut pinned_drop_idx = None; 16 - for (i, tt) in toks.iter().enumerate() { 17 - match tt { 18 - TokenTree::Punct(p) if p.as_char() == '<' => { 19 - nesting += 1; 9 + pub(crate) fn pinned_drop( 10 + _args: Nothing, 11 + mut input: ItemImpl, 12 + dcx: &mut DiagCtxt, 13 + ) -> Result<TokenStream, ErrorGuaranteed> { 14 + if let Some(unsafety) = input.unsafety { 15 + dcx.error(unsafety, "implementing `PinnedDrop` is safe"); 16 + } 17 + input.unsafety = Some(Token![unsafe](input.impl_token.span)); 18 + match &mut input.trait_ { 19 + Some((not, path, _for)) => { 20 + if let Some(not) = not { 21 + dcx.error(not, "cannot implement `!PinnedDrop`"); 20 22 } 21 - TokenTree::Punct(p) if p.as_char() == '>' => { 22 - nesting = nesting.checked_sub(1).unwrap(); 23 - continue; 23 + for (seg, expected) in path 24 + .segments 25 + .iter() 26 + .rev() 27 + .zip(["PinnedDrop", "pin_init", ""]) 28 + { 29 + if expected.is_empty() || seg.ident != expected { 30 + dcx.error(seg, "bad import path for `PinnedDrop`"); 31 + } 32 + if !seg.arguments.is_none() { 33 + dcx.error(&seg.arguments, "unexpected arguments for `PinnedDrop` path"); 34 + } 24 35 } 25 - _ => {} 36 + *path = parse_quote!(::pin_init::PinnedDrop); 26 37 } 27 - if i >= 1 && nesting == 0 { 28 - // Found the end of the generics, this should be `PinnedDrop`. 29 - assert!( 30 - matches!(tt, TokenTree::Ident(i) if i.to_string() == "PinnedDrop"), 31 - "expected 'PinnedDrop', found: '{tt:?}'" 38 + None => { 39 + let span = input 40 + .impl_token 41 + .span 42 + .join(input.self_ty.span()) 43 + .unwrap_or(input.impl_token.span); 44 + dcx.error( 45 + span, 46 + "expected `impl ... PinnedDrop for ...`, got inherent impl", 32 47 ); 33 - pinned_drop_idx = Some(i); 34 - break; 35 48 } 36 49 } 37 - let idx = pinned_drop_idx 38 - .unwrap_or_else(|| panic!("Expected an `impl` block implementing `PinnedDrop`.")); 39 - // Fully qualify the `PinnedDrop`, as to avoid any tampering. 40 - toks.splice(idx..idx, quote!(::pin_init::)); 41 - // Take the `{}` body and call the declarative macro. 42 - if let Some(TokenTree::Group(last)) = toks.pop() { 43 - let last = last.stream(); 44 - quote!(::pin_init::__pinned_drop! { 45 - @impl_sig(#(#toks)*), 46 - @impl_body(#last), 47 - }) 48 - } else { 49 - TokenStream::from_iter(toks) 50 + for item in &mut input.items { 51 + if let ImplItem::Fn(fn_item) = item { 52 + if fn_item.sig.ident == "drop" { 53 + fn_item 54 + .sig 55 + .inputs 56 + .push(parse_quote!(_: ::pin_init::__internal::OnlyCallFromDrop)); 57 + } 58 + } 50 59 } 60 + Ok(quote!(#input)) 51 61 }
+67 -90
rust/pin-init/internal/src/zeroable.rs
··· 1 1 // SPDX-License-Identifier: GPL-2.0 2 2 3 - #[cfg(not(kernel))] 4 - use proc_macro2 as proc_macro; 3 + use proc_macro2::TokenStream; 4 + use quote::quote; 5 + use syn::{parse_quote, Data, DeriveInput, Field, Fields}; 5 6 6 - use crate::helpers::{parse_generics, Generics}; 7 - use proc_macro::{TokenStream, TokenTree}; 7 + use crate::{diagnostics::ErrorGuaranteed, DiagCtxt}; 8 8 9 - pub(crate) fn parse_zeroable_derive_input( 10 - input: TokenStream, 11 - ) -> ( 12 - Vec<TokenTree>, 13 - Vec<TokenTree>, 14 - Vec<TokenTree>, 15 - Option<TokenTree>, 16 - ) { 17 - let ( 18 - Generics { 19 - impl_generics, 20 - decl_generics: _, 21 - ty_generics, 22 - }, 23 - mut rest, 24 - ) = parse_generics(input); 25 - // This should be the body of the struct `{...}`. 26 - let last = rest.pop(); 27 - // Now we insert `Zeroable` as a bound for every generic parameter in `impl_generics`. 28 - let mut new_impl_generics = Vec::with_capacity(impl_generics.len()); 29 - // Are we inside of a generic where we want to add `Zeroable`? 30 - let mut in_generic = !impl_generics.is_empty(); 31 - // Have we already inserted `Zeroable`? 32 - let mut inserted = false; 33 - // Level of `<>` nestings. 34 - let mut nested = 0; 35 - for tt in impl_generics { 36 - match &tt { 37 - // If we find a `,`, then we have finished a generic/constant/lifetime parameter. 38 - TokenTree::Punct(p) if nested == 0 && p.as_char() == ',' => { 39 - if in_generic && !inserted { 40 - new_impl_generics.extend(quote! { : ::pin_init::Zeroable }); 41 - } 42 - in_generic = true; 43 - inserted = false; 44 - new_impl_generics.push(tt); 45 - } 46 - // If we find `'`, then we are entering a lifetime. 47 - TokenTree::Punct(p) if nested == 0 && p.as_char() == '\'' => { 48 - in_generic = false; 49 - new_impl_generics.push(tt); 50 - } 51 - TokenTree::Punct(p) if nested == 0 && p.as_char() == ':' => { 52 - new_impl_generics.push(tt); 53 - if in_generic { 54 - new_impl_generics.extend(quote! { ::pin_init::Zeroable + }); 55 - inserted = true; 56 - } 57 - } 58 - TokenTree::Punct(p) if p.as_char() == '<' => { 59 - nested += 1; 60 - new_impl_generics.push(tt); 61 - } 62 - TokenTree::Punct(p) if p.as_char() == '>' => { 63 - assert!(nested > 0); 64 - nested -= 1; 65 - new_impl_generics.push(tt); 66 - } 67 - _ => new_impl_generics.push(tt), 9 + pub(crate) fn derive( 10 + input: DeriveInput, 11 + dcx: &mut DiagCtxt, 12 + ) -> Result<TokenStream, ErrorGuaranteed> { 13 + let fields = match input.data { 14 + Data::Struct(data_struct) => data_struct.fields, 15 + Data::Union(data_union) => Fields::Named(data_union.fields), 16 + Data::Enum(data_enum) => { 17 + return Err(dcx.error(data_enum.enum_token, "cannot derive `Zeroable` for an enum")); 68 18 } 19 + }; 20 + let name = input.ident; 21 + let mut generics = input.generics; 22 + for param in generics.type_params_mut() { 23 + param.bounds.insert(0, parse_quote!(::pin_init::Zeroable)); 69 24 } 70 - assert_eq!(nested, 0); 71 - if in_generic && !inserted { 72 - new_impl_generics.extend(quote! { : ::pin_init::Zeroable }); 73 - } 74 - (rest, new_impl_generics, ty_generics, last) 25 + let (impl_gen, ty_gen, whr) = generics.split_for_impl(); 26 + let field_type = fields.iter().map(|field| &field.ty); 27 + Ok(quote! { 28 + // SAFETY: Every field type implements `Zeroable` and padding bytes may be zero. 29 + #[automatically_derived] 30 + unsafe impl #impl_gen ::pin_init::Zeroable for #name #ty_gen 31 + #whr 32 + {} 33 + const _: () = { 34 + fn assert_zeroable<T: ?::core::marker::Sized + ::pin_init::Zeroable>() {} 35 + fn ensure_zeroable #impl_gen () 36 + #whr 37 + { 38 + #( 39 + assert_zeroable::<#field_type>(); 40 + )* 41 + } 42 + }; 43 + }) 75 44 } 76 45 77 - pub(crate) fn derive(input: TokenStream) -> TokenStream { 78 - let (rest, new_impl_generics, ty_generics, last) = parse_zeroable_derive_input(input); 79 - quote! { 80 - ::pin_init::__derive_zeroable!( 81 - parse_input: 82 - @sig(#(#rest)*), 83 - @impl_generics(#(#new_impl_generics)*), 84 - @ty_generics(#(#ty_generics)*), 85 - @body(#last), 86 - ); 46 + pub(crate) fn maybe_derive( 47 + input: DeriveInput, 48 + dcx: &mut DiagCtxt, 49 + ) -> Result<TokenStream, ErrorGuaranteed> { 50 + let fields = match input.data { 51 + Data::Struct(data_struct) => data_struct.fields, 52 + Data::Union(data_union) => Fields::Named(data_union.fields), 53 + Data::Enum(data_enum) => { 54 + return Err(dcx.error(data_enum.enum_token, "cannot derive `Zeroable` for an enum")); 55 + } 56 + }; 57 + let name = input.ident; 58 + let mut generics = input.generics; 59 + for param in generics.type_params_mut() { 60 + param.bounds.insert(0, parse_quote!(::pin_init::Zeroable)); 87 61 } 88 - } 89 - 90 - pub(crate) fn maybe_derive(input: TokenStream) -> TokenStream { 91 - let (rest, new_impl_generics, ty_generics, last) = parse_zeroable_derive_input(input); 92 - quote! { 93 - ::pin_init::__maybe_derive_zeroable!( 94 - parse_input: 95 - @sig(#(#rest)*), 96 - @impl_generics(#(#new_impl_generics)*), 97 - @ty_generics(#(#ty_generics)*), 98 - @body(#last), 99 - ); 62 + for Field { ty, .. } in fields { 63 + generics 64 + .make_where_clause() 65 + .predicates 66 + // the `for<'__dummy>` HRTB makes this not error without the `trivial_bounds` 67 + // feature <https://github.com/rust-lang/rust/issues/48214#issuecomment-2557829956>. 68 + .push(parse_quote!(#ty: for<'__dummy> ::pin_init::Zeroable)); 100 69 } 70 + let (impl_gen, ty_gen, whr) = generics.split_for_impl(); 71 + Ok(quote! { 72 + // SAFETY: Every field type implements `Zeroable` and padding bytes may be zero. 73 + #[automatically_derived] 74 + unsafe impl #impl_gen ::pin_init::Zeroable for #name #ty_gen 75 + #whr 76 + {} 77 + }) 101 78 }
+47 -153
rust/pin-init/src/lib.rs
··· 146 146 //! 147 147 //! impl DriverData { 148 148 //! fn new() -> impl PinInit<Self, Error> { 149 - //! try_pin_init!(Self { 149 + //! pin_init!(Self { 150 150 //! status <- CMutex::new(0), 151 151 //! buffer: Box::init(pin_init::init_zeroed())?, 152 152 //! }? Error) ··· 290 290 ptr::{self, NonNull}, 291 291 }; 292 292 293 + // This is used by doc-tests -- the proc-macros expand to `::pin_init::...` and without this the 294 + // doc-tests wouldn't have an extern crate named `pin_init`. 295 + #[allow(unused_extern_crates)] 296 + extern crate self as pin_init; 297 + 293 298 #[doc(hidden)] 294 299 pub mod __internal; 295 - #[doc(hidden)] 296 - pub mod macros; 297 300 298 301 #[cfg(any(feature = "std", feature = "alloc"))] 299 302 mod alloc; ··· 531 528 /// x: u32, 532 529 /// } 533 530 /// 534 - /// stack_try_pin_init!(let foo: Foo = try_pin_init!(Foo { 531 + /// stack_try_pin_init!(let foo: Foo = pin_init!(Foo { 535 532 /// a <- CMutex::new(42), 536 533 /// b: Box::try_new(Bar { 537 534 /// x: 64, ··· 558 555 /// x: u32, 559 556 /// } 560 557 /// 561 - /// stack_try_pin_init!(let foo: Foo =? try_pin_init!(Foo { 558 + /// stack_try_pin_init!(let foo: Foo =? pin_init!(Foo { 562 559 /// a <- CMutex::new(42), 563 560 /// b: Box::try_new(Bar { 564 561 /// x: 64, ··· 587 584 }; 588 585 } 589 586 590 - /// Construct an in-place, pinned initializer for `struct`s. 587 + /// Construct an in-place, fallible pinned initializer for `struct`s. 591 588 /// 592 - /// This macro defaults the error to [`Infallible`]. If you need a different error, then use 593 - /// [`try_pin_init!`]. 589 + /// The error type defaults to [`Infallible`]; if you need a different one, write `? Error` at the 590 + /// end, after the struct initializer. 594 591 /// 595 592 /// The syntax is almost identical to that of a normal `struct` initializer: 596 593 /// ··· 779 776 /// ``` 780 777 /// 781 778 /// [`NonNull<Self>`]: core::ptr::NonNull 782 - // For a detailed example of how this macro works, see the module documentation of the hidden 783 - // module `macros` inside of `macros.rs`. 784 - #[macro_export] 785 - macro_rules! pin_init { 786 - ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? { 787 - $($fields:tt)* 788 - }) => { 789 - $crate::try_pin_init!($(&$this in)? $t $(::<$($generics),*>)? { 790 - $($fields)* 791 - }? ::core::convert::Infallible) 792 - }; 793 - } 779 + pub use pin_init_internal::pin_init; 794 780 795 - /// Construct an in-place, fallible pinned initializer for `struct`s. 781 + /// Construct an in-place, fallible initializer for `struct`s. 796 782 /// 797 - /// If the initialization can complete without error (or [`Infallible`]), then use [`pin_init!`]. 798 - /// 799 - /// You can use the `?` operator or use `return Err(err)` inside the initializer to stop 800 - /// initialization and return the error. 801 - /// 802 - /// IMPORTANT: if you have `unsafe` code inside of the initializer you have to ensure that when 803 - /// initialization fails, the memory can be safely deallocated without any further modifications. 804 - /// 805 - /// The syntax is identical to [`pin_init!`] with the following exception: you must append `? $type` 806 - /// after the `struct` initializer to specify the error type you want to use. 807 - /// 808 - /// # Examples 809 - /// 810 - /// ```rust 811 - /// # #![feature(allocator_api)] 812 - /// # #[path = "../examples/error.rs"] mod error; use error::Error; 813 - /// use pin_init::{pin_data, try_pin_init, PinInit, InPlaceInit, init_zeroed}; 814 - /// 815 - /// #[pin_data] 816 - /// struct BigBuf { 817 - /// big: Box<[u8; 1024 * 1024 * 1024]>, 818 - /// small: [u8; 1024 * 1024], 819 - /// ptr: *mut u8, 820 - /// } 821 - /// 822 - /// impl BigBuf { 823 - /// fn new() -> impl PinInit<Self, Error> { 824 - /// try_pin_init!(Self { 825 - /// big: Box::init(init_zeroed())?, 826 - /// small: [0; 1024 * 1024], 827 - /// ptr: core::ptr::null_mut(), 828 - /// }? Error) 829 - /// } 830 - /// } 831 - /// # let _ = Box::pin_init(BigBuf::new()); 832 - /// ``` 833 - // For a detailed example of how this macro works, see the module documentation of the hidden 834 - // module `macros` inside of `macros.rs`. 835 - #[macro_export] 836 - macro_rules! try_pin_init { 837 - ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? { 838 - $($fields:tt)* 839 - }? $err:ty) => { 840 - $crate::__init_internal!( 841 - @this($($this)?), 842 - @typ($t $(::<$($generics),*>)? ), 843 - @fields($($fields)*), 844 - @error($err), 845 - @data(PinData, use_data), 846 - @has_data(HasPinData, __pin_data), 847 - @construct_closure(pin_init_from_closure), 848 - @munch_fields($($fields)*), 849 - ) 850 - } 851 - } 852 - 853 - /// Construct an in-place initializer for `struct`s. 854 - /// 855 - /// This macro defaults the error to [`Infallible`]. If you need a different error, then use 856 - /// [`try_init!`]. 783 + /// This macro defaults the error to [`Infallible`]; if you need a different one, write `? Error` 784 + /// at the end, after the struct initializer. 857 785 /// 858 786 /// The syntax is identical to [`pin_init!`] and its safety caveats also apply: 859 787 /// - `unsafe` code must guarantee either full initialization or return an error and allow ··· 817 883 /// } 818 884 /// # let _ = Box::init(BigBuf::new()); 819 885 /// ``` 820 - // For a detailed example of how this macro works, see the module documentation of the hidden 821 - // module `macros` inside of `macros.rs`. 822 - #[macro_export] 823 - macro_rules! init { 824 - ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? { 825 - $($fields:tt)* 826 - }) => { 827 - $crate::try_init!($(&$this in)? $t $(::<$($generics),*>)? { 828 - $($fields)* 829 - }? ::core::convert::Infallible) 830 - } 831 - } 832 - 833 - /// Construct an in-place fallible initializer for `struct`s. 834 - /// 835 - /// If the initialization can complete without error (or [`Infallible`]), then use 836 - /// [`init!`]. 837 - /// 838 - /// The syntax is identical to [`try_pin_init!`]. You need to specify a custom error 839 - /// via `? $type` after the `struct` initializer. 840 - /// The safety caveats from [`try_pin_init!`] also apply: 841 - /// - `unsafe` code must guarantee either full initialization or return an error and allow 842 - /// deallocation of the memory. 843 - /// - the fields are initialized in the order given in the initializer. 844 - /// - no references to fields are allowed to be created inside of the initializer. 845 - /// 846 - /// # Examples 847 - /// 848 - /// ```rust 849 - /// # #![feature(allocator_api)] 850 - /// # use core::alloc::AllocError; 851 - /// # use pin_init::InPlaceInit; 852 - /// use pin_init::{try_init, Init, init_zeroed}; 853 - /// 854 - /// struct BigBuf { 855 - /// big: Box<[u8; 1024 * 1024 * 1024]>, 856 - /// small: [u8; 1024 * 1024], 857 - /// } 858 - /// 859 - /// impl BigBuf { 860 - /// fn new() -> impl Init<Self, AllocError> { 861 - /// try_init!(Self { 862 - /// big: Box::init(init_zeroed())?, 863 - /// small: [0; 1024 * 1024], 864 - /// }? AllocError) 865 - /// } 866 - /// } 867 - /// # let _ = Box::init(BigBuf::new()); 868 - /// ``` 869 - // For a detailed example of how this macro works, see the module documentation of the hidden 870 - // module `macros` inside of `macros.rs`. 871 - #[macro_export] 872 - macro_rules! try_init { 873 - ($(&$this:ident in)? $t:ident $(::<$($generics:ty),* $(,)?>)? { 874 - $($fields:tt)* 875 - }? $err:ty) => { 876 - $crate::__init_internal!( 877 - @this($($this)?), 878 - @typ($t $(::<$($generics),*>)?), 879 - @fields($($fields)*), 880 - @error($err), 881 - @data(InitData, /*no use_data*/), 882 - @has_data(HasInitData, __init_data), 883 - @construct_closure(init_from_closure), 884 - @munch_fields($($fields)*), 885 - ) 886 - } 887 - } 886 + pub use pin_init_internal::init; 888 887 889 888 /// Asserts that a field on a struct using `#[pin_data]` is marked with `#[pin]` ie. that it is 890 889 /// structurally pinned. ··· 1277 1410 /// fn init_foo() -> impl PinInit<Foo, Error> { 1278 1411 /// pin_init_scope(|| { 1279 1412 /// let bar = lookup_bar()?; 1280 - /// Ok(try_pin_init!(Foo { a: bar.a.into(), b: bar.b }? Error)) 1413 + /// Ok(pin_init!(Foo { a: bar.a.into(), b: bar.b }? Error)) 1281 1414 /// }) 1282 1415 /// } 1283 1416 /// ``` 1284 1417 /// 1285 1418 /// This initializer will first execute `lookup_bar()`, match on it, if it returned an error, the 1286 1419 /// initializer itself will fail with that error. If it returned `Ok`, then it will run the 1287 - /// initializer returned by the [`try_pin_init!`] invocation. 1420 + /// initializer returned by the [`pin_init!`] invocation. 1288 1421 pub fn pin_init_scope<T, E, F, I>(make_init: F) -> impl PinInit<T, E> 1289 1422 where 1290 1423 F: FnOnce() -> Result<I, E>, ··· 1320 1453 /// fn init_foo() -> impl Init<Foo, Error> { 1321 1454 /// init_scope(|| { 1322 1455 /// let bar = lookup_bar()?; 1323 - /// Ok(try_init!(Foo { a: bar.a.into(), b: bar.b }? Error)) 1456 + /// Ok(init!(Foo { a: bar.a.into(), b: bar.b }? Error)) 1324 1457 /// }) 1325 1458 /// } 1326 1459 /// ``` 1327 1460 /// 1328 1461 /// This initializer will first execute `lookup_bar()`, match on it, if it returned an error, the 1329 1462 /// initializer itself will fail with that error. If it returned `Ok`, then it will run the 1330 - /// initializer returned by the [`try_init!`] invocation. 1463 + /// initializer returned by the [`init!`] invocation. 1331 1464 pub fn init_scope<T, E, F, I>(make_init: F) -> impl Init<T, E> 1332 1465 where 1333 1466 F: FnOnce() -> Result<I, E>, ··· 1401 1534 /// 1402 1535 /// Does not drop the current value and considers it as uninitialized memory. 1403 1536 fn write_pin_init<E>(self, init: impl PinInit<T, E>) -> Result<Pin<Self::Initialized>, E>; 1537 + } 1538 + 1539 + impl<T> InPlaceWrite<T> for &'static mut MaybeUninit<T> { 1540 + type Initialized = &'static mut T; 1541 + 1542 + fn write_init<E>(self, init: impl Init<T, E>) -> Result<Self::Initialized, E> { 1543 + let slot = self.as_mut_ptr(); 1544 + 1545 + // SAFETY: `slot` is a valid pointer to uninitialized memory. 1546 + unsafe { init.__init(slot)? }; 1547 + 1548 + // SAFETY: The above call initialized the memory. 1549 + unsafe { Ok(self.assume_init_mut()) } 1550 + } 1551 + 1552 + fn write_pin_init<E>(self, init: impl PinInit<T, E>) -> Result<Pin<Self::Initialized>, E> { 1553 + let slot = self.as_mut_ptr(); 1554 + 1555 + // SAFETY: `slot` is a valid pointer to uninitialized memory. 1556 + // 1557 + // The `'static` borrow guarantees the data will not be 1558 + // moved/invalidated until it gets dropped (which is never). 1559 + unsafe { init.__pinned_init(slot)? }; 1560 + 1561 + // SAFETY: The above call initialized the memory. 1562 + Ok(Pin::static_mut(unsafe { self.assume_init_mut() })) 1563 + } 1404 1564 } 1405 1565 1406 1566 /// Trait facilitating pinned destruction.
-1677
rust/pin-init/src/macros.rs
··· 1 - // SPDX-License-Identifier: Apache-2.0 OR MIT 2 - 3 - //! This module provides the macros that actually implement the proc-macros `pin_data` and 4 - //! `pinned_drop`. It also contains `__init_internal`, the implementation of the 5 - //! `{try_}{pin_}init!` macros. 6 - //! 7 - //! These macros should never be called directly, since they expect their input to be 8 - //! in a certain format which is internal. If used incorrectly, these macros can lead to UB even in 9 - //! safe code! Use the public facing macros instead. 10 - //! 11 - //! This architecture has been chosen because the kernel does not yet have access to `syn` which 12 - //! would make matters a lot easier for implementing these as proc-macros. 13 - //! 14 - //! Since this library and the kernel implementation should diverge as little as possible, the same 15 - //! approach has been taken here. 16 - //! 17 - //! # Macro expansion example 18 - //! 19 - //! This section is intended for readers trying to understand the macros in this module and the 20 - //! `[try_][pin_]init!` macros from `lib.rs`. 21 - //! 22 - //! We will look at the following example: 23 - //! 24 - //! ```rust,ignore 25 - //! #[pin_data] 26 - //! #[repr(C)] 27 - //! struct Bar<T> { 28 - //! #[pin] 29 - //! t: T, 30 - //! pub x: usize, 31 - //! } 32 - //! 33 - //! impl<T> Bar<T> { 34 - //! fn new(t: T) -> impl PinInit<Self> { 35 - //! pin_init!(Self { t, x: 0 }) 36 - //! } 37 - //! } 38 - //! 39 - //! #[pin_data(PinnedDrop)] 40 - //! struct Foo { 41 - //! a: usize, 42 - //! #[pin] 43 - //! b: Bar<u32>, 44 - //! } 45 - //! 46 - //! #[pinned_drop] 47 - //! impl PinnedDrop for Foo { 48 - //! fn drop(self: Pin<&mut Self>) { 49 - //! println!("{self:p} is getting dropped."); 50 - //! } 51 - //! } 52 - //! 53 - //! let a = 42; 54 - //! let initializer = pin_init!(Foo { 55 - //! a, 56 - //! b <- Bar::new(36), 57 - //! }); 58 - //! ``` 59 - //! 60 - //! This example includes the most common and important features of the pin-init API. 61 - //! 62 - //! Below you can find individual section about the different macro invocations. Here are some 63 - //! general things we need to take into account when designing macros: 64 - //! - use global paths, similarly to file paths, these start with the separator: `::core::panic!()` 65 - //! this ensures that the correct item is used, since users could define their own `mod core {}` 66 - //! and then their own `panic!` inside to execute arbitrary code inside of our macro. 67 - //! - macro `unsafe` hygiene: we need to ensure that we do not expand arbitrary, user-supplied 68 - //! expressions inside of an `unsafe` block in the macro, because this would allow users to do 69 - //! `unsafe` operations without an associated `unsafe` block. 70 - //! 71 - //! ## `#[pin_data]` on `Bar` 72 - //! 73 - //! This macro is used to specify which fields are structurally pinned and which fields are not. It 74 - //! is placed on the struct definition and allows `#[pin]` to be placed on the fields. 75 - //! 76 - //! Here is the definition of `Bar` from our example: 77 - //! 78 - //! ```rust,ignore 79 - //! #[pin_data] 80 - //! #[repr(C)] 81 - //! struct Bar<T> { 82 - //! #[pin] 83 - //! t: T, 84 - //! pub x: usize, 85 - //! } 86 - //! ``` 87 - //! 88 - //! This expands to the following code: 89 - //! 90 - //! ```rust,ignore 91 - //! // Firstly the normal definition of the struct, attributes are preserved: 92 - //! #[repr(C)] 93 - //! struct Bar<T> { 94 - //! t: T, 95 - //! pub x: usize, 96 - //! } 97 - //! // Then an anonymous constant is defined, this is because we do not want any code to access the 98 - //! // types that we define inside: 99 - //! const _: () = { 100 - //! // We define the pin-data carrying struct, it is a ZST and needs to have the same generics, 101 - //! // since we need to implement access functions for each field and thus need to know its 102 - //! // type. 103 - //! struct __ThePinData<T> { 104 - //! __phantom: ::core::marker::PhantomData<fn(Bar<T>) -> Bar<T>>, 105 - //! } 106 - //! // We implement `Copy` for the pin-data struct, since all functions it defines will take 107 - //! // `self` by value. 108 - //! impl<T> ::core::clone::Clone for __ThePinData<T> { 109 - //! fn clone(&self) -> Self { 110 - //! *self 111 - //! } 112 - //! } 113 - //! impl<T> ::core::marker::Copy for __ThePinData<T> {} 114 - //! // For every field of `Bar`, the pin-data struct will define a function with the same name 115 - //! // and accessor (`pub` or `pub(crate)` etc.). This function will take a pointer to the 116 - //! // field (`slot`) and a `PinInit` or `Init` depending on the projection kind of the field 117 - //! // (if pinning is structural for the field, then `PinInit` otherwise `Init`). 118 - //! #[allow(dead_code)] 119 - //! impl<T> __ThePinData<T> { 120 - //! unsafe fn t<E>( 121 - //! self, 122 - //! slot: *mut T, 123 - //! // Since `t` is `#[pin]`, this is `PinInit`. 124 - //! init: impl ::pin_init::PinInit<T, E>, 125 - //! ) -> ::core::result::Result<(), E> { 126 - //! unsafe { ::pin_init::PinInit::__pinned_init(init, slot) } 127 - //! } 128 - //! pub unsafe fn x<E>( 129 - //! self, 130 - //! slot: *mut usize, 131 - //! // Since `x` is not `#[pin]`, this is `Init`. 132 - //! init: impl ::pin_init::Init<usize, E>, 133 - //! ) -> ::core::result::Result<(), E> { 134 - //! unsafe { ::pin_init::Init::__init(init, slot) } 135 - //! } 136 - //! } 137 - //! // Implement the internal `HasPinData` trait that associates `Bar` with the pin-data struct 138 - //! // that we constructed above. 139 - //! unsafe impl<T> ::pin_init::__internal::HasPinData for Bar<T> { 140 - //! type PinData = __ThePinData<T>; 141 - //! unsafe fn __pin_data() -> Self::PinData { 142 - //! __ThePinData { 143 - //! __phantom: ::core::marker::PhantomData, 144 - //! } 145 - //! } 146 - //! } 147 - //! // Implement the internal `PinData` trait that marks the pin-data struct as a pin-data 148 - //! // struct. This is important to ensure that no user can implement a rogue `__pin_data` 149 - //! // function without using `unsafe`. 150 - //! unsafe impl<T> ::pin_init::__internal::PinData for __ThePinData<T> { 151 - //! type Datee = Bar<T>; 152 - //! } 153 - //! // Now we only want to implement `Unpin` for `Bar` when every structurally pinned field is 154 - //! // `Unpin`. In other words, whether `Bar` is `Unpin` only depends on structurally pinned 155 - //! // fields (those marked with `#[pin]`). These fields will be listed in this struct, in our 156 - //! // case no such fields exist, hence this is almost empty. The two phantomdata fields exist 157 - //! // for two reasons: 158 - //! // - `__phantom`: every generic must be used, since we cannot really know which generics 159 - //! // are used, we declare all and then use everything here once. 160 - //! // - `__phantom_pin`: uses the `'__pin` lifetime and ensures that this struct is invariant 161 - //! // over it. The lifetime is needed to work around the limitation that trait bounds must 162 - //! // not be trivial, e.g. the user has a `#[pin] PhantomPinned` field -- this is 163 - //! // unconditionally `!Unpin` and results in an error. The lifetime tricks the compiler 164 - //! // into accepting these bounds regardless. 165 - //! #[allow(dead_code)] 166 - //! struct __Unpin<'__pin, T> { 167 - //! __phantom_pin: ::core::marker::PhantomData<fn(&'__pin ()) -> &'__pin ()>, 168 - //! __phantom: ::core::marker::PhantomData<fn(Bar<T>) -> Bar<T>>, 169 - //! // Our only `#[pin]` field is `t`. 170 - //! t: T, 171 - //! } 172 - //! #[doc(hidden)] 173 - //! impl<'__pin, T> ::core::marker::Unpin for Bar<T> 174 - //! where 175 - //! __Unpin<'__pin, T>: ::core::marker::Unpin, 176 - //! {} 177 - //! // Now we need to ensure that `Bar` does not implement `Drop`, since that would give users 178 - //! // access to `&mut self` inside of `drop` even if the struct was pinned. This could lead to 179 - //! // UB with only safe code, so we disallow this by giving a trait implementation error using 180 - //! // a direct impl and a blanket implementation. 181 - //! trait MustNotImplDrop {} 182 - //! // Normally `Drop` bounds do not have the correct semantics, but for this purpose they do 183 - //! // (normally people want to know if a type has any kind of drop glue at all, here we want 184 - //! // to know if it has any kind of custom drop glue, which is exactly what this bound does). 185 - //! #[expect(drop_bounds)] 186 - //! impl<T: ::core::ops::Drop> MustNotImplDrop for T {} 187 - //! impl<T> MustNotImplDrop for Bar<T> {} 188 - //! // Here comes a convenience check, if one implemented `PinnedDrop`, but forgot to add it to 189 - //! // `#[pin_data]`, then this will error with the same mechanic as above, this is not needed 190 - //! // for safety, but a good sanity check, since no normal code calls `PinnedDrop::drop`. 191 - //! #[expect(non_camel_case_types)] 192 - //! trait UselessPinnedDropImpl_you_need_to_specify_PinnedDrop {} 193 - //! impl< 194 - //! T: ::pin_init::PinnedDrop, 195 - //! > UselessPinnedDropImpl_you_need_to_specify_PinnedDrop for T {} 196 - //! impl<T> UselessPinnedDropImpl_you_need_to_specify_PinnedDrop for Bar<T> {} 197 - //! }; 198 - //! ``` 199 - //! 200 - //! ## `pin_init!` in `impl Bar` 201 - //! 202 - //! This macro creates an pin-initializer for the given struct. It requires that the struct is 203 - //! annotated by `#[pin_data]`. 204 - //! 205 - //! Here is the impl on `Bar` defining the new function: 206 - //! 207 - //! ```rust,ignore 208 - //! impl<T> Bar<T> { 209 - //! fn new(t: T) -> impl PinInit<Self> { 210 - //! pin_init!(Self { t, x: 0 }) 211 - //! } 212 - //! } 213 - //! ``` 214 - //! 215 - //! This expands to the following code: 216 - //! 217 - //! ```rust,ignore 218 - //! impl<T> Bar<T> { 219 - //! fn new(t: T) -> impl PinInit<Self> { 220 - //! { 221 - //! // We do not want to allow arbitrary returns, so we declare this type as the `Ok` 222 - //! // return type and shadow it later when we insert the arbitrary user code. That way 223 - //! // there will be no possibility of returning without `unsafe`. 224 - //! struct __InitOk; 225 - //! // Get the data about fields from the supplied type. 226 - //! // - the function is unsafe, hence the unsafe block 227 - //! // - we `use` the `HasPinData` trait in the block, it is only available in that 228 - //! // scope. 229 - //! let data = unsafe { 230 - //! use ::pin_init::__internal::HasPinData; 231 - //! Self::__pin_data() 232 - //! }; 233 - //! // Ensure that `data` really is of type `PinData` and help with type inference: 234 - //! let init = ::pin_init::__internal::PinData::make_closure::< 235 - //! _, 236 - //! __InitOk, 237 - //! ::core::convert::Infallible, 238 - //! >(data, move |slot| { 239 - //! { 240 - //! // Shadow the structure so it cannot be used to return early. If a user 241 - //! // tries to write `return Ok(__InitOk)`, then they get a type error, 242 - //! // since that will refer to this struct instead of the one defined 243 - //! // above. 244 - //! struct __InitOk; 245 - //! // This is the expansion of `t,`, which is syntactic sugar for `t: t,`. 246 - //! { 247 - //! unsafe { ::core::ptr::write(::core::addr_of_mut!((*slot).t), t) }; 248 - //! } 249 - //! // Since initialization could fail later (not in this case, since the 250 - //! // error type is `Infallible`) we will need to drop this field if there 251 - //! // is an error later. This `DropGuard` will drop the field when it gets 252 - //! // dropped and has not yet been forgotten. 253 - //! let __t_guard = unsafe { 254 - //! ::pin_init::__internal::DropGuard::new(::core::addr_of_mut!((*slot).t)) 255 - //! }; 256 - //! // Expansion of `x: 0,`: 257 - //! // Since this can be an arbitrary expression we cannot place it inside 258 - //! // of the `unsafe` block, so we bind it here. 259 - //! { 260 - //! let x = 0; 261 - //! unsafe { ::core::ptr::write(::core::addr_of_mut!((*slot).x), x) }; 262 - //! } 263 - //! // We again create a `DropGuard`. 264 - //! let __x_guard = unsafe { 265 - //! ::pin_init::__internal::DropGuard::new(::core::addr_of_mut!((*slot).x)) 266 - //! }; 267 - //! // Since initialization has successfully completed, we can now forget 268 - //! // the guards. This is not `mem::forget`, since we only have 269 - //! // `&DropGuard`. 270 - //! ::core::mem::forget(__x_guard); 271 - //! ::core::mem::forget(__t_guard); 272 - //! // Here we use the type checker to ensure that every field has been 273 - //! // initialized exactly once, since this is `if false` it will never get 274 - //! // executed, but still type-checked. 275 - //! // Additionally we abuse `slot` to automatically infer the correct type 276 - //! // for the struct. This is also another check that every field is 277 - //! // accessible from this scope. 278 - //! #[allow(unreachable_code, clippy::diverging_sub_expression)] 279 - //! let _ = || { 280 - //! unsafe { 281 - //! ::core::ptr::write( 282 - //! slot, 283 - //! Self { 284 - //! // We only care about typecheck finding every field 285 - //! // here, the expression does not matter, just conjure 286 - //! // one using `panic!()`: 287 - //! t: ::core::panic!(), 288 - //! x: ::core::panic!(), 289 - //! }, 290 - //! ); 291 - //! }; 292 - //! }; 293 - //! } 294 - //! // We leave the scope above and gain access to the previously shadowed 295 - //! // `__InitOk` that we need to return. 296 - //! Ok(__InitOk) 297 - //! }); 298 - //! // Change the return type from `__InitOk` to `()`. 299 - //! let init = move | 300 - //! slot, 301 - //! | -> ::core::result::Result<(), ::core::convert::Infallible> { 302 - //! init(slot).map(|__InitOk| ()) 303 - //! }; 304 - //! // Construct the initializer. 305 - //! let init = unsafe { 306 - //! ::pin_init::pin_init_from_closure::< 307 - //! _, 308 - //! ::core::convert::Infallible, 309 - //! >(init) 310 - //! }; 311 - //! init 312 - //! } 313 - //! } 314 - //! } 315 - //! ``` 316 - //! 317 - //! ## `#[pin_data]` on `Foo` 318 - //! 319 - //! Since we already took a look at `#[pin_data]` on `Bar`, this section will only explain the 320 - //! differences/new things in the expansion of the `Foo` definition: 321 - //! 322 - //! ```rust,ignore 323 - //! #[pin_data(PinnedDrop)] 324 - //! struct Foo { 325 - //! a: usize, 326 - //! #[pin] 327 - //! b: Bar<u32>, 328 - //! } 329 - //! ``` 330 - //! 331 - //! This expands to the following code: 332 - //! 333 - //! ```rust,ignore 334 - //! struct Foo { 335 - //! a: usize, 336 - //! b: Bar<u32>, 337 - //! } 338 - //! const _: () = { 339 - //! struct __ThePinData { 340 - //! __phantom: ::core::marker::PhantomData<fn(Foo) -> Foo>, 341 - //! } 342 - //! impl ::core::clone::Clone for __ThePinData { 343 - //! fn clone(&self) -> Self { 344 - //! *self 345 - //! } 346 - //! } 347 - //! impl ::core::marker::Copy for __ThePinData {} 348 - //! #[allow(dead_code)] 349 - //! impl __ThePinData { 350 - //! unsafe fn b<E>( 351 - //! self, 352 - //! slot: *mut Bar<u32>, 353 - //! init: impl ::pin_init::PinInit<Bar<u32>, E>, 354 - //! ) -> ::core::result::Result<(), E> { 355 - //! unsafe { ::pin_init::PinInit::__pinned_init(init, slot) } 356 - //! } 357 - //! unsafe fn a<E>( 358 - //! self, 359 - //! slot: *mut usize, 360 - //! init: impl ::pin_init::Init<usize, E>, 361 - //! ) -> ::core::result::Result<(), E> { 362 - //! unsafe { ::pin_init::Init::__init(init, slot) } 363 - //! } 364 - //! } 365 - //! unsafe impl ::pin_init::__internal::HasPinData for Foo { 366 - //! type PinData = __ThePinData; 367 - //! unsafe fn __pin_data() -> Self::PinData { 368 - //! __ThePinData { 369 - //! __phantom: ::core::marker::PhantomData, 370 - //! } 371 - //! } 372 - //! } 373 - //! unsafe impl ::pin_init::__internal::PinData for __ThePinData { 374 - //! type Datee = Foo; 375 - //! } 376 - //! #[allow(dead_code)] 377 - //! struct __Unpin<'__pin> { 378 - //! __phantom_pin: ::core::marker::PhantomData<fn(&'__pin ()) -> &'__pin ()>, 379 - //! __phantom: ::core::marker::PhantomData<fn(Foo) -> Foo>, 380 - //! b: Bar<u32>, 381 - //! } 382 - //! #[doc(hidden)] 383 - //! impl<'__pin> ::core::marker::Unpin for Foo 384 - //! where 385 - //! __Unpin<'__pin>: ::core::marker::Unpin, 386 - //! {} 387 - //! // Since we specified `PinnedDrop` as the argument to `#[pin_data]`, we expect `Foo` to 388 - //! // implement `PinnedDrop`. Thus we do not need to prevent `Drop` implementations like 389 - //! // before, instead we implement `Drop` here and delegate to `PinnedDrop`. 390 - //! impl ::core::ops::Drop for Foo { 391 - //! fn drop(&mut self) { 392 - //! // Since we are getting dropped, no one else has a reference to `self` and thus we 393 - //! // can assume that we never move. 394 - //! let pinned = unsafe { ::core::pin::Pin::new_unchecked(self) }; 395 - //! // Create the unsafe token that proves that we are inside of a destructor, this 396 - //! // type is only allowed to be created in a destructor. 397 - //! let token = unsafe { ::pin_init::__internal::OnlyCallFromDrop::new() }; 398 - //! ::pin_init::PinnedDrop::drop(pinned, token); 399 - //! } 400 - //! } 401 - //! }; 402 - //! ``` 403 - //! 404 - //! ## `#[pinned_drop]` on `impl PinnedDrop for Foo` 405 - //! 406 - //! This macro is used to implement the `PinnedDrop` trait, since that trait is `unsafe` and has an 407 - //! extra parameter that should not be used at all. The macro hides that parameter. 408 - //! 409 - //! Here is the `PinnedDrop` impl for `Foo`: 410 - //! 411 - //! ```rust,ignore 412 - //! #[pinned_drop] 413 - //! impl PinnedDrop for Foo { 414 - //! fn drop(self: Pin<&mut Self>) { 415 - //! println!("{self:p} is getting dropped."); 416 - //! } 417 - //! } 418 - //! ``` 419 - //! 420 - //! This expands to the following code: 421 - //! 422 - //! ```rust,ignore 423 - //! // `unsafe`, full path and the token parameter are added, everything else stays the same. 424 - //! unsafe impl ::pin_init::PinnedDrop for Foo { 425 - //! fn drop(self: Pin<&mut Self>, _: ::pin_init::__internal::OnlyCallFromDrop) { 426 - //! println!("{self:p} is getting dropped."); 427 - //! } 428 - //! } 429 - //! ``` 430 - //! 431 - //! ## `pin_init!` on `Foo` 432 - //! 433 - //! Since we already took a look at `pin_init!` on `Bar`, this section will only show the expansion 434 - //! of `pin_init!` on `Foo`: 435 - //! 436 - //! ```rust,ignore 437 - //! let a = 42; 438 - //! let initializer = pin_init!(Foo { 439 - //! a, 440 - //! b <- Bar::new(36), 441 - //! }); 442 - //! ``` 443 - //! 444 - //! This expands to the following code: 445 - //! 446 - //! ```rust,ignore 447 - //! let a = 42; 448 - //! let initializer = { 449 - //! struct __InitOk; 450 - //! let data = unsafe { 451 - //! use ::pin_init::__internal::HasPinData; 452 - //! Foo::__pin_data() 453 - //! }; 454 - //! let init = ::pin_init::__internal::PinData::make_closure::< 455 - //! _, 456 - //! __InitOk, 457 - //! ::core::convert::Infallible, 458 - //! >(data, move |slot| { 459 - //! { 460 - //! struct __InitOk; 461 - //! { 462 - //! unsafe { ::core::ptr::write(::core::addr_of_mut!((*slot).a), a) }; 463 - //! } 464 - //! let __a_guard = unsafe { 465 - //! ::pin_init::__internal::DropGuard::new(::core::addr_of_mut!((*slot).a)) 466 - //! }; 467 - //! let init = Bar::new(36); 468 - //! unsafe { data.b(::core::addr_of_mut!((*slot).b), b)? }; 469 - //! let __b_guard = unsafe { 470 - //! ::pin_init::__internal::DropGuard::new(::core::addr_of_mut!((*slot).b)) 471 - //! }; 472 - //! ::core::mem::forget(__b_guard); 473 - //! ::core::mem::forget(__a_guard); 474 - //! #[allow(unreachable_code, clippy::diverging_sub_expression)] 475 - //! let _ = || { 476 - //! unsafe { 477 - //! ::core::ptr::write( 478 - //! slot, 479 - //! Foo { 480 - //! a: ::core::panic!(), 481 - //! b: ::core::panic!(), 482 - //! }, 483 - //! ); 484 - //! }; 485 - //! }; 486 - //! } 487 - //! Ok(__InitOk) 488 - //! }); 489 - //! let init = move | 490 - //! slot, 491 - //! | -> ::core::result::Result<(), ::core::convert::Infallible> { 492 - //! init(slot).map(|__InitOk| ()) 493 - //! }; 494 - //! let init = unsafe { 495 - //! ::pin_init::pin_init_from_closure::<_, ::core::convert::Infallible>(init) 496 - //! }; 497 - //! init 498 - //! }; 499 - //! ``` 500 - 501 - #[cfg(kernel)] 502 - pub use ::macros::paste; 503 - #[cfg(not(kernel))] 504 - pub use ::paste::paste; 505 - 506 - /// Creates a `unsafe impl<...> PinnedDrop for $type` block. 507 - /// 508 - /// See [`PinnedDrop`] for more information. 509 - /// 510 - /// [`PinnedDrop`]: crate::PinnedDrop 511 - #[doc(hidden)] 512 - #[macro_export] 513 - macro_rules! __pinned_drop { 514 - ( 515 - @impl_sig($($impl_sig:tt)*), 516 - @impl_body( 517 - $(#[$($attr:tt)*])* 518 - fn drop($($sig:tt)*) { 519 - $($inner:tt)* 520 - } 521 - ), 522 - ) => { 523 - // SAFETY: TODO. 524 - unsafe $($impl_sig)* { 525 - // Inherit all attributes and the type/ident tokens for the signature. 526 - $(#[$($attr)*])* 527 - fn drop($($sig)*, _: $crate::__internal::OnlyCallFromDrop) { 528 - $($inner)* 529 - } 530 - } 531 - } 532 - } 533 - 534 - /// This macro first parses the struct definition such that it separates pinned and not pinned 535 - /// fields. Afterwards it declares the struct and implement the `PinData` trait safely. 536 - #[doc(hidden)] 537 - #[macro_export] 538 - macro_rules! __pin_data { 539 - // Proc-macro entry point, this is supplied by the proc-macro pre-parsing. 540 - (parse_input: 541 - @args($($pinned_drop:ident)?), 542 - @sig( 543 - $(#[$($struct_attr:tt)*])* 544 - $vis:vis struct $name:ident 545 - $(where $($whr:tt)*)? 546 - ), 547 - @impl_generics($($impl_generics:tt)*), 548 - @ty_generics($($ty_generics:tt)*), 549 - @decl_generics($($decl_generics:tt)*), 550 - @body({ $($fields:tt)* }), 551 - ) => { 552 - // We now use token munching to iterate through all of the fields. While doing this we 553 - // identify fields marked with `#[pin]`, these fields are the 'pinned fields'. The user 554 - // wants these to be structurally pinned. The rest of the fields are the 555 - // 'not pinned fields'. Additionally we collect all fields, since we need them in the right 556 - // order to declare the struct. 557 - // 558 - // In this call we also put some explaining comments for the parameters. 559 - $crate::__pin_data!(find_pinned_fields: 560 - // Attributes on the struct itself, these will just be propagated to be put onto the 561 - // struct definition. 562 - @struct_attrs($(#[$($struct_attr)*])*), 563 - // The visibility of the struct. 564 - @vis($vis), 565 - // The name of the struct. 566 - @name($name), 567 - // The 'impl generics', the generics that will need to be specified on the struct inside 568 - // of an `impl<$ty_generics>` block. 569 - @impl_generics($($impl_generics)*), 570 - // The 'ty generics', the generics that will need to be specified on the impl blocks. 571 - @ty_generics($($ty_generics)*), 572 - // The 'decl generics', the generics that need to be specified on the struct 573 - // definition. 574 - @decl_generics($($decl_generics)*), 575 - // The where clause of any impl block and the declaration. 576 - @where($($($whr)*)?), 577 - // The remaining fields tokens that need to be processed. 578 - // We add a `,` at the end to ensure correct parsing. 579 - @fields_munch($($fields)* ,), 580 - // The pinned fields. 581 - @pinned(), 582 - // The not pinned fields. 583 - @not_pinned(), 584 - // All fields. 585 - @fields(), 586 - // The accumulator containing all attributes already parsed. 587 - @accum(), 588 - // Contains `yes` or `` to indicate if `#[pin]` was found on the current field. 589 - @is_pinned(), 590 - // The proc-macro argument, this should be `PinnedDrop` or ``. 591 - @pinned_drop($($pinned_drop)?), 592 - ); 593 - }; 594 - (find_pinned_fields: 595 - @struct_attrs($($struct_attrs:tt)*), 596 - @vis($vis:vis), 597 - @name($name:ident), 598 - @impl_generics($($impl_generics:tt)*), 599 - @ty_generics($($ty_generics:tt)*), 600 - @decl_generics($($decl_generics:tt)*), 601 - @where($($whr:tt)*), 602 - // We found a PhantomPinned field, this should generally be pinned! 603 - @fields_munch($field:ident : $($($(::)?core::)?marker::)?PhantomPinned, $($rest:tt)*), 604 - @pinned($($pinned:tt)*), 605 - @not_pinned($($not_pinned:tt)*), 606 - @fields($($fields:tt)*), 607 - @accum($($accum:tt)*), 608 - // This field is not pinned. 609 - @is_pinned(), 610 - @pinned_drop($($pinned_drop:ident)?), 611 - ) => { 612 - ::core::compile_error!(concat!( 613 - "The field `", 614 - stringify!($field), 615 - "` of type `PhantomPinned` only has an effect, if it has the `#[pin]` attribute.", 616 - )); 617 - $crate::__pin_data!(find_pinned_fields: 618 - @struct_attrs($($struct_attrs)*), 619 - @vis($vis), 620 - @name($name), 621 - @impl_generics($($impl_generics)*), 622 - @ty_generics($($ty_generics)*), 623 - @decl_generics($($decl_generics)*), 624 - @where($($whr)*), 625 - @fields_munch($($rest)*), 626 - @pinned($($pinned)* $($accum)* $field: ::core::marker::PhantomPinned,), 627 - @not_pinned($($not_pinned)*), 628 - @fields($($fields)* $($accum)* $field: ::core::marker::PhantomPinned,), 629 - @accum(), 630 - @is_pinned(), 631 - @pinned_drop($($pinned_drop)?), 632 - ); 633 - }; 634 - (find_pinned_fields: 635 - @struct_attrs($($struct_attrs:tt)*), 636 - @vis($vis:vis), 637 - @name($name:ident), 638 - @impl_generics($($impl_generics:tt)*), 639 - @ty_generics($($ty_generics:tt)*), 640 - @decl_generics($($decl_generics:tt)*), 641 - @where($($whr:tt)*), 642 - // We reached the field declaration. 643 - @fields_munch($field:ident : $type:ty, $($rest:tt)*), 644 - @pinned($($pinned:tt)*), 645 - @not_pinned($($not_pinned:tt)*), 646 - @fields($($fields:tt)*), 647 - @accum($($accum:tt)*), 648 - // This field is pinned. 649 - @is_pinned(yes), 650 - @pinned_drop($($pinned_drop:ident)?), 651 - ) => { 652 - $crate::__pin_data!(find_pinned_fields: 653 - @struct_attrs($($struct_attrs)*), 654 - @vis($vis), 655 - @name($name), 656 - @impl_generics($($impl_generics)*), 657 - @ty_generics($($ty_generics)*), 658 - @decl_generics($($decl_generics)*), 659 - @where($($whr)*), 660 - @fields_munch($($rest)*), 661 - @pinned($($pinned)* $($accum)* $field: $type,), 662 - @not_pinned($($not_pinned)*), 663 - @fields($($fields)* $($accum)* $field: $type,), 664 - @accum(), 665 - @is_pinned(), 666 - @pinned_drop($($pinned_drop)?), 667 - ); 668 - }; 669 - (find_pinned_fields: 670 - @struct_attrs($($struct_attrs:tt)*), 671 - @vis($vis:vis), 672 - @name($name:ident), 673 - @impl_generics($($impl_generics:tt)*), 674 - @ty_generics($($ty_generics:tt)*), 675 - @decl_generics($($decl_generics:tt)*), 676 - @where($($whr:tt)*), 677 - // We reached the field declaration. 678 - @fields_munch($field:ident : $type:ty, $($rest:tt)*), 679 - @pinned($($pinned:tt)*), 680 - @not_pinned($($not_pinned:tt)*), 681 - @fields($($fields:tt)*), 682 - @accum($($accum:tt)*), 683 - // This field is not pinned. 684 - @is_pinned(), 685 - @pinned_drop($($pinned_drop:ident)?), 686 - ) => { 687 - $crate::__pin_data!(find_pinned_fields: 688 - @struct_attrs($($struct_attrs)*), 689 - @vis($vis), 690 - @name($name), 691 - @impl_generics($($impl_generics)*), 692 - @ty_generics($($ty_generics)*), 693 - @decl_generics($($decl_generics)*), 694 - @where($($whr)*), 695 - @fields_munch($($rest)*), 696 - @pinned($($pinned)*), 697 - @not_pinned($($not_pinned)* $($accum)* $field: $type,), 698 - @fields($($fields)* $($accum)* $field: $type,), 699 - @accum(), 700 - @is_pinned(), 701 - @pinned_drop($($pinned_drop)?), 702 - ); 703 - }; 704 - (find_pinned_fields: 705 - @struct_attrs($($struct_attrs:tt)*), 706 - @vis($vis:vis), 707 - @name($name:ident), 708 - @impl_generics($($impl_generics:tt)*), 709 - @ty_generics($($ty_generics:tt)*), 710 - @decl_generics($($decl_generics:tt)*), 711 - @where($($whr:tt)*), 712 - // We found the `#[pin]` attr. 713 - @fields_munch(#[pin] $($rest:tt)*), 714 - @pinned($($pinned:tt)*), 715 - @not_pinned($($not_pinned:tt)*), 716 - @fields($($fields:tt)*), 717 - @accum($($accum:tt)*), 718 - @is_pinned($($is_pinned:ident)?), 719 - @pinned_drop($($pinned_drop:ident)?), 720 - ) => { 721 - $crate::__pin_data!(find_pinned_fields: 722 - @struct_attrs($($struct_attrs)*), 723 - @vis($vis), 724 - @name($name), 725 - @impl_generics($($impl_generics)*), 726 - @ty_generics($($ty_generics)*), 727 - @decl_generics($($decl_generics)*), 728 - @where($($whr)*), 729 - @fields_munch($($rest)*), 730 - // We do not include `#[pin]` in the list of attributes, since it is not actually an 731 - // attribute that is defined somewhere. 732 - @pinned($($pinned)*), 733 - @not_pinned($($not_pinned)*), 734 - @fields($($fields)*), 735 - @accum($($accum)*), 736 - // Set this to `yes`. 737 - @is_pinned(yes), 738 - @pinned_drop($($pinned_drop)?), 739 - ); 740 - }; 741 - (find_pinned_fields: 742 - @struct_attrs($($struct_attrs:tt)*), 743 - @vis($vis:vis), 744 - @name($name:ident), 745 - @impl_generics($($impl_generics:tt)*), 746 - @ty_generics($($ty_generics:tt)*), 747 - @decl_generics($($decl_generics:tt)*), 748 - @where($($whr:tt)*), 749 - // We reached the field declaration with visibility, for simplicity we only munch the 750 - // visibility and put it into `$accum`. 751 - @fields_munch($fvis:vis $field:ident $($rest:tt)*), 752 - @pinned($($pinned:tt)*), 753 - @not_pinned($($not_pinned:tt)*), 754 - @fields($($fields:tt)*), 755 - @accum($($accum:tt)*), 756 - @is_pinned($($is_pinned:ident)?), 757 - @pinned_drop($($pinned_drop:ident)?), 758 - ) => { 759 - $crate::__pin_data!(find_pinned_fields: 760 - @struct_attrs($($struct_attrs)*), 761 - @vis($vis), 762 - @name($name), 763 - @impl_generics($($impl_generics)*), 764 - @ty_generics($($ty_generics)*), 765 - @decl_generics($($decl_generics)*), 766 - @where($($whr)*), 767 - @fields_munch($field $($rest)*), 768 - @pinned($($pinned)*), 769 - @not_pinned($($not_pinned)*), 770 - @fields($($fields)*), 771 - @accum($($accum)* $fvis), 772 - @is_pinned($($is_pinned)?), 773 - @pinned_drop($($pinned_drop)?), 774 - ); 775 - }; 776 - (find_pinned_fields: 777 - @struct_attrs($($struct_attrs:tt)*), 778 - @vis($vis:vis), 779 - @name($name:ident), 780 - @impl_generics($($impl_generics:tt)*), 781 - @ty_generics($($ty_generics:tt)*), 782 - @decl_generics($($decl_generics:tt)*), 783 - @where($($whr:tt)*), 784 - // Some other attribute, just put it into `$accum`. 785 - @fields_munch(#[$($attr:tt)*] $($rest:tt)*), 786 - @pinned($($pinned:tt)*), 787 - @not_pinned($($not_pinned:tt)*), 788 - @fields($($fields:tt)*), 789 - @accum($($accum:tt)*), 790 - @is_pinned($($is_pinned:ident)?), 791 - @pinned_drop($($pinned_drop:ident)?), 792 - ) => { 793 - $crate::__pin_data!(find_pinned_fields: 794 - @struct_attrs($($struct_attrs)*), 795 - @vis($vis), 796 - @name($name), 797 - @impl_generics($($impl_generics)*), 798 - @ty_generics($($ty_generics)*), 799 - @decl_generics($($decl_generics)*), 800 - @where($($whr)*), 801 - @fields_munch($($rest)*), 802 - @pinned($($pinned)*), 803 - @not_pinned($($not_pinned)*), 804 - @fields($($fields)*), 805 - @accum($($accum)* #[$($attr)*]), 806 - @is_pinned($($is_pinned)?), 807 - @pinned_drop($($pinned_drop)?), 808 - ); 809 - }; 810 - (find_pinned_fields: 811 - @struct_attrs($($struct_attrs:tt)*), 812 - @vis($vis:vis), 813 - @name($name:ident), 814 - @impl_generics($($impl_generics:tt)*), 815 - @ty_generics($($ty_generics:tt)*), 816 - @decl_generics($($decl_generics:tt)*), 817 - @where($($whr:tt)*), 818 - // We reached the end of the fields, plus an optional additional comma, since we added one 819 - // before and the user is also allowed to put a trailing comma. 820 - @fields_munch($(,)?), 821 - @pinned($($pinned:tt)*), 822 - @not_pinned($($not_pinned:tt)*), 823 - @fields($($fields:tt)*), 824 - @accum(), 825 - @is_pinned(), 826 - @pinned_drop($($pinned_drop:ident)?), 827 - ) => { 828 - // Declare the struct with all fields in the correct order. 829 - $($struct_attrs)* 830 - $vis struct $name <$($decl_generics)*> 831 - where $($whr)* 832 - { 833 - $($fields)* 834 - } 835 - 836 - $crate::__pin_data!(make_pin_projections: 837 - @vis($vis), 838 - @name($name), 839 - @impl_generics($($impl_generics)*), 840 - @ty_generics($($ty_generics)*), 841 - @decl_generics($($decl_generics)*), 842 - @where($($whr)*), 843 - @pinned($($pinned)*), 844 - @not_pinned($($not_pinned)*), 845 - ); 846 - 847 - // We put the rest into this const item, because it then will not be accessible to anything 848 - // outside. 849 - const _: () = { 850 - // We declare this struct which will host all of the projection function for our type. 851 - // it will be invariant over all generic parameters which are inherited from the 852 - // struct. 853 - $vis struct __ThePinData<$($impl_generics)*> 854 - where $($whr)* 855 - { 856 - __phantom: ::core::marker::PhantomData< 857 - fn($name<$($ty_generics)*>) -> $name<$($ty_generics)*> 858 - >, 859 - } 860 - 861 - impl<$($impl_generics)*> ::core::clone::Clone for __ThePinData<$($ty_generics)*> 862 - where $($whr)* 863 - { 864 - fn clone(&self) -> Self { *self } 865 - } 866 - 867 - impl<$($impl_generics)*> ::core::marker::Copy for __ThePinData<$($ty_generics)*> 868 - where $($whr)* 869 - {} 870 - 871 - // Make all projection functions. 872 - $crate::__pin_data!(make_pin_data: 873 - @pin_data(__ThePinData), 874 - @impl_generics($($impl_generics)*), 875 - @ty_generics($($ty_generics)*), 876 - @where($($whr)*), 877 - @pinned($($pinned)*), 878 - @not_pinned($($not_pinned)*), 879 - ); 880 - 881 - // SAFETY: We have added the correct projection functions above to `__ThePinData` and 882 - // we also use the least restrictive generics possible. 883 - unsafe impl<$($impl_generics)*> 884 - $crate::__internal::HasPinData for $name<$($ty_generics)*> 885 - where $($whr)* 886 - { 887 - type PinData = __ThePinData<$($ty_generics)*>; 888 - 889 - unsafe fn __pin_data() -> Self::PinData { 890 - __ThePinData { __phantom: ::core::marker::PhantomData } 891 - } 892 - } 893 - 894 - // SAFETY: TODO. 895 - unsafe impl<$($impl_generics)*> 896 - $crate::__internal::PinData for __ThePinData<$($ty_generics)*> 897 - where $($whr)* 898 - { 899 - type Datee = $name<$($ty_generics)*>; 900 - } 901 - 902 - // This struct will be used for the unpin analysis. Since only structurally pinned 903 - // fields are relevant whether the struct should implement `Unpin`. 904 - #[allow(dead_code)] 905 - struct __Unpin <'__pin, $($impl_generics)*> 906 - where $($whr)* 907 - { 908 - __phantom_pin: ::core::marker::PhantomData<fn(&'__pin ()) -> &'__pin ()>, 909 - __phantom: ::core::marker::PhantomData< 910 - fn($name<$($ty_generics)*>) -> $name<$($ty_generics)*> 911 - >, 912 - // Only the pinned fields. 913 - $($pinned)* 914 - } 915 - 916 - #[doc(hidden)] 917 - impl<'__pin, $($impl_generics)*> ::core::marker::Unpin for $name<$($ty_generics)*> 918 - where 919 - __Unpin<'__pin, $($ty_generics)*>: ::core::marker::Unpin, 920 - $($whr)* 921 - {} 922 - 923 - // We need to disallow normal `Drop` implementation, the exact behavior depends on 924 - // whether `PinnedDrop` was specified as the parameter. 925 - $crate::__pin_data!(drop_prevention: 926 - @name($name), 927 - @impl_generics($($impl_generics)*), 928 - @ty_generics($($ty_generics)*), 929 - @where($($whr)*), 930 - @pinned_drop($($pinned_drop)?), 931 - ); 932 - }; 933 - }; 934 - // When no `PinnedDrop` was specified, then we have to prevent implementing drop. 935 - (drop_prevention: 936 - @name($name:ident), 937 - @impl_generics($($impl_generics:tt)*), 938 - @ty_generics($($ty_generics:tt)*), 939 - @where($($whr:tt)*), 940 - @pinned_drop(), 941 - ) => { 942 - // We prevent this by creating a trait that will be implemented for all types implementing 943 - // `Drop`. Additionally we will implement this trait for the struct leading to a conflict, 944 - // if it also implements `Drop` 945 - trait MustNotImplDrop {} 946 - #[expect(drop_bounds)] 947 - impl<T: ::core::ops::Drop> MustNotImplDrop for T {} 948 - impl<$($impl_generics)*> MustNotImplDrop for $name<$($ty_generics)*> 949 - where $($whr)* {} 950 - // We also take care to prevent users from writing a useless `PinnedDrop` implementation. 951 - // They might implement `PinnedDrop` correctly for the struct, but forget to give 952 - // `PinnedDrop` as the parameter to `#[pin_data]`. 953 - #[expect(non_camel_case_types)] 954 - trait UselessPinnedDropImpl_you_need_to_specify_PinnedDrop {} 955 - impl<T: $crate::PinnedDrop> 956 - UselessPinnedDropImpl_you_need_to_specify_PinnedDrop for T {} 957 - impl<$($impl_generics)*> 958 - UselessPinnedDropImpl_you_need_to_specify_PinnedDrop for $name<$($ty_generics)*> 959 - where $($whr)* {} 960 - }; 961 - // When `PinnedDrop` was specified we just implement `Drop` and delegate. 962 - (drop_prevention: 963 - @name($name:ident), 964 - @impl_generics($($impl_generics:tt)*), 965 - @ty_generics($($ty_generics:tt)*), 966 - @where($($whr:tt)*), 967 - @pinned_drop(PinnedDrop), 968 - ) => { 969 - impl<$($impl_generics)*> ::core::ops::Drop for $name<$($ty_generics)*> 970 - where $($whr)* 971 - { 972 - fn drop(&mut self) { 973 - // SAFETY: Since this is a destructor, `self` will not move after this function 974 - // terminates, since it is inaccessible. 975 - let pinned = unsafe { ::core::pin::Pin::new_unchecked(self) }; 976 - // SAFETY: Since this is a drop function, we can create this token to call the 977 - // pinned destructor of this type. 978 - let token = unsafe { $crate::__internal::OnlyCallFromDrop::new() }; 979 - $crate::PinnedDrop::drop(pinned, token); 980 - } 981 - } 982 - }; 983 - // If some other parameter was specified, we emit a readable error. 984 - (drop_prevention: 985 - @name($name:ident), 986 - @impl_generics($($impl_generics:tt)*), 987 - @ty_generics($($ty_generics:tt)*), 988 - @where($($whr:tt)*), 989 - @pinned_drop($($rest:tt)*), 990 - ) => { 991 - compile_error!( 992 - "Wrong parameters to `#[pin_data]`, expected nothing or `PinnedDrop`, got '{}'.", 993 - stringify!($($rest)*), 994 - ); 995 - }; 996 - (make_pin_projections: 997 - @vis($vis:vis), 998 - @name($name:ident), 999 - @impl_generics($($impl_generics:tt)*), 1000 - @ty_generics($($ty_generics:tt)*), 1001 - @decl_generics($($decl_generics:tt)*), 1002 - @where($($whr:tt)*), 1003 - @pinned($($(#[$($p_attr:tt)*])* $pvis:vis $p_field:ident : $p_type:ty),* $(,)?), 1004 - @not_pinned($($(#[$($attr:tt)*])* $fvis:vis $field:ident : $type:ty),* $(,)?), 1005 - ) => { 1006 - $crate::macros::paste! { 1007 - #[doc(hidden)] 1008 - $vis struct [< $name Projection >] <'__pin, $($decl_generics)*> { 1009 - $($(#[$($p_attr)*])* $pvis $p_field : ::core::pin::Pin<&'__pin mut $p_type>,)* 1010 - $($(#[$($attr)*])* $fvis $field : &'__pin mut $type,)* 1011 - ___pin_phantom_data: ::core::marker::PhantomData<&'__pin mut ()>, 1012 - } 1013 - 1014 - impl<$($impl_generics)*> $name<$($ty_generics)*> 1015 - where $($whr)* 1016 - { 1017 - /// Pin-projects all fields of `Self`. 1018 - /// 1019 - /// These fields are structurally pinned: 1020 - $(#[doc = ::core::concat!(" - `", ::core::stringify!($p_field), "`")])* 1021 - /// 1022 - /// These fields are **not** structurally pinned: 1023 - $(#[doc = ::core::concat!(" - `", ::core::stringify!($field), "`")])* 1024 - #[inline] 1025 - $vis fn project<'__pin>( 1026 - self: ::core::pin::Pin<&'__pin mut Self>, 1027 - ) -> [< $name Projection >] <'__pin, $($ty_generics)*> { 1028 - // SAFETY: we only give access to `&mut` for fields not structurally pinned. 1029 - let this = unsafe { ::core::pin::Pin::get_unchecked_mut(self) }; 1030 - [< $name Projection >] { 1031 - $( 1032 - // SAFETY: `$p_field` is structurally pinned. 1033 - $(#[$($p_attr)*])* 1034 - $p_field : unsafe { ::core::pin::Pin::new_unchecked(&mut this.$p_field) }, 1035 - )* 1036 - $( 1037 - $(#[$($attr)*])* 1038 - $field : &mut this.$field, 1039 - )* 1040 - ___pin_phantom_data: ::core::marker::PhantomData, 1041 - } 1042 - } 1043 - } 1044 - } 1045 - }; 1046 - (make_pin_data: 1047 - @pin_data($pin_data:ident), 1048 - @impl_generics($($impl_generics:tt)*), 1049 - @ty_generics($($ty_generics:tt)*), 1050 - @where($($whr:tt)*), 1051 - @pinned($($(#[$($p_attr:tt)*])* $pvis:vis $p_field:ident : $p_type:ty),* $(,)?), 1052 - @not_pinned($($(#[$($attr:tt)*])* $fvis:vis $field:ident : $type:ty),* $(,)?), 1053 - ) => { 1054 - $crate::macros::paste! { 1055 - // For every field, we create a projection function according to its projection type. If a 1056 - // field is structurally pinned, then it must be initialized via `PinInit`, if it is not 1057 - // structurally pinned, then it can be initialized via `Init`. 1058 - // 1059 - // The functions are `unsafe` to prevent accidentally calling them. 1060 - #[allow(dead_code)] 1061 - #[expect(clippy::missing_safety_doc)] 1062 - impl<$($impl_generics)*> $pin_data<$($ty_generics)*> 1063 - where $($whr)* 1064 - { 1065 - $( 1066 - $(#[$($p_attr)*])* 1067 - $pvis unsafe fn $p_field<E>( 1068 - self, 1069 - slot: *mut $p_type, 1070 - init: impl $crate::PinInit<$p_type, E>, 1071 - ) -> ::core::result::Result<(), E> { 1072 - // SAFETY: TODO. 1073 - unsafe { $crate::PinInit::__pinned_init(init, slot) } 1074 - } 1075 - 1076 - $(#[$($p_attr)*])* 1077 - $pvis unsafe fn [<__project_ $p_field>]<'__slot>( 1078 - self, 1079 - slot: &'__slot mut $p_type, 1080 - ) -> ::core::pin::Pin<&'__slot mut $p_type> { 1081 - ::core::pin::Pin::new_unchecked(slot) 1082 - } 1083 - )* 1084 - $( 1085 - $(#[$($attr)*])* 1086 - $fvis unsafe fn $field<E>( 1087 - self, 1088 - slot: *mut $type, 1089 - init: impl $crate::Init<$type, E>, 1090 - ) -> ::core::result::Result<(), E> { 1091 - // SAFETY: TODO. 1092 - unsafe { $crate::Init::__init(init, slot) } 1093 - } 1094 - 1095 - $(#[$($attr)*])* 1096 - $fvis unsafe fn [<__project_ $field>]<'__slot>( 1097 - self, 1098 - slot: &'__slot mut $type, 1099 - ) -> &'__slot mut $type { 1100 - slot 1101 - } 1102 - )* 1103 - } 1104 - } 1105 - }; 1106 - } 1107 - 1108 - /// The internal init macro. Do not call manually! 1109 - /// 1110 - /// This is called by the `{try_}{pin_}init!` macros with various inputs. 1111 - /// 1112 - /// This macro has multiple internal call configurations, these are always the very first ident: 1113 - /// - nothing: this is the base case and called by the `{try_}{pin_}init!` macros. 1114 - /// - `with_update_parsed`: when the `..Zeroable::init_zeroed()` syntax has been handled. 1115 - /// - `init_slot`: recursively creates the code that initializes all fields in `slot`. 1116 - /// - `make_initializer`: recursively create the struct initializer that guarantees that every 1117 - /// field has been initialized exactly once. 1118 - #[doc(hidden)] 1119 - #[macro_export] 1120 - macro_rules! __init_internal { 1121 - ( 1122 - @this($($this:ident)?), 1123 - @typ($t:path), 1124 - @fields($($fields:tt)*), 1125 - @error($err:ty), 1126 - // Either `PinData` or `InitData`, `$use_data` should only be present in the `PinData` 1127 - // case. 1128 - @data($data:ident, $($use_data:ident)?), 1129 - // `HasPinData` or `HasInitData`. 1130 - @has_data($has_data:ident, $get_data:ident), 1131 - // `pin_init_from_closure` or `init_from_closure`. 1132 - @construct_closure($construct_closure:ident), 1133 - @munch_fields(), 1134 - ) => { 1135 - $crate::__init_internal!(with_update_parsed: 1136 - @this($($this)?), 1137 - @typ($t), 1138 - @fields($($fields)*), 1139 - @error($err), 1140 - @data($data, $($use_data)?), 1141 - @has_data($has_data, $get_data), 1142 - @construct_closure($construct_closure), 1143 - @init_zeroed(), // Nothing means default behavior. 1144 - ) 1145 - }; 1146 - ( 1147 - @this($($this:ident)?), 1148 - @typ($t:path), 1149 - @fields($($fields:tt)*), 1150 - @error($err:ty), 1151 - // Either `PinData` or `InitData`, `$use_data` should only be present in the `PinData` 1152 - // case. 1153 - @data($data:ident, $($use_data:ident)?), 1154 - // `HasPinData` or `HasInitData`. 1155 - @has_data($has_data:ident, $get_data:ident), 1156 - // `pin_init_from_closure` or `init_from_closure`. 1157 - @construct_closure($construct_closure:ident), 1158 - @munch_fields(..Zeroable::init_zeroed()), 1159 - ) => { 1160 - $crate::__init_internal!(with_update_parsed: 1161 - @this($($this)?), 1162 - @typ($t), 1163 - @fields($($fields)*), 1164 - @error($err), 1165 - @data($data, $($use_data)?), 1166 - @has_data($has_data, $get_data), 1167 - @construct_closure($construct_closure), 1168 - @init_zeroed(()), // `()` means zero all fields not mentioned. 1169 - ) 1170 - }; 1171 - ( 1172 - @this($($this:ident)?), 1173 - @typ($t:path), 1174 - @fields($($fields:tt)*), 1175 - @error($err:ty), 1176 - // Either `PinData` or `InitData`, `$use_data` should only be present in the `PinData` 1177 - // case. 1178 - @data($data:ident, $($use_data:ident)?), 1179 - // `HasPinData` or `HasInitData`. 1180 - @has_data($has_data:ident, $get_data:ident), 1181 - // `pin_init_from_closure` or `init_from_closure`. 1182 - @construct_closure($construct_closure:ident), 1183 - @munch_fields($ignore:tt $($rest:tt)*), 1184 - ) => { 1185 - $crate::__init_internal!( 1186 - @this($($this)?), 1187 - @typ($t), 1188 - @fields($($fields)*), 1189 - @error($err), 1190 - @data($data, $($use_data)?), 1191 - @has_data($has_data, $get_data), 1192 - @construct_closure($construct_closure), 1193 - @munch_fields($($rest)*), 1194 - ) 1195 - }; 1196 - (with_update_parsed: 1197 - @this($($this:ident)?), 1198 - @typ($t:path), 1199 - @fields($($fields:tt)*), 1200 - @error($err:ty), 1201 - // Either `PinData` or `InitData`, `$use_data` should only be present in the `PinData` 1202 - // case. 1203 - @data($data:ident, $($use_data:ident)?), 1204 - // `HasPinData` or `HasInitData`. 1205 - @has_data($has_data:ident, $get_data:ident), 1206 - // `pin_init_from_closure` or `init_from_closure`. 1207 - @construct_closure($construct_closure:ident), 1208 - @init_zeroed($($init_zeroed:expr)?), 1209 - ) => {{ 1210 - // We do not want to allow arbitrary returns, so we declare this type as the `Ok` return 1211 - // type and shadow it later when we insert the arbitrary user code. That way there will be 1212 - // no possibility of returning without `unsafe`. 1213 - struct __InitOk; 1214 - // Get the data about fields from the supplied type. 1215 - // 1216 - // SAFETY: TODO. 1217 - let data = unsafe { 1218 - use $crate::__internal::$has_data; 1219 - // Here we abuse `paste!` to retokenize `$t`. Declarative macros have some internal 1220 - // information that is associated to already parsed fragments, so a path fragment 1221 - // cannot be used in this position. Doing the retokenization results in valid rust 1222 - // code. 1223 - $crate::macros::paste!($t::$get_data()) 1224 - }; 1225 - // Ensure that `data` really is of type `$data` and help with type inference: 1226 - let init = $crate::__internal::$data::make_closure::<_, __InitOk, $err>( 1227 - data, 1228 - move |slot| { 1229 - { 1230 - // Shadow the structure so it cannot be used to return early. 1231 - struct __InitOk; 1232 - // If `$init_zeroed` is present we should zero the slot now and not emit an 1233 - // error when fields are missing (since they will be zeroed). We also have to 1234 - // check that the type actually implements `Zeroable`. 1235 - $({ 1236 - fn assert_zeroable<T: $crate::Zeroable>(_: *mut T) {} 1237 - // Ensure that the struct is indeed `Zeroable`. 1238 - assert_zeroable(slot); 1239 - // SAFETY: The type implements `Zeroable` by the check above. 1240 - unsafe { ::core::ptr::write_bytes(slot, 0, 1) }; 1241 - $init_zeroed // This will be `()` if set. 1242 - })? 1243 - // Create the `this` so it can be referenced by the user inside of the 1244 - // expressions creating the individual fields. 1245 - $(let $this = unsafe { ::core::ptr::NonNull::new_unchecked(slot) };)? 1246 - // Initialize every field. 1247 - $crate::__init_internal!(init_slot($($use_data)?): 1248 - @data(data), 1249 - @slot(slot), 1250 - @guards(), 1251 - @munch_fields($($fields)*,), 1252 - ); 1253 - // We use unreachable code to ensure that all fields have been mentioned exactly 1254 - // once, this struct initializer will still be type-checked and complain with a 1255 - // very natural error message if a field is forgotten/mentioned more than once. 1256 - #[allow(unreachable_code, clippy::diverging_sub_expression)] 1257 - let _ = || { 1258 - $crate::__init_internal!(make_initializer: 1259 - @slot(slot), 1260 - @type_name($t), 1261 - @munch_fields($($fields)*,), 1262 - @acc(), 1263 - ); 1264 - }; 1265 - } 1266 - Ok(__InitOk) 1267 - } 1268 - ); 1269 - let init = move |slot| -> ::core::result::Result<(), $err> { 1270 - init(slot).map(|__InitOk| ()) 1271 - }; 1272 - // SAFETY: TODO. 1273 - let init = unsafe { $crate::$construct_closure::<_, $err>(init) }; 1274 - init 1275 - }}; 1276 - (init_slot($($use_data:ident)?): 1277 - @data($data:ident), 1278 - @slot($slot:ident), 1279 - @guards($($guards:ident,)*), 1280 - @munch_fields($(..Zeroable::init_zeroed())? $(,)?), 1281 - ) => { 1282 - // Endpoint of munching, no fields are left. If execution reaches this point, all fields 1283 - // have been initialized. Therefore we can now dismiss the guards by forgetting them. 1284 - $(::core::mem::forget($guards);)* 1285 - }; 1286 - (init_slot($($use_data:ident)?): 1287 - @data($data:ident), 1288 - @slot($slot:ident), 1289 - @guards($($guards:ident,)*), 1290 - // arbitrary code block 1291 - @munch_fields(_: { $($code:tt)* }, $($rest:tt)*), 1292 - ) => { 1293 - { $($code)* } 1294 - $crate::__init_internal!(init_slot($($use_data)?): 1295 - @data($data), 1296 - @slot($slot), 1297 - @guards($($guards,)*), 1298 - @munch_fields($($rest)*), 1299 - ); 1300 - }; 1301 - (init_slot($use_data:ident): // `use_data` is present, so we use the `data` to init fields. 1302 - @data($data:ident), 1303 - @slot($slot:ident), 1304 - @guards($($guards:ident,)*), 1305 - // In-place initialization syntax. 1306 - @munch_fields($field:ident <- $val:expr, $($rest:tt)*), 1307 - ) => { 1308 - let init = $val; 1309 - // Call the initializer. 1310 - // 1311 - // SAFETY: `slot` is valid, because we are inside of an initializer closure, we 1312 - // return when an error/panic occurs. 1313 - // We also use the `data` to require the correct trait (`Init` or `PinInit`) for `$field`. 1314 - unsafe { $data.$field(::core::ptr::addr_of_mut!((*$slot).$field), init)? }; 1315 - // SAFETY: 1316 - // - the project function does the correct field projection, 1317 - // - the field has been initialized, 1318 - // - the reference is only valid until the end of the initializer. 1319 - #[allow(unused_variables)] 1320 - let $field = $crate::macros::paste!(unsafe { $data.[< __project_ $field >](&mut (*$slot).$field) }); 1321 - 1322 - // Create the drop guard: 1323 - // 1324 - // We rely on macro hygiene to make it impossible for users to access this local variable. 1325 - // We use `paste!` to create new hygiene for `$field`. 1326 - $crate::macros::paste! { 1327 - // SAFETY: We forget the guard later when initialization has succeeded. 1328 - let [< __ $field _guard >] = unsafe { 1329 - $crate::__internal::DropGuard::new(::core::ptr::addr_of_mut!((*$slot).$field)) 1330 - }; 1331 - 1332 - $crate::__init_internal!(init_slot($use_data): 1333 - @data($data), 1334 - @slot($slot), 1335 - @guards([< __ $field _guard >], $($guards,)*), 1336 - @munch_fields($($rest)*), 1337 - ); 1338 - } 1339 - }; 1340 - (init_slot(): // No `use_data`, so we use `Init::__init` directly. 1341 - @data($data:ident), 1342 - @slot($slot:ident), 1343 - @guards($($guards:ident,)*), 1344 - // In-place initialization syntax. 1345 - @munch_fields($field:ident <- $val:expr, $($rest:tt)*), 1346 - ) => { 1347 - let init = $val; 1348 - // Call the initializer. 1349 - // 1350 - // SAFETY: `slot` is valid, because we are inside of an initializer closure, we 1351 - // return when an error/panic occurs. 1352 - unsafe { $crate::Init::__init(init, ::core::ptr::addr_of_mut!((*$slot).$field))? }; 1353 - 1354 - // SAFETY: 1355 - // - the field is not structurally pinned, since the line above must compile, 1356 - // - the field has been initialized, 1357 - // - the reference is only valid until the end of the initializer. 1358 - #[allow(unused_variables)] 1359 - let $field = unsafe { &mut (*$slot).$field }; 1360 - 1361 - // Create the drop guard: 1362 - // 1363 - // We rely on macro hygiene to make it impossible for users to access this local variable. 1364 - // We use `paste!` to create new hygiene for `$field`. 1365 - $crate::macros::paste! { 1366 - // SAFETY: We forget the guard later when initialization has succeeded. 1367 - let [< __ $field _guard >] = unsafe { 1368 - $crate::__internal::DropGuard::new(::core::ptr::addr_of_mut!((*$slot).$field)) 1369 - }; 1370 - 1371 - $crate::__init_internal!(init_slot(): 1372 - @data($data), 1373 - @slot($slot), 1374 - @guards([< __ $field _guard >], $($guards,)*), 1375 - @munch_fields($($rest)*), 1376 - ); 1377 - } 1378 - }; 1379 - (init_slot(): // No `use_data`, so all fields are not structurally pinned 1380 - @data($data:ident), 1381 - @slot($slot:ident), 1382 - @guards($($guards:ident,)*), 1383 - // Init by-value. 1384 - @munch_fields($field:ident $(: $val:expr)?, $($rest:tt)*), 1385 - ) => { 1386 - { 1387 - $(let $field = $val;)? 1388 - // Initialize the field. 1389 - // 1390 - // SAFETY: The memory at `slot` is uninitialized. 1391 - unsafe { ::core::ptr::write(::core::ptr::addr_of_mut!((*$slot).$field), $field) }; 1392 - } 1393 - 1394 - #[allow(unused_variables)] 1395 - // SAFETY: 1396 - // - the field is not structurally pinned, since no `use_data` was required to create this 1397 - // initializer, 1398 - // - the field has been initialized, 1399 - // - the reference is only valid until the end of the initializer. 1400 - let $field = unsafe { &mut (*$slot).$field }; 1401 - 1402 - // Create the drop guard: 1403 - // 1404 - // We rely on macro hygiene to make it impossible for users to access this local variable. 1405 - // We use `paste!` to create new hygiene for `$field`. 1406 - $crate::macros::paste! { 1407 - // SAFETY: We forget the guard later when initialization has succeeded. 1408 - let [< __ $field _guard >] = unsafe { 1409 - $crate::__internal::DropGuard::new(::core::ptr::addr_of_mut!((*$slot).$field)) 1410 - }; 1411 - 1412 - $crate::__init_internal!(init_slot(): 1413 - @data($data), 1414 - @slot($slot), 1415 - @guards([< __ $field _guard >], $($guards,)*), 1416 - @munch_fields($($rest)*), 1417 - ); 1418 - } 1419 - }; 1420 - (init_slot($use_data:ident): 1421 - @data($data:ident), 1422 - @slot($slot:ident), 1423 - @guards($($guards:ident,)*), 1424 - // Init by-value. 1425 - @munch_fields($field:ident $(: $val:expr)?, $($rest:tt)*), 1426 - ) => { 1427 - { 1428 - $(let $field = $val;)? 1429 - // Initialize the field. 1430 - // 1431 - // SAFETY: The memory at `slot` is uninitialized. 1432 - unsafe { ::core::ptr::write(::core::ptr::addr_of_mut!((*$slot).$field), $field) }; 1433 - } 1434 - // SAFETY: 1435 - // - the project function does the correct field projection, 1436 - // - the field has been initialized, 1437 - // - the reference is only valid until the end of the initializer. 1438 - #[allow(unused_variables)] 1439 - let $field = $crate::macros::paste!(unsafe { $data.[< __project_ $field >](&mut (*$slot).$field) }); 1440 - 1441 - // Create the drop guard: 1442 - // 1443 - // We rely on macro hygiene to make it impossible for users to access this local variable. 1444 - // We use `paste!` to create new hygiene for `$field`. 1445 - $crate::macros::paste! { 1446 - // SAFETY: We forget the guard later when initialization has succeeded. 1447 - let [< __ $field _guard >] = unsafe { 1448 - $crate::__internal::DropGuard::new(::core::ptr::addr_of_mut!((*$slot).$field)) 1449 - }; 1450 - 1451 - $crate::__init_internal!(init_slot($use_data): 1452 - @data($data), 1453 - @slot($slot), 1454 - @guards([< __ $field _guard >], $($guards,)*), 1455 - @munch_fields($($rest)*), 1456 - ); 1457 - } 1458 - }; 1459 - (make_initializer: 1460 - @slot($slot:ident), 1461 - @type_name($t:path), 1462 - @munch_fields(_: { $($code:tt)* }, $($rest:tt)*), 1463 - @acc($($acc:tt)*), 1464 - ) => { 1465 - // code blocks are ignored for the initializer check 1466 - $crate::__init_internal!(make_initializer: 1467 - @slot($slot), 1468 - @type_name($t), 1469 - @munch_fields($($rest)*), 1470 - @acc($($acc)*), 1471 - ); 1472 - }; 1473 - (make_initializer: 1474 - @slot($slot:ident), 1475 - @type_name($t:path), 1476 - @munch_fields(..Zeroable::init_zeroed() $(,)?), 1477 - @acc($($acc:tt)*), 1478 - ) => { 1479 - // Endpoint, nothing more to munch, create the initializer. Since the users specified 1480 - // `..Zeroable::init_zeroed()`, the slot will already have been zeroed and all field that have 1481 - // not been overwritten are thus zero and initialized. We still check that all fields are 1482 - // actually accessible by using the struct update syntax ourselves. 1483 - // We are inside of a closure that is never executed and thus we can abuse `slot` to 1484 - // get the correct type inference here: 1485 - #[allow(unused_assignments)] 1486 - unsafe { 1487 - let mut zeroed = ::core::mem::zeroed(); 1488 - // We have to use type inference here to make zeroed have the correct type. This does 1489 - // not get executed, so it has no effect. 1490 - ::core::ptr::write($slot, zeroed); 1491 - zeroed = ::core::mem::zeroed(); 1492 - // Here we abuse `paste!` to retokenize `$t`. Declarative macros have some internal 1493 - // information that is associated to already parsed fragments, so a path fragment 1494 - // cannot be used in this position. Doing the retokenization results in valid rust 1495 - // code. 1496 - $crate::macros::paste!( 1497 - ::core::ptr::write($slot, $t { 1498 - $($acc)* 1499 - ..zeroed 1500 - }); 1501 - ); 1502 - } 1503 - }; 1504 - (make_initializer: 1505 - @slot($slot:ident), 1506 - @type_name($t:path), 1507 - @munch_fields($(,)?), 1508 - @acc($($acc:tt)*), 1509 - ) => { 1510 - // Endpoint, nothing more to munch, create the initializer. 1511 - // Since we are in the closure that is never called, this will never get executed. 1512 - // We abuse `slot` to get the correct type inference here: 1513 - // 1514 - // SAFETY: TODO. 1515 - unsafe { 1516 - // Here we abuse `paste!` to retokenize `$t`. Declarative macros have some internal 1517 - // information that is associated to already parsed fragments, so a path fragment 1518 - // cannot be used in this position. Doing the retokenization results in valid rust 1519 - // code. 1520 - $crate::macros::paste!( 1521 - ::core::ptr::write($slot, $t { 1522 - $($acc)* 1523 - }); 1524 - ); 1525 - } 1526 - }; 1527 - (make_initializer: 1528 - @slot($slot:ident), 1529 - @type_name($t:path), 1530 - @munch_fields($field:ident <- $val:expr, $($rest:tt)*), 1531 - @acc($($acc:tt)*), 1532 - ) => { 1533 - $crate::__init_internal!(make_initializer: 1534 - @slot($slot), 1535 - @type_name($t), 1536 - @munch_fields($($rest)*), 1537 - @acc($($acc)* $field: ::core::panic!(),), 1538 - ); 1539 - }; 1540 - (make_initializer: 1541 - @slot($slot:ident), 1542 - @type_name($t:path), 1543 - @munch_fields($field:ident $(: $val:expr)?, $($rest:tt)*), 1544 - @acc($($acc:tt)*), 1545 - ) => { 1546 - $crate::__init_internal!(make_initializer: 1547 - @slot($slot), 1548 - @type_name($t), 1549 - @munch_fields($($rest)*), 1550 - @acc($($acc)* $field: ::core::panic!(),), 1551 - ); 1552 - }; 1553 - } 1554 - 1555 - #[doc(hidden)] 1556 - #[macro_export] 1557 - macro_rules! __derive_zeroable { 1558 - (parse_input: 1559 - @sig( 1560 - $(#[$($struct_attr:tt)*])* 1561 - $vis:vis struct $name:ident 1562 - $(where $($whr:tt)*)? 1563 - ), 1564 - @impl_generics($($impl_generics:tt)*), 1565 - @ty_generics($($ty_generics:tt)*), 1566 - @body({ 1567 - $( 1568 - $(#[$($field_attr:tt)*])* 1569 - $field_vis:vis $field:ident : $field_ty:ty 1570 - ),* $(,)? 1571 - }), 1572 - ) => { 1573 - // SAFETY: Every field type implements `Zeroable` and padding bytes may be zero. 1574 - #[automatically_derived] 1575 - unsafe impl<$($impl_generics)*> $crate::Zeroable for $name<$($ty_generics)*> 1576 - where 1577 - $($($whr)*)? 1578 - {} 1579 - const _: () = { 1580 - fn assert_zeroable<T: ?::core::marker::Sized + $crate::Zeroable>() {} 1581 - fn ensure_zeroable<$($impl_generics)*>() 1582 - where $($($whr)*)? 1583 - { 1584 - $(assert_zeroable::<$field_ty>();)* 1585 - } 1586 - }; 1587 - }; 1588 - (parse_input: 1589 - @sig( 1590 - $(#[$($struct_attr:tt)*])* 1591 - $vis:vis union $name:ident 1592 - $(where $($whr:tt)*)? 1593 - ), 1594 - @impl_generics($($impl_generics:tt)*), 1595 - @ty_generics($($ty_generics:tt)*), 1596 - @body({ 1597 - $( 1598 - $(#[$($field_attr:tt)*])* 1599 - $field_vis:vis $field:ident : $field_ty:ty 1600 - ),* $(,)? 1601 - }), 1602 - ) => { 1603 - // SAFETY: Every field type implements `Zeroable` and padding bytes may be zero. 1604 - #[automatically_derived] 1605 - unsafe impl<$($impl_generics)*> $crate::Zeroable for $name<$($ty_generics)*> 1606 - where 1607 - $($($whr)*)? 1608 - {} 1609 - const _: () = { 1610 - fn assert_zeroable<T: ?::core::marker::Sized + $crate::Zeroable>() {} 1611 - fn ensure_zeroable<$($impl_generics)*>() 1612 - where $($($whr)*)? 1613 - { 1614 - $(assert_zeroable::<$field_ty>();)* 1615 - } 1616 - }; 1617 - }; 1618 - } 1619 - 1620 - #[doc(hidden)] 1621 - #[macro_export] 1622 - macro_rules! __maybe_derive_zeroable { 1623 - (parse_input: 1624 - @sig( 1625 - $(#[$($struct_attr:tt)*])* 1626 - $vis:vis struct $name:ident 1627 - $(where $($whr:tt)*)? 1628 - ), 1629 - @impl_generics($($impl_generics:tt)*), 1630 - @ty_generics($($ty_generics:tt)*), 1631 - @body({ 1632 - $( 1633 - $(#[$($field_attr:tt)*])* 1634 - $field_vis:vis $field:ident : $field_ty:ty 1635 - ),* $(,)? 1636 - }), 1637 - ) => { 1638 - // SAFETY: Every field type implements `Zeroable` and padding bytes may be zero. 1639 - #[automatically_derived] 1640 - unsafe impl<$($impl_generics)*> $crate::Zeroable for $name<$($ty_generics)*> 1641 - where 1642 - $( 1643 - // the `for<'__dummy>` HRTB makes this not error without the `trivial_bounds` 1644 - // feature <https://github.com/rust-lang/rust/issues/48214#issuecomment-2557829956>. 1645 - $field_ty: for<'__dummy> $crate::Zeroable, 1646 - )* 1647 - $($($whr)*)? 1648 - {} 1649 - }; 1650 - (parse_input: 1651 - @sig( 1652 - $(#[$($struct_attr:tt)*])* 1653 - $vis:vis union $name:ident 1654 - $(where $($whr:tt)*)? 1655 - ), 1656 - @impl_generics($($impl_generics:tt)*), 1657 - @ty_generics($($ty_generics:tt)*), 1658 - @body({ 1659 - $( 1660 - $(#[$($field_attr:tt)*])* 1661 - $field_vis:vis $field:ident : $field_ty:ty 1662 - ),* $(,)? 1663 - }), 1664 - ) => { 1665 - // SAFETY: Every field type implements `Zeroable` and padding bytes may be zero. 1666 - #[automatically_derived] 1667 - unsafe impl<$($impl_generics)*> $crate::Zeroable for $name<$($ty_generics)*> 1668 - where 1669 - $( 1670 - // the `for<'__dummy>` HRTB makes this not error without the `trivial_bounds` 1671 - // feature <https://github.com/rust-lang/rust/issues/48214#issuecomment-2557829956>. 1672 - $field_ty: for<'__dummy> $crate::Zeroable, 1673 - )* 1674 - $($($whr)*)? 1675 - {} 1676 - }; 1677 - }
-1
samples/rust/rust_driver_auxiliary.rs
··· 16 16 }; 17 17 18 18 use core::any::TypeId; 19 - use pin_init::PinInit; 20 19 21 20 const MODULE_NAME: &CStr = <LocalModule as kernel::ModuleMetadata>::NAME; 22 21 const AUXILIARY_NAME: &CStr = c_str!("auxiliary");
+3 -4
samples/rust/rust_driver_i2c.rs
··· 4 4 5 5 use kernel::{ 6 6 acpi, 7 - c_str, 8 7 device::Core, 9 8 i2c, 10 9 of, ··· 16 17 ACPI_TABLE, 17 18 MODULE_ACPI_TABLE, 18 19 <SampleDriver as i2c::Driver>::IdInfo, 19 - [(acpi::DeviceId::new(c_str!("LNUXBEEF")), 0)] 20 + [(acpi::DeviceId::new(c"LNUXBEEF"), 0)] 20 21 } 21 22 22 23 kernel::i2c_device_table! { 23 24 I2C_TABLE, 24 25 MODULE_I2C_TABLE, 25 26 <SampleDriver as i2c::Driver>::IdInfo, 26 - [(i2c::DeviceId::new(c_str!("rust_driver_i2c")), 0)] 27 + [(i2c::DeviceId::new(c"rust_driver_i2c"), 0)] 27 28 } 28 29 29 30 kernel::of_device_table! { 30 31 OF_TABLE, 31 32 MODULE_OF_TABLE, 32 33 <SampleDriver as i2c::Driver>::IdInfo, 33 - [(of::DeviceId::new(c_str!("test,rust_driver_i2c")), 0)] 34 + [(of::DeviceId::new(c"test,rust_driver_i2c"), 0)] 34 35 } 35 36 36 37 impl i2c::Driver for SampleDriver {
+3 -4
samples/rust/rust_i2c_client.rs
··· 69 69 70 70 use kernel::{ 71 71 acpi, 72 - c_str, 73 72 device, 74 73 devres::Devres, 75 74 i2c, ··· 89 90 OF_TABLE, 90 91 MODULE_OF_TABLE, 91 92 <SampleDriver as platform::Driver>::IdInfo, 92 - [(of::DeviceId::new(c_str!("test,rust-device")), ())] 93 + [(of::DeviceId::new(c"test,rust-device"), ())] 93 94 ); 94 95 95 96 kernel::acpi_device_table!( 96 97 ACPI_TABLE, 97 98 MODULE_ACPI_TABLE, 98 99 <SampleDriver as platform::Driver>::IdInfo, 99 - [(acpi::DeviceId::new(c_str!("LNUXBEEF")), ())] 100 + [(acpi::DeviceId::new(c"LNUXBEEF"), ())] 100 101 ); 101 102 102 103 const SAMPLE_I2C_CLIENT_ADDR: u16 = 0x30; 103 104 const SAMPLE_I2C_ADAPTER_INDEX: i32 = 0; 104 105 const BOARD_INFO: i2c::I2cBoardInfo = 105 - i2c::I2cBoardInfo::new(c_str!("rust_driver_i2c"), SAMPLE_I2C_CLIENT_ADDR); 106 + i2c::I2cBoardInfo::new(c"rust_driver_i2c", SAMPLE_I2C_CLIENT_ADDR); 106 107 107 108 impl platform::Driver for SampleDriver { 108 109 type IdInfo = ();
-2
samples/rust/rust_misc_device.rs
··· 95 95 //! } 96 96 //! ``` 97 97 98 - use core::pin::Pin; 99 - 100 98 use kernel::{ 101 99 c_str, 102 100 device::Device,
+1 -1
samples/rust/rust_print_main.rs
··· 101 101 } 102 102 103 103 mod trace { 104 - use kernel::ffi::c_int; 104 + use kernel::prelude::*; 105 105 106 106 kernel::declare_trace! { 107 107 /// # Safety
+1 -1
scripts/generate_rust_analyzer.py
··· 147 147 append_crate( 148 148 "pin_init_internal", 149 149 srctree / "rust" / "pin-init" / "internal" / "src" / "lib.rs", 150 - ["std", "proc_macro"], 150 + ["std", "proc_macro", "proc_macro2", "quote", "syn"], 151 151 cfg=["kernel"], 152 152 is_proc_macro=True, 153 153 )
+2 -2
scripts/rustdoc_test_gen.rs
··· 174 174 macro_rules! assert {{ 175 175 ($cond:expr $(,)?) => {{{{ 176 176 ::kernel::kunit_assert!( 177 - "{kunit_name}", "{real_path}", __DOCTEST_ANCHOR - {line}, $cond 177 + "{kunit_name}", c"{real_path}", __DOCTEST_ANCHOR - {line}, $cond 178 178 ); 179 179 }}}} 180 180 }} ··· 184 184 macro_rules! assert_eq {{ 185 185 ($left:expr, $right:expr $(,)?) => {{{{ 186 186 ::kernel::kunit_assert_eq!( 187 - "{kunit_name}", "{real_path}", __DOCTEST_ANCHOR - {line}, $left, $right 187 + "{kunit_name}", c"{real_path}", __DOCTEST_ANCHOR - {line}, $left, $right 188 188 ); 189 189 }}}} 190 190 }}