rust: devres: fix race in Devres::drop()

In Devres::drop() we first remove the devres action and then drop the
wrapped device resource.

The design goal is to give the owner of a Devres object control over when
the device resource is dropped, but limit the overall scope to the
corresponding device being bound to a driver.

However, there's a race that was introduced with commit 8ff656643d30
("rust: devres: remove action in `Devres::drop`"), but also has been
(partially) present from the initial version on.

In Devres::drop(), the devres action is removed successfully and
subsequently the destructor of the wrapped device resource runs.
However, there is no guarantee that the destructor of the wrapped device
resource completes before the driver core is done unbinding the
corresponding device.

If in Devres::drop(), the devres action can't be removed, it means that
the devres callback has been executed already, or is still running
concurrently. In case of the latter, either Devres::drop() wins revoking
the Revocable or the devres callback wins revoking the Revocable. If
Devres::drop() wins, we (again) have no guarantee that the destructor of
the wrapped device resource completes before the driver core is done
unbinding the corresponding device.

CPU0 CPU1
------------------------------------------------------------------------
Devres::drop() { Devres::devres_callback() {
self.data.revoke() { this.data.revoke() {
is_available.swap() == true
is_available.swap == false
}
}

// [...]
// device fully unbound
drop_in_place() {
// release device resource
}
}
}

Depending on the specific device resource, this can potentially lead to
user-after-free bugs.

In order to fix this, implement the following logic.

In the devres callback, we're always good when we get to revoke the
device resource ourselves, i.e. Revocable::revoke() returns true.

If Revocable::revoke() returns false, it means that Devres::drop(),
concurrently, already drops the device resource and we have to wait for
Devres::drop() to signal that it finished dropping the device resource.

Note that if we hit the case where we need to wait for the completion of
Devres::drop() in the devres callback, it means that we're actually
racing with a concurrent Devres::drop() call, which already started
revoking the device resource for us. This is rather unlikely and means
that the concurrent Devres::drop() already started doing our work and we
just need to wait for it to complete it for us. Hence, there should not
be any additional overhead from that.

(Actually, for now it's even better if Devres::drop() does the work for
us, since it can bypass the synchronize_rcu() call implied by
Revocable::revoke(), but this goes away anyways once I get to implement
the split devres callback approach, which allows us to first flip the
atomics of all registered Devres objects of a certain device, execute a
single synchronize_rcu() and then drop all revocable objects.)

In Devres::drop() we try to revoke the device resource. If that is *not*
successful, it means that the devres callback already did and we're good.

Otherwise, we try to remove the devres action, which, if successful,
means that we're good, since the device resource has just been revoked
by us *before* we removed the devres action successfully.

If the devres action could not be removed, it means that the devres
callback must be running concurrently, hence we signal that the device
resource has been revoked by us, using the completion.

This makes it safe to drop a Devres object from any task and at any point
of time, which is one of the design goals.

Fixes: 76c01ded724b ("rust: add devres abstraction")
Reported-by: Alice Ryhl <aliceryhl@google.com>
Closes: https://lore.kernel.org/lkml/aD64YNuqbPPZHAa5@google.com/
Reviewed-by: Benno Lossin <lossin@kernel.org>
Link: https://lore.kernel.org/r/20250612121817.1621-4-dakr@kernel.org
Signed-off-by: Danilo Krummrich <dakr@kernel.org>

Danilo Krummrich 1 year ago f744201c 4b76fafb

+29 -8

1 changed file

expand all

rust

kernel

devres.rs

+29 -8

rust/kernel/devres.rs

··· 13 13 ffi::c_void, 14 14 prelude::*, 15 15 revocable::Revocable, 16 - sync::Arc, 16 + sync::{Arc, Completion}, 17 17 types::ARef, 18 18 }; 19 19 ··· 25 25 callback: unsafe extern "C" fn(*mut c_void), 26 26 #[pin] 27 27 data: Revocable<T>, 28 + #[pin] 29 + revoke: Completion, 28 30 } 29 31 30 32 /// This abstraction is meant to be used by subsystems to containerize [`Device`] bound resources to 31 33 /// manage their lifetime. 32 34 /// 33 35 /// [`Device`] bound resources should be freed when either the resource goes out of scope or the 34 - /// [`Device`] is unbound respectively, depending on what happens first. 36 + /// [`Device`] is unbound respectively, depending on what happens first. In any case, it is always 37 + /// guaranteed that revoking the device resource is completed before the corresponding [`Device`] 38 + /// is unbound. 35 39 /// 36 40 /// To achieve that [`Devres`] registers a devres callback on creation, which is called once the 37 41 /// [`Device`] is unbound, revoking access to the encapsulated resource (see also [`Revocable`]). ··· 106 102 dev: dev.into(), 107 103 callback: Self::devres_callback, 108 104 data <- Revocable::new(data), 105 + revoke <- Completion::new(), 109 106 }), 110 107 flags, 111 108 )?; ··· 135 130 self as _ 136 131 } 137 132 138 - fn remove_action(this: &Arc<Self>) { 133 + fn remove_action(this: &Arc<Self>) -> bool { 139 134 // SAFETY: 140 135 // - `self.inner.dev` is a valid `Device`, 141 136 // - the `action` and `data` pointers are the exact same ones as given to devm_add_action() 142 137 // previously, 143 138 // - `self` is always valid, even if the action has been released already. 144 - let ret = unsafe { 139 + let success = unsafe { 145 140 bindings::devm_remove_action_nowarn( 146 141 this.dev.as_raw(), 147 142 Some(this.callback), 148 143 this.as_ptr() as _, 149 144 ) 150 - }; 145 + } == 0; 151 146 152 - if ret == 0 { 147 + if success { 153 148 // SAFETY: We leaked an `Arc` reference to devm_add_action() in `DevresInner::new`; if 154 149 // devm_remove_action_nowarn() was successful we can (and have to) claim back ownership 155 150 // of this reference. 156 151 let _ = unsafe { Arc::from_raw(this.as_ptr()) }; 157 152 } 153 + 154 + success 158 155 } 159 156 160 157 #[allow(clippy::missing_safety_doc)] ··· 168 161 // `DevresInner::new`. 169 162 let inner = unsafe { Arc::from_raw(ptr) }; 170 163 171 - inner.data.revoke(); 164 + if !inner.data.revoke() { 165 + // If `revoke()` returns false, it means that `Devres::drop` already started revoking 166 + // `inner.data` for us. Hence we have to wait until `Devres::drop()` signals that it 167 + // completed revoking `inner.data`. 168 + inner.revoke.wait_for_completion(); 169 + } 172 170 } 173 171 } 174 172 ··· 244 232 245 233 impl<T> Drop for Devres<T> { 246 234 fn drop(&mut self) { 247 - DevresInner::remove_action(&self.0); 235 + // SAFETY: When `drop` runs, it is guaranteed that nobody is accessing the revocable data 236 + // anymore, hence it is safe not to wait for the grace period to finish. 237 + if unsafe { self.revoke_nosync() } { 238 + // We revoked `self.0.data` before the devres action did, hence try to remove it. 239 + if !DevresInner::remove_action(&self.0) { 240 + // We could not remove the devres action, which means that it now runs concurrently, 241 + // hence signal that `self.0.data` has been revoked successfully. 242 + self.0.revoke.complete_all(); 243 + } 244 + } 248 245 } 249 246 }

Configure Feed

Configure Feed