Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
1:orphan:
2
3Making Filesystems Exportable
4=============================
5
6Overview
7--------
8
9All filesystem operations require a dentry (or two) as a starting
10point. Local applications have a reference-counted hold on suitable
11dentries via open file descriptors or cwd/root. However remote
12applications that access a filesystem via a remote filesystem protocol
13such as NFS may not be able to hold such a reference, and so need a
14different way to refer to a particular dentry. As the alternative
15form of reference needs to be stable across renames, truncates, and
16server-reboot (among other things, though these tend to be the most
17problematic), there is no simple answer like 'filename'.
18
19The mechanism discussed here allows each filesystem implementation to
20specify how to generate an opaque (outside of the filesystem) byte
21string for any dentry, and how to find an appropriate dentry for any
22given opaque byte string.
23This byte string will be called a "filehandle fragment" as it
24corresponds to part of an NFS filehandle.
25
26A filesystem which supports the mapping between filehandle fragments
27and dentries will be termed "exportable".
28
29
30
31Dcache Issues
32-------------
33
34The dcache normally contains a proper prefix of any given filesystem
35tree. This means that if any filesystem object is in the dcache, then
36all of the ancestors of that filesystem object are also in the dcache.
37As normal access is by filename this prefix is created naturally and
38maintained easily (by each object maintaining a reference count on
39its parent).
40
41However when objects are included into the dcache by interpreting a
42filehandle fragment, there is no automatic creation of a path prefix
43for the object. This leads to two related but distinct features of
44the dcache that are not needed for normal filesystem access.
45
461. The dcache must sometimes contain objects that are not part of the
47 proper prefix. i.e that are not connected to the root.
482. The dcache must be prepared for a newly found (via ->lookup) directory
49 to already have a (non-connected) dentry, and must be able to move
50 that dentry into place (based on the parent and name in the
51 ->lookup). This is particularly needed for directories as
52 it is a dcache invariant that directories only have one dentry.
53
54To implement these features, the dcache has:
55
56a. A dentry flag DCACHE_DISCONNECTED which is set on
57 any dentry that might not be part of the proper prefix.
58 This is set when anonymous dentries are created, and cleared when a
59 dentry is noticed to be a child of a dentry which is in the proper
60 prefix. If the refcount on a dentry with this flag set
61 becomes zero, the dentry is immediately discarded, rather than being
62 kept in the dcache. If a dentry that is not already in the dcache
63 is repeatedly accessed by filehandle (as NFSD might do), an new dentry
64 will be a allocated for each access, and discarded at the end of
65 the access.
66
67 Note that such a dentry can acquire children, name, ancestors, etc.
68 without losing DCACHE_DISCONNECTED - that flag is only cleared when
69 subtree is successfully reconnected to root. Until then dentries
70 in such subtree are retained only as long as there are references;
71 refcount reaching zero means immediate eviction, same as for unhashed
72 dentries. That guarantees that we won't need to hunt them down upon
73 umount.
74
75b. A primitive for creation of secondary roots - d_obtain_root(inode).
76 Those do _not_ bear DCACHE_DISCONNECTED. They are placed on the
77 per-superblock list (->s_roots), so they can be located at umount
78 time for eviction purposes.
79
80c. Helper routines to allocate anonymous dentries, and to help attach
81 loose directory dentries at lookup time. They are:
82
83 d_obtain_alias(inode) will return a dentry for the given inode.
84 If the inode already has a dentry, one of those is returned.
85
86 If it doesn't, a new anonymous (IS_ROOT and
87 DCACHE_DISCONNECTED) dentry is allocated and attached.
88
89 In the case of a directory, care is taken that only one dentry
90 can ever be attached.
91
92 d_splice_alias(inode, dentry) will introduce a new dentry into the tree;
93 either the passed-in dentry or a preexisting alias for the given inode
94 (such as an anonymous one created by d_obtain_alias), if appropriate.
95 It returns NULL when the passed-in dentry is used, following the calling
96 convention of ->lookup.
97
98Filesystem Issues
99-----------------
100
101For a filesystem to be exportable it must:
102
103 1. provide the filehandle fragment routines described below.
104 2. make sure that d_splice_alias is used rather than d_add
105 when ->lookup finds an inode for a given parent and name.
106
107 If inode is NULL, d_splice_alias(inode, dentry) is equivalent to::
108
109 d_add(dentry, inode), NULL
110
111 Similarly, d_splice_alias(ERR_PTR(err), dentry) = ERR_PTR(err)
112
113 Typically the ->lookup routine will simply end with a::
114
115 return d_splice_alias(inode, dentry);
116 }
117
118
119
120A file system implementation declares that instances of the filesystem
121are exportable by setting the s_export_op field in the struct
122super_block. This field must point to a struct export_operations
123which has the following members:
124
125.. kernel-doc:: include/linux/exportfs.h
126 :identifiers: struct export_operations
127
128A filehandle fragment consists of an array of 1 or more 4byte words,
129together with a one byte "type".
130The decode_fh routine should not depend on the stated size that is
131passed to it. This size may be larger than the original filehandle
132generated by encode_fh, in which case it will have been padded with
133nuls. Rather, the encode_fh routine should choose a "type" which
134indicates the decode_fh how much of the filehandle is valid, and how
135it should be interpreted.
136
137Export Operations Flags
138-----------------------
139In addition to the operation vector pointers, struct export_operations also
140contains a "flags" field that allows the filesystem to communicate to nfsd
141that it may want to do things differently when dealing with it. The
142following flags are defined:
143
144 EXPORT_OP_NOWCC - disable NFSv3 WCC attributes on this filesystem
145 RFC 1813 recommends that servers always send weak cache consistency
146 (WCC) data to the client after each operation. The server should
147 atomically collect attributes about the inode, do an operation on it,
148 and then collect the attributes afterward. This allows the client to
149 skip issuing GETATTRs in some situations but means that the server
150 is calling vfs_getattr for almost all RPCs. On some filesystems
151 (particularly those that are clustered or networked) this is expensive
152 and atomicity is difficult to guarantee. This flag indicates to nfsd
153 that it should skip providing WCC attributes to the client in NFSv3
154 replies when doing operations on this filesystem. Consider enabling
155 this on filesystems that have an expensive ->getattr inode operation,
156 or when atomicity between pre and post operation attribute collection
157 is impossible to guarantee.
158
159 EXPORT_OP_NOSUBTREECHK - disallow subtree checking on this fs
160 Many NFS operations deal with filehandles, which the server must then
161 vet to ensure that they live inside of an exported tree. When the
162 export consists of an entire filesystem, this is trivial. nfsd can just
163 ensure that the filehandle live on the filesystem. When only part of a
164 filesystem is exported however, then nfsd must walk the ancestors of the
165 inode to ensure that it's within an exported subtree. This is an
166 expensive operation and not all filesystems can support it properly.
167 This flag exempts the filesystem from subtree checking and causes
168 exportfs to get back an error if it tries to enable subtree checking
169 on it.
170
171 EXPORT_OP_CLOSE_BEFORE_UNLINK - always close cached files before unlinking
172 On some exportable filesystems (such as NFS) unlinking a file that
173 is still open can cause a fair bit of extra work. For instance,
174 the NFS client will do a "sillyrename" to ensure that the file
175 sticks around while it's still open. When reexporting, that open
176 file is held by nfsd so we usually end up doing a sillyrename, and
177 then immediately deleting the sillyrenamed file just afterward when
178 the link count actually goes to zero. Sometimes this delete can race
179 with other operations (for instance an rmdir of the parent directory).
180 This flag causes nfsd to close any open files for this inode _before_
181 calling into the vfs to do an unlink or a rename that would replace
182 an existing file.
183
184 EXPORT_OP_REMOTE_FS - Backing storage for this filesystem is remote
185 PF_LOCAL_THROTTLE exists for loopback NFSD, where a thread needs to
186 write to one bdi (the final bdi) in order to free up writes queued
187 to another bdi (the client bdi). Such threads get a private balance
188 of dirty pages so that dirty pages for the client bdi do not imact
189 the daemon writing to the final bdi. For filesystems whose durable
190 storage is not local (such as exported NFS filesystems), this
191 constraint has negative consequences. EXPORT_OP_REMOTE_FS enables
192 an export to disable writeback throttling.
193
194 EXPORT_OP_NOATOMIC_ATTR - Filesystem does not update attributes atomically
195 EXPORT_OP_NOATOMIC_ATTR indicates that the exported filesystem
196 cannot provide the semantics required by the "atomic" boolean in
197 NFSv4's change_info4. This boolean indicates to a client whether the
198 returned before and after change attributes were obtained atomically
199 with the respect to the requested metadata operation (UNLINK,
200 OPEN/CREATE, MKDIR, etc).
201
202 EXPORT_OP_FLUSH_ON_CLOSE - Filesystem flushes file data on close(2)
203 On most filesystems, inodes can remain under writeback after the
204 file is closed. NFSD relies on client activity or local flusher
205 threads to handle writeback. Certain filesystems, such as NFS, flush
206 all of an inode's dirty data on last close. Exports that behave this
207 way should set EXPORT_OP_FLUSH_ON_CLOSE so that NFSD knows to skip
208 waiting for writeback when closing such files.
209
210Signed Filehandles
211------------------
212
213To protect against filehandle guessing attacks, the Linux NFS server can be
214configured to sign filehandles with a Message Authentication Code (MAC).
215
216Standard NFS filehandles are often predictable. If an attacker can guess
217a valid filehandle for a file they do not have permission to access via
218directory traversal, they may be able to bypass path-based permissions
219(though they still remain subject to inode-level permissions).
220
221Signed filehandles prevent this by appending a MAC to the filehandle
222before it is sent to the client. Upon receiving a filehandle back from a
223client, the server re-calculates the MAC using its internal key and
224verifies it against the one provided. If the signatures do not match,
225the server treats the filehandle as invalid (returning NFS[34]ERR_STALE).
226
227Note that signing filehandles provides integrity and authenticity but
228not confidentiality. The contents of the filehandle remain visible to
229the client; they simply cannot be forged or modified.
230
231Configuration
232~~~~~~~~~~~~~
233
234To enable signed filehandles, the administrator must provide a signing
235key to the kernel and enable the "sign_fh" export option.
236
2371. Providing a Key
238 The signing key is managed via the nfsd netlink interface. This key
239 is per-network-namespace and must be set before any exports using
240 "sign_fh" become active.
241
2422. Export Options
243 The feature is controlled on a per-export basis in /etc/exports:
244
245 sign_fh
246 Enables signing for all filehandles generated under this export.
247
248 no_sign_fh
249 (Default) Disables signing.
250
251Key Management and Rotation
252~~~~~~~~~~~~~~~~~~~~~~~~~~~
253
254The security of this mechanism relies entirely on the secrecy of the
255signing key.
256
257Initial Setup:
258 The key should be generated using a high-quality random source and
259 loaded early in the boot process or during the nfs-server startup
260 sequence.
261
262Changing Keys:
263 If a key is changed while clients have active mounts, existing
264 filehandles held by those clients will become invalid, resulting in
265 "Stale file handle" errors on the client side.
266
267Safe Rotation:
268 Currently, there is no mechanism for "graceful" key rotation
269 (maintaining multiple valid keys). Changing the key is an atomic
270 operation that immediately invalidates all previous signatures.
271
272Transitioning Exports
273~~~~~~~~~~~~~~~~~~~~~
274
275When adding or removing the "sign_fh" flag from an active export, the
276following behaviors should be expected:
277
278+-------------------+---------------------------------------------------+
279| Change | Result for Existing Clients |
280+===================+===================================================+
281| Adding sign_fh | Clients holding unsigned filehandles will find |
282| | them rejected, as the server now expects a |
283| | signature. |
284+-------------------+---------------------------------------------------+
285| Removing sign_fh | Clients holding signed filehandles will find them |
286| | rejected, as the server now expects the |
287| | filehandle to end at its traditional boundary |
288| | without a MAC. |
289+-------------------+---------------------------------------------------+
290
291Because filehandles are often cached persistently by clients, adding or
292removing this option should generally be done during a scheduled maintenance
293window involving a NFS client unmount/remount.