Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
Merge tag 'netfs-prep-20220318' of git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs
Pull netfs updates from David Howells:
"Netfs prep for write helpers.
Having had a go at implementing write helpers and content encryption
support in netfslib, it seems that the netfs_read_{,sub}request
structs and the equivalent write request structs were almost the same
and so should be merged, thereby requiring only one set of
alloc/get/put functions and a common set of tracepoints.
Merging the structs also has the advantage that if a bounce buffer is
added to the request struct, a read operation can be performed to fill
the bounce buffer, the contents of the buffer can be modified and then
a write operation can be performed on it to send the data wherever it
needs to go using the same request structure all the way through. The
I/O handlers would then transparently perform any required crypto.
This should make it easier to perform RMW cycles if needed.
The potentially common functions and structs, however, by their names
all proclaim themselves to be associated with the read side of things.
The bulk of these changes alter this in the following ways:
- Rename struct netfs_read_{,sub}request to netfs_io_{,sub}request.
- Rename some enums, members and flags to make them more appropriate.
- Adjust some comments to match.
- Drop "read"/"rreq" from the names of common functions. For
instance, netfs_get_read_request() becomes netfs_get_request().
- The ->init_rreq() and ->issue_op() methods become ->init_request()
and ->issue_read(). I've kept the latter as a read-specific
function and in another branch added an ->issue_write() method.
The driver source is then reorganised into a number of files:
fs/netfs/buffered_read.c Create read reqs to the pagecache
fs/netfs/io.c Dispatchers for read and write reqs
fs/netfs/main.c Some general miscellaneous bits
fs/netfs/objects.c Alloc, get and put functions
fs/netfs/stats.c Optional procfs statistics.
and future development can be fitted into this scheme, e.g.:
fs/netfs/buffered_write.c Modify the pagecache
fs/netfs/buffered_flush.c Writeback from the pagecache
fs/netfs/direct_read.c DIO read support
fs/netfs/direct_write.c DIO write support
fs/netfs/unbuffered_write.c Write modifications directly back
Beyond the above changes, there are also some changes that affect how
things work:
- Make fscache_end_operation() generally available.
- In the netfs tracing header, generate enums from the symbol ->
string mapping tables rather than manually coding them.
- Add a struct for filesystems that uses netfslib to put into their
inode wrapper structs to hold extra state that netfslib is
interested in, such as the fscache cookie. This allows netfslib
functions to be set in filesystem operation tables and jumped to
directly without having to have a filesystem wrapper.
- Add a member to the struct added above to track the remote inode
length as that may differ if local modifications are buffered. We
may need to supply an appropriate EOF pointer when storing data (in
AFS for example).
- Pass extra information to netfs_alloc_request() so that the
->init_request() hook can access it and retain information to
indicate the origin of the operation.
- Make the ->init_request() hook return an error, thereby allowing a
filesystem that isn't allowed to cache an inode (ceph or cifs, for
example) to skip readahead.
- Switch to using refcount_t for subrequests and add tracepoints to
log refcount changes for the request and subrequest structs.
- Add a function to consolidate dispatching a read request. Similar
code is used in three places and another couple are likely to be
added in the future"
Link: https://lore.kernel.org/all/2639515.1648483225@warthog.procyon.org.uk/
* tag 'netfs-prep-20220318' of git://git.kernel.org/pub/scm/linux/kernel/git/dhowells/linux-fs:
afs: Maintain netfs_i_context::remote_i_size
netfs: Keep track of the actual remote file size
netfs: Split some core bits out into their own file
netfs: Split fs/netfs/read_helper.c
netfs: Rename read_helper.c to io.c
netfs: Prepare to split read_helper.c
netfs: Add a function to consolidate beginning a read
netfs: Add a netfs inode context
ceph: Make ceph_init_request() check caps on readahead
netfs: Change ->init_request() to return an error code
netfs: Refactor arguments for netfs_alloc_read_request
netfs: Adjust the netfs_failure tracepoint to indicate non-subreq lines
netfs: Trace refcounting on the netfs_io_subrequest struct
netfs: Trace refcounting on the netfs_io_request struct
netfs: Adjust the netfs_rreq tracepoint slightly
netfs: Split netfs_io_* object handling out
netfs: Finish off rename of netfs_read_request to netfs_io_request
netfs: Rename netfs_read_*request to netfs_io_*request
netfs: Generate enums from trace symbol mapping lists
fscache: export fscache_end_operation()
+1868
-1628
+92
-48
Documentation/filesystems/netfs_library.rst
+92
-48
Documentation/filesystems/netfs_library.rst
···
7
7
.. Contents:
8
8
9
9
- Overview.
10
+
- Per-inode context.
11
+
- Inode context helper functions.
10
12
- Buffered read helpers.
11
13
- Read helper functions.
12
14
- Read helper structures.
···
28
26
29
27
Note that the library module doesn't link against local caching directly, so
30
28
access must be provided by the netfs.
29
+
30
+
31
+
Per-Inode Context
32
+
=================
33
+
34
+
The network filesystem helper library needs a place to store a bit of state for
35
+
its use on each netfs inode it is helping to manage. To this end, a context
36
+
structure is defined::
37
+
38
+
struct netfs_i_context {
39
+
const struct netfs_request_ops *ops;
40
+
struct fscache_cookie *cache;
41
+
};
42
+
43
+
A network filesystem that wants to use netfs lib must place one of these
44
+
directly after the VFS ``struct inode`` it allocates, usually as part of its
45
+
own struct. This can be done in a way similar to the following::
46
+
47
+
struct my_inode {
48
+
struct {
49
+
/* These must be contiguous */
50
+
struct inode vfs_inode;
51
+
struct netfs_i_context netfs_ctx;
52
+
};
53
+
...
54
+
};
55
+
56
+
This allows netfslib to find its state by simple offset from the inode pointer,
57
+
thereby allowing the netfslib helper functions to be pointed to directly by the
58
+
VFS/VM operation tables.
59
+
60
+
The structure contains the following fields:
61
+
62
+
* ``ops``
63
+
64
+
The set of operations provided by the network filesystem to netfslib.
65
+
66
+
* ``cache``
67
+
68
+
Local caching cookie, or NULL if no caching is enabled. This field does not
69
+
exist if fscache is disabled.
70
+
71
+
72
+
Inode Context Helper Functions
73
+
------------------------------
74
+
75
+
To help deal with the per-inode context, a number helper functions are
76
+
provided. Firstly, a function to perform basic initialisation on a context and
77
+
set the operations table pointer::
78
+
79
+
void netfs_i_context_init(struct inode *inode,
80
+
const struct netfs_request_ops *ops);
81
+
82
+
then two functions to cast between the VFS inode structure and the netfs
83
+
context::
84
+
85
+
struct netfs_i_context *netfs_i_context(struct inode *inode);
86
+
struct inode *netfs_inode(struct netfs_i_context *ctx);
87
+
88
+
and finally, a function to get the cache cookie pointer from the context
89
+
attached to an inode (or NULL if fscache is disabled)::
90
+
91
+
struct fscache_cookie *netfs_i_cookie(struct inode *inode);
31
92
32
93
33
94
Buffered Read Helpers
···
135
70
136
71
Three read helpers are provided::
137
72
138
-
void netfs_readahead(struct readahead_control *ractl,
139
-
const struct netfs_read_request_ops *ops,
140
-
void *netfs_priv);
73
+
void netfs_readahead(struct readahead_control *ractl);
141
74
int netfs_readpage(struct file *file,
142
-
struct folio *folio,
143
-
const struct netfs_read_request_ops *ops,
144
-
void *netfs_priv);
75
+
struct page *page);
145
76
int netfs_write_begin(struct file *file,
146
77
struct address_space *mapping,
147
78
loff_t pos,
148
79
unsigned int len,
149
80
unsigned int flags,
150
81
struct folio **_folio,
151
-
void **_fsdata,
152
-
const struct netfs_read_request_ops *ops,
153
-
void *netfs_priv);
82
+
void **_fsdata);
154
83
155
-
Each corresponds to a VM operation, with the addition of a couple of parameters
156
-
for the use of the read helpers:
84
+
Each corresponds to a VM address space operation. These operations use the
85
+
state in the per-inode context.
157
86
158
-
* ``ops``
159
-
160
-
A table of operations through which the helpers can talk to the filesystem.
161
-
162
-
* ``netfs_priv``
163
-
164
-
Filesystem private data (can be NULL).
165
-
166
-
Both of these values will be stored into the read request structure.
167
-
168
-
For ->readahead() and ->readpage(), the network filesystem should just jump
169
-
into the corresponding read helper; whereas for ->write_begin(), it may be a
87
+
For ->readahead() and ->readpage(), the network filesystem just point directly
88
+
at the corresponding read helper; whereas for ->write_begin(), it may be a
170
89
little more complicated as the network filesystem might want to flush
171
90
conflicting writes or track dirty data and needs to put the acquired folio if
172
91
an error occurs after calling the helper.
···
165
116
166
117
Additionally, there is::
167
118
168
-
* void netfs_subreq_terminated(struct netfs_read_subrequest *subreq,
119
+
* void netfs_subreq_terminated(struct netfs_io_subrequest *subreq,
169
120
ssize_t transferred_or_error,
170
121
bool was_async);
171
122
···
181
132
The read helpers make use of a couple of structures to maintain the state of
182
133
the read. The first is a structure that manages a read request as a whole::
183
134
184
-
struct netfs_read_request {
135
+
struct netfs_io_request {
185
136
struct inode *inode;
186
137
struct address_space *mapping;
187
138
struct netfs_cache_resources cache_resources;
···
189
140
loff_t start;
190
141
size_t len;
191
142
loff_t i_size;
192
-
const struct netfs_read_request_ops *netfs_ops;
143
+
const struct netfs_request_ops *netfs_ops;
193
144
unsigned int debug_id;
194
145
...
195
146
};
···
236
187
The second structure is used to manage individual slices of the overall read
237
188
request::
238
189
239
-
struct netfs_read_subrequest {
240
-
struct netfs_read_request *rreq;
190
+
struct netfs_io_subrequest {
191
+
struct netfs_io_request *rreq;
241
192
loff_t start;
242
193
size_t len;
243
194
size_t transferred;
···
293
244
The network filesystem must provide the read helpers with a table of operations
294
245
through which it can issue requests and negotiate::
295
246
296
-
struct netfs_read_request_ops {
297
-
void (*init_rreq)(struct netfs_read_request *rreq, struct file *file);
298
-
bool (*is_cache_enabled)(struct inode *inode);
299
-
int (*begin_cache_operation)(struct netfs_read_request *rreq);
300
-
void (*expand_readahead)(struct netfs_read_request *rreq);
301
-
bool (*clamp_length)(struct netfs_read_subrequest *subreq);
302
-
void (*issue_op)(struct netfs_read_subrequest *subreq);
303
-
bool (*is_still_valid)(struct netfs_read_request *rreq);
247
+
struct netfs_request_ops {
248
+
void (*init_request)(struct netfs_io_request *rreq, struct file *file);
249
+
int (*begin_cache_operation)(struct netfs_io_request *rreq);
250
+
void (*expand_readahead)(struct netfs_io_request *rreq);
251
+
bool (*clamp_length)(struct netfs_io_subrequest *subreq);
252
+
void (*issue_read)(struct netfs_io_subrequest *subreq);
253
+
bool (*is_still_valid)(struct netfs_io_request *rreq);
304
254
int (*check_write_begin)(struct file *file, loff_t pos, unsigned len,
305
255
struct folio *folio, void **_fsdata);
306
-
void (*done)(struct netfs_read_request *rreq);
256
+
void (*done)(struct netfs_io_request *rreq);
307
257
void (*cleanup)(struct address_space *mapping, void *netfs_priv);
308
258
};
309
259
310
260
The operations are as follows:
311
261
312
-
* ``init_rreq()``
262
+
* ``init_request()``
313
263
314
264
[Optional] This is called to initialise the request structure. It is given
315
265
the file for reference and can modify the ->netfs_priv value.
316
-
317
-
* ``is_cache_enabled()``
318
-
319
-
[Required] This is called by netfs_write_begin() to ask if the file is being
320
-
cached. It should return true if it is being cached and false otherwise.
321
266
322
267
* ``begin_cache_operation()``
323
268
···
348
305
349
306
This should return 0 on success and an error code on error.
350
307
351
-
* ``issue_op()``
308
+
* ``issue_read()``
352
309
353
310
[Required] The helpers use this to dispatch a subrequest to the server for
354
311
reading. In the subrequest, ->start, ->len and ->transferred indicate what
···
463
420
cache and this must call into the cache to do the work. If using fscache, for
464
421
example, the cache would call::
465
422
466
-
int fscache_begin_read_operation(struct netfs_read_request *rreq,
423
+
int fscache_begin_read_operation(struct netfs_io_request *rreq,
467
424
struct fscache_cookie *cookie);
468
425
469
426
passing in the request pointer and the cookie corresponding to the file.
470
427
471
-
The netfs_read_request object contains a place for the cache to hang its
428
+
The netfs_io_request object contains a place for the cache to hang its
472
429
state::
473
430
474
431
struct netfs_cache_resources {
···
486
443
void (*expand_readahead)(struct netfs_cache_resources *cres,
487
444
loff_t *_start, size_t *_len, loff_t i_size);
488
445
489
-
enum netfs_read_source (*prepare_read)(struct netfs_read_subrequest *subreq,
446
+
enum netfs_io_source (*prepare_read)(struct netfs_io_subrequest *subreq,
490
447
loff_t i_size);
491
448
492
449
int (*read)(struct netfs_cache_resources *cres,
···
605
562
======================
606
563
607
564
.. kernel-doc:: include/linux/netfs.h
608
-
.. kernel-doc:: fs/netfs/read_helper.c
565
+
.. kernel-doc:: fs/netfs/buffered_read.c
566
+
.. kernel-doc:: fs/netfs/io.c
+4
-6
fs/9p/cache.c
+4
-6
fs/9p/cache.c
···
49
49
50
50
void v9fs_cache_inode_get_cookie(struct inode *inode)
51
51
{
52
-
struct v9fs_inode *v9inode;
52
+
struct v9fs_inode *v9inode = V9FS_I(inode);
53
53
struct v9fs_session_info *v9ses;
54
54
__le32 version;
55
55
__le64 path;
56
56
57
57
if (!S_ISREG(inode->i_mode))
58
58
return;
59
-
60
-
v9inode = V9FS_I(inode);
61
-
if (WARN_ON(v9inode->fscache))
59
+
if (WARN_ON(v9fs_inode_cookie(v9inode)))
62
60
return;
63
61
64
62
version = cpu_to_le32(v9inode->qid.version);
65
63
path = cpu_to_le64(v9inode->qid.path);
66
64
v9ses = v9fs_inode2v9ses(inode);
67
-
v9inode->fscache =
65
+
v9inode->netfs_ctx.cache =
68
66
fscache_acquire_cookie(v9fs_session_cache(v9ses),
69
67
0,
70
68
&path, sizeof(path),
···
70
72
i_size_read(&v9inode->vfs_inode));
71
73
72
74
p9_debug(P9_DEBUG_FSC, "inode %p get cookie %p\n",
73
-
inode, v9inode->fscache);
75
+
inode, v9fs_inode_cookie(v9inode));
74
76
}
+1
-3
fs/9p/v9fs.c
+1
-3
fs/9p/v9fs.c
···
623
623
static void v9fs_inode_init_once(void *foo)
624
624
{
625
625
struct v9fs_inode *v9inode = (struct v9fs_inode *)foo;
626
-
#ifdef CONFIG_9P_FSCACHE
627
-
v9inode->fscache = NULL;
628
-
#endif
626
+
629
627
memset(&v9inode->qid, 0, sizeof(v9inode->qid));
630
628
inode_init_once(&v9inode->vfs_inode);
631
629
}
+8
-5
fs/9p/v9fs.h
+8
-5
fs/9p/v9fs.h
···
9
9
#define FS_9P_V9FS_H
10
10
11
11
#include <linux/backing-dev.h>
12
+
#include <linux/netfs.h>
12
13
13
14
/**
14
15
* enum p9_session_flags - option flags for each 9P session
···
109
108
#define V9FS_INO_INVALID_ATTR 0x01
110
109
111
110
struct v9fs_inode {
112
-
#ifdef CONFIG_9P_FSCACHE
113
-
struct fscache_cookie *fscache;
114
-
#endif
111
+
struct {
112
+
/* These must be contiguous */
113
+
struct inode vfs_inode; /* the VFS's inode record */
114
+
struct netfs_i_context netfs_ctx; /* Netfslib context */
115
+
};
115
116
struct p9_qid qid;
116
117
unsigned int cache_validity;
117
118
struct p9_fid *writeback_fid;
118
119
struct mutex v_mutex;
119
-
struct inode vfs_inode;
120
120
};
121
121
122
122
static inline struct v9fs_inode *V9FS_I(const struct inode *inode)
···
128
126
static inline struct fscache_cookie *v9fs_inode_cookie(struct v9fs_inode *v9inode)
129
127
{
130
128
#ifdef CONFIG_9P_FSCACHE
131
-
return v9inode->fscache;
129
+
return netfs_i_cookie(&v9inode->vfs_inode);
132
130
#else
133
131
return NULL;
134
132
#endif
···
165
163
extern const struct inode_operations v9fs_dir_inode_operations_dotl;
166
164
extern const struct inode_operations v9fs_file_inode_operations_dotl;
167
165
extern const struct inode_operations v9fs_symlink_inode_operations_dotl;
166
+
extern const struct netfs_request_ops v9fs_req_ops;
168
167
extern struct inode *v9fs_inode_from_fid_dotl(struct v9fs_session_info *v9ses,
169
168
struct p9_fid *fid,
170
169
struct super_block *sb, int new);
+14
-48
fs/9p/vfs_addr.c
+14
-48
fs/9p/vfs_addr.c
···
28
28
#include "fid.h"
29
29
30
30
/**
31
-
* v9fs_req_issue_op - Issue a read from 9P
31
+
* v9fs_issue_read - Issue a read from 9P
32
32
* @subreq: The read to make
33
33
*/
34
-
static void v9fs_req_issue_op(struct netfs_read_subrequest *subreq)
34
+
static void v9fs_issue_read(struct netfs_io_subrequest *subreq)
35
35
{
36
-
struct netfs_read_request *rreq = subreq->rreq;
36
+
struct netfs_io_request *rreq = subreq->rreq;
37
37
struct p9_fid *fid = rreq->netfs_priv;
38
38
struct iov_iter to;
39
39
loff_t pos = subreq->start + subreq->transferred;
···
52
52
}
53
53
54
54
/**
55
-
* v9fs_init_rreq - Initialise a read request
55
+
* v9fs_init_request - Initialise a read request
56
56
* @rreq: The read request
57
57
* @file: The file being read from
58
58
*/
59
-
static void v9fs_init_rreq(struct netfs_read_request *rreq, struct file *file)
59
+
static int v9fs_init_request(struct netfs_io_request *rreq, struct file *file)
60
60
{
61
61
struct p9_fid *fid = file->private_data;
62
62
63
63
refcount_inc(&fid->count);
64
64
rreq->netfs_priv = fid;
65
+
return 0;
65
66
}
66
67
67
68
/**
68
-
* v9fs_req_cleanup - Cleanup request initialized by v9fs_init_rreq
69
+
* v9fs_req_cleanup - Cleanup request initialized by v9fs_init_request
69
70
* @mapping: unused mapping of request to cleanup
70
71
* @priv: private data to cleanup, a fid, guaranted non-null.
71
72
*/
···
78
77
}
79
78
80
79
/**
81
-
* v9fs_is_cache_enabled - Determine if caching is enabled for an inode
82
-
* @inode: The inode to check
83
-
*/
84
-
static bool v9fs_is_cache_enabled(struct inode *inode)
85
-
{
86
-
struct fscache_cookie *cookie = v9fs_inode_cookie(V9FS_I(inode));
87
-
88
-
return fscache_cookie_enabled(cookie) && cookie->cache_priv;
89
-
}
90
-
91
-
/**
92
80
* v9fs_begin_cache_operation - Begin a cache operation for a read
93
81
* @rreq: The read request
94
82
*/
95
-
static int v9fs_begin_cache_operation(struct netfs_read_request *rreq)
83
+
static int v9fs_begin_cache_operation(struct netfs_io_request *rreq)
96
84
{
97
85
#ifdef CONFIG_9P_FSCACHE
98
86
struct fscache_cookie *cookie = v9fs_inode_cookie(V9FS_I(rreq->inode));
···
92
102
#endif
93
103
}
94
104
95
-
static const struct netfs_read_request_ops v9fs_req_ops = {
96
-
.init_rreq = v9fs_init_rreq,
97
-
.is_cache_enabled = v9fs_is_cache_enabled,
105
+
const struct netfs_request_ops v9fs_req_ops = {
106
+
.init_request = v9fs_init_request,
98
107
.begin_cache_operation = v9fs_begin_cache_operation,
99
-
.issue_op = v9fs_req_issue_op,
108
+
.issue_read = v9fs_issue_read,
100
109
.cleanup = v9fs_req_cleanup,
101
110
};
102
-
103
-
/**
104
-
* v9fs_vfs_readpage - read an entire page in from 9P
105
-
* @file: file being read
106
-
* @page: structure to page
107
-
*
108
-
*/
109
-
static int v9fs_vfs_readpage(struct file *file, struct page *page)
110
-
{
111
-
struct folio *folio = page_folio(page);
112
-
113
-
return netfs_readpage(file, folio, &v9fs_req_ops, NULL);
114
-
}
115
-
116
-
/**
117
-
* v9fs_vfs_readahead - read a set of pages from 9P
118
-
* @ractl: The readahead parameters
119
-
*/
120
-
static void v9fs_vfs_readahead(struct readahead_control *ractl)
121
-
{
122
-
netfs_readahead(ractl, &v9fs_req_ops, NULL);
123
-
}
124
111
125
112
/**
126
113
* v9fs_release_page - release the private state associated with a page
···
275
308
* file. We need to do this before we get a lock on the page in case
276
309
* there's more than one writer competing for the same cache block.
277
310
*/
278
-
retval = netfs_write_begin(filp, mapping, pos, len, flags, &folio, fsdata,
279
-
&v9fs_req_ops, NULL);
311
+
retval = netfs_write_begin(filp, mapping, pos, len, flags, &folio, fsdata);
280
312
if (retval < 0)
281
313
return retval;
282
314
···
336
370
#endif
337
371
338
372
const struct address_space_operations v9fs_addr_operations = {
339
-
.readpage = v9fs_vfs_readpage,
340
-
.readahead = v9fs_vfs_readahead,
373
+
.readpage = netfs_readpage,
374
+
.readahead = netfs_readahead,
341
375
.dirty_folio = v9fs_dirty_folio,
342
376
.writepage = v9fs_vfs_writepage,
343
377
.write_begin = v9fs_write_begin,
+10
-3
fs/9p/vfs_inode.c
+10
-3
fs/9p/vfs_inode.c
···
231
231
v9inode = alloc_inode_sb(sb, v9fs_inode_cache, GFP_KERNEL);
232
232
if (!v9inode)
233
233
return NULL;
234
-
#ifdef CONFIG_9P_FSCACHE
235
-
v9inode->fscache = NULL;
236
-
#endif
237
234
v9inode->writeback_fid = NULL;
238
235
v9inode->cache_validity = 0;
239
236
mutex_init(&v9inode->v_mutex);
···
245
248
void v9fs_free_inode(struct inode *inode)
246
249
{
247
250
kmem_cache_free(v9fs_inode_cache, V9FS_I(inode));
251
+
}
252
+
253
+
/*
254
+
* Set parameters for the netfs library
255
+
*/
256
+
static void v9fs_set_netfs_context(struct inode *inode)
257
+
{
258
+
netfs_i_context_init(inode, &v9fs_req_ops);
248
259
}
249
260
250
261
int v9fs_init_inode(struct v9fs_session_info *v9ses,
···
343
338
err = -EINVAL;
344
339
goto error;
345
340
}
341
+
342
+
v9fs_set_netfs_context(inode);
346
343
error:
347
344
return err;
348
345
+1
fs/afs/dynroot.c
+1
fs/afs/dynroot.c
+10
-31
fs/afs/file.c
+10
-31
fs/afs/file.c
···
19
19
#include "internal.h"
20
20
21
21
static int afs_file_mmap(struct file *file, struct vm_area_struct *vma);
22
-
static int afs_readpage(struct file *file, struct page *page);
23
22
static int afs_symlink_readpage(struct file *file, struct page *page);
24
23
static void afs_invalidate_folio(struct folio *folio, size_t offset,
25
24
size_t length);
26
25
static int afs_releasepage(struct page *page, gfp_t gfp_flags);
27
26
28
-
static void afs_readahead(struct readahead_control *ractl);
29
27
static ssize_t afs_file_read_iter(struct kiocb *iocb, struct iov_iter *iter);
30
28
static void afs_vm_open(struct vm_area_struct *area);
31
29
static void afs_vm_close(struct vm_area_struct *area);
···
50
52
};
51
53
52
54
const struct address_space_operations afs_file_aops = {
53
-
.readpage = afs_readpage,
54
-
.readahead = afs_readahead,
55
+
.readpage = netfs_readpage,
56
+
.readahead = netfs_readahead,
55
57
.dirty_folio = afs_dirty_folio,
56
58
.launder_folio = afs_launder_folio,
57
59
.releasepage = afs_releasepage,
···
238
240
static void afs_fetch_data_notify(struct afs_operation *op)
239
241
{
240
242
struct afs_read *req = op->fetch.req;
241
-
struct netfs_read_subrequest *subreq = req->subreq;
243
+
struct netfs_io_subrequest *subreq = req->subreq;
242
244
int error = op->error;
243
245
244
246
if (error == -ECONNABORTED)
···
308
310
return afs_do_sync_operation(op);
309
311
}
310
312
311
-
static void afs_req_issue_op(struct netfs_read_subrequest *subreq)
313
+
static void afs_issue_read(struct netfs_io_subrequest *subreq)
312
314
{
313
315
struct afs_vnode *vnode = AFS_FS_I(subreq->rreq->inode);
314
316
struct afs_read *fsreq;
···
357
359
return ret;
358
360
}
359
361
360
-
static void afs_init_rreq(struct netfs_read_request *rreq, struct file *file)
362
+
static int afs_init_request(struct netfs_io_request *rreq, struct file *file)
361
363
{
362
364
rreq->netfs_priv = key_get(afs_file_key(file));
365
+
return 0;
363
366
}
364
367
365
-
static bool afs_is_cache_enabled(struct inode *inode)
366
-
{
367
-
struct fscache_cookie *cookie = afs_vnode_cache(AFS_FS_I(inode));
368
-
369
-
return fscache_cookie_enabled(cookie) && cookie->cache_priv;
370
-
}
371
-
372
-
static int afs_begin_cache_operation(struct netfs_read_request *rreq)
368
+
static int afs_begin_cache_operation(struct netfs_io_request *rreq)
373
369
{
374
370
#ifdef CONFIG_AFS_FSCACHE
375
371
struct afs_vnode *vnode = AFS_FS_I(rreq->inode);
···
388
396
key_put(netfs_priv);
389
397
}
390
398
391
-
const struct netfs_read_request_ops afs_req_ops = {
392
-
.init_rreq = afs_init_rreq,
393
-
.is_cache_enabled = afs_is_cache_enabled,
399
+
const struct netfs_request_ops afs_req_ops = {
400
+
.init_request = afs_init_request,
394
401
.begin_cache_operation = afs_begin_cache_operation,
395
402
.check_write_begin = afs_check_write_begin,
396
-
.issue_op = afs_req_issue_op,
403
+
.issue_read = afs_issue_read,
397
404
.cleanup = afs_priv_cleanup,
398
405
};
399
-
400
-
static int afs_readpage(struct file *file, struct page *page)
401
-
{
402
-
struct folio *folio = page_folio(page);
403
-
404
-
return netfs_readpage(file, folio, &afs_req_ops, NULL);
405
-
}
406
-
407
-
static void afs_readahead(struct readahead_control *ractl)
408
-
{
409
-
netfs_readahead(ractl, &afs_req_ops, NULL);
410
-
}
411
406
412
407
int afs_write_inode(struct inode *inode, struct writeback_control *wbc)
413
408
{
+21
-11
fs/afs/inode.c
+21
-11
fs/afs/inode.c
···
54
54
}
55
55
56
56
/*
57
+
* Set parameters for the netfs library
58
+
*/
59
+
static void afs_set_netfs_context(struct afs_vnode *vnode)
60
+
{
61
+
netfs_i_context_init(&vnode->vfs_inode, &afs_req_ops);
62
+
}
63
+
64
+
/*
57
65
* Initialise an inode from the vnode status.
58
66
*/
59
67
static int afs_inode_init_from_status(struct afs_operation *op,
···
136
128
}
137
129
138
130
afs_set_i_size(vnode, status->size);
131
+
afs_set_netfs_context(vnode);
139
132
140
133
vnode->invalid_before = status->data_version;
141
134
inode_set_iversion_raw(&vnode->vfs_inode, status->data_version);
···
246
237
* idea of what the size should be that's not the same as
247
238
* what's on the server.
248
239
*/
240
+
vnode->netfs_ctx.remote_i_size = status->size;
249
241
if (change_size) {
250
242
afs_set_i_size(vnode, status->size);
251
243
inode->i_ctime = t;
···
430
420
struct afs_vnode_cache_aux aux;
431
421
432
422
if (vnode->status.type != AFS_FTYPE_FILE) {
433
-
vnode->cache = NULL;
423
+
vnode->netfs_ctx.cache = NULL;
434
424
return;
435
425
}
436
426
···
440
430
key.vnode_id_ext[1] = htonl(vnode->fid.vnode_hi);
441
431
afs_set_cache_aux(vnode, &aux);
442
432
443
-
vnode->cache = fscache_acquire_cookie(
444
-
vnode->volume->cache,
445
-
vnode->status.type == AFS_FTYPE_FILE ? 0 : FSCACHE_ADV_SINGLE_CHUNK,
446
-
&key, sizeof(key),
447
-
&aux, sizeof(aux),
448
-
vnode->status.size);
433
+
afs_vnode_set_cache(vnode,
434
+
fscache_acquire_cookie(
435
+
vnode->volume->cache,
436
+
vnode->status.type == AFS_FTYPE_FILE ?
437
+
0 : FSCACHE_ADV_SINGLE_CHUNK,
438
+
&key, sizeof(key),
439
+
&aux, sizeof(aux),
440
+
vnode->status.size));
449
441
#endif
450
442
}
451
443
···
540
528
541
529
vnode = AFS_FS_I(inode);
542
530
vnode->cb_v_break = as->volume->cb_v_break,
531
+
afs_set_netfs_context(vnode);
543
532
544
533
op = afs_alloc_operation(key, as->volume);
545
534
if (IS_ERR(op)) {
···
799
786
afs_put_wb_key(wbk);
800
787
}
801
788
802
-
#ifdef CONFIG_AFS_FSCACHE
803
-
fscache_relinquish_cookie(vnode->cache,
789
+
fscache_relinquish_cookie(afs_vnode_cache(vnode),
804
790
test_bit(AFS_VNODE_DELETED, &vnode->flags));
805
-
vnode->cache = NULL;
806
-
#endif
807
791
808
792
afs_prune_wb_keys(vnode);
809
793
afs_put_permits(rcu_access_pointer(vnode->permit_cache));
+16
-7
fs/afs/internal.h
+16
-7
fs/afs/internal.h
···
207
207
loff_t file_size; /* File size returned by server */
208
208
struct key *key; /* The key to use to reissue the read */
209
209
struct afs_vnode *vnode; /* The file being read into. */
210
-
struct netfs_read_subrequest *subreq; /* Fscache helper read request this belongs to */
210
+
struct netfs_io_subrequest *subreq; /* Fscache helper read request this belongs to */
211
211
afs_dataversion_t data_version; /* Version number returned by server */
212
212
refcount_t usage;
213
213
unsigned int call_debug_id;
···
619
619
* leak from one inode to another.
620
620
*/
621
621
struct afs_vnode {
622
-
struct inode vfs_inode; /* the VFS's inode record */
622
+
struct {
623
+
/* These must be contiguous */
624
+
struct inode vfs_inode; /* the VFS's inode record */
625
+
struct netfs_i_context netfs_ctx; /* Netfslib context */
626
+
};
623
627
624
628
struct afs_volume *volume; /* volume on which vnode resides */
625
629
struct afs_fid fid; /* the file identifier for this inode */
626
630
struct afs_file_status status; /* AFS status info for this file */
627
631
afs_dataversion_t invalid_before; /* Child dentries are invalid before this */
628
-
#ifdef CONFIG_AFS_FSCACHE
629
-
struct fscache_cookie *cache; /* caching cookie */
630
-
#endif
631
632
struct afs_permits __rcu *permit_cache; /* cache of permits so far obtained */
632
633
struct mutex io_lock; /* Lock for serialising I/O on this mutex */
633
634
struct rw_semaphore validate_lock; /* lock for validating this vnode */
···
675
674
static inline struct fscache_cookie *afs_vnode_cache(struct afs_vnode *vnode)
676
675
{
677
676
#ifdef CONFIG_AFS_FSCACHE
678
-
return vnode->cache;
677
+
return netfs_i_cookie(&vnode->vfs_inode);
679
678
#else
680
679
return NULL;
680
+
#endif
681
+
}
682
+
683
+
static inline void afs_vnode_set_cache(struct afs_vnode *vnode,
684
+
struct fscache_cookie *cookie)
685
+
{
686
+
#ifdef CONFIG_AFS_FSCACHE
687
+
vnode->netfs_ctx.cache = cookie;
681
688
#endif
682
689
}
683
690
···
1072
1063
extern const struct address_space_operations afs_symlink_aops;
1073
1064
extern const struct inode_operations afs_file_inode_operations;
1074
1065
extern const struct file_operations afs_file_operations;
1075
-
extern const struct netfs_read_request_ops afs_req_ops;
1066
+
extern const struct netfs_request_ops afs_req_ops;
1076
1067
1077
1068
extern int afs_cache_wb_key(struct afs_vnode *, struct afs_file *);
1078
1069
extern void afs_put_wb_key(struct afs_wb_key *);
+1
-3
fs/afs/super.c
+1
-3
fs/afs/super.c
···
688
688
/* Reset anything that shouldn't leak from one inode to the next. */
689
689
memset(&vnode->fid, 0, sizeof(vnode->fid));
690
690
memset(&vnode->status, 0, sizeof(vnode->status));
691
+
afs_vnode_set_cache(vnode, NULL);
691
692
692
693
vnode->volume = NULL;
693
694
vnode->lock_key = NULL;
694
695
vnode->permit_cache = NULL;
695
-
#ifdef CONFIG_AFS_FSCACHE
696
-
vnode->cache = NULL;
697
-
#endif
698
696
699
697
vnode->flags = 1 << AFS_VNODE_UNSET;
700
698
vnode->lock_state = AFS_VNODE_LOCK_NONE;
+4
-6
fs/afs/write.c
+4
-6
fs/afs/write.c
···
60
60
* file. We need to do this before we get a lock on the page in case
61
61
* there's more than one writer competing for the same cache block.
62
62
*/
63
-
ret = netfs_write_begin(file, mapping, pos, len, flags, &folio, fsdata,
64
-
&afs_req_ops, NULL);
63
+
ret = netfs_write_begin(file, mapping, pos, len, flags, &folio, fsdata);
65
64
if (ret < 0)
66
65
return ret;
67
66
···
354
355
static int afs_store_data(struct afs_vnode *vnode, struct iov_iter *iter, loff_t pos,
355
356
bool laundering)
356
357
{
358
+
struct netfs_i_context *ictx = &vnode->netfs_ctx;
357
359
struct afs_operation *op;
358
360
struct afs_wb_key *wbk = NULL;
359
-
loff_t size = iov_iter_count(iter), i_size;
361
+
loff_t size = iov_iter_count(iter);
360
362
int ret = -ENOKEY;
361
363
362
364
_enter("%s{%llx:%llu.%u},%llx,%llx",
···
379
379
return -ENOMEM;
380
380
}
381
381
382
-
i_size = i_size_read(&vnode->vfs_inode);
383
-
384
382
afs_op_set_vnode(op, 0, vnode);
385
383
op->file[0].dv_delta = 1;
386
384
op->file[0].modification = true;
387
385
op->store.write_iter = iter;
388
386
op->store.pos = pos;
389
387
op->store.size = size;
390
-
op->store.i_size = max(pos + size, i_size);
388
+
op->store.i_size = max(pos + size, ictx->remote_i_size);
391
389
op->store.laundering = laundering;
392
390
op->mtime = vnode->vfs_inode.i_mtime;
393
391
op->flags |= AFS_OPERATION_UNINTR;
+5
-5
fs/cachefiles/io.c
+5
-5
fs/cachefiles/io.c
···
380
380
* Prepare a read operation, shortening it to a cached/uncached
381
381
* boundary as appropriate.
382
382
*/
383
-
static enum netfs_read_source cachefiles_prepare_read(struct netfs_read_subrequest *subreq,
383
+
static enum netfs_io_source cachefiles_prepare_read(struct netfs_io_subrequest *subreq,
384
384
loff_t i_size)
385
385
{
386
386
enum cachefiles_prepare_read_trace why;
387
-
struct netfs_read_request *rreq = subreq->rreq;
387
+
struct netfs_io_request *rreq = subreq->rreq;
388
388
struct netfs_cache_resources *cres = &rreq->cache_resources;
389
389
struct cachefiles_object *object;
390
390
struct cachefiles_cache *cache;
391
391
struct fscache_cookie *cookie = fscache_cres_cookie(cres);
392
392
const struct cred *saved_cred;
393
393
struct file *file = cachefiles_cres_file(cres);
394
-
enum netfs_read_source ret = NETFS_DOWNLOAD_FROM_SERVER;
394
+
enum netfs_io_source ret = NETFS_DOWNLOAD_FROM_SERVER;
395
395
loff_t off, to;
396
396
ino_t ino = file ? file_inode(file)->i_ino : 0;
397
397
···
404
404
}
405
405
406
406
if (test_bit(FSCACHE_COOKIE_NO_DATA_TO_READ, &cookie->flags)) {
407
-
__set_bit(NETFS_SREQ_WRITE_TO_CACHE, &subreq->flags);
407
+
__set_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
408
408
why = cachefiles_trace_read_no_data;
409
409
goto out_no_object;
410
410
}
···
473
473
goto out;
474
474
475
475
download_and_store:
476
-
__set_bit(NETFS_SREQ_WRITE_TO_CACHE, &subreq->flags);
476
+
__set_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
477
477
out:
478
478
cachefiles_end_secure(cache, saved_cred);
479
479
out_no_object:
+53
-63
fs/ceph/addr.c
+53
-63
fs/ceph/addr.c
···
182
182
return 1;
183
183
}
184
184
185
-
static void ceph_netfs_expand_readahead(struct netfs_read_request *rreq)
185
+
static void ceph_netfs_expand_readahead(struct netfs_io_request *rreq)
186
186
{
187
187
struct inode *inode = rreq->inode;
188
188
struct ceph_inode_info *ci = ceph_inode(inode);
···
199
199
rreq->len = roundup(rreq->len, lo->stripe_unit);
200
200
}
201
201
202
-
static bool ceph_netfs_clamp_length(struct netfs_read_subrequest *subreq)
202
+
static bool ceph_netfs_clamp_length(struct netfs_io_subrequest *subreq)
203
203
{
204
204
struct inode *inode = subreq->rreq->inode;
205
205
struct ceph_fs_client *fsc = ceph_inode_to_client(inode);
···
218
218
{
219
219
struct ceph_fs_client *fsc = ceph_inode_to_client(req->r_inode);
220
220
struct ceph_osd_data *osd_data = osd_req_op_extent_osd_data(req, 0);
221
-
struct netfs_read_subrequest *subreq = req->r_priv;
221
+
struct netfs_io_subrequest *subreq = req->r_priv;
222
222
int num_pages;
223
223
int err = req->r_result;
224
224
···
244
244
iput(req->r_inode);
245
245
}
246
246
247
-
static bool ceph_netfs_issue_op_inline(struct netfs_read_subrequest *subreq)
247
+
static bool ceph_netfs_issue_op_inline(struct netfs_io_subrequest *subreq)
248
248
{
249
-
struct netfs_read_request *rreq = subreq->rreq;
249
+
struct netfs_io_request *rreq = subreq->rreq;
250
250
struct inode *inode = rreq->inode;
251
251
struct ceph_mds_reply_info_parsed *rinfo;
252
252
struct ceph_mds_reply_info_in *iinfo;
···
258
258
size_t len;
259
259
260
260
__set_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags);
261
-
__clear_bit(NETFS_SREQ_WRITE_TO_CACHE, &subreq->flags);
261
+
__clear_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
262
262
263
263
if (subreq->start >= inode->i_size)
264
264
goto out;
···
297
297
return true;
298
298
}
299
299
300
-
static void ceph_netfs_issue_op(struct netfs_read_subrequest *subreq)
300
+
static void ceph_netfs_issue_read(struct netfs_io_subrequest *subreq)
301
301
{
302
-
struct netfs_read_request *rreq = subreq->rreq;
302
+
struct netfs_io_request *rreq = subreq->rreq;
303
303
struct inode *inode = rreq->inode;
304
304
struct ceph_inode_info *ci = ceph_inode(inode);
305
305
struct ceph_fs_client *fsc = ceph_inode_to_client(inode);
···
353
353
dout("%s: result %d\n", __func__, err);
354
354
}
355
355
356
+
static int ceph_init_request(struct netfs_io_request *rreq, struct file *file)
357
+
{
358
+
struct inode *inode = rreq->inode;
359
+
int got = 0, want = CEPH_CAP_FILE_CACHE;
360
+
int ret = 0;
361
+
362
+
if (rreq->origin != NETFS_READAHEAD)
363
+
return 0;
364
+
365
+
if (file) {
366
+
struct ceph_rw_context *rw_ctx;
367
+
struct ceph_file_info *fi = file->private_data;
368
+
369
+
rw_ctx = ceph_find_rw_context(fi);
370
+
if (rw_ctx)
371
+
return 0;
372
+
}
373
+
374
+
/*
375
+
* readahead callers do not necessarily hold Fcb caps
376
+
* (e.g. fadvise, madvise).
377
+
*/
378
+
ret = ceph_try_get_caps(inode, CEPH_CAP_FILE_RD, want, true, &got);
379
+
if (ret < 0) {
380
+
dout("start_read %p, error getting cap\n", inode);
381
+
return ret;
382
+
}
383
+
384
+
if (!(got & want)) {
385
+
dout("start_read %p, no cache cap\n", inode);
386
+
return -EACCES;
387
+
}
388
+
if (ret == 0)
389
+
return -EACCES;
390
+
391
+
rreq->netfs_priv = (void *)(uintptr_t)got;
392
+
return 0;
393
+
}
394
+
356
395
static void ceph_readahead_cleanup(struct address_space *mapping, void *priv)
357
396
{
358
397
struct inode *inode = mapping->host;
···
402
363
ceph_put_cap_refs(ci, got);
403
364
}
404
365
405
-
static const struct netfs_read_request_ops ceph_netfs_read_ops = {
406
-
.is_cache_enabled = ceph_is_cache_enabled,
366
+
const struct netfs_request_ops ceph_netfs_ops = {
367
+
.init_request = ceph_init_request,
407
368
.begin_cache_operation = ceph_begin_cache_operation,
408
-
.issue_op = ceph_netfs_issue_op,
369
+
.issue_read = ceph_netfs_issue_read,
409
370
.expand_readahead = ceph_netfs_expand_readahead,
410
371
.clamp_length = ceph_netfs_clamp_length,
411
372
.check_write_begin = ceph_netfs_check_write_begin,
412
373
.cleanup = ceph_readahead_cleanup,
413
374
};
414
-
415
-
/* read a single page, without unlocking it. */
416
-
static int ceph_readpage(struct file *file, struct page *subpage)
417
-
{
418
-
struct folio *folio = page_folio(subpage);
419
-
struct inode *inode = file_inode(file);
420
-
struct ceph_inode_info *ci = ceph_inode(inode);
421
-
struct ceph_vino vino = ceph_vino(inode);
422
-
size_t len = folio_size(folio);
423
-
u64 off = folio_file_pos(folio);
424
-
425
-
dout("readpage ino %llx.%llx file %p off %llu len %zu folio %p index %lu\n inline %d",
426
-
vino.ino, vino.snap, file, off, len, folio, folio_index(folio),
427
-
ci->i_inline_version != CEPH_INLINE_NONE);
428
-
429
-
return netfs_readpage(file, folio, &ceph_netfs_read_ops, NULL);
430
-
}
431
-
432
-
static void ceph_readahead(struct readahead_control *ractl)
433
-
{
434
-
struct inode *inode = file_inode(ractl->file);
435
-
struct ceph_file_info *fi = ractl->file->private_data;
436
-
struct ceph_rw_context *rw_ctx;
437
-
int got = 0;
438
-
int ret = 0;
439
-
440
-
if (ceph_inode(inode)->i_inline_version != CEPH_INLINE_NONE)
441
-
return;
442
-
443
-
rw_ctx = ceph_find_rw_context(fi);
444
-
if (!rw_ctx) {
445
-
/*
446
-
* readahead callers do not necessarily hold Fcb caps
447
-
* (e.g. fadvise, madvise).
448
-
*/
449
-
int want = CEPH_CAP_FILE_CACHE;
450
-
451
-
ret = ceph_try_get_caps(inode, CEPH_CAP_FILE_RD, want, true, &got);
452
-
if (ret < 0)
453
-
dout("start_read %p, error getting cap\n", inode);
454
-
else if (!(got & want))
455
-
dout("start_read %p, no cache cap\n", inode);
456
-
457
-
if (ret <= 0)
458
-
return;
459
-
}
460
-
netfs_readahead(ractl, &ceph_netfs_read_ops, (void *)(uintptr_t)got);
461
-
}
462
375
463
376
#ifdef CONFIG_CEPH_FSCACHE
464
377
static void ceph_set_page_fscache(struct page *page)
···
1318
1327
struct folio *folio = NULL;
1319
1328
int r;
1320
1329
1321
-
r = netfs_write_begin(file, inode->i_mapping, pos, len, 0, &folio, NULL,
1322
-
&ceph_netfs_read_ops, NULL);
1330
+
r = netfs_write_begin(file, inode->i_mapping, pos, len, 0, &folio, NULL);
1323
1331
if (r == 0)
1324
1332
folio_wait_fscache(folio);
1325
1333
if (r < 0) {
···
1372
1382
}
1373
1383
1374
1384
const struct address_space_operations ceph_aops = {
1375
-
.readpage = ceph_readpage,
1376
-
.readahead = ceph_readahead,
1385
+
.readpage = netfs_readpage,
1386
+
.readahead = netfs_readahead,
1377
1387
.writepage = ceph_writepage,
1378
1388
.writepages = ceph_writepages_start,
1379
1389
.write_begin = ceph_write_begin,
+14
-14
fs/ceph/cache.c
+14
-14
fs/ceph/cache.c
···
29
29
if (!(inode->i_state & I_NEW))
30
30
return;
31
31
32
-
WARN_ON_ONCE(ci->fscache);
32
+
WARN_ON_ONCE(ci->netfs_ctx.cache);
33
33
34
-
ci->fscache = fscache_acquire_cookie(fsc->fscache, 0,
35
-
&ci->i_vino, sizeof(ci->i_vino),
36
-
&ci->i_version, sizeof(ci->i_version),
37
-
i_size_read(inode));
34
+
ci->netfs_ctx.cache =
35
+
fscache_acquire_cookie(fsc->fscache, 0,
36
+
&ci->i_vino, sizeof(ci->i_vino),
37
+
&ci->i_version, sizeof(ci->i_version),
38
+
i_size_read(inode));
38
39
}
39
40
40
-
void ceph_fscache_unregister_inode_cookie(struct ceph_inode_info* ci)
41
+
void ceph_fscache_unregister_inode_cookie(struct ceph_inode_info *ci)
41
42
{
42
-
struct fscache_cookie *cookie = ci->fscache;
43
-
44
-
fscache_relinquish_cookie(cookie, false);
43
+
fscache_relinquish_cookie(ceph_fscache_cookie(ci), false);
45
44
}
46
45
47
46
void ceph_fscache_use_cookie(struct inode *inode, bool will_modify)
48
47
{
49
48
struct ceph_inode_info *ci = ceph_inode(inode);
50
49
51
-
fscache_use_cookie(ci->fscache, will_modify);
50
+
fscache_use_cookie(ceph_fscache_cookie(ci), will_modify);
52
51
}
53
52
54
53
void ceph_fscache_unuse_cookie(struct inode *inode, bool update)
···
57
58
if (update) {
58
59
loff_t i_size = i_size_read(inode);
59
60
60
-
fscache_unuse_cookie(ci->fscache, &ci->i_version, &i_size);
61
+
fscache_unuse_cookie(ceph_fscache_cookie(ci),
62
+
&ci->i_version, &i_size);
61
63
} else {
62
-
fscache_unuse_cookie(ci->fscache, NULL, NULL);
64
+
fscache_unuse_cookie(ceph_fscache_cookie(ci), NULL, NULL);
63
65
}
64
66
}
65
67
···
69
69
struct ceph_inode_info *ci = ceph_inode(inode);
70
70
loff_t i_size = i_size_read(inode);
71
71
72
-
fscache_update_cookie(ci->fscache, &ci->i_version, &i_size);
72
+
fscache_update_cookie(ceph_fscache_cookie(ci), &ci->i_version, &i_size);
73
73
}
74
74
75
75
void ceph_fscache_invalidate(struct inode *inode, bool dio_write)
76
76
{
77
77
struct ceph_inode_info *ci = ceph_inode(inode);
78
78
79
-
fscache_invalidate(ceph_inode(inode)->fscache,
79
+
fscache_invalidate(ceph_fscache_cookie(ci),
80
80
&ci->i_version, i_size_read(inode),
81
81
dio_write ? FSCACHE_INVAL_DIO_WRITE : 0);
82
82
}
+3
-12
fs/ceph/cache.h
+3
-12
fs/ceph/cache.h
···
26
26
void ceph_fscache_update(struct inode *inode);
27
27
void ceph_fscache_invalidate(struct inode *inode, bool dio_write);
28
28
29
-
static inline void ceph_fscache_inode_init(struct ceph_inode_info *ci)
30
-
{
31
-
ci->fscache = NULL;
32
-
}
33
-
34
29
static inline struct fscache_cookie *ceph_fscache_cookie(struct ceph_inode_info *ci)
35
30
{
36
-
return ci->fscache;
31
+
return netfs_i_cookie(&ci->vfs_inode);
37
32
}
38
33
39
34
static inline void ceph_fscache_resize(struct inode *inode, loff_t to)
···
57
62
return fscache_dirty_folio(mapping, folio, ceph_fscache_cookie(ci));
58
63
}
59
64
60
-
static inline int ceph_begin_cache_operation(struct netfs_read_request *rreq)
65
+
static inline int ceph_begin_cache_operation(struct netfs_io_request *rreq)
61
66
{
62
67
struct fscache_cookie *cookie = ceph_fscache_cookie(ceph_inode(rreq->inode));
63
68
···
83
88
}
84
89
85
90
static inline void ceph_fscache_unregister_fs(struct ceph_fs_client* fsc)
86
-
{
87
-
}
88
-
89
-
static inline void ceph_fscache_inode_init(struct ceph_inode_info *ci)
90
91
{
91
92
}
92
93
···
135
144
return false;
136
145
}
137
146
138
-
static inline int ceph_begin_cache_operation(struct netfs_read_request *rreq)
147
+
static inline int ceph_begin_cache_operation(struct netfs_io_request *rreq)
139
148
{
140
149
return -ENOBUFS;
141
150
}
+3
-3
fs/ceph/inode.c
+3
-3
fs/ceph/inode.c
···
459
459
460
460
dout("alloc_inode %p\n", &ci->vfs_inode);
461
461
462
+
/* Set parameters for the netfs library */
463
+
netfs_i_context_init(&ci->vfs_inode, &ceph_netfs_ops);
464
+
462
465
spin_lock_init(&ci->i_ceph_lock);
463
466
464
467
ci->i_version = 0;
···
547
544
INIT_WORK(&ci->i_work, ceph_inode_work);
548
545
ci->i_work_mask = 0;
549
546
memset(&ci->i_btime, '\0', sizeof(ci->i_btime));
550
-
551
-
ceph_fscache_inode_init(ci);
552
-
553
547
return &ci->vfs_inode;
554
548
}
555
549
+8
-9
fs/ceph/super.h
+8
-9
fs/ceph/super.h
···
17
17
#include <linux/posix_acl.h>
18
18
#include <linux/refcount.h>
19
19
#include <linux/security.h>
20
+
#include <linux/netfs.h>
21
+
#include <linux/fscache.h>
20
22
21
23
#include <linux/ceph/libceph.h>
22
-
23
-
#ifdef CONFIG_CEPH_FSCACHE
24
-
#include <linux/fscache.h>
25
-
#endif
26
24
27
25
/* large granularity for statfs utilization stats to facilitate
28
26
* large volume sizes on 32-bit machines. */
···
316
318
* Ceph inode.
317
319
*/
318
320
struct ceph_inode_info {
321
+
struct {
322
+
/* These must be contiguous */
323
+
struct inode vfs_inode;
324
+
struct netfs_i_context netfs_ctx; /* Netfslib context */
325
+
};
319
326
struct ceph_vino i_vino; /* ceph ino + snap */
320
327
321
328
spinlock_t i_ceph_lock;
···
431
428
432
429
struct work_struct i_work;
433
430
unsigned long i_work_mask;
434
-
435
-
#ifdef CONFIG_CEPH_FSCACHE
436
-
struct fscache_cookie *fscache;
437
-
#endif
438
-
struct inode vfs_inode; /* at end */
439
431
};
440
432
441
433
static inline struct ceph_inode_info *
···
1214
1216
1215
1217
/* addr.c */
1216
1218
extern const struct address_space_operations ceph_aops;
1219
+
extern const struct netfs_request_ops ceph_netfs_ops;
1217
1220
extern int ceph_mmap(struct file *file, struct vm_area_struct *vma);
1218
1221
extern int ceph_uninline_data(struct file *file);
1219
1222
extern int ceph_pool_perm_check(struct inode *inode, int need);
+6
-4
fs/cifs/cifsglob.h
+6
-4
fs/cifs/cifsglob.h
···
16
16
#include <linux/mempool.h>
17
17
#include <linux/workqueue.h>
18
18
#include <linux/utsname.h>
19
+
#include <linux/netfs.h>
19
20
#include "cifs_fs_sb.h"
20
21
#include "cifsacl.h"
21
22
#include <crypto/internal/hash.h>
···
1403
1402
*/
1404
1403
1405
1404
struct cifsInodeInfo {
1405
+
struct {
1406
+
/* These must be contiguous */
1407
+
struct inode vfs_inode; /* the VFS's inode record */
1408
+
struct netfs_i_context netfs_ctx; /* Netfslib context */
1409
+
};
1406
1410
bool can_cache_brlcks;
1407
1411
struct list_head llist; /* locks helb by this inode */
1408
1412
/*
···
1438
1432
u64 uniqueid; /* server inode number */
1439
1433
u64 createtime; /* creation time on server */
1440
1434
__u8 lease_key[SMB2_LEASE_KEY_SIZE]; /* lease key for this inode */
1441
-
#ifdef CONFIG_CIFS_FSCACHE
1442
-
struct fscache_cookie *fscache;
1443
-
#endif
1444
-
struct inode vfs_inode;
1445
1435
struct list_head deferred_closes; /* list of deferred closes */
1446
1436
spinlock_t deferred_lock; /* protection on deferred list */
1447
1437
bool lease_granted; /* Flag to indicate whether lease or oplock is granted. */
+6
-13
fs/cifs/fscache.c
+6
-13
fs/cifs/fscache.c
···
103
103
104
104
cifs_fscache_fill_coherency(&cifsi->vfs_inode, &cd);
105
105
106
-
cifsi->fscache =
106
+
cifsi->netfs_ctx.cache =
107
107
fscache_acquire_cookie(tcon->fscache, 0,
108
108
&cifsi->uniqueid, sizeof(cifsi->uniqueid),
109
109
&cd, sizeof(cd),
···
126
126
void cifs_fscache_release_inode_cookie(struct inode *inode)
127
127
{
128
128
struct cifsInodeInfo *cifsi = CIFS_I(inode);
129
+
struct fscache_cookie *cookie = cifs_inode_cookie(inode);
129
130
130
-
if (cifsi->fscache) {
131
-
cifs_dbg(FYI, "%s: (0x%p)\n", __func__, cifsi->fscache);
132
-
fscache_relinquish_cookie(cifsi->fscache, false);
133
-
cifsi->fscache = NULL;
131
+
if (cookie) {
132
+
cifs_dbg(FYI, "%s: (0x%p)\n", __func__, cookie);
133
+
fscache_relinquish_cookie(cookie, false);
134
+
cifsi->netfs_ctx.cache = NULL;
134
135
}
135
-
}
136
-
137
-
static inline void fscache_end_operation(struct netfs_cache_resources *cres)
138
-
{
139
-
const struct netfs_cache_ops *ops = fscache_operation_valid(cres);
140
-
141
-
if (ops)
142
-
ops->end_operation(cres);
143
136
}
144
137
145
138
/*
+1
-1
fs/cifs/fscache.h
+1
-1
fs/cifs/fscache.h
-11
fs/fscache/internal.h
-11
fs/fscache/internal.h
···
71
71
}
72
72
73
73
/*
74
-
* io.c
75
-
*/
76
-
static inline void fscache_end_operation(struct netfs_cache_resources *cres)
77
-
{
78
-
const struct netfs_cache_ops *ops = fscache_operation_valid(cres);
79
-
80
-
if (ops)
81
-
ops->end_operation(cres);
82
-
}
83
-
84
-
/*
85
74
* main.c
86
75
*/
87
76
extern unsigned fscache_debug;
+7
-1
fs/netfs/Makefile
+7
-1
fs/netfs/Makefile
+428
fs/netfs/buffered_read.c
+428
fs/netfs/buffered_read.c
···
1
+
// SPDX-License-Identifier: GPL-2.0-or-later
2
+
/* Network filesystem high-level buffered read support.
3
+
*
4
+
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
5
+
* Written by David Howells (dhowells@redhat.com)
6
+
*/
7
+
8
+
#include <linux/export.h>
9
+
#include <linux/task_io_accounting_ops.h>
10
+
#include "internal.h"
11
+
12
+
/*
13
+
* Unlock the folios in a read operation. We need to set PG_fscache on any
14
+
* folios we're going to write back before we unlock them.
15
+
*/
16
+
void netfs_rreq_unlock_folios(struct netfs_io_request *rreq)
17
+
{
18
+
struct netfs_io_subrequest *subreq;
19
+
struct folio *folio;
20
+
unsigned int iopos, account = 0;
21
+
pgoff_t start_page = rreq->start / PAGE_SIZE;
22
+
pgoff_t last_page = ((rreq->start + rreq->len) / PAGE_SIZE) - 1;
23
+
bool subreq_failed = false;
24
+
25
+
XA_STATE(xas, &rreq->mapping->i_pages, start_page);
26
+
27
+
if (test_bit(NETFS_RREQ_FAILED, &rreq->flags)) {
28
+
__clear_bit(NETFS_RREQ_COPY_TO_CACHE, &rreq->flags);
29
+
list_for_each_entry(subreq, &rreq->subrequests, rreq_link) {
30
+
__clear_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags);
31
+
}
32
+
}
33
+
34
+
/* Walk through the pagecache and the I/O request lists simultaneously.
35
+
* We may have a mixture of cached and uncached sections and we only
36
+
* really want to write out the uncached sections. This is slightly
37
+
* complicated by the possibility that we might have huge pages with a
38
+
* mixture inside.
39
+
*/
40
+
subreq = list_first_entry(&rreq->subrequests,
41
+
struct netfs_io_subrequest, rreq_link);
42
+
iopos = 0;
43
+
subreq_failed = (subreq->error < 0);
44
+
45
+
trace_netfs_rreq(rreq, netfs_rreq_trace_unlock);
46
+
47
+
rcu_read_lock();
48
+
xas_for_each(&xas, folio, last_page) {
49
+
unsigned int pgpos = (folio_index(folio) - start_page) * PAGE_SIZE;
50
+
unsigned int pgend = pgpos + folio_size(folio);
51
+
bool pg_failed = false;
52
+
53
+
for (;;) {
54
+
if (!subreq) {
55
+
pg_failed = true;
56
+
break;
57
+
}
58
+
if (test_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags))
59
+
folio_start_fscache(folio);
60
+
pg_failed |= subreq_failed;
61
+
if (pgend < iopos + subreq->len)
62
+
break;
63
+
64
+
account += subreq->transferred;
65
+
iopos += subreq->len;
66
+
if (!list_is_last(&subreq->rreq_link, &rreq->subrequests)) {
67
+
subreq = list_next_entry(subreq, rreq_link);
68
+
subreq_failed = (subreq->error < 0);
69
+
} else {
70
+
subreq = NULL;
71
+
subreq_failed = false;
72
+
}
73
+
if (pgend == iopos)
74
+
break;
75
+
}
76
+
77
+
if (!pg_failed) {
78
+
flush_dcache_folio(folio);
79
+
folio_mark_uptodate(folio);
80
+
}
81
+
82
+
if (!test_bit(NETFS_RREQ_DONT_UNLOCK_FOLIOS, &rreq->flags)) {
83
+
if (folio_index(folio) == rreq->no_unlock_folio &&
84
+
test_bit(NETFS_RREQ_NO_UNLOCK_FOLIO, &rreq->flags))
85
+
_debug("no unlock");
86
+
else
87
+
folio_unlock(folio);
88
+
}
89
+
}
90
+
rcu_read_unlock();
91
+
92
+
task_io_account_read(account);
93
+
if (rreq->netfs_ops->done)
94
+
rreq->netfs_ops->done(rreq);
95
+
}
96
+
97
+
static void netfs_cache_expand_readahead(struct netfs_io_request *rreq,
98
+
loff_t *_start, size_t *_len, loff_t i_size)
99
+
{
100
+
struct netfs_cache_resources *cres = &rreq->cache_resources;
101
+
102
+
if (cres->ops && cres->ops->expand_readahead)
103
+
cres->ops->expand_readahead(cres, _start, _len, i_size);
104
+
}
105
+
106
+
static void netfs_rreq_expand(struct netfs_io_request *rreq,
107
+
struct readahead_control *ractl)
108
+
{
109
+
/* Give the cache a chance to change the request parameters. The
110
+
* resultant request must contain the original region.
111
+
*/
112
+
netfs_cache_expand_readahead(rreq, &rreq->start, &rreq->len, rreq->i_size);
113
+
114
+
/* Give the netfs a chance to change the request parameters. The
115
+
* resultant request must contain the original region.
116
+
*/
117
+
if (rreq->netfs_ops->expand_readahead)
118
+
rreq->netfs_ops->expand_readahead(rreq);
119
+
120
+
/* Expand the request if the cache wants it to start earlier. Note
121
+
* that the expansion may get further extended if the VM wishes to
122
+
* insert THPs and the preferred start and/or end wind up in the middle
123
+
* of THPs.
124
+
*
125
+
* If this is the case, however, the THP size should be an integer
126
+
* multiple of the cache granule size, so we get a whole number of
127
+
* granules to deal with.
128
+
*/
129
+
if (rreq->start != readahead_pos(ractl) ||
130
+
rreq->len != readahead_length(ractl)) {
131
+
readahead_expand(ractl, rreq->start, rreq->len);
132
+
rreq->start = readahead_pos(ractl);
133
+
rreq->len = readahead_length(ractl);
134
+
135
+
trace_netfs_read(rreq, readahead_pos(ractl), readahead_length(ractl),
136
+
netfs_read_trace_expanded);
137
+
}
138
+
}
139
+
140
+
/**
141
+
* netfs_readahead - Helper to manage a read request
142
+
* @ractl: The description of the readahead request
143
+
*
144
+
* Fulfil a readahead request by drawing data from the cache if possible, or
145
+
* the netfs if not. Space beyond the EOF is zero-filled. Multiple I/O
146
+
* requests from different sources will get munged together. If necessary, the
147
+
* readahead window can be expanded in either direction to a more convenient
148
+
* alighment for RPC efficiency or to make storage in the cache feasible.
149
+
*
150
+
* The calling netfs must initialise a netfs context contiguous to the vfs
151
+
* inode before calling this.
152
+
*
153
+
* This is usable whether or not caching is enabled.
154
+
*/
155
+
void netfs_readahead(struct readahead_control *ractl)
156
+
{
157
+
struct netfs_io_request *rreq;
158
+
struct netfs_i_context *ctx = netfs_i_context(ractl->mapping->host);
159
+
int ret;
160
+
161
+
_enter("%lx,%x", readahead_index(ractl), readahead_count(ractl));
162
+
163
+
if (readahead_count(ractl) == 0)
164
+
return;
165
+
166
+
rreq = netfs_alloc_request(ractl->mapping, ractl->file,
167
+
readahead_pos(ractl),
168
+
readahead_length(ractl),
169
+
NETFS_READAHEAD);
170
+
if (IS_ERR(rreq))
171
+
return;
172
+
173
+
if (ctx->ops->begin_cache_operation) {
174
+
ret = ctx->ops->begin_cache_operation(rreq);
175
+
if (ret == -ENOMEM || ret == -EINTR || ret == -ERESTARTSYS)
176
+
goto cleanup_free;
177
+
}
178
+
179
+
netfs_stat(&netfs_n_rh_readahead);
180
+
trace_netfs_read(rreq, readahead_pos(ractl), readahead_length(ractl),
181
+
netfs_read_trace_readahead);
182
+
183
+
netfs_rreq_expand(rreq, ractl);
184
+
185
+
/* Drop the refs on the folios here rather than in the cache or
186
+
* filesystem. The locks will be dropped in netfs_rreq_unlock().
187
+
*/
188
+
while (readahead_folio(ractl))
189
+
;
190
+
191
+
netfs_begin_read(rreq, false);
192
+
return;
193
+
194
+
cleanup_free:
195
+
netfs_put_request(rreq, false, netfs_rreq_trace_put_failed);
196
+
return;
197
+
}
198
+
EXPORT_SYMBOL(netfs_readahead);
199
+
200
+
/**
201
+
* netfs_readpage - Helper to manage a readpage request
202
+
* @file: The file to read from
203
+
* @subpage: A subpage of the folio to read
204
+
*
205
+
* Fulfil a readpage request by drawing data from the cache if possible, or the
206
+
* netfs if not. Space beyond the EOF is zero-filled. Multiple I/O requests
207
+
* from different sources will get munged together.
208
+
*
209
+
* The calling netfs must initialise a netfs context contiguous to the vfs
210
+
* inode before calling this.
211
+
*
212
+
* This is usable whether or not caching is enabled.
213
+
*/
214
+
int netfs_readpage(struct file *file, struct page *subpage)
215
+
{
216
+
struct folio *folio = page_folio(subpage);
217
+
struct address_space *mapping = folio_file_mapping(folio);
218
+
struct netfs_io_request *rreq;
219
+
struct netfs_i_context *ctx = netfs_i_context(mapping->host);
220
+
int ret;
221
+
222
+
_enter("%lx", folio_index(folio));
223
+
224
+
rreq = netfs_alloc_request(mapping, file,
225
+
folio_file_pos(folio), folio_size(folio),
226
+
NETFS_READPAGE);
227
+
if (IS_ERR(rreq)) {
228
+
ret = PTR_ERR(rreq);
229
+
goto alloc_error;
230
+
}
231
+
232
+
if (ctx->ops->begin_cache_operation) {
233
+
ret = ctx->ops->begin_cache_operation(rreq);
234
+
if (ret == -ENOMEM || ret == -EINTR || ret == -ERESTARTSYS)
235
+
goto discard;
236
+
}
237
+
238
+
netfs_stat(&netfs_n_rh_readpage);
239
+
trace_netfs_read(rreq, rreq->start, rreq->len, netfs_read_trace_readpage);
240
+
return netfs_begin_read(rreq, true);
241
+
242
+
discard:
243
+
netfs_put_request(rreq, false, netfs_rreq_trace_put_discard);
244
+
alloc_error:
245
+
folio_unlock(folio);
246
+
return ret;
247
+
}
248
+
EXPORT_SYMBOL(netfs_readpage);
249
+
250
+
/*
251
+
* Prepare a folio for writing without reading first
252
+
* @folio: The folio being prepared
253
+
* @pos: starting position for the write
254
+
* @len: length of write
255
+
* @always_fill: T if the folio should always be completely filled/cleared
256
+
*
257
+
* In some cases, write_begin doesn't need to read at all:
258
+
* - full folio write
259
+
* - write that lies in a folio that is completely beyond EOF
260
+
* - write that covers the folio from start to EOF or beyond it
261
+
*
262
+
* If any of these criteria are met, then zero out the unwritten parts
263
+
* of the folio and return true. Otherwise, return false.
264
+
*/
265
+
static bool netfs_skip_folio_read(struct folio *folio, loff_t pos, size_t len,
266
+
bool always_fill)
267
+
{
268
+
struct inode *inode = folio_inode(folio);
269
+
loff_t i_size = i_size_read(inode);
270
+
size_t offset = offset_in_folio(folio, pos);
271
+
size_t plen = folio_size(folio);
272
+
273
+
if (unlikely(always_fill)) {
274
+
if (pos - offset + len <= i_size)
275
+
return false; /* Page entirely before EOF */
276
+
zero_user_segment(&folio->page, 0, plen);
277
+
folio_mark_uptodate(folio);
278
+
return true;
279
+
}
280
+
281
+
/* Full folio write */
282
+
if (offset == 0 && len >= plen)
283
+
return true;
284
+
285
+
/* Page entirely beyond the end of the file */
286
+
if (pos - offset >= i_size)
287
+
goto zero_out;
288
+
289
+
/* Write that covers from the start of the folio to EOF or beyond */
290
+
if (offset == 0 && (pos + len) >= i_size)
291
+
goto zero_out;
292
+
293
+
return false;
294
+
zero_out:
295
+
zero_user_segments(&folio->page, 0, offset, offset + len, plen);
296
+
return true;
297
+
}
298
+
299
+
/**
300
+
* netfs_write_begin - Helper to prepare for writing
301
+
* @file: The file to read from
302
+
* @mapping: The mapping to read from
303
+
* @pos: File position at which the write will begin
304
+
* @len: The length of the write (may extend beyond the end of the folio chosen)
305
+
* @aop_flags: AOP_* flags
306
+
* @_folio: Where to put the resultant folio
307
+
* @_fsdata: Place for the netfs to store a cookie
308
+
*
309
+
* Pre-read data for a write-begin request by drawing data from the cache if
310
+
* possible, or the netfs if not. Space beyond the EOF is zero-filled.
311
+
* Multiple I/O requests from different sources will get munged together. If
312
+
* necessary, the readahead window can be expanded in either direction to a
313
+
* more convenient alighment for RPC efficiency or to make storage in the cache
314
+
* feasible.
315
+
*
316
+
* The calling netfs must provide a table of operations, only one of which,
317
+
* issue_op, is mandatory.
318
+
*
319
+
* The check_write_begin() operation can be provided to check for and flush
320
+
* conflicting writes once the folio is grabbed and locked. It is passed a
321
+
* pointer to the fsdata cookie that gets returned to the VM to be passed to
322
+
* write_end. It is permitted to sleep. It should return 0 if the request
323
+
* should go ahead; unlock the folio and return -EAGAIN to cause the folio to
324
+
* be regot; or return an error.
325
+
*
326
+
* The calling netfs must initialise a netfs context contiguous to the vfs
327
+
* inode before calling this.
328
+
*
329
+
* This is usable whether or not caching is enabled.
330
+
*/
331
+
int netfs_write_begin(struct file *file, struct address_space *mapping,
332
+
loff_t pos, unsigned int len, unsigned int aop_flags,
333
+
struct folio **_folio, void **_fsdata)
334
+
{
335
+
struct netfs_io_request *rreq;
336
+
struct netfs_i_context *ctx = netfs_i_context(file_inode(file ));
337
+
struct folio *folio;
338
+
unsigned int fgp_flags;
339
+
pgoff_t index = pos >> PAGE_SHIFT;
340
+
int ret;
341
+
342
+
DEFINE_READAHEAD(ractl, file, NULL, mapping, index);
343
+
344
+
retry:
345
+
fgp_flags = FGP_LOCK | FGP_WRITE | FGP_CREAT | FGP_STABLE;
346
+
if (aop_flags & AOP_FLAG_NOFS)
347
+
fgp_flags |= FGP_NOFS;
348
+
folio = __filemap_get_folio(mapping, index, fgp_flags,
349
+
mapping_gfp_mask(mapping));
350
+
if (!folio)
351
+
return -ENOMEM;
352
+
353
+
if (ctx->ops->check_write_begin) {
354
+
/* Allow the netfs (eg. ceph) to flush conflicts. */
355
+
ret = ctx->ops->check_write_begin(file, pos, len, folio, _fsdata);
356
+
if (ret < 0) {
357
+
trace_netfs_failure(NULL, NULL, ret, netfs_fail_check_write_begin);
358
+
if (ret == -EAGAIN)
359
+
goto retry;
360
+
goto error;
361
+
}
362
+
}
363
+
364
+
if (folio_test_uptodate(folio))
365
+
goto have_folio;
366
+
367
+
/* If the page is beyond the EOF, we want to clear it - unless it's
368
+
* within the cache granule containing the EOF, in which case we need
369
+
* to preload the granule.
370
+
*/
371
+
if (!netfs_is_cache_enabled(ctx) &&
372
+
netfs_skip_folio_read(folio, pos, len, false)) {
373
+
netfs_stat(&netfs_n_rh_write_zskip);
374
+
goto have_folio_no_wait;
375
+
}
376
+
377
+
rreq = netfs_alloc_request(mapping, file,
378
+
folio_file_pos(folio), folio_size(folio),
379
+
NETFS_READ_FOR_WRITE);
380
+
if (IS_ERR(rreq)) {
381
+
ret = PTR_ERR(rreq);
382
+
goto error;
383
+
}
384
+
rreq->no_unlock_folio = folio_index(folio);
385
+
__set_bit(NETFS_RREQ_NO_UNLOCK_FOLIO, &rreq->flags);
386
+
387
+
if (ctx->ops->begin_cache_operation) {
388
+
ret = ctx->ops->begin_cache_operation(rreq);
389
+
if (ret == -ENOMEM || ret == -EINTR || ret == -ERESTARTSYS)
390
+
goto error_put;
391
+
}
392
+
393
+
netfs_stat(&netfs_n_rh_write_begin);
394
+
trace_netfs_read(rreq, pos, len, netfs_read_trace_write_begin);
395
+
396
+
/* Expand the request to meet caching requirements and download
397
+
* preferences.
398
+
*/
399
+
ractl._nr_pages = folio_nr_pages(folio);
400
+
netfs_rreq_expand(rreq, &ractl);
401
+
402
+
/* We hold the folio locks, so we can drop the references */
403
+
folio_get(folio);
404
+
while (readahead_folio(&ractl))
405
+
;
406
+
407
+
ret = netfs_begin_read(rreq, true);
408
+
if (ret < 0)
409
+
goto error;
410
+
411
+
have_folio:
412
+
ret = folio_wait_fscache_killable(folio);
413
+
if (ret < 0)
414
+
goto error;
415
+
have_folio_no_wait:
416
+
*_folio = folio;
417
+
_leave(" = 0");
418
+
return 0;
419
+
420
+
error_put:
421
+
netfs_put_request(rreq, false, netfs_rreq_trace_put_failed);
422
+
error:
423
+
folio_unlock(folio);
424
+
folio_put(folio);
425
+
_leave(" = %d", ret);
426
+
return ret;
427
+
}
428
+
EXPORT_SYMBOL(netfs_write_begin);
+49
-1
fs/netfs/internal.h
+49
-1
fs/netfs/internal.h
···
5
5
* Written by David Howells (dhowells@redhat.com)
6
6
*/
7
7
8
+
#include <linux/netfs.h>
9
+
#include <linux/fscache.h>
10
+
#include <trace/events/netfs.h>
11
+
8
12
#ifdef pr_fmt
9
13
#undef pr_fmt
10
14
#endif
···
16
12
#define pr_fmt(fmt) "netfs: " fmt
17
13
18
14
/*
19
-
* read_helper.c
15
+
* buffered_read.c
16
+
*/
17
+
void netfs_rreq_unlock_folios(struct netfs_io_request *rreq);
18
+
19
+
/*
20
+
* io.c
21
+
*/
22
+
int netfs_begin_read(struct netfs_io_request *rreq, bool sync);
23
+
24
+
/*
25
+
* main.c
20
26
*/
21
27
extern unsigned int netfs_debug;
28
+
29
+
/*
30
+
* objects.c
31
+
*/
32
+
struct netfs_io_request *netfs_alloc_request(struct address_space *mapping,
33
+
struct file *file,
34
+
loff_t start, size_t len,
35
+
enum netfs_io_origin origin);
36
+
void netfs_get_request(struct netfs_io_request *rreq, enum netfs_rreq_ref_trace what);
37
+
void netfs_clear_subrequests(struct netfs_io_request *rreq, bool was_async);
38
+
void netfs_put_request(struct netfs_io_request *rreq, bool was_async,
39
+
enum netfs_rreq_ref_trace what);
40
+
struct netfs_io_subrequest *netfs_alloc_subrequest(struct netfs_io_request *rreq);
41
+
42
+
static inline void netfs_see_request(struct netfs_io_request *rreq,
43
+
enum netfs_rreq_ref_trace what)
44
+
{
45
+
trace_netfs_rreq_ref(rreq->debug_id, refcount_read(&rreq->ref), what);
46
+
}
22
47
23
48
/*
24
49
* stats.c
···
87
54
#define netfs_stat(x) do {} while(0)
88
55
#define netfs_stat_d(x) do {} while(0)
89
56
#endif
57
+
58
+
/*
59
+
* Miscellaneous functions.
60
+
*/
61
+
static inline bool netfs_is_cache_enabled(struct netfs_i_context *ctx)
62
+
{
63
+
#if IS_ENABLED(CONFIG_FSCACHE)
64
+
struct fscache_cookie *cookie = ctx->cache;
65
+
66
+
return fscache_cookie_valid(cookie) && cookie->cache_priv &&
67
+
fscache_cookie_enabled(cookie);
68
+
#else
69
+
return false;
70
+
#endif
71
+
}
90
72
91
73
/*****************************************************************************/
92
74
/*
+657
fs/netfs/io.c
+657
fs/netfs/io.c
···
1
+
// SPDX-License-Identifier: GPL-2.0-or-later
2
+
/* Network filesystem high-level read support.
3
+
*
4
+
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
5
+
* Written by David Howells (dhowells@redhat.com)
6
+
*/
7
+
8
+
#include <linux/module.h>
9
+
#include <linux/export.h>
10
+
#include <linux/fs.h>
11
+
#include <linux/mm.h>
12
+
#include <linux/pagemap.h>
13
+
#include <linux/slab.h>
14
+
#include <linux/uio.h>
15
+
#include <linux/sched/mm.h>
16
+
#include <linux/task_io_accounting_ops.h>
17
+
#include "internal.h"
18
+
19
+
/*
20
+
* Clear the unread part of an I/O request.
21
+
*/
22
+
static void netfs_clear_unread(struct netfs_io_subrequest *subreq)
23
+
{
24
+
struct iov_iter iter;
25
+
26
+
iov_iter_xarray(&iter, READ, &subreq->rreq->mapping->i_pages,
27
+
subreq->start + subreq->transferred,
28
+
subreq->len - subreq->transferred);
29
+
iov_iter_zero(iov_iter_count(&iter), &iter);
30
+
}
31
+
32
+
static void netfs_cache_read_terminated(void *priv, ssize_t transferred_or_error,
33
+
bool was_async)
34
+
{
35
+
struct netfs_io_subrequest *subreq = priv;
36
+
37
+
netfs_subreq_terminated(subreq, transferred_or_error, was_async);
38
+
}
39
+
40
+
/*
41
+
* Issue a read against the cache.
42
+
* - Eats the caller's ref on subreq.
43
+
*/
44
+
static void netfs_read_from_cache(struct netfs_io_request *rreq,
45
+
struct netfs_io_subrequest *subreq,
46
+
enum netfs_read_from_hole read_hole)
47
+
{
48
+
struct netfs_cache_resources *cres = &rreq->cache_resources;
49
+
struct iov_iter iter;
50
+
51
+
netfs_stat(&netfs_n_rh_read);
52
+
iov_iter_xarray(&iter, READ, &rreq->mapping->i_pages,
53
+
subreq->start + subreq->transferred,
54
+
subreq->len - subreq->transferred);
55
+
56
+
cres->ops->read(cres, subreq->start, &iter, read_hole,
57
+
netfs_cache_read_terminated, subreq);
58
+
}
59
+
60
+
/*
61
+
* Fill a subrequest region with zeroes.
62
+
*/
63
+
static void netfs_fill_with_zeroes(struct netfs_io_request *rreq,
64
+
struct netfs_io_subrequest *subreq)
65
+
{
66
+
netfs_stat(&netfs_n_rh_zero);
67
+
__set_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags);
68
+
netfs_subreq_terminated(subreq, 0, false);
69
+
}
70
+
71
+
/*
72
+
* Ask the netfs to issue a read request to the server for us.
73
+
*
74
+
* The netfs is expected to read from subreq->pos + subreq->transferred to
75
+
* subreq->pos + subreq->len - 1. It may not backtrack and write data into the
76
+
* buffer prior to the transferred point as it might clobber dirty data
77
+
* obtained from the cache.
78
+
*
79
+
* Alternatively, the netfs is allowed to indicate one of two things:
80
+
*
81
+
* - NETFS_SREQ_SHORT_READ: A short read - it will get called again to try and
82
+
* make progress.
83
+
*
84
+
* - NETFS_SREQ_CLEAR_TAIL: A short read - the rest of the buffer will be
85
+
* cleared.
86
+
*/
87
+
static void netfs_read_from_server(struct netfs_io_request *rreq,
88
+
struct netfs_io_subrequest *subreq)
89
+
{
90
+
netfs_stat(&netfs_n_rh_download);
91
+
rreq->netfs_ops->issue_read(subreq);
92
+
}
93
+
94
+
/*
95
+
* Release those waiting.
96
+
*/
97
+
static void netfs_rreq_completed(struct netfs_io_request *rreq, bool was_async)
98
+
{
99
+
trace_netfs_rreq(rreq, netfs_rreq_trace_done);
100
+
netfs_clear_subrequests(rreq, was_async);
101
+
netfs_put_request(rreq, was_async, netfs_rreq_trace_put_complete);
102
+
}
103
+
104
+
/*
105
+
* Deal with the completion of writing the data to the cache. We have to clear
106
+
* the PG_fscache bits on the folios involved and release the caller's ref.
107
+
*
108
+
* May be called in softirq mode and we inherit a ref from the caller.
109
+
*/
110
+
static void netfs_rreq_unmark_after_write(struct netfs_io_request *rreq,
111
+
bool was_async)
112
+
{
113
+
struct netfs_io_subrequest *subreq;
114
+
struct folio *folio;
115
+
pgoff_t unlocked = 0;
116
+
bool have_unlocked = false;
117
+
118
+
rcu_read_lock();
119
+
120
+
list_for_each_entry(subreq, &rreq->subrequests, rreq_link) {
121
+
XA_STATE(xas, &rreq->mapping->i_pages, subreq->start / PAGE_SIZE);
122
+
123
+
xas_for_each(&xas, folio, (subreq->start + subreq->len - 1) / PAGE_SIZE) {
124
+
/* We might have multiple writes from the same huge
125
+
* folio, but we mustn't unlock a folio more than once.
126
+
*/
127
+
if (have_unlocked && folio_index(folio) <= unlocked)
128
+
continue;
129
+
unlocked = folio_index(folio);
130
+
folio_end_fscache(folio);
131
+
have_unlocked = true;
132
+
}
133
+
}
134
+
135
+
rcu_read_unlock();
136
+
netfs_rreq_completed(rreq, was_async);
137
+
}
138
+
139
+
static void netfs_rreq_copy_terminated(void *priv, ssize_t transferred_or_error,
140
+
bool was_async)
141
+
{
142
+
struct netfs_io_subrequest *subreq = priv;
143
+
struct netfs_io_request *rreq = subreq->rreq;
144
+
145
+
if (IS_ERR_VALUE(transferred_or_error)) {
146
+
netfs_stat(&netfs_n_rh_write_failed);
147
+
trace_netfs_failure(rreq, subreq, transferred_or_error,
148
+
netfs_fail_copy_to_cache);
149
+
} else {
150
+
netfs_stat(&netfs_n_rh_write_done);
151
+
}
152
+
153
+
trace_netfs_sreq(subreq, netfs_sreq_trace_write_term);
154
+
155
+
/* If we decrement nr_copy_ops to 0, the ref belongs to us. */
156
+
if (atomic_dec_and_test(&rreq->nr_copy_ops))
157
+
netfs_rreq_unmark_after_write(rreq, was_async);
158
+
159
+
netfs_put_subrequest(subreq, was_async, netfs_sreq_trace_put_terminated);
160
+
}
161
+
162
+
/*
163
+
* Perform any outstanding writes to the cache. We inherit a ref from the
164
+
* caller.
165
+
*/
166
+
static void netfs_rreq_do_write_to_cache(struct netfs_io_request *rreq)
167
+
{
168
+
struct netfs_cache_resources *cres = &rreq->cache_resources;
169
+
struct netfs_io_subrequest *subreq, *next, *p;
170
+
struct iov_iter iter;
171
+
int ret;
172
+
173
+
trace_netfs_rreq(rreq, netfs_rreq_trace_copy);
174
+
175
+
/* We don't want terminating writes trying to wake us up whilst we're
176
+
* still going through the list.
177
+
*/
178
+
atomic_inc(&rreq->nr_copy_ops);
179
+
180
+
list_for_each_entry_safe(subreq, p, &rreq->subrequests, rreq_link) {
181
+
if (!test_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags)) {
182
+
list_del_init(&subreq->rreq_link);
183
+
netfs_put_subrequest(subreq, false,
184
+
netfs_sreq_trace_put_no_copy);
185
+
}
186
+
}
187
+
188
+
list_for_each_entry(subreq, &rreq->subrequests, rreq_link) {
189
+
/* Amalgamate adjacent writes */
190
+
while (!list_is_last(&subreq->rreq_link, &rreq->subrequests)) {
191
+
next = list_next_entry(subreq, rreq_link);
192
+
if (next->start != subreq->start + subreq->len)
193
+
break;
194
+
subreq->len += next->len;
195
+
list_del_init(&next->rreq_link);
196
+
netfs_put_subrequest(next, false,
197
+
netfs_sreq_trace_put_merged);
198
+
}
199
+
200
+
ret = cres->ops->prepare_write(cres, &subreq->start, &subreq->len,
201
+
rreq->i_size, true);
202
+
if (ret < 0) {
203
+
trace_netfs_failure(rreq, subreq, ret, netfs_fail_prepare_write);
204
+
trace_netfs_sreq(subreq, netfs_sreq_trace_write_skip);
205
+
continue;
206
+
}
207
+
208
+
iov_iter_xarray(&iter, WRITE, &rreq->mapping->i_pages,
209
+
subreq->start, subreq->len);
210
+
211
+
atomic_inc(&rreq->nr_copy_ops);
212
+
netfs_stat(&netfs_n_rh_write);
213
+
netfs_get_subrequest(subreq, netfs_sreq_trace_get_copy_to_cache);
214
+
trace_netfs_sreq(subreq, netfs_sreq_trace_write);
215
+
cres->ops->write(cres, subreq->start, &iter,
216
+
netfs_rreq_copy_terminated, subreq);
217
+
}
218
+
219
+
/* If we decrement nr_copy_ops to 0, the usage ref belongs to us. */
220
+
if (atomic_dec_and_test(&rreq->nr_copy_ops))
221
+
netfs_rreq_unmark_after_write(rreq, false);
222
+
}
223
+
224
+
static void netfs_rreq_write_to_cache_work(struct work_struct *work)
225
+
{
226
+
struct netfs_io_request *rreq =
227
+
container_of(work, struct netfs_io_request, work);
228
+
229
+
netfs_rreq_do_write_to_cache(rreq);
230
+
}
231
+
232
+
static void netfs_rreq_write_to_cache(struct netfs_io_request *rreq)
233
+
{
234
+
rreq->work.func = netfs_rreq_write_to_cache_work;
235
+
if (!queue_work(system_unbound_wq, &rreq->work))
236
+
BUG();
237
+
}
238
+
239
+
/*
240
+
* Handle a short read.
241
+
*/
242
+
static void netfs_rreq_short_read(struct netfs_io_request *rreq,
243
+
struct netfs_io_subrequest *subreq)
244
+
{
245
+
__clear_bit(NETFS_SREQ_SHORT_IO, &subreq->flags);
246
+
__set_bit(NETFS_SREQ_SEEK_DATA_READ, &subreq->flags);
247
+
248
+
netfs_stat(&netfs_n_rh_short_read);
249
+
trace_netfs_sreq(subreq, netfs_sreq_trace_resubmit_short);
250
+
251
+
netfs_get_subrequest(subreq, netfs_sreq_trace_get_short_read);
252
+
atomic_inc(&rreq->nr_outstanding);
253
+
if (subreq->source == NETFS_READ_FROM_CACHE)
254
+
netfs_read_from_cache(rreq, subreq, NETFS_READ_HOLE_CLEAR);
255
+
else
256
+
netfs_read_from_server(rreq, subreq);
257
+
}
258
+
259
+
/*
260
+
* Resubmit any short or failed operations. Returns true if we got the rreq
261
+
* ref back.
262
+
*/
263
+
static bool netfs_rreq_perform_resubmissions(struct netfs_io_request *rreq)
264
+
{
265
+
struct netfs_io_subrequest *subreq;
266
+
267
+
WARN_ON(in_interrupt());
268
+
269
+
trace_netfs_rreq(rreq, netfs_rreq_trace_resubmit);
270
+
271
+
/* We don't want terminating submissions trying to wake us up whilst
272
+
* we're still going through the list.
273
+
*/
274
+
atomic_inc(&rreq->nr_outstanding);
275
+
276
+
__clear_bit(NETFS_RREQ_INCOMPLETE_IO, &rreq->flags);
277
+
list_for_each_entry(subreq, &rreq->subrequests, rreq_link) {
278
+
if (subreq->error) {
279
+
if (subreq->source != NETFS_READ_FROM_CACHE)
280
+
break;
281
+
subreq->source = NETFS_DOWNLOAD_FROM_SERVER;
282
+
subreq->error = 0;
283
+
netfs_stat(&netfs_n_rh_download_instead);
284
+
trace_netfs_sreq(subreq, netfs_sreq_trace_download_instead);
285
+
netfs_get_subrequest(subreq, netfs_sreq_trace_get_resubmit);
286
+
atomic_inc(&rreq->nr_outstanding);
287
+
netfs_read_from_server(rreq, subreq);
288
+
} else if (test_bit(NETFS_SREQ_SHORT_IO, &subreq->flags)) {
289
+
netfs_rreq_short_read(rreq, subreq);
290
+
}
291
+
}
292
+
293
+
/* If we decrement nr_outstanding to 0, the usage ref belongs to us. */
294
+
if (atomic_dec_and_test(&rreq->nr_outstanding))
295
+
return true;
296
+
297
+
wake_up_var(&rreq->nr_outstanding);
298
+
return false;
299
+
}
300
+
301
+
/*
302
+
* Check to see if the data read is still valid.
303
+
*/
304
+
static void netfs_rreq_is_still_valid(struct netfs_io_request *rreq)
305
+
{
306
+
struct netfs_io_subrequest *subreq;
307
+
308
+
if (!rreq->netfs_ops->is_still_valid ||
309
+
rreq->netfs_ops->is_still_valid(rreq))
310
+
return;
311
+
312
+
list_for_each_entry(subreq, &rreq->subrequests, rreq_link) {
313
+
if (subreq->source == NETFS_READ_FROM_CACHE) {
314
+
subreq->error = -ESTALE;
315
+
__set_bit(NETFS_RREQ_INCOMPLETE_IO, &rreq->flags);
316
+
}
317
+
}
318
+
}
319
+
320
+
/*
321
+
* Assess the state of a read request and decide what to do next.
322
+
*
323
+
* Note that we could be in an ordinary kernel thread, on a workqueue or in
324
+
* softirq context at this point. We inherit a ref from the caller.
325
+
*/
326
+
static void netfs_rreq_assess(struct netfs_io_request *rreq, bool was_async)
327
+
{
328
+
trace_netfs_rreq(rreq, netfs_rreq_trace_assess);
329
+
330
+
again:
331
+
netfs_rreq_is_still_valid(rreq);
332
+
333
+
if (!test_bit(NETFS_RREQ_FAILED, &rreq->flags) &&
334
+
test_bit(NETFS_RREQ_INCOMPLETE_IO, &rreq->flags)) {
335
+
if (netfs_rreq_perform_resubmissions(rreq))
336
+
goto again;
337
+
return;
338
+
}
339
+
340
+
netfs_rreq_unlock_folios(rreq);
341
+
342
+
clear_bit_unlock(NETFS_RREQ_IN_PROGRESS, &rreq->flags);
343
+
wake_up_bit(&rreq->flags, NETFS_RREQ_IN_PROGRESS);
344
+
345
+
if (test_bit(NETFS_RREQ_COPY_TO_CACHE, &rreq->flags))
346
+
return netfs_rreq_write_to_cache(rreq);
347
+
348
+
netfs_rreq_completed(rreq, was_async);
349
+
}
350
+
351
+
static void netfs_rreq_work(struct work_struct *work)
352
+
{
353
+
struct netfs_io_request *rreq =
354
+
container_of(work, struct netfs_io_request, work);
355
+
netfs_rreq_assess(rreq, false);
356
+
}
357
+
358
+
/*
359
+
* Handle the completion of all outstanding I/O operations on a read request.
360
+
* We inherit a ref from the caller.
361
+
*/
362
+
static void netfs_rreq_terminated(struct netfs_io_request *rreq,
363
+
bool was_async)
364
+
{
365
+
if (test_bit(NETFS_RREQ_INCOMPLETE_IO, &rreq->flags) &&
366
+
was_async) {
367
+
if (!queue_work(system_unbound_wq, &rreq->work))
368
+
BUG();
369
+
} else {
370
+
netfs_rreq_assess(rreq, was_async);
371
+
}
372
+
}
373
+
374
+
/**
375
+
* netfs_subreq_terminated - Note the termination of an I/O operation.
376
+
* @subreq: The I/O request that has terminated.
377
+
* @transferred_or_error: The amount of data transferred or an error code.
378
+
* @was_async: The termination was asynchronous
379
+
*
380
+
* This tells the read helper that a contributory I/O operation has terminated,
381
+
* one way or another, and that it should integrate the results.
382
+
*
383
+
* The caller indicates in @transferred_or_error the outcome of the operation,
384
+
* supplying a positive value to indicate the number of bytes transferred, 0 to
385
+
* indicate a failure to transfer anything that should be retried or a negative
386
+
* error code. The helper will look after reissuing I/O operations as
387
+
* appropriate and writing downloaded data to the cache.
388
+
*
389
+
* If @was_async is true, the caller might be running in softirq or interrupt
390
+
* context and we can't sleep.
391
+
*/
392
+
void netfs_subreq_terminated(struct netfs_io_subrequest *subreq,
393
+
ssize_t transferred_or_error,
394
+
bool was_async)
395
+
{
396
+
struct netfs_io_request *rreq = subreq->rreq;
397
+
int u;
398
+
399
+
_enter("[%u]{%llx,%lx},%zd",
400
+
subreq->debug_index, subreq->start, subreq->flags,
401
+
transferred_or_error);
402
+
403
+
switch (subreq->source) {
404
+
case NETFS_READ_FROM_CACHE:
405
+
netfs_stat(&netfs_n_rh_read_done);
406
+
break;
407
+
case NETFS_DOWNLOAD_FROM_SERVER:
408
+
netfs_stat(&netfs_n_rh_download_done);
409
+
break;
410
+
default:
411
+
break;
412
+
}
413
+
414
+
if (IS_ERR_VALUE(transferred_or_error)) {
415
+
subreq->error = transferred_or_error;
416
+
trace_netfs_failure(rreq, subreq, transferred_or_error,
417
+
netfs_fail_read);
418
+
goto failed;
419
+
}
420
+
421
+
if (WARN(transferred_or_error > subreq->len - subreq->transferred,
422
+
"Subreq overread: R%x[%x] %zd > %zu - %zu",
423
+
rreq->debug_id, subreq->debug_index,
424
+
transferred_or_error, subreq->len, subreq->transferred))
425
+
transferred_or_error = subreq->len - subreq->transferred;
426
+
427
+
subreq->error = 0;
428
+
subreq->transferred += transferred_or_error;
429
+
if (subreq->transferred < subreq->len)
430
+
goto incomplete;
431
+
432
+
complete:
433
+
__clear_bit(NETFS_SREQ_NO_PROGRESS, &subreq->flags);
434
+
if (test_bit(NETFS_SREQ_COPY_TO_CACHE, &subreq->flags))
435
+
set_bit(NETFS_RREQ_COPY_TO_CACHE, &rreq->flags);
436
+
437
+
out:
438
+
trace_netfs_sreq(subreq, netfs_sreq_trace_terminated);
439
+
440
+
/* If we decrement nr_outstanding to 0, the ref belongs to us. */
441
+
u = atomic_dec_return(&rreq->nr_outstanding);
442
+
if (u == 0)
443
+
netfs_rreq_terminated(rreq, was_async);
444
+
else if (u == 1)
445
+
wake_up_var(&rreq->nr_outstanding);
446
+
447
+
netfs_put_subrequest(subreq, was_async, netfs_sreq_trace_put_terminated);
448
+
return;
449
+
450
+
incomplete:
451
+
if (test_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags)) {
452
+
netfs_clear_unread(subreq);
453
+
subreq->transferred = subreq->len;
454
+
goto complete;
455
+
}
456
+
457
+
if (transferred_or_error == 0) {
458
+
if (__test_and_set_bit(NETFS_SREQ_NO_PROGRESS, &subreq->flags)) {
459
+
subreq->error = -ENODATA;
460
+
goto failed;
461
+
}
462
+
} else {
463
+
__clear_bit(NETFS_SREQ_NO_PROGRESS, &subreq->flags);
464
+
}
465
+
466
+
__set_bit(NETFS_SREQ_SHORT_IO, &subreq->flags);
467
+
set_bit(NETFS_RREQ_INCOMPLETE_IO, &rreq->flags);
468
+
goto out;
469
+
470
+
failed:
471
+
if (subreq->source == NETFS_READ_FROM_CACHE) {
472
+
netfs_stat(&netfs_n_rh_read_failed);
473
+
set_bit(NETFS_RREQ_INCOMPLETE_IO, &rreq->flags);
474
+
} else {
475
+
netfs_stat(&netfs_n_rh_download_failed);
476
+
set_bit(NETFS_RREQ_FAILED, &rreq->flags);
477
+
rreq->error = subreq->error;
478
+
}
479
+
goto out;
480
+
}
481
+
EXPORT_SYMBOL(netfs_subreq_terminated);
482
+
483
+
static enum netfs_io_source netfs_cache_prepare_read(struct netfs_io_subrequest *subreq,
484
+
loff_t i_size)
485
+
{
486
+
struct netfs_io_request *rreq = subreq->rreq;
487
+
struct netfs_cache_resources *cres = &rreq->cache_resources;
488
+
489
+
if (cres->ops)
490
+
return cres->ops->prepare_read(subreq, i_size);
491
+
if (subreq->start >= rreq->i_size)
492
+
return NETFS_FILL_WITH_ZEROES;
493
+
return NETFS_DOWNLOAD_FROM_SERVER;
494
+
}
495
+
496
+
/*
497
+
* Work out what sort of subrequest the next one will be.
498
+
*/
499
+
static enum netfs_io_source
500
+
netfs_rreq_prepare_read(struct netfs_io_request *rreq,
501
+
struct netfs_io_subrequest *subreq)
502
+
{
503
+
enum netfs_io_source source;
504
+
505
+
_enter("%llx-%llx,%llx", subreq->start, subreq->start + subreq->len, rreq->i_size);
506
+
507
+
source = netfs_cache_prepare_read(subreq, rreq->i_size);
508
+
if (source == NETFS_INVALID_READ)
509
+
goto out;
510
+
511
+
if (source == NETFS_DOWNLOAD_FROM_SERVER) {
512
+
/* Call out to the netfs to let it shrink the request to fit
513
+
* its own I/O sizes and boundaries. If it shinks it here, it
514
+
* will be called again to make simultaneous calls; if it wants
515
+
* to make serial calls, it can indicate a short read and then
516
+
* we will call it again.
517
+
*/
518
+
if (subreq->len > rreq->i_size - subreq->start)
519
+
subreq->len = rreq->i_size - subreq->start;
520
+
521
+
if (rreq->netfs_ops->clamp_length &&
522
+
!rreq->netfs_ops->clamp_length(subreq)) {
523
+
source = NETFS_INVALID_READ;
524
+
goto out;
525
+
}
526
+
}
527
+
528
+
if (WARN_ON(subreq->len == 0))
529
+
source = NETFS_INVALID_READ;
530
+
531
+
out:
532
+
subreq->source = source;
533
+
trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
534
+
return source;
535
+
}
536
+
537
+
/*
538
+
* Slice off a piece of a read request and submit an I/O request for it.
539
+
*/
540
+
static bool netfs_rreq_submit_slice(struct netfs_io_request *rreq,
541
+
unsigned int *_debug_index)
542
+
{
543
+
struct netfs_io_subrequest *subreq;
544
+
enum netfs_io_source source;
545
+
546
+
subreq = netfs_alloc_subrequest(rreq);
547
+
if (!subreq)
548
+
return false;
549
+
550
+
subreq->debug_index = (*_debug_index)++;
551
+
subreq->start = rreq->start + rreq->submitted;
552
+
subreq->len = rreq->len - rreq->submitted;
553
+
554
+
_debug("slice %llx,%zx,%zx", subreq->start, subreq->len, rreq->submitted);
555
+
list_add_tail(&subreq->rreq_link, &rreq->subrequests);
556
+
557
+
/* Call out to the cache to find out what it can do with the remaining
558
+
* subset. It tells us in subreq->flags what it decided should be done
559
+
* and adjusts subreq->len down if the subset crosses a cache boundary.
560
+
*
561
+
* Then when we hand the subset, it can choose to take a subset of that
562
+
* (the starts must coincide), in which case, we go around the loop
563
+
* again and ask it to download the next piece.
564
+
*/
565
+
source = netfs_rreq_prepare_read(rreq, subreq);
566
+
if (source == NETFS_INVALID_READ)
567
+
goto subreq_failed;
568
+
569
+
atomic_inc(&rreq->nr_outstanding);
570
+
571
+
rreq->submitted += subreq->len;
572
+
573
+
trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
574
+
switch (source) {
575
+
case NETFS_FILL_WITH_ZEROES:
576
+
netfs_fill_with_zeroes(rreq, subreq);
577
+
break;
578
+
case NETFS_DOWNLOAD_FROM_SERVER:
579
+
netfs_read_from_server(rreq, subreq);
580
+
break;
581
+
case NETFS_READ_FROM_CACHE:
582
+
netfs_read_from_cache(rreq, subreq, NETFS_READ_HOLE_IGNORE);
583
+
break;
584
+
default:
585
+
BUG();
586
+
}
587
+
588
+
return true;
589
+
590
+
subreq_failed:
591
+
rreq->error = subreq->error;
592
+
netfs_put_subrequest(subreq, false, netfs_sreq_trace_put_failed);
593
+
return false;
594
+
}
595
+
596
+
/*
597
+
* Begin the process of reading in a chunk of data, where that data may be
598
+
* stitched together from multiple sources, including multiple servers and the
599
+
* local cache.
600
+
*/
601
+
int netfs_begin_read(struct netfs_io_request *rreq, bool sync)
602
+
{
603
+
unsigned int debug_index = 0;
604
+
int ret;
605
+
606
+
_enter("R=%x %llx-%llx",
607
+
rreq->debug_id, rreq->start, rreq->start + rreq->len - 1);
608
+
609
+
if (rreq->len == 0) {
610
+
pr_err("Zero-sized read [R=%x]\n", rreq->debug_id);
611
+
netfs_put_request(rreq, false, netfs_rreq_trace_put_zero_len);
612
+
return -EIO;
613
+
}
614
+
615
+
INIT_WORK(&rreq->work, netfs_rreq_work);
616
+
617
+
if (sync)
618
+
netfs_get_request(rreq, netfs_rreq_trace_get_hold);
619
+
620
+
/* Chop the read into slices according to what the cache and the netfs
621
+
* want and submit each one.
622
+
*/
623
+
atomic_set(&rreq->nr_outstanding, 1);
624
+
do {
625
+
if (!netfs_rreq_submit_slice(rreq, &debug_index))
626
+
break;
627
+
628
+
} while (rreq->submitted < rreq->len);
629
+
630
+
if (sync) {
631
+
/* Keep nr_outstanding incremented so that the ref always belongs to
632
+
* us, and the service code isn't punted off to a random thread pool to
633
+
* process.
634
+
*/
635
+
for (;;) {
636
+
wait_var_event(&rreq->nr_outstanding,
637
+
atomic_read(&rreq->nr_outstanding) == 1);
638
+
netfs_rreq_assess(rreq, false);
639
+
if (!test_bit(NETFS_RREQ_IN_PROGRESS, &rreq->flags))
640
+
break;
641
+
cond_resched();
642
+
}
643
+
644
+
ret = rreq->error;
645
+
if (ret == 0 && rreq->submitted < rreq->len) {
646
+
trace_netfs_failure(rreq, NULL, ret, netfs_fail_short_read);
647
+
ret = -EIO;
648
+
}
649
+
netfs_put_request(rreq, false, netfs_rreq_trace_put_hold);
650
+
} else {
651
+
/* If we decrement nr_outstanding to 0, the ref belongs to us. */
652
+
if (atomic_dec_and_test(&rreq->nr_outstanding))
653
+
netfs_rreq_assess(rreq, false);
654
+
ret = 0;
655
+
}
656
+
return ret;
657
+
}
+20
fs/netfs/main.c
+20
fs/netfs/main.c
···
1
+
// SPDX-License-Identifier: GPL-2.0-or-later
2
+
/* Miscellaneous bits for the netfs support library.
3
+
*
4
+
* Copyright (C) 2022 Red Hat, Inc. All Rights Reserved.
5
+
* Written by David Howells (dhowells@redhat.com)
6
+
*/
7
+
8
+
#include <linux/module.h>
9
+
#include <linux/export.h>
10
+
#include "internal.h"
11
+
#define CREATE_TRACE_POINTS
12
+
#include <trace/events/netfs.h>
13
+
14
+
MODULE_DESCRIPTION("Network fs support");
15
+
MODULE_AUTHOR("Red Hat, Inc.");
16
+
MODULE_LICENSE("GPL");
17
+
18
+
unsigned netfs_debug;
19
+
module_param_named(debug, netfs_debug, uint, S_IWUSR | S_IRUGO);
20
+
MODULE_PARM_DESC(netfs_debug, "Netfs support debugging mask");
+160
fs/netfs/objects.c
+160
fs/netfs/objects.c
···
1
+
// SPDX-License-Identifier: GPL-2.0-only
2
+
/* Object lifetime handling and tracing.
3
+
*
4
+
* Copyright (C) 2022 Red Hat, Inc. All Rights Reserved.
5
+
* Written by David Howells (dhowells@redhat.com)
6
+
*/
7
+
8
+
#include <linux/slab.h>
9
+
#include "internal.h"
10
+
11
+
/*
12
+
* Allocate an I/O request and initialise it.
13
+
*/
14
+
struct netfs_io_request *netfs_alloc_request(struct address_space *mapping,
15
+
struct file *file,
16
+
loff_t start, size_t len,
17
+
enum netfs_io_origin origin)
18
+
{
19
+
static atomic_t debug_ids;
20
+
struct inode *inode = file ? file_inode(file) : mapping->host;
21
+
struct netfs_i_context *ctx = netfs_i_context(inode);
22
+
struct netfs_io_request *rreq;
23
+
int ret;
24
+
25
+
rreq = kzalloc(sizeof(struct netfs_io_request), GFP_KERNEL);
26
+
if (!rreq)
27
+
return ERR_PTR(-ENOMEM);
28
+
29
+
rreq->start = start;
30
+
rreq->len = len;
31
+
rreq->origin = origin;
32
+
rreq->netfs_ops = ctx->ops;
33
+
rreq->mapping = mapping;
34
+
rreq->inode = inode;
35
+
rreq->i_size = i_size_read(inode);
36
+
rreq->debug_id = atomic_inc_return(&debug_ids);
37
+
INIT_LIST_HEAD(&rreq->subrequests);
38
+
refcount_set(&rreq->ref, 1);
39
+
__set_bit(NETFS_RREQ_IN_PROGRESS, &rreq->flags);
40
+
if (rreq->netfs_ops->init_request) {
41
+
ret = rreq->netfs_ops->init_request(rreq, file);
42
+
if (ret < 0) {
43
+
kfree(rreq);
44
+
return ERR_PTR(ret);
45
+
}
46
+
}
47
+
48
+
netfs_stat(&netfs_n_rh_rreq);
49
+
return rreq;
50
+
}
51
+
52
+
void netfs_get_request(struct netfs_io_request *rreq, enum netfs_rreq_ref_trace what)
53
+
{
54
+
int r;
55
+
56
+
__refcount_inc(&rreq->ref, &r);
57
+
trace_netfs_rreq_ref(rreq->debug_id, r + 1, what);
58
+
}
59
+
60
+
void netfs_clear_subrequests(struct netfs_io_request *rreq, bool was_async)
61
+
{
62
+
struct netfs_io_subrequest *subreq;
63
+
64
+
while (!list_empty(&rreq->subrequests)) {
65
+
subreq = list_first_entry(&rreq->subrequests,
66
+
struct netfs_io_subrequest, rreq_link);
67
+
list_del(&subreq->rreq_link);
68
+
netfs_put_subrequest(subreq, was_async,
69
+
netfs_sreq_trace_put_clear);
70
+
}
71
+
}
72
+
73
+
static void netfs_free_request(struct work_struct *work)
74
+
{
75
+
struct netfs_io_request *rreq =
76
+
container_of(work, struct netfs_io_request, work);
77
+
78
+
netfs_clear_subrequests(rreq, false);
79
+
if (rreq->netfs_priv)
80
+
rreq->netfs_ops->cleanup(rreq->mapping, rreq->netfs_priv);
81
+
trace_netfs_rreq(rreq, netfs_rreq_trace_free);
82
+
if (rreq->cache_resources.ops)
83
+
rreq->cache_resources.ops->end_operation(&rreq->cache_resources);
84
+
kfree(rreq);
85
+
netfs_stat_d(&netfs_n_rh_rreq);
86
+
}
87
+
88
+
void netfs_put_request(struct netfs_io_request *rreq, bool was_async,
89
+
enum netfs_rreq_ref_trace what)
90
+
{
91
+
unsigned int debug_id = rreq->debug_id;
92
+
bool dead;
93
+
int r;
94
+
95
+
dead = __refcount_dec_and_test(&rreq->ref, &r);
96
+
trace_netfs_rreq_ref(debug_id, r - 1, what);
97
+
if (dead) {
98
+
if (was_async) {
99
+
rreq->work.func = netfs_free_request;
100
+
if (!queue_work(system_unbound_wq, &rreq->work))
101
+
BUG();
102
+
} else {
103
+
netfs_free_request(&rreq->work);
104
+
}
105
+
}
106
+
}
107
+
108
+
/*
109
+
* Allocate and partially initialise an I/O request structure.
110
+
*/
111
+
struct netfs_io_subrequest *netfs_alloc_subrequest(struct netfs_io_request *rreq)
112
+
{
113
+
struct netfs_io_subrequest *subreq;
114
+
115
+
subreq = kzalloc(sizeof(struct netfs_io_subrequest), GFP_KERNEL);
116
+
if (subreq) {
117
+
INIT_LIST_HEAD(&subreq->rreq_link);
118
+
refcount_set(&subreq->ref, 2);
119
+
subreq->rreq = rreq;
120
+
netfs_get_request(rreq, netfs_rreq_trace_get_subreq);
121
+
netfs_stat(&netfs_n_rh_sreq);
122
+
}
123
+
124
+
return subreq;
125
+
}
126
+
127
+
void netfs_get_subrequest(struct netfs_io_subrequest *subreq,
128
+
enum netfs_sreq_ref_trace what)
129
+
{
130
+
int r;
131
+
132
+
__refcount_inc(&subreq->ref, &r);
133
+
trace_netfs_sreq_ref(subreq->rreq->debug_id, subreq->debug_index, r + 1,
134
+
what);
135
+
}
136
+
137
+
static void netfs_free_subrequest(struct netfs_io_subrequest *subreq,
138
+
bool was_async)
139
+
{
140
+
struct netfs_io_request *rreq = subreq->rreq;
141
+
142
+
trace_netfs_sreq(subreq, netfs_sreq_trace_free);
143
+
kfree(subreq);
144
+
netfs_stat_d(&netfs_n_rh_sreq);
145
+
netfs_put_request(rreq, was_async, netfs_rreq_trace_put_subreq);
146
+
}
147
+
148
+
void netfs_put_subrequest(struct netfs_io_subrequest *subreq, bool was_async,
149
+
enum netfs_sreq_ref_trace what)
150
+
{
151
+
unsigned int debug_index = subreq->debug_index;
152
+
unsigned int debug_id = subreq->rreq->debug_id;
153
+
bool dead;
154
+
int r;
155
+
156
+
dead = __refcount_dec_and_test(&subreq->ref, &r);
157
+
trace_netfs_sreq_ref(debug_id, debug_index, r - 1, what);
158
+
if (dead)
159
+
netfs_free_subrequest(subreq, was_async);
160
+
}
-1205
fs/netfs/read_helper.c
-1205
fs/netfs/read_helper.c
···
1
-
// SPDX-License-Identifier: GPL-2.0-or-later
2
-
/* Network filesystem high-level read support.
3
-
*
4
-
* Copyright (C) 2021 Red Hat, Inc. All Rights Reserved.
5
-
* Written by David Howells (dhowells@redhat.com)
6
-
*/
7
-
8
-
#include <linux/module.h>
9
-
#include <linux/export.h>
10
-
#include <linux/fs.h>
11
-
#include <linux/mm.h>
12
-
#include <linux/pagemap.h>
13
-
#include <linux/slab.h>
14
-
#include <linux/uio.h>
15
-
#include <linux/sched/mm.h>
16
-
#include <linux/task_io_accounting_ops.h>
17
-
#include <linux/netfs.h>
18
-
#include "internal.h"
19
-
#define CREATE_TRACE_POINTS
20
-
#include <trace/events/netfs.h>
21
-
22
-
MODULE_DESCRIPTION("Network fs support");
23
-
MODULE_AUTHOR("Red Hat, Inc.");
24
-
MODULE_LICENSE("GPL");
25
-
26
-
unsigned netfs_debug;
27
-
module_param_named(debug, netfs_debug, uint, S_IWUSR | S_IRUGO);
28
-
MODULE_PARM_DESC(netfs_debug, "Netfs support debugging mask");
29
-
30
-
static void netfs_rreq_work(struct work_struct *);
31
-
static void __netfs_put_subrequest(struct netfs_read_subrequest *, bool);
32
-
33
-
static void netfs_put_subrequest(struct netfs_read_subrequest *subreq,
34
-
bool was_async)
35
-
{
36
-
if (refcount_dec_and_test(&subreq->usage))
37
-
__netfs_put_subrequest(subreq, was_async);
38
-
}
39
-
40
-
static struct netfs_read_request *netfs_alloc_read_request(
41
-
const struct netfs_read_request_ops *ops, void *netfs_priv,
42
-
struct file *file)
43
-
{
44
-
static atomic_t debug_ids;
45
-
struct netfs_read_request *rreq;
46
-
47
-
rreq = kzalloc(sizeof(struct netfs_read_request), GFP_KERNEL);
48
-
if (rreq) {
49
-
rreq->netfs_ops = ops;
50
-
rreq->netfs_priv = netfs_priv;
51
-
rreq->inode = file_inode(file);
52
-
rreq->i_size = i_size_read(rreq->inode);
53
-
rreq->debug_id = atomic_inc_return(&debug_ids);
54
-
INIT_LIST_HEAD(&rreq->subrequests);
55
-
INIT_WORK(&rreq->work, netfs_rreq_work);
56
-
refcount_set(&rreq->usage, 1);
57
-
__set_bit(NETFS_RREQ_IN_PROGRESS, &rreq->flags);
58
-
if (ops->init_rreq)
59
-
ops->init_rreq(rreq, file);
60
-
netfs_stat(&netfs_n_rh_rreq);
61
-
}
62
-
63
-
return rreq;
64
-
}
65
-
66
-
static void netfs_get_read_request(struct netfs_read_request *rreq)
67
-
{
68
-
refcount_inc(&rreq->usage);
69
-
}
70
-
71
-
static void netfs_rreq_clear_subreqs(struct netfs_read_request *rreq,
72
-
bool was_async)
73
-
{
74
-
struct netfs_read_subrequest *subreq;
75
-
76
-
while (!list_empty(&rreq->subrequests)) {
77
-
subreq = list_first_entry(&rreq->subrequests,
78
-
struct netfs_read_subrequest, rreq_link);
79
-
list_del(&subreq->rreq_link);
80
-
netfs_put_subrequest(subreq, was_async);
81
-
}
82
-
}
83
-
84
-
static void netfs_free_read_request(struct work_struct *work)
85
-
{
86
-
struct netfs_read_request *rreq =
87
-
container_of(work, struct netfs_read_request, work);
88
-
netfs_rreq_clear_subreqs(rreq, false);
89
-
if (rreq->netfs_priv)
90
-
rreq->netfs_ops->cleanup(rreq->mapping, rreq->netfs_priv);
91
-
trace_netfs_rreq(rreq, netfs_rreq_trace_free);
92
-
if (rreq->cache_resources.ops)
93
-
rreq->cache_resources.ops->end_operation(&rreq->cache_resources);
94
-
kfree(rreq);
95
-
netfs_stat_d(&netfs_n_rh_rreq);
96
-
}
97
-
98
-
static void netfs_put_read_request(struct netfs_read_request *rreq, bool was_async)
99
-
{
100
-
if (refcount_dec_and_test(&rreq->usage)) {
101
-
if (was_async) {
102
-
rreq->work.func = netfs_free_read_request;
103
-
if (!queue_work(system_unbound_wq, &rreq->work))
104
-
BUG();
105
-
} else {
106
-
netfs_free_read_request(&rreq->work);
107
-
}
108
-
}
109
-
}
110
-
111
-
/*
112
-
* Allocate and partially initialise an I/O request structure.
113
-
*/
114
-
static struct netfs_read_subrequest *netfs_alloc_subrequest(
115
-
struct netfs_read_request *rreq)
116
-
{
117
-
struct netfs_read_subrequest *subreq;
118
-
119
-
subreq = kzalloc(sizeof(struct netfs_read_subrequest), GFP_KERNEL);
120
-
if (subreq) {
121
-
INIT_LIST_HEAD(&subreq->rreq_link);
122
-
refcount_set(&subreq->usage, 2);
123
-
subreq->rreq = rreq;
124
-
netfs_get_read_request(rreq);
125
-
netfs_stat(&netfs_n_rh_sreq);
126
-
}
127
-
128
-
return subreq;
129
-
}
130
-
131
-
static void netfs_get_read_subrequest(struct netfs_read_subrequest *subreq)
132
-
{
133
-
refcount_inc(&subreq->usage);
134
-
}
135
-
136
-
static void __netfs_put_subrequest(struct netfs_read_subrequest *subreq,
137
-
bool was_async)
138
-
{
139
-
struct netfs_read_request *rreq = subreq->rreq;
140
-
141
-
trace_netfs_sreq(subreq, netfs_sreq_trace_free);
142
-
kfree(subreq);
143
-
netfs_stat_d(&netfs_n_rh_sreq);
144
-
netfs_put_read_request(rreq, was_async);
145
-
}
146
-
147
-
/*
148
-
* Clear the unread part of an I/O request.
149
-
*/
150
-
static void netfs_clear_unread(struct netfs_read_subrequest *subreq)
151
-
{
152
-
struct iov_iter iter;
153
-
154
-
iov_iter_xarray(&iter, READ, &subreq->rreq->mapping->i_pages,
155
-
subreq->start + subreq->transferred,
156
-
subreq->len - subreq->transferred);
157
-
iov_iter_zero(iov_iter_count(&iter), &iter);
158
-
}
159
-
160
-
static void netfs_cache_read_terminated(void *priv, ssize_t transferred_or_error,
161
-
bool was_async)
162
-
{
163
-
struct netfs_read_subrequest *subreq = priv;
164
-
165
-
netfs_subreq_terminated(subreq, transferred_or_error, was_async);
166
-
}
167
-
168
-
/*
169
-
* Issue a read against the cache.
170
-
* - Eats the caller's ref on subreq.
171
-
*/
172
-
static void netfs_read_from_cache(struct netfs_read_request *rreq,
173
-
struct netfs_read_subrequest *subreq,
174
-
enum netfs_read_from_hole read_hole)
175
-
{
176
-
struct netfs_cache_resources *cres = &rreq->cache_resources;
177
-
struct iov_iter iter;
178
-
179
-
netfs_stat(&netfs_n_rh_read);
180
-
iov_iter_xarray(&iter, READ, &rreq->mapping->i_pages,
181
-
subreq->start + subreq->transferred,
182
-
subreq->len - subreq->transferred);
183
-
184
-
cres->ops->read(cres, subreq->start, &iter, read_hole,
185
-
netfs_cache_read_terminated, subreq);
186
-
}
187
-
188
-
/*
189
-
* Fill a subrequest region with zeroes.
190
-
*/
191
-
static void netfs_fill_with_zeroes(struct netfs_read_request *rreq,
192
-
struct netfs_read_subrequest *subreq)
193
-
{
194
-
netfs_stat(&netfs_n_rh_zero);
195
-
__set_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags);
196
-
netfs_subreq_terminated(subreq, 0, false);
197
-
}
198
-
199
-
/*
200
-
* Ask the netfs to issue a read request to the server for us.
201
-
*
202
-
* The netfs is expected to read from subreq->pos + subreq->transferred to
203
-
* subreq->pos + subreq->len - 1. It may not backtrack and write data into the
204
-
* buffer prior to the transferred point as it might clobber dirty data
205
-
* obtained from the cache.
206
-
*
207
-
* Alternatively, the netfs is allowed to indicate one of two things:
208
-
*
209
-
* - NETFS_SREQ_SHORT_READ: A short read - it will get called again to try and
210
-
* make progress.
211
-
*
212
-
* - NETFS_SREQ_CLEAR_TAIL: A short read - the rest of the buffer will be
213
-
* cleared.
214
-
*/
215
-
static void netfs_read_from_server(struct netfs_read_request *rreq,
216
-
struct netfs_read_subrequest *subreq)
217
-
{
218
-
netfs_stat(&netfs_n_rh_download);
219
-
rreq->netfs_ops->issue_op(subreq);
220
-
}
221
-
222
-
/*
223
-
* Release those waiting.
224
-
*/
225
-
static void netfs_rreq_completed(struct netfs_read_request *rreq, bool was_async)
226
-
{
227
-
trace_netfs_rreq(rreq, netfs_rreq_trace_done);
228
-
netfs_rreq_clear_subreqs(rreq, was_async);
229
-
netfs_put_read_request(rreq, was_async);
230
-
}
231
-
232
-
/*
233
-
* Deal with the completion of writing the data to the cache. We have to clear
234
-
* the PG_fscache bits on the folios involved and release the caller's ref.
235
-
*
236
-
* May be called in softirq mode and we inherit a ref from the caller.
237
-
*/
238
-
static void netfs_rreq_unmark_after_write(struct netfs_read_request *rreq,
239
-
bool was_async)
240
-
{
241
-
struct netfs_read_subrequest *subreq;
242
-
struct folio *folio;
243
-
pgoff_t unlocked = 0;
244
-
bool have_unlocked = false;
245
-
246
-
rcu_read_lock();
247
-
248
-
list_for_each_entry(subreq, &rreq->subrequests, rreq_link) {
249
-
XA_STATE(xas, &rreq->mapping->i_pages, subreq->start / PAGE_SIZE);
250
-
251
-
xas_for_each(&xas, folio, (subreq->start + subreq->len - 1) / PAGE_SIZE) {
252
-
/* We might have multiple writes from the same huge
253
-
* folio, but we mustn't unlock a folio more than once.
254
-
*/
255
-
if (have_unlocked && folio_index(folio) <= unlocked)
256
-
continue;
257
-
unlocked = folio_index(folio);
258
-
folio_end_fscache(folio);
259
-
have_unlocked = true;
260
-
}
261
-
}
262
-
263
-
rcu_read_unlock();
264
-
netfs_rreq_completed(rreq, was_async);
265
-
}
266
-
267
-
static void netfs_rreq_copy_terminated(void *priv, ssize_t transferred_or_error,
268
-
bool was_async)
269
-
{
270
-
struct netfs_read_subrequest *subreq = priv;
271
-
struct netfs_read_request *rreq = subreq->rreq;
272
-
273
-
if (IS_ERR_VALUE(transferred_or_error)) {
274
-
netfs_stat(&netfs_n_rh_write_failed);
275
-
trace_netfs_failure(rreq, subreq, transferred_or_error,
276
-
netfs_fail_copy_to_cache);
277
-
} else {
278
-
netfs_stat(&netfs_n_rh_write_done);
279
-
}
280
-
281
-
trace_netfs_sreq(subreq, netfs_sreq_trace_write_term);
282
-
283
-
/* If we decrement nr_wr_ops to 0, the ref belongs to us. */
284
-
if (atomic_dec_and_test(&rreq->nr_wr_ops))
285
-
netfs_rreq_unmark_after_write(rreq, was_async);
286
-
287
-
netfs_put_subrequest(subreq, was_async);
288
-
}
289
-
290
-
/*
291
-
* Perform any outstanding writes to the cache. We inherit a ref from the
292
-
* caller.
293
-
*/
294
-
static void netfs_rreq_do_write_to_cache(struct netfs_read_request *rreq)
295
-
{
296
-
struct netfs_cache_resources *cres = &rreq->cache_resources;
297
-
struct netfs_read_subrequest *subreq, *next, *p;
298
-
struct iov_iter iter;
299
-
int ret;
300
-
301
-
trace_netfs_rreq(rreq, netfs_rreq_trace_write);
302
-
303
-
/* We don't want terminating writes trying to wake us up whilst we're
304
-
* still going through the list.
305
-
*/
306
-
atomic_inc(&rreq->nr_wr_ops);
307
-
308
-
list_for_each_entry_safe(subreq, p, &rreq->subrequests, rreq_link) {
309
-
if (!test_bit(NETFS_SREQ_WRITE_TO_CACHE, &subreq->flags)) {
310
-
list_del_init(&subreq->rreq_link);
311
-
netfs_put_subrequest(subreq, false);
312
-
}
313
-
}
314
-
315
-
list_for_each_entry(subreq, &rreq->subrequests, rreq_link) {
316
-
/* Amalgamate adjacent writes */
317
-
while (!list_is_last(&subreq->rreq_link, &rreq->subrequests)) {
318
-
next = list_next_entry(subreq, rreq_link);
319
-
if (next->start != subreq->start + subreq->len)
320
-
break;
321
-
subreq->len += next->len;
322
-
list_del_init(&next->rreq_link);
323
-
netfs_put_subrequest(next, false);
324
-
}
325
-
326
-
ret = cres->ops->prepare_write(cres, &subreq->start, &subreq->len,
327
-
rreq->i_size, true);
328
-
if (ret < 0) {
329
-
trace_netfs_failure(rreq, subreq, ret, netfs_fail_prepare_write);
330
-
trace_netfs_sreq(subreq, netfs_sreq_trace_write_skip);
331
-
continue;
332
-
}
333
-
334
-
iov_iter_xarray(&iter, WRITE, &rreq->mapping->i_pages,
335
-
subreq->start, subreq->len);
336
-
337
-
atomic_inc(&rreq->nr_wr_ops);
338
-
netfs_stat(&netfs_n_rh_write);
339
-
netfs_get_read_subrequest(subreq);
340
-
trace_netfs_sreq(subreq, netfs_sreq_trace_write);
341
-
cres->ops->write(cres, subreq->start, &iter,
342
-
netfs_rreq_copy_terminated, subreq);
343
-
}
344
-
345
-
/* If we decrement nr_wr_ops to 0, the usage ref belongs to us. */
346
-
if (atomic_dec_and_test(&rreq->nr_wr_ops))
347
-
netfs_rreq_unmark_after_write(rreq, false);
348
-
}
349
-
350
-
static void netfs_rreq_write_to_cache_work(struct work_struct *work)
351
-
{
352
-
struct netfs_read_request *rreq =
353
-
container_of(work, struct netfs_read_request, work);
354
-
355
-
netfs_rreq_do_write_to_cache(rreq);
356
-
}
357
-
358
-
static void netfs_rreq_write_to_cache(struct netfs_read_request *rreq)
359
-
{
360
-
rreq->work.func = netfs_rreq_write_to_cache_work;
361
-
if (!queue_work(system_unbound_wq, &rreq->work))
362
-
BUG();
363
-
}
364
-
365
-
/*
366
-
* Unlock the folios in a read operation. We need to set PG_fscache on any
367
-
* folios we're going to write back before we unlock them.
368
-
*/
369
-
static void netfs_rreq_unlock(struct netfs_read_request *rreq)
370
-
{
371
-
struct netfs_read_subrequest *subreq;
372
-
struct folio *folio;
373
-
unsigned int iopos, account = 0;
374
-
pgoff_t start_page = rreq->start / PAGE_SIZE;
375
-
pgoff_t last_page = ((rreq->start + rreq->len) / PAGE_SIZE) - 1;
376
-
bool subreq_failed = false;
377
-
378
-
XA_STATE(xas, &rreq->mapping->i_pages, start_page);
379
-
380
-
if (test_bit(NETFS_RREQ_FAILED, &rreq->flags)) {
381
-
__clear_bit(NETFS_RREQ_WRITE_TO_CACHE, &rreq->flags);
382
-
list_for_each_entry(subreq, &rreq->subrequests, rreq_link) {
383
-
__clear_bit(NETFS_SREQ_WRITE_TO_CACHE, &subreq->flags);
384
-
}
385
-
}
386
-
387
-
/* Walk through the pagecache and the I/O request lists simultaneously.
388
-
* We may have a mixture of cached and uncached sections and we only
389
-
* really want to write out the uncached sections. This is slightly
390
-
* complicated by the possibility that we might have huge pages with a
391
-
* mixture inside.
392
-
*/
393
-
subreq = list_first_entry(&rreq->subrequests,
394
-
struct netfs_read_subrequest, rreq_link);
395
-
iopos = 0;
396
-
subreq_failed = (subreq->error < 0);
397
-
398
-
trace_netfs_rreq(rreq, netfs_rreq_trace_unlock);
399
-
400
-
rcu_read_lock();
401
-
xas_for_each(&xas, folio, last_page) {
402
-
unsigned int pgpos = (folio_index(folio) - start_page) * PAGE_SIZE;
403
-
unsigned int pgend = pgpos + folio_size(folio);
404
-
bool pg_failed = false;
405
-
406
-
for (;;) {
407
-
if (!subreq) {
408
-
pg_failed = true;
409
-
break;
410
-
}
411
-
if (test_bit(NETFS_SREQ_WRITE_TO_CACHE, &subreq->flags))
412
-
folio_start_fscache(folio);
413
-
pg_failed |= subreq_failed;
414
-
if (pgend < iopos + subreq->len)
415
-
break;
416
-
417
-
account += subreq->transferred;
418
-
iopos += subreq->len;
419
-
if (!list_is_last(&subreq->rreq_link, &rreq->subrequests)) {
420
-
subreq = list_next_entry(subreq, rreq_link);
421
-
subreq_failed = (subreq->error < 0);
422
-
} else {
423
-
subreq = NULL;
424
-
subreq_failed = false;
425
-
}
426
-
if (pgend == iopos)
427
-
break;
428
-
}
429
-
430
-
if (!pg_failed) {
431
-
flush_dcache_folio(folio);
432
-
folio_mark_uptodate(folio);
433
-
}
434
-
435
-
if (!test_bit(NETFS_RREQ_DONT_UNLOCK_FOLIOS, &rreq->flags)) {
436
-
if (folio_index(folio) == rreq->no_unlock_folio &&
437
-
test_bit(NETFS_RREQ_NO_UNLOCK_FOLIO, &rreq->flags))
438
-
_debug("no unlock");
439
-
else
440
-
folio_unlock(folio);
441
-
}
442
-
}
443
-
rcu_read_unlock();
444
-
445
-
task_io_account_read(account);
446
-
if (rreq->netfs_ops->done)
447
-
rreq->netfs_ops->done(rreq);
448
-
}
449
-
450
-
/*
451
-
* Handle a short read.
452
-
*/
453
-
static void netfs_rreq_short_read(struct netfs_read_request *rreq,
454
-
struct netfs_read_subrequest *subreq)
455
-
{
456
-
__clear_bit(NETFS_SREQ_SHORT_READ, &subreq->flags);
457
-
__set_bit(NETFS_SREQ_SEEK_DATA_READ, &subreq->flags);
458
-
459
-
netfs_stat(&netfs_n_rh_short_read);
460
-
trace_netfs_sreq(subreq, netfs_sreq_trace_resubmit_short);
461
-
462
-
netfs_get_read_subrequest(subreq);
463
-
atomic_inc(&rreq->nr_rd_ops);
464
-
if (subreq->source == NETFS_READ_FROM_CACHE)
465
-
netfs_read_from_cache(rreq, subreq, NETFS_READ_HOLE_CLEAR);
466
-
else
467
-
netfs_read_from_server(rreq, subreq);
468
-
}
469
-
470
-
/*
471
-
* Resubmit any short or failed operations. Returns true if we got the rreq
472
-
* ref back.
473
-
*/
474
-
static bool netfs_rreq_perform_resubmissions(struct netfs_read_request *rreq)
475
-
{
476
-
struct netfs_read_subrequest *subreq;
477
-
478
-
WARN_ON(in_interrupt());
479
-
480
-
trace_netfs_rreq(rreq, netfs_rreq_trace_resubmit);
481
-
482
-
/* We don't want terminating submissions trying to wake us up whilst
483
-
* we're still going through the list.
484
-
*/
485
-
atomic_inc(&rreq->nr_rd_ops);
486
-
487
-
__clear_bit(NETFS_RREQ_INCOMPLETE_IO, &rreq->flags);
488
-
list_for_each_entry(subreq, &rreq->subrequests, rreq_link) {
489
-
if (subreq->error) {
490
-
if (subreq->source != NETFS_READ_FROM_CACHE)
491
-
break;
492
-
subreq->source = NETFS_DOWNLOAD_FROM_SERVER;
493
-
subreq->error = 0;
494
-
netfs_stat(&netfs_n_rh_download_instead);
495
-
trace_netfs_sreq(subreq, netfs_sreq_trace_download_instead);
496
-
netfs_get_read_subrequest(subreq);
497
-
atomic_inc(&rreq->nr_rd_ops);
498
-
netfs_read_from_server(rreq, subreq);
499
-
} else if (test_bit(NETFS_SREQ_SHORT_READ, &subreq->flags)) {
500
-
netfs_rreq_short_read(rreq, subreq);
501
-
}
502
-
}
503
-
504
-
/* If we decrement nr_rd_ops to 0, the usage ref belongs to us. */
505
-
if (atomic_dec_and_test(&rreq->nr_rd_ops))
506
-
return true;
507
-
508
-
wake_up_var(&rreq->nr_rd_ops);
509
-
return false;
510
-
}
511
-
512
-
/*
513
-
* Check to see if the data read is still valid.
514
-
*/
515
-
static void netfs_rreq_is_still_valid(struct netfs_read_request *rreq)
516
-
{
517
-
struct netfs_read_subrequest *subreq;
518
-
519
-
if (!rreq->netfs_ops->is_still_valid ||
520
-
rreq->netfs_ops->is_still_valid(rreq))
521
-
return;
522
-
523
-
list_for_each_entry(subreq, &rreq->subrequests, rreq_link) {
524
-
if (subreq->source == NETFS_READ_FROM_CACHE) {
525
-
subreq->error = -ESTALE;
526
-
__set_bit(NETFS_RREQ_INCOMPLETE_IO, &rreq->flags);
527
-
}
528
-
}
529
-
}
530
-
531
-
/*
532
-
* Assess the state of a read request and decide what to do next.
533
-
*
534
-
* Note that we could be in an ordinary kernel thread, on a workqueue or in
535
-
* softirq context at this point. We inherit a ref from the caller.
536
-
*/
537
-
static void netfs_rreq_assess(struct netfs_read_request *rreq, bool was_async)
538
-
{
539
-
trace_netfs_rreq(rreq, netfs_rreq_trace_assess);
540
-
541
-
again:
542
-
netfs_rreq_is_still_valid(rreq);
543
-
544
-
if (!test_bit(NETFS_RREQ_FAILED, &rreq->flags) &&
545
-
test_bit(NETFS_RREQ_INCOMPLETE_IO, &rreq->flags)) {
546
-
if (netfs_rreq_perform_resubmissions(rreq))
547
-
goto again;
548
-
return;
549
-
}
550
-
551
-
netfs_rreq_unlock(rreq);
552
-
553
-
clear_bit_unlock(NETFS_RREQ_IN_PROGRESS, &rreq->flags);
554
-
wake_up_bit(&rreq->flags, NETFS_RREQ_IN_PROGRESS);
555
-
556
-
if (test_bit(NETFS_RREQ_WRITE_TO_CACHE, &rreq->flags))
557
-
return netfs_rreq_write_to_cache(rreq);
558
-
559
-
netfs_rreq_completed(rreq, was_async);
560
-
}
561
-
562
-
static void netfs_rreq_work(struct work_struct *work)
563
-
{
564
-
struct netfs_read_request *rreq =
565
-
container_of(work, struct netfs_read_request, work);
566
-
netfs_rreq_assess(rreq, false);
567
-
}
568
-
569
-
/*
570
-
* Handle the completion of all outstanding I/O operations on a read request.
571
-
* We inherit a ref from the caller.
572
-
*/
573
-
static void netfs_rreq_terminated(struct netfs_read_request *rreq,
574
-
bool was_async)
575
-
{
576
-
if (test_bit(NETFS_RREQ_INCOMPLETE_IO, &rreq->flags) &&
577
-
was_async) {
578
-
if (!queue_work(system_unbound_wq, &rreq->work))
579
-
BUG();
580
-
} else {
581
-
netfs_rreq_assess(rreq, was_async);
582
-
}
583
-
}
584
-
585
-
/**
586
-
* netfs_subreq_terminated - Note the termination of an I/O operation.
587
-
* @subreq: The I/O request that has terminated.
588
-
* @transferred_or_error: The amount of data transferred or an error code.
589
-
* @was_async: The termination was asynchronous
590
-
*
591
-
* This tells the read helper that a contributory I/O operation has terminated,
592
-
* one way or another, and that it should integrate the results.
593
-
*
594
-
* The caller indicates in @transferred_or_error the outcome of the operation,
595
-
* supplying a positive value to indicate the number of bytes transferred, 0 to
596
-
* indicate a failure to transfer anything that should be retried or a negative
597
-
* error code. The helper will look after reissuing I/O operations as
598
-
* appropriate and writing downloaded data to the cache.
599
-
*
600
-
* If @was_async is true, the caller might be running in softirq or interrupt
601
-
* context and we can't sleep.
602
-
*/
603
-
void netfs_subreq_terminated(struct netfs_read_subrequest *subreq,
604
-
ssize_t transferred_or_error,
605
-
bool was_async)
606
-
{
607
-
struct netfs_read_request *rreq = subreq->rreq;
608
-
int u;
609
-
610
-
_enter("[%u]{%llx,%lx},%zd",
611
-
subreq->debug_index, subreq->start, subreq->flags,
612
-
transferred_or_error);
613
-
614
-
switch (subreq->source) {
615
-
case NETFS_READ_FROM_CACHE:
616
-
netfs_stat(&netfs_n_rh_read_done);
617
-
break;
618
-
case NETFS_DOWNLOAD_FROM_SERVER:
619
-
netfs_stat(&netfs_n_rh_download_done);
620
-
break;
621
-
default:
622
-
break;
623
-
}
624
-
625
-
if (IS_ERR_VALUE(transferred_or_error)) {
626
-
subreq->error = transferred_or_error;
627
-
trace_netfs_failure(rreq, subreq, transferred_or_error,
628
-
netfs_fail_read);
629
-
goto failed;
630
-
}
631
-
632
-
if (WARN(transferred_or_error > subreq->len - subreq->transferred,
633
-
"Subreq overread: R%x[%x] %zd > %zu - %zu",
634
-
rreq->debug_id, subreq->debug_index,
635
-
transferred_or_error, subreq->len, subreq->transferred))
636
-
transferred_or_error = subreq->len - subreq->transferred;
637
-
638
-
subreq->error = 0;
639
-
subreq->transferred += transferred_or_error;
640
-
if (subreq->transferred < subreq->len)
641
-
goto incomplete;
642
-
643
-
complete:
644
-
__clear_bit(NETFS_SREQ_NO_PROGRESS, &subreq->flags);
645
-
if (test_bit(NETFS_SREQ_WRITE_TO_CACHE, &subreq->flags))
646
-
set_bit(NETFS_RREQ_WRITE_TO_CACHE, &rreq->flags);
647
-
648
-
out:
649
-
trace_netfs_sreq(subreq, netfs_sreq_trace_terminated);
650
-
651
-
/* If we decrement nr_rd_ops to 0, the ref belongs to us. */
652
-
u = atomic_dec_return(&rreq->nr_rd_ops);
653
-
if (u == 0)
654
-
netfs_rreq_terminated(rreq, was_async);
655
-
else if (u == 1)
656
-
wake_up_var(&rreq->nr_rd_ops);
657
-
658
-
netfs_put_subrequest(subreq, was_async);
659
-
return;
660
-
661
-
incomplete:
662
-
if (test_bit(NETFS_SREQ_CLEAR_TAIL, &subreq->flags)) {
663
-
netfs_clear_unread(subreq);
664
-
subreq->transferred = subreq->len;
665
-
goto complete;
666
-
}
667
-
668
-
if (transferred_or_error == 0) {
669
-
if (__test_and_set_bit(NETFS_SREQ_NO_PROGRESS, &subreq->flags)) {
670
-
subreq->error = -ENODATA;
671
-
goto failed;
672
-
}
673
-
} else {
674
-
__clear_bit(NETFS_SREQ_NO_PROGRESS, &subreq->flags);
675
-
}
676
-
677
-
__set_bit(NETFS_SREQ_SHORT_READ, &subreq->flags);
678
-
set_bit(NETFS_RREQ_INCOMPLETE_IO, &rreq->flags);
679
-
goto out;
680
-
681
-
failed:
682
-
if (subreq->source == NETFS_READ_FROM_CACHE) {
683
-
netfs_stat(&netfs_n_rh_read_failed);
684
-
set_bit(NETFS_RREQ_INCOMPLETE_IO, &rreq->flags);
685
-
} else {
686
-
netfs_stat(&netfs_n_rh_download_failed);
687
-
set_bit(NETFS_RREQ_FAILED, &rreq->flags);
688
-
rreq->error = subreq->error;
689
-
}
690
-
goto out;
691
-
}
692
-
EXPORT_SYMBOL(netfs_subreq_terminated);
693
-
694
-
static enum netfs_read_source netfs_cache_prepare_read(struct netfs_read_subrequest *subreq,
695
-
loff_t i_size)
696
-
{
697
-
struct netfs_read_request *rreq = subreq->rreq;
698
-
struct netfs_cache_resources *cres = &rreq->cache_resources;
699
-
700
-
if (cres->ops)
701
-
return cres->ops->prepare_read(subreq, i_size);
702
-
if (subreq->start >= rreq->i_size)
703
-
return NETFS_FILL_WITH_ZEROES;
704
-
return NETFS_DOWNLOAD_FROM_SERVER;
705
-
}
706
-
707
-
/*
708
-
* Work out what sort of subrequest the next one will be.
709
-
*/
710
-
static enum netfs_read_source
711
-
netfs_rreq_prepare_read(struct netfs_read_request *rreq,
712
-
struct netfs_read_subrequest *subreq)
713
-
{
714
-
enum netfs_read_source source;
715
-
716
-
_enter("%llx-%llx,%llx", subreq->start, subreq->start + subreq->len, rreq->i_size);
717
-
718
-
source = netfs_cache_prepare_read(subreq, rreq->i_size);
719
-
if (source == NETFS_INVALID_READ)
720
-
goto out;
721
-
722
-
if (source == NETFS_DOWNLOAD_FROM_SERVER) {
723
-
/* Call out to the netfs to let it shrink the request to fit
724
-
* its own I/O sizes and boundaries. If it shinks it here, it
725
-
* will be called again to make simultaneous calls; if it wants
726
-
* to make serial calls, it can indicate a short read and then
727
-
* we will call it again.
728
-
*/
729
-
if (subreq->len > rreq->i_size - subreq->start)
730
-
subreq->len = rreq->i_size - subreq->start;
731
-
732
-
if (rreq->netfs_ops->clamp_length &&
733
-
!rreq->netfs_ops->clamp_length(subreq)) {
734
-
source = NETFS_INVALID_READ;
735
-
goto out;
736
-
}
737
-
}
738
-
739
-
if (WARN_ON(subreq->len == 0))
740
-
source = NETFS_INVALID_READ;
741
-
742
-
out:
743
-
subreq->source = source;
744
-
trace_netfs_sreq(subreq, netfs_sreq_trace_prepare);
745
-
return source;
746
-
}
747
-
748
-
/*
749
-
* Slice off a piece of a read request and submit an I/O request for it.
750
-
*/
751
-
static bool netfs_rreq_submit_slice(struct netfs_read_request *rreq,
752
-
unsigned int *_debug_index)
753
-
{
754
-
struct netfs_read_subrequest *subreq;
755
-
enum netfs_read_source source;
756
-
757
-
subreq = netfs_alloc_subrequest(rreq);
758
-
if (!subreq)
759
-
return false;
760
-
761
-
subreq->debug_index = (*_debug_index)++;
762
-
subreq->start = rreq->start + rreq->submitted;
763
-
subreq->len = rreq->len - rreq->submitted;
764
-
765
-
_debug("slice %llx,%zx,%zx", subreq->start, subreq->len, rreq->submitted);
766
-
list_add_tail(&subreq->rreq_link, &rreq->subrequests);
767
-
768
-
/* Call out to the cache to find out what it can do with the remaining
769
-
* subset. It tells us in subreq->flags what it decided should be done
770
-
* and adjusts subreq->len down if the subset crosses a cache boundary.
771
-
*
772
-
* Then when we hand the subset, it can choose to take a subset of that
773
-
* (the starts must coincide), in which case, we go around the loop
774
-
* again and ask it to download the next piece.
775
-
*/
776
-
source = netfs_rreq_prepare_read(rreq, subreq);
777
-
if (source == NETFS_INVALID_READ)
778
-
goto subreq_failed;
779
-
780
-
atomic_inc(&rreq->nr_rd_ops);
781
-
782
-
rreq->submitted += subreq->len;
783
-
784
-
trace_netfs_sreq(subreq, netfs_sreq_trace_submit);
785
-
switch (source) {
786
-
case NETFS_FILL_WITH_ZEROES:
787
-
netfs_fill_with_zeroes(rreq, subreq);
788
-
break;
789
-
case NETFS_DOWNLOAD_FROM_SERVER:
790
-
netfs_read_from_server(rreq, subreq);
791
-
break;
792
-
case NETFS_READ_FROM_CACHE:
793
-
netfs_read_from_cache(rreq, subreq, NETFS_READ_HOLE_IGNORE);
794
-
break;
795
-
default:
796
-
BUG();
797
-
}
798
-
799
-
return true;
800
-
801
-
subreq_failed:
802
-
rreq->error = subreq->error;
803
-
netfs_put_subrequest(subreq, false);
804
-
return false;
805
-
}
806
-
807
-
static void netfs_cache_expand_readahead(struct netfs_read_request *rreq,
808
-
loff_t *_start, size_t *_len, loff_t i_size)
809
-
{
810
-
struct netfs_cache_resources *cres = &rreq->cache_resources;
811
-
812
-
if (cres->ops && cres->ops->expand_readahead)
813
-
cres->ops->expand_readahead(cres, _start, _len, i_size);
814
-
}
815
-
816
-
static void netfs_rreq_expand(struct netfs_read_request *rreq,
817
-
struct readahead_control *ractl)
818
-
{
819
-
/* Give the cache a chance to change the request parameters. The
820
-
* resultant request must contain the original region.
821
-
*/
822
-
netfs_cache_expand_readahead(rreq, &rreq->start, &rreq->len, rreq->i_size);
823
-
824
-
/* Give the netfs a chance to change the request parameters. The
825
-
* resultant request must contain the original region.
826
-
*/
827
-
if (rreq->netfs_ops->expand_readahead)
828
-
rreq->netfs_ops->expand_readahead(rreq);
829
-
830
-
/* Expand the request if the cache wants it to start earlier. Note
831
-
* that the expansion may get further extended if the VM wishes to
832
-
* insert THPs and the preferred start and/or end wind up in the middle
833
-
* of THPs.
834
-
*
835
-
* If this is the case, however, the THP size should be an integer
836
-
* multiple of the cache granule size, so we get a whole number of
837
-
* granules to deal with.
838
-
*/
839
-
if (rreq->start != readahead_pos(ractl) ||
840
-
rreq->len != readahead_length(ractl)) {
841
-
readahead_expand(ractl, rreq->start, rreq->len);
842
-
rreq->start = readahead_pos(ractl);
843
-
rreq->len = readahead_length(ractl);
844
-
845
-
trace_netfs_read(rreq, readahead_pos(ractl), readahead_length(ractl),
846
-
netfs_read_trace_expanded);
847
-
}
848
-
}
849
-
850
-
/**
851
-
* netfs_readahead - Helper to manage a read request
852
-
* @ractl: The description of the readahead request
853
-
* @ops: The network filesystem's operations for the helper to use
854
-
* @netfs_priv: Private netfs data to be retained in the request
855
-
*
856
-
* Fulfil a readahead request by drawing data from the cache if possible, or
857
-
* the netfs if not. Space beyond the EOF is zero-filled. Multiple I/O
858
-
* requests from different sources will get munged together. If necessary, the
859
-
* readahead window can be expanded in either direction to a more convenient
860
-
* alighment for RPC efficiency or to make storage in the cache feasible.
861
-
*
862
-
* The calling netfs must provide a table of operations, only one of which,
863
-
* issue_op, is mandatory. It may also be passed a private token, which will
864
-
* be retained in rreq->netfs_priv and will be cleaned up by ops->cleanup().
865
-
*
866
-
* This is usable whether or not caching is enabled.
867
-
*/
868
-
void netfs_readahead(struct readahead_control *ractl,
869
-
const struct netfs_read_request_ops *ops,
870
-
void *netfs_priv)
871
-
{
872
-
struct netfs_read_request *rreq;
873
-
unsigned int debug_index = 0;
874
-
int ret;
875
-
876
-
_enter("%lx,%x", readahead_index(ractl), readahead_count(ractl));
877
-
878
-
if (readahead_count(ractl) == 0)
879
-
goto cleanup;
880
-
881
-
rreq = netfs_alloc_read_request(ops, netfs_priv, ractl->file);
882
-
if (!rreq)
883
-
goto cleanup;
884
-
rreq->mapping = ractl->mapping;
885
-
rreq->start = readahead_pos(ractl);
886
-
rreq->len = readahead_length(ractl);
887
-
888
-
if (ops->begin_cache_operation) {
889
-
ret = ops->begin_cache_operation(rreq);
890
-
if (ret == -ENOMEM || ret == -EINTR || ret == -ERESTARTSYS)
891
-
goto cleanup_free;
892
-
}
893
-
894
-
netfs_stat(&netfs_n_rh_readahead);
895
-
trace_netfs_read(rreq, readahead_pos(ractl), readahead_length(ractl),
896
-
netfs_read_trace_readahead);
897
-
898
-
netfs_rreq_expand(rreq, ractl);
899
-
900
-
atomic_set(&rreq->nr_rd_ops, 1);
901
-
do {
902
-
if (!netfs_rreq_submit_slice(rreq, &debug_index))
903
-
break;
904
-
905
-
} while (rreq->submitted < rreq->len);
906
-
907
-
/* Drop the refs on the folios here rather than in the cache or
908
-
* filesystem. The locks will be dropped in netfs_rreq_unlock().
909
-
*/
910
-
while (readahead_folio(ractl))
911
-
;
912
-
913
-
/* If we decrement nr_rd_ops to 0, the ref belongs to us. */
914
-
if (atomic_dec_and_test(&rreq->nr_rd_ops))
915
-
netfs_rreq_assess(rreq, false);
916
-
return;
917
-
918
-
cleanup_free:
919
-
netfs_put_read_request(rreq, false);
920
-
return;
921
-
cleanup:
922
-
if (netfs_priv)
923
-
ops->cleanup(ractl->mapping, netfs_priv);
924
-
return;
925
-
}
926
-
EXPORT_SYMBOL(netfs_readahead);
927
-
928
-
/**
929
-
* netfs_readpage - Helper to manage a readpage request
930
-
* @file: The file to read from
931
-
* @folio: The folio to read
932
-
* @ops: The network filesystem's operations for the helper to use
933
-
* @netfs_priv: Private netfs data to be retained in the request
934
-
*
935
-
* Fulfil a readpage request by drawing data from the cache if possible, or the
936
-
* netfs if not. Space beyond the EOF is zero-filled. Multiple I/O requests
937
-
* from different sources will get munged together.
938
-
*
939
-
* The calling netfs must provide a table of operations, only one of which,
940
-
* issue_op, is mandatory. It may also be passed a private token, which will
941
-
* be retained in rreq->netfs_priv and will be cleaned up by ops->cleanup().
942
-
*
943
-
* This is usable whether or not caching is enabled.
944
-
*/
945
-
int netfs_readpage(struct file *file,
946
-
struct folio *folio,
947
-
const struct netfs_read_request_ops *ops,
948
-
void *netfs_priv)
949
-
{
950
-
struct netfs_read_request *rreq;
951
-
unsigned int debug_index = 0;
952
-
int ret;
953
-
954
-
_enter("%lx", folio_index(folio));
955
-
956
-
rreq = netfs_alloc_read_request(ops, netfs_priv, file);
957
-
if (!rreq) {
958
-
if (netfs_priv)
959
-
ops->cleanup(folio_file_mapping(folio), netfs_priv);
960
-
folio_unlock(folio);
961
-
return -ENOMEM;
962
-
}
963
-
rreq->mapping = folio_file_mapping(folio);
964
-
rreq->start = folio_file_pos(folio);
965
-
rreq->len = folio_size(folio);
966
-
967
-
if (ops->begin_cache_operation) {
968
-
ret = ops->begin_cache_operation(rreq);
969
-
if (ret == -ENOMEM || ret == -EINTR || ret == -ERESTARTSYS) {
970
-
folio_unlock(folio);
971
-
goto out;
972
-
}
973
-
}
974
-
975
-
netfs_stat(&netfs_n_rh_readpage);
976
-
trace_netfs_read(rreq, rreq->start, rreq->len, netfs_read_trace_readpage);
977
-
978
-
netfs_get_read_request(rreq);
979
-
980
-
atomic_set(&rreq->nr_rd_ops, 1);
981
-
do {
982
-
if (!netfs_rreq_submit_slice(rreq, &debug_index))
983
-
break;
984
-
985
-
} while (rreq->submitted < rreq->len);
986
-
987
-
/* Keep nr_rd_ops incremented so that the ref always belongs to us, and
988
-
* the service code isn't punted off to a random thread pool to
989
-
* process.
990
-
*/
991
-
do {
992
-
wait_var_event(&rreq->nr_rd_ops, atomic_read(&rreq->nr_rd_ops) == 1);
993
-
netfs_rreq_assess(rreq, false);
994
-
} while (test_bit(NETFS_RREQ_IN_PROGRESS, &rreq->flags));
995
-
996
-
ret = rreq->error;
997
-
if (ret == 0 && rreq->submitted < rreq->len) {
998
-
trace_netfs_failure(rreq, NULL, ret, netfs_fail_short_readpage);
999
-
ret = -EIO;
1000
-
}
1001
-
out:
1002
-
netfs_put_read_request(rreq, false);
1003
-
return ret;
1004
-
}
1005
-
EXPORT_SYMBOL(netfs_readpage);
1006
-
1007
-
/*
1008
-
* Prepare a folio for writing without reading first
1009
-
* @folio: The folio being prepared
1010
-
* @pos: starting position for the write
1011
-
* @len: length of write
1012
-
*
1013
-
* In some cases, write_begin doesn't need to read at all:
1014
-
* - full folio write
1015
-
* - write that lies in a folio that is completely beyond EOF
1016
-
* - write that covers the folio from start to EOF or beyond it
1017
-
*
1018
-
* If any of these criteria are met, then zero out the unwritten parts
1019
-
* of the folio and return true. Otherwise, return false.
1020
-
*/
1021
-
static bool netfs_skip_folio_read(struct folio *folio, loff_t pos, size_t len)
1022
-
{
1023
-
struct inode *inode = folio_inode(folio);
1024
-
loff_t i_size = i_size_read(inode);
1025
-
size_t offset = offset_in_folio(folio, pos);
1026
-
1027
-
/* Full folio write */
1028
-
if (offset == 0 && len >= folio_size(folio))
1029
-
return true;
1030
-
1031
-
/* pos beyond last folio in the file */
1032
-
if (pos - offset >= i_size)
1033
-
goto zero_out;
1034
-
1035
-
/* Write that covers from the start of the folio to EOF or beyond */
1036
-
if (offset == 0 && (pos + len) >= i_size)
1037
-
goto zero_out;
1038
-
1039
-
return false;
1040
-
zero_out:
1041
-
zero_user_segments(&folio->page, 0, offset, offset + len, folio_size(folio));
1042
-
return true;
1043
-
}
1044
-
1045
-
/**
1046
-
* netfs_write_begin - Helper to prepare for writing
1047
-
* @file: The file to read from
1048
-
* @mapping: The mapping to read from
1049
-
* @pos: File position at which the write will begin
1050
-
* @len: The length of the write (may extend beyond the end of the folio chosen)
1051
-
* @aop_flags: AOP_* flags
1052
-
* @_folio: Where to put the resultant folio
1053
-
* @_fsdata: Place for the netfs to store a cookie
1054
-
* @ops: The network filesystem's operations for the helper to use
1055
-
* @netfs_priv: Private netfs data to be retained in the request
1056
-
*
1057
-
* Pre-read data for a write-begin request by drawing data from the cache if
1058
-
* possible, or the netfs if not. Space beyond the EOF is zero-filled.
1059
-
* Multiple I/O requests from different sources will get munged together. If
1060
-
* necessary, the readahead window can be expanded in either direction to a
1061
-
* more convenient alighment for RPC efficiency or to make storage in the cache
1062
-
* feasible.
1063
-
*
1064
-
* The calling netfs must provide a table of operations, only one of which,
1065
-
* issue_op, is mandatory.
1066
-
*
1067
-
* The check_write_begin() operation can be provided to check for and flush
1068
-
* conflicting writes once the folio is grabbed and locked. It is passed a
1069
-
* pointer to the fsdata cookie that gets returned to the VM to be passed to
1070
-
* write_end. It is permitted to sleep. It should return 0 if the request
1071
-
* should go ahead; unlock the folio and return -EAGAIN to cause the folio to
1072
-
* be regot; or return an error.
1073
-
*
1074
-
* This is usable whether or not caching is enabled.
1075
-
*/
1076
-
int netfs_write_begin(struct file *file, struct address_space *mapping,
1077
-
loff_t pos, unsigned int len, unsigned int aop_flags,
1078
-
struct folio **_folio, void **_fsdata,
1079
-
const struct netfs_read_request_ops *ops,
1080
-
void *netfs_priv)
1081
-
{
1082
-
struct netfs_read_request *rreq;
1083
-
struct folio *folio;
1084
-
struct inode *inode = file_inode(file);
1085
-
unsigned int debug_index = 0, fgp_flags;
1086
-
pgoff_t index = pos >> PAGE_SHIFT;
1087
-
int ret;
1088
-
1089
-
DEFINE_READAHEAD(ractl, file, NULL, mapping, index);
1090
-
1091
-
retry:
1092
-
fgp_flags = FGP_LOCK | FGP_WRITE | FGP_CREAT | FGP_STABLE;
1093
-
if (aop_flags & AOP_FLAG_NOFS)
1094
-
fgp_flags |= FGP_NOFS;
1095
-
folio = __filemap_get_folio(mapping, index, fgp_flags,
1096
-
mapping_gfp_mask(mapping));
1097
-
if (!folio)
1098
-
return -ENOMEM;
1099
-
1100
-
if (ops->check_write_begin) {
1101
-
/* Allow the netfs (eg. ceph) to flush conflicts. */
1102
-
ret = ops->check_write_begin(file, pos, len, folio, _fsdata);
1103
-
if (ret < 0) {
1104
-
trace_netfs_failure(NULL, NULL, ret, netfs_fail_check_write_begin);
1105
-
if (ret == -EAGAIN)
1106
-
goto retry;
1107
-
goto error;
1108
-
}
1109
-
}
1110
-
1111
-
if (folio_test_uptodate(folio))
1112
-
goto have_folio;
1113
-
1114
-
/* If the page is beyond the EOF, we want to clear it - unless it's
1115
-
* within the cache granule containing the EOF, in which case we need
1116
-
* to preload the granule.
1117
-
*/
1118
-
if (!ops->is_cache_enabled(inode) &&
1119
-
netfs_skip_folio_read(folio, pos, len)) {
1120
-
netfs_stat(&netfs_n_rh_write_zskip);
1121
-
goto have_folio_no_wait;
1122
-
}
1123
-
1124
-
ret = -ENOMEM;
1125
-
rreq = netfs_alloc_read_request(ops, netfs_priv, file);
1126
-
if (!rreq)
1127
-
goto error;
1128
-
rreq->mapping = folio_file_mapping(folio);
1129
-
rreq->start = folio_file_pos(folio);
1130
-
rreq->len = folio_size(folio);
1131
-
rreq->no_unlock_folio = folio_index(folio);
1132
-
__set_bit(NETFS_RREQ_NO_UNLOCK_FOLIO, &rreq->flags);
1133
-
netfs_priv = NULL;
1134
-
1135
-
if (ops->begin_cache_operation) {
1136
-
ret = ops->begin_cache_operation(rreq);
1137
-
if (ret == -ENOMEM || ret == -EINTR || ret == -ERESTARTSYS)
1138
-
goto error_put;
1139
-
}
1140
-
1141
-
netfs_stat(&netfs_n_rh_write_begin);
1142
-
trace_netfs_read(rreq, pos, len, netfs_read_trace_write_begin);
1143
-
1144
-
/* Expand the request to meet caching requirements and download
1145
-
* preferences.
1146
-
*/
1147
-
ractl._nr_pages = folio_nr_pages(folio);
1148
-
netfs_rreq_expand(rreq, &ractl);
1149
-
netfs_get_read_request(rreq);
1150
-
1151
-
/* We hold the folio locks, so we can drop the references */
1152
-
folio_get(folio);
1153
-
while (readahead_folio(&ractl))
1154
-
;
1155
-
1156
-
atomic_set(&rreq->nr_rd_ops, 1);
1157
-
do {
1158
-
if (!netfs_rreq_submit_slice(rreq, &debug_index))
1159
-
break;
1160
-
1161
-
} while (rreq->submitted < rreq->len);
1162
-
1163
-
/* Keep nr_rd_ops incremented so that the ref always belongs to us, and
1164
-
* the service code isn't punted off to a random thread pool to
1165
-
* process.
1166
-
*/
1167
-
for (;;) {
1168
-
wait_var_event(&rreq->nr_rd_ops, atomic_read(&rreq->nr_rd_ops) == 1);
1169
-
netfs_rreq_assess(rreq, false);
1170
-
if (!test_bit(NETFS_RREQ_IN_PROGRESS, &rreq->flags))
1171
-
break;
1172
-
cond_resched();
1173
-
}
1174
-
1175
-
ret = rreq->error;
1176
-
if (ret == 0 && rreq->submitted < rreq->len) {
1177
-
trace_netfs_failure(rreq, NULL, ret, netfs_fail_short_write_begin);
1178
-
ret = -EIO;
1179
-
}
1180
-
netfs_put_read_request(rreq, false);
1181
-
if (ret < 0)
1182
-
goto error;
1183
-
1184
-
have_folio:
1185
-
ret = folio_wait_fscache_killable(folio);
1186
-
if (ret < 0)
1187
-
goto error;
1188
-
have_folio_no_wait:
1189
-
if (netfs_priv)
1190
-
ops->cleanup(mapping, netfs_priv);
1191
-
*_folio = folio;
1192
-
_leave(" = 0");
1193
-
return 0;
1194
-
1195
-
error_put:
1196
-
netfs_put_read_request(rreq, false);
1197
-
error:
1198
-
folio_unlock(folio);
1199
-
folio_put(folio);
1200
-
if (netfs_priv)
1201
-
ops->cleanup(mapping, netfs_priv);
1202
-
_leave(" = %d", ret);
1203
-
return ret;
1204
-
}
1205
-
EXPORT_SYMBOL(netfs_write_begin);
-1
fs/netfs/stats.c
-1
fs/netfs/stats.c
-8
fs/nfs/fscache.c
-8
fs/nfs/fscache.c
···
238
238
}
239
239
}
240
240
241
-
static inline void fscache_end_operation(struct netfs_cache_resources *cres)
242
-
{
243
-
const struct netfs_cache_ops *ops = fscache_operation_valid(cres);
244
-
245
-
if (ops)
246
-
ops->end_operation(cres);
247
-
}
248
-
249
241
/*
250
242
* Fallback page reading interface.
251
243
*/
+14
include/linux/fscache.h
+14
include/linux/fscache.h
···
457
457
}
458
458
459
459
/**
460
+
* fscache_end_operation - End the read operation for the netfs lib
461
+
* @cres: The cache resources for the read operation
462
+
*
463
+
* Clean up the resources at the end of the read request.
464
+
*/
465
+
static inline void fscache_end_operation(struct netfs_cache_resources *cres)
466
+
{
467
+
const struct netfs_cache_ops *ops = fscache_operation_valid(cres);
468
+
469
+
if (ops)
470
+
ops->end_operation(cres);
471
+
}
472
+
473
+
/**
460
474
* fscache_read - Start a read from the cache.
461
475
* @cres: The cache resources to use
462
476
* @start_pos: The beginning file offset in the cache file
+125
-37
include/linux/netfs.h
+125
-37
include/linux/netfs.h
···
18
18
#include <linux/fs.h>
19
19
#include <linux/pagemap.h>
20
20
21
+
enum netfs_sreq_ref_trace;
22
+
21
23
/*
22
24
* Overload PG_private_2 to give us PG_fscache - this is used to indicate that
23
25
* a page is currently backed by a local disk cache
···
108
106
return folio_wait_private_2_killable(page_folio(page));
109
107
}
110
108
111
-
enum netfs_read_source {
109
+
enum netfs_io_source {
112
110
NETFS_FILL_WITH_ZEROES,
113
111
NETFS_DOWNLOAD_FROM_SERVER,
114
112
NETFS_READ_FROM_CACHE,
···
117
115
118
116
typedef void (*netfs_io_terminated_t)(void *priv, ssize_t transferred_or_error,
119
117
bool was_async);
118
+
119
+
/*
120
+
* Per-inode description. This must be directly after the inode struct.
121
+
*/
122
+
struct netfs_i_context {
123
+
const struct netfs_request_ops *ops;
124
+
#if IS_ENABLED(CONFIG_FSCACHE)
125
+
struct fscache_cookie *cache;
126
+
#endif
127
+
loff_t remote_i_size; /* Size of the remote file */
128
+
};
120
129
121
130
/*
122
131
* Resources required to do operations on a cache.
···
143
130
/*
144
131
* Descriptor for a single component subrequest.
145
132
*/
146
-
struct netfs_read_subrequest {
147
-
struct netfs_read_request *rreq; /* Supervising read request */
133
+
struct netfs_io_subrequest {
134
+
struct netfs_io_request *rreq; /* Supervising I/O request */
148
135
struct list_head rreq_link; /* Link in rreq->subrequests */
149
136
loff_t start; /* Where to start the I/O */
150
137
size_t len; /* Size of the I/O */
151
138
size_t transferred; /* Amount of data transferred */
152
-
refcount_t usage;
139
+
refcount_t ref;
153
140
short error; /* 0 or error that occurred */
154
141
unsigned short debug_index; /* Index in list (for debugging output) */
155
-
enum netfs_read_source source; /* Where to read from */
142
+
enum netfs_io_source source; /* Where to read from/write to */
156
143
unsigned long flags;
157
-
#define NETFS_SREQ_WRITE_TO_CACHE 0 /* Set if should write to cache */
144
+
#define NETFS_SREQ_COPY_TO_CACHE 0 /* Set if should copy the data to the cache */
158
145
#define NETFS_SREQ_CLEAR_TAIL 1 /* Set if the rest of the read should be cleared */
159
-
#define NETFS_SREQ_SHORT_READ 2 /* Set if there was a short read from the cache */
146
+
#define NETFS_SREQ_SHORT_IO 2 /* Set if the I/O was short */
160
147
#define NETFS_SREQ_SEEK_DATA_READ 3 /* Set if ->read() should SEEK_DATA first */
161
148
#define NETFS_SREQ_NO_PROGRESS 4 /* Set if we didn't manage to read any data */
162
149
};
163
150
151
+
enum netfs_io_origin {
152
+
NETFS_READAHEAD, /* This read was triggered by readahead */
153
+
NETFS_READPAGE, /* This read is a synchronous read */
154
+
NETFS_READ_FOR_WRITE, /* This read is to prepare a write */
155
+
} __mode(byte);
156
+
164
157
/*
165
-
* Descriptor for a read helper request. This is used to make multiple I/O
166
-
* requests on a variety of sources and then stitch the result together.
158
+
* Descriptor for an I/O helper request. This is used to make multiple I/O
159
+
* operations to a variety of data stores and then stitch the result together.
167
160
*/
168
-
struct netfs_read_request {
161
+
struct netfs_io_request {
169
162
struct work_struct work;
170
163
struct inode *inode; /* The file being accessed */
171
164
struct address_space *mapping; /* The mapping being accessed */
172
165
struct netfs_cache_resources cache_resources;
173
-
struct list_head subrequests; /* Requests to fetch I/O from disk or net */
166
+
struct list_head subrequests; /* Contributory I/O operations */
174
167
void *netfs_priv; /* Private data for the netfs */
175
168
unsigned int debug_id;
176
-
atomic_t nr_rd_ops; /* Number of read ops in progress */
177
-
atomic_t nr_wr_ops; /* Number of write ops in progress */
169
+
atomic_t nr_outstanding; /* Number of ops in progress */
170
+
atomic_t nr_copy_ops; /* Number of copy-to-cache ops in progress */
178
171
size_t submitted; /* Amount submitted for I/O so far */
179
172
size_t len; /* Length of the request */
180
173
short error; /* 0 or error that occurred */
174
+
enum netfs_io_origin origin; /* Origin of the request */
181
175
loff_t i_size; /* Size of the file */
182
176
loff_t start; /* Start position */
183
177
pgoff_t no_unlock_folio; /* Don't unlock this folio after read */
184
-
refcount_t usage;
178
+
refcount_t ref;
185
179
unsigned long flags;
186
180
#define NETFS_RREQ_INCOMPLETE_IO 0 /* Some ioreqs terminated short or with error */
187
-
#define NETFS_RREQ_WRITE_TO_CACHE 1 /* Need to write to the cache */
181
+
#define NETFS_RREQ_COPY_TO_CACHE 1 /* Need to write to the cache */
188
182
#define NETFS_RREQ_NO_UNLOCK_FOLIO 2 /* Don't unlock no_unlock_folio on completion */
189
183
#define NETFS_RREQ_DONT_UNLOCK_FOLIOS 3 /* Don't unlock the folios on completion */
190
184
#define NETFS_RREQ_FAILED 4 /* The request failed */
191
185
#define NETFS_RREQ_IN_PROGRESS 5 /* Unlocked when the request completes */
192
-
const struct netfs_read_request_ops *netfs_ops;
186
+
const struct netfs_request_ops *netfs_ops;
193
187
};
194
188
195
189
/*
196
190
* Operations the network filesystem can/must provide to the helpers.
197
191
*/
198
-
struct netfs_read_request_ops {
199
-
bool (*is_cache_enabled)(struct inode *inode);
200
-
void (*init_rreq)(struct netfs_read_request *rreq, struct file *file);
201
-
int (*begin_cache_operation)(struct netfs_read_request *rreq);
202
-
void (*expand_readahead)(struct netfs_read_request *rreq);
203
-
bool (*clamp_length)(struct netfs_read_subrequest *subreq);
204
-
void (*issue_op)(struct netfs_read_subrequest *subreq);
205
-
bool (*is_still_valid)(struct netfs_read_request *rreq);
192
+
struct netfs_request_ops {
193
+
int (*init_request)(struct netfs_io_request *rreq, struct file *file);
194
+
int (*begin_cache_operation)(struct netfs_io_request *rreq);
195
+
void (*expand_readahead)(struct netfs_io_request *rreq);
196
+
bool (*clamp_length)(struct netfs_io_subrequest *subreq);
197
+
void (*issue_read)(struct netfs_io_subrequest *subreq);
198
+
bool (*is_still_valid)(struct netfs_io_request *rreq);
206
199
int (*check_write_begin)(struct file *file, loff_t pos, unsigned len,
207
200
struct folio *folio, void **_fsdata);
208
-
void (*done)(struct netfs_read_request *rreq);
201
+
void (*done)(struct netfs_io_request *rreq);
209
202
void (*cleanup)(struct address_space *mapping, void *netfs_priv);
210
203
};
211
204
···
254
235
/* Prepare a read operation, shortening it to a cached/uncached
255
236
* boundary as appropriate.
256
237
*/
257
-
enum netfs_read_source (*prepare_read)(struct netfs_read_subrequest *subreq,
238
+
enum netfs_io_source (*prepare_read)(struct netfs_io_subrequest *subreq,
258
239
loff_t i_size);
259
240
260
241
/* Prepare a write operation, working out what part of the write we can
···
273
254
};
274
255
275
256
struct readahead_control;
276
-
extern void netfs_readahead(struct readahead_control *,
277
-
const struct netfs_read_request_ops *,
278
-
void *);
279
-
extern int netfs_readpage(struct file *,
280
-
struct folio *,
281
-
const struct netfs_read_request_ops *,
282
-
void *);
257
+
extern void netfs_readahead(struct readahead_control *);
258
+
extern int netfs_readpage(struct file *, struct page *);
283
259
extern int netfs_write_begin(struct file *, struct address_space *,
284
260
loff_t, unsigned int, unsigned int, struct folio **,
285
-
void **,
286
-
const struct netfs_read_request_ops *,
287
-
void *);
261
+
void **);
288
262
289
-
extern void netfs_subreq_terminated(struct netfs_read_subrequest *, ssize_t, bool);
263
+
extern void netfs_subreq_terminated(struct netfs_io_subrequest *, ssize_t, bool);
264
+
extern void netfs_get_subrequest(struct netfs_io_subrequest *subreq,
265
+
enum netfs_sreq_ref_trace what);
266
+
extern void netfs_put_subrequest(struct netfs_io_subrequest *subreq,
267
+
bool was_async, enum netfs_sreq_ref_trace what);
290
268
extern void netfs_stats_show(struct seq_file *);
269
+
270
+
/**
271
+
* netfs_i_context - Get the netfs inode context from the inode
272
+
* @inode: The inode to query
273
+
*
274
+
* Get the netfs lib inode context from the network filesystem's inode. The
275
+
* context struct is expected to directly follow on from the VFS inode struct.
276
+
*/
277
+
static inline struct netfs_i_context *netfs_i_context(struct inode *inode)
278
+
{
279
+
return (struct netfs_i_context *)(inode + 1);
280
+
}
281
+
282
+
/**
283
+
* netfs_inode - Get the netfs inode from the inode context
284
+
* @ctx: The context to query
285
+
*
286
+
* Get the netfs inode from the netfs library's inode context. The VFS inode
287
+
* is expected to directly precede the context struct.
288
+
*/
289
+
static inline struct inode *netfs_inode(struct netfs_i_context *ctx)
290
+
{
291
+
return ((struct inode *)ctx) - 1;
292
+
}
293
+
294
+
/**
295
+
* netfs_i_context_init - Initialise a netfs lib context
296
+
* @inode: The inode with which the context is associated
297
+
* @ops: The netfs's operations list
298
+
*
299
+
* Initialise the netfs library context struct. This is expected to follow on
300
+
* directly from the VFS inode struct.
301
+
*/
302
+
static inline void netfs_i_context_init(struct inode *inode,
303
+
const struct netfs_request_ops *ops)
304
+
{
305
+
struct netfs_i_context *ctx = netfs_i_context(inode);
306
+
307
+
memset(ctx, 0, sizeof(*ctx));
308
+
ctx->ops = ops;
309
+
ctx->remote_i_size = i_size_read(inode);
310
+
}
311
+
312
+
/**
313
+
* netfs_resize_file - Note that a file got resized
314
+
* @inode: The inode being resized
315
+
* @new_i_size: The new file size
316
+
*
317
+
* Inform the netfs lib that a file got resized so that it can adjust its state.
318
+
*/
319
+
static inline void netfs_resize_file(struct inode *inode, loff_t new_i_size)
320
+
{
321
+
struct netfs_i_context *ctx = netfs_i_context(inode);
322
+
323
+
ctx->remote_i_size = new_i_size;
324
+
}
325
+
326
+
/**
327
+
* netfs_i_cookie - Get the cache cookie from the inode
328
+
* @inode: The inode to query
329
+
*
330
+
* Get the caching cookie (if enabled) from the network filesystem's inode.
331
+
*/
332
+
static inline struct fscache_cookie *netfs_i_cookie(struct inode *inode)
333
+
{
334
+
#if IS_ENABLED(CONFIG_FSCACHE)
335
+
struct netfs_i_context *ctx = netfs_i_context(inode);
336
+
return ctx->cache;
337
+
#else
338
+
return NULL;
339
+
#endif
340
+
}
291
341
292
342
#endif /* _LINUX_NETFS_H */
+3
-3
include/trace/events/cachefiles.h
+3
-3
include/trace/events/cachefiles.h
···
426
426
);
427
427
428
428
TRACE_EVENT(cachefiles_prep_read,
429
-
TP_PROTO(struct netfs_read_subrequest *sreq,
430
-
enum netfs_read_source source,
429
+
TP_PROTO(struct netfs_io_subrequest *sreq,
430
+
enum netfs_io_source source,
431
431
enum cachefiles_prepare_read_trace why,
432
432
ino_t cache_inode),
433
433
···
437
437
__field(unsigned int, rreq )
438
438
__field(unsigned short, index )
439
439
__field(unsigned short, flags )
440
-
__field(enum netfs_read_source, source )
440
+
__field(enum netfs_io_source, source )
441
441
__field(enum cachefiles_prepare_read_trace, why )
442
442
__field(size_t, len )
443
443
__field(loff_t, start )
+124
-66
include/trace/events/netfs.h
+124
-66
include/trace/events/netfs.h
···
15
15
/*
16
16
* Define enums for tracing information.
17
17
*/
18
-
#ifndef __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY
19
-
#define __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY
20
-
21
-
enum netfs_read_trace {
22
-
netfs_read_trace_expanded,
23
-
netfs_read_trace_readahead,
24
-
netfs_read_trace_readpage,
25
-
netfs_read_trace_write_begin,
26
-
};
27
-
28
-
enum netfs_rreq_trace {
29
-
netfs_rreq_trace_assess,
30
-
netfs_rreq_trace_done,
31
-
netfs_rreq_trace_free,
32
-
netfs_rreq_trace_resubmit,
33
-
netfs_rreq_trace_unlock,
34
-
netfs_rreq_trace_unmark,
35
-
netfs_rreq_trace_write,
36
-
};
37
-
38
-
enum netfs_sreq_trace {
39
-
netfs_sreq_trace_download_instead,
40
-
netfs_sreq_trace_free,
41
-
netfs_sreq_trace_prepare,
42
-
netfs_sreq_trace_resubmit_short,
43
-
netfs_sreq_trace_submit,
44
-
netfs_sreq_trace_terminated,
45
-
netfs_sreq_trace_write,
46
-
netfs_sreq_trace_write_skip,
47
-
netfs_sreq_trace_write_term,
48
-
};
49
-
50
-
enum netfs_failure {
51
-
netfs_fail_check_write_begin,
52
-
netfs_fail_copy_to_cache,
53
-
netfs_fail_read,
54
-
netfs_fail_short_readpage,
55
-
netfs_fail_short_write_begin,
56
-
netfs_fail_prepare_write,
57
-
};
58
-
59
-
#endif
60
-
61
18
#define netfs_read_traces \
62
19
EM(netfs_read_trace_expanded, "EXPANDED ") \
63
20
EM(netfs_read_trace_readahead, "READAHEAD") \
64
21
EM(netfs_read_trace_readpage, "READPAGE ") \
65
22
E_(netfs_read_trace_write_begin, "WRITEBEGN")
66
23
24
+
#define netfs_rreq_origins \
25
+
EM(NETFS_READAHEAD, "RA") \
26
+
EM(NETFS_READPAGE, "RP") \
27
+
E_(NETFS_READ_FOR_WRITE, "RW")
28
+
67
29
#define netfs_rreq_traces \
68
-
EM(netfs_rreq_trace_assess, "ASSESS") \
69
-
EM(netfs_rreq_trace_done, "DONE ") \
70
-
EM(netfs_rreq_trace_free, "FREE ") \
71
-
EM(netfs_rreq_trace_resubmit, "RESUBM") \
72
-
EM(netfs_rreq_trace_unlock, "UNLOCK") \
73
-
EM(netfs_rreq_trace_unmark, "UNMARK") \
74
-
E_(netfs_rreq_trace_write, "WRITE ")
30
+
EM(netfs_rreq_trace_assess, "ASSESS ") \
31
+
EM(netfs_rreq_trace_copy, "COPY ") \
32
+
EM(netfs_rreq_trace_done, "DONE ") \
33
+
EM(netfs_rreq_trace_free, "FREE ") \
34
+
EM(netfs_rreq_trace_resubmit, "RESUBMT") \
35
+
EM(netfs_rreq_trace_unlock, "UNLOCK ") \
36
+
E_(netfs_rreq_trace_unmark, "UNMARK ")
75
37
76
38
#define netfs_sreq_sources \
77
39
EM(NETFS_FILL_WITH_ZEROES, "ZERO") \
···
56
94
EM(netfs_fail_check_write_begin, "check-write-begin") \
57
95
EM(netfs_fail_copy_to_cache, "copy-to-cache") \
58
96
EM(netfs_fail_read, "read") \
59
-
EM(netfs_fail_short_readpage, "short-readpage") \
60
-
EM(netfs_fail_short_write_begin, "short-write-begin") \
97
+
EM(netfs_fail_short_read, "short-read") \
61
98
E_(netfs_fail_prepare_write, "prep-write")
62
99
100
+
#define netfs_rreq_ref_traces \
101
+
EM(netfs_rreq_trace_get_hold, "GET HOLD ") \
102
+
EM(netfs_rreq_trace_get_subreq, "GET SUBREQ ") \
103
+
EM(netfs_rreq_trace_put_complete, "PUT COMPLT ") \
104
+
EM(netfs_rreq_trace_put_discard, "PUT DISCARD") \
105
+
EM(netfs_rreq_trace_put_failed, "PUT FAILED ") \
106
+
EM(netfs_rreq_trace_put_hold, "PUT HOLD ") \
107
+
EM(netfs_rreq_trace_put_subreq, "PUT SUBREQ ") \
108
+
EM(netfs_rreq_trace_put_zero_len, "PUT ZEROLEN") \
109
+
E_(netfs_rreq_trace_new, "NEW ")
110
+
111
+
#define netfs_sreq_ref_traces \
112
+
EM(netfs_sreq_trace_get_copy_to_cache, "GET COPY2C ") \
113
+
EM(netfs_sreq_trace_get_resubmit, "GET RESUBMIT") \
114
+
EM(netfs_sreq_trace_get_short_read, "GET SHORTRD") \
115
+
EM(netfs_sreq_trace_new, "NEW ") \
116
+
EM(netfs_sreq_trace_put_clear, "PUT CLEAR ") \
117
+
EM(netfs_sreq_trace_put_failed, "PUT FAILED ") \
118
+
EM(netfs_sreq_trace_put_merged, "PUT MERGED ") \
119
+
EM(netfs_sreq_trace_put_no_copy, "PUT NO COPY") \
120
+
E_(netfs_sreq_trace_put_terminated, "PUT TERM ")
121
+
122
+
#ifndef __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY
123
+
#define __NETFS_DECLARE_TRACE_ENUMS_ONCE_ONLY
124
+
125
+
#undef EM
126
+
#undef E_
127
+
#define EM(a, b) a,
128
+
#define E_(a, b) a
129
+
130
+
enum netfs_read_trace { netfs_read_traces } __mode(byte);
131
+
enum netfs_rreq_trace { netfs_rreq_traces } __mode(byte);
132
+
enum netfs_sreq_trace { netfs_sreq_traces } __mode(byte);
133
+
enum netfs_failure { netfs_failures } __mode(byte);
134
+
enum netfs_rreq_ref_trace { netfs_rreq_ref_traces } __mode(byte);
135
+
enum netfs_sreq_ref_trace { netfs_sreq_ref_traces } __mode(byte);
136
+
137
+
#endif
63
138
64
139
/*
65
140
* Export enum symbols via userspace.
···
107
108
#define E_(a, b) TRACE_DEFINE_ENUM(a);
108
109
109
110
netfs_read_traces;
111
+
netfs_rreq_origins;
110
112
netfs_rreq_traces;
111
113
netfs_sreq_sources;
112
114
netfs_sreq_traces;
113
115
netfs_failures;
116
+
netfs_rreq_ref_traces;
117
+
netfs_sreq_ref_traces;
114
118
115
119
/*
116
120
* Now redefine the EM() and E_() macros to map the enums to the strings that
···
125
123
#define E_(a, b) { a, b }
126
124
127
125
TRACE_EVENT(netfs_read,
128
-
TP_PROTO(struct netfs_read_request *rreq,
126
+
TP_PROTO(struct netfs_io_request *rreq,
129
127
loff_t start, size_t len,
130
128
enum netfs_read_trace what),
131
129
···
158
156
);
159
157
160
158
TRACE_EVENT(netfs_rreq,
161
-
TP_PROTO(struct netfs_read_request *rreq,
159
+
TP_PROTO(struct netfs_io_request *rreq,
162
160
enum netfs_rreq_trace what),
163
161
164
162
TP_ARGS(rreq, what),
165
163
166
164
TP_STRUCT__entry(
167
165
__field(unsigned int, rreq )
168
-
__field(unsigned short, flags )
166
+
__field(unsigned int, flags )
167
+
__field(enum netfs_io_origin, origin )
169
168
__field(enum netfs_rreq_trace, what )
170
169
),
171
170
172
171
TP_fast_assign(
173
172
__entry->rreq = rreq->debug_id;
174
173
__entry->flags = rreq->flags;
174
+
__entry->origin = rreq->origin;
175
175
__entry->what = what;
176
176
),
177
177
178
-
TP_printk("R=%08x %s f=%02x",
178
+
TP_printk("R=%08x %s %s f=%02x",
179
179
__entry->rreq,
180
+
__print_symbolic(__entry->origin, netfs_rreq_origins),
180
181
__print_symbolic(__entry->what, netfs_rreq_traces),
181
182
__entry->flags)
182
183
);
183
184
184
185
TRACE_EVENT(netfs_sreq,
185
-
TP_PROTO(struct netfs_read_subrequest *sreq,
186
+
TP_PROTO(struct netfs_io_subrequest *sreq,
186
187
enum netfs_sreq_trace what),
187
188
188
189
TP_ARGS(sreq, what),
···
195
190
__field(unsigned short, index )
196
191
__field(short, error )
197
192
__field(unsigned short, flags )
198
-
__field(enum netfs_read_source, source )
193
+
__field(enum netfs_io_source, source )
199
194
__field(enum netfs_sreq_trace, what )
200
195
__field(size_t, len )
201
196
__field(size_t, transferred )
···
216
211
217
212
TP_printk("R=%08x[%u] %s %s f=%02x s=%llx %zx/%zx e=%d",
218
213
__entry->rreq, __entry->index,
219
-
__print_symbolic(__entry->what, netfs_sreq_traces),
220
214
__print_symbolic(__entry->source, netfs_sreq_sources),
215
+
__print_symbolic(__entry->what, netfs_sreq_traces),
221
216
__entry->flags,
222
217
__entry->start, __entry->transferred, __entry->len,
223
218
__entry->error)
224
219
);
225
220
226
221
TRACE_EVENT(netfs_failure,
227
-
TP_PROTO(struct netfs_read_request *rreq,
228
-
struct netfs_read_subrequest *sreq,
222
+
TP_PROTO(struct netfs_io_request *rreq,
223
+
struct netfs_io_subrequest *sreq,
229
224
int error, enum netfs_failure what),
230
225
231
226
TP_ARGS(rreq, sreq, error, what),
232
227
233
228
TP_STRUCT__entry(
234
229
__field(unsigned int, rreq )
235
-
__field(unsigned short, index )
230
+
__field(short, index )
236
231
__field(short, error )
237
232
__field(unsigned short, flags )
238
-
__field(enum netfs_read_source, source )
233
+
__field(enum netfs_io_source, source )
239
234
__field(enum netfs_failure, what )
240
235
__field(size_t, len )
241
236
__field(size_t, transferred )
···
244
239
245
240
TP_fast_assign(
246
241
__entry->rreq = rreq->debug_id;
247
-
__entry->index = sreq ? sreq->debug_index : 0;
242
+
__entry->index = sreq ? sreq->debug_index : -1;
248
243
__entry->error = error;
249
244
__entry->flags = sreq ? sreq->flags : 0;
250
245
__entry->source = sreq ? sreq->source : NETFS_INVALID_READ;
251
246
__entry->what = what;
252
-
__entry->len = sreq ? sreq->len : 0;
247
+
__entry->len = sreq ? sreq->len : rreq->len;
253
248
__entry->transferred = sreq ? sreq->transferred : 0;
254
249
__entry->start = sreq ? sreq->start : 0;
255
250
),
256
251
257
-
TP_printk("R=%08x[%u] %s f=%02x s=%llx %zx/%zx %s e=%d",
252
+
TP_printk("R=%08x[%d] %s f=%02x s=%llx %zx/%zx %s e=%d",
258
253
__entry->rreq, __entry->index,
259
254
__print_symbolic(__entry->source, netfs_sreq_sources),
260
255
__entry->flags,
···
263
258
__entry->error)
264
259
);
265
260
261
+
TRACE_EVENT(netfs_rreq_ref,
262
+
TP_PROTO(unsigned int rreq_debug_id, int ref,
263
+
enum netfs_rreq_ref_trace what),
264
+
265
+
TP_ARGS(rreq_debug_id, ref, what),
266
+
267
+
TP_STRUCT__entry(
268
+
__field(unsigned int, rreq )
269
+
__field(int, ref )
270
+
__field(enum netfs_rreq_ref_trace, what )
271
+
),
272
+
273
+
TP_fast_assign(
274
+
__entry->rreq = rreq_debug_id;
275
+
__entry->ref = ref;
276
+
__entry->what = what;
277
+
),
278
+
279
+
TP_printk("R=%08x %s r=%u",
280
+
__entry->rreq,
281
+
__print_symbolic(__entry->what, netfs_rreq_ref_traces),
282
+
__entry->ref)
283
+
);
284
+
285
+
TRACE_EVENT(netfs_sreq_ref,
286
+
TP_PROTO(unsigned int rreq_debug_id, unsigned int subreq_debug_index,
287
+
int ref, enum netfs_sreq_ref_trace what),
288
+
289
+
TP_ARGS(rreq_debug_id, subreq_debug_index, ref, what),
290
+
291
+
TP_STRUCT__entry(
292
+
__field(unsigned int, rreq )
293
+
__field(unsigned int, subreq )
294
+
__field(int, ref )
295
+
__field(enum netfs_sreq_ref_trace, what )
296
+
),
297
+
298
+
TP_fast_assign(
299
+
__entry->rreq = rreq_debug_id;
300
+
__entry->subreq = subreq_debug_index;
301
+
__entry->ref = ref;
302
+
__entry->what = what;
303
+
),
304
+
305
+
TP_printk("R=%08x[%x] %s r=%u",
306
+
__entry->rreq,
307
+
__entry->subreq,
308
+
__print_symbolic(__entry->what, netfs_sreq_ref_traces),
309
+
__entry->ref)
310
+
);
311
+
312
+
#undef EM
313
+
#undef E_
266
314
#endif /* _TRACE_NETFS_H */
267
315
268
316
/* This part must be outside protection */