Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
lib/crypto: sha3: Add SHA-3 support
Add SHA-3 support to lib/crypto/. All six algorithms in the SHA-3
family are supported: four digests (SHA3-224, SHA3-256, SHA3-384, and
SHA3-512) and two extendable-output functions (SHAKE128 and SHAKE256).
The SHAKE algorithms will be required for ML-DSA.
[EB: simplified the API to use fewer types and functions, fixed bug that
sometimes caused incorrect SHAKE output, cleaned up the
documentation, dropped an ad-hoc test that was inconsistent with
the rest of lib/crypto/, and many other cleanups]
Signed-off-by: David Howells <dhowells@redhat.com>
Co-developed-by: Eric Biggers <ebiggers@kernel.org>
Tested-by: Harald Freudenberger <freude@linux.ibm.com>
Reviewed-by: Ard Biesheuvel <ardb@kernel.org>
Link: https://lore.kernel.org/r/20251026055032.1413733-4-ebiggers@kernel.org
Signed-off-by: Eric Biggers <ebiggers@kernel.org>
authored by
David Howells
and committed by
Eric Biggers
05934472
41412119
+812
-5
+1
Documentation/crypto/index.rst
+1
Documentation/crypto/index.rst
+119
Documentation/crypto/sha3.rst
+119
Documentation/crypto/sha3.rst
···
1
+
.. SPDX-License-Identifier: GPL-2.0-or-later
2
+
3
+
==========================
4
+
SHA-3 Algorithm Collection
5
+
==========================
6
+
7
+
.. contents::
8
+
9
+
Overview
10
+
========
11
+
12
+
The SHA-3 family of algorithms, as specified in NIST FIPS-202 [1]_, contains six
13
+
algorithms based on the Keccak sponge function. The differences between them
14
+
are: the "rate" (how much of the state buffer gets updated with new data between
15
+
invocations of the Keccak function and analogous to the "block size"), what
16
+
domain separation suffix gets appended to the input data, and how much output
17
+
data is extracted at the end. The Keccak sponge function is designed such that
18
+
arbitrary amounts of output can be obtained for certain algorithms.
19
+
20
+
Four digest algorithms are provided:
21
+
22
+
- SHA3-224
23
+
- SHA3-256
24
+
- SHA3-384
25
+
- SHA3-512
26
+
27
+
Additionally, two Extendable-Output Functions (XOFs) are provided:
28
+
29
+
- SHAKE128
30
+
- SHAKE256
31
+
32
+
The SHA-3 library API supports all six of these algorithms. The four digest
33
+
algorithms are also supported by the crypto_shash and crypto_ahash APIs.
34
+
35
+
This document describes the SHA-3 library API.
36
+
37
+
38
+
Digests
39
+
=======
40
+
41
+
The following functions compute SHA-3 digests::
42
+
43
+
void sha3_224(const u8 *in, size_t in_len, u8 out[SHA3_224_DIGEST_SIZE]);
44
+
void sha3_256(const u8 *in, size_t in_len, u8 out[SHA3_256_DIGEST_SIZE]);
45
+
void sha3_384(const u8 *in, size_t in_len, u8 out[SHA3_384_DIGEST_SIZE]);
46
+
void sha3_512(const u8 *in, size_t in_len, u8 out[SHA3_512_DIGEST_SIZE]);
47
+
48
+
For users that need to pass in data incrementally, an incremental API is also
49
+
provided. The incremental API uses the following struct::
50
+
51
+
struct sha3_ctx { ... };
52
+
53
+
Initialization is done with one of::
54
+
55
+
void sha3_224_init(struct sha3_ctx *ctx);
56
+
void sha3_256_init(struct sha3_ctx *ctx);
57
+
void sha3_384_init(struct sha3_ctx *ctx);
58
+
void sha3_512_init(struct sha3_ctx *ctx);
59
+
60
+
Input data is then added with any number of calls to::
61
+
62
+
void sha3_update(struct sha3_ctx *ctx, const u8 *in, size_t in_len);
63
+
64
+
Finally, the digest is generated using::
65
+
66
+
void sha3_final(struct sha3_ctx *ctx, u8 *out);
67
+
68
+
which also zeroizes the context. The length of the digest is determined by the
69
+
initialization function that was called.
70
+
71
+
72
+
Extendable-Output Functions
73
+
===========================
74
+
75
+
The following functions compute the SHA-3 extendable-output functions (XOFs)::
76
+
77
+
void shake128(const u8 *in, size_t in_len, u8 *out, size_t out_len);
78
+
void shake256(const u8 *in, size_t in_len, u8 *out, size_t out_len);
79
+
80
+
For users that need to provide the input data incrementally and/or receive the
81
+
output data incrementally, an incremental API is also provided. The incremental
82
+
API uses the following struct::
83
+
84
+
struct shake_ctx { ... };
85
+
86
+
Initialization is done with one of::
87
+
88
+
void shake128_init(struct shake_ctx *ctx);
89
+
void shake256_init(struct shake_ctx *ctx);
90
+
91
+
Input data is then added with any number of calls to::
92
+
93
+
void shake_update(struct shake_ctx *ctx, const u8 *in, size_t in_len);
94
+
95
+
Finally, the output data is extracted with any number of calls to::
96
+
97
+
void shake_squeeze(struct shake_ctx *ctx, u8 *out, size_t out_len);
98
+
99
+
and telling it how much data should be extracted. Note that performing multiple
100
+
squeezes, with the output laid consecutively in a buffer, gets exactly the same
101
+
output as doing a single squeeze for the combined amount over the same buffer.
102
+
103
+
More input data cannot be added after squeezing has started.
104
+
105
+
Once all the desired output has been extracted, zeroize the context::
106
+
107
+
void shake_zeroize_ctx(struct shake_ctx *ctx);
108
+
109
+
110
+
References
111
+
==========
112
+
113
+
.. [1] https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.202.pdf
114
+
115
+
116
+
API Function Reference
117
+
======================
118
+
119
+
.. kernel-doc:: include/crypto/sha3.h
+321
-5
include/crypto/sha3.h
+321
-5
include/crypto/sha3.h
···
1
1
/* SPDX-License-Identifier: GPL-2.0 */
2
2
/*
3
3
* Common values for SHA-3 algorithms
4
+
*
5
+
* See also Documentation/crypto/sha3.rst
4
6
*/
5
7
#ifndef __CRYPTO_SHA3_H__
6
8
#define __CRYPTO_SHA3_H__
7
9
8
10
#include <linux/types.h>
11
+
#include <linux/string.h>
9
12
10
13
#define SHA3_224_DIGEST_SIZE (224 / 8)
11
14
#define SHA3_224_BLOCK_SIZE (200 - 2 * SHA3_224_DIGEST_SIZE)
···
26
23
#define SHA3_512_BLOCK_SIZE (200 - 2 * SHA3_512_DIGEST_SIZE)
27
24
#define SHA3_512_EXPORT_SIZE SHA3_STATE_SIZE + SHA3_512_BLOCK_SIZE + 1
28
25
26
+
/*
27
+
* SHAKE128 and SHAKE256 actually have variable output size, but this is used to
28
+
* calculate the block size (rate) analogously to the above.
29
+
*/
30
+
#define SHAKE128_DEFAULT_SIZE (128 / 8)
31
+
#define SHAKE128_BLOCK_SIZE (200 - 2 * SHAKE128_DEFAULT_SIZE)
32
+
#define SHAKE256_DEFAULT_SIZE (256 / 8)
33
+
#define SHAKE256_BLOCK_SIZE (200 - 2 * SHAKE256_DEFAULT_SIZE)
34
+
29
35
#define SHA3_STATE_SIZE 200
30
36
31
37
struct shash_desc;
32
38
33
-
struct sha3_state {
34
-
u64 st[SHA3_STATE_SIZE / 8];
35
-
};
36
-
37
39
int crypto_sha3_init(struct shash_desc *desc);
38
40
39
-
#endif
41
+
/*
42
+
* State for the Keccak-f[1600] permutation: 25 64-bit words.
43
+
*
44
+
* We usually keep the state words as little-endian, to make absorbing and
45
+
* squeezing easier. (It means that absorbing and squeezing can just treat the
46
+
* state as a byte array.) The state words are converted to native-endian only
47
+
* temporarily by implementations of the permutation that need native-endian
48
+
* words. Of course, that conversion is a no-op on little-endian machines.
49
+
*/
50
+
struct sha3_state {
51
+
union {
52
+
u64 st[SHA3_STATE_SIZE / 8]; /* temporarily retained for compatibility purposes */
53
+
54
+
__le64 words[SHA3_STATE_SIZE / 8];
55
+
u8 bytes[SHA3_STATE_SIZE];
56
+
57
+
u64 native_words[SHA3_STATE_SIZE / 8]; /* see comment above */
58
+
};
59
+
};
60
+
61
+
/* Internal context, shared by the digests (SHA3-*) and the XOFs (SHAKE*) */
62
+
struct __sha3_ctx {
63
+
struct sha3_state state;
64
+
u8 digest_size; /* Digests only: the digest size in bytes */
65
+
u8 block_size; /* Block size in bytes */
66
+
u8 absorb_offset; /* Index of next state byte to absorb into */
67
+
u8 squeeze_offset; /* XOFs only: index of next state byte to extract */
68
+
};
69
+
70
+
void __sha3_update(struct __sha3_ctx *ctx, const u8 *in, size_t in_len);
71
+
72
+
/**
73
+
* struct sha3_ctx - Context for SHA3-224, SHA3-256, SHA3-384, or SHA3-512
74
+
* @ctx: private
75
+
*/
76
+
struct sha3_ctx {
77
+
struct __sha3_ctx ctx;
78
+
};
79
+
80
+
/**
81
+
* sha3_zeroize_ctx() - Zeroize a SHA-3 context
82
+
* @ctx: The context to zeroize
83
+
*
84
+
* This is already called by sha3_final(). Call this explicitly when abandoning
85
+
* a context without calling sha3_final().
86
+
*/
87
+
static inline void sha3_zeroize_ctx(struct sha3_ctx *ctx)
88
+
{
89
+
memzero_explicit(ctx, sizeof(*ctx));
90
+
}
91
+
92
+
/**
93
+
* struct shake_ctx - Context for SHAKE128 or SHAKE256
94
+
* @ctx: private
95
+
*/
96
+
struct shake_ctx {
97
+
struct __sha3_ctx ctx;
98
+
};
99
+
100
+
/**
101
+
* shake_zeroize_ctx() - Zeroize a SHAKE context
102
+
* @ctx: The context to zeroize
103
+
*
104
+
* Call this after the last squeeze.
105
+
*/
106
+
static inline void shake_zeroize_ctx(struct shake_ctx *ctx)
107
+
{
108
+
memzero_explicit(ctx, sizeof(*ctx));
109
+
}
110
+
111
+
/**
112
+
* sha3_224_init() - Initialize a context for SHA3-224
113
+
* @ctx: The context to initialize
114
+
*
115
+
* This begins a new SHA3-224 message digest computation.
116
+
*
117
+
* Context: Any context.
118
+
*/
119
+
static inline void sha3_224_init(struct sha3_ctx *ctx)
120
+
{
121
+
*ctx = (struct sha3_ctx){
122
+
.ctx.digest_size = SHA3_224_DIGEST_SIZE,
123
+
.ctx.block_size = SHA3_224_BLOCK_SIZE,
124
+
};
125
+
}
126
+
127
+
/**
128
+
* sha3_256_init() - Initialize a context for SHA3-256
129
+
* @ctx: The context to initialize
130
+
*
131
+
* This begins a new SHA3-256 message digest computation.
132
+
*
133
+
* Context: Any context.
134
+
*/
135
+
static inline void sha3_256_init(struct sha3_ctx *ctx)
136
+
{
137
+
*ctx = (struct sha3_ctx){
138
+
.ctx.digest_size = SHA3_256_DIGEST_SIZE,
139
+
.ctx.block_size = SHA3_256_BLOCK_SIZE,
140
+
};
141
+
}
142
+
143
+
/**
144
+
* sha3_384_init() - Initialize a context for SHA3-384
145
+
* @ctx: The context to initialize
146
+
*
147
+
* This begins a new SHA3-384 message digest computation.
148
+
*
149
+
* Context: Any context.
150
+
*/
151
+
static inline void sha3_384_init(struct sha3_ctx *ctx)
152
+
{
153
+
*ctx = (struct sha3_ctx){
154
+
.ctx.digest_size = SHA3_384_DIGEST_SIZE,
155
+
.ctx.block_size = SHA3_384_BLOCK_SIZE,
156
+
};
157
+
}
158
+
159
+
/**
160
+
* sha3_512_init() - Initialize a context for SHA3-512
161
+
* @ctx: The context to initialize
162
+
*
163
+
* This begins a new SHA3-512 message digest computation.
164
+
*
165
+
* Context: Any context.
166
+
*/
167
+
static inline void sha3_512_init(struct sha3_ctx *ctx)
168
+
{
169
+
*ctx = (struct sha3_ctx){
170
+
.ctx.digest_size = SHA3_512_DIGEST_SIZE,
171
+
.ctx.block_size = SHA3_512_BLOCK_SIZE,
172
+
};
173
+
}
174
+
175
+
/**
176
+
* sha3_update() - Update a SHA-3 digest context with input data
177
+
* @ctx: The context to update; must have been initialized
178
+
* @in: The input data
179
+
* @in_len: Length of the input data in bytes
180
+
*
181
+
* This can be called any number of times to add data to a SHA3-224, SHA3-256,
182
+
* SHA3-384, or SHA3-512 digest (depending on which init function was called).
183
+
*
184
+
* Context: Any context.
185
+
*/
186
+
static inline void sha3_update(struct sha3_ctx *ctx,
187
+
const u8 *in, size_t in_len)
188
+
{
189
+
__sha3_update(&ctx->ctx, in, in_len);
190
+
}
191
+
192
+
/**
193
+
* sha3_final() - Finish computing a SHA-3 message digest
194
+
* @ctx: The context to finalize; must have been initialized
195
+
* @out: (output) The resulting SHA3-224, SHA3-256, SHA3-384, or SHA3-512
196
+
* message digest, matching the init function that was called. Note that
197
+
* the size differs for each one; see SHA3_*_DIGEST_SIZE.
198
+
*
199
+
* After finishing, this zeroizes @ctx. So the caller does not need to do it.
200
+
*
201
+
* Context: Any context.
202
+
*/
203
+
void sha3_final(struct sha3_ctx *ctx, u8 *out);
204
+
205
+
/**
206
+
* shake128_init() - Initialize a context for SHAKE128
207
+
* @ctx: The context to initialize
208
+
*
209
+
* This begins a new SHAKE128 extendable-output function (XOF) computation.
210
+
*
211
+
* Context: Any context.
212
+
*/
213
+
static inline void shake128_init(struct shake_ctx *ctx)
214
+
{
215
+
*ctx = (struct shake_ctx){
216
+
.ctx.block_size = SHAKE128_BLOCK_SIZE,
217
+
};
218
+
}
219
+
220
+
/**
221
+
* shake256_init() - Initialize a context for SHAKE256
222
+
* @ctx: The context to initialize
223
+
*
224
+
* This begins a new SHAKE256 extendable-output function (XOF) computation.
225
+
*
226
+
* Context: Any context.
227
+
*/
228
+
static inline void shake256_init(struct shake_ctx *ctx)
229
+
{
230
+
*ctx = (struct shake_ctx){
231
+
.ctx.block_size = SHAKE256_BLOCK_SIZE,
232
+
};
233
+
}
234
+
235
+
/**
236
+
* shake_update() - Update a SHAKE context with input data
237
+
* @ctx: The context to update; must have been initialized
238
+
* @in: The input data
239
+
* @in_len: Length of the input data in bytes
240
+
*
241
+
* This can be called any number of times to add more input data to SHAKE128 or
242
+
* SHAKE256. This cannot be called after squeezing has begun.
243
+
*
244
+
* Context: Any context.
245
+
*/
246
+
static inline void shake_update(struct shake_ctx *ctx,
247
+
const u8 *in, size_t in_len)
248
+
{
249
+
__sha3_update(&ctx->ctx, in, in_len);
250
+
}
251
+
252
+
/**
253
+
* shake_squeeze() - Generate output from SHAKE128 or SHAKE256
254
+
* @ctx: The context to squeeze; must have been initialized
255
+
* @out: Where to write the resulting output data
256
+
* @out_len: The amount of data to extract to @out in bytes
257
+
*
258
+
* This may be called multiple times. A number of consecutive squeezes laid
259
+
* end-to-end will yield the same output as one big squeeze generating the same
260
+
* total amount of output. More input cannot be provided after squeezing has
261
+
* begun. After the last squeeze, call shake_zeroize_ctx().
262
+
*
263
+
* Context: Any context.
264
+
*/
265
+
void shake_squeeze(struct shake_ctx *ctx, u8 *out, size_t out_len);
266
+
267
+
/**
268
+
* sha3_224() - Compute SHA3-224 digest in one shot
269
+
* @in: The input data to be digested
270
+
* @in_len: Length of the input data in bytes
271
+
* @out: The buffer into which the digest will be stored
272
+
*
273
+
* Convenience function that computes a SHA3-224 digest. Use this instead of
274
+
* the incremental API if you're able to provide all the input at once.
275
+
*
276
+
* Context: Any context.
277
+
*/
278
+
void sha3_224(const u8 *in, size_t in_len, u8 out[SHA3_224_DIGEST_SIZE]);
279
+
280
+
/**
281
+
* sha3_256() - Compute SHA3-256 digest in one shot
282
+
* @in: The input data to be digested
283
+
* @in_len: Length of the input data in bytes
284
+
* @out: The buffer into which the digest will be stored
285
+
*
286
+
* Convenience function that computes a SHA3-256 digest. Use this instead of
287
+
* the incremental API if you're able to provide all the input at once.
288
+
*
289
+
* Context: Any context.
290
+
*/
291
+
void sha3_256(const u8 *in, size_t in_len, u8 out[SHA3_256_DIGEST_SIZE]);
292
+
293
+
/**
294
+
* sha3_384() - Compute SHA3-384 digest in one shot
295
+
* @in: The input data to be digested
296
+
* @in_len: Length of the input data in bytes
297
+
* @out: The buffer into which the digest will be stored
298
+
*
299
+
* Convenience function that computes a SHA3-384 digest. Use this instead of
300
+
* the incremental API if you're able to provide all the input at once.
301
+
*
302
+
* Context: Any context.
303
+
*/
304
+
void sha3_384(const u8 *in, size_t in_len, u8 out[SHA3_384_DIGEST_SIZE]);
305
+
306
+
/**
307
+
* sha3_512() - Compute SHA3-512 digest in one shot
308
+
* @in: The input data to be digested
309
+
* @in_len: Length of the input data in bytes
310
+
* @out: The buffer into which the digest will be stored
311
+
*
312
+
* Convenience function that computes a SHA3-512 digest. Use this instead of
313
+
* the incremental API if you're able to provide all the input at once.
314
+
*
315
+
* Context: Any context.
316
+
*/
317
+
void sha3_512(const u8 *in, size_t in_len, u8 out[SHA3_512_DIGEST_SIZE]);
318
+
319
+
/**
320
+
* shake128() - Compute SHAKE128 in one shot
321
+
* @in: The input data to be used
322
+
* @in_len: Length of the input data in bytes
323
+
* @out: The buffer into which the output will be stored
324
+
* @out_len: Length of the output to produce in bytes
325
+
*
326
+
* Convenience function that computes SHAKE128 in one shot. Use this instead of
327
+
* the incremental API if you're able to provide all the input at once as well
328
+
* as receive all the output at once. All output lengths are supported.
329
+
*
330
+
* Context: Any context.
331
+
*/
332
+
void shake128(const u8 *in, size_t in_len, u8 *out, size_t out_len);
333
+
334
+
/**
335
+
* shake256() - Compute SHAKE256 in one shot
336
+
* @in: The input data to be used
337
+
* @in_len: Length of the input data in bytes
338
+
* @out: The buffer into which the output will be stored
339
+
* @out_len: Length of the output to produce in bytes
340
+
*
341
+
* Convenience function that computes SHAKE256 in one shot. Use this instead of
342
+
* the incremental API if you're able to provide all the input at once as well
343
+
* as receive all the output at once. All output lengths are supported.
344
+
*
345
+
* Context: Any context.
346
+
*/
347
+
void shake256(const u8 *in, size_t in_len, u8 *out, size_t out_len);
348
+
349
+
#endif /* __CRYPTO_SHA3_H__ */
+7
lib/crypto/Kconfig
+7
lib/crypto/Kconfig
···
195
195
default y if SPARC64
196
196
default y if X86_64
197
197
198
+
config CRYPTO_LIB_SHA3
199
+
tristate
200
+
select CRYPTO_LIB_UTILS
201
+
help
202
+
The SHA3 library functions. Select this if your module uses any of
203
+
the functions from <crypto/sha3.h>.
204
+
198
205
config CRYPTO_LIB_SM3
199
206
tristate
200
207
+5
lib/crypto/Makefile
+5
lib/crypto/Makefile
···
278
278
279
279
################################################################################
280
280
281
+
obj-$(CONFIG_CRYPTO_LIB_SHA3) += libsha3.o
282
+
libsha3-y := sha3.o
283
+
284
+
################################################################################
285
+
281
286
obj-$(CONFIG_MPILIB) += mpi/
282
287
283
288
obj-$(CONFIG_CRYPTO_SELFTESTS_FULL) += simd.o
+359
lib/crypto/sha3.c
+359
lib/crypto/sha3.c
···
1
+
// SPDX-License-Identifier: GPL-2.0-or-later
2
+
/*
3
+
* SHA-3, as specified in
4
+
* https://nvlpubs.nist.gov/nistpubs/FIPS/NIST.FIPS.202.pdf
5
+
*
6
+
* SHA-3 code by Jeff Garzik <jeff@garzik.org>
7
+
* Ard Biesheuvel <ard.biesheuvel@linaro.org>
8
+
* David Howells <dhowells@redhat.com>
9
+
*
10
+
* See also Documentation/crypto/sha3.rst
11
+
*/
12
+
13
+
#define pr_fmt(fmt) KBUILD_MODNAME ": " fmt
14
+
#include <crypto/sha3.h>
15
+
#include <crypto/utils.h>
16
+
#include <linux/export.h>
17
+
#include <linux/kernel.h>
18
+
#include <linux/module.h>
19
+
#include <linux/unaligned.h>
20
+
21
+
/*
22
+
* On some 32-bit architectures, such as h8300, GCC ends up using over 1 KB of
23
+
* stack if the round calculation gets inlined into the loop in
24
+
* sha3_keccakf_generic(). On the other hand, on 64-bit architectures with
25
+
* plenty of [64-bit wide] general purpose registers, not inlining it severely
26
+
* hurts performance. So let's use 64-bitness as a heuristic to decide whether
27
+
* to inline or not.
28
+
*/
29
+
#ifdef CONFIG_64BIT
30
+
#define SHA3_INLINE inline
31
+
#else
32
+
#define SHA3_INLINE noinline
33
+
#endif
34
+
35
+
#define SHA3_KECCAK_ROUNDS 24
36
+
37
+
static const u64 sha3_keccakf_rndc[SHA3_KECCAK_ROUNDS] = {
38
+
0x0000000000000001ULL, 0x0000000000008082ULL, 0x800000000000808aULL,
39
+
0x8000000080008000ULL, 0x000000000000808bULL, 0x0000000080000001ULL,
40
+
0x8000000080008081ULL, 0x8000000000008009ULL, 0x000000000000008aULL,
41
+
0x0000000000000088ULL, 0x0000000080008009ULL, 0x000000008000000aULL,
42
+
0x000000008000808bULL, 0x800000000000008bULL, 0x8000000000008089ULL,
43
+
0x8000000000008003ULL, 0x8000000000008002ULL, 0x8000000000000080ULL,
44
+
0x000000000000800aULL, 0x800000008000000aULL, 0x8000000080008081ULL,
45
+
0x8000000000008080ULL, 0x0000000080000001ULL, 0x8000000080008008ULL
46
+
};
47
+
48
+
/*
49
+
* Perform a single round of Keccak mixing.
50
+
*/
51
+
static SHA3_INLINE void sha3_keccakf_one_round_generic(u64 st[25])
52
+
{
53
+
u64 t[5], tt, bc[5];
54
+
55
+
/* Theta */
56
+
bc[0] = st[0] ^ st[5] ^ st[10] ^ st[15] ^ st[20];
57
+
bc[1] = st[1] ^ st[6] ^ st[11] ^ st[16] ^ st[21];
58
+
bc[2] = st[2] ^ st[7] ^ st[12] ^ st[17] ^ st[22];
59
+
bc[3] = st[3] ^ st[8] ^ st[13] ^ st[18] ^ st[23];
60
+
bc[4] = st[4] ^ st[9] ^ st[14] ^ st[19] ^ st[24];
61
+
62
+
t[0] = bc[4] ^ rol64(bc[1], 1);
63
+
t[1] = bc[0] ^ rol64(bc[2], 1);
64
+
t[2] = bc[1] ^ rol64(bc[3], 1);
65
+
t[3] = bc[2] ^ rol64(bc[4], 1);
66
+
t[4] = bc[3] ^ rol64(bc[0], 1);
67
+
68
+
st[0] ^= t[0];
69
+
70
+
/* Rho Pi */
71
+
tt = st[1];
72
+
st[ 1] = rol64(st[ 6] ^ t[1], 44);
73
+
st[ 6] = rol64(st[ 9] ^ t[4], 20);
74
+
st[ 9] = rol64(st[22] ^ t[2], 61);
75
+
st[22] = rol64(st[14] ^ t[4], 39);
76
+
st[14] = rol64(st[20] ^ t[0], 18);
77
+
st[20] = rol64(st[ 2] ^ t[2], 62);
78
+
st[ 2] = rol64(st[12] ^ t[2], 43);
79
+
st[12] = rol64(st[13] ^ t[3], 25);
80
+
st[13] = rol64(st[19] ^ t[4], 8);
81
+
st[19] = rol64(st[23] ^ t[3], 56);
82
+
st[23] = rol64(st[15] ^ t[0], 41);
83
+
st[15] = rol64(st[ 4] ^ t[4], 27);
84
+
st[ 4] = rol64(st[24] ^ t[4], 14);
85
+
st[24] = rol64(st[21] ^ t[1], 2);
86
+
st[21] = rol64(st[ 8] ^ t[3], 55);
87
+
st[ 8] = rol64(st[16] ^ t[1], 45);
88
+
st[16] = rol64(st[ 5] ^ t[0], 36);
89
+
st[ 5] = rol64(st[ 3] ^ t[3], 28);
90
+
st[ 3] = rol64(st[18] ^ t[3], 21);
91
+
st[18] = rol64(st[17] ^ t[2], 15);
92
+
st[17] = rol64(st[11] ^ t[1], 10);
93
+
st[11] = rol64(st[ 7] ^ t[2], 6);
94
+
st[ 7] = rol64(st[10] ^ t[0], 3);
95
+
st[10] = rol64( tt ^ t[1], 1);
96
+
97
+
/* Chi */
98
+
bc[ 0] = ~st[ 1] & st[ 2];
99
+
bc[ 1] = ~st[ 2] & st[ 3];
100
+
bc[ 2] = ~st[ 3] & st[ 4];
101
+
bc[ 3] = ~st[ 4] & st[ 0];
102
+
bc[ 4] = ~st[ 0] & st[ 1];
103
+
st[ 0] ^= bc[ 0];
104
+
st[ 1] ^= bc[ 1];
105
+
st[ 2] ^= bc[ 2];
106
+
st[ 3] ^= bc[ 3];
107
+
st[ 4] ^= bc[ 4];
108
+
109
+
bc[ 0] = ~st[ 6] & st[ 7];
110
+
bc[ 1] = ~st[ 7] & st[ 8];
111
+
bc[ 2] = ~st[ 8] & st[ 9];
112
+
bc[ 3] = ~st[ 9] & st[ 5];
113
+
bc[ 4] = ~st[ 5] & st[ 6];
114
+
st[ 5] ^= bc[ 0];
115
+
st[ 6] ^= bc[ 1];
116
+
st[ 7] ^= bc[ 2];
117
+
st[ 8] ^= bc[ 3];
118
+
st[ 9] ^= bc[ 4];
119
+
120
+
bc[ 0] = ~st[11] & st[12];
121
+
bc[ 1] = ~st[12] & st[13];
122
+
bc[ 2] = ~st[13] & st[14];
123
+
bc[ 3] = ~st[14] & st[10];
124
+
bc[ 4] = ~st[10] & st[11];
125
+
st[10] ^= bc[ 0];
126
+
st[11] ^= bc[ 1];
127
+
st[12] ^= bc[ 2];
128
+
st[13] ^= bc[ 3];
129
+
st[14] ^= bc[ 4];
130
+
131
+
bc[ 0] = ~st[16] & st[17];
132
+
bc[ 1] = ~st[17] & st[18];
133
+
bc[ 2] = ~st[18] & st[19];
134
+
bc[ 3] = ~st[19] & st[15];
135
+
bc[ 4] = ~st[15] & st[16];
136
+
st[15] ^= bc[ 0];
137
+
st[16] ^= bc[ 1];
138
+
st[17] ^= bc[ 2];
139
+
st[18] ^= bc[ 3];
140
+
st[19] ^= bc[ 4];
141
+
142
+
bc[ 0] = ~st[21] & st[22];
143
+
bc[ 1] = ~st[22] & st[23];
144
+
bc[ 2] = ~st[23] & st[24];
145
+
bc[ 3] = ~st[24] & st[20];
146
+
bc[ 4] = ~st[20] & st[21];
147
+
st[20] ^= bc[ 0];
148
+
st[21] ^= bc[ 1];
149
+
st[22] ^= bc[ 2];
150
+
st[23] ^= bc[ 3];
151
+
st[24] ^= bc[ 4];
152
+
}
153
+
154
+
/* Generic implementation of the Keccak-f[1600] permutation */
155
+
static void sha3_keccakf_generic(struct sha3_state *state)
156
+
{
157
+
/*
158
+
* Temporarily convert the state words from little-endian to native-
159
+
* endian so that they can be operated on. Note that on little-endian
160
+
* machines this conversion is a no-op and is optimized out.
161
+
*/
162
+
163
+
for (int i = 0; i < ARRAY_SIZE(state->words); i++)
164
+
state->native_words[i] = le64_to_cpu(state->words[i]);
165
+
166
+
for (int round = 0; round < SHA3_KECCAK_ROUNDS; round++) {
167
+
sha3_keccakf_one_round_generic(state->native_words);
168
+
/* Iota */
169
+
state->native_words[0] ^= sha3_keccakf_rndc[round];
170
+
}
171
+
172
+
for (int i = 0; i < ARRAY_SIZE(state->words); i++)
173
+
state->words[i] = cpu_to_le64(state->native_words[i]);
174
+
}
175
+
176
+
/*
177
+
* Generic implementation of absorbing the given nonzero number of full blocks
178
+
* into the sponge function Keccak[r=8*block_size, c=1600-8*block_size].
179
+
*/
180
+
static void __maybe_unused
181
+
sha3_absorb_blocks_generic(struct sha3_state *state, const u8 *data,
182
+
size_t nblocks, size_t block_size)
183
+
{
184
+
do {
185
+
for (size_t i = 0; i < block_size; i += 8)
186
+
state->words[i / 8] ^= get_unaligned((__le64 *)&data[i]);
187
+
sha3_keccakf_generic(state);
188
+
data += block_size;
189
+
} while (--nblocks);
190
+
}
191
+
192
+
#ifdef CONFIG_CRYPTO_LIB_SHA3_ARCH
193
+
#include "sha3.h" /* $(SRCARCH)/sha3.h */
194
+
#else
195
+
#define sha3_keccakf sha3_keccakf_generic
196
+
#define sha3_absorb_blocks sha3_absorb_blocks_generic
197
+
#endif
198
+
199
+
void __sha3_update(struct __sha3_ctx *ctx, const u8 *in, size_t in_len)
200
+
{
201
+
const size_t block_size = ctx->block_size;
202
+
size_t absorb_offset = ctx->absorb_offset;
203
+
204
+
/* Warn if squeezing has already begun. */
205
+
WARN_ON_ONCE(absorb_offset >= block_size);
206
+
207
+
if (absorb_offset && absorb_offset + in_len >= block_size) {
208
+
crypto_xor(&ctx->state.bytes[absorb_offset], in,
209
+
block_size - absorb_offset);
210
+
in += block_size - absorb_offset;
211
+
in_len -= block_size - absorb_offset;
212
+
sha3_keccakf(&ctx->state);
213
+
absorb_offset = 0;
214
+
}
215
+
216
+
if (in_len >= block_size) {
217
+
size_t nblocks = in_len / block_size;
218
+
219
+
sha3_absorb_blocks(&ctx->state, in, nblocks, block_size);
220
+
in += nblocks * block_size;
221
+
in_len -= nblocks * block_size;
222
+
}
223
+
224
+
if (in_len) {
225
+
crypto_xor(&ctx->state.bytes[absorb_offset], in, in_len);
226
+
absorb_offset += in_len;
227
+
}
228
+
ctx->absorb_offset = absorb_offset;
229
+
}
230
+
EXPORT_SYMBOL_GPL(__sha3_update);
231
+
232
+
void sha3_final(struct sha3_ctx *sha3_ctx, u8 *out)
233
+
{
234
+
struct __sha3_ctx *ctx = &sha3_ctx->ctx;
235
+
236
+
ctx->state.bytes[ctx->absorb_offset] ^= 0x06;
237
+
ctx->state.bytes[ctx->block_size - 1] ^= 0x80;
238
+
sha3_keccakf(&ctx->state);
239
+
memcpy(out, ctx->state.bytes, ctx->digest_size);
240
+
sha3_zeroize_ctx(sha3_ctx);
241
+
}
242
+
EXPORT_SYMBOL_GPL(sha3_final);
243
+
244
+
void shake_squeeze(struct shake_ctx *shake_ctx, u8 *out, size_t out_len)
245
+
{
246
+
struct __sha3_ctx *ctx = &shake_ctx->ctx;
247
+
const size_t block_size = ctx->block_size;
248
+
size_t squeeze_offset = ctx->squeeze_offset;
249
+
250
+
if (ctx->absorb_offset < block_size) {
251
+
/* First squeeze: */
252
+
253
+
/* Add the domain separation suffix and padding. */
254
+
ctx->state.bytes[ctx->absorb_offset] ^= 0x1f;
255
+
ctx->state.bytes[block_size - 1] ^= 0x80;
256
+
257
+
/* Indicate that squeezing has begun. */
258
+
ctx->absorb_offset = block_size;
259
+
260
+
/*
261
+
* Indicate that no output is pending yet, i.e. sha3_keccakf()
262
+
* will need to be called before the first copy.
263
+
*/
264
+
squeeze_offset = block_size;
265
+
}
266
+
while (out_len) {
267
+
if (squeeze_offset == block_size) {
268
+
sha3_keccakf(&ctx->state);
269
+
squeeze_offset = 0;
270
+
}
271
+
size_t copy = min(out_len, block_size - squeeze_offset);
272
+
273
+
memcpy(out, &ctx->state.bytes[squeeze_offset], copy);
274
+
out += copy;
275
+
out_len -= copy;
276
+
squeeze_offset += copy;
277
+
}
278
+
ctx->squeeze_offset = squeeze_offset;
279
+
}
280
+
EXPORT_SYMBOL_GPL(shake_squeeze);
281
+
282
+
void sha3_224(const u8 *in, size_t in_len, u8 out[SHA3_224_DIGEST_SIZE])
283
+
{
284
+
struct sha3_ctx ctx;
285
+
286
+
sha3_224_init(&ctx);
287
+
sha3_update(&ctx, in, in_len);
288
+
sha3_final(&ctx, out);
289
+
}
290
+
EXPORT_SYMBOL_GPL(sha3_224);
291
+
292
+
void sha3_256(const u8 *in, size_t in_len, u8 out[SHA3_256_DIGEST_SIZE])
293
+
{
294
+
struct sha3_ctx ctx;
295
+
296
+
sha3_256_init(&ctx);
297
+
sha3_update(&ctx, in, in_len);
298
+
sha3_final(&ctx, out);
299
+
}
300
+
EXPORT_SYMBOL_GPL(sha3_256);
301
+
302
+
void sha3_384(const u8 *in, size_t in_len, u8 out[SHA3_384_DIGEST_SIZE])
303
+
{
304
+
struct sha3_ctx ctx;
305
+
306
+
sha3_384_init(&ctx);
307
+
sha3_update(&ctx, in, in_len);
308
+
sha3_final(&ctx, out);
309
+
}
310
+
EXPORT_SYMBOL_GPL(sha3_384);
311
+
312
+
void sha3_512(const u8 *in, size_t in_len, u8 out[SHA3_512_DIGEST_SIZE])
313
+
{
314
+
struct sha3_ctx ctx;
315
+
316
+
sha3_512_init(&ctx);
317
+
sha3_update(&ctx, in, in_len);
318
+
sha3_final(&ctx, out);
319
+
}
320
+
EXPORT_SYMBOL_GPL(sha3_512);
321
+
322
+
void shake128(const u8 *in, size_t in_len, u8 *out, size_t out_len)
323
+
{
324
+
struct shake_ctx ctx;
325
+
326
+
shake128_init(&ctx);
327
+
shake_update(&ctx, in, in_len);
328
+
shake_squeeze(&ctx, out, out_len);
329
+
shake_zeroize_ctx(&ctx);
330
+
}
331
+
EXPORT_SYMBOL_GPL(shake128);
332
+
333
+
void shake256(const u8 *in, size_t in_len, u8 *out, size_t out_len)
334
+
{
335
+
struct shake_ctx ctx;
336
+
337
+
shake256_init(&ctx);
338
+
shake_update(&ctx, in, in_len);
339
+
shake_squeeze(&ctx, out, out_len);
340
+
shake_zeroize_ctx(&ctx);
341
+
}
342
+
EXPORT_SYMBOL_GPL(shake256);
343
+
344
+
#ifdef sha3_mod_init_arch
345
+
static int __init sha3_mod_init(void)
346
+
{
347
+
sha3_mod_init_arch();
348
+
return 0;
349
+
}
350
+
subsys_initcall(sha3_mod_init);
351
+
352
+
static void __exit sha3_mod_exit(void)
353
+
{
354
+
}
355
+
module_exit(sha3_mod_exit);
356
+
#endif
357
+
358
+
MODULE_DESCRIPTION("SHA-3 library functions");
359
+
MODULE_LICENSE("GPL");