Merge tag 'dmaengine-7.0-rc1' of git://git.kernel.org/pub/scm/linux/kernel/git/vkoul/dmaengine

+15

Documentation/ABI/stable/sysfs-driver-dma-idxd

··· 136 136 Also last configuration error overloaded. 137 137 Writing to it will clear the status. 138 138 139 + What: /sys/bus/dsa/devices/dsa<m>/dsacaps 140 + Date: April 5, 2026 141 + KernelVersion: 6.20.0 142 + Contact: dmaengine@vger.kernel.org 143 + Description: The DSA3 specification introduces three new capability 144 + registers: dsacap[0-2]. User components (e.g., configuration 145 + libraries and workload applications) require this information 146 + to properly utilize the DSA3 features. 147 + This includes SGL capability support, Enabling hardware-specific 148 + optimizations, Configuring memory, etc. 149 + The output format is '<dsacap2>,<dsacap1>,<dsacap0>' where each 150 + DSA cap value is a 64 bit hex value. 151 + This attribute should only be visible on DSA devices of version 152 + 3 or later. 153 + 139 154 What: /sys/bus/dsa/devices/dsa<m>/iaa_cap 140 155 Date: Sept 14, 2022 141 156 KernelVersion: 6.0.0

+1 -1

Documentation/devicetree/bindings/dma/arm-pl08x.yaml

··· 4 4 $id: http://devicetree.org/schemas/dma/arm-pl08x.yaml# 5 5 $schema: http://devicetree.org/meta-schemas/core.yaml# 6 6 7 - title: ARM PrimeCells PL080 and PL081 and derivatives DMA controller 7 + title: ARM PrimeCell PL080 and PL081 and derivatives DMA controller 8 8 9 9 maintainers: 10 10 - Vinod Koul <vkoul@kernel.org>

+3 -1

Documentation/devicetree/bindings/dma/atmel,sama5d4-dma.yaml

··· 33 33 - microchip,sam9x7-dma 34 34 - const: atmel,sama5d4-dma 35 35 - items: 36 - - const: microchip,sama7d65-dma 36 + - enum: 37 + - microchip,lan9691-dma 38 + - microchip,sama7d65-dma 37 39 - const: microchip,sama7g5-dma 38 40 39 41 "#dma-cells":

+20

Documentation/devicetree/bindings/dma/mediatek,uart-dma.yaml

··· 7 7 title: MediaTek UART APDMA controller 8 8 9 9 maintainers: 10 + - AngeloGioacchino Del Regno <angelogioacchino.delregno@collabora.com> 10 11 - Long Cheng <long.cheng@mediatek.com> 11 12 12 13 description: | ··· 24 23 - enum: 25 24 - mediatek,mt2712-uart-dma 26 25 - mediatek,mt6795-uart-dma 26 + - mediatek,mt8173-uart-dma 27 + - mediatek,mt8183-uart-dma 27 28 - mediatek,mt8365-uart-dma 28 29 - mediatek,mt8516-uart-dma 29 30 - const: mediatek,mt6577-uart-dma 31 + - items: 32 + - enum: 33 + - mediatek,mt7988-uart-dma 34 + - mediatek,mt8186-uart-dma 35 + - mediatek,mt8188-uart-dma 36 + - mediatek,mt8192-uart-dma 37 + - mediatek,mt8195-uart-dma 38 + - const: mediatek,mt6835-uart-dma 39 + - items: 40 + - enum: 41 + - mediatek,mt6991-uart-dma 42 + - mediatek,mt8196-uart-dma 43 + - const: mediatek,mt6985-uart-dma 30 44 - enum: 31 45 - mediatek,mt6577-uart-dma 46 + - mediatek,mt6795-uart-dma 47 + - mediatek,mt6835-uart-dma 48 + - mediatek,mt6985-uart-dma 32 49 33 50 reg: 34 51 minItems: 1 ··· 77 58 78 59 mediatek,dma-33bits: 79 60 type: boolean 61 + deprecated: true 80 62 description: Enable 33-bits UART APDMA support 81 63 82 64 required:

+3 -1

Documentation/devicetree/bindings/dma/qcom,gpi.yaml

··· 24 24 - qcom,sm6350-gpi-dma 25 25 - items: 26 26 - enum: 27 + - qcom,glymur-gpi-dma 28 + - qcom,kaanapali-gpi-dma 27 29 - qcom,milos-gpi-dma 28 30 - qcom,qcm2290-gpi-dma 29 31 - qcom,qcs8300-gpi-dma ··· 60 58 description: 61 59 Interrupt lines for each GPI instance 62 60 minItems: 1 63 - maxItems: 13 61 + maxItems: 16 64 62 65 63 "#dma-cells": 66 64 const: 3

+1

Documentation/devicetree/bindings/dma/renesas,rz-dmac.yaml

··· 24 24 - items: 25 25 - enum: 26 26 - renesas,r9a09g047-dmac # RZ/G3E 27 + - renesas,r9a09g056-dmac # RZ/V2N 27 28 - const: renesas,r9a09g057-dmac 28 29 29 30 - const: renesas,r9a09g057-dmac # RZ/V2H(P)

+9 -5

Documentation/devicetree/bindings/dma/snps,dw-axi-dmac.yaml

··· 17 17 18 18 properties: 19 19 compatible: 20 - enum: 21 - - snps,axi-dma-1.01a 22 - - intel,kmb-axi-dma 23 - - starfive,jh7110-axi-dma 24 - - starfive,jh8100-axi-dma 20 + oneOf: 21 + - enum: 22 + - snps,axi-dma-1.01a 23 + - intel,kmb-axi-dma 24 + - starfive,jh7110-axi-dma 25 + - starfive,jh8100-axi-dma 26 + - items: 27 + - const: altr,agilex5-axi-dma 28 + - const: snps,axi-dma-1.01a 25 29 26 30 reg: 27 31 minItems: 1

+4 -1

Documentation/driver-api/dmaengine/provider.rst

··· 411 411 - This structure can be initialized using the function 412 412 ``dma_async_tx_descriptor_init``. 413 413 414 - - You'll also need to set two fields in this structure: 414 + - You'll also need to set following fields in this structure: 415 415 416 416 - flags: 417 417 TODO: Can it be modified by the driver itself, or ··· 420 420 - tx_submit: A pointer to a function you have to implement, 421 421 that is supposed to push the current transaction descriptor to a 422 422 pending queue, waiting for issue_pending to be called. 423 + 424 + - phys: Physical address of the descriptor which is used later by 425 + the dma engine to read the descriptor and initiate transfer. 423 426 424 427 - In this structure the function pointer callback_result can be 425 428 initialized in order for the submitter to be notified that a

+1

MAINTAINERS

··· 7542 7542 7543 7543 DMA GENERIC OFFLOAD ENGINE SUBSYSTEM 7544 7544 M: Vinod Koul <vkoul@kernel.org> 7545 + R: Frank Li <Frank.Li@kernel.org> 7545 7546 L: dmaengine@vger.kernel.org 7546 7547 S: Maintained 7547 7548 Q: https://patchwork.kernel.org/project/linux-dmaengine/list/

+1 -1

drivers/dma/Kconfig

··· 590 590 591 591 config ST_FDMA 592 592 tristate "ST FDMA dmaengine support" 593 - depends on ARCH_STI 593 + depends on ARCH_STI || COMPILE_TEST 594 594 depends on REMOTEPROC 595 595 select ST_SLIM_REMOTEPROC 596 596 select DMA_ENGINE

+2 -4

drivers/dma/altera-msgdma.c

··· 396 396 void *desc = NULL; 397 397 size_t len, avail; 398 398 dma_addr_t dma_dst, dma_src; 399 - u32 desc_cnt = 0, i; 400 - struct scatterlist *sg; 399 + u32 desc_cnt; 401 400 u32 stride; 402 401 unsigned long irqflags; 403 402 404 - for_each_sg(sgl, sg, sg_len, i) 405 - desc_cnt += DIV_ROUND_UP(sg_dma_len(sg), MSGDMA_MAX_TRANS_LEN); 403 + desc_cnt = sg_nents_for_dma(sgl, sg_len, MSGDMA_MAX_TRANS_LEN); 406 404 407 405 spin_lock_irqsave(&mdev->lock, irqflags); 408 406 if (desc_cnt > mdev->desc_free_cnt) {

+2 -2

drivers/dma/amba-pl08x.c

··· 1010 1010 /* 1011 1011 * Remove all src, dst and transfer size bits, then set the 1012 1012 * width and size according to the parameters. The bit offsets 1013 - * are different in the FTDMAC020 so we need to accound for this. 1013 + * are different in the FTDMAC020 so we need to account for this. 1014 1014 */ 1015 1015 if (pl08x->vd->ftdmac020) { 1016 1016 retbits &= ~FTDMAC020_LLI_DST_WIDTH_MSK; ··· 2978 2978 return ret; 2979 2979 } 2980 2980 2981 - /* PL080 has 8 channels and the PL080 have just 2 */ 2981 + /* PL080 has 8 channels and the PL081 have just 2 */ 2982 2982 static struct vendor_data vendor_pl080 = { 2983 2983 .config_offset = PL080_CH_CONFIG, 2984 2984 .channels = 8,

+23 -14

drivers/dma/at_xdmac.c

··· 379 379 if (!desc->active_xfer) 380 380 continue; 381 381 382 - pm_runtime_mark_last_busy(atxdmac->dev); 383 382 pm_runtime_put_autosuspend(atxdmac->dev); 384 383 } 385 384 } ··· 412 413 413 414 ret = !!(at_xdmac_chan_read(atchan, AT_XDMAC_GS) & atchan->mask); 414 415 415 - pm_runtime_mark_last_busy(atxdmac->dev); 416 416 pm_runtime_put_autosuspend(atxdmac->dev); 417 417 418 418 return ret; ··· 444 446 } 445 447 } 446 448 447 - pm_runtime_mark_last_busy(atxdmac->dev); 448 449 pm_runtime_put_autosuspend(atxdmac->dev); 449 450 } 450 451 ··· 1673 1676 1674 1677 spin_unlock: 1675 1678 spin_unlock_irqrestore(&atchan->lock, flags); 1676 - pm_runtime_mark_last_busy(atxdmac->dev); 1677 1679 pm_runtime_put_autosuspend(atxdmac->dev); 1678 1680 return ret; 1679 1681 } ··· 1754 1758 __func__, &bad_desc->lld.mbr_sa, &bad_desc->lld.mbr_da, 1755 1759 bad_desc->lld.mbr_ubc); 1756 1760 1757 - pm_runtime_mark_last_busy(atxdmac->dev); 1758 1761 pm_runtime_put_autosuspend(atxdmac->dev); 1759 1762 1760 1763 /* Then continue with usual descriptor management */ ··· 1817 1822 * Decrement runtime PM ref counter incremented in 1818 1823 * at_xdmac_start_xfer(). 1819 1824 */ 1820 - pm_runtime_mark_last_busy(atxdmac->dev); 1821 1825 pm_runtime_put_autosuspend(atxdmac->dev); 1822 1826 } 1823 1827 ··· 1948 1954 1949 1955 spin_unlock_irqrestore(&atchan->lock, flags); 1950 1956 1951 - pm_runtime_mark_last_busy(atxdmac->dev); 1952 1957 pm_runtime_put_autosuspend(atxdmac->dev); 1953 1958 1954 1959 return 0; ··· 1991 1998 1992 1999 unlock: 1993 2000 spin_unlock_irqrestore(&atchan->lock, flags); 1994 - pm_runtime_mark_last_busy(atxdmac->dev); 1995 2001 pm_runtime_put_autosuspend(atxdmac->dev); 1996 2002 1997 2003 return ret; ··· 2033 2041 clear_bit(AT_XDMAC_CHAN_IS_CYCLIC, &atchan->status); 2034 2042 spin_unlock_irqrestore(&atchan->lock, flags); 2035 2043 2036 - pm_runtime_mark_last_busy(atxdmac->dev); 2037 2044 pm_runtime_put_autosuspend(atxdmac->dev); 2038 2045 2039 2046 return 0; ··· 2226 2235 } 2227 2236 } 2228 2237 2229 - pm_runtime_mark_last_busy(atxdmac->dev); 2230 2238 pm_runtime_put_autosuspend(atxdmac->dev); 2231 2239 2232 2240 return 0; ··· 2247 2257 return clk_enable(atxdmac->clk); 2248 2258 } 2249 2259 2260 + static inline int at_xdmac_get_channel_number(struct platform_device *pdev, 2261 + u32 reg, u32 *pchannels) 2262 + { 2263 + int ret; 2264 + 2265 + if (reg) { 2266 + *pchannels = AT_XDMAC_NB_CH(reg); 2267 + return 0; 2268 + } 2269 + 2270 + ret = of_property_read_u32(pdev->dev.of_node, "dma-channels", pchannels); 2271 + if (ret) 2272 + dev_err(&pdev->dev, "can't get number of channels\n"); 2273 + 2274 + return ret; 2275 + } 2276 + 2250 2277 static int at_xdmac_probe(struct platform_device *pdev) 2251 2278 { 2252 2279 struct at_xdmac *atxdmac; 2253 - int irq, nr_channels, i, ret; 2280 + int irq, ret; 2254 2281 void __iomem *base; 2255 - u32 reg; 2282 + u32 nr_channels, i, reg; 2256 2283 2257 2284 irq = platform_get_irq(pdev, 0); 2258 2285 if (irq < 0) ··· 2285 2278 * of channels to do the allocation. 2286 2279 */ 2287 2280 reg = readl_relaxed(base + AT_XDMAC_GTYPE); 2288 - nr_channels = AT_XDMAC_NB_CH(reg); 2281 + ret = at_xdmac_get_channel_number(pdev, reg, &nr_channels); 2282 + if (ret) 2283 + return ret; 2284 + 2289 2285 if (nr_channels > AT_XDMAC_MAX_CHAN) { 2290 2286 dev_err(&pdev->dev, "invalid number of channels (%u)\n", 2291 2287 nr_channels); ··· 2422 2412 2423 2413 at_xdmac_axi_config(pdev); 2424 2414 2425 - pm_runtime_mark_last_busy(&pdev->dev); 2426 2415 pm_runtime_put_autosuspend(&pdev->dev); 2427 2416 2428 2417 return 0;

+1 -18

drivers/dma/bcm2835-dma.c

··· 260 260 control_block->info |= finalextrainfo; 261 261 } 262 262 263 - static inline size_t bcm2835_dma_count_frames_for_sg( 264 - struct bcm2835_chan *c, 265 - struct scatterlist *sgl, 266 - unsigned int sg_len) 267 - { 268 - size_t frames = 0; 269 - struct scatterlist *sgent; 270 - unsigned int i; 271 - size_t plength = bcm2835_dma_max_frame_length(c); 272 - 273 - for_each_sg(sgl, sgent, sg_len, i) 274 - frames += bcm2835_dma_frames_for_length( 275 - sg_dma_len(sgent), plength); 276 - 277 - return frames; 278 - } 279 - 280 263 /** 281 264 * bcm2835_dma_create_cb_chain - create a control block and fills data in 282 265 * ··· 655 672 } 656 673 657 674 /* count frames in sg list */ 658 - frames = bcm2835_dma_count_frames_for_sg(c, sgl, sg_len); 675 + frames = sg_nents_for_dma(sgl, sg_len, bcm2835_dma_max_frame_length(c)); 659 676 660 677 /* allocate the CB chain */ 661 678 d = bcm2835_dma_create_cb_chain(chan, direction, false,

+34 -19

drivers/dma/dma-axi-dmac.c

··· 8 8 9 9 #include <linux/adi-axi-common.h> 10 10 #include <linux/bitfield.h> 11 + #include <linux/cleanup.h> 11 12 #include <linux/clk.h> 12 13 #include <linux/device.h> 13 14 #include <linux/dma-mapping.h> ··· 70 69 #define AXI_DMAC_REG_START_TRANSFER 0x408 71 70 #define AXI_DMAC_REG_FLAGS 0x40c 72 71 #define AXI_DMAC_REG_DEST_ADDRESS 0x410 72 + #define AXI_DMAC_REG_DEST_ADDRESS_HIGH 0x490 73 73 #define AXI_DMAC_REG_SRC_ADDRESS 0x414 74 + #define AXI_DMAC_REG_SRC_ADDRESS_HIGH 0x494 74 75 #define AXI_DMAC_REG_X_LENGTH 0x418 75 76 #define AXI_DMAC_REG_Y_LENGTH 0x41c 76 77 #define AXI_DMAC_REG_DEST_STRIDE 0x420 ··· 236 233 unsigned int flags = 0; 237 234 unsigned int val; 238 235 239 - if (!chan->hw_sg) { 240 - val = axi_dmac_read(dmac, AXI_DMAC_REG_START_TRANSFER); 241 - if (val) /* Queue is full, wait for the next SOT IRQ */ 242 - return; 243 - } 236 + val = axi_dmac_read(dmac, AXI_DMAC_REG_START_TRANSFER); 237 + if (val) /* Queue is full, wait for the next SOT IRQ */ 238 + return; 244 239 245 240 desc = chan->next_desc; 246 241 ··· 248 247 return; 249 248 list_move_tail(&vdesc->node, &chan->active_descs); 250 249 desc = to_axi_dmac_desc(vdesc); 250 + chan->next_desc = desc; 251 251 } 252 252 sg = &desc->sg[desc->num_submitted]; 253 253 ··· 267 265 else 268 266 chan->next_desc = NULL; 269 267 flags |= AXI_DMAC_FLAG_LAST; 270 - } else { 271 - chan->next_desc = desc; 272 268 } 273 269 274 270 sg->hw->id = axi_dmac_read(dmac, AXI_DMAC_REG_TRANSFER_ID); ··· 274 274 if (!chan->hw_sg) { 275 275 if (axi_dmac_dest_is_mem(chan)) { 276 276 axi_dmac_write(dmac, AXI_DMAC_REG_DEST_ADDRESS, sg->hw->dest_addr); 277 + axi_dmac_write(dmac, AXI_DMAC_REG_DEST_ADDRESS_HIGH, 278 + sg->hw->dest_addr >> 32); 277 279 axi_dmac_write(dmac, AXI_DMAC_REG_DEST_STRIDE, sg->hw->dst_stride); 278 280 } 279 281 280 282 if (axi_dmac_src_is_mem(chan)) { 281 283 axi_dmac_write(dmac, AXI_DMAC_REG_SRC_ADDRESS, sg->hw->src_addr); 284 + axi_dmac_write(dmac, AXI_DMAC_REG_SRC_ADDRESS_HIGH, sg->hw->src_addr >> 32); 282 285 axi_dmac_write(dmac, AXI_DMAC_REG_SRC_STRIDE, sg->hw->src_stride); 283 286 } 284 287 } ··· 677 674 if (direction != chan->direction) 678 675 return NULL; 679 676 680 - num_sgs = 0; 681 - for_each_sg(sgl, sg, sg_len, i) 682 - num_sgs += DIV_ROUND_UP(sg_dma_len(sg), chan->max_length); 683 - 677 + num_sgs = sg_nents_for_dma(sgl, sg_len, chan->max_length); 684 678 desc = axi_dmac_alloc_desc(chan, num_sgs); 685 679 if (!desc) 686 680 return NULL; ··· 925 925 926 926 static int axi_dmac_parse_dt(struct device *dev, struct axi_dmac *dmac) 927 927 { 928 - struct device_node *of_channels, *of_chan; 929 928 int ret; 930 929 931 - of_channels = of_get_child_by_name(dev->of_node, "adi,channels"); 930 + struct device_node *of_channels __free(device_node) = of_get_child_by_name(dev->of_node, 931 + "adi,channels"); 932 932 if (of_channels == NULL) 933 933 return -ENODEV; 934 934 935 - for_each_child_of_node(of_channels, of_chan) { 935 + for_each_child_of_node_scoped(of_channels, of_chan) { 936 936 ret = axi_dmac_parse_chan_dt(of_chan, &dmac->chan); 937 - if (ret) { 938 - of_node_put(of_chan); 939 - of_node_put(of_channels); 937 + if (ret) 940 938 return -EINVAL; 941 - } 942 939 } 943 - of_node_put(of_channels); 944 940 945 941 return 0; 946 942 } ··· 989 993 static int axi_dmac_detect_caps(struct axi_dmac *dmac, unsigned int version) 990 994 { 991 995 struct axi_dmac_chan *chan = &dmac->chan; 996 + struct device *dev = dmac->dma_dev.dev; 997 + u32 mask; 998 + int ret; 992 999 993 1000 axi_dmac_write(dmac, AXI_DMAC_REG_FLAGS, AXI_DMAC_FLAG_CYCLIC); 994 1001 if (axi_dmac_read(dmac, AXI_DMAC_REG_FLAGS) == AXI_DMAC_FLAG_CYCLIC) ··· 1024 1025 dev_err(dmac->dma_dev.dev, 1025 1026 "Source memory-mapped interface not supported."); 1026 1027 return -ENODEV; 1028 + } 1029 + 1030 + if (axi_dmac_dest_is_mem(chan)) { 1031 + axi_dmac_write(dmac, AXI_DMAC_REG_DEST_ADDRESS_HIGH, 0xffffffff); 1032 + mask = axi_dmac_read(dmac, AXI_DMAC_REG_DEST_ADDRESS_HIGH); 1033 + } else { 1034 + axi_dmac_write(dmac, AXI_DMAC_REG_SRC_ADDRESS_HIGH, 0xffffffff); 1035 + mask = axi_dmac_read(dmac, AXI_DMAC_REG_SRC_ADDRESS_HIGH); 1036 + } 1037 + 1038 + mask = 32 + fls(mask); 1039 + 1040 + ret = dma_set_mask_and_coherent(dev, DMA_BIT_MASK(mask)); 1041 + if (ret) { 1042 + dev_err(dev, "DMA mask set error %d\n", ret); 1043 + return ret; 1027 1044 } 1028 1045 1029 1046 if (version >= ADI_AXI_PCORE_VER(4, 2, 'a'))

+2 -4

drivers/dma/dw-axi-dmac/dw-axi-dmac-platform.c

··· 850 850 unsigned int loop = 0; 851 851 struct scatterlist *sg; 852 852 size_t axi_block_len; 853 - u32 len, num_sgs = 0; 853 + u32 len, num_sgs; 854 854 unsigned int i; 855 855 dma_addr_t mem; 856 856 int status; ··· 867 867 if (axi_block_len == 0) 868 868 return NULL; 869 869 870 - for_each_sg(sgl, sg, sg_len, i) 871 - num_sgs += DIV_ROUND_UP(sg_dma_len(sg), axi_block_len); 872 - 870 + num_sgs = sg_nents_for_dma(sgl, sg_len, axi_block_len); 873 871 desc = axi_desc_alloc(num_sgs); 874 872 if (unlikely(!desc)) 875 873 goto err_desc_get;

+2 -2

drivers/dma/dw-edma/dw-edma-pcie.c

··· 161 161 const struct pci_device_id *pid) 162 162 { 163 163 struct dw_edma_pcie_data *pdata = (void *)pid->driver_data; 164 - struct dw_edma_pcie_data *vsec_data __free(kfree) = NULL; 165 164 struct device *dev = &pdev->dev; 166 165 struct dw_edma_chip *chip; 167 166 int err, nr_irqs; 168 167 int i, mask; 169 168 170 - vsec_data = kmalloc(sizeof(*vsec_data), GFP_KERNEL); 169 + struct dw_edma_pcie_data *vsec_data __free(kfree) = 170 + kmalloc(sizeof(*vsec_data), GFP_KERNEL); 171 171 if (!vsec_data) 172 172 return -ENOMEM; 173 173

-1

drivers/dma/fsl-edma-main.c

··· 915 915 of_dma_controller_free(np); 916 916 dma_async_device_unregister(&fsl_edma->dma_dev); 917 917 fsl_edma_cleanup_vchan(&fsl_edma->dma_dev); 918 - fsl_disable_clocks(fsl_edma, fsl_edma->drvdata->dmamuxs); 919 918 } 920 919 921 920 static int fsl_edma_suspend_late(struct device *dev)

+5

drivers/dma/idxd/device.c

··· 390 390 memset(wq->name, 0, WQ_NAME_SIZE); 391 391 wq->max_xfer_bytes = WQ_DEFAULT_MAX_XFER; 392 392 idxd_wq_set_max_batch_size(idxd->data->type, wq, WQ_DEFAULT_MAX_BATCH); 393 + idxd_wq_set_init_max_sgl_size(idxd, wq); 393 394 if (wq->opcap_bmap) 394 395 bitmap_copy(wq->opcap_bmap, idxd->opcap_bmap, IDXD_MAX_OPCAP_BITS); 395 396 } ··· 990 989 /* bytes 12-15 */ 991 990 wq->wqcfg->max_xfer_shift = ilog2(wq->max_xfer_bytes); 992 991 idxd_wqcfg_set_max_batch_shift(idxd->data->type, wq->wqcfg, ilog2(wq->max_batch_size)); 992 + if (idxd_sgl_supported(idxd)) 993 + wq->wqcfg->max_sgl_shift = ilog2(wq->max_sgl_size); 993 994 994 995 /* bytes 32-63 */ 995 996 if (idxd->hw.wq_cap.op_config && wq->opcap_bmap) { ··· 1170 1167 1171 1168 wq->max_xfer_bytes = 1ULL << wq->wqcfg->max_xfer_shift; 1172 1169 idxd_wq_set_max_batch_size(idxd->data->type, wq, 1U << wq->wqcfg->max_batch_shift); 1170 + if (idxd_sgl_supported(idxd)) 1171 + wq->max_sgl_size = 1U << wq->wqcfg->max_sgl_shift; 1173 1172 1174 1173 for (i = 0; i < WQCFG_STRIDES(idxd); i++) { 1175 1174 wqcfg_offset = WQCFG_OFFSET(idxd, wq->id, i);

+19

drivers/dma/idxd/idxd.h

··· 227 227 char name[WQ_NAME_SIZE + 1]; 228 228 u64 max_xfer_bytes; 229 229 u32 max_batch_size; 230 + u32 max_sgl_size; 230 231 231 232 /* Lock to protect upasid_xa access. */ 232 233 struct mutex uc_lock; ··· 253 252 struct opcap opcap; 254 253 u32 cmd_cap; 255 254 union iaa_cap_reg iaa_cap; 255 + union dsacap0_reg dsacap0; 256 + union dsacap1_reg dsacap1; 257 + union dsacap2_reg dsacap2; 256 258 }; 257 259 258 260 enum idxd_device_state { ··· 349 345 350 346 u64 max_xfer_bytes; 351 347 u32 max_batch_size; 348 + u32 max_sgl_size; 352 349 int max_groups; 353 350 int max_engines; 354 351 int max_rdbufs; ··· 692 687 wq->max_batch_size = 0; 693 688 else 694 689 wq->max_batch_size = max_batch_size; 690 + } 691 + 692 + static bool idxd_sgl_supported(struct idxd_device *idxd) 693 + { 694 + return idxd->data->type == IDXD_TYPE_DSA && 695 + idxd->hw.version >= DEVICE_VERSION_3 && 696 + idxd->hw.dsacap0.sgl_formats; 697 + } 698 + 699 + static inline void idxd_wq_set_init_max_sgl_size(struct idxd_device *idxd, 700 + struct idxd_wq *wq) 701 + { 702 + if (idxd_sgl_supported(idxd)) 703 + wq->max_sgl_size = 1U << idxd->hw.dsacap0.max_sgl_shift; 695 704 } 696 705 697 706 static inline void idxd_wqcfg_set_max_batch_shift(int idxd_type, union wqcfg *wqcfg,

+11

drivers/dma/idxd/init.c

··· 222 222 init_completion(&wq->wq_resurrect); 223 223 wq->max_xfer_bytes = WQ_DEFAULT_MAX_XFER; 224 224 idxd_wq_set_max_batch_size(idxd->data->type, wq, WQ_DEFAULT_MAX_BATCH); 225 + idxd_wq_set_init_max_sgl_size(idxd, wq); 225 226 wq->enqcmds_retries = IDXD_ENQCMDS_RETRIES; 226 227 wq->wqcfg = kzalloc_node(idxd->wqcfg_size, GFP_KERNEL, dev_to_node(dev)); 227 228 if (!wq->wqcfg) { ··· 585 584 dev_dbg(dev, "opcap[%d]: %#llx\n", i, idxd->hw.opcap.bits[i]); 586 585 } 587 586 multi_u64_to_bmap(idxd->opcap_bmap, &idxd->hw.opcap.bits[0], 4); 587 + 588 + if (idxd->hw.version >= DEVICE_VERSION_3) { 589 + idxd->hw.dsacap0.bits = ioread64(idxd->reg_base + IDXD_DSACAP0_OFFSET); 590 + idxd->hw.dsacap1.bits = ioread64(idxd->reg_base + IDXD_DSACAP1_OFFSET); 591 + idxd->hw.dsacap2.bits = ioread64(idxd->reg_base + IDXD_DSACAP2_OFFSET); 592 + } 593 + if (idxd_sgl_supported(idxd)) { 594 + idxd->max_sgl_size = 1U << idxd->hw.dsacap0.max_sgl_shift; 595 + dev_dbg(dev, "max sgl size: %u\n", idxd->max_sgl_size); 596 + } 588 597 589 598 /* read iaa cap */ 590 599 if (idxd->data->type == IDXD_TYPE_IAX && idxd->hw.version >= DEVICE_VERSION_2)

+27 -1

drivers/dma/idxd/registers.h

··· 18 18 19 19 #define DEVICE_VERSION_1 0x100 20 20 #define DEVICE_VERSION_2 0x200 21 + #define DEVICE_VERSION_3 0x300 21 22 22 23 #define IDXD_MMIO_BAR 0 23 24 #define IDXD_WQ_BAR 2 ··· 390 389 /* bytes 12-15 */ 391 390 u32 max_xfer_shift:5; 392 391 u32 max_batch_shift:4; 393 - u32 rsvd4:23; 392 + u32 max_sgl_shift:4; 393 + u32 rsvd4:19; 394 394 395 395 /* bytes 16-19 */ 396 396 u16 occupancy_inth; ··· 586 584 u32 bits_lower32; 587 585 u32 bits_upper32; 588 586 }; 587 + u64 bits; 588 + }; 589 + 590 + #define IDXD_DSACAP0_OFFSET 0x180 591 + union dsacap0_reg { 592 + u64 bits; 593 + struct { 594 + u64 max_sgl_shift:4; 595 + u64 max_gr_block_shift:4; 596 + u64 ops_inter_domain:7; 597 + u64 rsvd1:17; 598 + u64 sgl_formats:16; 599 + u64 max_sg_process:8; 600 + u64 rsvd2:8; 601 + }; 602 + }; 603 + 604 + #define IDXD_DSACAP1_OFFSET 0x188 605 + union dsacap1_reg { 606 + u64 bits; 607 + }; 608 + 609 + #define IDXD_DSACAP2_OFFSET 0x190 610 + union dsacap2_reg { 589 611 u64 bits; 590 612 }; 591 613

+24

drivers/dma/idxd/sysfs.c

··· 1713 1713 } 1714 1714 static DEVICE_ATTR_RW(event_log_size); 1715 1715 1716 + static ssize_t dsacaps_show(struct device *dev, 1717 + struct device_attribute *attr, char *buf) 1718 + { 1719 + struct idxd_device *idxd = confdev_to_idxd(dev); 1720 + 1721 + return sysfs_emit(buf, "%016llx,%016llx,%016llx\n", 1722 + (u64)idxd->hw.dsacap2.bits, 1723 + (u64)idxd->hw.dsacap1.bits, 1724 + (u64)idxd->hw.dsacap0.bits); 1725 + } 1726 + static DEVICE_ATTR_RO(dsacaps); 1727 + 1716 1728 static bool idxd_device_attr_max_batch_size_invisible(struct attribute *attr, 1717 1729 struct idxd_device *idxd) 1718 1730 { ··· 1762 1750 !idxd->hw.gen_cap.evl_support); 1763 1751 } 1764 1752 1753 + static bool idxd_device_attr_dsacaps_invisible(struct attribute *attr, 1754 + struct idxd_device *idxd) 1755 + { 1756 + return attr == &dev_attr_dsacaps.attr && 1757 + (idxd->data->type != IDXD_TYPE_DSA || 1758 + idxd->hw.version < DEVICE_VERSION_3); 1759 + } 1760 + 1765 1761 static umode_t idxd_device_attr_visible(struct kobject *kobj, 1766 1762 struct attribute *attr, int n) 1767 1763 { ··· 1786 1766 return 0; 1787 1767 1788 1768 if (idxd_device_attr_event_log_size_invisible(attr, idxd)) 1769 + return 0; 1770 + 1771 + if (idxd_device_attr_dsacaps_invisible(attr, idxd)) 1789 1772 return 0; 1790 1773 1791 1774 return attr->mode; ··· 1818 1795 &dev_attr_cmd_status.attr, 1819 1796 &dev_attr_iaa_cap.attr, 1820 1797 &dev_attr_event_log_size.attr, 1798 + &dev_attr_dsacaps.attr, 1821 1799 NULL, 1822 1800 }; 1823 1801

+2 -7

drivers/dma/k3dma.c

··· 536 536 size_t len, avail, total = 0; 537 537 struct scatterlist *sg; 538 538 dma_addr_t addr, src = 0, dst = 0; 539 - int num = sglen, i; 539 + int num, i; 540 540 541 541 if (sgl == NULL) 542 542 return NULL; 543 543 544 544 c->cyclic = 0; 545 545 546 - for_each_sg(sgl, sg, sglen, i) { 547 - avail = sg_dma_len(sg); 548 - if (avail > DMA_MAX_SIZE) 549 - num += DIV_ROUND_UP(avail, DMA_MAX_SIZE) - 1; 550 - } 551 - 546 + num = sg_nents_for_dma(sgl, sglen, DMA_MAX_SIZE); 552 547 ds = k3_dma_alloc_desc_resource(num, chan); 553 548 if (!ds) 554 549 return NULL;

+2 -7

drivers/dma/lgm/lgm-dma.c

··· 1164 1164 struct dw2_desc *hw_ds; 1165 1165 struct dw2_desc_sw *ds; 1166 1166 struct scatterlist *sg; 1167 - int num = sglen, i; 1168 1167 dma_addr_t addr; 1168 + int num, i; 1169 1169 1170 1170 if (!sgl) 1171 1171 return NULL; ··· 1173 1173 if (d->ver > DMA_VER22) 1174 1174 return ldma_chan_desc_cfg(chan, sgl->dma_address, sglen); 1175 1175 1176 - for_each_sg(sgl, sg, sglen, i) { 1177 - avail = sg_dma_len(sg); 1178 - if (avail > DMA_MAX_SIZE) 1179 - num += DIV_ROUND_UP(avail, DMA_MAX_SIZE) - 1; 1180 - } 1181 - 1176 + num = sg_nents_for_dma(sgl, sglen, DMA_MAX_SIZE); 1182 1177 ds = dma_alloc_desc_resource(num, c); 1183 1178 if (!ds) 1184 1179 return NULL;

+18 -17

drivers/dma/mediatek/mtk-uart-apdma.c

··· 41 41 #define VFF_STOP_CLR_B 0 42 42 #define VFF_EN_CLR_B 0 43 43 #define VFF_INT_EN_CLR_B 0 44 - #define VFF_4G_SUPPORT_CLR_B 0 44 + #define VFF_ADDR2_CLR_B 0 45 45 46 46 /* 47 47 * interrupt trigger level for tx ··· 72 72 /* TX: the buffer size SW can write. RX: the buffer size HW can write. */ 73 73 #define VFF_LEFT_SIZE 0x40 74 74 #define VFF_DEBUG_STATUS 0x50 75 - #define VFF_4G_SUPPORT 0x54 75 + #define VFF_ADDR2 0x54 76 76 77 77 struct mtk_uart_apdmadev { 78 78 struct dma_device ddev; 79 79 struct clk *clk; 80 - bool support_33bits; 80 + bool support_ext_addr; 81 81 unsigned int dma_requests; 82 82 }; 83 83 ··· 148 148 mtk_uart_apdma_write(c, VFF_WPT, 0); 149 149 mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_TX_INT_CLR_B); 150 150 151 - if (mtkd->support_33bits) 152 - mtk_uart_apdma_write(c, VFF_4G_SUPPORT, VFF_4G_EN_B); 151 + if (mtkd->support_ext_addr) 152 + mtk_uart_apdma_write(c, VFF_ADDR2, upper_32_bits(d->addr)); 153 153 } 154 154 155 155 mtk_uart_apdma_write(c, VFF_EN, VFF_EN_B); ··· 191 191 mtk_uart_apdma_write(c, VFF_RPT, 0); 192 192 mtk_uart_apdma_write(c, VFF_INT_FLAG, VFF_RX_INT_CLR_B); 193 193 194 - if (mtkd->support_33bits) 195 - mtk_uart_apdma_write(c, VFF_4G_SUPPORT, VFF_4G_EN_B); 194 + if (mtkd->support_ext_addr) 195 + mtk_uart_apdma_write(c, VFF_ADDR2, upper_32_bits(d->addr)); 196 196 } 197 197 198 198 mtk_uart_apdma_write(c, VFF_INT_EN, VFF_RX_INT_EN_B); ··· 297 297 goto err_pm; 298 298 } 299 299 300 - if (mtkd->support_33bits) 301 - mtk_uart_apdma_write(c, VFF_4G_SUPPORT, VFF_4G_SUPPORT_CLR_B); 300 + if (mtkd->support_ext_addr) 301 + mtk_uart_apdma_write(c, VFF_ADDR2, VFF_ADDR2_CLR_B); 302 302 303 303 err_pm: 304 304 pm_runtime_put_noidle(mtkd->ddev.dev); ··· 468 468 } 469 469 470 470 static const struct of_device_id mtk_uart_apdma_match[] = { 471 - { .compatible = "mediatek,mt6577-uart-dma", }, 471 + { .compatible = "mediatek,mt6577-uart-dma", .data = (void *)32 }, 472 + { .compatible = "mediatek,mt6795-uart-dma", .data = (void *)33 }, 473 + { .compatible = "mediatek,mt6835-uart-dma", .data = (void *)34 }, 474 + { .compatible = "mediatek,mt6985-uart-dma", .data = (void *)35 }, 472 475 { /* sentinel */ }, 473 476 }; 474 477 MODULE_DEVICE_TABLE(of, mtk_uart_apdma_match); ··· 480 477 { 481 478 struct device_node *np = pdev->dev.of_node; 482 479 struct mtk_uart_apdmadev *mtkd; 483 - int bit_mask = 32, rc; 484 480 struct mtk_chan *c; 485 - unsigned int i; 481 + unsigned int bit_mask, i; 482 + int rc; 486 483 487 484 mtkd = devm_kzalloc(&pdev->dev, sizeof(*mtkd), GFP_KERNEL); 488 485 if (!mtkd) ··· 495 492 return rc; 496 493 } 497 494 498 - if (of_property_read_bool(np, "mediatek,dma-33bits")) 499 - mtkd->support_33bits = true; 500 - 501 - if (mtkd->support_33bits) 502 - bit_mask = 33; 495 + bit_mask = (unsigned int)(uintptr_t)of_device_get_match_data(&pdev->dev); 496 + if (bit_mask > 32) 497 + mtkd->support_ext_addr = true; 503 498 504 499 rc = dma_set_mask_and_coherent(&pdev->dev, DMA_BIT_MASK(bit_mask)); 505 500 if (rc)

+1 -7

drivers/dma/pl330.c

··· 2133 2133 spin_unlock_irqrestore(&pch->lock, flags); 2134 2134 2135 2135 /* If work list empty, power down */ 2136 - if (power_down) { 2137 - pm_runtime_mark_last_busy(pch->dmac->ddma.dev); 2136 + if (power_down) 2138 2137 pm_runtime_put_autosuspend(pch->dmac->ddma.dev); 2139 - } 2140 2138 } 2141 2139 2142 2140 static struct dma_chan *of_dma_pl330_xlate(struct of_phandle_args *dma_spec, ··· 2311 2313 list_splice_tail_init(&pch->work_list, &pl330->desc_pool); 2312 2314 list_splice_tail_init(&pch->completed_list, &pl330->desc_pool); 2313 2315 spin_unlock_irqrestore(&pch->lock, flags); 2314 - pm_runtime_mark_last_busy(pl330->ddma.dev); 2315 2316 if (power_down) 2316 2317 pm_runtime_put_autosuspend(pl330->ddma.dev); 2317 2318 pm_runtime_put_autosuspend(pl330->ddma.dev); ··· 2344 2347 desc->status = PAUSED; 2345 2348 } 2346 2349 spin_unlock_irqrestore(&pch->lock, flags); 2347 - pm_runtime_mark_last_busy(pl330->ddma.dev); 2348 2350 pm_runtime_put_autosuspend(pl330->ddma.dev); 2349 2351 2350 2352 return 0; ··· 2367 2371 list_splice_tail_init(&pch->work_list, &pch->dmac->desc_pool); 2368 2372 2369 2373 spin_unlock_irqrestore(&pl330->lock, flags); 2370 - pm_runtime_mark_last_busy(pch->dmac->ddma.dev); 2371 2374 pm_runtime_put_autosuspend(pch->dmac->ddma.dev); 2372 2375 pl330_unprep_slave_fifo(pch); 2373 2376 } ··· 3171 3176 pm_runtime_irq_safe(&adev->dev); 3172 3177 pm_runtime_use_autosuspend(&adev->dev); 3173 3178 pm_runtime_set_autosuspend_delay(&adev->dev, PL330_AUTOSUSPEND_DELAY); 3174 - pm_runtime_mark_last_busy(&adev->dev); 3175 3179 pm_runtime_put_autosuspend(&adev->dev); 3176 3180 3177 3181 return 0;

+2 -3

drivers/dma/pxa_dma.c

··· 970 970 struct scatterlist *sg; 971 971 dma_addr_t dma; 972 972 u32 dcmd, dsadr = 0, dtadr = 0; 973 - unsigned int nb_desc = 0, i, j = 0; 973 + unsigned int nb_desc, i, j = 0; 974 974 975 975 if ((sgl == NULL) || (sg_len == 0)) 976 976 return NULL; ··· 979 979 dev_dbg(&chan->vc.chan.dev->device, 980 980 "%s(): dir=%d flags=%lx\n", __func__, dir, flags); 981 981 982 - for_each_sg(sgl, sg, sg_len, i) 983 - nb_desc += DIV_ROUND_UP(sg_dma_len(sg), PDMA_MAX_DESC_BYTES); 982 + nb_desc = sg_nents_for_dma(sgl, sg_len, PDMA_MAX_DESC_BYTES); 984 983 sw_desc = pxad_alloc_desc(chan, nb_desc + 1); 985 984 if (!sw_desc) 986 985 return NULL;

+71 -90

drivers/dma/qcom/bam_dma.c

··· 23 23 * indication of where the hardware is currently working. 24 24 */ 25 25 26 - #include <linux/kernel.h> 27 - #include <linux/io.h> 28 - #include <linux/init.h> 29 - #include <linux/slab.h> 30 - #include <linux/module.h> 31 - #include <linux/interrupt.h> 32 - #include <linux/dma-mapping.h> 33 - #include <linux/scatterlist.h> 34 - #include <linux/device.h> 35 - #include <linux/platform_device.h> 36 - #include <linux/of.h> 37 - #include <linux/of_address.h> 38 - #include <linux/of_irq.h> 39 - #include <linux/of_dma.h> 40 26 #include <linux/circ_buf.h> 27 + #include <linux/cleanup.h> 41 28 #include <linux/clk.h> 29 + #include <linux/device.h> 30 + #include <linux/dma-mapping.h> 42 31 #include <linux/dmaengine.h> 32 + #include <linux/init.h> 33 + #include <linux/interrupt.h> 34 + #include <linux/io.h> 35 + #include <linux/kernel.h> 36 + #include <linux/module.h> 37 + #include <linux/of_address.h> 38 + #include <linux/of_dma.h> 39 + #include <linux/of_irq.h> 40 + #include <linux/of.h> 41 + #include <linux/platform_device.h> 43 42 #include <linux/pm_runtime.h> 43 + #include <linux/scatterlist.h> 44 + #include <linux/slab.h> 44 45 45 46 #include "../dmaengine.h" 46 47 #include "../virt-dma.h" ··· 571 570 struct bam_chan *bchan = to_bam_chan(chan); 572 571 struct bam_device *bdev = bchan->bdev; 573 572 u32 val; 574 - unsigned long flags; 575 573 int ret; 576 574 577 575 ret = pm_runtime_get_sync(bdev->dev); ··· 584 584 goto err; 585 585 } 586 586 587 - spin_lock_irqsave(&bchan->vc.lock, flags); 588 - bam_reset_channel(bchan); 589 - spin_unlock_irqrestore(&bchan->vc.lock, flags); 587 + scoped_guard(spinlock_irqsave, &bchan->vc.lock) 588 + bam_reset_channel(bchan); 590 589 591 590 dma_free_wc(bdev->dev, BAM_DESC_FIFO_SIZE, bchan->fifo_virt, 592 591 bchan->fifo_phys); ··· 623 624 struct dma_slave_config *cfg) 624 625 { 625 626 struct bam_chan *bchan = to_bam_chan(chan); 626 - unsigned long flag; 627 627 628 - spin_lock_irqsave(&bchan->vc.lock, flag); 628 + guard(spinlock_irqsave)(&bchan->vc.lock); 629 + 629 630 memcpy(&bchan->slave, cfg, sizeof(*cfg)); 630 631 bchan->reconfigure = 1; 631 - spin_unlock_irqrestore(&bchan->vc.lock, flag); 632 632 633 633 return 0; 634 634 } ··· 653 655 struct scatterlist *sg; 654 656 u32 i; 655 657 struct bam_desc_hw *desc; 656 - unsigned int num_alloc = 0; 657 - 658 + unsigned int num_alloc; 658 659 659 660 if (!is_slave_direction(direction)) { 660 661 dev_err(bdev->dev, "invalid dma direction\n"); 661 662 return NULL; 662 663 } 663 664 664 - /* calculate number of required entries */ 665 - for_each_sg(sgl, sg, sg_len, i) 666 - num_alloc += DIV_ROUND_UP(sg_dma_len(sg), BAM_FIFO_SIZE); 667 - 668 665 /* allocate enough room to accommodate the number of entries */ 666 + num_alloc = sg_nents_for_dma(sgl, sg_len, BAM_FIFO_SIZE); 669 667 async_desc = kzalloc(struct_size(async_desc, desc, num_alloc), 670 668 GFP_NOWAIT); 671 - 672 669 if (!async_desc) 673 670 return NULL; 674 671 ··· 719 726 { 720 727 struct bam_chan *bchan = to_bam_chan(chan); 721 728 struct bam_async_desc *async_desc, *tmp; 722 - unsigned long flag; 723 729 LIST_HEAD(head); 724 730 725 731 /* remove all transactions, including active transaction */ 726 - spin_lock_irqsave(&bchan->vc.lock, flag); 727 - /* 728 - * If we have transactions queued, then some might be committed to the 729 - * hardware in the desc fifo. The only way to reset the desc fifo is 730 - * to do a hardware reset (either by pipe or the entire block). 731 - * bam_chan_init_hw() will trigger a pipe reset, and also reinit the 732 - * pipe. If the pipe is left disabled (default state after pipe reset) 733 - * and is accessed by a connected hardware engine, a fatal error in 734 - * the BAM will occur. There is a small window where this could happen 735 - * with bam_chan_init_hw(), but it is assumed that the caller has 736 - * stopped activity on any attached hardware engine. Make sure to do 737 - * this first so that the BAM hardware doesn't cause memory corruption 738 - * by accessing freed resources. 739 - */ 740 - if (!list_empty(&bchan->desc_list)) { 741 - async_desc = list_first_entry(&bchan->desc_list, 742 - struct bam_async_desc, desc_node); 743 - bam_chan_init_hw(bchan, async_desc->dir); 744 - } 732 + scoped_guard(spinlock_irqsave, &bchan->vc.lock) { 733 + /* 734 + * If we have transactions queued, then some might be committed to the 735 + * hardware in the desc fifo. The only way to reset the desc fifo is 736 + * to do a hardware reset (either by pipe or the entire block). 737 + * bam_chan_init_hw() will trigger a pipe reset, and also reinit the 738 + * pipe. If the pipe is left disabled (default state after pipe reset) 739 + * and is accessed by a connected hardware engine, a fatal error in 740 + * the BAM will occur. There is a small window where this could happen 741 + * with bam_chan_init_hw(), but it is assumed that the caller has 742 + * stopped activity on any attached hardware engine. Make sure to do 743 + * this first so that the BAM hardware doesn't cause memory corruption 744 + * by accessing freed resources. 745 + */ 746 + if (!list_empty(&bchan->desc_list)) { 747 + async_desc = list_first_entry(&bchan->desc_list, 748 + struct bam_async_desc, desc_node); 749 + bam_chan_init_hw(bchan, async_desc->dir); 750 + } 745 751 746 - list_for_each_entry_safe(async_desc, tmp, 747 - &bchan->desc_list, desc_node) { 748 - list_add(&async_desc->vd.node, &bchan->vc.desc_issued); 749 - list_del(&async_desc->desc_node); 750 - } 752 + list_for_each_entry_safe(async_desc, tmp, 753 + &bchan->desc_list, desc_node) { 754 + list_add(&async_desc->vd.node, &bchan->vc.desc_issued); 755 + list_del(&async_desc->desc_node); 756 + } 751 757 752 - vchan_get_all_descriptors(&bchan->vc, &head); 753 - spin_unlock_irqrestore(&bchan->vc.lock, flag); 758 + vchan_get_all_descriptors(&bchan->vc, &head); 759 + } 754 760 755 761 vchan_dma_desc_free_list(&bchan->vc, &head); 756 762 ··· 765 773 { 766 774 struct bam_chan *bchan = to_bam_chan(chan); 767 775 struct bam_device *bdev = bchan->bdev; 768 - unsigned long flag; 769 776 int ret; 770 777 771 778 ret = pm_runtime_get_sync(bdev->dev); 772 779 if (ret < 0) 773 780 return ret; 774 781 775 - spin_lock_irqsave(&bchan->vc.lock, flag); 776 - writel_relaxed(1, bam_addr(bdev, bchan->id, BAM_P_HALT)); 777 - bchan->paused = 1; 778 - spin_unlock_irqrestore(&bchan->vc.lock, flag); 782 + scoped_guard(spinlock_irqsave, &bchan->vc.lock) { 783 + writel_relaxed(1, bam_addr(bdev, bchan->id, BAM_P_HALT)); 784 + bchan->paused = 1; 785 + } 779 786 pm_runtime_mark_last_busy(bdev->dev); 780 787 pm_runtime_put_autosuspend(bdev->dev); 781 788 ··· 790 799 { 791 800 struct bam_chan *bchan = to_bam_chan(chan); 792 801 struct bam_device *bdev = bchan->bdev; 793 - unsigned long flag; 794 802 int ret; 795 803 796 804 ret = pm_runtime_get_sync(bdev->dev); 797 805 if (ret < 0) 798 806 return ret; 799 807 800 - spin_lock_irqsave(&bchan->vc.lock, flag); 801 - writel_relaxed(0, bam_addr(bdev, bchan->id, BAM_P_HALT)); 802 - bchan->paused = 0; 803 - spin_unlock_irqrestore(&bchan->vc.lock, flag); 808 + scoped_guard(spinlock_irqsave, &bchan->vc.lock) { 809 + writel_relaxed(0, bam_addr(bdev, bchan->id, BAM_P_HALT)); 810 + bchan->paused = 0; 811 + } 804 812 pm_runtime_mark_last_busy(bdev->dev); 805 813 pm_runtime_put_autosuspend(bdev->dev); 806 814 ··· 816 826 static u32 process_channel_irqs(struct bam_device *bdev) 817 827 { 818 828 u32 i, srcs, pipe_stts, offset, avail; 819 - unsigned long flags; 820 829 struct bam_async_desc *async_desc, *tmp; 821 830 822 831 srcs = readl_relaxed(bam_addr(bdev, 0, BAM_IRQ_SRCS_EE)); ··· 835 846 836 847 writel_relaxed(pipe_stts, bam_addr(bdev, i, BAM_P_IRQ_CLR)); 837 848 838 - spin_lock_irqsave(&bchan->vc.lock, flags); 849 + guard(spinlock_irqsave)(&bchan->vc.lock); 839 850 840 851 offset = readl_relaxed(bam_addr(bdev, i, BAM_P_SW_OFSTS)) & 841 852 P_SW_OFSTS_MASK; ··· 874 885 } 875 886 list_del(&async_desc->desc_node); 876 887 } 877 - 878 - spin_unlock_irqrestore(&bchan->vc.lock, flags); 879 888 } 880 889 881 890 return srcs; ··· 937 950 int ret; 938 951 size_t residue = 0; 939 952 unsigned int i; 940 - unsigned long flags; 941 953 942 954 ret = dma_cookie_status(chan, cookie, txstate); 943 955 if (ret == DMA_COMPLETE) ··· 945 959 if (!txstate) 946 960 return bchan->paused ? DMA_PAUSED : ret; 947 961 948 - spin_lock_irqsave(&bchan->vc.lock, flags); 949 - vd = vchan_find_desc(&bchan->vc, cookie); 950 - if (vd) { 951 - residue = container_of(vd, struct bam_async_desc, vd)->length; 952 - } else { 953 - list_for_each_entry(async_desc, &bchan->desc_list, desc_node) { 954 - if (async_desc->vd.tx.cookie != cookie) 955 - continue; 962 + scoped_guard(spinlock_irqsave, &bchan->vc.lock) { 963 + vd = vchan_find_desc(&bchan->vc, cookie); 964 + if (vd) { 965 + residue = container_of(vd, struct bam_async_desc, vd)->length; 966 + } else { 967 + list_for_each_entry(async_desc, &bchan->desc_list, desc_node) { 968 + if (async_desc->vd.tx.cookie != cookie) 969 + continue; 956 970 957 - for (i = 0; i < async_desc->num_desc; i++) 958 - residue += le16_to_cpu( 959 - async_desc->curr_desc[i].size); 971 + for (i = 0; i < async_desc->num_desc; i++) 972 + residue += le16_to_cpu( 973 + async_desc->curr_desc[i].size); 974 + } 960 975 } 961 976 } 962 - 963 - spin_unlock_irqrestore(&bchan->vc.lock, flags); 964 977 965 978 dma_set_residue(txstate, residue); 966 979 ··· 1101 1116 { 1102 1117 struct bam_device *bdev = from_tasklet(bdev, t, task); 1103 1118 struct bam_chan *bchan; 1104 - unsigned long flags; 1105 1119 unsigned int i; 1106 1120 1107 1121 /* go through the channels and kick off transactions */ 1108 1122 for (i = 0; i < bdev->num_channels; i++) { 1109 1123 bchan = &bdev->channels[i]; 1110 - spin_lock_irqsave(&bchan->vc.lock, flags); 1124 + 1125 + guard(spinlock_irqsave)(&bchan->vc.lock); 1111 1126 1112 1127 if (!list_empty(&bchan->vc.desc_issued) && !IS_BUSY(bchan)) 1113 1128 bam_start_dma(bchan); 1114 - spin_unlock_irqrestore(&bchan->vc.lock, flags); 1115 1129 } 1116 1130 1117 1131 } ··· 1124 1140 static void bam_issue_pending(struct dma_chan *chan) 1125 1141 { 1126 1142 struct bam_chan *bchan = to_bam_chan(chan); 1127 - unsigned long flags; 1128 1143 1129 - spin_lock_irqsave(&bchan->vc.lock, flags); 1144 + guard(spinlock_irqsave)(&bchan->vc.lock); 1130 1145 1131 1146 /* if work pending and idle, start a transaction */ 1132 1147 if (vchan_issue_pending(&bchan->vc) && !IS_BUSY(bchan)) 1133 1148 bam_start_dma(bchan); 1134 - 1135 - spin_unlock_irqrestore(&bchan->vc.lock, flags); 1136 1149 } 1137 1150 1138 1151 /**

+4 -5

drivers/dma/qcom/qcom_adm.c

··· 390 390 } 391 391 392 392 /* iterate through sgs and compute allocation size of structures */ 393 - for_each_sg(sgl, sg, sg_len, i) { 394 - if (achan->slave.device_fc) { 393 + if (achan->slave.device_fc) { 394 + for_each_sg(sgl, sg, sg_len, i) { 395 395 box_count += DIV_ROUND_UP(sg_dma_len(sg) / burst, 396 396 ADM_MAX_ROWS); 397 397 if (sg_dma_len(sg) % burst) 398 398 single_count++; 399 - } else { 400 - single_count += DIV_ROUND_UP(sg_dma_len(sg), 401 - ADM_MAX_XFER); 402 399 } 400 + } else { 401 + single_count = sg_nents_for_dma(sgl, sg_len, ADM_MAX_XFER); 403 402 } 404 403 405 404 async_desc = kzalloc(sizeof(*async_desc), GFP_NOWAIT);

+2 -4

drivers/dma/sa11x0-dma.c

··· 526 526 struct sa11x0_dma_chan *c = to_sa11x0_dma_chan(chan); 527 527 struct sa11x0_dma_desc *txd; 528 528 struct scatterlist *sgent; 529 - unsigned i, j = sglen; 529 + unsigned int i, j; 530 530 size_t size = 0; 531 531 532 532 /* SA11x0 channels can only operate in their native direction */ ··· 542 542 543 543 for_each_sg(sg, sgent, sglen, i) { 544 544 dma_addr_t addr = sg_dma_address(sgent); 545 - unsigned int len = sg_dma_len(sgent); 546 545 547 - if (len > DMA_MAX_SIZE) 548 - j += DIV_ROUND_UP(len, DMA_MAX_SIZE & ~DMA_ALIGN) - 1; 549 546 if (addr & DMA_ALIGN) { 550 547 dev_dbg(chan->device->dev, "vchan %p: bad buffer alignment: %pad\n", 551 548 &c->vc, &addr); ··· 550 553 } 551 554 } 552 555 556 + j = sg_nents_for_dma(sg, sglen, DMA_MAX_SIZE & ~DMA_ALIGN); 553 557 txd = kzalloc(struct_size(txd, sg, j), GFP_ATOMIC); 554 558 if (!txd) { 555 559 dev_dbg(chan->device->dev, "vchan %p: kzalloc failed\n", &c->vc);

+6 -9

drivers/dma/sh/rz-dmac.c

··· 65 65 void __iomem *ch_base; 66 66 void __iomem *ch_cmn_base; 67 67 unsigned int index; 68 - int irq; 69 68 struct rz_dmac_desc *desc; 70 69 int descs_allocated; 71 70 ··· 799 800 struct rz_lmdesc *lmdesc; 800 801 char pdev_irqname[6]; 801 802 char *irqname; 802 - int ret; 803 + int irq, ret; 803 804 804 805 channel->index = index; 805 806 channel->mid_rid = -EINVAL; 806 807 807 808 /* Request the channel interrupt. */ 808 809 scnprintf(pdev_irqname, sizeof(pdev_irqname), "ch%u", index); 809 - channel->irq = platform_get_irq_byname(pdev, pdev_irqname); 810 - if (channel->irq < 0) 811 - return channel->irq; 810 + irq = platform_get_irq_byname(pdev, pdev_irqname); 811 + if (irq < 0) 812 + return irq; 812 813 813 814 irqname = devm_kasprintf(dmac->dev, GFP_KERNEL, "%s:%u", 814 815 dev_name(dmac->dev), index); 815 816 if (!irqname) 816 817 return -ENOMEM; 817 818 818 - ret = devm_request_threaded_irq(dmac->dev, channel->irq, 819 - rz_dmac_irq_handler, 819 + ret = devm_request_threaded_irq(dmac->dev, irq, rz_dmac_irq_handler, 820 820 rz_dmac_irq_handler_thread, 0, 821 821 irqname, channel); 822 822 if (ret) { 823 - dev_err(dmac->dev, "failed to request IRQ %u (%d)\n", 824 - channel->irq, ret); 823 + dev_err(dmac->dev, "failed to request IRQ %u (%d)\n", irq, ret); 825 824 return ret; 826 825 } 827 826

+3 -4

drivers/dma/sh/shdma-base.c

··· 143 143 } 144 144 145 145 schan->pm_state = SHDMA_PM_ESTABLISHED; 146 - ret = pm_runtime_put(schan->dev); 146 + pm_runtime_put(schan->dev); 147 147 148 148 spin_unlock_irq(&schan->chan_lock); 149 149 return ret; ··· 577 577 struct scatterlist *sg; 578 578 struct shdma_desc *first = NULL, *new = NULL /* compiler... */; 579 579 LIST_HEAD(tx_list); 580 - int chunks = 0; 580 + int chunks; 581 581 unsigned long irq_flags; 582 582 int i; 583 583 584 - for_each_sg(sgl, sg, sg_len, i) 585 - chunks += DIV_ROUND_UP(sg_dma_len(sg), schan->max_xfer_len); 584 + chunks = sg_nents_for_dma(sgl, sg_len, schan->max_xfer_len); 586 585 587 586 /* Have to lock the whole loop to protect against concurrent release */ 588 587 spin_lock_irqsave(&schan->chan_lock, irq_flags);

+1 -1

drivers/dma/st_fdma.c

··· 68 68 { 69 69 struct st_fdma_dev *fdev = fchan->fdev; 70 70 71 - dev_dbg(fdev->dev, "put dreq_line:%#x\n", fchan->dreq_line); 71 + dev_dbg(fdev->dev, "put dreq_line:%#lx\n", fchan->dreq_line); 72 72 clear_bit(fchan->dreq_line, &fdev->dreq_mask); 73 73 } 74 74

+1 -1

drivers/dma/st_fdma.h

··· 120 120 struct dma_slave_config scfg; 121 121 struct st_fdma_cfg cfg; 122 122 123 - int dreq_line; 123 + long dreq_line; 124 124 125 125 struct virt_dma_chan vchan; 126 126 struct st_fdma_desc *fdesc;

+2 -12

drivers/dma/ste_dma40.c

··· 1452 1452 1453 1453 res = d40_channel_execute_command(d40c, D40_DMA_SUSPEND_REQ); 1454 1454 1455 - pm_runtime_mark_last_busy(d40c->base->dev); 1456 1455 pm_runtime_put_autosuspend(d40c->base->dev); 1457 1456 spin_unlock_irqrestore(&d40c->lock, flags); 1458 1457 return res; ··· 1478 1479 if (d40_residue(d40c) || d40_tx_is_linked(d40c)) 1479 1480 res = d40_channel_execute_command(d40c, D40_DMA_RUN); 1480 1481 1481 - pm_runtime_mark_last_busy(d40c->base->dev); 1482 1482 pm_runtime_put_autosuspend(d40c->base->dev); 1483 1483 spin_unlock_irqrestore(&d40c->lock, flags); 1484 1484 return res; ··· 1579 1581 if (d40_queue_start(d40c) == NULL) { 1580 1582 d40c->busy = false; 1581 1583 1582 - pm_runtime_mark_last_busy(d40c->base->dev); 1583 1584 pm_runtime_put_autosuspend(d40c->base->dev); 1584 1585 } 1585 1586 ··· 2051 2054 else 2052 2055 d40c->base->lookup_phy_chans[phy->num] = NULL; 2053 2056 2054 - if (d40c->busy) { 2055 - pm_runtime_mark_last_busy(d40c->base->dev); 2057 + if (d40c->busy) 2056 2058 pm_runtime_put_autosuspend(d40c->base->dev); 2057 - } 2058 2059 2059 2060 d40c->busy = false; 2060 2061 d40c->phy_chan = NULL; 2061 2062 d40c->configured = false; 2062 2063 mark_last_busy: 2063 - pm_runtime_mark_last_busy(d40c->base->dev); 2064 2064 pm_runtime_put_autosuspend(d40c->base->dev); 2065 2065 return res; 2066 2066 } ··· 2460 2466 if (is_free_phy) 2461 2467 d40_config_write(d40c); 2462 2468 mark_last_busy: 2463 - pm_runtime_mark_last_busy(d40c->base->dev); 2464 2469 pm_runtime_put_autosuspend(d40c->base->dev); 2465 2470 spin_unlock_irqrestore(&d40c->lock, flags); 2466 2471 return err; ··· 2611 2618 chan_err(d40c, "Failed to stop channel\n"); 2612 2619 2613 2620 d40_term_all(d40c); 2614 - pm_runtime_mark_last_busy(d40c->base->dev); 2615 2621 pm_runtime_put_autosuspend(d40c->base->dev); 2616 - if (d40c->busy) { 2617 - pm_runtime_mark_last_busy(d40c->base->dev); 2622 + if (d40c->busy) 2618 2623 pm_runtime_put_autosuspend(d40c->base->dev); 2619 - } 2620 2624 d40c->busy = false; 2621 2625 2622 2626 spin_unlock_irqrestore(&d40c->lock, flags);

+136 -29

drivers/dma/stm32/stm32-dma3.c

··· 288 288 u32 fifo_size; 289 289 u32 max_burst; 290 290 bool semaphore_mode; 291 + bool semaphore_taken; 291 292 struct stm32_dma3_dt_conf dt_config; 292 293 struct dma_slave_config dma_config; 293 294 u8 config_set; ··· 331 330 static struct device *chan2dev(struct stm32_dma3_chan *chan) 332 331 { 333 332 return &chan->vchan.chan.dev->device; 333 + } 334 + 335 + static struct device *ddata2dev(struct stm32_dma3_ddata *ddata) 336 + { 337 + return ddata->dma_dev.dev; 334 338 } 335 339 336 340 static void stm32_dma3_chan_dump_reg(struct stm32_dma3_chan *chan) ··· 1069 1063 return IRQ_HANDLED; 1070 1064 } 1071 1065 1066 + static int stm32_dma3_get_chan_sem(struct stm32_dma3_chan *chan) 1067 + { 1068 + struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan); 1069 + u32 csemcr, ccid; 1070 + 1071 + csemcr = readl_relaxed(ddata->base + STM32_DMA3_CSEMCR(chan->id)); 1072 + /* Make an attempt to take the channel semaphore if not already taken */ 1073 + if (!(csemcr & CSEMCR_SEM_MUTEX)) { 1074 + writel_relaxed(CSEMCR_SEM_MUTEX, ddata->base + STM32_DMA3_CSEMCR(chan->id)); 1075 + csemcr = readl_relaxed(ddata->base + STM32_DMA3_CSEMCR(chan->id)); 1076 + } 1077 + 1078 + /* Check if channel is under CID1 control */ 1079 + ccid = FIELD_GET(CSEMCR_SEM_CCID, csemcr); 1080 + if (!(csemcr & CSEMCR_SEM_MUTEX) || ccid != CCIDCFGR_CID1) 1081 + goto bad_cid; 1082 + 1083 + chan->semaphore_taken = true; 1084 + dev_dbg(chan2dev(chan), "under CID1 control (semcr=0x%08x)\n", csemcr); 1085 + 1086 + return 0; 1087 + 1088 + bad_cid: 1089 + chan->semaphore_taken = false; 1090 + dev_err(chan2dev(chan), "not under CID1 control (in-use by CID%d)\n", ccid); 1091 + 1092 + return -EACCES; 1093 + } 1094 + 1095 + static void stm32_dma3_put_chan_sem(struct stm32_dma3_chan *chan) 1096 + { 1097 + struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan); 1098 + 1099 + if (chan->semaphore_taken) { 1100 + writel_relaxed(0, ddata->base + STM32_DMA3_CSEMCR(chan->id)); 1101 + chan->semaphore_taken = false; 1102 + dev_dbg(chan2dev(chan), "no more under CID1 control\n"); 1103 + } 1104 + } 1105 + 1072 1106 static int stm32_dma3_alloc_chan_resources(struct dma_chan *c) 1073 1107 { 1074 1108 struct stm32_dma3_chan *chan = to_stm32_dma3_chan(c); 1075 1109 struct stm32_dma3_ddata *ddata = to_stm32_dma3_ddata(chan); 1076 - u32 id = chan->id, csemcr, ccid; 1077 1110 int ret; 1078 1111 1079 - ret = pm_runtime_resume_and_get(ddata->dma_dev.dev); 1112 + ret = pm_runtime_resume_and_get(ddata2dev(ddata)); 1080 1113 if (ret < 0) 1081 1114 return ret; 1082 1115 ··· 1137 1092 1138 1093 /* Take the channel semaphore */ 1139 1094 if (chan->semaphore_mode) { 1140 - writel_relaxed(CSEMCR_SEM_MUTEX, ddata->base + STM32_DMA3_CSEMCR(id)); 1141 - csemcr = readl_relaxed(ddata->base + STM32_DMA3_CSEMCR(id)); 1142 - ccid = FIELD_GET(CSEMCR_SEM_CCID, csemcr); 1143 - /* Check that the channel is well taken */ 1144 - if (ccid != CCIDCFGR_CID1) { 1145 - dev_err(chan2dev(chan), "Not under CID1 control (in-use by CID%d)\n", ccid); 1146 - ret = -EPERM; 1095 + ret = stm32_dma3_get_chan_sem(chan); 1096 + if (ret) 1147 1097 goto err_pool_destroy; 1148 - } 1149 - dev_dbg(chan2dev(chan), "Under CID1 control (semcr=0x%08x)\n", csemcr); 1150 1098 } 1151 1099 1152 1100 return 0; ··· 1149 1111 chan->lli_pool = NULL; 1150 1112 1151 1113 err_put_sync: 1152 - pm_runtime_put_sync(ddata->dma_dev.dev); 1114 + pm_runtime_put_sync(ddata2dev(ddata)); 1153 1115 1154 1116 return ret; 1155 1117 } ··· 1173 1135 1174 1136 /* Release the channel semaphore */ 1175 1137 if (chan->semaphore_mode) 1176 - writel_relaxed(0, ddata->base + STM32_DMA3_CSEMCR(chan->id)); 1138 + stm32_dma3_put_chan_sem(chan); 1177 1139 1178 - pm_runtime_put_sync(ddata->dma_dev.dev); 1140 + pm_runtime_put_sync(ddata2dev(ddata)); 1179 1141 1180 1142 /* Reset configuration */ 1181 1143 memset(&chan->dt_config, 0, sizeof(chan->dt_config)); ··· 1242 1204 bool prevent_refactor = !!FIELD_GET(STM32_DMA3_DT_NOPACK, chan->dt_config.tr_conf) || 1243 1205 !!FIELD_GET(STM32_DMA3_DT_NOREFACT, chan->dt_config.tr_conf); 1244 1206 1207 + /* Semaphore could be lost during suspend/resume */ 1208 + if (chan->semaphore_mode && !chan->semaphore_taken) 1209 + return NULL; 1210 + 1245 1211 count = stm32_dma3_get_ll_count(chan, len, prevent_refactor); 1246 1212 1247 1213 swdesc = stm32_dma3_chan_desc_alloc(chan, count); ··· 1305 1263 bool prevent_refactor = !!FIELD_GET(STM32_DMA3_DT_NOPACK, chan->dt_config.tr_conf) || 1306 1264 !!FIELD_GET(STM32_DMA3_DT_NOREFACT, chan->dt_config.tr_conf); 1307 1265 int ret; 1266 + 1267 + /* Semaphore could be lost during suspend/resume */ 1268 + if (chan->semaphore_mode && !chan->semaphore_taken) 1269 + return NULL; 1308 1270 1309 1271 count = 0; 1310 1272 for_each_sg(sgl, sg, sg_len, i) ··· 1395 1349 dma_addr_t src, dst; 1396 1350 u32 count, i, ctr1, ctr2; 1397 1351 int ret; 1352 + 1353 + /* Semaphore could be lost during suspend/resume */ 1354 + if (chan->semaphore_mode && !chan->semaphore_taken) 1355 + return NULL; 1398 1356 1399 1357 if (!buf_len || !period_len || period_len > STM32_DMA3_MAX_BLOCK_SIZE) { 1400 1358 dev_err(chan2dev(chan), "Invalid buffer/period length\n"); ··· 1615 1565 if (!(mask & BIT(chan->id))) 1616 1566 return false; 1617 1567 1618 - ret = pm_runtime_resume_and_get(ddata->dma_dev.dev); 1568 + ret = pm_runtime_resume_and_get(ddata2dev(ddata)); 1619 1569 if (ret < 0) 1620 1570 return false; 1621 1571 semcr = readl_relaxed(ddata->base + STM32_DMA3_CSEMCR(chan->id)); 1622 - pm_runtime_put_sync(ddata->dma_dev.dev); 1572 + pm_runtime_put_sync(ddata2dev(ddata)); 1623 1573 1624 1574 /* Check if chan is free */ 1625 1575 if (semcr & CSEMCR_SEM_MUTEX) ··· 1641 1591 struct dma_chan *c; 1642 1592 1643 1593 if (dma_spec->args_count < 3) { 1644 - dev_err(ddata->dma_dev.dev, "Invalid args count\n"); 1594 + dev_err(ddata2dev(ddata), "Invalid args count\n"); 1645 1595 return NULL; 1646 1596 } 1647 1597 ··· 1650 1600 conf.tr_conf = dma_spec->args[2]; 1651 1601 1652 1602 if (conf.req_line >= ddata->dma_requests) { 1653 - dev_err(ddata->dma_dev.dev, "Invalid request line\n"); 1603 + dev_err(ddata2dev(ddata), "Invalid request line\n"); 1654 1604 return NULL; 1655 1605 } 1656 1606 1657 1607 /* Request dma channel among the generic dma controller list */ 1658 1608 c = dma_request_channel(mask, stm32_dma3_filter_fn, &conf); 1659 1609 if (!c) { 1660 - dev_err(ddata->dma_dev.dev, "No suitable channel found\n"); 1610 + dev_err(ddata2dev(ddata), "No suitable channel found\n"); 1661 1611 return NULL; 1662 1612 } 1663 1613 ··· 1670 1620 1671 1621 static u32 stm32_dma3_check_rif(struct stm32_dma3_ddata *ddata) 1672 1622 { 1623 + struct device *dev = ddata2dev(ddata); 1673 1624 u32 chan_reserved, mask = 0, i, ccidcfgr, invalid_cid = 0; 1674 1625 1675 1626 /* Reserve Secure channels */ ··· 1682 1631 * In case CID filtering is not configured, dma-channel-mask property can be used to 1683 1632 * specify available DMA channels to the kernel. 1684 1633 */ 1685 - of_property_read_u32(ddata->dma_dev.dev->of_node, "dma-channel-mask", &mask); 1634 + of_property_read_u32(dev->of_node, "dma-channel-mask", &mask); 1686 1635 1687 1636 /* Reserve !CID-filtered not in dma-channel-mask, static CID != CID1, CID1 not allowed */ 1688 1637 for (i = 0; i < ddata->dma_channels; i++) { ··· 1702 1651 ddata->chans[i].semaphore_mode = true; 1703 1652 } 1704 1653 } 1705 - dev_dbg(ddata->dma_dev.dev, "chan%d: %s mode, %s\n", i, 1654 + dev_dbg(dev, "chan%d: %s mode, %s\n", i, 1706 1655 !(ccidcfgr & CCIDCFGR_CFEN) ? "!CID-filtered" : 1707 1656 ddata->chans[i].semaphore_mode ? "Semaphore" : "Static CID", 1708 1657 (chan_reserved & BIT(i)) ? "denied" : ··· 1710 1659 } 1711 1660 1712 1661 if (invalid_cid) 1713 - dev_warn(ddata->dma_dev.dev, "chan%*pbl have invalid CID configuration\n", 1662 + dev_warn(dev, "chan%*pbl have invalid CID configuration\n", 1714 1663 ddata->dma_channels, &invalid_cid); 1715 1664 1716 1665 return chan_reserved; ··· 1950 1899 return ret; 1951 1900 } 1952 1901 1902 + static int stm32_dma3_pm_suspend(struct device *dev) 1903 + { 1904 + struct stm32_dma3_ddata *ddata = dev_get_drvdata(dev); 1905 + struct dma_device *dma_dev = &ddata->dma_dev; 1906 + struct dma_chan *c; 1907 + int ccr, ret; 1908 + 1909 + ret = pm_runtime_resume_and_get(dev); 1910 + if (ret < 0) 1911 + return ret; 1912 + 1913 + list_for_each_entry(c, &dma_dev->channels, device_node) { 1914 + struct stm32_dma3_chan *chan = to_stm32_dma3_chan(c); 1915 + 1916 + ccr = readl_relaxed(ddata->base + STM32_DMA3_CCR(chan->id)); 1917 + if (ccr & CCR_EN) { 1918 + dev_warn(dev, "Suspend is prevented: %s still in use by %s\n", 1919 + dma_chan_name(c), dev_name(c->slave)); 1920 + pm_runtime_put_sync(dev); 1921 + return -EBUSY; 1922 + } 1923 + } 1924 + 1925 + pm_runtime_put_sync(dev); 1926 + 1927 + pm_runtime_force_suspend(dev); 1928 + 1929 + return 0; 1930 + } 1931 + 1932 + static int stm32_dma3_pm_resume(struct device *dev) 1933 + { 1934 + struct stm32_dma3_ddata *ddata = dev_get_drvdata(dev); 1935 + struct dma_device *dma_dev = &ddata->dma_dev; 1936 + struct dma_chan *c; 1937 + int ret; 1938 + 1939 + ret = pm_runtime_force_resume(dev); 1940 + if (ret < 0) 1941 + return ret; 1942 + 1943 + ret = pm_runtime_resume_and_get(dev); 1944 + if (ret < 0) 1945 + return ret; 1946 + 1947 + /* 1948 + * Channel semaphores need to be restored in case of registers reset during low power. 1949 + * stm32_dma3_get_chan_sem() will prior check the semaphore status. 1950 + */ 1951 + list_for_each_entry(c, &dma_dev->channels, device_node) { 1952 + struct stm32_dma3_chan *chan = to_stm32_dma3_chan(c); 1953 + 1954 + if (chan->semaphore_mode && chan->semaphore_taken) 1955 + stm32_dma3_get_chan_sem(chan); 1956 + } 1957 + 1958 + pm_runtime_put_sync(dev); 1959 + 1960 + return 0; 1961 + } 1962 + 1953 1963 static const struct dev_pm_ops stm32_dma3_pm_ops = { 1954 - SYSTEM_SLEEP_PM_OPS(pm_runtime_force_suspend, pm_runtime_force_resume) 1964 + SYSTEM_SLEEP_PM_OPS(stm32_dma3_pm_suspend, stm32_dma3_pm_resume) 1955 1965 RUNTIME_PM_OPS(stm32_dma3_runtime_suspend, stm32_dma3_runtime_resume, NULL) 1956 1966 }; 1957 1967 ··· 2026 1914 }, 2027 1915 }; 2028 1916 2029 - static int __init stm32_dma3_init(void) 2030 - { 2031 - return platform_driver_register(&stm32_dma3_driver); 2032 - } 2033 - 2034 - subsys_initcall(stm32_dma3_init); 1917 + module_platform_driver(stm32_dma3_driver); 2035 1918 2036 1919 MODULE_DESCRIPTION("STM32 DMA3 controller driver"); 2037 1920 MODULE_AUTHOR("Amelie Delaunay <amelie.delaunay@foss.st.com>");

+1 -1

drivers/dma/stm32/stm32-mdma.c

··· 731 731 struct stm32_mdma_chan_config *chan_config = &chan->chan_config; 732 732 struct scatterlist *sg; 733 733 dma_addr_t src_addr, dst_addr; 734 - u32 m2m_hw_period, ccr, ctcr, ctbr; 734 + u32 m2m_hw_period = 0, ccr = 0, ctcr, ctbr; 735 735 int i, ret = 0; 736 736 737 737 if (chan_config->m2m_hw)

+30 -6

drivers/dma/sun6i-dma.c

··· 583 583 return ret; 584 584 } 585 585 586 + static u32 find_burst_size(const u32 burst_lengths, u32 maxburst) 587 + { 588 + if (!maxburst) 589 + return 1; 590 + 591 + if (BIT(maxburst) & burst_lengths) 592 + return maxburst; 593 + 594 + /* Hardware only does power-of-two bursts. */ 595 + for (u32 burst = rounddown_pow_of_two(maxburst); burst > 0; burst /= 2) 596 + if (BIT(burst) & burst_lengths) 597 + return burst; 598 + 599 + return 1; 600 + } 601 + 586 602 static int set_config(struct sun6i_dma_dev *sdev, 587 603 struct dma_slave_config *sconfig, 588 604 enum dma_transfer_direction direction, ··· 632 616 return -EINVAL; 633 617 if (!(BIT(dst_addr_width) & sdev->slave.dst_addr_widths)) 634 618 return -EINVAL; 635 - if (!(BIT(src_maxburst) & sdev->cfg->src_burst_lengths)) 636 - return -EINVAL; 637 - if (!(BIT(dst_maxburst) & sdev->cfg->dst_burst_lengths)) 638 - return -EINVAL; 639 619 640 620 src_width = convert_buswidth(src_addr_width); 641 621 dst_width = convert_buswidth(dst_addr_width); 642 - dst_burst = convert_burst(dst_maxburst); 643 - src_burst = convert_burst(src_maxburst); 622 + src_burst = find_burst_size(sdev->cfg->src_burst_lengths, src_maxburst); 623 + dst_burst = find_burst_size(sdev->cfg->dst_burst_lengths, dst_maxburst); 624 + dst_burst = convert_burst(dst_burst); 625 + src_burst = convert_burst(src_burst); 644 626 645 627 *p_cfg = DMA_CHAN_CFG_SRC_WIDTH(src_width) | 646 628 DMA_CHAN_CFG_DST_WIDTH(dst_width); ··· 840 826 v_lli->cfg = lli_cfg; 841 827 sdev->cfg->set_drq(&v_lli->cfg, DRQ_SDRAM, vchan->port); 842 828 sdev->cfg->set_mode(&v_lli->cfg, LINEAR_MODE, IO_MODE); 829 + dev_dbg(chan2dev(chan), 830 + "%s; chan: %d, dest: %pad, src: %pad, len: %zu. flags: 0x%08lx\n", 831 + __func__, vchan->vc.chan.chan_id, 832 + &sconfig->dst_addr, &buf_addr, 833 + buf_len, flags); 843 834 } else { 844 835 sun6i_dma_set_addr(sdev, v_lli, 845 836 sconfig->src_addr, ··· 852 833 v_lli->cfg = lli_cfg; 853 834 sdev->cfg->set_drq(&v_lli->cfg, vchan->port, DRQ_SDRAM); 854 835 sdev->cfg->set_mode(&v_lli->cfg, IO_MODE, LINEAR_MODE); 836 + dev_dbg(chan2dev(chan), 837 + "%s; chan: %d, dest: %pad, src: %pad, len: %zu. flags: 0x%08lx\n", 838 + __func__, vchan->vc.chan.chan_id, 839 + &buf_addr, &sconfig->src_addr, 840 + buf_len, flags); 855 841 } 856 842 857 843 prev = sun6i_dma_lli_add(prev, v_lli, p_lli, txd);

+3 -2

drivers/dma/ti/Kconfig

··· 36 36 37 37 config TI_K3_UDMA 38 38 tristate "Texas Instruments UDMA support" 39 - depends on ARCH_K3 39 + depends on ARCH_K3 || COMPILE_TEST 40 40 depends on TI_SCI_PROTOCOL 41 41 depends on TI_SCI_INTA_IRQCHIP 42 42 select DMA_ENGINE 43 43 select DMA_VIRTUAL_CHANNELS 44 + select SOC_TI 44 45 select TI_K3_RINGACC 45 46 select TI_K3_PSIL 46 47 help ··· 50 49 51 50 config TI_K3_UDMA_GLUE_LAYER 52 51 tristate "Texas Instruments UDMA Glue layer for non DMAengine users" 53 - depends on ARCH_K3 52 + depends on ARCH_K3 || COMPILE_TEST 54 53 depends on TI_K3_UDMA 55 54 help 56 55 Say y here to support the K3 NAVSS DMA glue interface

-5

drivers/dma/ti/cppi41.c

··· 390 390 if (!c->is_tx) 391 391 cppi_writel(c->q_num, c->gcr_reg + RXHPCRA0); 392 392 393 - pm_runtime_mark_last_busy(cdd->ddev.dev); 394 393 pm_runtime_put_autosuspend(cdd->ddev.dev); 395 394 396 395 return 0; ··· 410 411 411 412 WARN_ON(!list_empty(&cdd->pending)); 412 413 413 - pm_runtime_mark_last_busy(cdd->ddev.dev); 414 414 pm_runtime_put_autosuspend(cdd->ddev.dev); 415 415 } 416 416 ··· 507 509 cppi41_run_queue(cdd); 508 510 spin_unlock_irqrestore(&cdd->lock, flags); 509 511 510 - pm_runtime_mark_last_busy(cdd->ddev.dev); 511 512 pm_runtime_put_autosuspend(cdd->ddev.dev); 512 513 } 513 514 ··· 624 627 txd = &c->txd; 625 628 626 629 err_out_not_ready: 627 - pm_runtime_mark_last_busy(cdd->ddev.dev); 628 630 pm_runtime_put_autosuspend(cdd->ddev.dev); 629 631 630 632 return txd; ··· 1135 1139 if (ret) 1136 1140 goto err_of; 1137 1141 1138 - pm_runtime_mark_last_busy(dev); 1139 1142 pm_runtime_put_autosuspend(dev); 1140 1143 1141 1144 return 0;

+2 -4

drivers/dma/xilinx/xdma.c

··· 605 605 struct xdma_chan *xdma_chan = to_xdma_chan(chan); 606 606 struct dma_async_tx_descriptor *tx_desc; 607 607 struct xdma_desc *sw_desc; 608 - u32 desc_num = 0, i; 609 608 u64 addr, dev_addr, *src, *dst; 609 + u32 desc_num, i; 610 610 struct scatterlist *sg; 611 611 612 - for_each_sg(sgl, sg, sg_len, i) 613 - desc_num += DIV_ROUND_UP(sg_dma_len(sg), XDMA_DESC_BLEN_MAX); 614 - 612 + desc_num = sg_nents_for_dma(sgl, sg_len, XDMA_DESC_BLEN_MAX); 615 613 sw_desc = xdma_alloc_desc(xdma_chan, desc_num, false); 616 614 if (!sw_desc) 617 615 return NULL;

+21

drivers/dma/xilinx/xilinx_dma.c

··· 1022 1022 return residue; 1023 1023 } 1024 1024 1025 + static u32 1026 + xilinx_dma_get_residue_axidma_direct_s2mm(struct xilinx_dma_chan *chan, 1027 + struct xilinx_dma_tx_descriptor *desc) 1028 + { 1029 + struct xilinx_axidma_tx_segment *seg; 1030 + struct xilinx_axidma_desc_hw *hw; 1031 + u32 finished_len; 1032 + 1033 + finished_len = dma_ctrl_read(chan, XILINX_DMA_REG_BTT); 1034 + 1035 + seg = list_first_entry(&desc->segments, struct xilinx_axidma_tx_segment, 1036 + node); 1037 + 1038 + hw = &seg->hw; 1039 + 1040 + return hw->control - finished_len; 1041 + } 1042 + 1025 1043 /** 1026 1044 * xilinx_dma_chan_handle_cyclic - Cyclic dma callback 1027 1045 * @chan: Driver specific dma channel ··· 1751 1733 if (chan->has_sg && chan->xdev->dma_config->dmatype != 1752 1734 XDMA_TYPE_VDMA) 1753 1735 desc->residue = xilinx_dma_get_residue(chan, desc); 1736 + else if (chan->xdev->dma_config->dmatype == XDMA_TYPE_AXIDMA && 1737 + chan->direction == DMA_DEV_TO_MEM && !chan->has_sg) 1738 + desc->residue = xilinx_dma_get_residue_axidma_direct_s2mm(chan, desc); 1754 1739 else 1755 1740 desc->residue = 0; 1756 1741 desc->err = chan->err;

-2

drivers/dma/xilinx/zynqmp_dma.c

··· 695 695 (2 * ZYNQMP_DMA_DESC_SIZE(chan) * ZYNQMP_DMA_NUM_DESCS), 696 696 chan->desc_pool_v, chan->desc_pool_p); 697 697 kfree(chan->sw_desc_pool); 698 - pm_runtime_mark_last_busy(chan->dev); 699 698 pm_runtime_put_autosuspend(chan->dev); 700 699 } 701 700 ··· 1144 1145 goto free_chan_resources; 1145 1146 } 1146 1147 1147 - pm_runtime_mark_last_busy(zdev->dev); 1148 1148 pm_runtime_put_sync_autosuspend(zdev->dev); 1149 1149 1150 1150 return 0;

+11 -13

include/linux/dma/edma.h

··· 27 27 }; 28 28 29 29 /** 30 - * struct dw_edma_core_ops - platform-specific eDMA methods 30 + * struct dw_edma_plat_ops - platform-specific eDMA methods 31 31 * @irq_vector: Get IRQ number of the passed eDMA channel. Note the 32 32 * method accepts the channel id in the end-to-end 33 33 * numbering with the eDMA write channels being placed ··· 63 63 /** 64 64 * struct dw_edma_chip - representation of DesignWare eDMA controller hardware 65 65 * @dev: struct device of the eDMA controller 66 - * @id: instance ID 67 66 * @nr_irqs: total number of DMA IRQs 68 - * @ops DMA channel to IRQ number mapping 69 - * @flags dw_edma_chip_flags 70 - * @reg_base DMA register base address 71 - * @ll_wr_cnt DMA write link list count 72 - * @ll_rd_cnt DMA read link list count 73 - * @rg_region DMA register region 74 - * @ll_region_wr DMA descriptor link list memory for write channel 75 - * @ll_region_rd DMA descriptor link list memory for read channel 76 - * @dt_region_wr DMA data memory for write channel 77 - * @dt_region_rd DMA data memory for read channel 78 - * @mf DMA register map format 67 + * @ops: DMA channel to IRQ number mapping 68 + * @flags: dw_edma_chip_flags 69 + * @reg_base: DMA register base address 70 + * @ll_wr_cnt: DMA write link list count 71 + * @ll_rd_cnt: DMA read link list count 72 + * @ll_region_wr: DMA descriptor link list memory for write channel 73 + * @ll_region_rd: DMA descriptor link list memory for read channel 74 + * @dt_region_wr: DMA data memory for write channel 75 + * @dt_region_rd: DMA data memory for read channel 76 + * @mf: DMA register map format 79 77 * @dw: struct dw_edma that is filled by dw_edma_probe() 80 78 */ 81 79 struct dw_edma_chip {

+2

include/linux/scatterlist.h

··· 441 441 442 442 int sg_nents(struct scatterlist *sg); 443 443 int sg_nents_for_len(struct scatterlist *sg, u64 len); 444 + int sg_nents_for_dma(struct scatterlist *sgl, unsigned int sglen, size_t len); 445 + 444 446 struct scatterlist *sg_last(struct scatterlist *s, unsigned int); 445 447 void sg_init_table(struct scatterlist *, unsigned int); 446 448 void sg_init_one(struct scatterlist *, const void *, unsigned int);

+14 -14

include/linux/shdma-base.h

··· 19 19 #include <linux/types.h> 20 20 21 21 /** 22 - * shdma_pm_state - DMA channel PM state 23 - * SHDMA_PM_ESTABLISHED: either idle or during data transfer 24 - * SHDMA_PM_BUSY: during the transfer preparation, when we have to 22 + * enum shdma_pm_state - DMA channel PM state 23 + * @SHDMA_PM_ESTABLISHED: either idle or during data transfer 24 + * @SHDMA_PM_BUSY: during the transfer preparation, when we have to 25 25 * drop the lock temporarily 26 - * SHDMA_PM_PENDING: transfers pending 26 + * @SHDMA_PM_PENDING: transfers pending 27 27 */ 28 28 enum shdma_pm_state { 29 29 SHDMA_PM_ESTABLISHED, ··· 74 74 75 75 /** 76 76 * struct shdma_ops - simple DMA driver operations 77 - * desc_completed: return true, if this is the descriptor, that just has 77 + * @desc_completed: return true, if this is the descriptor, that just has 78 78 * completed (atomic) 79 - * halt_channel: stop DMA channel operation (atomic) 80 - * channel_busy: return true, if the channel is busy (atomic) 81 - * slave_addr: return slave DMA address 82 - * desc_setup: set up the hardware specific descriptor portion (atomic) 83 - * set_slave: bind channel to a slave 84 - * setup_xfer: configure channel hardware for operation (atomic) 85 - * start_xfer: start the DMA transfer (atomic) 86 - * embedded_desc: return Nth struct shdma_desc pointer from the 79 + * @halt_channel: stop DMA channel operation (atomic) 80 + * @channel_busy: return true, if the channel is busy (atomic) 81 + * @slave_addr: return slave DMA address 82 + * @desc_setup: set up the hardware specific descriptor portion (atomic) 83 + * @set_slave: bind channel to a slave 84 + * @setup_xfer: configure channel hardware for operation (atomic) 85 + * @start_xfer: start the DMA transfer (atomic) 86 + * @embedded_desc: return Nth struct shdma_desc pointer from the 87 87 * descriptor array 88 - * chan_irq: process channel IRQ, return true if a transfer has 88 + * @chan_irq: process channel IRQ, return true if a transfer has 89 89 * completed (atomic) 90 90 */ 91 91 struct shdma_ops {

+133 -137

include/uapi/linux/idxd.h

··· 3 3 #ifndef _USR_IDXD_H_ 4 4 #define _USR_IDXD_H_ 5 5 6 - #ifdef __KERNEL__ 7 6 #include <linux/types.h> 8 - #else 9 - #include <stdint.h> 10 - #endif 11 7 12 8 /* Driver command error status */ 13 9 enum idxd_scmd_stat { ··· 172 176 #define DSA_COMP_STATUS(status) ((status) & DSA_COMP_STATUS_MASK) 173 177 174 178 struct dsa_hw_desc { 175 - uint32_t pasid:20; 176 - uint32_t rsvd:11; 177 - uint32_t priv:1; 178 - uint32_t flags:24; 179 - uint32_t opcode:8; 180 - uint64_t completion_addr; 179 + __u32 pasid:20; 180 + __u32 rsvd:11; 181 + __u32 priv:1; 182 + __u32 flags:24; 183 + __u32 opcode:8; 184 + __u64 completion_addr; 181 185 union { 182 - uint64_t src_addr; 183 - uint64_t rdback_addr; 184 - uint64_t pattern; 185 - uint64_t desc_list_addr; 186 - uint64_t pattern_lower; 187 - uint64_t transl_fetch_addr; 186 + __u64 src_addr; 187 + __u64 rdback_addr; 188 + __u64 pattern; 189 + __u64 desc_list_addr; 190 + __u64 pattern_lower; 191 + __u64 transl_fetch_addr; 188 192 }; 189 193 union { 190 - uint64_t dst_addr; 191 - uint64_t rdback_addr2; 192 - uint64_t src2_addr; 193 - uint64_t comp_pattern; 194 + __u64 dst_addr; 195 + __u64 rdback_addr2; 196 + __u64 src2_addr; 197 + __u64 comp_pattern; 194 198 }; 195 199 union { 196 - uint32_t xfer_size; 197 - uint32_t desc_count; 198 - uint32_t region_size; 200 + __u32 xfer_size; 201 + __u32 desc_count; 202 + __u32 region_size; 199 203 }; 200 - uint16_t int_handle; 201 - uint16_t rsvd1; 204 + __u16 int_handle; 205 + __u16 rsvd1; 202 206 union { 203 - uint8_t expected_res; 207 + __u8 expected_res; 204 208 /* create delta record */ 205 209 struct { 206 - uint64_t delta_addr; 207 - uint32_t max_delta_size; 208 - uint32_t delt_rsvd; 209 - uint8_t expected_res_mask; 210 + __u64 delta_addr; 211 + __u32 max_delta_size; 212 + __u32 delt_rsvd; 213 + __u8 expected_res_mask; 210 214 }; 211 - uint32_t delta_rec_size; 212 - uint64_t dest2; 215 + __u32 delta_rec_size; 216 + __u64 dest2; 213 217 /* CRC */ 214 218 struct { 215 - uint32_t crc_seed; 216 - uint32_t crc_rsvd; 217 - uint64_t seed_addr; 219 + __u32 crc_seed; 220 + __u32 crc_rsvd; 221 + __u64 seed_addr; 218 222 }; 219 223 /* DIF check or strip */ 220 224 struct { 221 - uint8_t src_dif_flags; 222 - uint8_t dif_chk_res; 223 - uint8_t dif_chk_flags; 224 - uint8_t dif_chk_res2[5]; 225 - uint32_t chk_ref_tag_seed; 226 - uint16_t chk_app_tag_mask; 227 - uint16_t chk_app_tag_seed; 225 + __u8 src_dif_flags; 226 + __u8 dif_chk_res; 227 + __u8 dif_chk_flags; 228 + __u8 dif_chk_res2[5]; 229 + __u32 chk_ref_tag_seed; 230 + __u16 chk_app_tag_mask; 231 + __u16 chk_app_tag_seed; 228 232 }; 229 233 /* DIF insert */ 230 234 struct { 231 - uint8_t dif_ins_res; 232 - uint8_t dest_dif_flag; 233 - uint8_t dif_ins_flags; 234 - uint8_t dif_ins_res2[13]; 235 - uint32_t ins_ref_tag_seed; 236 - uint16_t ins_app_tag_mask; 237 - uint16_t ins_app_tag_seed; 235 + __u8 dif_ins_res; 236 + __u8 dest_dif_flag; 237 + __u8 dif_ins_flags; 238 + __u8 dif_ins_res2[13]; 239 + __u32 ins_ref_tag_seed; 240 + __u16 ins_app_tag_mask; 241 + __u16 ins_app_tag_seed; 238 242 }; 239 243 /* DIF update */ 240 244 struct { 241 - uint8_t src_upd_flags; 242 - uint8_t upd_dest_flags; 243 - uint8_t dif_upd_flags; 244 - uint8_t dif_upd_res[5]; 245 - uint32_t src_ref_tag_seed; 246 - uint16_t src_app_tag_mask; 247 - uint16_t src_app_tag_seed; 248 - uint32_t dest_ref_tag_seed; 249 - uint16_t dest_app_tag_mask; 250 - uint16_t dest_app_tag_seed; 245 + __u8 src_upd_flags; 246 + __u8 upd_dest_flags; 247 + __u8 dif_upd_flags; 248 + __u8 dif_upd_res[5]; 249 + __u32 src_ref_tag_seed; 250 + __u16 src_app_tag_mask; 251 + __u16 src_app_tag_seed; 252 + __u32 dest_ref_tag_seed; 253 + __u16 dest_app_tag_mask; 254 + __u16 dest_app_tag_seed; 251 255 }; 252 256 253 257 /* Fill */ 254 - uint64_t pattern_upper; 258 + __u64 pattern_upper; 255 259 256 260 /* Translation fetch */ 257 261 struct { 258 - uint64_t transl_fetch_res; 259 - uint32_t region_stride; 262 + __u64 transl_fetch_res; 263 + __u32 region_stride; 260 264 }; 261 265 262 266 /* DIX generate */ 263 267 struct { 264 - uint8_t dix_gen_res; 265 - uint8_t dest_dif_flags; 266 - uint8_t dif_flags; 267 - uint8_t dix_gen_res2[13]; 268 - uint32_t ref_tag_seed; 269 - uint16_t app_tag_mask; 270 - uint16_t app_tag_seed; 268 + __u8 dix_gen_res; 269 + __u8 dest_dif_flags; 270 + __u8 dif_flags; 271 + __u8 dix_gen_res2[13]; 272 + __u32 ref_tag_seed; 273 + __u16 app_tag_mask; 274 + __u16 app_tag_seed; 271 275 }; 272 276 273 - uint8_t op_specific[24]; 277 + __u8 op_specific[24]; 274 278 }; 275 279 } __attribute__((packed)); 276 280 277 281 struct iax_hw_desc { 278 - uint32_t pasid:20; 279 - uint32_t rsvd:11; 280 - uint32_t priv:1; 281 - uint32_t flags:24; 282 - uint32_t opcode:8; 283 - uint64_t completion_addr; 284 - uint64_t src1_addr; 285 - uint64_t dst_addr; 286 - uint32_t src1_size; 287 - uint16_t int_handle; 282 + __u32 pasid:20; 283 + __u32 rsvd:11; 284 + __u32 priv:1; 285 + __u32 flags:24; 286 + __u32 opcode:8; 287 + __u64 completion_addr; 288 + __u64 src1_addr; 289 + __u64 dst_addr; 290 + __u32 src1_size; 291 + __u16 int_handle; 288 292 union { 289 - uint16_t compr_flags; 290 - uint16_t decompr_flags; 293 + __u16 compr_flags; 294 + __u16 decompr_flags; 291 295 }; 292 - uint64_t src2_addr; 293 - uint32_t max_dst_size; 294 - uint32_t src2_size; 295 - uint32_t filter_flags; 296 - uint32_t num_inputs; 296 + __u64 src2_addr; 297 + __u32 max_dst_size; 298 + __u32 src2_size; 299 + __u32 filter_flags; 300 + __u32 num_inputs; 297 301 } __attribute__((packed)); 298 302 299 303 struct dsa_raw_desc { 300 - uint64_t field[8]; 304 + __u64 field[8]; 301 305 } __attribute__((packed)); 302 306 303 307 /* ··· 305 309 * volatile and prevent the compiler from optimize the read. 306 310 */ 307 311 struct dsa_completion_record { 308 - volatile uint8_t status; 312 + volatile __u8 status; 309 313 union { 310 - uint8_t result; 311 - uint8_t dif_status; 314 + __u8 result; 315 + __u8 dif_status; 312 316 }; 313 - uint8_t fault_info; 314 - uint8_t rsvd; 317 + __u8 fault_info; 318 + __u8 rsvd; 315 319 union { 316 - uint32_t bytes_completed; 317 - uint32_t descs_completed; 320 + __u32 bytes_completed; 321 + __u32 descs_completed; 318 322 }; 319 - uint64_t fault_addr; 323 + __u64 fault_addr; 320 324 union { 321 325 /* common record */ 322 326 struct { 323 - uint32_t invalid_flags:24; 324 - uint32_t rsvd2:8; 327 + __u32 invalid_flags:24; 328 + __u32 rsvd2:8; 325 329 }; 326 330 327 - uint32_t delta_rec_size; 328 - uint64_t crc_val; 331 + __u32 delta_rec_size; 332 + __u64 crc_val; 329 333 330 334 /* DIF check & strip */ 331 335 struct { 332 - uint32_t dif_chk_ref_tag; 333 - uint16_t dif_chk_app_tag_mask; 334 - uint16_t dif_chk_app_tag; 336 + __u32 dif_chk_ref_tag; 337 + __u16 dif_chk_app_tag_mask; 338 + __u16 dif_chk_app_tag; 335 339 }; 336 340 337 341 /* DIF insert */ 338 342 struct { 339 - uint64_t dif_ins_res; 340 - uint32_t dif_ins_ref_tag; 341 - uint16_t dif_ins_app_tag_mask; 342 - uint16_t dif_ins_app_tag; 343 + __u64 dif_ins_res; 344 + __u32 dif_ins_ref_tag; 345 + __u16 dif_ins_app_tag_mask; 346 + __u16 dif_ins_app_tag; 343 347 }; 344 348 345 349 /* DIF update */ 346 350 struct { 347 - uint32_t dif_upd_src_ref_tag; 348 - uint16_t dif_upd_src_app_tag_mask; 349 - uint16_t dif_upd_src_app_tag; 350 - uint32_t dif_upd_dest_ref_tag; 351 - uint16_t dif_upd_dest_app_tag_mask; 352 - uint16_t dif_upd_dest_app_tag; 351 + __u32 dif_upd_src_ref_tag; 352 + __u16 dif_upd_src_app_tag_mask; 353 + __u16 dif_upd_src_app_tag; 354 + __u32 dif_upd_dest_ref_tag; 355 + __u16 dif_upd_dest_app_tag_mask; 356 + __u16 dif_upd_dest_app_tag; 353 357 }; 354 358 355 359 /* DIX generate */ 356 360 struct { 357 - uint64_t dix_gen_res; 358 - uint32_t dix_ref_tag; 359 - uint16_t dix_app_tag_mask; 360 - uint16_t dix_app_tag; 361 + __u64 dix_gen_res; 362 + __u32 dix_ref_tag; 363 + __u16 dix_app_tag_mask; 364 + __u16 dix_app_tag; 361 365 }; 362 366 363 - uint8_t op_specific[16]; 367 + __u8 op_specific[16]; 364 368 }; 365 369 } __attribute__((packed)); 366 370 367 371 struct dsa_raw_completion_record { 368 - uint64_t field[4]; 372 + __u64 field[4]; 369 373 } __attribute__((packed)); 370 374 371 375 struct iax_completion_record { 372 - volatile uint8_t status; 373 - uint8_t error_code; 374 - uint8_t fault_info; 375 - uint8_t rsvd; 376 - uint32_t bytes_completed; 377 - uint64_t fault_addr; 378 - uint32_t invalid_flags; 379 - uint32_t rsvd2; 380 - uint32_t output_size; 381 - uint8_t output_bits; 382 - uint8_t rsvd3; 383 - uint16_t xor_csum; 384 - uint32_t crc; 385 - uint32_t min; 386 - uint32_t max; 387 - uint32_t sum; 388 - uint64_t rsvd4[2]; 376 + volatile __u8 status; 377 + __u8 error_code; 378 + __u8 fault_info; 379 + __u8 rsvd; 380 + __u32 bytes_completed; 381 + __u64 fault_addr; 382 + __u32 invalid_flags; 383 + __u32 rsvd2; 384 + __u32 output_size; 385 + __u8 output_bits; 386 + __u8 rsvd3; 387 + __u16 xor_csum; 388 + __u32 crc; 389 + __u32 min; 390 + __u32 max; 391 + __u32 sum; 392 + __u64 rsvd4[2]; 389 393 } __attribute__((packed)); 390 394 391 395 struct iax_raw_completion_record { 392 - uint64_t field[8]; 396 + __u64 field[8]; 393 397 } __attribute__((packed)); 394 398 395 399 #endif

+26

lib/scatterlist.c

··· 65 65 EXPORT_SYMBOL(sg_nents_for_len); 66 66 67 67 /** 68 + * sg_nents_for_dma - return the count of DMA-capable entries in scatterlist 69 + * @sgl: The scatterlist 70 + * @sglen: The current number of entries 71 + * @len: The maximum length of DMA-capable block 72 + * 73 + * Description: 74 + * Determines the number of entries in @sgl which would be permitted in 75 + * DMA-capable transfer if list had been split accordingly, taking into 76 + * account chaining as well. 77 + * 78 + * Returns: 79 + * the number of sgl entries needed 80 + * 81 + **/ 82 + int sg_nents_for_dma(struct scatterlist *sgl, unsigned int sglen, size_t len) 83 + { 84 + struct scatterlist *sg; 85 + int i, nents = 0; 86 + 87 + for_each_sg(sgl, sg, sglen, i) 88 + nents += DIV_ROUND_UP(sg_dma_len(sg), len); 89 + return nents; 90 + } 91 + EXPORT_SYMBOL(sg_nents_for_dma); 92 + 93 + /** 68 94 * sg_last - return the last scatterlist entry in a list 69 95 * @sgl: First entry in the scatterlist 70 96 * @nents: Number of entries in the scatterlist

Configure Feed

Configure Feed