Linux kernel mirror (for testing) git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel os linux
1
fork

Configure Feed

Select the types of activity you want to include in your feed.

Merge branch 'build-script' into docs-mw

Quoth Mauro:

This series should probably be called:

"Move the trick-or-treat build hacks accumulated over time
into a single place and document them."

as this reflects its main goal. As such:

- it places the jobserver logic on a library;
- it removes sphinx/parallel-wrapper.sh;
- the code now properly implements a jobserver-aware logic
to do the parallelism when called via GNU make, failing back to
"-j" when there's no jobserver;
- converts check-variable-fonts.sh to Python and uses it via
function call;
- drops an extra script to generate man pages, adding a makefile
target for it;
- ensures that return code is 0 when PDF successfully builds;
- about half of the script is comments and documentation.

I tried to do my best to document all tricks that are inside the
script. This way, the docs build steps is now documented.

It should be noticed that it is out of the scope of this series
to change the implementation. Surely the process can be improved,
but first let's consolidate and document everything on a single
place.

Such script was written in a way that it can be called either
directly or via a Makefile. Running outside Makefile is
interesting specially when debug is needed. The command line
interface replaces the need of having lots of env vars before
calling sphinx-build:

$ ./tools/docs/sphinx-build-wrapper --help
usage: sphinx-build-wrapper [-h]
[--sphinxdirs SPHINXDIRS [SPHINXDIRS ...]] [--conf CONF]
[--builddir BUILDDIR] [--theme THEME] [--css CSS] [--paper {,a4,letter}] [-v]
[-j JOBS] [-i] [-V [VENV]]
{cleandocs,linkcheckdocs,htmldocs,epubdocs,texinfodocs,infodocs,mandocs,latexdocs,pdfdocs,xmldocs}

Kernel documentation builder

positional arguments:
{cleandocs,linkcheckdocs,htmldocs,epubdocs,texinfodocs,infodocs,mandocs,latexdocs,pdfdocs,xmldocs}
Documentation target to build

options:
-h, --help show this help message and exit
--sphinxdirs SPHINXDIRS [SPHINXDIRS ...]
Specific directories to build
--conf CONF Sphinx configuration file
--builddir BUILDDIR Sphinx configuration file
--theme THEME Sphinx theme to use
--css CSS Custom CSS file for HTML/EPUB
--paper {,a4,letter} Paper size for LaTeX/PDF output
-v, --verbose place build in verbose mode
-j, --jobs JOBS Sets number of jobs to use with sphinx-build
-i, --interactive Change latex default to run in interactive mode
-V, --venv [VENV] If used, run Sphinx from a venv dir (default dir: sphinx_latest)

the only mandatory argument is the target, which is identical with
"make" targets.

The call inside Makefile doesn't use the last four arguments. They're
there to help identifying problems at the build:

-v makes the output verbose;
-j helps to test parallelism;
-i runs latexmk in interactive mode, allowing to debug PDF
build issues;
-V is useful when testing it with different venvs.

When used with GNU make (or some other make which implements jobserver),
a call like:

make -j <targets> htmldocs

will make the wrapper to automatically use POSIX jobserver to claim
the number of available job slots, calling sphinx-build with a
"-j" parameter reflecting it. ON such case, the default can be
overriden via SPHINXDIRS argument.

Visiable changes when compared with the old behavior:

When V=0, the only visible difference is that:
- pdfdocs target now returns 0 on success, 1 on failures.
This addresses an issue over the current process where we
it always return success even on failures;
- it will now print the name of PDF files that failed to build,
if any.

In verbose mode, sphinx-build-wrapper and sphinx-build command lines
are now displayed.

+1584 -1297
+35 -121
Documentation/Makefile
··· 23 23 SPHINXDIRS = . 24 24 DOCS_THEME = 25 25 DOCS_CSS = 26 - _SPHINXDIRS = $(sort $(patsubst $(srctree)/Documentation/%/index.rst,%,$(wildcard $(srctree)/Documentation/*/index.rst))) 27 - SPHINX_CONF = conf.py 26 + RUSTDOC = 28 27 PAPER = 29 28 BUILDDIR = $(obj)/output 30 29 PDFLATEX = xelatex 31 30 LATEXOPTS = -interaction=batchmode -no-shell-escape 32 31 32 + PYTHONPYCACHEPREFIX ?= $(abspath $(BUILDDIR)/__pycache__) 33 + 34 + # Wrapper for sphinx-build 35 + 36 + BUILD_WRAPPER = $(srctree)/tools/docs/sphinx-build-wrapper 37 + 33 38 # For denylisting "variable font" files 34 39 # Can be overridden by setting as an env variable 35 40 FONTS_CONF_DENY_VF ?= $(HOME)/deny-vf 36 41 37 - ifeq ($(findstring 1, $(KBUILD_VERBOSE)),) 38 - SPHINXOPTS += "-q" 39 - endif 40 - 41 42 # User-friendly check for sphinx-build 42 43 HAVE_SPHINX := $(shell if which $(SPHINXBUILD) >/dev/null 2>&1; then echo 1; else echo 0; fi) 44 + 45 + ifneq ($(wildcard $(srctree)/.config),) 46 + ifeq ($(CONFIG_RUST),y) 47 + RUSTDOC=--rustdoc 48 + endif 49 + endif 43 50 44 51 ifeq ($(HAVE_SPHINX),0) 45 52 46 53 .DEFAULT: 47 54 $(warning The '$(SPHINXBUILD)' command was not found. Make sure you have Sphinx installed and in PATH, or set the SPHINXBUILD make variable to point to the full path of the '$(SPHINXBUILD)' executable.) 48 55 @echo 49 - @$(srctree)/scripts/sphinx-pre-install 56 + @$(srctree)/tools/docs/sphinx-pre-install 50 57 @echo " SKIP Sphinx $@ target." 51 58 52 59 else # HAVE_SPHINX 53 60 54 - # User-friendly check for pdflatex and latexmk 55 - HAVE_PDFLATEX := $(shell if which $(PDFLATEX) >/dev/null 2>&1; then echo 1; else echo 0; fi) 56 - HAVE_LATEXMK := $(shell if which latexmk >/dev/null 2>&1; then echo 1; else echo 0; fi) 61 + # Common documentation targets 62 + htmldocs mandocs infodocs texinfodocs latexdocs epubdocs xmldocs pdfdocs linkcheckdocs: 63 + $(Q)PYTHONPYCACHEPREFIX="$(PYTHONPYCACHEPREFIX)" \ 64 + $(srctree)/tools/docs/sphinx-pre-install --version-check 65 + +$(Q)PYTHONPYCACHEPREFIX="$(PYTHONPYCACHEPREFIX)" \ 66 + $(PYTHON3) $(BUILD_WRAPPER) $@ \ 67 + --sphinxdirs="$(SPHINXDIRS)" $(RUSTDOC) \ 68 + --builddir="$(BUILDDIR)" --deny-vf=$(FONTS_CONF_DENY_VF) \ 69 + --theme=$(DOCS_THEME) --css=$(DOCS_CSS) --paper=$(PAPER) 57 70 58 - ifeq ($(HAVE_LATEXMK),1) 59 - PDFLATEX := latexmk -$(PDFLATEX) 60 - endif #HAVE_LATEXMK 61 71 62 - # Internal variables. 63 - PAPEROPT_a4 = -D latex_elements.papersize=a4paper 64 - PAPEROPT_letter = -D latex_elements.papersize=letterpaper 65 - ALLSPHINXOPTS = -D kerneldoc_srctree=$(srctree) -D kerneldoc_bin=$(KERNELDOC) 66 - ALLSPHINXOPTS += $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) 67 - ifneq ($(wildcard $(srctree)/.config),) 68 - ifeq ($(CONFIG_RUST),y) 69 - # Let Sphinx know we will include rustdoc 70 - ALLSPHINXOPTS += -t rustdoc 71 72 endif 72 - endif 73 - # the i18n builder cannot share the environment and doctrees with the others 74 - I18NSPHINXOPTS = $(PAPEROPT_$(PAPER)) $(SPHINXOPTS) . 75 - 76 - # commands; the 'cmd' from scripts/Kbuild.include is not *loopable* 77 - loop_cmd = $(echo-cmd) $(cmd_$(1)) || exit; 78 - 79 - # $2 sphinx builder e.g. "html" 80 - # $3 name of the build subfolder / e.g. "userspace-api/media", used as: 81 - # * dest folder relative to $(BUILDDIR) and 82 - # * cache folder relative to $(BUILDDIR)/.doctrees 83 - # $4 dest subfolder e.g. "man" for man pages at userspace-api/media/man 84 - # $5 reST source folder relative to $(src), 85 - # e.g. "userspace-api/media" for the linux-tv book-set at ./Documentation/userspace-api/media 86 - 87 - PYTHONPYCACHEPREFIX ?= $(abspath $(BUILDDIR)/__pycache__) 88 - 89 - quiet_cmd_sphinx = SPHINX $@ --> file://$(abspath $(BUILDDIR)/$3/$4) 90 - cmd_sphinx = \ 91 - PYTHONPYCACHEPREFIX="$(PYTHONPYCACHEPREFIX)" \ 92 - BUILDDIR=$(abspath $(BUILDDIR)) SPHINX_CONF=$(abspath $(src)/$5/$(SPHINX_CONF)) \ 93 - $(PYTHON3) $(srctree)/scripts/jobserver-exec \ 94 - $(CONFIG_SHELL) $(srctree)/Documentation/sphinx/parallel-wrapper.sh \ 95 - $(SPHINXBUILD) \ 96 - -b $2 \ 97 - -c $(abspath $(src)) \ 98 - -d $(abspath $(BUILDDIR)/.doctrees/$3) \ 99 - -D version=$(KERNELVERSION) -D release=$(KERNELRELEASE) \ 100 - $(ALLSPHINXOPTS) \ 101 - $(abspath $(src)/$5) \ 102 - $(abspath $(BUILDDIR)/$3/$4) && \ 103 - if [ "x$(DOCS_CSS)" != "x" ]; then \ 104 - cp $(if $(patsubst /%,,$(DOCS_CSS)),$(abspath $(srctree)/$(DOCS_CSS)),$(DOCS_CSS)) $(BUILDDIR)/$3/_static/; \ 105 - fi 106 - 107 - htmldocs: 108 - @$(srctree)/scripts/sphinx-pre-install --version-check 109 - @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,html,$(var),,$(var))) 110 - 111 - htmldocs-redirects: $(srctree)/Documentation/.renames.txt 112 - @tools/docs/gen-redirects.py --output $(BUILDDIR) < $< 113 - 114 - # If Rust support is available and .config exists, add rustdoc generated contents. 115 - # If there are any, the errors from this make rustdoc will be displayed but 116 - # won't stop the execution of htmldocs 117 - 118 - ifneq ($(wildcard $(srctree)/.config),) 119 - ifeq ($(CONFIG_RUST),y) 120 - $(Q)$(MAKE) rustdoc || true 121 - endif 122 - endif 123 - 124 - texinfodocs: 125 - @$(srctree)/scripts/sphinx-pre-install --version-check 126 - @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,texinfo,$(var),texinfo,$(var))) 127 - 128 - # Note: the 'info' Make target is generated by sphinx itself when 129 - # running the texinfodocs target define above. 130 - infodocs: texinfodocs 131 - $(MAKE) -C $(BUILDDIR)/texinfo info 132 - 133 - linkcheckdocs: 134 - @$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,linkcheck,$(var),,$(var))) 135 - 136 - latexdocs: 137 - @$(srctree)/scripts/sphinx-pre-install --version-check 138 - @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,latex,$(var),latex,$(var))) 139 - 140 - ifeq ($(HAVE_PDFLATEX),0) 141 - 142 - pdfdocs: 143 - $(warning The '$(PDFLATEX)' command was not found. Make sure you have it installed and in PATH to produce PDF output.) 144 - @echo " SKIP Sphinx $@ target." 145 - 146 - else # HAVE_PDFLATEX 147 - 148 - pdfdocs: DENY_VF = XDG_CONFIG_HOME=$(FONTS_CONF_DENY_VF) 149 - pdfdocs: latexdocs 150 - @$(srctree)/scripts/sphinx-pre-install --version-check 151 - $(foreach var,$(SPHINXDIRS), \ 152 - $(MAKE) PDFLATEX="$(PDFLATEX)" LATEXOPTS="$(LATEXOPTS)" $(DENY_VF) -C $(BUILDDIR)/$(var)/latex || sh $(srctree)/scripts/check-variable-fonts.sh || exit; \ 153 - mkdir -p $(BUILDDIR)/$(var)/pdf; \ 154 - mv $(subst .tex,.pdf,$(wildcard $(BUILDDIR)/$(var)/latex/*.tex)) $(BUILDDIR)/$(var)/pdf/; \ 155 - ) 156 - 157 - endif # HAVE_PDFLATEX 158 - 159 - epubdocs: 160 - @$(srctree)/scripts/sphinx-pre-install --version-check 161 - @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,epub,$(var),epub,$(var))) 162 - 163 - xmldocs: 164 - @$(srctree)/scripts/sphinx-pre-install --version-check 165 - @+$(foreach var,$(SPHINXDIRS),$(call loop_cmd,sphinx,xml,$(var),xml,$(var))) 166 - 167 - endif # HAVE_SPHINX 168 73 169 74 # The following targets are independent of HAVE_SPHINX, and the rules should 170 75 # work or silently pass without Sphinx. 76 + 77 + htmldocs-redirects: $(srctree)/Documentation/.renames.txt 78 + @tools/docs/gen-redirects.py --output $(BUILDDIR) < $< 171 79 172 80 refcheckdocs: 173 81 $(Q)cd $(srctree);scripts/documentation-file-ref-check ··· 83 175 cleandocs: 84 176 $(Q)rm -rf $(BUILDDIR) 85 177 178 + # Used only on help 179 + _SPHINXDIRS = $(sort $(patsubst $(srctree)/Documentation/%/index.rst,%,$(wildcard $(srctree)/Documentation/*/index.rst))) 180 + 86 181 dochelp: 87 182 @echo ' Linux kernel internal documentation in different formats from ReST:' 88 183 @echo ' htmldocs - HTML' 89 184 @echo ' htmldocs-redirects - generate HTML redirects for moved pages' 90 185 @echo ' texinfodocs - Texinfo' 91 186 @echo ' infodocs - Info' 187 + @echo ' mandocs - Man pages' 92 188 @echo ' latexdocs - LaTeX' 93 189 @echo ' pdfdocs - PDF' 94 190 @echo ' epubdocs - EPUB' ··· 106 194 @echo ' make SPHINXDIRS="s1 s2" [target] Generate only docs of folder s1, s2' 107 195 @echo ' valid values for SPHINXDIRS are: $(_SPHINXDIRS)' 108 196 @echo 109 - @echo ' make SPHINX_CONF={conf-file} [target] use *additional* sphinx-build' 110 - @echo ' configuration. This is e.g. useful to build with nit-picking config.' 111 - @echo 112 197 @echo ' make DOCS_THEME={sphinx-theme} selects a different Sphinx theme.' 113 198 @echo 114 199 @echo ' make DOCS_CSS={a .css file} adds a DOCS_CSS override file for html/epub output.' 200 + @echo 201 + @echo ' make PAPER={a4|letter} Specifies the paper size used for LaTeX/PDF output.' 202 + @echo 203 + @echo ' make FONTS_CONF_DENY_VF={path} sets a deny list to block variable Noto CJK fonts' 204 + @echo ' for PDF build. See tools/docs/lib/latex_fonts.py for more details' 115 205 @echo 116 206 @echo ' Default location for the generated documents is Documentation/output'
+5 -10
Documentation/conf.py
··· 18 18 # documentation root, use os.path.abspath to make it absolute, like shown here. 19 19 sys.path.insert(0, os.path.abspath("sphinx")) 20 20 21 - from load_config import loadConfig # pylint: disable=C0413,E0401 22 - 23 21 # Minimal supported version 24 22 needs_sphinx = "3.4.3" 25 23 ··· 91 93 # LaTeX and PDF output require a list of documents with are dependent 92 94 # of the app.srcdir. Add them here 93 95 94 - # When SPHINXDIRS is used, we just need to get index.rst, if it exists 96 + # Handle the case where SPHINXDIRS is used 95 97 if not os.path.samefile(doctree, app.srcdir): 98 + # Add a tag to mark that the build is actually a subproject 99 + tags.add("subproject") 100 + 101 + # get index.rst, if it exists 96 102 doc = os.path.basename(app.srcdir) 97 103 fname = "index" 98 104 if os.path.exists(os.path.join(app.srcdir, fname + ".rst")): ··· 584 582 # line arguments. 585 583 kerneldoc_bin = "../scripts/kernel-doc.py" 586 584 kerneldoc_srctree = ".." 587 - 588 - # ------------------------------------------------------------------------------ 589 - # Since loadConfig overwrites settings from the global namespace, it has to be 590 - # the last statement in the conf.py file 591 - # ------------------------------------------------------------------------------ 592 - loadConfig(globals()) 593 - 594 585 595 586 def setup(app): 596 587 """Patterns need to be updated at init time on older Sphinx versions"""
+16 -13
Documentation/doc-guide/kernel-doc.rst
··· 579 579 How to use kernel-doc to generate man pages 580 580 ------------------------------------------- 581 581 582 - If you just want to use kernel-doc to generate man pages you can do this 583 - from the kernel git tree:: 582 + To generate man pages for all files that contain kernel-doc markups, run:: 584 583 585 - $ scripts/kernel-doc -man \ 586 - $(git grep -l '/\*\*' -- :^Documentation :^tools) \ 587 - | scripts/split-man.pl /tmp/man 584 + $ make mandocs 588 585 589 - Some older versions of git do not support some of the variants of syntax for 590 - path exclusion. One of the following commands may work for those versions:: 586 + Or calling ``script-build-wrapper`` directly:: 591 587 592 - $ scripts/kernel-doc -man \ 593 - $(git grep -l '/\*\*' -- . ':!Documentation' ':!tools') \ 594 - | scripts/split-man.pl /tmp/man 588 + $ ./tools/docs/sphinx-build-wrapper mandocs 595 589 596 - $ scripts/kernel-doc -man \ 597 - $(git grep -l '/\*\*' -- . ":(exclude)Documentation" ":(exclude)tools") \ 598 - | scripts/split-man.pl /tmp/man 590 + The output will be at ``/man`` directory inside the output directory 591 + (by default: ``Documentation/output``). 592 + 593 + Optionally, it is possible to generate a partial set of man pages by 594 + using SPHINXDIRS: 595 + 596 + $ make SPHINXDIRS=driver-api/media mandocs 597 + 598 + .. note:: 599 + 600 + When SPHINXDIRS={subdir} is used, it will only generate man pages for 601 + the files explicitly inside a ``Documentation/{subdir}/.../*.rst`` file.
+2 -2
Documentation/doc-guide/sphinx.rst
··· 106 106 recognize your distribution, it will also give a hint about the install 107 107 command line options for your distro:: 108 108 109 - $ ./scripts/sphinx-pre-install 109 + $ ./tools/docs/sphinx-pre-install 110 110 Checking if the needed tools for Fedora release 26 (Twenty Six) are available 111 111 Warning: better to also install "texlive-luatex85". 112 112 You should run: ··· 116 116 . sphinx_2.4.4/bin/activate 117 117 pip install -r Documentation/sphinx/requirements.txt 118 118 119 - Can't build as 1 mandatory dependency is missing at ./scripts/sphinx-pre-install line 468. 119 + Can't build as 1 mandatory dependency is missing at ./tools/docs/sphinx-pre-install line 468. 120 120 121 121 By default, it checks all the requirements for both html and PDF, including 122 122 the requirements for images, math expressions and LaTeX build, and assumes
+1 -1
Documentation/sphinx/kerneldoc-preamble.sty
··· 220 220 If you want them, please install non-variable ``Noto Sans CJK'' 221 221 font families along with the texlive-xecjk package by following 222 222 instructions from 223 - \sphinxcode{./scripts/sphinx-pre-install}. 223 + \sphinxcode{./tools/docs/sphinx-pre-install}. 224 224 Having optional non-variable ``Noto Serif CJK'' font families will 225 225 improve the looks of those translations. 226 226 \end{sphinxadmonition}}
-60
Documentation/sphinx/load_config.py
··· 1 - # -*- coding: utf-8; mode: python -*- 2 - # SPDX-License-Identifier: GPL-2.0 3 - # pylint: disable=R0903, C0330, R0914, R0912, E0401 4 - 5 - import os 6 - import sys 7 - from sphinx.util.osutil import fs_encoding 8 - 9 - # ------------------------------------------------------------------------------ 10 - def loadConfig(namespace): 11 - # ------------------------------------------------------------------------------ 12 - 13 - """Load an additional configuration file into *namespace*. 14 - 15 - The name of the configuration file is taken from the environment 16 - ``SPHINX_CONF``. The external configuration file extends (or overwrites) the 17 - configuration values from the origin ``conf.py``. With this you are able to 18 - maintain *build themes*. """ 19 - 20 - config_file = os.environ.get("SPHINX_CONF", None) 21 - if (config_file is not None 22 - and os.path.normpath(namespace["__file__"]) != os.path.normpath(config_file) ): 23 - config_file = os.path.abspath(config_file) 24 - 25 - # Let's avoid one conf.py file just due to latex_documents 26 - start = config_file.find('Documentation/') 27 - if start >= 0: 28 - start = config_file.find('/', start + 1) 29 - 30 - end = config_file.rfind('/') 31 - if start >= 0 and end > 0: 32 - dir = config_file[start + 1:end] 33 - 34 - print("source directory: %s" % dir) 35 - new_latex_docs = [] 36 - latex_documents = namespace['latex_documents'] 37 - 38 - for l in latex_documents: 39 - if l[0].find(dir + '/') == 0: 40 - has = True 41 - fn = l[0][len(dir) + 1:] 42 - new_latex_docs.append((fn, l[1], l[2], l[3], l[4])) 43 - break 44 - 45 - namespace['latex_documents'] = new_latex_docs 46 - 47 - # If there is an extra conf.py file, load it 48 - if os.path.isfile(config_file): 49 - sys.stdout.write("load additional sphinx-config: %s\n" % config_file) 50 - config = namespace.copy() 51 - config['__file__'] = config_file 52 - with open(config_file, 'rb') as f: 53 - code = compile(f.read(), fs_encoding, 'exec') 54 - exec(code, config) 55 - del config['__file__'] 56 - namespace.update(config) 57 - else: 58 - config = namespace.copy() 59 - config['tags'].add("subproject") 60 - namespace.update(config)
-33
Documentation/sphinx/parallel-wrapper.sh
··· 1 - #!/bin/sh 2 - # SPDX-License-Identifier: GPL-2.0+ 3 - # 4 - # Figure out if we should follow a specific parallelism from the make 5 - # environment (as exported by scripts/jobserver-exec), or fall back to 6 - # the "auto" parallelism when "-jN" is not specified at the top-level 7 - # "make" invocation. 8 - 9 - sphinx="$1" 10 - shift || true 11 - 12 - parallel="$PARALLELISM" 13 - if [ -z "$parallel" ] ; then 14 - # If no parallelism is specified at the top-level make, then 15 - # fall back to the expected "-jauto" mode that the "htmldocs" 16 - # target has had. 17 - auto=$(perl -e 'open IN,"'"$sphinx"' --version 2>&1 |"; 18 - while (<IN>) { 19 - if (m/([\d\.]+)/) { 20 - print "auto" if ($1 >= "1.7") 21 - } 22 - } 23 - close IN') 24 - if [ -n "$auto" ] ; then 25 - parallel="$auto" 26 - fi 27 - fi 28 - # Only if some parallelism has been determined do we add the -jN option. 29 - if [ -n "$parallel" ] ; then 30 - parallel="-j$parallel" 31 - fi 32 - 33 - exec "$sphinx" $parallel "$@"
+2 -2
Documentation/translations/it_IT/doc-guide/sphinx.rst
··· 109 109 sarà in grado di darvi dei suggerimenti su come procedere per completare 110 110 l'installazione:: 111 111 112 - $ ./scripts/sphinx-pre-install 112 + $ ./tools/docs/sphinx-pre-install 113 113 Checking if the needed tools for Fedora release 26 (Twenty Six) are available 114 114 Warning: better to also install "texlive-luatex85". 115 115 You should run: ··· 119 119 . sphinx_2.4.4/bin/activate 120 120 pip install -r Documentation/sphinx/requirements.txt 121 121 122 - Can't build as 1 mandatory dependency is missing at ./scripts/sphinx-pre-install line 468. 122 + Can't build as 1 mandatory dependency is missing at ./tools/docs/sphinx-pre-install line 468. 123 123 124 124 L'impostazione predefinita prevede il controllo dei requisiti per la generazione 125 125 di documenti html e PDF, includendo anche il supporto per le immagini, le
+2 -2
Documentation/translations/zh_CN/doc-guide/sphinx.rst
··· 84 84 这有一个脚本可以自动检查Sphinx依赖项。如果它认得您的发行版,还会提示您所用发行 85 85 版的安装命令:: 86 86 87 - $ ./scripts/sphinx-pre-install 87 + $ ./tools/docs/sphinx-pre-install 88 88 Checking if the needed tools for Fedora release 26 (Twenty Six) are available 89 89 Warning: better to also install "texlive-luatex85". 90 90 You should run: ··· 94 94 . sphinx_2.4.4/bin/activate 95 95 pip install -r Documentation/sphinx/requirements.txt 96 96 97 - Can't build as 1 mandatory dependency is missing at ./scripts/sphinx-pre-install line 468. 97 + Can't build as 1 mandatory dependency is missing at ./tools/docs/sphinx-pre-install line 468. 98 98 99 99 默认情况下,它会检查html和PDF的所有依赖项,包括图像、数学表达式和LaTeX构建的 100 100 需求,并假设将使用虚拟Python环境。html构建所需的依赖项被认为是必需的,其他依
+1 -1
Documentation/translations/zh_CN/how-to.rst
··· 64 64 :: 65 65 66 66 cd linux 67 - ./scripts/sphinx-pre-install 67 + ./tools/docs/sphinx-pre-install 68 68 69 69 以 Fedora 为例,它的输出是这样的:: 70 70
+1 -3
MAINTAINERS
··· 7411 7411 P: Documentation/doc-guide/maintainer-profile.rst 7412 7412 T: git git://git.lwn.net/linux.git docs-next 7413 7413 F: Documentation/ 7414 - F: scripts/check-variable-fonts.sh 7415 7414 F: scripts/checktransupdate.py 7416 7415 F: scripts/documentation-file-ref-check 7417 7416 F: scripts/get_abi.py ··· 7419 7420 F: scripts/lib/kdoc/* 7420 7421 F: tools/docs/* 7421 7422 F: tools/net/ynl/pyynl/lib/doc_generator.py 7422 - F: scripts/sphinx-pre-install 7423 7423 X: Documentation/ABI/ 7424 7424 X: Documentation/admin-guide/media/ 7425 7425 X: Documentation/devicetree/ ··· 7453 7455 S: Maintained 7454 7456 F: Documentation/sphinx/parse-headers.pl 7455 7457 F: scripts/documentation-file-ref-check 7456 - F: scripts/sphinx-pre-install 7458 + F: tools/docs/sphinx-pre-install 7457 7459 7458 7460 DOCUMENTATION/ITALIAN 7459 7461 M: Federico Vaga <federico.vaga@vaga.pv.it>
+4 -3
Makefile
··· 1797 1797 1798 1798 # Documentation targets 1799 1799 # --------------------------------------------------------------------------- 1800 - DOC_TARGETS := xmldocs latexdocs pdfdocs htmldocs htmldocs-redirects \ 1801 - epubdocs cleandocs linkcheckdocs dochelp refcheckdocs \ 1802 - texinfodocs infodocs 1800 + DOC_TARGETS := xmldocs latexdocs pdfdocs htmldocs epubdocs cleandocs \ 1801 + linkcheckdocs dochelp refcheckdocs texinfodocs infodocs mandocs \ 1802 + htmldocs-redirects 1803 + 1803 1804 PHONY += $(DOC_TARGETS) 1804 1805 $(DOC_TARGETS): 1805 1806 $(Q)$(MAKE) $(build)=Documentation $@
-115
scripts/check-variable-fonts.sh
··· 1 - #!/bin/sh 2 - # SPDX-License-Identifier: GPL-2.0-only 3 - # Copyright (C) Akira Yokosawa, 2024 4 - # 5 - # For "make pdfdocs", reports of build errors of translations.pdf started 6 - # arriving early 2024 [1, 2]. It turned out that Fedora and openSUSE 7 - # tumbleweed have started deploying variable-font [3] format of "Noto CJK" 8 - # fonts [4, 5]. For PDF, a LaTeX package named xeCJK is used for CJK 9 - # (Chinese, Japanese, Korean) pages. xeCJK requires XeLaTeX/XeTeX, which 10 - # does not (and likely never will) understand variable fonts for historical 11 - # reasons. 12 - # 13 - # The build error happens even when both of variable- and non-variable-format 14 - # fonts are found on the build system. To make matters worse, Fedora enlists 15 - # variable "Noto CJK" fonts in the requirements of langpacks-ja, -ko, -zh_CN, 16 - # -zh_TW, etc. Hence developers who have interest in CJK pages are more 17 - # likely to encounter the build errors. 18 - # 19 - # This script is invoked from the error path of "make pdfdocs" and emits 20 - # suggestions if variable-font files of "Noto CJK" fonts are in the list of 21 - # fonts accessible from XeTeX. 22 - # 23 - # References: 24 - # [1]: https://lore.kernel.org/r/8734tqsrt7.fsf@meer.lwn.net/ 25 - # [2]: https://lore.kernel.org/r/1708585803.600323099@f111.i.mail.ru/ 26 - # [3]: https://en.wikipedia.org/wiki/Variable_font 27 - # [4]: https://fedoraproject.org/wiki/Changes/Noto_CJK_Variable_Fonts 28 - # [5]: https://build.opensuse.org/request/show/1157217 29 - # 30 - #=========================================================================== 31 - # Workarounds for building translations.pdf 32 - #=========================================================================== 33 - # 34 - # * Denylist "variable font" Noto CJK fonts. 35 - # - Create $HOME/deny-vf/fontconfig/fonts.conf from template below, with 36 - # tweaks if necessary. Remove leading "# ". 37 - # - Path of fontconfig/fonts.conf can be overridden by setting an env 38 - # variable FONTS_CONF_DENY_VF. 39 - # 40 - # * Template: 41 - # ----------------------------------------------------------------- 42 - # <?xml version="1.0"?> 43 - # <!DOCTYPE fontconfig SYSTEM "urn:fontconfig:fonts.dtd"> 44 - # <fontconfig> 45 - # <!-- 46 - # Ignore variable-font glob (not to break xetex) 47 - # --> 48 - # <selectfont> 49 - # <rejectfont> 50 - # <!-- 51 - # for Fedora 52 - # --> 53 - # <glob>/usr/share/fonts/google-noto-*-cjk-vf-fonts</glob> 54 - # <!-- 55 - # for openSUSE tumbleweed 56 - # --> 57 - # <glob>/usr/share/fonts/truetype/Noto*CJK*-VF.otf</glob> 58 - # </rejectfont> 59 - # </selectfont> 60 - # </fontconfig> 61 - # ----------------------------------------------------------------- 62 - # 63 - # The denylisting is activated for "make pdfdocs". 64 - # 65 - # * For skipping CJK pages in PDF 66 - # - Uninstall texlive-xecjk. 67 - # Denylisting is not needed in this case. 68 - # 69 - # * For printing CJK pages in PDF 70 - # - Need non-variable "Noto CJK" fonts. 71 - # * Fedora 72 - # - google-noto-sans-cjk-fonts 73 - # - google-noto-serif-cjk-fonts 74 - # * openSUSE tumbleweed 75 - # - Non-variable "Noto CJK" fonts are not available as distro packages 76 - # as of April, 2024. Fetch a set of font files from upstream Noto 77 - # CJK Font released at: 78 - # https://github.com/notofonts/noto-cjk/tree/main/Sans#super-otc 79 - # and at: 80 - # https://github.com/notofonts/noto-cjk/tree/main/Serif#super-otc 81 - # , then uncompress and deploy them. 82 - # - Remember to update fontconfig cache by running fc-cache. 83 - # 84 - # !!! Caution !!! 85 - # Uninstalling "variable font" packages can be dangerous. 86 - # They might be depended upon by other packages important for your work. 87 - # Denylisting should be less invasive, as it is effective only while 88 - # XeLaTeX runs in "make pdfdocs". 89 - 90 - # Default per-user fontconfig path (overridden by env variable) 91 - : ${FONTS_CONF_DENY_VF:=$HOME/deny-vf} 92 - 93 - export XDG_CONFIG_HOME=${FONTS_CONF_DENY_VF} 94 - 95 - notocjkvffonts=`fc-list : file family variable | \ 96 - grep 'variable=True' | \ 97 - grep -E -e 'Noto (Sans|Sans Mono|Serif) CJK' | \ 98 - sed -e 's/^/ /' -e 's/: Noto S.*$//' | sort | uniq` 99 - 100 - if [ "x$notocjkvffonts" != "x" ] ; then 101 - echo '=============================================================================' 102 - echo 'XeTeX is confused by "variable font" files listed below:' 103 - echo "$notocjkvffonts" 104 - echo 105 - echo 'For CJK pages in PDF, they need to be hidden from XeTeX by denylisting.' 106 - echo 'Or, CJK pages can be skipped by uninstalling texlive-xecjk.' 107 - echo 108 - echo 'For more info on denylisting, other options, and variable font, see header' 109 - echo 'comments of scripts/check-variable-fonts.sh.' 110 - echo '=============================================================================' 111 - fi 112 - 113 - # As this script is invoked from Makefile's error path, always error exit 114 - # regardless of whether any variable font is discovered or not. 115 - exit 1
+23 -65
scripts/jobserver-exec
··· 1 1 #!/usr/bin/env python3 2 2 # SPDX-License-Identifier: GPL-2.0+ 3 - # 4 - # This determines how many parallel tasks "make" is expecting, as it is 5 - # not exposed via an special variables, reserves them all, runs a subprocess 6 - # with PARALLELISM environment variable set, and releases the jobs back again. 7 - # 8 - # https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html#POSIX-Jobserver 9 - from __future__ import print_function 10 - import os, sys, errno 11 - import subprocess 12 3 13 - # Extract and prepare jobserver file descriptors from environment. 14 - claim = 0 15 - jobs = b"" 16 - try: 17 - # Fetch the make environment options. 18 - flags = os.environ['MAKEFLAGS'] 4 + """ 5 + Determines how many parallel tasks "make" is expecting, as it is 6 + not exposed via any special variables, reserves them all, runs a subprocess 7 + with PARALLELISM environment variable set, and releases the jobs back again. 19 8 20 - # Look for "--jobserver=R,W" 21 - # Note that GNU Make has used --jobserver-fds and --jobserver-auth 22 - # so this handles all of them. 23 - opts = [x for x in flags.split(" ") if x.startswith("--jobserver")] 9 + See: 10 + https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html#POSIX-Jobserver 11 + """ 24 12 25 - # Parse out R,W file descriptor numbers and set them nonblocking. 26 - # If the MAKEFLAGS variable contains multiple instances of the 27 - # --jobserver-auth= option, the last one is relevant. 28 - fds = opts[-1].split("=", 1)[1] 13 + import os 14 + import sys 29 15 30 - # Starting with GNU Make 4.4, named pipes are used for reader and writer. 31 - # Example argument: --jobserver-auth=fifo:/tmp/GMfifo8134 32 - _, _, path = fds.partition('fifo:') 16 + LIB_DIR = "lib" 17 + SRC_DIR = os.path.dirname(os.path.realpath(__file__)) 33 18 34 - if path: 35 - reader = os.open(path, os.O_RDONLY | os.O_NONBLOCK) 36 - writer = os.open(path, os.O_WRONLY) 37 - else: 38 - reader, writer = [int(x) for x in fds.split(",", 1)] 39 - # Open a private copy of reader to avoid setting nonblocking 40 - # on an unexpecting process with the same reader fd. 41 - reader = os.open("/proc/self/fd/%d" % (reader), 42 - os.O_RDONLY | os.O_NONBLOCK) 19 + sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR)) 43 20 44 - # Read out as many jobserver slots as possible. 45 - while True: 46 - try: 47 - slot = os.read(reader, 8) 48 - jobs += slot 49 - except (OSError, IOError) as e: 50 - if e.errno == errno.EWOULDBLOCK: 51 - # Stop at the end of the jobserver queue. 52 - break 53 - # If something went wrong, give back the jobs. 54 - if len(jobs): 55 - os.write(writer, jobs) 56 - raise e 57 - # Add a bump for our caller's reserveration, since we're just going 58 - # to sit here blocked on our child. 59 - claim = len(jobs) + 1 60 - except (KeyError, IndexError, ValueError, OSError, IOError) as e: 61 - # Any missing environment strings or bad fds should result in just 62 - # not being parallel. 63 - pass 21 + from jobserver import JobserverExec # pylint: disable=C0415 64 22 65 - # We can only claim parallelism if there was a jobserver (i.e. a top-level 66 - # "-jN" argument) and there were no other failures. Otherwise leave out the 67 - # environment variable and let the child figure out what is best. 68 - if claim > 0: 69 - os.environ['PARALLELISM'] = '%d' % (claim) 70 23 71 - rc = subprocess.call(sys.argv[1:]) 24 + def main(): 25 + """Main program""" 26 + if len(sys.argv) < 2: 27 + name = os.path.basename(__file__) 28 + sys.exit("usage: " + name +" command [args ...]\n" + __doc__) 72 29 73 - # Return all the reserved slots. 74 - if len(jobs): 75 - os.write(writer, jobs) 30 + with JobserverExec() as jobserver: 31 + jobserver.run(sys.argv[1:]) 76 32 77 - sys.exit(rc) 33 + 34 + if __name__ == "__main__": 35 + main()
+149
scripts/lib/jobserver.py
··· 1 + #!/usr/bin/env python3 2 + # SPDX-License-Identifier: GPL-2.0+ 3 + # 4 + # pylint: disable=C0103,C0209 5 + # 6 + # 7 + 8 + """ 9 + Interacts with the POSIX jobserver during the Kernel build time. 10 + 11 + A "normal" jobserver task, like the one initiated by a make subrocess would do: 12 + 13 + - open read/write file descriptors to communicate with the job server; 14 + - ask for one slot by calling: 15 + claim = os.read(reader, 1) 16 + - when the job finshes, call: 17 + os.write(writer, b"+") # os.write(writer, claim) 18 + 19 + Here, the goal is different: This script aims to get the remaining number 20 + of slots available, using all of them to run a command which handle tasks in 21 + parallel. To to that, it has a loop that ends only after there are no 22 + slots left. It then increments the number by one, in order to allow a 23 + call equivalent to make -j$((claim+1)), e.g. having a parent make creating 24 + $claim child to do the actual work. 25 + 26 + The end goal here is to keep the total number of build tasks under the 27 + limit established by the initial make -j$n_proc call. 28 + 29 + See: 30 + https://www.gnu.org/software/make/manual/html_node/POSIX-Jobserver.html#POSIX-Jobserver 31 + """ 32 + 33 + import errno 34 + import os 35 + import subprocess 36 + import sys 37 + 38 + class JobserverExec: 39 + """ 40 + Claim all slots from make using POSIX Jobserver. 41 + 42 + The main methods here are: 43 + - open(): reserves all slots; 44 + - close(): method returns all used slots back to make; 45 + - run(): executes a command setting PARALLELISM=<available slots jobs + 1> 46 + """ 47 + 48 + def __init__(self): 49 + """Initialize internal vars""" 50 + self.claim = 0 51 + self.jobs = b"" 52 + self.reader = None 53 + self.writer = None 54 + self.is_open = False 55 + 56 + def open(self): 57 + """Reserve all available slots to be claimed later on""" 58 + 59 + if self.is_open: 60 + return 61 + 62 + try: 63 + # Fetch the make environment options. 64 + flags = os.environ["MAKEFLAGS"] 65 + # Look for "--jobserver=R,W" 66 + # Note that GNU Make has used --jobserver-fds and --jobserver-auth 67 + # so this handles all of them. 68 + opts = [x for x in flags.split(" ") if x.startswith("--jobserver")] 69 + 70 + # Parse out R,W file descriptor numbers and set them nonblocking. 71 + # If the MAKEFLAGS variable contains multiple instances of the 72 + # --jobserver-auth= option, the last one is relevant. 73 + fds = opts[-1].split("=", 1)[1] 74 + 75 + # Starting with GNU Make 4.4, named pipes are used for reader 76 + # and writer. 77 + # Example argument: --jobserver-auth=fifo:/tmp/GMfifo8134 78 + _, _, path = fds.partition("fifo:") 79 + 80 + if path: 81 + self.reader = os.open(path, os.O_RDONLY | os.O_NONBLOCK) 82 + self.writer = os.open(path, os.O_WRONLY) 83 + else: 84 + self.reader, self.writer = [int(x) for x in fds.split(",", 1)] 85 + # Open a private copy of reader to avoid setting nonblocking 86 + # on an unexpecting process with the same reader fd. 87 + self.reader = os.open("/proc/self/fd/%d" % (self.reader), 88 + os.O_RDONLY | os.O_NONBLOCK) 89 + 90 + # Read out as many jobserver slots as possible 91 + while True: 92 + try: 93 + slot = os.read(self.reader, 8) 94 + self.jobs += slot 95 + except (OSError, IOError) as e: 96 + if e.errno == errno.EWOULDBLOCK: 97 + # Stop at the end of the jobserver queue. 98 + break 99 + # If something went wrong, give back the jobs. 100 + if self.jobs: 101 + os.write(self.writer, self.jobs) 102 + raise e 103 + 104 + # Add a bump for our caller's reserveration, since we're just going 105 + # to sit here blocked on our child. 106 + self.claim = len(self.jobs) + 1 107 + 108 + except (KeyError, IndexError, ValueError, OSError, IOError): 109 + # Any missing environment strings or bad fds should result in just 110 + # not being parallel. 111 + self.claim = None 112 + 113 + self.is_open = True 114 + 115 + def close(self): 116 + """Return all reserved slots to Jobserver""" 117 + 118 + if not self.is_open: 119 + return 120 + 121 + # Return all the reserved slots. 122 + if len(self.jobs): 123 + os.write(self.writer, self.jobs) 124 + 125 + self.is_open = False 126 + 127 + def __enter__(self): 128 + self.open() 129 + return self 130 + 131 + def __exit__(self, exc_type, exc_value, exc_traceback): 132 + self.close() 133 + 134 + def run(self, cmd, *args, **pwargs): 135 + """ 136 + Run a command setting PARALLELISM env variable to the number of 137 + available job slots (claim) + 1, e.g. it will reserve claim slots 138 + to do the actual build work, plus one to monitor its children. 139 + """ 140 + self.open() # Ensure that self.claim is set 141 + 142 + # We can only claim parallelism if there was a jobserver (i.e. a 143 + # top-level "-jN" argument) and there were no other failures. Otherwise 144 + # leave out the environment variable and let the child figure out what 145 + # is best. 146 + if self.claim: 147 + os.environ["PARALLELISM"] = str(self.claim) 148 + 149 + return subprocess.call(cmd, *args, **pwargs)
+4 -1
scripts/lib/kdoc/kdoc_files.py
··· 275 275 self.config.log.warning("No kernel-doc for file %s", fname) 276 276 continue 277 277 278 - for arg in self.results[fname]: 278 + symbols = self.results[fname] 279 + self.out_style.set_symbols(symbols) 280 + 281 + for arg in symbols: 279 282 m = self.out_msg(fname, arg.name, arg) 280 283 281 284 if m is None:
+2 -1
scripts/lib/kdoc/kdoc_item.py
··· 5 5 # 6 6 7 7 class KdocItem: 8 - def __init__(self, name, type, start_line, **other_stuff): 8 + def __init__(self, name, fname, type, start_line, **other_stuff): 9 9 self.name = name 10 + self.fname = fname 10 11 self.type = type 11 12 self.declaration_start_line = start_line 12 13 self.sections = {}
+80 -5
scripts/lib/kdoc/kdoc_output.py
··· 215 215 216 216 # Virtual methods to be overridden by inherited classes 217 217 # At the base class, those do nothing. 218 + def set_symbols(self, symbols): 219 + """Get a list of all symbols from kernel_doc""" 220 + 218 221 def out_doc(self, fname, name, args): 219 222 """Outputs a DOC block""" 220 223 ··· 580 577 581 578 super().__init__() 582 579 self.modulename = modulename 580 + self.symbols = [] 583 581 584 582 dt = None 585 583 tstamp = os.environ.get("KBUILD_BUILD_TIMESTAMP") ··· 596 592 dt = datetime.now() 597 593 598 594 self.man_date = dt.strftime("%B %Y") 595 + 596 + def arg_name(self, args, name): 597 + """ 598 + Return the name that will be used for the man page. 599 + 600 + As we may have the same name on different namespaces, 601 + prepend the data type for all types except functions and typedefs. 602 + 603 + The doc section is special: it uses the modulename. 604 + """ 605 + 606 + dtype = args.type 607 + 608 + if dtype == "doc": 609 + return self.modulename 610 + 611 + if dtype in ["function", "typedef"]: 612 + return name 613 + 614 + return f"{dtype} {name}" 615 + 616 + def set_symbols(self, symbols): 617 + """ 618 + Get a list of all symbols from kernel_doc. 619 + 620 + Man pages will uses it to add a SEE ALSO section with other 621 + symbols at the same file. 622 + """ 623 + self.symbols = symbols 624 + 625 + def out_tail(self, fname, name, args): 626 + """Adds a tail for all man pages""" 627 + 628 + # SEE ALSO section 629 + self.data += f'.SH "SEE ALSO"' + "\n.PP\n" 630 + self.data += (f"Kernel file \\fB{args.fname}\\fR\n") 631 + if len(self.symbols) >= 2: 632 + cur_name = self.arg_name(args, name) 633 + 634 + related = [] 635 + for arg in self.symbols: 636 + out_name = self.arg_name(arg, arg.name) 637 + 638 + if cur_name == out_name: 639 + continue 640 + 641 + related.append(f"\\fB{out_name}\\fR(9)") 642 + 643 + self.data += ",\n".join(related) + "\n" 644 + 645 + # TODO: does it make sense to add other sections? Maybe 646 + # REPORTING ISSUES? LICENSE? 647 + 648 + def msg(self, fname, name, args): 649 + """ 650 + Handles a single entry from kernel-doc parser. 651 + 652 + Add a tail at the end of man pages output. 653 + """ 654 + super().msg(fname, name, args) 655 + self.out_tail(fname, name, args) 656 + 657 + return self.data 599 658 600 659 def output_highlight(self, block): 601 660 """ ··· 685 618 if not self.check_doc(name, args): 686 619 return 687 620 688 - self.data += f'.TH "{self.modulename}" 9 "{self.modulename}" "{self.man_date}" "API Manual" LINUX' + "\n" 621 + out_name = self.arg_name(args, name) 622 + 623 + self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n" 689 624 690 625 for section, text in args.sections.items(): 691 626 self.data += f'.SH "{section}"' + "\n" ··· 696 627 def out_function(self, fname, name, args): 697 628 """output function in man""" 698 629 699 - self.data += f'.TH "{name}" 9 "{name}" "{self.man_date}" "Kernel Hacker\'s Manual" LINUX' + "\n" 630 + out_name = self.arg_name(args, name) 631 + 632 + self.data += f'.TH "{name}" 9 "{out_name}" "{self.man_date}" "Kernel Hacker\'s Manual" LINUX' + "\n" 700 633 701 634 self.data += ".SH NAME\n" 702 635 self.data += f"{name} \\- {args['purpose']}\n" ··· 742 671 self.output_highlight(text) 743 672 744 673 def out_enum(self, fname, name, args): 745 - self.data += f'.TH "{self.modulename}" 9 "enum {name}" "{self.man_date}" "API Manual" LINUX' + "\n" 674 + out_name = self.arg_name(args, name) 675 + 676 + self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n" 746 677 747 678 self.data += ".SH NAME\n" 748 679 self.data += f"enum {name} \\- {args['purpose']}\n" ··· 776 703 def out_typedef(self, fname, name, args): 777 704 module = self.modulename 778 705 purpose = args.get('purpose') 706 + out_name = self.arg_name(args, name) 779 707 780 - self.data += f'.TH "{module}" 9 "{name}" "{self.man_date}" "API Manual" LINUX' + "\n" 708 + self.data += f'.TH "{module}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n" 781 709 782 710 self.data += ".SH NAME\n" 783 711 self.data += f"typedef {name} \\- {purpose}\n" ··· 791 717 module = self.modulename 792 718 purpose = args.get('purpose') 793 719 definition = args.get('definition') 720 + out_name = self.arg_name(args, name) 794 721 795 - self.data += f'.TH "{module}" 9 "{args.type} {name}" "{self.man_date}" "API Manual" LINUX' + "\n" 722 + self.data += f'.TH "{module}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n" 796 723 797 724 self.data += ".SH NAME\n" 798 725 self.data += f"{args.type} {name} \\- {purpose}\n"
+11 -4
scripts/lib/kdoc/kdoc_parser.py
··· 254 254 255 255 class KernelEntry: 256 256 257 - def __init__(self, config, ln): 257 + def __init__(self, config, fname, ln): 258 258 self.config = config 259 + self.fname = fname 259 260 260 261 self._contents = [] 261 262 self.prototype = "" ··· 351 350 self.section = SECTION_DEFAULT 352 351 self._contents = [] 353 352 353 + python_warning = False 354 354 355 355 class KernelDoc: 356 356 """ ··· 385 383 # We need Python 3.7 for its "dicts remember the insertion 386 384 # order" guarantee 387 385 # 388 - if sys.version_info.major == 3 and sys.version_info.minor < 7: 386 + global python_warning 387 + if (not python_warning and 388 + sys.version_info.major == 3 and sys.version_info.minor < 7): 389 + 389 390 self.emit_msg(0, 390 391 'Python 3.7 or later is required for correct results') 392 + python_warning = True 391 393 392 394 def emit_msg(self, ln, msg, warning=True): 393 395 """Emit a message""" ··· 423 417 The actual output and output filters will be handled elsewhere 424 418 """ 425 419 426 - item = KdocItem(name, dtype, self.entry.declaration_start_line, **args) 420 + item = KdocItem(name, self.fname, dtype, 421 + self.entry.declaration_start_line, **args) 427 422 item.warnings = self.entry.warnings 428 423 429 424 # Drop empty sections ··· 447 440 variables used by the state machine. 448 441 """ 449 442 450 - self.entry = KernelEntry(self.config, ln) 443 + self.entry = KernelEntry(self.config, self.fname, ln) 451 444 452 445 # State flags 453 446 self.state = state.NORMAL
-719
scripts/sphinx-build-wrapper
··· 1 - #!/usr/bin/env python3 2 - # SPDX-License-Identifier: GPL-2.0 3 - # Copyright (C) 2025 Mauro Carvalho Chehab <mchehab+huawei@kernel.org> 4 - # 5 - # pylint: disable=R0902, R0912, R0913, R0914, R0915, R0917, C0103 6 - # 7 - # Converted from docs Makefile and parallel-wrapper.sh, both under 8 - # GPLv2, copyrighted since 2008 by the following authors: 9 - # 10 - # Akira Yokosawa <akiyks@gmail.com> 11 - # Arnd Bergmann <arnd@arndb.de> 12 - # Breno Leitao <leitao@debian.org> 13 - # Carlos Bilbao <carlos.bilbao@amd.com> 14 - # Dave Young <dyoung@redhat.com> 15 - # Donald Hunter <donald.hunter@gmail.com> 16 - # Geert Uytterhoeven <geert+renesas@glider.be> 17 - # Jani Nikula <jani.nikula@intel.com> 18 - # Jan Stancek <jstancek@redhat.com> 19 - # Jonathan Corbet <corbet@lwn.net> 20 - # Joshua Clayton <stillcompiling@gmail.com> 21 - # Kees Cook <keescook@chromium.org> 22 - # Linus Torvalds <torvalds@linux-foundation.org> 23 - # Magnus Damm <damm+renesas@opensource.se> 24 - # Masahiro Yamada <masahiroy@kernel.org> 25 - # Mauro Carvalho Chehab <mchehab+huawei@kernel.org> 26 - # Maxim Cournoyer <maxim.cournoyer@gmail.com> 27 - # Peter Foley <pefoley2@pefoley.com> 28 - # Randy Dunlap <rdunlap@infradead.org> 29 - # Rob Herring <robh@kernel.org> 30 - # Shuah Khan <shuahkh@osg.samsung.com> 31 - # Thorsten Blum <thorsten.blum@toblux.com> 32 - # Tomas Winkler <tomas.winkler@intel.com> 33 - 34 - 35 - """ 36 - Sphinx build wrapper that handles Kernel-specific business rules: 37 - 38 - - it gets the Kernel build environment vars; 39 - - it determines what's the best parallelism; 40 - - it handles SPHINXDIRS 41 - 42 - This tool ensures that MIN_PYTHON_VERSION is satisfied. If version is 43 - below that, it seeks for a new Python version. If found, it re-runs using 44 - the newer version. 45 - """ 46 - 47 - import argparse 48 - import locale 49 - import os 50 - import re 51 - import shlex 52 - import shutil 53 - import subprocess 54 - import sys 55 - 56 - from concurrent import futures 57 - from glob import glob 58 - 59 - LIB_DIR = "lib" 60 - SRC_DIR = os.path.dirname(os.path.realpath(__file__)) 61 - 62 - sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR)) 63 - 64 - from jobserver import JobserverExec # pylint: disable=C0413 65 - 66 - 67 - def parse_version(version): 68 - """Convert a major.minor.patch version into a tuple""" 69 - return tuple(int(x) for x in version.split(".")) 70 - 71 - def ver_str(version): 72 - """Returns a version tuple as major.minor.patch""" 73 - 74 - return ".".join([str(x) for x in version]) 75 - 76 - # Minimal supported Python version needed by Sphinx and its extensions 77 - MIN_PYTHON_VERSION = parse_version("3.7") 78 - 79 - # Default value for --venv parameter 80 - VENV_DEFAULT = "sphinx_latest" 81 - 82 - # List of make targets and its corresponding builder and output directory 83 - TARGETS = { 84 - "cleandocs": { 85 - "builder": "clean", 86 - }, 87 - "htmldocs": { 88 - "builder": "html", 89 - }, 90 - "epubdocs": { 91 - "builder": "epub", 92 - "out_dir": "epub", 93 - }, 94 - "texinfodocs": { 95 - "builder": "texinfo", 96 - "out_dir": "texinfo", 97 - }, 98 - "infodocs": { 99 - "builder": "texinfo", 100 - "out_dir": "texinfo", 101 - }, 102 - "latexdocs": { 103 - "builder": "latex", 104 - "out_dir": "latex", 105 - }, 106 - "pdfdocs": { 107 - "builder": "latex", 108 - "out_dir": "latex", 109 - }, 110 - "xmldocs": { 111 - "builder": "xml", 112 - "out_dir": "xml", 113 - }, 114 - "linkcheckdocs": { 115 - "builder": "linkcheck" 116 - }, 117 - } 118 - 119 - # Paper sizes. An empty value will pick the default 120 - PAPER = ["", "a4", "letter"] 121 - 122 - class SphinxBuilder: 123 - """ 124 - Handles a sphinx-build target, adding needed arguments to build 125 - with the Kernel. 126 - """ 127 - 128 - def is_rust_enabled(self): 129 - """Check if rust is enabled at .config""" 130 - config_path = os.path.join(self.srctree, ".config") 131 - if os.path.isfile(config_path): 132 - with open(config_path, "r", encoding="utf-8") as f: 133 - return "CONFIG_RUST=y" in f.read() 134 - return False 135 - 136 - def get_path(self, path, abs_path=False): 137 - """ 138 - Ancillary routine to handle patches the right way, as shell does. 139 - 140 - It first expands "~" and "~user". Then, if patch is not absolute, 141 - join self.srctree. Finally, if requested, convert to abspath. 142 - """ 143 - 144 - path = os.path.expanduser(path) 145 - if not path.startswith("/"): 146 - path = os.path.join(self.srctree, path) 147 - 148 - if abs_path: 149 - return os.path.abspath(path) 150 - 151 - return path 152 - 153 - def __init__(self, venv=None, verbose=False, n_jobs=None, interactive=None): 154 - """Initialize internal variables""" 155 - self.venv = venv 156 - self.verbose = None 157 - 158 - # Normal variables passed from Kernel's makefile 159 - self.kernelversion = os.environ.get("KERNELVERSION", "unknown") 160 - self.kernelrelease = os.environ.get("KERNELRELEASE", "unknown") 161 - self.pdflatex = os.environ.get("PDFLATEX", "xelatex") 162 - 163 - if not interactive: 164 - self.latexopts = os.environ.get("LATEXOPTS", "-interaction=batchmode -no-shell-escape") 165 - else: 166 - self.latexopts = os.environ.get("LATEXOPTS", "") 167 - 168 - if not verbose: 169 - verbose = bool(os.environ.get("KBUILD_VERBOSE", "") != "") 170 - 171 - # Handle SPHINXOPTS evironment 172 - sphinxopts = shlex.split(os.environ.get("SPHINXOPTS", "")) 173 - 174 - # As we handle number of jobs and quiet in separate, we need to pick 175 - # it the same way as sphinx-build would pick, so let's use argparse 176 - # do to the right argument expansion 177 - parser = argparse.ArgumentParser() 178 - parser.add_argument('-j', '--jobs', type=int) 179 - parser.add_argument('-q', '--quiet', type=int) 180 - 181 - # Other sphinx-build arguments go as-is, so place them 182 - # at self.sphinxopts 183 - sphinx_args, self.sphinxopts = parser.parse_known_args(sphinxopts) 184 - if sphinx_args.quiet == True: 185 - self.verbose = False 186 - 187 - if sphinx_args.jobs: 188 - self.n_jobs = sphinx_args.jobs 189 - 190 - # Command line arguments was passed, override SPHINXOPTS 191 - if verbose is not None: 192 - self.verbose = verbose 193 - 194 - self.n_jobs = n_jobs 195 - 196 - # Source tree directory. This needs to be at os.environ, as 197 - # Sphinx extensions and media uAPI makefile needs it 198 - self.srctree = os.environ.get("srctree") 199 - if not self.srctree: 200 - self.srctree = "." 201 - os.environ["srctree"] = self.srctree 202 - 203 - # Now that we can expand srctree, get other directories as well 204 - self.sphinxbuild = os.environ.get("SPHINXBUILD", "sphinx-build") 205 - self.kerneldoc = self.get_path(os.environ.get("KERNELDOC", 206 - "scripts/kernel-doc.py")) 207 - self.obj = os.environ.get("obj", "Documentation") 208 - self.builddir = self.get_path(os.path.join(self.obj, "output"), 209 - abs_path=True) 210 - 211 - # Media uAPI needs it 212 - os.environ["BUILDDIR"] = self.builddir 213 - 214 - # Detect if rust is enabled 215 - self.config_rust = self.is_rust_enabled() 216 - 217 - # Get directory locations for LaTeX build toolchain 218 - self.pdflatex_cmd = shutil.which(self.pdflatex) 219 - self.latexmk_cmd = shutil.which("latexmk") 220 - 221 - self.env = os.environ.copy() 222 - 223 - # If venv parameter is specified, run Sphinx from venv 224 - if venv: 225 - bin_dir = os.path.join(venv, "bin") 226 - if os.path.isfile(os.path.join(bin_dir, "activate")): 227 - # "activate" virtual env 228 - self.env["PATH"] = bin_dir + ":" + self.env["PATH"] 229 - self.env["VIRTUAL_ENV"] = venv 230 - if "PYTHONHOME" in self.env: 231 - del self.env["PYTHONHOME"] 232 - print(f"Setting venv to {venv}") 233 - else: 234 - sys.exit(f"Venv {venv} not found.") 235 - 236 - def run_sphinx(self, sphinx_build, build_args, *args, **pwargs): 237 - """ 238 - Executes sphinx-build using current python3 command and setting 239 - -j parameter if possible to run the build in parallel. 240 - """ 241 - 242 - with JobserverExec() as jobserver: 243 - if jobserver.claim: 244 - n_jobs = str(jobserver.claim) 245 - else: 246 - n_jobs = "auto" # Supported since Sphinx 1.7 247 - 248 - cmd = [] 249 - 250 - if self.venv: 251 - cmd.append("python") 252 - else: 253 - cmd.append(sys.executable) 254 - 255 - cmd.append(sphinx_build) 256 - 257 - # if present, SPHINXOPTS or command line --jobs overrides default 258 - if self.n_jobs: 259 - n_jobs = str(self.n_jobs) 260 - 261 - if n_jobs: 262 - cmd += [f"-j{n_jobs}"] 263 - 264 - if not self.verbose: 265 - cmd.append("-q") 266 - 267 - cmd += self.sphinxopts 268 - 269 - cmd += build_args 270 - 271 - if self.verbose: 272 - print(" ".join(cmd)) 273 - 274 - rc = subprocess.call(cmd, *args, **pwargs) 275 - 276 - def handle_html(self, css, output_dir): 277 - """ 278 - Extra steps for HTML and epub output. 279 - 280 - For such targets, we need to ensure that CSS will be properly 281 - copied to the output _static directory 282 - """ 283 - 284 - if not css: 285 - return 286 - 287 - css = os.path.expanduser(css) 288 - if not css.startswith("/"): 289 - css = os.path.join(self.srctree, css) 290 - 291 - static_dir = os.path.join(output_dir, "_static") 292 - os.makedirs(static_dir, exist_ok=True) 293 - 294 - try: 295 - shutil.copy2(css, static_dir) 296 - except (OSError, IOError) as e: 297 - print(f"Warning: Failed to copy CSS: {e}", file=sys.stderr) 298 - 299 - def build_pdf_file(self, latex_cmd, from_dir, path): 300 - """Builds a single pdf file using latex_cmd""" 301 - try: 302 - subprocess.run(latex_cmd + [path], 303 - cwd=from_dir, check=True) 304 - 305 - return True 306 - except subprocess.CalledProcessError: 307 - # LaTeX PDF error code is almost useless: it returns 308 - # error codes even when build succeeds but has warnings. 309 - # So, we'll ignore the results 310 - return False 311 - 312 - def pdf_parallel_build(self, tex_suffix, latex_cmd, tex_files, n_jobs): 313 - """Build PDF files in parallel if possible""" 314 - builds = {} 315 - build_failed = False 316 - max_len = 0 317 - has_tex = False 318 - 319 - # Process files in parallel 320 - with futures.ThreadPoolExecutor(max_workers=n_jobs) as executor: 321 - jobs = {} 322 - 323 - for from_dir, pdf_dir, entry in tex_files: 324 - name = entry.name 325 - 326 - if not name.endswith(tex_suffix): 327 - continue 328 - 329 - name = name[:-len(tex_suffix)] 330 - 331 - max_len = max(max_len, len(name)) 332 - 333 - has_tex = True 334 - 335 - future = executor.submit(self.build_pdf_file, latex_cmd, 336 - from_dir, entry.path) 337 - jobs[future] = (from_dir, name, entry.path) 338 - 339 - for future in futures.as_completed(jobs): 340 - from_dir, name, path = jobs[future] 341 - 342 - pdf_name = name + ".pdf" 343 - pdf_from = os.path.join(from_dir, pdf_name) 344 - 345 - try: 346 - success = future.result() 347 - 348 - if success and os.path.exists(pdf_from): 349 - pdf_to = os.path.join(pdf_dir, pdf_name) 350 - 351 - os.rename(pdf_from, pdf_to) 352 - builds[name] = os.path.relpath(pdf_to, self.builddir) 353 - else: 354 - builds[name] = "FAILED" 355 - build_failed = True 356 - except Exception as e: 357 - builds[name] = f"FAILED ({str(e)})" 358 - build_failed = True 359 - 360 - # Handle case where no .tex files were found 361 - if not has_tex: 362 - name = "Sphinx LaTeX builder" 363 - max_len = max(max_len, len(name)) 364 - builds[name] = "FAILED (no .tex file was generated)" 365 - build_failed = True 366 - 367 - return builds, build_failed, max_len 368 - 369 - def handle_pdf(self, output_dirs): 370 - """ 371 - Extra steps for PDF output. 372 - 373 - As PDF is handled via a LaTeX output, after building the .tex file, 374 - a new build is needed to create the PDF output from the latex 375 - directory. 376 - """ 377 - builds = {} 378 - max_len = 0 379 - tex_suffix = ".tex" 380 - 381 - # Get all tex files that will be used for PDF build 382 - tex_files = [] 383 - for from_dir in output_dirs: 384 - pdf_dir = os.path.join(from_dir, "../pdf") 385 - os.makedirs(pdf_dir, exist_ok=True) 386 - 387 - if self.latexmk_cmd: 388 - latex_cmd = [self.latexmk_cmd, f"-{self.pdflatex}"] 389 - else: 390 - latex_cmd = [self.pdflatex] 391 - 392 - latex_cmd.extend(shlex.split(self.latexopts)) 393 - 394 - # Get a list of tex files to process 395 - with os.scandir(from_dir) as it: 396 - for entry in it: 397 - if entry.name.endswith(tex_suffix): 398 - tex_files.append((from_dir, pdf_dir, entry)) 399 - 400 - # When using make, this won't be used, as the number of jobs comes 401 - # from POSIX jobserver. So, this covers the case where build comes 402 - # from command line. On such case, serialize by default, except if 403 - # the user explicitly sets the number of jobs. 404 - n_jobs = 1 405 - 406 - # n_jobs is either an integer or "auto". Only use it if it is a number 407 - if self.n_jobs: 408 - try: 409 - n_jobs = int(self.n_jobs) 410 - except ValueError: 411 - pass 412 - 413 - # When using make, jobserver.claim is the number of jobs that were 414 - # used with "-j" and that aren't used by other make targets 415 - with JobserverExec() as jobserver: 416 - n_jobs = 1 417 - 418 - # Handle the case when a parameter is passed via command line, 419 - # using it as default, if jobserver doesn't claim anything 420 - if self.n_jobs: 421 - try: 422 - n_jobs = int(self.n_jobs) 423 - except ValueError: 424 - pass 425 - 426 - if jobserver.claim: 427 - n_jobs = jobserver.claim 428 - 429 - # Build files in parallel 430 - builds, build_failed, max_len = self.pdf_parallel_build(tex_suffix, 431 - latex_cmd, 432 - tex_files, 433 - n_jobs) 434 - 435 - msg = "Summary" 436 - msg += "\n" + "=" * len(msg) 437 - print() 438 - print(msg) 439 - 440 - for pdf_name, pdf_file in builds.items(): 441 - print(f"{pdf_name:<{max_len}}: {pdf_file}") 442 - 443 - print() 444 - 445 - # return an error if a PDF file is missing 446 - 447 - if build_failed: 448 - sys.exit(f"PDF build failed: not all PDF files were created.") 449 - else: 450 - print("All PDF files were built.") 451 - 452 - def handle_info(self, output_dirs): 453 - """ 454 - Extra steps for Info output. 455 - 456 - For texinfo generation, an additional make is needed from the 457 - texinfo directory. 458 - """ 459 - 460 - for output_dir in output_dirs: 461 - try: 462 - subprocess.run(["make", "info"], cwd=output_dir, check=True) 463 - except subprocess.CalledProcessError as e: 464 - sys.exit(f"Error generating info docs: {e}") 465 - 466 - def cleandocs(self, builder): 467 - 468 - shutil.rmtree(self.builddir, ignore_errors=True) 469 - 470 - def build(self, target, sphinxdirs=None, conf="conf.py", 471 - theme=None, css=None, paper=None): 472 - """ 473 - Build documentation using Sphinx. This is the core function of this 474 - module. It prepares all arguments required by sphinx-build. 475 - """ 476 - 477 - builder = TARGETS[target]["builder"] 478 - out_dir = TARGETS[target].get("out_dir", "") 479 - 480 - # Cleandocs doesn't require sphinx-build 481 - if target == "cleandocs": 482 - self.cleandocs(builder) 483 - return 484 - 485 - # Other targets require sphinx-build 486 - sphinxbuild = shutil.which(self.sphinxbuild, path=self.env["PATH"]) 487 - if not sphinxbuild: 488 - sys.exit(f"Error: {self.sphinxbuild} not found in PATH.\n") 489 - 490 - if builder == "latex": 491 - if not self.pdflatex_cmd and not self.latexmk_cmd: 492 - sys.exit("Error: pdflatex or latexmk required for PDF generation") 493 - 494 - docs_dir = os.path.abspath(os.path.join(self.srctree, "Documentation")) 495 - 496 - # Prepare base arguments for Sphinx build 497 - kerneldoc = self.kerneldoc 498 - if kerneldoc.startswith(self.srctree): 499 - kerneldoc = os.path.relpath(kerneldoc, self.srctree) 500 - 501 - # Prepare common Sphinx options 502 - args = [ 503 - "-b", builder, 504 - "-c", docs_dir, 505 - ] 506 - 507 - if builder == "latex": 508 - if not paper: 509 - paper = PAPER[1] 510 - 511 - args.extend(["-D", f"latex_elements.papersize={paper}paper"]) 512 - 513 - if self.config_rust: 514 - args.extend(["-t", "rustdoc"]) 515 - 516 - if conf: 517 - self.env["SPHINX_CONF"] = self.get_path(conf, abs_path=True) 518 - 519 - if not sphinxdirs: 520 - sphinxdirs = os.environ.get("SPHINXDIRS", ".") 521 - 522 - # The sphinx-build tool has a bug: internally, it tries to set 523 - # locale with locale.setlocale(locale.LC_ALL, ''). This causes a 524 - # crash if language is not set. Detect and fix it. 525 - try: 526 - locale.setlocale(locale.LC_ALL, '') 527 - except Exception: 528 - self.env["LC_ALL"] = "C" 529 - self.env["LANG"] = "C" 530 - 531 - # sphinxdirs can be a list or a whitespace-separated string 532 - sphinxdirs_list = [] 533 - for sphinxdir in sphinxdirs: 534 - if isinstance(sphinxdir, list): 535 - sphinxdirs_list += sphinxdir 536 - else: 537 - for name in sphinxdir.split(" "): 538 - sphinxdirs_list.append(name) 539 - 540 - # Build each directory 541 - output_dirs = [] 542 - for sphinxdir in sphinxdirs_list: 543 - src_dir = os.path.join(docs_dir, sphinxdir) 544 - doctree_dir = os.path.join(self.builddir, ".doctrees") 545 - output_dir = os.path.join(self.builddir, sphinxdir, out_dir) 546 - 547 - # Make directory names canonical 548 - src_dir = os.path.normpath(src_dir) 549 - doctree_dir = os.path.normpath(doctree_dir) 550 - output_dir = os.path.normpath(output_dir) 551 - 552 - os.makedirs(doctree_dir, exist_ok=True) 553 - os.makedirs(output_dir, exist_ok=True) 554 - 555 - output_dirs.append(output_dir) 556 - 557 - build_args = args + [ 558 - "-d", doctree_dir, 559 - "-D", f"kerneldoc_bin={kerneldoc}", 560 - "-D", f"version={self.kernelversion}", 561 - "-D", f"release={self.kernelrelease}", 562 - "-D", f"kerneldoc_srctree={self.srctree}", 563 - src_dir, 564 - output_dir, 565 - ] 566 - 567 - # Execute sphinx-build 568 - try: 569 - self.run_sphinx(sphinxbuild, build_args, env=self.env) 570 - except Exception as e: 571 - sys.exit(f"Build failed: {e}") 572 - 573 - # Ensure that html/epub will have needed static files 574 - if target in ["htmldocs", "epubdocs"]: 575 - self.handle_html(css, output_dir) 576 - 577 - # PDF and Info require a second build step 578 - if target == "pdfdocs": 579 - self.handle_pdf(output_dirs) 580 - elif target == "infodocs": 581 - self.handle_info(output_dirs) 582 - 583 - @staticmethod 584 - def get_python_version(cmd): 585 - """ 586 - Get python version from a Python binary. As we need to detect if 587 - are out there newer python binaries, we can't rely on sys.release here. 588 - """ 589 - 590 - result = subprocess.run([cmd, "--version"], check=True, 591 - stdout=subprocess.PIPE, stderr=subprocess.PIPE, 592 - universal_newlines=True) 593 - version = result.stdout.strip() 594 - 595 - match = re.search(r"(\d+\.\d+\.\d+)", version) 596 - if match: 597 - return parse_version(match.group(1)) 598 - 599 - print(f"Can't parse version {version}") 600 - return (0, 0, 0) 601 - 602 - @staticmethod 603 - def find_python(): 604 - """ 605 - Detect if are out there any python 3.xy version newer than the 606 - current one. 607 - 608 - Note: this routine is limited to up to 2 digits for python3. We 609 - may need to update it one day, hopefully on a distant future. 610 - """ 611 - patterns = [ 612 - "python3.[0-9]", 613 - "python3.[0-9][0-9]", 614 - ] 615 - 616 - # Seek for a python binary newer than MIN_PYTHON_VERSION 617 - for path in os.getenv("PATH", "").split(":"): 618 - for pattern in patterns: 619 - for cmd in glob(os.path.join(path, pattern)): 620 - if os.path.isfile(cmd) and os.access(cmd, os.X_OK): 621 - version = SphinxBuilder.get_python_version(cmd) 622 - if version >= MIN_PYTHON_VERSION: 623 - return cmd 624 - 625 - return None 626 - 627 - @staticmethod 628 - def check_python(): 629 - """ 630 - Check if the current python binary satisfies our minimal requirement 631 - for Sphinx build. If not, re-run with a newer version if found. 632 - """ 633 - cur_ver = sys.version_info[:3] 634 - if cur_ver >= MIN_PYTHON_VERSION: 635 - return 636 - 637 - python_ver = ver_str(cur_ver) 638 - 639 - new_python_cmd = SphinxBuilder.find_python() 640 - if not new_python_cmd: 641 - sys.exit(f"Python version {python_ver} is not supported anymore.") 642 - 643 - # Restart script using the newer version 644 - script_path = os.path.abspath(sys.argv[0]) 645 - args = [new_python_cmd, script_path] + sys.argv[1:] 646 - 647 - print(f"Python {python_ver} not supported. Changing to {new_python_cmd}") 648 - 649 - try: 650 - os.execv(new_python_cmd, args) 651 - except OSError as e: 652 - sys.exit(f"Failed to restart with {new_python_cmd}: {e}") 653 - 654 - def jobs_type(value): 655 - """ 656 - Handle valid values for -j. Accepts Sphinx "-jauto", plus a number 657 - equal or bigger than one. 658 - """ 659 - if value is None: 660 - return None 661 - 662 - if value.lower() == 'auto': 663 - return value.lower() 664 - 665 - try: 666 - if int(value) >= 1: 667 - return value 668 - 669 - raise argparse.ArgumentTypeError(f"Minimum jobs is 1, got {value}") 670 - except ValueError: 671 - raise argparse.ArgumentTypeError(f"Must be 'auto' or positive integer, got {value}") 672 - 673 - def main(): 674 - """ 675 - Main function. The only mandatory argument is the target. If not 676 - specified, the other arguments will use default values if not 677 - specified at os.environ. 678 - """ 679 - parser = argparse.ArgumentParser(description="Kernel documentation builder") 680 - 681 - parser.add_argument("target", choices=list(TARGETS.keys()), 682 - help="Documentation target to build") 683 - parser.add_argument("--sphinxdirs", nargs="+", 684 - help="Specific directories to build") 685 - parser.add_argument("--conf", default="conf.py", 686 - help="Sphinx configuration file") 687 - 688 - parser.add_argument("--theme", help="Sphinx theme to use") 689 - 690 - parser.add_argument("--css", help="Custom CSS file for HTML/EPUB") 691 - 692 - parser.add_argument("--paper", choices=PAPER, default=PAPER[0], 693 - help="Paper size for LaTeX/PDF output") 694 - 695 - parser.add_argument("-v", "--verbose", action='store_true', 696 - help="place build in verbose mode") 697 - 698 - parser.add_argument('-j', '--jobs', type=jobs_type, 699 - help="Sets number of jobs to use with sphinx-build") 700 - 701 - parser.add_argument('-i', '--interactive', action='store_true', 702 - help="Change latex default to run in interactive mode") 703 - 704 - parser.add_argument("-V", "--venv", nargs='?', const=f'{VENV_DEFAULT}', 705 - default=None, 706 - help=f'If used, run Sphinx from a venv dir (default dir: {VENV_DEFAULT})') 707 - 708 - args = parser.parse_args() 709 - 710 - SphinxBuilder.check_python() 711 - 712 - builder = SphinxBuilder(venv=args.venv, verbose=args.verbose, 713 - n_jobs=args.jobs, interactive=args.interactive) 714 - 715 - builder.build(args.target, sphinxdirs=args.sphinxdirs, conf=args.conf, 716 - theme=args.theme, css=args.css, paper=args.paper) 717 - 718 - if __name__ == "__main__": 719 - main()
+27 -108
scripts/sphinx-pre-install tools/docs/sphinx-pre-install
··· 26 26 """ 27 27 28 28 import argparse 29 + import locale 29 30 import os 30 31 import re 31 32 import subprocess 32 33 import sys 33 34 from glob import glob 34 35 36 + from lib.python_version import PythonVersion 35 37 36 - def parse_version(version): 37 - """Convert a major.minor.patch version into a tuple""" 38 - return tuple(int(x) for x in version.split(".")) 39 - 40 - 41 - def ver_str(version): 42 - """Returns a version tuple as major.minor.patch""" 43 - 44 - return ".".join([str(x) for x in version]) 45 - 46 - 47 - RECOMMENDED_VERSION = parse_version("3.4.3") 48 - MIN_PYTHON_VERSION = parse_version("3.7") 38 + RECOMMENDED_VERSION = PythonVersion("3.4.3").version 39 + MIN_PYTHON_VERSION = PythonVersion("3.7").version 49 40 50 41 51 42 class DepManager: ··· 227 236 return None 228 237 229 238 @staticmethod 230 - def get_python_version(cmd): 231 - """ 232 - Get python version from a Python binary. As we need to detect if 233 - are out there newer python binaries, we can't rely on sys.release here. 234 - """ 235 - 236 - result = SphinxDependencyChecker.run([cmd, "--version"], 237 - capture_output=True, text=True) 238 - version = result.stdout.strip() 239 - 240 - match = re.search(r"(\d+\.\d+\.\d+)", version) 241 - if match: 242 - return parse_version(match.group(1)) 243 - 244 - print(f"Can't parse version {version}") 245 - return (0, 0, 0) 246 - 247 - @staticmethod 248 - def find_python(): 249 - """ 250 - Detect if are out there any python 3.xy version newer than the 251 - current one. 252 - 253 - Note: this routine is limited to up to 2 digits for python3. We 254 - may need to update it one day, hopefully on a distant future. 255 - """ 256 - patterns = [ 257 - "python3.[0-9]", 258 - "python3.[0-9][0-9]", 259 - ] 260 - 261 - # Seek for a python binary newer than MIN_PYTHON_VERSION 262 - for path in os.getenv("PATH", "").split(":"): 263 - for pattern in patterns: 264 - for cmd in glob(os.path.join(path, pattern)): 265 - if os.path.isfile(cmd) and os.access(cmd, os.X_OK): 266 - version = SphinxDependencyChecker.get_python_version(cmd) 267 - if version >= MIN_PYTHON_VERSION: 268 - return cmd 269 - 270 - @staticmethod 271 - def check_python(): 272 - """ 273 - Check if the current python binary satisfies our minimal requirement 274 - for Sphinx build. If not, re-run with a newer version if found. 275 - """ 276 - cur_ver = sys.version_info[:3] 277 - if cur_ver >= MIN_PYTHON_VERSION: 278 - ver = ver_str(cur_ver) 279 - print(f"Python version: {ver}") 280 - 281 - # This could be useful for debugging purposes 282 - if SphinxDependencyChecker.which("docutils"): 283 - result = SphinxDependencyChecker.run(["docutils", "--version"], 284 - capture_output=True, text=True) 285 - ver = result.stdout.strip() 286 - match = re.search(r"(\d+\.\d+\.\d+)", ver) 287 - if match: 288 - ver = match.group(1) 289 - 290 - print(f"Docutils version: {ver}") 291 - 292 - return 293 - 294 - python_ver = ver_str(cur_ver) 295 - 296 - new_python_cmd = SphinxDependencyChecker.find_python() 297 - if not new_python_cmd: 298 - print(f"ERROR: Python version {python_ver} is not spported anymore\n") 299 - print(" Can't find a new version. This script may fail") 300 - return 301 - 302 - # Restart script using the newer version 303 - script_path = os.path.abspath(sys.argv[0]) 304 - args = [new_python_cmd, script_path] + sys.argv[1:] 305 - 306 - print(f"Python {python_ver} not supported. Changing to {new_python_cmd}") 307 - 308 - try: 309 - os.execv(new_python_cmd, args) 310 - except OSError as e: 311 - sys.exit(f"Failed to restart with {new_python_cmd}: {e}") 312 - 313 - @staticmethod 314 239 def run(*args, **kwargs): 315 240 """ 316 241 Excecute a command, hiding its output by default. 317 - Preserve comatibility with older Python versions. 242 + Preserve compatibility with older Python versions. 318 243 """ 319 244 320 245 capture_output = kwargs.pop('capture_output', False) ··· 423 516 """ 424 517 Gets sphinx-build version. 425 518 """ 519 + env = os.environ.copy() 520 + 521 + # The sphinx-build tool has a bug: internally, it tries to set 522 + # locale with locale.setlocale(locale.LC_ALL, ''). This causes a 523 + # crash if language is not set. Detect and fix it. 426 524 try: 427 - result = self.run([cmd, "--version"], 525 + locale.setlocale(locale.LC_ALL, '') 526 + except Exception: 527 + env["LC_ALL"] = "C" 528 + env["LANG"] = "C" 529 + 530 + try: 531 + result = self.run([cmd, "--version"], env=env, 428 532 stdout=subprocess.PIPE, 429 533 stderr=subprocess.STDOUT, 430 534 text=True, check=True) ··· 445 527 for line in result.stdout.split("\n"): 446 528 match = re.match(r"^sphinx-build\s+([\d\.]+)(?:\+(?:/[\da-f]+)|b\d+)?\s*$", line) 447 529 if match: 448 - return parse_version(match.group(1)) 530 + return PythonVersion.parse_version(match.group(1)) 449 531 450 532 match = re.match(r"^Sphinx.*\s+([\d\.]+)\s*$", line) 451 533 if match: 452 - return parse_version(match.group(1)) 534 + return PythonVersion.parse_version(match.group(1)) 453 535 454 536 def check_sphinx(self, conf): 455 537 """ ··· 460 542 for line in f: 461 543 match = re.match(r"^\s*needs_sphinx\s*=\s*[\'\"]([\d\.]+)[\'\"]", line) 462 544 if match: 463 - self.min_version = parse_version(match.group(1)) 545 + self.min_version = PythonVersion.parse_version(match.group(1)) 464 546 break 465 547 except IOError: 466 548 sys.exit(f"Can't open {conf}") ··· 480 562 sys.exit(f"{sphinx} didn't return its version") 481 563 482 564 if self.cur_version < self.min_version: 483 - curver = ver_str(self.cur_version) 484 - minver = ver_str(self.min_version) 565 + curver = PythonVersion.ver_str(self.cur_version) 566 + minver = PythonVersion.ver_str(self.min_version) 485 567 486 568 print(f"ERROR: Sphinx version is {curver}. It should be >= {minver}") 487 569 self.need_sphinx = 1 ··· 1222 1304 else: 1223 1305 if self.need_sphinx and ver >= self.min_version: 1224 1306 return (f, ver) 1225 - elif parse_version(ver) > self.cur_version: 1307 + elif PythonVersion.parse_version(ver) > self.cur_version: 1226 1308 return (f, ver) 1227 1309 1228 1310 return ("", ver) ··· 1329 1411 return 1330 1412 1331 1413 if self.latest_avail_ver: 1332 - latest_avail_ver = ver_str(self.latest_avail_ver) 1414 + latest_avail_ver = PythonVersion.ver_str(self.latest_avail_ver) 1333 1415 1334 1416 if not self.need_sphinx: 1335 1417 # sphinx-build is present and its version is >= $min_version ··· 1425 1507 else: 1426 1508 print("Unknown OS") 1427 1509 if self.cur_version != (0, 0, 0): 1428 - ver = ver_str(self.cur_version) 1510 + ver = PythonVersion.ver_str(self.cur_version) 1429 1511 print(f"Sphinx version: {ver}\n") 1430 1512 1431 1513 # Check the type of virtual env, depending on Python version ··· 1531 1613 1532 1614 checker = SphinxDependencyChecker(args) 1533 1615 1534 - checker.check_python() 1616 + PythonVersion.check_python(MIN_PYTHON_VERSION, 1617 + bail_out=True, success_on_error=True) 1535 1618 checker.check_needs() 1536 1619 1537 1620 # Call main if not used as module
-28
scripts/split-man.pl
··· 1 - #!/usr/bin/env perl 2 - # SPDX-License-Identifier: GPL-2.0 3 - # 4 - # Author: Mauro Carvalho Chehab <mchehab+samsung@kernel.org> 5 - # 6 - # Produce manpages from kernel-doc. 7 - # See Documentation/doc-guide/kernel-doc.rst for instructions 8 - 9 - if ($#ARGV < 0) { 10 - die "where do I put the results?\n"; 11 - } 12 - 13 - mkdir $ARGV[0],0777; 14 - $state = 0; 15 - while (<STDIN>) { 16 - if (/^\.TH \"[^\"]*\" 9 \"([^\"]*)\"/) { 17 - if ($state == 1) { close OUT } 18 - $state = 1; 19 - $fn = "$ARGV[0]/$1.9"; 20 - print STDERR "Creating $fn\n"; 21 - open OUT, ">$fn" or die "can't open $fn: $!\n"; 22 - print OUT $_; 23 - } elsif ($state != 0) { 24 - print OUT $_; 25 - } 26 - } 27 - 28 - close OUT;
+33
tools/docs/check-variable-fonts.py
··· 1 + #!/usr/bin/env python3 2 + # SPDX-License-Identifier: GPL-2.0-only 3 + # Copyright (C) Akira Yokosawa, 2024 4 + # 5 + # Ported to Python by (c) Mauro Carvalho Chehab, 2025 6 + # 7 + # pylint: disable=C0103 8 + 9 + """ 10 + Detect problematic Noto CJK variable fonts. 11 + 12 + or more details, see lib/latex_fonts.py. 13 + """ 14 + 15 + import argparse 16 + import sys 17 + 18 + from lib.latex_fonts import LatexFontChecker 19 + 20 + checker = LatexFontChecker() 21 + 22 + parser=argparse.ArgumentParser(description=checker.description(), 23 + formatter_class=argparse.RawTextHelpFormatter) 24 + parser.add_argument("--deny-vf", 25 + help="XDG_CONFIG_HOME dir containing fontconfig/fonts.conf file") 26 + 27 + args=parser.parse_args() 28 + 29 + msg = LatexFontChecker(args.deny_vf).check() 30 + if msg: 31 + print(msg) 32 + 33 + sys.exit(1)
+167
tools/docs/lib/latex_fonts.py
··· 1 + #!/usr/bin/env python3 2 + # SPDX-License-Identifier: GPL-2.0-only 3 + # Copyright (C) Akira Yokosawa, 2024 4 + # 5 + # Ported to Python by (c) Mauro Carvalho Chehab, 2025 6 + 7 + """ 8 + Detect problematic Noto CJK variable fonts. 9 + 10 + For "make pdfdocs", reports of build errors of translations.pdf started 11 + arriving early 2024 [1, 2]. It turned out that Fedora and openSUSE 12 + tumbleweed have started deploying variable-font [3] format of "Noto CJK" 13 + fonts [4, 5]. For PDF, a LaTeX package named xeCJK is used for CJK 14 + (Chinese, Japanese, Korean) pages. xeCJK requires XeLaTeX/XeTeX, which 15 + does not (and likely never will) understand variable fonts for historical 16 + reasons. 17 + 18 + The build error happens even when both of variable- and non-variable-format 19 + fonts are found on the build system. To make matters worse, Fedora enlists 20 + variable "Noto CJK" fonts in the requirements of langpacks-ja, -ko, -zh_CN, 21 + -zh_TW, etc. Hence developers who have interest in CJK pages are more 22 + likely to encounter the build errors. 23 + 24 + This script is invoked from the error path of "make pdfdocs" and emits 25 + suggestions if variable-font files of "Noto CJK" fonts are in the list of 26 + fonts accessible from XeTeX. 27 + 28 + References: 29 + [1]: https://lore.kernel.org/r/8734tqsrt7.fsf@meer.lwn.net/ 30 + [2]: https://lore.kernel.org/r/1708585803.600323099@f111.i.mail.ru/ 31 + [3]: https://en.wikipedia.org/wiki/Variable_font 32 + [4]: https://fedoraproject.org/wiki/Changes/Noto_CJK_Variable_Fonts 33 + [5]: https://build.opensuse.org/request/show/1157217 34 + 35 + #=========================================================================== 36 + Workarounds for building translations.pdf 37 + #=========================================================================== 38 + 39 + * Denylist "variable font" Noto CJK fonts. 40 + - Create $HOME/deny-vf/fontconfig/fonts.conf from template below, with 41 + tweaks if necessary. Remove leading "". 42 + - Path of fontconfig/fonts.conf can be overridden by setting an env 43 + variable FONTS_CONF_DENY_VF. 44 + 45 + * Template: 46 + ----------------------------------------------------------------- 47 + <?xml version="1.0"?> 48 + <!DOCTYPE fontconfig SYSTEM "urn:fontconfig:fonts.dtd"> 49 + <fontconfig> 50 + <!-- 51 + Ignore variable-font glob (not to break xetex) 52 + --> 53 + <selectfont> 54 + <rejectfont> 55 + <!-- 56 + for Fedora 57 + --> 58 + <glob>/usr/share/fonts/google-noto-*-cjk-vf-fonts</glob> 59 + <!-- 60 + for openSUSE tumbleweed 61 + --> 62 + <glob>/usr/share/fonts/truetype/Noto*CJK*-VF.otf</glob> 63 + </rejectfont> 64 + </selectfont> 65 + </fontconfig> 66 + ----------------------------------------------------------------- 67 + 68 + The denylisting is activated for "make pdfdocs". 69 + 70 + * For skipping CJK pages in PDF 71 + - Uninstall texlive-xecjk. 72 + Denylisting is not needed in this case. 73 + 74 + * For printing CJK pages in PDF 75 + - Need non-variable "Noto CJK" fonts. 76 + * Fedora 77 + - google-noto-sans-cjk-fonts 78 + - google-noto-serif-cjk-fonts 79 + * openSUSE tumbleweed 80 + - Non-variable "Noto CJK" fonts are not available as distro packages 81 + as of April, 2024. Fetch a set of font files from upstream Noto 82 + CJK Font released at: 83 + https://github.com/notofonts/noto-cjk/tree/main/Sans#super-otc 84 + and at: 85 + https://github.com/notofonts/noto-cjk/tree/main/Serif#super-otc 86 + , then uncompress and deploy them. 87 + - Remember to update fontconfig cache by running fc-cache. 88 + 89 + !!! Caution !!! 90 + Uninstalling "variable font" packages can be dangerous. 91 + They might be depended upon by other packages important for your work. 92 + Denylisting should be less invasive, as it is effective only while 93 + XeLaTeX runs in "make pdfdocs". 94 + """ 95 + 96 + import os 97 + import re 98 + import subprocess 99 + import textwrap 100 + import sys 101 + 102 + class LatexFontChecker: 103 + """ 104 + Detect problems with CJK variable fonts that affect PDF builds for 105 + translations. 106 + """ 107 + 108 + def __init__(self, deny_vf=None): 109 + if not deny_vf: 110 + deny_vf = os.environ.get('FONTS_CONF_DENY_VF', "~/deny-vf") 111 + 112 + self.environ = os.environ.copy() 113 + self.environ['XDG_CONFIG_HOME'] = os.path.expanduser(deny_vf) 114 + 115 + self.re_cjk = re.compile(r"([^:]+):\s*Noto\s+(Sans|Sans Mono|Serif) CJK") 116 + 117 + def description(self): 118 + return __doc__ 119 + 120 + def get_noto_cjk_vf_fonts(self): 121 + """Get Noto CJK fonts""" 122 + 123 + cjk_fonts = set() 124 + cmd = ["fc-list", ":", "file", "family", "variable"] 125 + try: 126 + result = subprocess.run(cmd,stdout=subprocess.PIPE, 127 + stderr=subprocess.PIPE, 128 + universal_newlines=True, 129 + env=self.environ, 130 + check=True) 131 + 132 + except subprocess.CalledProcessError as exc: 133 + sys.exit(f"Error running fc-list: {repr(exc)}") 134 + 135 + for line in result.stdout.splitlines(): 136 + if 'variable=True' not in line: 137 + continue 138 + 139 + match = self.re_cjk.search(line) 140 + if match: 141 + cjk_fonts.add(match.group(1)) 142 + 143 + return sorted(cjk_fonts) 144 + 145 + def check(self): 146 + """Check for problems with CJK fonts""" 147 + 148 + fonts = textwrap.indent("\n".join(self.get_noto_cjk_vf_fonts()), " ") 149 + if not fonts: 150 + return None 151 + 152 + rel_file = os.path.relpath(__file__, os.getcwd()) 153 + 154 + msg = "=" * 77 + "\n" 155 + msg += 'XeTeX is confused by "variable font" files listed below:\n' 156 + msg += fonts + "\n" 157 + msg += textwrap.dedent(f""" 158 + For CJK pages in PDF, they need to be hidden from XeTeX by denylisting. 159 + Or, CJK pages can be skipped by uninstalling texlive-xecjk. 160 + 161 + For more info on denylisting, other options, and variable font, run: 162 + 163 + tools/docs/check-variable-fonts.py -h 164 + """) 165 + msg += "=" * 77 166 + 167 + return msg
+178
tools/docs/lib/python_version.py
··· 1 + #!/usr/bin/env python3 2 + # SPDX-License-Identifier: GPL-2.0-or-later 3 + # Copyright (c) 2017-2025 Mauro Carvalho Chehab <mchehab+huawei@kernel.org> 4 + 5 + """ 6 + Handle Python version check logic. 7 + 8 + Not all Python versions are supported by scripts. Yet, on some cases, 9 + like during documentation build, a newer version of python could be 10 + available. 11 + 12 + This class allows checking if the minimal requirements are followed. 13 + 14 + Better than that, PythonVersion.check_python() not only checks the minimal 15 + requirements, but it automatically switches to a the newest available 16 + Python version if present. 17 + 18 + """ 19 + 20 + import os 21 + import re 22 + import subprocess 23 + import shlex 24 + import sys 25 + 26 + from glob import glob 27 + from textwrap import indent 28 + 29 + class PythonVersion: 30 + """ 31 + Ancillary methods that checks for missing dependencies for different 32 + types of types, like binaries, python modules, rpm deps, etc. 33 + """ 34 + 35 + def __init__(self, version): 36 + """Ïnitialize self.version tuple from a version string""" 37 + self.version = self.parse_version(version) 38 + 39 + @staticmethod 40 + def parse_version(version): 41 + """Convert a major.minor.patch version into a tuple""" 42 + return tuple(int(x) for x in version.split(".")) 43 + 44 + @staticmethod 45 + def ver_str(version): 46 + """Returns a version tuple as major.minor.patch""" 47 + return ".".join([str(x) for x in version]) 48 + 49 + @staticmethod 50 + def cmd_print(cmd, max_len=80): 51 + cmd_line = [] 52 + 53 + for w in cmd: 54 + w = shlex.quote(w) 55 + 56 + if cmd_line: 57 + if not max_len or len(cmd_line[-1]) + len(w) < max_len: 58 + cmd_line[-1] += " " + w 59 + continue 60 + else: 61 + cmd_line[-1] += " \\" 62 + cmd_line.append(w) 63 + else: 64 + cmd_line.append(w) 65 + 66 + return "\n ".join(cmd_line) 67 + 68 + def __str__(self): 69 + """Returns a version tuple as major.minor.patch from self.version""" 70 + return self.ver_str(self.version) 71 + 72 + @staticmethod 73 + def get_python_version(cmd): 74 + """ 75 + Get python version from a Python binary. As we need to detect if 76 + are out there newer python binaries, we can't rely on sys.release here. 77 + """ 78 + 79 + kwargs = {} 80 + if sys.version_info < (3, 7): 81 + kwargs['universal_newlines'] = True 82 + else: 83 + kwargs['text'] = True 84 + 85 + result = subprocess.run([cmd, "--version"], 86 + stdout = subprocess.PIPE, 87 + stderr = subprocess.PIPE, 88 + **kwargs, check=False) 89 + 90 + version = result.stdout.strip() 91 + 92 + match = re.search(r"(\d+\.\d+\.\d+)", version) 93 + if match: 94 + return PythonVersion.parse_version(match.group(1)) 95 + 96 + print(f"Can't parse version {version}") 97 + return (0, 0, 0) 98 + 99 + @staticmethod 100 + def find_python(min_version): 101 + """ 102 + Detect if are out there any python 3.xy version newer than the 103 + current one. 104 + 105 + Note: this routine is limited to up to 2 digits for python3. We 106 + may need to update it one day, hopefully on a distant future. 107 + """ 108 + patterns = [ 109 + "python3.[0-9][0-9]", 110 + "python3.[0-9]", 111 + ] 112 + 113 + python_cmd = [] 114 + 115 + # Seek for a python binary newer than min_version 116 + for path in os.getenv("PATH", "").split(":"): 117 + for pattern in patterns: 118 + for cmd in glob(os.path.join(path, pattern)): 119 + if os.path.isfile(cmd) and os.access(cmd, os.X_OK): 120 + version = PythonVersion.get_python_version(cmd) 121 + if version >= min_version: 122 + python_cmd.append((version, cmd)) 123 + 124 + return sorted(python_cmd, reverse=True) 125 + 126 + @staticmethod 127 + def check_python(min_version, show_alternatives=False, bail_out=False, 128 + success_on_error=False): 129 + """ 130 + Check if the current python binary satisfies our minimal requirement 131 + for Sphinx build. If not, re-run with a newer version if found. 132 + """ 133 + cur_ver = sys.version_info[:3] 134 + if cur_ver >= min_version: 135 + ver = PythonVersion.ver_str(cur_ver) 136 + return 137 + 138 + python_ver = PythonVersion.ver_str(cur_ver) 139 + 140 + available_versions = PythonVersion.find_python(min_version) 141 + if not available_versions: 142 + print(f"ERROR: Python version {python_ver} is not spported anymore\n") 143 + print(" Can't find a new version. This script may fail") 144 + return 145 + 146 + script_path = os.path.abspath(sys.argv[0]) 147 + 148 + # Check possible alternatives 149 + if available_versions: 150 + new_python_cmd = available_versions[0][1] 151 + else: 152 + new_python_cmd = None 153 + 154 + if show_alternatives and available_versions: 155 + print("You could run, instead:") 156 + for _, cmd in available_versions: 157 + args = [cmd, script_path] + sys.argv[1:] 158 + 159 + cmd_str = indent(PythonVersion.cmd_print(args), " ") 160 + print(f"{cmd_str}\n") 161 + 162 + if bail_out: 163 + msg = f"Python {python_ver} not supported. Bailing out" 164 + if success_on_error: 165 + print(msg, file=sys.stderr) 166 + sys.exit(0) 167 + else: 168 + sys.exit(msg) 169 + 170 + print(f"Python {python_ver} not supported. Changing to {new_python_cmd}") 171 + 172 + # Restart script using the newer version 173 + args = [new_python_cmd, script_path] + sys.argv[1:] 174 + 175 + try: 176 + os.execv(new_python_cmd, args) 177 + except OSError as e: 178 + sys.exit(f"Failed to restart with {new_python_cmd}: {e}")
+841
tools/docs/sphinx-build-wrapper
··· 1 + #!/usr/bin/env python3 2 + # SPDX-License-Identifier: GPL-2.0 3 + # Copyright (C) 2025 Mauro Carvalho Chehab <mchehab+huawei@kernel.org> 4 + # 5 + # pylint: disable=R0902, R0912, R0913, R0914, R0915, R0917, C0103 6 + # 7 + # Converted from docs Makefile and parallel-wrapper.sh, both under 8 + # GPLv2, copyrighted since 2008 by the following authors: 9 + # 10 + # Akira Yokosawa <akiyks@gmail.com> 11 + # Arnd Bergmann <arnd@arndb.de> 12 + # Breno Leitao <leitao@debian.org> 13 + # Carlos Bilbao <carlos.bilbao@amd.com> 14 + # Dave Young <dyoung@redhat.com> 15 + # Donald Hunter <donald.hunter@gmail.com> 16 + # Geert Uytterhoeven <geert+renesas@glider.be> 17 + # Jani Nikula <jani.nikula@intel.com> 18 + # Jan Stancek <jstancek@redhat.com> 19 + # Jonathan Corbet <corbet@lwn.net> 20 + # Joshua Clayton <stillcompiling@gmail.com> 21 + # Kees Cook <keescook@chromium.org> 22 + # Linus Torvalds <torvalds@linux-foundation.org> 23 + # Magnus Damm <damm+renesas@opensource.se> 24 + # Masahiro Yamada <masahiroy@kernel.org> 25 + # Mauro Carvalho Chehab <mchehab+huawei@kernel.org> 26 + # Maxim Cournoyer <maxim.cournoyer@gmail.com> 27 + # Peter Foley <pefoley2@pefoley.com> 28 + # Randy Dunlap <rdunlap@infradead.org> 29 + # Rob Herring <robh@kernel.org> 30 + # Shuah Khan <shuahkh@osg.samsung.com> 31 + # Thorsten Blum <thorsten.blum@toblux.com> 32 + # Tomas Winkler <tomas.winkler@intel.com> 33 + 34 + 35 + """ 36 + Sphinx build wrapper that handles Kernel-specific business rules: 37 + 38 + - it gets the Kernel build environment vars; 39 + - it determines what's the best parallelism; 40 + - it handles SPHINXDIRS 41 + 42 + This tool ensures that MIN_PYTHON_VERSION is satisfied. If version is 43 + below that, it seeks for a new Python version. If found, it re-runs using 44 + the newer version. 45 + """ 46 + 47 + import argparse 48 + import locale 49 + import os 50 + import re 51 + import shlex 52 + import shutil 53 + import subprocess 54 + import sys 55 + 56 + from concurrent import futures 57 + from glob import glob 58 + 59 + from lib.python_version import PythonVersion 60 + from lib.latex_fonts import LatexFontChecker 61 + 62 + LIB_DIR = "../../scripts/lib" 63 + SRC_DIR = os.path.dirname(os.path.realpath(__file__)) 64 + 65 + sys.path.insert(0, os.path.join(SRC_DIR, LIB_DIR)) 66 + 67 + from jobserver import JobserverExec # pylint: disable=C0413,C0411,E0401 68 + 69 + # 70 + # Some constants 71 + # 72 + VENV_DEFAULT = "sphinx_latest" 73 + MIN_PYTHON_VERSION = PythonVersion("3.7").version 74 + PAPER = ["", "a4", "letter"] 75 + 76 + TARGETS = { 77 + "cleandocs": { "builder": "clean" }, 78 + "linkcheckdocs": { "builder": "linkcheck" }, 79 + "htmldocs": { "builder": "html" }, 80 + "epubdocs": { "builder": "epub", "out_dir": "epub" }, 81 + "texinfodocs": { "builder": "texinfo", "out_dir": "texinfo" }, 82 + "infodocs": { "builder": "texinfo", "out_dir": "texinfo" }, 83 + "mandocs": { "builder": "man", "out_dir": "man" }, 84 + "latexdocs": { "builder": "latex", "out_dir": "latex" }, 85 + "pdfdocs": { "builder": "latex", "out_dir": "latex" }, 86 + "xmldocs": { "builder": "xml", "out_dir": "xml" }, 87 + } 88 + 89 + 90 + # 91 + # SphinxBuilder class 92 + # 93 + 94 + class SphinxBuilder: 95 + """ 96 + Handles a sphinx-build target, adding needed arguments to build 97 + with the Kernel. 98 + """ 99 + 100 + def get_path(self, path, use_cwd=False, abs_path=False): 101 + """ 102 + Ancillary routine to handle patches the right way, as shell does. 103 + 104 + It first expands "~" and "~user". Then, if patch is not absolute, 105 + join self.srctree. Finally, if requested, convert to abspath. 106 + """ 107 + 108 + path = os.path.expanduser(path) 109 + if not path.startswith("/"): 110 + if use_cwd: 111 + base = os.getcwd() 112 + else: 113 + base = self.srctree 114 + 115 + path = os.path.join(base, path) 116 + 117 + if abs_path: 118 + return os.path.abspath(path) 119 + 120 + return path 121 + 122 + def get_sphinx_extra_opts(self, n_jobs): 123 + """ 124 + Get the number of jobs to be used for docs build passed via command 125 + line and desired sphinx verbosity. 126 + 127 + The number of jobs can be on different places: 128 + 129 + 1) It can be passed via "-j" argument; 130 + 2) The SPHINXOPTS="-j8" env var may have "-j"; 131 + 3) if called via GNU make, -j specifies the desired number of jobs. 132 + with GNU makefile, this number is available via POSIX jobserver; 133 + 4) if none of the above is available, it should default to "-jauto", 134 + and let sphinx decide the best value. 135 + """ 136 + 137 + # 138 + # SPHINXOPTS env var, if used, contains extra arguments to be used 139 + # by sphinx-build time. Among them, it may contain sphinx verbosity 140 + # and desired number of parallel jobs. 141 + # 142 + parser = argparse.ArgumentParser() 143 + parser.add_argument('-j', '--jobs', type=int) 144 + parser.add_argument('-q', '--quiet', action='store_true') 145 + 146 + # 147 + # Other sphinx-build arguments go as-is, so place them 148 + # at self.sphinxopts, using shell parser 149 + # 150 + sphinxopts = shlex.split(os.environ.get("SPHINXOPTS", "")) 151 + 152 + # 153 + # Build a list of sphinx args, honoring verbosity here if specified 154 + # 155 + 156 + verbose = self.verbose 157 + sphinx_args, self.sphinxopts = parser.parse_known_args(sphinxopts) 158 + if sphinx_args.quiet is True: 159 + verbose = False 160 + 161 + # 162 + # If the user explicitly sets "-j" at command line, use it. 163 + # Otherwise, pick it from SPHINXOPTS args 164 + # 165 + if n_jobs: 166 + self.n_jobs = n_jobs 167 + elif sphinx_args.jobs: 168 + self.n_jobs = sphinx_args.jobs 169 + else: 170 + self.n_jobs = None 171 + 172 + if not verbose: 173 + self.sphinxopts += ["-q"] 174 + 175 + def __init__(self, builddir, venv=None, verbose=False, n_jobs=None, 176 + interactive=None): 177 + """Initialize internal variables""" 178 + self.venv = venv 179 + self.verbose = None 180 + 181 + # 182 + # Normal variables passed from Kernel's makefile 183 + # 184 + self.kernelversion = os.environ.get("KERNELVERSION", "unknown") 185 + self.kernelrelease = os.environ.get("KERNELRELEASE", "unknown") 186 + self.pdflatex = os.environ.get("PDFLATEX", "xelatex") 187 + 188 + # 189 + # Kernel main Makefile defines a PYTHON3 variable whose default is 190 + # "python3". When set to a different value, it allows running a 191 + # diferent version than the default official python3 package. 192 + # Several distros package python3xx-sphinx packages with newer 193 + # versions of Python and sphinx-build. 194 + # 195 + # Honor such variable different than default 196 + # 197 + self.python = os.environ.get("PYTHON3") 198 + if self.python == "python3": 199 + self.python = None 200 + 201 + if not interactive: 202 + self.latexopts = os.environ.get("LATEXOPTS", "-interaction=batchmode -no-shell-escape") 203 + else: 204 + self.latexopts = os.environ.get("LATEXOPTS", "") 205 + 206 + if not verbose: 207 + verbose = bool(os.environ.get("KBUILD_VERBOSE", "") != "") 208 + 209 + if verbose is not None: 210 + self.verbose = verbose 211 + 212 + # 213 + # Source tree directory. This needs to be at os.environ, as 214 + # Sphinx extensions use it 215 + # 216 + self.srctree = os.environ.get("srctree") 217 + if not self.srctree: 218 + self.srctree = "." 219 + os.environ["srctree"] = self.srctree 220 + 221 + # 222 + # Now that we can expand srctree, get other directories as well 223 + # 224 + self.sphinxbuild = os.environ.get("SPHINXBUILD", "sphinx-build") 225 + self.kerneldoc = self.get_path(os.environ.get("KERNELDOC", 226 + "scripts/kernel-doc.py")) 227 + self.builddir = self.get_path(builddir, use_cwd=True, abs_path=True) 228 + 229 + # 230 + # Get directory locations for LaTeX build toolchain 231 + # 232 + self.pdflatex_cmd = shutil.which(self.pdflatex) 233 + self.latexmk_cmd = shutil.which("latexmk") 234 + 235 + self.env = os.environ.copy() 236 + 237 + self.get_sphinx_extra_opts(n_jobs) 238 + 239 + # 240 + # If venv command line argument is specified, run Sphinx from venv 241 + # 242 + if venv: 243 + bin_dir = os.path.join(venv, "bin") 244 + if not os.path.isfile(os.path.join(bin_dir, "activate")): 245 + sys.exit(f"Venv {venv} not found.") 246 + 247 + # "activate" virtual env 248 + self.env["PATH"] = bin_dir + ":" + self.env["PATH"] 249 + self.env["VIRTUAL_ENV"] = venv 250 + if "PYTHONHOME" in self.env: 251 + del self.env["PYTHONHOME"] 252 + print(f"Setting venv to {venv}") 253 + 254 + def run_sphinx(self, sphinx_build, build_args, *args, **pwargs): 255 + """ 256 + Executes sphinx-build using current python3 command. 257 + 258 + When calling via GNU make, POSIX jobserver is used to tell how 259 + many jobs are still available from a job pool. claim all remaining 260 + jobs, as we don't want sphinx-build to run in parallel with other 261 + jobs. 262 + 263 + Despite that, the user may actually force a different value than 264 + the number of available jobs via command line. 265 + 266 + The "with" logic here is used to ensure that the claimed jobs will 267 + be freed once subprocess finishes 268 + """ 269 + 270 + with JobserverExec() as jobserver: 271 + if jobserver.claim: 272 + # 273 + # when GNU make is used, claim available jobs from jobserver 274 + # 275 + n_jobs = str(jobserver.claim) 276 + else: 277 + # 278 + # Otherwise, let sphinx decide by default 279 + # 280 + n_jobs = "auto" 281 + 282 + # 283 + # If explicitly requested via command line, override default 284 + # 285 + if self.n_jobs: 286 + n_jobs = str(self.n_jobs) 287 + 288 + # 289 + # We can't simply call python3 sphinx-build, as OpenSUSE 290 + # Tumbleweed uses an ELF binary file (/usr/bin/alts) to switch 291 + # between different versions of sphinx-build. So, only call it 292 + # prepending "python3.xx" when PYTHON3 variable is not default. 293 + # 294 + if self.python: 295 + cmd = [self.python] 296 + else: 297 + cmd = [] 298 + 299 + cmd += [sphinx_build] 300 + cmd += [f"-j{n_jobs}"] 301 + cmd += self.sphinxopts 302 + cmd += build_args 303 + 304 + if self.verbose: 305 + print(" ".join(cmd)) 306 + 307 + return subprocess.call(cmd, *args, **pwargs) 308 + 309 + def handle_html(self, css, output_dir, rustdoc): 310 + """ 311 + Extra steps for HTML and epub output. 312 + 313 + For such targets, we need to ensure that CSS will be properly 314 + copied to the output _static directory 315 + """ 316 + 317 + if css: 318 + css = os.path.expanduser(css) 319 + if not css.startswith("/"): 320 + css = os.path.join(self.srctree, css) 321 + 322 + static_dir = os.path.join(output_dir, "_static") 323 + os.makedirs(static_dir, exist_ok=True) 324 + 325 + try: 326 + shutil.copy2(css, static_dir) 327 + except (OSError, IOError) as e: 328 + print(f"Warning: Failed to copy CSS: {e}", file=sys.stderr) 329 + 330 + if rustdoc: 331 + if "MAKE" in self.env: 332 + cmd = [self.env["MAKE"]] 333 + else: 334 + cmd = ["make", "LLVM=1"] 335 + 336 + cmd += [ "rustdoc"] 337 + if self.verbose: 338 + print(" ".join(cmd)) 339 + 340 + try: 341 + subprocess.run(cmd, check=True) 342 + except subprocess.CalledProcessError as e: 343 + print(f"Ignored errors when building rustdoc: {e}. Is RUST enabled?", 344 + file=sys.stderr) 345 + 346 + def build_pdf_file(self, latex_cmd, from_dir, path): 347 + """Builds a single pdf file using latex_cmd""" 348 + try: 349 + subprocess.run(latex_cmd + [path], 350 + cwd=from_dir, check=True, env=self.env) 351 + 352 + return True 353 + except subprocess.CalledProcessError: 354 + return False 355 + 356 + def pdf_parallel_build(self, tex_suffix, latex_cmd, tex_files, n_jobs): 357 + """Build PDF files in parallel if possible""" 358 + builds = {} 359 + build_failed = False 360 + max_len = 0 361 + has_tex = False 362 + 363 + # 364 + # LaTeX PDF error code is almost useless for us: 365 + # any warning makes it non-zero. For kernel doc builds it always return 366 + # non-zero even when build succeeds. So, let's do the best next thing: 367 + # Ignore build errors. At the end, check if all PDF files were built, 368 + # printing a summary with the built ones and returning 0 if all of 369 + # them were actually built. 370 + # 371 + with futures.ThreadPoolExecutor(max_workers=n_jobs) as executor: 372 + jobs = {} 373 + 374 + for from_dir, pdf_dir, entry in tex_files: 375 + name = entry.name 376 + 377 + if not name.endswith(tex_suffix): 378 + continue 379 + 380 + name = name[:-len(tex_suffix)] 381 + has_tex = True 382 + 383 + future = executor.submit(self.build_pdf_file, latex_cmd, 384 + from_dir, entry.path) 385 + jobs[future] = (from_dir, pdf_dir, name) 386 + 387 + for future in futures.as_completed(jobs): 388 + from_dir, pdf_dir, name = jobs[future] 389 + 390 + pdf_name = name + ".pdf" 391 + pdf_from = os.path.join(from_dir, pdf_name) 392 + pdf_to = os.path.join(pdf_dir, pdf_name) 393 + out_name = os.path.relpath(pdf_to, self.builddir) 394 + max_len = max(max_len, len(out_name)) 395 + 396 + try: 397 + success = future.result() 398 + 399 + if success and os.path.exists(pdf_from): 400 + os.rename(pdf_from, pdf_to) 401 + 402 + # 403 + # if verbose, get the name of built PDF file 404 + # 405 + if self.verbose: 406 + builds[out_name] = "SUCCESS" 407 + else: 408 + builds[out_name] = "FAILED" 409 + build_failed = True 410 + except futures.Error as e: 411 + builds[out_name] = f"FAILED ({repr(e)})" 412 + build_failed = True 413 + 414 + # 415 + # Handle case where no .tex files were found 416 + # 417 + if not has_tex: 418 + out_name = "LaTeX files" 419 + max_len = max(max_len, len(out_name)) 420 + builds[out_name] = "FAILED: no .tex files were generated" 421 + build_failed = True 422 + 423 + return builds, build_failed, max_len 424 + 425 + def handle_pdf(self, output_dirs, deny_vf): 426 + """ 427 + Extra steps for PDF output. 428 + 429 + As PDF is handled via a LaTeX output, after building the .tex file, 430 + a new build is needed to create the PDF output from the latex 431 + directory. 432 + """ 433 + builds = {} 434 + max_len = 0 435 + tex_suffix = ".tex" 436 + tex_files = [] 437 + 438 + # 439 + # Since early 2024, Fedora and openSUSE tumbleweed have started 440 + # deploying variable-font format of "Noto CJK", causing LaTeX 441 + # to break with CJK. Work around it, by denying the variable font 442 + # usage during xelatex build by passing the location of a config 443 + # file with a deny list. 444 + # 445 + # See tools/docs/lib/latex_fonts.py for more details. 446 + # 447 + if deny_vf: 448 + deny_vf = os.path.expanduser(deny_vf) 449 + if os.path.isdir(deny_vf): 450 + self.env["XDG_CONFIG_HOME"] = deny_vf 451 + 452 + for from_dir in output_dirs: 453 + pdf_dir = os.path.join(from_dir, "../pdf") 454 + os.makedirs(pdf_dir, exist_ok=True) 455 + 456 + if self.latexmk_cmd: 457 + latex_cmd = [self.latexmk_cmd, f"-{self.pdflatex}"] 458 + else: 459 + latex_cmd = [self.pdflatex] 460 + 461 + latex_cmd.extend(shlex.split(self.latexopts)) 462 + 463 + # Get a list of tex files to process 464 + with os.scandir(from_dir) as it: 465 + for entry in it: 466 + if entry.name.endswith(tex_suffix): 467 + tex_files.append((from_dir, pdf_dir, entry)) 468 + 469 + # 470 + # When using make, this won't be used, as the number of jobs comes 471 + # from POSIX jobserver. So, this covers the case where build comes 472 + # from command line. On such case, serialize by default, except if 473 + # the user explicitly sets the number of jobs. 474 + # 475 + n_jobs = 1 476 + 477 + # n_jobs is either an integer or "auto". Only use it if it is a number 478 + if self.n_jobs: 479 + try: 480 + n_jobs = int(self.n_jobs) 481 + except ValueError: 482 + pass 483 + 484 + # 485 + # When using make, jobserver.claim is the number of jobs that were 486 + # used with "-j" and that aren't used by other make targets 487 + # 488 + with JobserverExec() as jobserver: 489 + n_jobs = 1 490 + 491 + # 492 + # Handle the case when a parameter is passed via command line, 493 + # using it as default, if jobserver doesn't claim anything 494 + # 495 + if self.n_jobs: 496 + try: 497 + n_jobs = int(self.n_jobs) 498 + except ValueError: 499 + pass 500 + 501 + if jobserver.claim: 502 + n_jobs = jobserver.claim 503 + 504 + builds, build_failed, max_len = self.pdf_parallel_build(tex_suffix, 505 + latex_cmd, 506 + tex_files, 507 + n_jobs) 508 + 509 + # 510 + # In verbose mode, print a summary with the build results per file. 511 + # Otherwise, print a single line with all failures, if any. 512 + # On both cases, return code 1 indicates build failures, 513 + # 514 + if self.verbose: 515 + msg = "Summary" 516 + msg += "\n" + "=" * len(msg) 517 + print() 518 + print(msg) 519 + 520 + for pdf_name, pdf_file in builds.items(): 521 + print(f"{pdf_name:<{max_len}}: {pdf_file}") 522 + 523 + print() 524 + if build_failed: 525 + msg = LatexFontChecker().check() 526 + if msg: 527 + print(msg) 528 + 529 + sys.exit("Error: not all PDF files were created.") 530 + 531 + elif build_failed: 532 + n_failures = len(builds) 533 + failures = ", ".join(builds.keys()) 534 + 535 + msg = LatexFontChecker().check() 536 + if msg: 537 + print(msg) 538 + 539 + sys.exit(f"Error: Can't build {n_failures} PDF file(s): {failures}") 540 + 541 + def handle_info(self, output_dirs): 542 + """ 543 + Extra steps for Info output. 544 + 545 + For texinfo generation, an additional make is needed from the 546 + texinfo directory. 547 + """ 548 + 549 + for output_dir in output_dirs: 550 + try: 551 + subprocess.run(["make", "info"], cwd=output_dir, check=True) 552 + except subprocess.CalledProcessError as e: 553 + sys.exit(f"Error generating info docs: {e}") 554 + 555 + def handle_man(self, kerneldoc, docs_dir, src_dir, output_dir): 556 + """ 557 + Create man pages from kernel-doc output 558 + """ 559 + 560 + re_kernel_doc = re.compile(r"^\.\.\s+kernel-doc::\s*(\S+)") 561 + re_man = re.compile(r'^\.TH "[^"]*" (\d+) "([^"]*)"') 562 + 563 + if docs_dir == src_dir: 564 + # 565 + # Pick the entire set of kernel-doc markups from the entire tree 566 + # 567 + kdoc_files = set([self.srctree]) 568 + else: 569 + kdoc_files = set() 570 + 571 + for fname in glob(os.path.join(src_dir, "**"), recursive=True): 572 + if os.path.isfile(fname) and fname.endswith(".rst"): 573 + with open(fname, "r", encoding="utf-8") as in_fp: 574 + data = in_fp.read() 575 + 576 + for line in data.split("\n"): 577 + match = re_kernel_doc.match(line) 578 + if match: 579 + if os.path.isfile(match.group(1)): 580 + kdoc_files.add(match.group(1)) 581 + 582 + if not kdoc_files: 583 + sys.exit(f"Directory {src_dir} doesn't contain kernel-doc tags") 584 + 585 + cmd = [ kerneldoc, "-m" ] + sorted(kdoc_files) 586 + try: 587 + if self.verbose: 588 + print(" ".join(cmd)) 589 + 590 + result = subprocess.run(cmd, stdout=subprocess.PIPE, text= True) 591 + 592 + if result.returncode: 593 + print(f"Warning: kernel-doc returned {result.returncode} warnings") 594 + 595 + except (OSError, ValueError, subprocess.SubprocessError) as e: 596 + sys.exit(f"Failed to create man pages for {src_dir}: {repr(e)}") 597 + 598 + fp = None 599 + try: 600 + for line in result.stdout.split("\n"): 601 + match = re_man.match(line) 602 + if not match: 603 + if fp: 604 + fp.write(line + '\n') 605 + continue 606 + 607 + if fp: 608 + fp.close() 609 + 610 + fname = f"{output_dir}/{match.group(2)}.{match.group(1)}" 611 + 612 + if self.verbose: 613 + print(f"Creating {fname}") 614 + fp = open(fname, "w", encoding="utf-8") 615 + fp.write(line + '\n') 616 + finally: 617 + if fp: 618 + fp.close() 619 + 620 + def cleandocs(self, builder): # pylint: disable=W0613 621 + """Remove documentation output directory""" 622 + shutil.rmtree(self.builddir, ignore_errors=True) 623 + 624 + def build(self, target, sphinxdirs=None, 625 + theme=None, css=None, paper=None, deny_vf=None, rustdoc=False, 626 + skip_sphinx=False): 627 + """ 628 + Build documentation using Sphinx. This is the core function of this 629 + module. It prepares all arguments required by sphinx-build. 630 + """ 631 + 632 + builder = TARGETS[target]["builder"] 633 + out_dir = TARGETS[target].get("out_dir", "") 634 + 635 + # 636 + # Cleandocs doesn't require sphinx-build 637 + # 638 + if target == "cleandocs": 639 + self.cleandocs(builder) 640 + return 641 + 642 + if theme: 643 + os.environ["DOCS_THEME"] = theme 644 + 645 + # 646 + # Other targets require sphinx-build, so check if it exists 647 + # 648 + if not skip_sphinx: 649 + sphinxbuild = shutil.which(self.sphinxbuild, path=self.env["PATH"]) 650 + if not sphinxbuild and target != "mandocs": 651 + sys.exit(f"Error: {self.sphinxbuild} not found in PATH.\n") 652 + 653 + if target == "pdfdocs": 654 + if not self.pdflatex_cmd and not self.latexmk_cmd: 655 + sys.exit("Error: pdflatex or latexmk required for PDF generation") 656 + 657 + docs_dir = os.path.abspath(os.path.join(self.srctree, "Documentation")) 658 + 659 + # 660 + # Fill in base arguments for Sphinx build 661 + # 662 + kerneldoc = self.kerneldoc 663 + if kerneldoc.startswith(self.srctree): 664 + kerneldoc = os.path.relpath(kerneldoc, self.srctree) 665 + 666 + args = [ "-b", builder, "-c", docs_dir ] 667 + 668 + if builder == "latex": 669 + if not paper: 670 + paper = PAPER[1] 671 + 672 + args.extend(["-D", f"latex_elements.papersize={paper}paper"]) 673 + 674 + if rustdoc: 675 + args.extend(["-t", "rustdoc"]) 676 + 677 + if not sphinxdirs: 678 + sphinxdirs = os.environ.get("SPHINXDIRS", ".") 679 + 680 + # 681 + # The sphinx-build tool has a bug: internally, it tries to set 682 + # locale with locale.setlocale(locale.LC_ALL, ''). This causes a 683 + # crash if language is not set. Detect and fix it. 684 + # 685 + try: 686 + locale.setlocale(locale.LC_ALL, '') 687 + except locale.Error: 688 + self.env["LC_ALL"] = "C" 689 + 690 + # 691 + # sphinxdirs can be a list or a whitespace-separated string 692 + # 693 + sphinxdirs_list = [] 694 + for sphinxdir in sphinxdirs: 695 + if isinstance(sphinxdir, list): 696 + sphinxdirs_list += sphinxdir 697 + else: 698 + sphinxdirs_list += sphinxdir.split() 699 + 700 + # 701 + # Step 1: Build each directory in separate. 702 + # 703 + # This is not the best way of handling it, as cross-references between 704 + # them will be broken, but this is what we've been doing since 705 + # the beginning. 706 + # 707 + output_dirs = [] 708 + for sphinxdir in sphinxdirs_list: 709 + src_dir = os.path.join(docs_dir, sphinxdir) 710 + doctree_dir = os.path.join(self.builddir, ".doctrees") 711 + output_dir = os.path.join(self.builddir, sphinxdir, out_dir) 712 + 713 + # 714 + # Make directory names canonical 715 + # 716 + src_dir = os.path.normpath(src_dir) 717 + doctree_dir = os.path.normpath(doctree_dir) 718 + output_dir = os.path.normpath(output_dir) 719 + 720 + os.makedirs(doctree_dir, exist_ok=True) 721 + os.makedirs(output_dir, exist_ok=True) 722 + 723 + output_dirs.append(output_dir) 724 + 725 + build_args = args + [ 726 + "-d", doctree_dir, 727 + "-D", f"kerneldoc_bin={kerneldoc}", 728 + "-D", f"version={self.kernelversion}", 729 + "-D", f"release={self.kernelrelease}", 730 + "-D", f"kerneldoc_srctree={self.srctree}", 731 + src_dir, 732 + output_dir, 733 + ] 734 + 735 + if target == "mandocs": 736 + self.handle_man(kerneldoc, docs_dir, src_dir, output_dir) 737 + elif not skip_sphinx: 738 + try: 739 + result = self.run_sphinx(sphinxbuild, build_args, 740 + env=self.env) 741 + 742 + if result: 743 + sys.exit(f"Build failed: return code: {result}") 744 + 745 + except (OSError, ValueError, subprocess.SubprocessError) as e: 746 + sys.exit(f"Build failed: {repr(e)}") 747 + 748 + # 749 + # Ensure that each html/epub output will have needed static files 750 + # 751 + if target in ["htmldocs", "epubdocs"]: 752 + self.handle_html(css, output_dir, rustdoc) 753 + 754 + # 755 + # Step 2: Some targets (PDF and info) require an extra step once 756 + # sphinx-build finishes 757 + # 758 + if target == "pdfdocs": 759 + self.handle_pdf(output_dirs, deny_vf) 760 + elif target == "infodocs": 761 + self.handle_info(output_dirs) 762 + 763 + def jobs_type(value): 764 + """ 765 + Handle valid values for -j. Accepts Sphinx "-jauto", plus a number 766 + equal or bigger than one. 767 + """ 768 + if value is None: 769 + return None 770 + 771 + if value.lower() == 'auto': 772 + return value.lower() 773 + 774 + try: 775 + if int(value) >= 1: 776 + return value 777 + 778 + raise argparse.ArgumentTypeError(f"Minimum jobs is 1, got {value}") 779 + except ValueError: 780 + raise argparse.ArgumentTypeError(f"Must be 'auto' or positive integer, got {value}") # pylint: disable=W0707 781 + 782 + def main(): 783 + """ 784 + Main function. The only mandatory argument is the target. If not 785 + specified, the other arguments will use default values if not 786 + specified at os.environ. 787 + """ 788 + parser = argparse.ArgumentParser(description="Kernel documentation builder") 789 + 790 + parser.add_argument("target", choices=list(TARGETS.keys()), 791 + help="Documentation target to build") 792 + parser.add_argument("--sphinxdirs", nargs="+", 793 + help="Specific directories to build") 794 + parser.add_argument("--builddir", default="output", 795 + help="Sphinx configuration file") 796 + 797 + parser.add_argument("--theme", help="Sphinx theme to use") 798 + 799 + parser.add_argument("--css", help="Custom CSS file for HTML/EPUB") 800 + 801 + parser.add_argument("--paper", choices=PAPER, default=PAPER[0], 802 + help="Paper size for LaTeX/PDF output") 803 + 804 + parser.add_argument('--deny-vf', 805 + help="Configuration to deny variable fonts on pdf builds") 806 + 807 + parser.add_argument('--rustdoc', action="store_true", 808 + help="Enable rustdoc build. Requires CONFIG_RUST") 809 + 810 + parser.add_argument("-v", "--verbose", action='store_true', 811 + help="place build in verbose mode") 812 + 813 + parser.add_argument('-j', '--jobs', type=jobs_type, 814 + help="Sets number of jobs to use with sphinx-build") 815 + 816 + parser.add_argument('-i', '--interactive', action='store_true', 817 + help="Change latex default to run in interactive mode") 818 + 819 + parser.add_argument('-s', '--skip-sphinx-build', action='store_true', 820 + help="Skip sphinx-build step") 821 + 822 + parser.add_argument("-V", "--venv", nargs='?', const=f'{VENV_DEFAULT}', 823 + default=None, 824 + help=f'If used, run Sphinx from a venv dir (default dir: {VENV_DEFAULT})') 825 + 826 + args = parser.parse_args() 827 + 828 + PythonVersion.check_python(MIN_PYTHON_VERSION, show_alternatives=True, 829 + bail_out=True) 830 + 831 + builder = SphinxBuilder(builddir=args.builddir, venv=args.venv, 832 + verbose=args.verbose, n_jobs=args.jobs, 833 + interactive=args.interactive) 834 + 835 + builder.build(args.target, sphinxdirs=args.sphinxdirs, 836 + theme=args.theme, css=args.css, paper=args.paper, 837 + rustdoc=args.rustdoc, deny_vf=args.deny_vf, 838 + skip_sphinx=args.skip_sphinx_build) 839 + 840 + if __name__ == "__main__": 841 + main()