Linux kernel mirror (for testing)
git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git
kernel
os
linux
Merge branch 'mauro' into docs-mw
Mauro says:
The first patches on this series are focused mostly on .TH
(troff header) line, but, as a side effect, it also change
the name of man pages generated from DOC kernel-doc annotations.
At the previous state, those were overriden due to lots of
duplicated names.
The rationale for most of such changes is that modern troff/man
page specs say that .TH has up to 5 arguments,, as defined at [1]:
.TH topic section [footer-middle] [footer-inside] [header-middle]
[1] https://man7.org/linux/man-pages/man7/groff_man_style.7.html
Right now, Kernel uses 6 arguments, probably due to some legacy
man page definitions.
After double checking, modern man pages use this format:
.TH "{name}" {section} "{date}" "{modulename}" "{manual}"
Right now, man pages generation are messing up on how it encodes
each position at .TH, depending on the type of object it emits.
After this series, the definition is more consistent and file
output is better named.
It also fixes two issues at sphinx-build-wrapper related to how
it generate files names from the .TH header.
The last 4 patches on this series are new: they fix lots of issues
related to groff format: there, new lines continue the test from
previous pagragraph. This cause issues mainly on:
- tables;
- code blocks;
- lists
With the changes, the output now looks a lot better.
Please notice that the code there is not meant to fully implement
rst -> troff/groff conversion. Instead, it is meant to make the
output reasonable.
A more complete approach would be to use docutils or Sphinx
libraries, but that would likely require to also write a troff
output plugin, as the "man" builder is very limited. Also,
this could be problematic, as kernel-doc classes can be called
from Sphinx. I don't think we need that much complexity, as what
we mainly need is to avoid bad line grouping when generating
man pages.
This series should not affect HTML documentation. It only affect
man page generation and ManFormat output class.
+283
-25
-1
tools/docs/kernel-doc
-1
tools/docs/kernel-doc
+6
-4
tools/docs/sphinx-build-wrapper
+6
-4
tools/docs/sphinx-build-wrapper
···
576
576
"""
577
577
578
578
re_kernel_doc = re.compile(r"^\.\.\s+kernel-doc::\s*(\S+)")
579
-
re_man = re.compile(r'^\.TH "[^"]*" (\d+) "([^"]*)"')
580
579
581
580
if docs_dir == src_dir:
582
581
#
···
615
616
fp = None
616
617
try:
617
618
for line in result.stdout.split("\n"):
618
-
match = re_man.match(line)
619
-
if not match:
619
+
if not line.startswith(".TH"):
620
620
if fp:
621
621
fp.write(line + '\n')
622
622
continue
···
623
625
if fp:
624
626
fp.close()
625
627
626
-
fname = f"{output_dir}/{match.group(2)}.{match.group(1)}"
628
+
# Use shlex here, as it handles well parameters with commas
629
+
args = shlex.split(line)
630
+
fname = f"{args[1]}.{args[2]}"
631
+
fname = fname.replace("/", " ")
632
+
fname = f"{output_dir}/{fname}"
627
633
628
634
if self.verbose:
629
635
print(f"Creating {fname}")
+277
-20
tools/lib/python/kdoc/kdoc_output.py
+277
-20
tools/lib/python/kdoc/kdoc_output.py
···
580
580
581
581
582
582
class ManFormat(OutputFormat):
583
-
"""Consts and functions used by man pages output."""
583
+
"""
584
+
Consts and functions used by man pages output.
585
+
586
+
This class has one mandatory parameter and some optional ones, which
587
+
are needed to define the title header contents:
588
+
589
+
``modulename``
590
+
Defines the module name to be used at the troff ``.TH`` output.
591
+
592
+
This argument is optional. If not specified, it will be filled
593
+
with the directory which contains the documented file.
594
+
595
+
``section``
596
+
Usually a numeric value from 0 to 9, but man pages also accept
597
+
some strings like "p".
598
+
599
+
Defauls to ``9``
600
+
601
+
``manual``
602
+
Defaults to ``Kernel API Manual``.
603
+
604
+
The above controls the output of teh corresponding fields on troff
605
+
title headers, which will be filled like this::
606
+
607
+
.TH "{name}" {section} "{date}" "{modulename}" "{manual}"
608
+
609
+
where ``name``` will match the API symbol name, and ``date`` will be
610
+
either the date where the Kernel was compiled or the current date
611
+
"""
584
612
585
613
highlights = (
586
614
(type_constant, r"\1"),
···
635
607
"%m %d %Y",
636
608
]
637
609
638
-
def __init__(self, modulename):
610
+
def modulename(self, args):
611
+
if self._modulename:
612
+
return self._modulename
613
+
614
+
return os.path.dirname(args.fname)
615
+
616
+
def emit_th(self, name, args):
617
+
"""Emit a title header line."""
618
+
title = name.strip()
619
+
module = self.modulename(args)
620
+
621
+
self.data += f'.TH "{title}" {self.section} "{self.date}" '
622
+
self.data += f'"{module}" "{self.manual}"\n'
623
+
624
+
def __init__(self, modulename=None, section="9", manual="Kernel API Manual"):
639
625
"""
640
626
Creates class variables.
641
627
···
658
616
"""
659
617
660
618
super().__init__()
661
-
self.modulename = modulename
619
+
620
+
self._modulename = modulename
621
+
self.section = section
622
+
self.manual = manual
623
+
662
624
self.symbols = []
663
625
664
626
dt = None
···
678
632
if not dt:
679
633
dt = datetime.now()
680
634
681
-
self.man_date = dt.strftime("%B %Y")
635
+
self.date = dt.strftime("%B %Y")
682
636
683
637
def arg_name(self, args, name):
684
638
"""
···
693
647
dtype = args.type
694
648
695
649
if dtype == "doc":
696
-
return self.modulename
650
+
return name
651
+
# return os.path.basename(self.modulename(args))
697
652
698
653
if dtype in ["function", "typedef"]:
699
654
return name
···
744
697
745
698
return self.data
746
699
700
+
def emit_table(self, colspec_row, rows):
701
+
702
+
if not rows:
703
+
return ""
704
+
705
+
out = ""
706
+
colspec = "\t".join(["l"] * len(rows[0]))
707
+
708
+
out += "\n.TS\n"
709
+
out += "box;\n"
710
+
out += f"{colspec}.\n"
711
+
712
+
if colspec_row:
713
+
out_row = []
714
+
715
+
for text in colspec_row:
716
+
out_row.append(f"\\fB{text}\\fP")
717
+
718
+
out += "\t".join(out_row) + "\n_\n"
719
+
720
+
for r in rows:
721
+
out += "\t".join(r) + "\n"
722
+
723
+
out += ".TE\n"
724
+
725
+
return out
726
+
727
+
def grid_table(self, lines, start):
728
+
"""
729
+
Ancillary function to help handling a grid table inside the text.
730
+
"""
731
+
732
+
i = start + 1
733
+
rows = []
734
+
colspec_row = None
735
+
736
+
while i < len(lines):
737
+
line = lines[i]
738
+
739
+
if KernRe(r"^\s*\|.*\|\s*$").match(line):
740
+
parts = []
741
+
742
+
for p in line.strip('|').split('|'):
743
+
parts.append(p.strip())
744
+
745
+
rows.append(parts)
746
+
747
+
elif KernRe(r'^\+\=[\+\=]+\+\s*$').match(line):
748
+
if rows and rows[0]:
749
+
if not colspec_row:
750
+
colspec_row = [""] * len(rows[0])
751
+
752
+
for j in range(0, len(rows[0])):
753
+
content = []
754
+
for row in rows:
755
+
content.append(row[j])
756
+
757
+
colspec_row[j] = " ".join(content)
758
+
759
+
rows = []
760
+
761
+
elif KernRe(r"^\s*\+[-+]+\+.*$").match(line):
762
+
pass
763
+
764
+
else:
765
+
break
766
+
767
+
i += 1
768
+
769
+
return i, self.emit_table(colspec_row, rows)
770
+
771
+
def simple_table(self, lines, start):
772
+
"""
773
+
Ancillary function to help handling a simple table inside the text.
774
+
"""
775
+
776
+
i = start
777
+
rows = []
778
+
colspec_row = None
779
+
780
+
pos = []
781
+
for m in KernRe(r'\-+').finditer(lines[i]):
782
+
pos.append((m.start(), m.end() - 1))
783
+
784
+
i += 1
785
+
while i < len(lines):
786
+
line = lines[i]
787
+
788
+
if KernRe(r"^\s*[\-]+[ \t\-]+$").match(line):
789
+
i += 1
790
+
break
791
+
792
+
elif KernRe(r'^[\s=]+$').match(line):
793
+
if rows and rows[0]:
794
+
if not colspec_row:
795
+
colspec_row = [""] * len(rows[0])
796
+
797
+
for j in range(0, len(rows[0])):
798
+
content = []
799
+
for row in rows:
800
+
content.append(row[j])
801
+
802
+
colspec_row[j] = " ".join(content)
803
+
804
+
rows = []
805
+
806
+
else:
807
+
row = [""] * len(pos)
808
+
809
+
for j in range(0, len(pos)):
810
+
start, end = pos[j]
811
+
812
+
row[j] = line[start:end].strip()
813
+
814
+
rows.append(row)
815
+
816
+
i += 1
817
+
818
+
return i, self.emit_table(colspec_row, rows)
819
+
820
+
def code_block(self, lines, start):
821
+
"""
822
+
Ensure that code blocks won't be messed up at the output.
823
+
824
+
By default, troff join lines at the same paragraph. Disable it,
825
+
on code blocks.
826
+
"""
827
+
828
+
line = lines[start]
829
+
830
+
if "code-block" in line:
831
+
out = "\n.nf\n"
832
+
elif line.startswith("..") and line.endswith("::"):
833
+
#
834
+
# Handle note, warning, error, ... markups
835
+
#
836
+
line = line[2:-1].strip().upper()
837
+
out = f"\n.nf\n\\fB{line}\\fP\n"
838
+
elif line.endswith("::"):
839
+
out = line[:-1]
840
+
out += "\n.nf\n"
841
+
else:
842
+
# Just in case. Should never happen in practice
843
+
out = "\n.nf\n"
844
+
845
+
i = start + 1
846
+
ident = None
847
+
848
+
while i < len(lines):
849
+
line = lines[i]
850
+
851
+
m = KernRe(r"\S").match(line)
852
+
if not m:
853
+
out += line + "\n"
854
+
i += 1
855
+
continue
856
+
857
+
pos = m.start()
858
+
if not ident:
859
+
if pos > 0:
860
+
ident = pos
861
+
else:
862
+
out += "\n.fi\n"
863
+
if i > start + 1:
864
+
return i - 1, out
865
+
else:
866
+
# Just in case. Should never happen in practice
867
+
return i, out
868
+
869
+
if pos >= ident:
870
+
out += line + "\n"
871
+
i += 1
872
+
continue
873
+
874
+
break
875
+
876
+
out += "\n.fi\n"
877
+
return i, out
878
+
747
879
def output_highlight(self, block):
748
880
"""
749
881
Outputs a C symbol that may require being highlighted with
···
934
708
if isinstance(contents, list):
935
709
contents = "\n".join(contents)
936
710
937
-
for line in contents.strip("\n").split("\n"):
938
-
line = KernRe(r"^\s*").sub("", line)
939
-
if not line:
940
-
continue
711
+
lines = contents.strip("\n").split("\n")
712
+
i = 0
941
713
942
-
if line[0] == ".":
943
-
self.data += "\\&" + line + "\n"
714
+
while i < len(lines):
715
+
org_line = lines[i]
716
+
717
+
line = KernRe(r"^\s*").sub("", org_line)
718
+
719
+
if line:
720
+
if KernRe(r"^\+\-[-+]+\+.*$").match(line):
721
+
i, text = self.grid_table(lines, i)
722
+
self.data += text
723
+
continue
724
+
725
+
if KernRe(r"^\-+[ \t]\-[ \t\-]+$").match(line):
726
+
i, text = self.simple_table(lines, i)
727
+
self.data += text
728
+
continue
729
+
730
+
if line.endswith("::") or KernRe(r"\.\.\s+code-block.*::").match(line):
731
+
i, text = self.code_block(lines, i)
732
+
self.data += text
733
+
continue
734
+
735
+
if line[0] == ".":
736
+
self.data += "\\&" + line + "\n"
737
+
i += 1
738
+
continue
739
+
740
+
#
741
+
# Handle lists
742
+
#
743
+
line = KernRe(r'^[-*]\s+').sub(r'.IP \[bu]\n', line)
744
+
line = KernRe(r'^(\d+|a-z)[\.\)]\s+').sub(r'.IP \1\n', line)
944
745
else:
945
-
self.data += line + "\n"
746
+
line = ".PP\n"
747
+
748
+
i += 1
749
+
750
+
self.data += line + "\n"
946
751
947
752
def out_doc(self, fname, name, args):
948
753
if not self.check_doc(name, args):
···
981
724
982
725
out_name = self.arg_name(args, name)
983
726
984
-
self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
727
+
self.emit_th(out_name, args)
985
728
986
729
for section, text in args.sections.items():
987
730
self.data += f'.SH "{section}"' + "\n"
···
991
734
992
735
out_name = self.arg_name(args, name)
993
736
994
-
self.data += f'.TH "{name}" 9 "{out_name}" "{self.man_date}" "Kernel Hacker\'s Manual" LINUX' + "\n"
737
+
self.emit_th(out_name, args)
995
738
996
739
self.data += ".SH NAME\n"
997
740
self.data += f"{name} \\- {args['purpose']}\n"
···
1037
780
def out_enum(self, fname, name, args):
1038
781
out_name = self.arg_name(args, name)
1039
782
1040
-
self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
783
+
self.emit_th(out_name, args)
1041
784
1042
785
self.data += ".SH NAME\n"
1043
786
self.data += f"enum {name} \\- {args['purpose']}\n"
···
1070
813
out_name = self.arg_name(args, name)
1071
814
full_proto = args.other_stuff["full_proto"]
1072
815
1073
-
self.data += f'.TH "{self.modulename}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
816
+
self.emit_th(out_name, args)
1074
817
1075
818
self.data += ".SH NAME\n"
1076
819
self.data += f"{name} \\- {args['purpose']}\n"
···
1087
830
self.output_highlight(text)
1088
831
1089
832
def out_typedef(self, fname, name, args):
1090
-
module = self.modulename
833
+
module = self.modulename(args)
1091
834
purpose = args.get('purpose')
1092
835
out_name = self.arg_name(args, name)
1093
836
1094
-
self.data += f'.TH "{module}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
837
+
self.emit_th(out_name, args)
1095
838
1096
839
self.data += ".SH NAME\n"
1097
840
self.data += f"typedef {name} \\- {purpose}\n"
···
1101
844
self.output_highlight(text)
1102
845
1103
846
def out_struct(self, fname, name, args):
1104
-
module = self.modulename
847
+
module = self.modulename(args)
1105
848
purpose = args.get('purpose')
1106
849
definition = args.get('definition')
1107
850
out_name = self.arg_name(args, name)
1108
851
1109
-
self.data += f'.TH "{module}" 9 "{out_name}" "{self.man_date}" "API Manual" LINUX' + "\n"
852
+
self.emit_th(out_name, args)
1110
853
1111
854
self.data += ".SH NAME\n"
1112
855
self.data += f"{args.type} {name} \\- {purpose}\n"