On Wed, Aug 1, 2018 at 8:56 AM, Gedare Bloom <ged...@rtems.org> wrote:
> On Tue, Jul 31, 2018 at 11:50 PM, Dannie Huang <danxue.hu...@gmail.com> > wrote: > > Hi, > > > > This is the initial implementation commit of converting newlib markup to > > rst, the example .rst file looks like this photo (see attached photo > > please). > > > > Can we see a side-by-side comparison with how newlib's documentation looks? > Sure, see attached photo please. Newlib's documentation is on the left and the rst output is on the right. > > > Code is pushed on my github: > > > > https://github.com/dh0072/NewlibMarkup2SphinxConverter > > > > Best, > > Dannie > > > > On Tue, Jul 31, 2018 at 10:42 PM, Dannie Huang <danxue.hu...@gmail.com> > > wrote: > >> > >> From: Danxue Huang <36866155+dh0...@users.noreply.github.com> > >> > >> --- > >> .gitignore | 2 + > >> README.md | 2 + > >> gen_rst_from_makedoc.py | 125 > >> ++++++++++++++++++++++++++++++++++++++++++++++++ > >> rst.py | 104 ++++++++++++++++++++++++++++++++++++++++ > >> strcmp.rst | 47 ++++++++++++++++++ > >> 5 files changed, 280 insertions(+) > >> create mode 100644 .gitignore > >> create mode 100644 README.md > >> create mode 100755 gen_rst_from_makedoc.py > >> create mode 100644 rst.py > >> create mode 100644 strcmp.rst > >> > >> diff --git a/.gitignore b/.gitignore > >> new file mode 100644 > >> index 0000000..9f90354 > >> --- /dev/null > >> +++ b/.gitignore > >> @@ -0,0 +1,2 @@ > >> +.idea/* > >> +__pycache__/* > >> diff --git a/README.md b/README.md > >> new file mode 100644 > >> index 0000000..8ebb224 > >> --- /dev/null > >> +++ b/README.md > >> @@ -0,0 +1,2 @@ > >> +# NewlibMarkup2SphinxConvertorPrivate > >> +(PRIVATE) This repo contains code for > >> NewlibMarkup2SphinxConvertorPrivate. > >> diff --git a/gen_rst_from_makedoc.py b/gen_rst_from_makedoc.py > >> new file mode 100755 > >> index 0000000..da69c80 > >> --- /dev/null > >> +++ b/gen_rst_from_makedoc.py > >> @@ -0,0 +1,125 @@ > >> +#!/usr/bin/env python > >> +# > >> +# RTEMS Tools Project (http://www.rtems.org/) > >> +# Copyright 2018 Danxue Huang (danxue.hu...@gmail.com) > >> +# All rights reserved. > >> +# > >> +# This file is part of the RTEMS Tools package in 'rtems-tools'. > >> +# > >> +# Redistribution and use in source and binary forms, with or without > >> +# modification, are permitted provided that the following conditions > are > >> met: > >> +# > >> +# 1. Redistributions of source code must retain the above copyright > >> notice, > >> +# this list of conditions and the following disclaimer. > >> +# > >> +# 2. Redistributions in binary form must reproduce the above copyright > >> notice, > >> +# this list of conditions and the following disclaimer in the > >> documentation > >> +# and/or other materials provided with the distribution. > >> +# > >> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > "AS > >> IS" > >> +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, > >> THE > >> +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR > >> PURPOSE > >> +# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR > CONTRIBUTORS > >> BE > >> +# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR > >> +# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF > >> +# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR > >> BUSINESS > >> +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER > IN > >> +# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR > OTHERWISE) > >> +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED > OF > >> THE > >> +# POSSIBILITY OF SUCH DAMAGE. > >> +# > >> + > >> + > >> +import argparse > >> +import re > >> +import rst > >> + > >> + > >> +def is_command(s): > >> + """ > >> + A command is a single word of at least 3 characters, all uppercase > >> + :param s: input string > >> + :return: True if s is a single command, otherwise False > >> + """ > >> + return True if re.match('^[A-Z_]{3,}\s*$', s) else False > >> + > >> + > >> +def extract_comments(content): > >> + """ > >> + Extract content inside of /* */ > >> + :param content: input file content > >> + :return: extracted comments > >> + """ > >> + comments = '' > >> + comments_match = re.match('/\*(\*(?!/)|[^*])*\*/', content) > >> + if comments_match: > >> + wrapped_comments = comments_match.group() > >> + comments = > >> wrapped_comments.lstrip('/*').rstrip('*/').lstrip().rstrip() > >> + return comments > >> + > >> + > >> +def extract_command_and_text(content): > >> + """ > >> + Extract command and text from input string content > >> + :param content: input string containing command and text > >> + :return: a tuple containing command and text > >> + """ > >> + command = '' > >> + text = '' > >> + command_text_dict = {} > >> + for line in content.splitlines(): > >> + if is_command(line): > >> + if command and text: > >> + command_text_dict[command] = text > >> + command = line.rstrip() > >> + text = '' > >> + else: > >> + text += line + '\n' > >> + return command_text_dict > >> + > >> + > >> +def generate_rst(command_text_dict): > >> + rst_str = '' > >> + for command, text in command_text_dict.items(): > >> + rst_str += rst.get_command_processor(command)(command, text) > >> + return rst_str > >> + > >> + > >> +def get_parser(): > >> + parser = argparse.ArgumentParser( > >> + description='Convert newlib style markup to rst markup' > >> + ) > >> + parser.add_argument( > >> + '-s', > >> + '--source_file_dir', > >> + type=str, > >> + help='Source file directory with newlib style comments', > >> + ) > >> + parser.add_argument( > >> + '-d', > >> + '--dest_file_dir', > >> + type=str, > >> + help='Destination directory for converted rst markup file', > >> + ) > >> + return parser > >> + > >> + > >> +def main(source_file_dir, dest_file_dir): > >> + with open(source_file_dir, 'r') as source_file, open(dest_file_dir, > >> 'w') as dest_file: > >> + file_content = source_file.read() > >> + > >> + # Get comments inside of /* */ > >> + comments = extract_comments(file_content) > >> + > >> + # Parse comments > >> + command_text_dict = extract_command_and_text(comments) > >> + > >> + # Process text based on command type > >> + rst_str = generate_rst(command_text_dict) > >> + > >> + dest_file.write(rst_str) > >> + > >> + > >> +if __name__ == '__main__': > >> + args = get_parser().parse_args() > >> + main(args.source_file_dir, args.dest_file_dir) > >> diff --git a/rst.py b/rst.py > >> new file mode 100644 > >> index 0000000..2531f46 > >> --- /dev/null > >> +++ b/rst.py > >> @@ -0,0 +1,104 @@ > >> +# > >> +# RTEMS Tools Project (http://www.rtems.org/) > >> +# Copyright 2018 Danxue Huang (danxue.hu...@gmail.com) > >> +# All rights reserved. > >> +# > >> +# This file is part of the RTEMS Tools package in 'rtems-tools'. > >> +# > >> +# Redistribution and use in source and binary forms, with or without > >> +# modification, are permitted provided that the following conditions > are > >> met: > >> +# > >> +# 1. Redistributions of source code must retain the above copyright > >> notice, > >> +# this list of conditions and the following disclaimer. > >> +# > >> +# 2. Redistributions in binary form must reproduce the above copyright > >> notice, > >> +# this list of conditions and the following disclaimer in the > >> documentation > >> +# and/or other materials provided with the distribution. > >> +# > >> +# THIS SOFTWARE IS PROVIDED BY THE COPYRIGHT HOLDERS AND CONTRIBUTORS > "AS > >> IS" > >> +# AND ANY EXPRESS OR IMPLIED WARRANTIES, INCLUDING, BUT NOT LIMITED TO, > >> THE > >> +# IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR > >> PURPOSE > >> +# ARE DISCLAIMED. IN NO EVENT SHALL THE COPYRIGHT HOLDER OR > CONTRIBUTORS > >> BE > >> +# LIABLE FOR ANY DIRECT, INDIRECT, INCIDENTAL, SPECIAL, EXEMPLARY, OR > >> +# CONSEQUENTIAL DAMAGES (INCLUDING, BUT NOT LIMITED TO, PROCUREMENT OF > >> +# SUBSTITUTE GOODS OR SERVICES; LOSS OF USE, DATA, OR PROFITS; OR > >> BUSINESS > >> +# INTERRUPTION) HOWEVER CAUSED AND ON ANY THEORY OF LIABILITY, WHETHER > IN > >> +# CONTRACT, STRICT LIABILITY, OR TORT (INCLUDING NEGLIGENCE OR > OTHERWISE) > >> +# ARISING IN ANY WAY OUT OF THE USE OF THIS SOFTWARE, EVEN IF ADVISED > OF > >> THE > >> +# POSSIBILITY OF SUCH DAMAGE. > >> +# > >> + > >> + > >> +import re > >> + > >> + > >> +def register_processor_command(): > >> + return { > >> + 'FUNCTION': gen_function_summary, > >> + 'SYNOPSIS': gen_code_block, > >> + 'ANSI_SYNOPSIS': gen_code_block, > >> + 'TRAD_SYNOPSIS': gen_code_block, > >> + 'TYPEDEF': gen_code_block, > >> + 'DESCRIPTION': gen_custom_directives, > >> + 'INDEX': gen_custom_directives, > >> + 'RETURNS': gen_custom_directives, > >> + 'PORTABILITY': gen_custom_directives, > >> + 'NOTES': gen_custom_directives, > >> + 'ERRORS': gen_custom_directives, > >> + 'BUGS': gen_custom_directives, > >> + 'WARNINGS': gen_custom_directives, > >> + 'QUICKREF': gen_nothing, > >> + 'MATHREF': gen_nothing, > >> + 'NEWPAGE': gen_nothing, > >> + 'START': gen_nothing, > >> + 'END': gen_nothing, > >> + 'SEEALSO': gen_custom_directives, > >> + } > >> + > >> + > >> +def get_command_processor(command): > >> + command_processor_dict = register_processor_command() > >> + if command in command_processor_dict: > >> + return command_processor_dict[command] > >> + else: > >> + print('Command {c} is not recognized, skip > it'.format(c=command)) > >> + return gen_nothing > >> + > >> + > >> +def gen_function_summary(command, text): > >> + function_name = extract_function_name(text) > >> + summary = extract_summary(text) > >> + > >> + title = '.. {f}:\n\n{f} - {s}\n'.format( > >> + f=function_name, > >> + s=summary.capitalize() > >> + ) > >> + dashes = '-' * len(text) + '\n' > >> + func_name_index = '.. index:: {name}\n'.format(name=function_name) > >> + summary_index = '.. index:: {summary}\n\n'.format(summary=summary) > >> + return title + dashes + func_name_index + summary_index > >> + > >> + > >> +def extract_function_name(text): > >> + function_name = '' > >> + function_name_match = re.match('\s+(<<(>(?!>)|[^>])*>>)', text) > >> + if function_name_match: > >> + function_name = > >> function_name_match.group(1).lstrip('<<').rstrip('>>') > >> + return function_name > >> + > >> + > >> +def extract_summary(text): > >> + return text.split('---')[-1].rstrip() > >> + > >> + > >> +def gen_code_block(command, text): > >> + return '**{c}:**\n\n.. code-block:: c\n\n{t}\n\n'.format(c = > command, > >> t=text) > >> + > >> + > >> +def gen_nothing(command, text): > >> + return '\n\n' > >> + > >> + > >> +def gen_custom_directives(command, text): > >> + return '**{c}:**\n\n{t}\n\n'.format(c=command, t=text) > >> + > >> diff --git a/strcmp.rst b/strcmp.rst > >> new file mode 100644 > >> index 0000000..c1dc543 > >> --- /dev/null > >> +++ b/strcmp.rst > >> @@ -0,0 +1,47 @@ > >> +.. strcmp: > >> + > >> +strcmp - Character string compare > >> +----------------------------------------- > >> +.. index:: strcmp > >> +.. index:: character string compare > >> + > >> +**INDEX:** > >> + > >> + strcmp > >> + > >> + > >> + > >> +**SYNOPSIS:** > >> + > >> +.. code-block:: c > >> + > >> + #include <string.h> > >> + int strcmp(const char *<[a]>, const char *<[b]>); > >> + > >> + > >> + > >> +**DESCRIPTION:** > >> + > >> + <<strcmp>> compares the string at <[a]> to > >> + the string at <[b]>. > >> + > >> + > >> + > >> +**RETURNS:** > >> + > >> + If <<*<[a]>>> sorts lexicographically after <<*<[b]>>>, > >> + <<strcmp>> returns a number greater than zero. If the two > >> + strings match, <<strcmp>> returns zero. If <<*<[a]>>> > >> + sorts lexicographically before <<*<[b]>>>, <<strcmp>> returns a > >> + number less than zero. > >> + > >> + > >> + > >> +**PORTABILITY:** > >> + > >> +<<strcmp>> is ANSI C. > >> + > >> +<<strcmp>> requires no supporting OS subroutines. > >> + > >> + > >> + > >> -- > >> 2.7.4 > >> > > > > > > _______________________________________________ > > devel mailing list > > devel@rtems.org > > http://lists.rtems.org/mailman/listinfo/devel >
_______________________________________________ devel mailing list devel@rtems.org http://lists.rtems.org/mailman/listinfo/devel