Automation API documentation generation

[campb303]

Python in this project adheres to section 3.8 of Google's guidelines for documenting python.

Sphinx is a popular documentation tool for Python and could be used to auto generate documentation.

This will depend on #24

[benne238]

Using Sphinx

Sphinx is essentially a file generator. Given inputs, it automatically creates human readable files (html being one of the type of files it can generate)
However, for our purposes, we want to place a heavier emphasis on the autogenerated documentation that sphinx creates.
Sphinx can pull information directly from python docstring comments for functions within a given python script. This seems to be the best advantage of creating documentation for the api (a basic demonstration of the autogenerate capabilities can be found here (start at 6:25)

Note:
It would seem however that anything we want to create that isn't a python docstring would have to be done manually using sphinx. Currently looking to see if that is actually the case...

[benne238]

Some more videos and documentation for sphinx

https://www.youtube.com/watch?v=b4iFyrLQQh4: a simple demonstration on creating an auto generation doc

https://realpython.com/lessons/setting-sphinx-example-project-and-scaffolding/: similar to the above, focuses on the sphinx-quickstart command (which we will most likely use) and goes in depth on the different options that can be enabled with the command

https://www.sphinx-doc.org/en/master/usage/quickstart.html: Sphinx's documentation on how to get started with Sphinx (which looks like it was generated using Sphinx I might add). Is complex for documentation, but makes sense after watching some of the tutorial videos before hand.

[benne238]

Installed sphinx in the virtual environment using pip

Note: might need to add sphinx to the requirements list depending on how we plan to use it...

Worked on trying to build an html autogenerated file based on the docstrings of the Item class in ECNQueue.py, though will have to do more research on how to automatically produce out put for "private" functions for the Item class.
Generally speaking, the only thing that can be automatically generated are functions and the docstrings to those corresponding functions. Sphinx does help with styling to make everythig look uniform, but any other documentation that would need to be done might be best done using a markdown README file

[benne238]

Docusaurus seems to be an alternative to Sphinx in that it can autogenerate a documentation website. However, Docusaurus relies on manual documentation in that Sphinx autodoc feature that collects the docstrings from python functions.

https://v2.docusaurus.io/docs/installation: installation of Docusaurus
https://www.youtube.com/watch?v=dn4dgA51WNg: demonstration of Docusaurus

[benne238]

Installation and configuration of Sphinx

1. Run npm run venv:reset
2. Run pip install Sphinx in the virtual environment
3. Within the project directory, create a documentation directory that will be used by sphinx for build and configuration files.
4. cd into that newly created directory and run sphinx-quickstart
5. Accept all of the default values (the values in brackets) and give the project a title, author and version number
6. The following step will create several files and directories within the documentation folder that was created. Open the conf.py file.
7. Within the conf.py file uncomment lines 13, 14, and 15:

# import os
# import sys
# sys.path.insert(0, os.path.abspath('..'))

Ensure that the path points to the location of the script to be documented (in my case, it is in the parent directory .. from the current directory
8. Also within conf.py, ensure that the extensions list on lines 33 and 34 includes 'sphinx.ext.autodoc':

extensions = [
'sphinx.ext.autodoc'
]

9. Save the configuration and close the file

[benne238]

Creating Sphinx Auto Documentation

1. index.rst determines what files and folders are processed when generating documentation.
2. Within the documentation folder, create an additional folder called docfiles or something meaningful. This folder will ultimately contain the rst files that autogenerate documentation for python scripts by using doc strings.
   *Any rst files created within the parent documentation folder will automatically be used by sphinx, but the docfiles adds a level of organization to the documentation folder.
3. To create an autodoc, create a new rst file in the docfiles directory.
4. Within the newly created rst file, add the following:

Code documentation
=========================================

.. autoclass:: ECNQueue.Item
    :members:
    :private-members:

.. autoclass:: ECNQueue.Queue
    :members:
    :private-members:

.. automodule:: ECNQueue
    :members:

This tells the sphinx parser to parse all of the private functions in the Item and Queue classes within ECNQueue.py, but also any other functions not included within a specific class. Save this file.

5. Within the terminal, ensure you are in the parent documentation directory and run make clean. This will remove any items in the build directory and ensures that the next command will execute without errors caused by lingering files within build.

6. Run the command make html. This will generate several files in the build directory. Navigate to the build/html/docfiles directory. Within this directory, there will be an html file with the same name as the rst file created in step 4. This html file contains all of the functions within ECNQueue.py as well as their respective docstrings in a relatively easy to read format.

[benne238]

Sphinx Themes

Additional themes are available for sphinx through pip
In this case, pip install sphinx_rtd_theme
To configure this theme, go to conf.py in the documentation folder and edit line 51

html_theme = 'alabaster'

to this:

html_theme = 'sphinx_rtd_theme'

[benne238]

Markdown and Sphinx

Still looking into this, but it seems that at the very least, markdown documents can be put into the sphinx directory. However, it may be possible to output auto generated code as markdown. Will look into this tomorrow.

Allowing markdown to be parsed by Sphinx

1. run pip install recommonmark
2. edit conf.py in the documentation directory to include these lines at the end:

# The file extensions of source files. Sphinx considers the files with this suffix as sources. 
# The value is a dictionary mapping file extensions to file types
source_suffix = {
    '.rst': 'restructuredtext',
    '.md': 'markdown'
}

This tells sphinx to not delete any .md files in the documents directory and to treat .md files as markdown files.

[benne238]

Markdown in Sphinx

Following the steps above does allow for markdown to be parsed directly by sphinx and thus parsed into an html page. However, documentation doesn't appear to be able to automatically generated directly from a markdown file like what can be done with an rst file.

[benne238]

Python Docstring Format

One of the "issues" with the Sphinx auto-doc feature is that the output format looks terrible:

However, there is a relatively easy way to fix this problem which is by reformatting the python docstrings so that the output sphinx produces looks significantly better:

When outputting, Sphinx considers multiple things from the string:

1. The indention level of different lines in the docstring
2. New lines within docstrings
3. Special characters, such as asterisks

All of which affect the output format of the html files

[benne238]

Markdown and Sphinx

Using the guide here, it is possible to create auto generated markdown files from the python docstrings.

There is some additional configuration that has to be done, which includes running this in the terminal

pip install sphinx-markdown-builder

and adding this information to conf.py for sphinx

# add 'sphinx_markdown_builder` to the existing extensions list
extensions = [
    'sphinx_markdown_builder'
]

# add the exclude_patterns list with this entry:
exclude_patterns = [
    'build/*'
]

[benne238]

Installation and configuration of Sphinx

Sphinx is a documentation generation tool that pulls content from reStructuredText, or rst files. rst is similar to html in that it is a markup language, but is designed specifically for documentation. Sphinx is used to convert rst within code files and docstrings into html, which is easily readable. Sphinx has a variety of built-in themes that dictate how the html output will be formatted and any additional themes must be installed along with sphinx. In this guide we will be using and installing the sphinx_rtd_theme.

Note: The starting directory for the following commands will be the webqueue2 project directory.
1. Create a clean environment in which Sphinx can be installed by running this command in the terminal:

npm run venv:reset

Note: This command is a part of the dev. environment and can only be run within the /home/benne238/webqueue2 directory

2. Next, install sphinx and the sphinx rtd theme by running these commands:

pip install Sphinx

pip install sphinx_rtd_theme

3. Within the webqueue2/api directory, create a documentation directory. This directory will be used by sphinx for build and configuration files

mkdir api/documentation

Note: This guide will refer to this directory as the parent documentation directory. Everything related to sphinx will be located and generated here.

4. Next, create the sphinx configuration files and dependencies with in the documentation folder:

cd api/documentation

Sphinx-quick start generates all of the dependencies and directories needed set up and use Sphinx. These configurations are generated in the directory where the sphinx-quickstart command is run.

sphinx-quickstart

5. Accept all of the default values (the values in brackets) and give the project a title, author and version number

After accepting the output of the sphinx-quickstart command should look something similar to this:

Creating file /home/pier/e/benne238/webqueue2/api/documentation/conf.py.
Creating file /home/pier/e/benne238/webqueue2/api/documentation/index.rst.
Creating file /home/pier/e/benne238/webqueue2/api/documentation/Makefile.
Creating file /home/pier/e/benne238/webqueue2/api/documentation/make.bat.

Finished: An initial directory structure has been created.

You should now populate your master file /home/pier/e/benne238/webqueue2/api/documentation/index.rst and create other documentation
source files. Use the Makefile to build the docs, like so:
   make builder
where "builder" is one of the supported builders, e.g. html, latex or linkcheck.

6. Edit the main sphinx configuration file, conf.py.

nano conf.py

Many settings and defaults that sphinx has will be located here and can also be modified in this file.

7. Uncomment these lines within the conf.py file:

...

import os
import sys
sys.path.insert(0, os.path.abspath('..'))

...

Note:
The current (simplified) directory of webqueue2 looks something like this:

webqueue2
    └───api
        ├──API.py
        ├──ECNQueue.py
        │
        └───documentation
             └──conf.py

Ensure that the path in sys.path.insert(0, os.pat =h.abspath('..')) points to the location of the python script(s) to be documented (in this case the scripts are located in webqueue2/api, the current directory of conf.py is webqueue2/api/documentation so using a relative path of .. will point to the correct directory

8. Also within conf.py, ensure that the extensions list, includes 'sphinx.ext.autodoc':

...

extensions = [
'sphinx.ext.autodoc'
]

...

9. Finally, to configure the sphinx_rtd_theme theme, modify html_theme within conf.py to this:

...

html_theme = 'sphinx_rtd_theme'

...

10. Save the configuration and close the file

Sphinx is now configured to use autodocumentation and output html with the rtd theme, however, no files exist yet to document the existing python scripts so any documentation generated now will not be useful.

Sphinx Table of Contents creation

Sphinx generates html output based on a directory structure within the main documentation directory, thus the need arises to create a logical structure for all of the docs called a table of contents or TOC.
To do this:

Note: *The starting directory for the following commands will be the webqueue2/api/documentation directory.

webqueue2
    └───api
        ├──API.py
        ├──ECNQueue.py
        │
        └───documentation  <<
             └─ conf.py

1. Create a directory named docfiles within the parent documentation directory

mkdir docfiles

2. Create two directories named ecnqueue and api within the docfiles directory

mkdir docfiles/ecnqueue

mkdir docfiles/api

Note: Now the (simplified) directory structure within the webqueue2 should look like this:

webqueue2
    └───api
        ├──API.py
        ├──ECNQueue.py
        │
        └───documentation 
             ├──conf.py
             │
             └───docfiles
                  │
                  ├───api
                  └───ecnqueue

This directory structure mirrors the structure of the API in which there are two scripts: the ECNQueue.py and api.py scripts. Similarly, if one wanted to document a third script within the API, an additional directory would need to be created in the docfiles directory that refelcts the name of the additional script. All of these names are arbitray and do not affect the way the html output looks, this is just a logical way to store the rst files that will be created in the next step.

---

Sphinx RST file configuration

Within webqueue2/api/documentation, there is an index.rst file. This will serve as the "backbone" for the Sphinx autodoc feature.
The contents of the file look something like this (depending on what you input for the sphinx-quickstart):

Note: The starting directory for the following commands will be the webqueue2/api/documentation directory unless stated otherwise.

webqueue2
    └───api
        ├──API.py
        ├──ECNQueue.py
        │
        └───documentation <<
             ├──conf.py
             ├──index.rst
             │
             └───docfiles
                  │
                  ├───api
                  └───ecnqueue

1. Open webqueue2/api/documentation/index.rst:

nano index.rst

webqueue2/api/documentation/index.rst:

.. webqueue2 api documentation master file, created by
   sphinx-quickstart on Wed Nov  4 15:15:58 2020.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Welcome to webqueue2 api's documentation!
=========================================

This rst file will do several things:

• Create a title (Welcome to webqueue2 api's documentation!) for the home documentation page
• Create a visible structure in which documentation is separated by python script by making a separate API and ECNQueue heading.
• Tell Sphinx to look in the webqueue2/api/documentation/docfiles/ecnqueue and webqueue2/api/documentation/docfiles/api directoriess for any .rst files and if any are found, include the title of those documents in the TOC.

2. Modify the index.rst file to this:

.. webqueue2 api documentation master file, created by
   sphinx-quickstart on Wed Nov  4 15:15:58 2020.
   You can adapt this file completely to your liking, but it should at least
   contain the root `toctree` directive.

Welcome to webqueue2 api's documentation!
=========================================

.. toctree::
   :caption: ECNQueue:
   :glob:
   
   docfiles/ecnqueue/*

.. toctree::
   :caption: API:
   :glob:
   
   docfiles/api/*

Indices and tables
==================

* :ref:`genindex`
* :ref:`modindex`
* :ref:`search`

However nothing in index.rst actually documents any of the docstrings within the python scripts. To do that, we will need to create several more rst files in their respective directories. Unfortunately, one of the shortcomings of sphinx is to automatically detect classes within a given python script, so each class within any script must have its own specific entry within its own rst file. However, where the autodoc feature comes in handy is that if a class is explicitly mentioned, all of it's functions' docstrings are documented.

---

Sphinx Autodoc

Autodoc is a Sphinx extension that allows sphinx to pull rst formatted docstrings from python scripts and automatically format the docstrings into html.

Within the webqueue2/api/documentation/docfiles/ecnqueue directory, three files will be created:

• item.rst
• queue.rst
• ecnqueue.rst

These files ultimately allow for sphinx to document three different aspects of the ECNQueue.py script:

1. All of the functions within the Item class
2. All of the functions within the Queue class
3. All of the functions outside of Item and Queue but within the ECNQueue.py script.

Create these files as described below:
Note: The starting directory for the following commands will be the webqueue2/api/documentation/docfiles directory:

webqueue2
    └───api
        ├──API.py
        ├──ECNQueue.py
        │
        └───documentation 
             ├──conf.py
             ├──index.rst
             │
             └───docfiles  <<
                  │
                  ├───api
                  └───ecnqueue

1. Create the item.rst file:

nano ecnqueue/item.rst

and add these lines:

webqueue2/api/documentation/docfiles/ecnqueue/item.rst:

Item class Code documentation
=========================================

.. autoclass:: ECNQueue.Item
    :members:
    :private-members:

2. Create the queue.rst file:

nano ecnqueue/queue.rst

and add these lines:

webqueue2/api/documentation/docfiles/ecnqueue/queue.rst:

Queue class Code documentation
=========================================

.. autoclass:: ECNQueue.Queue
    :members:
    :private-members:

1. Create the ecnqueue.rst file:

nano ecnqueue/ecnqueue.rst

and add these lines:

webqueue2/api/documentation/docfiles/ecnqueue/ecnqueue.rst:

ECNQueue Script
=========================================

.. automodule:: ECNQueue
    :members:

The .. autoclass:: line in each of these files will tell the sphinx parser to look for all of the members and private-members of their respective classes. members will autodocument all non-private functions within a class, and private-memebers will autodocument all private functions within a specified class.
The .. automodule:: line in a file will tell the sphinx parser to look for all of the members (or functions) that are not in a specific class within the specified python script.

Syntax for autoclass: .. autoclass:: python_script.class_name
Syntax for automodule: .. automodule:: python_sctipt

For now, we will skip the api rst file creation, but it is the same principle as creating rst files for the ECNQueue.py.

---

Make HTML

At this point, all previous steps have been only for the configuration for sphinx autodoc as no documentation for the API has been created yet. Lets change that.
Note: The starting directory will be the webqueue2/api/documentation/docfiles directory from the previous step

webqueue2
    └───api
        ├──API.py
        ├──ECNQueue.py
        │
        └───documentation 
             ├──conf.py
             ├──index.rst
             │
             └───docfiles  <<
                  │
                  ├───api
                  └───ecnqueue

1. In the terminal, navigate back to the documentation directory (where sphinx-quickstart was initially run)

cd ..

2. Create the the documentation by running

make html

Note: ensure you are still within the python virtual environment.

This will create several directories within the webqueue2/api/documentation/build directory. That is the auto-documentation.

webqueue2
    └───api
        ├──API.py
        ├──ECNQueue.py
        │
        └───documentation 
             ├──conf.py
             ├──index.rst
             │
             ├───build
             │   └───html
             │        └───docfiles <<
             │             ├───api
             │             └───docfiles
             │
             └───docfiles
                  │
                  ├───api
                  └───ecnqueue

Note: *The html documentation will always be created in the docfiles directory as indicated in the tree structure above *

1. Navigate to webqueue2/api/documentation/build/html/docfiles/ecnqueue directory.

cd build/html/docfiles/ecnqueue

The ecnqueue directory contains three html files that resemble the three rst files that were created earlier.

2. Open one of the files and preview it.
The html output, which includes all of the docstrings within each of the functions in the scope of the rst files created from earlier. (ie: Queue.html displays docstrings for all ECNQueue.Queue functions, Item.html displays docstrings for all EcnQueue.Item functions)

---

Making Changes to the html output

The docstrings need to be written in rst in order to provide better looking output. This "cheatsheet" provides a fairly in-depth and easy to understand syntax outline for rst.

Anytime the docstrings are edited, the entire project needs be rebuilt, luckily this can be accomplished very easily.

1. Make a change to the docstrings in either the api or ECNQueue scripts

Note: The starting directory for the following commands will be webqueue/api.
2. Within the terminal, navigate to the documentation directory

cd documentation

3. Ensure the virtual environment is running

4. Run this command in the terminal:

make clean && make html

Note: make clean and make html can be run as two separate commands, the && operator just simplifies the process.

This will delete all of the files within the webqueue2/api/documentation/build directory and recrate them, implementing any of the saved changes to the docstrings within the ecnqueue and api scripts.

[benne238]

Mkdocs

Due to the complexity and messiness of Sphinx, we will now attempt to make documentation using mkdocs along with a feature called mkautodocs to automatically document the docstrings of the api in markdown similarly to Sphinx autodoc. Issues and information regarding mkdocs will be posted here.

[benne238]

Mkdocs configuration and installation

Mkdocs is a html documentation generator which parses markdown files and outputs the contents of markdown into html. Mkautodoc is an extension of mkdocs that pulls markdown formated docstrings from python scripts without having to copy the docstrings manually. This project will make use of the mkdocs material theme.

Note: The starting directory for commands will be the webqueue2 project directory

1. Mkdocs, mkautodoc, and the mkdoc material theme can be installed directly with pip (ensure that the python virtual environment is running)

pip install mkdocs

pip install mkautodoc

pip install mkdocs-material

2. Mkdocs needs a directory to contain all of the documentation related files and subdirectories, to do that:

cd api

This next command will create several things:

• a directory called documentation with a subdirectory called docs
• a mkdocs.yml file located within the newly created documentation directory
• an index.md file located within the newly created docs directory

mkdocs new documentation

Output:

INFO    -  Creating project directory: documentation 
INFO    -  Writing config file: documentation/mkdocs.yml 
INFO    -  Writing initial docs: docuentation/docs/index.md 

3. Edit the mkdocs.yml file

nano documentation/mkdocs.yml

mkdocs.yml

site_name: Webqueue2

markdown_extensions:
  - admonition
  - codehilite
  - mkautodoc

theme:
  name: material

This file is used for a number of different configurations which change the output format of the html files produced my mkdocs, a list of all the flags can be found here

---

Prepping the Python Scripts

In order for mkautodoc to function, the ecnqueue.py and api.py scripts must be importable, meaning they need to be packaged with pip.

Note: The starting directory for commands will be the api directory

webqueue2
    └───api  <<
        ├──API.py
        ├──ECNQueue.py
        │
        └───documentation 
             ├──mkdocs.yml
             │
             └───docs
                  └───index.md

1. Create a setup.py document in the api directory
This document tells pip basic information about the package and is required to package the ECNqueue.py and the api.py scripts.

nano setup.py

2. Copy these lines into the newly created setup.py script:
This script uses the disutils.core setup function to:

• Create a basic package named webqueueapi
• Defines a version of 1.0
• Defines api.py and ECNqueue.py as the modules included within the package

webqueue2/api/setup.py

from distutils.core import setup

setup(name='webqueueapi',
      version='1.0',
      py_modules=['api', 'ECNqueue'],
      )

Note: This guide gives a more in depth explanation on the functionality and configuration of the setup.py script

3. Save the setup.py scripts and exit the nano editor.

4. Package and install the api directory with pip:
This will install the current directory hence the . signifying the current directory, and the -e flag installs the directory as editable? (need to find info on the -e flag)

pip install -e .

Note: Alternatively, the direct path to the webqueue2/api directory would work instead of .
Output:

Obtaining file:///home/pier/e/benne238/webqueue2/api
Installing collected packages: webqueueapi
  Running setup.py develop for webqueueapi
Successfully installed webqueueapi

The ECNqueue.py and api.py are now able to be imported and thus, able to be located by mkautodoc.

---

Documentation

These next steps will create the documentation from the doc strings in the python scripts.
Note: The starting directory for commands should be the webqueue2/api directory

webqueue2
    └───api  <<
        ├──API.py
        ├──ECNQueue.py
        ├──setup.py
        │
        └───documentation 
             ├──mkdocs.yml
             │
             └───docs
                  └───index.md

1. Navigate to the webqueue2/api/documentation/docs directory
This is where all of the markdown documentation files will be kept

cd documentation/docs

2. Create a markdown file to document the ECNqueue.py script
This file will make use of mkautodoc to pull from the ECNqueue.py script but additional information can be manually added to the markdown file.

nano ecnqueue.md

3. Copy these lines into the editor:
This will tell mkautodoc where and what docstrings to pull and include in the documentation.

# ECNQueue documentation

## Item Class

::: ECNQueue.Item
    :docstring:

### Item functions

::: ECNQueue.Item
    :members:

## Queue Class

::: ECNQueue.Queue
    :docstring:

### Queue Functions

::: ECNQueue.Queue
    :memebers:

This specific markdown file does several things:

• Defines a page name of ECNQueue Documentation
• Defines a structure with sections and subsections by using the the # markdown header

ECNQueue
        ├──Item Class
        │        └───Item Class Functions
        │
        └──Queue Class
                  └───Queue Class Functions

• Uses the ::: flag to signifies which docstrings to pull from the ECNQueue module, in this case:
  • The docstring from the Item class
  • The docstring from any public functions in the Item class
  • The Docstring from the Queue class
  • The docstring from any public functions in the Queue class

4. Save the file and close the editor

Note: The same process can be used for documenting the api.py script, however, mkautodoc has a couple of limitations

1. Mkautodoc does not document private functions (any function begining with __)
2. Mkautodoc requires an explicit mention of each class to autogenerate html based on the docstrings within that class, meaning it is not possible to automatically pull all of the classes within ECNQueue by just including ::: ECNQueue in the markdown file.

---

Viewing the Documentation

Now that mkdocs is configured to use mkautodoc and now that there are markdown documents in the webqueue2/api/documentation/docs directory, it is possible to see the html output of mkdocs.

Note: *The starting directory for commands should be the webqueue2/api/documentation/docs directory

webqueue2
    └───api 
        ├──API.py
        ├──ECNQueue.py
        ├──setup.py
        │
        └───documentation 
             ├──mkdocs.yml
             │
             └───docs <<
                  ├───ecnqueue.md
                  └───index.md

1. Navigate to the webqueue2/api/documentation directory, which contains the mkdocs.yml file
To use mkdocs commands, you must be in a directory that was created with the mkdocs new [dir_name] command

cd ..

2. Create a webpage using mkdocs
This command creates a locally available website by interpreting everything within the webqueue2/api/documentation directory.

mkdocs serve

Output:

INFO    -  Building documentation... 
INFO    -  Cleaning site directory 
INFO    -  Documentation built in 0.37 seconds 
[I 201218 15:57:24 server:335] Serving on http://127.0.0.1:8000
INFO    -  Serving on http://127.0.0.1:8000
[I 201218 15:57:24 handlers:62] Start watching changes
INFO    -  Start watching changes
[I 201218 15:57:24 handlers:64] Start detecting changes
INFO    -  Start detecting changes

This creates a locally available website at http://127.0.0.1:8000 that can be used to preview changes made to any of the markdown files. If a change is made to a markdown file and saved while the mkdocs serve command is running, the website will be automatically updated to reflect the newly saved changes. The home page of the website is index.md, which can be modified to better reflect the purpose of the website. The previously created autodocumentation can now be viewed from the ECNQueue Documentation page on the website.

Note: if a change is made to the docstrings of one of the python scripts being autodocumented, there are a couple more steps to get the changes to take affect on the website preview.

---

Modifying the Docstrings

Unfortunately mkautodoc does not detect any changes made within the docstrings of the python scripts, and the solution includes stopping the mkdocs serve command and reinstalling the webqueueapi package. Here is a simplified way to do that:

1. Stop the mkdocs serve command with ctrl + c within the terminal
This will stop broadcasting the website and you will not be able to preview the website if this command is not running

2. Execute this command:
Note: This command should be run from the webqueue2/api/documentation directory, otherwise it will throw an error

This command will automatically reinstall the webqueueapi package and re-run the mkdocs serve command

cd .. && pip install -e . && cd documentation && mkdocs serve

[campb303]

mkdocs w/ material theme has been configured for baseline operations and currently looks like this:

On Desktop:

Docs	Search

On Mobile:

Docs	Search

This tools is extraordinarily easy to use and produces a quality site. I think it would be worth the effort to make this work for all docs related to this project.

I'm currently working on getting the navigation menu to work with a mixture of explicit document links and auto generated directory structures so that we can control the order of sections when we want to and place things like the Getting Started page first but not have to list every single file in each subsection.

[campb303]

Progress has been made on a robust nav section generator but it is still a work in progress.

My first approach was to use a Python script to build a nav structure for the mkdocs.yml config file and convert the Python data structure into YAML using PyYAML. However, I ran into several issues with this route including:

• PyYAML does not preserve YAML comments between loading the source file and dumping new content back. This means we'd lost the whitespace formatting and explanations for settings which would be a step backward.
• PyYAML does not preserve the original order of the source YAML which also reduced readability. This can technically be addressed by using PyYAML data representers as explained in this Stack Overflow post but this method requires additional data to be stored in the config file that mkdocs doesn't support.

My current thinking is to avoid loading the entire configuration file and instead generate just the nav section of the config file and append it to the end of the config file before running mkdocs. This could be automated with an npm task entry. I will try this tomorrow.

[campb303]

Work was done in #157 to help with automated docs generation.

[campb303]

Next Steps:

• Implement offline search See: https://squidfunk.github.io/mkdocs-material/setup/setting-up-site-search/#offline-search
• Add docs installation guide to docs site
• Add docs for adding docs to docs site
• Look at mkautodoc config

[campb303]

I'd originally thought that the offline search mentioned above was required for search capabilities to be usable when serving the docs site from anything other than the development server available by running mkdocs serve but I was wrong. Search is only unavailable when viewing the site from a local filesystem file the file:// protocol due to how modern browsers restrict JavaScript. Therefore, making the built docs site available on a web server will allow for search.

While it might be nice to have a locally searchable copy of the docs, enabling local search via the file:// protocol would require a change in the URL structure for MkDocs. I think it is more important that the docs be easily linkable than locally searchable and subsequently won't implement local search.

[campb303]

This is waiting on me.

[campb303]

Next steps:

• Integrate mkdocs with npm scripts
• Add docs installation guide to docs site
• Add docs for adding docs to docs site
• Add support for versioned docs
• Add docs building to the release routine

[campb303]

npm scripts for interacting with mkdocs have been added:

Command	Action
api:start:docs	This will start a local server on localhost:8000 to access the API docs. As you change API documentation files in api/documentation/docs you'll see your changes in the browser.
api:build:docs	This will output a static bundle of the API documentation in api/documentation/site that can be put on any webserver.
api:kill:docs	Kills the runaway API docs process(es).

[benne238]

Versioning

The feature for adding a dropdown menu to allow easy access to different versions of documentation is a premium feature and the benefit of implementing a simple way to access previous versions of docs does not out-weigh the cost of doing so.

To mitigate this problem, the workflow for making docs will go as follows:

• Create a feature branch
• Make necessary code changes and update documentation in feature branch
• Merge the updated documentation and code from the feature branch back into staging

This workflow will ensure that the most up-to-date code will always have the most up-to-date docs associated with it

[campb303]

Moving

Add docs building to the release routine

to #11

[campb303]

Next Steps (after API docs #195 ):

• Add docs installation guide to docs site
• Add docs for adding docs to docs site

[campb303]

Moved to ECN/webqueue2-api#8