Backend Documentation structure

[benne238]

The backend documentation structure needs to be clean and avoid transferring unnecessary files when ever the package is installed. This required defining and managing a documentation either by making a separate branch or by some other means. Currently looking into how other github projects host their documentation.

Anything relating to the implementation of backend documentation will be kept in this issue.

[benne238]

Documentation for Other Projects hosted on GitHub

To get an understanding of how to structure our own docs, the documentation for 6 other projects was compared to understand how docs are structured and to hopefully incorporate that within our own documentation. The 6 projects outlined in this document are:

• Flask
• Flask-RESTful
• Flask-JWT-Extended
• python-dotenv
• python-ldap
• MkDocs

It is worth noting that all of these projects are hosted on pypi as well

Flask

https://github.com/pallets/flask/tree/1.1.x
Note that the current version of flask is kept on the 1.1x branch and not the master branch, which is used for development

This project includes a main readme.md within the root of the repository, which is standard for pretty much all projects. The readme give a basic overview of the project which includes:

• how to install the project
• a simple example of how to use the project
• how to contribute or donate to the project
• external links relating to the project (including an external documentation webpage)

While the documentation is meant to be accessed through the external documentation page, it is worth noting that the documentation that is located on this website is pulled directly from a documentation directory located directly in the repo. This directory seems to be the host for the documentation and it is possible to read the documentation directly from the git repository, but it is written in rst which makes it more effective in an environment like sphinx, which seems to be how the documentation is generated in the external website.

Flask-RESTful

https://github.com/flask-restful/flask-restful
Note: unlike Flask, the current version is kept directly on the master branch

This project does include a readme, however, it is very minimal and serves only to point the reader to the official documentation page. Like Flask, the documentation is written in rst and is meant to viewed through the external documentation webpage. Also like Flask, there is a dedicated documentation directory directly in the repo that seems to be the source for the external documentation webpage

Flask-JWT-Extended

https://github.com/vimalloc/flask-jwt-extended/tree/master
Note the latest stable version is kept in the master branch

This project is very similar to flask in almost every way including:

• A basic readme that outlines the basic features (and points to an external documentation webpage)
• An rst and sphinx based external-documentation-page
• A documentation directory directly in the repo that hosts the docs for the external webpage

python-dotenv

https://github.com/theskumar/python-dotenv

This project differs from the previous three projects in several ways:

• The readme is longer and more in-depth
• There is no docs webpage that the readme points to
• There is no documentation directory in the repo that hosts the documentation for the project

It seems that this project is small enough and it's uses are limited enough for the documentation to be a single readme.

python-ldap

https://github.com/python-ldap/python-ldap

This project is similar to Flask in pretty much everyway including:

• A basic readme that outlines the basic features (and points to an external documentation webpage)
• An rst and sphinx based external-documentation-page
• A documentation directory directly in the repo that hosts the docs for the external webpage

MkDocs

https://github.com/mkdocs/mkdocs

This project is also similar to Flask in pretty much everyway including:

• A basic readme that outlines the basic features (and points to an external documentation webpage)
• An external-documentation-page (based in markdown instead of rst and sphinx like Flask)
• A documentation directory directly in the repo that hosts the docs for the external webpage

Conclusion

It seems like it is common practice to store documentation for a given project in the branch that the documentation corresponds to, and in all 6 projects, no documentation is stored on a separate branch.

However, it is also common practice to serve out the documentation on an external website as opposed to making the user look through the documentation directly in the repository. So it would seem that the purpose for keeping the documentation directly in the git repository might be for consolidation, but the documentation in almost all the above projects (except for python-dotenv) is the external webpage that the readme's point to.

Where the source files for the external webpage are kept could theoretically be anywhere.

[campb303]

It appears Flask keeps its source code in a src/ directory and uses a function called find_packages with an argument of that directory path to get source code. See: https://github.com/pallets/flask/blob/1.1.x/setup.py#L51-L52

Flask RESTful does a similar thing but passes an exclude kwarg to the find_packages function. See: https://github.com/flask-restful/flask-restful/blob/master/setup.py#L39

Flask JWT Extended explicitly lists the name of the folder that contains the source code files. See: https://github.com/vimalloc/flask-jwt-extended/blob/master/setup.py#L25

python-dotenv also manually lists its module in its setup.py file. See:https://github.com/theskumar/python-dotenv/blob/master/setup.py#L31 It seems that the docs are available via markdown in the repo but also host it via MkDocs at https://saurabh-kumar.com/python-dotenv/. I don't see how they build their docs though.

mkdocs uses get_packages instead of find_packages in its setup.py file. See: https://github.com/mkdocs/mkdocs/blob/master/setup.py#L54 It also has a separeate docs folder for its documentation. assumedly this is built with mkdocs.

So it seems that storing docs within the git repo shouldn't cause any issues. After a quick test, it appears that only the package itself gets installed into a python virtual environment anyhow.

We should rearrange out GitHub repo to look like this:

.
├── ECNQueue.py
├── README.md
├── docs/
├── setup.py
└── webqueue2-api/
    ├── ECNQueue.py
    ├── __init__.py
    └── api.py

[campb303]

Starting to work on this.

[campb303]

This was resolved in #18 by adding a docs folder to the root of of the repo. The workflow for updating docs is now:

1. Clone GitHub repo:

git clone https://github.itap.purdue.edu/ECN/webqueue2-api.git

2. Enter the webqueue2-api directory:

cd webqueue2-api

3. Create a virtual environment:

python3 -m venv venv

4. Activate the virtual environment:

source venv/bin/activate

5. Update pip on ECN hosts b/c it is out of date:

pip install -U pip

6. Install the complete webqueue2-api metapackage:

# The . (dot) references the setup.py script in the same directory.
# The [all] suffix specifies the metapackage. Other options include [docs] and [dev].
# Omitting the suffix and just using . (dot) will install the production version.
pip install .[all]

7. Start MkDocs in dev mode:

mkdocs serve

8. Edit files under docs/ and view them at http://localhost:8000
9. Build a production version of docs for hosting:

# Outputs to `site/` directory.
mkdocs build