Skip to content

Update dev environment setup guide #28

Closed
wants to merge 71 commits into from
Closed
Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
71 commits
Select commit Hold shift + click to select a range
52061e8
Move package files to src dir
campb303 Mar 18, 2021
07ee438
Add installation instructions to readme
benne238 Mar 12, 2021
85b6f4c
Move package files to its own directory
campb303 Mar 19, 2021
0801ed9
Remove logging and update package path
campb303 Mar 19, 2021
3fa5e70
Add ability to install all conditional dependencies
campb303 Mar 19, 2021
b590473
Add docs from frontend
campb303 Mar 19, 2021
9e8802d
Remove venv-manager docs
campb303 Mar 19, 2021
a9aeef9
Add mkdocs plugin to setup
campb303 Mar 19, 2021
2fe4944
Remove symlink ref
campb303 Mar 19, 2021
38a74e3
Update README with contribution details
campb303 Mar 19, 2021
c5fcdc0
Add venv activation step to README
campb303 Mar 19, 2021
0b4c76a
mixed bad import statement in ecnqueue.md
benne238 Mar 19, 2021
2e9b148
added import statment to __init__.py to make python modules more in s…
benne238 Mar 22, 2021
26a3dff
Add easyad to requirements
campb303 Mar 22, 2021
4430d31
Fix module imports
campb303 Mar 22, 2021
e64f8a9
Change cURL config filename
campb303 Mar 22, 2021
b566910
Add check to prevent admin accounts from signing in
campb303 Mar 22, 2021
896e360
Add note about valid users
campb303 Mar 22, 2021
79c9ba4
Merge pull request #18 from ECN/create-docs-structure
campb303 Mar 24, 2021
c894551
Add variable support to mkdocs
campb303 Mar 24, 2021
b3efaf4
Update docs to include production URL variable
campb303 Mar 24, 2021
1a20353
Update auth docs to include fetch/curl examples for getting, passing …
campb303 Mar 24, 2021
0d7ccfa
style changes to the backend documentation
benne238 Mar 29, 2021
4838cac
fix for .autodoc-docstring css
benne238 Mar 29, 2021
cf7ebff
removed mentions of :private-memebers: from markdown files as autodoc…
benne238 Mar 29, 2021
f3b9d67
added additional information for the overview
benne238 Mar 29, 2021
fdf4aeb
update api.py docstrings
benne238 Mar 29, 2021
a5a185d
Remove JWT Refresh path restriction
campb303 Mar 29, 2021
1fe26c6
Create get_access_token snippet
campb303 Apr 1, 2021
f631916
Merge branch 'update-docs' of github.itap.purdue.edu:ECN/webqueue2-ap…
campb303 Apr 1, 2021
f52d74f
Replace token with include
campb303 Apr 1, 2021
858dd3b
Remove cURL examples, finish basic usage examples
campb303 Apr 2, 2021
d5f573d
Remove cURL examples, simplify javascript examples
campb303 Apr 2, 2021
28ad8a1
Simplify examples, reference mkdocs variables
campb303 Apr 2, 2021
3d6f940
Add API section to table of contents
campb303 Apr 2, 2021
f4b0b89
Remove versioning TODO
campb303 Apr 2, 2021
72c0b8b
Update GitHub link
campb303 Apr 2, 2021
eea653c
Add comment for variables
campb303 Apr 2, 2021
57452f9
Remove unnecessary file
campb303 Apr 2, 2021
e3d40b5
Minor wording changes
campb303 Apr 2, 2021
034bb2d
Create new index file
campb303 Apr 2, 2021
5fdecb5
Clean comments
campb303 Apr 2, 2021
8336ab6
Finish installation docs
campb303 Apr 2, 2021
3f3f241
Remove code formatting for URL
campb303 Apr 2, 2021
47595e1
Update repo URL
campb303 Apr 2, 2021
a6740a4
Finish Items docs
campb303 Apr 3, 2021
468b23a
Finish Queues docs
campb303 Apr 3, 2021
59aea43
Remove MkAutoDocs file
campb303 Apr 4, 2021
897a976
Remove rest operator in nav
campb303 Apr 4, 2021
e1bea81
Add endpoint docs to Item
campb303 Apr 4, 2021
52d91ef
Add enpoint docs to Queue
campb303 Apr 4, 2021
d972fd8
Remove overview doc
campb303 Apr 4, 2021
d635c86
Remove MkAutoDocs plugin
campb303 Apr 4, 2021
4e67580
Remove MkAutoDocs from MkDocs
campb303 Apr 4, 2021
5d885fa
Remove MkAutoDocs styles
campb303 Apr 4, 2021
651da82
Add basic home page
campb303 Apr 4, 2021
ecbfbd6
Merge pull request #22 from ECN/update-docs
campb303 Apr 4, 2021
80ee836
preliminary changes to the dev enviornment setup guide
benne238 May 4, 2021
da4fbb1
making docs preliminary docs
benne238 May 7, 2021
c41325c
preliminary package structure docs
benne238 May 7, 2021
a8b8eae
package structure diagram for the docs
benne238 May 7, 2021
6b9721f
migrate docstring format file from frontend
benne238 May 18, 2021
82b7e95
add dev environment to nav
benne238 May 18, 2021
c78973c
added restart to list of api commands
benne238 May 18, 2021
824a5a7
Add note admonition for tilda usage in shells
campb303 May 18, 2021
22179e8
Update ssh config ports
campb303 May 18, 2021
fd52b94
Remove unneccesary table padding
campb303 May 18, 2021
de34425
Add tip for SSH proxy jump hosts
campb303 May 18, 2021
9f3f413
Simplify extensions list
campb303 May 18, 2021
485de8a
Add more details about virtual environment setup
campb303 May 18, 2021
1fddf8e
Correct and simplify API commands
campb303 May 18, 2021
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
41 changes: 40 additions & 1 deletion README.md
Original file line number Diff line number Diff line change
@@ -1 +1,40 @@
# webqueue2-api
# webqueue2 API
A Python based parser and RESTful API for ECN's webqueue.

## Usage
### Install via pip:
```
pip install git+https://github.itap.purdue.edu/ECN/webqueue2-api@<VERSION>#egg=webqueue2-api
```
For example, to install version 0.9.1:
```
pip install git+https://github.itap.purdue.edu/ECN/webqueue2-api@0.9.1#egg=webqueue2-api
```

### Install via requirements file:
```
git+https://github.itap.purdue.edu/ECN/webqueue2-api@<VERSION>#egg=webqueue2-api
```
For example, to install version 0.9.1:
```
git+https://github.itap.purdue.edu/ECN/webqueue2-api@0.9.1#egg=webqueue2-api
```

## Contributing
1. Clone the git repo:
```bash
git clone https://github.itap.purdue.edu/ECN/webqueue2-api.git
```
2. Create and activate Python virtual environment:
```bash
python3 -m venv venv
source venv/bin/activate
```
3. Install dependencies:
```bash
# Older versions of pip may fail to install newer packages.
pip install -U pip
# Install the webqueue2-api package in editable mode with all extra packages.
pip install -e .[all]
```
4. Make changes and create a PR.
Empty file removed __init__.py
Empty file.
299 changes: 299 additions & 0 deletions docs/Dev Environment Setup Guide.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,299 @@
# Dev Environment Setup Guide
This document will walk you through the setup and configuration of a development environment for webqueue2 using the provided development machines, SSH authentication, GitHub and VS Code.

## Prerequisites

### On Your Computer
- [VS Code](https://code.visualstudio.com/download)
- [Cisco AnyConnect for WebVPN](https://webvpn.purdue.edu/)

### On The Development Computer

!!! Note
These are already installed on the provided development machines.

- [git](https://git-scm.com/downloads)
- [Python](https://www.python.org/downloads/) == 3.6.9

## Configuring Your SSH Keys
We will be using SSH keys to authenticate to both the development machines and GitHub.

In either PowerShell on Windows or bash on macOS/Linux, run the following command and accept all defaults by pressing <kbd>Enter</kbd>:
```none
ssh-keygen
```
This will create the files `id_rsa` and `id_rsa.pub` in the `.ssh` folder inside your user's folder. Your user's folder can usually be found at:

=== "Windows"
```none
C:\Users\<user_name>
```
=== "macOS"
```none
/Users/<user_name>
```
=== "Linux"
```none
/home/<user_name>
```

!!! note
Most CLI shells like PowerShell and bash use the <kbd> ~ </kbd> (tilda) key as a shortcut for your user's folder.

You can confirm these files were created by running:
```none
ls ~/.ssh/
```

## Configuring SSH
In your editor of choice, create the file `~/.ssh/config` and add this:

!!! tip
Replace `campb303` with your career account username and replace `w2vm1` with the name of the provided development machine you're connecting to.

```none
Host campb303-w2vm1
HostName w2vm1.ecn.purdue.edu
User campb303
# Forward webqueue2 API Port
LocalForward 5000 localhost:5000
# Forward webqueue2 API Docs Port
LocalForward 8000 localhost:8000
```

The configuration above will allow you to connect to the development machine and automatically forward ports for development tools to work. Here's a bit more detail about what's going on:

| Key | Value |
| - | - |
| Host | A friendly name to identify an SSH host by. This can be anything. |
| HostName | The DNS name or IP address of the computer you're connecting to. |
| User | The name of your user on the computer you're connecting to. |
| LocalForward | Forwards a port on your computer to a port on the computer you're connecting to. |

Development machines are not publicly accessible. You'll need to be connected to WebVPN to connect to them.

??? tip "SSH Jump Hosts"
WebVPN works on all platforms but requires an extra step to get to work. On macOS/Linux, `pier.ecn.purdue.edu` can be used as an [SSH jump host](https://www.tecmint.com/access-linux-server-using-a-jump-host/) allowing for development machines to be connected to automatically when openning VS Code.

Assuming you already have an SSH key configured for `pier.ecn.purdue.edu`, you can use a jump host by adding the following to `~/.ssh/config`:

```
Host campb303-pier
HostName pier.ecn.purdue.edu
User campb303

Host campb303-w2vm1
HostName w2vm1.ecn.purdue.edu
ProxyJump campb303-pier
User campb303
# Forward webqueue2 API Port
LocalForward 5000 localhost:5000
# Forward webqueue2 API Docs Port
LocalForward 8000 localhost:8000
```

To test your configuration, run `ssh` followed by the `Host` value. When prompted for your password, enter your career account password and press <kbd>Enter</kbd>.

!!! warning
Most shell environemnts like PowerShell and bash do not show your password being typed but it is being typed.

For the configuration above you would run:

```none
ssh campb303-w2vm1
campb303@w2vm1's password:
```

## Adding SSH Keys
Now that we've generated SSH keys and configured our host entries, we need to add our SSH key to the host. This will allow us to connect to these machines without passwords later.

!!! tip
Replace `HOST` below with the value from above. _Example:_ `campb303-w2vm1`

=== "PowerShell on Windows"
```powershell
type $env:USERPROFILE\.ssh\id_rsa.pub | ssh HOST "cat >> .ssh/authorized_keys"
```
=== "bash on macOS/Linux"
```bash
ssh-copy-id HOST
```

If the key was added successfully, you can login without entering a password by running:
```
ssh HOST
```

## Installing VS Code
Download and install [VS Code](https://code.visualstudio.com/download). Be sure to add `code` to your PATH.

=== "Windows"

Adding `code` to your PATH on Windows is a checkbox in the installer:

![VS Code Install Options on Windows](https://i0.wp.com/www.techomoro.com/wp-content/uploads/2019/06/5.jpg?w=596&ssl=1)

_Image from [this article on Techomoro](https://www.techomoro.com/installing-visual-studio-code-on-windows-10/)_

=== "macOS/Linux"

Adding `code` to your PATH on macOS/Linux is accessible by searching for "PATH" in the Command Pallete. You can access the Command Pallete with the keyboard shortcut <kbd>Command</kbd>/<kbd>Ctrl</kbd> + <kbd>Shift</kbd> + <kbd>P</kbd>:

![VS Code Install Options on macOS/Linux](https://i.stack.imgur.com/Ng886.png)

_Image from [this StackOverflow thread](https://stackoverflow.com/questions/30065227/run-open-vscode-from-mac-terminal)_

## Connecting To The Development Machine
Install the [Remote - SSH](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh) plugin. After installation a new icon will appear in the sidebar:

![Remote SSH Plugin Icon](images/Dev%20Environment%20Setup%20Guide/Remote%20SSH%20Icon.png)

- Click on the Remote SSH icon and you'll see the SSH host you just defined
- Select the host and click the New Window icon to connect. A new window connected to that host will appear

!!! note
This may take a couple of minutes on the first connection while VS Code installs its server.

![Remote SSH Connection Example](images/Dev%20Environment%20Setup%20Guide/Connect%20to%20Remote%20SSH%20Host.gif)

If prompted for a platform, select Linux:
![VS Code Select Platform](images/Dev%20Environment%20Setup%20Guide/VS%20Code%20Select%20Platform.png)

## Adding Your SSH Keys to GitHub
Because the development machine will be the computer that connects to GitHub, we need to create another SSH key and add it to our GitHub profile:

First, open VS Code's integrated terminal by pressing <kbd>Control</kbd> + <kbd>~</kbd> (tilda)

![Open VS Code's integrated terminal](images/Dev%20Environment Setup%20Guide/Open%20VS%20Code%20Integrated%20Terminal.gif)

Now run the following command and accept all defaults by pressing <kbd>Enter</kbd>:
```bash
ssh-keygen
```

This will create the files `id_rsa` and `id_rsa.pub` in `~/.ssh/`. You can confirm this by running:
```bash
ls ~/.ssh/
```

Now copy the public key from `id_rsa.pub` by openning the file in VS Code, selecting its contents and copying it. You can open the file in VS Code by running the following in the integrated terminal:

!!! danger
Do not copy your private key from `id_rsa`! This key should never leave your machine.

```bash
code ~/.ssh/id_rsa.pub
```

Now go to [github.itap.purdue.edu/settings/keys](https://github.itap.purdue.edu/settings/keys). You may be prompted to login using your career account username and password.

- Click the "New SSH Key" button
- Give the key a title that tells you what device the key is from (e.g. Desktop or Dev Machine)
- Paste the contents of `id_rsa.pub` into the key box
- Click the "Add SSH Key" button

![Add SSH Key to GitHub](images/Dev%20Environment%20Setup%20Guide/Add%20SSH%20Key%20to%20GitHub.gif)

## Cloning and Opening the Repository
Using the integrated terminal in VS Code, run:
```none
git clone git@github.itap.purdue.edu:ECN/webqueue2-api.git
```
This will create a `webqueue2-api` folder in your user's home directory. Open this directory using the "Open Folder" button:

![Open Repo](images/Dev%20Environment%20Setup%20Guide/Open%20Repo.gif)

!!! note
VS Code will automatically reconnect to the last remote host used when it reopens.

!!! tip
If you need to reconnect manually, there will now be an option to open the webqueue2-api folder directly from the Remote Hosts menu:

![Remote Folder Open](images/Dev%20Environment%20Setup%20Guide/Remote%20Folder%20Open.png)

## Configuring VS Code

### Installing Extensions
- [Python](https://marketplace.visualstudio.com/items?itemName=ms-python.python) for Python language support, IntelliSense, virtual environments and debugging
- [Git Graph](https://marketplace.visualstudio.com/items?itemName=mhutchie.git-graph) for a more detailed view of git info
- [Markdown Preview GitHub Styling](https://marketplace.visualstudio.com/items?itemName=bierner.markdown-preview-github-styles) for previewing Markdown as it will appear on GitHub
- [Python Docstring Generator](https://marketplace.visualstudio.com/items?itemName=njpwerner.autodocstring) for generating Google style docstrings according to section 3.8 of [Google's Python style guide](https://google.github.io/styleguide/pyguide.html#s3.8-comments-and-docstrings)

### Configuring Extensions

#### Python
The Python extension supports virtual environments but needs to be told where the virtual environment is.

Create or modify the file `.vscode/settings.json` and add the following:
```none
"python.pythonPath": "./venv/bin/python3"
```

!!! tip
The Python extension may complain and tell you to select another interpreter. Ignore this warning for now.

#### Python Docstring Generator
For consistentcy, we'll use a docstring template. Create or modify the file `.vscode/settings.json` and add the following:
```none
"autoDocstring.customTemplatePath": "./docstring-format.mustache"
```

At this point, your `.vscode/settings.json` file should look like this:
```none
{
"python.pythonPath": "./venv/bin/python3",
"autoDocstring.customTemplatePath": "./docstring-format.mustache"
}
```

## Setting Up the Virtual Environment
For development, we'll use a [Python Virtual Environment](https://realpython.com/python-virtual-environments-a-primer/) to isolate out changes and have a reproducible dev environment.

In VS Code's integrated terminal:

Create a virtual environment at `./venv/`:
```bash
python3 -m venv venv
```

Activate the virtual environment:
```bash
source venv/bin/activate
```

!!! tip
To deactivate the virtual environment and use your system's Python interpreter, run:

```bash
deactivate
```

Update pip within the virtual environment:
```none
pip install -U pip
```

Install the webqueue2 API within the virtual environemt:
```none
pip install -e .[all]
```

`-e` installs a package in [editable mode](https://pip.pypa.io/en/stable/cli/pip_install/#cmdoption-e) which allows code changes to take effect without reinstalling a package.

webqueue2 API has multiple [conditional dependencies](https://setuptools.readthedocs.io/en/latest/userguide/dependency_management.html#id7):

| Condition | Installation Command | Description |
| - | - | - |
| Production | `pip install webqueue2-api` | For use in production. Only installed needed packages. |
| Development | `pip install webqueue2-api[dev]` | For use in development. Installs everything for production and extra packages for development. |
| Documentation | `pip install webqueue2-api[docs]` | For use in creating documentation. Installs everything for production and extra packages for documentation. |
| All | `pip install webqueue2-api[all]` | A shortcut for installing production, development and documentation dependencies. |

### API Commands
| Command | Description |
| - | - |
| `python3 -m webqueue2_api start-api` | Starts the API on [localhost:5000](http://localhost:5000). See [Getting Started](./api/Getting Started.md) for more info. |
| `python3 -m webqueue2_api stop-api` | Stops the API. |
| `python3 -m webqueue2_api restart-api` | Stops and starts the API. |
| `mkdocs serve` | This will start a local server on [localhost:8000](http://localhost:8000) to access the API docs. As you change API documentation files in `./docs/` you'll see your changes in the browser. |
| `mkdocs build` | This will output a static bundle of the API documentation in `./site/` that can be put on any webserver. |
Loading