diff --git a/Dev Environment Setup Guide.md b/Dev Environment Setup Guide.md index 21789fa..f2e6be4 100644 --- a/Dev Environment Setup Guide.md +++ b/Dev Environment Setup Guide.md @@ -1,119 +1,250 @@ -# Dev Environment Setup Guice +# Dev Environment Setup Guide +This document will walk you through the setup and configuration of a development environment for webqueue2 using the provided development machines, SSH authentication, GitHub and VS Code. + +# Table of Contents + + + +- [Prerequisites](#prerequisites) + - [On Your Computer](#on-your-computer) + - [On The Development Computer](#on-the-development-computer) +- [Configuring Your SSH Keys](#configuring-your-ssh-keys) +- [Configuring SSH](#configuring-ssh) +- [Adding SSH Keys](#adding-ssh-keys) +- [Installing VS Code](#installing-vs-code) +- [Connecting To The Development Machine](#connecting-to-the-development-machine) +- [Adding Your SSH Keys to GitHub](#adding-your-ssh-keys-to-github) +- [Cloning and Opening the Repository](#cloning-and-opening-the-repository) +- [Configuring VS Code](#configuring-vs-code) + - [Installing Extensions](#installing-extensions) + - [Frontend Extensions:](#frontend-extensions) + - [Backend/API Extensions:](#backendapi-extensions) + - [Configuring Extensions](#configuring-extensions) + - [Python](#python) + - [Python Docstring Generator](#python-docstring-generator) +- [Configuring Tools](#configuring-tools) +- [Using Tools](#using-tools) + + ## Prerequisites + +### On Your Computer - [VS Code](https://code.visualstudio.com/download) +- [Cisco AnyConnect for WebVPN](https://webvpn.purdue.edu/) + +### On The Development Computer +**Note:** These are already installed on the provided development machines. - [git](https://git-scm.com/downloads) -- [NodeJS](https://nodejs.org/en/) >= v14.2.0 -- npm >= v6.14.4 -- npx >= v6.14.4 -- [Python](https://www.python.org/downloads/) >= 3.7 +- [NodeJS](https://nodejs.org/en/) >= v14.11.0 +- npm >= v6.14.8 +- npx >= v6.14.8 +- [Python](https://www.python.org/downloads/) == 3.6.9 +## Configuring Your SSH Keys +We will be using SSH keys to authenticate to both the development machines and GitHub. -## Setup -- The webqueue2 API uses `gunicorn` as a WSGI server and this does not run on Windows. If you're developing on Windows use a virtual machine. +In either PowerShell on Windows or bash on Linux, run: +``` +ssh-keygen +``` +This will create the files `id_rsa` and `id_rsa.pub` in the `.ssh` folder inside your user's folder. Your user's folder can usually be found at: -- webqueue2 uses npm and create-react-app to manage itself. You can learn more about npm [here](https://nodesource.com/blog/an-absolute-beginners-guide-to-using-npm/) and create-react-app [here](https://create-react-app.dev/docs/getting-started/). +- `C:\Users\` (Windows) +- `/Users/` (macOS) +- `/home/` (Linux) ---- +Most CLI shells like PowerShell and bash use the tilda ~ key as a shortcut for your user's folder. -### Step 1: Close the Reposistory +You can confirm these files were created by running: ``` -git clone https://github.itap.purdue.edu/ECN/webqueue2.git +ls ~/.ssh/ ``` -**Note:** github.itap.purdue.edu is only accessible on Purdue's campus or via the [AnyConnect webvpn](https://webvpn.purdue.edu/). -### Step 2: Move to the project directory -``` -cd webqueue2 -``` +## Configuring SSH +In your editor of choice, create the file `~/.ssh/config` and add this: -### Step 3: Install npm Packages -``` -npm install +**Important:** Replace `campb303` with your career account username and replace `w2vm1` with the name of the provided development machine you're connecting to. + +```bash +Host campb303-w2vm1 + HostName w2vm1.ecn.purdue.edu + User campb303 + # Forward webqueue2 Frontend Port + LocalForward 3000 localhost:3000 + # Forward webqueue2 API Port + LocalForward 5000 localhost:5000 + # Forward webqueue2 Docs Port + LocalForward 6060 localhost:6060 ``` -#### Using npm -There are four npm scripts included in this project: +The configuration above will allow you to connect to the development machine and automatically forward ports for development tools to work. Here's a bit more detail about what's going on: -- `npm run start:frontend`: This will start a development server on [localhost:3000](http://localhost:3000). (If the server is on your local machine, this will also launch your default browser at that address.) As you save changes in /public and/or /src you'll see your changes in the browser. (API requests are automatically proxied to the API server.) +| Key | Value | +| - | - | +| Host | A friendly name to identify an SSH host by. This can be anything. | +| HostName | The DNS name or IP address of the computer you're connecting to. | +| User | The name of your user on the computer you're connecting to. | +| LocalForward | Forwards a port on your computer to a port on the computer you're connecting to. | -- `npm run start:api`: This will start a local WSGI server on [localhost:5000](http://localhost:5000) to access the API. **Note:** You will need to add Python to your PATH variable if it is not already. +**Note:** You need to be connected to WebVPN before SSH'ing into a the development machine. -- `npm run build`: This will build a static version of the site in /build ready to be placed in the document root of any web server. +To test your configuration, run `ssh` followed by the `Host`value. When prompted for your password, enter your career account password and press Enter. -- `npm run test`: This will run any tests (using the [Jest](https://jestjs.io/) tester). There are currently no defined tests. +**Note:** You will not see your password being typed but it is being typed. -### Step 4: Setup Python for API -Go to the API directory +For the configuration above you would run: ``` -cd api/ +ssh campb303-w2vm1 +campb303@w2vm1's password: ``` -Create Python virtual environment +## Adding SSH Keys +Replace `HOST` below with the value from above. + +**In PowerShell on Windows:** +```powershell +type $env:USERPROFILE\.ssh\id_rsa.pub | ssh HOST "cat >> .ssh/authorized_keys" ``` -python3 -m venv venv + +**In bash on macOS/Linux:** +```bash +ssh-copy-id HOST ``` -Activate the Python virtual environment +If the key was added successfully, you can login without entering a password by running: ``` -source venv/bin/activate +ssh HOST ``` -If you're pip version is less than 19, upgrade pip -``` -(venv) campb303@acererak [~/webqueue2/api] -$ pip --version -pip 9.0.1 from /home/pier/e/campb303/webqueue2/api/venv/lib/python3.6/site-packages (python 3.6) +## Installing VS Code +Download and install [VS Code](https://code.visualstudio.com/download). Be sure to add `code` to your PATH. + +**On Windows:** this is a a checkbox in the installer. + +![VS Code Install Options on Windows](https://i0.wp.com/www.techomoro.com/wp-content/uploads/2019/06/5.jpg?w=596&ssl=1) + +_Image from [this article on Techomoro](https://www.techomoro.com/installing-visual-studio-code-on-windows-10/)_ + +**On macOS/Linux:** this is accessible by searching for "PATH" in the Command Pallete. You can access the Command Pallete with the keyboard shortcut Command/Ctrl + Shift + P: + +![VS Code Install Options on macOS/Linux](https://i.stack.imgur.com/Ng886.png) + +_Image from [this StackOverflow thread](https://stackoverflow.com/questions/30065227/run-open-vscode-from-mac-terminal)_ + +## Connecting To The Development Machine +Install the [Remote - SSH](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh) plugin. After installation a new icon will appear in the sidebar: + +![Remote SSH Plugin Icon](./docs/Dev%20Environment%20Setup%20Guide/Remote%20SSH%20Icon.png) + +- Click on the Remote SSH icon and you'll see the SSH host you just defined +- Select the host and click the New Window icon to connect. A new window connected to that host will appear + +**Note:** This may take a couple of minutes on the first connection while VS Code installs its server + +![Remote SSH Connection Example](./docs/Dev%20Environment%20Setup%20Guide/Connect%20to%20Remote%20SSH%20Host.gif) + +## Adding Your SSH Keys to GitHub +Because the development machine will be the computer that connects to GitHub, we need to create another SSH key and add it to our GitHub profile: -(venv) campb303@acererak [~/webqueue2/api] -$ pip install --upgrade pip -... -Successfully installed pip-20.2.2 +Using the integrated terminal in VS Code, run: +```bash +ssh-keygen ``` -Install Python requiements +This will create the files `id_rsa` and `id_rsa.pub` in `~/.ssh/`. You can confirm this by running: ``` -pip install -r requirements.txt +ls ~/.ssh/ ``` -**Note:** You can deactivate the Python virtual environment by running `deactivate`. +- Go to https://github.itap.purdue.edu/settings/keys +- Click the "New SSH Key" button +- Give the key a title that tells you what device the key is from (e.g. Desktop or Dev Machine) +- Paste the contents of `id_rsa.pub` into the key box +- Click the "Add SSH Key" button -### Step 5: Install VS Code extentions +![Add SSH Key to GitHub](docs/Dev%20Environment%20Setup%20Guide/Add%20SSH%20Key%20to%20GitHub.gif) -#### Installing Plugins -- [ES Lint](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint) for JavaScript linting -- [Babel JavaScript](https://marketplace.visualstudio.com/items?itemName=mgmcdermott.vscode-language-babel) for JavaScript and JSX syntac highlighting and error checking +## Cloning and Opening the Repository +Using the integrated terminal in VS Code, run: +```bash +git clone git@github.itap.purdue.edu:ECN/webqueue2.git +``` +This will create a `webqueue2` folder in your user's home directory. Open this directory using the "Open Folder" button: + +![Open Repo](./docs/Dev%20Environment%20Setup%20Guide/Open%20Repo.gif) + +**Note:** When you open VS Code next it will try to reconnect to the last SSH host it was connected to. + +**Note:** If you need to reconnect manually, there will not be an option to open this folder directly from the Remote Hosts menu: + +![Remote Folder Open](./docs/Dev%20Environment%20Setup%20Guide/Remote%20Folder%20Open.png) + +## Configuring VS Code + +### Installing Extensions +Install the following extensions: - [Python](https://marketplace.visualstudio.com/items?itemName=ms-python.python) for working with Python virtual environments and debugging +- [Git Graph](https://marketplace.visualstudio.com/items?itemName=mhutchie.git-graph) for a more detailed view of git info +- [Markdown Preview GitHub Styling](https://marketplace.visualstudio.com/items?itemName=bierner.markdown-preview-github-styles) for previewing Markdown as it will appear on GitHub + +#### Frontend Extensions: +- [ES Lint](https://marketplace.visualstudio.com/items?itemName=dbaeumer.vscode-eslint) for JavaScript linting +- [Babel JavaScript](https://marketplace.visualstudio.com/items?itemName=mgmcdermott.vscode-language-babel) for JavaScript and JSX syntax highlighting and error checking +- [MDX](https://marketplace.visualstudio.com/items?itemName=silvenon.mdx) to provide MDX syntax support + +#### Backend/API Extensions: - [Python Docstring Generator](https://marketplace.visualstudio.com/items?itemName=njpwerner.autodocstring) for generating Google style docstrings according to section 3.8 of [Google's Python style guide](https://google.github.io/styleguide/pyguide.html#s3.8-comments-and-docstrings) -Optionally: -- [Remote - SSH](https://marketplace.visualstudio.com/items?itemName=ms-vscode-remote.remote-ssh) for directly editing files on remote machine -- [Markdown Preview GitHub Styling](https://marketplace.visualstudio.com/items?itemName=bierner.markdown-preview-github-styles) for previewing Markdown as it will appear on GitHub +### Configuring Extensions -## Configuring Environemnt -ES Lint, Babel JavaScript and Markdown Preview GitHub Styling will work without further configuration +#### Python +The Python extension supports virtual environments but needs to be told where the virtual environment is. -### Step 1: Configure VS Code's Python Extension -Create a folder called .vscode in the project root -``` -mkdir .vscode +Create or modify the file `.vscode/settings.json` and add the following: +```json +"python.pythonPath": "./api/venv/bin/python3" ``` -Create a file in the .vscode folder called settings.json -``` -nano .vscode/settings.json -``` +**Note:** The Python extension may complain and tell you to select another interpreter. Ignore this warning for now. + +#### Python Docstring Generator +For consistentcy, we'll use a docstring template. -Add the path to the virtual environment Python interpreter +Create or modify the file `.vscode/settings.json` and add the following: +```json +"autoDocstring.customTemplatePath": "./docstring-format.mustache" ``` + +At this point, your `.vscode/settings.json` file should look like this: +```json { - "python.pythonPath": "./api/venv/bin/python3" + "python.pythonPath": "/Users/justincampbell/GitHub/webqueue2/api/venv/bin/python3", + "autoDocstring.customTemplatePath": "./docstring-format.mustache" } ``` -For more information on the Python extension and details on how to use the debugger see: [Getting Started with Python in VS Code](https://code.visualstudio.com/docs/python/python-tutorial) for more info. -### Step 2: Configure Python Docstring Generator -In Settings > Extensions> Python Docstring Generator set the Custom Template Path to `./docstring-format.mustache` +## Configuring Tools +Install npm dependencies: +```bash +npm install +``` + +Create the Python virtual environment: +```bash +npm run venv:create +``` + +## Using Tools +All of the tools in this project are accessible as an npm task so you can interact with them by running `npm run `. The tasks are: -### Step 3 (Optional): Configure Remote - SSH Plugin -Remote - SSH will requires hosts to be added to your SSH config file. See [this guide](https://linuxize.com/post/using-the-ssh-config-file/) for more info. \ No newline at end of file +| Task | Description | +| - | - | +| `start:frontend` | This will start a development server on [localhost:3000](http://localhost:3000). (If the server is on your local machine, this will also launch your default browser at that address.) As you save changes in the `/public` or `/src` directories you'll see your changes in the browser. (API requests are automatically proxied to the API server.) | +| `start:api` | This will start a local WSGI server on [localhost:5000](http://localhost:5000) to access the API. You can interact with the API using `curl` or [Postman](https://www.postman.com/) | +| `start:docs` | This will start a local server on [localhost:6060](http://localhost:6060) showing you the React component documentation. As you change a components documentation file in `/src/components//.md` you'll see your changes in the browser. | +| `build:frontend` | This will output a static bundle of the frontend in `/build` that can be put on any webserver. | +| `build:docs` | This will output a static bundle of the frontend documentation in `/styleguidist/frontend-docs` that can be put on any webserver. | +| `venv:create` | This will create a virtual environment in `/api/venv` and install requirements from `/api/requirements.txt`. | +| `venv:delete` | This will delete the folder `/api/venv`. | +| `venv:reset` | This will run `venv:delete` then `venv:create`. | \ No newline at end of file diff --git a/README.md b/README.md index d7aa1ad..36e6329 100644 --- a/README.md +++ b/README.md @@ -1,15 +1,12 @@ # webqueue2 A re-write of Purdue ECN's webqueue -![UI Snapshot](./docs/UI%20Snapshots/UI-Snapshot%202020-08-19%20at%202.10.48%20PM.png) +![UI Snapshot](./docs/UI%20Snapshots/UI-Snapshot%202020-09-22%20at%201.48.58%20PM.png) ## Stay Up To Date -See what's being worked on with [the webqueue2 Project](https://github.itap.purdue.edu/ECN/webqueue2/projects/4). +See what's being worked on with [the webqueue2 Project](https://github.itap.purdue.edu/ECN/webqueue2/projects/). See and participate in the conversation by viewing the [issues](https://github.itap.purdue.edu/ECN/webqueue2/issues). -## Tech Specs -Stay up to date with the [Dev Logs](https://github.itap.purdue.edu/ECN/webqueue2/blob/master/Dev%20Logs.md). - ## Get Involved Reading the [Dev Environment Setup Guide](https://github.itap.purdue.edu/ECN/webqueue2/blob/master/Dev%20Environment%20Setup%20Guide.md) diff --git a/docs/Dev Environment Setup Guide/Add SSH Key to GitHub.gif b/docs/Dev Environment Setup Guide/Add SSH Key to GitHub.gif new file mode 100644 index 0000000..b812c5b Binary files /dev/null and b/docs/Dev Environment Setup Guide/Add SSH Key to GitHub.gif differ diff --git a/docs/Dev Environment Setup Guide/Connect to Remote SSH Host.gif b/docs/Dev Environment Setup Guide/Connect to Remote SSH Host.gif new file mode 100644 index 0000000..3d644dd Binary files /dev/null and b/docs/Dev Environment Setup Guide/Connect to Remote SSH Host.gif differ diff --git a/docs/Dev Environment Setup Guide/Open Repo.gif b/docs/Dev Environment Setup Guide/Open Repo.gif new file mode 100644 index 0000000..08cd8e3 Binary files /dev/null and b/docs/Dev Environment Setup Guide/Open Repo.gif differ diff --git a/docs/Dev Environment Setup Guide/Remote Folder Open.png b/docs/Dev Environment Setup Guide/Remote Folder Open.png new file mode 100644 index 0000000..6a53201 Binary files /dev/null and b/docs/Dev Environment Setup Guide/Remote Folder Open.png differ diff --git a/docs/Dev Environment Setup Guide/Remote SSH Icon.png b/docs/Dev Environment Setup Guide/Remote SSH Icon.png new file mode 100644 index 0000000..32f210c Binary files /dev/null and b/docs/Dev Environment Setup Guide/Remote SSH Icon.png differ diff --git a/docs/Example GIF/Example GIF.gif b/docs/Example GIF/Example GIF.gif new file mode 100644 index 0000000..d1fea10 Binary files /dev/null and b/docs/Example GIF/Example GIF.gif differ diff --git a/docs/UI Snapshots/UI-Snapshot 2020-09-22 at 1.48.58 PM.png b/docs/UI Snapshots/UI-Snapshot 2020-09-22 at 1.48.58 PM.png new file mode 100644 index 0000000..f9226f2 Binary files /dev/null and b/docs/UI Snapshots/UI-Snapshot 2020-09-22 at 1.48.58 PM.png differ diff --git a/utils/venv-manager.py b/utils/venv-manager.py index d19da5c..6de0bd8 100644 --- a/utils/venv-manager.py +++ b/utils/venv-manager.py @@ -102,7 +102,7 @@ def run_logged_subprocess(command: Union[str, list], timeout: int = 10, shell: b """ logger.debug(f"Entering subprocess for '{command}'") with subprocess.Popen(command,\ - stderr=subprocess.STDOUT, stdout=subprocess.PIPE, shell=shell, text=True)\ + stderr=subprocess.STDOUT, stdout=subprocess.PIPE, shell=shell, universal_newlines=True)\ as logged_shell_process: subprocess_log_prefix = f"(PID: {logged_shell_process.pid})"