NCEP_NCAR_Reanalysis_air.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Creating Interactive Plots of Mean Air Temperature\n",
    "\n",
    "[Video walkthrough](https://mediaspace.itap.purdue.edu/media/Dask%20Demo%202%20-%20Creating%20Interactive%20Plots%20of%20Mean%20Air%20Temperature/1_2u3j7kyk)\n",
    "\n",
    "This is a demo that involves reading multiple netCDF files, calculating means, downsampling data, and creating interactive plots using `matplotlib` and `hvPlot`.\n",
    "\n",
    "The demo notebook uses the following Python dependencies:\n",
    "\n",
    "```\n",
    "dask==2021.6.0\n",
    "dask-core==2021.6.0\n",
    "distributed==2021.6.0\n",
    "hvplot\n",
    "ipywidgets\n",
    "matplotlib\n",
    "netcdf4\n",
    "xarray\n",
    "```\n",
    "\n",
    "**If you're running this notebook on Binder, no setup is needed.** \n",
    "\n",
    "If you're running in another notebook environment, you need to install some dependencies and download data files. Please follow the [setup guide](https://pages.github.itap.purdue.edu/xiao253/dask-demo/#user-guides)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Dataset\n",
    "\n",
    "This demo notebook uses the NCEP/NCAR Reanalysis dataset. Learn more about the dataset at: https://www.climate.gov/maps-data/dataset/average-wind-speeds-map-viewer"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Start a Local Dask Cluster\n",
    "\n",
    "The \"local\" Dask cluster runs on the notebook server."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Suppress warnings, which are generally safe to ignore\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from dask.distributed import Client, LocalCluster, progress\n",
    "import xarray as xr\n",
    "\n",
    "# This will automatically determine the number of workers based on available CPU cores\n",
    "cluster = LocalCluster()\n",
    "# Or, manually specify the desired number of workers\n",
    "# cluster = LocalCluster(n_workers=12)\n",
    "\n",
    "client = Client()\n",
    "client"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load Multiple netCDF Files\n",
    "\n",
    "Here we load all NetCDF files that contains data from 2010 to 2019 (10 files total).\n",
    "\n",
    "Note: the following steps use Dask."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Open all netCDF files and load sea ice concentration variable "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import glob\n",
    "\n",
    "# build file list\n",
    "netcdf_list = glob.glob('data/air.sig995.*.nc') + glob.glob('data/pres.sfc.*.nc')\n",
    "netcdf_list"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import xarray as xr\n",
    "\n",
    "# read files into xarray Dataset\n",
    "ds = xr.open_mfdataset(paths=netcdf_list)\n",
    "ds"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# load mean daily air temperature variable into xarray DataArray\n",
    "da = ds['air']\n",
    "da"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Calculate & Plot Mean *Monthly* Air Temperature"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Calculate mean *monthly* air temperature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "da_month = da.groupby('time.month').mean()\n",
    "\n",
    "da_month = da_month.persist()  # persist mean DataArray in memory\n",
    "progress(da_month)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "da_month"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot mean *monthly* air temperature with `hvPlot`\n",
    "\n",
    "`hvPlot` creates nice interactive plots with many features, including pan, zoom, and hover to view raw data.\n",
    "\n",
    "Performance is good (plot generation takes less than 1 second) when `da_month` DataArray is persisted in memory."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Interative plot**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import hvplot.xarray\n",
    "import panel.widgets as pnw\n",
    "\n",
    "slider = pnw.IntSlider(name='month', start=1, end=12)\n",
    "\n",
    "da_month.interactive.sel(month=slider).hvplot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Looping plot**\n",
    "\n",
    "The player is set to step once every second and loop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import hvplot.xarray\n",
    "import panel.widgets as pnw\n",
    "\n",
    "player = pnw.Player(name='month', start=1, end=12, loop_policy='loop', interval=1000)\n",
    "\n",
    "da_month.interactive.sel(month=player).hvplot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot mean *monthly* air temperature with `matplotlib`\n",
    "\n",
    "`matplotlib` works well as a simple, standalone plot in Jupyter Notebook (legacy, not JupyterLab) environments.\n",
    "\n",
    "However, when used in JupyterLab or with interactive elements (`Slider` or `Player`), the plots are not interactive. To create interactive plots in these environments, consider `hvPlot`.\n",
    "\n",
    "Performance is good (plot generation takes less than 1 second) when `da_month` DataArray is persisted in memory."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Interative plot**"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot is not interactive when Player or Slider is added\n",
    "%matplotlib notebook\n",
    "import panel.widgets as pnw\n",
    "\n",
    "slider = pnw.IntSlider(name='month', start=1, end=12)\n",
    "\n",
    "da_month.interactive.sel(month=slider).plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Looping plot**\n",
    "\n",
    "The player is set to step once every second."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "import panel.widgets as pnw\n",
    "\n",
    "player = pnw.Player(name='month', start=1, end=12, loop_policy='loop', interval=1000)\n",
    "\n",
    "da_month.interactive().sel(month=player).plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Downsample & Plot Mean *Monthly* Air Temperature"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Downsample mean *daily* air temperature variable to *monthly*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "da_month = da.resample(time='1m').mean()\n",
    "\n",
    "da_month = da_month.persist()  # persist mean DataArray in memory\n",
    "progress(da_month)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "da_month"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot mean *monthly* air temperature with `hvPlot`\n",
    "\n",
    "`hvPlot` creates nice interactive plots with many features, including pan, zoom, and hover to view raw data.\n",
    "\n",
    "Performance is good (plot generation takes less than 1 second) when `da_month` DataArray is persisted in memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import hvplot.xarray\n",
    "import panel.widgets as pnw\n",
    "\n",
    "da_month.interactive.sel(time=pnw.DiscreteSlider).hvplot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Plot mean *monthly* air temperature with `matplotlib`\n",
    "\n",
    "`matplotlib` works well as a simple, standalone plot in Jupyter Notebook (legacy, not JupyterLab) environments.\n",
    "\n",
    "However, when used in JupyterLab or with interactive elements (`Slider` or `Player`), the plots are not interactive. To create interactive plots in these environments, consider `hvPlot`.\n",
    "\n",
    "Performance is good (plot generation takes less than 1 second) when `da_month` DataArray is persisted in memory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot is not interactive when Player or Slider is added\n",
    "%matplotlib notebook\n",
    "import panel.widgets as pnw\n",
    "\n",
    "da_month.interactive.sel(time=pnw.DiscreteSlider).plot()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "dask-demo",
   "language": "python",
   "name": "dask-demo"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}