{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "<style>div.container { width: 100% }</style>\n",
    "<img style=\"float:left;  vertical-align:text-bottom;\" height=\"65\" width=\"172\" src=\"../assets/holoviz-logo-unstacked.svg\" />\n",
    "<div style=\"float:right; vertical-align:text-bottom;\"><h2>Tutorial 5. Composing Plots</h2></div>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "So far we have generated plots using [hvPlot](http://hvplot.pyviz.org), but we\n",
    "haven't discussed what exactly these plots are and how they differ from the\n",
    "output of other libraries offering the `.plot` API. It turns out that as in the\n",
    "previous `pn.interact` example, the `.hvplot()` output is actually a rich,\n",
    "compositional object that can be used in many different ways, not just as an\n",
    "immediate plot. Specifically, hvPlot generates\n",
    "[HoloViews](https://holoviews.org) objects rendered using the\n",
    "[bokeh](https://bokeh.org) backend. In the previous notebook we saw that these\n",
    "objects are rendered as interactive Bokeh plots that support hovering, panning,\n",
    "and zooming.\n",
    "\n",
    "In this notebook, we'll examine the output of hvPlot calls to take a look at\n",
    "individual HoloViews objects. Then we will see how these \"elements\" offer us\n",
    "powerful ways of combining and composing layered visualizations.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Read in the data\n",
    "\n",
    "We'll read in the data as before, and also reindex by time so that we can do\n",
    "more easily do resampling.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import hvplot.dask  # noqa: adds hvplot method to dask objects\n",
    "import hvplot.pandas  # noqa"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from load_data import *\n",
    "\n",
    "df = load_data()\n",
    "df = df.sample(frac=0.01)\n",
    "df.time = df.time.astype(\"datetime64[ns]\")\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "cleaned_df = df.copy()\n",
    "cleaned_df[\"wspd\"] = df.wspd.where(df.wspd > 0)\n",
    "cleaned_reindexed_df = cleaned_df.set_index(cleaned_df.time)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Composing plots\n",
    "\n",
    "In this section we'll start looking at how we can group plots to gain a deeper\n",
    "understanding of the data. We'll start by resampling the data to explore\n",
    "patterns in magnitude of waves over time.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "weekly_count = (\n",
    "    cleaned_reindexed_df.station.resample(\"1W\").count().rename(\"count\")\n",
    ")\n",
    "weekly_count_plot = weekly_count.hvplot(title=\"Count of waves by week\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first thing to note is that with `hvplot`, it is common to grab a handle on\n",
    "the returned output. Unlike the matplotlib based `.plot` API of pandas, where an\n",
    "axis object is returned (with plotting display occuring as a side-effect if\n",
    "matplotlib inline is loaded), grabbing the output of `hvplot` has no\n",
    "side-effects at all (as would be true for typical Python objects as well).\n",
    "\n",
    "When working with the HoloViews object returned by `hvplot`, plotting only\n",
    "occurs when we look at the object itself:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "weekly_count_plot"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we have a handle on this object, we can look at its textual representation\n",
    "by printing it:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(weekly_count_plot)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is HoloViews notation for saying that the plot is a `Curve` element with\n",
    "`time` as the key dimension (`kdim`) and `count` as the value dimension\n",
    "(`vdim`).\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "weekly_mean_wspd = cleaned_reindexed_df.wspd.resample(\"1W\").mean()\n",
    "weekly_mean_wspd_plot = weekly_mean_wspd.hvplot(title=\"Weekly Mean Wind Speed\")\n",
    "weekly_mean_wspd_plot"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(weekly_mean_wspd_plot)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This plot has time on the x axis like the other, but the value dimension is\n",
    "magnitude rather than count. Holoviews supports composing plots from individual\n",
    "elements using the `+` symbol to see them side-by-side with linked axes for any\n",
    "shared dimensions:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "(weekly_mean_wspd_plot + weekly_count_plot).cols(1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Try zooming in and out to explore the linking between the plots above.\n",
    "\n",
    "Interestingly, there are three clear peaks in the monthy counts, and two of them\n",
    "correspond to sudden dips in the mean magnitude, while the third corresponds to\n",
    "a peak in the mean magnitude.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise\n",
    "\n",
    "Use tab completion to explore `weekly_count_plot`.\n",
    "\n",
    "<details><summary>Hint</summary><br>\n",
    "\n",
    "Try accessing .data:\n",
    "\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Adding a third dimension\n",
    "\n",
    "Now let's filter the waves to only include the really gusty ones. We can add\n",
    "extra dimensions to the visualization by using color in addition to x and y.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import hvplot.pandas"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is how you can color by wave height using pandas `.plot` API:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "most_severe = df[df.gst >= 10]\n",
    "%matplotlib inline\n",
    "most_severe.plot.scatter(x='longitude', y='latitude', c='gst')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here is the analogous version using `hvplot` where we grab the handle\n",
    "`high_wspd_scatter` so we can inspect the return value:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "high_wspd_scatter = most_severe.hvplot.scatter(\n",
    "    x=\"longitude\", y=\"latitude\", c=\"gst\"\n",
    ")\n",
    "high_wspd_scatter"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As always, this return value is actually a HoloViews element which has a printed\n",
    "representation:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(high_wspd_scatter)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As mentioned earlier, the notion of a 'scatter' plot implies that there is an\n",
    "_independent variable_ and at least one _dependent variable_. This is reflected\n",
    "in the printed representation where the independent variables are in the square\n",
    "brackets and the dependent ones are in parentheses - we can now see that this\n",
    "scatter object implies that latitude is dependent on longitude, which is\n",
    "incorrect. We will learn more about HoloViews objects in the next notebook, and\n",
    "we'll fix the dimensions below.\n",
    "\n",
    "But first, let's adjust the options to create a better plot. First we'll use\n",
    "[colorcet](https://colorcet.pyviz.org) to get a colormap that doesn't have white\n",
    "at one end, to avoid ambiguity with the page background. We can choose one from\n",
    "the website and use the HoloViews/Bokeh-based colorcet plotting module to make\n",
    "sure it looks good.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import colorcet as cc"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from colorcet.plotting import swatch\n",
    "\n",
    "swatch(\"CET_L4\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll reverse the colors to align dark reds with gustier waves.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "wspd_cmap = cc.CET_L4[::-1]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition to fixing the colormap, we will now switch from `scatter` to using\n",
    "`points` to correctly reflect that longitude and latitude are independent\n",
    "variables, as well as add some additional columns to the hover text, and add a\n",
    "title.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "gusty_points = most_severe.hvplot.points(\n",
    "    x=\"longitude\",\n",
    "    y=\"latitude\",\n",
    "    c=\"gst\",\n",
    "    hover_cols=[\"place\", \"time\"],\n",
    "    cmap=wspd_cmap,\n",
    "    title=\"Wave Heights with gusts >= 10\",\n",
    ")\n",
    "\n",
    "gusty_points"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When you hover over the points you'll see the place and time of the waves in\n",
    "addition to the wind speed and lat/lon. This is reflected in the dimensions that\n",
    "HoloViews is keeping track of:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(gusty_points)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Exercise\n",
    "\n",
    "Compare this `Points` printed representation to the `Scatter` printed\n",
    "representation and note the differences in how the dimensions are grouped\n",
    "together.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the colorcet plotting module `swatches(group='linear')` to choose a\n",
    "different colormap.\n",
    "\n",
    "<details><summary>Hint</summary><br>\n",
    "\n",
    "```python\n",
    "from colorcet.plotting import swatches\n",
    "swatches(group='linear')\n",
    "```\n",
    "\n",
    "</details>\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Overlay with a tiled map\n",
    "\n",
    "That colormap is better, and we can kind of see the outlines of the continents,\n",
    "but the visualization would be much easier to parse if we added a base map\n",
    "underneath. To do this, we'll import a tile element from HoloViews, namely the\n",
    "`OSM` tile from [openstreetmap](https://www.openstreetmap.org/) using the Web\n",
    "Mercator projection:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from holoviews.element.tiles import OSM\n",
    "\n",
    "OSM()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that when you zoom the map becomes more and more detailed, downloading\n",
    "tiles as necessary. In order to overlay on this basemap, we need to project our\n",
    "waves to the Web Mercator projection system.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd\n",
    "\n",
    "from datashader.utils import lnglat_to_meters"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To do this we will use the `lnglat_to_meters` function in the `datashader.geo`\n",
    "module to map `longitude` and `latitude` to `easting` and `northing`\n",
    "respectively:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "x, y = lnglat_to_meters(most_severe.longitude, most_severe.latitude)\n",
    "most_severe_projected = most_severe.join(\n",
    "    [pd.DataFrame({\"easting\": x}), pd.DataFrame({\"northing\": y})]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can now overlay our points on top of the `OSM` tile source but instead of\n",
    "overlaying the tile source explicitly we can also just specify `tiles='OSM'` as\n",
    "a string:\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "most_severe_projected.hvplot.points(\n",
    "    x=\"easting\",\n",
    "    y=\"northing\",\n",
    "    c=\"wspd\",\n",
    "    hover_cols=[\"place\", \"time\"],\n",
    "    cmap=wspd_cmap,\n",
    "    title=\"Waves with gusts >= 10\",\n",
    "    tiles=\"OSM\",\n",
    "    line_color=\"black\",\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that the Web Mercator projection is only one of many possible projections\n",
    "used when working with geospatial data. If you need to work with these different\n",
    "projections, you can use the [GeoViews](http://geoviews.org) extension to\n",
    "HoloViews that makes elements aware of the projection they are defined in and\n",
    "automatically projects into whatever coordinates are needed for display.\n",
    "\n",
    "#### Exercise\n",
    "\n",
    "Import and use different tiles.\n",
    "\n",
    "<details><summary>Hint</summary><br>\n",
    "\n",
    "EsriImagery or Wikipedia.\n",
    "\n",
    "</details>\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python [conda env:root] *",
   "language": "python",
   "name": "conda-root-py"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}