examples/notebooks/walkthrough_community_python_event.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "8b4405f3-ff20-45fc-b83b-8ab976285d28",
   "metadata": {},
   "source": [
    "# Exploring Crypto Data Using the Coin Metrics Python API Client \n",
    "## Using Coin Metrics Community Network and Market Data\n",
    "\n",
    "*February 17, 2022*\n",
    "\n",
    "![CM](https://cdn.substack.com/image/fetch/w_96,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F4430351a-a92c-4505-8c8f-3822d76715df_256x256.png)\n",
    "\n",
    "This notebook provides some examples using the Coin Metrics' Python API Client analyzing both on-chain (network) data and market data. For more information on Coin Metrics, the API client, and general Python information, see the resources below and make sure to follow us on Twitter [@coinmetrics](https://twitter.com/coinmetrics) and check out our free [charting tools](https://charts.coinmetrics.io/network-data/).\n",
    "\n",
    "## Resources\n",
    "\n",
    "- The [Coin Metrics API v4](https://docs.coinmetrics.io/api/v4) website contains the full set of endpoints and data offered by Coin Metrics.\n",
    "- The [API Spec](https://coinmetrics.github.io/api-client-python/site/api_client.html) contains a full list of functions, documentation, and the syntax for the Python API client.\n",
    "- The [Coin Metrics Knowledge Base](https://docs.coinmetrics.io/info) provides a list of available assets, metrics, and what is available to Community and Pro users.\n",
    "- The [Coin Metrics Data Encyclopedia](https://docs.coinmetrics.io/) gives detailed, conceptual explanations of the data and metrics that Coin Metrics offers.\n",
    "\n",
    "## Coin Metrics Research \n",
    "- [Coin Metrics State of the Network](https://coinmetrics.substack.com/), our weekly data-driven newsletter highlighting on-chain (network) data\n",
    "- [Coin Metrics State of the Market](https://coinmetrics.io/insights/state-of-the-market/), weekly report contextualizing the week’s crypto market movement\n",
    "- [Other Original Research](https://coinmetrics.io/insights/original-research/) and Long-form Reports\n",
    "\n",
    "## General Python / Data Science Resources\n",
    "- [Kaggle Courses](https://www.kaggle.com/learn) on Python & Pandas \n",
    "- [Pandas Tutorials](https://pandas.pydata.org/pandas-docs/stable/getting_started/tutorials.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "16c3476d-c33a-421e-811e-344b59735ec0",
   "metadata": {},
   "source": [
    "## Setup"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "7fbd2927-9184-41fd-8403-b259146ee1a2",
   "metadata": {},
   "outputs": [],
   "source": [
    "from os import environ\n",
    "import sys\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import seaborn as sns\n",
    "import matplotlib.pyplot as plt\n",
    "import matplotlib.image as image\n",
    "from matplotlib.ticker import FuncFormatter\n",
    "import matplotlib.dates as mdates\n",
    "import plotly.express as px\n",
    "import logging\n",
    "from datetime import date, datetime, timedelta\n",
    "from coinmetrics.api_client import CoinMetricsClient\n",
    "import json\n",
    "import logging\n",
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "acb66361",
   "metadata": {},
   "outputs": [],
   "source": [
    "# for plotting\n",
    "sns.set_theme()\n",
    "sns.set(rc={'figure.figsize':(12,8)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "abc14b24-03f4-457c-94aa-0c33ff01c79b",
   "metadata": {},
   "outputs": [],
   "source": [
    "logging.basicConfig(\n",
    "    format='%(asctime)s %(levelname)-8s %(message)s',\n",
    "    level=logging.INFO,\n",
    "    datefmt='%Y-%m-%d %H:%M:%S'\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ecbe22c4-ee5a-4130-bc7f-9bf9a61b4bc1",
   "metadata": {},
   "outputs": [],
   "source": [
    "# We recommend privately storing your API key in your local environment.\n",
    "try:\n",
    "    api_key = environ[\"CM_API_KEY\"]\n",
    "    logging.info(\"Using API key found in environment\")\n",
    "except KeyError:\n",
    "    api_key = \"\"\n",
    "    logging.info(\"API key not found. Using community client\")\n",
    "\n",
    "# Initialize the client\n",
    "client = CoinMetricsClient(api_key)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32cfcda6-2546-4965-82a6-bdf857ffc251",
   "metadata": {},
   "source": [
    "## Example 1: Pulling Crypto Prices using Coin Metrics' Reference Rates\n",
    "\n",
    "Data on crypto asset prices are incredibly important. CM's Reference Rates provide prices for over 300 crypto assets calculated in U.S. Dollars and Euros using a transparent and independent methodology, robust to manipulation and derived from high quality constituent markets.\n",
    "\n",
    "The code below shows how to pull CM reference rates for a given asset and time period using the `get_asset_metrics` [endpoint.](https://coinmetrics.github.io/api-client-python/site/api_client.html#get_asset_metrics)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e36f5181",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Query API for bitcoin (BTC) prices, daily CM reference rates as dataframe\n",
    "metrics    = \"ReferenceRateUSD\"\n",
    "frequency  = \"1d\"\n",
    "start_time = \"2021-01-01\"\n",
    "end_time   = \"2022-02-16\"\n",
    "asset      = [\"btc\"]\n",
    "\n",
    "logging.info(\"Getting prices...\")\n",
    "df_prices = client.get_asset_metrics(assets    =asset,\n",
    "                                     metrics   =metrics,\n",
    "                                     frequency =frequency,\n",
    "                                     start_time=start_time,\n",
    "                                     end_time  =end_time).to_dataframe()\n",
    "# Assign datatypes\n",
    "df_prices[\"ReferenceRateUSD\"] = df_prices[\"ReferenceRateUSD\"].astype(float)\n",
    "df_prices[\"time\"] = pd.to_datetime(df_prices[\"time\"])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3e170297",
   "metadata": {},
   "outputs": [],
   "source": [
    "# lets take a look at the data... note that the data will be as of midnight UTC\n",
    "df_prices.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eaa3081b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# plot\n",
    "df_prices.plot(kind='line',x=\"time\",y=\"ReferenceRateUSD\",color=\"orange\",linewidth=3,fontsize=20,xlabel=\"\",ylabel=\"Price ($)\", title=\"BTC Price (USD), Coin Metrics Reference Rate\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdf755a0",
   "metadata": {},
   "source": [
    "## Example 2: Getting Returns for Many Assets\n",
    "\n",
    "One of the key benefits to using the API client is the ability to pull large amounts of data quickly and manipulate it efficiently. The example below shows how to pull reference rates for many more assets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "23fe7091",
   "metadata": {},
   "outputs": [],
   "source": [
    "#Query API for prices, daily CM reference rates as dataframe\n",
    "assets = ['1INCH','ADA', 'ALGO', 'ATOM', 'AVAX','AXS','BCH',\n",
    "         'BNB','BSV','BTC','CELO','COMP','CRO','CRV','DASH','DOGE',\n",
    "         'DOT','ETC','ETH','FIL','FTM','FTT','GRT','HBAR','HNT',\n",
    "         'HT','LINK','LTC', 'LUNA','MANA','MATIC','MKR', 'NEAR',\n",
    "         'SAND','SNX','SOL','SUSHI','TRX','UNI','XLM','XMR','XRP','XTZ','ZEC']\n",
    "\n",
    "# Note if you want to see all assets that have a reference rate you can use the catalog endpoints, the lines below get all assets with a ref rate\n",
    "# assets_refrate = client.catalog_metrics(\"ReferenceRateUSD\")\n",
    "# asset_with_ref_rates = assets_refrate[0][\"frequencies\"][0][\"assets\"]\n",
    "\n",
    "metrics = \"ReferenceRateUSD\"\n",
    "frequency = \"1d\"\n",
    "start_time = \"2021-01-01\"\n",
    "end_time = \"2021-12-31\"\n",
    "\n",
    "logging.info(\"Getting prices...\")\n",
    "df_prices = client.get_asset_metrics(assets=assets,\n",
    "                                     metrics=metrics,\n",
    "                                     frequency=frequency,\n",
    "                                     start_time=start_time,\n",
    "                                     end_time=end_time).to_dataframe()\n",
    "# Assign datatypes\n",
    "df_prices[\"time\"] = pd.to_datetime(df_prices.time)\n",
    "df_prices[\"ReferenceRateUSD\"] = df_prices.ReferenceRateUSD.astype(float)\n",
    "# Sort on time and asset\n",
    "df_prices.sort_values([\"asset\",\"time\"],inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "bfdf1ec7",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_prices.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "df7d03f0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Manipulate data set, get returns from prices\n",
    "\n",
    "# Reshape dataset so assets are in columns, date is row, and values are prices\n",
    "df_prices_pivot = df_prices.pivot(index=\"time\",\n",
    "                                  columns=\"asset\",\n",
    "                                  values=\"ReferenceRateUSD\")\n",
    "\n",
    "# Create monthly and full year returns\n",
    "df_prices_pivot[\"month\"] = df_prices_pivot.index.month\n",
    "monthly_returns = ((df_prices_pivot.groupby(\"month\").last() / df_prices_pivot.groupby(\"month\").first())-1)\n",
    "year_returns     = ((df_prices_pivot.iloc[-1]/df_prices_pivot.iloc[0])-1)\n",
    "monthly_returns.loc[\"Year\"]=year_returns"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5d02733b",
   "metadata": {},
   "source": [
    "The following heat map of returns by month in 2021 was used in [*State of the Network* Issue 134](https://coinmetrics.substack.com/p/state-of-the-network-issue-134) for our 2021 year in review summary."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "25fa221e",
   "metadata": {},
   "outputs": [],
   "source": [
    "sns.set(font_scale=1.9)\n",
    "d = monthly_returns.transpose()    \n",
    "d.columns = [\"Jan\",\"Feb\",\"Mar\",\"Apr\",\"May\",\"Jun\",\"Jul\",\"Aug\",\"Sep\",\"Oct\",\"Nov\",\"Dec\",\"2021\"]\n",
    "\n",
    "plt.subplots(figsize=(24,20))\n",
    "sns.heatmap(d,\n",
    "            annot=True,\n",
    "            linewidths=.5,\n",
    "            cbar=False,\n",
    "            cmap=\"RdYlGn\",\n",
    "            vmin=-1,\n",
    "            vmax=1,\n",
    "            fmt=\",.0%\")\n",
    "plt.xticks(fontsize=30)\n",
    "plt.yticks(fontsize=26)\n",
    "plt.ylabel(\"\")\n",
    "plt.title(\"Selected Crypto Asset Returns by Month, 2021\",fontsize=35,y=1.04,fontweight='bold')\n",
    "\n",
    "g_fig = plt.gcf()\n",
    "im = image.imread('https://cdn.substack.com/image/fetch/w_96,c_limit,f_auto,q_auto:good,fl_progressive:steep/https%3A%2F%2Fbucketeer-e05bbc84-baa3-437e-9518-adb32be77984.s3.amazonaws.com%2Fpublic%2Fimages%2F4430351a-a92c-4505-8c8f-3822d76715df_256x256.png')\n",
    "g_fig.figimage(im, 1320, 1130, zorder=3)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "12ad7231",
   "metadata": {},
   "source": [
    "## Example 3: Analyzing the Bitcoin Mining Crackdown in China\n",
    "\n",
    "The Python API client can be very helpful in pulling CM network data to research and add context to important events. In this example, we pull Bitcoin mining data and analyze the data around May 2021 when miners were fleeing China due to new regulations. This was one of the most significant developments in crypto in 2021, shifting the map on Bitcoin mining and giving the Bitcoin network a stress test to abrupt, wide-sweeping change.\n",
    "\n",
    "![alt](https://miro.medium.com/max/1400/1*VtWg4MwkOsNlig3NVCGauQ.png)\n",
    "*Photo by [郑 无忌](https://unsplash.com/@godslar)*\n",
    "\n",
    "We again use the `get_asset_metrics` endpoint to pull `HashRate`, `HashRate30d` (30d Moving Average), and `DiffLast`.\n",
    "\n",
    "Note that Hash Rate is the speed at which miners are solving hashes. Hash rate provides an estimate of the total computational resources being allocated to the proof-of-work network. Difficulty is a network parameter that sets how hard it is to find a new block. It adjusts roughly every 2 weeks (2,016 Bitcoin blocks) to target a 10-min block time."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5d796ef1",
   "metadata": {},
   "outputs": [],
   "source": [
    "metrics = [\"HashRate\",\"HashRate30d\",\"DiffLast\"]\n",
    "frequency = \"1d\"\n",
    "assets  = \"btc\"\n",
    "\n",
    "logging.info(\"Getting mining data...\")\n",
    "df_btc_mining_data = client.get_asset_metrics(assets=assets, #note that omitting start/end time will return full history\n",
    "                                              metrics=metrics,\n",
    "                                              frequency=frequency).to_dataframe()\n",
    "\n",
    "# Assign Data Types\n",
    "df_btc_mining_data[\"time\"]=pd.to_datetime(df_btc_mining_data.time)\n",
    "df_btc_mining_data[\"HashRate\"]=df_btc_mining_data[\"HashRate\"].astype(float)\n",
    "df_btc_mining_data[\"HashRate30d\"]=df_btc_mining_data[\"HashRate30d\"].astype(float)\n",
    "df_btc_mining_data[\"DiffLast\"]   = df_btc_mining_data[\"DiffLast\"].astype(float)\n",
    "# Sort on time\n",
    "df_btc_mining_data.sort_values(\"time\",inplace=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d6429544",
   "metadata": {},
   "outputs": [],
   "source": [
    "df_btc_mining_data.tail()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "4693f005",
   "metadata": {},
   "source": [
    "The graph below shows the significant drop in Bitcoin hash rate around the May 2021 China Crackdown."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "5aad3b6f",
   "metadata": {},
   "outputs": [],
   "source": [
    "def human_format(num,pos):\n",
    "    \"\"\" Return human-readable suffixed data\"\"\"\n",
    "    magnitude = 0\n",
    "    while abs(num) >= 1000:\n",
    "        magnitude += 1\n",
    "        num /= 1000.0\n",
    "    # add more suffixes if you need them\n",
    "    return '%.0f%s' % (num, ['', 'K', 'M', 'B', 'T', 'P'][magnitude])\n",
    "\n",
    "sns.set(font_scale=1)\n",
    "sns.set(rc={'figure.figsize':(12,8)})\n",
    "formatter = FuncFormatter(human_format)\n",
    "\n",
    "fig,ax = plt.subplots(figsize=(12,8))\n",
    "\n",
    "# Plot zoomed in \n",
    "plt.plot(df_btc_mining_data.time,df_btc_mining_data.HashRate,label=\"Hash Rate (1d)\",linewidth=3,alpha=0.6)\n",
    "plt.plot(df_btc_mining_data.time,df_btc_mining_data.HashRate30d, label=\"Hash Rate (30d Moving Avg.)\",linewidth=3)\n",
    "plt.xlim(datetime(2021,1,1), datetime(2021,12,31))\n",
    "plt.ylim(.5e8,2e8)\n",
    "plt.title(\"Bitcoin Hash Rate, 2021\",fontsize=20)\n",
    "plt.ylabel('TH/s',fontsize=20)\n",
    "plt.xticks(fontsize=20)\n",
    "plt.yticks(fontsize=20)\n",
    "ax.xaxis.set_major_formatter(mdates.DateFormatter('%b\\n%Y'))\n",
    "ax.yaxis.set_major_formatter(formatter)\n",
    "plt.legend(fontsize=18)\n",
    "\n",
    "# Create zoom-out plot\n",
    "ax_new = fig.add_axes([1, 0.4, 0.4, 0.4]) # the position of zoom-out plot compare to the ratio of zoom-in plot \n",
    "plt.plot(df_btc_mining_data.time,df_btc_mining_data.HashRate)\n",
    "plt.plot(df_btc_mining_data.time,df_btc_mining_data.HashRate30d)\n",
    "plt.fill_between([datetime(2021,1,1), datetime(2021,12,31)],2.5e8,alpha=0.4,color='tan',label=\"2021\")\n",
    "plt.title(\"Bitcoin Hash Rate, 2009-2022\")\n",
    "ax_new.yaxis.set_major_formatter(formatter)\n",
    "plt.ylabel('TH/s')\n",
    "plt.legend(fontsize=16)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8074b18f",
   "metadata": {},
   "source": [
    "Note that Hash Rate is inferred from on-chain data and statistically noisy in short-term intervals, so it is always best for researchers to look at **moving averages**. For more information, check out Coin Metrics' own Lucas Nuzzi's article on estimating Hash Rate and the implications of the exodus of Bitcoin mining out of China [here.](https://medium.com/coinmetrics/bitcoin-miners-are-escaping-china-d3937e8f018c)\n",
    "![*hash rate formula*](https://miro.medium.com/max/1400/1*TxF9YP2Vpq5WNFOSkw8AFg.png)\n",
    "*hash rate formula*"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b69ba1a0",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get days where mining difficulty changed\n",
    "df_btc_mining_data[\"DiffLast_lag\"] = df_btc_mining_data[\"DiffLast\"].shift(1) \n",
    "df_btc_mining_data[\"DiffLast_chg_pct\"] = (df_btc_mining_data[\"DiffLast\"] / df_btc_mining_data[\"DiffLast_lag\"])-1\n",
    "df_btc_difficulty_change_days = df_btc_mining_data[(df_btc_mining_data[\"DiffLast_chg_pct\"] != 0)].iloc[1:].copy()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0e28ad8b",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Plot change in mining difficulty\n",
    "plt.style.use('seaborn-whitegrid')\n",
    "ax = df_btc_difficulty_change_days.plot(kind='line', x='time',y='DiffLast_chg_pct', label=\"% Change in Difficulty\", figsize=(10, 7.5), linewidth=2, alpha=0.9, color=[247/255,147/255,26/255])\n",
    "\n",
    "plt.title(\"Percent Change in Bitcoin Mining Difficulty\",fontsize=20)\n",
    "plt.xlabel(\"\",fontsize=20)\n",
    "plt.ylabel('',fontsize=20)\n",
    "plt.ylim(-.4,.4)\n",
    "plt.xlim(datetime(2010,1,1),datetime(2022,2,15))\n",
    "plt.hlines(0,datetime(2010,1,1),datetime(2022,2,15),color='black')\n",
    "\n",
    "plt.gca().set_yticklabels(['{:,.0f}%'.format(x*100) for x in plt.gca().get_yticks()])\n",
    "plt.xticks(fontsize=15,rotation = 45)\n",
    "plt.yticks(fontsize=15)\n",
    "\n",
    "#biggest decline \n",
    "point = (datetime(2021,7,3),-.28)\n",
    "circle_rad = 15  # This is the radius, in points\n",
    "ax.plot(point[0], point[1], 'o',\n",
    "        ms=circle_rad * 2, mec='r', mfc='none', mew=2)\n",
    "ax.annotate('July 3, 2021\\n28% Decline', xy=point, xytext=(30, 60),\n",
    "            textcoords='offset points',\n",
    "            color='black', size='large',\n",
    "            arrowprops=dict(\n",
    "                arrowstyle='simple,tail_width=0.3,head_width=0.8,head_length=0.8',\n",
    "                facecolor='black', shrinkB=circle_rad * 1.2)\n",
    ")\n",
    "\n",
    "plt.gca().xaxis.grid(False)\n",
    "plt.legend(fontsize=15)\n",
    "plt.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e1016d41",
   "metadata": {},
   "source": [
    "# Example 4: Analyzing Market Candles of Listed Assets on Coinbase\n",
    "\n",
    "The Coin Metrics Market Data Feed offers highly detailed data on specific exchanges and asset pairs. In 2021, we [launched](https://coinmetrics.io/coin-metrics-launches-a-market-data-community-offering/) a market data community offering covering many different types of market data including:\n",
    "\n",
    "- Market trades, order books, quotes, open interest, funding rates, and liquidations are available for all markets for the past 24 hours\n",
    "- Market candles are available for all markets with full history at daily frequency and the past 24 hours for sub-daily frequencies\n",
    "- Reference rates are available for all assets in our coverage universe with full history at daily frequency and the past 24 hours for sub-daily frequencies\n",
    "- CMBI index levels are available with full history at daily frequency and the past 24 hours for sub-daily frequencies\n",
    "- CMBI constituents and weighting data is available at hourly frequency for the past 60 days\n",
    "\n",
    "In the example below, we use the `get_market_candles` endpoint to analyze the breakdown of daily volume on Coinbase by asset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a7c0cbfc-ae7f-4271-9604-ee4c6567a22c",
   "metadata": {},
   "outputs": [],
   "source": [
    "candles_coinbase = client.get_market_candles(\n",
    "    markets=\"coinbase-*-spot\", # wildcards can be passed to get all asset pairs \n",
    "    start_time=\"2022-01-01\",\n",
    "    end_time=\"2022-02-15\",\n",
    "    frequency=\"1d\"\n",
    ").to_dataframe()\n",
    "candles_coinbase[\"candle_usd_volume\"] = candles_coinbase.candle_usd_volume.astype(float)\n",
    "candles_coinbase[\"time\"] = pd.to_datetime(candles_coinbase.time)\n",
    "candles_coinbase.sort_values([\"market\",\"time\"],inplace=True)\n",
    "\n",
    "# Create Addt. Cols\n",
    "candles_coinbase['exchange'] = candles_coinbase.market.apply(lambda x: x.split(\"-\")[0])\n",
    "candles_coinbase['exchange-base'] = candles_coinbase.market.apply(lambda x: x.split(\"-\")[0]+\"-\"+x.split(\"-\")[1])\n",
    "candles_coinbase['market_type'] = candles_coinbase.market.apply(lambda x: x.split(\"-\")[-1])\n",
    "candles_coinbase['base'] = candles_coinbase.market.apply(lambda x: x.split(\"-\")[1])\n",
    "candles_coinbase['quote'] = candles_coinbase.market.apply(lambda x: x.split(\"-\")[2])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "8b6c9fc2",
   "metadata": {},
   "outputs": [],
   "source": [
    "candles_coinbase.tail()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "50b7d8da",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get volume by base asset by day\n",
    "\n",
    "# Get top 10 assets by volume\n",
    "total_vol_by_base = candles_coinbase.groupby('base',as_index=False).candle_usd_volume.sum()\n",
    "total_vol_by_base.sort_values(by=\"candle_usd_volume\",inplace=True)\n",
    "base_top_list = total_vol_by_base.tail(10).base.tolist()\n",
    "candles_coinbase[\"base2\"] = np.where(candles_coinbase.base.isin(base_top_list),candles_coinbase.base,f\"{len(total_vol_by_base)-10} others\")\n",
    "    \n",
    "# Get sum by base asset by day\n",
    "df_vol_by_base = candles_coinbase.groupby([\"time\",\"base2\"],as_index=False).candle_usd_volume.sum()\n",
    "df_vol_by_base['total_vol'] = df_vol_by_base.groupby(\"time\").candle_usd_volume.transform(sum)\n",
    "df_vol_by_base.columns=[\"time\",\"base_asset\",\"vol\",\"total_vol\"]\n",
    "df_vol_by_base[\"vol_pct\"]=df_vol_by_base.vol/df_vol_by_base.total_vol\n",
    "df_vol_by_base.sort_values([\"base_asset\",\"time\"],inplace=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01acd65a",
   "metadata": {},
   "source": [
    "### Volume Broken Down by Asset, %"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a590190e",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pivot back to assets in columns\n",
    "df_vol_pivot = df_vol_by_base.pivot(index='time',\n",
    "                                 columns=\"base_asset\",\n",
    "                                 values=\"vol_pct\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "701e0ff5",
   "metadata": {},
   "outputs": [],
   "source": [
    "color_map = {\"btc\": \"#ff9900\",\"eth\":  \"#37367b\",f\"{len(total_vol_by_base)-10} others\":\"lightsteelblue\",\"ada\":  \"palegoldenrod\",\"link\": \"blue\",\n",
    "            \"ltc\":  \"cyan\",\"matic\":\"deeppink\",\"sol\":  \"darkmagenta\",\"xlm\":  \"dimgrey\",\"bch\":  \"springgreen\",\n",
    "            \"algo\": \"maroon\",\"shib\": \"red\",\"doge\":\"gold\",\"etc\": \"darkgreen\",\"xrp\": \"darkgrey\",\n",
    "             \"grt\": \"indigo\",\"mana\":\"teal\",\"dot\": \"purple\"}\n",
    "top_assets  = df_vol_by_base[~df_vol_by_base.base_asset.isin([\"btc\",\"eth\",f\"{len(total_vol_by_base)-10} others\"])].groupby(\"base_asset\").vol.sum().sort_values(ascending=False).index.tolist()\n",
    "top_assets  = [\"btc\",\"eth\",f\"{len(total_vol_by_base)-10} others\"] + top_assets\n",
    "\n",
    "fig = px.area(df_vol_pivot,color_discrete_map=color_map,category_orders={\"base_asset\":top_assets},title=\"Percentage of Daily USD Spot Volume on Coinbase by Asset\")\n",
    "fig.update_yaxes(tickformat=\"%\",range=[0,1],title=\"% of Daily USD Spot Volume\")\n",
    "fig.update_xaxes(title=\"\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "adb10dd4",
   "metadata": {},
   "source": [
    "### Volume Broken Down by Asset, All USD"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "aaaa8848",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Pivot back to assets in columns\n",
    "df_vol_pivot = df_vol_by_base.pivot(index='time',\n",
    "                                 columns=\"base_asset\",\n",
    "                                 values=\"vol\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "938fbad7",
   "metadata": {},
   "outputs": [],
   "source": [
    "color_map = {\"btc\": \"#ff9900\",\"eth\":  \"#37367b\",f\"{len(total_vol_by_base)-10} others\":\"lightsteelblue\",\"ada\":  \"palegoldenrod\",\"link\": \"blue\",\n",
    "            \"ltc\":  \"cyan\",\"matic\":\"deeppink\",\"sol\":  \"darkmagenta\",\"xlm\":  \"dimgrey\",\"bch\":  \"springgreen\",\n",
    "            \"algo\": \"maroon\",\"shib\": \"red\",\"doge\":\"gold\",\"etc\": \"darkgreen\",\"xrp\": \"darkgrey\",\n",
    "             \"grt\": \"indigo\",\"mana\":\"teal\",\"dot\": \"purple\"}\n",
    "top_assets  = df_vol_by_base[~df_vol_by_base.base_asset.isin([\"btc\",\"eth\",f\"{len(total_vol_by_base)-10} others\"])].groupby(\"base_asset\").vol.sum().sort_values(ascending=False).index.tolist()\n",
    "top_assets  = [\"btc\",\"eth\",f\"{len(total_vol_by_base)-10} others\"] + top_assets\n",
    "\n",
    "fig = px.bar(df_vol_pivot,color_discrete_map=color_map,category_orders={\"base_asset\":top_assets},title=\"Daily USD Spot Volume on Coinbase by Asset\")\n",
    "fig.update_yaxes(title=\"Daily USD Spot Volume\")\n",
    "fig.update_xaxes(title=\"\")\n",
    "fig.show()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45791747",
   "metadata": {},
   "source": [
    "With so much community data, there is plenty to work with. In the past members of the CM community have created dashboards ([BitcoinKPIs](http://www.bitcoinkpis.com/security)) and have used CM data in original research.\n",
    "\n",
    "Our data might also be useful to journalists, academics, and more.\n",
    "\n",
    "For any additional questions feel free to email us [here](https://coinmetrics.io/support/) or give us a shout on Twitter @coinmetrics. "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}