{ "cells": [ { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "# Composable Blocks & Interactive Charts" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "This notebook is a simple illustration of the python API for Blocks and Highcharts interactive charts.\n", "\n", "**What are Blocks?**\n", "Blocks are composable layout elements for easily building HTML, PDF, PNG and JPG based reports. At the same time, all block constructs can be rendered in-line in IPython Notebooks (as will be shown later). This has practical benefits like a single backtest function that can be used for quick analysis during research work in a notebook, but also directly injected into more formal reports with having to fumble around with intermediate formats.\n", "\n", "Practically all the functionality is based on HTML rendering. HTML is a declarative, tree based language that is easy to work with and fungible. Blocks are also declarative, composable into a tree and are meant to be dead simple. The match was thus quite natural.\n", "\n", "The blocks do not try to match the power and precision of latex. Such an undertaking would be not only out of the scope of a simple library, but would mean the reinvention of latex with all the gnarliness that comes with it.\n", "\n", "This notebook aims to showcase the functionality and offers some examples to help people get started." ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "### Imports & Data" ] }, { "cell_type": "code", "execution_count": 9, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
Interactive mode initialized successfully
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import numpy as np\n", "import pandas as pd\n", "import pandas.util.testing as pt\n", "from datetime import datetime\n", "\n", "# === Begin bits required for interactive plotting and reports ===\n", "from pybloqs import *\n", "from pybloqs.plot import *\n", "\n", "interactive()\n", "# === End interactive bits ===" ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [], "source": [ "wp = pt.makePanel().ix[:, :, :2]\n", "\n", "df = pd.DataFrame((np.random.rand(200, 4)-0.5)/10,\n", " columns=list(\"ABCD\"),\n", " index=pd.date_range(datetime(2000,1,1), periods=200))\n", "\n", "df_cr = (df + 1).cumprod()\n", "\n", "a = df_cr.A\n", "b = df_cr.B\n", "c = df_cr.C\n", "c.name = \"C\"" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "### Using Blocks\n", "Obligatory \"Hello World!\"" ] }, { "cell_type": "code", "execution_count": 11, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", " Hello World!\n", "
" ], "text/plain": [ "" ] }, "execution_count": 11, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Block(\"Hello World!\")" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "Play around with alignment" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", " Hello World!\n", "
" ], "text/plain": [ "" ] }, "execution_count": 12, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Block(\"Hello World!\", h_align=\"left\")" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "Adding a title" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", "

\n", " Announcement\n", "

\n", " Hello World!\n", "
" ], "text/plain": [ "" ] }, "execution_count": 13, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Block(\"Hello World!\", title=\"Announcement\", h_align=\"left\")" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "Writing out dataframes" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", " \n", " A\n", " \n", " B\n", " \n", " C\n", " \n", " D\n", "
\n", " 2000-01-01 00:00:00\n", " \n", " -0.03\n", " \n", " 0.03\n", " \n", " 0.02\n", " \n", " 0.02\n", "
\n", " 2000-01-02 00:00:00\n", " \n", " -0.04\n", " \n", " -0.02\n", " \n", " -0.01\n", " \n", " -0.01\n", "
\n", " 2000-01-03 00:00:00\n", " \n", " -0.01\n", " \n", " 0.04\n", " \n", " -0.01\n", " \n", " 0.03\n", "
\n", " 2000-01-04 00:00:00\n", " \n", " 0.00\n", " \n", " 0.03\n", " \n", " -0.03\n", " \n", " 0.00\n", "
\n", " 2000-01-05 00:00:00\n", " \n", " 0.03\n", " \n", " -0.03\n", " \n", " -0.03\n", " \n", " -0.02\n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Block(df.head())" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "Writing out matplotlib plots" ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", " \n", "
" ], "text/plain": [ "" ] }, "execution_count": 15, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Block(df.A.plot())" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "Raw HTML output" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", " \n", " this text is bold\n", " \n", "
" ], "text/plain": [ "" ] }, "execution_count": 16, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Block(\"this text is bold\")" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "Composing blocks" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", "
\n", "
\n", "

\n", " Announcement\n", "

\n", " Hello World!\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " \n", " this text is bold\n", " \n", "
\n", "
\n", "
\n", "
\n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 17, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Block([Block(\"Hello World!\", title=\"Announcement\"), Block(\"this text is bold\")])" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "In most cases, one does not need to explicitly wrap elements in blocks" ] }, { "cell_type": "code", "execution_count": 18, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", "
\n", "
\n", " Block 0\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " Block 1\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " Block 2\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " Block 3\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " Block 4\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " Block 5\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " Block 6\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " Block 7\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 18, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Block([\"Block %s\" % i for i in xrange(8)])" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "Splitting composite blocks into columns" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", "
\n", "
\n", " Block 0\n", "
\n", "
\n", "
\n", "
\n", " Block 1\n", "
\n", "
\n", "
\n", "
\n", " Block 2\n", "
\n", "
\n", "
\n", "
\n", " Block 3\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " Block 4\n", "
\n", "
\n", "
\n", "
\n", " Block 5\n", "
\n", "
\n", "
\n", "
\n", " Block 6\n", "
\n", "
\n", "
\n", "
\n", " Block 7\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 19, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Block([\"Block %s\" % i for i in xrange(8)], cols=4)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "Layout styling is cascading - styles will cascade from parent blocks to child blocks by default. This behavior can be disabled by setting *inherit_cfg* to false on the child blocks, or simply specifying the desired settings explicitly." ] }, { "cell_type": "code", "execution_count": 20, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", "
\n", "
\n", " Block 0\n", "
\n", "
\n", "
\n", "
\n", " Block 1\n", "
\n", "
\n", "
\n", "
\n", " Block 2\n", "
\n", "
\n", "
\n", "
\n", " Block 3\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
\n", " Block 4\n", "
\n", "
\n", "
\n", "
\n", " Block 5\n", "
\n", "
\n", "
\n", "
\n", " Block 6\n", "
\n", "
\n", "
\n", "
\n", " Block 7\n", "
\n", "
\n", "
\n", "
\n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Block([\"Block %s\" % i for i in xrange(8)], cols=4, text_align=\"right\")" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "Using specific block types is simple as well. As an example - the Paragraph block:" ] }, { "cell_type": "code", "execution_count": 21, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", "
\n", "

\n", "

\n", " First paragraph.\n", "
\n", "

\n", "
\n", "
\n", "
\n", "
\n", "

\n", "

\n", " Second paragraph.\n", "
\n", "

\n", "
\n", "
\n", "
\n", "
\n", "

\n", "

\n", " Third paragraph.\n", "
\n", "

\n", "
\n", "
\n", "
\n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Block([Paragraph(\"First paragraph.\"),\n", " Paragraph(\"Second paragraph.\"),\n", " Paragraph(\"Third paragraph.\")], text_align=\"right\")" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "The Pre block preserves whitespace formatting and is rendered using a fixed width font. Useful for rendering code-like text." ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n",
       "some:\n",
       "  example:\n",
       "    yaml: [1,2,3]\n",
       "  data: \"text\"\n",
       "
" ], "text/plain": [ "" ] }, "execution_count": 22, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Pre(\"\"\"\n", "some:\n", " example:\n", " yaml: [1,2,3]\n", " data: \"text\"\n", "\"\"\")" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "Creating custom blocks is trivial. For the majority of the cases, one can just inherit from the Container block, which has most of the plumbing already in place:" ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", " THIS HERE TEXT SHOULD LOOK LIKE SHOUTING!\n", "
" ], "text/plain": [ "<__main__.Capitalize at 0x7f65950cd790>" ] }, "execution_count": 23, "metadata": {}, "output_type": "execute_result" } ], "source": [ "class Capitalize(Raw):\n", " def __init__(self, contents, **kwargs):\n", " # Stringify and capitalize\n", " contents = str(contents).upper()\n", "\n", " super(Capitalize, self).__init__(contents, **kwargs)\n", " \n", "Capitalize(\"this here text should look like shouting!\")" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "### Simple line chart\n", "When evaluated as the last expression in a Notebook Cell, the plot is automatically displayed inline.\n", "\n", "Note how the plot name (hover over the line to see the little popup) is taken from the input data (if available)." ] }, { "cell_type": "code", "execution_count": 24, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", " \n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Plot(a)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "### Saving as interactive HTML" ] }, { "cell_type": "code", "execution_count": 25, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/plain": [ "'chart_sample.html'" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from pybloqs.plot import Plot\n", "Plot(a).save(\"chart_sample.html\")" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "### Scatter Plot\n", "Regular scatter plot, with zooming on both the x and y axes." ] }, { "cell_type": "code", "execution_count": 26, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", " \n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Plot(df.values[:,:2], Scatter(Marker(enabled=True)), Chart(zoom_type=\"xy\"))" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "###Bar Charts\n", "Notice how when viewing all the data at once, the chart shows monthly data, yet zooming in reveals additional detail at up to daily resolution. This is accomplished by using a custom data grouping." ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [], "source": [ "bar_grouping = DataGrouping(approximation=\"open\", enabled=True, group_pixel_width=100)" ] }, { "cell_type": "code", "execution_count": 28, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", " \n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Bar chart from a dataframe\n", "Plot(df, Column(bar_grouping))" ] }, { "cell_type": "markdown", "metadata": { "collapsed": false, "deletable": true, "editable": true }, "source": [ "# Stacked bar chart\n", "Plot(df, Column(bar_grouping, stacking=\"normal\"))" ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", " \n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# Composite bar chart from two separate plots.\n", "s2 = Plot([Plot(a, Column(bar_grouping)),\n", " Plot(b, Column(bar_grouping))])\n", "s2" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "### Comparing series in a dataframe\n", "Plot the cumulative percentage difference between input series (or columns of a dataframe). The cumulative difference is always calculated from the start of the observation window. This results in intuitively correct behavior when zooming in or sliding the window, as the chart will dynamically recalculate the values. Incredibly useful for comparing model performance over time for example as one doesn't need to manually normalize money curves for different periods." ] }, { "cell_type": "code", "execution_count": 30, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", " \n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "s3 = Plot(df_cr,\n", " PlotOptions(Series(compare=\"percent\")),\n", " TooltipPct(),\n", " YAxisPct())\n", "s3" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "### Three series on separate side-by-side Y axes" ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", " \n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 31, "metadata": {}, "output_type": "execute_result" } ], "source": [ "s4 = Plot([Plot(a),\n", " Plot(b, YAxis(Title(text=\"B Axis\"), opposite=True)),\n", " Plot(c, YAxis(Title(text=\"C Axis\"), opposite=True, offset=40))])\n", "s4" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "### Two series on separate subplots" ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", " \n", "
\n", "
" ], "text/plain": [ "" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "s5 = Plot([Plot(a, Line(), YAxis(Title(text=\"a only\"), height=150)),\n", " Plot(b, Column(), YAxis(Title(text=\"b only\"),\n", " top=200, height=100, offset=0))],\n", " Tooltip(value_decimals=2), height=\"400px\")\n", "s5" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "### Writing out multiple charts as HTML" ] }, { "cell_type": "code", "execution_count": 33, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [], "source": [ "b = Block([Block(pt.makeTimeDataFrame().tail(10), title=\"A table\", title_level=1),\n", " Block([s2, s3], title=\"Side-by-side Plots\", cols=2),\n", " Block(title=\"Full Width Plots\", break_before=\"always\"),\n", " Block(s4, title=\"Side by Side Axes\"),\n", " Block(s5, title=\"Composite Plots\")], title=\"Dynamic Reports\")" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "['/users/is/dchrist/pyenvs/dev_RHEL7_pydev/bin/wkhtmltopdf', '--debug-javascript', '--page-size', 'A4', '--orientation', 'Portrait', '/tmp/xppdu10cy40s.html', 'charts_test.pdf'] returned:\n", "\n", "None\n" ] }, { "data": { "text/plain": [ "'charts_test.html'" ] }, "execution_count": 34, "metadata": {}, "output_type": "execute_result" } ], "source": [ "b.save(\"charts_test.pdf\")\n", "b.save(\"charts_test.html\")" ] }, { "cell_type": "code", "execution_count": 35, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Please create ~/.pybloqs.cfg with entry for smtp_server . See README.md and pybloqs/config.py for details.\n" ] } ], "source": [ "# Emails the report. The emailing is independent of previous reports being saved (e.g. there is no need to call save\n", "# before emailing).\n", "from smtplib import SMTPServerDisconnected\n", "\n", "try:\n", " b.email(fmt=\"html\")\n", "except SMTPServerDisconnected:\n", " print \"Please create ~/.pybloqs.cfg with entry for smtp_server . See README.md and pybloqs/config.py for details.\"" ] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.11" } }, "nbformat": 4, "nbformat_minor": 0 }