The Plotly plotting library is really cool. It’s actually written in javascript so it’s interactive, integrates well with python and jupyter notebooks and it stays intact when converting the notebook to html like in this post.

It’s possible to attach to plotly events, like clicking a point. They actually provide a python callback when running inside the notebook which means the javascript event triggers a python script. Astonishing! But that won’t work when exporting to html. Of course they also provide a javascript callback so here’s a way to use it in notebook and html export.

from typing import Optional
import plotly
import plotly.graph_objects

# A small helper function to render a plotly
# Figure with attached javascript
def html_plotly(
        fig: plotly.graph_objects.Figure, 
        post_script: Optional[str] = None, 
        post_html: Optional[str] = None, 
        display: bool = True,
) -> Optional[str]:
    html = fig.to_html(
        full_html=False, 
        include_mathjax=False, 
        include_plotlyjs="require", 
        post_script=post_script,
    )
    if post_html:
        html += post_html
    
    if display:
        from IPython.display import display, HTML
        display(HTML(html))
    else:
        return html

Plotly’s to_html method is quite powerful and convenient. And it allows passing javascript code where the special template tag {plot_id} will be replaced by the id of the plot <div> and more is not needed for catching the events.

This function can be called on a plotly figure, so let’s create one using data from … öhmm .. maybe .. Überwachung für Alle!, more precisely my personal keystrokes inside the browser during the last month.

from elastipy import Search

keystrokes = (
    Search(index="ufa-events-keyboard")
        .range("timestamp", gte="2021-02-25", lte="2021-03-25")
        .agg_date_histogram("day", calendar_interval="day")
        .agg_terms("key", field="key", size=50)
        .execute().df()
        .replace({" ": "Space"})
        .set_index(["day", "key"])
)

This is asking elasticsearch for two aggregations: the number of keystrokes per day and the top 50 keys per day. The search is executed, converted to a pandas dataframe and beautified a bit.

keystrokes
day.doc_count key.doc_count
day key
2021-02-25 Control 1768 137
Backspace 1768 130
ArrowRight 1768 117
ArrowLeft 1768 97
Shift 1768 84
... ... ... ...
2021-03-25 - 18132 55
Home 18132 47
< 18132 46
> 18132 43
/ 18132 40

1443 rows × 2 columns

Then we build a python object that can be passed to javascript to enhance the plot experience.

keystrokes_per_day = [
    {
        "date": date.to_pydatetime().isoformat(),
        "count": int(keystrokes.loc[date]["day.doc_count"][0]),
        "keys": list(keystrokes.xs(date)["key.doc_count"].items()),
    }
    for date in keystrokes.index.unique(level=0)
]

(This was done with some terrible code before reading Tom Augspurger’s blog and more pandas documentation..)

Now create some piece of html/javascript that displays the top keys for each clicked day.

import secrets
import json

# I'm writing these things mostly at night, so 
# dark mode is much more eye-friendly
plotly.io.templates.default = "plotly_dark"

# a random id to identify our data object and the html container 
ID = secrets.token_hex(8)

# a place-holder for the data
post_html = f"""
<div id="post-plot-{ID}">
    <div class="info"></div>
</div>
"""

# The usual vanilla-javascript DOM-mangling code 
# But one can use jquery or whatever
post_script = """
const data_%(ID)s = %(data)s;
function on_click(point_index) {
    const 
        data = data_%(ID)s[point_index],
        date_str = new Date(Date.parse(data.date)).toDateString();
    
    let html = `<h3>${date_str}: ${data.count} keystrokes</h3>`;
    html += `<table><tbody>`;
    html += `<tr><td>key</td> <td>count</td> <td>percent</td> </tr>`;
    html += data.keys.map(function(key) {
        return `<tr><td>${key[0]}</td> <td>${key[1]}</td>`
             + `<td>${Math.round(key[1] * 10000 / data.count) / 100}%%</td> </tr>`;
    }).join("") + "</tbody></table>";
    
    document.querySelector("#post-plot-%(ID)s .info").innerHTML = html;
}

// attach to plotly
document.getElementById("{plot_id}").on("plotly_click", function(click_data) {
    on_click(click_data.points[0].pointIndex);
});
""" % {"ID": ID, "data": json.dumps(keystrokes_per_day)}

And finally create an actual plot and attach the code.

import plotly.express as px

fig = px.bar(
    keystrokes.groupby("day").first().rename(columns={"day.doc_count": "keystrokes"}),
    y="keystrokes",
    title="Click me!",
)

html_plotly(fig, post_script=post_script, post_html=post_html)

Obviously, i’m a right-arrow affine, 4-day-interval writer.