IEX, the Investors Exchange, is a transparent stock exchange that discourages high-frequency trading and makes historical trading data publicly available. The data is offered in the form of daily pcap files where each single packet corresponds to a stock trade.
Even with this specialized pcap file format, these daily records can exceed a gigabyte in size on a given day. In this notebook, we will develop a dashboard that will allow us to explore every single trade that happened in a day, including the associated metadata. To visualize all this data at once both rapidly and interactively, we will use datashader via the HoloViews API.
Loading the data¶
The IEX stock data is saved in two formats of pcap file called TOPS and DEEP. These formats are complex enough to make it non trivial to parse the trades with standard packet loading tools. For this reason, the trades for Monday 21st of October 2019 are supplied as a CSV file that has been generated from the original pcap file using the IEXTools library.
import datetime import pandas as pd df = pd.read_csv('./data/IEX_2019-10-21.csv') print('Dataframe loaded containing %d events' % len(df))
Dataframe loaded containing 1222412 events
We can now look at the head of this DataFrame to see its structure:
Each row above corresponds to a stock trade where
the stock price, the
size indicates the size of the trade and the
symbol specifies which stock was traded. Every trade also has a
timestamp specified in nanoseconds.
Note that multiple trades can occur on the same timestamp.
Visualizing trade with
We can now load HoloViews with the Bokeh plotting extension to start visualizing some of this data:
import holoviews as hv from bokeh.models import HoverTool from holoviews.operation.datashader import spikes_aggregate hv.config.image_rtol = 10e-3 # Fixes datetime issue at high zoom level hv.extension('bokeh')