Happy reading!

Below you can find a pdf of the textbook and my notes:

• Market Microstructure in Practice
• Chapter 1 Notes
• Chapter 2 Notes
• Chapter 3 Notes

Organizing a Trading Structure to Answer a Fragmented Landscape

Main Inputs of Trading Tools
Large asset managers concentrate on tasks of mounting and unwinding positions to dealing desks, which takes care of:

• The relationships with intermediaries.
• Measuring trading costs and comparing broker performances thanks to Transaction Cost Analysis (TCA).
• Choosing benchmark to target (VWAP, TWAP, Implementation shortfall, Target Close) according to investment style that generated the trade.

Dealing desks demand accuracy. Exchanges provide services around best execution (SOR), vendors provide better performance measurements and comparison features.

Market Data

• Real time market data feeds are the eyes of a trading algorithm. Consolidated data is needed and the source should be easily disaggregated to know which trading venue is currently offering this price.

Market data contains:

• Market trades, described by price, quantity, and timestamp
• Limit orderbooks (bid-ask prices, quantities available, number of different market participants contributing at each price level), or sequences of modifications of the orderbook

Data is enriched with analysis with bid-ask spread, qualifying the observed market trades, intraday volatility, efficient bid-ask spread, or estimates of hidden quantities in the book.

Connection to Venues Algorithms need to be connected to exchanges. Connections are not natively synchronized with the market data and are two-way:

• One to send messages to the trading venue: Creation of a new order, cancellation, or modification of another order.
• The other to receive messages from the trading venue about the orders of the trading algorithm. Acknowledgements of messages sent by the algo, rejections, cancellations, partial fills, ect.

Incoming messages can be used as information probes. In dark pools, given no pre-trade transparency, orders sent by algos can be used as probes coming back with information about the state of the orderbook. When a dark buy order is filled at a given price, the orderbook of the dark pool contains resting sell quantities at the price or better.

Historical Data Optimal trading relies on intraday risk control and usual rhythms and behaviors of market features that are captured by models whose parameters are estimated by stock-by-stock or sector-by-sector basis. Historical data is needed and should contain:

• Market Data (tick-by-tick or order-by-order)
• Analytics that have been computed in real time
• Recordings of the messages sent and received by the algorithms of the firm
• Context-related information lik news, corporate events, activation of circuit breakers, outages of trading venues, ect.

Models A model is based off of three components:

• The Formula: Takes in input parameters and variables. Parameters can be estimated once per day and sorted, the variables are known in real time and have to be plugged in the formula on the fly.
• Estimation: A mathematical procedure taking as input historical data on a given universe and a given horizon and computing estimated values of the parameters of the model.
• The accuracy computation: A mathematical procedure to quantify the quality of the model and its parameters according to the immediate past or in real time.

Components of a Trading Algorithm
Trading algos can be launched by a external user or by other algorithms. The different components are:

• The execution parameters: Defines the execution conditions that apply to the algorithm
• The risk control layer: In charge of computing reasonable maximum and minimum trading rates to target during the trading: This is called the trading curve. The layer uses models and expected market contexts to produce minimum nd maximum trading envelopes to follow in order to make the balance between two main sources of risk: market impact and market risk.
• Trading robots or liquidity-seeking tactics: Like SOR, these are more sophisticated opportunistic tactics dedicated to navigate inside the trading envelope defined by the risk control layer. These robots have to take into account the immediate state of the liquidity to extract as much value as possible from venues.

The two layers, risk control and tactics, communicate through the risk control layer by sending messages to tactics to assign them short-term goals.

In figure 1 the trajectory of a trading algo is shown with the trading envelope and the real trajectory of the algo.

Main Outputs of an Automated Trading System

Pre-trade Analytics: The historical data, the models, and their parameters have to be used to provide decision support before the effective launch of a trading algorithm.

Pre-trade analytics are useful for the portfolios to check consistency of a portfolio and generate automated directives to speed up lines and slow down other according to pre-computation of the balance between expected market risk and expected market impact.

Typical pre-trade analysis includes:

• Estimation and comparison of expected trading cost components (spread, traded volume, quantities on the books)
• Expected trading profiles
• Expected context (news, volatility levels)
• Breakdown by sector, currency, country, ect.

Monitoring Indicators
Once an algo has been launched it has to be monitored by traders. Traders usually monitored several hundreds of working algos - indicators allow traders to build diagnoses of the algos on the fly.

Performance Indicators
A specific subset of monitoring indicators are direct measures of what the performance would be if trading should stop now and others include estimates of the performance when it will end, seen from now.

Typical performance indicators come from the decomposition of the average price of the trading algorithm according to different effects.

AVG price = Immediate Price + Market Moves + Market Impact.

On a benchmark-by-benchmark bases, such a decomposition can be refined towards

AVG Price = Immediate Price + Planned Price + Unexpected Market Moves + Trading Efficiency + Market Impact.

Planned price is what the risk control part of the trading algo expected on average. Trading efficiency rates the success of the liquidity-seeking tactics used by the trading algo.

Where to Produce Real Time Indicators
Algorithms themselves produce the indicators to publish. Indicator publishers are compatible with the fact that trading algos can be modeled as stateless agents and should have introspection capabilities; publishing views on their own state in real time to share with final users of companion als is an important component of modern trading algos.

Post Trade Analysis
Three components should be compatible:

For the spread:

• The pre-trade analysis gives an expected average value.
• The monitoring gives the current value, the updated expected average value, and the likelihood of the current value with respect to what happened so far. It gives the ability to quantify the relationship between the current trading performances and the trajectory of the spread.
• The post-trade analysis gives an assessment of the value of the spread during the trading and its influence on trading performance.

Transaction Cost Analysis (TCA)
TCA is used by different algo providers to compare the efficiency of their trading over several weeks.

Post-trade analysis (day to day) aims to understand what happened because of the market context.

• Requires fine grained analysis of market behavior. TCA aims to to explain the part of performance comes from the design o the trading algorithms.
• Requires on averaging huge datasets to cancel most of market effects.

TCA associates a given trading algo with a give market context of trading style so that the user of the algo will send flow to the proper algo provider given market conditions. TCA involves breakdowns according to the characteristics of the market (volatility, jumps, liquidity, occurrence of news, etc.).

Liquidity is important to consider given it is useless to compare the efficient of an execution on a very liquid stock with one illiquid stock.

Typical indicators used to quantify liquidity are:

• bid-ask spread
• tick size in basis points
• quantity on the books
• daily turnover
• average trade size
• free float
• trading rate

Market Impact Measurements: Understanding the PFP from the Viewpoint of One Investor

Factors that drive market impact:

• liquidity of traded stock
• quantity which is executed relative tot he overall quantity traded by other participants
• aggressiveness of the trading algorithm
• duration of the execution

Trade timing describes how an order is being traded compared to what happens in the market. Timing tells you if you are a trend follower or mean reverter. Timing tells you if the way you trade makes you visible to HFT eyes.

Market Impact Over the Trading Period
The benefit of intraday analysis of market impact is to quantify the changes in the market impact as an order is being executed and the dilution of this impact after the end of an execution.

The difference between the price at any given time and the arrival price is a good and frequently used measure of market impact.

Figure 2 shows the price of a stock while an order is being executed. If the returns of the stock is in the perspective of a buy order, prices above 0 means the investor will pay more to purchase a share or will be paid less to sell vs the beginning of execution. The grey bands represent the 95% confidence interval;the white line is the average. At 0 to 100 the impact curve is concave. At 100 to 200 shows the returns revert back.

Market impact may depend on several parameters.

Stock specific characteristics:

• liquidity of stock
• volatility
• spread size

Order-specific characteristics:

• aggressiveness of execution
• quantity to execute
• expected duration

Structure of the Dependency Between Market Impact and Parameters
Period Liquidity Ratio (PLR): participation rate

In figure 3 we plot the market impact for different values of the participation rate.

It can be seen that market impact curves get steeper as the participation rate increases.

In figure 4 we can see two impact surfaces for two different sets of durations. The surface corresponding to the long duration is always above the shorter duration - ie more market impact. Hence the longer the duration the bigger the impact.

With both participation rate and duration contributing to market impact, we can build the model:

The dependency of the market impact on the two parameters separately is concave. This means the impact of one additional percent of participation rate for an order of a given duration is lower for higher participation vs lower participation.

Market Impact on a Longer Horizon
Within daily data the permanent market impact is the remaining price shift after the decay has taken place. The decay phase starts at the end of the execution period and can not be exactly determined - but we consider that the price reaches its final level after one day. Liquidity providers mostly react at the intraday timescale. After one day the temporary market impact of the metaorder execution vanishes.

Price Anticipation or Permanent Market Impact
There are two positions when it comes to permanent market impact.

Position 1: Permanent impact as the consequence of a mechanical process.

• Stock prices move because of the trading of all the market participants. If the selling pressure is stronger than the buying pressure, price goes down and vice versa if buying pressure is the strongest.
• We want to understand the law at the root of the impact of buying or selling pressure on price dynamics.

Position 2: Permanent impact as the trace of new information in the price.

• Informational vision says stock prices move because new information is made available to market participants. New information allows investors to update expectations which leads to new global equilibriums and price levels.

Permanent Market Impact on a Long-term Horizon

We will look at three studies that de-bias the effect of the signal that triggers price variations. This separates the information effect from the pure execution effect from permanent market impact.

[Bacry et al., 2015]: Reports measures made on a database of electronically traded metaorders from the brokerage arm of a large European investment bank during 2010.

• The broker execution database studied does not provide the informational content at the root of the trades.
• Authors assumed that the clients of the broker globally constitute a good representative sample of the diversity of market investors (total portfolio not far from CAPM)
• The study of price moves on the post-execution period net of market portfolio moves is the so called idiosyncratic moves of stock prices.

[Waelbroeck and Gomes, 2015]: Uses a database of trades from 112 portfolio managers across several large institutional asset managers between 2009 and 2012.

• Each trade is labeled as ‘liquidity’ or ‘informed’ trades to separate trades triggered by informational events from the rest.
  • Informed: Metaorders coming from portfolio rebalancing.
  • Uninformed: Cash trades that are related to new subscriptions of redemptions triggered by heterogeneous and relatively exogenous to the market information.
• On a daily basis informed trades have a permanent market impact but uninformed trades do not.

[Brokmann et al., 2015]: Studies a proprietary CFM database consisting of an exhaustive list of 1.6 million metaorders together with the daily signals that lead to these trades over three full years (2011-2013) in a variety of equity markets.

• Uses for each trade the signal that triggered it.
• Uses the intensity of the trading signal which triggered each metaorder to remove the informational content of their studied proprietary metaorder database.
• Divergence of effective and expected price moves gives measure of permanent impact in the absence of informational effect.
• Authors observed no permanent impact on the residual price moves.

De-biasing the Temporary Market IMpact of Autocorrelated Execution During th ePOst-Execution Period

All three studies aggregate signed metaorder flows show autocorrelations even several days after execution. This has an effect of post-execution price variations and needs to be filtered out in order to make an accurate estimation of the permanent market impact.

Aggregated daily participation rates are computed at the stock scale as the sum of the signed traded volume divided by the total market volume for the stock on that day.

Total autocorrelations are the equi-weighted average of the autocorrelations across all the stocks of the studied universe.

Finding a unbiased long-term impact picture is not straightforward. Market impact of metaorders according to daily participation rate turns out to be far from linear.

Withdrawing impact of metaorders traded over the post-execution period uses a simple market impact model in [Waelbroeck and Gomes, 2015]. In [Bacry et al., 2015] a square root model is fit. Price variations and the post-execution period are debiased by subtracting the market impact associated to the aggregated daily participation rate on that date applying the square root model.

CAPM Decomposition of Price Variations
[Bacry et al., 2015] uses a CAPM framework to disentangle the systematic component of the market impact form the idiosyncratic components. For each metaorder the decomposition is centered on teh execution dat over a 41-day period:

$D$ is the metaorder execution date, $P_d$ is the stock’s close price on date $d$, $beta$ designates the beta of the traded stock on the period from $D-20$ to $D+20$, and $I_d$ is the reference index price on date $d$.

$\beta$ is estimated through simple least mean square regression and the idiosyncratic component starting one day before the execution at date $D-1$ is defined as the cumulative sum of the residuals $\Delta W_d$:

The systematic component is defined as $\beta(\log(I_{d-1}) - \log(I_{D-1}))$.

In figure 5 we see the idiosyncratic, systematic, and daily post-execution profiles of price moves. The post-execution profiles of price moves are not yet debiased - the price jump between day 0 and 1 correspond to the market impact of the execution on day 1. Over the time horizon prices slowly trend in the same direction.

After debiasing the post-execution profile the market impact of metaorders executed the day after the execution has been removed. We can see the effect in figure 6.

Over the post-execution period the price converges back to a level lower than the one reached on execution day. The idiosyncratic post-execution profile reaches its initial level before the end of the observed period - this permanent market impact that remains 20 days after the execution is entirely explained by the systematic component (average level of the market).

Based on this, [Bacry et al., 2015] concludes that once temporary market impact of the correlated metaorders executed has been removed and aggregated information (systematic) has been isolated and removed there is no permanent market impact due to the metaorder itself.

The PFP and Orderbooks Dynamics

On equity markets most trading takes place on LOB that are often called CLOB. This is opposite of a quote driven system (used on fixed income markets).

Attractiveness of Limit Orderbooks
Limit orderbooks are convenient for several reasons:

• They are anonymous and provide MLT, leveling the playing field between traders and market-makers.
• They are fast and synchronized, easily supporting fragmentation and competition between trading platforms.

By understanding orderbook dynamics one can answer the question: How can practitioners extract information form orderbook dynamics?

Information Reaching Orderbooks
At the start of the day an orderbook is empty. Traders and market makers send orders that are stored in the orderbook if they do not match resting orders - orders are matched if they are compatible in price. Each match generates a transaction and gives birth to three messages:

• Public trade
• Two private trades: one for each party of the trade

Because the orderbook is hosted on a server in a data center, the localization of the data center is paramount because it influences latency.

Members of the trading facility are connected to the server ina secured way, often via tow channels: one private (SLE) allowing them to send and receive messages to the server and one public channel (SLC).

• On the private connection the trader will send specific order to the matching engine and receive answers and messages concerning his orders.
• On the public channel they will see the transactions and the state of the orderbook.

The Orderbook in Motion
If the fair price of a stock is around $10, the first trader with intentions to buy will buy cheaper and a selling trader will try to sell for more. This message of buying low and selling high can last long as no transaction occurs. It shapes the orderbook and keeps record (limit-by-limit view) of the sum of all orders at each level of price.

To see the trade the order must be fulfilled. At any point in the life cycle of the orderbook a trader can insert, modify, or cancel an order.

The effect of messages can:

• Remove liquidity via cancellation or transaction (Market order or limit marketable order).
• Add liquidity via an insertion.

Transactions can only occur at the first limits of the LOB.

Understanding via Conditioning
To use conditioning of the orderbook we focus on the probability that a limit order touches the first limit only when its size is large. Is the probability different from the probability of an insert at the second limit when it is small?

This conditioning can be split into three trends:

• Zero-intelligence: Has few conditioning and instead focuses on identifying the distribution of variables of interest.
• Game theoretic: Explains the behavior of participants with accuracy. Tries to deduce the theoretical laws for probability of each type of order.
• Empirical: Authors estimate transitions empirically.

Poisson Modeling
We can use the point process to model the probability of occurrence of an event. Poisson processes are standard point processes with intensity $\lambda$. Let $N$ be the number of occurrences of an event, then we can same for some time $t$:

If we know $\lambda$ then the average number of events observed during $\Delta T$ can be approximated using $\lambda \times \Delta T$.

Hence we can isolate three intensities in the orderbook:

• $\lambda^+$: limit order
• $\lambda^c$: cancellation
• $\lambda^x$: transaction

Plain Homogenous Poisson Modeling
The simplest model for the three intensities is to use the homogenous Poisson framework. The probability to observe a liquidity consuming or liquidity providing event is independent of the state of the orderbook.

We can estimate the intensities by starting at time $t$ and record the timestamps of the next event in each category, which we denote $\tau^+, \tau^c, \tau^x$. Obtain the observations of time to wait $\tau - t$ then average them out over the whole database for an estimate of $1/\lambda$.

With the three intensities we can simply run a simulation of the random dynamics of an orderbook and look at the distributions of the queue sizes we obtain during the simulation.

The homogenous Poisson process is very far away from reality. It does not take into account the state of the orderbook. Modeling arrival rate of atomic events (insert, trade, cancel) does not succeed in recovering the observed configurations of queue sizes on the same data.

Improving the Model Based on Conditioning
[Huang et al., 2015] shows a good way to improve the modeling by simply conditioning the three intensities by the size of the considered queue. This is called the Queue Reactive Model.

Model I Instead of avenging the time before events to find the intensity, we keep record of one intensity for each queue size:

1. Start at time $t$.
2. Record the queue size $Q_t$ at $t$ in a generic unit.
3. Observe an event (say $c$) at $\tau^c$.
4. Keep track of $\tau^c - t$ in a register associated to the size of the queue $Q_t^c$ at $t$.

Iterating over a lot of starting times $t$ gives us a collection of $\tau - t$ for each event. We can now average wait times by queue size.

Model II \& III The distribution of sizes at other limits are not good with Model I, so Model II and III improve the statistics by conditioning intensities by the current size of other queues.

These models condition intensities by the size of the concerned queue but also by the size of its two neighbors. For example the intensity of events ont he first queue should be a function of the current size of the first, the second, adn the opposite first queue. To predict probability $p$ of the number of cancellations int he next 10 seconds on the first bid queue of size 4, we can write:

The improvement can be describes by two facts:

• When the first queue is smaller than its opposite queue, it will most probably be consumed.
• When a first queue is large, the second queue will probably see more cancellations than if the first queue is small.

This is called predictive power of imbalance: liquidity imbalance at the first limit gives a god idea of future price direction.

In most cases the amplitude of this future price movement is lower than a tick size, hence it is not possible to build a rewarding arbitrage on this predictor.

Asymptotic Behaviors of the First Two Queues
A/R ratio (Arrival over Departure ratio): This measures the intensity of insertion over te sum of the intensities of cancellation and trades.

When the ratio is greater thn one,the queue will explode (size will never stop increasing) and when its ratio is lower than one the queue will disappear (size will go to zero).

In figure 8 we can see

• The first limit is always depleting to 0.
• The second limit explodes.

In the context of stable sizes of the orderbook:

• The first queue goes to 0 while the second queue explodes.
• Once the first queue disappears the formerly second queue is now promoted as a first queue.
• The promoted queue depletes to prevent explosion.

Liquidity Dynamics vs. Price Dynamics

Note an observation: Now that we have a model reproducing with accuracy liquidity dynamics, you can generate realistic orderbook dynamics - but what about price trajectories you generated?

The volatility of these trajectories is far too low compared to the ones observed in the real database used to fit the intensities. This is because in the simulations where a queue is promoted, the sizing of the new first queue is very large. This leads to price reversion - the negative autocorrelation of intraday price returns is a well-known stylized fact, but for an orderbook driven mode, the effect is too strong.

To compensate for the mean reversion we can introduce the following mechanisms:

• Time to time: when a first queue fully depletes.
• All market participants accept the newly set price as a new ‘fair price: as a consequence the model has to reset the orderbook state to a randomly chosen one in the database.

Summary
The Queue Reactive Model of orderbook dynamics shows the following:

• Predictive power of the imbalance of provided liquidity (sizes of first two limits)
• The usefulness of the heterogenous Poisson process to roughly model the occurrence of orderbook events.
• The need to condition the intensity of the arrival rates of such events byt he size of the concerned queue (and its neighbors).
• The need to add a reset of orderbook state after price movement to prevent intense price mean reversion.

When tick size is large the liquidity contained in each queue will be more meaningful than when the tick is small.

The point process in the orderbook is important but other models exist. Instead of counting instance in time, we can count events - changing the meaning of the obtained models. These are called propagator models.

Optimal Trading Methods

Algorithmic trading: Adapting Trading Style to Investor Needs
The following can be noted in electronic execution from investors:

• In the US, HFTs are almost everywhere - stealth trading is the main point of focus.
• In Asia the comparison between the efficient of trading in each local market is very importance.
• In Europe the need to mix between avoiding being detected by HFTs and understanding the efficient of interacting with a set of orderbooks to adjust order routing policies as fast as possible.

European investors should target Implemented Shortfall benchmarks, Asian ones should focus on VWAP or PoV, and US ones should look at liquidity seekers.

Liquidity seekers are more suitable to trading orders without minimum participation rate constrains, IS are for small orders, VWAP are for larger ones and PoV for very large ones.

The textbook summarizes stock features and features of trading algos in the tables below:

Customization Offers Multi-feature Trading Styles
These benchmarks can be combined: we can add maximum and minimum participation rates to an algo to make it behave like a PoV in the case of a really high or low market volume during the trading.

You can match sophisticated benchmarks like ‘follow market flows if there is high volume, finish fast if the price is really low, and use as much liquidity as possible’ using VWAP or TWAP as a backbone with a minimum participation rate.

Market data needed to build benchmark based policy are noisy because of HFT activity in the orderbook and uncertainty on some important parameters.

From a practical standpoint the optimal trading curve o follow is expressed as a trade rate that has to be represented by the algorithm. If it obtains trades too fast with respect to this curve, it has to stop posting; if it does not obtain enough it needs to obtain trades more aggressively, paying the spread. The curve is not a thin line but a thick one (given noisy data) such that it can define a trading envelope. Inside the envelope any trading trajectory cannot a priori be said to be more efficient than another from a risk-control perspective.

Inside a trading envelope any combination of liquidity capturing techniques can be plugged in without harming the optimality of the risk control layer (strategic layer). It is also possible to inject any volume or price-driven constraints if needed into the way they are built. Customization of a trading algo follows a two step process:

1. Understanding the desired risk profile and injecting the needed constraints into the trading envelope building process
2. Understanding the mix of liquidity adapted to the investment style and plugging the associated tactics into the envelopes with respect to properly defined market conditions.

Liquidity Seeking Algorithms are no Longer Nice to Have
Any market participant should be able to evenly access all of the trading venue quoting a given instrument. Any investor should be able to access all the available liquidity at a given marketable price.

In the US the trade-through rule demands that trading venues themselves reroute orders that are marketable at a better price elsewhere. In Europe, the executing broker is in charge of the kind of rerouting - they have to disclose to their clients the best execution policy explaining his rerouting methodology.

The difference is important:

• In the US the trading venues have to be able to access an official consolidated orderbook. The clearing and settlement fees/costs have to be the same for all trading venues otherwise a market participant could direct a marketable order on a venue that seems to offer a cheaper price than another (but the participant will pay more, all costs included).
• In Europe the executing brokers are in charge to select the way they offer consolidated access to liquidity and have to prove it. It increases the cost of fragmentation for the intermediaries and demands that the investors are able to understand the stakes of these ‘consolidated services’.

Impact of Trading Costs/Fees
Let us denote venue $V$ at price $P$ with fees $f$. Is it cheaper to remove liquidity on another venue $V’$ at a higher price (for a buy order) $P(1+\delta)$, where $P\delta$ is the tick size in euro, because of cheaper fees (say the fees on $V’$ are $f’$ and on $V$ they are $f$ such that $f’ = f - \phi$).

If we have a better price on venue $V’$ than on $V$ we can write: \(\begin{align} P(1+\delta)(1+f-\phi)Q <P(1+f)Q. \end{align}\)

This is equivalent to having the tick size (in basis points) small enough such that:

This means that the tick size should be as small as the fee difference.

A liquidity provider has to be sensitive to the fees he or she will have to pay once this order will be executed. Most times an observer can expect to see limit orders going first on the cheapest venue.

Impact of Latency
It can be said that what is important is not the liquidity that can be seen in the orderbooks but the probability that you will be able to obtain liquidity at a given price level when your order reaches the orderbooks, taking into account the latency discrepancies between venues, the mirrored orders, and hidden liquidity.

Seeking an Optimal Liquidity Capturing Scheme
Dealing with a marketable order (buy market order or buy order with a limit price higher than the best ask price on at least one trading venue) has shown that effects to take into account are as follows:

• Probability of hidden liquidity in the orderbooks.
• The probability that when the latency to all venues is not the same, HFTs take advantage of information conveyed by market orders reaching one venue to cancel their liquidity providing orders.

Building a Liquidity Seeker
A liquidity seeker has to take care of the following:

• Removing liquidity optimally using estimates of the hidden liquidity, mirrored orders, and latency to each venue open to trading.
• Posting limit orders optimally to end an order as fast as possible.

Like in dark pools, liquidity in any pool is clustered. Once a trade has occurred, the probability of observing another one increases.

Using fill rates and real liquidity estimates, we can find the optimized allocation of quantity to trade across different trading venues.

Well, this is the end of my mini series! Don’t fret, I have more microstructure posts coming soon; specifically look out for some analysis on the papers mentioned in the textbook.

But anyways, I hope you enjoyed my aggregated notes, both transcribed and hand written, and that you learned something cool.

As a reward for staying till the end, here’s a picture of Linnea’s cat:

I’ve omitted most of the appendix material, historical analysis of European alternative markets, and the Spanish Experiment. For the curious reader more details can be found in Chapter 2.5.

This post is a continuation of Chapter 1, which can be found under ‘Market Microstructure’ on my ‘Blog Posts’ tab. Happy reading!

Below you can find a pdf of the textbook and my notes:

• Market Microstructure in Practice
• Chapter 1 Notes
• Chapter 2 Notes
• Chapter 3 Notes

Intraday Market Share to Volume Curves: Stationary Issues

Not all market participants trade for the same reasons. Hence, when liquidity providers create bridges among pools of liquidity, liquidity remains concentrated.

To catch the reference price, primary markets are preferred because they create these reference prices.

Primary markets maintain a monopoly on the flow of another reference price effect called the Exchange Delivery Settlement Price (EDSP). This is the reference price that will be used for the valuation of the derivatives at the expiry.

Inventory Driven Investors Need Fixing Auctions
Equity trading consists of two types of trading sequences:

• Continuous phase: This consists of immediate trade execution where both sides of the orderbook match.
• Fixing Auction phase: this phase includes both opening and closing auctions that are important to fill and clear the orderbook. One can have up to four auctions at once. Some exchanges omit the continuous phase.

Fixing auctions are used for consensual pricing and to unwind orders with huge volume. During witching days fixing turnover is high given that derivative products that expire are taken or closed during these auctions to benefit form same price.

By concentrating more agents, fixing auctions are allowed to form a more consensual price for the whole volume traded. This is very useful if the price is a reference for many valuations (closing prices usually are) and to build orderbooks before the opening of the market.

Fixing auctions also reduce price impact of a trade given high volatility - enhancement of info revealed during the auction stabilizes volatility and increases price stability.

Rules of Call Auctions
Fixing price is determined by four steps - if the price is not clear by the first step the next step is applied. If the last step is not applicable the fixing price can not be determined.

1. Maximum Executable Volume: Establish the price at which the highest quantity will be executed.
2. Minimum Surplus: Determine price based on the minimum quantity left in the market at the fixing price.
3. Market Pressure: If surplus is wholly on the buy side then the highest potential price is the fixing price. Vice versa is true. If surplus is on both sides then the next rule is applied.
4. Reference Price: The reference price is the last price traded.

Different methodologies exist for different exchanges.

Theoretical Matching Curves
The composition of participants is different with opening and closing auctions.

Matching Curves: Relative difference between the volume that would be matched at time $t$ if the auctions ended at time $t$ with the final volume matched at the real fixing for each time $t$.

When a matching curve is zero this means total end volume has been sent to the market - every participant revealed its desire to trade.

In the pre-opening period book building accelerates as time goes on. Matched volume only reaches its final volume in the last few moments. The curve is convex in figure 1.

During closing call, matched volume rapidly increases to its final level as agents already present must participate in the fixing order to close positions. The curve is concave.

Participants take longer to intervene in opening vs closing because there is less uncertainty but also less patience at the end of the day. Participants face most constraints in the closing session given its the last time to open or close a position for the day.

The Opaque German Orderbook
For closing call matching curves in other markets, more than 90% of the distribution remains below the final value. This means that 50% of the time the final volume will be reached before the end of th fixing auction. The upper quartile will go higher than the zero threshold - and given our fixing pricing rules of maximizing volume - this means that a certain amounts of orders needed to be cancelled or modified before the end (around 25%).

In German auctions the orderbook is not revealed. The orders cancelled are probably players fishing for information by cancelling and sending orders to test fill rates. Exploitation/exploration tradeoffs are needed to optimize information levels in the orderbook.

Information Peaks
The pace of information sent into closing and opening auctions follow certain patterns. These updates make matched volume closer to final volume.

Investor Optimal Trading Rate
Market design and information flow timing imply liquidity patterns.

Intraday volume patters allow us to compute execution benchmarks like VWAP and measure how informative traded prices are during intervals. Informed and liquidity traders have in best interest to time their trades together. The meeting of these points create specific intraday volume patterns.

The U and J shapes seen in figure 2 should be analyzed with different causes in context:

• Opening of a linked market.
• Release of news at a predetermined time of day on a regular basis.
• Computation of reference prices and if fixing phase is used for this purpose.
  • For many European funds the reference price is the closing price. India does not have a closing price and instead uses VWAP over the last half-hour of trading.

Hong Kong’s volume pattern is shown to be highly dependent with the way reference price is computed.

Brazil’s opening hours are slow until US markets open 30 minutes later. At this time the Brazilian markets will have real price formation; Brazil’s markets are heavily dependent on the US’, instilling winter and summer hours to be more closely matched with US hours. This same market dependency can be said about South African markets and European markets.

Effects of Volume Patterns
Opening of the US Markets:

• NY shift: Computed volume pattern using days where there is only a 5-hour difference between French time and US time.
  • Increase in volume occurs when the US markets are open one hour earlier.
  • Spikes on the third Friday of March and select months are due to expiry dates.
• NY Closed: Volume pattern on the days where the US market is closed but the French market is open. Activity volume is much lower.

US macroeconomic news:

• Immediate release of economic activity reports on volume depend on how surprising it is for the market.

Equity derivatives expires:

• Witching days and expiry days make investors who do not want to roll over positions unwind cash positions and generate huge volumes during certain time stamps.
• The exchange date settlement price (EDSP) for single stock futures and options is the closing call auction price. This is why closing fixings have large volume as well.

Fragmentation and Evolution of Intraday Volume Patterns
Market share is more important at the beginning vs end of the day. This is because beginning PFP is taking place on the primary market from unmatched opening fixing orders. As well, mature players will time SORs to not route orders to MTFs at time if not needed.

Chi-X’s Jan-Feb 2012 dip (below the 50% threshold) can be explained by the influx of liquidity providers who need to increase activity at the end of the day.

Participants unwinding their cash hedging positions at a price close to EDSP will want to trade on markets where the price will be observed to target reference prices and directly participate in its fixing with their own execution prices.

Four Main Liquidity Variables

The four main liquidity variables are:

• Traded Volume
• Bid-Ask Spread
• Volatility
• Quoted Quantities

Given the state of microstructure and associated trading offers, liquidity providers and consumers explicitly or implicitly accept to trade. The amount traded reflects fragmentation.

Offered liquidity

• Offered liquidity serves as the counter part for traded volume as a variable for liquidity. It can be approximated as the first limit of the books.

Book size, liquidity in the book:

• The book size is the average between the quantity at the best bid and at the best ask. (for smaller tick sizes several limits will need to be summed compared to larger ticks).
• Book size is significant for maximum tradable quantity and serves as a lower bound for traded volume. This concepts is more complex given orderbooks replenish.

All in all book size is an indication of interest of liquidity providers who are ready to trade at the current prices (around mid-price).

The bid-ask spread is the best lowest ask and the highest buy. An instrument is less liquid if the spread is tighter and vice versa.

Comparing bid-ask spread $\psi_1$ vs $\psi_2$ you will find that

$N_i$ is the average number of traders per day. $a, b$ are constants that seem to be shared by all instruments given an asset class.

Using $\eta$ we can compute the bid-ask bounce to be,

$\eta$ is estimated using traded price and

• $N^a$ of alternations of price over the day
• $N^c$ of continuations of price over the day

Then,

$\eta$ is influenced by the bid-ask bounces. The more bounces the more alternations and the smaller the parameter. If prices move randomly then $\eta$ is balanced. This parameter can be used to predict the effect of a tick change.

Intraday Seasonalities
Traded volumes, bid-ask spreads, quantities at first limits, and volatility exhibit intraday patterns. They correspond to more uncertainty on the price after the opening, less activity during mid-day, and a run to end trading before the close. Scheduled events like news influence these patterns.

Volatility is estimated by different models such as Brownian models. Implied volatility can be used as a proxy for volatility but this measure looks at uncertainty of future prices. In the context of microstructure we need to understand the frictions of past trading activity.

The naive estimator of volatility is not a good estimator because of pollution of bid-ask bounce.

In figure 3 we see that the four curves exhibit intraday seasonality. Traded volume, bid-ask spread, and volatility are high at the start of the day. By mid day the price formation process readjusts prices and movement has calmed. By the end of the day volume increases given participants trying to close positions.

When bid-ask spread is large the volume on the book is rather small and vice versa. At the end of the day the spread is at its minimum and when it is at its minimum the book size increases to compensate. Liquidity providers who can’t step into the spread (given its size is the tick size) will have to increase sizes at best bid and ask to provide liquidity, hence increasing the book size.

When the spread is stuck to one tick an increase in the book size plays a role similar to a spread decrease.

Does More Liquidity Guarantee a Better Market Share?

Before MiFID, transactions took place on single national exchanges where liquidity providers and investors were mandated to interact in the same place. Now, with the introduction of competition, exchanges have to attract and keep market-makers, transforming investors into connected members.

Market-makers are attracted through maker/taker fees. Members are attracted through connections costs.

The number of connected member and the satisfaction of market makers is conditional on the liquidity of the exchange

Liquid stock: High volume combined with low volatility and bid ask spread.

Creating and maintaining a trading venue requires:

• Liquidity providers
• Competitive bid-ask spreads
• Connected members

Bid-Ask Spread’s Influence on Liquidity

• The higher the volume on a trading venue the more its market share.
• Volatility cannot be considered as an element of market design choices for an exchange.

The bid-ask spread represents the reward given to the market maker for the risk of providing liquidity.

The bid-ask spread is a way for exchanges to grab market share by increasing the chances the venue hosts a transaction. This is different in the US where mandates for best executions are implemented by consolidated tape.

The Bid-Ask Spread and Volatility Move Accordingly

The daily bid-ask spread increases with daily volatility.

The higher the volatility the higher market makers ask to compensate for their possible loses. This can be modeled with a simple linear regression where volatility can be adjusted per trade (seen in figure 4).

For contributions of microstructure, short term intervals need to be taken into account. After the opening of the market, the market suffers through lots of noise; hence the bid-ask spread and volatility are high. Movement dies down in the middle of the day but picks up by closing because of HFTs needs to flatten residual inventory, making them accept tighter bid-ask spreads regardless of risk.

Bid-Ask Spread and Market Share are Deeply LInked
Market maker activity is more risky on less traded stocks - hence their reward is higher. The market share of a venue moves according to the quality of its bid-ask spread in relative terms.

Tight relative bid-ask spreads imply an increasing market share because SORs are mandated to generate transactions on the venue providing the best price.

Exchanges Need to Show Volatility Resistance
The dependence of the bid-ask spread on volatility is taken into account because it has impact on market makers. Because market makers have the duty to provide liquidity to other participants, they will choose markets that are resilient to relatively high levels of volatility.

Resilience can be measure with the relationship between the number of trades on a trading venue and its price volatility.

In figure 5(a) the primary market has the highest slope, meaning that it offers a far higher relative number of trades during the volatile periods than other markets. The PFP tends to occur where transactions have the most resilience to volatility, hence Chi-X takes a greater part in the PFP vs. BATS and Turquoise.

But this behavior differs from liquid to illiquid stocks.

Indexes and venues can be compared using the correlation between the number of trades and high frequency volatility. This is called flow reactivity which dictates the hierarchy of destinations. When a venue has low flow reactivity, high frequency market makers stop offering as much liquidity as they do during calmer periods.

There is a strong relationship between quality of trading venue, success, and market makers that operate it. Venues have the obligation to keep market makers active in order to improve market share through competitive spreads and resilience to volatility (better liquidity).

The Agenda of HFTs

The main question concerning HFTs is: Does this market design enable competition between several markets have negative externalities?

Fragmentation was in response to financial market monopolies to decrease prices and increase trading quality.

Metric for the Balance in liquidity Among Indexes Turnover is chosen as a measure of heterogeneity of liquidity among stocks.

We can then ask the question: How well is turnover dispatched between components of indexes? Are only a few stocks responsible for activity on one index or do all components contribute to it equally?

The coverage metric and Gini coefficient evaluate this level of inequity.

Coverage metric:

• This indicates which percentage of the number of stocks is needed to realize 80% of the total turnover.
• This is a static viewpoint that does not take into account the shape of repartition of turnover across stocks. Repartition meaning if the 20% is concentrated over medium traded stocks or spread thinly over low activity ones.

Gini coefficient:

• The Gini coefficient uses the whole repartition curve to estimate the degree of inequality among stocks.
• The repartition curve is found by summing all intraday turnover on each component after the components have been sorted in decreasing order and diving by sum (to normalize).

The Gini index and Kolmogorov–Smirnov test is covered in chapter 1 notes. It can also be reviewed in A.9.

The higher the coverage or the lower the coefficient the more the investor can find an equivalent quantity of liquidity on every stock of the index for a specific exchange.

In figure 6 the turnover for 45 FTSE 100 stocks equates to 80% of the total turnover of the entire index on the LSE. By tracing a horizontal line from the y axis where it reads 80% we can find the corresponding x axis value that intersects with the repartition curve. The area under the curve multiplied by two represents the Gini index.

LSE’s coverage liquidity metric is around 45%. The Gini index, which indicates how far the repartition of turnover is from a uniform repartition, is equal to 0.99.

The History of Coverage
Between 2009 - 2012 coverage values increased, decreasing the discrepancies between venues. This created positive homogeneity but did not impact illiquid stocks given HFTs could not increase their activity on them given hedging and short selling is incredibly expensive with large risk.

With the evolution of Multilateral Trading facilities (MTFs), market share became dependent on stock liquidity. Because HFTs have higher risk when extending their activity to less liquid stocks, MTFs increase coverage for wide indexes. Therefore, HFTs extend their perimeter of action for a symbiotic relationship.

High frequency market makers increase uniformity among indexes coverage by spreading volume and ensure the contribution of MTFs to the price formation process by arbitrage.

HFTs Do Not Impact All Investors Equally
Platforms reward trading strategies that make use of the maker/taker schedule. Arbitrageurs use technology to lower inventory risk by trading very small orders very frequently.

The SEC definition for HFTs is as follows:

1. Very high speed and sophisticated computer programs for generating, routing, and executing orders.
2. Use of co-location services and individual data feeds offered by exchanges and MTFs to minimize network and other types of latencies.
3. Very short timeframes for establishing and liquidating positions.
4. Submission of numerous orders that are cancelled after submission.
5. Ending the trading day in as close to flat position as possible.

How HFTs Lower Spread Without Changing Execution Costs

The effective spread is reduced on an overall basis, but the costs for other participants remain unchanged or slightly increased.

If we consider a market with player A and B, seen in figure 7(a), A becomes impatient and hits player B’s price in order to trade. His cost is the bid-ask spread $\psi$.

In 7(b) player B becomes impatient and hits player A’s price in order to trade. His cost is the bid-ask spread $\psi$. Hence A and B will bear the cost of $\psi / 2$.

In 7(c) a HFT intervenes on the market and takes a position on the outside of B and the first limit against A. A hits the HFT price and his cost is slightly more: $\psi / 2 + \epsilon$. The HFT takes a new position rapidly in front of B, and since B’s order is not matched, B needs to hit the HFTs price. His cost then becomes $\psi / 2 + \epsilon$.

The final effective bid-ask spread is reduced since it represents slightly more than half the formal bid-ask spread. The HFT forces player A and B to become more aggressive, but given trade sizes are small, prompts other participants to fraction their orders, incurring cost. These fixed costs are unevenly distributed and lead to a deterioration in trading conditions for final investors.

Bid-Ask Spread: Cost and Uncertainty for Investors
We now consider if bid-ask spreads would have been unchanged if HFTs had not appeared. If we consider HFTs involved in $q$% of trades, the new bid-ask spread is

The proportion of HFTs among trades is assumed to be 40% and the spreads take the values,

For the cost $C$ the investor has to look at the probability $r$ of meeting a HFT. Three scenarios are outlined:

• If the investor encounters a HFT and is willing to be aggressive the his cost is $C = \psi_{HFT} / 2 + \psi_{B} / 2$.
• If the investor does not encounter a HFT then his cost is $C = \psi_B$.
• If he does not encounter a HFT and is passive he pays $C = 0$.

In figure 8 the curve represents $C$ and the uncertainty that relates to $C$. The smaller the cost the higher the uncertainty about it (like the efficient fronter portfolio). Points on the line vary depending on the probability $r$ of meeting an HFT. Increasing this probability increases cost by reduces uncertainty.

For large numbers of trades the Central Limit Theorem (CLT) is used such that,

The uncertainty is lower with the increase of trades.

The bid-ask spread is a source of cost for investors in the market. Both the expected value of this cost and the uncertainty are key figures. Depending on the number of trades and the probability of meeting an HFT actor expectation will not vary but uncertainty will significantly.

The Link Between Fragmentation and Systemic Risk

Systematic risk can be seen as the residual risk that is left in a portfolio when all the other risks have been hedged through all possible financial instruments.

Investors ask if systematic risk has increased with the rise fo HFTs. This gives way to the relationship between extra and intraday volatility. At what extent does extraday volatility effect intraday volatility?

Overnight vs Intraday Correlation
We examine the trends of the average intraday and overnight correlations among the components of an index.

Correlation spikes are often well explained by overnight ones.

Intraday correlation reflects to some extent the amount of co-movement between securities at intraday scale. This is an intermediary marker in the contagion process which leads microscopic causes for macroscopic effects.

To calculate intraday correlation:

where

During the night the market is organized as an auction fixing and during the day as continuos trading. Hence opening fixing is less liable to be contaminated by loud microstructure noise and tends to reflect consensus of price vs closing fixing. The overnight corrections are closer to extraday correlation.

Flash Crash Analysis
On May 6, 2012 the US stock market suffered a sudden and rapid drop following a rapid recovery.

The SEC and CFTC suggested several reasons. The large sale of US indexes on May 6th initiated by a trader who did not put a limit price on the algorithmic order caused an excess of volume that brought HFTs to hedge themselves in light of high risk that day. The snowball effect of hedging eventually returned to its starting point and caused escalations of volume while liquidity offered remained unchanged. The artificially increased volume caused traders to increase flow sent to the market and fed the vicious cycle.

The jump in volume in the market was followed by a large number of HFT players liquidating their positions, increasing imbalance between the demand and supply of liquidly.

The E-mini and SPY showed desynchronization. With the nigh volatility b=on both derivatives, this caused an interruption in the number of orders which transmitted a shock to the stock market. With a pause on orders and the re-routing of OTC orders to primary markets, the pressure on the aggressive side of the orderbook was rising.

As well the Liquidity Replenishment POint (LRP) that slackens the market for a period of time for liquidity to return exacerbated the imbalance between supply and demand. When LRP is triggered all aggressive orders are rerouted to other exchanges - the immediate effect of removing passive liquidity in the NYSE but retaining aggressive liquidity is disastrous.

Systematic Risk and Circuit Breaking
The Flash Crash raised new concerns about the stability of markets. Automation of market making seems to allow liquidity providers to suffer from less adverse selection (because they process information faster). If adverse selection of market makers is a stabilizing force for the PFP, then it can be expected that the more aware adverse selection providers are, the less stable the markets.

Circuit Breakers
Circuit Breakers may be crucial protection against sudden demand of liquidity by market participants. In principle there is a need to stop the PFP in a local or global scale. For a micro scale how can we differentiae between usual liquidity imbalance vs an unusual one that halts trading? Is it even a good idea to halt trading?

Circuit breaks are expected:

• To be activated ini case of liquidity imbalance, but all of them loo at prices. Instead looking at the conjunction of price moves ad poor liquidity can help define more adequate thresholds.
• To be activated in case of abnormal price swings. One should take into account two levels of volatility (daily and intraday variations) using intraday seasonalites.
• Instruments whose prices are indexed on halted tradable assets should be halted when trading on the latter has been halted. This ensures fair trade.
• Highly correlated instrument should be halted to stop propagation of trading impact.
• Synchronization of circuit breakers should be considered and their propagation effect.

Beyond Equity Markets

Trading Futures
Futures are highly electronified. They have no call auction and participants have to agree on liquid hours during which they trade. Futures usually correlate to trading on equities but some studies suggest PFP on futures leads PFP on their components. Tick sizes of futures are usually larger and cause lots of bid-ask bounce and price reversion. Sizes of first limits of futures are therefore much larger compared to equities.

Future markets are almost not fragmented given post-trading processes would be more complex if fragmented.

Trading Options
The post-trading process on options is more complex than futures. US options markets are fragmented (European ones are note). The current challenge for options markets is to attract market makers given they only post liquidity in limit orderbooks and the buyside consumes liquidity.

Trading Fixed-Income Products
Fixed income products cover a wide variety of instruments: swaps, bonds, foreign exchange credit default swaps, ect. These highly liquid products are traded on electronic platforms close to equites.

Chapter 2 notes are complete!

The penultimate chapter of Market Microstructure in Practice will focus on optimal organizations for optimal trading - specifically targeting trading structure, market impact measurements, and the PFP dynamics on orderbooks.

Fenrir’s large black body covers the canvas while a pale limb in the corner entices the jaws of the beast to sever it completely off the limb. He is bound by a silky pink ribbon called Gleipnir, whose strength seems inconsequential next to the hulking figure of the animal.

The painting is both mythic and unsettling, blurring the line between predator and prey, reality and fable. This tension lies at the heart of Walton Ford’s piece Gleipnir, where the fantastical collides with naturalist narratives for a spin of the unusual.

For the American artist Walton Ford, this is commonplace. Ford’s art is vivid and wild, pushing the viewer to find the storyline embedded into his subjects. His landscapes seems to grapple with the confines of their canvas, exhibiting the punishing nature of animal order and unabashed pride for the untamed.

Selfishly, in hopes of bringing some of his genius with me, I had went to CVS to print out a selection of his work before the start of junior year. My favorite out of the set, Delirium (2004), was a vertical painting of a bald eagle in flight, intended to hang in my room above my desk. When my roommates inquired about my selection of posters, I ecstatically showed them the poster of the bird.

“Oh so your room is going to be hunting themed?”

No you cretin, it is not. Safe to say they sorely misunderstood the subject, so I relocated the poster to my childhood bedroom for the pleasure of more discerning eyes.

But for the more perspicacious reader, it would be my honor to introduce the history, background, and interpretations of Delirium (2004), and why it represents such a piece of brilliance to me.

To start, we must turn our attention to an ornithologist by the name of John James Audubon.

The Natural Order of Birds, Skulls, and Race

“I must acknowledge that as I watched his eye, and observed his looks of proud disdain, I felt towards him not so generously as I ought to have done” - John James Audubon, 1835

John James Audubon was a naturalist and ornithologist credited for documenting and introducing the public to the biodiversity of America. His seminal The Birds of America was his magnum opus and response to his life’s objective of illustrating all bird species in America.

Comprising 435 life-size prints, The Birds of America graces museums across the country, reverently coddled in air conditioned rooms and treated to the silk gloves as museum conservatives riffle through their pages. Audubon too is graced with such admiration; buildings, neighborhoods, and streets sport his name - a testament to his contributions to science and a promise to withhold his legacy.

But every coin has two sides. The naturalist was a staunch anti-abolitionist who embraced racial pseudoscience. Famously, Audubon collected five skulls of the unburied on a battlefield during the Texas War to be sent to Samuel Morton. Morton used these skulls to “define racial categories and hierarchies” by studying cranial differences that he thought to define human races. Unsurprisingly, Morton thought that the race with the largest brains, and consequently had the highest intelligence, belonged to Caucasians¹².

Audubon “had no stance on slavery” according to earlier biographers by…owning slaves. It is known that the Audubon household continually bought and sold slaves as financial constraints allowed.

Audubon never saw people of color, African Americans and the Indigenous, as equals. Even when outsourcing assistance and observations to communities who were adept with the local flora and fauna, John Audubon refused to acknowledge or credit their expertise. Instead, he coined multiple racially charged short stories depicting Black men as menacing and dangerous, hoping to captivate readers with the ‘savageness’ of the ‘other race’.

It is still not understood if Audubon was mixed race having self cited (and lied) about his mothers European origins³.

Trapping a Golden Eagle

According to the New England Journal of Aesthetic Research, “…on February 24, 1833, Ethan Allen Greenwood asked Audubon to help him identify a live eagle he purchased from a man who hunted fox with spring traps in New Hampshire’s White Mountains.”

Audubon recounts his experience of setting the trap and waiting for it to catch in his 1835 Ornithological Biography, citing by the morning the bird was caught by one of its talons, notwithstanding a struggle. Audubon convinced Greenwood to sell the 3-foot-long female golden eagle to him, hoping to capture it’s 7-foot wingspan in his The Birds of America etchings⁴.

In his biography Audubon oscillated between wanting to set the bird free “might he return to his native mountains…spread out his broad wings and sail away towards the rocks of his wild haunts” or pin his wings onto a board “for the express purpose of shewing you his semblance”.

The more he observed the bird, the more his ambivalence grew, and in the end, Audubon’s own fever to capture the bird’s natural elegance and pride won out over “the more generous design of setting him at liberty”.

The next day he locked the bird in a small shed, sealing it with blankets and a pan of charcoal. He waited for the bird to inhale the fumes and suffocate, listening for the telltale sign of its body falling off its perch.

After hours of waiting Audubon finally checked on the bird - only to find its unflinching eyes staring back at him. This time, he intensified his efforts by adding sulfur to the shed to a point of unbearability even for himself. But alas the bird still lived, so Audubon, as a last resort, pierced the bald eagle’s heart with a pointed piece of steel. His “proud prisoner instantly fell dead, without even ruffling a feather”.

While working on the painting of the dead pinned golden eagle, Audubon was “suddenly seized with a spasmodic affection”. The stroke rendered much of his lip paralyzed and his hands unusable.

The Descent Into Delirium

In Delirium (2004), painted by Walton Ford, we see the same bald eagle from Audubon’s illustrations. Her chest heaves from the metal spike driven into her heart and she chokes on the noxious fumes of charcoal and sulfur from the shed. Yet, even chained onto her perch, she is free from death. On the bottom left we see a rendition of Audubon lying on the snowy ground. He is dead, supposedly succumbing to the stroke that almost took his life in 1833.

The Folly of Animal and Man

This painting is striking to me on so many levels. The bird seems to cry out into the snowy landscape, avenging her past reality to set herself free from the chains of human brutality. She transcends her bindings as she escapes with her shackles.

Her life in the hands of her prisoner is nothing more than an accessory of human greed. Her captor, in hopes of capturing her animal presence for himself, intends to discard her body, letting her rot as he parades his imposed image of her. He forces a reinvention of the bird through his illustration, and to make sure his representation is superior, kills her. He trifles with the idea of mercy, but in his mind only one can exist, and with the power to terminate others, willingly does so.

But Walton Ford punishes Audubon for his cruel folly by reversing their subject’s interactions with death. Balancing the scales, he allows for one moment in time, for an irrational - delirious - instance, to let animal triumph over man. But the bird is not freed unscathed as she flees with the accessories of chains on her legs and a spike through her heart.

She herself is crazed from the smoke and realistically faces demise. Instead of her wings tacked to a board, her body will be pinned by the gravity of death. Her delirium is inevitable - her strength to hold out among the smoke is foolishness. Ford gives her the moment of success, but illustrates her impending doom on the horizon of her new chance of life.

In contrast to Ford’s Delirium, Audubon’s Golden Eagle is staid in nature. It’s a polite rendition of a majestic eagle that is barely off the ground. Ford understands these absences of - ironically - life, by foiling his painting with Audubon’s.

Ford’s eagle faces the other way, her wings are open, and instead of clutching sustenance in her talons, is bound by the metal of her captor. Instead of pandering to Audubon’s self eulogization of his adventurous acts - seen by his own self portrait in the landscape straddling a tree and wrangling with birds - Ford depicts Audubon inflicted by his own actions, too blind to see past his delirium of conquering the landscape around him to feel his heart fail.

I like to think of Walton Ford’s revival of the Golden Eagle as a metaphorical unraveling of John James Audubon’s pursuit of dominance in race, achievement, and intellectualism.

Audubon’s foray into conquering mother nature is not unlike his campaign for racial hierarchy. By trying to tame the wilderness and natural order around him, Audubon tries to mimic the superiority he finds throughout his racial rhetoric by wrestling with his subject’s nature in hopes of molding their presence under his own. He, the artist, wields the brush but also the sword - two instruments he uses to reroute and rewrite history.

In the end delirium is inflicted on both bird and human - a small reprieve of balance that Ford allots us among the scenery of human barbarity.

Touted as a beginner friendly introduction to microstructure and its mechanisms, Market Microstructure in Practice primarily focuses on European market timelines and the evolution of Markets in Financial Instruments Directive (MiFID) regulation. Even if some microstructure design is not directly applicable to US counterparts, it’s still a very interesting read that has benefited my understanding.

I’ve decided to split this book into three separate posts, each dedicated to its own chapter. This series of blog posts aims to compress all my hand written notes and concepts introduced in Market Microstructure in Practice into an abridged notebook of sorts. I’ve outlined important concepts and some technical appendix insights that I found interesting and applicable while skipping some eulogies on the history of European markets and the birth of MiFID. Happy reading!

Below you can find a pdf of the textbook and my notes:

• Market Microstructure in Practice
• Chapter 1 Notes
• Chapter 2 Notes
• Chapter 3 Notes

Liquidity in Question

Market Impact: If information symmetry occurs between buyers and sellers, sellers offer to sell at worse prices if the buyer is not stealthy enough.

Bid-ask spread: Is the distance between bid and ask price.

Slices: When an investor is seeking liquidity given a large order they will have to splice such order through venues. Optimal slicing is in reference to an optimal trading theory.

Decay: Decay of market impact serves to answer the question - if I consume liquidity on an asset at time $t$ how much time is needed for the price to revert?

Synchronization: Large buy orders can be obscured with sell orders if there are high levels of synchronization. Hence, market impact of a large order is the average over enough market configurations.

Price Formation Process (PFP): Buying or selling pressure that is not consistent with market participant consensus will only generate a temporary impact. The opposite is true for aligning motives of participants.

Microstructure of a Market: Defined as the way different traders share their responsibilities and roles.

How the Market Consumes Liquidity
Regulatory changes in 2005 have lead to new price formation processes that have shaped the market landscape. Now, HFTs act as market makers, a consequence of fragmentation within venues and products.

Because liquidity bridges need to be established to ensure orders are filled, arbitrageurs (HFTs) will take half the difference of the bid-ask spread and improve the level of information. Acting as information delegates, the price of HFTs adding to the fragmentation of markets is the increased flexibility of cross-trading between venues.

The Price Formation Process
Driven by information, the price formation process is a paradox of its own. Sharing information is adverse for an investor, yet it is essential to use information from other participants to make informed trades.

In the US this conundrum has manifested into regulations across asset classes. For fixed income and equities markets, the Centralized Limit Order Book (CLOB) or Request for Quotes (RFQ) is used. The differences are aggregated below.

RFQs:

• Bilateral
• Trader has fundamental information and dealer has information on flows (asymmetry of information)
• Trades are more exposed to opportunity cost while dealers are exposed to adverse selection cost
• Market makers widely use conditional orders RFQs to standardize refusal to trade.
• Stalled quotes are more present

CLOBs:

• Multilateral
• Participants are exposed to both opportunity cost and adverse selection cost depending on limit or market order
• Vanishing liquidity is more present

European Market Fragmentation
In the US, options are traded electronically or by using limit and market orders. This multilateral approach differs from European regulation which instead hinges on fragmentation of Central Counter Parties (CCPs) that make it virtually impossible to net-zero buy and sell the same option on different venues. Without normalization of CCPs, competition on the derivatives market is impossible.

When the Markets in Financial Instruments Directive (MiFID) was created in 2007, the European Union sought to change equities markets by liberalizing trading venues and creating a ‘best execution policy’. However, the application of the directive was surprising - liberalization created only a small number of alternative trading venues (Chi-X, Turquoise, BATS Europe, NASDAQ-OMX) which created high levels of competition - splitting markets into niches.

The ‘best execution policy’ was a matter of offering consolidated access to liquidity by both passive and aggressive orders through Broker Crossing Networks (BCNs), Smart Order Routers (SORs), and others.

The Importance of Market Microstructure
At its core, market microstructure generates the price formation process. This process is the “heart of capitalism” that balances supply and demand through the context of information. The more information available to trades, the better the PFP becomes. Conversely, information leakage and front running can create negative price impacts.

The general consensus of the PFP is that it encompasses each trade occurring in the market combined with “fair price” of the listed asset. This can be split into two phases,

• Phase 1 deals with the impact of price given movement with buys and sellers, liquidity consumers and takers. This is coined the immediate price impact.
• Phase 2 consists of liquidity returns in concordance with spread and fair price. The decay of impact from these liquidity behaviors is also known as the resilience of the limit orderbook.

Fluctuations of Market Shares: A First Look at Liquidity

Levels of Market Fragmentation

• Market operators
• Trading Venues
• Orderbooks
• Orders themselves
• Technology

When price of stock $k$ is relatively constant such that $\forall l: P^k_l \approx P$, we can reformulate market share as

If we understand that fixing auctions change market share (more on this later), we take into account the monopoly the venue has over said auction. If venue $\mathcal{T_n}$ has an average market share of $m(n)$ during a fixing auction with weight $p$ of overall venue activity and average daily turnover $\mathbb{T} = \sum_{1 \leq l \leq T}P^k_lV^k_l$, then the average market share of a fixing auction is

Market Weights
When computing the market share of multiple stocks we look at weighting each stock by daily turnover. We can represent our market share equation as,

Weighting by market share of of stock $k$ according to weight $w_k$ in a given index $J$ is given by

The index is often proportional to the market capitalization of the stock and is not usually interpreted with daily turnover.

Measuring Metric Adequacy
Analyze the positioning and revenue of each trading venue.

• The market share (all included) can be important even in terms of venue.

*Measuring the probability when a trade occurs for security $k$ is on venue *$\mathcal{T}(n)$.

• Instead of summing fixing auctions with the continuous ones, use market share during the fixing auctions and ex-fixing as separate indicators.

Measuring the probability that, when a unit of currency (euro, dollar) is traded on security k, it is on trading venue $\mathcal{T}(n)$.

• The classical market share value $\mathcal{M}^{k}(n)$ is the best given we want to make one measurement during the fixing auction and another for intraday.

Appendix A.8. Estimation of Proportion and its Confidence Intervals
In microstructure we have to estimate proportions and probabilities using a binomial distribution (collection of bernoulli trials). To estimate the success the probability of success we use the empirical mean of the trials where $\hat{p} = \frac{1}{n}\sum^n_{k=1}X_k$. This unbiased estimator ($\mathbb{E}[\hat{p}]=p$) and variance $Var(\hat{p}) = \frac{p(1-p)}{n}$.

Given SLLN (Strong Law of Large Numbers) the estimation converges to $p$ as $n \to \infty$. By the central limit theorem (CLT), the empirical mean converges to a normal distribution. With a level of $1-\alpha$ we can approximate a confidence interval of $p$ such that,

where $z_{1-\alpha / 2}$ is the $1-\alpha / 2$ percentile of a standard normal distribution.

First Attempts of Fragmentation

A simple statistic to follow the fragmentation of European markets is the market share of the historical markets on each main index of each country. Given the increased competition and decreased market share, MiFID’s foray into fragmentation:

• Halted the monopolistic position of historical exchanges
• Promoted pan-European microstructure
• Created LSE Group
• Merged Chi-X and BATS

These niche markets offer two specific mixes of liquidity:

• Natural liquidity: comes directly from investors buying and selling for medium to long-term held positions.
• Opportunistic liquidity: which are offered by HFTs and market makers.

These types of liquidity allow for investors to create their own mix of liquidity and segmenting needs, proliferating the creations of mid-point dark pools, buy-side only pools, size-priority books, and random-auctions. Increasing the number of liquidity sources (like increasing the degrees of freedom) increases the complexity of allocation and cost of providing an aggregate view.

Best Execution
The European Commission does not provide a consolidated tape or route orders to be filled elsewhere for European markets. This diversity of clearing and settlement channels brings cause to potential inversion of best prices. Combined with intermediary best execution policy of routing procedures, it proliferated the creation of a two layered competition model designed to put pressure on intermediaries and venues to offer the best mix of liquidity and improving the PFP.

Trading Venue States
MiFID constrained executions on four markets:

• Regular markets (RM): host a matching engine that executes orders on a public order book. These markets are responsible for integrity of PFP.
• Multi-lateral Trading Facilities (MTF)s: transactions are matching in a engine on public orderbooks more often pegged to the state of orderbooks in RMs.
• Systematic Internalizes (SIs): executes orders based on bilateral agreements between counterparts (one of them being a market maker)
• Over the counter

Pre-trade Transparency
MiFID allowed for the disclosure of the orderbooks state to traders, also known as pre-trade transparency. This demand comes from the need for information that is disclosed to decision-makers; the more rational participant actions become from this information the more efficient the PFP will be - increasing a principle demand for pre-trade transparency.

A price can be formed by executing a liquidity demanding order against a resting order only if the state of the orderbook is disclosed to all market participants. Some exceptions exist:

• A large in scale waiver can rest in orderbooks without transparency
• Order management facility waivers exist to protect trades from information leaks
• Negotiated trade waivers that are priced within VWS
• Reference price waivers allow hidden orders of any size provided they do not form a price via confrontation of offer and demand. They can be traded at a fixed price - if this happens the transparency is delegated to the reference venue (dark pools).

Post-trade Transparency
Another obligation of the PFP process is post-trade transparency. This trade dissemination ensures investors and traders make rational choices and have the proper level of information on the last price at which the participant agreed to trade upon. But again, this transparency comes with a cost - more information increases rational decisions yet investors are worried about information leakage that shorten investment horizons and profit.

Entropy of Market Microstructure
Applying statistical thermodynamics, it’s worth noting that measuring the dispersion of matter or the level of randomness is called entropy. If the consolidated market is considered a system, “order” is all orders on one venue wile “disorder” increases with fragmentation (A.1.).

The formula for entropy is used to define the Fragmentation Efficiency Index (FEI) which models the microstructure of $N$ trading venues (assuming in relaxed state) where liquidity could find its way to any venue at the same cost. An ideal or minimal fragmentation is $1/N$.

• When only one liquidity pool is available the entropy of the system is zero
• When all liquidity is concentrated in one pool the entropy for the system is zero
• When there is equal allocation of liquidity in each pool $H=\log N$.

The FEI of a distribution of liquidity $q_1, \dots q_n$ for $N$ pools is the ratio of entropy $H(q_1, \dots q_n)$ divided by the possible entropy over $N$ pools.

The FEI measures the efficiency of the repartitions across liquidity pools compared to a uniform one. The closer to zero, the more concentrated liquidity, the more inefficient the microstructure is. When closer to one the fragmentation is efficient - a trader does not need to pay much attention to the smart routing given all sources are valued equivalently.

HFT Activity Outside of Visible Pools
The rise of HFTs in conjunction of the global decrease of European liquidity dulled the confidence of the price formation process. In light of the Flash Crash, many were hooked on the idea that HFTs earned a part of the bid-ask spread by taking profit from other participants.

Fragmentation created different pools of liquidity to host specific investors. MTFs offered second hand liquidity where historical pools offered more first hand liquidity; this, coupled with increased fix cost from fragmentation pushed demand on pools marketed as wholely first hand liquidity. This caused the birth of Broker Crossing Networks (BCNs).

Catching Up Main Indexes
The evolution of market share for 600 of the most liquid European stocks can be seen in figure 1. We see that in the mid-2009s large caps were much more fragmented - by 2012 the difference between mid and large caps is marginal.

The differences of the three segments can be explained by

1. Liquidity providing microstructure: Providing liquidity on the ask side without officially being a market making meant borrowing shares. Borrowing costs decrease with the number of shares available, hence asymmetry for the provision of liquidity that caused fragmentation among caps.
2. Unbalanced inventories: For HFTs and market makers to buy and sell correlated instruments hedging using futures usually had a smaller bid-ask spread. This meant that it was easier to sustain unbalanced inventories on large market caps given their underlying feature on future contracts.

These two reason allowed for liquidity arbitrageurs to accept inventory risk on large market caps, increasing fragmentation levels for cap levels.

Measuring Liquidity Discrepancies
The Gini-coefficient and the Kolmogorov-Smirnov test allows us to measure repatriations of the different types of actors in the market (A.9.).

The Gini coefficient is the measure of degree of inequality of income distribution in a given society. Zero represents total equality while one represents total inequality.

In statistics the Kolmogorov-Smirnov test is used to determine if a sample follows a known probability distributution given by a continuous distribution function. The empirical distribution $F_n$ for $n$ iid. observations $X_k$, $1 \leq k \leq n$ is defined as,

$\delta_{\lbrace X_k \leq x\rbrace}$ is the indicator function that equals one if $X_k \leq x$ and zero otherwise. The Kolmogorov–Smirnov statistic for a given cumulative distribution function $F(x)$ is,

Given the Glivenko–Cantelli theorem the sample comes from distribution $F(x)$ then $D_n$ converges to $0$ almost surely.

Plotting the evolution of distributions between 2008 and 2012 for market share of SBF 120 on Euronext Paris and Chi-X and their associated Gini index we note Chi-X’s increase in market share Conducting a Kolmogorov–Smirnov test checks if the market shares of the venues have the same distribution. By the given data the null hypothesis is rejected in 2008 and 2009 but accepted in 2012 and 2012.

SOR (Smart Order Routing) and the European PFP

Routing Orders in a Fragmented Market
Large orders are spliced with consideration on market impact and market risk given adverse moves in prices. Risk Aversion is used as a parameter to measure optimal trading schedules. Given market risk and impact are not linear and volume and volatility experience intraday variation, optimizing leads to non-linear trading curves.

Focusing on the best way to split the quantity of a large trade among venues will be optimal if

• It obtains a price better than or equal to the asked price.
• It finishes the order as soon as possible.

The best way allocate orders is to not consume a whole limit on a price in the main destination if volume is available in alternative venues. A SOR completes this optimization as an agnostic device - it takes a limit or market order and splits it with advanced knowledge of the state of liquidity on the destinations that are ready to trade.

Aggregating Liquidity to Minimize Impact of Order
A consequence of electronic trading is the increase of deals with current trading sizes conditioned on the size of passive orders. The average trading size of orders have decreased, signaling an increase in market fragmentation and more passive orders spread over more destinations.

Duplicate Liquidity
HFTs and market makers are simultaneously at bid and asks of a security on several destinations. They agree to take on market risk and are rewarded with the fraction of the spread - but when liquidity is posted on the same side of the orderbook on different venues, as soon as one is executed, the other orders are cancelled. This leads to “duplicate” quantities in the orderbook, undermining pre-trade transparency.

SORs dynamically split an order across venues to take into account historical performance of their executions. An order with large variance will be minimized by a rational SOR

Optimal Tick Size

Tick size defines the price grid on which a limit order must lie. Depending on the market it can be applicable to price and liquidity or dependent on price range and quotation group of the stock.

Tick sizes set the discreteness of traded prices. Larger tick sizes with a constant spread will base volatility as a function of tick size - uncertainty on the text traded price in context depends on sizing of ticks.

Measuring Spread and Volatility based on Tick Size
The Harris Model serves to understand the underlying continuous spread discretized by tick.

Some caveats are highlighted below:

• Turnover $T_j$: Determinant of spread given proxy for liquidity
• Volatility $\sigma_{j, d}$: Determinant of spread because of extra risk taken by limit order traders when providing liquidity.
• Price $P_{j, d}$: Not a determinant of relative spread because providers are indifferent about nominal price of supplied liquidity.
• $\tilde{\mathcal{f}}_{s, \theta}$: density of underlying continuous spread.
• $\psi_{j, t} = \frac{\text{Ask - Bid}_{j, t}}{\text{Tick}_d}$: This is the observed spread in ticks for stock $j$ on the $t$th trade on day $d$.

Seen in figure 3, the model is consistent with the discreteness of observed spread and will quantify the increase of expected spread from continuous to discrete distribution. The discretized distribution $\mathcal{f}_{s, \theta}$ is made up of Dirac densities (used to model spike functions).

If we reformulate the expectation of spread measured in tick size conditioned on turnover and volatility as a log linear relationship we use the maximum log likelihood to attain the maximum likelihood estimator $\hat{\lambda} = arg\max_\lambda l(\lambda)$:

For a numerical example of the harris model, we can use the 60 most liquid stocks on the FTSE, an underlying spread gamma distribution, and maximize likelihood to estimate the parameters.

The pseudo R-Squared value is 0.8.

How Tick Size Affects Market Quality

• Decreasing te tick size lowers spreads when tick size is a constraint

Tick sizes will eventually act as a minimal bound for the bid-ask spread acting as a constraint in two ways:

1. Binding probability: Percentage of trades or duration of quotes with a spread equal to one tick.
2. Spread leeway: The spread expressed as tick units minus one.

Does increasing the tick size increase the cost for a liquidity taker?

A smaller tick size will split limit orders over more price points - this causes less quoted size at each limit price. Queue jumping commences with small upticks in price to gain queue priority. Quoted sizes on the best limits are therefore much smaller with a small tick size. Furthermore, the mechanical consequences of having more price points to spread your order on is the incentive to conceal full sizes of orders. Queue jumping makes participants less inclined to show size of orders to void being front run.

Cumulative Depth and Order Exposure Incentive
Dissecting execution cost can be achieved through comparing post and pre event costs for any volume to tell which orderbook is sounder. This approach takes into account hidden liquidity.

Some other models were reviewed:

• Bacidore 2003: Showed for any given size the expected cost of a marketable order was lower after decimalization
  • Found that at orderbook at anytime with and without conditional of the occurrence of a trade. This gave insight ont he resiliency of the orderbook and its ability to replenish queues with liquidity after consumer traffic.
  • Found that liquidity was worse after tick size change for deeper limits.
• Goldstein, Kavajecz 2000: They found that the publicly available orderbook was less deep after decimalization and the cost of a virtual large trade for frequently traded stocks was greater.
  • Found that the cost for small-sized orders decreased but there was no improvement for trading a large size on a high price frequently traded stock. This gives us insight on order exposure incentive.

In summary, tick size reduction harms the liquidity by increasing costs.

Unlike European markets, US’s large tick sizes have larger spreads so that someone able to undercut a limit order with small price concessions will be able to make the market, hence earning a large spread rent. Given maker-taker fees (rebate to provider and fees on consumer) are expressed in dollar per share - not the relative trade value traded - rebates are much higher on lower priced stocks. This is a sound incentive to provide liquidity for low priced stocks which helps competition for liquidity provision with small price improvement.

Notes on Queue Jumping
With huge volume at the first limit and a spread equal to tick size, fairly fast execution will need to be in the form of a market order. Providers will have to be careful about queues given the timing for a limit buy. Small tick sizes can achieve fast execution through price improvement – shifting it’s focus from queueing problems to limit price issues.

Can We see in the Dark?

Dark pools in Europe were created as a result of MiFID to increase competition and consumer protection in investment services.

Dark pool is the generic name for an exchange that does not publicly announce available liquidity. Almost like a spin off of manual telephone trading, dark pools make the PFP much more complex.

Mechanisms of Dark Pools
The main types of dark pools are listed below:

• Crossing Networks: Pools of liquidity that cross buy and sell orders for the same stock using mid-prices derived from the primary market. WHen we talk about dark pools this is the general pool we refer to.
• Internalization Pools: Pools where a Broker internalizes its own flow.
• Electronic Market-Makers: These pools do not cross orders with other client orders but with the operators flow. Electronic market-makers immediately reject or accept orders.
• Exchange-based pools: This is dark liquidity hidden on lit pools. These orders are executed when the counterpart is larger that a certain threshold.
• Dark Aggregators: This provides to all dark pools simultaneously to create a single point of fragmented liquidity. They receive an order and split it across various pools.
• Size Discovery Pools: These are large block dark pools that host natural liquidity (from institutional investors only).

Distinct orderbooks (MiFID regulated):

• Integrated book: Accepts all visible orders and hidden orders that are “large in scale” (LIS).
• Dark or mid-point book: Accepts hidden orders smaller than the LIS metric and matches only at the mid-point of the primary best bid and offer.

Hence order routing depends on order size - LIS orders participate in PFP while smaller orders are matched at mid-point.

Dark Pool Price Discovery
Prices can not be created in dark pools and only serve as liquidity reservoirs. Transactions are made at mid-price of the primary market and do not directly (but they do indirectly) participate in PFP.

How do we understand the role of dark pools in the price formation process?

• What are the types and quality of information present in crossing networks?
• What are factors that motivate someone to choose a lit or dark pool?
• What is the relationship between dark vs. lit pool activity?

Based on Buti 2010:

• Dark pool activity is concentrated in liquid stocks.
• Activity is higher in days with high share volume, depth, and lower intraday volatility.
• Activity is lower in days with large order imbalances relative to share volume.
• Activity is associated with short-term overreaction.

Main Characteristics of Dark Liquidity
Based on Rosenblatt Securities 2012,

• Execution sizes decrease compared with preceding years.
• There is no clear relationship between market share and volatility.
• Broker-dealer pools dominate exchange-based dark pools.
• There was a modest dark pool market share growth in the US during 2012.

Reasons to trade in dark pools:

• Fear of revealing trading intentions. Institutional traders want to keep their orders secret and allow them to to be crossed against other institutions
• Enable investors to be more aggressive without paying the whole spread (trading at mid-point)

Hypothesis: liquidity is mainly driven by the appeal of earning part of the spread.

To set out to prove the hypothesis, we looked at data consisting of VWAS, volatility, and turnover traded in the dark. For a single stock the VWAS and volatility was split into ten deciles and summed over the turnover traded according to said deciles. Summing all the turnovers on all stocks, we can view the joint distribution of VWAS, dark pool turnover, and the distribution of dark turnover, conditional on VWAS.

Given figure 6 and 7, either the spread or the volatility is related to the volume traded in dark pools. The link between volume and volatility being stronger that spread and volatility implies we must moderate the conclusions we draw from a sole volume-volatility correspondence.

If we focused on dark pool market share and volatility or spread we can see volatility is not the strongest incentive for investors to use dark pools. Figure 8 illustrates this by breaking the volatility and turnover for a single stock into deciles and dividing the former over the latter, repeating the computation over all stocks of an index.

In contrast, relationships between spread are high. Indeed, the median dark pool market share is three times as high in the last decile of the spread as in the first decile - the higher the spread the easier it will be to find liquidity in dark pools.

This makes sense given the fact that when an investor really needs to trade a stock fast that has a high spread they will look at dark pools before sending a market order. This is because earning a half spread when the spread is consequential is beneficial. Dark pools are also used for reducing market impact if lit pools have lesser liquidity.

Chapter 1 notes are complete! There was quite a bit of timeline information on MiFID which I chose to omit - the curious reader can find more about that at the end of the ‘Introduction’ section. Appendix material I omitted included A.10 Simple Linear Regression Model and A.3 Natural Fragmentation of Market Microstructure.

Chapter 2 and 3 consist of understanding the stakes of fragmentation and the optimal organizations for trading. Those will be posted soon, but in the mean time feel free to take a look at my notes or the textbook.

Over winter break, I opened the kitchen fridge to see, on the middle shelf to the left, two lines of LaCroix. The left most can was a guava while the flavor besides it was key lime. I certainly knew my choice, but as I reached towards my preferred flavor (key lime of course), I heard a screech in my ear.

“Don’t pick that one!”

It was my brother who was also home for winter break. I rolled my eyes - attributing his diva-ness to the fact that he had to spend the better part of the year in wintery New Jersey hell, but he knocked my hand aside and pointed to the pink guava can, screeching in his superior holier-than-thou-tone (paraphrased because I filter out most of what he says anyways),

“You have to drink it symmetrically - you see the guava one has an extra can. You’re going to drink all the key lime and I won’t have any options to choose from!”

What in the world…drink symmetrically? Anyways, its not like you could taste the flavors in LaCroix. Maybe whatever he tasting was the byproduct of the chemical imbalance (more like illness) corroding his brain.

I made sure he sashayed away before I reached for the other can and flung myself on the couch. Cracking open the tab, I took my first sip and cheered myself to the liberation from the carbonated water police.

Even so, his slight OCD proposed a rather interesting question. How many times could I get away drinking asymmetrically without him ever noticing, ie. always having two unique drink flavors left for him? Did the count of each flavor contribute to the probability of me being yelled at? How many ways could my brother and I drink LaCroix such that there would always be more guava? Could I say the same for key lime?

And herein I introduce the topic of this post - a stupid real life application of very non-stupid combinatorics principles that involve counting valid sequences of drawing drinks. I hope you enjoy this exercise of my free will.

The LaCroix Problem

The Problem Statement

• How many valid permutations of drawing an equal number of pink (guava) or green (key lime) cans will result in the last two cans being unique flavors given we always have more pink cans, $\mathcal{P} > \mathcal{G}$?

Variations:

1. What if I loosened the inequality to $\mathcal{P} \geq \mathcal{G}$?
2. What if I drank multiple green cans per visit to the fridge?
3. What if I completely scrap the restrictive inequality such that I only care that the last two drinks are green and pink?

For a simple combinatorics exercise, let’s calculate the total number of sequences without any restrictions. Let’s say I have six green cans $(\mathcal{G})$ and six pink cans $(\mathcal{P})$. Drawing all 12 is a permutation of uniquely ordering either the six cans in $\mathcal{G}$ or $\mathcal{P}$. Therefore the total number of ways to arrange these is with a simple 12 choose 6,

To aid our journey of solving our LaCroix problem, it’s useful to be able to geometrically visualize our permutations. Without further ado, let me introduce lattice walks.

Lattice Walks

Our counting of total sequences can be visualized with a lattice walk where each step (reaching for a drink) represents either or of a flavor. This can be visualized by a 6x6 grid for our six green and six pink cans where moving horizontally represents choosing $\mathcal{P}$ while moving vertically represents choosing $\mathcal{G}$. Counting such lattice walks so we reach $(6, 6)$ (or any arbitrary cartesian point we assign to each grid intersection) is represented by how many ways we can travel right $x$ times and up $y$ times. Our linear line $y=x$ represents the balanced count between flavors; each vertex of our path touching the line symbolizes equal counts of pink and green cans for that given step.

Intuitively we know we have drank symmetrically if we reach $(5,5)$ of our lattice walk. We can observe, under a prior step, that five pink and five green cans have been consumed, leaving us with one pink and one green can left. We can further analyze these patterns, but first I need to introduce some foundational building blocks of paths.

An Introduction to Catalan Numbers

Lattice paths allow us to enter a peculiar world. Visualizations of sequences into paths start as a generalized lattice but can morph into complex and elegant representations of different path dynamics under $y=x$. These paths, which will help us efficiently count our LaCroix problem, are built upon Catalan numbers¹. But first, let me introduce you to Dyck paths and an arsenal of propositions that will help us build intuition.

Dyck Paths
A Dyck path is a lattice path that doesn’t cross the lime of $y = x$ from below. A visual representation can be seen in our drink path in figure 1.

In addition to Dyck paths I’ll introduce some propositions that you’ll convince yourself are factual; you can take my word for it or look at this heinous 113 long slide deck ².

Proof. We can easily see this by flattening the dimensionality of the problem. Imagine the sum $m+n$ as a row of slots. In each slot we have six green and six pink chips to enter in any particular order. Hence for any six green chips the complement of the other six pink chips in inevitable. Hence 12 choose 6.

Proof. Let $Q$ be the lattice path in the $(n-1)$ x $(n+1)$ grid and $P$ of the nxn grid. We can construct $Q$ as the same as $P$ up to a certain point $p$. After $p$ let us denote this path $Q_p$ and swap path $P$ directions for every step for $Q_p$’s path. We can see that $Q$ has $n-1$ east steps and $n+1$ north steps. For $P$ there must be n east and n north steps, hence, there must be exactly one more east step than north steps after $p$. This intuition is classically called the Andre Reflection Method³. More on this later.

This proposition allows us to reduce the complexity of counting ‘bad’ paths that cross $y=x$ by mapping unique paths on the nxn grid to unique paths on the skewed grid, creating a one-to-one correspondence.

Now we’re ready to dive into Catalan numbers.

Other forms include:

Proof. $C_n$ counts the number of lattice paths that do not cross $y=x$ on the nxn grid. From our previous exploration we know there are $\binom{2n}{n}$ total paths with the number of’bad’ paths in the nxn grid bijecting onto the skewed $(n-1)$x$(n+1)$ grid. We therefore can measure ‘bad’ paths as the total amount of paths in the $(n-1)$x$(n+1)$ grid.

Hence $C_n =$ {number of total paths} - {number of bad paths}, reaching our decomposition of $C_n$. In summary we combined the principles of counting symmetry and the skewed grid bijection to decompose Catalan numbers as the subtraction of the total number paths with the total number of ‘bad’ paths.

Combining It All Together
In many ways a Dyck sequence is a geometrical representation of balance, like a surface of a pond. A fish, water bug, or duck that does not disrupt the well-formed waterline is a representation of our one-to-one correspondence, called a bijection, to each other. Their path is the same, but its meaning changes depending on the lens you use. A fish’s underwater trail can parallel a string of kelp’s sway to the current, but they serve as mathematical refractions caused by the watery environment they live in, their form as Dyck paths casting long shadows onto the pond floor.

Catalan numbers are the representation of the pond depths. They count of all valid paths our pond inhabitants and the refractions of these paths, tallying each specular reflection to hold a single balanced truth on top of its surface.

Objects Counted by Catalan Numbers

With this obscure and seemingly tepid equation we can do some real magic. Catalan numbers can count many objects, from chords to arcs to binary trees¹, but we’ll shift our focus onto two that are seemingly divergent in their relation but provide us with a way to bring this back full circle. To do this I’ll need to introduce some more propositions and definitions.

Firstly, we’ll look at a Ballot sequence.

Proof. First we convert the Ballot sequence to a lattice path where 1’s are green and -1’s are pink. Given that we have a nonnegative sequence, this means we have more or equal to the number of horizontal moves that vertical in an given path subset. Hence we never cross $y=x$, essentially mapping the Ballot sequence to a Dyck path.

To chain it together:

• Dyck paths are paths that do not cross $y=x$
• Catalan paths are composed of up of Dyck paths
• Ballot sequences are sequences where 1 is never behind (in count) of -1

We can show that A = B by bijection, and B = C by bijection, then A = C. Hence we can show Ballot sequences are counted by Catalan numbers.

Parenthesization
Now we can introduce parenthesization, a way for us to model hierarchies and nested structures to help us count more efficiently that are also counted by Catalan numbers ⁴.

An example of this can be seen with the term ‘abcd’. With $n=4$ we have 3 sets of parentheses that allow us to subset the statement like,

It all connects back I promise! Because these parentheses are - as you probably guessed - Dyck paths where each right and left parentheses corresponds to a horizontal movement and a letter a vertical one. A example can be seen in the statement ‘abcdef’ mapped onto a 6x6 grid with the parenthesization of $(a((bc)d))(ef)$,

A invalid parenthesization corresponds to a Dyck path that crosses $y=x$ by having more green vs pink moves (more vertical moves); in translation this means that the parenthesization has more opening parenthesis than letters. This is nonsensical.

The Ballot Theorem

We’re almost at the home stretch! Our LaCroix problem, if you even remember, is a variation of the Ballot problem which combines lattice walks, Catalan numbers, and skewed grid bijections for a magus opus of combinatorics and logic. The diversity and depth of proofs for this theorem are really quite beautiful³,⁵, but for our instances we’ll focus on the Andre-Reflection Method.

The Ballot Problem
In 1887 Joseph Bertrand introduced the Ballot problem where candidates $(\mathcal{A})$ and $(\mathcal{B})$ are in election. $(\mathcal{A})$ receives $a$ total votes and $(\mathcal{B})$ receives $b$ total votes with the caveat $\mathcal{A} > \mathcal{B}$ at all times. How many ballot permutations could be made?

Desire Andre produced a combinatorics proof for $k=1$ downturns (under the generalized Ballot theorem problem) by counting the number of bad ballot permutations and subtracting that from the total amount of ballot permutations (not unlike of composition of Catalan numbers)⁶. Andre understood that unfavorable permutations could be stratified into permutations starting with the starting vote as either $(\mathcal{B})$ or $(\mathcal{A})$, and for permutations starting with $(\mathcal{B})$, a bijection existed with all permutations of $b-1$ ballots $(\mathcal{B})$ and $a$ ballots of $(\mathcal{A})$. The same was understood for permutations starting with $(\mathcal{A})$.

While Andre’s proof has no geometric reasoning, we’ll use a combination of both Andre’s symmetry logic to sketch out the Andre-Reflection Method.

The Andre-Reflection Method

Think back to Proposition 1. where we state that the lattice paths in a nxn grid that cross the line $y=x$ are in bijection with the lattice paths in the $(n-1)$ x $(n+1)$ grid. In essence we’re mapping a bad path (one where $b > a$, crossing $b = a$) to a good path onto a different grid that semantically still represent an unbalanced outcome.

By reflecting the bad Ballot sequence at point $p$ where it crosses $b=a$, we rebalance upturns and downturns to make the path valid again. We can bijectively map the path to another with $a-1$ x $b+1$ grid for a total of $a+b$ steps. Using this reflection method we don’t count the paths directly - instead we allow ourselves to take advantage of this peculiar correspondence to count paths in the skewed grid. For bad permutations that start with $(\mathcal{A})$ we must subtract one from $a$ such given this step is not reflected. For all bad paths this gives us:

• Starting with $\mathcal{A}$: $(a-1) + b = a + b - 1$
• Starting with $\mathcal{B}$: $a + (b-1) = a + b - 1$

Hence the bad paths with different starts are symmetric; starting with either candidate essentially leads to the same problem given symmetry, so by subtracting the bad paths with a multiplier of 2 (for $\mathcal{A}$ and $\mathcal{B}$ starts) we achieve the solution to the Ballot problem.

Back to the LaCroix Problem

Finally! Let’s answer our problem statement and variants.

The Problem Statement

• How many valid permutations of drawing a equal number of pink (guava) or green (key lime) cans will result in the last two cans being unique flavors given we always have more pink cans $\mathcal{P} > \mathcal{G}$?

Our original problem statement hinges on the condition of always having two unique drink flavors left from the original six. Given $\mathcal{P} > \mathcal{G}$, we can directly apply the generalized Ballot theorem where the number permutations is, assuming I only drink one at a given instance,

Catalan Numbers
Weird, but this actually makes a lot of sense. If we want two unique flavors for the last two cans we need the previous draw to balance the count ie. reach $(5, 5)$ on our grid. This isn’t possible with our inequality constraint of $\mathcal{P} > \mathcal{G}$, hence we have 0 valid permutations.

If we had a mismatched number of cans, lets say six pink and 5 green, then the number of drink permutations is given as,

Variation 1

• What if I loosened the inequality to $\mathcal{P} \geq \mathcal{G}$?

Assuming we have the same number of cans for each flavor, let’s say six, then the loosened inequality lets us touch but not cross the line $\mathcal{G} = \mathcal{P}$ from under. Therefore counting valid paths is just a process of counting all Dyck paths on the grid. Hence we can use a Catalan number and our number of drinking permutations is,

The Weak Ballot Theorem
If we had different numbers of cans per flavor our approach is different such that we use the Weak Ballot theorem. Any ballot permutation where $a$ maintains at least $b$ can be converted to the traditional Ballot Theorem where $\mathcal{A} > \mathcal{B}$ by simply appending a vote for $\mathcal{A}$ at the beginning of the permutation⁵. Therefore the weak Ballot theorem is the same as the “strict” version when $\mathcal{A}$ receives $a+1$ votes and $\mathcal{B}$ receives $b$ votes.

If we have six pink cans and five green cans given $\mathcal{P} \geq \mathcal{G}$, our number of valid permutations becomes,

A Map from Strong to Weak
This sorcery would surely resurrect another witch burning! But just as it seems, the reason why the Ballot theorem has such a nice closed form solution is because of its inherent bijection with constrained lattice paths.

By adding the additional $\mathcal{A}$ vote we added a buffer such that:

• Every strict path from $(0,0) \rightarrow (a+1, b)$ with $\mathcal{A} > \mathcal{B}$ starts with A being ahead
• If we ignore the first path step of $\mathcal{A}$, the path becomes $(0,0) \rightarrow (a, b)$ for $\mathcal{A} \geq \mathcal{B}$

In other words each unique valid weak path is mapped to a unique strong path by just the addition of an $\mathcal{A}$ vote. We can therefore, not unlike how we used our skewed grid in proposition 1, utilize this bijection and make our counting lives easier.

Catalan Numbers Revisited
Why does our number of drink permutations for unequal counts among flavors match the instance when we have the same number of drinks for each flavor counted by $C_6$? It’s a coincidence in value, but not in structure. If we let $a=b=n$ for $\mathcal{A} \geq \mathcal{B}$ we also find that our Weak Ballot theorem is generalized by a Catalan number,

While this observation seems redundant, this gives us an algebraic understanding of the Weak Ballot theorem and further strengthens our intuition of Dyck paths and Catalan numbers.

Variation 2

• What if I drank multiple green cans per visit to the fridge?

Generalized Ballot Theorem
With our initial condition of $\mathcal{P} > \mathcal{G}$, multiple green drinks taken out of the fridge for each visit is a downturn step that has a greater magnitude that an upturn. Hence, we can reformulate our Ballot theorem for some relative size $k$ such that $\mathcal{A} > k\mathcal{B}$. This is called the generalized Ballot theorem⁵and introduces new caveats with lead margins and rotated lattice walks.

For this theorem we hinge our linear line of $B=A$ on the grid such that our x-axis is our edict for a good or bad path. For example if a path dips below the x-axis then it is considered bad, while a path that stays above the x-axis is good. A downturn that starts above the x-axis and ends below it (negative slope) is a bad step.

Hence for each candidate step we have,

• $\mathcal{A}$ upturn: $(1, 1)$
• $\mathcal{B}$ downturn: $(1, -k)$

Let $B_0, \dots, B_k$ represent the number of bad paths where $B_i$ is a path that lands exactly $i$ units below the x-axis. These sets are disjoint and their union $B_0 \cup \dots \cup B_k$ is the collection of all bad paths. $B_k$ is all paths that start with a downturn where the first step lands $k$ units below the x-axis. Therefore, if we let the x-axis denote $\mathcal{A}$ and the y-axis $\mathcal{B}$, we can see that the number of $B_k$ paths is the total $a+b$ subtracted by the first initial step, hence $a+b-1$. It follows that $B_k$ is,

Interestingly enough we can show for $k \neq i$ that

To prove this let $P$ be the path in $B_i$ where $k \neq i$ and $X$ the initial segment of P that ends $i$ units below the x-axis. Let path $P=XY$ where $Y$ is the rest of the path not including $X$. Let $\tilde{X}$ represent the path that results from rotating $X$ 180 degrees. Given $X$ ends with a downturn $\tilde{X}$ starts with a downturn.

Our new rotated path of $\tilde{X}Y \in B_k$. We can reverse this process from $B_k$ to $B_i$ but in any case, we model a bijection where

Given there are $k+1$ such bad path sets given $0 \leq i \leq k$ we have

Subtracting good paths from bad paths we finally achieve

Applied to our problem statement let us assume we take two green cans ($k=2$) at a given time given 12 pink and five green cans. Our total number of drinking permutations such that $\mathcal{P} > k\mathcal{G}$ is,

Uniform Partitioning
It can be shown that

The same cardinality of the sets of $B_i$ regardless of the value of $i$ is a example of uniform partitioning. Instead of counting all paths separately we can group each path by definitive characteristics (like downturn location on the x-axis) and construct a bijection. This principle is coined the Chung-Feller theorem but was originally introduced by MacMahon in 1909⁷.

Generalized Catalan Numbers
If $a = kb$ we can see the generalized form of Catalan numbers called k-Catalan numbers,

Variation 3

• What if I completely scrap the restrictive inequality such that I only care that the last two drinks are green and pink?

This problem lets us step away from the Ballot theorem to look at some good old lattice walks. If we assume equal number of cans per flavor, let’s say six, then we know that we must reach coordinate $(5, 5)$. We are restricted by the the $(5,5)$ grid where we can take five steps up (green can) or 5 steps horizontally (pink can). It’s a simple scenario of how many ways we can order the five steps vertically or horizontally. Hence the number of valid drinking permutations is given as,

Even if our number of drinks per flavor differ we can still use our strategy. If we have six pink and five green cans we must reach the coordinates $(5, 4)$. Therefore our valid drinking paths can be represented as,

Conclusions

Well well well… what a great exercise of free will. Trust humans to make something as simple as counting so pedantic.

Safe to say this post had me feeling like:

Next steps? Maybe I’ll dabble in chords or triangulations - those seem stupidly fun. Heck I might even take a look into knot theory; it’ll be fun to tell my friends I’m studying something that’ll acquire me a girl scout badge.

This algorithm exists in the field of reinforcement learning and implements exploration-exploitation tradeoffs under different parameters compared to the ETC algorithm.

With two or more players, information asymmetry and, by extension, reward asymmetry occurs. We’ll be looking at how we tackle such imbalances and how we can still achieve desired sets.

Prerequisite Posts (if you care to follow along):

• Beware of Bandits!

Source Material:

• Optimal Cooperative Multiplayer Learning Bandits with Noisy Rewards and No Communication
• Bandit Algorithms

Review of Regret and Reward

Although we looked at regret and rewards in Beware of Bandits, I’ll reiterate some terminology that the paper has in different notation.

Joint Action Consider a set of $M$ players $P_1, \dots P_n$ in which player $P_i$ has a set of $\mathcal{K}$ of $K_i$ arms to pick from. At each round, a player can pick an arm independently and simultaneously from other players from their set of $\mathcal{K}_i$. This joint action can be represented as a $M$-tuple of arms picked and denoted by $a = (a_1\dots a_M)$¹. This generates $M$ iid. random rewards $X_a^i \in$[0,1] with $i \in$ [M] from a 1-subgaussian distribtuion $F_a$ with mean $\mu_a$.

Reward Asymmetry The idea that each player $P_i$ can only observe their rewards $X_i^a$ but are oblivious to all the other rewards is called reward asymmetry. Only after joint action is taken can they observe the actions of players. All players can determine a strategy and know the joint action space beforehand. However, during the learning process, they are not allowed to communicate.

Regret The optimal arm among all arm tuples is denoted as $\Delta _a = \mu*-\mu_a$ where $\mu*$ is the highest reward mean. The goal of these players is to pull the optimal arm as often as possible.

Let $a_t(i) \in (1, \dots K_i)$ be the arm chosen by player $i$ at time $t$ and deonte $a_t = (a_t(1), \dots, a_t(m))$. For the players to pull the optimal arm, the high-level strategy is to collectively identify the best set of arms $a$ corresponding to mean reward $\mu*$ given that they don’t know the distribution or the means $\mu_a$. To do this, they must learn by playing and exploring. The learning efficiency can be represented as expected regret,

$T$ is the number of learning instances and $X_{a(t)} (t)$ is the random reward if $a(t)$ is pulled. We let $\eta_a(t)$ be the number of times joint arm $a$ has been pulled up to but not including time $t$. Let $\hat{\mu_a^i}(\eta_a(t))$ to be the empirical mean of arm $a$ for player $i$ at time $t$ after $\eta_a(t)$ pulls the joint arm $a$. This empircal mean is indexed by the player since each get their own iid reward. In contrast, the regret $R_T$ is the same for each player because the reward for each joint arm has the same mean across all players ². $\eta_a$ is not indexed by the player $i$ becuase each player contributes to the same joint action $a$ every round.

While each player will likely have different rewards, the rewards are iid and the regret will therefore be the same across all players. The paper uses the the suggested lower regret bound $O(logT)$ for the multi-player decentralized learning algorithm with the same regret order.

The left matrix is the joint action space of player a and b with each cell corresponding to a subgaussian reward distribution for some mean. The table on the right lists different types of information asymmetry. In this post, we focus on Reward Asymmetry in grey.

Main Results of mUCB-Intervals

The UCB Index

To maximize the cumulative rewards we define the *UCB index, also known as the Upper Confidence Bound for each joint action. The player’s objective is to pull the largest UCB index as often as possible. For each player $i$ we can define the UCB index as,

Let $\epsilon_a(\dots)$ be the constant added to the empirical mean for arm $a$ in calculating the UCB index. This constant is the same for every player and therefore not indexed by $i$. Our algorithm introduced further in the post will be of length $2\epsilon_a(\dots)$. Because of this we add a tuning parameter $\gamma$ such that,

Using Hoeffding’s bound for subgaussian variables³, the true mean $\mu_a$ is within the interval

with high probability. We can use this confidence interval to create a set of arms that are likely to be optimal and to make this selection from this set. While this interval is different for each player $i$ given some joint action $a$ due to the empirical means $\hat{\mu_a^i}$ being different, $\epsilon_a)$ is not indexed by $i$.

Reward Asymmetry

To deal with information asymmetry, all players will maintain a desired set that contains the joint arms that are candidates for the optimal arm. Initially, $K^m$ joint actions are in this desired set; this desirability is predetermined by an arbitrary ordering.

A joint action is considered if it is the arm in the desired set that supposed to be pulled given the order that was agreed by all players. We denote this joint action in the desired action as $c$. If there is $l$ joint actions $c_1, \dots, c_l$ then the ordering of the desired set can be viewed by the following flow chart as,

A joint arm is eliminated from the set if it does not fall within the UCB interval and is disjoint from another arm. Player $i$ effectively “communicates” this elimination to the other players by not pulling $c_t[i]$; the other players will also eliminate this arm from their desired set while maintaining order. Pulling a joint arm is not considered the same as removing a considered arm from a set.

The net flow can be visualized as,

Run-through Example

Consider a two-player setting where each player has two arms. We will represent each joint as as a matrix where each cell is the UCB interval defined in equation 4 (general visualization of the player environment can be seen in figure 1).

If the first and second row and column correspond to player 1 and player 2 respectively,

Suppose the order of the joint action is $(1, 1), (1, 2), (2, 1), (2,2)$. Initially, all UCB intervals are infinite as each arm gets pulled at least once. Then after the joint action flow where $t = 4$, the matrices become,

For round $t=5$, we consider $(1,1)$ again. But because the UCB interval is disjoint some arm, that being (2,1), player 1 knows with high probability that this arm is not optimal. To communicate this to player 2, player 1 will instead choose arm 2. Player 2 does not know player 1’s actions during the round, so it still considers (1,1); player 2 will choose 1. The joint action of these two players will be $(2, 1)$.

After choosing these arms, player 2 sees that arm 1 is suboptimal and now understands $(1,1)$ should be eliminated. The flow of the desired set now becomes $(1, 2), (2, 1), (2,2)$.

For round $t=6$, the next considered arm is $(1, 2)$ and the process repeats until we are left with the most optimal arm with the highest probability. In this case, this is $(2, 2)$.

Regret Bound Analysis

Theorem 1 For any choices of tuning parameter $\gamma > 0$, the gap dependent of algorithm mUCB-Intervals is

Theorem 2 For any choices of $gamma > 0$ the gap-independent regret bound of algorithm 1 is

Lemma 4 ith regret defined by equation $1$< the regret decomposition for player $i$ follows,

where $\eta_a(T)$ is the number of times the arm $a$ has been pulled up to round $T$.

Lemma 5 Under the event $G \cap G^{cross}_a$, the number of pulls of arm $a$ is at most $u_a$.⁴

The paper introduces a cooperative multiplayer bandit learning problem where players are only allowed to agree on a strategy beforehand but can not communicate during the learning process. Each player receives a noisy version of the reward that cannot be shared with other players. mUCB-Intervals based on upper and lower confidence bounds scaled by $\gamma$ can be used to select optimal actions despite asymmetry. This asymmetry can be ultimately removed and be used to derive an algorithm that gives asymptotically optimal regret.

Now that we understand the basics of bandits, we can start exploring some basic algorithms that utilize the idea of regret and reward! The algorithm we’ll be exploring is called the Explore-then-Commit algorithm (ETC). ETC is split into two phases: exploring and committing by playing each arm a fixed number of times and committing to that arm for the rest of the rounds.

Note the prerequisite for this blog post:

• Beware of Bandits!

Source material:

• Bandit Algorithms

The Algorithm

The Cramer-Chernoff Method and Subgaussian Random Variables

Before we can jump into the algorithm, we need to brush up on some assumptions and distributions. For the rest of this post, we assume that all bandit instances are $\xi_{SG}^k(1)$ where the reward distribution for all arms is 1-subgaussian.

Definition of Subgaussianity A random varibale $X$ is $\sigma$-subgaussian if for all $\lambda \in \mathbb{R}$ it holds that $\mathbb{E}[ exp(\lambda X)] \leq exp(\lambda^2\sigma^2/2)$.

The sub-gaussian distribution has strong tail decay and can also be characterized by its variance proxy¹,

This property bounds the tails of the distribution implying that the random variable $X$ does not produce extreme outliers too frequently. For 1-subgaussian random variables, and in this context our reward distribution, this inequality holds for $\sigma = 1$.

Let’s also define an important theorem that allows us to use the Cramer-Chernoff method.

Theorem 2.1

If $X$ is $\sigma$-subgaussian, then for any $\epsilon \geq 0$,

Proof. Using the Cramer-Chernoff method, let $\lambda > 0$ to be some constant later. Then,

We can choose $\lambda = \epsilon// \sigma^2$ to complete the inequality and proof.

All this notation and definition ultimately boils down to analyzing the bounds of the tails of $\hat{\mu} - \mu$. The last corollary for this section introduces two inequalities that are important when analyzing ETC.

Corollary 5.5 Assume that $X_i - \mu$ are independent, $\sigma$-subgaussian random variables. Then for any $\epsilon \geq 0$,

where $\hat{\mu} = \frac{1}{n}\sum_{t=1}^{n}X_t$.

The ETC Policy

Ok let’s get back to the algorithm. ETC is characterized by the number of times it explores each arm, denoted $m$. Because there are $k$ actions, the total number of times ETC explores is $mk$ rounds until committing. The average reward is denoted as $\hat{\mu}_i(t)$ such that,

The term $T_i(t) = \sum_{s=1}^{t}T{A_s=i}$ is the number of times action $i$ has been played after round $t$ where ties are broken arbitrarily. The policy is given as:

1. Input $m$
2. In round $t$ choose action

With this we can bound our regret given our suboptimality gap $\Delta_i = \mu* - \mu_i$ and mean reward $\mu_i$².

Theorem 2.2 When ETC is interacting with any 1-subguassian bandit and $1\leq m \leq n{}k$,

Proof. The proof is quite interesting but long so buckle up! W.l.o.g. that the first arm is optimal. This means $\mu_1 = \mu*=max_i\mu_i$. By decomposing Lemma 1.4 $R_n= \sum_{a \in \mathcal{A}} \Delta _a \mathbb{E}{[}T_a(n){]}$ we have.

In the first $mk$ rounds the policy is deterministic (ie produces the same action each time because our first arm is optimal). Therefore, the single action chosen maximizes the average reward during the exploration phase,

The probability on the right side is bounded by,

By theorem 2.1, we can use the properties of the subgaussian random variable to construct the inequality,

\(\tag{2.3} \mathbb{P}(\hat{\mu_i}(mk) - \mu_i - (\hat{mu_1}(mk) - \mu_1 \geq \Delta_i) \leq exp(\frac{-m\Delta_i^2}{4})\).

Substitute $2.3$ into $2.1$ $R_n$ and $2.2$ $\mathbb{E}[T_i(n)]$ for the result.

Theorem $2.2$ exhibits the trade-off between exploration and exploitation. If $m$ is large then the policy explores for too long and the first term will be too large and we have some semblance of linear regret. If $m$ is too small, then the probability that the algorithm commits to the wrong arm will grow and the second term becomes large.

So how do we choose $m$?

Assume that $k=2$ (we have two actions) and the first arm is optimal such that $\Delta_1 = 0$, we can abbreviate $\Delta= \Delta_2$. The bound given in 2.1 can be simplified to,

We can show that for a large $n$ the right-hand side of $2.4$ is minimized up to a possible rounding error³ - for this choice and any $n$ the regret is bounded by

where $C > 0$ is a universal constant and when $\Delta \leq 1$ it is often assumed. We also have another bound,

This bound is called the worst-case, problem-free, or problem-independent bound because it only depends on the horizon and class of bandits for which the algorithm is designed, not a specific instance of the class. Without $\Delta \leq 1$ the worst case bound for ETC is $\infty$.

The Bound in Action

The figure above shoes the expected regret of ETC when playing a Gaussian Bandit with $k=2$ (two actions) for means $\mu_1 = 0$ and $\mu_2 = -\Delta$. The horizon is set to $n = 1000$ and the suboptimality gap $\Delta \in [0,1]$. Each data point is the average of $10^5$ simulations (that’s why there are no error bars). We can visually see that our constructed bound in Theorem $2.1$ is quite close.

This is the simplest and what my mentor calls the most “stupid” bandit algorithm but it’s still good to review for my own sake. In some ways, this parallels any betting game: when do we access the trade-off to commit to a strategy versus explore different strategies? What’s our EV (or in this case our constructed bound) for finding the optimal reward and sticking to such reward? Interesting thoughts I might explore later.

Much of what I synthesize comes from the holy grail text some affectionately coin as the Bandit Bible. Written by Tor Lattimore and Csaba Szepesvari, this textbook is widely regarded as as a ground up manual and gateway into bandits research.

An old bandit adage: A bell is a cup until it is struck.

Colin Meloy

Core Assumptions

A different kind of "Bandit"

Definition: Stochastic Bandits

A stochastic bandit is a collection of distributions $\mathcal{v} = (P_a:a \in \mathcal{A})$, where $\mathcal{A}$ is the set of available actions. In this environment, we have two players: the learner and the environment. In this ecosystem the following actions and states exist:

• In an environment there are $t$ rounds such that $t \in { 1, \dots, n }$.
  • The horizon n, or total encompassing rounds, is finite (sometimes we allow $n=\infty$).
• On each round a learner chooses an action $A_t \in \mathcal{A}$.
• The environment then samples a reward $X_t \in \mathbb{R}$ from distribution $P_{A_t}$ and reveals the reward $X_t$ to the learner.
• This interaction between learner and environment induces a probability measure on a sequence of outcomes $A_1, X_1, \dots, A_n, X_n$.

We also impose some assumptions on our environment:

• The conditional distribution of reward $X_t$ given our sequence of outcomes is $P_{A_t}$. Intuitively this makes sense as we sample our reward from $P_{A_t}$ in some round $t$.

• On action $A_t$ the player chooses from $\mathbb{R}$ is $\pi_t ( \cdot | A_1, X_1, \dots, A_{t-1}, X_{t-1})$ ¹. $\pi_t$ where $t \in { 1, \dots, n }$ is the sequence of probability kernels that characterize the learner.

$\pi_t$ is essentially the function that provides the probability distribution over $A_t$ given some past action sequence $A_1, X_1, \dots, A_{t-1}, X_{t-1}$. Think of $\pi_t$ as the describing function that characterizes how the learner decides on an action at round $t$ based on historical rounds.

The Learning Objective

Like any good self-optimizing student, the goal of the learner is to maximize the reward $X_t$ across the horizon $n$. This is a random quantity that depends on the interaction between the environment and player and interestingly enough is NOT an optimization problem. Why?

1. Maximizing rewards is a process built on incomplete information. We don’t know the value of $n$ rounds we will need.
2. Our reward value itself is random.
3. Our distributions that govern each reward is unknown.

So basically we know nothing…right? Well like everything in math, we can scoot some things around to adapt to this lack of information. To do this, we construct a policy that assumes this loss of information and proves that this loss is minimal anyway.

But to assign utility² to our distirbutions of $S_n = \sum _{t=1}^{n} X_t$ is harder. By convention, we choose the largest expected value of the reward distributions³.

The Regret

No Regrets!

Unlike the tattoo artist in the above Snickers commercial, bandit’s have regrets! Many of them to be precise. Regret is a central principle of bandits and is how we measure the performance of bandit algorithms. In the most basic terms regret is the deficit suffered by the learner relative to the optimal policy. If, by earlier definitions, $\mathcal{v} = (P_a: a \in \mathcal{A})$ is a stochastic bandit, then we define our expected or mean reward⁴ given $P_a(x) as

The regret of policy $\pi$ on bandit instance $\mathcal{v}$ is

Therefore, the regret after $n$ time steps over the policy $\pi$ and bandit instance $\mathcal{v}$ is:

• $\mu(\mathcal{v})$: the *optimal reward term
• $\mathbb{E}[\sum_{t=1}^{n}X_t]$: the total expected reward

Lemma 1.3

• $R_n(\pi, \mathcal{v}) \geq 0$ for all policies $\pi$
• The policy $\pi$ choosing $A_t \in argmax_a\mu_a$ for all $t$ satisfies $R_n(\pi, \mathcal{v}) = 0$
• If $R_n(\pi, \mathcal{v}) = 0$ for some $\pi$ then $\mathbb{P}(\mu_{A_t} = \mu*) = 1 \forall t \in [n]$

Decomposing the Regret

Now let’s introduce the suboptimality gap, also known as the action gap or immediate regret of action $a$,

We interpret $T_a(t)$ as the number of times action $a$ is chosen by the learner after the end of round $t$. $A_t$ inherits randomness from past observational randomness.

Now let’s look at the dependence of the various quantities of policy $\pi$ and the environment $\mathcal{v}$ is suppressed.

Lemma 1.4 For any policy $\pi$ and stochastic bandit environment $\mathcal{v}$ with countable $\mathcal{A}$ and horizon $n \in \mathbb{N}$, the regret $R_n$ of policy $\pi$ in $\mathcal{v}$ satisfies

This decomposition decomposes regret in terms of loss by using each of the arms. This tells us that to keep the regret small the learner should aim to use an arm with a larger suboptimality gap fewer times (sorta like “learning from its mistakes”). For the most optimal arm, suboptimality gaps are zero.

Now you know the bare bones of stochastic bandits! From how we formally define bandits to the introduction of decomposing regret, we can now explore more complex and accurate algorithms and their bounds as well as create new bounds with different environments and bandit parameters.