Danial Dervovic

Branch and Bound (and Cut), Visualised

Mon, 22 Jun 2026 12:00:00 +0000

Following on from my blog post The Simplex Algorithm Visualised I thought it was time to look at integer linear programs. Namely, what happens when we take a standard LP and demand that the solution have integer-valued variables? It turns out there are several basic approaches that make sense to use in different contexts; specifically branches and cuts. In the animation we see examples where each approach makes sense - it is nice to get a visceral image of why. Have fun playing around with the problem inputs.

As before, animation and write-up below generated by Claude.

Companion to the Interactive Visualiser

This document walks through every concept behind the visualiser: how the LP relaxation gives a bound, how branching carves up the feasible region, why pruning is what makes the search tractable, what cutting planes add, and the trade-offs between the three methods — with attention to the geometry and the edge cases where things get interesting.

Throughout, the running example is the visualiser’s Textbook preset:

\[\max \; 3x_1 + 2x_2 \quad \text{s.t.} \quad 3x_1 + 4x_2 \le 16, \;\; 4x_1 + x_2 \le 12, \;\; x_1, x_2 \ge 0, \;\; x_1, x_2 \in \mathbb{Z}.\]

Its integer optimum is $(2,2)$ — a point that sits strictly inside the polyhedron, touching neither constraint, which makes it a good illustration of how an integer optimum differs from the fractional LP vertex.

1. Why Integer Programming Is Hard

A linear program (LP) optimises a linear objective over a polyhedron, and simplex solves it efficiently by walking the vertices. An integer program (IP) adds one innocent-looking requirement — the variables must be integers:

\[\max \; \mathbf{c}^\top \mathbf{x} \quad \text{s.t.} \quad A\mathbf{x} \le \mathbf{b}, \;\; \mathbf{x} \ge \mathbf{0}, \;\; \mathbf{x} \in \mathbb{Z}^n.\]

We write $z = \mathbf{c}^\top \mathbf{x}$ for the objective value — the quantity being maximised (in the Textbook example, $z = 3x_1 + 2x_2$). It is the number reported at each node and tracked in the status strip.

That single change moves the problem from “polynomial-time solvable” to NP-hard. The feasible set is no longer a convex polyhedron but a scatter of lattice points inside it, and the optimum can sit deep in the interior of the polyhedron, at no vertex simplex would ever visit.

The tempting shortcut — solve the LP and round — is unreliable. In the Textbook example the relaxation optimum is $(2.46,\,2.15)$. The objective rewards a larger $x_1$, so the natural rounding is $(3,2)$ — but that is infeasible, since $3\cdot 3 + 4\cdot 2 = 17 > 16$. In fact three of the four ways to round $(2.46,\,2.15)$ leave the feasible region; the survivor $(2,2)$ happens to be optimal here, but you only learn that by checking, and with $n$ variables there are $2^n$ candidate roundings that may all be infeasible or all suboptimal. Rounding is a guess, not an algorithm.

We need a method that searches the lattice points intelligently. That is branch and bound.

2. The LP Relaxation and the Bound

Drop the integrality requirement and you are left with the LP relaxation:

\[\max \; \mathbf{c}^\top \mathbf{x} \quad \text{s.t.} \quad A\mathbf{x} \le \mathbf{b}, \;\; \mathbf{x} \ge \mathbf{0}.\]

Because the relaxation optimises over a superset of the integer-feasible points, its optimal value is an optimistic bound on the IP: for a maximisation problem,

\[z_{\text{IP}}^* \;\le\; z_{\text{LP}}^*.\]

This is the single most important fact here. Every node of the search will solve a relaxation, and the bound it returns tells us the best we could possibly hope for in that part of the search — which is exactly what lets us discard parts of the search without exploring them.

In the visualiser, the relaxation is solved (by the very simplex routine from the previous post) at every node; the result is the diamond marker in the geometry pane — hollow when the relaxation optimum is fractional, filled gold when it happens to be integer.

3. The Search Tree

Branch and bound organises the search as a tree of subproblems. The root is the original problem. Each node is the original IP plus some extra bound constraints accumulated on the way down from the root (things like $x_1 \le 3$ or $x_2 \ge 2$). Every node owns its own LP relaxation.

The tree is, in principle, astronomically large — but we never build all of it. The art is to grow only the branches that could contain the optimum and to cut off the rest as early as possible. The left pane of the visualiser draws this tree as it is discovered: each circle is a node labelled with its LP bound, and the edges are labelled with the branching constraint that created the child.

A node is open (or active) once its relaxation has been solved but it has not yet been dealt with — it is waiting on the frontier to be processed. It becomes closed once we have finished with it, in one of two ways: it can be branched (split into two children, which become open in its place), or it can become a leaf by being pruned. The algorithm keeps a pool of open nodes, repeatedly picks one to process, and terminates when the pool is empty — every node closed. There are three distinct reasons a node gets pruned, which we come to in §5; the “global bound” in the status strip is computed over exactly the open nodes.

4. Branching: Slicing Out the Fractional Gap

Suppose a node’s relaxation optimum has a fractional component, say $x_j = f \notin \mathbb{Z}$. We pick that variable and branch: create two child subproblems,

\[x_j \le \lfloor f \rfloor \qquad \text{and} \qquad x_j \ge \lceil f \rceil.\]

The key observation is that no integer-feasible point is lost. Any integer solution has $x_j$ either $\le \lfloor f \rfloor$ or $\ge \lceil f \rceil$; the open strip $\lfloor f \rfloor < x_j < \lceil f \rceil$ contains no integers at all. We have thrown away a slab of the polyhedron that held only fractional points — including the current fractional optimum, which is now excluded from both children, forcing progress.

In the Textbook example the root optimum is $(2.46,\,2.15)$, so we branch on $x_1$ (the more fractional coordinate): the strip $2 < x_1 < 3$ is discarded and we get the children $x_1 \le 2$ and $x_1 \ge 3$. Watch the geometry pane redraw the shaded feasible region for each child as you step into it — the branch constraint appears as a new wall, and the region shrinks.

5. Bounding and Pruning

Branching alone would just enumerate everything. Pruning is what makes branch and bound efficient: it discards subtrees we can prove are not worth exploring. A node is pruned for one of three reasons.

Prune by bound. If the node’s LP bound is no better than the best integer solution found so far (the incumbent), nothing in this subtree can beat what we already have, so we discard it unexplored. In the Textbook run, once the incumbent $(2,2)$ with $z=10$ is found, the $x_1 \ge 3$ node — whose relaxation is worth only $9$ — is pruned by bound: even its relaxation can’t reach $10$, so no integer point below it can either.

Prune by infeasibility. If the node’s added constraints make its LP relaxation infeasible, the subtree is empty and we discard it.

Prune by integrality. If the relaxation optimum happens to be all-integer, it is a genuine feasible solution to the IP. There is nothing left to branch on, so the node becomes a leaf — and if it improves on the incumbent, it becomes the new incumbent.

These three cases are colour-coded in the tree pane (pruned-by-bound ✂, pruned-infeasible ✗, integer/incumbent ★), so you can see at a glance why each leaf closed.

6. The Incumbent and the Optimality Gap

Two running quantities drive the search, shown in the status strip:

The incumbent is the best integer-feasible solution found so far. It is a lower bound on the optimum (for a max problem) — a value we know is achievable.
The global bound is the best (largest) LP bound among all still-open nodes, together with the incumbent. It is an upper bound — no integer solution anywhere can exceed it.

The distance between them is the optimality gap:

\[\text{gap} = \frac{\text{global bound} - \text{incumbent}}{|\text{incumbent}|}.\]

The search is provably done when the gap reaches zero — every open node has a bound no better than the incumbent, so the incumbent is optimal. In practice, large industrial problems are often stopped early at a small non-zero gap (say 1%), trading a certificate of optimality for time. This is one of the great practical virtues of branch and bound: it produces a feasible solution and a bound on how far that solution could possibly be from optimal, at every moment during the search.

7. Search Strategy and Branching Rules

Branch and bound leaves two choices open, and they matter a great deal in practice.

Which open node to explore next? The visualiser uses best-bound search: always expand the open node with the strongest LP bound. This tends to raise the incumbent quickly and prove optimality with few nodes. The main alternative is depth-first search, which dives to leaves fast (finding feasible incumbents early, which sharpens pruning) and uses little memory, at the cost of sometimes exploring more nodes. Real solvers blend the two.

Which fractional variable to branch on? The visualiser uses the most-fractional rule — branch on the variable whose value is closest to a half-integer. It is simple and intuitive but, perhaps surprisingly, often mediocre. Better rules estimate the objective degradation each branch would cause: pseudocost branching learns these from past branches, and strong branching actually tentatively solves both child LPs to measure it. The choice of branching variable can swing the node count by orders of magnitude.

This mirrors the situation with simplex’s pivot rules: the algorithm is correct under many choices, but the choice governs how fast it runs.

8. Cutting Planes and the Integer Hull

Now we change tack. Instead of splitting the problem, can we tighten the relaxation itself?

Imagine the integer hull: the convex hull of all integer-feasible points. If we could describe it with linear inequalities, a single LP over it would hand us the integer optimum directly — its vertices are integer points. The integer hull is generally too complex to write down fully, but we don’t need all of it; we only need it near the optimum.

A cutting plane is a valid inequality $\boldsymbol{\alpha}^\top \mathbf{x} \le \beta$ that

is satisfied by every integer-feasible point (so we lose no real solutions), but
is violated by the current fractional LP optimum (so it actually does something).

Add such a cut to the relaxation, re-solve, and the LP optimum is forced to move — closer to the integer hull. Repeat, and the relaxation tightens around the integer optimum. In the visualiser these appear as dashed magenta lines slicing across the geometry pane, with the LP optimum diamond jumping to a new vertex after each cut.

9. Gomory Cuts from the Simplex Tableau

Where do valid cuts come from? The visualiser uses Gomory fractional cuts, which are read directly off the optimal simplex tableau — a satisfying callback to the simplex post.

Take a row of the optimal tableau whose basic variable $x_{B(i)}$ is fractional:

\[x_{B(i)} + \sum_{j \in \mathcal{N}} \bar{a}_{ij}\, x_j = \bar{b}_i,\]

where $\mathcal{N}$ indexes the non-basic variables — in our 2-D setting, the slack variables of the two binding constraints. Split each coefficient into its integer floor and a fractional remainder,

\[\bar a_{ij} = \lfloor \bar a_{ij} \rfloor + f_{ij}, \quad 0 \le f_{ij} < 1, \qquad \bar b_i = \lfloor \bar b_i \rfloor + f_i, \quad 0 < f_i < 1,\]

(the strict $f_i > 0$ is exactly the assumption that this row is fractional). Substitute these into the row and collect every term that is guaranteed to be an integer onto the right-hand side:

\[\underbrace{\sum_{j \in \mathcal{N}} f_{ij}\, x_j - f_i}_{\text{left-hand side}} \;=\; \underbrace{\lfloor \bar b_i \rfloor - x_{B(i)} - \sum_{j \in \mathcal{N}} \lfloor \bar a_{ij} \rfloor\, x_j}_{\text{an integer whenever } x \text{ is integer-feasible}}.\]

Now two observations pin down the left-hand side. First, the right-hand side is a sum and difference of integers (for any integer-feasible $x$ all the $x_j$ are integers, and the floors are integers by construction), so the left-hand side is an integer. Second, because every $x_j \ge 0$ and every $f_{ij} \ge 0$, the sum $\sum_{j} f_{ij}\, x_j \ge 0$, so the left-hand side is at least $-f_i$, and since $f_i < 1$ that means it is strictly greater than $-1$. An integer that is greater than $-1$ must be $\ge 0$. Putting the two together, $\sum_j f_{ij} x_j - f_i \ge 0$, i.e.

\[\sum_{j \in \mathcal{N}} f_{ij}\, x_j \;\ge\; f_i.\]

This is the Gomory cut. The argument above shows it holds at every integer point, so it is a valid inequality — adding it loses no integer-feasible solution. But a valid inequality is only useful if it actually does something: it must be violated by the current fractional optimum, or we have cut nothing away. For a Gomory cut this is guaranteed by construction, and it is worth seeing exactly why.

The key is that the cut is written entirely in terms of the non-basic variables $\mathcal{N}$. At the current LP optimal vertex every non-basic variable is zero by definition of a basic feasible solution, so the left-hand side $\sum_{j\in\mathcal N} f_{ij}\, x_j$ evaluates to exactly $0$ there. The cut demands this be $\ge f_i$, so at the current vertex it reads $0 \ge f_i$ — which is false, because we deliberately chose a source row whose basic value is fractional, making $f_i > 0$ strictly. The fractional vertex is therefore sliced off.

So the cut points “in the right direction” not by luck but by two design choices acting together: it lives in non-basic space — exactly the coordinates pinned to zero at the current vertex — and its right-hand side $f_i$ is strictly positive precisely because the source row is fractional. The contrast is instructive: had we picked a row whose RHS were already integer, then $f_i = 0$ and the cut would read $\sum_j f_{ij} x_j \ge 0$, trivially true everywhere (every term is non-negative) and cutting off nothing. Fractionality of the chosen row is the whole reason the cut bites.

So it slices off the fractional vertex without losing a single lattice point. The visualiser reconstructs this from the basis at the node’s optimal vertex and translates it from slack-space back into $(x_1, x_2)$ coordinates so it can be drawn. In Cutting Planes mode the left pane shows this tableau directly: the fractional source row is highlighted, its fractional parts $f_{ij}$ and $f_i$ are listed, and the resulting cut is assembled beneath — updating each round as the basis changes.

Reading a cut off the optimal tableau

To make the connection to the simplex post concrete, here is the actual optimal tableau for the Textbook root relaxation, in the same format as that post — basic variables labelling the rows, non-basic slacks $s_1, s_2$ as the free columns, the identity block sitting under the basic columns, and the shadow prices in the $z$-row:

\[\begin{array}{c|cc|cc|c} & x_1 & x_2 & \color{#1f77b4}{s_1} & \color{#1f77b4}{s_2} & \text{RHS} \\ \hline \bbox[6px,#fff3b0]{\color{#b8860b}{x_1}} & \bbox[6px,#fff3b0]{1} & \bbox[6px,#fff3b0]{0} & \bbox[6px,#fff3b0]{\color{#1f77b4}{-\tfrac{1}{13}}} & \bbox[6px,#fff3b0]{\color{#1f77b4}{\tfrac{4}{13}}} & \bbox[6px,#ffd6d6]{\color{#cc0000}{\tfrac{32}{13}}} \\[2pt] x_2 & 0 & 1 & \tfrac{4}{13} & -\tfrac{3}{13} & \tfrac{28}{13} \\ \hline z & 0 & 0 & \tfrac{5}{13} & \tfrac{6}{13} & \tfrac{152}{13} \end{array}\]

The $\bbox[4px,#fff3b0]{\color{#b8860b}{x_1}}$ row is highlighted in yellow because its basic value $x_1 = \tfrac{32}{13} \approx 2.46$ is fractional — it is a valid source row. (The $x_2$ row, with $x_2 = \tfrac{28}{13}\approx 2.15$, would do just as well.) The ingredients of the cut are exactly the highlighted cells:

the $\color{#cc0000}{\text{RHS in red}}$ supplies the fractional part $f_i = \tfrac{32}{13} - 2 = \color{#cc0000}{\tfrac{6}{13}}$;
the $\color{#1f77b4}{\text{non-basic coefficients in blue}}$, under the slack columns, supply $f_{s_1} = -\tfrac{1}{13} - (-1) = \color{#1f77b4}{\tfrac{12}{13}}$ and $f_{s_2} = \tfrac{4}{13} - 0 = \color{#1f77b4}{\tfrac{4}{13}}$.

Dropping these into $\sum_{j\in\mathcal N} f_{ij}\,x_j \ge f_i$ gives the cut in slack-space:

\[\color{#1f77b4}{\tfrac{12}{13}}\, s_1 + \color{#1f77b4}{\tfrac{4}{13}}\, s_2 \;\ge\; \color{#cc0000}{\tfrac{6}{13}}.\]

This cut is expressed in the slacks, but the picture is drawn in $(x_1, x_2)$, so we translate. Each slack is defined by writing its constraint as an equality — $\text{(constraint LHS)} + s_i = b_i$ — and solving for the slack, so for the two Textbook constraints

\[\begin{aligned} 3x_1 + 4x_2 \le 16 \;&\Longrightarrow\; s_1 = 16 - 3x_1 - 4x_2, \\ 4x_1 + x_2 \le 12 \;&\Longrightarrow\; s_2 = 12 - 4x_1 - x_2. \end{aligned}\]

Substituting these two identities into the slack-space cut and simplifying lands it back in the original coordinates:

\[2x_1 + 2x_2 \;\le\; 9.\]

It is worth being clear about what kind of operation each step is. This last substitution is just Gaussian elimination — using the constraint equalities $s_i = b_i - \mathbf{a}_i\cdot\mathbf{x}$ to eliminate the slacks is exactly the row-combination work from the simplex post, and it is legitimate because those equalities hold at every point, so it merely re-expresses the same inequality in different coordinates. The tableau row we started from was produced the same way — it is $B^{-1}$ applied to the original system. But the cut itself is not something elimination could ever have produced: any linear combination of the constraint equalities is satisfied at the current LP vertex, so row operations alone can never manufacture an inequality that excludes it. The step that does the real work is the rounding — splitting each coefficient into floor plus fractional part and invoking integrality. So the construction is bookended by elimination (build the source row, then translate coordinates) with the genuinely integer-programming step, the floor, in the middle.

This is the first cut the visualiser adds, drawn as the first dashed magenta line. Interestingly, if you keep adding cuts (the Cutting Planes mode), the iterates close in on the integer optimum, and the last two cuts generated are $x_1 + x_2 \le 4$ and $2x_1 + x_2 \le 6$ — both facets of the integer hull, and they intersect exactly at the integer optimum $(2,2)$. The procedure has, in effect, rediscovered the true integer description right where it matters.

10. Branch and Cut: The Best of Both

Branch and cut is the combination that powers every serious MIP solver: at each node, add a few cutting planes to tighten the relaxation before deciding whether to branch. Cuts raise the bound (sharpening pruning and sometimes producing an integer optimum outright); branching guarantees the search terminates even when cuts stall.

The precise rule the visualiser follows at each node makes the “when to cut, when to branch” decision concrete. Having selected the strongest open node (best-bound, as in §7) and survived the prune-by-bound test, it does the following:

Cut, up to a fixed budget. While the node’s LP optimum is fractional and a Gomory cut can be read off its tableau, add the cut and re-solve — but at most a fixed number of rounds per node (two, in the visualiser). The loop also stops early if the LP turns integer, if no valid cut exists, or if the tightened bound drops to the incumbent (in which case the node is pruned by bound right there).
Then branch on whatever fractionality survives. Once the cut budget is spent, inspect the resulting LP optimum: if it is integer, the node is a feasible solution (a new incumbent if it improves); if it is infeasible, prune; otherwise it is still fractional, and we branch on the most-fractional variable exactly as in §4 — handing the two children back to the open pool.

So cutting and branching are not alternatives chosen by some test; they are applied in sequence at every node — cut a bounded amount to tighten, then branch on whatever the cuts could not resolve. The cut budget is the single knob that interpolates between the three methods: a budget of zero rounds is pure branch and bound, an unbounded budget with branching switched off is pure cutting planes, and a small finite budget with branching on is branch and cut.

The visualiser lets you run the same instance three ways and compare. The contrast is the whole point, and the presets are chosen to make it vivid:

Preset	Branch & Bound	Branch & Cut	Cutting Planes
Textbook	5 nodes	3 nodes, 3 cuts	1 node, 5 cuts
Tight relaxation	1 node	1 node	1 node
Cuts win	25 nodes	1 node, 2 cuts	1 node, 2 cuts
Cuts stall	3 nodes	3 nodes, 2 cuts	27 cuts

A few lessons fall out:

Tight relaxation has an LP optimum that is already integral, so every method finishes at the root. When the relaxation is tight, integer programming is no harder than LP.
Cuts win is a thin, near-$45^\circ$ wedge. Branch and bound thrashes through 25 nodes chasing ever-smaller fractions near the apex, while two cutting planes carve straight to the integer optimum. This is the case for cuts.
Cuts stall is the cautionary tale: pure Gomory cuts “tail off,” adding 27 ever-shallower slices that each barely move the bound, while branch and bound finishes in 3 nodes. This slow convergence is a well-known weakness of pure cutting-plane methods — and exactly why branching is kept in the loop.

Branch and cut sits in the middle, robust across all four: a little cutting to tighten, a little branching to finish.

11. Edge Cases and Subtleties

Alternate optima. In the Cuts win preset, branch and bound returns $(3,4)$ while the cut-based methods return $(0,7)$ — both with $z = 7$. The objective is parallel to the wedge, so an entire edge of integer points is optimal, and different methods land on different ones. Same value, different vertex — the integer analogue of the alternate-optima case from the simplex post.

No feasible integer solution. A relaxation can be perfectly feasible while the IP is not — the polyhedron may simply contain no lattice point. Branch and bound discovers this by pruning every leaf without ever finding an integer incumbent; it terminates reporting infeasibility.

Unbounded relaxation. If the relaxation is unbounded in an improving direction, so is the IP (assuming it is feasible). The visualiser detects this and reports it rather than looping.

Cutting planes that never finish. Pure Gomory cuts are guaranteed to converge in theory (with care over the choice of source row), but convergence can be painfully slow and numerically delicate — the coefficients can grow and accumulate rounding error. The visualiser caps the number of cut rounds; if a custom problem exceeds it, the Cutting Planes mode honestly reports that it did not converge and suggests switching to branch and cut. This is not a defect of the demo so much as a faithful reflection of why nobody runs pure cutting planes in practice.

Worst-case explosion. Branch and bound is exponential in the worst case — the Cuts win preset already shows a 25-node tree for a two-variable problem. On real instances the tree can be enormous, which is why bounding quality (cuts), branching rules, and good incumbents (heuristics) matter so much.

12. What to Watch in the Visualiser

As you step through an instance, the two panes tell a coordinated story:

The geometry pane shows the current node’s feasible region (the shaded polygon), the integer lattice points (feasible ones highlighted), the objective direction, any cuts as dashed lines, the LP optimum as a diamond (hollow = fractional, filled = integer), and the incumbent as a ★. Watch the region shrink as you descend the tree and the diamond jump as cuts are added.
The tree pane shows the search: nodes coloured by fate (active, branched, incumbent ★, pruned ✂/✗), labelled with their LP bounds, the current node ringed in white.
The status strip tracks the incumbent, the global bound, and the gap closing to zero.
The mode switch re-runs the same instance as Branch & Bound, Branch & Cut, or Cutting Planes — flip between them on the Cuts win and Cuts stall presets to feel the trade-offs in §10.

13. Going Further

Topics beyond this two-variable visualiser, but natural next steps:

Stronger cut families. Gomory cuts are the classic; modern solvers lean on mixed-integer rounding (MIR), cover, clique, flow-cover, and lift-and-project cuts, separated only when violated.
Better branching. Pseudocost and reliability branching, and full strong branching, dramatically shrink trees compared to most-fractional.
Primal heuristics. The feasibility pump, RINS, and local branching find good incumbents early, which sharpens pruning throughout.
Warm-starting the children. A child LP differs from its parent by one bound, so the dual simplex re-optimises it in a handful of pivots instead of from scratch — the dual machinery from the simplex post earning its keep.
Presolve. Tightening bounds, removing redundant constraints, and detecting implications before the search even begins often matters more than anything during it.
Branch and price. For problems with enormous numbers of variables, column generation is interleaved with branching — branch and price — the cutting-plane idea applied to the dual.

The two-variable picture is a faithful miniature of all of this: bound, branch, cut, prune, repeat — until the gap closes.

The Simplex Algorithm Visualised

Mon, 16 Feb 2026 21:47:08 +0000

For many years I have used the simplex algorithm for Linear Programming (LP) but could never quite wrap my head around its inner workings. I knew it traversed the vertices of a polyhedron encoded by the problem constraints and it used this mysterious tableau to do so, but any explanation I read just didn’t quite click for me.

Fortunately, today we have amazing visualisation tools and LLMs to generate them quickly. See below the animation that finally did it for me, courtesy of Claude. Hopefully it can help someone else to understand too.

Explanation in detail below the animation (also Claude).

Companion to the Interactive Visualizer

This document walks through every concept behind the simplex method visualizer, from tableau construction to duality, with special attention to the geometric intuition and the edge cases where things get interesting.

1. The Setup

We’re solving a linear program (LP): optimize a linear objective function subject to linear inequality constraints.

Standard form (maximization):

\[\max \; \mathbf{c}^\top \mathbf{x} \quad \text{subject to} \quad A\mathbf{x} \leq \mathbf{b}, \;\; \mathbf{x} \geq \mathbf{0}\]

In two dimensions, this means: maximize $c_1 x_1 + c_2 x_2$ subject to a set of half-plane constraints. The feasible region is a convex polygon (or polyhedron in higher dimensions), and a fundamental theorem of LP guarantees the optimum occurs at a vertex — if one exists.

The simplex method exploits this by only ever visiting vertices, moving along edges in the direction that improves the objective.

2. Slack Variables and the Tableau

Each inequality constraint $a_{i1} x_1 + a_{i2} x_2 \leq b_i$ becomes an equality by introducing a slack variable $s_i \geq 0$:

\[a_{i1} x_1 + a_{i2} x_2 + s_i = b_i\]

The slack $s_i$ measures how much room is left before the constraint is tight. If $s_i = 0$, the constraint is binding — you’re on that boundary.

The objective is rewritten as an equation too: $z - c_1 x_1 - c_2 x_2 = 0$. All of this is packed into a tableau — a matrix where:

Each row (except the last) is a constraint equation
The last row is the objective equation
Each column corresponds to a variable ($x_1, x_2, s_1, \ldots, s_m$, RHS)

Initial Tableau Structure

For the general case with $n$ decision variables and $m$ constraints, the initial tableau is:

\[\begin{array}{c|cccc|cccc|c} & x_1 & x_2 & \cdots & x_n & s_1 & s_2 & \cdots & s_m & \text{RHS} \\ \hline s_1 & a_{11} & a_{12} & \cdots & a_{1n} & 1 & 0 & \cdots & 0 & b_1 \\ s_2 & a_{21} & a_{22} & \cdots & a_{2n} & 0 & 1 & \cdots & 0 & b_2 \\ \vdots & \vdots & \vdots & \ddots & \vdots & \vdots & \vdots & \ddots & \vdots & \vdots \\ s_m & a_{m1} & a_{m2} & \cdots & a_{mn} & 0 & 0 & \cdots & 1 & b_m \\ \hline z & -c_1 & -c_2 & \cdots & -c_n & 0 & 0 & \cdots & 0 & 0 \end{array}\]

The left column shows the basic variables — initially all slacks. The identity matrix under $s_1, \ldots, s_m$ means each slack is a unit vector, so the solution reads directly: $s_i = b_i$ and all $x_j = 0$. The objective row carries the negated costs $-c_j$; these are the reduced costs that drive the pivot selection.

After one pivot (say $x_k$ enters and $s_r$ leaves), the tableau transforms so that column $x_k$ becomes a unit vector with the 1 in row $r$, and the row label changes from $s_r$ to $x_k$:

\[\begin{array}{c|cccc|cccc|c} & x_1 & \cdots & x_k & \cdots & s_1 & \cdots & s_r & \cdots & \text{RHS} \\ \hline s_1 & \tilde{a}_{11} & \cdots & 0 & \cdots & 1 & \cdots & \tilde{a}_{1r} & \cdots & \tilde{b}_1 \\ \vdots & \vdots & & \vdots & & \vdots & & \vdots & & \vdots \\ x_k & \tilde{a}_{r1} & \cdots & 1 & \cdots & 0 & \cdots & \tilde{a}_{rr} & \cdots & \tilde{b}_r \\ \vdots & \vdots & & \vdots & & \vdots & & \vdots & & \vdots \\ \hline z & \bar{c}_1 & \cdots & 0 & \cdots & 0 & \cdots & \bar{c}_r & \cdots & \bar{z} \end{array}\]

where $\tilde{a}_{ij}$ denotes the updated coefficients after row reduction and $\bar{c}_r$ (the objective row entry under the departed slack $s_r$) is now the emerging shadow price for constraint $r$.

In the visualizer’s 2-variable case ($n = 2$), the initial tableau looks like:

\[\begin{array}{c|cc|ccc|c} & x_1 & x_2 & s_1 & s_2 & s_3 & \text{RHS} \\ \hline s_1 & a_{11} & a_{12} & 1 & 0 & 0 & b_1 \\ s_2 & a_{21} & a_{22} & 0 & 1 & 0 & b_2 \\ s_3 & a_{31} & a_{32} & 0 & 0 & 1 & b_3 \\ \hline z & -c_1 & -c_2 & 0 & 0 & 0 & 0 \end{array}\]

This is exactly what you see at Step 1 of the animation. As the simplex progresses, slacks leave the basis one by one and get replaced by $x_1$ or $x_2$ in the row labels, while the identity block under the slack columns gets scrambled by the row operations — but the columns under the basic variables always form an identity.

Why This Representation?

Here is the key idea that makes everything else click. Every row of the tableau is a linear equation that is true at every step of the algorithm. The slack-variable equations $a_{i1}x_1 + a_{i2}x_2 + s_i = b_i$ and the objective equation $z - c_1x_1 - c_2x_2 = 0$ hold simultaneously, and they continue to hold no matter what we do to the tableau — they are invariants. The tableau is just a compact way of writing this system down.

Now, a system of $m$ equations in $n + m$ unknowns is underdetermined: it has infinitely many solutions. To pin down a single one, we choose $m$ variables to solve for (the basic variables) and force the remaining $n$ to zero (the non-basic variables). This is exactly what a vertex is.

The reason we want the tableau in a special form is so that this solution can be read off without any algebra. If each basic variable’s column is a standard basis vector — a single 1, the rest 0s — then setting the non-basic variables to zero makes every equation collapse to “basic variable = its RHS entry.” The solution falls straight out of the RHS column. Achieving and preserving this clean, identity-pattern form is the entire point of the row operations that follow. We are never changing the underlying equations; we are only re-expressing them so that a different vertex becomes the one we can read directly.

3. Basic and Non-Basic Variables

With $m$ constraints and $n$ original variables, we have $n + m$ total variables and $m$ equations. A basic feasible solution (BFS) partitions the variables into:

$m$ basic variables: solved from the equations (their values come from the RHS)
$n$ non-basic variables: fixed at zero

Each BFS corresponds to a vertex of the feasible polyhedron. The key geometric insight: setting $n$ variables to zero means you’re at the intersection of $n$ hyperplane boundaries, which in $n$ dimensions pins you to a point — a vertex.

The row labels in the tableau tell you which variables are currently basic. As the algorithm progresses, these labels change: slack variables get swapped out for decision variables as we move from the origin toward the optimum.

4. The Simplex Algorithm

4.1 Dantzig’s Pivot Rule (Entering Variable)

The visualizer uses Dantzig’s rule (also called the largest coefficient rule): choose the non-basic variable with the most negative coefficient in the objective row as the entering variable.

Why? The objective row coefficients are the reduced costs — they tell you the rate of change of $z$ per unit increase in each non-basic variable. A negative reduced cost means increasing that variable will increase $z$. Dantzig’s rule picks the steepest rate, which is a greedy heuristic for fastest improvement.

Other pivot rules exist. Bland’s rule (choose the smallest index among negative coefficients) guarantees no cycling but may be slower. The steepest edge rule considers the actual step length, not just the rate. Dantzig’s rule is the simplest and most commonly taught.

4.2 Minimum Ratio Test (Leaving Variable)

Once we’ve chosen which variable enters, we need to know how far to increase it before some basic variable hits zero (goes infeasible). For each row $i$ where the entering column has a positive entry $a_{ij} > 0$, the ratio $b_i / a_{ij}$ tells us how far we can go.

The minimum ratio determines the tightest constraint — the first basic variable to hit zero. That variable leaves the basis. This is the leaving variable, and together with the entering variable, they define the pivot.

If no entry in the entering column is positive, the problem is unbounded — you can increase the entering variable forever without any constraint stopping you.

4.3 Pivoting (Row Reduction)

This is where the “why” from Section 2 pays off. We have chosen a new variable to bring into the basis (the entering variable) and one to remove (the leaving variable). But the tableau equations are still written in terms of the old basis — the entering variable’s column is not yet a unit vector, so we can’t just read its value off the RHS. Pivoting re-expresses the exact same system of equations so that the new basic set once again sits in identity-pattern form.

Crucially, the operations we use — scaling a row, and adding a multiple of one row to another — are reversible and preserve the solution set of the linear system. We are not changing what the equations say; we are only changing which variables they are solved for. That is why the answer we eventually read off is genuinely a solution to the original program.

The pivot operation is standard Gaussian elimination, targeted to restore the structure:

Divide the pivot row by the pivot element, making the entering variable’s coefficient 1
Eliminate the entering variable from all other rows (including the objective row) using row operations

After pivoting, the entering variable’s column has become a standard basis vector (with the 1 in the pivot row), the leaving variable’s column is no longer a unit vector, and the solution is again readable straight from the RHS. The row label updates to reflect the new basic variable — geometrically, we have slid to an adjacent vertex.

The algorithm terminates when the basis can no longer be improved (Section 4.4). At that point the basic variables — typically a mix of decision variables $x_j$ and any slacks still carrying spare capacity — form the identity block, and the optimal vertex is read directly from the RHS. Watch this happen in the visualizer: the identity columns march from the slack block over to the $x_1, x_2$ columns as the origin’s slacks are swapped out for genuine decision variables.

4.4 Optimality Check

After each pivot, check the objective row. If all coefficients are non-negative, no improvement is possible — we’re at the optimum. If any coefficient is still negative, we repeat.

5. The Objective Direction Arrow

On the graph, the arrow labeled “improve ↑” (or “improve ↓” for minimization) shows the gradient of the objective function $\nabla z = (c_1, c_2)$ — the direction in which $z$ increases most rapidly.

Geometrically, the simplex method is sliding along the boundary of the feasible polygon in the direction that has the largest positive component along this gradient. The optimal vertex is the one where you can’t move along any edge without going against the gradient (or leaving the feasible region).

6. The Dual Problem

Every LP has a dual. For the standard max problem, the dual is:

Primal:

\[\max \; \mathbf{c}^\top \mathbf{x} \quad \text{s.t.} \quad A\mathbf{x} \leq \mathbf{b}, \;\; \mathbf{x} \geq \mathbf{0}\]

Dual:

\[\min \; \mathbf{b}^\top \mathbf{y} \quad \text{s.t.} \quad A^\top \mathbf{y} \geq \mathbf{c}, \;\; \mathbf{y} \geq \mathbf{0}\]

The construction is mechanical: objective coefficients $\leftrightarrow$ constraint RHS, constraint matrix gets transposed, inequalities flip, and $\min \leftrightarrow \max$.

For a minimization primal, the dual is a maximization problem with $\leq$ constraints.

6.1 Key Duality Theorems

Weak duality: The dual objective always bounds the primal. For a max primal and min dual: any dual-feasible $\mathbf{y}$ gives $\mathbf{b}^\top \mathbf{y} \geq \mathbf{c}^\top \mathbf{x}$ for any primal-feasible $\mathbf{x}$. This means every dual solution provides a certificate that the primal can’t do better than $\mathbf{b}^\top \mathbf{y}$.

Strong duality: At optimality, the bound is tight — the primal and dual objectives are equal:

\[\mathbf{c}^\top \mathbf{x}^* = \mathbf{b}^\top \mathbf{y}^*\]

The visualizer verifies this at the optimal step.

Complementary slackness: If a primal constraint has slack ($s_i > 0$), the corresponding dual variable must be zero ($y_i = 0$), and vice versa:

\[s_i \cdot y_i = 0 \quad \forall \; i\]

The contrapositive: if $y_i > 0$, the primal constraint must be binding. Both being binding simultaneously is perfectly fine — this is the non-degenerate case.

6.2 Shadow Prices

The optimal dual variables $\mathbf{y}^*$ are the shadow prices. They appear in the simplex tableau as the objective row entries under the slack variable columns.

The shadow price $y_i^*$ equals the partial derivative of the optimal value function with respect to the $i$-th constraint’s RHS:

\[y_i^* = \frac{\partial \, z^*}{\partial \, b_i}\]

In economic terms: if constraint $i$ limits a resource, $y_i^*$ is the marginal value of one additional unit of that resource.

A zero shadow price means the constraint has slack — you already have more of that resource than you need, so more wouldn’t help. A positive shadow price identifies a bottleneck.

6.3 Why the Dual Lives in the Tableau

The simplex method solves the primal and dual simultaneously. The dual variables at any stage are $\mathbf{c}_B^\top B^{-1}$, where $B$ is the current basis matrix. Through the row operations of the simplex method, this vector is maintained in the objective row under the slack columns. At optimality, it gives the optimal dual solution — no separate computation needed.

More precisely, the optimal value function $z^\ast(\mathbf{b})$ is piecewise linear and concave, and the dual solution $\mathbf{y}^\ast$ is a subgradient:

\[z^*(\mathbf{b}) = \mathbf{c}_B^\top B^{-1} \mathbf{b}\]

This is linear in $\mathbf{b}$ as long as the current basis $B$ remains optimal.

7. Minimization

To minimize $c_1 x_1 + c_2 x_2$, the visualizer internally negates the objective and solves:

\[\max \; (-c_1) x_1 + (-c_2) x_2\]

The simplex algorithm is identical; only the interpretation changes:

The displayed objective value is negated back for the user
The gradient arrow flips direction (pointing toward decreasing $z$)
The dual switches from minimization to maximization

This is a standard trick — every minimization LP can be converted to maximization without loss of generality.

8. Edge Cases and Subtleties

8.1 The Origin Isn’t Always Feasible

The visualizer assumes all constraints are of the form $A\mathbf{x} \leq \mathbf{b}$ with $\mathbf{b} \geq \mathbf{0}$. This guarantees that $\mathbf{x} = \mathbf{0}$ (the origin) is feasible, because all slacks start positive. The initial basis is simply ${s_1, s_2, \ldots, s_m}$.

But what if some $b_i < 0$? Then $s_i = b_i < 0$ at the origin, which is infeasible. The standard simplex method can’t start.

Solutions:

Big-M method: Add artificial variables with a large penalty $M$ in the objective, driving them to zero if a feasible solution exists.
Two-phase simplex: Phase I minimizes the sum of artificial variables to find a feasible starting point (or proves infeasibility). Phase II then optimizes the real objective from there.
Dual simplex: Start with a dual-feasible (but primal-infeasible) tableau and pivot to achieve primal feasibility while maintaining dual feasibility. This is particularly useful for re-optimization after adding constraints.

The visualizer validates that all RHS values are non-negative before solving, which is why it rejects negative $b_i$.

8.2 Degeneracy

A basic feasible solution is degenerate if one or more basic variables equal zero. Geometrically, this means more than $n$ constraints pass through the vertex (it’s “over-determined”).

Degeneracy causes problems because a pivot may not actually move to a new vertex — the objective value stays the same. In theory, this can cause cycling (visiting the same sequence of bases forever). In practice this is rare, but Bland’s rule guarantees it never happens.

With degeneracy, shadow prices may not be unique: different optimal bases (all representing the same vertex) can give different dual values. The shadow price interpretation becomes a subgradient rather than a gradient — formally, $\mathbf{y}^\ast$ lies in the subdifferential $\partial z^\ast(\mathbf{b})$.

8.3 Objective Parallel to a Constraint (Alternate Optima)

If the gradient of the objective $\nabla z = (c_1, c_2)$ is perpendicular to a constraint boundary’s normal (i.e., the objective is parallel to that constraint), then every point along that constraint edge between two optimal vertices achieves the same objective value.

In simplex terms, this shows up as a zero reduced cost for a non-basic variable at optimality — bringing it into the basis wouldn’t change $z$. Pivoting on it gives a different optimal BFS, and the convex combination of all such BFS gives the full set of optimal solutions (an edge or face of the polyhedron):

\[\mathbf{x}^* = \lambda , \mathbf{x}_1^* + (1 - \lambda) , \mathbf{x}_2^* \quad \text{for any } \lambda \in [0, 1]\]

The visualizer will correctly find one optimal vertex, but won’t indicate that others exist. Look for zeros in the objective row (under non-basic variable columns) at the final step — that’s the signature of alternate optima.

8.4 Unbounded Problems

If during the minimum ratio test, no row has a positive entry in the entering column, the entering variable can increase without limit — the feasible region extends to infinity in a direction that improves the objective:

\[z \to \infty \quad (\text{or } z \to -\infty \text{ for min})\]

The visualizer detects this and displays an “Unbounded!” error. Note that an unbounded feasible region doesn’t necessarily mean an unbounded objective — it depends on whether the objective gradient points into the unbounded direction.

8.5 Infeasibility

If the constraints are contradictory (e.g., $x_1 \leq 5$ and $x_1 \geq 10$), no feasible solution exists. The simplex method on the original problem would never encounter this directly (it would just start infeasible), which is why Phase I / Big-M is needed.

The visualizer doesn’t handle this case since it requires $\mathbf{b} \geq \mathbf{0}$, which always gives a feasible origin. But if you enter constraints that make the feasible region empty while keeping $\mathbf{b} \geq \mathbf{0}$ (e.g., $x_1 + x_2 \leq 10$ and $-x_1 - x_2 \leq -20$, which requires negative $b$), the validator will catch it.

8.6 Objective Perpendicular to a Constraint

If the objective gradient $\nabla z$ is exactly perpendicular to a constraint boundary, that constraint’s normal is parallel to $\nabla z$, meaning that constraint contributes maximally to the objective improvement per unit of resource. This shows up as that constraint having the highest shadow price in the dual solution — it’s the most valuable bottleneck.

This is actually the common case: the optimal vertex typically sits at the intersection of constraints whose normals $\mathbf{a}_i$ span a cone containing the objective gradient:

\[\nabla z = \sum_{i \in \mathcal{B}} y_i^* , \mathbf{a}_i\]

where $\mathcal{B}$ is the set of binding constraints and $y_i^*$ are the shadow prices. This is precisely the KKT (Karush-Kuhn-Tucker) condition for optimality.

8.7 When Decision Variables Can Be Negative

The standard form — and the visualizer — assumes $\mathbf{x} \geq \mathbf{0}$. This non-negativity is not cosmetic: it is what makes the origin a vertex, what lets each non-basic variable sit at zero, and what gives the minimum-ratio test its meaning (variables increase from zero until a constraint stops them). A variable that is allowed to go negative — a free (or unrestricted) variable — breaks all three assumptions, because its $x_j \geq 0$ wall is gone.

The fix is a reformulation, not a new algorithm. Split each free variable into the difference of two non-negative parts:

\[x_j = x_j^+ - x_j^-, \qquad x_j^+, \, x_j^- \geq 0.\]

Now $x_j^+ - x_j^-$ can take any sign, while both pieces obey the usual non-negativity, so the ordinary simplex method applies unchanged. A positive $x_j$ shows up as $x_j^+ > 0,\ x_j^- = 0$; a negative $x_j$ as the reverse. (At any basic feasible solution the two never both go positive — their columns are negatives of each other, so a sensible pivot keeps at most one in the basis. If both were positive you could subtract the common part from each and lower the objective slack, so it never helps.)

The cost is one extra column per free variable, doubling that part of the tableau. A cheaper alternative when a variable is merely bounded below at some $\ell_j \neq 0$ (rather than truly free) is to shift it: substitute $x_j’ = x_j - \ell_j \geq 0$, solve, and shift back. A box constraint $\ell_j \leq x_j \leq u_j$ is handled the same way (shift to put the lower bound at zero), with the upper bound carried as an ordinary constraint — or, in production solvers, via the bounded-variable simplex that treats $\ell_j$ and $u_j$ implicitly and lets non-basic variables rest at either bound, not only zero.

Geometrically, dropping $x_j \geq 0$ removes one of the axes as a boundary, so the feasible region is no longer trapped in the first quadrant and the origin may not be a vertex at all. The visualizer always draws the first quadrant and treats $x_1, x_2 \geq 0$ as hard walls, so it cannot show a free variable directly; to model one you would split it as above and interpret the result through $x_j = x_j^+ - x_j^-$.

9. Summary: What to Watch in the Visualizer

As you step through the animation, pay attention to:

The arrow shows where the objective wants to go — the simplex path should generally trend in that direction along the polygon edges.
Row labels changing — each swap tells you a slack variable (resource with room) became tight, and a decision variable entered production.
Objective row coefficients — negative values drive the next pivot. When they all go non-negative, you’ve arrived.
Shadow prices appearing at optimality under the slack columns — they tell you which constraints are worth relaxing.
The dual panel tracking along — dual variables evolve with each pivot, converging to the shadow prices at the optimal step. Strong duality confirms $\mathbf{c}^\top \mathbf{x}^\ast = \mathbf{b}^\top \mathbf{y}^\ast$.
Zero reduced costs at optimality — if a non-basic variable has a zero in the objective row, alternate optima exist along a constraint edge.

10. Going Further

Topics beyond the scope of this visualizer but natural extensions:

Revised simplex method: avoids storing the full tableau by maintaining $B^{-1}$ directly — much more efficient for large problems.
Interior point methods: instead of walking along edges, these methods cut through the interior of the polyhedron. They have polynomial worst-case complexity (unlike simplex, which is exponential in the worst case but fast in practice).
Sensitivity analysis: how much can you change the coefficients before the optimal basis changes? The shadow prices tell you the rates of change; sensitivity analysis tells you the ranges over which those rates hold.
Integer programming: when variables must be integers, the LP relaxation (solved by simplex) provides bounds, and branch-and-bound or cutting plane methods find the integer optimum.

Building trust in AI - Transparent models for better decisions

Fri, 15 Nov 2024 20:56:08 +0000

I have recently published a blog post on AIHub.org, titled Building trust in AI: Transparent models for better decisions.

The post is a more digestible version of our recent paper, published at IJCAI 24, Are Logistic Models Really Interpretable? by Danial Dervovic, Freddy Lécué, Nicolás Marchesotti and Daniele Magazzeni.

Hope you enjoy it.

Last Man Standing - optimal strategies for betting games

Sun, 13 May 2018 14:55:01 +0000

A good friend of mine with a penchant for sports betting told me recently about a new game he’s been playing: Last Man Standing. In this blog post we’re going to develop strategies for playing this game. Let’s get to it.

The Rules.

The game starts at the beginning of the English Premier League (EPL) season (or other sports league of your choice).
There are $M$ players in the game. Each player puts in $x$ units of cash (say, twenty quid).
Each of the $M$ players chooses one of the $n$ teams in the league. Different players can choose the same team if they wish.
If a given player’s team wins, they go through to the next week; lose or draw, they’re out.
The remaining players choose another team that they think will win. Importantly, no player can choose a team they have chosen before.
This procedure continues until either: there is one player left and they take all the winnings; or, there are no players left and another game begins, starting the next week. The pot rolls over, everyone puts in $x$ monies again and they are each free to choose any team they like again.
Continue until everyone is broke, apart from the player with the best strategy! Hopefully this will be us by the end of this blog post 😉.

The moment my pal told me about this I was hooked. This seemed exactly like the kind of game amenable to some mathematical and algorithmic analysis – potentially giving the informed player an edge over their unwitting opponents. What follows is my winding path of thoughts and strategies on how to play this game.

Formalised model

The first thing to do when thinking about this was to create a mathematical model of the game.

First, we have $n$ teams. We impose a linear ordering of the teams, i.e., the $1$st team is Arsenal, the $2$nd team is Chelsea etc – we will refer to a team simply by its index $j \in [n]$ (where $n:=\{1,2,\ldots, n\}$). Naturally, a strategy is then defined as ordered list of integers from $[n]$, with no repeats. Optimistically, we hope we won’t lose at any round so we specify a strategy as the longest possible list. The resulting list is a permutation, $\sigma \in \mathbb{S}_n$, where $\mathbb{S}_n$ is the symmetric group on $n$ elements.

Note an individual season will require two consecutive strategies in the best case, as each team plays home and away (concretely, in the EPL each pair of teams plays a match against each other twice per season). Without loss of generality we can consider the single best strategy from any chosen starting point in the season. When we must start again (i.e. when all our choices win, or more likely, one loses and we start playing again), recalculate the best strategy from the new starting point.

Now we have a formal notion for a strategy (as a permutation $\sigma \in \mathbb{S}_n$) we need to specify how to evaluate the various strategies. Note that the number of possible strategies is large ($n! \sim 10^{18}$ for the EPL). Let’s take the most natural measure: how likely is it that a strategy wins? Namely,

\[\operatorname{VAL}(\sigma) := \mathbb{P}(\sigma \text{ wins} ) = \mathbb{P}( \omega^{(1)}_{\sigma(1)} \cap \omega^{(2)}_{\sigma(2)} \cap \cdots \cap \omega^{(n)}_{\sigma(n)} ),\]

where $\omega^{(i)}_{j}$ is the event that team $j$ wins in week $i$ and we define $\operatorname{VAL}(\sigma)$ as the value of a strategy $\sigma \in \mathbb{S}_n$. Of course, the best strategy $\sigma^\star$ is the one with highest probability of winning, i.e. $\sigma^\star := \arg \max_{\sigma \in \mathbb{S}_n} \operatorname{VAL}(\sigma)$.

Now we run into our first problem: how to evaluate $\operatorname{VAL}(\sigma)$. This involves computing $\mathbb{P}(\sigma \text{ wins} )$. Now we don’t even know if this probability is well defined, different probabilistic models will return different values. Moreover, there are $n!$ strategies, so evaluating each strategy separately will be intractable. We need a simplifying assumption. First, let’s introduce some notation:

\[p^{(i)}_j = \mathbb{P} \left( \text{team }j\text{ wins in week }i \right) = \mathbb{P}(\omega^{(i)}_j).\]

We’re going to make the following assumption:

Assumption 1. The winning probabilities $p^{(i)}_j$ are independent, for all $i,j\in [n]$.

This assumption clearly doesn’t hold in reality. Consider the following scenario: team $j$ plays team $j'$ in week $i$. Then $p^{(i)}_j + p^{(i)}_{j'} \leq 1$ since both teams can’t win at the same time! One could consider numerous other scenarios in which this model fails, for instance think about a ‘winning streak’ – teams that have in previous weeks are likely to keep winning, due to some underlying factor such as a new great manager etc.

Nevertheless, we are going to work within the independence model as it will make the winning probabilities tractable. We are going to approximate $\operatorname{VAL}(\sigma)$ in the following way:

\[\widetilde{\operatorname{VAL}}(\sigma) = \prod_{i \in [n]} p^{(i)}_{\sigma(i)},\]

i.e. using the product of the winning probabilities, which is just the joint winning probability $\mathbb{P}(\sigma \text{ wins})$ for the teams ordered according to the strategy $\sigma$, under the independence assumption (Assumption 1).

We can now formally define the Last Man Standing problem $(\mathsf{LMS})$.

Problem ($\mathsf{LMS}$). Let $p^{(i)}_j \in (0, 1)$ for $i,j \in [n]$. Then, find

\[\tag{$\mathsf{LMS}$} \widetilde{\sigma}^\star = \arg \max_{\sigma \in \mathbb{S_n}}\widetilde{\operatorname{VAL}}(\sigma).\]

Note. The domain of the $p^{(i)}_j$ is the open interval $(0,1)$ as opposed to the closed interval $[0,1]$ since a team is never 100% likely to win or lose.

Implementation

Great, we now have a concrete mathematical problem we would like to solve, but it would be good to apply to some concrete data. Namely, where do we get the winning probabilities $p^{(i)}_j$? I thought about this for a while and came to the conclusion that no one is going to be better at calculating the chances of different of outcomes of sports games than bookmakers. Duh! Now we just need bookies’ odds data from somewhere…

Fortunately, the excellent footballdata python package gives us everything we need in a nice pandas DataFrame format. Let’s explore a little bit.

%matplotlib inline
%config InlineBackend.figure_format = 'retina'

import footballdata as foo
import matplotlib.pyplot as plt
import numpy as np
np.random.seed(1234)
import pandas as pd
import itertools
import cvxpy as cvx

from matplotlib.pyplot import figure, show
import pprint as pp

print(foo.MatchHistory.__doc__)

Provides pandas.DataFrames from CSV files available at
    http://www.football-data.co.uk/data.php

    Column names are explained here: http://www.football-data.co.uk/notes.txt

    Data will be downloaded as necessary and cached locally in ./data

    Parameters
    ----------
    leagues : string or iterable of league-ids to include, None for all
    seasons : string, int or list of seasons. Examples:
              '16-17'; 2016; '2016-17'; [14, 15, 16]

foo.MatchHistory.available_leagues()

['BEL-Jupiler League',
 'ENG-Championship',
 'ENG-Conference',
 'ENG-League 1',
 'ENG-League 2',
 'ENG-Premier League',
 'ESP-La Liga',
 'ESP-La Liga 2',
 'FRA-Ligue 1',
 'FRA-Ligue 2',
 'GER-Bundesliga',
 'GER-Bundesliga 2',
 'GRE-Ethniki Katigoria',
 'ITA-Serie A',
 'ITA-Serie B',
 'NED-Eredivisie',
 'POR-Liga 1',
 'SCO-Division 1',
 'SCO-Division 2',
 'SCO-Division 3',
 'SCO-Premiership',
 'TUR-Ligi 1']

prem = foo.MatchHistory('ENG-Premier League', range(2016, 2017)).read_games()
prem.sample(5)

			date	home_team	away_team	FTHG	FTAG	FTR	HTHG	HTAG	HTR	Referee	...	BbAv<2.5	BbAH	BbAHh	BbMxAHH	BbAvAHH	BbMxAHA	BbAvAHA	PSCH	PSCD	PSCA
league	season	game_id
ENG-Premier League	1617	2017-05-12 West Bromwich Albion-Chelsea	2017-05-12	West Bromwich Albion	Chelsea	0	1	A	0	0	D	M Oliver	...	1.99	20	1.25	2.06	2.02	1.89	1.85	9.76	5.18	1.38
		2017-04-15 Tottenham Hotspur-AFC Bournemouth	2017-04-15	Tottenham Hotspur	AFC Bournemouth	4	0	H	2	0	H	M Oliver	...	2.80	21	-1.75	1.98	1.92	2.01	1.95	1.25	7.00	12.80
		2017-04-30 Manchester United-Swansea City	2017-04-30	Manchester United	Swansea City	1	1	D	1	0	H	N Swarbrick	...	2.03	21	-1.50	2.21	2.14	1.78	1.74	1.42	4.79	9.45
		2016-10-29 Manchester United-Burnley	2016-10-29	Manchester United	Burnley	0	0	D	0	0	D	M Clattenburg	...	2.12	36	-2.00	2.13	2.05	1.85	1.80	1.26	6.40	14.25
		2017-05-21 Leicester City-AFC Bournemouth	2017-05-21	Leicester City	AFC Bournemouth	1	1	D	0	1	A	l Mason	...	2.23	17	-0.50	1.95	1.89	2.03	1.98	1.69	4.50	4.81

5 rows × 64 columns

print(list(prem)) # columns of DataFrame

['date', 'home_team', 'away_team', 'FTHG', 'FTAG', 'FTR', 'HTHG', 'HTAG', 'HTR', 'Referee', 'HS', 'AS', 'HST', 'AST', 'HF', 'AF', 'HC', 'AC', 'HY', 'AY', 'HR', 'AR', 'B365H', 'B365D', 'B365A', 'BWH', 'BWD', 'BWA', 'IWH', 'IWD', 'IWA', 'LBH', 'LBD', 'LBA', 'PSH', 'PSD', 'PSA', 'WHH', 'WHD', 'WHA', 'VCH', 'VCD', 'VCA', 'Bb1X2', 'BbMxH', 'BbAvH', 'BbMxD', 'BbAvD', 'BbMxA', 'BbAvA', 'BbOU', 'BbMx>2.5', 'BbAv>2.5', 'BbMx<2.5', 'BbAv<2.5', 'BbAH', 'BbAHh', 'BbMxAHH', 'BbAvAHH', 'BbMxAHA', 'BbAvAHA', 'PSCH', 'PSCD', 'PSCA']

We can see for the 2016-2017 premiership season we have a variety of bookies we can go to – these are the fields ending in ‘H’, ‘D’ and ‘A’; corresponding to ‘home win’, ‘draw’ and ‘away win’ respectively. Here we are given the odds in European format. We will need to convert these odds into a probability.

Computing probabilities from odds

European odds are given as a decimal number equalling bookies_payout/betters_stake. As an example, European odds of 1.40 mean that if you stake £100, the bet (if successful) will payout £140, and your profit will be £40.

Odds don’t correspond directly with the probabilities, since the bookies need their cut for whichever outcome takes place! We assume that the bookies’ cut, or vig is factored in proportionally with the odds. The bookmaker prices the decimal odds $d_E$ for an event $E$ as

\[d_E = \frac{1}{p_E + o_E},\]

where $p_E$ is their estimated probability for $E$ and $o_E$ is the overround for $E$. The overround is the relative form of the vig, i.e. the vig is the bookmaker’s percentage profit on the total stakes made on the event, whereas the overround is the expected profit. For example, 20% overround is vigorish of 16 2/3%

We have $o_E = o$ for all $E \in \Omega$, where $\Omega$ is the event set, since we assume the vig is priced proportionally to the odds.

We can then solve for the probabilities like so:

\[\frac{1}{d_E}= p_E + o_E = p_E + o\]

and

\[\sum_{E' \in \Omega} \frac{1}{d_{E'}} = 1 + \sum_{E' \in \Omega}{o_{E'}} = 1 + \vert\Omega\vert o.\]

So, $o = \frac{1}{\vert\Omega\vert}\left( \sum_{E' \in \Omega} \frac{1}{d_{E'}} - 1 \right)$ and we have

\[p_E = \frac{1}{d_E} - \frac{1}{\vert\Omega\vert} \left(\sum_{E' \in \Omega} \frac{1}{d_{E'}} - 1 \right) = \frac{1}{\vert\Omega\vert} + \frac{1}{d_E} - \frac{1}{\vert\Omega\vert}\sum_{E' \in \Omega} \frac{1}{d_{E'}}.\]

def probs_from_odds(odds_home, odds_draw, odds_away):
    sum_reciprocol_odds = 1 / odds_home + 1 / odds_draw + 1 / odds_away
    summand = - (1 / 3) * sum_reciprocol_odds + (1 / 3)
    prob_home, prob_draw, prob_away = \
        map(lambda odds: 1 / odds +  summand, [odds_home, odds_draw, odds_away])
    return prob_home, prob_draw, prob_away

Team names

Below we number the teams, i.e. establish the correspondence $\text{team name} \mapsto j \in [n]$.

teams = list(prem.home_team.unique())
print(teams)

['Burnley', 'Crystal Palace', 'Everton', 'Hull City', 'Manchester City', 'Middlesbrough', 'Southampton', 'AFC Bournemouth', 'Arsenal', 'Chelsea', 'Manchester United', 'Leicester City', 'Stoke City', 'Swansea City', 'Tottenham Hotspur', 'Watford', 'West Bromwich Albion', 'Sunderland', 'West Ham United', 'Liverpool']

Winning probabilities matrix

What we want now is the matrix $X\in [0,1]^{38 \times 20}$, with elements defined by

\[X_{i,j} := p^{(i)}_j = \text{probability that team }j\text{ wins in week }i.\]

def build_X(league, teams, bookie_name):
    num_teams = len(teams)
    X = np.zeros( (2 * (num_teams - 1), num_teams))
    X_index = np.empty((2 * (num_teams - 1), num_teams), dtype=object)
    games_played_by_team = np.zeros(len(teams), dtype=int) # array keeps track of how many games each team has played

    for index, row in league.iterrows():
        j_home = teams.index(row["home_team"])
        j_away = teams.index(row["away_team"])
        i_home = games_played_by_team[j_home]
        i_away = games_played_by_team[j_away]
        home_prob, draw_prob, away_prob = probs_from_odds(row[bookie_name + "H"], row[bookie_name + "D"], row[bookie_name + "A"])
        X[i_home, j_home] = home_prob
        X[i_away, j_away] = away_prob
        X_index[i_home, j_home] = index
        X_index[i_away, j_away] = index
        games_played_by_team[j_home] += 1
        games_played_by_team[j_away] += 1
    return X    

X = build_X(prem, teams, "B365")

Practical concerns

The winning probabilities will be (very) small, even for the best ones. So let’s use the $\log(\widetilde{\text{VAL}})$ of the winning probability as our metric of ‘goodness’ for a strategy, since $\log(x)$ is a monotonic function of $x \in (0, 1]$.

We also want to be able to convert a permutation back into a list of teams

def evaluate_perm(perm_list, X):
    """
      Gives log of winning probability of a permutation
    """
    winning_probs = X[np.arange(len(perm_list)), np.array(perm_list)]
    return np.sum(np.log(winning_probs))

def team_order_from_perm_list(perm_list, teams):
    """
        takes a list of integers and returns corresponding list of teams
    """
    ## check list 'full'
    if( not (sorted(perm_list) == list(range(len(perm_list))))):
        raise(ValueError("perm_list doesn't contain all integers from 0 to %d" % (len(perm_list) -1 )))
    team_choice = [teams[i] for i in perm_list]
    return team_choice

Strategies

We now have all the necessary ingredients to start deriving and evaluating strategies.

Greedy strategy

First of all, let’s do the most simple thing we can do: at each time, pick the team out of those you are allowed to with the highest winning probability for that week. We’ll call this the greedy strategy.

def greedy_strategy(X, teams):
    """
        Given winning probability matrix X returns the "greedy" strategy
        permutation of teams - given as a list of integers.
    """
    ## Check dims of X ok
    num_weeks = X.shape[0]
    num_teams = X.shape[1]
    if (num_teams != len(teams)):
        raise(ValueError("X needs to have %d rows, has %d." % (num_teams, X.shape[1])))
    # generate sample
    visited = []
    for t in list(range(min(num_teams, num_weeks))):
        allowed = [i for i in list(range(num_teams)) if i not in visited]
        probs = X[t, allowed] / np.sum(X[t, allowed])
        sample = allowed[np.argmax(probs)]
        visited.append(sample)
    return visited

greedy_perm = greedy_strategy(X, teams)
greedy_strategy_result = {"opt_perm": greedy_perm, \
                          "opt_value": evaluate_perm(greedy_perm, X), \
                          "opt_team_list": team_order_from_perm_list(greedy_perm, teams)\
                         }
pp.pprint(greedy_strategy_result, compact=True)

{'opt_perm': [4, 14, 9, 8, 11, 19, 10, 6, 18, 15, 7, 2, 16, 12, 13, 1, 5, 0, 17,
              3],
 'opt_team_list': ['Manchester City', 'Tottenham Hotspur', 'Chelsea', 'Arsenal',
                   'Leicester City', 'Liverpool', 'Manchester United',
                   'Southampton', 'West Ham United', 'Watford',
                   'AFC Bournemouth', 'Everton', 'West Bromwich Albion',
                   'Stoke City', 'Swansea City', 'Crystal Palace',
                   'Middlesbrough', 'Burnley', 'Sunderland', 'Hull City'],
 'opt_value': -13.507040869365095}

So by our earlier reasoning, any permutation of teams we can find that is higher than greedy_strategy_result["opt_value"] is a success!

Greedy sampling strategy

Now let’s do the second simplest thing we can think of.

Greedy sampling (GS). At each timestep $t$, look at the teams that haven’t previously been chosen. Choose a team randomly, where the probability is proportional to the odds of that team winning in week $t$.

More formally, let $\mathcal{A}(t)$ be the set of teams allowed at time $t$, i.e. those that have not been chosen in times $1,2,\ldots,t-1$. Then,

\[\mathbb{P}\{ \text{choose team }j \text{ at time } t\} = \frac{p^{(t)}_j}{\sum_{k \in \mathcal{A}(t)} p^{(t)}_k}.\]

Let’s try and provide some rough theoretical justification for this method of generating permutations of teams (you can skip this part if you believe me that this is an OK thing to do).

Suppose we have have chosen the permutation $\sigma = (\sigma(1), \sigma(2), \ldots, \sigma(n))$ according to the greedy sampling scheme. The probability of choosing this permutation is

\[\mathbb{P}_{\sigma \sim \text{GS}}(\sigma) = \frac{p^{(1)}_{\sigma(1)}}{\sum_{k \in [n]} p^{(1)}_{k}} \cdot \frac{p^{(2)}_{\sigma(2)}}{\sum_{k \in [n]\setminus \{\sigma(1)\}} p^{(2)}_{k}} \cdot \frac{p^{(3)}_{\sigma(3)}}{\sum_{k \in [n]\setminus \{\sigma(1), \sigma(2)\}} p^{(3)}_{k}} \cdot \cdots \cdot \frac{p^{(n-1)}_{\sigma(n-1)}}{\sum_{k \in [n]\setminus \{\sigma(1), \sigma(2), \ldots, \sigma(n-2)\}} p^{(n-1)}_{k}} \cdot 1 .\]

We then have that

\[\mathbb{P}\{\sigma \text{ wins}\} = p^{(n)}_{\sigma(n)} \cdot \left( \sum_{k \in [n]\setminus \{\sigma(1), \sigma(2), \ldots, \sigma(n-2)\}} p^{(n-1)}_{k} \right) \cdot \cdots \cdot \left( \sum_{k \in [n]} p^{(1)}_{k} \right) \cdot \mathbb{P}_{\sigma \sim \text{GS}}(\sigma).\]

Let’s consider two permutations $\sigma_A$ and $\sigma_B$ and take the ratio of the winning probabilities:

\[\frac{\mathbb{P}\{\sigma_A \text{ wins}\}}{\mathbb{P}\{\sigma_B \text{ wins}\}} = \frac{p^{(n)}_{\sigma_A(n)} \cdot \left( \sum_{k \in [n]\setminus \{\sigma_A(1), \sigma_A(2), \ldots, \sigma_A(n-2)\}} p^{(n-1)}_{k} \right) \cdot \cdots \cdot \left( \sum_{k \in [n]} p^{(1)}_{k} \right)}{ p^{(n)}_{\sigma_B(n)} \cdot \left( \sum_{k \in [n]\setminus \{\sigma_B(1), \sigma_B(2), \ldots, \sigma_B(n-2)\}} p^{(n-1)}_{k} \right) \cdot \cdots \cdot \left( \sum_{k \in [n]} p^{(1)}_{k} \right) } \cdot \frac{\mathbb{P}_{\sigma_A \sim \text{GS}}(\sigma_A)}{\mathbb{P}_{\sigma_B \sim \text{GS}}(\sigma_B)}\]

That’s a pretty hairy equation, so let’s simplify with a bit of new notation: let $S_\sigma^{(t)} = [n]\setminus \{\sigma(1), \sigma(2), \ldots, \sigma(t)\}$ for $t\in\{0,1,\ldots, n\}$. Just to clarify:

\[S^{(n)}_\sigma = \emptyset, \ S^{(n-1)}_\sigma = \{ \sigma(n) \},\ \ldots, \ S^{1}_\sigma = [n] \setminus \{\sigma(1)\},\ S^{(0)}_\sigma = [n]\]

and $\left\vert S^{(t)}_\sigma\right\vert = n-t$. Moreover, let’s call the ratio

\[R^{(t)} = \frac{\sum_{k\in S^{(t -1 )}_{\sigma_A}} p_k^{(t)} } {\sum_{k\in S^{(t -1 )}_{\sigma_B}} p_k^{(t)}}\]

for $t \in \{1,\ldots, n\}$. We can now have the more succinct

\[\frac{\mathbb{P}\{\sigma_A \text{ wins}\}}{\mathbb{P}\{\sigma_B \text{ wins}\}} = \prod^{n}_{t=1}R^{(t)} \cdot \frac{\mathbb{P}_{\sigma_A \sim \text{GS}}(\sigma_A)}{\mathbb{P}_{\sigma_B \sim \text{GS}}(\sigma_B)}\]

Now for the dodgy bit: let’s now assume that we choose the permutations $\sigma_A, \sigma_B \sim \operatorname{Uni}(\mathbb{S}_n)$, i.e. uniformly at random; and that the $p^{(i)}_j$ are independent (Assumption 1) and identically distributed on the open interval $(0,1)$. Moreover, define $\mu = \mathbb{E}[p^{(i)}_j]$. Let us try and estimate the quantity $\mathbb{E}\left[ \frac{1}{\prod^{n}_{t=1}R^{(t)}} \cdot \frac{\mathbb{P}\{\sigma_A \text{ wins}\}}{\mathbb{P}\{\sigma_B \text{ wins}\}} \right]$ as a proxy for the ratio $\frac{\mathbb{P}_{\sigma_A \sim \text{GS}}(\sigma_A)}{\mathbb{P}_{\sigma_B \sim \text{GS}}(\sigma_B)}$. We then have

\[\mathbb{E}\left[ \frac{1}{\prod^{n}_{t=1}R^{(t)}} \cdot \frac{\mathbb{P}\{\sigma_A \text{ wins}\}}{\mathbb{P}\{\sigma_B \text{ wins}\}} \right] = \mathbb{E}\left[ \prod^{n}_{t=1}R^{(t)} \cdot \frac{\mathbb{P}\{\sigma_A \text{ wins}\}}{\mathbb{P}\{\sigma_B \text{ wins}\}} \right] = \mathbb{E}\left[ \prod^{n}_{t=1}R^{(t)}\right] \cdot \frac{\mathbb{P}\{\sigma_A \text{ wins}\}}{\mathbb{P}\{\sigma_B \text{ wins}\}},\]

where we obtain the first equality by symmetry and the second from linearity of expectation.

Let’s now take the expectation value of $\prod^{n}_{t=1}R^{(t)}$. By linearity of expectation, we have that

\[\mathbb{E} \left[ \prod^{n}_{t=1}R^{(t)} \right] = \prod^{n}_{t=1} \mathbb{E} \left[ R^{(t)}\right].\]

Now,

\[\mathbb{E} \left[ R^{(t)}\right] = \frac{1}{n!} \sum_{\sigma_A \in \mathbb{S}_n} \sum_{k \in S^{(t)}_{\sigma_A}} \mathbb{E}\left[ p^{(t)}_k \right] \cdot \frac{1}{n!} \sum_{\sigma_B \in \mathbb{S}_n}\frac{1}{ \sum_{k \in S^{(t)}_{\sigma_B}} \mathbb{E}\left[ p^{(t)}_k \right]} \\ = \frac{1}{n!} \sum_{\sigma_A \in \mathbb{S}_n} \sum_{k \in S^{(t)}_{\sigma_A}} \mu \cdot \frac{1}{n!} \sum_{\sigma_B \in \mathbb{S}_n} \frac{1}{\sum_{k \in S^{(t)}_{\sigma_B}} \mu} \\ \frac{1}{n!} \sum_{\sigma_A \in \mathbb{S}_n} (n-t) \mu \cdot \frac{1}{n!} \sum_{\sigma_B \in \mathbb{S}_n} \frac{1}{(n-t) \mu} = 1.\]

So that means, under some somewhat shaky assumptions, we have

\[\frac{\mathbb{P}_{\sigma_A \sim \text{GS}}(\sigma_A)}{\mathbb{P}_{\sigma_B \sim \text{GS}}(\sigma_B)} \approx \frac{\mathbb{P}\{\sigma_A \text{ wins}\}}{\mathbb{P}\{\sigma_B \text{ wins}\}} ,\]

so sampling permutations from the GS distribution hopefully gives us samples somewhat consistent with sampling permutations with probability proportional to their winning probabilities – that is – the permutation with the highest winning probability should be sampled with the highest probability according to GS.

def greedy_sample(X, teams):
    """
        Given winning probability matrix X returns a "greedy" sample
        permutation of teams - given as a list of integers.
    """
    ## Check dims of X ok
    num_weeks = X.shape[0]
    num_teams = X.shape[1]
    if (num_teams != len(teams)):
        raise(ValueError("X needs to have %d rows, has %d." % (num_teams, X.shape[1])))
    # generate samples
    visited = []
    for t in list(range(min(num_teams, num_weeks))):
        allowed = [i for i in list(range(num_teams)) if i not in visited]
        probs = X[t, allowed] / np.sum(X[t, allowed])
        sample = np.random.choice(allowed, p=probs)
        visited.append(sample)
    return visited

Take many greedy samples and save the best

def greedy_sample_strategy(X, teams, num_iterations):
    optimal_sampled_perm = []
    optimal_value = -float("inf")
    optimal_t = 0
    for t in range(num_iterations - 1):
        if (t % 1000 == 0):
            print("Run %d of %d. Current optimal value is %.4f observed at run %d." \
                  % (t, num_iterations, optimal_value, optimal_t))
        sample = greedy_sample(X, teams)
        value = evaluate_perm(sample, X)
        if (value > optimal_value):
            optimal_value = value
            optimal_sampled_perm = sample[:]
            optimal_t = t
    return optimal_sampled_perm, optimal_value

num_iterations = 10000
greedy_opt_perm, greedy_opt_val = greedy_sample_strategy(X, teams, num_iterations)
greedy_sample_result = {"opt_perm": greedy_opt_perm, \
                        "opt_value": greedy_opt_val, \
                        "opt_team_list": team_order_from_perm_list(greedy_opt_perm, teams)\
                       }

Run 0 of 10000. Current optimal value is -inf observed at run 0.
Run 1000 of 10000. Current optimal value is -14.4627 observed at run 89.
Run 2000 of 10000. Current optimal value is -14.3029 observed at run 1518.
Run 3000 of 10000. Current optimal value is -14.3029 observed at run 1518.
Run 4000 of 10000. Current optimal value is -13.9215 observed at run 3467.
Run 5000 of 10000. Current optimal value is -13.9215 observed at run 3467.
Run 6000 of 10000. Current optimal value is -13.5204 observed at run 5081.
Run 7000 of 10000. Current optimal value is -13.5204 observed at run 5081.
Run 8000 of 10000. Current optimal value is -13.5204 observed at run 5081.
Run 9000 of 10000. Current optimal value is -13.5204 observed at run 5081.

pp.pprint(greedy_sample_result, compact=True)

{'opt_perm': [11, 10, 9, 7, 16, 18, 2, 12, 8, 15, 6, 17, 3, 5, 19, 4, 14, 13, 0,
              1],
 'opt_team_list': ['Leicester City', 'Manchester United', 'Chelsea',
                   'AFC Bournemouth', 'West Bromwich Albion', 'West Ham United',
                   'Everton', 'Stoke City', 'Arsenal', 'Watford', 'Southampton',
                   'Sunderland', 'Hull City', 'Middlesbrough', 'Liverpool',
                   'Manchester City', 'Tottenham Hotspur', 'Swansea City',
                   'Burnley', 'Crystal Palace'],
 'opt_value': -12.625602276029568}

So it looks like greedy sampling has given us better results than the pure strategy! Can we do better though?

Simulated Annealing Strategy

Simulated annealing (SA) is a popular heuristic strategy for th travelling salesman problem and other combinatorial optimisation problems. The basic idea is to model the solution space of our problem as the physical states in a thermodynamical system. Higher cost solutions correspond to higher energy states and vice versa. We allow the system to ‘evolve’ under this thermodynamic model, while decreasing the virtual temperature, in analogy to a thermal annealing process.

We start at an inverse temperature $\beta_i$ and finish at $\beta_f$. Dynamics is modelled as a Markov chain with transition probabilities $\mathbb{P}(\sigma \to \sigma') = \min\{1,\ \exp(- \beta(\sigma - \sigma'))\}$. The temperature $\beta$ changes according to some fixed schedule. For the Markov chain we require a number of ‘burn-in’ steps to allow the chain to mix.

def metropolis_sample(X, teams, burn_in, start_perm, beta):
    """
    Returns a Metropolis-sampled permutation after `burn_in` steps, starting from `start_perm`.
    Acceptance probability ratio is winning probability ratio, beta is inverse temperature.
    """
    ## Check dims of X, start_perm ok
    num_weeks = X.shape[0]
    num_teams = X.shape[1]
    if (num_teams != len(teams)):
        raise(ValueError("X needs to have %d rows, has %d." % (num_teams, X.shape[1])))
    if( not (sorted(start_perm) == list(range(len(start_perm))))):
        raise(ValueError("start_perm doesn't contain all integers from 0 to %d" % (len(start_perm) -1 )))
    
    out_perm = start_perm[:]
    out_val = evaluate_perm(out_perm, X)
    for t in range(burn_in):
        idx1, idx2 = np.random.choice(len(teams), 2, replace=False)
        candidate_perm = out_perm[:]
        candidate_perm[idx1], candidate_perm[idx2] = candidate_perm[idx2], candidate_perm[idx1]
        candidate_val = evaluate_perm(candidate_perm, X)
        ratio = np.exp(candidate_val) / np.exp(out_val)  # evaluate_perm() is log of winnning probability
        
        accept = False
        if (ratio >= 1):
            accept = True
        else:
            accept = np.random.rand() <= np.exp(- beta * ratio)

        if (accept):
            # print("accept", t, candidate_val, out_val, (np.exp(candidate_val) / np.exp(out_val)), out_perm)
            out_perm = candidate_perm[:]
            out_val = candidate_val
            
    return out_perm, out_val        

def simulated_anneal_sample(X, teams, burn_in, start_perm, beta_i, beta_f, alpha):
    """
    Return a permutation sample according to the simulated annealing algorithm, 
    starting at inverse temperature `beta_i` and stopping at `beta_f`. Beta geometrically
    increases, by a factor `alpha` at each step, `burn_in` runs at each temperature.
    """
    current_perm = start_perm
    beta = beta_i
    while (beta <= beta_f):
        beta = alpha * beta
        out_perm, out_val = metropolis_sample(X, teams, burn_in, current_perm, beta)
        current_perm = out_perm[:]
    return out_perm, out_val    

def simulated_anneal_sample_strategy(X, teams, burn_in, start_perm, beta_i, beta_f, alpha, num_runs):
    """
    Return the permutation giving the maximum winning probability after `num_samples`
    SA samples.
    """
    optimal_perm = start_perm[:]
    optimal_val = evaluate_perm(optimal_perm, X)
    for t in range(num_runs):
        if (t % 5 == 0):
            print("Run %d of %d. Current optimal value is %.4f." \
                  % (t + 1, num_runs, optimal_val))
        sample_perm, sample_val = simulated_anneal_sample(X, teams, burn_in, start_perm, beta_i, beta_f, alpha)
        if (sample_val > optimal_val):
            optimal_val = sample_val
            optimal_perm = sample_perm[:]
    return optimal_perm, optimal_val    

Tune the hyperparameters

Naturally, we don’t just want to guess at the hyperparamters in the SA strategy, we need to select them somehow.

Here we’re going to use a very naive method: grid search. Let’s fix some of the hyperparameters to sensible values to not have to test too many combinations. These include the final inverse temperature $\beta_f= 50$ and the burn_in=200. We’ll set $\beta_f = 50$ because $e^{-50}$ is tiny so more or less zero acceptance probability for any transition lowering the energy and burn_in=200 since it is roughly the number of possible transitions from a given state $n(n-1)/2$. We’ll use the greedy stategy as a starting state.

We are going to vary the initial inverse temperature $\beta_i$ and the multiplier, $\alpha$.

We’ll give each of the $(\beta_i, \alpha)$ pairs 25 goes to show us what they’re made of. The best pair will then be used for more runs to give us the SA strategy generated permutation.

beta_f = 50
burn_in = 200
start_perm = greedy_perm
num_runs = 25

alphas = [1.02, 1.05, 1.1, 1.15, 1.2]
beta_is = [0.001, 0.005, 0.01, 0.1, 0.2, 0.3, 0.5]

SA_hyperparam_results = pd.DataFrame(columns = ['alpha', 'beta_i', 'SA_opt_val', 'SA_opt_perm'])

for alpha, beta_i in itertools.product(alphas, beta_is):
    print("Testing alpha = %.3f, beta_i = %.4f " % (alpha, beta_i))
    SA_opt_perm, SA_opt_val = \
            simulated_anneal_sample_strategy(X, teams, burn_in, start_perm, beta_i, beta_f, alpha, num_runs)
    SA_hyperparam_results = SA_hyperparam_results.append({'alpha': alpha, 'beta_i': beta_i, 'SA_opt_val': SA_opt_val, 'SA_opt_perm': SA_opt_perm},\
                                ignore_index=True)    

... lots of output ...

Whoa, that took a while! But it seems to have paid off

SA_hyperparam_results.iloc[SA_hyperparam_results['SA_opt_val'].idxmax()]

alpha                                                        1.2
beta_i                                                       0.5
SA_opt_val                                              -10.4662
SA_opt_perm    [0, 13, 4, 5, 11, 19, 2, 6, 18, 15, 7, 17, 3, ...
Name: 34, dtype: object

beta_f = 50
burn_in = 200
start_perm = greedy_perm
num_runs = 200
alpha = 1.2
beta_i = 0.5

SA_opt_perm, SA_opt_val = \
        simulated_anneal_sample_strategy(X, teams, burn_in, start_perm, beta_i, beta_f, alpha, num_runs)

... lots of output ...

SA_sample_result = {"opt_perm": SA_opt_perm, \
                    "opt_value": SA_opt_val, \
                    "opt_team_list": team_order_from_perm_list(SA_opt_perm, teams)\
                    }
pp.pprint(SA_sample_result, compact=True)

{'opt_perm': [4, 13, 2, 0, 11, 19, 8, 6, 18, 15, 7, 17, 3, 12, 1, 14, 5, 10, 9,
              16],
 'opt_team_list': ['Manchester City', 'Swansea City', 'Everton', 'Burnley',
                   'Leicester City', 'Liverpool', 'Arsenal', 'Southampton',
                   'West Ham United', 'Watford', 'AFC Bournemouth',
                   'Sunderland', 'Hull City', 'Stoke City', 'Crystal Palace',
                   'Tottenham Hotspur', 'Middlesbrough', 'Manchester United',
                   'Chelsea', 'West Bromwich Albion'],
 'opt_value': -10.540821987149268}

Convex Programming Strategy

The SA strategy did pretty well. There is one more direction of attack that I’d like us to go down, so let’s investigate.

Theory

Let $\widetilde{X}$ be the matrix with entries given by

\[[\widetilde{X}]_{i,j} = \ln X_{i,j} = \ln p^{(i)}_j\]

Now consider the optimisation problem

\[\max_{\sigma \in \mathbb{S}_n} \operatorname{Tr}(P_{\sigma} \widetilde{X} ), \tag{$\mathsf{X}$}\]

where $P_\sigma$ is the permutation matrix assoiated to the permutation $\sigma$, with elements $[P_\sigma]_{i,j} = \delta^{\sigma(i)}_j$. We’ll call this problem $\mathsf{X}$ with optimal value $\operatorname{OPT}(\mathsf{X})$.

The solution to $\mathsf{X}$ is also the permutation with maximum winning probability! Why?

\[\operatorname{Tr}(P_{\sigma} \widetilde{X} ) = \sum_{i,j} [P_{\sigma}]_{i,j} [\widetilde{X}]_{i,j} = \sum_{i,j} \delta^{\sigma(i)}_j \ln p^{(i)}_j = \sum_i \ln p^{(i)}_{\sigma(i)} = \ln \left( \prod_i p^{(i)}_{\sigma(i)} \right) = \ln \left( \mathbb{P}(\sigma \text{ wins}) \right).\]

Since $\ln(\,\cdot\,)$ is a monotonic function, maximising $\ln \left( \prod_i p^{(i)}_{\sigma(i)} \right)$ maximises $\prod_i p^{(i)}_{\sigma(i)} = \mathbb{P}(\sigma \text{ wins})$.

“So what?” one might think – we still have a search space of permutations on $n$ elements with size $n!$ – all we’ve done is rewrite the original problem using matrices. However, writing the problem this way suggests a new means of attack.

Let’s try taking the convex relaxation of $\mathsf{X}$, calling it $\mathsf{convX}$:

\[\max_{P \in \mathcal{B}_n} \operatorname{Tr}(P \widetilde{X} ), \tag{$\mathsf{convX}$}\]

We have a new object here, $\mathcal{B}_n$, which is known as the Birkhoff polytope. It is the convex hull of all $n \times n$ permutation matrices. This means that every $P\in \mathcal{B}_n$ is a doubly-stochastic matrix, a matrix where every element $[P_{i,j}]\in[0,1]$ and all of its rows and columns sum to $1$.

Let’s think about $\mathsf{convX}$ for a second here: we are optimising a linear function of nonnegative real variables subject to linear constraints; this is just a linear program! We can leverage the theory and algorithms for solving linear programs to help us find a good solution to $\mathsf{X}$.

Theorem. There is an optimal solution to $\mathsf{convX}$. Moreover, there is an optimal solution at an extreme point of $\mathcal{B}_n$.

Proof. The set $\mathcal{B}_n$ is compact and $\mathsf{convX}$ is a maximisation problem over $\mathcal{B}_n$, so there is a solution to $\mathsf{convX}$, call it $P^\star$. We can write $P^\star = \sum_{\sigma \in \mathbb{S}_n}\alpha^\star_\sigma P_\sigma$ from the definition of $\mathcal{B}_n$, where $\alpha^\star_\sigma \in [0,1]$ and $\sum_{\sigma \in \mathbb{S}_n} \alpha^\star_\sigma = 1$. By linearity we have $\operatorname{OPT}(\mathsf{convX}) = \operatorname{Tr}(P^\star \widetilde{X}) = \sum_{\sigma \in \mathbb{S}_n} \alpha^\star_\sigma \operatorname{Tr}(P_\sigma \widetilde{X})$. There is no $\sigma \in \mathbb{S}_n$ such that $\operatorname{Tr}(P_\sigma \widetilde{X}) > \operatorname{OPT}(\mathsf{convX})$, otherwise $P^\star$ is not optimal. Now, consider the set $S^\star = \left\{\sigma \in \mathbb{S}_n \, \middle\vert \, \operatorname{Tr}(P_\sigma \widetilde{X}) = \operatorname{OPT}(\mathsf{convX}) \right\}$. The set $S^\star$ is non-empty from the following: suppose $S^\star = \emptyset$. Then, $P^\star$ is a convex sum of terms $P_\sigma$ where each $\operatorname{Tr}(P_\sigma \widetilde{X}) < \operatorname{OPT}(\mathsf{convX})$, giving $\operatorname{Tr}(P^\star\widetilde{X}) < \operatorname{OPT}(\mathsf{convX})$ from linearity; but $\operatorname{OPT}(\mathsf{convX}) = \operatorname{Tr}(P^\star \widetilde{X})$ by definition. Thus we have that $P^\star = \sum_{\sigma \in S^\star}\alpha^\star_\sigma P_\sigma$, where $\alpha^\star_\sigma \in [0,1]$ and $\sum_{\sigma \in S^\star} \alpha^\star_\sigma = 1$. Clearly, any $P_\sigma$ for $\sigma \in S^\star$ is an optimal solution to $\mathsf{convX}$ and also an extreme point of $\mathcal{B}_n$, by definition.

Corollary. $\operatorname{OPT}(\mathsf{X}) = \operatorname{OPT}(\mathsf{convX})$.

This is great news! We can find an extremal solution to $\mathsf{convX}$ and we will get the solution to $\mathsf{X}$ – an optimal permutation. Since $\mathsf{convX}$ is a linear program we can solve it quickly. Moreover, typical LP solvers usually search through extreme points and give them as a solution, so we will most likely not need to work hard to project the returned $P^\star$ to the nearest extreme point of $\mathcal{B}_n$.

Implementation

X_tilde = np.log(X)[0:len(teams), 0:len(teams)]

P = cvx.Variable(*X_tilde.shape)
W = cvx.Parameter(*X_tilde.shape, sign="negative")
W.value = X_tilde
obj = cvx.Maximize(cvx.trace(W*P))

constraints = [0 <= P, P <= 1]

col_sums = cvx.sum_entries(P, axis=0) 
row_sums = cvx.sum_entries(P, axis=1)

for col_sum in col_sums:
    constraints += [col_sum == 1]

for row_sum in row_sums:
    constraints += [row_sum == 1]

prob = cvx.Problem(obj, constraints)
prob.solve()

print("status:", prob.status)
print("optimal value", prob.value)

status: optimal
optimal value -10.337805564701268

We have optimal value! Let’s take a look at the returned $P$ matrix.

from matplotlib.colors import LogNorm

plt.figure(figsize=(12, 4))
plt.subplot(1, 2, 1)
plt.matshow(P.value.T, cmap="binary", fignum=0)
plt.colorbar();
plt.title("optimal $$P$$\n");

plt.subplot(1, 2, 2)
plt.matshow(P.value.T, cmap="binary", fignum=0, norm=LogNorm(vmin=1e-12, vmax=1))
plt.colorbar();
plt.title("optimal $$P$$, log scale\n");

We see that it’s not quite a $\{0,1\}$ permutation matrix, so let’s round it and get the corrsponding team list.

rounded = np.rint(np.asarray(P.value.T))

# check if permutation
assert(np.all(np.sum(rounded, 0)) == True)
assert(np.all(np.sum(rounded, 1)) == True)

# convert permutation matrix to perm_list
cvx_opt_perm = list(np.where(rounded == 1)[1])

cvx_result = {"opt_perm": cvx_opt_perm, \
              "opt_value": evaluate_perm(cvx_opt_perm, X), \
              "opt_team_list": team_order_from_perm_list(cvx_opt_perm, teams) \
              }

pp.pprint(cvx_result, compact=True)

{'opt_perm': [1, 13, 4, 5, 11, 19, 2, 6, 8, 15, 7, 17, 3, 12, 9, 14, 18, 10, 0,
              16],
 'opt_team_list': ['Crystal Palace', 'Swansea City', 'Manchester City',
                   'Middlesbrough', 'Leicester City', 'Liverpool', 'Everton',
                   'Southampton', 'Arsenal', 'Watford', 'AFC Bournemouth',
                   'Sunderland', 'Hull City', 'Stoke City', 'Chelsea',
                   'Tottenham Hotspur', 'West Ham United', 'Manchester United',
                   'Burnley', 'West Bromwich Albion'],
 'opt_value': -10.337805543989093}

Strategy comparison

Let’s now compare the different strategies once and for all.

result_df = pd.DataFrame({"greedy": greedy_strategy_result, "GS": greedy_sample_result, "SA": SA_sample_result, "cvx": cvx_result})
result_df

	GS	SA	cvx	greedy
opt_perm	[11, 10, 9, 7, 16, 18, 2, 12, 8, 15, 6, 17, 3,...	[4, 13, 2, 0, 11, 19, 8, 6, 18, 15, 7, 17, 3, ...	[1, 13, 4, 5, 11, 19, 2, 6, 8, 15, 7, 17, 3, 1...	[4, 14, 9, 8, 11, 19, 10, 6, 18, 15, 7, 2, 16,...
opt_team_list	[Leicester City, Manchester United, Chelsea, A...	[Manchester City, Swansea City, Everton, Burnl...	[Crystal Palace, Swansea City, Manchester City...	[Manchester City, Tottenham Hotspur, Chelsea, ...
opt_value	-12.6256	-10.5408	-10.3378	-13.507

np.exp(result_df["cvx"]['opt_value'] / result_df["greedy"]['opt_value'])

2.1497772399422357

We have that the convex strategy wins out over the naive greedy strategy with more than twice the winning probability - and is computationally efficient!

Next time, we’ll back test these two strategies against one another to see how they differ on a more realistic setting. This also constitutes a test of the independence assumption we used at the beginning.

Markov decision processes and life - why we are all doomed by physics to make bad decisions

Tue, 01 May 2018 16:24:23 +0000

$\newcommand{\P}{\mathsf{P}}$ $\newcommand{\BQP}{\mathsf{BQP}}$ $\newcommand{\PSPACE}{\mathsf{PSPACE}}$

In life it seems as if it’s always tough to make the right decision, or at least it’s often difficult to do so due to some known or unknown incurred cost. Many a time the palm of my hand has swung lugubriously to my forehead after some blunder I’d like to forget about (usually it’s something I have forgotten about…). Alas, maybe there is hope, or at least consolation. Perhaps this isn’t due to some personal deficiency, but in fact follows inevitably from the laws of the universe, which conspire to make us bad decision-makers. In this post we will think about this question through the lens of Markov decision processes and, with some shaky reasoning, once and for all absolve ourselves of all responsibility for any of the poor decisions we make.

In a Markov decision process $M$ we are given a finite set of states $S$, and we start with a state $s_0 \in S$. Accordingly, at a time $t \in \mathbb{N}$ we are in the state $s_t \in S$. For every $s \in S$ we are allowed a finite set of decisions $D_s$. A particular decision $i\in D_s$ made at time $t$ incurs a cost $c(s, i ,t)$, and our next state $s’$ is decided randomly, according to a probability distribution $p(s, s’, i ,t)$. A policy $\delta: S \times \mathbb{N} \to \cup_s D_s$ is a mapping from a given state $s$ and time $t$ to a decision $\delta(s,t) \in D_s$.

The problem we are interested in solving is minimising the expectation of the total cost $\sum_{t=0}^T c(s_t, \delta(s_t, t), t )$ over some finite time horizon $T\in \mathbb{N}$, by finding an optimal policy $\delta$. We’ll call this problem of computing $\delta$ the finite-time Markov decision process problem (FTMDP). This problem has been known for a long time to be in $\P$, so polynomial-time solvable, via reduction to linear programming. So far this is good news, if we apply this as a model for real decision-making. However, there is a variant of this problem known as partially observed FTMDP (pFTMDP) that we are going to concern ourselves with. In this variant, at time $t$, we don’t know which state $s_t\in S$ we are in, rather we know that we lie in a subset of $S$, $z \in \Pi$, where $\Pi = {z_1, z_2, \ldots, z_k}$ is a partition of $S$, that is, the $z_i$’s are disjoint and their union is $S$.

Papadimitrou and Tsitsiklis in 1987 showed (thanks to Josh Lockhart for showing me this paper) that the computational complexity of pFTMDP is $\PSPACE$-complete. That is, unless $\P=\PSPACE$, there is no polynomial-time classical algorithm that can tell us if a particular expected cost can be achieved in the partial observation setting. Moreover, unless $\BQP=\PSPACE$ then not even a quantum computer can answer this question in polynomial time. Taken dramatically, this means that the best machines the universe has to offer are hopeless in the face of pFTMDP.

Now let us extrapolate this result wildly to the messy domain of meatspace, where we envisage a human life as some Markov decision process $M_{\text{life}}$. I will now make a lazy attempt to justify this model. Surely we are always unsure of which state precisely we are in, however we wish to define our state space $S$ (a numeric scale of 1-100, ASCII-encoded bitstring of our deepest, darkest thoughts) and cost $c$ (monetary cost, how many of our hairs fall out). We do probably know roughly which subset of states we are in (between 70 and 80 out of 100, generally feeling good, ‘I’m hungry’ etc…), with the collection of subsets corresponding to our partition $\Pi$. I think we can all at least to some extent agree that the next state of one’s life is decided at random, by some implicit probability distribution $p$.

It seems that even in the unlikely scenario that we know all of the state transition probabilities and incurred costs that it is hopeless for us to make good decisions. Why? Well we want to live our best lives, and of course that means computing an optimal policy $\delta_{\text{life}}$ for the Markov decision process $M_{\text{life}}$ described in the previous paragraph, taking $t=0$ as birth and $T$ as some finite time significantly larger than a typical lifetime (presumably moving to the state $s={\text{dead}}$ incurs a high cost). Now even asking if we can get a policy with expected total cost above or below a certain value seems to be forbidden by physics, since it’s overwhelmingly likely that $\BQP \neq \PSPACE$ and pFTMDP is $\PSPACE$-complete. So it’s curtains for producing the policy itself.

What do we take from this? I would take this as a consolation for any bad decisions taken and perhaps even as a hand-wavy excuse! I mean, if the universe wanted us to make good decisions, then surely pFTMDP would be in $\P$? Nowadays, if anyone asks why I’ve done something stupid now I simply tell them I didn’t have a $\PSPACE$ oracle to hand. Somehow that doesn’t seem to go down so well…

Is it a banger? Audio classification in tensorflow

Thu, 18 Jan 2018 12:14:01 +0000

In Parks and Recreation Season 6 Episode 18 “Prom”, Tom Haverford famously tells us about his test of whether a song is a “banger” or not. There are many questions in this test: “does it feature any acoutic instruments?”, “how many drops?”, “how dope are the drops?” etc.

I think we can make his test even more rigorous: why don’t we use a deep neural network, trained on examples of bangers (and non-bangers), to tell us if a song is banger or not?

In this jupyter notebook, we’re going to construct, train and test this neural network.

Initial Environment

import matplotlib.pyplot as plt
import librosa.display
import numpy as np
np.random.seed(1337)
import pandas as pd
%matplotlib inline

The Dataset

    df = pd.read_pickle("../data/processed_dataset.pkl")

This data set was generated using the instructions in this notebook. Let’s take a look.

    df[:9]

	audio	label	label_one_hot	log_specgram
Cliff Richard - Greatest Hits 1958-1962 (Not Now Music) [Full Album]_0415.wav	[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...	not_a_banger	[1.0, 0.0]	[[-80.0, -54.1524, -35.3907, -33.0633, -39.626...
Selected New Year Mix_0121.wav	[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...	banger	[0.0, 1.0]	[[-67.3112, -51.5708, -53.4622, -72.6484, -80....
Rihanna - Stay ft. Mikky Ekko_0036.wav	[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...	not_a_banger	[1.0, 0.0]	[[-64.2413, -50.564, -57.0061, -37.2135, -37.0...
The Lumineers - Slow It Down (Live on KEXP)_0049.wav	[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...	not_a_banger	[1.0, 0.0]	[[-80.0, -73.9336, -59.1297, -49.4456, -45.314...
Passenger _ Let Her Go (Official Video)_0016.wav	[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...	not_a_banger	[1.0, 0.0]	[[-80.0, -79.4122, -63.2455, -56.2228, -56.834...
Low Steppa - Vocal Loop (Premiere)_0032.wav	[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...	banger	[0.0, 1.0]	[[-65.6515, -31.3697, -21.9142, -25.2813, -61....
Stardust - Music Sounds Better (Mistrix Dub) (Free Download)_0049.wav	[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...	banger	[0.0, 1.0]	[[-80.0, -80.0, -78.6725, -79.2538, -80.0, -80...
Ed Sheeran - Thinking Out Loud [Official Video]_0033.wav	[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...	not_a_banger	[1.0, 0.0]	[[-80.0, -65.3586, -53.2574, -44.407, -50.1324...
Best Of 2017 Tech House Yearmix_0145.wav	[0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, 0.0, ...	banger	[0.0, 1.0]	[[-80.0, -57.0543, -39.8118, -61.7071, -38.360...

We can see in the first column the names of the tracks (in .wav format) with a numeric identifier at the end. Each track has been clipped into 5 second segments (at 22.05kHz sample rate) and the identifier tells us which segment we have.

The audio column is a numpy array with the audio sample values.

The label column tells us if the given file is labelled as a banger or not. For the most part, the labels are obvious to us (but not the machine): Ed Sheeran, The Lumineers, Cliff Richard… clearly NOT A BANGER. Various tech house mixes and artists - BANGERZ.

The label_one_hot column gives us the vectorised, “one-hot” encoding of the label. [0.0, 1.0] == banger, [1.0, 0.0] == not_a_banger.

The final column, log_specgram, is the most interesting and what will comprise our features input to the neural net. It comprises the log spectrogram of the audio signal. This is the absolute value squared Short Time Fourier Transform of the audio signal. This gives us the frequency content of the signal within short time windows.

We’re going to use a common image classification tool, a ConvNet, on the log spectrogram image to do our classification.

Let’s take a closer look at the dataset.

    bangerz = df.loc[df['label'] == "banger"]
    clangerz = df.loc[df['label'] == "not_a_banger"]
    num_bangerz = bangerz.index.size
    num_clangerz = clangerz.index.size
    
    print("Dataset has %g audio clips." % df.index.size)
    print( "This is split between %g \"banger\"s and %g \"not_a_banger\"s" % (num_bangerz, num_clangerz) )

Dataset has 875 audio clips.
This is split between 422 "banger"s and 453 "not_a_banger"s

So we are split more-or-less 50:50 between bangers and clangers. Now we want to look at the audio signal and log spectrogram for some examples.

    def plot_waveforms(df, idx):
        audio = df.iloc[idx].audio
        log_specgram = df.iloc[idx].log_specgram
        filename = df.iloc[idx].name
        label = df.iloc[idx].label
        # audio is np.array holding sample values, log_specgram is 2-dim np.array
        plt.figure(figsize=(15,6))
        plt.subplot(1, 2, 1)
        librosa.display.waveplot(audio, sr=22050)
        plt.subplot(1, 2, 2)
        librosa.display.specshow(log_specgram, x_axis='time',y_axis='log')
        plt.colorbar(format='%+2.0f dB')
        plt.suptitle(filename + ", label = \"" + label + "\".")



    [plot_waveforms(bangerz, i) for i in [0, 1, 2]];

We can see for the first two bangers sharp, rhythmic, percussive signal, focused on the low end of the frequency spectrum. This is the kick drum!

In the third banger, we are likely in a section where the producer has used a high-pass filter, since there is virtually no low-frequency content here, yet we can still see some regularity from the kick in the higher end of the spectrum.

Now for the clangers!

    [plot_waveforms(clangerz, i) for i in [0, 1, 2]];

Here we see a less percussive, rhythic signal across the board, with far less low-frequency content.

Hopefully our ConvNet will be able to use this to its advantage.

Establish baseline

We can calculate a baseline classification accuracy, if we just choose the majority label in the dataset for any example. This is the accuracy we need to beat.

Ideally, we should run “Haverford’s algorithm” and compare, but I really didn’t feel like doing this for 875 examples! Volunteers welcome…

    naive_accuracy = (max(num_bangerz, num_clangerz) / (float)(df.index.size))
    print ("This is the accuracy if we always guess max{#banger, #not_a_banger}: %.3f" % naive_accuracy)

This is the accuracy if we always guess max{#banger, #not_a_banger}: 0.518

Form the training and testing data sets¶

Let’s set aside 80% of the data for training and 20% for testing.

    train_frac = 0.8
    
    def split_train_test(df, train_frac=0.8):
        include = np.random.rand(*df.index.shape)
        is_train = include < train_frac
        train_data = df[is_train]
        test_data = df[~is_train]
        return train_data, test_data
                    
    train_data, test_data = split_train_test(df, train_frac)


    print( "Training data has %g clips, test data has %g clips." % (train_data.index.size, test_data.index.size))

Training data has 711 clips, test data has 164 clips.

Tensorflow

Having prepped the training and test datasets, we’re ready to set up our ConvNet. We will closely follow the structure of the tensorflow deep MNIST example neural net with some small modifications – if it ain’t broke, don’t fix it!

The deep network will look something like this:

We feed the image of the log spectrogram into a convolutional layer, conv1, followed by a max-pooling layer, hpool1, which reduces the size of the image. We then feed this image into another convolutional layer, conv2, followed by another max-pooling layer, hpool2, which reduces the image size further. We then have two consecutive fully connected layers, fc1 and fc2, between which we use dropout (this randomly removes edges during each epoch of training to mitigate overfitting). Finally, we classify.

We’re going to use the ADAM adaptive moment optimizer, with a cross-entropy cost function.

Setup

    import tensorflow as tf
    tf.set_random_seed(1234)


    # convolution params
    log_specgram_shape = df.iloc[0]["log_specgram"].shape
    CONV_STRIDE_LENGTH = 1
    CONV_WINDOW_LENGTH = 5
    MAX_POOL_STRIDE_LENGTH = 2
    
    # features
    CONV_1_NUM_FEATURES = 32
    CONV_2_NUM_FEATURES = 16
    DENSE_NUM_FEATURES = 256
    
    # training
    NUM_LABELS = df.label.unique().size
    BATCH_SIZE = 50
    NUM_EPOCHS = 1000
    LEARNING_RATE = 1e-4
    LOG_TRAIN_STEPS = 1

Draw the computational graph

    # This node is where we feed a batch of the training data and labels at each training step
    x = tf.placeholder(tf.float32,shape=(None, *log_specgram_shape, 1))
    y_ = tf.placeholder(tf.float32, shape=(None, len(df.label.unique())))


    # Weight initialisation functions
     
    # small noise for symmetry breaking and non-zero gradients
    def weight_variable(shape):
        initial = tf.truncated_normal(shape, stddev=0.1)
        return tf.Variable(initial)
    
    # ReLU neurons - initialise with small positive bias to stop 'dead' neurons
    def bias_variable(shape):
        initial = tf.constant(0.1, shape=shape)
        return tf.Variable(initial)


    def conv2d(x, W):  
        return tf.nn.conv2d(x, W, strides=[1, CONV_STRIDE_LENGTH, CONV_STRIDE_LENGTH, 1], padding='SAME')
    
    # ksize is filter size
    def max_pool_2x2(x):
        return tf.nn.max_pool(x, ksize=[1, MAX_POOL_STRIDE_LENGTH, MAX_POOL_STRIDE_LENGTH, 1],
                            strides=[1, MAX_POOL_STRIDE_LENGTH, MAX_POOL_STRIDE_LENGTH, 1], padding='SAME')

First Convolutional Layer

We can now implement our first layer. It will consist of convolution, followed by max pooling. The convolution will compute CONV_1_NUM_FEATURES features for each CONV_WINDOW_LENGTH $\times$ CONV_WINDOW_LENGTH patch. Its weight tensor will have a shape of [CONV_WINDOW_LENGTH, CONV_WINDOW_LENGTH, 1, CONV_1_NUM_FEATURES]. The first two dimensions are the patch size, the next is the number of input channels (mono audio, so 1), and the last is the number of output channels. We will also have a bias vector with a component for each output channel.

    W_conv1 = weight_variable([CONV_WINDOW_LENGTH, CONV_WINDOW_LENGTH, 1, CONV_1_NUM_FEATURES])
    b_conv1 = bias_variable([CONV_1_NUM_FEATURES])


    h_conv1 = tf.nn.relu(conv2d(x, W_conv1) + b_conv1)
    h_pool1 = max_pool_2x2(h_conv1)

Second Convolutional Layer

    W_conv2 = weight_variable([CONV_WINDOW_LENGTH, CONV_WINDOW_LENGTH, CONV_1_NUM_FEATURES, CONV_2_NUM_FEATURES])
    b_conv2 = bias_variable([CONV_2_NUM_FEATURES])
    
    h_conv2 = tf.nn.relu(conv2d(h_pool1, W_conv2) + b_conv2)
    h_pool2 = max_pool_2x2(h_conv2)
    
    # 2x2 maxpool gives image dimensions np.ceil(np.array(log_specgram_shape)/2).astype(int)

Densely Connected Layer

Now that the image size has been reduced, we add a fully-connected layer with 256 neurons. We reshape the tensor from the pooling layer into a batch of vectors, multiply by a weight matrix, add a bias, and apply a ReLU activation function.

    def scale_shape_maxpool2x2(shape_tuple):
        return np.ceil(np.array(shape_tuple)/2).astype(int)
    
    log_specgram_shape_reduced = scale_shape_maxpool2x2(scale_shape_maxpool2x2(log_specgram_shape))
    
    W_fc1 = weight_variable([np.prod(log_specgram_shape_reduced) * CONV_2_NUM_FEATURES, DENSE_NUM_FEATURES])
    b_fc1 = bias_variable([DENSE_NUM_FEATURES])
    
    h_pool2_flat = tf.reshape(h_pool2, [-1, np.prod(log_specgram_shape_reduced) * CONV_2_NUM_FEATURES])
    h_fc1 = tf.nn.relu(tf.matmul(h_pool2_flat, W_fc1) + b_fc1)

Dropout

To reduce overfitting, we will apply dropout before the readout layer. We create a placeholder for the probability that a neuron’s output is kept during dropout. This allows us to turn dropout on during training, and turn it off during testing.

    keep_prob = tf.placeholder(tf.float32)
    h_fc1_drop = tf.nn.dropout(h_fc1, keep_prob)

Readout Layer

    W_fc2 = weight_variable([DENSE_NUM_FEATURES, NUM_LABELS])
    b_fc2 = bias_variable([NUM_LABELS])
    
    y_conv = tf.matmul(h_fc1_drop, W_fc2) + b_fc2

Training

Batching function

We need a function to feed in batches of data for training.

    def return_batch(df, batch_size=10):
        batch_df = df.sample(batch_size)
        x = np.vstack(batch_df["log_specgram"]).reshape(batch_df.index.size, *log_specgram_shape, 1).astype(np.float32)
        y = np.vstack(batch_df["label_one_hot"]).astype(np.float32)
        return x, y

Time logging

We want some rough idea of how long training is going to take. On my laptop it was around 14 hours! 😱

    import time
    
    def estimate_time_remaining(time_in, current_step, steps_gap, total_steps):
        current_time = time.time() - time_in
        time_per_step = current_time / steps_gap
        time_remaining = (total_steps - current_step) * time_per_step
        m, s = divmod(time_remaining, 60)
        h, m = divmod(m, 60)
        print("Approximately %d hours, %02d minutes, %02d seconds remaining." % (h, m, s))

Train and Evaluate the Model

We’re using the numerically stable tf.nn.softmax_cross_entropy_with_logits function here. This is the long part.

    cross_entropy = tf.reduce_mean(
        tf.nn.softmax_cross_entropy_with_logits(labels=y_, logits=y_conv))
    train_step = tf.train.AdamOptimizer(LEARNING_RATE).minimize(cross_entropy)
    correct_prediction = tf.equal(tf.argmax(y_conv, 1), tf.argmax(y_, 1))
    accuracy = tf.reduce_mean(tf.cast(correct_prediction, tf.float32))
    
    sess = tf.Session()
    sess.run(tf.global_variables_initializer())
    
    with sess.as_default():
        current_time = time.time()
        for i in range(NUM_EPOCHS):
            batch = return_batch(train_data, BATCH_SIZE)
            
            # logging
            if i % LOG_TRAIN_STEPS == 0:
                train_accuracy = accuracy.eval(feed_dict={x: batch[0], y_: batch[1], keep_prob: 1.0})
                print('Epoch %d, training accuracy %.3f' % (i, train_accuracy))
                estimate_time_remaining(current_time, i, LOG_TRAIN_STEPS, NUM_EPOCHS)
                current_time = time.time()
    
            train_step.run(feed_dict={x: batch[0], y_: batch[1], keep_prob: 0.5})

Note: I’ve deleted the output of the above cell to keep the notebook short.

Save model and variables

    with sess.as_default():
        saver = tf.train.Saver()
        save_path = saver.save(sess, "../data/model.ckpt")
        print("Model saved in file: %s" % save_path)

    Model saved in file: ../data/model.ckpt

Testing

Now we have a trained model, we want to test out how well it works on the test set.

    with sess.as_default():
        test_batch = return_batch(test_data, test_data.index.size)
        test_accuracy = accuracy.eval(feed_dict={x: test_batch[0], y_: test_batch[1], keep_prob: 1.0})
        print("Test accuracy: %.3f" % test_accuracy)

    Test accuracy: 0.915

Yay! We have done a lot better than the baseline of 0.518.

What’s Next?

It looks like our initial attempt with a ConvNet trained on log spectrogram data has worked well as a first attempt. However, there are a bunch of things we could think about to improve things:

Feature selection: the librosa library which generated the log spectrogram can compute a whole host of different audio features such as mel spectrogram and decompositions of the signal into percussive and melodic components.
Inspecting misclassified data: digging in to which audio clips were misclassified might give us insight into why they were misclassified. We could use this information to improve the model.
ConvNet: There are plenty of hyperparameters to tune here and even the architecture can be changed. Thinking more carefully about the structure of the input features and what design to use could help here.
Other models: perhaps another machine learning model, such as SVM or nearest neighbours classification could be more effective (and certainly would be quicker!)

Thanks for reading, and as Tommy H would say, keep it 💯.

Is it a banger? Make your own dataset

Thu, 18 Jan 2018 12:14:01 +0000

These are some brief instructions on how to make the dataset I used in the article Is it a banger?.

I’m also going to assume you have downloaded the files in the GitHub repository.

Folder structure

We want to create a directory called data, with a subdirectory for each label, e.g.

data
├── label_1
├── label_2
├──    ·
├──    ·
├──    ·
└── label_k

In each label subdirectory, we have a text-file, where each line is the URL of a YouTube track or playlist with the relevant audio data.

For the article, we simply have

data
├── banger
│   └── URL_banger.txt
└── not_a_banger
    └── URL_not_a_banger.txt

You can see the URLs used in the article at URL_banger.txt and URL_not_a_banger.txt

We then need to run the following command in the directory is_it_a_banger/scripts/

./scripts/prepare_data_files.sh data 5

where 5 is the audio segment length in seconds. Note that this script requires ffmpeg and youtube-dl to work.

After running the script, you should have in each label subdirectory a bunch of 5 second .wav audio files.

We then need the following python to generate the pandas DataFrame from the generated audio files.

Imports

    import os
    import glob
    import librosa
    import numpy as np
    np.random.seed(1234)
    import pandas as pd

Get filenames and directories

    parent_dir = '../data'
    parent_dir_contents = [os.path.join(parent_dir, dirname) for dirname in os.listdir(parent_dir)]
    sub_dirs = [filename if os.path.isdir(filename) else None for filename in parent_dir_contents]
    sub_dirs = list(filter(None.__ne__, sub_dirs))
    labels_list = [os.path.relpath(path, parent_dir) for path in sub_dirs]

Extract Features

We’re going to use the librosa library for processing the audio signal. We’ll keep the raw audio samples and compute a log spectrogram.

Note that we clip samples at the end of the audio file, as the combination of running ffmpeg earlier and resampling to 22.05kHz means the audio sample arrays don’t have uniform length.

    def extract_features(file_name, sample_rate=22050, segment_time=5, samples_to_clip=500):
        audio, sample_rate = librosa.load(file_name, sr=sample_rate)
        end_idx = (sample_rate * segment_time) - samples_to_clip # remove some end samples as not strictly uniform size
        audio = audio[0:end_idx]
        log_specgram = librosa.logamplitude(np.abs(librosa.stft(audio))**2, ref_power=np.max)
        features = {"audio": audio, "log_specgram": log_specgram}
        return features

Turn labels into ‘one-hot’ vector encoding

    def one_hot_encode(label, labels_list):
        n_labels = len(labels_list)
        one_hot_encoded = np.zeros(n_labels)
        for idx, cmp in enumerate(labels_list):
            if label == cmp:
                one_hot_encoded[idx] = 1                     
        return one_hot_encoded

Trim file list

Only include a fraction of audio files for a given track to avoid training set 1) having too many highly correlated data points, and 2) having too large a file size.

    def trim_file_list(fnames_list, p_include=1.0):
        fnames_list = np.asarray(fnames_list)
        include = np.random.rand(*fnames_list.shape)
        fnames_list = fnames_list[include < p_include]
        return fnames_list

Build DataFrame from files

    def parse_audio_files(parent_dir, sub_dirs_list, labels_list, file_ext='*.wav', p_include=1.0,\
                          sample_rate=22050, segment_time=5, samples_to_clip=500):
        data = []
        index = []
        for label_idx, sub_dir in enumerate(sub_dirs_list):
            fnames_list = glob.glob(os.path.join(sub_dir, file_ext))
            fnames_list = trim_file_list(fnames_list, p_include=p_include)
            for fname in fnames_list:
                print("Processing " + os.path.basename(fname))
                features = extract_features(fname, segment_time=segment_time, \
                                            sample_rate=sample_rate, samples_to_clip=samples_to_clip)
                label = labels_list[label_idx]
                label_one_hot = one_hot_encode(label, labels_list)
                features['label'] = label
                features["label_one_hot"] = label_one_hot
                data.append(features)
                index.append(os.path.basename(fname))
        return pd.DataFrame(data, index=index)


    df = parse_audio_files(parent_dir, sub_dirs, labels_list, p_include=0.1, segment_time=5, samples_to_clip=1100)
    df = df.iloc[np.random.permutation(len(df))] # shuffle rows
    df.to_pickle(os.path.join(parent_dir, 'processed_dataset.pkl'))