Flash Boys

Michael Lewis is a prolific writer on all the dodgy dealings in the financial markets. He has authored Liar's Poker and The Big Short, which I have read and thoroughly recommend.

There is a lot to be learned by sports traders from good writers in the financial world and Michael is certainly one of those writers. His latest offering is Flash Boys, which details the nefarious goings on in High-Frequency Trading (HFT), those who benefit from it and those who fight against it.

The book tells us about how financial trading is now less about knowledge of fundamentals such as the data held in company reports but more about market structure. Knowledge of how the many electronic exchanges are linked and who is buying and selling which stocks before they are traded allows high-frequency traders to gain an advantage.

It's as though you could tap into Patrick Veitch's (see Predicting the Starting Price is NOT Possible (for us mortals)) mind to ascertain which horse he is going to perform a coup on and at what odds he has determined are fair. And, before he can get on the phone to his agents to place bets across the country, you can get online to the exchanges and back the horse all the way down to fair odds.

Naturally, that would make Patrick very angry and his ability to trade would be over. And yet, that is what happens in the financial markets everyday. From individuals wanting to play the market for a few hundred shares to the big funds wanting to trades hundreds of thousands of shares, the HFT firms get wind of this activity in advance and manipulate the market to make people pay more to buy and expect less when selling.

How is all this achieved? Again, it is the 'rocket scientists' at work. Since floor trading was superseded by electronic exchanges the market has fragmented. There are many more exchanges and big banks have their own private exchanges too called Dark Pools (see Dark Pools: The rise of A.I. trading machines and the looming threat to Wall Street). There is a time lag between each of these exchanges (depending on how far you are from each of them) as the fibre optic cable linking you to any one exchange might take a circuitous route.

To make a big trade means that you have to split an order up and spread it across many exchanges to find enough buyers or sellers. Of course, the exchange closest to your servers is going to get your first order but then that alerts the high-frequency traders so they cancel their offers on other exchanges and re-offer at a higher or lower price accordingly. The HFT firms do this to extort (in some people minds) a middleman fee from every trade. Instead of one entity buying from another, the HFT firms interject themselves and buy what is wanted from the seller and sell it to the buyer at an inflated price.

From analysts calculating fair value for a stock we now have computer programmers building bots using speed and cunning to extract fractions of a penny, many thousands of times a second. As they say, "Look after the pennies and the pounds will look after themselves."

This book is a good read on taking advantage of market structure. Something for the bot building sports trader to learn from.

Amazon - Flash Boys

Calculated Bets: Computers, Gambling, and Mathematical Modeling to Win

Steven Skiena of Stony Brook University has a lifelong passion for the sport of Jai Alai, a ball game from the Basque region of Spain. Akin to squash, the game is played with baskets attached to the hand rather than with racquets held by the hand. The book details Steven's building of a model for the game of Jai Alai with which he used for betting

The book explains the rules of Jai Alai and how betting is incorporated into the game. It is important to understand how betting and a sport's rules are related, which can reveal quirks that can be utilised for arbitrage.

In the following chapter Steven builds a Monte Carlo simulation of Jai Alai so that he can test his theories about the likelihood of any one particular player in a matchup winning the event. With 8 players vying to be the best there is an advantage in being one of the first two paired players to play. Steven's simulation proves this hypothesis.

Using available player ratings Steven builds a model for each player and for the payoffs for various kinds of Pari-Mutuel bets. He then goes on to build an automated betting system, which was rather sophisticated for its time. Finally, Steven discusses how to approach other sports and gambling games using his methodology.

All-in-all this book is a fascinating read. Some find it hard going but I can appreciate all the hard work that Steven has put into mastering modelling a sport and betting on it. If you want to know what it takes to model a sport and build a bot to place bets then you need this book at hand. You won't learn how to program specifically for your requirements and this book was written before Betfair took off. However, you will learn what it takes to dedicate yourself to a related task.

Amazon - Calculated Bets: Computers, Gambling, and Mathematical Modeling to Win

Read My Articles in SmartSigger Magazine

I have been invited to write articles for SmartSigger, a magazine dedicated to bringing you the best articles in professional betting.

My first article is entitled Trading Bots: An Introduction and introduces the newcomer to bots, their use in sports betting and how to program a simple bot. I have pitched the article at beginners to bots and programming, and I hope anyone reading the article will have a go.

The article requires no prior knowledge, just a desire to learn. The reader will be shown how to install Microsoft's free Visual Studio programming environment and produce a simple bot that grabs and updates Betfair charts.

In future articles, I will give an in-depth introduction to Betfair's new API-NG and how to use it to program bots of increasing complexity. Again, the articles will assume no prior knowledge and I hope you enjoy reading them. I will continue to write this blog and hope to bring you additional articles to the ones I write for SmartSigger.

Subscription - SmartSigger Magazine (first issue free)

Craft Betfair JSON Requests with the Aid of Google Chrome

Even with Betfair's API-NG documentation at hand it can be difficult to understand exactly how a well-formed JSON request should look like. By using the JavaScript console on Google Chrome you can see what is expected from Betfair's servers.

Open Betfair's API-NG Visualiser (https://api-ng.betstores.com/betting) in Google Chrome and open the JavaScript console.


You then login to the server with your application key and ssoid as in the previous Betfair API-NG Tutorial #1 - Getting an application key. As you execute JSON requests with the analyser you will see JSON requests being sent to the Betfair server and responses returned in the console.

With the online JSON parser at http://json.parser.online.fr you can then see the structure of JSON requests and responses and how they look as objects. Knowing what your JSON request is supposed to look like will help you to craft your program code.


Calculating Return on Investment

ROI (Return on Investment) is a subject that comes up frequently on betting forums. How do you calculate your return? This is an important question as only then can you determine if your investment is worthwhile or not.

There are many ways of performing the calculation and each one has its pros and cons. The simplest is just the percentage increase in your bank roll at the end of each day. This is an easy calculation to perform. If your bankroll is £100 and you made £10 profit today then your ROI is simply

ROI = Today's Profit / (Today's Bankroll - Yesterday's Bankroll)

In our example this would be 10 / (110 - 100) = 10%.

The problem with this is that it does not take into account your investment activity. You may have invested the full £100 or you may have invested £5 of your bankroll in a back bet. If the latter then you really made 200% ROI on just £5 risked.

The second way of calculating ROI is by way of turnover. This is the measure of actual money being invested. In our example, turnover was just £5 risked in a back bet. Alternatively, you may have laid a bet of £5 at odds of 3.0. Now, if you lose then you will have to pay out £10. You liability is £10 and so your turnover is also £10.

ROI = Today's Profit / Total Turnover

This brings in the concept of drawdown, the maximum possible loss from an investment. In our previous lay example, assuming you do not trade off any liability with a back bet then the maximum you can lose is £10. Your maximum possible drawdown is £10.

However, if you are a trader and you intend backing and laying the same selection then things are different. Imagine an example whereby you back a horse for £100 at 1.90. Your maximum drawdown is £100 if you forget to lay off liability and the horse loses or £99.81 if the odds go out to 1000 just before the end of the race and you remember to hedge.

Of course, you would've hedged much sooner. Let's say you had a 10 tick stop loss and will hedge at 2.00 for a loss of £5.00. What is your ROI now?

Imagine the trade was a bad one and the odds lengthened so you closed out for the £5 loss. You risked £100 to back the horse and then laid the horse for £95. Did you really invest £195? I don't believe that you did. At any one time your maximum drawdown was £100 so you lost £5 on £100 risked. An ROI of -£5.00.

This is why I use the first bet to determine my maximum drawdown. The second bet is just to close the trade and does not increase drawdown. In fact it lowers it.

With multiple horses backed and laid in a race you can calculate the volume weighted average price for all selections. Betfair does this for you in your Betting Profit and Loss report. You will see a back and lay subtotal, which takes into account that any money laid on one horse can be bet on another horse at no risk to yourself. Either the back or lay subtotal will be positive (depending, I believe, on the order in which you placed your bets). I take the positive value as my total risk for the event and divide it by the Net Market Total to give what I believe to be a good representation of return on total risk.

ROI = Net Market Total / (positive)[Back or Lay Sub Total]

Betfair API-NG Tutorial #2 - Logging in to Betfair

The second part of this tutorial regards logging into Betfair and performing a simple request using VB.NET. By now we will have created our application key and are now ready to make our first JSON request.

We will now use the application key and username/password combination to login, get an ssoid and then perform a getAccountFunds request.

Using JSON is a much simpler task than using Betfair API 6.0's SOAP requests. So much so that this tutorial encapsulates virtually everything we need to know about Betfair NG.

As the author's applications are a hybrid of human interaction with bot assistants they use a manual login. In future this will allow for 2-factor authentication. The alternative is a certificate based authentication, which the author is not familiar with and won't be covering.

Also, because we don't want to give away any specifics about our applications and that we expect the reader to be a coder porting from API 6.0 to API NG then we will just give an outline of how to go about interacting with Betfair via API NG

We use a child form to login to Betfair from the main form's Load handler. We then call the login form with LoginForm.Show(). The class for the LoginForm is as follows and consists of a single handler for a WebBrowser component.

Public Class LoginForm

    Public ssoid As String

    Private Sub WebBrowser1_DocumentCompleted(ByVal sender As System.Object, ByVal e As System.Windows.Forms.WebBrowserDocumentCompletedEventArgs) Handles WebBrowser1.DocumentCompleted

        Dim CookiesArr As String() = WebBrowser1.Document.Cookie.Split(";")

        If WebBrowser1.Document.Cookie.Count > 0 Then  ' check for cookies
            For Each Cookie In CookiesArr
                Dim Idx As Long = Cookie.IndexOf("=")
                If Idx <> -1 Then
                    Dim CookieName As String = Cookie.Substring(0, Idx).Trim
                    Dim CookieValue As String = Cookie.Substring(Idx + 1).Trim

                    If CookieName = "ssoid" Then  ' look for ssoid
                        ssoid = CookieValue
                    End If

                End If
            Next
        End If

        If Not ssoid = "" Then
            MainWindow.ssoid = ssoid  ' copy ssoid to main form variable
           
MainWindow.Print("Login Successful " & ssoid)
           
MainWindow.initialise()  ' starts the ball rolling on the main form
            Me.Close()  ' login window no longer required
        End If

    End Sub


End Class


The webBrowser component is pre-initialised with the URL property set to https://identitysso.betfair.com/view/login which is a Betfair login screen (see next image). We enter our username and password into the login form and when authenticated we receive a cookie containing the ssoid.


With the ssoid we are now ready to make a JSON getAccountFunds request. The heart of API NG is the JSON request. JSON standards for JavaScript Object Notation and is a simple text format for passing objects over the Internet.

Betfair has two JSON servers, one for betting requests and the other for accounting requests. We shall target the accounting server at https://api.betfair.com/exchange/account/json-rpc/v1/ to make the getAccountFunds request.

The following function makes an accounting request to the accounting server. The function takes one variable, which is our chosen request method. In this case we are using the getAccountFunds method. All accounting and betting requests can be seen at https://api.developer.betfair.com/services/webapps/docs/display/1smk3cen4v3lu3yomq5qye0ni/Accounts+Operations

This function is genericised so that we can send any accounting request to the server. We will do the same with the sendBetReq in future tutorials.

The only difference between any application is the application's key and runtime ssoid. We can hard code the application key, as has been done so. Every ssoid will be different everytime we login. These two items authenticate our requests to the server.

The following code fragments are placed into our main Window class.

    Private Function sendAccReq(ByVal method As String)

        Dim request As WebRequest = WebRequest.Create("https://api.betfair.com/exchange/account/json-rpc/v1/")
        Dim postData As String = "{""jsonrpc"": ""2.0"", ""method"": ""AccountAPING/v1.0/" & method & """}"
        Dim byteArray As Byte() = Encoding.UTF8.GetBytes(postData)

        request.Method = "POST"
        request.ContentType = "application/json"
        request.ContentLength = byteArray.Length
        request.Headers.Add("X-Application: YOUR APP KEY HERE")
        request.Headers.Add("X-Authentication: " & ssoid)

        Dim dataStream As Stream = request.GetRequestStream()
        dataStream.Write(byteArray, 0, byteArray.Length)
        dataStream.Close()

        Dim response As WebResponse = request.GetResponse()
        dataStream = response.GetResponseStream()

        Dim reader As New StreamReader(dataStream)
        Dim responseFromServer As String = reader.ReadToEnd()

        reader.Close()
        dataStream.Close()
        response.Close()

        Return responseFromServer

    End Function


On calling the function above we receive a string of the following form

{"jsonrpc":"2.0","result":{"availableToBetBalance":15279.81,"exposure":0.0,"retainedCommission":0.0,"exposureLimit":-5000.0}}

and is a JSON object, which now needs to be parsed into an object class as below (this needs to go into a class module of its own)

'Classes for getAccountBalance

Public Class AccountBalanceClass
    Public jsonrpc As String
    Public result As AccountBalanceResultClass
End Class

Public Class AccountBalanceResultClass
    Public availableToBetBalance As Decimal
    Public exposure As Decimal
    Public retainedCommission As Decimal
    Public exposureLimit As Decimal
End Class


VB.NET has a JSON serialiser which we can instaniate with the following code

Dim jss As New JavaScriptSerializer() 'JSON serialiser

Stick it at the top somewhere, outside of a subroutine. The author is a messy programmer with plenty of global variables and little care for object oriented programming. He began programming in the early 80s and has no time for new things. He just wants to code and do good experiments.

A simple subroutine that sends a getAccountBalance request to the accounting server is as follows

Public Sub getAccountFunds()

        Dim accBal As AccountBalanceClass = jss.Deserialize(Of AccountBalanceClass)(sendAccReq("getAccountFunds"))
        Label1.Text = "Balance: £" & accBal.result.availableToBetBalance & "   Exposure: £" & accBal.result.exposure


End Sub

As we can see sending and receiving objects is very simple with JSON. When we receive an object from the server it is just a simple case of using a serialiser to load the plain text object into a VB.NET object from where we can access it in our program.

Free Computational Investing Course

I happened upon a computational investing course during a trawl for quant websites. I haven't looked at all of the course but it looks rigorous.

The course is run by Georgia Tech and the lectures are given by an ex-quant who worked for a hedge fun and is now a professor at the aforementioned university.

Although the course is directed towards the financial markets, the methodology is the same for anyone wishing to use computational methods for automated sports trading.

You will have to watch videos, do some programming and answer a few questions. Ideal for someone starting out in automated trading or for someone wanting to learn a little more.

It can't hurt to dip your toe in and see what it's like. Eventually, I shall do the whole course, when I have time to spare and will let you know what I think of it.

Computational Investing, Part I