Windows 10 Upgrade

I have taken the plunge and upgraded to Windows 10. One does have one's limit and after six months of messaging from Microsoft, after each morning's boot-up, I finally clicked the upgrade button. I lost a few hours of my day and updates have been uploading since.

The upgrade was left for about half a year whilst those braver than I found the majority of bugs and reported them to Microsoft so that updates could be implemented. After two days of using Windows 10, I feel as though the operating system runs more smoothly than Windows 7, although I might be imagining that. Maybe there is some psychological reason for doing so.

With many sales of Programming for Betfair, over the Christmas period, I assumed that people would also be upgrading their hardware. Therefore, I wanted to know if I would have to update the code in the book to accommodate the new operating system. Last night I ran the code in Programming for Betfair and discovered just the one bug, which has now been rectified and added to the addenda. Current owners of the book will find it in the usual place.

I have started coding a new version of the EDDIE software, which I created in the mid-90s. Called "EDDIE Baby" (a Mars Bar will be posted to the first person who can tell me why,) the platform will incorporate both Genetic Algorithm and Genetic Programming in an ingenious way. EDDIE Baby will provide algo-traders using Programming for Betfair with an automated design process for trading rule creation.

After a few revisions, I think my next book will be less ambitious than I had originally planned and will just be a book about artificial intelligence methods (I might add Monte Carlo too) for trading on Betfair. If I try to write a longer book then it may never get published. It was always my intention to publish my thirty plus years of sports trading research. Doing so in small tranches will probably ensure that it all gets done afore I walk through the doors of the great bookie in the sky.

Million Dollar Traders

Whilst intensively researching (aka skimming through) Green All Over I was reminded of a three-part reality mini-series from 2009 called Million Dollar Traders. In the spirit of Turtle Traders, eight aspiring but decidedly amateur people were given the chance to join the hedge fund of Lex Van Dam. The videos from the series are available on YouTube (links below) and so, last night, I decided to have another look rather than watch my nightly movie. 

The team was a disparate group. One of the eight was a shop keeper and had day-traded from home, losing heavily. In addition, there was an economics student, an ex-army Major, a female entrepreneur, an ex-veterinary, a fight promoter, a retired IT worker and an eco-warrior. We can be sure they were chosen to make a good story rather than for any great ability. It's the BBC way, make good telly, the truth can wait.

We were not shown the eight being trained up, which also adds to the belief that good telly over ability was the deciding factor in choosing the team. What can you teach people in so short a time? If anything then I am sure it was all very basic. The trades placed by the eight were all haphazard punts with the exception of one trade that I will detail later.

Watching the programme again, last night, I don't believe for one moment that these amateur traders were trading with real money. The small quantities of shares they were trading just wouldn't be handled by a City brokerage. I suspect they were calling employees of Lex, who "visited" every so often in a shirt (it was not summer) from, I suspect, another room rather than another office. There were also a few telling words of familiarity between traders and brokers suggesting that the whole team met up at the end of a day of filming.

There was a lot of writing down of trades and keeping track of figures. I also suspect this was to keep a tally of the trades and in so doing proves that the trades were not real otherwise they would be electronically accessible from the brokerage.

The Wild Bunch - Million Dollar Traders

Leaving aside the fact that the trading was not for real money the process was fairly realistic. The point of a hedge fund is to buy long and sell short financial instruments whilst hedging against failure. As far as I could see in the programme there was only one hedged trade placed by the team. 

The soldier had read about a rights issue by HBOS to raise funds through issuing new stock and so he was going to buy HBOS shares in the hope that they would go up in price. To protect himself the soldier was going to short the company performing the issuance as they were liable should the issue not be fully taken up by existing HBOS share holders. In the end only 8% of the new stock was taken and HBOS fell in price. However, the shares of the company doing the issuance did not fall and so the hedge failed.

The trades by the rest of the team were pretty speculative. The retired IT worker had no clue and left the team. The ex-veterinary was too frightened of failure to either place trades or stick with a trade that momentarily went against her. The fight promoter just bet on the companies that made the products he owned. The soldier used his knowledge of the military and conflict to buy the stocks of weapon manufacturers. This was much to the annoyance of the eco-warrior who spent most of the time moaning about the evils of capitalism. At least he made the lefties in the BBC happy.

The most successful trader was the entrepreneur but we never got to see any of her trades. The shop keeper was similar to the ex-veterinary and was too risk averse to perform. The economics student understood the game and eventually became the second best trader after the entrepreneur.

To add to the drama, and that is what these shows are all about, the fight promoter, shop keeper and eco-warrior walked out when the ex-veterinary was relieved of her duties and asked to leave. In the end the only trader to make a profit was the entrepreneur with 0.5%. The soldier and the economics student lost 1% and 0.5%. Lex had lost 1% in the same time period.

What can we conclude from the series? It was filmed in 2009 whilst there was much market volatility. Lex's hedge fund had made a loss in that time and for three people to equal or better him was an achievement. The others failed simply because they just didn't get it or did not have the right psychological profile. They let past trades affect future trades and were too frightened to fail, even with play money.


Million Dollar Traders - Part 1, Part 2 and Part 3

Artificially Intelligent Financial Trading

The Rise of the Artificially Intelligent Hedge Fund, on Wired magazine's website, describes a new hedge fund running solely on AI decision making. There is no human discretion at all that can modify bad decisions, which are all made by machine learning techniques. At least, that is what the article implies but I know that all algo-trading systems have a circuit breaker for those inevitable Black Swan days.

For me the article is particularly interesting because it mentions evolutionary computation, which I describe in my recent article EDDIE Beats The Bookies, detailing my own research. The company building this AI financial trading engine is called Aidyia and uses both evolutionary computation and probabilistic logic.

The fund made 2% on its first day of trading. Most sports algo-traders would be happy with that figure. Even the majority of manual sports traders would be happy with that figure too. Other companies using AI for fund management include; Sentient Technologies, Two Sigma and Renaissance Technologies.

Many trading firms use teams of quants to build computer models to decide where to place funds but there is always a degree of human discretion involved. Removing human discretion is a natural progress as it removes emotion, human error and speeds up the trading process to get the best prices available. Other automated trading systems will be competing in the market and the first to act is the first (often the only one) to profit.

Computer (quant) models tend to be rather static and do not adapt to changes in market sentiment too well due to the human input required to update them. AI systems are adaptive and run in real time, constantly updating as markets change and evolve. Indeed, AI systems also try to predict these changes before they occur, which allow for impressive market coups.

By using evolutionary computing, a population of virtual stock traders are evaluated, the superior traders pass their genes on to a new generation of traders whilst the old one is culled. This process continues until the population of traders learn traits that adapt them to become profitable traders. Whenever these traders start losing money then it is assumed the market has changed and new virtual traders can be evolved to replace them.

In sports trading I use a similar process and create a population of sports traders that evolve traits that allow them to generalise about movements on Betfair and BetDAQs exchanges. The task is never-ending with hardware and software changes to decrease latency, increase speed and updates for the latest technology; price and order streaming being the latest.

Further Reading

Programming for Betfair is a guide to creating applications for direct access to Betfair's exchange and will therefore be useful to those wishing to implement an algorithmic trading set up using the other books listed here.

No previous programming experience is necessary to build the applications in the book. After completing the programming exercises the reader will have a powerful tool for gathering prices for database creation, strategy building and algorithmic trade placement. Beginner programmers and experienced programmers have informed me that the book is easy to understand and that it has assisted them in creating algorithmic trading platforms.

Pension Building - Fixing at the Australian Open?

Thanks to Cassini (a supposed international criminal based in California - he really isn't),  I was alerted to an article in the New York Times about possible match fixing in this year's Australian Open. In the mixed doubles tournament, Lara Arruabarrena and David Marrero were up against Andrea Hlavackova and Lukasz Kubot. For an unseeded tie in the first round the heavy betting for this match looked suspicious to Pinnacle on whose website the betting was taking place.

Pinnacle suspended the market for this match because most money was going on Andrea Hlavackova and Lukasz Kubot. I assume they had priced up the match to be close-run and would lose heavily if most money went on to the eventual winners. Hlavackova and Kubot went on to win. Arruabarrena said she was injured to  account for her team's loss.

It is almost impossible to prove any fixing in this case. If some "insiders" (people at the tennis complex; employees, coaches, friends and family in addition to bettors) realise that one team is suffering from injuries then that is enough information to move a market. That could be just as valid a reason for the betting seen as much as one team being paid to lose.

As in horse racing, insider money can move a market and in a market of two it is either going on one runner or the other. Insider betting is more obvious in tennis than in horse racing. In a horse race if you see the favourite acting up or doing something else that suggests it has no chance then you either lay that horse to lose or spread your money around a few of the other horses to win. In tennis the money can only go on one other competitor.

The case mentioned in this article appears to me to be a case of insider trading rather than a fixed match. As we know from horse racing, insider trading in sports is not illegal. It's just that bookmakers don't like being cleaned out by people who know more about an event than they do. If the shoe had been on the other foot you can be sure that Pinnacle would have kept their book open.

Liam Broady (UK tennis player) in Match fixing: Liam Broady says authorities face 'losing battle' said that stamping down on fixing/insiders was a losing battle. The prize money is very lop-sided in tennis. You can plod along and become a coach when you retire. That might be the only money you ever make from tennis because anything you made as a player went on expenses. The temptation to accept a bribe or to throw a game of your own volition, whilst getting friends to bet against you when you know you are going to lose, is very tempting as it guarantees part of your pension. A pension is very important in a person's life and is something that is often in my thoughts.

These days young people have no idea how to make provision for their retirement with most of them not having any savings at all. They spend a lot of their money on micro-payments and subscriptions for instant gratification with no care for the future. Older people too are beginning to worry that their government might have squandered their soft landing by the time they retire. A pension, no matter how far into the future, should never be too far from your mind.

Not for one moment have I considered that I am going to continue making money (in whatever form) from now on until the day I die. Judging by the deterioration in my father's health, and that a lot of predisposition to illness is passed on genetically by ones parents, then there are many dangers ahead of me. If I am not careful and/or unlucky then diabetes, failing eyesight and heart problems will mean that I will be unable to do anything to earn a living. Already, I have noticed that I am not as physically able as I once was. The memory of sitting on a road bike and effortlessly cycling up a mountain pass whilst other team members gave up is very distant indeed.

To all those younger sports traders who are starting out on their sports trading careers I issue these words of advice;

  1. You are probably not going to make as much money as you imagine you will.
  2. You will probably not even make any profit at all (90% is often quoted as a figure for the proportion of losing traders).
  3. Until you have proven without doubt that you are in the top 10% of traders, you should consider sports trading as a hobby with which to lose a little money each time for the fun of it and never to chase your losses.
  4. You are not going to be as fit and strong when you are older as you are now. Incapacity is common amongst older people. Mental and physical incapacity impairs your ability to earn money.
  5. "Gambling" related activities are not looked on by many women as a positive trait for a future husband. Many gamblers and traders live a solitary life and there will be nobody to look after you in old age other than whomever you pay to look after you.
  6. You will need a lot of money to retire on in the form of public and private pensions, in addition to savings.

I once had a friend whom I introduced to poker in London's casinos when he reached 18 years of age. He became a very good player. So good that he is has made around £1 million in career winnings. In his twenties he would think nothing of living the high-life in a London apartment. Now, he is in his thirties, married, has a child on the way and without asking him I know that he has his head screwed on and will do all in his power to make sure the life that he has is one that will be there until his dying days.

Be careful when sports trading and consider worst case scenarios if life does not go to plan.

EDDIE Beats The Bookies

At the University of Essex, during the mid 1990s, I performed research in machine learning for financial decision making. My work spawned the Centre for Computational Finance and Economic Agents, an advert for which can be seen to the right.

I recommend the courses at the CCFEA to those looking to study higher educational qualifications for a career in finance or who just have an interest in mathematical and computational applications in finance.

My research centred around genetic programming, which I introduced to the computing department at the university, although I was not the originator of the technique. Genetic programming is a field in evolutionary computation whereby a population of decision making trees are evolved using (rather like biological evolution) mutation and competition in a constrained environment to optimise a solution for a decision making problem.

To this day, every now and again, my supervising professor at Essex University contacts me about yet another member of the public asking him for information on the research that led to the paper EDDIE Beats the Bookies, which used horse race betting markets as a proxy for financial markets. Judging by the continued interest, over twenty years later, the paper has gained a cult status amongst those using machine learning applications for horse race betting.

EDDIE (Evolutionary Dynamic Data Investment Evaluator - I like inventing catchy acronyms) came out of the third-year project of my bachelor's degree. I scraped through the degree with a Desmond (Tutu - 2:2) with all my focus on that one project. I had no interest in most of the courses.

First year philosophy I enjoyed for the chance to debate with other students about machine intelligence. Computability I enjoyed because it all referenced back to the work of Alan Turing. The bottom-up AI I understood before I entered university (and is why I taught genetic programming to the university and not vice-versa). The rest of the courses were of no interest to me, including the top-down AI course, which I never liked as nature didn't create human intelligence in such a God-like manner.

After graduating I stayed in Essex to continue my research in a PhD. A colleague mentioned my work to a friend of his working for a subsidiary (Equisoft) of Reuters that happened to be in Colchester, where the university is located. I was invited, along with my professor, to talk to Equisoft about our research. Unknown to me, Equisoft was working on a project for Reuters called Reuters 3000 Xtra and wanted to add machine learning to the trading software for optimising trading systems.

Rather than offering a collaboration to Essex University, I was offered a job in a blue-skies research group within Reuters instead and so I left the PhD after a year of study. I did a little more work on an EDDIE clone for Reuters but then moved on to machine translation of financial news stories. I had the foresight to see that computers could automatically trade events without human interaction through machine translation of the news.

My thoughts went further in that I thought that the best way to translate news stories was to write them in a simplified manner that machines could understand more easily and then apply a template to the story to make it look more flowery for human consumption. Unfortunately, the people I reported to at Reuters didn't have my foresight and nothing came of it. Today, all of the big trading firms will have trading systems based around machine translation of financial news.

I also had ideas about RFID technology for tracking goods and their interactions with people in shops. Also shops that had no frontage, just a dark storage facility with robots picking items and shipping them to customers. For the 1990s I was in the wrong company for all these ideas!

Now, I find myself writing about my research. Teaching, writing and research is what I enjoy doing the most. To all those who have asked for more details about EDDIE then please wait a little longer as it will be in my forthcoming book, which will detail machine learning applications for sports trading. The book will include EDDIE and other machine learning methodologies applied to system building, trading and money management.

Acknowledgement from Betfair for my Book

I am gratified to learn that Betfair has read my book, Programming for Betfair, and has now linked to it in the Learning section of their website. The Learning section has articles teaching aspects of sports trading from various experts in their field and is well worth a read.

Also, I have received a request from Betfair with regards to collaborative work relating to the future release of market and order streaming, which will be an addition to the existing API-NG. This addition will offer great efficiencies with market changes being streamed to a user's application rather than the application having to request them.

At present a trading application has a timer which polls Betfair's servers for data updates. With streaming a user subscribes to a market and any updates are served automatically to the user's application the moment any change occurs.. For example, you can subscribe to all of the day's races and when an order is placed on a market or an order is taken on the market's order book then your application will be notified. 

Streaming offers sports traders a more CPU efficient method for accessing Betfair's exchange so that on the traders' side the CPU has more time for calculating trading algorithms and presenting data to aid decision making.

The addition of streaming to API-NG will augment the existing API and not replace it. The current API-NG will still be the core of the API and will be necessary for placing and monitoring bets. I will keep you all informed as and when I am able to.

See also

Game, Set and Fixed - Match Fixing in Tennis

I enjoy watching tennis and also playing it. Since I am no longer able to ride a bike at the level I used to in my racing days, I play tennis as an enjoyable way of staying fit. Tennis is all action, points are scored all of the time and there is a guaranteed winner at the end of every match.

I would rather watch tennis than torture myself watching a game of soccer (association football, one of the many varieties of football games). Maybe soccer is some sort of S&M activity with 89 minutes of tedium followed by a combined total of 60 seconds of ecstatic release. Mind you I can still get tied in knots watching Andy Murray playing tennis. I have only watched half the coverage of Andy's major title victories because he appears to like labouring the task. The only field game I can watch from start to finish is hurling but then coming from a Kilkenny family makes the process rather easy as we rarely lose.

For most viewers of a tennis match their only interest is seeing a good game, although they might cheer on a favoured player. Most watchers of a tennis match have no interest in wagering on the contest. They would naturally want to see a fair game so that they'd know their time was well spent watching the match. 

Currently, the men's game has two titans (one more titan than the other), another two players waning in their abilities and then we have Mr Wawrinka filling the gap between the aforementioned and the rest. There are well over a thousand professional players in the men's game who have no hope of being a tennis star. These journeymen scratch around for a living in Challenger and Futures tournaments, rarely getting the chance to play the top players in the ATP events. Equipment, training and coaching staff, travel, accommodation and nutrition have all to be paid for and so it must be very tempting to guarantee an income through fixing a game for a betting syndicate.

However, most betting is done on the higher ranked ATP tournaments because these are the only events that get televised. And yet, it is in these tournaments that some players in the higher rankings will agree to fix a match. An interesting (albeit very long) article has appeared on Green All Over (original article, The Tennis Racket, acknowledged) with regards to match fixing in tennis. The article is so long that I will summarise it with a series of bullet points that I wish to discuss.

  • 16 players in the top 50 are accused of match fixing.
  • These players are in the pay of match fixing syndicates who bet on the outcomes of fixed games.
  • Investigators devised an algorithm and applied it to 26,000 matches to identify fixing.
  • The algorithm identified lopsided betting patterns e.g. a player is far ahead in the match, the syndicate bet against him at favourable odds and that player then throws the game.
  • Presented with the information, tennis authorities appear not to have done enough (if anything) to stamp out match fixing.

There are two perspectives from which you can view these allegations; one based on the integrity of the game and another from the gambling aspect.

Firstly, the non-betting public expect to see a fair match. We can probably be sure that the players who contest the later stages of the Grand Slams are not involved in match fixing. Such players are not going to throw a match for tens of thousands of dollars when they have a chance of winning hundreds of thousands of dollars in prize money plus all the advertising and merchandising contracts that come with being a top player. 

Players in the Challenger and Futures events, which have no attached betting markets, are not going to fix games either. It's the players caught between the élite and the Challenger/Futures also-rans that seem amenable to a little extra in their pay packets. What can be done about this? I am not sure. Other than a sting operation that videos a player accepting money before a match it is hard to prove match fixing.

Does the fixing really affect the rankings at the top of tennis? I don't think so. The players you would expect to see winning Grand Slams and the ATP events do so. Can you really stop a player from playing within himself to move the odds in a tennis game, to be a set down and then comfortably win the match?

Of course, when gangsters get involved in betting they do so from a fixing perspective. Why bother creating a trading algorithm when you can bribe someone. Of course, if gangsters don't make money from their activities then their first recourse is to violence. As mentioned in the article on Green All Over, even Betfair employees have been threatened by these syndicates.

Now let's look at tennis from another perspective, the betting one. In particular we will look at it as a fundamental trader and as an algorithmic trader. The fundamental trader on hearing the news of match fixing will be angry. After all, they have put a lot of work into creating rankings and algorithms to determine their own odds line with which to perform a statistical arbitrage. For the old-school bettor this must be disheartening. 

For myself, as an algo-trader, I am thinking of what is this algorithm that can spot a fixed match? Can I replicate that code? How can I determine who these 16 match fixing players are and bet against them? Will a Weight of Flow analysis help me to determine when one of these 16 players is going to create an opportunity for me to short the market?

Yes, it all sounds very devious but this is what the markets have presented to me and I should do all I can to take advantage of it. If the tennis authorities are not interested in stopping match fixing then I have a simple decision to make. Do I wish to continue profiting from tennis or not? It's the same in financial trading, the authorities make up rules and regulations and the traders have to find ways of profiting within that construct, no matter how devious those methods might be.

I get a lot of people saying to me, "I want to make money on Betfair. How do I pick the winner of a horse race?" I answer with, "Why do you only want to make money on the winner? What about the losers or opposing the winner or trading market trends?" Yes, they are all inter-related trades but they all offer opportunities at different times during the life-cycle of a market.

For a fundamental trader in tennis the question is easy. Does match fixing remove my edge? If so then you immediately stop trading tennis or you stop being a fundamental trader. Such traders use past form to model future events. If you cannot trust past or future events then don't trade them.

It's the old-school approach to betting that I try to push people away from on this website. That is why I prefer to say trading rather than betting. In the past you only had the option of betting on a sports event. Now, there are so many other ways of making a profit. When you say "bet" you are saying one thing and one thing only. When you trade you have so many more options.

By adding a technical aspect to fundamental trading you may spot betting anomalies. The old-school approach would be to predict who is going to win a tennis match, betting on that match and then waiting for the match to be completed. By adding a technical aspect to the trade you would have additional rules for monitoring the market and looking for suspicious activity. You could then trade out of the market for a smaller loss rather than losing the whole bet.

I have joked about Casinni's age on Green All Over but only in jest as I am over 50 too. Old-school betting is not practised just by old men who formerly habituated betting shops and now begrudgingly use Betfair. Even young newcomers to sports exchanges find themselves presented with the latest books discussing how to determine the winners of all manner of sporting events when they would be far better off not reading those books and looking at sports from new and different angles. 

When I am building a trading algorithm I look at what is presented to me and go from there. If match fixing is not taken seriously by the tennis authorities and it is going to skew my ratings then what do I do? Do I trade only the bigger ATP tournaments and Grand Slams? Do I not trade in matches with a list of players that I do not trust? If court-siders are beating me to the trade in some markets are there others I can trade? Do I become a court-sider too? Can I make money from spotting suspicious betting patterns or suspicious players?

Most certainly, I do not say, "Oh well. I need to find another sport to trade." At least, not until all possibilities have been looked at. After all, I am not a connection (insider) in horse racing but I still trade. There are still angles for me to approach the horse race trading problem. I don't bet on the winner of a horse race, I trade any horse that will yield me an edge.