Username   Password       Forgot your password?  Forgot your username?  Create an account

A Free Template From Joomlashack



They recommend us:




Interactive Broker




Ninja Trader



Podcast: Interview of Perry Kaufman, author of the bible of the trading systems
Saturday, 05 January 2013 14:01

We had the opportunity to discuss with Perry Kaufman, a famous systematic trader and author of what is often called the "bible of the trading systems": the book "New Trading Systems and Methods".

Perry is also the director of Kaufman Analytics which provides trading strategies and consulting to institutions and individual investors.

There is a lot of meat in this interview so I hope you will appreciate it. Thanks a lot to Perry who managed to stay passionate and humble despite more than 40 years on the markets...

If you do not understand french, you can skip the very first seconds of the podcast where I present Perry Kaufman in french. You can find the transcript below or the french translated version here.



Below is the interview in plain text.

NV (Nicolas Vitale) - Hi Perry, how are you?

Perry Kaufman - Fine, thank you. A pleasure to hear from you.

NV - Thank you a lot for being with us today! I know you have been in the market for many many years, but you started as what we can call a rocket scientist. Could you explain us that a bit?

Perry KaufmanPerry Kaufman - I found when I make presentations to clients, they are surprisingly interested in my background in aerospace. It was a very interesting time because I worked on the navigation for Gemini which was a two man space vehicle, and the navigation for that was used in Apollo, because you always develop the technology in advance. And you know, I was quite young and very excited, and at that time we were at the state-of-art in mathematics, but now, I think anyone getting out of college knows more than we did. But it was all very exciting! And I worked on this matter for quite until, in 1969, I founded my own computer company.

And after that, someone came to me with a problem in options. It was London options, a long time ago. And it was so interesting that we never stopped. We just switched over what we were doing, which was medical reimbursement. Of course, that would have turned out to be highly profitable. But we switched to options and then into futures and other instruments in the financial markets... and we just kept going! It was a field that you had to fall into. No training was available, and everybody in the field came from different backgrounds and different skills. It was a fascinating time! Perhaps it still is..., but with much more competition then there used to be.

NV - And did you find some similarities between what you were doing in the space sector and financial engineering? That is, in term of mathematics.

Perry Kaufman - Yes, actually, it may seem very simple now, but in the old days, besides the space program, I also worked in missiles and reconnaissance. And you know, we used exponential smoothing to figure out where a missile would land, what would be its trajectory. And a lot of fellows in the aerospace program had time to trade their own stock portfolio. And I used to sit there and apply exponential averages and moving averages to the stock market. And the stock market was so much smoother then, not as noisy as it is now. Many of the engineers made substantial amounts of money with very simple methods. I know even in 1980, which was even a long time for you (ndlr - yes indeed I was not born yet ;-) when I started consulting for an oil company and we were trading a lot of money, the oil market was not used by anyone other than commercials. And you could use a 3-day moving average... and make a lot of money. When prices started going up, they would kept going up and when they went down they would kept going down. The simplest methods trending methods were highly profitable.

NV - Do you think the market today is more difficult than what it used to be?

Perry Kaufman - Much more difficult. With all of the pension programs and all of the different types of investors and competition, the volume is much higher. People have different opinions whether it is going up or down and all these orders placed at different times for different reasons cause a substantial amount of market noise. And that noise makes it difficult to recognize the trend. And so you find that you can not apply trend following in the short term. It is just too noisy. The only trend following that is successful is macro-trend, where you try to take longer-term position that represents a fundamental view of the stock market or interest rate policy. Something that would develop over time. Because in the short term, prices just move up and down too much to be able to really identify a trend with reliability.

NV - I know you are more into strategies with important underlying economic reason, something linked to the market, rather than the new kind of artificial intelligent, computerized methods. Could you just explain why?

Perry Kaufman - Well, you're not completely correct. My methods are all algorithmic. I only like methods that have what I call a sound premise. I use the computer to explore solutions, but I need to have a premise which I believe is true before I start working on a system.

I like seasonality in agriculture markets and we can develop a program that says “if prices are low in the spring before plantation and high in the summer, then the market is seasonal.” Once you have decided that this year has a seasonal pattern, then you can figure out how to trade it. And I can explore that algorithmically. Also, I like arbitrage and I do a lot of pairs trading. Pairs trading also have a sound premise. It takes markets that have some relationships to one another, and when they move apart, I sell one and buy the other expecting them to move back together. Those are very, very popular methods that can be fully automated. And each one has some reason that you believe is true.

Just taking an indicator like a Relative Strength (cf RSI), and trying to figure out where to buy and sell the market based on overbought or oversold levels, is just not a successful approach. Perhaps it’s just too simple. But mostly, there is no sound premise. Just because prices are overbought at some arbitrary level doesn’t mean they will reverse within a few days. They can continue to go up for a longer period of time. So they are not good on their own as strategies. And finding one that appears to work is generally a formula for failures.

NV - And speaking about computerized methods, do you optimize your strategies?

Perry Kaufman - That is a very difficult question. Everybody optimizes to a certain degree. For example, when I am looking to take profits in a particular system, I use the Average True Range as an indicator of volatility. I know, because I have done that for too long, that targeting an average true range times a factor of 3 or 4 from the trade entry will work. I don't even need to test it to find out this is the right answer. So when you have enough experience, you always optimize. You also know that long-term moving averages work but short-term moving averages don’t work. You do not even need to bother trying them. So my first guess as to how to specify the parameters of the system are going to be pretty close of what somebody might find when they optimize. And so I can never say that I do not optimize.

Of course I will take my final strategy and run it through some backtests on data. Because if it does not work, or the profit per trade is too small, then I have to look for an alternative. It may be that the strategy is sound but I need more profit per trades to overcome cost of slippage and commissions. As you know, if a strategy is robust then you should have the choice of slowing it down or making more per trades, or speeding it up and making less per trades. I think those are valid choices without calling optimizations.

But it is certainly tuning it to your needs.

NV - Do you use computerized methods like walk–forward analysis, or you do not because as you explained, you base your decisions on experience to prevent over optimization.

Perry KaufmanPerry Kaufman - When I do a test of a system, let's say I am interested into finding out if the method I just created is robust. And that is a very important issue. What I do is, in advance, I decide a range of parameters that makes sense for that method. For example, looking at a macro trend system, I am going to test trend speeds of say 40 to 120 days. Or I might test different factors for taking profits. For any optimization to be successful, I want about 70% of all of the tests to be profitable. So I could just close my eyes and pick one combination and expect it to be profitable.

But I do not pick the parameters with the best test result. What I do is I pick a sample of a number of combinations and I trade all of them. What I am really doing is looking for an average performance. If I do an optimization on a thousand combinations, the only result in which I am really interested is how many of these combinations are successful and what is the average return of all of the tests. And that is my target goal, the average return. And so when I choose to trade a number of different parameters, I only expect the average return.

That is going to be disappointing for a lot of people, but I think it is realistic. We really don't know what we will get in the future. But we can be pretty sure we will get the average of a number of these reasonable tests.

NV - Thank you for all the details. When it comes to finding ideas of new trading systems, do you have a method or starting point? Where do you find inspiration globally? I’m sure every new trader is interested.

Perry Kaufman - An other difficult question... Finding trading ideas is perhaps the most difficult of all things (ndlr Perry's book would help you definitely...). Because once you have an idea, you can make it a successful system (ndlr... and Alpha Novae can help you for that...). So first you have to be an observer of the market. Then, there are so many things that happen in economics or in seasonality, or things that you see in the newspaper, about why the price of oil goes up or why the dollar has gone down. You need to be able to identify them and say aha!, here is an idea that I can do something with. Or you see that there are many books on pairs trading and you know believe that arbitrage is a sound approach, but people have already done that already. So you have to study it and decide how your ideas may differ from others.

For example, everybody knows you can arbitrage a stock within the pharmaceutical sector or auto industry. But so many others are doing this that the profit per trade is very small and not profitable enough. However, there is a bigger picture. The entire market has become more correlated because of index programs. An index often is a group of stocks within the S&P. Investors will place an order to by or sell the index. The index company goes in and buys all of the stocks in that sector, and that drives the entire group in the same direction. But there are some companies in that group don't deserve to be driven up along of the others, and so there is an opportunity to sell that company and buy the sector or a different stock in that sector. In a broader sense, you can now arbitrage nearly any two items in the S&P because index markets drive them back together and then apart and then back together. There are plenty of opportunities.

NV - Overall are you more into intraday or swing, mid-term strategies?

Perry Kaufman - I prefer daily strategies because they have larger profits per trade. There is a lot of competition everywhere but I think it is easier to deal with daily data. And I suspect for most investors, even many professional traders, daily is much easier to deal with. With intraday trading you need to have much more data. You need to be able to place orders during the day and that is inconvenient for many people; and, profits per trade are much smaller. So intraday trading is subject to slippage and other issues that become a real problem. I believe there are plenty of opportunities in daily. In fact, I do intraday arbitrage, but I don't do it the way high frequency does it, which is in microseconds. I do it in hours.

NV - Speaking about high frequency trading, everyone is talking about it, even at the political level. Do you have an opinion about that? Should it be banned or regulated more? (ndlr you can see an here an extract of our conference on High Frequency Trading at the Salon du Trading, and more extracts from the Salon du Trading here).

Perry Kaufman - If high frequency trading makes money, it does extract profits from the market that somebody else could possibly get. I don't really care who makes money as long as it is done honestly. It's fine to find a method to arbitrage in micro seconds. They will create their own competition. Up to now, the high-frequency people have made an arrangement with the exchanges to place their computers as close as possible to the transmission source in order to beat competition by a fraction of a millisecond. The exchanges are now rejecting that and say no, you cannot be closer than, say 100 feet, and cannot share the same space as the exchange. As with all of these great methods of trading, competition will eventually minimize the profits and many companies will stop. I think the market will take care of itself.

NV - To change the subject a bit, I know, having read some of your books, than you emphasize risk and money management. Is it at the core of your systems? And do you build specific money management modules for each strategy? Can you detail a bit how you handle money management?

Perry Kaufman - Risk management became even more important in the new book that is just coming out within the next weeks or two. I have really gone through it and made sure that risk is explained almost everywhere throughout the book. It is mostly because we were horribly surprised by the subprime crisis. I am one of those who did not really expect the magnitude of the risk in 2008. I kept saying that it gone far enough, it will stop and recover and things will be back to normal. We did not expect anything this big and this long. And that requires you rethink all of your concepts of risk. You don't want to be wiped out because you managed the risk incorrectly.

So I have spent at least five years focusing on risk and decided that risk is as important as the underlying system. When the market becomes more volatile, we must reduce our position size. But even more important, when the market becomes less volatile, you need to increase your position size because there are long periods of low volatility which make nice money. If you do not have a big enough position during periods of low volatility, you can't get the return that you want. So for both high and low volatility, you want to do what we call “volatility stabilization.” It is a concept which is familiar to most professionals, where you keep adjusting your size, always conscious of the cost of doing this. Because whenever you switch position size there is a cost of what we call “rebalancing.” Then you have to be careful how often you rebalance, but you still must rebalance.

Investors that put money into a program deserve risk control. Without it, there is a much higher chance being wiped out. Diversification is also important, but almost everyone already understands that. Traders should also look at Value at Risk. Value at Risk is a formula that tells you the probability of a loss greater than some value, given the positions you are holding. It’s not perfect, but it is a help controlling risk. I know the formula is available in many books, and it may also be in Excel. If you are smart, you will reduce you position when VaR shows extreme risk.

Besides portfolio risk, you have risk on individual trades. Some people deal with that with stop losses but I prefer taking profit and getting out of the market whenever possible. Taking profits has the added advantage of keeping you out of the market. For example, if you are not in the market one third of the time, then you have avoided unexpected price shocks for one third of the time, and your overall risk will go down. Then one of the rules of choosing the best trading method should be that you are out of the market as much as possible. By doing that you avoid unexpected an unpleasant situation.

(ndlr - By not being in the market, you also get extra liquidity for other strategies that would need being in the market at this time. So you get a better cash flexibility for optimal allocation.)

NV - Thank you for everything. To speak more technically now, do you use specific platforms or tools to automate your strategy or to backtest?

Perry KaufmanPerry Kaufman - Well, I do, mostly a platform that I developed myself, and so I have 40 years now of developing that.

One of the problems with the platforms that you buy, and some of them are quite nice and very convenient, is that they can be slow, especially if you’re working with a lot of intraday data. On the other hand, I often use TradeStation to get a fast look at whether a new idea seems reasonable. I learned to use TradeStation because it was the first platform you could really program and it’s very convenient. So that is why there are many example of TradeStation in my book (ndlr we recommend MultiCharts too as an equivalent multi broker and actively developed). We have also started using MetaStock because Thomson Reuters has done a great job upgrading it. Of course, there are many other platforms that will work and a far less expensive.

NV - Yes, there are hundreds and hundreds platforms of course. (ndlr, most used platforms from our forum)

Perry Kaufman - Yes of course, but I am not in the business of building platforms. The greatest advantage of any platform is to see if a new idea make any sense at all, I would use TradeStation or MetaStock to see where the signals occur on the screen and make sure they do not look stupid.

NV - Do you try to optimize your execution quality when you are trading? Or is it not really the most important point because you do not really trade a lot in small timeframes?

Perry Kaufman - That is correct, but I found that when using trend following, I prefer not entering on a trend signal because the markets tend to be pushed and they are many orders clustering at the trend change area. I found that my best approach is to wait until the market reverses after the signal. That turns out to be a good technique because sometimes the trend reverses again and you never have to actually change your position. The other method would be averaging in over whatever the timeframe you're using.

For example, if you are a macro trend trader and you're going to hold the trade for 3 to 6 months, you can take a week to get in. And if you are a shorter-term trader, you can take an hour or two to get in. When you average in, you find that your results are much more stable than if you just take the original trading signal.

NV - I know you trade in stocks, futures, and commodities. Are you also trading in Foreign Exchange (forex)?

Perry Kaufman - Yes, I have a major program in FX Carry. But again, FX Carry is a sound premise. It is not the concept that there is money in this arbitrage, but that investors put money into countries with higher interest rates net of inflation. And so it is the movement of money that makes FX Carry successful. But it is also a complicated program because it is all about controlling risk. You need to avoid begin caught in the Japanese intervention which turns your entire profits over the year into a loss. So that is my focus in FX, although FX is part of all the other trend programs and everything else I do. But specifically I like the Carry program.

NV - You just spoke before about your book and a lot of people especially know the "New Trading Systems and Methods". We refer quite often to it as the bible of the trading systems (ndlr more than 1200 pages!). You were telling us we would have a new edition soon?

Perry Kaufman - A new edition within weeks! It should be released soon (ndlr begining february according to Amazon). It is a good upgrade. All the examples and charts have been upgraded to current examples. There is a lot more on arbitrage, more trading systems, and a lot more on risk throughout the book. It is also a better organized. I have started to give more analysis on whether or not particular methods are good for certain users. In the past I treated it more like an encyclopedia. You could pick what you wanted and see how it worked. Now I am trying to give you a better understanding on when to use it, when it does not work, and when other methods which are simpler may better work. So I hope the users will like those changes.

NV - Yes I think they will. Personally, I became interested in automated trading strategies some years ago partly thanks to your book, so I will be happy to see the new version. And do you have other projects for the future?

Perry Kaufman - Yes, I am working on some retails programs for both UBS and Thomson Reuters and I have a program running at RBS. So I am trying to stay what we call “relevant,” staying in the mainstream with these companies. The program for UBS will be a relative value arbitrage similar to pair trading, while the one with Thomson Reuters will be both proprietary trading strategies and a toolbox of strategies. So I hope investors will find them interesting.

NV - Thank you very much for all these details and for the time you have spent with us today. I know that you are a very busy man so thanks a lot!

livre perry kaufman

Minutes of the Global Algorithmic Trading Conference at UCL


You probably knew that the Alpha Novae's team was represented at the Global Algorithmic Trading Competition conference at UCL. For you readers of Automated Trading we decided to make a small report of what we saw and heard.

For those who didn't follow, the conference took place at UCL ( University College of London ), a prestigious university well known for the quality of its education and its researches as well. The conference was mainly focused on "professional" algorithmic trading so more on institutional algorithmic trading or High Frequency trading than on automated trading as we know it on MT4 ;). All along the day, Microsoft, Barclays Capital, City Knight and LMAX followed one another to offer us twenty minutes presentations. I will try here to give you a brief feedback of the conference and to reveal what all these people are working on ( at least what they're willing to tell us )

First of all let's start with a short visit of the UCL campus

You can see here the main entry of the famous university. All the students who are reading us should note that UCL is, at this date, the only university ( at least european ) to offer education in the algorithmic trading sector with, among others, a MSc in Finantial Computing. In 2010, UCL is ranked 21st in the world (and 3rd in Europe) in the Academic Ranking of World Universities, 4th in the world (and 2nd in Europe) in the QS World University Rankings and 22nd in the world (and 5th in Europe) in the Times Higher Education World University Rankings. If you are a student passionate by automate trading this seems to be a proper education ( however, be careful about the admission conditions and the education fees ). As concerned the conference itself, it naturally took place in the computer science department of the university.


Let's now get to the heart of the matter. Here is my report of the different presentations we attended :

The first intervention was presented by a UCL teacher named Philip Treleaven. He started by remind us how much algorithmic trading was an integrant part of the markets since it represents in the US between 60 et 70% of the shares trades volume. If all this positions were in fact opened automatically we should differentiate the simple execution algorithms, Market Making or arbitrage activities from the famous Black Boxes. Because what is actually interesting us here is the Alpha and the modelling of profits making strategies. Here is an automated trading system diagram as it was presented to us.

So according to this teacher, the data feeds would be for our black boxes what the blood is for human beings. "the more data you have, the more succes" added. And if the classical strategies use today the prices of the instruments traded, the speaker insisted on the opportunity that represents the use of "social" data ( news informations or data extracted from social networks such as twitter ).


The second presentation was held by Chris Donnan from Barclays Capital. We learned here that to make profits in the verry competitive market of the algorithmic trading, Barclays works mainly on 4 factors : the hardware ( more than 800 machines dedicated ) the software ( with a framework developped by and for Barclays ), the networking ( trading data as fast as possible ) and the relationships ( the exchange of ideas and knowledge between traders, quants and other analysts ). The quant desk manages to open 8 positions by second and realizes between 200 and 250 thousands transactions a day( if this is not HFT.. ). The man concluded this way : ' the technology is really the core of the business" even if he underlined that the 3 other factors we were talking about earlier were also important pieces of the automated trading puzzle. To conclude i will add another quote " Working in automated trading is exciting as they are big hairy puzzles to solve "


The third intervention introduced the competition launched by UCL and all its sponsors around automated trading. It's a very attractive competition since, as the man pointed out, the 20 000 pounds of cash prize are nothing compared to the wage of the one who will be able to show off on his CV for having arrived first. I will thus encourage all the interested members of Automated Trading to compete by their own, or even better, to work together for the win since the team projects are allowed. This is now the prizes list :

- The best algorithm Microsoft Prize : 5000 pounds (+ 2500GBP depending on the performance)

- The best use of the Microsoft Technology : 2500 pounds

- Barclays Capital Prize for best competition entry by a female contestant : 2500 pounds

- CitiFX Prize for best trading model : 5000 pounds

- Knight Capital's Prize for Best Use of Data : 5000 pounds


The fourth intervention was held by Dr John Loizides from City and dealt with High Frequency Trading on Forex. The Forex is a quite young sector for automated and algorithmic trading, unlike equities and CFD wich are the "source" markets. In order to trade on UTs shorter and shorter, the algorithms are analyzing more and more data feeds as you can see here


The capacity to process and analyze these data is a decisive parameter in the modelling of algorithms. Today more than 100GB of data are available per day to institutionals. As concerned the programming language, it varies a lot depending on the needs (FPGA’s, C/C++, C#, KDB+, Q, JAVA). I will finally add a quote which attracted my attention "The ability to find real market indicators within the high frequency world with a respectable profit and loss profile is difficult, but very achievable with the correct mathematical and technological approach."

The fifth presentation was the Microsoft's one which presented to us its "new" programming language : the F#. F# can be integrated to an environment using standards for language. It's a mathematical oriented language. This makes F# a verry attractive tool for quants but which can also reveals itself useful for any other financial application.

Let's note also that it allows quants to directly contribute to the tools development ( no need to code in mathematical language and then in C++ )

Finally F# is located somewhere between C++ and a pure mathematical language ( closer from python for example )

To conclude here are the Hardware trends for the future according Microsoft

- Clouding
- Multicore

Finally, LMAX, the London Multiple Asset Exchange, which is the first exchange (MTF) for Forex and CFD trading, gave a conference on its capacity to handle the global trading risk for its clients tick by tick, which is a real innovation in the field. In fact, the platform LMAX based on the Betfaire technology manages everyday more trades than the all the professional market trades. In fact, the retail world multiply the number of traders, who have the tendency to take much more risk than the professionals. Each transaction has thus to be validated, and at each tick the global risk calculated. Knowing that LMAX is oriented to High Frequency Trading, this was impossible with the standard tools. They thus created an inovating low latency environment and were professionally prized for this. And, even more surprising, they opened the code of their framework entitled "disruptor".

For information, Alpha Novae is a partner of LMAX and realizes currently a bridge allowing to duplicate the orders executed via MT4 on LMAX.

Finally the day ended between petit-four and passionate talks. We really enjoyed being able to meet some clients and talk about interesting partnership projects.

Jeremy for Automated Trading


Joomla 1.5 Templates by Joomlashack