Are you an Amibroker user who wants to improve the way you do backtests and optimizations? In this video taken from a one-on-one mentoring session from one of my students with the trader success system, I take you through exactly how to use the backtest report, the trade log that you get during the back test, and also the optimization report to make better decisions about your trading systems in Amibroker. So stick around.

So I’ve run a back test. The system is called formula 112. That’s how random it is. And what you can see on the screen here is the trade database that the backtest generates. So first we’ll do a normal back test, then we’ll do an optimization, and we compare the columns and so on. Okay. So if we look at the trade database that we get, when we run a standard backtest with no optimization, basically tells you everything Amibroker needs to know about that trade in order to assemble it. So you get the entry date, the exit date, the entry price, the exit price, and more or less everything else is calculated based on those things. So we know for each row, it’ll tell us the price change from start to finish, the percent profit, the number of shares you bought according to your position sizing formula, which is defined in the system, the value of that position, as in how many dollars that was worth.

This cumulative profit, I don’t actually understand that. I think it’s the system’s cumulative profit on the day that signal, that stock was triggered, but I don’t look at that one. Then there’s number of bars, which is how long the trade was open for. So you can see here in this system, that ranges from about six bars up to over 200 bars, because this is a trend following system. So the losing trades get stopped out pretty quickly, but the winning trades can go on for long, so let’s see, this trade here was open for 274 bars, and it went up 61% near enough.

The profit per bar is really quite useful when you’re optimizing a system. One of the things that you want to make sure of is that your capital is effectively utilized. So you can increase the average profit per trade buying using a longer term exit. But let’s say for example, what’s the example, lets say you’re using a trend following system and the price crossing below the 200 day moving average is your long-term exit. If you made it price crossing below the 300 day moving average, you’d probably get a higher average profit per trade. Which sounds like a good thing, but the profit per bar, the profit per day you’re the trade may actually go down because it’s taking so much longer to get your exit.

And so you’ve got to be a bit mindful. Not that we’re in a rush, but you’ve got to be a bit mindful of how quickly the system makes the money. Because if one system, if two systems have the same average profit per trade, but one has shorter trades. That second system will compound more quickly. And so your account will grow more quickly. So that’s quite useful. Maximum adverse excursion and maximum favorable excursion are the last columns I’ve got here. There’s some other columns you can probably add, but these are the only ones I see. So maximum adverse excursion is the worst distance, in percentage terms, the stock moves against you in the life of the trade.

So if you enter at $10 and the stock goes down to $5 before closing out at 15, the drop from 10 to five is a 50% adverse excursion. That’s the maximum adverse excursion. Does that make sense? And then favorably to, as an MFE, maximum favorable excursion, is how far it goes in your favor before it ultimately closes out. So if you buy a stock at $10 and it goes to 20 before closing out at 15, the favorable excursion is from 10 to 20, which is a hundred percent maximum favorable excursion. And the term maximum just means in the life of the trade, what was the very best or very worst that it got to.

And I don’t use adverse excursion all that much. It’s useful for setting a stop losses a little bit, but favorable excursion is really interesting because if you have a, let’s say you got an average profit of trade of like 5%. The system could be quite profitable, but if heaps of trades have a favorable excursion of 20, 30, 50, 100% before closing out on average at five. You may be missing out on a lot of profit because your exit is no good. Yeah. So if you’re favorable excursion super high, or much higher than your average profit per trade, sometimes it’s worth looking at a bunch of example trades and trying to find a better exit that captures more of that favorable move because you’re giving up too much profit from between the high point and your exit point.

So this is for, this is the trade database. The next thing you see when you run a normal back test is the trade report. Now, can you see my trade report here? Okay. So this back test report and the results you get when you’re doing an optimization, the columns you get in optimization are very similar, almost identical. Okay. So if you understand one, you understand the other. So in the backtest report, this is basically giving you all of the statistics of the system so that you can evaluate whether or not it’s good enough, or whether you’ve improved it, or made it worse. Now I look at a few things, the most important ones we can go through now. First thing I generally look at is the compound annual return, for obvious reasons, because we want to make money. I’m also interested in the exposure.

Exposure is the percentage of the time that your money is invested. So let’s say you’ve got $100,000 dollars in your account. And on average, over the life of the backtest, it was only 50% invested. So half the time you were fully in the market, half the time you were in cash. Your exposure would be 50%. If you a 100% invested all of the time, your exposure would be 100%. So why is that interesting? Well, if you’ve got two systems, they both make 20% return per year, but one has an exposure of 50% and one has never exposure of 80%. Which one has the higher risk?

80%.

Right, because your cash is exposed to the market moves more of the time. So for the same level of return, you want a lower exposure. The other thing that exposure is useful for is to see how well you can use that system to layer on top of your other systems. I have a few systems where the exposure is very low, like five or 6%, because they don’t trade very often. And if you could make 10% return per year on average, with an average of 5% exposure, that’s pretty amazing. Because you’re only putting a teeny bit of money into the market every now and then, 5% of the time, and you’re making 10% return per year. The great thing about that is you could take that system because the exposure is low and you can combine it with another system with low exposure, and another system with low exposure, and another system with low exposure, and pretty soon you’ve got some magic happening in portfolio.

Does that make sense? Okay, so let’s say…

Hold on

You’ve got, you’ve got $10,000 in your account. Let’s say you’ve got one system that only signals every now and then. What if it takes one trade at a time, each trade is 500 bucks, and it only signals one trade at a time on average. So the maximum cash you’re using is $500 out of 10,000. Right, 5%. So even if you got a signal and you’ve taken a trade, you’ve still got $9,500 cash sitting around doing nothing. So you could take another system and trade it as well, using your whole $10,000 [inaudible 00:09:21]. And when you, if you’re making 5% or 10% per year on that first system, and then another 10% per year in the second system, those 10% are additives, so you’ll be making 20% on your portfolio. So this, this system here has a 68% exposure, which is fairly high, but typical for trend follow.

So you can’t really add two of these together without taking on leverage. You have to sort of give a bit of capital to one, a bit of capital to another. But in the mean reversion system, you might, it’s more often like the other example I gave you where it’s you, you make 10% return per year, but you’re only using five or 10% exposure. So you can add two of them together pretty easily. Or you could add a meager version to a trend following system and not really use up any extra capital.

I also look at the average profit or loss percentage. I don’t really look at the average profit or loss in dollar terms, because in dollar terms, that depends on how much capital you start with. And it depends on how long the backtest is and how big your account grows to. I’m much more interested in the percentage number, because it’s normalized at a time. 15% at the start of backtest is 15% at the end of your back test. It doesn’t matter about the equity. So I’ll look at that. I look at the average boss held, because I want to make my money more efficiently if I can. Or if there’s a reason to hold for a long term, like tax considerations, I want to know that my average profit is held for a long time.

And these two stats, the average profit loss and the average bars, are broken down for winners and losers. So a little further down, you’ve got the average profit percent and average bars held for the winners and the percentage of winners. So I’ll look at those, and then same underneath for the losers. And then down the bottom, there’s a couple of things I’ll look at which are the most interesting. One is the max system drawdown. The compound annual return divided by the max system drawdown I quite like as a visual. And the reason I like that is if this is your equity curve, the squiggly line in red, let’s say you make 20% per year and you suffer a worst case, drawdown of 20%.

The compound annual return divided by the max drawdown is 20 other 20 equals one. A ratio one is pretty phenomenal, particularly for trend following. If your maximum drawdown on another system was, let’s say it did this, this other system made 20%, but I had a max drawdown of 40, that’s systems much harder to tolerate. Yeah. Right. Yeah. And too painful is a term for a judgment that is relevant for you. Someone else may be fine with it. And someone else might be freaking out at the 20% drawdown of the first system.

Yeah.

Right. So this is why and Michael, this is why the T6 trader profile, the drawdown tolerance is also useful because based on all of that, you can decide whether a system is right for you. And if the drawdown is too high, it won’t be, and you need to modify it before you can trade it. So carryover max drawdown, compound annual return divided by max drawdown, is quite useful because you always want to be basically the objective of your optimization and your system testing and everything is to push that up, to make it better.

Now I’ve tried in systems with CAR over max drawdown low as about 0.3, but I prefer sort of 0.6 or above for the individual system. But if I’m combining systems into a portfolio, you don’t have to evaluate the system individually. So let’s say you’ve got a bull market system that makes money when the market’s going up. And you’ve got a short-selling system that makes money when the market’s going down. The stats of the bear market system that makes money when the market’s going down, don’t have to be great on their own. It can have a fairly big drawdown on a fairly modest return, like big drawdown small return, because when you add it to your other system, the combined effect is really good because they’re diversified. They sort of run counter to each other. These are the stats that I focus on most. There’s a lot of other stuff in there, but a lot of it is unnecessary.

Okay, and so the same thing goes in your optimization. I’ll just, I have to go into the code of this system and just run a quick change something to it and optimize so we can look at that. Okay. So I just made one variable an optimization variable. So I’m just going to press optimize now so we can get all of those columns up. And this might take a little while because I haven’t done an optimization in this system before, but here we go. So you’ll see, the first thing you should notice is that the columns are pretty much the same as in that backtest report that we just saw. Net profit, percent net profit dollars, compound annual return, risk adjusted return, exposure, maximum system drawdown percent. They’re all, basically the report for the backtest gives it to you in a vertical format. And when you run an optimization, it gives you all those same numbers in a horizontal format.

That makes sense? See the columns that you’ve got here. So net profit, CAR, risk adjusted return, exposure, and everything. They’re exactly the same headings as we’re down the side in the facts of what we just looked at. So if you understand the backs of support, then you understand the optimization, but all you’re doing in the optimization is comparing one road to the next, to make a judgment about which is better. So it’s lining them all up you so you can make that comparison. So when you look at the optimization report, you want to look at the same columns that I highlighted in the backtest report, because everything is a trade-off. You know, one might come with a really high percentage profit, but a really high drop down. One might come with a lower profit, but a nice smooth equity curve, a higher win rate means smaller winners, typically.

A low win rate, mean bigger winners typically. So you kind of want to understand the trade-offs and find the one that best suits you. Now, generally, what we’re doing is trying to make a smooth equity curve. So I usually start with compound annual return to the max drawdown. So comparing that first to see if one, if there’s a clear winner or if there’s a clear pattern of improvement or deterioration in the smoothest of the equity curve, as we change that optimization value. So on this, in the system, we changed the short-term moving average. That’s what optimized. And we changed it from actually, I don’t know what we changed it from. Let me just go over here and have a look.

We changed it from 20 up to 100 in steps of 10. Okay. And well, we can see in the compound annual return of a max drawdown is the shape of the equity curve didn’t change much. They basically 0.4 to 0.44 current max draw down for every value of that short term, moving average. So compound annual return is the growth. The percent, it’s like interest rate. It’s the equivalent interest rate from the start of your backtest to the end of your backtest. That’s what compound annual return is. And then max drawdown is the biggest percentage drop at any one point in your backtest.

So compound annual return over max drawdown. So let’s take a simple example. If we made 20% return for a year, and we had a 40% drawdown, then CAR of MDD will be 0.5. And so the bigger that number, the better, the smoother the system. If you, let’s say we let’s say we, we did this backtest and we got 20% and 40% drawdown. Let’s say we change the system, make it better. And all of a sudden we don’t have 40% drawdown, we’ve got 10, but you’re still making 20% return per year. Is the system better? A lot right? What is the CAR over max drawdown now? So we get two. So that’s substantially better than the previous example. Yeah. Now often it’s not so stark, but if you add a new rule, or take a rule away. The CAR over max drawdown will tend to change.

Yeah. And so when you do your optimization, you want to look at this because if it gets materially smoother, that’s a good thing. So the CAR over max drawdown will go up. Okay. So that’s one thing that you can look at in the backtest, in your optimization, CAR over max drawdown. But, you can’t just look at that. Sadly. There’s no one magic number that we can, we can look at, but it’s not just the bad numbers you’re looking at. You’re also looking for like step changes or spikes or instability. If you’re, if I can, let me draw on screen again for a sec here.

Cause basically with optimization system design, one of the most important principles we’re trying to adhere to is to find a system which is stable over time and stable across different parameter values. But if you’ve got, let’s say this is the compound annual return on the vertical axis. And on the horizontal axis is all the different values of the parameter that you’re testing. Right? Let’s yeah imagine the chart was like this, it was kind of all over the place. It doesn’t give you any confidence that that system is working, it’s kind of random. But if instead, the profile looked kind of like this. Well then it’s pretty stable, right? And the answer is you choose the value for the parameters in the middle of the [inaudible 00:21:42] stability.

And so you’ve kind of got to look at the performance stats and the optimization and the parameter, how they vary as you change the parameters to make your judgment about where you should put your parameters at. And so I did this in my head, like I sort of visualize looking at the numbers. Some people put it into Excel and draw the charts. I used to do that. To be honest, I find it a little slow now. So I really just kind of look at the trend in the numbers and pick the value. But I do that for compound annual return. I do it for max system drawdown. I do it for the CAR over max drawdown, which is further to the right and all of the stats I’m interested in. Right? So I’ll look at the CAR over max drawdown, which is here.

I look at the number of trades, which is actually quite important. If you’ve got, if you do an optimization and the number of trades falls off dramatically and ends up, you don’t want to choose an optimum value that produces very few trades. Because it’s probably not reliable. So at one end of the optimization you’re producing a thousand trades and at the other end of the optimization, you’re producing 50, you know that this end of the optimization is kind of suspicious. You can’t really rely on it, it’s not enough data. Yeah.

I also look at the average percent profit loss, average bars held, all the same things I mentioned earlier, the percent winners, you have percent winners, average win, average loss. All of the same things I highlight on the performance report. I’ll look at it in the optimization report and I’ll choose the road that represents the best trade off. So one of the most challenging things, I guess, to get our heads around for optimization is there’s not a right answer. But we’re looking for things that we clearly want to avoid, and we’re looking for areas of better performance, and areas of worst performance, and we obviously are to position our parameter values around the areas of better performance, as long as they’re stable.

The mistake that most people make when optimizing a trading system or working with a trading system is pressing optimize and using one performance measure to tell them which is the best value. But it’s too simplistic. Like it doesn’t really work that way because we’re not after best. We’re after most stable and profitable. If we can improve the performance and get more stable, great. But if the best performance is an unstable area, like if you vary the parameter value just a little bit and the performance falls to crap, then that’s no good. We don’t want that. So what I would suggest as a next step, is to practice this. So run an optimization, get this table and say, okay, I think this is the best value. This is the best trade off.