Why bother backtesting?

[dmedin]

From my experience so far, backtesting trades is worthless because live trading produces totally different results.

Why do people bother backtesting at all?

[jlz]

It depends on the type of backtesting that you are doing. If you are doing it manually in one market it won't give you great results.

The point of backtesting your algo is to be able to run it against multiples markets with different timeframes, then you can compare the results and see failures. You need multiple runs against different datasets and from the delta differences you will see where your technique is weaker.

[dmedin]

5 minutes ago, jlz said:

It depends on the type of backtesting that you are doing. If you are doing it manually in one market it won't give you great results.

The point of backtesting your algo is to be able to run it against multiples markets with different timeframes, then you can compare the results and see failures. You need multiple runs against different datasets and from the delta differences you will see where your technique is weaker.

 

I tested simple a simple moving average strategy (with multiple values for the MAs) on several indices and got completely different results for all of them. 

I suppose you mean an algo that is more sophisticated, which is beyond me

 

[jlz]

A backtest is never going to give you a final algorithm with one run. You are defining there one run in your text. You need a least a thousand runs with multiple parameters. Even if you change the MA range values that would be considered one run, because you are changing one parameter. Different runs have different input parameters.

If you have time, read about how genetic algorithms work. They start being the silliest form of live and they evolve into a very complex entity by asking questions to themselves. The very same way a human being would learn through experience.

First you need to define your input and output parameters and make permutations between them.

Let's say your population is 10 markets and your input parameter is an MA to enter. Like you said.

From those 10  markets you can create multiple parameters:

- Time related: Start Date, End Date, candle time frames

- Price action related: High, Low, Open, Close, Volumes

- Account related: PnLs, Margins, Required Funds

- Indicator related: Value of the MA, Crossovers with price actions

- Exit related: Stops, Trailing levels, Take profit levels

Once you've got those set up you can change their values multiple times and compare the results.

Your outputs will give you a report that you have to analyse. From that report you will see which markets are more likely to give you better results for your technique and then you select the best of them.

You are basically selecting the best individual from a population, very much like a genetic algorithm would do.

Then continue adding more parameters to those markets that won the selection.

Eventually after multiple runs you will see the performance of your technique and if it is worth using it, then you can try it on the demo account just to see if when you are involved as a person you can get the same results. A person is likely to decrease the performance of the algorithm.

 

 

[jlz]

This is an example of how a genetic algorithm would find the best path.

https://www.youtube.com/watch?v=wL7tSgUpy8w

You can see how they trigger generations (Gen) with 650 cars each time trying to find which parameters are best to no to crash the car.

If you multiply 650 by the number of generations you will end with tens of thousands of runs. And that is to teach a car to follow a static path.

How many runs would you need to set up a trading algorithm? Since the market is a random walk and your space is not static, you need way more runs than the example in the video.

[dmedin]

So basically, I'm never going to make money in the markets.

 

[dmedin]

Soo ... trading using the 'replay' function is totally unrepresentative of live trading.

What is the point of any of this?

Is it really just designed to get money off desperate people?

[THT]

1 hour ago, dmedin said:

Soo ... trading using the 'replay' function is totally unrepresentative of live trading.

What is the point of any of this?

Is it really just designed to get money off desperate people?

Its betting, gambling - I've said on more than 1 occasion that the laws of probability and probabilistic returns have to be abided by to win in this game and I've said that you really need to skew R value well upwards of 2 for success

The only way to truly know if a method really works or not is to manually back test it over years of data - most of the **** out there in trader land don't work

[dmedin]

39 minutes ago, THT said:

Its betting, gambling - I've said on more than 1 occasion that the laws of probability and probabilistic returns have to be abided by to win in this game and I've said that you really need to skew R value well upwards of 2 for success

The only way to truly know if a method really works or not is to manually back test it over years of data - most of the **** out there in trader land don't work

 

I've done two runs of backtesting using the replay function on TradingView (100 manual trades each time), approx 50% win rate in each and both profitable because of the 1:1.5 risk to reward ratio.

When I trade LIVE, I don't even come close to a 50% win rate lol.

What the f*ck is going on... maybe I'm just not thinking straight when I'm trading live?!

I feel like such a dumb c&nt

Edited October 2, 2020 by dmedin

[Kodiak]

As i said before

Markets changes all the time and you cant test a month or 2 and expect it to work for a longer time

and april may this year was a period when almost everything worked but then changed during the summer

I show a picture of a very simple system based on the stochastic indicator 

[dmedin]

8 minutes ago, Kodiak said:

As i said before

Markets changes all the time and you cant test a month or 2 and expect it to work for a longer time

and april may this year was a period when almost everything worked but then changed during the summer

I show a picture of a very simple system based on the stochastic indicator 

 

 

Nothing 'simple' about that to me.

If market constantly changes to the extent of invalidating a system then that just confirms that backtesting is pointless, then.

[dmedin]

If I can win ~50% of the time using discretionary trading on historical data, why not live data?

[dmedin]

I've never seen anyone demonstrate a consistently profitable system using stochastics.

[jlz]

2 hours ago, dmedin said:

Soo ... trading using the 'replay' function is totally unrepresentative of live trading.

What is the point of any of this?

Is it really just designed to get money off desperate people?

The replay button is useful at the last stage, when your algorithm has been tested against a large dataset of data. Then you can test yourself executing it and see what would happen if you are the one in charge of triggering every action. 

Until we have lost and understood how insignificant are our chances to win, we are not able to see where we are against the market. The major problem is that we are stupid enough to think that we can beat the market when in reality we are playing a game that is way against us, and where we always have the worst cards with very small chances to win.

So your backtesting is essential to create a working ground where you can develop your style. An algorithm is not going to give you a winning strategy, it will only maximise your chances to win. But until you execute it perfectly, that technique will not yield results like it should.

When senior trades are telling us"control your emotions" , "be disciplined", "be methodic" , they are really telling us "become a bot", "behave like a computer". Backtesting can give you a view of what would happen if you were a bot.

 

[jlz]

1 hour ago, THT said:

I've said on more than 1 occasion that the laws of probability and probabilistic returns have to be abided by to win in this game and I've said that you really need to skew R value well upwards of 2 for success

This sentence should be framed and hung in every trading room. But of course people keep looking at charts and indicators instead of developing a risk management strategy.

[dmedin]

3 hours ago, jlz said:

This sentence should be framed and hung in every trading room. But of course people keep looking at charts and indicators instead of developing a risk management strategy.

 

There you go again with the bullsh!t ...

So all you need to make money is a 'risk management strategy'?

Tell us all about it.

[dmedin]

3 hours ago, jlz said:

The replay button is useful at the last stage, when your algorithm has been tested against a large dataset of data. Then you can test yourself executing it and see what would happen if you are the one in charge of triggering every action. 

Until we have lost and understood how insignificant are our chances to win, we are not able to see where we are against the market. The major problem is that we are stupid enough to think that we can beat the market when in reality we are playing a game that is way against us, and where we always have the worst cards with very small chances to win.

So your backtesting is essential to create a working ground where you can develop your style. An algorithm is not going to give you a winning strategy, it will only maximise your chances to win. But until you execute it perfectly, that technique will not yield results like it should.

When senior trades are telling us"control your emotions" , "be disciplined", "be methodic" , they are really telling us "become a bot", "behave like a computer". Backtesting can give you a view of what would happen if you were a bot.

 

 

That's all just a w4nky way of saying that you have no chance of making any money trading.

[jlz]

4 hours ago, dmedin said:

 

There you go again with the bullsh!t ...

So all you need to make money is a 'risk management strategy'?

Tell us all about it.

TA is like a religion. If I challenge it the response will contain an insult. People cannot admit they have been wasting their time looking at flashing colours when they use indicators. You can call it bullsh*t but sooner or later if you want to be successful you have to develop your own risk management strategy. Then you will realise how much time you have wasted looking at those flashing charts. 

[THT]

13 hours ago, dmedin said:

When I trade LIVE, I don't even come close to a 50% win rate lol

Emotions and reacting to a live situation - I missed a massive position back in 2011 when on holiday, could see it building up prev month, then went on hols, move happened perfectly, I'm sat in a tent in France = real life

You need to shift your personality to computer mode when trading, just do the processes exactly or automate

[THT]

5 hours ago, jlz said:

TA is like a religion. If I challenge it the response will contain an insult. People cannot admit they have been wasting their time looking at flashing colours when they use indicators. You can call it bullsh*t but sooner or later if you want to be successful you have to develop your own risk management strategy. Then you will realise how much time you have wasted looking at those flashing charts. 

To be fair there's 5% of TA out there that does work - I have fantastic money management but its backed up by using TA methods that work 

[dmedin]

On 03/10/2020 at 04:07, jlz said:

TA is like a religion. If I challenge it the response will contain an insult. People cannot admit they have been wasting their time looking at flashing colours when they use indicators. You can call it bullsh*t but sooner or later if you want to be successful you have to develop your own risk management strategy. Then you will realise how much time you have wasted looking at those flashing charts. 

 

... if I can't trade visually how the f*k am I going to do it writing code?

[jlz]

Trading Systems are not profitable because they enter at the right time, they are because they exit at the right time.

Let's say you are going to write code and need to tell the algorithm when to enter. Do you think that you can tell a computer to follow an indicator? Does a computer have a sense of uptrend or downtrend? Does it understand when the market is reaching a "support" or a "resistance" level? Is it able to follow price action techniques?

Every developer that is using an indicator to trade what is really telling the computer is to enter at an exact point in time, that can be or can be not supported by the indicator. They don't really care when to enter because they cannot prove when they are right. They could just tell the algorithm to enter randomly and the result would be very similar.

All of those concepts that are part of TA including the charts they are based on are just visual entertainments for traders. They are happy looking at flashing colors so they can justify their entries in a market. You can remove your charts and indicators completely and your entries will be the same, random. If you analyse the number of variables that you need to remove randomness from a "chaotic random walk" such a stock market you will see how they are in the number of thousands. And you are going to tell me that you can justify when to enter in a market by following an indicator that use just one variable. Good luck with that. Your trades are still random.

Once you realise that you are entering in a market randomly, every single time, you start to realise that you need something way more robust than a MA crossover. And that is risk management combined with bank roll sizing.

There are many brokers that are "execution only" and don't offer any charts, simply because you don't need them to trade. The only reason why brokers like IG offer charts and indicators is because people ask for them. The very same people that are part of that infamous 80% of non-profitable traders.

There are many books on risk management for stock trading and they all talk about very similar questions:

- Why do you choose a market against any other to trade?

- What is your optimal bet size looking at your funds?

- Margin and its relation to risk.

- Leveraged products and its relation to risk.

- What is your risk at any point in time?

- What is your optimal exit? When would you take loses? When would you take profits?

They go on and on around bank roll management and risk profiles, and you won't see a singular reference to an indicator. 

If you are able to answer all of the questions above your entry does not matter because your exit is covered, so you can enter randomly on any side of the trade.

Once you have achieved that level you can continue with game theory and its relation to gambling.

https://en.wikipedia.org/wiki/Game_theory

If you follow that path you will never look at a chart again.

 

 

[dmedin]

Not a chance in h3ll, in other words.

[jlz]

Let me post it again, so people don't miss it and can challenge it if they wish.

Trading Systems are not profitable because they enter at the right time, they are because they exit at the right time.

Let's say you are going to write code and need to tell the algorithm when to enter. Do you think that you can tell a computer to follow an indicator? Does a computer have a sense of uptrend or downtrend? Does it understand when the market is reaching a "support" or a "resistance" level? Is it able to follow price action techniques?

Every developer that is using an indicator to trade what is really telling the computer is to enter at an exact point in time, that can be or can be not supported by the indicator. They don't really care when to enter because they cannot prove when they are right. They could just tell the algorithm to enter randomly and the result would be very similar.

All of those concepts that are part of TA including the charts they are based on are just visual entertainments for traders. They are happy looking at flashing colors so they can justify their entries in a market. You can remove your charts and indicators completely and your entries will be the same, random. If you analyse the number of variables that you need to remove randomness from a "chaotic random walk" such a stock market you will see how they are in the number of thousands. And you are going to tell me that you can justify when to enter in a market by following an indicator that use just one variable. Good luck with that. Your trades are still random.

Once you realise that you are entering in a market randomly, every single time, you start to realise that you need something way more robust than a MA crossover. And that is risk management combined with bank roll sizing.

There are many brokers that are "execution only" and don't offer any charts, simply because you don't need them to trade. The only reason why brokers like IG offer charts and indicators is because people ask for them. The very same people that are part of that infamous 80% of non-profitable traders.

There are many books on risk management for stock trading and they all talk about very similar questions:

- Why do you choose a market against any other to trade?

- What is your optimal bet size looking at your funds?

- Margin and its relation to risk.

- Leveraged products and its relation to risk.

- What is your risk at any point in time?

- What is your optimal exit? When would you take loses? When would you take profits?

They go on and on around bank roll management and risk profiles, and you won't see a singular reference to an indicator. 

If you are able to answer all of the questions above your entry does not matter because your exit is covered, so you can enter randomly on any side of the trade.

Once you have achieved that level you can continue with game theory and its relation to gambling.

https://en.wikipedia.org/wiki/Game_theory

If you follow that path you will never look at a chart again.

[THT]

56 minutes ago, jlz said:

Trading Systems are not profitable because they enter at the right time, they are because they exit at the right time.

Let's say you are going to write code and need to tell the algorithm when to enter. Do you think that you can tell a computer to follow an indicator? Does a computer have a sense of uptrend or downtrend? Does it understand when the market is reaching a "support" or a "resistance" level? Is it able to follow price action techniques?

Every developer that is using an indicator to trade what is really telling the computer is to enter at an exact point in time, that can be or can be not supported by the indicator. They don't really care when to enter because they cannot prove when they are right. They could just tell the algorithm to enter randomly and the result would be very similar.

All of those concepts that are part of TA including the charts they are based on are just visual entertainments for traders. They are happy looking at flashing colors so they can justify their entries in a market. You can remove your charts and indicators completely and your entries will be the same, random. If you analyse the number of variables that you need to remove randomness from a "chaotic random walk" such a stock market you will see how they are in the number of thousands. And you are going to tell me that you can justify when to enter in a market by following an indicator that use just one variable. Good luck with that. Your trades are still random.

Once you realise that you are entering in a market randomly, every single time, you start to realise that you need something way more robust than a MA crossover. And that is risk management combined with bank roll sizing.

There are many brokers that are "execution only" and don't offer any charts, simply because you don't need them to trade. The only reason why brokers like IG offer charts and indicators is because people ask for them. The very same people that are part of that infamous 80% of non-profitable traders.

There are many books on risk management for stock trading and they all talk about very similar questions:

- Why do you choose a market against any other to trade?

- What is your optimal bet size looking at your funds?

- Margin and its relation to risk.

- Leveraged products and its relation to risk.

- What is your risk at any point in time?

- What is your optimal exit? When would you take loses? When would you take profits?

They go on and on around bank roll management and risk profiles, and you won't see a singular reference to an indicator. 

If you are able to answer all of the questions above your entry does not matter because your exit is covered, so you can enter randomly on any side of the trade.

Once you have achieved that level you can continue with game theory and its relation to gambling.

https://en.wikipedia.org/wiki/Game_theory

If you follow that path you will never look at a chart again.

 

 

I understand where you're coming from

Are you willing to elaborate on what you do? - don't feel obliged to as - but purely from a technical and risk management based aspect I'm intrigued!

 

[dmedin]

Why would he?  He just wants to show off how 'enlightened' he is ...

[THT]

5 minutes ago, dmedin said:

Why would he?  He just wants to show off how 'enlightened' he is ...

He is though - if he's got a method that wins makes money and consistently then he's in the hallowed 5% and anyone in the hallowed 5% is worth listening to

[dmedin]

1 minute ago, THT said:

He is though - if he's got a method that wins makes money and consistently then he's in the hallowed 5% and anyone in the hallowed 5% is worth listening to

 

Why?  Are you going to write an algorithm too?

Theoretical physicists are clever people but you won't get anything useful out of listening to them.  

[dmedin]

Just to be clear, by backtesting I meant simple MA strategies or manual backtesting.  I have zero ability and zero interest in 'writing algorithms.'  So if I can't trade using charts, I can't trade at all and S&P 500 ETF beckons.

[jlz]

17 minutes ago, THT said:

I understand where you're coming from

Are you willing to elaborate on what you do? - don't feel obliged to as - but purely from a technical and risk management based aspect I'm intrigued!

 

My approach is manly looking to minimise my chances to loose. I do not aim to make any profit so I look primarily at bet sizing to maximise  the number of trades I can send. Reports from my backtesting showed that I can't predict any trade to be profitable so I just make sure that I have enough funds to try again if I am wrong.

I choose markets that won't make huge changes so I only trade in Indices. I chose those markets because of their natural inclination to stay in a range of values so my gains won't be huge but since the aim is to minimise loses they will work best for the algorithm.

The bet size will be the minimal allowed or very close to minimal with a small increment if the previous trades have been profitable. I have been trading with the minimal size for about a year now, and I don't see the size increasing any time soon, so we can count that the bet size is the minimum.

I do not use more than 30% of the equity at anytime. If the account reaches that value it will stop and will take loses. 

I send trades randomly to both sides of the book without taking any criteria to enter, it only checks that there is a trade already or not to trigger another one.

It started with one Index and it is trading now in 5 indexes, any new market is being added if the funds allow the equity level to be below 30%. 

I do not set stops as such at any level, the equity limit will trigger closing orders to gain the margin required back in order to reach that level of equity. So if there is any trade incurring in a loss it will close it  regardless of the amount, the aim is to maintain the account alive to trade with enough equity.

Profits are taken following basic market volatility. This volatility is calculated every hour, so the "take profit order" limit will change depending on the market and its action. Since the volatility calculation is random, profits are random as well. Looking at previous trades they vary from 10 to 60 points the most.

The result is a very very conservative algorithm that yields small profits after "surviving" many losing attempts.