The Gateway to Algorithmic and Automated Trading

Reducing the lag of exponential moving averages

Published in Automated Trader Magazine Issue 40 Q3 2016

Moving average-based filtering techniques bring much needed smoothing to noisy time series. But they do so at the unavoidable cost of an introduced lag. There are a number of ways of trying to mitigate this. One approach, aptly called 'twicing', is to simply add the lag itself back into the equation

Moving averages and exponential moving averages (EMAs) are ubiquitous in automated trading. They are used both as a graphical tool and in the quantitative treatment of numerical data since they are a simple and computationally efficient way of filtering out high frequency noise in the data. They act as a low pass filter. Technicians and traders alike use them on charts and as building blocks in their quantitative backtests. An EMA can simply be written recursively as the weighted combination of its previous value and the latest data:

EMA(t) = alpha*x(t) + (1-alpha)*EMA(t-1).

The smaller the alpha parameter is, the more efficiently noise is going to be filtered out - but the less responsive the EMA is going to be when a genuine move appears in the data.

Over the years, numerous schemes of various complexity have been designed to reduce the EMA lag, i.e. the average delay between the signal and the filtered signal for a given amount of smoothing. In this article we are going to focus on the approach developed by Tim Tillson [Tillson, 98] building on the idea of 'twicing' by John Tukey [Tukey, 76]. Let's introduce the idea step-by-step: First let's define lag as the difference between signal and the output of an EMA:

lag = x - EMA(x).

Now we consider an EMA with a modified input: Instead of using the raw signal x, we add the lag to the latter and use this as an input of the EMA, to give it some sort of 'head start':

EMA(x+lag) = EMA(x + x - EMA(x))

= EMA(2*x - EMA(x)).

Given that the EMA is a linear operation

EMA(a*x+y) = a*EMA(x) + EMA(y)

we have

EMA(x+lag) = 2*EMA(x) - EMA(EMA(x))

= Twiced_EMA(x).

We can quantitatively study the behavior of this new EMA filter. A filter is characterized by two quantities: The 'lag' or average delay for a signal of various frequency, and the 'gain' or response of the filter to the signal amplitude. The smaller the lag and the closer the gain to 1.0, the better the filter. The lag plot on Figure 01 compares the lag of an EMA and the lag of the twicing EMA - it clearly shows that the lag has been reduced.

Figure 01: Lag plot of the standard EMA and the twicing EMA. Higher is better as it means less lag

Figure 01: Lag plot of the standard EMA and the twicing EMA.
Higher is better as it means less lag

The gain plot on Figure 02 shows an interesting feature:

Figure 02: Gain plot of the EMA and the twicing EMA

Figure 02: Gain plot of the EMA and the twicing EMA

For a range of frequencies, the gain of the twiced EMA is actually larger than one! In practice, this can be highly unsettling especially on a chart, as this means that the filtered output can 'overshoot' or 'over-react' to the input, as shown in Figure 03.

The twicing EMA keeps going in the signal direction after the signal has reverted, which is unnatural when one is used to standard moving averages. This filter with gain larger than one is called 'active', as opposed to a 'passive' filter like the moving average. This overshooting clearly is a behavior we would like to avoid when we aim at building a passive filter, like the moving average, that does not overreact to the price and yet has a reduced lag. The idea of Tillson is to modulate the portion of the lag that we add to the input signal, that means we do not add the whole lag to the input signal but only a fraction 'v' of the lag.

Figure 03: Graphical display of various reduced lag moving average to a given input. As the price stops increasing and goes down, the filtered price keeps going up. Source: Lyons, 2010

Figure 03: Graphical display of various reduced lag moving average to a given input. As the price stops increasing and goes down, the filtered price keeps going up. Source: Lyons, 2010

Figure 04: Maximum gain value as a function of the parameter v

Figure 04: Maximum gain value as a function of the parameter v

Figure 05: Filter gain as a function of the modulating parameter v and the signal frequency omega. Note how the surface adopts a bump shape for v > 0.4.

Figure 05: Filter gain as a function of the modulating parameter v and the signal frequency omega. Note how the surface adopts a bump shape for v > 0.4.

Figure 06: Lag of the EMA filter and the modulated twiced filter. Higher is better.

Figure 06: Lag of the EMA filter and the modulated twiced filter.
Higher is better.

The input x is replaced the following way:

x -> x + v*(x - EMA(x))

EMA(x+v*(x-EMA(x)))= EMA((1+v)*x - v*EMA(x))

=(1+v)*EMA(x)-v*EMA(EMA(x))

The new filter is therefore:

Modulated_Twiced_EMA(x) =

(1+v)*EMA(x) -v*EMA(EMA(x)).

This new filter has a free parameter, v, which controls the amount of lag that is added to the original signal. For v = 0 we do not add any lag and the modulated twiced EMA is identical to an EMA. For v=1 it is identical to the twiced EMA of Tukey. The idea now is to study the lag and maximum gain as a function of this parameter. The gain study that was done in Figure 02 can now be repeated for all values of v between 0.0 and 1.0, and the maximum gain value for every v value is shown in Figure 04.

This result is striking as it shows that there is a value of v for which the maximum gain becomes larger than 1. This can also be seen on Figure 05, which shows the maximum gain as a function of v and the signal frequency.

The maximum gain surface adopts a different shape for values of v > 0.4 as can be shown on the contour plot of Figure 05. It is therefore recommended to use v=0.4 and this will guarantee that the gain remains lower than one. We can also check that there is indeed a lag improvement of v=0.4 as shown in Figure 06.

Many 'reduced lag' schemes exist in the literature, but one needs to consider if an over-reaction to the price is present in the filter, and if it is possible to fix it. We have presented a simple filter - based on Tillson's twicing study - that is a lag improvement over an EMA. It has one extra free parameter for which 'safe' values allow the gain of the filter to stay well-behaved and to avoid overshooting.

References

Lyons, Richard
2010
Reduced Data Smoothing
bit.ly/29DSxld

Raudys, A.
Malcius, E.
Lenciauskas, V.
2013
Moving Averages for Financial Data Smoothing
bit.ly/29PGzsg

Tillson, Tim
1988
Smoothing Techniques For More Accurate Signals
Stocks & Commodities V16:1 (33-37)

Hamming, R.W.
1989
Digital Filters, 3rd Edition

Mulloy, Patrick G.
Feb 1994
Smoothing Data with less Lag
Technical Analysis of Stocks & Commodities, V12:2
Jan 1994
Smoothing Data with faster Moving Averages
Technical Analysis of Stocks & Commodities, V12:1

Tukey, John
1976
Exploratory Data Analysis, Addison-Wesley: Reading, MA