PST Backtest Questions

[kylemarino22]

Hi all,

I'm relatively new to this project but have spent the last couple of months reading AFTS (and Rob's blog) and bringing up PST. I'm currently trying to get a backtest of the dynamic system with good performance through 2024, but I keep seeing it level off for the last few years despite good overall SR. I'm finding that my results with current forecast and weight estimates perform similarly to Rob's prebaked system which is a year out of date, so it's possible I have it configured wrong. I'm wondering if other people are seeing similar results or have backtests with good SR for the last few years.

I have a few other questions and general comments:

• I see that bootstrapping has been removed as an optimizer so are other people using handcrafting or shrinkage? I'm curious how these compare.
• Is it intended to go through interactive_update_roll_status daily as well as always running monitor.py to close scripts that have crashed? I might not be using this system exactly as intended.
• I found some slight numerical stability issues in the dynamic optimization code, which again, I'm wondering if others have experienced this too. (I've fixed the problems in my version, can make a PR)
• I changed a few parameters in the rollconfig.csv, seems some instruments had mistakes.
• I have written a script that pulls data from IB and updates multiple prices.
• I have also made a cache checkpointing system so that if a backtest crashes it can be resumed.

Thanks in advance for any help,
Kyle

[vishalg]

• It may be helpful to compare your backtest results with the BTOP50 or SocGen CTA indices. I think this strategy it as a "crisis-alpha"/diversifier strategy which affords one higher leverage in other potentially racier parts of the portfolio - as a standalone it probably won't beat simpler strategies (buy and hold/tactical asset allocation etc). The positive skew and negative correlation with most other asset classes is what makes it so attractive.
• Instrument weights - I use fixed weights calculated using the asset class based top-down methodology
• Roll status update - I run it daily when there are futures which need to be rolled but don't have enough contract/relative volume. Then I'd leave it for a 3-4 days and revisit. Rolling in my view is very much a subjective process, you can have useful thumbrules but eventually futures are not entirely hands-off trading instruments
• Please could you share what you have discovered re the instability in the dynamic optimisation code? I have been running the dynamic optimisation system without the execution stack and optimized positions do change a fair bit on a daily basis but that's because the system does not find any current positions in the portfolio
• Again helpful if you can share the mistakes you have spotted
• Isn't this the same as what Rob's scripts do (seed_from_IB and the daily update scripts)?
• This would be helpful as well
  The code may not be accepted in the project - but I am curious to learn what you have spotted.

[kylemarino22]

I did see his published results on his blog, and I was mostly comparing my own with the backtest in this post (https://qoppac.blogspot.com/2021/12/my-trading-system.html) but it's from 2021. I get a similar overall sharpe of 1.3, but the curve has a downward concavity, which I'm not sure if it's from poor parameter optimization or from a general deceleration of carry/trend profitability. Maybe I'll look into adding more instruments once I get more of the system running.

Ideally, I would like to fully automate rolls. I know there's an option to do this in interactive_update_roll_status, but I haven't really looked much into what the code actually does and what the drawbacks are of mindlessly using that feature.

I looked into the volatility stability issue some more, and it comes from trying to backfill the first nan volatility estimates with the minimum volatility value (1e-10). This seems to be a bit of an edge case where only if the first few days after the nan estimates have 0 volatility they'll get replaced with the minimum value. Setting small volatility values to nan in get_average_position_at_subsystem_level seems to work ok since it'll prevent the optimization alg from looking at the instrument for that day, but it's a bit of a hacky solution.

[vishalg]

• A CTA firm and Rob's latest published numbers to compare against. I'd focus more on correlation with return streams rather than distributional profile of returns (sharpe/skew etc). Again I must stress that trend following is not a pure alpha strategy in the sense of say a long-short equity.
• Not saying that rolls can't be automated, but I believe there will be instances (frequently enough) where you would want to roll earlier than a static rule based procedure. That said, you could also implement a backtest optimised roll strategy
• That rings a bell but I can't seem to replicate it anymore. Edge cases are always worth fixing if you can come up with a solution.

[vishalg]

@kylemarino22 - Have you tried running the backtest with a very large capital, say about $1bn? Just a thought, but it may be the case that as prices go up with time it becomes harder to hit optimal allocation to an instrument as the contract sizes become larger (and we can't trade fractional contracts). Although, it still remains the case that there has been a qualitative difference between trend following performance pre and post GFC, which may simply due to the macro economic regime - in which case things should look a lot more like pre-GFC era in the next decade.

[kylemarino22]

Just tried the $1bn backtest. I had to use the static system since the dynamic alg was too slow, and I get an overall sharpe of 1.375 and from 2000 on it's 1.06. It seems that with larger capital, the volatility target is hit more precisely due to closer-to-optimal instrument allocations.

I keep finding that the idm estimate is too low for small accounts in the dynamic backtest. I think it's because the idm is being calculated on the full portfolio and not the small subset after dynamic optimization, but since this value doesn't change much it's probably ok to hardcode it.

I think there's room improvement wrt forecast and instrument weight estimates. I might try to get a version of bootstrapping working if I have the time.

[vishalg]

Should have warned you! With £1bn you are not capital constrained anymore so dynamic optimisation is essentially recreating the original portfolio every day.

I also use a fixed value of IDM - 1) it should be mainly a function of number of instruments and fairly stable w.r.t choice of instruments 2) I do find it a bit redundant as you can achieve the same effect by changing the volatility target even though it is conceptually cleaner to separate the two.

Bootstrapping will be very slow for weight estimation to run on a live system but will be a useful research exercise.

[anna-gie]

What backtest results are you getting using the provided rob_system?

After a fresh git clone of pysystemtrade, I ran a backtest using the provided CSV files with the following parameters:

notional_trading_capital: 500000  
base_currency: "USD"  
forecast_post_ceiling_cost_SR: 0.13  

Since there was no data available for some instruments, I added them to the ignore_instruments list as follows:

ignore_instruments:
  - 'NIFTY'
  - 'USIRS10'
  - 'USIRS2ERIS'
  - 'USIRS5'
  - 'USIRS5ERIS'

The backtest produced the following results:

1) Ideal portfolio(non DO):

from systems.provided.rob_system.run_system import futures_system
from sysdata.sim.csv_futures_sim_data import csvFuturesSimData
import matplotlib.pyplot as plt 
import pandas as pd

data = csvFuturesSimData()
system = futures_system(sim_data=data)

non_do_portfolio = system.accounts.portfolio() #no DO

non_do_portfolio.percent.curve().plot()
plt.show()
non_do_portfolio.percent.curve().tail(2500).plot()
plt.show()

print(non_do_portfolio.percent.stats())

Produces following:

Last 9 years:

('min', '-8.706'),
('max', '9.235'),
('median', '0.09021'),
('mean', '0.1101'),
('std', '1.183'),
('skew', '-0.03901'),
('ann_mean', '28.18'),
('ann_std', '18.93'),
('sharpe', '1.489'),
('sortino', '2.079'),
('avg_drawdown', '-7.33'),
('time_in_drawdown', '0.9044'),
('calmar', '0.7666'),
('avg_return_to_drawdown', '3.845'),
('avg_loss', '-0.803'),
('avg_gain', '0.8658'),
('gaintolossratio', '1.078'),
('profitfactor', '1.303'),
('hitrate', '0.5472'),
('t_stat', '11.06'),
('p_value', '2.533e-28')

2) Portfolio with DO:

from systems.provided.rob_system.run_system import futures_system
from sysdata.sim.csv_futures_sim_data import csvFuturesSimData
import matplotlib.pyplot as plt 
import pandas as pd

data = csvFuturesSimData()
system = futures_system(sim_data=data)

portfolio = system.accounts.optimised_portfolio() #with DO

portfolio.percent.curve().plot()
plt.show()
portfolio.percent.curve().tail(2500).plot()
plt.show()

print(portfolio.percent.stats())

Produces:

Last 9 years:

('min', '-9.027'),
('max', '9.352'),
('median', '0.06928'),
('mean', '0.08798'),
('std', '1.302'),
('skew', '0.01656'),
('ann_mean', '22.52'),
('ann_std', '20.83'),
('sharpe', '1.081'),
('sortino', '1.56'),
('avg_drawdown', '-15.26'),
('time_in_drawdown', '0.9227'),
('calmar', '0.2782'),
('avg_return_to_drawdown', '1.476'),
('avg_loss', '-0.922'),
('avg_gain', '0.9763'),
('gaintolossratio', '1.059'),
('profitfactor', '1.206'),
('hitrate', '0.5325'),
('t_stat', '8.032'),
('p_value', '1.037e-15')

The optimisation settings were left at their default values:

small_system:
  shadow_cost: 50
  cost_multiplier: 1.0
  tracking_error_buffer: 0.0125
  shrink_instrument_returns_correlation: 0.5

If I remove the forecast_post_ceiling_cost_SR: 0.13 parameter, the Sharpe ratio improves slightly, but the performance discrepancy remains.

The DO portfolio shows a significant drop in performance compared to the ideal (Non-DO) portfolio.
I’ve noticed that others have shared account curves similar to my Non-DO results, but after using DO.

Additionally, I checked the list of "expensive" instruments, and they are as follows:

expensive_instruments=[] 
for instrument in system.data.get_instrument_list():
    cost = system.accounts.get_SR_cost_per_trade_for_instrument(instrument)
    if cost>0.01:
        expensive_instruments.append(instrument)    
print(expensive_instruments)  

Results:
SGX', 'VNKI', 'IRS', 'TIN_LME', 'WHEY', 'MILKDRY', 'SMI-MID', 'BOVESPA', 'BUTTER', 'COAL-GEORDIE', 'FTSEINDO', 'MILKWET', 'CZK', 'STEEL', 'EURIBOR', 'CH10', 'COTTON', 'SUGAR16', 'HOUSE-US', 'FTSEVIET', 'COAL', 'GASOILINE_micro', 'JGB-mini', 'BB3M', 'AEX_mini', 'CNH-onshore', 'ETHANOL', 'OMX', 'WHEAT_ICE', 'SWISSLEAD'

No positions are taken in these instruments, so I don't think they are causing the issue.

Costs and Turnover:

portfolio=system.accounts.optimised_portfolio().percent

portfolio.costs.ann_mean()
portfolio.gross.ann_mean()
system.accounts.total_optimised_portfolio_level_turnover()

Results:
Costs annual mean:
-0.829160430870585

Gross annual mean:
23.350083163583164

Portfolio Level Turnover:
15.450447236495762

My question is:

Did you achieve this out-of-the-box, or did you tweak parameters (e.g., shadow_cost) to bring the DO portfolio closer to the ideal portfolio? Maybe a lower shadow_cost?

Could this large drop in DO performance be expected due to default parameters, or could there be a deeper issue with the data or configuration?

Any insights or suggestions would be greatly appreciated.

[kylemarino22]

Could you share the full config file that you're using? It's worth trying the backtest with all of the estimates set to false since rob's provided config file is manually tuned for whenever he published it. My own csv data is a little messed up since I've updated it and only use db sim data for my own tests so I can't exactly recreate your results, but I remember running into similar issues. I also haven't messed with the DO parameters, but that might be something worth tweaking.

[anna-gie]

The /provided/rob_system/config.yaml is the original one, the estimates are set to false, only use_instrument_div_mult_estimates: is set to True. Forecast scalars, instrument and forecast weights are not changed.

Here the head of config.yaml:

#YAML
# start_date: 2000-01-19
use_forecast_weight_estimates: False
use_forecast_div_mult_estimates: False
use_instrument_weight_estimates: False
use_instrument_div_mult_estimates: True
use_forecast_scale_estimates: False
#
# Raw data
#
#
# Trading rules
#
volatility_calculation:
  func: "sysquant.estimators.vol.mixed_vol_calc"
  days: 35
  min_periods: 10
  vol_abs_min: 0.0000000001
  slow_vol_years: 20
  proportion_of_slow_vol: 0.35
  backfill: True

Here the interesting parts of the /sysdata/config/defaults.yaml. I changed the capital to 500000 USD, the rest should be unchanged:

volatility_calculation:
  func: "sysquant.estimators.vol.mixed_vol_calc"
  name_returns_attr_in_rawdata: "daily_returns"
  multiplier_to_get_daily_vol: 1.0
  days: 35
  min_periods: 10
  slow_vol_years: 10
  proportion_of_slow_vol: 0.3
  vol_abs_min: 0.0000000001
#
forecast_scalar: 1.0
#
instruments_with_threshold: []
use_forecast_scale_estimates: False
#
forecast_scalar_estimate:
   pool_instruments: True
   func: "sysquant.estimators.forecast_scalar.forecast_scalar"
   window: 250000
   min_periods: 500
   backfill: True
#
# capping
forecast_cap: 20.0
#
average_absolute_forecast: 10.0
#
forecast_div_multiplier: 1.0
#
use_forecast_div_mult_estimates: False
#
forecast_correlation_estimate:
   pool_instruments: True
   func: sysquant.estimators.pooled_correlation.pooled_correlation_estimator
   frequency: "W"
   date_method: "expanding"
   using_exponent: True
   ew_lookback: 250
   min_periods: 20
   cleaning: True
   rollyears: 20
   floor_at_zero: True
   forward_fill_data: True
#
forecast_div_mult_estimate:
   func: sysquant.estimators.diversification_multipliers.diversification_multiplier_from_list
   # this moving average is in business day space to smooth jumps
   ewma_span: 125
   dm_max: 2.5
#
use_forecast_weight_estimates: False
#
forecast_cost_estimates:
   use_pooled_costs: False
   use_pooled_turnover: True
#
# Smooth forecast weights; weights are in business days
forecast_weight_ewma_span: 125
#
# Set this to 0.13 to enforce the speed limit even for fixed weights or
#   regardless of what an estimator does
forecast_post_ceiling_cost_SR: 999
#
forecast_weight_estimate:
   func: sysquant.optimisation.generic_optimiser.genericOptimiser
   pool_gross_returns: True
   equalise_gross: False
   cost_multiplier: 2.0
   apply_cost_weight: False
   ceiling_cost_SR: 9999
   frequency: "W"
   date_method: "expanding"
   rollyears: 20
   method: handcraft
   cleaning: True
   equalise_SR: False
   ann_target_SR: 0.5
   equalise_vols: True
   shrinkage_SR: 0.9
   shrinkage_corr: 0.5
   correlation_estimate:
     func: sysquant.estimators.correlation_estimator.correlationEstimator
     using_exponent: True
     ew_lookback: 50000
     min_periods: 10
     cleaning: False
     floor_at_zero: False
   mean_estimate:
     func: sysquant.estimators.mean_estimator.meanEstimator
     using_exponent: True
     ew_lookback: 50000
     min_periods: 5
   vol_estimate:
     func: sysquant.estimators.stdev_estimator.stdevEstimator
     using_exponent: True
     ew_lookback: 50000
     min_periods: 5
#
# Capital correction
#
percentage_vol_target: 25.0
notional_trading_capital: 500000
base_currency: "USD"
# other options are full_compounding and half_compounding, but see syscore.capital.py
capital_multiplier:
   func: syscore.capital.fixed_capital
#
# add instrument codes here to make long only at position sizing
long_only_instruments:
  - "an_instrument"
  - "another_instrument"
# Portfolio creation
#
instrument_div_multiplier: 1.0
#
use_instrument_div_mult_estimates: False
#
instrument_correlation_estimate:
   func: sysquant.estimators.correlation_over_time.correlation_over_time_for_returns
   frequency: "W"
   date_method: "expanding"
   using_exponent: True
   ew_lookback: 25
   min_periods: 20
   cleaning: True
   rollyears: 20
   floor_at_zero: True
   forward_fill_price_index: True
#
instrument_div_mult_estimate:
   func: sysquant.estimators.diversification_multipliers.diversification_multiplier_from_list
   # smooth IDM in daily space
   ewma_span: 125
   dm_max: 2.5
#
use_instrument_weight_estimates: False
#
# smooth weights they will are in daily space
instrument_weight_ewma_span: 125
#
instrument_weight_estimate:
   func: sysquant.optimisation.generic_optimiser.genericOptimiser
   method: handcraft
   frequency: "W"
   equalise_gross: False
   cost_multiplier: 1.0
   apply_cost_weight: False
   date_method: "expanding"
   rollyears: 20
   cleaning: True
   equalise_SR: True
   ann_target_SR: 0.5
   equalise_vols: True
   shrinkage_mean: 1.00
   shrinkage_corr: 0.50
   monte_runs: 100
   correlation_estimate:
     func: sysquant.estimators.correlation_estimator.correlationEstimator
     using_exponent: True
     ew_lookback: 500000
     min_periods: 10
     floor_at_zero: True
     cleaning: False
   mean_estimate:
     func: sysquant.estimators.mean_estimator.meanEstimator
     using_exponent: True
     ew_lookback: 50000
     min_periods: 5
   vol_estimate:
     func: sysquant.estimators.stdev_estimator.stdevEstimator
     using_exponent: True
     ew_lookback: 50000
     min_periods: 5
#
# buffering / position inertia
buffer_method: forecast
buffer_size: 0.10
buffer_trade_to_edge: True
# costs and accounting
use_SR_costs: False
vol_normalise_currency_costs: True
multiply_roll_costs_by: 0.5
#
# risk overlay - make these numbers and uncomment
#risk_overlay:
#  max_risk_fraction_normal_risk: 99999
#  max_risk_fraction_stdev_risk: 99999
#  max_risk_limit_sum_abs_risk: 99999
#  max_risk_leverage: 99999
#
instrument_returns_correlation:
  func: sysquant.estimators.correlation_over_time.correlation_over_time_for_returns
  interval_frequency: "W"
  frequency: "7D"
  date_method: "rolling"
  using_exponent: True
  ew_lookback: 75
  min_periods: 10
  cleaning: True
  offdiag: 0.0
  rollyears: 5
  floor_at_zero: False
  forward_fill_price_index: True
  clip: 0.99
# small system optimisation
small_system:
  shadow_cost: 40
  cost_multiplier: 1.0
  tracking_error_buffer: 0.0125
  shrink_instrument_returns_correlation: 0.5

And here the /private/private_config.yaml with the forecast_post_ceiling_cost_SR:parameter set:

forecast_post_ceiling_cost_SR: 0.13
# Capital correction
#
percentage_vol_target: 25.0
notional_trading_capital: 500000
base_currency: "USD"

duplicate_instruments:
  include:
    things: 'thing_we_want'
  exclude:
    things: ['bad_thing', 'Another_thing']

exclude_instrument_lists: 
  # ignore_instruments are ignored in backtests
  # we still collect price data for them in production, do rolls etc
  # You can override this by modifying the backtest config .yaml file used in either sim or production
  # If you want to ignore because a duplicate market, use duplicate_instruments
  # If you want to include in sim data, but not actually trade, use trading_restrictions
  #
  # BEST FOR INSTRUMENTS THAT LOOK EXPENSIVE, BUT WHERE WE HAVEN'T GOT ACCURATE DATA YET
  #
  # REVIEW REGULARLY AND CHECK THESE INSTRUMENTS AREN'T BETTER ELSEWHERE...
  ignore_instruments:
    - 'NIFTY'
    - 'USIRS10'
    - 'USIRS2ERIS'
    - 'USIRS5'
    - 'USIRS5ERIS'

Tweaking the shadow_cost to 40 improves the DO result slightly, but still there is a significant performance drop in comparison to non-DO(sharpe 1.489):

('min', '-9.073'),
('max', '11.73'),
('median', '0.07073'),
('mean', '0.09152'),
('std', '1.322'),
('skew', '0.0127'),
('ann_mean', '23.43'),
('ann_std', '21.16'),
('sharpe', '1.107'),
('sortino', '1.591'),
('avg_drawdown', '-13.26'),
('time_in_drawdown', '0.9209'),
('calmar', '0.2978'),
('avg_return_to_drawdown', '1.766'),
('avg_loss', '-0.9357'),
('avg_gain', '0.9905'),
('gaintolossratio', '1.059'),
('profitfactor', '1.212'),
('hitrate', '0.5337'),
('t_stat', '8.226'),
('p_value', '2.099e-16')

Do others experience similar issues with the out-of-the-box settings?

[vishalg]

Can you try with higher capital value e.g $10m? Don't set it too high otherwise the optimisation may take too long. I have a similar drop in sharpe between optimised and non optimised backtests for my customised version of the provided system.

[anna-gie]

After 8 hours of running the backtest(rob_system) using the following settings:

• percentage_vol_target: 25.0
• notional_trading_capital: $10,000,000
• forecast_post_ceiling_cost_SR: 0.13

Here the results:

Trading Capital:
10000000.0

Total Turnover:
14.681128185149532

Costs in % annual mean:
-0.8757409484482601

Sharpe Ratio with DO:
1.0761904339347232

Gross annual mean:
23.72926012153914

('min', '-8.879'),
('max', '10.71'),
('median', '0.07142'),
('mean', '0.08928'),
('std', '1.327'),
('skew', '0.01638'),
('ann_mean', '22.86'),
('ann_std', '21.24'),
('sharpe', '1.076'),
('sortino', '1.551'),
('avg_drawdown', '-15.55'),
('time_in_drawdown', '0.9207'),
('calmar', '0.2385'),
('avg_return_to_drawdown', '1.47'),
('avg_loss', '-0.9315'),
('avg_gain', '0.9984'),
('gaintolossratio', '1.072'),
('profitfactor', '1.205'),
('hitrate', '0.5292'),
('t_stat', '7.995'),
('p_value', '1.394e-15')

I checked the position sizes, and it turns out we do have more positions open across more instruments.

For example, here are the position sizes from the last day of the provided CSV data:

system.optimisedPositions.get_optimised_position_df()

2024-03-29 00:00:00:

• FEEDCOW: 65
• WHEAT: -23
• CORN: -118
• SP400: -1
• SEK: -4
• PLN: 5
• EURCHF: 26
• US10: 8
• DOW: -108
• PLAT: 16
• GICS: 1
• JPY: -8
• ZAR: 14
• SOFR: -1
• CRUDE_W: 15
• DAX: 57
• MSCISING: 57
• US5: 12
• CAD: -16
• GOLD_micro: -518
• LIVECOW: 57
• SOYBEAN_mini: 17
• GBPEUR: -18
• PALLAD: -6
• LEANHOG: 12
• EUR_micro: -83
• AUD: 6
• SOYMEAL: 11
• YENEUR: -4
• JGB-SGX-mini: 165
• US2: -1
• BRE: 29
• GAS_US_mini: -30
• NASDAQ_micro: -16
• SOYOIL: -20
• SP500_micro: 6
• RUR: 30
• COPPER-micro: 136
• GBP: -4
• US20: 14
• RICE: 40
• SILVER: -20
• CHF: -26
• HEATOIL: 4
• NZD: 15
• GASOILINE: -7
• FED: -1
• MXP: 44
• REDWHEAT: -9

It’s astonishing that increasing the capital to $10,000,000 does not result in a performance increase, even though this allows for trading more instruments.

Does anyone else face similar challenges getting the system to perform as expected, or have insights into what might be limiting the results despite increasing capital and trading more instruments?

[kylemarino22]

I forgot about another thing I did which was to use instrument restrictions from an old commit. Not sure why they got removed, but I took them from this commit in the defaults.yaml file: e478a91#diff-975e19401715695535d58466a96564179e02f0f8452c5601c05f40a55a448c1f

[vishalg]

I get similar results with my set-up (i.e drop in sharpe from c 1.45 to 0.95). This is very much in contrast to Rob's results when he published this on his blog.

[anna-gie]

From around 2005 onward, the account curve of DO remains flat, and it would be great to identify the reason behind this.

I've been exploring ways to analyze the results of the Dynamic Optimization (DO) portfolio, and I've encountered an issue with the methods designed for analyzing the "ideal" portfolio.

For example, methods like pandl_for_instrument() or pandl_for_trading_rule_weighted() work well when applied to the ideal portfolio system.accounts.portfolio().
However, these methods don't seem to work on the Dynamic Optimisation object system.accounts.optimised_portfolio().

How do you typically analyze the results of the Dynamic Optimisation portfolio?

Are there alternative methods, workarounds, or workflows to evaluate the performance of the DO portfolio effectively?

[vishalg]

For my configuration, the returns since late 2009 have been lower than in the past but the not the kind of flattening you are seeing.
Red curve is the optimised portfolio and green one is the raw portfolio

The above is with

• capital: £500k
• vol target: 25%
• forecast_post_ceiling_cost_SR: 0.13
• forecast weights using auto_weight_from_grouping, 69% divergent, 21% convergent and 10% skewrv
• instrument_weights: manual top-down weighting (equal weight to each asset class and within each asset class equal weight to all the assets)
• forecast scalars and forecast div multiplier estimated from historical data
• instrument div multiplier of 2.5 (which is very close to the estimate from data)
• c150 instruments in the raw backtest out of which c40 are filtered out via bad_instruments for the optimised portfolio

One does need to take backtests on older data with a pinch of salt though, maybe more instructive to compare vs the performance of an actual trend following CTA and/or the CTA indices.

[anna-gie]

Thank you so much for your advice!
I think the forecast_scalars did cause the issue. After switching them from fixed values to ones estimated from historical data, the results have improved significantly.

[robcarver17]

Another good test is to run DO with a very large amount of capital ($$$ billions) and check the results are similar to static. They won't be identical because the cost penalty and buffering works differently but should be pretty close.