Can ChatGPT backtest a trading strategy?

The short answer

Partly. An AI assistant such as ChatGPT can help with the thinking around a backtest: it can turn your rules into a clear checklist, point out where they are ambiguous, draft code and explain what a figure means.

A backtest also needs verified price data, a simulation that follows your rules literally, a cost model and a record of every trade. A chat window on its own supplies none of those. The numbers have to come from a run on real data.

Backtest

A run of fixed trading rules over past market data, producing a record of every trade the rules would have taken.

Fill

The price at which an order is assumed to execute.

Look-ahead

A rule or a line of code that uses information which did not exist yet at the moment it acts.

Data source

Where the price bars come from. Its bar size, history and treatment of contract rolls decide what can be tested.

What an AI assistant can do

Used as a helper, it can save time on the parts that are about language and code.

• Restate your rules back in plain words, so you can see whether it understood them.
• Find the places where a rule can be read two ways, such as whether a time range includes its last minute.
• Draft code for a platform’s scripting language or for Python.
• Explain terms and figures, such as R, drawdown or expectancy, in simple language.
• Suggest questions to ask of a result, such as how it looks year by year.

What it cannot do on its own

It cannot hand you verified price bars. If it states how a strategy would have done, that statement is text. It is not the output of a simulation on real data unless a tool actually ran one, and even then the data and the fill rules behind it decide whether you can trust the answer.

Code it writes can contain mistakes you cannot see by reading it. A rule can quietly use information from after the moment it fires. A fill can be placed at a price that never traded. You find those by running the code on data you trust and reading the trades.

A backtest result has to come from a run on real price data. A number that was only written out in a chat is not one.

Why fills and costs change the answer

Two people can run the same rules and get different results because their fill rules differ. Where does an entry fill? What happens when the stop and the target are in the same bar? What does a gap do?

In our method a target fills on touch at the exact price. A signal that appears when a bar closes can first be acted on at the next bar’s open. A stop fills at the stop price plus slippage, or at the bar’s own open if price gapped through it. When a stop and a target share a bar on a timeframe above one minute, the 1-minute bars decide which was hit first, and if they share a 1-minute bar the stop is taken.

Costs matter just as much. Commissions and fees vary by broker; a backtest should use your own. A result with no stated costs is a result no account could have had.

If you use an AI assistant anyway

A few habits keep it useful.

• Ask it to list every assumption it made about fills, costs and time zones.
• Run the code on data you know, and compare a few trades against the chart by hand.
• Check every input for look-ahead: could this have been known at that bar?
• Read the trades, not only the summary.

How we do it

You describe your rules in plain language, the way you would explain them to another trader. If something is ambiguous, we normally ask you the question first.

We run them on historical CME futures data licensed from Databento, with the assumptions written out, and you get every trade back. A version is $19. A run takes two to three business days.