Frosty's trading bot goes live part 2

i think sim data vs live data played a factor.
based on what i see on my on setup..it coulda been crucial. prolly sounds crazy, but for example.. my sim ats was long yesterday til close. (NQ) - same ats on live data entered short position yesterday afternoon, hence missing the big uptrend. sure..it's just 1 instance, but from what i gather from frostys posts, he may not have been prepared for the subtle changes seen moving to live data from sim..
 
Quote from frostengine:

btw just noticed today and yesterday were glitches, didn't check the backtest either day until now.. and the results DID Not match.... backtest shows complety different entry/exit areas.... so somethign is amiss in the software.... this is the only times i've seen this since being live so doesnt explain everything, only that there is a glitch that i need to work on before next live venture
More...

I once had a 'misplaced' line of code cost me tens of thousands of dollars. I never noticed it until later, when I was testing a different parameter set and realized that the results were coming out TOO GOOD.
 
BigBubba,

I believe you are right.. there seems to be some differences in the data between the two that may actually be playing a VERY big role in my live failures.... the data differences may not be huge, but when trading on a small time frame such as the case with my bot.. its possible for the sim account to be one direction and the live account to not be in a trade at all or even the opposite direction...

I have been looking closely at this the past several hours and I really think its a HUGE piece of the puzzle.. glad to see it isn't just my ATS that is noticing the issue.

I am starting to think that forward testing results on the sim account cannot truly be trusted due to different trade decisions could be triggered from slightly different data.

I think it may be more beneficial for me to run the system on the live account and just record all the data it has and have my backtesting engine tell me what i "should" have made... like this I still get an accurate picture of the amount it should have made that day.. and my data wont experience the small differences between the accounts.
 
it's a silly oversight, 1 that i made as well. i'm now rebuilding my ats on live data. not a biggie.. but a lil set back i suppose. only solution i see.. unless your willing to setup on sim data .. trade the signals from there to your live account. sounds a bit complicated. sad part is, i have an ats that is killing the sim data :p (i guess you did too)
 
yup, it was a monster on the sim account ;)

Perhaps this is IB's way of getting people to feel comfortable with their system and switching over to the live account ;)
 
For $50-60 a month you can get DTNIQ data.

I'm running with filtered data from Tradestation currently, it will be interesting to see how sim results compare to real results with the data held stable.
 
Quote from GTS:

Some good discussion here, I'd like to focus on the above statement because its something that I've come back to again and again in my tiny quest to carve out my piece of the pie - is there any hope for a lowly retail ATS trader to compete against the big boys?

My thought is that one advantage retailer bot trades have is that some strategies that can be used by them would be useless to large firms because they do not scale to the level that a large firm requires.

Meaning if you spend millions on developing a system then you need to trade large size to recoup your development cost and that kind of trade size limits the type of strategies you can employ (because you can end up moving the market with your entries and exits).

A retail ATS bot does not face that issue as long as they stick to trading liquid products during RTH.
More...

You don't need to spend millions.

Traders in firms run a bunch of small systems.

Seems to me, people assume that 1 player = 1 system. That's not the case.
 
Quote from GTS:

I don't know why we keep coming back to this point again and again.

Frosty already said that backtesting over the same period as the live forward testing gave the same results (except for some anomaly over the last 2 days where the trades didnt match up- different problem).

If he did not consider slippage correctly in backtesting then backtesting results would not have matched live forward testing results. That is not what happened.

Slippage or any other problem attributed to incorrect backtesting simulation is not the problem.
More...

Is backtesting the only "TEST" you run?

Is Monte-Carlo the only non-backtesting "TEST" you run?

There's "TEST" environment vs. "PRODUCTION" (live) environment.

===================

Another food for thought:

1. Let's say your broker uses some algo. to execute bulk orders (They exist, like MB Trading) Would you test the performance against these algos. to see if your trading model fits it?

2. Let's say your trading Forex. You are using multiple brokers and you can clearly see that each broker's Bid/Ask quotes different. What kind of datafeed would you use to develop your models? How would you execute your trades?

3. Programming and computer skills are one thing. But there's also Risk Management (Math) stuff you need to learn. Risk Management is by far... the most important for any kind of trader regardless of their trading style.

If you can come in and say, "This particular model has xx% chance of working for next xx months." You've passed the sucker level of systematic trading. (You a bit more than Monte Carlo VaR. So the first step is to understand Value At Risk.)

RISK IS KING.

RISK RULES THE MARKET.
 
Quote from frostengine:

I have a few changes. #1 I am going to optimize for lower draw down instead of max PL.
...
More...

As a discretionary (intraday) index futures trader myself, "optimizing" for PnL vs loss containment is not the sign of a professional, skilled trader.

It's good to see you've recognized this.

All the best
Osorico :)


For the automation gurus here:
Assuming apples to apples strategy and trade style, discretionary vs bot...

Is bot money/risk management "different" than discretionary? Are discretionary money/risk management practices worthless in bot construction?
 
Quote from osorico:

For the automation gurus here: Assuming apples to apples strategy and trade style, discretionary vs bot...

Is bot money/risk management "different" than discretionary? Are discretionary money/risk management practices worthless in bot construction?
More...

Sometimes people get mystified by the notion of the machine making trading decisons and managing the risk. In reality, it's not much different from multiplying 345 by 789 by hand versus using a calculator do do the same. If you are a discretional trader and use certain indicators and rules for entries/exits and controlling the risk, there is not much science in automating these rules. The main difference is that the bot would execute them in a predicable and systematic fashion.

Are bot money/risk management "different" than discretionary? Yes, if you want them to be different. Are discretionary money/risk management practices worthless in bot construction? Some of them may be. For example, as a number of ATSers noted, in most cases, adding stop losses to a "stopless" strategy neither reduces the max DD nor improves the P&L. The value of "stops" may as well be a myth propagated by the discretional traders. "Cutting your losses short" sounds sensible, but it just doesn't hold water in systematic tests.
 
You must log in or register to reply here.
Back
Top