Deciding on a Backtesting and Trading Platform

Since, in between the flow of the "Review for the TradersStudio" was broken; so just to help in putting the thoughts together, am putting all the earlier parts of the reviews together for your kind reference, before I continue with the rest of the review.

The links of the above said review are given below in the proper order:


Please have a look at the above and give your feedback.
 
Second Part of the Set of “Review of TradersStudio”

Part 2 of 4

In the earlier review, we discussed about stocks. Now, we shall discuss about using money management, which is a method by which you can judge how much you should or can invest in a market filled with uncertainty in order to maximize the profits.

In TradersStudio, you can optimize money management in both Trade Plan and Session Level. Moreover, you have the option to assign data like S&P500 for this purpose and also use the same with Forex.

In Equity stocks analysis we had used a NASDAQ value as an Intermarket to trade the basket of NASDAQ stocks, which sized the shares using the “split-level” method. This is okay for a Session level trading (“buy-and-hold”). However, for a Trade Plan level, it is better have the system trade a single unit and let the “money management” calculations do the sizing. A session to demonstrate the same is available in the software known as “TSStockSystem”.

In order to create an Equal Equity Trade Plan, there is an in-built high-level Trade Plan in TradersStudio, which invests equally in each stock of the portfolio. You can define different types of Equity like the following:

  • Normal Equity: It is the sum of “Current Equity” and “Starting Equity”
  • Available Cash: The sum of (“Current Equity” & “Starting Equity”) less the (“Total Holding”)
  • Core Equity: The value of the account if all stops are hit at the same time.

Another interesting feature of TradersStudio is the Stock Screens. Here, you can select baskets of stocks to run and view a few selected trades in the screens using different criteria. However, the software can not directly perform the screening at the Session level. You can do the same at the Trade Plan level by canceling those stocks that do not meet the criteria selected for screening purposes.

The adding of screening to the systems is made simple by the low level functions and sorting capabilities provided by TradersStudio. There are filters to use for screening such as:

  • FilterType is “0”: A classic relative strength type filter where percent return over Per Period is looked at.
  • FilterType is “1”: It is Volatility over Per Period.
  • FilterType is “2”: It lets the user develop custom relative measure filters.

However, having a “relative strength” as a filter might not work due to various reasons.

Firstly, large cap stocks behave differently from small cap stocks. The best example is that of the ‘Dogs of the Dow”, wherein the above worked well on the Dow 30, but failed on the Russell 2000. This was so, as the “Dow 30” stocks are strong, so there is no scare of them going out of business. However, large caps companies do fail, for instance the WorldCom and Enron are the best examples. Though, such cases are rare and drastic too.

Secondly, the bull market of the late 1990s was such that it was not possible to beat the NASDAQ 100 stocks (all are large caps companies), due to the large moves that occurred throughout the portfolio.

There is a reason for the bias behavior of the NASDAQ 100 index data, the “survivorship bias". It means that there are a number of stocks which have not been included (like WorldCom) as they are under performing ones and not included in the NASDAQ 100 listing. Thus, including only those stocks of the NASDAQ 100 index, means that the back-test results will not be accurate. Instead it shall be biased and shall show more profits than actual as the under-performing stocks are not included.

TradersStudio Stock supports a powerful method for money management which is unique to it only. However, before explaining this, we need to discuss the “Percent Risk money management in commodities”. In commodities trading, the margin is the cash required to hold a position, which is generally a very small percentage of the actual value of the contract. Thus, cash for margin is rarely an issue no matter how small the risk is.

It means that if for example, you wanted to risk $500 on each Ten-Year Note that is being traded; it would represent a very tight stop of less than the average daily range. However, if you wanted to risk 2% of a $100,000 account, it would mean that you could trade 4 contracts (4 times $500). Since the margin on Ten-Year Notes is $1,800, the margin requirement would be $7,200. Thus, it can be concluded that if you were trading 10 commodities markets with similar margin requirements, you could trade aggressively on a margin basis but still have enough money in your account to fund trading.

However, in the case of stocks, the situation is different as you need money in the Trading Account. It means that if you are trading on stock “X”, priced at $100 per share with a stop of $1. Now, if you have $100,00 in your account plus 2% risk, you have enough funds after adjusting the risk to buy 2,000 shares of the stock. However, the glitch is that you need $2 million to buy 2000 shares! Thus, the real stock world is different. Hence, if we trade multiple stocks, we would be buying different dollar values of stock based on how far the exit stop is from the entry price. Hence, if we use a technical stop (like the lowest of the low in the past 3 days); the level of the risk on a percentage basis would be different. Hence, we can conclude that “Percent Risk money management” does not work in stock trading without advanced forms of analysis.

However, with TradersStudio you can handle these issues can be handled properly, and this is the unique powerful feature mentioned earlier. Thus, our analysis can buy based on Percent Risk but limit how much is spent to some multiple of capital available. This number (limit) should be a maximum of 2-3 times dollars available for a position, if equity was being divided evenly among positions.

The above is possible in TradersStudio with its in-built functions, a powerful money management method provided for the stock traders. It allows them to limit the number of markets traded and use Percent Risk money management based on correct dollars since it is defined for use only with TradersStudio stock data. It also allows filtering based on relative strength. However, you can override the filter can by customizing the settings

Please note that using percent risk in stocks is a little dangerous because it is possible to have the algorithm buy more shares of stock than there is money available. Thus to avoid this kind of situation, a limit has been added to the above which limits as to how many shares of a stock you can trade. It means that with a value of $100,000 in the trading capital and a market portfolio of 100 markets, with an equal division of funds if the account were fully invested, position size would be limited to $1,000 per position.

Hence, if 2% risk is assigned for a system, the position risk shall be $2,000. Thus, if the stock is $5.00 per share, 2% risk means that we can risk up to $0.10 per share. Hence, we can buy 20,000 shares. Therefore, what the algorithm is trying to tell is that we can purchase as many shares of stocks as we have in funds. Thus, the in-built limit prevents this situation. Taking the same case as above, if we were to set the limit to ‘2,’ it means that we can buy only up to $20,000 of this stock i.e. 4,000 shares only (and not 20,000).

Moreover, the above can be viewed in the reports like the “Equity Report”, “Summary Report”, “Annual Break down Report”, and “Yearly Report”.


Thus, we can conclude that TradersStudio has incorporated the powerful features and also overcome the shortcomings of money management in the software. The simplicity by which we could do this was a plus feature along with the screens presented.


To be Continued.....
 
Second Part of the Set of “Review of TradersStudio”

Since the length of this “Part 3 of 4”was too long to post in one go, hence, I have split the same into two sections. Following is “Part 3 of 4 (Section A)” and the post after this is “Part 3 of 4 (Section B)”

Part 3 of 4 (Section A)

In the earlier review, we discussed about stocks. Now, we shall discuss about using walk forward analysis, an important feature of trading and which is missed out by many developers in the making of the software. The fact that TradersStudio offers a well thought out process for walk forward analysis which ensures its success in the future is admirable.

Walk-forward optimization is a process where the system values are optimized on previous market data and then tested on test data (can be real time) which is not part of the data selected earlier. Hence, we have two data segments, the optimized one and the test one and both can be used to test the system accuracy and consistency. Thus, in summary, when we roll the windows forward over time and repeat the process, we only look at the test data for the results and do not use the results of the optimized period.

Please note that both data sets are used in the Walk-forward optimization. Thus, we can derive the parameters from the optimized set and use in the test set. After the optimization of the test set, we need to re-optimize and roll both the time-periods (relating to both data sets) forwards a given number of days for every set of days. It means that supposing we roll the optimized parameters and the test period of 250 days forward. When we now observe the results, we shall use the results of the test period in the reports. It means that if the “old parameters” had us long and the “new parameters” had us short; we would exit at the end of the “old test period” and wait for a new signal on the “new test period”.

The above is desirable since, we traders want one that has worked earlier and shall work in the future. However, we traders are not programmers or developers to know the system working, thus the developers need to take into account every possible situation that can arise so that the system is consistent backwards and forward. A system is judged as to how it performs on the test data and not the optimized one.

Development of trading systems is complex and the developers need to make sure that their system gives consistent results. However, trading systems test a set of data via different sets of trading rules and parameters, while it should be the other way around, namely: testing same rules on different set of data. Thus, systems developed by testing different data tend to fail quickly when they are traded moving forward in real-time. This is a major and common problem in this type of system development and is considered “too few degrees of freedom” (statistical term by mathematicians) or the “curve fitting” (a trading term).

Hence, in the above testing, where same data is tested using different parameters, the “degrees of freedom” of the results are reduced. Although, using historical data does improve performance, but it reduces the chance that these measures shall be profitable in the future.

Hence, a question arises as to how to develop a Trading System and maintain a reasonable “degree of freedom?” The simplest way is to use “out-of-sample testing”, i.e. the data series is divided into two segments covering different time-periods. The system(s) are developed using the first or “in-sample” time-period and then the system(s) is tested to see how it performs on the second or “out-of-sample” period.

Thus, one test is performed on the “out-of-sample” data irrespective as to how many ideas are tried on the “in-sample” data. The “degrees of freedom” are preserved and the “out-of-sample” performance provides a much better indication of real-time system performance. However, there is a point that every time the testing process is repeated with a different system, the “degrees of freedom” are lost. If enough testing is done using the data, the “out-of-sample” data becomes part of the “in-sample” data and the advantage of this method of testing is lost.

Coming back to “Walk-forward testing”, it takes the idea of “out-of-sample” testing to the next level, like an “out-of-sample” testing for steroids. The same can be clarified by an example. We have 12 years of data extending from 1995 to 2006 for the markets that we want to trade. We shall assume that our trading strategy needs a minimum of three years of data for testing and optimization.

Now, we develop and optimize the system in as many ways we wish to using the period of 1995-1997. When we find the optimum method, record the strategies, rules and parameters for future reference and testing with new data for the period from 1998. Now, in the same period, shift the time period to a month ahead, so the first month of the earlier period is left out. It means that if started from Jan ‘95, now start from Feb ‘95. The same process of optimization is to be done and record the rules once more which shall be used in the next month for the period after 1997.

The same process to be repeated till we cross the period of 2006, after that we need to re-test the system as a whole, for the entire period from 1998 to 2006, using the rules and parameters recorded earlier. Now, here we are in fact doing a “new out-of-sample test”. Hence, the system performance for these sample years shall be a much better indication of how a system will perform in real time than the performance of any single time period used for optimization.

Please note that taking the above assumed time periods (three years for system development and one month for the walk-forward interval), was simply a trade-off between optimization time and statistical validity of the results. Practically, it was found out that using about 20% of the optimization period for the walk- forward window worked reasonably.

Moreover, time periods can be different, saying a 10 year period is taken with 1 year shift and testing phase is done accordingly on whole system and in 1 year shift phase.


To be Continued.....
 
Second Part of the Set of “Review of TradersStudio”

This is the “Section B” of the same part “Part 3 of 4”.

Part 3 of 4 (Section A)

Advantages of this Walk-forward optimization process are:

  • If the above results (“out-of-sample” months) turned out well, the same walk-forward process can be used in real time to find the parameters to use with real money.
  • Our system shall better adapt to changes in market behavior over time, since the markets do change with time. Many systems worked well for years but failed when markets changed.

Walk-forward testing is a hot topic and has many issues to be addressed such as the following:

What if the optimal parameter value changes as the testing moves to a different time window, how do we change the parameter used in the “out-of-sample” walk-forward period? If a parameter value changes from “20” to “22,” it is not much of a problem but if there are two peaks with one at “20” and the other at “50” and the optimal value changes from “20” to “50” on a single walk-forward step, what value do we use?

The above problem is called “transitional trade” or “boundary trades” in walk-forward analysis and is easily taken care of by TradersStudio by setting up a “transitional trade” that is handled by different rules.

An important point to note is that we cannot expect walk-forward results to be as impressive as the results from in-sample optimization. It means that if the results from walk-forward testing are 50%-75% of the in-sample testing profits, it shows that the system is working as expected and gives realistic numbers as to how the system will perform moving forward in real time.

Moreover, it is imperative to understand that the “out-of-sample performance” is used to judge how good the system is. However, a major problem with doing this type of testing is that there is currently no software (except TradersStudio) available that does walk-forward optimization. Hence, the fact that TradersStudio provides for this is a big advantage.

Thus, the powerful features of the TradersStudio walk-forward optimizer are:

  • It has numerous options, like a length entry for each optimization window plus the length for the out-of-sample run window. It also includes the variable search criteria for isolating the best parameter set.
  • It can also be used to generate orders for the next day trading and can be used for much more than just back-testing.
  • We can also optimize the best system walking forward by using “Optimal f” as the search method. “Optimal f” is normally used as a money management methodology, but it has the trade distribution of the system and gauges system performance. A higher “Optimal f” value means that the system is performing better. Using “Optimal f” as an optimization search method illustrates that it is possible to create complex measures of performance and use them in a systematic walk-forward testing approach.
  • The ability to export the trade history TradersStudio has run as a system to recreate the results.
  • Moreover, it uses the “Export Trade-by-Trade technology”; it produces a Text file that TradersStudio can use to recreate a full normal system run. Thus, a trade plan can be imported to our walk-forward analysis with a little macro programming.
  • We can export the Walk-Forward trades.
  • It can work on a single market or a portfolio of markets.

Besides the above features, the Walk-Forward test shows many useful reports as described below:

  • The Walk-Forward Periods Report shows the start and end dates for the trading and the run out-of-sample periods. It also shows the Net Profit, Percent Winning Trades, and Drawdown for each time-period.
  • The Raw Trades Report shows all trades for the trading period and the run period for each window.
  • The Clean Trades Report shows only the out-of-sample trades with the boundary points between resolved trades.
  • The Clean Trades by Period Report labels each walk-forward period and all transitional trades. This window tells what parameters were used in each walk-forward session and how the boundary trades were handled.
  • The Active Order Report shows the orders to be placed for tomorrow’s trades.

It also provides for Charts like the “Start Trade Drawdown” and “Underwater Equity Curve”.

Thus, a simple conclusion is that although, “Walk-forward analysis” is an important and powerful requirement in Trading Systems, it was not available in any of the products, until TradersStudio included it. TradersStudio present version 2.5 has a fully integrated walk-forward analysis program; a major plus point over the other software. Moreover, it is imperative to have this as with the present changing markets; we require walk forward testing to develop reliable and robust systems.

Hence, this is another reason, why I chose this over the others.


To be Continued.....
 
Quote from Gyles:

Second Part of the Set of “Review of TradersStudio”

This is the “Section B” of the same part “Part 3 of 4”.

Part 3 of 4 (Section A)

.......................................
More...


Please read the last line shown above as:

Part 3 of 4 (Section B)

Sorry and thanks for reading.
 
Second Part of the Set of “Review of TradersStudio”

Part 4 of 4

It has been noticed that many back-testing platforms claim to support Forex (Foreign Exchange) currency trading, however many issues have not been dealt by them. For example, the margin of an account is based on the leverage that the account was set up with i.e. the margin for ‘Trader A’ can differ from that of ‘Trader B.’

Moreover, in present times and new trading conditions, it is at times overlooked that trading Forex is entirely different from those of trading futures or equities as seen below:

  • The exchanges markets manage what is viewed as the historical data bank, while in Forex each broker maintains their own historical database and trade in isolation.
  • Moreover, the retail trader’s order is sold at the ‘bid’ price and bought at the ‘ask’ price i.e. the spread is in ‘pips’. This is the way these broker(s) earn instead of the commission charged on a per trade basis like futures or equities.

There are many issues to be dealt with trading Forex, as follows:

  • The sources for composite Forex data like E-Signal Forex with its composite quote of multiple Forex brokers and banks. It has a problem as the broadcast price is based on an average bid (or offer) making it possible to have a trade that occurs outside the daily range since that range is composed of average bids and offers while a fill may be at the actual highest ask or lowest offer of the day.
  • The margin is not established by the exchanges. It is possible to establish an account with a pre-arranged leverage level, which can be as high as 400 to 1 for some brokers and lot sizes can vary from full lots to various sized mini lots.
  • Forex trading involves pairs of currencies. The first member of the pair is called the base currency and the second member is called the quote currency. The value of the pair at any time indicates how much of the quote currency is required to buy one unit of the base currency.

Thus, Trading Forex requires the currency conversion like trading foreign futures plus the interest profit or loss if trades are held overnight. If you wish to calculate the interest incurred by a trade, you need two rates, the rate for the “base currency” (B), and the rate for the “quote currency” (Q).

Thus, considering the above pair of currencies as “B/Q”, the following possibilities exist:

  1. If the trade is long “B/Q”, interest is received for borrowing ‘B’ with the rate determined by ‘B’s’ borrowing rate. Interest is charged from the lending interest for ‘Q’ determined by ‘Q’s’ lending rate.
  2. If the trade is short “B/Q”, then interest is charged from the lending interest for ‘B’ and received the borrowing interest for ‘Q.’

In both above cases, the difference between lending interest and borrowing interest or swap is incurred. If lending interest is greater than borrowing interest, then interest is paid to us; else, an interest cost is incurred. However, to calculate the difference, both interest values need to be converted into the account currency. If the account currency is the same as either the base or the quote currency, then no conversion is required for that currency.

Please note that for a particular currency, the borrowing interest rate is always higher than the lending interest rate and interest applies if a trade is held overnight (considered as 5 PM EST time).

As a conclusion, for Forex trading, you require historical data provided by your Forex broker if you want realistic results. You also need to include the effects of interest rates on your trades, which can be sizable.

Mostly, Forex brokers allow data to be downloaded in .CSV file format. Some people trade, called the “End-of-Day Forex”, wherein they place their trades around 5 PM EST rollover time.

As seen in the earlier reviews, among the many other features of TradersStudio is that they have designed Forex trading in such a manner that it addresses the issues (mentioned above), which people dealing with Forex Trading wish to be addressed to. However, the same is not the case with other platforms like TradeStation.

Thus, the issues addressed by TradersStudio are:

  • Relating to the borrowing and lending rate, so that we can see how our system profits are affected by the cost of carry. Thus, if we have a Forex system which holds trades for several days, this effect can have a 20%-30% effect on the final profit or losses, not a small margin, if you see. Moreover, the effect is more, if the system is for a longer term.
  • Relating to the money management. Thus, when we apply the same to Forex, the margin is based on the leverage applied to the account. Moreover, TradersStudio allows the setting of the same when a session is set up. In addition to this, we can set up the data to use a given pip spread and even add slippage and commission in pips if required. Hence, the results of the system are shown in both dollars and pips.

As noticed that TradersStudio is an offline product that supports intra-day back testing. It has the following features:

  1. It supports back testing using daily Forex data and Intra-day data.
  2. While setting up new Forex data in TradersStudio, you can add the data and configure the same.
  3. It has standard reports available for Forex analysis, like the Active Orders report, Monthly Breakdown, Annual Breakdown, Correlation Report, Custom Report and Color Report. Others include the Summary Report and the Consolidated Report. The consolidated one is similar to the Trade-by-Trade Report for futures trading plus an interest column calculated which is used to adjust the trade P/L based on whether interest was earned or paid. This report is in dollars. Another report is the Trade-by-Trade Report for Forex in “PIPs”.

    Reports shall be different from the Trading ones as the working of the Forex Traders are different from the others as they do not use dollars or percents. Instead, they use ‘PIPs’, (like a ‘min move’ in futures).
  4. It also has futures settings, which allows you to create a setup for “back testing or forward testing Forex systems”. Hence, it is possible to assign an exchange file (sometimes two are required, one for the minimum move value and other for the margin) to a pair that allows an accurate calculation of returns for pairs that do not have the U.S. Dollar as the quote currency. These exchange files are regular Forex files, which can be used for used for back testing.

    Once more, it is emphasized that for real-time operation, the data source needs to be updated continuously. If a particular data source is associated via a Data Universe with a particular symbol, then it should be possible to determine the necessary exchange files (or data sources) automatically.

    The above can be clarified by the following example, supposing, the U.S. dollar is the currency of the trader’s account. Then, we need the data for the “quote currency” against the U.S. dollar for the calculation of the Min Move Value and the data for the “base currency” against the U.S. dollar for the calculation of the Margin. The operation is determined by the position of the U.S. dollar in the pair. If it is the base currency, the calculation is to divide; while if it is the quote currency, the calculation is to be multiplied.
  5. Pinnacle Data CLC database, includes the “End-of-Day Forex and Interest Rate Forex files” in the TradersStudio special version of the CLC database.

As a conclusion, although at present the TradersStudio is not real time, it can be used only for the “End of Day Forex Trading” i.e. we need to enter the trades at around 5:00 pm (the standard for End of Day). However, when TradersStudio does becomes a real time product (which it shall be very soon), it will be a very powerful Forex tool, as the real world issues dealing with the Forex Trading has already been addressed with.
 
CONCLUSION

Now, the much awaited conclusion of the whole thread is as follows. I did realize that the thread and reviews are becoming too long and thought of winding it off fast, so that we can discuss the working of TradersStudio more practically via examples.

We have reviewed many trading platforms namely in this thread Getting Started Developing and Trading System

AND

Deciding on a Backtesting and Trading Platform

However, as a note it is important to understand that several factors determine the choice of the product selected by you; such as whether you are trading “Intra-Day” or using “End of Day” data, how much money you can afford to spend on purchasing any software etc.

Now, each of the products reviewed has it own plus and minus points and to refresh the memory I am enclosing the links of the same:


After we go through each of the reviews, the following few points become apparent:

  • Firstly, if you desire a real time product, then TradeStation would be a good choice despite its weakness. However, it would not have been the case had Trading Blox or TradersStudio also had real time support.
  • Secondly, supposing that you are interested in trading system and in 100% objective methodologies, then Trading Blox and TradersStudio are your two best choices.

Now, if we were to compare “Trading Blox” and TradersStudio”, the following points arise:

  • Trading Blox is a pure system platform, however its’ chart support is weak. TradersStudio charts are good, however might not be as good as TradeStation, but close enough in terms of its features like, expert commentary etc. which is supported by TradeStation.
  • On the basis of System Testing and Development, both products are about even. However, TradersStudio has better add-in support plus the useful feature the “Walk Forward Testing”.
  • Considering the price, here is way the main disparity occurs. TradersStudio cost $ 599.00, while the comparable version of Trading Blox cost is $ 2995.00. Thus, the price made a big difference in the final decision. Moreover, I have worked with the Trading Blox demo and at present own TradersStudio. I found out in my research that TradersStudio can do anything which Trading Blox can do. Moreover, I find TradersStudio more flexible.

At present, I am planning to my System Development Research and shall use TradersStudio for the same. As a conclusion, I would like to say that based on the research done in the past few months or so, I have found a suitable product i.e. TradersStudio.

I find it suitable as:

  • It delivers what it says
  • Its features are a lot more than what one might expect at its price.
  • It is also the only product reviewed so far, which comes with Printed Bound Manuals (this made researching easy).
  • The best part is their customer service and technical support very good, much better than I could have expected (as it is a small company).

As a whole, I am satisfied and happy to have purchased TradersStudio.
 
You must log in or register to reply here.
Back
Top