MQL -> C# converter

I

InvBox

Hello guys,

I have spare time and like to discuss subj feature. For a few years, I've been developed some algos for FX market and now have a couple robust strategies. I am not much involved in this market and most of my codes strategies related with stock market where I'm using pure C# and MultiCharts (sometimes NinjaTrader). So, for now, I have a mixed solution and would like to prefer to migrate it into single codes uses C#.

There are more reasons to do that:

1) Reduce transaction cost. Unfortunately, my broker charges a little bit more for MT orders than his own native connection.

2) Reduce slippage. Major reason. Using native connection I receive prices much faster than in MQL.

3) Want to start my trading on stocks without modification. My stock broker is IBKR where is not MT.

Brute force - rewrite my codes to C# manually. I've started 2 weeks ago and... goes very slooooow. Monkey work, no interesting. So I ask you guys who did that before. Did you use some automation?

I've seen in google a few abandoned projects. Do not understand why no one supports that. I think it will be a very useful feature. MT now looks very out of date platform and most of brokers now prefer to provide their own solution. Based on FIX/REST connections.

I'm thinking right now about open source project. Maybe I can try to make it. Is anyone who can interesting to use or collaborate? Here is small screen how it may looks like:

mql_csharp-png.195903


What you think about? Let's discuss. :-)
 

Attachments

  • MQL_CSharp.png
    MQL_CSharp.png
    39.6 KB · Views: 571
InvBox said:
Hello guys,

I have spare time and like to discuss subj feature. For a few years, I've been developed some algos for FX market and now have a couple robust strategies. I am not much involved in this market and most of my codes strategies related with stock market where I'm using pure C# and MultiCharts (sometimes NinjaTrader). So, for now, I have a mixed solution and would like to prefer to migrate it into single codes uses C#.

There are more reasons to do that:

1) Reduce transaction cost. Unfortunately, my broker charges a little bit more for MT orders than his own native connection.

2) Reduce slippage. Major reason. Using native connection I receive prices much faster than in MQL.

3) Want to start my trading on stocks without modification. My stock broker is IBKR where is not MT.

Brute force - rewrite my codes to C# manually. I've started 2 weeks ago and... goes very slooooow. Monkey work, no interesting. So I ask you guys who did that before. Did you use some automation?

I've seen in google a few abandoned projects. Do not understand why no one supports that. I think it will be a very useful feature. MT now looks very out of date platform and most of brokers now prefer to provide their own solution. Based on FIX/REST connections.

I'm thinking right now about open source project. Maybe I can try to make it. Is anyone who can interesting to use or collaborate? Here is small screen how it may looks like:

mql_csharp-png.195903


What you think about? Let's discuss. :)
More...
I have built something similar to this but it is not a line by line parser/compiler as shown in your diagram. What I did was to generate a data structure which represents a strategy and then use template and definition files to generate code for a specific platform (AlgoTerminal). Given the following data structure, I can relatively easily write code generators for any platform. This approach works for my use case because I can easily generate the data structure from my research platform. I would be happy to share what I have if you are interested.

Code: 
{
  "ConditionExpressions": {
    "0": [
      {
        "LeftExpressionDataPart": {
          "DataDefinition": {
            "Name": "Close",
            "Property": null,
            "ValueParamaters": null,
            "DataParamaters": null
          },
          "Offset": 0,
          "TimeFrameConfiguration": {
            "Interval": 0,
            "TimeFrame": 2
          }
        },
        "Operator": ">",
        "RightExpressionDataPart": {
          "DataDefinition": {
            "Name": "Sma",
            "Property": "Sma",
            "ValueParamaters": [
              {
                "Name": "period",
                "Type": "int",
                "Value": "200"
              }
            ],
            "DataParamaters": [
              {
                "Name": "dataItem",
                "Type": "double[]",
                "Value": "Close"
              }
            ]
          },
          "Offset": 0,
          "TimeFrameConfiguration": {
            "Interval": 0,
            "TimeFrame": 2
          }
        },
        "RightExpressionValuePart": null
      }
    ],
    "1": [
      {
        "LeftExpressionDataPart": {
          "DataDefinition": {
            "Name": "Close",
            "Property": null,
            "ValueParamaters": null,
            "DataParamaters": null
          },
          "Offset": 0,
          "TimeFrameConfiguration": {
            "Interval": 0,
            "TimeFrame": 2
          }
        },
        "Operator": "<",
        "RightExpressionDataPart": {
          "DataDefinition": {
            "Name": "Sma",
            "Property": "Sma",
            "ValueParamaters": [
              {
                "Name": "period",
                "Type": "int",
                "Value": "10"
              }
            ],
            "DataParamaters": [
              {
                "Name": "dataItem",
                "Type": "double[]",
                "Value": "Close"
              }
            ]
          },
          "Offset": 0,
          "TimeFrameConfiguration": {
            "Interval": 0,
            "TimeFrame": 2
          }
        },
        "RightExpressionValuePart": null
      }
    ],
    "2": [
      {
        "LeftExpressionDataPart": {
          "DataDefinition": {
            "Name": "Macd",
            "Property": "Macd",
            "ValueParamaters": [
              {
                "Name": "fastPeriod",
                "Type": "int",
                "Value": "12"
              },
              {
                "Name": "slowPeriod",
                "Type": "int",
                "Value": "26"
              },
              {
                "Name": "signalPeriod",
                "Type": "int",
                "Value": "9"
              }
            ],
            "DataParamaters": null
          },
          "Offset": 0,
          "TimeFrameConfiguration": {
            "Interval": 0,
            "TimeFrame": 2
          }
        },
        "Operator": "<",
        "RightExpressionDataPart": {
          "DataDefinition": {
            "Name": "Macd",
            "Property": "Signal",
            "ValueParamaters": [
              {
                "Name": "fastPeriod",
                "Type": "int",
                "Value": "12"
              },
              {
                "Name": "slowPeriod",
                "Type": "int",
                "Value": "26"
              },
              {
                "Name": "signalPeriod",
                "Type": "int",
                "Value": "9"
              }
            ],
            "DataParamaters": null
          },
          "Offset": 0,
          "TimeFrameConfiguration": {
            "Interval": 0,
            "TimeFrame": 2
          }
        },
        "RightExpressionValuePart": null
      }
    ],
    "3": [
      {
        "LeftExpressionDataPart": {
          "DataDefinition": {
            "Name": "ChangeInValue",
            "Property": "ChangeInValue",
            "ValueParamaters": [
              {
                "Name": "atrPeriod",
                "Type": "int",
                "Value": "14"
              },
              {
                "Name": "changeBigMultiplier",
                "Type": "double",
                "Value": "1"
              },
              {
                "Name": "changeVeryBigMultiplier",
                "Type": "double",
                "Value": "1.5"
              }
            ],
            "DataParamaters": [
              {
                "Name": "dataItem",
                "Type": "double[]",
                "Value": "Close"
              }
            ]
          },
          "Offset": 0,
          "TimeFrameConfiguration": {
            "Interval": 0,
            "TimeFrame": 2
          }
        },
        "Operator": "==",
        "RightExpressionDataPart": null,
        "RightExpressionValuePart": {
          "Type": "string",
          "Value": "Up"
        }
      },
      {
        "LeftExpressionDataPart": {
          "DataDefinition": {
            "Name": "ChangeInValue",
            "Property": "ChangeInValue",
            "ValueParamaters": [
              {
                "Name": "atrPeriod",
                "Type": "int",
                "Value": "14"
              },
              {
                "Name": "changeBigMultiplier",
                "Type": "double",
                "Value": "1"
              },
              {
                "Name": "changeVeryBigMultiplier",
                "Type": "double",
                "Value": "1.5"
              }
            ],
            "DataParamaters": [
              {
                "Name": "dataItem",
                "Type": "double[]",
                "Value": "Close"
              }
            ]
          },
          "Offset": 1,
          "TimeFrameConfiguration": {
            "Interval": 0,
            "TimeFrame": 2
          }
        },
        "Operator": "==",
        "RightExpressionDataPart": null,
        "RightExpressionValuePart": {
          "Type": "string",
          "Value": "Up"
        }
      }
    ],
    "4": [
      {
        "LeftExpressionDataPart": {
          "DataDefinition": {
            "Name": "ChangeInValue",
            "Property": "ChangeInValue",
            "ValueParamaters": [
              {
                "Name": "atrPeriod",
                "Type": "int",
                "Value": "14"
              },
              {
                "Name": "changeBigMultiplier",
                "Type": "double",
                "Value": "1.5"
              },
              {
                "Name": "changeVeryBigMultiplier",
                "Type": "double",
                "Value": "2"
              }
            ],
            "DataParamaters": [
              {
                "Name": "dataItem",
                "Type": "double[]",
                "Value": "Close"
              }
            ]
          },
          "Offset": 0,
          "TimeFrameConfiguration": {
            "Interval": 1,
            "TimeFrame": 1
          }
        },
        "Operator": "==",
        "RightExpressionDataPart": null,
        "RightExpressionValuePart": {
          "Type": "string",
          "Value": "Up"
        }
      }
    ],
    "5": [
      {
        "LeftExpressionDataPart": {
          "DataDefinition": {
            "Name": "IsTimeInRange",
            "Property": "IsTimeInRange",
            "ValueParamaters": [
              {
                "Name": "startHour",
                "Type": "int",
                "Value": "9"
              },
              {
                "Name": "startMinute",
                "Type": "int",
                "Value": "30"
              },
              {
                "Name": "endHour",
                "Type": "int",
                "Value": "10"
              },
              {
                "Name": "endMinute",
                "Type": "int",
                "Value": "30"
              }
            ],
            "DataParamaters": null
          },
          "Offset": 0,
          "TimeFrameConfiguration": {
            "Interval": 1,
            "TimeFrame": 1
          }
        },
        "Operator": "==",
        "RightExpressionDataPart": null,
        "RightExpressionValuePart": {
          "Type": "bool",
          "Value": "True"
        }
      }
    ]
  },
  "OrderDefinitions": [
    {
      "DataDefinition": {
        "Name": "StopForRangeConfiguration",
        "Property": "StopForRangeConfiguration",
        "ValueParamaters": [
          {
            "Name": "multiplier",
            "Type": "double",
            "Value": "4"
          },
          {
            "Name": "isTrailingStop",
            "Type": "bool",
            "Value": "false"
          }
        ],
        "DataParamaters": null
      },
      "ParamaterDataDefinitions": [
        {
          "Name": "Atr",
          "Property": "Atr",
          "ValueParamaters": [
            {
              "Name": "period",
              "Type": "int",
              "Value": "14"
            }
          ],
          "DataParamaters": null
        }
      ]
    },
    {
      "DataDefinition": {
        "Name": "LimitForRangeConfiguration",
        "Property": "LimitForRangeConfiguration",
        "ValueParamaters": [
          {
            "Name": "multiplier",
            "Type": "double",
            "Value": "4"
          }
        ],
        "DataParamaters": null
      },
      "ParamaterDataDefinitions": [
        {
          "Name": "Atr",
          "Property": "Atr",
          "ValueParamaters": [
            {
              "Name": "period",
              "Type": "int",
              "Value": "14"
            }
          ],
          "DataParamaters": null
        }
      ]
    }
  ],
  "RiskDataDefinitions": null,
  "StrategyId": 0,
  "StrategyName": null
}
 
"generate a data structure which represents a strategy" not sure I've got the idea. Did you generate structures for MQL codes? Or you work with scripts from other platforms?
 
InvBox said:
"generate a data structure which represents a strategy" not sure I've got the idea. Did you generate structures for MQL codes? Or you work with scripts from other platforms?
More...
Not MQL. I generate the data structure from my custom platform.
 
Hi InvBox,

I understand your pain in regards to porting your code to a new platform. But as you've probably already noticed while doing this manually: it is not trivial. This implies that writing a tool to do so is going to be even more complicated.

While NinjaTrader and MultiCharts both run on C#, there are many subtle differences between the platforms. Just some examples: MultiCharts requires you to manually instantiate indicators, NinjaTrader does not; the way how indicators are connected is completely different between the two. MultiCharts separates portfolio management from trading strategies, with NinjaTrader a strategy is monolythic. MultiCharts instantiates one object per traded instrument, NinjaTrader shares a single object. And on top of all this comes a library of indicators, that is probably similar but not identical. What you will find is, that getting an algorithm to 'kind of work' is manageable, but when you insist on identical results as proof that you've ported things correctly it becomes a nightmare.

I really don't want to discourage you, but pouring the effort into an open source backtesting engine/ trading platform likely offers better return in the long run. I have personally chosen that route, which is why I have started www.TuringTrader.org, which is the engine I am using for all of my ongoing research and strategy development.

Cheers, Felix
 
Felix168 said:
Hi InvBox,
More...

Hi Felix,

Appreciate your feedback.

Felix168 said:
While NinjaTrader and MultiCharts both run on C#, there are many subtle differences between the platforms.
More...

Sure I know that. I have some experience support multi platform scripts. What I've got from those days.

The core logic was extracted to dedicated files (later, I've reused them directly in Visual Studio). For use this logic I've write some adapting codes for MC and NT independently. So most of those codes was reused and work in both platforms in parallel.

Felix168 said:
I really don't want to discourage you, but pouring the effort into an open source backtesting engine/ trading platform likely offers better return in the long run.
More...

Hey, you read my mind. Sure I have some stuff for own execution engine built on several open source projects. But some things better use on in-box solutions. For example to put a code to a partner who isn't much experience with coding but knows trading and money management well.

Maybe someday I fully migrate my solution to provide it not for me only but also for my colleagues. But currently, I have what I have.

Felix168 said:
I have personally chosen that route, which is why I have started www.TuringTrader.org, which is the engine I am using for all of my ongoing research and strategy development.

Cheers, Felix
More...

I use StockSharp and QuantConnect for the same manner. Finally, I've done my own backtesting engine what is calc everything what I need (commission, network latency, order books density, brokerage ordering flow). I wanna migrate all my MQL codes to C# to work with that too.
 
InvBox said:
derstand why no one supports that. I think it will be a very useful feature. MT now looks very out of date platform and most of brokers now prefer to provide their own solution. Based on FIX/REST connections.
More...

Your project is ambitious and interesting. I would think it would be impossible for a single person to do it. In computer science we generally have to write a compiler to graduate. If you have no formal CS training (you didn't specify) you probably won't be able to get this off the ground. Please do not waste your time. Do not be offended by this. I have written two compilers (one for school, one I re-implemented a subset of a dead programming language). I don't recommend the task to anyone and even though I excelled in this area of CS I ended up abandoning the second compiler I tried to write due to the complexity of execution. I will try to outline for you how I would approach this project given what I know and what you've posted above:


MQL

As you have stated you wish to convert MQL into some intermediary language. We will call this T. T will be some intermediate representation form of the entirety of your MQL program. This is similar to the way LLVM works. This is your intermediate target language. If you are familiar with the rise of Javascript - most languages such as CoffeeScript, PureScript, etc re-target to Javascript using this idea. So breaking this stage down into bite-sized chunks here's what you have to do:

1. Develop an Intermediate Target Language (ITL)

This is necessary because your goal is to target multiple platforms (Python, C#, etc). Don't do this yourself if you can avoid it. Convert it to something with a very, very simple grammar that won't lose the computational expressiveness of the parent (MQL).

This step is incredibly difficult. It would take a while to even think of how to tackle this.

Let's assume you manage to get this far. You have developed a turing complete ITL. Now what?

2. Find a Copy of the MQL Grammar

A cursory Google search didn't turn up anything for me. It's likely proprietary. You could attempt to email MetaTrader but you may not get far. If you can't get the Grammar this project is dead in the water. Stop here - do not pass go. Heuristically turning MQL code into a target is a futile effort and it will be riddled with show stopping bugs.

So let's say you even found the MQL grammar - nice work! What next?

3. Write a compiler for the Full MQL Grammar

In order to capture any program you need to write a compiler that implements the grammar faithfully. This task will likely take years as you tweak the compiler to output your ITL. There will be hundreds of bugs. I will be intentionally vague here because you likely won't get this far. If you do, there are other choices you need to make including compiler architecture, design of your parser (LL, LALR, SLR, etc), etc etc. There are nuances to each choice (especially choosing a parser...) and you will likely need to seek a trained computer scientist here.

But wow you even did that - just incredible...

Now you have your code in your ITL. We need to solve the next hardest problem... getting it into other languages.

Converting to Python, C#, Pinescript, etc

For each target you will need to find their grammar and implement it, as above, faithfully. You are now building a second compiler. The first compiler (stage 1) converted your MQL to your ITL, and now you need to compile your ITL to your target.

To do this, you need to determine the proper conversions from your ITL to your target and implement them with as few errors as possible. PhD Computer Scientists can barely pull this off on a single re-targeting. This is borderline impossible to do as one person and even a small team of people would struggle to pull this off.

I'll leave this off here. I would be extremely surprised if you got anywhere near the second stage.


Good luck though. You're wasting your time in my opinion. You could rewrite your code into any other format yourself faster than implementing a generic solution to one of the harder problems in Computer Science.
 
"I would be extremely surprised if you got anywhere near the second stage."

Frankly speaking I've already some workable solution (MQL->NT).:D It is not ideal and not universal. But it is works for some my scripts.
 
As far as I know one of the platforms that use C# is cTrader, and they provide an auto code converter. http://2calgo.com/
 
You must log in or register to reply here.
Back
Top