MQL5 supports object oriented paradigm of programming from very beginning, and MQL4 has been improved to meet MQL5, so basically there is no difference between the languages (only core APIs of MetaTrader 4 and MetaTrader 5 are different).

This series of articles try to answer some of these questions, explain applicational value of OOP, and provide examples of using it in MetaTrader.

Definition

This works in many aspects, such as:

1. brevity - lower number of code lines needed to implement a required product via reuse of existing classes (without any changes in the classes);
2. reliability - error minimization as a consequence of decomposed code with managable access rules for different contexts (classes, variables);
3. readability - short and clear presentation of algorithm in a notation, which is close to natural language and hierarchy of real-life entities;
4. maintainability - when it becomes necessary, one need to change as little code lines as possible;
5. standartization - best practices of solving common programming tasks are collected and available as compendiums of design patterns;

Main question here is - how can one know whether a specific code is complex enough to make it a good candidate for simplification by using OOP? Or maybe it will only become more complex?

Well, indeed, OOP can seem an overkill for a single buffer indicator, but if you have a bunch of similar indicators it is already an occasion for extracting common structures and behavioral patterns into classes. Anytime you find yourself making "copy&paste" of code you're probably missing the point to get the benefits of OOP world. You may say you could use plain old functions and includes for eliminating "copy&paste", and this works in many cases, but then it usually gets more and more agglomerated over time. And this is when OOP comes in handy. Just a simple example: when you include a library of functions, you have only 2 options - either use it "as is" (if it meets your needs entirely), or customize for specific purpose (with these specific changes affecting every other code relying on the library, which can be error prone). With OOP, you can include a class with base functionality, derive new descendant class from it, and customize it specifically for your code.

Just as a short reminder, let us formulate main principles of OOP. This is the last time we speak about theory, and will from now on concentrate on practice.

Main Principles 

In OOP you define a class with an interface, beyond which all implementation details are hidden from the rest of code. This makes it easy to improve or completely change implementation when necessary without affecting other parts of code.

All tasks to be performed by a code are distributed among logically decoupled code parts - classes with clear and unique responsibility (purpose) and minimal dependencies.

An object pointer or reference can hold instance of a derived class. This allows you to call methods declared in the base class and actually invoke an overridden implemenation, thus customize base behavior partially and utilize all the other useful features of the parent.

If you try to use a natural way of printing:

Of course one can invent some shorthands, such as a popular DS(number, digits) macro for DoubleToString, but they do not actually solve the problem, but make it less obtrusive.  

First start the class and its private member variables.

class OutputStream
{
  private:
    int digits;      // default number of digits for floating point numbers
    char delimiter;  // default separator between values
    int timemode;    // default format for datetime
    string format;   // format string for numbers with floating point
    string line;     // internal buffer, where all printed data is accumulated

    void createFormat()
    {
      format = "%." + (string)digits + "f";
    }

  public:
    OutputStream(): digits(_Digits), timemode(TIME_DATE|TIME_MINUTES)
    {
      createFormat();
    }
    OutputStream(int p): digits(p), timemode(TIME_DATE|TIME_MINUTES)
    {
      createFormat();
    }
    OutputStream(int p, char d): digits(p), delimiter(d), timemode(TIME_DATE|TIME_MINUTES)
    {
      createFormat();
    }
    OutputStream(int p, char d, int t): digits(p), delimiter(d), timemode(t)
    {
      createFormat();
    }

Default value for digits (if it's not specified) is the _Digits of current _Symbol. Default datetime format is without seconds. Default separator is empty.

    template<typename T>
    OutputStream *operator<<(const T v)
    {
      if(typename(v) == "int" || typename(v) == "uint"
      || typename(v) == "long" || typename(v) == "ulong"
      || typename(v) == "short" || typename(v) == "ushort")
      {
        if(typename(v) == "ushort" && v == '\n')
        {
          Print(trimmedLine());
          line = NULL;
          return GetPointer(this);
        }
        else
        {
          line += IntegerToString((int)v);
        }
      }
      else
      if(typename(v) == "double" || typename(v) == "float")
      {
        if(v == EMPTY_VALUE) line += "<EMPTY_VALUE>";
        else line += StringFormat(format, v);
      }
      else
      if(typename(v) == "datetime")
      {
        line += TimeToString((datetime)v, timemode);
      }
      else
      if(typename(v) == "color")
      {
        line += (string)v;
      }
      else
      if(typename(v) == "char" || typename(v) == "uchar")
      {
        if(v == '\n')
        {
          Print(trimmedLine());
          line = NULL;
          return GetPointer(this);
        }
        else
        {
          line += CharToString((char)v);
        }
      }
      else
      {
        line += (string)v;
      }
      if(delimiter != 0)
      {
        line += CharToString(delimiter);
      }
      return GetPointer(this);
    }

In this case it's implemented as a single templatized method which can accept value of any built-in type. Instead of this you could define multiple methods with different parameter types - one per every built-in type, but I decided the single universal one. In this method we process passed variable according to its type determined by typename function. For every type, specific API function - such as DoubleToString, TimeToString, CharToString - is used to get its string representation, which is added to the line member variable. The process uses predefined settings specified only once in constructor. When the input contains single newline character '\n', all the contents of the line varibale is flushed by Print call and the line is emptied. Printing uses a private method trimmedLine, which cuts latest separator. Separators are added automatically after every streamed value, because we assume a next value to come, but the latest separator makes no sense. Thus this method is required. Alternatively one could insert separator before every new value except very first one, but then you should check if the line is empty before every concatenation.

Now we can create an object of this class.

And then use it in the following way

which outputs

The EN helper is a shorthand for enumeration printing.

One important thing to note is that operator << defined above can process only built-in types. For objects we should add another templates.

    template<typename T>
    OutputStream *operator<<(const T &v)
    {
      line += typename(v) + StringFormat("%X", GetPointer(v));
      if(delimiter != 0)
      {
        line += CharToString(delimiter);
      }
      return GetPointer(this);
    }

    template<typename T>
    OutputStream *operator<<(const T *v)
    {
      line += typename(v) + StringFormat("%X", v);
      if(delimiter != 0)
      {
        line += CharToString(delimiter);
      }
      return GetPointer(this);
    }

The type T here is resolved to any object - they are passed by reference or as pointers. This implementation prints out not so much information about the objects. If you want objects to provide additional information, you can define a class Printable like that

class Printable
{
  public:
    virtual string toString() const
    {
      return "<?>";
    };
};

Please, use recent versions of MQL compiler to eliminate possible errors which may arise on older versions. MQL is constantly extending its support of more and more OOP features. 

Table of contents

One simple use case where OOP can be helpful is related to resource control. A file is a resource of a special kind. It should be allocated (created), used, and finally released (closed).

This problem can be easily solved by OOP. Let us define a class which wraps file handle and takes care about (potentially, all) manipulations with it. To open a file one will create an automatic object of this class, for example inside a function. And when the flow control will leave the context (the function in this case, or other block of code where the file variable is declared), the object destructor will be called automatically and close file handle. It's important to note that the variable should be declared as automatic, not via a dynamically allocated pointer using operator new (because such a stuff must be deleted explicitly by programmer).  

Here is the simple self-explanatory class.

class File
{
  private:
    int file; // file handle
  
  public:
    File(const string name, const int flags, const short delimiter)
    {
      file = FileOpen(name, flags, delimiter);
    }

    File(const string name, const int flags)
    {
      file = FileOpen(name, flags);
    }
    
    bool isOpened() const
    {
      return (file != INVALID_HANDLE);
    }
    
    bool operator!() const
    {
      return !isOpened();
    }
    
    int handle() const
    {
      return file;
    }
    
    virtual ~File()
    {
      if(file != INVALID_HANDLE)
      {
        Print("File is closed automatically"); // this is just a debug log
        FileClose(file);
      }
    }
};

int processFile(const string inputFileName = "abc.csv")
{
  // declare automatic variable of class File
  File f(inputFileName, FILE_READ | FILE_TXT | FILE_SHARE_READ, ',');
  if(!f)
  {
    Alert("Can't read file " + inputFileName);
    return 1;
  }
  
  int handle = f.handle();

  // these are just placeholders for some actual conditions  
  bool requiredCondition1, requiredCondition2;
  
  Print("Processing started...");
  
  if(!requiredCondition1)
  {
    // FileClose(handle) call is not required
    return 2;
  }
  
  string line = FileReadString(handle);

  Print("More in-depth processing...");

  if(!requiredCondition2)
  {
    // FileClose(handle) call is not required
    return 3;
  }
  
  // ...
  
  Print("Processing done");
  
  // FileClose(handle) call is not required

  return 0;
}

double buffer1[];
double buffer2[];
double buffer3[];

Why not to write simply:

???

for(int i = 0; i < Bars; ++i)
{
  for(int j = 0; j < ArrayRange(buffers, 0); ++j)
  {
    buffers[j][i] = iClose(Symbols[j], 0 , i) * getIndex(j);
  }
}

All these years and up to the current moment, you need to replace the inner loop with N assignments for different buffers.

The indicator will have 4 buffers, 3 of which are simple moving averages with given periods and the 4-th is a weighted sum of them.

class Indicator
{
  private:
    double buffer[];
    int cursor;
    
  public:
    Indicator(int i)
    {
      SetIndexBuffer(i, buffer);
    }

It holds an array of doubles for later binding as indicator buffer in the class contructor. The cursor is an integer pointer for this buffer. We want the class to behave exactly as an array, this is why we need to overload operator[].

    Indicator *operator[](int b)
    {
      cursor = (int)b;
      return GetPointer(this);
    }

    double operator=(double x)
    {
      buffer[cursor] = x;
      return x;
    }

This is where the bar index (cursor) saved in the previous method is used to update an element in the buffer. Among assigment operator there mus be overloaded many other arithmetic operators in order to mimic ordinary array. 

    double operator+(double x) const
    {
      return buffer[cursor] + x;
    }
    
    double operator-(double x) const
    {
      return buffer[cursor] - x;
    }
    
    double operator*(double x) const
    {
      return buffer[cursor] * x;
    }
    
    double operator/(double x) const
    {
      return buffer[cursor] / x;
    }

    double operator+=(double x)
    {
      buffer[cursor] += x;
      return buffer[cursor];
    }
    
    double operator-=(double x)
    {
      buffer[cursor] -= x;
      return buffer[cursor];
    }
    
    double operator*=(double x)
    {
      buffer[cursor] *= x;
      return buffer[cursor];
    }
    
    double operator/=(double x)
    {
      buffer[cursor] /= x;
      return buffer[cursor];
    }
};

Now let us define another class which manages an array of the objects of the class Indicator. Remember? - We're after an array of buffers.

class IndicatorArray
{
  private:
    Indicator *array[];
    
  public:
    IndicatorArray(int n)
    {
      ArrayResize(array, n);
      for(int i = 0; i < n; ++i)
      {
        array[i] = new Indicator(i);
      }
    }
    
    virtual ~IndicatorArray()
    {
      int n = ArraySize(array);
      for(int i = 0; i < n; ++i)
      {
        if(CheckPointer(array[i]) == POINTER_DYNAMIC)
        {
          delete array[i];
        }
      }
      ArrayResize(array, 0);
    }
    
    Indicator *operator[](int n) const
    {
      return array[n];
    }
    
    int size() const
    {
      return ArraySize(array);
    }
};

Having these 2 classes we can start coding the indicator itself (I use old-fashined event handlers for simplicity).

#property indicator_chart_window
#property indicator_buffers 4

#property indicator_color1 Red
#property indicator_color2 Green
#property indicator_color3 Blue
#property indicator_color4 Yellow
#property indicator_width4 3

#property strict

extern int Period1 = 5;
extern int Period2 = 11;
extern int Period3 = 21;

IndicatorArray buffers(4); // 1 single line magic!

int periods[3];
double periodsSum;

int init()
{
  periods[0] = Period1;
  periods[1] = Period2;
  periods[2] = Period3;
  
  periodsSum = 1.0/Period1 + 1.0/Period2 + 1.0/Period3;
  
  SetIndexLabel(0, "MA " + IntegerToString(Period1));
  SetIndexLabel(1, "MA " + IntegerToString(Period2));
  SetIndexLabel(2, "MA " + IntegerToString(Period3));
  SetIndexLabel(3, "MA SUM");
  
  return 0;
}

int start()
{
  int limit = MathMax(Bars - IndicatorCounted(), 1);
  
  for(int i = limit - 1; i >= 0; --i)
  {
    buffers[3][i] = 0;
    for(int j = 0; j < 3; ++j)
    {
      buffers[j][i] = iMA(_Symbol, _Period, periods[j], 0, MODE_EMA, PRICE_TYPICAL, i);
      buffers[3][i] += buffers[j][i] / periods[j];
    }
    buffers[3][i] /= periodsSum;
  }

  return 0;
}

will output strange integer number, which is obviously not a value of the 0-th element. If you try

the things get even worse - you'll get a code generation error (older versions of compiler generate bad ex4, while newer can't compile the code). 

    double operator[](int b)
    {
      return buffer[b];
    }

but we can't do that, because such method is already defined with different return type (remember Indicator *operator[](int b)?). We can use the following trick here:

    double operator[](uint b)
    {
      return buffer[b];
    }

class IndicatorGetter
{
  private:
    Indicator *owner;
    int cursor;
    
  public:
    IndicatorGetter(Indicator &o)
    {
      owner = GetPointer(o);
    }
    
    double operator[](int b)
    {
      return owner[(uint)b];
    }
};

And the class Indicator creates an instance of IndicatorGetter internally in its contructor and makes it available outside via special method edit - here is the updated Indicator definition:

class Indicator
{
  private:
    double buffer[];
    int cursor;
    IndicatorGetter *instance;
    
  public:
    Indicator(int i)
    {
      SetIndexBuffer(i, buffer);
      instance = new IndicatorGetter(this);
    }

    IndicatorGetter *edit() const
    {
      return instance;
    }
    
    virtual ~Indicator()
    {
      delete instance;
    }
    ...

Finally, for convenience the helper class is accompanied by the class IndicatorArrayGetter supporting an array metaphor for the helper buffers, in the same way as IndicatorArray does for Indicator objects. Please, consult with the source code for further details. Also note that not all operators are overloaded in the example: if you use other ones, you need to extend the classes.

Table of contents

The example will demonstrate the whole idea based on EventSetTimer function which supports intervals starting from 1 second. You can modify it for more fine-grained timers using EventSetMillisecondTimer. 

Surely the class should have a method to signal the timer event and a member variable storing a time of previous notification.

class TimerNotification
{
  private:
    datetime lastNotification;
    
  public:
    TimerNotification(): lastNotification(0)
    {
    }
  
    virtual void notify()
    {
      lastNotification = TimeLocal();
    }

    virtual int getSeconds() const = 0;

Timer period is the main and the only property of a timer, but we do not define a variable for it in the abstract class - insteed we allow derived classes to provide this information via the virtual method, and manage the period on their own, including probably a floating period with changes in time.

    bool isTimeCome()
    {
      return lastNotification + getSeconds() < TimeLocal();
    }
};

class TimerNotification
{
  private:
    datetime lastNotification;
    bool checkedNotifyCall;
    
  public:
    TimerNotification(): lastNotification(0), checkedNotifyCall(false)
    {
    }
  
    virtual void notify()
    {
      lastNotification = TimeLocal();
    }
    
    virtual int getSeconds() const = 0;
    
    bool isTimeCome()
    {
      if(lastNotification == 0)
      {
        if(!checkedNotifyCall)
        {
          checkedNotifyCall = true;
          return true;
        }
        else
        {
          Print("ERROR: call to TimerNotification::notify() is missing in concrete timer class");
        }
      }
        
      return lastNotification + getSeconds() < TimeLocal();
    }
};

If descendant "forgets" to invoke the base notify from its own notify, the error will be shown on logs.

class Timer: public TimerNotification
{
  private:
    int seconds;
    
  public:
    Timer(const int s)
    {
      seconds = s;
    }
    
    virtual int getSeconds() const
    {
      return seconds;
    }
    
    virtual void notify()
    {
      Print(__FUNCSIG__, seconds);
      TimerNotification::notify();
    }
};

The constructor accepts a number of seconds as the timer period and then returns this number from getSeconds. Do not forget to call notify method of the base class.

class MainTimer
{
  private:
    TimerNotification *subscribers[];

This array will store pointers to all timers which registered themselves in the manager. 

  public:
    MainTimer()
    {
      EventSetTimer(1);

We use 1 second period here as the least possible period for timers with second granularity. If you plan to work with milliseconds consider the value carefully, because too small value may increase CPU overheads.  

    }
    
    ~MainTimer()
    {
      EventKillTimer();
    }

Finally - the method to add a timer object in the internal array. This is a subscription.

    void bind(TimerNotification &tn)
    {
      int i, n = ArraySize(subscribers);
      for(i = 0; i < n; ++i)
      {
        if(subscribers[i] == &tn) return; // already added
        if(subscribers[i] == NULL) break; // empty slot
      }
      
      if(i == n) // no empty slots, extend the array
      {
        ArrayResize(subscribers, n + 1);
      }
      else // use empty slot (if any)
      {
        n = i;
      }
      subscribers[n] = &tn;
    }

The next method does the contrary - unsubscribes (disables) a timer.

    void unbind(TimerNotification &tn)
    {
      int n = ArraySize(subscribers);
      for(int i = 0; i < n; ++i)
      {
        if(subscribers[i] == &tn)
        {
          subscribers[i] = NULL; // mark the slot as empty
          return;
        }
      }
    }

And this is the method which should be called from OnTimer event handler in global context. It publishes information for existing subscribers. The timer manager class is the publisher.

    void onTimer()
    {
      int n = ArraySize(subscribers);
      for(int i = 0; i < n; ++i)
      {
        if(CheckPointer(subscribers[i]) != POINTER_INVALID)
        {
          if(subscribers[i].isTimeCome())
          {
            subscribers[i].notify();
          }
        }
      }
    }
};

Now we need to go back to our Timer class and insert some lines in order to make subscription/unsubscription. The final variant of code is the following:

class Timer: public TimerNotification
{
  private:
    int seconds;
    MainTimer *owner;
    
  public:
    Timer(MainTimer &main, const int s)
    {
      seconds = s;
      owner = &main;
      owner.bind(this); // subscribe itself
    }
    
    ~Timer()
    {
      owner.unbind(this); // unsubscribe itself
    }
    
    virtual int getSeconds() const
    {
      return seconds;
    }
    
    virtual void notify()
    {
      Print(__FUNCSIG__, seconds);
      TimerNotification::notify();
    }
};

To test the classes write a simple example with 2 timers.

const int seconds = 2;

MainTimer mt();
Timer t1(mt, seconds);
Timer t2(mt, seconds * 2);

void OnTimer()
{
  mt.onTimer();
}

The output looks like this:

The timer with 2 seconds period is fired 2 times faster than the timer with 4 seconds period. Notifications do not come exactly with 1 second accuracy, but this is normal for standard EventSetTimer. You can make sure that every next notification comes not ealier than the specified number of seconds for every timer.

Bingo.

Table of contents

The process implies that there should be a header file (.mqh) with all required stuff, which could be sufficient to include into a mq4-file, and after changing its extension to .mq5 compilation should produce a ready-made indicator for MetaTrader 5.

Of course, the mqh-file should contain some non-OOP defines and functions. For example, here is a part of them:

#define StrToDouble StringToDouble
#define DoubleToStr DoubleToString
#define TimeToStr TimeToString
#define Ask SymbolInfoDouble(_Symbol, SYMBOL_ASK)
#define Bid SymbolInfoDouble(_Symbol, SYMBOL_BID)

#define Digits _Digits
#define Point _Point

#define extern input

...

int ObjectsTotal()
{
  return ObjectsTotal(0);
}

bool ObjectCreate(const string name, const ENUM_OBJECT type, const int subwindow, const datetime time1, const double price1)
{
  return ObjectCreate(0, name, type, subwindow, time1, price1);
};

...

bool ObjectSetText(const string name, const string text, const int fontsize = 0)
{
  bool b = ObjectSetString(0, name, OBJPROP_TEXT, text);
  if(fontsize != 0) ObjectSetInteger(0, name, OBJPROP_FONTSIZE, fontsize);
  return b;
}

...

int iBarShift(string symbol, ENUM_TIMEFRAMES timeframe, datetime time, bool Exact = true)
{
  datetime lastBar;
  SeriesInfoInteger(symbol, timeframe, SERIES_LASTBAR_DATE, lastBar);
  return(Bars(symbol, timeframe, time, lastBar) - 1);
}

...

long iVolume(string symbol, ENUM_TIMEFRAMES tf, int b)
{
  long result[1];
  return CopyTickVolume(symbol, tf, b, 1, result) > 0 ? result[0] : 0;
}

...

bool _SetIndexBuffer(const int index, double &buffer[], const ENUM_INDEXBUFFER_TYPE type = INDICATOR_DATA)
{
  bool b = ::SetIndexBuffer(index, buffer, type);
  ArraySetAsSeries(buffer, true);
  ArrayInitialize(buffer, EMPTY_VALUE); // default filling
  return b;
}

#define SetIndexBuffer _SetIndexBuffer

void SetIndexStyle(const int index, const int type, const int style = EMPTY, const int width = EMPTY, const color clr = clrNONE)
{
  PlotIndexSetInteger(index, PLOT_DRAW_TYPE, type);
  if(style != EMPTY) PlotIndexSetInteger(index, PLOT_LINE_STYLE, style);
  if(width != EMPTY) PlotIndexSetInteger(index, PLOT_LINE_WIDTH, width);
  if(clr != clrNONE) PlotIndexSetInteger(index, PLOT_LINE_COLOR, clr);
}

All of these is obvious. You'll find complete code in the attachment.

First, remember MT4 accessors for timeseries - Open[], High[], Low[], Close[], etc. We can mimic this by objects with overloaded operator[]. For example, for volume data we can use the class:

class VolumeBroker
{
  public:
    long operator[](int b)
    {
      return iVolume(_Symbol, _Period, b);
    }
};

VolumeBroker Volume;

Actually, as all these classes are very similar to each other, one can generate them by preprocessor using one define:

#define DefineBroker(NAME,TYPE) \
class NAME##Broker \
{ \
  public: \
    TYPE operator[](int b) \
    { \
      return i##NAME(_Symbol, _Period, b); \
    } \
}; \
NAME##Broker NAME;

DefineBroker(Time, datetime);
DefineBroker(Open, double);
DefineBroker(High, double);
DefineBroker(Low, double);
DefineBroker(Close, double);
DefineBroker(Volume, long);

Second, remember that object property setter function ObjectSet has changed in MT5 to a set of functions ObjectSetInteger, ObjectSetDouble, etc, accepting different number of parameters. For example, mql4-code:

ObjectSet(name, OBJPROP_TIME1, Time[0]);
ObjectSet(name, OBJPROP_PRICE1, price);

should be somehow translated into mql5:

ObjectSetInteger(0, name, OBJPROP_TIME, 0, Time[0]);
ObjectSetDouble(0, name, OBJPROP_PRICE, 0, price);

It's important that there are no constants such as OBJPROP_TIME1 and OBJPROP_PRICE1 in mql5, and instead of OBJPROP_TIME1, OBJPROP_TIME2, OBJPROP_TIME3 one need to use the single constant OBJPROP_TIME and additional index. Price-related constants changed in the same way. How to emulate this? Well, by objects. Let's define 2 classes for properties of integer and double types (other types you can add yourself).

class OBJPROP_INTEGER_BROKER
{
  public:
    ENUM_OBJECT_PROPERTY_INTEGER p;
    int i;
    
    OBJPROP_INTEGER_BROKER(const ENUM_OBJECT_PROPERTY_INTEGER property, const int modifier)
    {
      p = property;
      i = modifier;
    }
};

class OBJPROP_DOUBLE_BROKER
{
  public:
    ENUM_OBJECT_PROPERTY_DOUBLE p;
    int i;
    
    OBJPROP_DOUBLE_BROKER(const ENUM_OBJECT_PROPERTY_DOUBLE property, const int modifier)
    {
      p = property;
      i = modifier;
    }
};

OBJPROP_INTEGER_BROKER OBJPROP_TIME1(OBJPROP_TIME, 0);
OBJPROP_DOUBLE_BROKER OBJPROP_PRICE1(OBJPROP_PRICE, 0);
OBJPROP_INTEGER_BROKER OBJPROP_TIME2(OBJPROP_TIME, 1);
OBJPROP_DOUBLE_BROKER OBJPROP_PRICE2(OBJPROP_PRICE, 1);
OBJPROP_INTEGER_BROKER OBJPROP_TIME3(OBJPROP_TIME, 2);
OBJPROP_DOUBLE_BROKER OBJPROP_PRICE3(OBJPROP_PRICE, 2);

Look how constants OBJPROP_TIMEn/OBJPROP_PRICEn "became" the objects storing corresponding index inside. Now we can implement new ObjectSet functions, and pass broker objects as parameters:

bool ObjectSet(const string name, const OBJPROP_INTEGER_BROKER &property, const long value)
{
  return ObjectSetInteger(0, name, property.p, property.i, value);
}

bool ObjectSet(const string name, const OBJPROP_DOUBLE_BROKER &property, const double value)
{
  return ObjectSetDouble(0, name, property.p, property.i, value);
}

Voila!

ObjectSet(name, OBJPROP_TIME1, Time[0]);
ObjectSet(name, OBJPROP_PRICE1, price);

works again in MetaTrader 5. 

Let us consider a process of convertion of several indicators.

Let's take standard ZigZag.mq4 from MetaQuotes (specifically, dated 2014). You should add indicator_plots directive and change indicator_buffers number:

#property indicator_plots 1   // added
#property indicator_buffers 3 // was 1

because this is how MetaTrader 5 adds auxiliary buffers, and IndicatorBuffers is not supported anymore (it should be deleted in the source code). Also lines:

SetIndexBuffer(1, ExtHighBuffer);
SetIndexBuffer(2, ExtLowBuffer);

should be edited as follows:

SetIndexBuffer(1, ExtHighBuffer, INDICATOR_CALCULATIONS);
SetIndexBuffer(2, ExtLowBuffer, INDICATOR_CALCULATIONS);

As the indicator uses new style of event handlers (OnInit, OnCalculate) add MT4_NEW_EVENT_HANDLERS define before include: 

#define MT4_NEW_EVENT_HANDLERS
#include <ind4to5.mqh>

That's it - the indicator is ready to compile (3 lines added, 1 removed, 2 changed).

#property indicator_plots 1

and

#include <ind4to5.mqh> 

MetaTrader 5 is not back compatible with MetaTrader 4 in many aspects. This is bad, but we can do nothing with this. Yet we can do something to simplify translation of MetaTrader 4 products into MetaTrader 5.

One of the problems that arises when we move from MetaTrader 4 to MetaTrader 5 is the way how latter uses different method of indicator invocation. In MetaTrader 4, a call to an indicator returns a value right away, whereas in MetaTrader 5 the similar call returns an indicator handle, which should be passed later to CopyBuffer function in order to get the value. We faced this problem in the previous blog post about converting indicators. Actually, the problem is general for any MQL program using indicators, including expert advisers. So if we solve it, we'll kill 2 birds in one shot.

We have chosen MaBased-ZigZag.mq4 (from the codebase) for example translation, and this indicator uses iMA internally (2 different instances, in fact). Let's think of a way how to emulate the old-fashined iMA behaviour in MetaTrader 5.

class Indicators
{
  public:
    static double jMA(const string symbol, const ENUM_TIMEFRAMES period, const int ma_period, const int ma_shift, const ENUM_MA_METHOD ma_method, const ENUM_APPLIED_PRICE applied_price, const int bar)
    {
      // ???
      return EMPTY_VALUE;
    }

Here we define a static function jMA which replicates iMA prototype from MetaTrader 4. When this function is called first time, we need to create an indicator and save the handle. When this function is called second and all successive times, we use the handle to call CopyBuffer and return values (if they are ready). A draft of the code could look like this.

  private:
    static int handle = -1; // this is not a MQL ;-)

    static double jMA(const string symbol, const ENUM_TIMEFRAMES period, const int ma_period, const int ma_shift, const ENUM_MA_METHOD ma_method, const ENUM_APPLIED_PRICE applied_price, const int bar)
    {
      if(handle == -1)
      {
        handle = iMA(symbol, period, ma_period, ma_shift, ma_method, applied_price);
      }
      else
      {
        double result[1];
        if(CopyBuffer(handle, 0, bar, 1, result) == 1)
        {
          return result[0];
        }
      }
      return EMPTY_VALUE;
    }

But wait - there may exist many different instances of the indicator, every one with its own set of parameters. So instead of the single handle variable we need a kind of array which saves not only the handles but the sets of corresponding parameters. In other words, we need an associative array with elements which can be addressed by a composite index containing all parameters. For that purpose a hashmap can be used. The name comes from the fact that indices are usually calculated as a hash of the data, but this is not a requirement.

template<typename K, typename V>
class HashMapSimple
{
  private:
    K keys[];
    V values[];
    int count;
    K emptyKey;
    V emptyValue;

  public:
    HashMapSimple(): count(0), emptyKey(NULL), emptyValue((V)EMPTY_VALUE){};
    HashMapSimple(const K nokey, const V empty): count(0), emptyKey(nokey), emptyValue(empty){};

Keys and values can be of a different type, this is why there are 2 arrays for underlying data (just to clarify - there is a single type for keys, and single type for values, but these 2 types can differ). For both arrays we need to provide constants denoting empty values.

    void add(const K key, const V value)
    {
      ArrayResize(keys, count + 1);
      ArrayResize(values, count + 1);
      keys[count] = key;
      values[count] = value;
      count++;
    }

Get index by key:

    int getIndex(K key) const
    {
      for(int i = 0; i < count; i++)
      {
        if(keys[i] == key) return(i);
      }
      return -1;
    }

Get key by index:

    K getKey(int index) const
    {
      if(index < 0 || index >= count)
      {
        Print(__FUNCSIG__, ": index out of bounds = ", index);
        return NULL;
      }
      return(keys[index]);
    }

Get value by index (as in conventional arrays):

    V operator[](int index) const
    {
      if(index < 0 || index >= count)
      {
        Print(__FUNCSIG__, ": index out of bounds = ", index);
        return(emptyValue);
      }
      return(values[index]);
    }

Get value by key (associative array feature):

    V operator[](K key) const
    {
      for(int i = 0; i < count; i++)
      {
        if(keys[i] == key) return(values[i]);
      }
      Print(__FUNCSIG__, ": no key=", key);
      return(emptyValue);
    }

Also we provide methods to update and remove (key;value) pairs.

    void set(const K key, const V value)
    {
      int index = getIndex(key);
      if(index != -1)
      {
        values[index] = value;
      }
      else
      {
        add(key, value);
      }
    }

Removal does actually mark an element as removed and does not resize the arrays. This allows you to remove many elements in a row, before the arrays will be shrinked by purge method shown below.

    void remove(const K key)
    {
      int index = getIndex(key);
      if(index != -1)
      {
        keys[index] = emptyKey;
        values[index] = emptyValue;
      }
    }

Purge wipes out elements marked as removed and squeezes the arrays:

    void purge()
    {
      int write = 0;
      int i = 0;
      while(i < count)
      {
        while(keys[i] == emptyKey)
        {
          i++;
        }
        
        while(keys[i] != emptyKey)
        {
          if(write < i)
          {
            values[write] = values[i];
            keys[write] = keys[i];
          }
          i++;
          write++;
        }
      }
      count = write;
      ArrayResize(keys, count);
      ArrayResize(values, count);
    }

All this is not optimized in any way but sufficient for our purpose. Types should be standard MQL types, not classes. Also type of keys can not be int because int is already used for accessing elements by indices. If necessary though, one may use long or uint for keys as a replacement.

HashMapSimple<datetime,double> map;

The resulting header file HashMapSimple.mqh is attached below.

#include <HashMapSimple.mqh>

class Indicators
{
  private:
    static HashMapSimple<string,int> mapOfInstances;

In our simple implementation we'll use string keys. As the map is static it's used for all instances of MA indicators and even can be used for different types of indicators (such as WPR, ATR, etc). For that purpose we should include indicator name into the key along with parameters. The values are int handles.

HashMapSimple<string,int> Indicators::mapOfInstances("", NO_VALUE);

The NO_VALUE is defined as -1:

#define NO_VALUE -1

This is invalid handle.

    static double jMA(const string symbol, const ENUM_TIMEFRAMES period, const int ma_period, const int ma_shift, const ENUM_MA_METHOD ma_method, const ENUM_APPLIED_PRICE applied_price, const int bar)
    {
      string key = StringFormat("iMA-%s-%d-%d-%d-%d-%d", symbol, period, ma_period, ma_shift, ma_method, applied_price);
      int handle = mapOfInstances[key];
      if(handle == NO_VALUE)
      {
        handle = iMA(symbol, period, ma_period, ma_shift, ma_method, applied_price);
        if(handle != INVALID_HANDLE)
        {
          mapOfInstances.add(key, handle);
        }
      }
      else
      {
        double result[1];
        if(CopyBuffer(handle, 0, bar, 1, result) == 1)
        {
          return result[0];
        }
      }
      return EMPTY_VALUE;
    }

Please notice how the key is constructed from "iMA" name, symbol, period and other input parameters. Other indicators can be implemented in the similar way, but as an additional demonstration of the idea let us support iCustom function.

    template<typename T1, typename T2>
    static double jCustom(const string symbol, const ENUM_TIMEFRAMES period, const string name, T1 t1, T2 t2, const int buffer, const int bar)
    {
      string key = StringFormat("iCustom-%s-%d-%s-%s-%s", symbol, period, name, (string)t1, (string)t2);
      int index = mapOfInstances[key];
      if(index == NO_VALUE)
      {
        int handle = iCustom(symbol, period, name, t1, t2);
        if(handle != INVALID_HANDLE)
        {
          mapOfInstances.add(key, handle);
        }
      }
      else
      {
        double result[1];
        if(CopyBuffer(index, buffer, bar, 1, result) == 1)
        {
          return result[0];
        }
      }
      return EMPTY_VALUE;
    }

The key comprises "iCustom", symbol, period, custom indicator name, and 2 templatized parameters. For, say, 5 parameters we need similar but more lengthier implemenation:

    template<typename T1, typename T2, typename T3, typename T4, typename T5>
    static double jCustom(const string symbol, const ENUM_TIMEFRAMES period, const string name, T1 t1, T2 t2, T3 t3, T4 t4, T5 t5, const int buffer, const int bar)
    {
      string key = StringFormat("iCustom-%s-%d-%s-%s-%s-%s-%s-%s", symbol, period, name, (string)t1, (string)t2, (string)t3, (string)t4, (string)t5);
      int index = mapOfInstances[key];
      if(index == NO_VALUE)
      {
        int handle = iCustom(symbol, period, name, t1, t2, t3, t4, t5);
        if(handle != INVALID_HANDLE)
        {
          mapOfInstances.add(key, handle);
        }
      }
      else
      {
        double result[1];
        if(CopyBuffer(index, buffer, bar, 1, result) == 1)
        {
          return result[0];
        }
      }
      return EMPTY_VALUE;
    }

To tie up this methods with calls of standard iMA or iCustom use simple defines:

#define iMA Indicators::jMA
#define iCustom Indicators::jCustom

Finally, we can return to the custom indicator MaBased-ZigZag.mq4 and finish its adaptation to MetaTrader 5. Actually, it's sufficient to include a header file with new classes discussed above. Here is the side-by-side comparison of 2 files.

As you may see some changes are related to the transformation, while the others not.

For the convertion we added

#property indicator_plots 5

and then 

#define MT4_OLD_EVENT_HANDLERS
#include <ind4to5.mqh>
#include <MT4Indicators.mqh>

MT4Indicators.mqh is attached. ind4to5.mqh was published in the previous post.

One important thing to note is that our new iMA implementation returns EMPTY_VALUE until the plot is calculated. This is a side effect of the fact that MetaTrader 5 builds indicators asynchronously. As a result, we need to skip all ticks while our iMA value is unavailable and return -1 from start (in old event handlers 0 means success, and other values denote errors). Consequently the event wrapper OnCalculate in ind4to5.mqh will return 0, and recalculation will continue until iMA is ready. To achieve this behaviour, we added the line

if(ma[i] == EMPTY_VALUE || ma_fast[i] == EMPTY_VALUE) return -1;

in the main loop of the indicator.

• Colors are changed for better visibility (for example, black line is invisible on black background used for charts by default).
• Empty values of the buffers should be left EMPTY_VALUE because otherwise (when set to 0) the picture is screwed up.
• Input parameters with MA types and prices should use special enumerations ENUM_MA_METHOD and ENUM_APPLIED_PRICE.
• Limit of bar to process was incorrect.
• Buffers must be filled with empty values explicitly in MetaTrader 5.

 P.S. There is still one more bug in the source code of the original indicator, but it's not marked in the comparison screenshots. You may try to figure it out. This should be easy because MetaTrader 5 will show the error dynamically, if it'll occur. I hope you'll be able to fix it yourself - homework, so to speak ;-).

Introduction

The errors can be divided into 2 distinct categories: permanent (critical) and temporary (manageable). Example of the 1-st kind could be ERR_NOT_ENOUGH_MONEY and of the 2-nd - ERR_OFF_QUOTES. The difference between them is that temporary errors can vanish themselves after one or more retries and refreshes of quotes, while critical errors can be solved only by deliberate change in trading function parameters or trading environment (for example, trader could deposit more money on his account or change its leverage).  

Most of MQL programmers try to handle temporary errors by running a loop with a trading function call and refreshing rates until it succeeds or a predefined number of iterations achieved. This appoach works but has many drawbacks. If you retry the call too often, this may overload you local trade context (which is only one for the terminal) or your server (which can give you another error - too many requests). And if you call Sleep function in between, your expert will miss any other events (such as chart events) or receive them with a lag.

The logic of using commands is asynchronous. We'll create a command, pass it to a manager (which we'll discuss below) for execution, and then allow our code to do some other tasks (if any) or keep idling until the command will be finally completed or failed, and then the manager will notify our code about result. 

The command

    static int cmdIncrementedID;

The variable cmdIncrementedID must be initialized - here we start ID numbering from 1.

int OrderCommand::cmdIncrementedID = 1;

Every command can have custom deviation and magic number, though we'll provide another common place for these settings later.

    string symbol;
    int cmd;
    double volume;
    double price;
    double stoploss;
    double takeprofit;
    string comment;
    datetime expiration;
    color arrow_color;

As far as the command is asynchronous let's provide a field for resulting value and public method to access it:

    int getResult() const
    {
      return result;
    }

The command should handle errors, specifically "decide" if an error can be fixed by retry or not. For that purpose we add an array with temporary error codes.

    static int HandledErrorCodes[];

and the function for checking this array:

    int handledError(string opName, int errCode, bool &priceRequote)
    {
      int handled = -1;
      priceRequote = false;
      for(int i = 0; i < ArraySize(HandledErrorCodes); i++)
      {
        if(HandledErrorCodes[i] == errCode)
        {
          handled = i;
          priceRequote = (i >= 7);
          break;
        }
      }
      Print(opName + " error: ", errCode, " handled:", (handled != -1));
      return handled;
    }

The array should be filled after the class definition:

int OrderCommand::HandledErrorCodes[] = {0, ERR_SERVER_BUSY, ERR_NO_CONNECTION, ERR_TRADE_TIMEOUT, ERR_OFF_QUOTES, ERR_BROKER_BUSY, ERR_TRADE_CONTEXT_BUSY,
                           ERR_INVALID_PRICE, ERR_PRICE_CHANGED, ERR_REQUOTE};

The errors starting from 7-th position (2-nd line) implies that price refreshing may help.

Now we can code the constructor:

  public:
    OrderCommand(string symbol = "", int cmd = -1, double volume = 0, double price = 0, int deviation = 0, double stoploss = 0, double takeprofit = 0,
             string comment = "", int magicId = 0, datetime expiration = 0, color arrow_color = CLR_NONE)
    {
      this.symbol = symbol == "" || symbol == NULL ? _Symbol : symbol;
      int d = (int)MarketInfo(this.symbol, MODE_DIGITS);
      
      this.cmd = cmd;
      this.volume = volume;
      this.price = NormalizeDouble(price, d);
      this.stoploss = NormalizeDouble(stoploss, d);
      this.takeprofit = NormalizeDouble(takeprofit, d);
      this.expiration = expiration;
      this.comment = comment;
      
      this.deviation = deviation;
      this.magicId = magicId;
      
      cmdID = cmdIncrementedID++;
      created = TimeCurrent();
    }

All fields are assigned and ID is incremented for every new object. The fields are similar to properties of MT4 orders, so please consult with documentation if necessary.

A method for reading every field value should be implemented, for example:

    int getCmdID() const
    {
      return cmdID;
    }
 
    static int getNextCmdID()
    {
      return cmdIncrementedID;
    }
    
    ...
    
    double getVolume() const
    {
      return volume;
    }

Now let us consider error handling functions provided by MT4 and try to make use of them. Most impostant thing is that GetLastError function not only returns last error code but also clears it internally, so that successive calls will return 0. I don't think this is a good design. For example, if you use a 3-d party library which calls GetLastError inside, your code will never detect which error has actually happened. So let us save error code in the object as integer _LastErrorOverride and implement the wrapper function:

    int getLastError()
    {
      if(_LastErrorOverride != 0) return _LastErrorOverride;
      int e = GetLastError();
      if(e != 0)
      {
        _LastErrorOverride = e;
      }
      return _LastErrorOverride;
    }

The function returns last error code from the variable if it's assigned, or requests the code from the built-in GetLastError and then saves result in the variable. Also we need to wrap another standard function:

    void resetLastError()
    {
      ResetLastError();
      _LastErrorOverride = 0;
    }

After all the preparations we've made by abovementioned code, we are ready to deal with main task of any command - its execution. Our base class should provide a method which establishes some default process in a step-by-step manner. Derived classes will be able to customize every step. This design pattern is called "template method".

For the simplicity we'll use the former approach with a loop - this is harmless as far as we can control it by parameters and even make as trivial as single pass with zero sleeping time. Inside the loop there will be the following steps:

1. reset error code;
2. execute command specific action;
3. analyze error code;
4. if error is temporary - retry (go to step 1), otherwise - stop the process with the error.

    int process(const int maxRetries, const int timeWaitMsecs)
    {
      int i;
      for(i = 0; i < maxRetries; i++)
      {
        resetLastError();
        
        int code = execute();
        if(code > 0) return code; // success
        
        Print("DATA:", symbol, " ", cmd, " p=", D2S(price), " sl=", D2S(stoploss), " tp=", D2S(takeprofit),
        " / ask=", D2S(MarketInfo(symbol, MODE_ASK)), " bid=", D2S(MarketInfo(symbol, MODE_BID)), " V=", D2S(volume),
        " s=", (int)MarketInfo(symbol, MODE_SPREAD), " ", (int)MarketInfo(symbol, MODE_STOPLEVEL));
        
        int err = getLastError();
        bool priceRequote = false;
        
        if(handledError(classname(), err, priceRequote) == -1) break;

        Sleep(timeWaitMsecs); // NB: doesn't work in the tester
        RefreshRates();
        
        if(cmd == OP_BUY) price = MarketInfo(symbol, MODE_ASK);
        else if(cmd == OP_SELL) price = MarketInfo(symbol, MODE_BID);
        
        // we can't deduce updated price for pending orders and order deletion
        if((cmd > OP_SELL && priceRequote) || cmd < 0) break;
        
        Print("New price=", D2S(price));
      }
      
      if(i == maxRetries)
      {
        Print("Max retries count ", maxRetries, " reached, give up");
        return 0; // neutral, retry later
      }
   
      return -1;
    }

The loop runs maxRetries times. On every interation we start from resetting error code and then call yet unknown method execute which returns an integer number. This method should be virtual and overridden in descendants. This is where every specific command will perform its actual task, such as opening or closing an order. The resulting value should be positive on success, negative on error and 0 in a neutral situation. If it's positive we exit the method immediately returning the code (the meaning of the code may depend from specific command). Otherwise we output command details in log and read last error code by means of getLastError. Next we use handledError (described several paragraphs above) to discover if the error can be handled by waiting a bit or not. If not, we break the loop and exit with result -1. If the error seems temporary, we Sleep for timeWaitMsecs period, refresh rates and update command's price if it's for new market order. Then the cycle repeats if attempts count does not exceed maxRetries. If it is, we return 0, signalling outside that the command is not completed yet, but there are chances to make it next time.

In the call of handledError there is another yet unknown virtual function classname. Both virtual functions - execute and classname - can be made pure abstract in the base command class or provide minimal default implementation, for example:

    virtual int execute() // = 0;
    {
      _LastErrorOverride = ERR_NO_RESULT;
      return -1;
    }
    
    virtual string classname() const // = 0;
    {
      return typename(this);
    }

Execute does actually nothing and returns error (because we don't want to run the dummy process again and again).

Classname returns a name of current class. You'll see that we will override the function in all descendants with very same implemenation - so in every case typename will return actual name of a derived class.

class OrderCreateCommand: public OrderCommand
{
  public:
    OrderCreateCommand(int cmd, double volume, string symbol, double price = 0, int deviation = 0, double stoploss = 0, double takeprofit = 0,
      string comment = "", int magicId = 0, datetime expiration = 0, color arrow_color = CLR_NONE):
      OrderCommand(symbol, cmd, volume, price, deviation, stoploss, takeprofit, comment, magicId, expiration, arrow_color)
    {
    }

    virtual int execute()
    {
      if(price == 0) // use current price for market/instant orders
      {
        if(cmd == OP_BUY) price = MarketInfo(symbol, MODE_ASK);
        else if(cmd == OP_SELL) price = MarketInfo(symbol, MODE_BID);
      }
      
      result = OrderSend(symbol, cmd, volume, price, deviation, stoploss, takeprofit, comment, magicId, expiration, arrow_color);
      return result;
    }
    
    virtual string classname() const
    {
      return typename(this);
    }
};

This is a simplified version of the code. At the bottom you'll find a file with complete code attached.

And here is how a command for order modification can look like.

class OrderModifyCommand: public OrderCommand
{
  private:
    int ticket;
    
  public:
    OrderModifyCommand(int ticket, double price = 0, double stoploss = 0, double takeprofit = 0,
      datetime expiration = 0, color arrow_color = CLR_NONE):
      OrderCommand("", -1, 0, price, 0, stoploss, takeprofit, NULL, 0, expiration, arrow_color)
    {
      this.ticket = ticket;
      if(OrderSelect(ticket, SELECT_BY_TICKET))
      {
        if(this.price == 0) this.price = OrderOpenPrice();
        this.symbol = OrderSymbol();
        this.cmd = OrderType();
        this.volume = OrderLots();
        this.comment = OrderComment();
        this.magicId = OrderMagicNumber();
      }
    }
    
    virtual int execute()
    {
      result = OrderModify(ticket, price, stoploss, takeprofit, expiration, arrow_color);
      return result;
    }
    
    virtual string classname() const
    {
      return typename(this);
    }
};

You'll find all the other commands in the attached file as well.

The manager

class OrderManager
{
  private:
    int slippage;
    int magic;
  
    int maxRetries;
    int timeWaitMsecs;

The manager should store all pending commands somewhere.

    OrderCommand *commands[];

And the manager should be most likely instantiated only once. In fact, there could be cases when you want to have many managers in a single expert adviser, but for the simplicity we decide to use only one instance in this framework. This is why we use the "singleton" design pattern.

    static OrderManager *manager; // singleton object instance

    OrderManager() // constructor is hidden from outer space
    {
      maxRetries = 10;
      timeWaitMsecs = 5000;
      slippage = 10;
      
      manager = GetPointer(this);
    }

  public:

    static OrderManager *getInstance()
    {
      if(manager == NULL) manager = new OrderManager();
      return manager;
    }

The manager can only be created by the public factory method getInstance, which either creates a new manager object (if it does not yet exist) or returns already existing one.

As you remember the manager stores deviation and magic number, which are default values for all commands. The manager can "publish" them via getters:

    int getDeviation() const
    {
      return slippage;
    }
    
    int getMagic() const
    {
      return magic;
    }

As a result, we can use the default values in OrderCommand constructor and thus allow a caller to omit these specific properties in constructor call.

    // updated lines in the OrderCommand constructor:
    // use default values if deviation or magic are not specified  
    this.deviation = deviation > 0 ? deviation : OrderManager::getInstance().getDeviation();
    this.magicId = magicId > 0 ? magicId : OrderManager::getInstance().getMagic();

But lets go back to the manager.

To initiate execution of a command we pass its object (pointer) to the manager.

    void place(OrderCommand *cmd)
    {
      int i, n = ArraySize(commands);
      for(i = 0; i < n; i++)
      {
        if(commands[i] == NULL) break;
      }
      
      if(i == n)
      {
        ArrayResize(commands, n + 1);
      }
      else
      {
        n = i;
      }
      commands[n] = cmd;
    }

The method finds an empty element (if any) in the array of commands or extends the array and reserves new empty slot for the command. The commands stored in the array should be processed in a dedicated method. 

    void processAll()
    {
      int n = ArraySize(commands);
      for(int i = 0; i < n; i++)
      {
        if(CheckPointer(commands[i]) != POINTER_INVALID)
        {
          int r = commands[i].process(maxRetries, timeWaitMsecs);
          if(r != 0) // succeeded or failed, otherwise 0 - still pending
          {
            commands[i] = NULL;
          }
        }
      }
    }

For every not yet completed command it calls its method process, and if it fails unrecoverably or succeeds the command is removed from the queue.

The method processAll should be called periodically, for example from a timer event. For that purpose we'll define:

    void onTimer()
    {
      processAll();
    }

which should be invoked from standard OnTimer function of EA. Other important events should be also utilized.

    void onInit()
    {
      // TODO: restore command queue from Global Variables
    }
    
    void onDeinit()
    {
      // TODO: serialize pending commands to Global Variables
      // TODO: clean up the queue
      if(CheckPointer(&this) == POINTER_DYNAMIC) delete &this;
    }

Lets wire up all things together using new order creation as example.

To open a new order we have to create an object of the class OrderCreateCommand and pass it to the manager via its place method. Next time the timer event fires, the manager executes all pending commands including our OrderCreateCommand. This will try to do OrderSend, error analysis, and then either leave the command in the queue for more trials or exclude the command as completed or failed (depending from the result).  

Refinement

The first issue is not an issue if we save the pointer to our command in the EA an delete it ourselves. But this is not convenient in many cases. For example, it's much simplier to have an option to write:

manangerObject.place(new OrderCreateCommand(OP_BUY, 0.1, Symbol()));

and "forget" about the object allowing the manager to release the memory.

As far as use-cases may vary, automatic deletion of processed commands should be enabled or disabled by a flag. For this purpose let's add a protected variable selfDestruction to OrderCommand class and corresponding accessor method:

    bool isSelfDestructing()
    {
      return selfDestruction && (CheckPointer(&this) == POINTER_DYNAMIC);
    }

Here we added the check if the pointer is dynamic, because only dynamically allocated objects can be deleted explicitly (we can't guarantee that some programmer will not set selfDestruction to true mistakenly for an automatically allocated object).

The variable can be initialized through a constructor parameter (in OrderCommand or in derived classes), so that caller can specify if he/she wants the object to devastate itself. Actually we can provide helper constructors with shorter syntax, for example: 

    OrderCreateCommand(const int cmd, const double volume):
      OrderCommand(_Symbol, cmd, volume)
    {
      selfDestruction = CheckPointer(&this) == POINTER_DYNAMIC; // convenient but tricky implicit initialization
    }

This way we have no need to pass additional parameter to the constructor. You may look through the attached code for more possibilities, including the "builder" pattern (which allows you to setup object properties on demand step by step, by chainable simple methods).

Once the variable selfDestruction is assigned, we can use it for object "finalization". In the manager's processAll method change the lines to:

    if(r != 0) // succeeded or failed, can be deleted
    {
      if(commands[i].isSelfDestructing())
      {
        delete commands[i];
      }
      commands[i] = NULL;
    }

In the attached file this part is more eleborated, but it's simplified here for readability. In any way, we can now delegate destruction of the command objects to the framework.

The second issue was related to returning result of a command execution back to EA. When it comes to asyncronous commands the best way for passing information is a notification interface.

class OrderCallback
{
  public:
    virtual void onSuccess(const int result, const OrderCommand *cmd)    // = 0;
    {
      Print("Completed ", cmd.toString(), ", result: ", result);
    }
    virtual void onFailure(const int error, const OrderCommand *cmd)     // = 0;
    {
      Print("Failed ", cmd.toString(), ", error: ", error);
    }
    virtual void onTimeout(const int timeout, const OrderCommand *cmd)   // = 0;
    {
      Print("Timeout ", cmd.toString(), ", timeout: ", timeout);
    }
};

Again, it could be a pure interface, that is an abstract class with pure virtual methods and no data, but we provide a minimal base implementation. The methods allows us to send 3 notifications: on successfull task completion, on insolvable problem, and on timeout (the usage of the later one is shown in the code only).

The variable of the type OrderCallback should be added to the manager class.

  private:
    OrderCallback *pCallback;

Then EA can implement an object of this class and assign it via special method in the manager:

    void bindCallback(OrderCallback *c)
    {
      pCallback = c;
    }

Now we can elaborate processAll method by adding the notifications: 

    void processAll()
    {
      int n = ArraySize(commands);
      for(int i = 0; i < n; i++)
      {
        if(CheckPointer(commands[i]) != POINTER_INVALID)
        {
          int r = commands[i].process(maxRetries, timeWaitMsecs);
          if(r != 0) // succeeded or failed, otherwise 0 - still pending
          {
            if(CheckPointer(pCallback) != POINTER_INVALID)
            {
              if(r > 0)
              {
                pCallback.onSuccess(r, commands[i]);
              }
              else // r < 0
              {
                pCallback.onFailure(commands[i].getLastError(), commands[i]);
              }
            }

            if(commands[i].isSelfDestructing())
            {
              delete commands[i];
            }
            commands[i] = NULL;
          }
        }
      }
    }

As you see, successfully completed command will send back the result of its execution. For OrderCreateCommand this is a ticket, and for other commands the content will vary. In case of a fatal error its code is sent with the notification.

Finally, for convenience we can add shorthand methods which will mimic standard functions such OrderSend, OrderModify etc. Here is a helper method in the manager:

    int Send(string symbol, int cmd, double volume, double price, int deviation, double stoploss, double takeprofit,
             string comment = "", int magicId = 0, datetime expiration = 0, color arrow_color = CLR_NONE)
    {
      OrderCreateCommand create(cmd, volume, symbol, price, deviation, stoploss, takeprofit, comment, magicId, expiration, arrow_color);
      return create.process(maxRetries, timeWaitMsecs);
    }

And accompanying define:

#define OrderSend OrderManager::getInstance().Send

More features are actually implemented in the code of the manager, but they are left uncovered in the text due to the lack of space. For example, it's possible to try to execute commands synchronously (as shown above). You may inspect the source code for details. 

The command with delays

class OrderCreateCommandWithDelay: public OrderCreateCommand
{
  private:
    int delay; // seconds
    
  public:
    OrderCreateCommandWithDelay(int delay, int cmd, double volume):
      OrderCreateCommand(cmd, volume)
    {
      this.delay = delay;
      if(delay > 0) Print("delay till ", (string)(created + delay));
    }
    
    virtual int execute()
    {
      if(created + delay > TimeCurrent())
      {
        Print("delay");
        // randomly switch between handled (ERR_BROKER_BUSY) and unhandled (ERR_MARKET_CLOSED) simulated error
        this._LastErrorOverride = MathRand() > 32767 / 2 ? ERR_MARKET_CLOSED : ERR_BROKER_BUSY;
        return -1;
      }

      // after the specified delay, execute order as usual
      if(price == 0)
      {
        if(cmd == OP_BUY) price = MarketInfo(symbol, MODE_ASK);
        else if(cmd == OP_SELL) price = MarketInfo(symbol, MODE_BID);
      }
      
      result = OrderSend(symbol, cmd, volume, price, deviation, stoploss, takeprofit, comment, magicId, expiration, arrow_color);
      return result;
    }

    virtual string classname() const
    {
      return typename(this);
    }
};

Look how this code works according to testing logs.

The Expert Adviser

It works. It has been tested on EURUSD H1. You may compare execution without and with delays. You may also try to embed the framework in your existing EA since it supports regular syntax of OrderXXX functions and synchronous execution. Yet its advantage is asynchrony, and to enable it you need only to add ASYNC_ORDERS define before include. In this case, the manager will try to execute commands instantly and return their appropriate results, but if a minor error occurs, the command will be added to the queue for execution in future. If you need to get asynchronous results (such as ticket number), override notification callbacks as appropriate.

This is a draft rather than a real life expert adviser, but it covers many practical aspects making it a good starting point for further development.

Let us look at the CObject first, and then try to design our own base class for MQL.

The methods working with files are also inapproriate here. I agree that the object should have methods for serialization and deserializtion - that is saving and restoring it to/from some persistent storage. But why this feature is bound to files and specifically to file handles? What if I like to save object in global variables or on my server via WebRequest? What if someone will pass a handle of a file opened for reading into Save method? And what if the file is in text mode but object wants to write binary data? Such base class can become a cause for many potential errors. 

Last but not the least, every object should have a method to get its string represenation, and such thing is missing here.

You may ask what are the classes OutputStream and InputStream. Here is a draft.

I hope the idea behind them is clear now. They are abstract input/output "interfaces" without dependency to a concrete storage type. The object stored in such external streams can be restored lately using InputStreams. The set of writeXXX functions can be extended with a templatized function for convenience (please note, it's applicable only for textual representation, as it uses string):

Now lets us return back to the Object class. Every object has a specification, including - but not limited to - which data does it contain. If you're going to save an object instance for longer persistence (for example, between teminal sessions) you should think about possible changes in specification. If you have a stored object and then extend its class with a new field which should be stored as well, then the old object will lack the new field, and your code should probably perform special actions for proper object initialization. This is why we have the method version - it allows you to return version number of current object specification and compare it with a version of any object being restored from an external storage.

Here is a simple example, which demonstrates the whole idea.

This class comprises 2 fields, which can be serialized and deserialized. At the beginning of every object stream we output the class name and version. When the object is being read from a stream, we check if the class name and version correspond to current values. In a real world case one should somehow upgrade the loaded objects with a lesser version up to the current version.

The complete source code is attached. Place it in MQL4/Scripts folder. The code provides different stream classes to store objects to the log (as a debug output) or text files. For example, we can save an object to a file and then restore it.

Unfortunately, the built-in profiler works only in online charts, where inefficient parts of code are not so easy to spot. It would be much more convenient to profile code in the tester (during single tests or even batch optimization of an expert adviser). For that purpose we can implement a profiler ourselves as a set of classes. It will be simple yet sufficient enough to perform main profiler tasks.

• Profiler - implements the main functionality;
• ProfilerLocator - is intended for pointing program contexts or parts (such as functions or sequences of code lines), where we want to measure performance;
• ProfilerObject - is a helper class, which allows for automating starts/stops of profiling process in accordance to its objects lifetime cycles; for example, you can declare such an object in a function (specifically, create an automatic instance on the stack), and then - no matter how many exit points the function has - every time control flow leaves the scope of the function, the object will be destroyed, which in turn will stop profiling (until the next time the function is entered again, which in turn will start profiling automatically again).

The Profiler object is already declared in the header. By default it's initialized with 100 points for measurements, that is one can evaluate duration of execution of 100 code fragments. Every point is basically backed with an element in internal arrays which store time lapse and count of times of code execution. If you need more points, adjust the initialization: the required number of measurement points is passed as the parameter of the Profiler constructor.

The simpliest, but slowest (and hence not recommended) method is:

The fragment above is identified by a unique name, specified in the StartProfiler method. There are more efficient forms of the method StartProfiler, for example the one which identifies code fragments by integer numbers. For this variant to use, one could first register all code fragments in the profiler (for example, in OnInit):

Then you can start and stop profiling by the following calls:

As a convenient way of defining profiling points one may use ProfilerLocator objects inline - just before corresponding code fragments: 

The fragment above is identified by the locator object, which generates a unique integer identifier internally. Please note that the locator object is declared static, because it should be created only once. This guarantees that every execution of the code fragment will use the same profiling element accumulating statistics. If the object would not be static, every single pass would generate (reserve) new measurement point, and everyone would have the same name "CalculateProfit". 

Automatic creation of the ProfilerObject object starts profiling of the entire function scope automatically, and its deletion upon function termination stops profiling automatically as well. A shorthand macro _PROFILEFUNC_ is defined for this in the header file.

Here is an example.

This script will produce something like this:

Overview

The reason why exceptions are so important can be easily seen in the following 2 axioms.

The new strict policy of MQL execution was not there from very beginning. Some time ago (before OOP epoch in MQL4) MQL4 could silently suppress part of errors. For example, when accessing an array element by incorrect index in a legacy MT4 indicator, it ignores the error and continues running. Then MQ introduced new property strict directive in MQL4, when it was upgraded to MQL5, and made the directive mandatory for the Market products. Since that moment both MQL4 and MQL5 programs (where the strict mode is always on) can not survive on errors.

First let us decide which types of critical errors to handle. Most frequent errors are "index out of bounds" and "divide by zero". There is a number of others of course, but part of them can not be handled internally by MQL (for example, if a requested library or indicator is unavailable) and the rest can be addressed later in the similar way to the two most frequent ones, which are covered below. All beyond this is left for homework ;-). And here is a small hint: next best candidate could be "invalid pointer access" error, which can be processed by a kind of safe pointers class.

Rubber array

Placing such stuff inside an implicit helper method can simplify our code significantly. Of course, the class should be templatized because we want a universal array capable of storing values of different types. So start with:

Data will be actually stored in the protected data array. We hide the method add in protected part as well, because no one would normally like to call it, provided that other shorthand notations can be easily implemeted. For example, in PHP we can declare and fill arrays like this:

Here we added 3 elements into the array. Let's mimic this in RubbArray class.

Now we can use the assignment operator to add new elements into the array. Similar operator can be overloaded for copying arrays.

In addition to operator= we can provide another way of adding elements. For example: 

This is almost the same as operator=, but we return the array object itself, making it possible to chain calls in a single line:

Now, when the array is filled with some data, code a method to access elements, specifically let's overload operator[].

The lines marked by comments are just placeholders for something we'll need to elaborate on later. But it's already a safe code: for incorrect indices it returns 0 silently, as former MQL did for time series (it actually returned EMPTY_VALUE, but we use 0 here as more universal for different types, and the value can be customized in future, as you'll see). 

Notice how we used operator[] internally.

Now we need somehow call it in a neat way. This can be done in 3 steps. First create a helper class:

As you may see, its sole purpose is to override operator[] and operator= to pass required index and value to the change method of RubbArray. Editor is created for specific RubbArray and stores a reference for the parent.

and constructor/destructor: 

Step 3 - adding a method and new overload of operator[] to access the editor.

For convenient usage of the new overload one may define a macro:

Now we can code:

The complete class is attached in SafeArray.mqh at the bottom of the page.

Safe numbers

The class includes several things not related to error handling just for making hook-ups, and can be extended further. For example, the eps variable is intended for comparison of 2 numbers with a given accuracy, which is usually important for doubles. We can construct or assign a value to the Number in straightforward manner:

Also Number should overload all possible operations, including division, which is actually our main concern.

Again, the commented lines of code will require error handling in future. Currently we return 0, suppressing the error.

Now all is ready to deal with exceptions. But we already typed a lot of code, so let us take a break and continue with this in the part 2.

Unfortunately exceptions are missing in MQL. Which is why we're going to implement a surrogate, and have already prepared helper classes of rubber arrays and safe numbers in the previous publication. These classes will provide a playground for implementing our own exception handlers.

Exceptions

Many OOP languages provide very similar syntax.

Our exception handler will work slightly different, because MQL does not provide much freedom in what we can do in general. 

The object contains a variable of type string: an identifier of location. Also the class contains the static array landmarks where we will save references for all instances of try objects, and cursor which points to specific try (if any), containing lines of code being executed (of course, try blocks may not cover all the code, and there can be moments of executing code outside any try block). We need to initialize the static stuff outside the class definition.

And then get back to the class, specifically to its constructor.

The contructor accepts a single parameter with an identifier. Then current instance - this - is added to the array of landmarks. We don't need destructor, which could remove the reference from the array, because all instances of try will be static. This makes sense since the objects are landmarks ;-). They will be created automatically by the platform before program initialization. But we need to know when a specific try block becomes active, so add appropriate method.

Here we search through the array of landmarks and set cursor to found index. In addition we return previous value of the cursor, making it possible to save and restore code execution context.

Who will switch the context, you ask? This should be objects of another class, and they can not be static this time, because they should be created and deleted locally and ad hoc, according to control flow, that is how program enters and leaves functions, loops and other code blocks defined by curly brackets. Here is the class. 

Every time an object context is created, it accepts a reference to nearby try block landmark and calls its activate method. As soon as it's deleted (when control flow leaves current code context), the index of previously active landamrk is restored.

Now we're approaching most tricky point of the scheme. Having a try block identified and activated, we need to intercept and handle exceptions somehow. The interception part is relatively simple. In the previous publication we created classes of rubber arrays and safe numbers, which can intercept specific errors, such as "index out of bounds" and "divide by zero". So, they can throw an exception and pass it to our try block. All we need is a method accepting the exception. Lets add it to the try class.

Having this method we need to jump for a moment back to the classes RubbArray and Number to refine their methods controlling possible errors. For example, RubbArray could have:

The method throw does nothing at the moment. Some error handler is required. This could be just a global function, for example OnError, but it would be better to try to mimic catch block syntax as much as possible, and define handlers locally for every block. In comparison with a single global function, this approach will give one more benefit of splitting error handling code to small parts, specific for every particular error and location.  

Of course, the catch object should be bound to try, this is why constructor accepts a reference for one. But what is the method bind? We don't have it yet. Lets add it to the class try.

And now it's time to elaborate the method throw.

Here we use another undefined yet class - Exception. It's that is easy to conceive.

This is just a storage of all information about current error, and also a value, which the handler can pass via Exception object back to protected code, as a result of error processing. The member variables made public for simplicity.

This example just prints information about the error and returns EMPTY_VALUE to protected block of code. If, for example, the origin of the error is a line trying to access a rubber array element by incorrect index, then it will get EMPTY_VALUE as a result, and the code continue execution smoothly. This is the difference between conventional exception handlers and our MQL exception handler. We don't have a reliable way to jump out of the block in pure MQL.  

Cryptic? Well, this may take a while to understand (though all parts have been already described above), but what it actually means - we can now write something like this:

Magic? No - OOP. 

The complete code of exception handling classes is attached below in the file Exception.mqh. Also an example script exception.mq4 is provided. Its output is:

Have fun.

Ideally, MetaTrader could provide automatic self-optimization of EAs by design. The built-in tester does work, and only thing which is missing is an API, exposing the settings of the tester for MQL code. For example, there could be a couple of functions - TesterSetSettings, TesterGetSettings, accepting a structure with date ranges (how many bars from history to use), optimization mode (open prices, control points, ticks), spread, method and goal of optimization (straightforward/genetic, balance/profit-factor/drawdown/etc.), and a reference to a file with parameters optimization set (which parameters to optimize, in which ranges, with which increments). This set-file can be prepared by user only once (just by clicking Save button on Inputs tab of the optimization dialog), and then he can start optimization by preferred schedule from MQL code by calling a dedicated method like this: TesterStartOptimization. The results of optimization could be then accessed via similar API, and EA could select a better parameter set on the fly. Unfortunately, neither MT4 nor MT5 can afford this.

The library will work in MetaTrader 4. I have many reasons to choose it over MetaTrader 5 - its API is more simple and I use it more - just to name a few. If you like MetaTrader 5, you can develop similar library - it should be easy considering the fact that this library provides a detailed roadmap.   

Interface Design

The code of ordinary EA is running in the global context, where API functions such as OrderSend, OrderClose, iOpen or iClose are defined and provided by the platform. If we move the code inside a class and define our own methods imitating OrderSend, OrderClose, iOpen, iClose and all the others, inside the class, then the original algorithm will work as usual, without noticing the deception.   

This is just a simplified scheme for explanation. In reality we'll need to use slightly different hierarchy of classes.  

Interface blueprint

Which information do we need to control an optimization process from within EA? Here is the main points:

• a simple way to enable/disable the library as a whole
• history depth to use for virtual trading (ending datetime is always now)
• how often to repeat optimization (for example, if optimization runs on 2 months history, it probably makes sense to update it every week)
• initial deposit size (remember, some EAs prefer to take risks only on a part of account balance)
• work spread (current spread can be inappropriate if optimization is performed on weekends)
• optimization goal (which value to optimize: balance, profit factor, sharpe ratio, etc)
• and finally, parameters - an array of named entities (inputs) with a range of values to probe, and increments

All the methods are pure virtual, because we're defining an abstract interface. If it wouldn't be so, then after the header is included into EA source code, compiler would give the error "function must have a body" for every non-pure virtual method. Compiler needs a complete implementation of every concrete method (function) or requires an import directive pointing to a library, where to get it from. But in MQL it's not possible to import classes. Libraries can only export plain old functions. This is why we need to introduce a so-called factory function, which can create an instance of our class and provide it for EA.

This "says" to compiler: this function will return a runtime object with specified virtual functions, but their implementation is not your bussiness at the moment.

We can specify time ranges either in bars or in days.

These constants allow us to choose one of most popular optimization criteria. I think their names are self-explaining.

The first 4 methods just returns the properties that we passed to the library via addParameter method. getValue is a current value, and getBest is an optimal value (if any). Also we should have an option to exclude (disable) specific parameter from optimization temporary. This will allow us to add all parameters to the library in a batch and then play with them without recompiling EA - just by switching inputs.

The method getParameter is also pure virtual (and its implementation is postponed for a while), but the helper overload of operator[] is defined inline using the former method.  

Finally, let's add a method which will be an entry point for the library on every tick.

It should be called inside EA's OnTick event handler.

The class contains not only the callback trade, but also stubs for most of standard API functions. It's not yet important what will be inside them. The main point to note here, is that from the method trade's point of view, all standard names will lead to the protected substitutes.

If you remember, we created OptimysticCallback in order to provide the callback method in EA, which would be called from the library. It means that we need to pass a reference to the object OptimysticCallback into the library somehow. The best solution is to add it as parameter into our factory function:

Interface refinement

The control flow is normally as follows:

1. EA creates a custom callback object derived from OptimysticCallback;
2. EA creates an instance of Optimystic and passes the callback object to it;
3. EA adjusts settings of the library; 
4. On every tick EA calls the library via Optimystic::onTick method;
5. The library does some work behind the scenes and calls MyOptimysticCallback::trade method once for current datetime and parameter set;
6. EA trades using OrderSend inside MyOptimysticCallback::trade; 
7. As far as the library is in real trading mode, OrderSend goes to the market;

5. The library does some work behind the scenes and calls MyOptimysticCallback::trade method many times for different bars in past and different parameter sets;
6. EA trades using OrderSend inside MyOptimysticCallback::trade; 
7. As far as the library is in optimization mode, OrderSend performs virtual trades;
8. The library calculates performance indicators for all trades and finds best parameter set;
9. ...

Actually, we have added 4 methods at once, because they are logically interconnected. It's likely that EA will need to make some preparations before every optimization, this is why we introduced onBeforeOptimization. And we do already know what to place inside its concrete implementation:

As you remember, we have reserved setDeposit and setSpread in Optimystic. The method onBeforeOptimization returns boolean to let the library know if EA is ready to start optimization: by returning false EA can deny pending optimization. 

As you remember, operator[] is overloaded for Optimystic to return specific parameter, and it returns an object OptimysticParameter. In turn, it provides current parameter value via getValue.

Just as a brief reminder, the method getOptimalPerformance has been added to Optimystic a few paragraphs above. 

So get back to the control flow and complement it:

What is the work behind the scenes that we mention all the time? This implies switching the trading context between real and virtual one. Most simple way to explain this is to look inside any wrapper function we have added into OptimysticCallback.

What should we write instead the ellipsis?

This is a contract for concrete implemention of our simulated trade context (I mean we will someday create a class, derived from the interface and populate all its method with complete meaning). We should somehow acquire a pointer to such implementation in OptimysticCallback and use it every time when optimization mode is on. For the simplicity, we can use the pointer itself as a flag: if it's empty - real trading is on, and if it's assigned to a virtual implementation - optimization mode is on. 

The implementation of OptimysticTradeContext is hidden inside the library, and the library should call special setContext method to control current trade context. This is the work behind the scenes.

There is only one last touch that should be made upon the header. As you remember, we have the import directive which binds our factory method with ex4 library file. We can't leave it as is, because we're going to write implemetation for our classes, and the same header file should be included into the library source Optimystic.mq4 as well. But it can't import from itself. To solve the problem we add a macro definition:

In EA source code OPTIMYSTIC_LIBRARY is undefined, so the library is imported. In our library surce code we'll define it as simple as:

and the header will be processed without an issue.

Implementation of the interfaces and example of EA with on the fly optimization will be studied in the part 2.

Table of contents 

The library source code resides in the file Optimystic.mq4, and it's currently almost empty: in the part 1 we've left it on:

The only function that is exported from the library is the factory:

It should return an object which implements Optimystic interface. Let's start coding it.

We're currently missing another class - a class with an implementation of OptimysticParameter. Let's code it as well.

To store array of parameters inside OptimysticImplementation I'll use helper class RubbArray (it was described in the blog ealier, but also attached below).

Now we can complete implementation of the methods addParameter and getParameter:

Now we can focus on the main method onTick. We can't write it at once, so start with a schematic draft. 

This will not compile, because some things here are undefined yet. For example, we need a private variable lastOptimization in the class to store a time of last optimization. We will assign it in the private method optimize, which should be also added to perform actual work. And we don't yet know what is the context. Let's fix the first 2 points:

As for the context, it should be a variable with implementation of the interface OptimysticTradeContext, which is left the only unimplemented one.

It must contain a lot of methods. You can just copy and paste the declaration of the interface into the class and code all methods scrupulously, one by one. Here, in the blog, it's not possible to cover all the changes, so please consult with the attached source code to get the final picture.

If you remember, we have a method optimize in OptimysticImplementation. This is where we place a cycle through bars on backtest period, and it will call setBar. Something like this:

Having the current bar in the context we can implement specific methods, for example, TimeCurrent:

This is the first method of the abstract interface OptimysticTradeContext which got a concrete implementation in OptimysticTradeContextImplementation. Some of the other methods can be implemented in the similar way:

But the others require additional work to be done. For example, before we can create a new order with OrderSend we surely need to define a special class.

The context should maintain an array of all market and history orders. So let's add them using RubbArray again:

And implement OrderSend:

To close an order we could code something like this:

We use new private helper here - findByTicket. It's code is more or less obvious:

The calculation of order's profit (that shown as TODO in OrderClose) can be coded inside the Order class itself:

Now we can calculate profit in OrderClose. The profit should update balance curve, which we don't have yet. Actually we need to collect many more statistics about trading, such as order counter, equity curve, etc. Let's add them to the context:

Unfortunately we use yet another new class here - BatState. The good news is that this is the last class in our implementation. And it's simple.

Get back to the context, and we can finally complete OrderClose (only profit-related part is shown):

You may see how we update balance and the counters on current bar. Of course, they should be initalized somewhere. So we need to add such a method in the context:

And it should be called from OptimysticImplementation::optimize method before we run the cycle through history bars. All input values are already known there.

In addition to initialize you see a bunch of other new methods called on the context. I hope their names are self-explaining. We need to implement them all, but let us skip details here - you may consult with the source code. The only thing which can fit is a note about income array. We need to add new element into it on every bar, this is why we extend the function setBar a bit:

This code simply copies last known balance to the new bar, and if some orders will be closed (as you saw in OrderClose), their profit will update the balance.

As soon as we have total profit and other stats after the loop through the bars, the context can calculate performance indicators. So let's add a method which return them.

This method can be used to provide implementation for Optimystic::getOptimalPerformance, which is very important for the client EA.

With the help of getPerformance optimization process (in OptimysticImplementation::optimize) can be refined further. 

All calls related to the parameter set enumeration and selection are still marked as TODO. Due to the lack of space their implementation is not covered here and can be studied in the source code.

And that's actually all. The library is ready. It works.

• incomplete multicurrency support: specifically we need to have separate spread values for every symbol;
• no swaps calculation; 
• no fast optimization by means of genetics or other algorithm (so don't specify too many combinations of parameters to check, because straightforward approach is currently used, and it's slow);
• no validation of function parameters and error generation (such as, invalid stops);
• tick values are taken for current exchange rates, whereas they should be recalculated using historic quotes;
• rough price simulation mode (by open prices);

Expert Adviser Example

After you add the include:

Declare your own class derived from OptimysticCallback:

and implement all the methods. Part of them was already described in the part 1. Most interesting is that code for the method trade should be taken completely from the former OnTick event handler. The lines that invoke indicators should be changed to take specific bar number passed via the first parameter:

All other calls made in the source to seemingly standard functions are intercepted by OptimysticCallback and redirected to appropriate backend: either the platfrom or our virtual trade context. And even Ask and Bid are replaced with correct values for the bar bar. 

Then in the method trade:

Hence, new orders can be opened only if virtual context is not NULL (that is in optimization mode) or an applicable parameter set exists (TradeEnabled is true in real trading mode). 

Please, note, that since we need to change input parameters from MQL code they should be defined as extern instead of input. Also when on the fly optimization is enabled, input parameters which undergo optimization specify minimal value and step for optimization (they are the same for simplicity), and their maximal value is 10 times larger the minimal value. As a result, if you set 3 parameters for optimization, they will make 1000 combinations.

And here is the final warning: the MACD Sample was taken as example due to its simplicity and availability, but the EA itself is not very good (for example, stoploss is applied on orders only after they go in sufficient profit, so if price moves in unfavourable direction from very beginning, you'll get large loss), as a result you should not expect much from the EA performance. Try the library with your own EAs.

I'm sure there is still much to discuss on the subject, but the blog post can't last for ever, even if it's a very very long story. You may explore the library yourself and try MQL OOP in the action.

Table of contents 

There is a number of methods of fast optimization. One of them is genetic optimization, which is used by the tester in MetaTrader. The other one is simulated annealing. And yet another one is particle swarm optimization. This method is relatively simple to implement. Let's code it in MQL.

According to the algorithm every particle should have current position, velocity, and memory about its best position in past. The "best" means that EA demonstarted highest (known to the particle) performance at those position (parameter set). Let's put this in a class.

All arrays have size of parameter space dimension, that is the number of parameters being optimized. As far as our optimization implies that better performance is denoted by higher values we initialize bestValue with most minimal possible value -DBL_MAX. As you remember, EA performance can hold any of main trade performance indicators such as profit, profit factor, sharpe ratio and etc.

In addition to single particles the algorithm utilizes so called topologies or subsets of particles. They can vary, but we will use topology which simulates social groups. The group of particles will hold knowledge about best known position among all particles in the group.

Now we can start implementing the swarm algorithm itself. Of course, we pack it in a class.

Here we have a swarm of particles and a set of groups.

In addition we want to save optimal parameter set somewhere.

Finally we need an index array that would mark every particle as a member of specific social group.

The static array should be instantiated after the class definition.

As far as we will probably have several different constructors of the swarm, let's define a unified method for its initialization.

All arrays are resized to given dimensions and filled with provided data. Group membership is initialized randomly.

The first one uses some empirical rules to deduce size of the swarm and number of the groups from the number of parameters. The second allows you to specify all values explicitly.

And this is the moment when the main method of the class should be coded - the method performing optimization itself.

Let's skip the first parameter for a while. The second one is a number of cycles which the algorithm will run. The next 3 arguments are parameters of the swarm algorithm: inertia is a coefficient to preserve particle's velocity (usually fading out, as you see from the default value 0.8), selfBoost and groupBoost define how sensitive a particle will be to requests for changing its movement in direction to best known position of the particle and its group respectively.

The method accepts an array with parameters and returns a single double value. In future we can implement a derived class in our EA, and perform virtual back-testing inside the method calculate.

First initialize result (objective function value) and coordinates in the parameter space.

Second, initialize particles. Their positions and velocities are selected randomly, and then we call the functor for every particle and save best values into groups and global solution.

Then we repeat the cycle of optimization (it looks a bit lengthy but reproduces the pseudo-code almost exactly):

That's all. To finalize the method we should return the result.

To read found solution (parameter set) from outer world we need another method.

Is it really all? It's too simple to be true. ;-) Actually we have a problem here.

This solves the problem only partially. The functor will be executed with proper parameters, but there is no guarantee that algorithm will not call it many times with the same values (due to the rounding by the step sizes). To prevent this we need to learn to identify somehow every set of values and skip repetitive calls.

It accepts CRC of a previous block of data (if any, can be 0 if only one block of data is processed; allows you to chain many blocks), the array of bytes to process and its length. The function returns 64-bit CRC.

Having this library included, we can write the following helper function to get CRC64 for an array of parameters:  

The next question is where to store the hashes and how to check them against every new parameter set generated by the swarm. The answer is binary tree. This is a data structure which provides very fast operations for adding new values and checking for existing ones. Actually the fastness relies on the tree property called balance. In other words the tree should be balanced to ensure the most possible quick operations. Fortunately, the fact that we'll store CRC hashes in the tree plays into our hands. Here is a short definition of a hash.

As a result, adding hashes into the binary tree makes it balanced, and automatically most efficient.

To implement this logic we need 2 classes: the node and the tree. It makes sense to use templates for both.

I hope the algorithm is clear enough. Please note that deletion of root in the destructor will be automatically unwound to deletion of all subsidiary nodes. 

In the method optimize we should elaborate those parts where we move particles to new positions.

We added an auxiliary array next, where we save newly generated position and check it for uniqueness. If new position was not processed yet, we add it into the tree, copy to the particle's position and continue the procedure. If new position exists in the tree (hence has been already processed), we skip this iteration.

There exist many popular test objective functions (or benchmarks), such as "rosenbrock", "griewank", "sphere", etc. For example, for the sphere we can define the functor's calculate method as follows.

Please note, that conventional formulae of the test objective functions implies minimization, whereas we coded the algorithm for maximization (because we want maximal performance from EA). This is why you need to negate result in the calculation. This is important to remember if you decide to test our implementation on other popular objective functions. 

Finally, all is ready to embed the fast optimization algorithm into Optimystic library for on the fly optimization of EAs. But let us address this question in the next publication.

Table of contents 

Next step is to embed the particle swarm into the library. Let's go back to Optimystic.mqh.

And we need a method to switch either of the methods on.

That's it! We're done with the header file. Let's move to the implementation part in Optimystic.mq4.

And add new declarations to the class.

The class should change optimization approach according to the selected method, so onTick should be updated. 

Here we added a new procedure optimizeByPSO. Let's define it using already existing optimize as a draft. 

We marked the time to compare performance of the new and old algorithms later.

We modified the function resetParameters to return number of active parameters (as you remember, we can disable specific parameters by calling setEnabled). Normally all parameters are enabled, so count will be equal to the size of parameters array.

The reason why we're bothering with this is our need to pass work ranges of parameters values to the swarm on the next lines.

And here is the problems begin. Next thing that we should write is somethig like this:

But we need a functor instead of ellipsis. The first thought you may have - let's derive the class OptimysticImplementation from the swarm's Functor interface, but this is not possible. The class is already derived from Optimystic, and MQL does not allow multiple inheritance. This is a classical dilemma for many OOP languages, because multiple inheritance is a double-edged sword.

When multiple inheritance is not supported one can apply a composition. This is a widely used technique.

This class does minimal job. It's only purpose is to accept calls from the swarm and redirect them to OptimysticImplementation, where we'll need to add new function calculatePS. This is done so, because only OptimysticImplementation manages all information required to perform EA testing. We'll consider the function calculatePS in a few minutes, but first let's continue the composition.

And now we can return back to the method optimizeByPSO which we have left on half way. Let's create an instance of SwarmFunctor and pass it to the swarm.