Digital Assets Matching Engine: Technical Considerations for Operating in the Cloud

[ajacobson]

April 16, 2021

Digital Assets Matching Engine: Technical Considerations for Operating in the Cloud

Magnus Almqvist

Exberry

Follow | Profile | More

Share

What does it take to set up a digital exchange? Magnus Almqvist, Head of Exchange Development at Exberry, lays out a roadmap of the requirements and options that a firm faces in this quest to set up an exchange in the cloud. The article is a compilation of answers to the questions he most frequently encounters from clients and prospects looking to take the digital asset leap.

A growing number of professional investors are willing to have exposure to digital assets in their portfolios, but what of the underlying technology behind the infrastructure that these increased volumes will need to rely on? Is it really possible that a fully functional, multi-asset exchange platform for multiple assets, for example, can scale on the cloud at an affordable cost? Won’t such a fully functional proposal that measures up to regulatory scrutiny, on par with the likes of that offered by incumbent technology providers, usually be accompanied with a prohibitive price point?

Incumbent capital markets are mired by legacy technology and costly operational overheads, as well as lengthy change processes coupled with interests vested in conserving the status quo. Yet with the rise of digital assets and advances in technology, more and more people are coming up with new ideas and services to help the real economy. As a result, they are naturally asking whether flexible, scalable functionality as described above actually exists today.

Indeed, a central matching engine and order book system delivered as a service in the cloud has the potential to transform stock exchange operations, reduce the cost base, and enable a new level of innovation with very short turnaround times.


Nevertheless, can such a system actually deliver a matching engine application within the capital markets space, especially due to stringent requirements around security, high-volumes, low-latency and high availability? In order to please market makers, it is essential that the platform delivers messages in a fair and transparent way, as well as being low jitter.

Technical considerations

Happily, recent innovative solutions have come to the rescue. Agile matching engines have recently surfaced that allow exchange operators to respond nimbly to new market opportunities. They provide levels of corporate governance, reliability, stability, and they are auditable – factors that are frequently absent from other cryptocurrency and digital assets platforms.

Here are some questions to consider whether a cloud-based matching engine fits your specific requirements:

1. Will the delivery model of a central matching engine in the cloud suit the needs of my platform as it grows?

Small- and large-scale customers have different needs – some are on-cloud, or on-premise only. Solutions need to be flexible in terms of method of deployment and delivery. For example, small customers are more concerned about time to market, ease of integration, and overall costs. Larger customers are often more concerned about performance, low jitter and latency, and the ability to run on proprietary data centres (“bare metal”) or hybrid solutions, or enable co-location in the cloud.

The use of microservices such as Kubernetes, if done properly, ensures scalability and elasticity beyond anything possible in traditional systems. This enables day-to-day efficiencies as system resources are expanded and reduced intra-day, and the scalability enables long-term growth from microsystems up to international scale markets.

1. What about integration with blockchains?

Because the use of blockchain is maturing, users are finding real use cases that centre on differentiating technologies. For example, the use of smart contracts to define a tradable financial asset is the driver behind today’s digital asset boom. While a traditional matching engine relies on streaming protocols, modern cloud-based matching engines provide WebSocket-based APIs and store all historical data. That means an API can be built based on data types rather than messages.

In addition, the matching engine can also be queried in order to retrieve historical data, which makes integration with blockchain much easier. For instance, the JSON API over web socket is easy to use, extremely fast, and can be integrated directly into a web-based architecture. For larger customers more concerned with latency, UDP gateways via different APIs (such as binary) can also be provided.

1. How accessible are these solutions?

For smaller enterprise-type customers, a solution can be delivered immediately with integration beginning the next day. This low-cost software is exactly the same matching engine delivered for larger customers, with the option to scale solutions when needed. On the other hand, larger customers can benefit from the fact that the technology is cloud-ready and can be easily integrated with legacy ecosystems. This includes a private cloud or even hardware and co-located with other systems. Such access to top-notch technology with a better total cost of ownership provides larger customers with a bridge to help accelerate any roadmap to the cloud and technology upgrade. This avoids the all-too-risky ‘big bang’ approach.

1. How do you build Disaster Recovery (DR) capabilities?

The key here again is flexibility – and not to be dependent on one specific cloud – but be able to run on a number of different clouds, such as Amazon and Google. Ideally, a provider should have DR capabilities in different zones with hardware deployed on different segments, in addition to different regions. There is also the ability to run on bare metal if so desired. Sensitivity to latency is also a consideration, meaning matching engines need to be highly available in a single site with full redundancy, as well as implementing a “warm Disaster Recovery site” with servers set up in advance without a database (meaning production work will only take place once a disaster occurs).

1. Does UDP really work in the cloud?

It is indeed possible to set up UDP in the cloud by setting up a network using a transport layer protocol called Reliable User Datagram Protocol (RUPD). A great example is Aeron Reliable UDP transports, which provide high-capacity resilient and deterministic throughput and minimal overhead, as well as the highest possible throughput for an application’s data, while excluding other overheads such as signalling.

1. How about jitter and low latency, surely it will be slow and high jitter?

Matching engines in the cloud need to be built from the ground up, to support low latency – this requires a reactive type of architecture that has the ability to unblock low jitter. It also has the advantage of having relatively short round trip times, which can succeed in delivering production environments that are actually twice as fast as those used today by existing Tier 1 stock exchange systems.

1. How secure is a matching engine in the cloud?

Security is paramount. The software should be comprised of nodes to manage role-based access control and be multi-tenant. Deployment is also a factor, so consideration needs to be given for restricted access on web resources and connections with digital signatures. Cloud infrastructure by such companies as Amazon are well known for their robust security infrastructure, incorporating system-wide standard protocols and security mechanisms.

Adapt for opportunity

Cloud-installed systems delivered as a service have many compelling advantages, including short delivery timescales, the removal of operational headaches, lower impact upgrade, and the ability to enable elastic and scalable solutions. In addition, the ease of access and connectivity is more important than ever in times of high demand for flexible work locations.

Further, easy access to sandboxes to quickly test the viability of processes is vital. The inherent low operational costs allow vendors to offer live continuous access to public sandbox environments for immediate zero-barrier prototyping, and even more advanced MVP builds before migrating to a proper production environment.

Ultimately, cloud-based exchange infrastructure can help deliver initiatives designed to implement processes that will drive down costs and increase operational efficiencies with the flexibility to add new markets when opportunities arise.

Photo Credit: Photo by JJ Ying on Unsplash

• • •

Magnus Almqvist is Head of Exchange Development at Exberry. In his role, Mr. Almqvist builds and nourishes contacts with exchange pioneers around the globe with over 20 years’ experience behind him. With a background as Head of Sales, Aquis Exchange, Senior Business Development Manager at FIS (SunGard) and prior to that heading up Smarts Group’s (part of NASDAQ OMX) London office, and as the CTO of the London Stock Exchange Group Company EDX, he has a solid understanding of capital markets exchanges, their ecosystem and their evolving universe.

From TABB