Cloud platforms have become the dominant force in the technology industry. In fact, the public cloud is where the majority of new firms deploy their applications and servers. Much has been written in the last decade about the numerous benefits of cloud computing, and the drawbacks are mitigated more and more each year.
FINCAD has embraced cloud platforms and for many years used them to develop, test, and host our products. Today, I’d like to share some personal observations, as I walk you through the number of choices to consider, from private and public cloud to multi- and hybrid cloud solutions. So grab a coffee and a comfortable chair, as this is one of our ‘meatier’ blog posts.
Private Cloud
In-house infrastructure remains widely prevalent, and more so within the financial industry. Practically every large organization still manages huge pools of hardware internally, which would almost universally be referred to as the “private cloud”. However, while virtualization has become ubiquitous in such settings, what is often lacking is good support for Infrastructure-as-Code (IaC), which can be seen as one of the foundational elements of modern cloud.
Even worse, many organizations do not have suitable processes or technology in place to allocate additional resources quickly and automatically. Without an ability to scale usage elastically, and without IaC tooling to control such processes, one cannot derive maximum benefits from a cloud platform. As such, many or perhaps most "private clouds" one encounters are more akin to traditional data centers.
In addition to these issues, even the largest financial firms are starting to recognize that they cannot maintain large scale computing infrastructure at the same level of quality and cost effectiveness as cloud platform giants like Microsoft, Amazon or Google. Therefore all organizations are starting to make at least some inroads into the public cloud space. Cloud providers are offering huge discounts and other incentives to accelerate that trend.
Public Cloud
From our observations, hedge funds and new asset managers are the most eager adopters of public cloud platforms, and these days it is very typical for such firms to deploy all of their new systems this way. One reason for this is cost optimization, as usually these firms do not have the budgets or the personnel to invest upfront into IT. Perhaps even more importantly, however, is that these organizations face huge pressure from their customers to create value quickly. Therefore, they need to be very agile and develop and deploy new solutions with minimal delays. The best way to achieve that is via public cloud.
In contrast, large financial organizations have customers who value stability and continuity; and the regulatory scrutiny is much greater. As a result, we are seeing these firms move much more cautiously. It is likely that at the vast majority of these institutions the core systems all run on premises. But cloud is increasingly popular for non-core systems and ad-hoc solutions. In addition, industry experts widely expect that even core technology will be increasingly deployed on public cloud in the coming years. The COVID crisis has added additional momentum to this trend.
Regulation and Security
Regulatory concerns do seem to be a factor deterring adoption of public cloud solutions. While no financial regulation explicitly discourages the use of cloud providers, the indirect effect is obvious due to the serious responsibility that regulations place on financial firms to ensure security of their data and customer information in particular. This effectively prevents regulation-bound firms from using the cloud in the same agile manner that technology startups do, as any use of third-party cloud technology has to go through necessary reviews. The need for additional security and governance oversight significantly erodes the cost and time-to-market advantages of the cloud.
Regulations tend to put the heaviest emphasis on storage and transmission of data. Thus, many firms choose the middle ground, where they keep as much of their data on premises as possible. They then leverage cloud for its sheer computing power, and use minimal amounts of cloud data, which are typically not sufficient enough to identify customers.
Cloud providers are keenly aware of these concerns, and have been working on various ways to alleviate them. One example is Amazon launching private network link capabilities.
Cloud Provider Choice
It is well known that AWS dominates cloud market share overall across the tech industry. However, what we are observing within the financial industry is that AWS does not enjoy nearly the same lead. And, Microsoft's Azure cloud has at least similar if not larger market share, and has been making rapid gains in recent years. Other providers are trailing far behind, although Google Cloud Platform deserves a mention.
However, this broad view masks very stark differences between varied jurisdictions all over the world, where the market share, reputation and adoption for cloud providers is nothing like the global picture.
As an example, in some countries in Asia, financial firms are almost exclusively using MS Azure cloud with almost no market penetration from other cloud providers. In these jurisdictions, even convincing a firm to use a vendor managed solution running on a different cloud platform is an uphill battle.
Multi-Cloud Cloud
The term, multi-cloud, refers to the simultaneous usage of multiple cloud providers. There are a number of benefits that drive cloud users towards it, such as an ability to pick and choose best services based either on price or functionality. In addition, any organization that can seamlessly transfer application load and infrastructure from one provider to another can gain significant leverage in price negotiations.
Software industry reports suggest that multi-cloud IT strategies are widely used, however we have not seen them being adopted very broadly when working with financial institutions. There is a cost associated with multi-cloud strategies, as the tooling, services and products are not consistent or standardized between different providers. Therefore we consistently see smaller firms choose one provider and invest in building infrastructure with them. Larger organizations are considering multi-cloud strategies, as ROI is much more lucrative for them. But as I mentioned above, they are often only starting to make tentative steps in the cloud and that may explain why we haven’t seen many of them put comprehensive multi-cloud initiatives into production.
Hybrid Cloud
Hybrid cloud solutions are a mix of cloud and on-premises solutions. This means that only some of the firm's systems are running in the public cloud. Such configurations are quite widespread, mostly because of difficulty transforming and moving a large system fully to the cloud (the concept of ‘lift-and-shift’ only sounds simple!). Some systems are seen as safer and easier to move, and they are migrated first. Alternatively, old systems may be allowed to live out their age in an existing stable configuration, while new solutions are built in the cloud.
One typical hybrid cloud pattern we see is when a firm chooses to carve out a portion of a system that benefits most from the move to the cloud, leaving the rest of the system running on premises. A classic example would be a solution that needs to perform end of day processing of the firm's portfolio and therefore requires a large amount of compute resources to be provisioned for a few hours each day. By allocating this compute capacity on the public cloud, the firm would only pay for the resources it needs, releasing them as soon as calculations are done. This can result in huge cost savings compared to keeping all the necessary hardware on premises. Even more beneficial is an ability to cheaply reduce the calculation time. For instance, reducing the calculation time in half may mean paying for twice the CPUs, but only for half of the duration, yielding roughly the same cost.
In this example only the part of the system that can easily consume additional CPU resources is put on the cloud, while the other parts of the system, such as business workflow services, databases, integration and UI layers, could all remain on premises. This could be appealing as a means of avoiding what is often seen as a costly and risky technology transformation project on a system that may already be viewed as working well. These other parts of the system typically have very inelastic resource requirements and therefore no dramatic savings could be expected from shifting them to the cloud.
Another reason you often see hybrid cloud solutions is a desire to put existing (often significant) on premises resources to use. Any hardware that already exists is seen as essentially a free resource with corresponding pressure to utilize it in conjunction with the cloud.
FINCAD’s Use of Cloud Platforms
Stay tuned. In a later post we will discuss how we use cloud platforms at FINCAD, and some of the lessons we’ve learned in the process.
