Midgard, the international collaborative research initiative, has completed its run of five years. The technology pioneered in this collaboration represents a paradigm shift, questioning the fundamental virtual memory system that has endured since the 1960s. At the same time it has demonstrated a unique form of research funding, coming to light at a time when Swiss universities had lost the right to lead most European research projects.
All modern computers run a trusted piece of software called the operating system, that enables the safe and secure sharing of hardware. Computer memory, which accounts for half of the cost of a server's hardware today, is where a program code and its data are kept. Modern operating systems are based on principles of "virtual memory" that were formed in the 1960s to bulid operating systems for safe and secure use of memory.
What happened? First, CPUs grew exponentially faster and memory became exponentially bigger. Second, there are now tens of thousands of services (programs) running on the CPU that need to be directed to their own space.
Today, the process of ensuring and managing protected access to memory forms a bottleneck.
Researchers at EPFL saw the possibility to innovate a new virtual memory design that, with CPU support, can accelerate managing and securing access to memory. This new design would be available to application developers on mobiles or servers, allowing full compatability with today's software running on Linux, Android, iOS/MacOS and HarmonyOS.
A paradigm shift like this required a high level of collaboration between many institutes. When Midgard started, Switzerland had lost the right (recently regained) to lead projects in almost all European Union research funding organisations. For its time, Midgard was therefore an outlier in being able to federate a global team of experts, for a simple reason: the funding came from industry.
Transformative research needs transformative financing
"For a project in tranformative architecture we needed a big funding instrument, in terms of the amount of funding and in terms of the wide range of institutions we wanted to get involved," explains Prof. Babak Falsafi, founder of both EcoCloud and Midgard. "EcoCloud had secured funding from Microsoft and Facebook in the past, but this time it was Intel who came forward. So we could make a collaboration between Yale, Edinburgh University, Korea University, HES-SO and EPFL, with the backing of the world's biggest CPU manufacturer.
"That was the whole purpose of creating EcoCloud: to be able to work with companies, while at the same time creating funds that would allow us to work with other Academic institutions."
In a recent paper, presented at the 52nd Annual International Symposium on Computer Architecture in Tokyo, it was shown how Midgard technology could be applied to FaaS (Function as a Service), a popular software development paradigm offered by cloud service providers such as Amazon, Microsoft and Google.
FaaS was originally conceived so developers from various organizations (and potentially untrusted parties) could write programs (in the form of just a software function) that worked together seamlessly with the operating system managing secure access to memory. "Unfortunately, today's FaaS systems suffer from millisecond-level performance bottlenecks that arise from isolating functions using conventional virtual memory," explains Prof. Falsafi.
"We strove for efficient in-process memory isolation — a new hardware mechanism that exceeds the capability of traditional virtual memory systems and forgoes syscalls, page table manipulation, and TLB shootdown: processes that take tens or even thousands of microseconds to complete.
"We demonstrated that our implementation could meet microsecond-level Service Level Objectives for microservice workloads while performing within 16% of an idealized but insecure baseline, delivering over two times higher throughput compared to enhanced state-of-the-art academic operating system proposals."
The future lies in efficient rack technology
Marking the end of Midgard's five year run, as the largest computer systems project in the history of EPFL, are a raft of publications, an in-house demonstrator showing order of magnitude gains, a showcase presentation at Intel and - of course - talk of a sequel.
"We are looking at rack technology," explains Prof. Falsafi. "Apple make purpose-built processors for their smartphones and laptops. Most data centers, however, have been using desktop hardware and operating system technology. It's now time to create cloud-native hardware and software technologies. The next wave of innovation is in "rack-scale" integration. Today's cloud datacenters bring together several dozens of nearly identical servers into a 'rack'. Because the services have diverse use for hardware, much of this hardware ends up being fragmented and not used across the dozens of servers (not only wasting silicon but also idle power).
Rack-scale integration with the help of Midgard can help improve utilization of the hardware. As far as software is concerned, Midgard is compatible with modern operating systems including Linux, Apple iOS/MacOS and HarmonyOS.
"There's always left-over memory that data centers don't use," concludes Falsafi. "Microsoft reported last year, in a paper, that 50% of GPUs are underutilised in AI. We're trying to create technologies to help operators use their hardware more efficiently. That is the goal that drives us on."
So, will there be Midgard 2? No name as yet. "Midgard" comes from the ancient Norse, meaning Middle Earth. Whether the next one is Asgard, or even Valhalla, we can be sure that the paradigm will shift again.
