Source: https://k21academy.com/azure-cloud/azure-serverless-computing-architecture-advantages-azur e-function/ The paradigm of serverless and Function-as-a-Service has enabled many programming paradigms by abstracting away the server maintenance and capacity planning overheads. The ability of the cloud infrastructure to scale up or down individual serverless API endpoints from zero to thousands of concurrently executing instances presents a unique performance bottleneck called a cold start. The cloud platform needs to provision a new execution environment, download the source code and runtime dependencies from persistent storage, launch the execution context, and execute the initialization logic on every new request arrival at an unloaded API endpoint. Such cold start delays are unacceptable for latency-critical applications, such as trading engines, responsive web backends, or user analytics pipelines. The problem of reducing the cold start penalties needs novel algorithmic approaches for container warm-up strategies and application state persistence. The cold start initialization overheads are better characterized by analyzing the execution flow of a program inside the serverless container. Given that no warm instance is available to process an incoming request, the serverless orchestration engine needs to perform a number of preparatory steps before invoking the user-defined program handler. First, it provisions a set of network namespaces and virtual Ethernet pairs to establish the execution environment for the new container. Second, it downloads the ZIP archive with the program code and third-party libraries over the internal network. Third, the execution environment is initialized by loading runtime-specific dependencies and executing global initialization handlers. Therefore, the total cold start penalty is determined by the sum of execution times of these initialization steps. On the other hand, no such overheads are incurred when a warm instance is available to process the request. The processing delay is instead dominated by the execution time of the user-defined handler. Modern applications require predictable end-to-end delays under sub-millisecond levels, making it paramount to reduce serverless initialization overheads. The most straightforward approach to reducing cold start penalties is through the concept of provisioned concurrency. It is defined as the number of function instances launched and kept in a warm state to immediately process incoming requests. The provisioning of such instances, however, can be extremely costly for applications that experience sporadic traffic patterns. At the same time, insufficient provisioned concurrency will result in high request latencies for sporadic workloads. Serverless platforms, therefore, employ various model-driven approaches to utilize the domain knowledge of the application traffic patterns to predict the occurrence of new requests. Given the characteristics of request inter-arrival times, represented as a time-series forecasting problem, serverless orchestration engines can employ Long Short-Term Memory networks to predict future system load and provision function instances in advance. In addition to reducing the number of cold starts through provisioned concurrency, the initialization overheads can be reduced by minimizing the amount of code and initialization logic executed per function call. The process of spinning up a new function execution instance can be optimized with state-of-the-art snapshot-restore techniques. At a high level, such methods allow execution of the function handler in a lazy-initialized environment. Before spinning up a new function instance, the serverless orchestration engine executes all global initialization handlers and loads the function code into memory. It then saves the memory state of the current execution environment as a compressed archive to persistent storage. Upon the next cold start request, the engine can restore the execution environment from memory to process the request faster. By avoiding the costly code loading and initialization steps, such snapshot-based approaches reduce cold start penalties by an order of magnitude. The problem of state persistence in the serverless paradigm is closely related to the challenges of maintaining application states across container lifecycle events. Due to the nature of the Function-as-a-Service abstraction, the serverless orchestration engine can terminate a function instance at any given moment to reduce compute costs. Therefore, the only reliable way to store session state, mutable data structures, or other information across function calls is to persist such data to a distributed database. The latency overheads of such an approach, however, can be unacceptable for latency-critical applications that need to minimize the number of round trips to the database. For such applications, critical path latency can be reduced with the use of in-memory caching and auxiliary storage mechanisms. By storing frequently accessed records in cache or utilizing edge-tier key-value stores, the serverless application can reduce the tail latencies of state access while minimizing the overheads of data persistence. The problem of reducing cold start penalties in the serverless paradigm requires a system-level approach and involves reducing the initialization overheads as well as optimizing the storage and retrieval of stateful session data. As enterprises begin to migrate latency-critical applications to cloud-native platforms, the discussed optimization techniques will become paramount in reducing end-to-end request processing delays.
Sources
https://www.usenix.org/system/files/hotcloud19-paper-mohan.pdf https://dev.to/vaib/conquering-cold-starts-strategies-for-high-performance-serverless-application s-59eg