on this page
an architecture from the pastthe serverless erawhat is serverless?the scaling problem and optimization?requests from multiple regionsthe little big thingswrapping uppractice problemsHow fast can we deliver data to the users of our application? Why is there such importance in moving data processing closer to our end users? What are the implications of moving data closer to our end users? These are some questions that come to mind as developers are building on products and applications that have the potential to impact a larger scale audience. Technology is advancing at a rapid pace, and thus the demand for faster and more efficient data processing is on the rise. To meet such a demand, we as developers must build the tools and infrastructure that allow for fast and efficient data delivery.
The "Edge" is a solution that has the potential to address many of these concerns as it allows for the processing of data closer to our end-user. It divorces the idea that processing data must occur on a single centralized cloud infrastructure and instills the idea that data processing can occur at the edge of many networks. The edge is an umbrella term that encompasses many different technologies like storage, compute, and most recently data. Thus it's not surprising for someone to refer to the edge as edge compute, edge storage, or edge data.
Edge computing has a lot of benefits and it opens up an expansive array of new questions as it also has involvement with IOT (Internet of Things) devices -- this brings hardware and software closer together. Although edge computing is an exciting area of conversation, there are, however, some potential areas where it falls short. Nonetheless, the Edge is something we should be excited about and start adopting within the application we build today. Throughout this article, we will explore the inner workings of edge computing, its advantages and disadvantages, and the potential areas in which it has the most impact.
Before we dive further into edge computing, it's important to recognize and understand the processes that led us to this point. More specifically, how can we now ask questions that allow us to probe the impact of processing data closer to our end user?
An architecture from the past
Traditionally, when we think of technical architectures, we may imagine a host of computers in one region serving information to visiting users or other machines. This type of architecture is shared amongst various businesses alongside developers and although deployment stratergies have moved from on-prem to cloud, the fundemental idea of deploying services to one geographical location still remains.
Taking a further step back to a time before "the cloud" was a thing, if we wanted to run any code that required a server, we would have to run, manage, maintain, and scale our own physical hardware. For small projects, this was no problem as we most often would not run into a scaling issue due to limitations in hardware. However, in the context of larger projects and businesses where scaling was a critical factor, owning physical hardware/servers had a lot of downsides and got expensive relatively quickly.
We can see this depicted in the following example below; toggle between light traffic and heavy traffic to see what happens.
Note
The red dot represents a service that has been deployed to a physical server in north america while green user icons are individuals that are making requests to the service. Black arrows that originate from the green user icon and connect red dot represent a succeful response being returned from the server. Black arrows that originate from the green user icon and terminate at a red X represent an unsuccesful connection to the server.
As "the cloud" became a thing, businesses could now virtualize hardware and later software, therefore, making it easier to approach the scaling problem. The introduction of virtualization provided an abstraction for hardware and software that allowed businesses to move much faster on the development of their product. As a result, there was no longer a necessity for businesses to manage and maintain their own physical hardware. We speak more on the impact of virtualization and containerization in the following article: What does it mean to containerize an application?. It is important to emphasize that although virtualization solved a lot of problems, limitations still remained.
The serverless era
Like many, the first time I heard the term serverless I thought that it did not involve any servers. However, this is far from the case; as contrary to its name, serverless doesn't actually mean the absence of servers. Rather, it's an architecture that abstracts away the maintenance, and management of a sever. In short, a serverless architecture delegates the responsibility of managing and maintaining servers to someone else, thereby allowing you to focus on the business logic of your application.
What is serverless?
Serverless architectures encompass many components and can be derived from many abstraction layers, however, they all have one thing in common -- someone else manages the infrastructure. Today, we have cloud vendors like Google, Amazon, Microsoft, etc. that take care of the provisioning and management of infrastructure/resources. The payment structure also works out nicely as development teams are only responsible for covering the cost of their usage -- it's a pay-as-you-go service for servers.
One component when we think of serverless is known as functions as a service (Faas) -- colloquially known as serverless functions. FaaS is a cloud computing service that enables you to run code in response to requests or events without the need to specify the resources/infrastructure required to run the code.
But wait... you might be now thinking that our definition of FaaS is very similar to the definition of serverless and you wouldn't be entirely wrong. These terms are used interchangeably as FaaS is a central model to the serverless architecture. However, serverless entails much more than just FaaS as it involves an entire stack of services that respond to requests or events.
Another component of the serverless architecture is known as backend as a service (BaaS). It's any third-party service that is integrated with your application; think of services like Firebase or Supabase.
Serverless includes much more than FaaS & BaaS
Cloud providers also offer services that are included in the umbrella term, serverless computing. These services include Infrastructure as a Service (IaaS), Platform as a Service (PaaS), and Software as a Service (SaaS).
Databases can also be included within the umbrella term of serverless since we can now have serverless databases that do not require the provisioning of instances which have a defined capacity, connection and query limit *. Instead, serverless databases move towards models that scale with the demand.
The scaling problem and optimization?
The introduction of the serverless architecture solved a lot of the previous problems we experienced but most importantly it solved the scaling problem. As we've mentioned, the serverless architecture provided a method to elastically scale services with the needs of our customers as in this case servers could constantly "spin up" and "spin down" to meet the demands of customers. It eliminated the worry of spikes in traffic that could bring down an entire application.
While there are a lot of really good upsides to a serverless architecture, it's not all guns and roses as there are also many downsides to serverless. One in particular is known as the "cold-start" time of a serverless function. Before we explain cold-start times, let's see an illustration to try to figure out what's going on.
Consider the following:
Here we have a button that says "Send request" which when clicked activates a serverless function. We also have a progress bar that fills up when the serverless function has been activated. Let's press the button to see what happens.
As we can see, the bar is only filled up a few seconds after the button has been pressed instead of being filled immediately. Now if we were to press the button again, you can see that the bar fills up a lot faster.
Serverless functions are analogous to our example above in that when they are first activated/or called, the environment they are run in needs some time to provision the resources to run the function. This so-called time is known as a cold start time and it's one major drawback for serverless functions.
We previously mentioned that serverless functions are very good at spinning up and spinning down to match the demands of our users. So you may now be wondering if serverless functions have to pay the cost of "cold start times" whenever they are activated.
In short yes but it's more nuanced than that. Recall our example above of clicking the buttons. You may have noticed that if you clicked the button a second time, the bar filled up a lot faster than the first time the button was pressed -- in this case the cost of the wait time was not paid twice. This is very similar to serverless functions as when they are first activated, they pay the cost of the cold start time. However, subsequent activations of the same function within a short period means that the cost of the cold start time is not paid. This occurs because after a function finishes executing, the container/environment required to run the function remains operational for a brief period in case it's needed again. Therefore, we get a "warm start" time on subsequent requests that are made to the same function within a short period. Awesome!! Shorter wait times if the function has multiple requests!
But we said that the cost had to be paid again?!?! If we have warm start times on subsequent requests, how is it the case the cost of cold start times is paid again? Recall that on subsequent requests the container/environment required to run the function remains operational and is kept alive. However, once the container is no longer needed (i.e. no new requests are being accepted within a window of time), the container/environment is destroyed. This means that when the function is requested again, the cost of the cold start time is paid again.
We can see this in the following example: Press the button once and then twice to see a warm start time. Wait 10 seconds and then press the button again. What do you notice??
After we pressed it proceeding with the 10-second wait time, it took a lot longer for the bar to fill up.
Requests from multiple regions
This is awesome! We've solved the ability for our services to easily scale elastically by moving to a serverless architecture but a bigger issue remains! What happens when function calls are being made from a different region than where the function is deployed? Is there a cost in response time for users?
To properly address this question, we have to recognize that similar to physical servers, serverless functions are mostly deployed to one region. For example, if we were a developer on the east coast of North America, we may opt to deploy our serverless function to the US-east servers. This means that users who live closer to the US-east servers would have faster response times than users who live further away from the US-east servers.
You may recall that we spoke about edge being the solution that addresses many of these concerns as it allows us to process data closer to our end-user. Moreover, we introduced the idea of edge compute -- the ability to execute code at the edge. Instead of deploying our function to one region, using edge computing, we can deploy our function at the edge of many regions.
The image below describes what we mean -- red dots represent edge servers in many regions and green user icons represent an individual user. Black arrows represent a successful connection (request and response) between user and server.
The image above showcases an edge function (red dots) that has been deployed at the edge of many regions and using this approach ultimately increases the response time for our end-users globally. In turn, it improves the user experience for users who are further away from the original deployed location as they now experience shorter loading times.
The edge is not just "compute"...
So far we've been talking about compute -- specifically edge compute. However, there is another type of edge that has existed for a while that escapes some of the limitations we're about to mention. This type of edge is known as edge storage but it's colloquially referred to as a CDN (content delivery network). These edge storage servers are responsible for storing and distributing static files at the edge of networks thereby making the delivery of content to end-users globally fast.
We can see this demonstrated in the interactive below. When we click on the single deployment (assumes that the services are deployed in us-east) location, we can see traffic to various countries at different speeds. However, when we click on the edge deployment strategy, we can traffic speed up to different countries.
You may now be wondering about the downsides of moving to the edge because as we alluded to earlier, it's not all guns and roses.
The little big things
When we speak about the edge in the context of using FaaS, we're moving the conversation to computing and executing functions/code at the edge. The downside here is that the resources that are available within a runtime on a traditional server may not be available in an edge runtime. This is the case as it is very difficult to distribute all of these resources across an edge network thus for edge compute to exist, the resources needed to make the runtime work have to be very light. Cloudflare Workers is a runtime that runs using the JavaScript V8 engine from Google Chrome and they provide access to several standardized APIs through Cloudflare’s global network.
We've mentioned the downsides of moving compute to the edge but to further illustrate them here, let's take a look at some of the APIs we get access to using Cloudflare Workers. The APIs are as follows:
- Fetch
- KV
- Request
- Response
- R2
- Stream
- Web Crypto
- WebSockets
The full list is available here. Cloudflare Workers are awesome and they allow us to do cool things with edge compute but as we've seen they do not give us access to the full runtime of NodeJS. It's important to note that this example is specifically looking at NodeJS but these limitations extend to other programming languages that require the execution of code at the edge.
Unfortunately, there's more. Yes, there are other downsides of edge computing that are not direct limitations of edge compute. Rather, these are side effects that require other technologies to advance so that we can truly harness the power of the edge on the web! One of these limitations is databases -- specifically when we speak about deploying serverful/serverless databases.
Question: What limitations do you think we run into with serverful/serverless databases in the context of edge computing?
Yes, we run into the exact same issues described above. Our database is deployed to one region (i.e. US-east) and thus the benefits of having the edge are quickly diminished. This is the case as our instance now has to connect from the edge to the database hosted in a specific region, grab the data and then return it to the edge function which then returns it to the the user. We can see this is illustrated in the diagram below.
When someone from Australia makes a request, that request goes to the edge function nearest to the user -- in this case, it would be Australia (dot in red) -- then the function from Australia makes a database call to servers that are on the other side of the world -- in this case somewhere in Europe, specifically eu-west-1 (dot in blue). We can see that the response times regarding the user receiving the data are relatively longer than what we'd expect using edge compute. This illustrates the need for innovative solutions to optimize data distribution and harness the full potential of edge computing in delivering exceptional user experiences.
Wrapping up
Sooooo What's at the edge?? In my opinion, speed... speed, efficiency, and improved user experiences are at the edge. The ability to process data closer to our end-users leads us to increase the speed at which they receive information, however, it doesn't stop there! Using edge compute, we can begin to tailor experiences to specific geographical regions. You can imagine an e-commerce store issuing specific gift cards depending on the geographic region a user is visiting from. Going beyond that, full page redirects that are evaluated at the edge are now possible. This unlocks the ability for A/B testing and feature flagging without the addition of third-party scripts or client-side JavaScript. The culmination of these features ultimately improves the user experience of your product as you can now personalize experiences.
By revisiting the architectural paradigms of the past, we see that edge computing builds on top of serverless architectures. While serverless computing excelled in scalability, it faced limitations when serving users distributed across various geographic locations. Edge computing came as a solution as it allowed us to deploy functions closer to end-users globally. This approach promised improved response times and enhanced user experiences, mitigating the impact of geographical distance.
However, we uncovered certain limitations, particularly to do with database location versus edge function deployments. We revealed that longer response times would result if the data was not globally distributed as well. Therefore, we would need to begin to think about having data at the edge and the resulting implications. Some Redis key-value stores are already implementing the idea of having data at the edge to enable localized data stores at the edge. This is similar to the efforts of upstash with their serverless Redis data platform. However, it would be exciting to see this same implementation for primary databases.
We also stumbled upon resource constraints within edge runtimes. Edge storage, or CDNs, excelled in delivering static content but faced different constraints than compute at the edge. Nevertheless, innovations like Cloudflare Workers offered access to standardized APIs and showcased the potential of edge compute, albeit with certain limitations compared to traditional server runtimes.
Earlier we mentioned that edge computing also had some involvement in IoT devices. Looking specifically at the amount of data generated on these devices, if not processed in real time, it could quickly become overwhelming for centralized cloud infrastructure to handle. Edge computing addresses this issue by processing data at the edge of the network, where it is generated. This allows for quick analysis and decision-making without the need to send the data to a centralized location for processing. Thereby reducing the time it takes for data to be analyzed and acted upon on these IoT devices.
Edge computing and serverless architectures have opened new doors for developers, addressing scalability and responsiveness challenges, while also presenting unique considerations and trade-offs. As we continue to innovate in this space, we must strike a balance between leveraging the speed of the edge and overcoming its limitations to deliver the best possible user experience in our interconnected world.
All right, I'm going to wrap it up there! Hope you found this useful, and I'll catch you in the next one... Peace!
Practice problems
PSSSST! Hey you! Yaa you! Enjoyed the article?? Here's a fun little exercise for you to try out! 👀
Exercise
These are a series of questions or mini-games that are associated with the article you just finished reading! They are meant to help solidify the concepts talked about in this article. Have fun!