In chapter 1 of this series, I talked about the changing relationship between companies and their digital infrastructure and code-creation capabilities today. In this chapter, I’d like to take a step back in time and investigate how that relationship has changed over time throughout the decades of the web.
The reason for this? To understand how and why the web has evolved over time to become what it is today will hopefully help you to understand what it is we want to achieve and the kind of service provider we want to be with Upsun.
So without further ado, allow us to take you back to where it all began: 1995.
The first decade of the web: 1995-2005
The web’s youthful years saw the democratization of personal computing as prior to this decade, most people did not have their own computers in their homes. From a dedicated business function, computing and the internet became pervasive. Concurrently two very strong and very contradictory phenomena happened.
On one side the web, there was a broadcast. As large corporations began talking to the public via a one-way conversation, they were not going to expose internal systems and transactional capabilities.
On the other hand, computing changed. You could have a computer under your desk that could be connected to from anywhere on Earth without having to ask anyone for permission. The barrier to entry running a networked computer program got so low that the reduction in cost allowed anyone to develop their own media. Leading to blogging and Wikipedias, as well as peer-to-peer ecommerce such as eBay and PayPal.
All of this was powered and accompanied by the growth of open source. During the years of 1995 to 2005, servers still ran Unix and Windows but Linux had started becoming the default choice. As were dynamic programming languages that were easy to deploy, particularly PHP. With little method to the madness these were the years of boom and bust (the dotcom crash) of fake-it-till-you-make-it. This is what allowed for the unit-economics to work with almost no cost to get started and very low marginal costs to consider. After all, when you don’t have to pay license fees for operating systems and programming languages, you can scale very quickly. Some of the current giants—like Amazon or Google—were created with almost no capital investment at that time.
The second decade of the web: 2005-2015
The second decade of the web saw the rise of centralized social media in its current form including Facebook. But also, with the growing number of available consumers, this decade welcomed the explosive growth of re-centralized ecommerce—such as Amazon Prime, Shopify, and Etsy. These kept some of the hallmarks of the peer-to-peer era as marketplaces that allowed others to play in their closed gardens but were centrally controlled.
Maybe more importantly, this decade was the moment where the internet became mobile with the apparition of the iPhone and the Android ecosystems. Suddenly with the scale of internet usage running a webserver under your desk was no longer an option. This was when server virtualization started happening with the likes of VMware, Solaris Zones, Xen, and KVM.
The unit-economics of building software kept trending down—these were the actual start-up years—while the unit economics of running hardware kept growing. So these were also the years in which the cloud appeared with again an incredible effect on democratization accompanied by a strong movement towards centralization. Led by Amazon Web Services (AWS) and slowly accompanied by Google and Microsoft.
2005 to 2015 were also the years of open source. Alongside profound changes in the way software was built and developed with agile methodologies, software testing, and automation. At the heart of the revolution was the arrival of a new distributed source code management tool, Git.
Git—initially developed for the Linux Kernel—allowed for much more scalable development compared to previous tools like SVN, CVS, and SourceSafe. Basically, it offered collaboration at scale. But at the same time, open source itself was centralized with Github becoming the centerpiece further democratizing software development. And while open source became dominant, most projects didn’t have a sustainable business model, other than serving some other business competitive goal—for example, Google with Android and Google with Chrome.
During this time, PHP, trivial to deploy and still running some of the largest sites—Wikipedia, Facebook, and Slack—and the most numerous ones—Drupal, and WordPress—starts getting displaced in the start-up world. Firstly by Ruby and then by Python, which was possibly a more productive option but much harder to deploy and manage.
These were the glory years of bespoke internet solutions. Frameworks appear—like Rails, Symfony, and Django—that allow the development of robust and scalable solutions at a rhythm unimaginable in the previous decade. These more productive, less trivial-to-operate solutions see operations automation tooling starting to appear—such as Chef, Puppet, and Ansible—which will soon be referred to as DevOps tools. While the “cloud” develops from a small number of low-level primitives to a large suite of high-level value-added tools. From proposing just-in-time provisioning and elastic capacity, Infrastructure-as-a-Service (IaaS) providers start building tightly-coupled services that produce strong vendor lock-in.
Courtesy of the growth of mobile usage and the reduction of complexity with the popularity of REST architectures displacing other forms of machine-to-machine communication, Web APIs become widespread. The web is no longer just something consumed by web browsers but something that is going to enable the interconnection of many different services from multiple providers in a predominately seamless way.
These are also the days of the divergence of hyperscalers: digital native companies that are going to define the cloud. Facebook, Amazon, and Google design their own hardware and build bespoke software architectures that have magical capabilities inaccessible to the common players. They continuously deploy systems that are highly distributed and redundant with elastic scalability. They will build and define what operations at-scale in the cloud mean. Node.js appears as another democratization allowing front-end developers to intermingle with the backend people. As well as enabling highly-concurrent use cases that were only accessible beforehand to hyperscalers and, to a degree, the good-old enterprises with Java and .NET. This is also the moment of copycat astronaut architects epitomized by MongoDB and Kafka.
A concept appeared in the early days of the second decade, a concept that would become dominant and yet disappear to the background shortly after. The concept involved a service by a computer system no longer being rendered by a single system, not even a single company. Many companies contributed their capabilities: first by simply injecting javascripts to a front-end; the “third party” ecosystem. Web pages are composed of resources coming from all over the place but also backends coordinating through APIs, often something that ends up with the GAFAMs.
As an echo to the first days of the web where web scale was accessible to all and IaaS providers and these new open-source technologies reduced the barrier to entry. But often at double the cost and a growing operational complexity.
Platforms-as-a-Service (PaaS) were created during this decade to reduce the cost of operations, mostly for monolithic applications with players such as Heroku and AppEngine. They promise to absorb this growing operational complexity by offering an opinionated runtime with some strong constraints. These constraints being no local files, only one supported database, and one way to deploy. Accompanied by the productivity of web frameworks, PaaSs reduced not only the initial time-to-market but also the time-to-scale. But both the architectural limitations and non-scalable pricing models meant that most companies migrated away from those after their initial growth.
Our decade of the web: 2015-2025
It’s the decade of 2015 to 2025, the internet of everything is ubiquitous. We no longer simply connect to the internet, most of the developed world lives and breathes it. Netflix, Uber, Airbnb, Booking, Amazon, Gmail, Google docs, Slack, and Teams are present in every waking hour of our lives. In business, these are the years of SaaS.
The unit economics of software development peaked bringing a new movement of centralisation. Encouraged by very few B2C platforms and the growing tendency to forgo bespoke software development and use SaaS solutions instead. Open source still powers all of these services in the backend but the value is captured by SaaS from one side and the cloud from the other.
As a counter-reaction, commercial open-source solutions start maturing with strong actors in the database (MongoDB, Elastic), DevOps (Hashicorp, Cloudbees, and Docker), and CMS/ecommerce spaces (Automattic, Acquia, and Magento).
From monolithic applications of the previous decade built with a simple, multi-tiered architecture—native to the internet, new solutions started to form. Often built with stateless microservices on one side, and a scalable data-fabric proposed by the cloud provider on the other. These new topologies brought with them an enormous level of operational complexity which led to Observability-as-a-Service—like NewRelic and Datadog—thriving as a remediation.
These new topologies see the need for other forms of deployment and workload isolation. Initially, as a development tool containers started to be democratized and the units of execution became smaller. Kubernetes quickly gained in popularity and in a way displaced Linux as the abstraction layer of the “thing you deploy to”. This is accompanied by a further explosion of operational complexity of the underlying infrastructure—on top of what was already contributed by the workloads.
New forms of PaaS appear that rather than manage the whole infrastructure—with an opinionated approach—only manage the smallest elements of the system. Small, stateless bits are tied together through service-meshes like Lambda and Serverless. Even though the inherent complexities of distributed systems make the adoption of those anecdotal at best. Most organizations either deploy actual monolithic applications to those or use them only at the very edge of the application for very simple tasks.
A lot of the current cloud is about the separation of the data-fabric which is the lock-in mechanism of the hyper-scales—BigQuery and RDS. But part of these trends is actually about the data layer moving to an isolated tier. Often “as a service” we can see that in the explosive growth of Snowflake, but also in the strong market position of MongoDB, Confluent, and Elastic search. This is even more true with the data-layer containing actual third-party data. Initially this was the OpenData paradigm but now this is all about machine learning and trained models. OpenAI and Anthropic are the main examples here of when “your” data becomes a fine-tuned layer on top of a closed-source large language model (LLM). Only accessible as a remote API, you are no longer in control of your data.
The first “decade” of production-grade LLMs was all-around proprietary SaaS-mode APIs—a decade that lasted all of a few months. The second and third quarter-length decades already saw the pendulum swing to open-source models with LLAMA generating an explosion there. We also saw the growing importance of on-device inference, private fine-tuned models, and explainable AI—all indicators that both approaches will be relevant.
Correlated to these growing usage patterns, cloud services expressed themselves as simple edge services. While another group of important players, CDN providers, introduced runtime capabilities beyond their initial caching features to run parts of applications at the edge.
And as this decade comes to an end, we are seeing the apparition of platform engineering as a response to the increasing complexity of modern software architectures, as Gartner puts it. Part of this trend is the failure of first-generation PaaS to scale beyond the initial simple monolithic use case. As a counterpart, we see MACH, a new moniker that we don’t expect to last, as potentially an ultimate example of the end of a decentralized cycle. The future is all about composability and will possibly end with companies running no IT infrastructure of their own—everything on SaaS.
Platform.sh, through Upsun, is in a very unique position regarding software composability, to propose a technical and organizational solution for product teams to both hedge the bets and integrate best-of-breed practices. Whether these teams are going fully SaaS/API, or are re-integrating capabilities in-house, or are early-adopters of AI-assisted coding and pervasive production-use of LLMs, or are waiting for the dust to settle and for technologies to mature.
