PaaS under pressure: secrets to stress testing before Black Friday

This blog is based on the Upsun live stream with Greg Qualls, Thomas di Luccio, and Guillaume Moigneu.

As the holiday season approaches, developers and technical teams face a familiar challenge: ensuring their applications can handle massive traffic spikes without breaking. During a recent Upsun live stream, Greg Qualls, Director of Product Marketing, Thomas di Luccio, Product Manager, and special guest Guillaume, Field Chief Technology Officer at Upsun, shared valuable insights from years of managing e-commerce platforms through Black Friday chaos.

The reality of high-traffic events

Guillaume, with over 25 years of development experience, has witnessed firsthand what happens when preparation meets reality. Working with e-commerce agencies that manage 30-40 major European retailers, he has seen companies scale from normal operations to handling 1,200 CPUs worth of traffic in a single day.

"We have clients that reached 1,200 CPUs just for the day, getting insane numbers of orders," Guillaume explained. "That's always a challenge because those applications sometimes are not really meant to scale that high."

The stakes are real. As Guillaume noted, there's no worse feeling than having your CEO tell you that the company missed a million dollars in revenue because the site went down, or hearing about a customer who tried to buy $20,000 worth of jewelry but couldn't complete the transaction.

Essential strategies for black friday traffic

1. Start months ahead

The most critical insight from these seasoned developers? Start your preparation months in advance, not weeks.

Guillaume recommends a strategic shift in priorities: "Every time you go for Black Friday, especially if that's a new application or something you haven't really battle-tested before, you need to take a few months to actually slow down on new development and new features."

This doesn't mean implementing a complete code freeze, but rather shifting focus from feature development to stability and performance optimization.

Master the art of caching

When asked what single thing developers should focus on if they have limited time and resources, Guillaume's answer was immediate: caching.

Caching can be amazing. Caching can remove a lot of bottlenecks in your application when it's done right. There's a famous saying: there are two things hard in computers - caching and naming.

The benefits extend beyond just performance. Proper caching improves user experience through faster loading times and reduces the risk of overloading backend resources like databases. Guillaume referenced the often-cited statistic that Amazon was losing $1 million in revenue for every 100 milliseconds added to their loading times. 

Use observability tools

Guillaume's second recommendation centers on observability tools that help you understand how your application actually behaves under stress.

Invest in observability - give yourself the ability to witness your application and infrastructure behaving and try to learn from this," he advised. "It will tap you on the shoulder and say, 'Look at this, look in this direction,' and show you processes that keep fetching all the resources or slow requests that need optimization.

Create realistic testing scenarios

Thomas di Luccio learned this lesson the hard way while working for a ticketing company. When they secured a deal with a large venue, everything seemed perfect until the season announcement day arrived.

"Everybody was buying tickets at the same time, big rush on the website, everyone hitting the same API, and everything collapsed," Thomas recalled.

The key insight? You need to create testing scenarios that actually mirror real user behavior. Users don't just browse catalogs - they add 100 products to their cart and then remove them, using it as a wishlist. They hit refresh repeatedly when pages load slowly. These behaviors need to be part of your testing strategy.

Test your processes, not just your code

One often-overlooked aspect of preparation is testing the human element. Regular testing serves two purposes:

• It helps new team members understand your systems and processes before they're needed in a crisis
• It reveals how changes in your codebase affect performance under load

As Greg pointed out, “You're not just testing the system, you're testing people against the systems and processes that you've put in place.”

Set up automated safeguards

Thomas advocates for automated testing that includes performance regression testing. "You can test regression - features not working, cosmetic regression - but performance regression is also super important to make sure that some feature you deploy doesn't ruin something else."

However, he warns against the temptation to "fix" failing tests by simply raising the limits. "I was expecting to do fewer than 50 SQL queries, and okay, 52 - I'll fix the test. You have to resist that temptation."

Don’t over-customize

Custom code and endless feature tweaks can be a performance killer.  One unexpected challenge comes from within your own organization. You need to fight some feature requests that might impact performance. I've seen clients asking for completely custom experiences, with numerous promotion rules. Every time you add those features, it uses more resources.

The solution? Work with stakeholders to rework feature requests in ways that satisfy their needs while meeting performance expectations.

Communicate with third-party service providers. 

Don't forget to communicate with your vendors and third-party service providers. Guillaume emphasized the importance of understanding what you can expect from partners when things go wrong: response times, escalation procedures, and backup plans.

"When you rely on independent partners, you need to make sure that they're actually up to the task," he noted. Companies should reach out to clients beforehand to prepare rather than realizing last-minute that something was missed.

Final thoughts

Black Friday can feel like the “worst-case scenario” for developers. For business leaders, though, it’s the best case, the dream of record traffic and booming sales. The disconnect often occurs when marketing and technology don’t align.

The takeaway? Planning, testing, and cross-team communication are the real Black Friday survival tools. Whether it’s through better observability, smarter caching, or simply running drills, preparation is the difference between celebrating record revenue and explaining outages to the CEO.

As Guillaume summarized: "If one Black Friday went bad, do not wait one year hoping that everything will be better." Now is the time to start preparing for next year's holiday rush.

Hope is not a strategy. In the high-stakes world of e-commerce and high-traffic applications, thorough preparation and testing are the only reliable paths to success.