How AppSignal's monitoring product found continuity

About AppSignal

AppSignal is a comprehensive monitoring solution for applications, with support for Ruby, Elixir, Node.js and JavaScript. It provides real-time monitoring of errors, performance and application health. AppSignal offers a wide range of functionality, including health monitoring, user sessions, call races, DB tracking, API monitoring and integration with other development tools. The all-in-one monitoring solution is suitable for a wide range of developers developing and maintaining Web applications.

Thijs Cadier, co-founder of AppSignal explains: "This is a monitoring product for all midsize organizations. We bring a handful of best practices that enterprises (hopefully) use. We put all those functionalities in a very convenient package where you don't have to set much. With this, even as a small team, you have access to a 360-degree solution to monitor your applications with."

From separate VPS services to consistent infrastructure with dedicated servers

Previously, AppSignal's IT infrastructure was housed with multiple VPS providers, where the monitoring software was hampered by so-called "noisy neighbors." In a shared cloud environment, one user taxes the usable resources to the point where you experience inconvenience. Think of resources like bandwidth, disk and processor consumption.

"We were on virtual private servers before this, we had to move away from that because it just didn't bring optimal performance. Basically, it was workable, but because of the fluctuating performance, we were constantly having to work with it," Thijs said.

Because of the type of load on our monitoring software, we suffer more from "noisy neighbors," because we don't want to experience many spikes. We usually have a fairly streamlined server load. The system is built in a way that makes it extra sensitive to anomalies. As a result, sometimes as many as twice a week the system became inconsistent because of those "noisy neighbors."

What others are doing on the same server impacts the performance at that VPS. This is especially prevalent when it is not convenient. "It's 4 o'clock on Friday afternoon and you think: it's been fine and suddenly that performance of one of the eight services collapsed. Then you do have to get to work, when it's actually not convenient."

Despite the recognizable drawbacks, Thijs remains realistic about solving them: `You always have to exchange different types of pains with each other. In this case, that's less flexibility in capacity planning, because we have to give earlier notice if we want to scale up." When you opt for an IT infrastructure that uses only dedicated servers, it gives more flexibility and control at the application level, whereas with a cloud infrastructure, the same goes again with the level of scalability.

Several solutions appeared to be possible, yet it felt like mopping up. "You can work around this annoyance, but then you are faced with the choice: are we going to put time and energy into engineering then?" For example, by automatically moving load? In the end, it doesn't outweigh the effort we put into making it manageable. For that reason, among others, we chose to move away from this type of infrastructure."

Furthermore, AppSignal's system is not designed for VPS machines with differences in configuration.With the Apache Kafka software, all load is distributed very evenly across the entire infrastructure.In this setup, it occurred that one of the machines was 20% slower than the other, creating a kind of imbalance. To restore balance, we had to constantly distribute traffic between machines evenly.In the end, it comes down to the physical hardware, which was the only way to make sure everything stayed consistent.So, four Dell servers with the exact same CPU and configuration."That simple fact is what saved us the most time," Thijs concludes.

We had actually kind of misjudged that SSDs were fine for our infrastructure. Eventually it turned out that we did really need NVMe SSDs, so we switched to single disk NVMe SSDs.

After a period of searching for parties offering dedicated servers, the answer came in the form of a message in a Slack group. Here Worldstream was mentioned, after which the team went to investigate. In particular, the promise of support with 24/7/365 response times proved to be a hit. "With other parties they may have solid hardware and solutions, but the technical support is then not something you can really rely on. Then we found out that you have excellent support, especially the fact that someone is there at night. That was actually exactly what we needed."

24/7/365 support, important but rather not necessary

The fact that experienced engineers are present in the data center day and night is reassuring, but according to AppSignal, it's even better: "Well, we almost never need your support in practice, because you have the basics very well in order. We found it very important that if there is something once, we can count on you." By this, Thijs is referring to aspects of the infrastructure, such as the network and hardware. "Because the network is solid and reliable, we have not experienced any outages in the good 1.5 years we have been with you."

When AppSignal did need support the following stood out: "It's nice to be talking to someone who understands your infra. "When we get a wake-up call because our own monitoring system is going off, we know there's someone with you who understands it. What you see with those big hosting parties is that they respond quickly, but they have staff who don't understand anything. An issue like this has to be escalated before anything actually happens. Before you know it, you're sitting for hours waiting for some clarity, not to mention waiting for a solution. What we like about Worldstream is that with you we don't need a special SLA (service level agreement), it's very nice to know that regardless of company size you are taken seriously."

AppSignal uses dedicated servers in Naaldwijk and Amsterdam for their infrastructure because of their high availability. All data processing of the AppSignal services takes place in Worldstream's data centers. Only for the DNS servers does AppSignal use AWS. Then you simply don't rely on a fiber connection. In addition, physical security played a role. ISO certifications 27001, 9001 and 14001 and PCI-DSS guarantee compliance with internationally recognized security standards. Furthermore, Worldstream employs strict physical security measures, including biometric access control, electric fencing, video surveillance and on-site security personnel 24/7/365.

AppSignal's future prospects

Now that AppSignal is spending less time troubleshooting their IT infrastructure, they can focus on the continued development of their solution. Eventually, AppSignal wants to offer support for all common programming languages. For example, a Python integration is coming and logging was recently added to the product. "What we think is more important is to make customers' lives better in all kinds of big and small ways. Our monitoring solution is now fairly complete, of which logging was the last piece of the puzzle. Within all the monitoring categories, we are now looking to find smarter ways again to make our customers' lives easier by continually making the software better."

"At the time we switched to Worldstream we had about the same spend as with that VPS provider," Thijs says. "By now they are 18 months on and it has not yet been necessary to scale up significantly. While at the time we were already approaching maximum capacity of the VPS setup. In the current setup we are only at 50% load. With the VPS provider, we would have spent much more by now."

We are satisfied with Worldstream because they understand our infrastructure and know who we are. If we email you then engineers present know right away who we are. In the end you choose a party that looks like you. We find that quite often our customers are somewhat similar to us in terms of corporate culture and size.