Built from the Roots of Synthetic Monitoring: Our Story

Synthetic monitoring has been around in the tech industry for over two decades and its adoption rate has only been rising year after year. With that, the number of companies who claim to offer synthetic monitoring has also been steadily increasing.

Prior to starting Catchpoint, our founders were responsible for some of the largest infrastructures in the world and used synthetic monitoring tools on a daily basis. After realizing there were certain challenges that needed to be addressed to provide an efficient digital experience, they decided to build their own tool to do the job.

According to our founders, there were four major challenges the current tools in the market could not overcome, as listed below:

Challenge #1: In the beginning stages of monitoring, there were a handful of companies who offered synthetic monitoring services that primarily focused on actively monitoring web applications from the end users’ perspectives, i.e. monitoring from different cities and ISPs. While the core monitoring methodology of these companies was correct, they failed to reduce the MTTR.

Challenge #2: The companies that emerged later adopted the strategy of building low-cost synthetic monitoring solutions. It failed to serve its purpose, though, because the adopted methodology was only partially correct.

Challenge #3: The Internet space evolved over the years with new technologies such as HTML5, single page apps, responsive websites, new caching methodologies, new protocols  (Websockets, HTTP/2, MQTT), and technology providers (DNS, ADC, CDN/Multi-CDN, Cloud/Multi-Cloud, SAAS, etc.) also evolved drastically. However, these synthetic monitoring services failed to adapt, in turn forcing companies to use services from multiple vendors.

Challenge #4: Finally, active monitoring/synthetic monitoring is all about collecting data actively from multiple perspectives and the collected data has multiple recipients – NOC, OPS, DEV, ENGG, marketing, business, vendors, etc. The users should be able to slice and dice the collected data based on their needs; however, almost all of the solutions in the market today have rigid reporting capabilities forcing users to explore yet another solution just to analyze the monitored data in a customized way.

This blog series is an effort to clearly define the synthetic monitoring methodology and establish an efficient way to measure digital experience by addressing all the four challenges highlighted above.

What is Synthetic Monitoring?

In three simple points:

1. Synthetic monitoring is all about simulating an end user’s action on an application by using real web browsers/applications or emulators
2. These actions/tests were scripted and built using programs such as Selenium and run on software agents/servers, which are distributed across multiple geographies and ISPs
3. The actions/tests were configured to run at regular intervals which enabled companies to monitor applications 24/7

While this core methodology worked well, companies who offered monitoring services based on this methodology failed to address the four challenges described above.

Challenge 1: Reduce the mean time to repair (MTTR)

While the traditional synthetic monitoring solutions were good at detecting a problem, they couldn’t pinpoint the root cause of the problem. Which means the users had to run additional tests to isolate the issue and this made it difficult to detect the problem quickly.

Example 1: If a user is unable to access a website because of a DNS error, most synthetic solutions can report the errors, but won’t be able to indicate exactly what is failing within the DNS.

Example 2: If the user’s connection to an application times out, the synthetic solution will not be able to identify the cause of the failure. If the timeout resulted from an ISP failure, multi ISP failure, application failure, etc.

This leads to an incomplete diagnosis of a problem and a poor user experience.

Solution:

We focused on reducing the MTTR while retaining the core methodology of synthetic monitoring. Catchpoint provides:

• Intelligent monitoring platform where our monitoring nodes are stateless, which means that if the testing environment is not healthy, the test will not run and this helps avoids false positives.
• Automated debugging. This replaces the manual troubleshooting required after an alert is triggered. Catchpoint automatically performs debugs such as DIG, DIG + Trace, Traceroute, Ping, Auto Verify tests, intelligent alerts, historical compare, and smartboards to look at all details at a single glance.

These features help reduce the MTTR significantly.

Challenge 2: Partial/incorrect adoption of synthetic monitoring methodology

While companies have adopted the approach of simulating end users actions, many hosted their monitoring nodes/servers on cloud services such as AWS, Softlayer, Azure, etc.

The issue with this approach is that end users don’t access your website or application from those servers. When deciding where to host our nodes, we considered these factors:

• End users are located across thousands of cities and hundreds of ISPs
• ISPs are not accountable for successfully delivering your content
• ISPs must talk to each other, leading to latency and connectivity issues
• Congested networks
• CDNs focus at last mile optimization, not monitoring from the ISP is like blindly trusting your vendors

Solution:

We knew that having a vast number of nodes and global coverage would enable our customers to accurately measure and monitor their applications from an end user’s perspective from any location including:

• 80+ Countries
• 180+ Cities
• 200+ ISPs

Challenge 3: Failing to adapt to evolving technologies and needs

The technology eco-space has evolved drastically and includes new technologies such as:

• Single Page Applications, IOT, Microservices
• New protocols Web Sockets, MQTT
• Simple DNS hosting to managed DNS providers, Application Delivered Controllers, Cloud Load Balancers
• CDNs have moved on from just network acceleration & caching services to Managed DNS, WAF, FEO, Cloud services, etc.
• REST & SOAP APIs: Companies are now providers and consumers of APIs, which enables them to integrate themselves with others and vice versa

Solution:

We created a comprehensive synthetic monitoring solution that will evolve with the industry and technology. Catchpoint provides a comprehensive set of 15 test types:

1. Web Tests
2. Web Transactions
3. API Transactions
4. DNS Experience
5. PING
6. TCP
7. Traceroute (ICMP, TCP, UDP)
8. FTP
9. SMTP
10. Websockets
11. SSH
12. IMAP
13. NTP
14. SMTP
15. MQTT

Additional features enable companies to capture custom metrics of your choice, evaluate vendor performance and your SLAs, and more.

Challenge 4: Rigid reporting

As discussed above, synthetic monitoring is all about collecting and analyzing data from multiple perspectives. Every company has a unique approach to analyze data, and catering to their needs is extremely important to ensure optimal value. This is the biggest limitation that is pushing users to either depend on their own spreadsheets and pivot charts or to buy third-party tools for data analysis.

Solution:

The fact that the founders of Catchpoint were already familiar with similar solutions enabled them to understand the features and flexibility that the user expected from these tools. Catchpoint data analysis and reporting were built to address these challenges.

We offer 35+ metrics to choose from and provide flexibility for slicing and dicing data within the UI.

In addition to this, Catchpoint offers custom visualizations to enable users to build custom charts and graphs. Users can access their data from Catchpoint by integrating third-party tools, such as DataDog or Slack, using Webhooks and REST API.

Conclusion

Today’s digital landscape requires a monitoring platform that can:

• Provide a comprehensive solution that covers the monitoring of all types of end points
• Monitor from cities across geographies and ISPs – monitoring from the cloud is not an accurate measure of end-user experience
• Modern monitoring features that can monitor and measure all the modern technologies
• Intelligent solutions that can auto debug issues and in turn reduce MTTR

In our next blog, we will discuss different types of monitoring nodes/servers and when to use them.