DevOps India Summit 2020

The DevOps India Summit brings together some of the best thought leaders in the DevOps space to discuss and debate everything DevOps related. This year, the event was a virtual one, held in a virtual venue with a virtual lobby, exhibit hall, booths, and a virtual auditorium for speakers presenting online. The DevOps India Summit 2020 focused on DevSecOps, experts from around the globe led discussions around the evolving security trends and tools and spoke about the innovations and transformations that they were driving within their own organizations.

The event featured keynotes from industry leaders to inspire, motivate, and share valuable insights. The virtual event was designed to provide an immersive learning experience for the attendees, allowing them to interact and collaborate with each other; DevOps practitioners and SMEs could share knowledge and understand industry trends.

Catchpoint presented two informative sessions on DevOps, DevSecOps, and the role of Digital Experience Monitoring (DEM) in ensuring a secure end-user experience. In this blog, we look at some of the key takeaways from these sessions.

Bulletproofing DevSecOps with End-User Monitoring

The first session was led by Navya Dwarakanath, Solutions Engineer, and Ankit Kumar, Performance Engineer, from Catchpoint. They spoke about the impact of cyberattacks and security breaches on end-user experience. The application development cycle involves various checkpoints to detect bugs and ensure the application is working as designed. And this is made possible with the use of multiple tools at different stages of the application development cycle.

The image below illustrates these tools, including those specific to application security, and where it is used in the development cycle. The security tools that are part of the DevSecOps pipeline are crucial for detecting vulnerabilities in the application. Monitoring is also an integral part of this process as it lets you analyze and evaluate security vulnerabilities from the end user’s perspective.

With so many different components in the application architecture, it helps to zoom in to each component, inevitably you will identify several potential security vulnerabilities. And then you have end users accessing the application from different locations over different connections, add to this the strategies in place to mitigate cyberattacks such as SQL injections, cross-site scripting, or a BGP hijack, at different layers of the application. In a compromised security scenario, working through all of these vulnerabilities to identify and resolve the issue can be a time consuming and daunting task.

In such situations, it is important to have the right data and tools at hand to quickly remediate the incident. Here are some real incidents that we helped our customers with, end-user monitoring helped fast-track detection and resolution.

1. DNS Spoofing

The image below shows DNS resolution from different locations, we can see how one of the assets in China is pointing to the wrong IP (a Facebook IP) and this does not happen from Mumbai. This indicates DNS spoofing which can divert traffic away from your application. Monitoring the DNS gives you visibility into one of the most crucial components that determines application reachability.

2. Expired SSL

Zoom had an outage for 8-10 minutes due to an expired SSL (as seen in the screenshot below). Monitoring SSL can alert you when an SSL certificate is set to expire so that you can avoid such sudden and complete outage, which can have a big impact on the end user.

3. Dev and Sec Miscommunication

Proactively monitoring user workflows can also help detect unexpected issues arising from miscommunication between different teams if they are working in silos. For example, the security team of one of our enterprise customers was in the process of implementing multi-factor authentication but the development team somehow missed the memo. The transaction tests (screenshot below) set up for the workflow started failing which alerted the development team and they had to roll back the changes. The development team had not prepared for the workflow changes and it was impacting the end-user experience. This example highlights the importance of maintaining communication between Dev and Sec teams and following the DevSecOps rules.

Identifying and Resolving Security Incidents

All the scenarios discussed here reinforces the need for the visibility that DEM provides. When dealing with cybersecurity, it is important to evaluate how secure the application currently is compared to how secure it was. Another important aspect is how security incidents are managed, is the approach proactive or reactive?

Proactively monitoring the application provides a performance baseline across different time periods which can then be used to alert any unusual change in performance metrics. The deviation in any metrics value can be an early signal into security issues so you can act on it quickly.

DEM can also play a role in resolving security incidents faster – pick up any specific pattern to an attack, the attack vectors being used, correlating metrics that are specific to indicate the type of attack vector. Breaking down how long it took, for example, the DNS and SSL handshake, makes it easy to identify where the issue is. Additionally, DEM also allows you to test how well equipped the application is and how it reacts to security attacks.

The examples we discussed illustrate how monitoring plays a significant role in the DevSecOps pipeline –

• how it helps you test your application security.
• how proactive monitoring can be effective in alerting potential incidents.
• how to isolate the root cause and resolve the issue faster.
• how you can mitigate the impacts on end-user experience.

The Nuances Behind the Buzz – DevOps, DevSecOps, and Observability

The second session was presented by Nithyanand Mehta, VP Tech Services, and General Manager of Catchpoint India. The session focused on understanding the buzz around DevOps, DevSecOps, and Observability. The concepts around each of these terms have evolved over the years but the importance of delivering customer experience remains a priority.

A lot has changed in every aspect of application development and deployment:

• Most organization are now relying on a hybrid infrastructure for better performance and availability
• Open Source is preferred as it reduces vendor lock-in and speeds your access to innovation
• Micro-Services and APIs enable scale-out, resilience, and recovery
• Continuous Development and delivery of updates and new features through automation

DevOps aims to shorten the development life cycle while delivering features, fixes, and updates frequently while in close alignment with business objectives. DevOps approach builds reliable applications quickly, the various stages in the DevOps lifecycle involves:

• continuous development
• continuous testing
• continuous integration
• continuous deployment
• continuous monitoring

In tandem with the changes in application architecture and infrastructure, the security aspects have also evolved and changed. The concept of DevSecOps was a result of these changes which required security to adapt to the DevOps approach. There are hundreds of different tools and practices being used to build and deploy and this is adding lots of new components between the end-user and the application itself.

DevSecOps has to account for the vulnerabilities brought in by the different layers and processes of the application including the last mile (ISP), backbone (internet transit), DNS, CDN, WAF services, and third-party integrations. So once again, there is a complex network of components that brings in its own set of vulnerabilities. Security is no longer an internal process, organizations should focus on the distributed architecture, infrastructure, and services to gain complete visibility and maintain control over the application delivery.

DevOps is all about shared ownership, but whenever an incident breaks, there is always some level of finger-pointing. Let us take the example of an AWS outage that happened last year. Catchpoint customers were alerted of DNS failures and latency reaching AWS S3 name servers across North America, the West Coast was badly impacted.

The service provider assumed they were in the green as nothing was reported in the logs or by their monitoring tools while the end users complained and vented their frustration online. It took the provider five hours to report the incident and another few hours to resolve it.

The early signals of a security breach or performance degradation must be a part of the DevSecOps strategy to gain better end-to-end visibility.

Application performance management (APM) goals are mainly to detect and resolve to reduce business impact. Application monitoring has transformed along with the changes in application technology. Effective APM requires careful selection of relevant metrics that indicate true end-user experience. Start with these basic questions:

• Is the app available?
• How soon does the app load?
• How soon can the user interact with the app?
• Are there any errors in the transaction workflow?

Once you have these answers, you can then define the different metrics/SLIs for each layer/component based on the purpose of those services.

• Front End
• Back End
• Network
• Third parties
• Delivery chain infrastructure – CDN, DNS, Cloud, Firewall, Load balancers, Proxy, etc.
• Design and Architecture

Monitoring the relevant metrics with relevant tools will give you a clear perspective of end-user experience and this visibility is considered true observability.

DevOps and Beyond

DevOps India Summit 2020 was a virtual event catering to the DevOps community. It addressed a lot of the hype and doubts around the adoption of DevSecOps with discussions centered around the challenges and opportunities associated with harnessing the power of DevSecOps, the topics ranged from the pillars of DevSecOps to implementing DevSecOps in large organizations. There were many engaging and insightful sessions throughout the day from thought leaders in the DevOps space.

DEM remains an important part of the DevOps, and DevSecOps, pipeline. The two sessions hosted by Catchpoint focused on the need to invest in monitoring especially with the adoption of new technologies and practices in application development and deployment.

Learn more about gaining visibility into application services starting with your CDN provider. Listen to the on-demand webinar now!