30% OPEX savings from AI-driven auto-recovery across 200+ microservices

An AIOps framework for a large telecommunications provider that identifies anomalies, classifies each problem, suggests the recovery, and auto-recovers known patterns.

Outcome

Incoming tickets down from 2,500 to 3,000 a month to 500 to 600, average process time cut from 30 minutes to 3, 30% of fallout handled automatically, 30% OPEX savings, and a 5% revenue increase.

See the results

Talk to an amp'd specialist

At a glance

Client & industry

Large telecommunications provider, with 200+ microservices and applications integrated

Engagement

AIOps framework build to identify anomalies, classify problems, suggest recovery, and provide single-click recovery action

Scale

5 to 6 million Splunk logs, with 500K to 1.2M exceptions and 300 to 600 incidents a day

Algorithms

Linear regression and SVM, with NLP for problem categorization

Maps to below services

MLOps & Scalable ML Platforms, Cloud & Platform Engineering

Where it started

Orders stuck in problem states and thousands of tickets a month, all resolved by hand.

Business problem

Between 3% and 10% of orders ran into problems and needed manual fixing. 2,500 to 3,000 tickets arrived every month, and every incident was resolved manually.

Technical barriers

Every monitoring tool had to feed its data in, with thresholds set carefully enough to avoid false-positive alerts across 200+ microservices and applications.

Agivant's role

AIOps architecture, ML and NLP model development, integration across 200+ microservices, and automated recovery

What Agivant built

Agivant built the framework on a BigPanda-based architecture for monitoring, event correlation, de-duplication, enrichment, and escalation, and integrated 200+ microservices and applications. Models train on data gathered from every relevant source and retrain regularly. In production, each incoming incident or alert runs through the models, which predict its category and its solution and publish both to a dashboard where the support team acts. Value in days and weeks.

BigPanda event pipeline

Monitoring, event correlation, de-duplication, enrichment, and escalation for 200+ integrated microservices and applications.

Anomaly detection

AIOps identifies anomalies across the integrated estate.

Problem categorization

AI, ML, and NLP models classify each incoming incident.

Solution prediction

Models predict the likely solution for each incident and suggest the recovery action.

Single-click recovery

The support team triggers the suggested recovery from the dashboard with one click.

Auto-recovery for known patterns

Known failure patterns recover automatically, with no manual step.

Defects from application logs

Code and data issues found in application logs are raised as defects, and the code fixes that follow reduce incoming tickets.

Single-pane dashboard

Health information on systems, applications, and incidents in one view.

Train and retrain loop

Models train on data from all relevant sources and retrain regularly, with support team feedback feeding each cycle.

How it is built

Monitoring and correlation	Microservices	Databases	Log source	AI and ML	Models	Recovery
BigPanda	Java and Spring Boot, 200+ integrated	MongoDB NoSQL, Oracle for order metrics	Splunk, 5 to 6 million logs	Python, linear regression, SVM, NLP	Multiple models, Model 1 through Model N, with a feedback loop	Auto-recovery for known patterns, single-click recovery for the rest

Before

Manual incident resolution, 2,500 to 3,000 tickets a month, and 3% to 10% of orders needing hand fixes.

After

Correlated events feeding ML predictions, auto-recovery for known patterns, defects raised from logs, and one health dashboard.

Proof in production

Data from every relevant source trains the models, and they retrain regularly. When an incident or alert comes in, the models predict its category and its solution and publish both to the dashboard. Known patterns recover automatically, and the support team acts on the rest with a single click. Code and data issues found in application logs become defects, and the code fixes that follow brought incoming tickets from 2,500 to 3,000 a month down to 500 to 600, with average process time down from 30 minutes to 3.

Measured results

Incident resolution moved from manual work to AIOps-driven handling, with 90% savings.

30%

OPEX savings from auto-recovery

500 to 600

Monthly tickets, down from 2,500 to 3,000

30 to 3 min

Average process time

30%

Fallout handled automatically

90%

Savings on incident resolution

+5%

Revenue

The repeatable pattern

Reusable asset

The trained models and their retraining loop stay with the framework, so every new incident and every support team action feeds the next prediction. Every build amplifies the next.

Challenges overcome

The AIOps service needed receiver and feeder APIs into a cloud-based solution, and those APIs had to clear security review before integration could proceed.

Enterprise movement

Foundational

Operational

Path to autonomous