Providing big data and data science talent to detect and address anomalies

Fortune 50 Telecommunications Company

Unusual network activity was a challenge for our client

NorthStar provided our client with big data and analytics talent for their network content delivery team. Our client was experiencing unusual network events that drastically degraded service for their customers. The objective was to detect these, and other, issues from the data to help catch and address issues. In a big data environment with millions of events occurring per second, analysts and teams can quickly become inundated and be unable to find what is needed in the middle of all of the chaos. Also, the client neither had a solid definition of what was customer-impacting nor how to identify their occurrence.

Identifying the culprits

Our data scientists quickly surveyed the team and identified a new dataset that wasn't well understood or used, and identified ways to analyze network events and developed anomaly-detection algorithms that would indicate when something abnormal occurred. They also identified ways to connect a separate dataset to know what anomalous events were customer impacting, and then predicted customer impacting events using machine learning.

Results

Our project reached several quantifiable goals using an analytical approach.

1. With the analytical approach we were able to identify anomalous events without a pre-defined event signature. This was done in a modular way that allowed various types of events to be detected again without knowing the details of what a future event would look like.

2. By identifying events impacting the client's ability to deliver their desired quality of service to their customers, these anomaly detection and prediction algorithms give insight to the client when issues begin to happen and often in advance of their occurrence so they can address them proactively rather than waiting to either receive reports of the issue from their customers because client personnel was not able to observe the issue amidst the overwhelming amount of data and activity of normal operations.