“We’ve always done it that way”. This is the most common response that we hear when we propose the concept of automating manual processes to our customers, regardless of the industry that the customer occupies .
We ask our customers to take a step back and look objectively at their Network Operations C entre (NOC) and how it is run day to day. From this, we can identify and speak to NOC staff who carry out inefficient practices such as manual processes that could in fact be automated. For example, we still find customers logging into servers manually to check configuration or status. We still find customers manually correlating event data with trouble tickets, all slowing down MTTR (Mean Time to Resolution) of events and increasing costs.
It is often the case that NOC operators follow long established and out of date processes. When acquisitions or mergers are thrown into the mix, such as when teams take on new staff or monitoring of new infrastructure, problems arise that can go unaddressed for many years. When a company grows, the number technical team members often grow with it. As a result, maintaining a view on efficiency and how teams operate day to day can be difficult.
These practices are tolerated to maintain old and often inefficient processes. The potential for slower resolution time, wasted work hours and the unseen knock on effects of not automating processes is real and can be very costly for an organisation.
One customer we partnered with had a major problem. Operators had no way of knowing if events from HFC (Hybrid Fibre Coax) cabinets were legitimate or caused by maintenance work being carried out on the cabinets. The data relating to the status of the cabinet maintenance was held in a separate system that was not accessible to the operators. This caused confusion for all involved and slowed down the time of resolution for the events. This issue also caused a knock on effect where operators were unable to authorise truck rolls (deploying a service team) to fix non-existent issues. This was extremely costly.
To tackle these issues, we implemented a simple automation solution to correlate the events with the relevant reference data and to populate the events with maintenance information. This allowed operators to easily identify if a cabinet was in a maintenance window or if an engineer needed to visit the cabinet. It also updated the IVR (Interactive Voice Response) system, allowing their customers to check on outage data in their area, further reducing the number of trouble tickets opened.
In just 6 months, the customer saved over 250K.
So the next time your NOC processes are up for review, be wary of someone saying “We’ve always done things this way”.