A night at the Global Operations Centre
Posted on July 30, 2015
4 min read
Ever wondered who is responsible for maintaining Telstra’s networks? Paul Harrison, Network Operations Manager at Telstra’s Global Operations Centre (GOC) in Melbourne shares what it’s like during a typical night shift.
I’d like to share with you what it’s like during a night shift at Telstra’s Global Operations Centre, the 24×7 monitoring hub of Telstra’s national and international networks. My work day starts at 11pm when most of the country is already in bed. Along with over 20 colleagues, our role is to monitor Telstra’s networks and identify any issues that may impact services. You could be forgiven for thinking not a great deal happens in Telstra’s network between 11pm and 7am, but this couldn’t be further from the truth. Most night shifts are a hype of activity, as this is when Telstra performs a lot of its maintenance activity to ensure the network is stable and resilient during peak times.
This night started off routinely with a handover from the previous shift, but it didn’t take long for the relative calm of the shift to be broken and the work area to come alive with technical chatter, telephone calls and an influx of network alarms – the official network maintenance window had begun.
Among the expected alarms generated by planned maintenance activities, I identified some high priority alarms that didn’t match the activities already underway. It didn’t take long to discover the impact was caused by an optical cable that had been cut in rural NSW which was affecting landline, ADSL and mobile services. It was going to take our technicians three hours to arrive at the site due to its remote location, so my team began reviewing the impact to customer services and developing a restoration plan.
By 2am the first few hours of the shift had passed quickly. I checked in with each technology group to understand the progress of their activities. An hour later, it was time to grab a bite to eat and a cuppa and take a break from the computer and general buzz of my work area for a few minutes. If I didn’t take the opportunity then it would have been lost as the second half of the shift usually ramps up and becomes quite hectic, as we work to ensure all changes are completed well before the start of the next business day.
As the alarms continued to clear throughout the early morning with planned maintenance activity closing and other incidents being resolved, I turned my attention back to the outage in NSW. By now, our technicians had arrived onsite and confirmed the damage had been caused by a third-party contractor performing civil works.
With only a few hours remaining before daybreak, I set up a phone conference with our Major Incident Management team, technology group and onsite technicians to establish the quickest way to restore services.
After listening to the onsite team’s assessment of the damaged cable, a plan to restore the affected services was agreed. It was going to take 2.5 hours to dig to the damaged cable and then prepare it for joining, so a priority restoration plan was agreed to ensure most of the impacted customer services were restored prior to the start of the business day. Over the next few hours, I was provided with regular updates to ensure the plan is still on track. Thanks to the collective efforts of the team, the majority of affected services were restored before day breaks. Shortly, after all services were restored.
Before I know it, my counterpart had arrived for the start of the day shift and it was time to run through a shift handover. At 7.15am, daylight is just beginning to peak over the horizon as I drive home. All I have on my mind is a well-earned sleep before I come back and do it all again later that night.