Update on service interruption
Posted on May 22, 2018
3 min read
Yesterday we experienced a service disruption that impacted a large number of our mobile and business data customers. Our world is increasingly connected and reliant on the breadth, speed and functionality of our network and our customers rightly have high expectations of Telstra.
We are very sorry for any inconvenience caused by yesterday’s issue. We provided regular updates during the interruption and we can now share more information on the technical cause.
Starting from approximately 10:00am some key network equipment failed causing a disruption to 4G voice and data services nationally. The impact was widespread and with a large number of customers dropping back to 3G there was significant disruption to 3G voice and data services as demand exceeded the capacity of our 3G network.
We have identified that the initial cause of the disruption was a software fault which triggered multiple elements across the network to fail.
The network is designed to switch onto standby hardware in such a situation which it did. However, following the failover a further fault caused an interruption which impacted 4G services. There is redundancy built into these systems but the system did not operate as intended.
By around 11:40am we had successfully isolated the impacted network infrastructure and services began to be progressively restored. It took about two hours for full service to be restored after that, although our team was working longer with some government and enterprise customers to help them with their systems recovery.
We are still investigating the root cause of the software fault. We are working closely with our technology vendors on the specific element of software which triggered the issue to correct that fault and to understand why the redundancy in our network did not prevent customer impact.
Our network is large and complex comprising of more than 500,000 individual network elements. Given this size and scale, no one can guarantee that disruptions won’t ever happen. What we can do is reduce the likelihood and the impact of those disruptions, especially as mobile services are more important than ever for our customers.
We know how important reliability is to our customers and we are continuing to invest in our network to meet the ever growing demand for connectivity at the highest levels of resilience. We are investing billions of dollars a year to deliver the best mobile network in Australia with the largest coverage footprint and faster speeds in more places.