Dear Customer,
We would like you to know about the following unplanned outage.
Service Affected: SIP entry server
Impact: Major Impact
Start Date: Friday, 14th of August 2015, 11:40 AM (BST)
End Date: See updates below
Status Report: Service Status
Description:
The SIP entry server seems to have run into some issues and appears down although it is up. All our technicians are on top of this and are trying to locate the issue and solve this asap.
We apologise for the inconvenience that this may cause and if there are any questions, please contact our technical support team.
Kindest Regards
Support team
Update @ 11:55
The SIP entry server is back online and we can see SIP registrations coming in and succeeding again.
Update @ 12:05
It seems that devices can register to the SIP entry server again, but not fully / correctly. Our engineers are continuing their combined effort to tackle this issue asap.
Update @ 12:09
The issue has been pinned down to the Master server suffering a critical error. All service are now switching over to it's fall-back server which should then automatically take over the services.
Update @ 12:12
Our engineers confirm that all services have now been switched over to the fall-back server and that services have been restored and should be publically available asap.
Update @ 12:14
We can see calls going through the system again. If your device doesn't seem to have re-registered yet, you might want to force it to do so. We will monitor closely if all goes well on our side, of course.
Update @ 12:35
We have now seen a multitude of devices logging in properly and calls are being made so the issue seems to be corrected and working.
We will now continue to monitor this issue and investigate how it could happen and what we can do to prevent this from happening again in the future. Another matter to investigate is why the fall-back server didn't take over straight away so that we can take action there as well.
Update @ 13:15
Further investigation has shown that the Master server had issues because of hardware problems. When the Master server was brought back online, it was available but the MySQL DB's weer inaccessible resulting in the services having to switch over to the fall-back server. With that switch, all services came back online and were available again.
We will now further repair the Master service and bring it back in service as soon as possible to maintain redundancy.
A new announcement will be created for this maintenance session but we expect it to be on the 16th of August during the night. No significant downtime is expected when we do so.
We truly apologise for the inconvenience that this may cause and if there are any questions or remaining issues with this, do not hesitate to contact our support team.
Friday, August 14, 2015
