VM Host Disk Failure (Was: Continued Maintenance)
Incident Report for Endless Group
Resolved
We have been monitoring our host systems and have not observed any further issues. We consider this incident to be resolved. If you are still experiencing problems, please contact our support.
Posted May 31, 2023 - 11:04 EDT
Monitoring
All host systems have been successfully restored and confirmed to be operational. New drives were installed in the failing machine. Additionally, our new host system has been joined to the cluster. We are now monitoring to ensure that all components are operating normally. All customer systems should be online at this time. If you are experiencing an issue with your system, please contact us using our support channels.
Posted May 28, 2023 - 04:17 EDT
Update
We are continuing to work on restoring the failed host. Most customer systems should be back online at this time.
Posted May 27, 2023 - 02:06 EDT
Update
We will be rebooting the remaining host system in order to finalize the update.
Posted May 27, 2023 - 01:38 EDT
Identified
We have identified the problem as a failing disk in one of our host machines. We are recovering the machine but as this may use a large portion of our in-network bandwidth, please expect degraded performance on your sites at this time.
Posted May 25, 2023 - 19:25 EDT
Investigating
We are experiencing a problem where one of our host machines is unable to successfully reboot following the upgrade. We are working on this issue as fast as possible.
Posted May 24, 2023 - 23:20 EDT
Monitoring
The previous maintenance tasks have not yet been completed.
Posted May 24, 2023 - 23:02 EDT
This incident affected: Customer Systems (DirectAdmin, Apache, FTP, MySQL, Exim, Customer VPS', Networking) and EH Systems (Homepage/Signups, Helpdesk, Email Server, Billing).