ALL EH Services Down
Incident Report for Endless Group
Postmortem

We first noticed our host was entirely down at around 00:05 UTC. This included our VM management system, Proxmox. This signaled us to contact our provider, InterConnX, about the issue. They provided the following response (PII Redacted):

Hi <Redacted> thanks for the buzz. Looks like the account is fully vetted now. Our upper management had a few things come back to them about servers being used for wrongful marketing and advertisement but it seems to have been closed out by upper management and the server is back online.

We do not know who may have submitted a report to our host, nor what the report consisted of, as we are not aware of any marketing or advertisement being done using our services. If you were the creator of this report and wished only to have a certain customer/website removed for this offense (which is also against our own guidelines), please open a support ticket with us.

Thanks,

EH Administration

Posted Apr 29, 2019 - 20:48 EDT

Resolved
Our host has identified the problem and has restarted our server. A postmortem will follow with further information on the cause of the issue.
Posted Apr 29, 2019 - 20:32 EDT
Investigating
We are investigating the issue with all services being down. The issue has been determined to be, in part, caused by our top-level host. Further details pending.
Posted Apr 29, 2019 - 20:05 EDT
This incident affected: Customer Systems (DirectAdmin, Apache, FTP, MySQL, Exim, Customer VPS', Networking) and EH Systems (Homepage/Signups, Helpdesk, Email Server, Billing).