 |
Dedicated Migration Update: 8/4/07 6PM
As of 6PM Saturday, we are continuing to identify and replace faulty components for the remaining few servers with hardware issues.
Call volume in our Dedicated Technical Support Department has returned to normal levels and individual outages should be reported to technical support.
We are still keeping all available staff in both our data center and our call center and will continue to work the remaining reported issues 24 hours per day throughout the weekend.
If your need your server restarted, please use the on-line reboot request form at https://secure.valueweb.net/cgi-bin/vw0202.exe, as these requests are being responded to in the order they are received.
Dedicated Migration Update: 8/2/07 12PM
As of noon today, fewer than 50 servers are still affected by a hardware failure of one form or another as we continue to systematically identify and replace faulty components. Some system have experienced more than one component failure, which has made their restoration more problematic. Additionally, we are concurrently working through the queue of reboot requests and other support issues, as we continue to address the critical hardware failures.
If your need your server restarted, please use the on-line reboot request form at https://secure.valueweb.net/cgi-bin/vw0202.exe, as these requests are being responded to in the order they are received.
Finally, we are continuing to keep all available staff in place both in the data center and at our call center facilities to handle every request and inquiry as quickly as humanly possible.
Dedicated Migration Update: 8/1/07 2PM
As of 2PM ET, there are fewer than 100 servers remaining in the hardware failure repair queue. We have also identified a possible source of some sporadic network latency and availability issues affecting certain IP blocks related to an upstream provider that was broadcasting outdated routing information. This issue is has been resolved, and we expect it to have a significant impact on network stability for those customers affected.
We are continuing to work around the clock on all remaining service issues, and have additional staff in place both at the data center, and our call center to expedite the restoration of service as much as possible.
Dedicated Migration Update: 7/31/07 9PM
As of 9pm this evening, we are continuing to make progress in restoring the remaining servers that were affected by the data center move. In the past hour, an additional 20 - 25 servers have been restored and are back on-line. Our engineers are continuing to systematicly work through each affected server, diagnose the root cause of the issue, replace hardware as needed, and bring each machine back up as fast as humanly possible. The admin team will continue to work around the clock, 24x7, until every single customer is back on-line.
Dedicated Migration Update: 7/31/07 4PM
First, we at Hostway sincerely apologize for the downtime experienced by customers during the recent server migration and we are doing everything within our power to restore service to those customers still affected by the move as quickly as possible. Overall, the migration involved a total of approximately 4,000 servers, approximately 500 of which experienced a hardware or networking configuration failure that caused the amount of downtime for these machines to exceed our original estimate of 12 - 15 hours. Of the 500 affected servers, approximately half have since been restored and are now operational.
The system administration team is working feverishly to systematically diagnose the root cause of the outage on each of the remaining servers, which range from network configuration issues to physical hard drive failures. Concurrently, our admins and support staff are working to proactively contact affected customers and work through each situation on a case by case basis. Finally, we have all available staff in place on the dedicated support queue to handle the volume of phone inquiries, and while the service level is still not where we'd like it to be, the average answer delay is now running between 10 - 20 minutes.
Going forward, we will be posting regular updates to this page in addition to updating the status message on the phone system.
Please be assured that this outage has 100% of the focus of the entire Hostway management team, and that our number one priority is to get each and every customer back up and running as soon as humanly possible.
