Nov 11, 2011: Bluemile has issued the following RFO for the issues this past week.
Bluemile has been experiencing connectivity issues within our dedicated and XEN server environments over the last week. Through research, it has been determined that a firmware upgrade that was loaded onto our DDOS boxes on August 18, 2011 was the root cause of the issue. Data analysis showed that network connectivity drops started right after we upgraded the firmware and the frequency of the drops increased over time.
On Wednesday, 11/9/2011, we downgraded the IOS of our DDOS boxes and ran limited traffic through them. During this testing, we did not experience any network drops.
On Thursday, 11/10/2011, we routed all production traffic back through the DDOS boxes at 12:55pm EST. There was connectivity/congestion issues when all traffic was initially routed back through the DDOS boxes. This was due to the initial packet analysis done by the DDOS boxes.
We apologize for any inconvenience this issue may have caused. We are working with our DDOS vendor to determine the issue with their new firmware revision.
Wizzsolutions Network Team