You don't appear to be logged in. Sign Up for a game account. It's free and only takes a few moments! Sign Up
Sedy Admin Sedy
posted on Friday, March 30 2012, 11:38 PM in Updates
Well our luck seems to just continue

The server is currently experiencing massive lag and lost packets due to some routing problem with our host.

They say it better than I could, so here it is.

Hello,

We had a routing problem last night due to a software bug affecting two core routers in Roubaix. These Cisco ASR 9010 provide bandwidth for data centers in Roubaix (RBX1 RBX2 RBX3 RBX4 RBX5) and the connection to Paris, Brussels, Amsterdam, London and Frankfurt. In short, the core routing in Roubaix.

This bug is known and was linked to new cards we have put into production in late January (24x10G slots). For some reason the random map will detect
ECC RAM errors and no longer route packets. But especially nevertheless the card do not have state "down" and remains in the router as if it was good.
Other routers continue to send packets but in front there is no one. Everything falls into a black hole and the network no longer works properly.
Worst case: a failure not net.

That night, three cards of 2 24x10G ASR 9010 routers had this bug and almost simultaneously. This has broken the network into 3 pieces: United States / London / Amsterdam / Warsaw Roubaix and Paris, Frankfurt, Madrid, Milan, by drawing the packets in Roubaix. Usually the traffic would
been rerouted but there it was aspirated and blocked in Roubaix.

So we were not able to administer the network and retrieve logs from all routers to know the origin of the problem. We sailed to the old, with connections
emergency / outside to connect to each backbone router to check if the router which is causing the problem. This operation took time, because in addition to two routers have been down and it had been slow to understand that it came not just router rbx-g2-a9 but also because of rbx-g1-a9. Once we restarted
the three cards all came back in 5 minutes.

There are about three weeks. We have already opened a ticket to Cisco about this problem of RAM ECC. Cisco has worked on the problem and we could
provide .. This morning the software patch to be applied on routers to fix this problem here. We will do this tonight. No failure to predict.

It also looks at how to improve the management of our routers in the case where the whole backbone is down for some reason that hopefully never comes. It can handle this case but it is slow. Very slow.

In all cases, the outage lasted more than 99.9% ie 1:22 when we have "right" in 43 min months of downtime. There is therefore the Penalties
triggers for exceeding the time allowed.Example: on OVH Dedicated Server is 5% per hour of downtime.


I can see from their status logs that they are still working on it.
Responses (7)
Sorry, but guests cannot view the replies here.
Your Reply

Latest Discussions

2 weeks ago 3 208
1 month ago 2 156
1 month ago 2 382

Latest Important Stuff

4 years ago 0 5335
5 years ago 0 3011

Latest Events

Latest Updates

3 months ago 0 472
5 months ago 2 527

Latest Replies

There is something else to say, it's tough to get points and reputation when you keep losing Battlef
2 weeks ago
Thank you! And also can you make a requirement for joining in the event? Like 20lvl or above. Ther
2 weeks ago
Yeh it's a good idea. I will look into it
2 weeks ago
Sedy wrote: They should have ended tomorrow, not today, my bad. I will extend them until Sunday.
1 month ago