General Network Challenges - CCIE Section 1.1.c
Section 1.1.c of the CCIE syllabus contains a few over looked subjects. I've already touched on Unicast flooding and Layer 2 asymmetric routing. This sections will talk about out of order packets and the impact of micro bursts.
Out of Order Packets
This is a scenario where packet arrive out of order from when they were sent. A TCP flow can generally handle out of order packets, assuming there are not an abundance and the packets arrive if quick secession (3 ms according to wireshark). UDP flows do not have a mechanism to compensate. If UDP flows are used, the application is responsible for handling the out of order data.
Causes of out of order packets are generally a flow taking multiple paths, asymmetric routing or poor queuing. In the case of multiple paths, this can be most apparent when one of paths experience a delay. Path delay can be caused by a device along a path not CEF routing, process vs interrupt switching, or path saturation.
As discussed in later sections, Cisco CEF aims to reduce the occurrence of out of order packets by load balancing ECMP paths on a per flow basis by default.
References:
http://wiresharkbook.com/tr_samplepages/978-1-893939-97-4ooo174.pdf
https://en.wikipedia.org/wiki/Out-of-order_delivery
Microbursts
A microburst is a very brief period of time, microseconds, where very large amounts of data are sent, generally leading to period of full line-rate transmission, overflowing buffers (on the RX side) and causing short-term packet loss. What makes microbursts special are the difficulty in detection via traditional methods. Traditional monitoring will show that an interface's utilization is normal. This is due to SNMP polling intervals for example. Hints to microbursts can be seen on network interfaces as ignores and/or overruns.
If microbursts are suspected, packet captures can verify and help determine the cause of microbursts. Other than increasing network interface buffers, modifying the offending application can help curb the issue of microbursts.
With the advent of modern cut-through / line rate switching hardware, the impact of microbursts are somewhat mitigated. Legacy store-and-forward switches, for example, are more susceptible due to interface buffer exhaustion.
References:
http://www.arista.com/assets/data/pdf/7148sx-ixnetwork-microburst.pdf
https://en.wikipedia.org/wiki/Head-of-line_blocking
