Routing protocols and architectures/Hints on the architecture of network devices

The architecture of routers has been evolving over time to increase more and more their packet processing capability:

• first generation (until early 1990): lower than 500 Mbps;
• second generation (early 1990): about 5 Gbps;
• third generation (late 1990): starting from 50 Gbps;
• multi-chassis (recent trend): of the order of Tbps.

First generation

First-generation router architecture.

First-generation routers were basically modified PCs:

• network interfaces: they are normal NICs, in a greater number and in manifold kinds with respect to the ones on a normal PC;
• memory: solid-state memory (e.g. SD card) is preferred because less subject to failures with respect to a mechanical hard disk;
• operating system: it is optimized specifically for network traffic processing.

Advantage

• economy of scale: it uses components manufactured on a large scale instead of components dedicated to networking world.

Disadvantages

• bottlenecks:
  • shared bus:
    • slow path: each packet transits twice over the bus;
    • arbitrage: one NIC at a time can use the bus → multiple interfaces can not work in parallel;
  • memory access: the routing table is stored in a data structure in the generic memory → accesses are not so localized as in traditional PCs, and the cache is little used;
  • processing capability: the CPU interrupts the operating system whenever a packet arrives;
• network interfaces: traditional network cards have not many ports and only support the most common kinds of interfaces;
• maintenance costs: using open-source components (e.g. PPP client) brings integration troubles and varied configuration ways.

Currently this architecture is used for mid-low-end routers, whose performance is adequate for small offices.

Second generation

Second-generation router architecture.

Second-generation routers try to solve the performance problem by moving the processing load from the core CPU to edge units: NICs are replaced by line card, 'smart' network cards which add forwarding modules to physical/MAC components:

• CPU: the packet processor is a dedicated processor for 'simple' packet processing;
• memory: each incoming packet is stored into a local memory split from the one (generally TCAM) containing the routing table;
• routing table: sometimes the updated routing table is copied from the central unit to line cards.

Packets can follow two ways:

• fast path: the packets falling in the typical case (e.g. IPv4) transit only once over the bus: they are processed directly by the packet processor and are immediately sent to the output interface;
• slow path: least frequent packets (e.g. IPv6, OSPF) are leaved to the core CPU for a more sophisticated processing, at the cost of a definitely lower throughput.

The work of line cards should be coordinated by the core CPU, on which routing protocols (e.g. OSPF) are running too, which is mounted on a card called supervisor in high-end systems.

Advantages

• core CPU: it works only for packets along the fast path → it can be less powerful;
• fast path optimization: the first packets in a connection follow the slow path, then the central system marks the connection as eligible for the fast path and all following packets will follow the fast path;
• flexibility: the introduction of a new protocol (e.g. IPv6) only requires updating the software on the central system.

Disadvantage

• shared bus arbitrage: one line card at a time can use the bus → multiple interfaces can not work in parallel.

Third generation

Third-generation router architecture.

Third-generation routers focus on the word parallelization problem by replacing the shared bus with a switching fabric (or crossbar) able to handle multiple transfers: a line card is connected to another line card by shorting the switch at the intersection of their 'wires' in the switching matrix.

Two line cards can not be connected to another same line card at the same time → an arbiter is needed which drives switching points and solves contention situations without collisions.

When output interfaces are slow in sending packets with respect to the switching speed of the switching fabric, packets can be queued in various ways:

• output queuing: buffers are placed on output interfaces → worst case: switching fabric N times faster than receiving speed, with N = number of input interfaces;
• input queuing: buffers are placed on input interfaces → the switching speed of the switching fabric does not depend on the number of input interfaces, but suffers from the head-of-line blocking problem:
  1. two packets in two input queues are destined to the same output queue;
  2. the arbiter lets one of the packets pass blocking the other one;
  3. all the following packets in the input queue, even the ones destined to free output queues, have to wait for the 'head of line';
• virtual output queuing: it solves the head-of-line blocking problems by keeping one queue per output at each input;
• buffered fabric: buffers are placed inside the crossbar at the spot of switching nodes.

Disadvantages

• arbiter:
  • it constitutes a bottleneck, especially in case of frequent decisions due to small packets (e.g. ATM);
  • scheduling policies for quality of service may make its hardware more complex;
• queue buffers: they are expensive memories because they should be fast.

Multi-chassis routers

Multi-chassis routers aim at scalability: multiple flanked racks are connected so as to appear as a single router:

• the processing power is multiplied by the number of chassis;
• there is more physical room for network interfaces → a large number of links can be connected to a same device;
• routing is concentrated in a single station (like in the telephone world) instead of a lot of small routers spread over the network → the data station can serve all the users within a wide geographical area (e.g. Piedmont).

Service cards

Another recent trend aims at flexibility: routers are no longer purely layer-3 devices, because a lot of other features not belonging to layer 3 are being added, such as cache, firewall, network monitor, ACL, etc. The goal is to customize the service offered by ISPs, bringing back a portion of the business currently in the hands of content providers.

A currently common solution are service cards, line cards enriched with value-added services pre-configured by the manufacturer. Service cards however are not so flexible: being not programmable, a new service card should be purchased to add a service.

Major current issues

Throughput is no longer a current issue by now, but the current major issues are:

• purchase costs: to reduce the cost of network devices dedicated components are being replaced by general-purpose components made on a large scale:
  • mainstream CPUs have too short product life cycles and too long replacement times → Intel recently proposed embedded CPUs with longer product life cycles and shorter replacement times with respect to mainstream CPUs;
  • in dedicated systems the programmer could exploit the memory hierarchy, by storing the RIB into a slow memory and the FIB into a fast memory (e.g. TCAMs), while general-purpose systems autonomously decide what to put into the cache → GPUs, having partitioned memories but limited processing versatility, should be addressed;
• operational costs: power consumption and heat dissipation may be significant;
• flexibility: it is required to move toward the separation of control features from physical hardware: C3. Introduction to Software-Defined Networks.