FPGA-Aided Microarchitecture Exploration

Increasing number of modern digital systems now use interconnection networks as the communication fabric. Simulation is an essential tool to evaluate the performance of interconnection networks, but detailed simulation in software is often slow. Since a network has few global states, and most nodes in the network are quasi-independent from one another, there is opportunity for hardware acceleration.

The goal is to build a timing simulator to study the behaviour of the type of interconnection networks used in CMPs. Modern on-chip interconnection networks are packet-switched and consist of a network of routers. The simulator will model such a network, with a focus on the details of the pipelines and buffers of the on-chip routers. The simulator accepts packet traffic from an external source (a host computer or a full-system simulator), and simulates the traversal of the packets in the network.

The simulator provides tools to meausre various metrics of the network performance, such as aggregated throughput, average end-to-end latency, etc.

A tentative Project Plan.

• Scalability
  • > 200 nodes on one chip (tentatively the Xilinx Virtex 5 on the ML507 board)
  • Scale to more nodes?
• Performance
  • Same or higher level of details compared to GARNET
  • Simulate 10M interconnect cycles per second (100x slow down compare to real time?)?

The interconnection network simulator should model the functionality of the following components:

• Flit-level traffic
• Pipelined VC router
  • Buffer contention
  • Switch and virtual channel arbitration
  • Link contention
  • Pipeline bubbles
• Routing protocol (dimension-order at least)
• Flow control (credit-based, etc.)
• Generic topology (at least mesh and torus)

The simulator should provide an interface to interact with an external processor or memory system simulator. The external simulator can be running on an embedded processor or a CPU off-chip. It provides stimulus traffic to the interconnection network simulator and receives latency numbers from the latter.

• User provides a configuration file for
  • topology
  • router parameters (buffer sizes, number of VCs ?)
  • traffic pattern (to test for pathological cases?)
• Router switching scheme / routing protocol
• Software configurable?

Research Survey

• 5-stage pipelined VC router
• Flit-level input buffers
• Crossbar allocator
• Routing algorithms

• Execution-driven: interface to external multiprocessor function simulator
• Synthetic workload can be provided by simple traffic generator nodes

On-chip interconnect

Our on-chip interconnect is similar to the one in Packet Network Simulator project. This architecture allows arbitrary communication pattern between the on-chip nodes (Routers, Traffic Generators, etc.), so we can simulate different topologies without changing the simulator hardware.

HDL specifications of the simulator components

• SVN: svn+ssh://navy.eecg/autofs/steffan/r/r1/wangda/svn/icsim
• Performance Simulator to model the cycle-accurate behaviour of the hardware design
• Progress summary 1: /doc/summary_0809.ppt

1. What's the performance penalty for using a generic on-chip network to simulate a specific topology instead of building the target topology directly?
   • ~50% speed reduction as measured on the network-simulator-on-chip simulator ¹⁾
2. Which full-system simulator to use?
   • Need to analyze the performance bottleneck in a full-system simulator
3. How to synchronize timing between external processor simulator and the interconnect simulator?
4. Any merit to not assume unit bandwidth and latency for on-chip links?
   • As long as everything operates based on flit-cycles, there's no benefit to assume non-unity bandwidth and latency. Links to memory controllers are also on-chip.