Research Survey

This page contains a summary of on-going research projects that use FPGAs to accelerate computer architecture simulation.

• Micro's special edition on on-chip interconnects
• On-chip interconnects at SoCCentral

• Virtual-channel flow control. Dally. Parallel and Distributed Systems, March 1992.
• Express virtual channels: towards the ideal interconnection fabric. Kumar, Peh et al. ISCA 2007.

• An 80-Tile 1.28TFLOPS Network-on-Chip in 65nm CMOS. Vangal, Howard, et al. ISSCC 2007.
• A third-generation 65nm 16-core 32-thread plus 32-scout-thread CMT SPARC processor. Tremblay, Chaudhry. ISSCC 2008.
• Introduction to the Cell multiprocessor. Kahle, Day et al. IBM 2005.
• The Alpha 21364 network architecture. Mukherjee, Bannon et al. Micro 2002.
• On-chip interconnection architecture of the Tile processor. Wentzlaff, Griffin et al. Micro 2007.

• Exploiting ILP, TLP and DLP using polymorphism in the TRIPS architecture. Sankaralingam, Nagarajan et al. ISCA 2003.
• Scalar operand networks: On-chip interconnects for ILP in partitioned architectures. Taylor, Lee et al. HPCA 2003.

• A network on chip architecture and design methodology. Kumar, Jantsch et al. VLSI 2002.
• AEthereal network on chip. Goosens, Radulescu. Design & Test of Computers, 2005.
• NoC Synthesis Flow for Customized Domain Specific Multiprocessor Systems-on-Chip. Bertozzi, Jalabert. Parallel and Distributed Systems, 2005.
• MPARM: Exploring the multiprocessor SoC design space with SystemC. Benini, Bertozzi et al. VLSI Signal Processing 2005.
• Efficient synthesis of networks on chip. Pinto, Carloni et al. Computer Design 2003.

• A complete network-on-chip emulation framework. Genko, Atienza et al. Design, Automation and Test in Europe, 2005.
• NoC emulation: a tool and design flow for MPSoC. Genko, Atienza et al. IEEE Circuits and Systems Magazine, 2007.

Genko and Antieza focused on the design of TG and TR (Traffic Receptor) nodes. Switches are generated using Xpipes compiler. As an emulation, the timing, resource contention and flow control details in the interconnects are not modeled. Consequently, the behaviour of the network presented may not be representative of real systems. Program traces are presented as an alternative way to load the network besides synthetic traffic; however, it is unclear how the traces are obtained (it is mentioned that it's a set of packet descriptors), what kind of traces they are, and what is the timing of the trace playback. In addition, only small test cases are implemented in this system (3×2 and 2×2).

• Fast, accurate and detailed NoC simulations. Wolkotte, Holzenspies, Smit. NoCS 2007.

Wolkotte et al. trades off simulation time and on-chip resources by time-multiplexing all N routers in an NoC onto a single HDL model on the FPGA, which implements a five-port 4VC wormhole Router. Because the simulation is completely serialized, each system clock requires 2N FPGA clocks to simulate. An on-board ARM processor generates and feeds the stimuli flits to the FPGA, collects results after a fixed number of simulated cycles and computes statistics from the results. The software components run in parallel with the hardware simulation to amortize the communication cost. The final FPGA Router runs at 6.6MHz while the ARM processor runs at 84MHz. The size of the simulation is limited only by the amount of on-chip block RAM available, which are used to stored the “contexts” of the Router. Even though only synthetic traffic is presented in the paper, it should not be hard to use a network trace instead. However, it is unclear how inter-router communication is handled during a simulation period. This could potentially lead to buffer overflow on congested links, which is not addressed in the paper.

• NoCs as hardware components of an OS for reconfigurable systems. Marescaux, Mignolet et al. FPL 2003.

Marescaux et al. describes an on-chip network for heterogenous systems-on-chip. The main concept is the use of three different networks to carry control, configuration and data communications. This seems rather straight forward.

• On-chip interconnect exploration for multicore processors utilizing FPGAs. Shelle, Grunwald. WARP 2006.

NoCem (OpenCore project) offers a synthesizable model for on-chip network. Supports configurable topology, bus width, VC etc. Resource utilization is the bottleneck; the XC2VP30 part only fits a 4×4 mesh with 32-bit datawidth. Traffic stimuli is provided through Microblaze processors to emulate real coherence traffic. The applications used are unknown.

• Exploring FPGA network on chip implementations across various application and network loads. Schelle, Grunwald. FPL 2008.

Schelle and Grunwald emulates NoCs by building VC switches on the FPGA. Resource utilization limits the size of their NoC to 4×4 (on XC2VP30). It appears that VC-based routers are expensive due to the control logic for arbitrating and allocating VC resources and crossbar. One way to alleviate resource usage of a VC router may be to decompose the parallel switching for all output ports into multiple clock cycles. This avoids the crossbar, and the other VC logic may be pipelined. This approach is not explored by the authors and need to be verified.

• Networking on chip with platform FPGAs. Brebner, Levi. FPT 2003.
• SoCIN: a parametric and scalable NoC. Zeferino, Susin. Intergrated Circuits and Systems Design, 2003.
• RASoC: a router soft-core for NoC. Zeferino, Kreutz et al. DATE 2004.

RASoC proposes a composable router that is build using individual input port and output port modules. The switching logic (crossbar, flow control) is distributed in each port module. The ports implement small buffers (2 or 4 flits, 32-bit flits).

• A Fast Emulation-Based NoC Prototyping Framework. Krasteva, Criado et al. ReConFig 2008.

Krasteva et al. uses partial reconfiguration to build a NoC emulation framework that can be reconfigured without resynthesis. DRNoC contains a fixed mesh network that connects reconfigurable blocks. Hard core libraries are used to populate the reconfigurable blocks to model routers and TGs. The fixed mesh network has cross links so it can emulate non-mesh topologies (such as star). It is mentioned that independent routers can be mapped to the same router-slot to save area, however the paper does not elaborate on how this is done. The final system built is a 2×2 on a XC2VP30 (although they claim a 4×4 system can be mapped to the same FPGA). A software tool is developed to configure and read data back from the emulator, which is similar to our approach.

Designing NoCs on FPGA

• Packet switched vs. time multiplexed FPGA overlay networks. Kapre, Mehta et al. FCCM 2006.
• CuNoC: A Scalable Dynamic NoC for Dynamically Reconfigurable FPGAs. Jovanovic, Tanougast et al. FPL 2007.

• PEPE: A trace-driven simulator to evaluate reconfigurable multicomputer architectures. Garcia, Sanchez, Gonzalez. Lecture Notes in Computer Science, 1996.

Uses communication trace to evaluate the interconnection network in a multicomputer system. Parallel programs are first simulated in a virtual multicomputer (seems to be a functional processor simulator) to generate the trace. The trace is a sequence of messages. Each message can be independent or dependent. The former is associated with an absolute timestamp of injection. The latter has a pointer to a trigger message and the delay before injection after receipt of the trigger message. Seems to be a simple method to provide some feedback without actually executing the program during network simulation.

• Edinet: An execution driven interconnection network simulator for DSM systems. Flich, Lopez et al. Lecture Notes in Computer Science, 1998.

Uses Limes (an execution-driven simulator for multiprocessors, sounds like a SMP version of SimpleScalar) to generate memory requests to be simulated by an interconnect simulator (NetSim). Limes has a memory system simulator (MemSim) and can run SPLASH-2. Instruction execution time is calculated at the end of each basic block, hence a rough estimate without regards to microarchitectural details such as out-of-order execution. The memory model is not cache coherent. Results show that adaptive routing algorithm does result in shorter execution time for the two applications examined (OCEAN and FFT).

• Limes: A multiprocessor simulation environment for PC platforms. Magdic. Internation Conference on Microelectronics, 1997.

Execution-drive SMP simulator that models a cache-coherent shared-memory system. It runs the SPLASH-2 parallel benchmark. Parallel execution is simulated one thread at a time, running natively on the PC, until it encounters a memory reference, at which point the operation is rescheduled until the its timestamp reaches the global simulation time. Simulation time is calculated from instruction counts at the end of each basic block. A centralized simulation kernel makes sure memory references from all threads are interleaved in the correct order and supply this trace to the back-end memory simulator.

• A methodology for mapping multiple use-caes onto networks on chips. Murali, Coenen et al. DATE 2006.
• An FPGA Approach to Quantifying Coherence Traffic Efficiency on Multiprocessor Systems. Suh, Lu, Lee. FPL 2007.

• A case for bufferless routing in on-chip networks. Moscibroda, Mutlu. ISCA 2009.
• Deadlock-free adaptive routing in multicomputer networks using virtual channels. Dally, Aoki. Parallel and Distributed Systems, 1993.
• Hermes: an infrastructure for low area overhead packet-switching networks on chip. Moraes, Calazans et al. Intergration, 2004.
• Adaptive bubble router: a design to improve performance in torus networks. Puente, Beivide et al. Parallel Processing, 2009.

• In-network cache coherence. Eisley, Peh, Shang. Micro 2006.
• Virtual circuit tree multicasting. Jerger, Peh, Lipasti. ISCA 2008.

• A survey of research and practices of network-on-chip. Bjerregaard, Mahadevan. Computing Surveys, 2006
• On-chip communication architecture exploration: A quantitative evaluation of point-to-point, bus, and network-on-chip approaches. Lee, Chang et al. Design Automation of Electronic Systems (TODAES) 2007.

• Homepage: http://www.ece.cmu.edu/~protoflex
• Brief summary [ PPT ]

• Homepage: http://users.ece.utexas.edu/~derek/FAST.html
• 1.2 MIPS (aiming for 5~10 MIPS ¹⁾ )
• Brief Summary [ PPT ]

• Asim: A Performance Model Framework. Computer, pp68-76, 2002.
• A-Ports: An Efficient Abstraction for Cycle-Accurate Performance Models on FPGAs. FPGA '08.
• (HASim) Quick Performance Models Quickly: Closely-Coupled Partitioned Simulation on FPGAs. ISPASS '08.
• Brief summary [ PPT ]

• Bochs: http://bochs.sourceforge.net/
• PTLsim: http://www.ptlsim.org/
• QEMU: http://bellard.org/qemu/
• FeS2: http://fes2.cs.uiuc.edu/

• DRAMsim: a memory system simulator. Wang, Ganesh et al. Computer Architecture News, 2005.
  • Maybe can port this to use our interconnect simulator
• How to simulate 1000 cores. Monchiero, Ahn et al. Computer Architecture News, 2005.
• COTSon: infrastructure for full system simulation. Argollo, Falcon et al. Operating Systems Review, 2009.

• Homepage: http://ramp.eecs.berkeley.edu/

• Homepage: http://www.cs.wisc.edu/gems/home.html
• Orion Power Simulator Summary [ PPT ]
• GEMS Interconnect Model Summary [ PPT ]
• Breakdown of topologies found in publications that use GEMS [ Topology ]

• Simics: http://www.virtutech.com/
• SESC: http://iacoma.cs.uiuc.edu/~paulsack/sescdoc/
  • Cycle accurate multiprocessor multithreading
  • Peak 1.5 MIPS
• UNISIM: http://unisim.org/site/start

• SICOSYS: http://www.atc.unican.es/SICOSYS/
• FlexSim: http://ceng.usc.edu/smart/tools.html
• Netmaker: synthesizable VC router (for ASIC)
  • Full RTL implementation of VC router with speculative routing
  • Traffic generation module

http://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=01639260

• Brief summary of the Nios II and MicroBlaze soft processor systems [ PPT ]

• On-Chip Network Research Resources Page

• Principles and Practices of Interconnection Networks. W. J. Dally and B. Towels. Morgan Kaufmann, San Fransisco, CA, 2004.