Testing

Before trying to test the supersmall soft processor, I recommend you run through the installation guide to make sure everything's set up properly first.

Simulation

simulate

There are numerous test applications contained in the testing/applications folder, each of which runs a representative software program, and then ends (if successful) by continuously printing 0xdeaddead to register 28. Unfortunately, depending on how fast your computer is, actually running these tests is going to take a significant amount of time. Let's try one of the smaller ones for starters. Using a handy script contained in the testing directory, we're going to run modelsim non-interactively and compare its results against the results of an object-code simulator.

testing/simulate des

You should get a long string of output, as modelsim executes the program instruction by instruction, ended by the happy words "Trace Validated." That means that the output from the simulated processor is the same as that given by a software MIPS simulator. Congratulations!

Mint

If you go exploring in the testing/applications directories, you'll notice that each one contains a mint folder, which contains static trace files. These are produced by a proprietary MIPS simulator that, while available to the University of Toronto Computer Engineering Group, isn't freely distributable. That means, if you want to add your own programs to the test suite, you're going to have to find another way to make the software traces.

simulate_gui

Now, "simulate" tells you whether or not the processor passes a certain test, but sometimes, while debugging, you need more output than that. In that case, use "simulate_gui", a shell script that behaves the same way, except it doesn't attempt to run the application until completion, and it opens it up in the modelsim gui, with a number of useful waveforms pre-added for your inspection. If "simulate" is reporting failures, use "simulate_gui" to get a better idea of what's going wrong. To try it right now, type:

testing/simulate_gui des

That command should open up modelsim, run the simulator for 800 us, and let you take a look at the waveforms. With "simulate_gui", don't be afraid to edit simulate_gui.do, also contained in the testing directory, to adjust the output according to your tastes. In particular, if you start adding or deleting waveforms, modelsim will likely give you errors unless you edit simulate_gui.do accordingly.

simulateall

One advantage of having multiple tests is that sometimes one test will pick up a bug that other ones don't. Following this pattern, the supersmall distribution package includes a range of test applications, which can all be run sequentially using the "simulateall" script. The output is rather terse, and unless you have a spectacular computer, it will likely take a significant amount of time to run (i.e. hours), but it's useful when you want to be absolutely sure that your modifications have still resulted in a fully working processor. As input, it takes a newline-separated list of benchmarks. The one normally used for nightly testing is included in the applications directory and named, strangely enough, "benchmarks".

testing/simulateall testing/applications/benchmarks

In-Hardware Testing

Ports Package

The supersmall DE3 testing framework is integrated using the University of Toronto USB Ports Package, an easy-to-use interface to the DE3 that allows physical FPGA pins to be read and written to by software running on a *nix PC.

To perform the same tests, except on actual FPGA hardware, two additional scripts have been provided: "hwtest" and "hwtestall", also contained within the testing directory. These scripts have an identical interface to the above "simulate" and "simulateall" scripts, but do require some extra setup. In particular, for these scripts to work, the following conditions must be met:

If those conditions have been met, you can try running them, and watch as the supersmall soft processor demonstrates its proficiencies on actual hardware.

Exceptions Testing

To test the supersmall's implementation of exceptions, a special test has been developed, called "exceptiontest" in the applications directory. This test, rather than comparing against a known good simulation, instead methodically checks whether each exception has fired properly, and continuously outputs 0xfadefade on failure. If this test completes successfully (i.e, outputs 0xdeaddead not 0xfadefade), you should be assured that exceptions are working properly. Obviously, if you disable exceptions in the processor, this test will fail.

Since the exceptions test is simply a specialized application, it works perfectly well with the "simulate" and "hwtest" scripts you already know about, excluding the fact that it doesn't have a software trace, due to the fact its exceptions support differs from our software simulator's. Therefore, "simulate" and "hwtest" will record every run as a failure. Don't worry about this though: as long as they don't get stuck outputting 0xfadefade, exceptions support should still be good.

testing/simulate exceptiontest

Size Testing

The stated goal of the supersmall soft processor is to be as small as possible. How small? Well, start up your version of Quartus II (which you installed back in the installation guide, right?), and open the "supersmall" project contained in the barequartus directory. Make sure the HARDWARE_DEBUG and CONTROL_DEBUG lines are commented out in the supersmall/defines.v file (if we're looking at size, we don't want to worry about all the extra debugging material), and press Ctrl-K in Quartus to run analysis and synthesis. After a brief wait, Quartus should spit out its report, telling you the resource utilization of the processor. In particular, check the ALUT and Register counts, which are going to really determine how much you can cram into an FPGA.

On a DE3, your numbers should be around 236 ALUTS/336 registers with exceptions enabled, and 142 ALUTS/168 registers with exceptions disabled. Obviously, there will probably be variation depending on your particular Quartus setup, but if you get wildly differing results, something's wrong: make sure that you don't have debugging enabled, or external memory support.

Of course, even ALUTs and registers are an abstraction of the actual FPGA logic architecture, and if you want truly precise results, you need to run a full compile and have Quartus tell you exactly how many of the FPGA's resources you're consuming. However, the supersmall isn't intended to go on an FPGA on its own, and you're unlikely to find these useful until you've actually integrated it with your design. Unless you're simply interested in hacking the supersmall by itself, for more accurate counts, integrate it with your design and then see what your results are like.

Next Steps

Now that you know the processor works, you probably want to start using it. In that case, you can read the compiler installation guide to install a mips cross-compiler, or, if you already have your own cross-compiler, go straight to the connection guide to find out how to attach this thing into your existing system.