NUMAchine ensures that deadlock will not occur by dealing with messages
that elicit responses from the target stations differently from those that
do not elicit a response; we refer to the former as nonsinkable and
the latter as sinkable. Sinkable messages include read responses,
write-backs, multicasts, and invalidation commands, while nonsinkable
messages include all types of read requests (including
interventions). To avoid deadlock, certain
functional modules in NUMAchine use separate queues to hold sinkable and
nonsinkable messages.
The following rules govern the handling of sinkable and nonsinkable messages once they have entered the network:
In addition to the deadlock avoidance scheme described above, NUMAchine has a simple flow control mechanism optimized for expected bandwidth requirements. Each ring interface has an input queue large enough to handle short term traffic bursts. Whenever the input queue is close to capacity, the operation of the ring that is feeding the buffer is temporarily halted; other rings are unaffected and continue to operate. Meanwhile, packets at the head of the full queue are processed until it is empty enough to start up the ring again. The flow control ensures that the order of sinkable requests is maintained, and it can be shown that this allows for important common case optimizations to the coherence protocol. However, it can result in poor performance under extreme conditions, such as when many processors simultaneously flush modified data from their caches to remote memory.