mem: Delay responses in the crossbar before forwarding

This patch changes how the crossbar classes deal with
responses. Instead of forwarding responses directly and burdening the
neighbouring modules in paying for the latency (through the
pkt->headerDelay), we now queue them before sending them.

The coherency protocol is not affected as requests and any snoop
requests/responses are still passed on in zero time. Thus, the
responses end up paying for any header delay accumulated when passing
through the crossbar. Any latency incurred on the request path will be
paid for on the response side, if no other module has dealt with it.

As a result of this patch, responses are returned at a later
point. This affects the number of outstanding transactions, and quite
a few regressions see an impact in blocking due to no MSHRs, increased
cache-miss latencies, etc.

Going forward we should be able to use the same concept also for snoop
responses, and any request that is not an express snoop.

1 files changed, 2 insertions, 1 deletions

diff --git a/src/mem/xbar.hh b/src/mem/xbar.hh
index ed678d9d0..ede60e666 100644
--- a/src/mem/xbar.hh
+++ b/src/mem/xbar.hh
@@ -57,6 +57,7 @@
 #include "base/hashmap.hh"
 #include "base/types.hh"
 #include "mem/mem_object.hh"
+#include "mem/qport.hh"
 #include "params/BaseXBar.hh"
 #include "sim/stats.hh"
 
@@ -427,7 +428,7 @@ class BaseXBar : public MemObject
     bool gotAllAddrRanges;
 
     /** The master and slave ports of the crossbar */
-    std::vector<SlavePort*> slavePorts;
+    std::vector<QueuedSlavePort*> slavePorts;
     std::vector<MasterPort*> masterPorts;
 
     /** Port that handles requests that don't match any of the interfaces.*/