arch: teach ISA parser how to split code across files

This patch encompasses several interrelated and interdependent changes
to the ISA generation step.  The end goal is to reduce the size of the
generated compilation units for instruction execution and decoding so
that batch compilation can proceed with all CPUs active without
exhausting physical memory.

The ISA parser (src/arch/isa_parser.py) has been improved so that it can
accept 'split [output_type];' directives at the top level of the grammar
and 'split(output_type)' python calls within 'exec {{ ... }}' blocks.
This has the effect of "splitting" the files into smaller compilation
units.  I use air-quotes around "splitting" because the files themselves
are not split, but preprocessing directives are inserted to have the same
effect.

Architecturally, the ISA parser has had some changes in how it works.
In general, it emits code sooner.  It doesn't generate per-CPU files,
and instead defers to the C preprocessor to create the duplicate copies
for each CPU type.  Likewise there are more files emitted and the C
preprocessor does more substitution that used to be done by the ISA parser.

Finally, the build system (SCons) needs to be able to cope with a
dynamic list of source files coming out of the ISA parser. The changes
to the SCons{cript,truct} files support this. In broad strokes, the
targets requested on the command line are hidden from SCons until all
the build dependencies are determined, otherwise it would try, realize
it can't reach the goal, and terminate in failure. Since build steps
(i.e. running the ISA parser) must be taken to determine the file list,
several new build stages have been inserted at the very start of the
build. First, the build dependencies from the ISA parser will be emitted
to arch/$ISA/generated/inc.d, which is then read by a new SCons builder
to finalize the dependencies. (Once inc.d exists, the ISA parser will not
need to be run to complete this step.) Once the dependencies are known,
the 'Environments' are made by the makeEnv() function. This function used
to be called before the build began but now happens during the build.
It is easy to see that this step is quite slow; this is a known issue
and it's important to realize that it was already slow, but there was
no obvious cause to attribute it to since nothing was displayed to the
terminal. Since new steps that used to be performed serially are now in a
potentially-parallel build phase, the pathname handling in the SCons scripts
has been tightened up to deal with chdir() race conditions. In general,
pathnames are computed earlier and more likely to be stored, passed around,
and processed as absolute paths rather than relative paths.  In the end,
some of these issues had to be fixed by inserting serializing dependencies
in the build.

Minor note:
For the null ISA, we just provide a dummy inc.d so SCons is never
compelled to try to generate it. While it seems slightly wrong to have
anything in src/arch/*/generated (i.e. a non-generated 'generated' file),
it's by far the simplest solution.

13 files changed, 47 insertions, 52 deletions

diff --git a/src/arch/mips/SConscript b/src/arch/mips/SConscript
index 944fc8e55..2bc7eca99 100644
--- a/src/arch/mips/SConscript
+++ b/src/arch/mips/SConscript
@@ -59,9 +59,4 @@ if env['TARGET_ISA'] == 'mips':
 
     DebugFlag('MipsPRA')
 
-    # Add in files generated by the ISA description.
-    isa_desc_files = env.ISADesc('isa/main.isa')
-    # Only non-header files need to be compiled.
-    for f in isa_desc_files:
-        if not f.path.endswith('.hh'):
-            Source(f)
+    env.ISADesc('isa/main.isa')
diff --git a/src/arch/mips/isa/formats/basic.isa b/src/arch/mips/isa/formats/basic.isa
index 46c48548c..98da450d8 100644
--- a/src/arch/mips/isa/formats/basic.isa
+++ b/src/arch/mips/isa/formats/basic.isa
@@ -61,7 +61,7 @@ def template BasicConstructor {{
 
 // Basic instruction class execute method template.
 def template BasicExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc, Trace::InstRecord *traceData) const
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc, Trace::InstRecord *traceData) const
         {
                 Fault fault = NoFault;
 
diff --git a/src/arch/mips/isa/formats/control.isa b/src/arch/mips/isa/formats/control.isa
index 1480a5589..c9ef6707f 100644
--- a/src/arch/mips/isa/formats/control.isa
+++ b/src/arch/mips/isa/formats/control.isa
@@ -80,7 +80,7 @@ output header {{
 
 // Basic instruction class execute method template.
 def template CP0Execute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc, Trace::InstRecord *traceData) const
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc, Trace::InstRecord *traceData) const
         {
                 Fault fault = NoFault;
                 %(op_decl)s;
@@ -101,7 +101,7 @@ def template CP0Execute {{
 }};
 
 def template CP1Execute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc, Trace::InstRecord *traceData) const
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc, Trace::InstRecord *traceData) const
         {
                 Fault fault = NoFault;
                 %(op_decl)s;
@@ -122,7 +122,7 @@ def template CP1Execute {{
 }};
 // Basic instruction class execute method template.
 def template ControlTLBExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc, Trace::InstRecord *traceData) const
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc, Trace::InstRecord *traceData) const
         {
             Fault fault = NoFault;
             %(op_decl)s;
@@ -181,7 +181,7 @@ output header {{
 
 output exec {{
         bool
-        isCoprocessorEnabled(%(CPU_exec_context)s *xc, unsigned cop_num)
+        isCoprocessorEnabled(CPU_EXEC_CONTEXT *xc, unsigned cop_num)
         {
             if (!FullSystem)
                 return true;
@@ -203,7 +203,7 @@ output exec {{
         }
 
         bool inline
-        isCoprocessor0Enabled(%(CPU_exec_context)s *xc)
+        isCoprocessor0Enabled(CPU_EXEC_CONTEXT *xc)
         {
             if (FullSystem) {
                 MiscReg Stat = xc->readMiscReg(MISCREG_STATUS);
@@ -219,7 +219,7 @@ output exec {{
         }
 
         bool
-        isMMUTLB(%(CPU_exec_context)s *xc)
+        isMMUTLB(CPU_EXEC_CONTEXT *xc)
         {
             MiscReg Config = xc->readMiscReg(MISCREG_CONFIG);
             return FullSystem && (Config & 0x380) == 0x80;
diff --git a/src/arch/mips/isa/formats/dsp.isa b/src/arch/mips/isa/formats/dsp.isa
index 9dfae3f44..39232bfe0 100755
--- a/src/arch/mips/isa/formats/dsp.isa
+++ b/src/arch/mips/isa/formats/dsp.isa
@@ -64,7 +64,7 @@ output header {{
 
 // Dsp instruction class execute method template.
 def template DspExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc, Trace::InstRecord *traceData) const
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc, Trace::InstRecord *traceData) const
         {
                 Fault fault = NoFault;
 
@@ -97,7 +97,7 @@ def template DspExecute {{
 
 // DspHiLo instruction class execute method template.
 def template DspHiLoExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc, Trace::InstRecord *traceData) const
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc, Trace::InstRecord *traceData) const
         {
                 Fault fault = NoFault;
 
@@ -147,7 +147,7 @@ output decoder {{
 
 output exec {{
     bool
-    isDspEnabled(%(CPU_exec_context)s *xc)
+    isDspEnabled(CPU_EXEC_CONTEXT *xc)
     {
         return !FullSystem || bits(xc->readMiscReg(MISCREG_STATUS), 24);
     }
@@ -155,7 +155,7 @@ output exec {{
 
 output exec {{
     bool
-    isDspPresent(%(CPU_exec_context)s *xc)
+    isDspPresent(CPU_EXEC_CONTEXT *xc)
     {
         return !FullSystem || bits(xc->readMiscReg(MISCREG_CONFIG3), 10);
     }
diff --git a/src/arch/mips/isa/formats/fp.isa b/src/arch/mips/isa/formats/fp.isa
index 1b061bc63..731c6c06a 100644
--- a/src/arch/mips/isa/formats/fp.isa
+++ b/src/arch/mips/isa/formats/fp.isa
@@ -92,7 +92,7 @@ output header {{
 }};
 
 output exec {{
-        inline Fault checkFpEnableFault(%(CPU_exec_context)s *xc)
+        inline Fault checkFpEnableFault(CPU_EXEC_CONTEXT *xc)
         {
             //@TODO: Implement correct CP0 checks to see if the CP1
             // unit is enable or not
@@ -105,7 +105,7 @@ output exec {{
         //If any operand is Nan return the appropriate QNaN
         template <class T>
         bool
-        fpNanOperands(FPOp *inst, %(CPU_exec_context)s *xc, const T &src_type,
+        fpNanOperands(FPOp *inst, CPU_EXEC_CONTEXT *xc, const T &src_type,
                       Trace::InstRecord *traceData)
         {
             uint64_t mips_nan = 0;
@@ -126,7 +126,7 @@ output exec {{
 
         template <class T>
         bool
-        fpInvalidOp(FPOp *inst, %(CPU_exec_context)s *cpu, const T dest_val,
+        fpInvalidOp(FPOp *inst, CPU_EXEC_CONTEXT *cpu, const T dest_val,
                     Trace::InstRecord *traceData)
         {
             uint64_t mips_nan = 0;
@@ -156,7 +156,7 @@ output exec {{
         }
 
         void
-        fpResetCauseBits(%(CPU_exec_context)s *cpu)
+        fpResetCauseBits(CPU_EXEC_CONTEXT *cpu)
         {
             //Read FCSR from FloatRegFile
             uint32_t fcsr = cpu->tcBase()->readFloatRegBits(FLOATREG_FCSR);
@@ -170,7 +170,7 @@ output exec {{
 }};
 
 def template FloatingPointExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc, Trace::InstRecord *traceData) const
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc, Trace::InstRecord *traceData) const
         {
                 Fault fault = NoFault;
 
diff --git a/src/arch/mips/isa/formats/int.isa b/src/arch/mips/isa/formats/int.isa
index 14dcc7259..42a1abfe6 100644
--- a/src/arch/mips/isa/formats/int.isa
+++ b/src/arch/mips/isa/formats/int.isa
@@ -133,7 +133,7 @@ output header {{
 
 // HiLo instruction class execute method template.
 def template HiLoExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc, Trace::InstRecord *traceData) const
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc, Trace::InstRecord *traceData) const
         {
                 Fault fault = NoFault;
 
@@ -152,7 +152,7 @@ def template HiLoExecute {{
 
 // HiLoRsSel instruction class execute method template.
 def template HiLoRsSelExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc, Trace::InstRecord *traceData) const
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc, Trace::InstRecord *traceData) const
         {
                 Fault fault = NoFault;
 
@@ -178,7 +178,7 @@ def template HiLoRsSelExecute {{
 
 // HiLoRdSel instruction class execute method template.
 def template HiLoRdSelExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc, Trace::InstRecord *traceData) const
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc, Trace::InstRecord *traceData) const
         {
                 Fault fault = NoFault;
 
diff --git a/src/arch/mips/isa/formats/mem.isa b/src/arch/mips/isa/formats/mem.isa
index 3e7a8de5a..80107be8b 100644
--- a/src/arch/mips/isa/formats/mem.isa
+++ b/src/arch/mips/isa/formats/mem.isa
@@ -105,7 +105,7 @@ output exec {{
     /** return data in cases where there the size of data is only
         known in the packet
     */
-    uint64_t getMemData(%(CPU_exec_context)s *xc, Packet *packet) {
+    uint64_t getMemData(CPU_EXEC_CONTEXT *xc, Packet *packet) {
         switch (packet->getSize())
         {
           case 1:
@@ -176,7 +176,7 @@ def template LoadStoreConstructor {{
 
 def template EACompExecute {{
     Fault
-    %(class_name)s::eaComp(%(CPU_exec_context)s *xc,
+    %(class_name)s::eaComp(CPU_EXEC_CONTEXT *xc,
                                    Trace::InstRecord *traceData) const
     {
         Addr EA;
@@ -203,7 +203,7 @@ def template EACompExecute {{
 }};
 
 def template LoadExecute {{
-    Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+    Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
                                   Trace::InstRecord *traceData) const
     {
         Addr EA;
@@ -235,7 +235,7 @@ def template LoadExecute {{
 
 
 def template LoadInitiateAcc {{
-    Fault %(class_name)s::initiateAcc(%(CPU_exec_context)s *xc,
+    Fault %(class_name)s::initiateAcc(CPU_EXEC_CONTEXT *xc,
                                       Trace::InstRecord *traceData) const
     {
         Addr EA;
@@ -262,7 +262,7 @@ def template LoadInitiateAcc {{
 
 def template LoadCompleteAcc {{
     Fault %(class_name)s::completeAcc(Packet *pkt,
-                                      %(CPU_exec_context)s *xc,
+                                      CPU_EXEC_CONTEXT *xc,
                                       Trace::InstRecord *traceData) const
     {
         Fault fault = NoFault;
@@ -292,7 +292,7 @@ def template LoadCompleteAcc {{
 }};
 
 def template StoreExecute {{
-    Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+    Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
                                   Trace::InstRecord *traceData) const
     {
         Addr EA;
@@ -326,7 +326,7 @@ def template StoreExecute {{
 
 
 def template StoreFPExecute {{
-    Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+    Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
                                   Trace::InstRecord *traceData) const
     {
         Addr EA;
@@ -361,7 +361,7 @@ def template StoreFPExecute {{
 }};
 
 def template StoreCondExecute {{
-    Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+    Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
                                   Trace::InstRecord *traceData) const
     {
         Addr EA;
@@ -395,7 +395,7 @@ def template StoreCondExecute {{
 }};
 
 def template StoreInitiateAcc {{
-    Fault %(class_name)s::initiateAcc(%(CPU_exec_context)s *xc,
+    Fault %(class_name)s::initiateAcc(CPU_EXEC_CONTEXT *xc,
                                       Trace::InstRecord *traceData) const
     {
         Addr EA;
@@ -422,7 +422,7 @@ def template StoreInitiateAcc {{
 
 def template StoreCompleteAcc {{
     Fault %(class_name)s::completeAcc(Packet *pkt,
-                                      %(CPU_exec_context)s *xc,
+                                      CPU_EXEC_CONTEXT *xc,
                                       Trace::InstRecord *traceData) const
     {
         return NoFault;
@@ -431,7 +431,7 @@ def template StoreCompleteAcc {{
 
 def template StoreCondCompleteAcc {{
     Fault %(class_name)s::completeAcc(Packet *pkt,
-                                      %(CPU_exec_context)s *xc,
+                                      CPU_EXEC_CONTEXT *xc,
                                       Trace::InstRecord *traceData) const
     {
         Fault fault = NoFault;
@@ -454,7 +454,7 @@ def template StoreCondCompleteAcc {{
 }};
 
 def template MiscExecute {{
-    Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+    Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
                                   Trace::InstRecord *traceData) const
     {
         Addr EA M5_VAR_USED = 0;
@@ -474,7 +474,7 @@ def template MiscExecute {{
 }};
 
 def template MiscInitiateAcc {{
-    Fault %(class_name)s::initiateAcc(%(CPU_exec_context)s *xc,
+    Fault %(class_name)s::initiateAcc(CPU_EXEC_CONTEXT *xc,
                                       Trace::InstRecord *traceData) const
     {
         panic("Misc instruction does not support split access method!");
@@ -485,7 +485,7 @@ def template MiscInitiateAcc {{
 
 def template MiscCompleteAcc {{
     Fault %(class_name)s::completeAcc(Packet *pkt,
-                                      %(CPU_exec_context)s *xc,
+                                      CPU_EXEC_CONTEXT *xc,
                                       Trace::InstRecord *traceData) const
     {
         panic("Misc instruction does not support split access method!");
diff --git a/src/arch/mips/isa/formats/mt.isa b/src/arch/mips/isa/formats/mt.isa
index 74163eebf..f3369edc0 100644
--- a/src/arch/mips/isa/formats/mt.isa
+++ b/src/arch/mips/isa/formats/mt.isa
@@ -96,7 +96,7 @@ output header {{
 }};
 
 output exec {{
-    void getThrRegExValues(%(CPU_exec_context)s *xc,
+    void getThrRegExValues(CPU_EXEC_CONTEXT *xc,
             VPEConf0Reg &vpe_conf0, TCBindReg &tc_bind_mt,
             TCBindReg &tc_bind, VPEControlReg &vpe_control,
             MVPConf0Reg &mvp_conf0)
@@ -108,14 +108,14 @@ output exec {{
         mvp_conf0 = xc->readMiscReg(MISCREG_MVP_CONF0);
     }
 
-    void getMTExValues(%(CPU_exec_context)s *xc, Config3Reg &config3)
+    void getMTExValues(CPU_EXEC_CONTEXT *xc, Config3Reg &config3)
     {
         config3 = xc->readMiscReg(MISCREG_CONFIG3);
     }
 }};
 
 def template ThreadRegisterExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc, Trace::InstRecord *traceData) const
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc, Trace::InstRecord *traceData) const
         {
             Fault fault = NoFault;
             int64_t data M5_VAR_USED;
@@ -153,7 +153,7 @@ def template ThreadRegisterExecute {{
 }};
 
 def template MTExecute{{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc, Trace::InstRecord *traceData) const
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc, Trace::InstRecord *traceData) const
         {
                 Fault fault = NoFault;
                 %(op_decl)s;
diff --git a/src/arch/mips/isa/formats/noop.isa b/src/arch/mips/isa/formats/noop.isa
index 3aea5a9c6..5964b0f0a 100644
--- a/src/arch/mips/isa/formats/noop.isa
+++ b/src/arch/mips/isa/formats/noop.isa
@@ -82,7 +82,7 @@ output decoder {{
 
 output exec {{
     Fault
-    Nop::execute(%(CPU_exec_context)s *, Trace::InstRecord *) const
+    Nop::execute(CPU_EXEC_CONTEXT *, Trace::InstRecord *) const
     {
         return NoFault;
     }
diff --git a/src/arch/mips/isa/formats/tlbop.isa b/src/arch/mips/isa/formats/tlbop.isa
index 368037955..bd27a347c 100644
--- a/src/arch/mips/isa/formats/tlbop.isa
+++ b/src/arch/mips/isa/formats/tlbop.isa
@@ -58,7 +58,7 @@ output decoder {{
 }};
 
 def template TlbOpExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc, Trace::InstRecord *traceData) const
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc, Trace::InstRecord *traceData) const
         {
                 //Write the resulting state to the execution context
                 %(op_wb)s;
diff --git a/src/arch/mips/isa/formats/trap.isa b/src/arch/mips/isa/formats/trap.isa
index 7ebe121aa..6f8275687 100644
--- a/src/arch/mips/isa/formats/trap.isa
+++ b/src/arch/mips/isa/formats/trap.isa
@@ -81,7 +81,7 @@ output decoder {{
 
 def template TrapExecute {{
         //Edit This Template When Traps Are Implemented
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc, Trace::InstRecord *traceData) const
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc, Trace::InstRecord *traceData) const
         {
                 //Write the resulting state to the execution context
                 %(op_wb)s;
diff --git a/src/arch/mips/isa/formats/unimp.isa b/src/arch/mips/isa/formats/unimp.isa
index d567a113f..b5bcb6e5a 100644
--- a/src/arch/mips/isa/formats/unimp.isa
+++ b/src/arch/mips/isa/formats/unimp.isa
@@ -180,7 +180,7 @@ output decoder {{
 
 output exec {{
     Fault
-    FailUnimplemented::execute(%(CPU_exec_context)s *xc,
+    FailUnimplemented::execute(CPU_EXEC_CONTEXT *xc,
                                Trace::InstRecord *traceData) const
     {
         panic("attempt to execute unimplemented instruction '%s' "
@@ -190,7 +190,7 @@ output exec {{
     }
 
     Fault
-    CP0Unimplemented::execute(%(CPU_exec_context)s *xc,
+    CP0Unimplemented::execute(CPU_EXEC_CONTEXT *xc,
                                Trace::InstRecord *traceData) const
     {
         if (FullSystem) {
@@ -207,7 +207,7 @@ output exec {{
     }
 
     Fault
-    CP1Unimplemented::execute(%(CPU_exec_context)s *xc,
+    CP1Unimplemented::execute(CPU_EXEC_CONTEXT *xc,
                                Trace::InstRecord *traceData) const
     {
         if (FullSystem) {
@@ -224,7 +224,7 @@ output exec {{
     }
 
     Fault
-    CP2Unimplemented::execute(%(CPU_exec_context)s *xc,
+    CP2Unimplemented::execute(CPU_EXEC_CONTEXT *xc,
                                Trace::InstRecord *traceData) const
     {
         if (FullSystem) {
@@ -241,7 +241,7 @@ output exec {{
     }
 
     Fault
-    WarnUnimplemented::execute(%(CPU_exec_context)s *xc,
+    WarnUnimplemented::execute(CPU_EXEC_CONTEXT *xc,
                                Trace::InstRecord *traceData) const
     {
         if (!warned) {
diff --git a/src/arch/mips/isa/formats/unknown.isa b/src/arch/mips/isa/formats/unknown.isa
index e4037477f..ba8fc5c07 100644
--- a/src/arch/mips/isa/formats/unknown.isa
+++ b/src/arch/mips/isa/formats/unknown.isa
@@ -69,7 +69,7 @@ output decoder {{
 
 output exec {{
     Fault
-    Unknown::execute(%(CPU_exec_context)s *xc,
+    Unknown::execute(CPU_EXEC_CONTEXT *xc,
                      Trace::InstRecord *traceData) const
     {
         return new ReservedInstructionFault;