arch: teach ISA parser how to split code across files

This patch encompasses several interrelated and interdependent changes
to the ISA generation step.  The end goal is to reduce the size of the
generated compilation units for instruction execution and decoding so
that batch compilation can proceed with all CPUs active without
exhausting physical memory.

The ISA parser (src/arch/isa_parser.py) has been improved so that it can
accept 'split [output_type];' directives at the top level of the grammar
and 'split(output_type)' python calls within 'exec {{ ... }}' blocks.
This has the effect of "splitting" the files into smaller compilation
units.  I use air-quotes around "splitting" because the files themselves
are not split, but preprocessing directives are inserted to have the same
effect.

Architecturally, the ISA parser has had some changes in how it works.
In general, it emits code sooner.  It doesn't generate per-CPU files,
and instead defers to the C preprocessor to create the duplicate copies
for each CPU type.  Likewise there are more files emitted and the C
preprocessor does more substitution that used to be done by the ISA parser.

Finally, the build system (SCons) needs to be able to cope with a
dynamic list of source files coming out of the ISA parser. The changes
to the SCons{cript,truct} files support this. In broad strokes, the
targets requested on the command line are hidden from SCons until all
the build dependencies are determined, otherwise it would try, realize
it can't reach the goal, and terminate in failure. Since build steps
(i.e. running the ISA parser) must be taken to determine the file list,
several new build stages have been inserted at the very start of the
build. First, the build dependencies from the ISA parser will be emitted
to arch/$ISA/generated/inc.d, which is then read by a new SCons builder
to finalize the dependencies. (Once inc.d exists, the ISA parser will not
need to be run to complete this step.) Once the dependencies are known,
the 'Environments' are made by the makeEnv() function. This function used
to be called before the build began but now happens during the build.
It is easy to see that this step is quite slow; this is a known issue
and it's important to realize that it was already slow, but there was
no obvious cause to attribute it to since nothing was displayed to the
terminal. Since new steps that used to be performed serially are now in a
potentially-parallel build phase, the pathname handling in the SCons scripts
has been tightened up to deal with chdir() race conditions. In general,
pathnames are computed earlier and more likely to be stored, passed around,
and processed as absolute paths rather than relative paths.  In the end,
some of these issues had to be fixed by inserting serializing dependencies
in the build.

Minor note:
For the null ISA, we just provide a dummy inc.d so SCons is never
compelled to try to generate it. While it seems slightly wrong to have
anything in src/arch/*/generated (i.e. a non-generated 'generated' file),
it's by far the simplest solution.

15 files changed, 23 insertions, 26 deletions

diff --git a/src/arch/x86/SConscript b/src/arch/x86/SConscript
index 6ec714e8e..c35ba608a 100644
--- a/src/arch/x86/SConscript
+++ b/src/arch/x86/SConscript
@@ -312,6 +312,3 @@ if env['TARGET_ISA'] == 'x86':
         # Add in python file dependencies that won't be caught otherwise
         for pyfile in python_files:
             env.Depends(f, "isa/insts/%s" % pyfile)
-        # Only non-header files need to be compiled.
-        if not f.path.endswith('.hh'):
-            Source(f)
diff --git a/src/arch/x86/isa/formats/basic.isa b/src/arch/x86/isa/formats/basic.isa
index d8f8592d7..a4b96c43e 100644
--- a/src/arch/x86/isa/formats/basic.isa
+++ b/src/arch/x86/isa/formats/basic.isa
@@ -77,7 +77,7 @@ def template BasicConstructor {{
 
 // Basic instruction class execute method template.
 def template BasicExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
                 Trace::InstRecord *traceData) const
         {
             Fault fault = NoFault;
diff --git a/src/arch/x86/isa/formats/cpuid.isa b/src/arch/x86/isa/formats/cpuid.isa
index 535f1e925..265d0f7ef 100644
--- a/src/arch/x86/isa/formats/cpuid.isa
+++ b/src/arch/x86/isa/formats/cpuid.isa
@@ -68,7 +68,7 @@ output decoder {{
 }};
 
 def template CPUIDExecute {{
-    Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+    Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
             Trace::InstRecord *traceData) const
     {
         // If the CPUID instruction used a valid function number, this will
diff --git a/src/arch/x86/isa/formats/nop.isa b/src/arch/x86/isa/formats/nop.isa
index 42d209748..d33529faa 100644
--- a/src/arch/x86/isa/formats/nop.isa
+++ b/src/arch/x86/isa/formats/nop.isa
@@ -72,7 +72,7 @@ output decoder {{
 }};
 
 def template NopExecute {{
-    Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+    Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
             Trace::InstRecord *traceData) const
     {
         return NoFault;
diff --git a/src/arch/x86/isa/formats/syscall.isa b/src/arch/x86/isa/formats/syscall.isa
index 8b6d469b2..6888af02c 100644
--- a/src/arch/x86/isa/formats/syscall.isa
+++ b/src/arch/x86/isa/formats/syscall.isa
@@ -74,7 +74,7 @@ output decoder {{
 }};
 
 def template SyscallExecute {{
-    Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+    Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
             Trace::InstRecord *traceData) const
     {
         Fault fault = NoFault;
diff --git a/src/arch/x86/isa/formats/unimp.isa b/src/arch/x86/isa/formats/unimp.isa
index be066acc2..7849a4970 100644
--- a/src/arch/x86/isa/formats/unimp.isa
+++ b/src/arch/x86/isa/formats/unimp.isa
@@ -122,7 +122,7 @@ output decoder {{
 
 output exec {{
     Fault
-    FailUnimplemented::execute(%(CPU_exec_context)s *xc,
+    FailUnimplemented::execute(CPU_EXEC_CONTEXT *xc,
                                Trace::InstRecord *traceData) const
     {
         panic("attempt to execute unimplemented instruction '%s' %s",
@@ -131,7 +131,7 @@ output exec {{
     }
 
     Fault
-    WarnUnimplemented::execute(%(CPU_exec_context)s *xc,
+    WarnUnimplemented::execute(CPU_EXEC_CONTEXT *xc,
                                Trace::InstRecord *traceData) const
     {
         if (!warned) {
diff --git a/src/arch/x86/isa/formats/unknown.isa b/src/arch/x86/isa/formats/unknown.isa
index 1108fd4a4..5911d5c00 100644
--- a/src/arch/x86/isa/formats/unknown.isa
+++ b/src/arch/x86/isa/formats/unknown.isa
@@ -74,7 +74,7 @@ output decoder {{
 }};
 
 output exec {{
-        Fault Unknown::execute(%(CPU_exec_context)s *xc,
+        Fault Unknown::execute(CPU_EXEC_CONTEXT *xc,
                 Trace::InstRecord *traceData) const
         {
             return new InvalidOpcode();
diff --git a/src/arch/x86/isa/microops/debug.isa b/src/arch/x86/isa/microops/debug.isa
index b9da9c602..650c8a5a3 100644
--- a/src/arch/x86/isa/microops/debug.isa
+++ b/src/arch/x86/isa/microops/debug.isa
@@ -84,7 +84,7 @@ def template MicroDebugDeclare {{
 
 def template MicroDebugExecute {{
         Fault
-        %(class_name)s::execute(%(CPU_exec_context)s *xc,
+        %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
                 Trace::InstRecord *traceData) const
         {
             %(op_decl)s
diff --git a/src/arch/x86/isa/microops/fpop.isa b/src/arch/x86/isa/microops/fpop.isa
index 131284076..4155fe3aa 100644
--- a/src/arch/x86/isa/microops/fpop.isa
+++ b/src/arch/x86/isa/microops/fpop.isa
@@ -44,7 +44,7 @@
 //////////////////////////////////////////////////////////////////////////
 
 def template MicroFpOpExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
                 Trace::InstRecord *traceData) const
         {
             Fault fault = NoFault;
diff --git a/src/arch/x86/isa/microops/ldstop.isa b/src/arch/x86/isa/microops/ldstop.isa
index b82cf57e0..c26e9932d 100644
--- a/src/arch/x86/isa/microops/ldstop.isa
+++ b/src/arch/x86/isa/microops/ldstop.isa
@@ -47,7 +47,7 @@
 // LEA template
 
 def template MicroLeaExecute {{
-    Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+    Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
           Trace::InstRecord *traceData) const
     {
         Fault fault = NoFault;
@@ -87,7 +87,7 @@ def template MicroLeaDeclare {{
 // Load templates
 
 def template MicroLoadExecute {{
-    Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+    Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
           Trace::InstRecord *traceData) const
     {
         Fault fault = NoFault;
@@ -116,7 +116,7 @@ def template MicroLoadExecute {{
 }};
 
 def template MicroLoadInitiateAcc {{
-    Fault %(class_name)s::initiateAcc(%(CPU_exec_context)s * xc,
+    Fault %(class_name)s::initiateAcc(CPU_EXEC_CONTEXT * xc,
             Trace::InstRecord * traceData) const
     {
         Fault fault = NoFault;
@@ -135,7 +135,7 @@ def template MicroLoadInitiateAcc {{
 
 def template MicroLoadCompleteAcc {{
     Fault %(class_name)s::completeAcc(PacketPtr pkt,
-            %(CPU_exec_context)s * xc,
+            CPU_EXEC_CONTEXT * xc,
             Trace::InstRecord * traceData) const
     {
         Fault fault = NoFault;
@@ -159,7 +159,7 @@ def template MicroLoadCompleteAcc {{
 // Store templates
 
 def template MicroStoreExecute {{
-    Fault %(class_name)s::execute(%(CPU_exec_context)s * xc,
+    Fault %(class_name)s::execute(CPU_EXEC_CONTEXT * xc,
             Trace::InstRecord *traceData) const
     {
         Fault fault = NoFault;
@@ -187,7 +187,7 @@ def template MicroStoreExecute {{
 }};
 
 def template MicroStoreInitiateAcc {{
-    Fault %(class_name)s::initiateAcc(%(CPU_exec_context)s * xc,
+    Fault %(class_name)s::initiateAcc(CPU_EXEC_CONTEXT * xc,
             Trace::InstRecord * traceData) const
     {
         Fault fault = NoFault;
@@ -211,7 +211,7 @@ def template MicroStoreInitiateAcc {{
 
 def template MicroStoreCompleteAcc {{
     Fault %(class_name)s::completeAcc(PacketPtr pkt,
-            %(CPU_exec_context)s * xc, Trace::InstRecord * traceData) const
+            CPU_EXEC_CONTEXT * xc, Trace::InstRecord * traceData) const
     {
         %(op_decl)s;
         %(op_rd)s;
diff --git a/src/arch/x86/isa/microops/limmop.isa b/src/arch/x86/isa/microops/limmop.isa
index e7cd548ec..cd282f67a 100644
--- a/src/arch/x86/isa/microops/limmop.isa
+++ b/src/arch/x86/isa/microops/limmop.isa
@@ -42,7 +42,7 @@
 //////////////////////////////////////////////////////////////////////////
 
 def template MicroLimmOpExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
                 Trace::InstRecord *traceData) const
         {
             %(op_decl)s;
diff --git a/src/arch/x86/isa/microops/mediaop.isa b/src/arch/x86/isa/microops/mediaop.isa
index fecdab863..e382151ef 100644
--- a/src/arch/x86/isa/microops/mediaop.isa
+++ b/src/arch/x86/isa/microops/mediaop.isa
@@ -27,7 +27,7 @@
 // Authors: Gabe Black
 
 def template MediaOpExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
                 Trace::InstRecord *traceData) const
         {
             Fault fault = NoFault;
diff --git a/src/arch/x86/isa/microops/regop.isa b/src/arch/x86/isa/microops/regop.isa
index b957a164a..759bffc3c 100644
--- a/src/arch/x86/isa/microops/regop.isa
+++ b/src/arch/x86/isa/microops/regop.isa
@@ -42,7 +42,7 @@
 //////////////////////////////////////////////////////////////////////////
 
 def template MicroRegOpExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
                 Trace::InstRecord *traceData) const
         {
             Fault fault = NoFault;
@@ -73,7 +73,7 @@ def template MicroRegOpExecute {{
 }};
 
 def template MicroRegOpImmExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
                 Trace::InstRecord *traceData) const
         {
             Fault fault = NoFault;
diff --git a/src/arch/x86/isa/microops/seqop.isa b/src/arch/x86/isa/microops/seqop.isa
index 72c28b1fe..76766e055 100644
--- a/src/arch/x86/isa/microops/seqop.isa
+++ b/src/arch/x86/isa/microops/seqop.isa
@@ -68,7 +68,7 @@ def template SeqOpDeclare {{
 }};
 
 def template SeqOpExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
                 Trace::InstRecord *traceData) const
         {
             %(op_decl)s;
diff --git a/src/arch/x86/isa/microops/specop.isa b/src/arch/x86/isa/microops/specop.isa
index a13250589..854d3910f 100644
--- a/src/arch/x86/isa/microops/specop.isa
+++ b/src/arch/x86/isa/microops/specop.isa
@@ -87,7 +87,7 @@ def template MicroFaultDeclare {{
 }};
 
 def template MicroFaultExecute {{
-        Fault %(class_name)s::execute(%(CPU_exec_context)s *xc,
+        Fault %(class_name)s::execute(CPU_EXEC_CONTEXT *xc,
                 Trace::InstRecord *traceData) const
         {
             %(op_decl)s;
@@ -103,7 +103,7 @@ def template MicroFaultExecute {{
 
 output exec {{
     Fault
-    MicroHalt::execute(%(CPU_exec_context)s *xc,
+    MicroHalt::execute(CPU_EXEC_CONTEXT *xc,
             Trace::InstRecord * traceData) const
     {
         xc->tcBase()->suspend();