syscall_emul: [patch 13/22] add system call retry capability

This changeset adds functionality that allows system calls to retry without
affecting thread context state such as the program counter or register values
for the associated thread context (when system calls return with a retry
fault).

This functionality is needed to solve problems with blocking system calls
in multi-process or multi-threaded simulations where information is passed
between processes/threads. Blocking system calls can cause deadlock because
the simulator itself is single threaded. There is only a single thread
servicing the event queue which can cause deadlock if the thread hits a
blocking system call instruction.

To illustrate the problem, consider two processes using the producer/consumer
sharing model. The processes can use file descriptors and the read and write
calls to pass information to one another. If the consumer calls the blocking
read system call before the producer has produced anything, the call will
block the event queue (while executing the system call instruction) and
deadlock the simulation.

The solution implemented in this changeset is to recognize that the system
calls will block and then generate a special retry fault. The fault will
be sent back up through the function call chain until it is exposed to the
cpu model's pipeline where the fault becomes visible. The fault will trigger
the cpu model to replay the instruction at a future tick where the call has
a chance to succeed without actually going into a blocking state.

In subsequent patches, we recognize that a syscall will block by calling a
non-blocking poll (from inside the system call implementation) and checking
for events. When events show up during the poll, it signifies that the call
would not have blocked and the syscall is allowed to proceed (calling an
underlying host system call if necessary). If no events are returned from the
poll, we generate the fault and try the instruction for the thread context
at a distant tick. Note that retrying every tick is not efficient.

As an aside, the simulator has some multi-threading support for the event
queue, but it is not used by default and needs work. Even if the event queue
was completely multi-threaded, meaning that there is a hardware thread on
the host servicing a single simulator thread contexts with a 1:1 mapping
between them, it's still possible to run into deadlock due to the event queue
barriers on quantum boundaries. The solution of replaying at a later tick
is the simplest solution and solves the problem generally.

5 files changed, 15 insertions, 10 deletions

diff --git a/src/arch/x86/isa/decoder/one_byte_opcodes.isa b/src/arch/x86/isa/decoder/one_byte_opcodes.isa
index 859d1f1b4..95bc7a5c1 100644
--- a/src/arch/x86/isa/decoder/one_byte_opcodes.isa
+++ b/src/arch/x86/isa/decoder/one_byte_opcodes.isa
@@ -400,8 +400,9 @@
                         // will sign extend it, and there's no easy way to
                         // specify only checking the first byte.
                         0xffffffffffffff80:
-                            SyscallInst::int80('xc->syscall(Rax)',
-                                 IsSyscall, IsNonSpeculative, IsSerializeAfter);
+                            SyscallInst::int80('xc->syscall(Rax, &fault)',
+                                               IsSyscall, IsNonSpeculative,
+                                               IsSerializeAfter);
                     }
 
                     default: Inst::INT(Ib);
diff --git a/src/arch/x86/isa/decoder/two_byte_opcodes.isa b/src/arch/x86/isa/decoder/two_byte_opcodes.isa
index 772177d42..97c3dd688 100644
--- a/src/arch/x86/isa/decoder/two_byte_opcodes.isa
+++ b/src/arch/x86/isa/decoder/two_byte_opcodes.isa
@@ -235,8 +235,9 @@
                 }
             }
             0x05: decode FullSystemInt {
-                0: SyscallInst::syscall('xc->syscall(Rax)',
-                        IsSyscall, IsNonSpeculative, IsSerializeAfter);
+                0: SyscallInst::syscall('xc->syscall(Rax, &fault)',
+                                        IsSyscall, IsNonSpeculative,
+                                        IsSerializeAfter);
                 default: decode MODE_MODE {
                     0x0: decode MODE_SUBMODE {
                         0x0: Inst::SYSCALL_64();
@@ -422,8 +423,9 @@
             0x2: Inst::RDMSR();
             0x3: rdpmc();
             0x4: decode FullSystemInt {
-                0: SyscallInst::sysenter('xc->syscall(Rax)',
-                        IsSyscall, IsNonSpeculative, IsSerializeAfter);
+                0: SyscallInst::sysenter('xc->syscall(Rax, &fault)',
+                                         IsSyscall, IsNonSpeculative,
+                                         IsSerializeAfter);
                 default: sysenter();
             }
             0x5: sysexit();
diff --git a/src/arch/x86/process.cc b/src/arch/x86/process.cc
index dfbd41e4e..c1e4f710a 100644
--- a/src/arch/x86/process.cc
+++ b/src/arch/x86/process.cc
@@ -134,7 +134,7 @@ X86_64Process::X86_64Process(ProcessParams *params, ObjectFile *objFile,
 }
 
 void
-I386Process::syscall(int64_t callnum, ThreadContext *tc)
+I386Process::syscall(int64_t callnum, ThreadContext *tc, Fault *fault)
 {
     TheISA::PCState pc = tc->pcState();
     Addr eip = pc.pc();
@@ -143,7 +143,7 @@ I386Process::syscall(int64_t callnum, ThreadContext *tc)
         pc.npc(vsyscallPage.base + vsyscallPage.vsysexitOffset);
         tc->pcState(pc);
     }
-    X86Process::syscall(callnum, tc);
+    X86Process::syscall(callnum, tc, fault);
 }
 
 
diff --git a/src/arch/x86/process.hh b/src/arch/x86/process.hh
index fa95b4ff4..ef0329329 100644
--- a/src/arch/x86/process.hh
+++ b/src/arch/x86/process.hh
@@ -130,7 +130,7 @@ namespace X86ISA
         void argsInit(int intSize, int pageSize);
         void initState();
 
-        void syscall(int64_t callnum, ThreadContext *tc);
+        void syscall(int64_t callnum, ThreadContext *tc, Fault *fault);
         X86ISA::IntReg getSyscallArg(ThreadContext *tc, int &i);
         X86ISA::IntReg getSyscallArg(ThreadContext *tc, int &i, int width);
         void setSyscallArg(ThreadContext *tc, int i, X86ISA::IntReg val);
diff --git a/src/arch/x86/pseudo_inst.cc b/src/arch/x86/pseudo_inst.cc
index acf310631..c0ec11059 100644
--- a/src/arch/x86/pseudo_inst.cc
+++ b/src/arch/x86/pseudo_inst.cc
@@ -49,7 +49,9 @@ m5Syscall(ThreadContext *tc)
 {
     DPRINTF(PseudoInst, "PseudoInst::m5Syscall()\n");
 
-    tc->syscall(tc->readIntReg(INTREG_RAX));
+    Fault fault;
+    tc->syscall(tc->readIntReg(INTREG_RAX), &fault);
+
     MiscReg rflags = tc->readMiscReg(MISCREG_RFLAGS);
     rflags &= ~(1 << 16);
     tc->setMiscReg(MISCREG_RFLAGS, rflags);