Update to ply 2.3

ext/ply/ply/lex.py:
ext/ply/ply/yacc.py:
ext/ply/CHANGES:
ext/ply/README:
ext/ply/TODO:
ext/ply/doc/ply.html:
ext/ply/example/ansic/clex.py:
ext/ply/example/ansic/cparse.py:
ext/ply/example/calc/calc.py:
ext/ply/example/hedit/hedit.py:
ext/ply/example/optcalc/calc.py:
ext/ply/test/README:
ext/ply/test/calclex.py:
ext/ply/test/lex_doc1.exp:
ext/ply/test/lex_doc1.py:
ext/ply/test/lex_dup1.exp:
ext/ply/test/lex_dup1.py:
ext/ply/test/lex_dup2.exp:
ext/ply/test/lex_dup2.py:
ext/ply/test/lex_dup3.exp:
ext/ply/test/lex_dup3.py:
ext/ply/test/lex_empty.py:
ext/ply/test/lex_error1.py:
ext/ply/test/lex_error2.py:
ext/ply/test/lex_error3.exp:
ext/ply/test/lex_error3.py:
ext/ply/test/lex_error4.exp:
ext/ply/test/lex_error4.py:
ext/ply/test/lex_hedit.exp:
ext/ply/test/lex_hedit.py:
ext/ply/test/lex_ignore.exp:
ext/ply/test/lex_ignore.py:
ext/ply/test/lex_re1.exp:
ext/ply/test/lex_re1.py:
ext/ply/test/lex_rule1.py:
ext/ply/test/lex_token1.py:
ext/ply/test/lex_token2.py:
ext/ply/test/lex_token3.py:
ext/ply/test/lex_token4.py:
ext/ply/test/lex_token5.exp:
ext/ply/test/lex_token5.py:
ext/ply/test/yacc_badargs.exp:
ext/ply/test/yacc_badargs.py:
ext/ply/test/yacc_badprec.exp:
ext/ply/test/yacc_badprec.py:
ext/ply/test/yacc_badprec2.exp:
ext/ply/test/yacc_badprec2.py:
ext/ply/test/yacc_badrule.exp:
ext/ply/test/yacc_badrule.py:
ext/ply/test/yacc_badtok.exp:
ext/ply/test/yacc_badtok.py:
ext/ply/test/yacc_dup.exp:
ext/ply/test/yacc_dup.py:
ext/ply/test/yacc_error1.exp:
ext/ply/test/yacc_error1.py:
ext/ply/test/yacc_error2.exp:
ext/ply/test/yacc_error2.py:
ext/ply/test/yacc_error3.exp:
ext/ply/test/yacc_error3.py:
ext/ply/test/yacc_inf.exp:
ext/ply/test/yacc_inf.py:
ext/ply/test/yacc_missing1.exp:
ext/ply/test/yacc_missing1.py:
ext/ply/test/yacc_nodoc.exp:
ext/ply/test/yacc_nodoc.py:
ext/ply/test/yacc_noerror.exp:
ext/ply/test/yacc_noerror.py:
ext/ply/test/yacc_nop.exp:
ext/ply/test/yacc_nop.py:
ext/ply/test/yacc_notfunc.exp:
ext/ply/test/yacc_notfunc.py:
ext/ply/test/yacc_notok.exp:
ext/ply/test/yacc_notok.py:
ext/ply/test/yacc_rr.exp:
ext/ply/test/yacc_rr.py:
ext/ply/test/yacc_simple.exp:
ext/ply/test/yacc_simple.py:
ext/ply/test/yacc_sr.exp:
ext/ply/test/yacc_sr.py:
ext/ply/test/yacc_term1.exp:
ext/ply/test/yacc_term1.py:
ext/ply/test/yacc_unused.exp:
ext/ply/test/yacc_unused.py:
ext/ply/test/yacc_uprec.exp:
ext/ply/test/yacc_uprec.py:
    Import patch ply.diff
src/arch/isa_parser.py:
    everything is now within the ply package

--HG--
rename : ext/ply/lex.py => ext/ply/ply/lex.py
rename : ext/ply/yacc.py => ext/ply/ply/yacc.py
extra : convert_revision : fca8deabd5c095bdeabd52a1f236ae1404ef106e

18 files changed, 1236 insertions, 0 deletions

diff --git a/ext/ply/example/BASIC/README b/ext/ply/example/BASIC/README
new file mode 100644
index 000000000..be24a3005
--- /dev/null
+++ b/ext/ply/example/BASIC/README
@@ -0,0 +1,79 @@
+Inspired by a September 14, 2006 Salon article "Why Johnny Can't Code" by
+David Brin (http://www.salon.com/tech/feature/2006/09/14/basic/index.html),
+I thought that a fully working BASIC interpreter might be an interesting,
+if not questionable, PLY example.  Uh, okay, so maybe it's just a bad idea,
+but in any case, here it is.
+
+In this example, you'll find a rough implementation of 1964 Dartmouth BASIC
+as described in the manual at:
+
+   http://www.bitsavers.org/pdf/dartmouth/BASIC_Oct64.pdf
+
+See also:
+
+  http://en.wikipedia.org/wiki/Dartmouth_BASIC
+
+This dialect is downright primitive---there are no string variables
+and no facilities for interactive input. Moreover, subroutines and functions
+are brain-dead even more than they usually are for BASIC. Of course,
+the GOTO statement is provided.
+
+Nevertheless, there are a few interesting aspects of this example:
+
+  - It illustrates a fully working interpreter including lexing, parsing,
+    and interpretation of instructions.
+ 
+  - The parser shows how to catch and report various kinds of parsing
+    errors in a more graceful way.
+
+  - The example both parses files (supplied on command line) and
+    interactive input entered line by line.
+
+  - It shows how you might represent parsed information.  In this case,
+    each BASIC statement is encoded into a Python tuple containing the
+    statement type and parameters.  These tuples are then stored in
+    a dictionary indexed by program line numbers.
+
+  - Even though it's just BASIC, the parser contains more than 80
+    rules and 150 parsing states. Thus, it's a little more meaty than
+    the calculator example.
+
+To use the example, run it as follows:
+
+   % python basic.py hello.bas
+   HELLO WORLD
+   %
+
+or use it interactively:
+
+   % python basic.py
+   [BASIC] 10 PRINT "HELLO WORLD"
+   [BASIC] 20 END
+   [BASIC] RUN
+   HELLO WORLD
+   [BASIC]
+
+The following files are defined:
+
+   basic.py         - High level script that controls everything
+   basiclex.py      - BASIC tokenizer
+   basparse.py      - BASIC parser
+   basinterp.py     - BASIC interpreter that runs parsed programs.
+
+In addition, a number of sample BASIC programs (.bas suffix) are
+provided.  These were taken out of the Dartmouth manual.
+
+Disclaimer: I haven't spent a ton of time testing this and it's likely that
+I've skimped here and there on a few finer details (e.g., strictly enforcing
+variable naming rules).  However, the interpreter seems to be able to run
+the examples in the BASIC manual.
+
+Have fun!
+
+-Dave
+
+
+
+
+
+
diff --git a/ext/ply/example/BASIC/basic.py b/ext/ply/example/BASIC/basic.py
new file mode 100644
index 000000000..3a07acdbf
--- /dev/null
+++ b/ext/ply/example/BASIC/basic.py
@@ -0,0 +1,68 @@
+# An implementation of Dartmouth BASIC (1964)
+#
+
+import sys
+sys.path.insert(0,"../..")
+
+import basiclex
+import basparse
+import basinterp
+
+# If a filename has been specified, we try to run it.
+# If a runtime error occurs, we bail out and enter
+# interactive mode below
+if len(sys.argv) == 2:
+    data = open(sys.argv[1]).read()
+    prog = basparse.parse(data)
+    if not prog: raise SystemExit
+    b = basinterp.BasicInterpreter(prog)
+    try:
+        b.run()
+        raise SystemExit
+    except RuntimeError:
+        pass
+
+else:
+    b = basinterp.BasicInterpreter({})
+
+# Interactive mode.  This incrementally adds/deletes statements
+# from the program stored in the BasicInterpreter object.  In
+# addition, special commands 'NEW','LIST',and 'RUN' are added.
+# Specifying a line number with no code deletes that line from
+# the program.
+
+while 1:
+    try:
+        line = raw_input("[BASIC] ")
+    except EOFError:
+        raise SystemExit
+    if not line: continue
+    line += "\n"
+    prog = basparse.parse(line)
+    if not prog: continue
+
+    keys = prog.keys()
+    if keys[0] > 0:
+         b.add_statements(prog)
+    else:
+         stat = prog[keys[0]]
+         if stat[0] == 'RUN':
+             try:
+                 b.run()
+             except RuntimeError:
+                 pass
+         elif stat[0] == 'LIST':
+             b.list()
+         elif stat[0] == 'BLANK':
+             b.del_line(stat[1])
+         elif stat[0] == 'NEW':
+             b.new()
+
+
+
+
+
+
+
+
+
diff --git a/ext/ply/example/BASIC/basiclex.py b/ext/ply/example/BASIC/basiclex.py
new file mode 100644
index 000000000..727383f2b
--- /dev/null
+++ b/ext/ply/example/BASIC/basiclex.py
@@ -0,0 +1,74 @@
+# An implementation of Dartmouth BASIC (1964)
+
+from ply import *
+
+keywords = (
+    'LET','READ','DATA','PRINT','GOTO','IF','THEN','FOR','NEXT','TO','STEP',
+    'END','STOP','DEF','GOSUB','DIM','REM','RETURN','RUN','LIST','NEW',
+)
+
+tokens = keywords + (
+     'EQUALS','PLUS','MINUS','TIMES','DIVIDE','POWER',
+     'LPAREN','RPAREN','LT','LE','GT','GE','NE',
+     'COMMA','SEMI', 'INTEGER','FLOAT', 'STRING',
+     'ID','NEWLINE'
+)
+
+t_ignore = ' \t'
+
+def t_REM(t):
+    r'REM .*'
+    return t
+
+def t_ID(t):
+    r'[A-Z][A-Z0-9]*'
+    if t.value in keywords:
+        t.type = t.value
+    return t
+
+t_EQUALS  = r'='
+t_PLUS    = r'\+'
+t_MINUS   = r'-'
+t_TIMES   = r'\*'
+t_POWER   = r'\^'
+t_DIVIDE  = r'/'
+t_LPAREN  = r'\('
+t_RPAREN  = r'\)'
+t_LT      = r'<'
+t_LE      = r'<='
+t_GT      = r'>'
+t_GE      = r'>='
+t_NE      = r'<>'
+t_COMMA   = r'\,'
+t_SEMI    = r';'
+t_INTEGER = r'\d+'
+t_FLOAT   = r'((\d*\.\d+)(E[\+-]?\d+)?|([1-9]\d*E[\+-]?\d+))'
+t_STRING  = r'\".*?\"'
+
+def t_NEWLINE(t):
+    r'\n'
+    t.lexer.lineno += 1
+    return t
+
+def t_error(t):
+    print "Illegal character", t.value[0]
+    t.lexer.skip(1)
+
+lex.lex()
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/ext/ply/example/BASIC/basinterp.py b/ext/ply/example/BASIC/basinterp.py
new file mode 100644
index 000000000..5850457cb
--- /dev/null
+++ b/ext/ply/example/BASIC/basinterp.py
@@ -0,0 +1,440 @@
+# This file provides the runtime support for running a basic program
+# Assumes the program has been parsed using basparse.py
+
+import sys
+import math
+import random
+
+class BasicInterpreter:
+
+    # Initialize the interpreter. prog is a dictionary
+    # containing (line,statement) mappings
+    def __init__(self,prog):
+         self.prog = prog
+
+         self.functions = {           # Built-in function table
+             'SIN' : lambda z: math.sin(self.eval(z)),
+             'COS' : lambda z: math.cos(self.eval(z)),
+             'TAN' : lambda z: math.tan(self.eval(z)),
+             'ATN' : lambda z: math.atan(self.eval(z)),
+             'EXP' : lambda z: math.exp(self.eval(z)),
+             'ABS' : lambda z: abs(self.eval(z)),
+             'LOG' : lambda z: math.log(self.eval(z)),
+             'SQR' : lambda z: math.sqrt(self.eval(z)),
+             'INT' : lambda z: int(self.eval(z)),
+             'RND' : lambda z: random.random()
+         }
+
+    # Collect all data statements
+    def collect_data(self):
+         self.data = []
+         for lineno in self.stat:
+              if self.prog[lineno][0] == 'DATA':
+                  self.data = self.data + self.prog[lineno][1]
+         self.dc = 0                  # Initialize the data counter
+
+    # Check for end statements
+    def check_end(self):
+         has_end = 0
+         for lineno in self.stat:
+             if self.prog[lineno][0] == 'END' and not has_end:
+                  has_end = lineno
+         if not has_end:
+             print "NO END INSTRUCTION"
+             self.error = 1
+         if has_end != lineno:
+             print "END IS NOT LAST"
+             self.error = 1
+
+    # Check loops
+    def check_loops(self):
+         for pc in range(len(self.stat)):
+             lineno = self.stat[pc]
+             if self.prog[lineno][0] == 'FOR':
+                  forinst = self.prog[lineno]
+                  loopvar = forinst[1]
+                  for i in range(pc+1,len(self.stat)):
+                       if self.prog[self.stat[i]][0] == 'NEXT':
+                            nextvar = self.prog[self.stat[i]][1]
+                            if nextvar != loopvar: continue
+                            self.loopend[pc] = i
+                            break
+                  else:
+                       print "FOR WITHOUT NEXT AT LINE" % self.stat[pc]
+                       self.error = 1
+
+    # Evaluate an expression
+    def eval(self,expr):
+        etype = expr[0]
+        if etype == 'NUM': return expr[1]
+        elif etype == 'GROUP': return self.eval(expr[1])
+        elif etype == 'UNARY':
+             if expr[1] == '-': return -self.eval(expr[2])
+        elif etype == 'BINOP':
+             if expr[1] == '+': return self.eval(expr[2])+self.eval(expr[3])
+             elif expr[1] == '-': return self.eval(expr[2])-self.eval(expr[3])
+             elif expr[1] == '*': return self.eval(expr[2])*self.eval(expr[3])
+             elif expr[1] == '/': return float(self.eval(expr[2]))/self.eval(expr[3])
+             elif expr[1] == '^': return abs(self.eval(expr[2]))**self.eval(expr[3])
+        elif etype == 'VAR':
+             var,dim1,dim2 = expr[1]
+             if not dim1 and not dim2:
+                  if self.vars.has_key(var):
+                       return self.vars[var]
+                  else:
+                       print "UNDEFINED VARIABLE", var, "AT LINE", self.stat[self.pc]
+                       raise RuntimeError
+             # May be a list lookup or a function evaluation
+             if dim1 and not dim2:
+                if self.functions.has_key(var):
+                      # A function
+                      return self.functions[var](dim1)
+                else:
+                      # A list evaluation
+                      if self.lists.has_key(var):
+                            dim1val = self.eval(dim1)
+                            if dim1val < 1 or dim1val > len(self.lists[var]):
+                                 print "LIST INDEX OUT OF BOUNDS AT LINE", self.stat[self.pc]
+                                 raise RuntimeError
+                            return self.lists[var][dim1val-1]
+             if dim1 and dim2:
+                 if self.tables.has_key(var):
+                      dim1val = self.eval(dim1)
+                      dim2val = self.eval(dim2)
+                      if dim1val < 1 or dim1val > len(self.tables[var]) or dim2val < 1 or dim2val > len(self.tables[var][0]):
+                           print "TABLE INDEX OUT OUT BOUNDS AT LINE", self.stat[self.pc]
+                           raise RuntimeError
+                      return self.tables[var][dim1val-1][dim2val-1]
+             print "UNDEFINED VARIABLE", var, "AT LINE", self.stat[self.pc]
+             raise RuntimeError
+
+    # Evaluate a relational expression
+    def releval(self,expr):
+         etype = expr[1]
+         lhs   = self.eval(expr[2])
+         rhs   = self.eval(expr[3])
+         if etype == '<':
+             if lhs < rhs: return 1
+             else: return 0
+
+         elif etype == '<=':
+             if lhs <= rhs: return 1
+             else: return 0
+
+         elif etype == '>':
+             if lhs > rhs: return 1
+             else: return 0
+
+         elif etype == '>=':
+             if lhs >= rhs: return 1
+             else: return 0
+
+         elif etype == '=':
+             if lhs == rhs: return 1
+             else: return 0
+
+         elif etype == '<>':
+             if lhs != rhs: return 1
+             else: return 0
+
+    # Assignment
+    def assign(self,target,value):
+        var, dim1, dim2 = target
+        if not dim1 and not dim2:
+            self.vars[var] = self.eval(value)
+        elif dim1 and not dim2:
+            # List assignment
+            dim1val = self.eval(dim1)
+            if not self.lists.has_key(var):
+                 self.lists[var] = [0]*10
+
+            if dim1val > len(self.lists[var]):
+                 print "DIMENSION TOO LARGE AT LINE", self.stat[self.pc]
+                 raise RuntimeError
+            self.lists[var][dim1val-1] = self.eval(value)
+        elif dim1 and dim2:
+            dim1val = self.eval(dim1)
+            dim2val = self.eval(dim2)
+            if not self.tables.has_key(var):
+                 temp = [0]*10
+                 v = []
+                 for i in range(10): v.append(temp[:])
+                 self.tables[var] = v
+            # Variable already exists
+            if dim1val > len(self.tables[var]) or dim2val > len(self.tables[var][0]):
+                 print "DIMENSION TOO LARGE AT LINE", self.stat[self.pc]
+                 raise RuntimeError
+            self.tables[var][dim1val-1][dim2val-1] = self.eval(value)
+
+    # Change the current line number
+    def goto(self,linenum):
+         if not self.prog.has_key(linenum):
+              print "UNDEFINED LINE NUMBER %d AT LINE %d" % (linenum, self.stat[self.pc])
+              raise RuntimeError
+         self.pc = self.stat.index(linenum)
+
+    # Run it
+    def run(self):
+        self.vars   = { }            # All variables
+        self.lists  = { }            # List variables
+        self.tables = { }            # Tables
+        self.loops  = [ ]            # Currently active loops
+        self.loopend= { }            # Mapping saying where loops end
+        self.gosub  = None           # Gosub return point (if any)
+        self.error  = 0              # Indicates program error
+
+        self.stat = self.prog.keys()      # Ordered list of all line numbers
+        self.stat.sort()
+        self.pc = 0                  # Current program counter
+
+        # Processing prior to running
+
+        self.collect_data()          # Collect all of the data statements
+        self.check_end()
+        self.check_loops()
+
+        if self.error: raise RuntimeError
+
+        while 1:
+            line  = self.stat[self.pc]
+            instr = self.prog[line]
+
+            op = instr[0]
+
+            # END and STOP statements
+            if op == 'END' or op == 'STOP':
+                 break           # We're done
+
+            # GOTO statement
+            elif op == 'GOTO':
+                 newline = instr[1]
+                 self.goto(newline)
+                 continue
+
+            # PRINT statement
+            elif op == 'PRINT':
+                 plist = instr[1]
+                 out = ""
+                 for label,val in plist:
+                     if out:
+                          out += ' '*(15 - (len(out) % 15))
+                     out += label
+                     if val:
+                          if label: out += " "
+                          eval = self.eval(val)
+                          out += str(eval)
+                 sys.stdout.write(out)
+                 end = instr[2]
+                 if not (end == ',' or end == ';'):
+                     sys.stdout.write("\n")
+                 if end == ',': sys.stdout.write(" "*(15-(len(out) % 15)))
+                 if end == ';': sys.stdout.write(" "*(3-(len(out) % 3)))
+
+            # LET statement
+            elif op == 'LET':
+                 target = instr[1]
+                 value  = instr[2]
+                 self.assign(target,value)
+
+            # READ statement
+            elif op == 'READ':
+                 for target in instr[1]:
+                      if self.dc < len(self.data):
+                          value = ('NUM',self.data[self.dc])
+                          self.assign(target,value)
+                          self.dc += 1
+                      else:
+                          # No more data.  Program ends
+                          return
+            elif op == 'IF':
+                 relop = instr[1]
+                 newline = instr[2]
+                 if (self.releval(relop)):
+                     self.goto(newline)
+                     continue
+
+            elif op == 'FOR':
+                 loopvar = instr[1]
+                 initval = instr[2]
+                 finval  = instr[3]
+                 stepval = instr[4]
+
+                 # Check to see if this is a new loop
+                 if not self.loops or self.loops[-1][0] != self.pc:
+                        # Looks like a new loop. Make the initial assignment
+                        newvalue = initval
+                        self.assign((loopvar,None,None),initval)
+                        if not stepval: stepval = ('NUM',1)
+                        stepval = self.eval(stepval)    # Evaluate step here
+                        self.loops.append((self.pc,stepval))
+                 else:
+                        # It's a repeat of the previous loop
+                        # Update the value of the loop variable according to the step
+                        stepval = ('NUM',self.loops[-1][1])
+                        newvalue = ('BINOP','+',('VAR',(loopvar,None,None)),stepval)
+
+                 if self.loops[-1][1] < 0: relop = '>='
+                 else: relop = '<='
+                 if not self.releval(('RELOP',relop,newvalue,finval)):
+                      # Loop is done. Jump to the NEXT
+                      self.pc = self.loopend[self.pc]
+                      self.loops.pop()
+                 else:
+                      self.assign((loopvar,None,None),newvalue)
+
+            elif op == 'NEXT':
+                 if not self.loops:
+                       print "NEXT WITHOUT FOR AT LINE",line
+                       return
+
+                 nextvar = instr[1]
+                 self.pc = self.loops[-1][0]
+                 loopinst = self.prog[self.stat[self.pc]]
+                 forvar = loopinst[1]
+                 if nextvar != forvar:
+                       print "NEXT DOESN'T MATCH FOR AT LINE", line
+                       return
+                 continue
+            elif op == 'GOSUB':
+                 newline = instr[1]
+                 if self.gosub:
+                       print "ALREADY IN A SUBROUTINE AT LINE", line
+                       return
+                 self.gosub = self.stat[self.pc]
+                 self.goto(newline)
+                 continue
+
+            elif op == 'RETURN':
+                 if not self.gosub:
+                      print "RETURN WITHOUT A GOSUB AT LINE",line
+                      return
+                 self.goto(self.gosub)
+                 self.gosub = None
+
+            elif op == 'FUNC':
+                 fname = instr[1]
+                 pname = instr[2]
+                 expr  = instr[3]
+                 def eval_func(pvalue,name=pname,self=self,expr=expr):
+                      self.assign((pname,None,None),pvalue)
+                      return self.eval(expr)
+                 self.functions[fname] = eval_func
+
+            elif op == 'DIM':
+                 for vname,x,y in instr[1]:
+                     if y == 0:
+                          # Single dimension variable
+                          self.lists[vname] = [0]*x
+                     else:
+                          # Double dimension variable
+                          temp = [0]*y
+                          v = []
+                          for i in range(x):
+                              v.append(temp[:])
+                          self.tables[vname] = v
+
+            self.pc += 1
+
+    # Utility functions for program listing
+    def expr_str(self,expr):
+        etype = expr[0]
+        if etype == 'NUM': return str(expr[1])
+        elif etype == 'GROUP': return "(%s)" % self.expr_str(expr[1])
+        elif etype == 'UNARY':
+             if expr[1] == '-': return "-"+str(expr[2])
+        elif etype == 'BINOP':
+             return "%s %s %s" % (self.expr_str(expr[2]),expr[1],self.expr_str(expr[3]))
+        elif etype == 'VAR':
+              return self.var_str(expr[1])
+
+    def relexpr_str(self,expr):
+         return "%s %s %s" % (self.expr_str(expr[2]),expr[1],self.expr_str(expr[3]))
+
+    def var_str(self,var):
+         varname,dim1,dim2 = var
+         if not dim1 and not dim2: return varname
+         if dim1 and not dim2: return "%s(%s)" % (varname, self.expr_str(dim1))
+         return "%s(%s,%s)" % (varname, self.expr_str(dim1),self.expr_str(dim2))
+
+    # Create a program listing
+    def list(self):
+         stat = self.prog.keys()      # Ordered list of all line numbers
+         stat.sort()
+         for line in stat:
+             instr = self.prog[line]
+             op = instr[0]
+             if op in ['END','STOP','RETURN']:
+                   print line, op
+                   continue
+             elif op == 'REM':
+                   print line, instr[1]
+             elif op == 'PRINT':
+                   print line, op,
+                   first = 1
+                   for p in instr[1]:
+                         if not first: print ",",
+                         if p[0] and p[1]: print '"%s"%s' % (p[0],self.expr_str(p[1])),
+                         elif p[1]: print self.expr_str(p[1]),
+                         else: print '"%s"' % (p[0],),
+                         first = 0
+                   if instr[2]: print instr[2]
+                   else: print
+             elif op == 'LET':
+                   print line,"LET",self.var_str(instr[1]),"=",self.expr_str(instr[2])
+             elif op == 'READ':
+                   print line,"READ",
+                   first = 1
+                   for r in instr[1]:
+                         if not first: print ",",
+                         print self.var_str(r),
+                         first = 0
+                   print ""
+             elif op == 'IF':
+                   print line,"IF %s THEN %d" % (self.relexpr_str(instr[1]),instr[2])
+             elif op == 'GOTO' or op == 'GOSUB':
+                   print line, op, instr[1]
+             elif op == 'FOR':
+                   print line,"FOR %s = %s TO %s" % (instr[1],self.expr_str(instr[2]),self.expr_str(instr[3])),
+                   if instr[4]: print "STEP %s" % (self.expr_str(instr[4])),
+                   print
+             elif op == 'NEXT':
+                   print line,"NEXT", instr[1]
+             elif op == 'FUNC':
+                   print line,"DEF %s(%s) = %s" % (instr[1],instr[2],self.expr_str(instr[3]))
+             elif op == 'DIM':
+                   print line,"DIM",
+                   first = 1
+                   for vname,x,y in instr[1]:
+                         if not first: print ",",
+                         first = 0
+                         if y == 0:
+                               print "%s(%d)" % (vname,x),
+                         else:
+                               print "%s(%d,%d)" % (vname,x,y),
+
+                   print
+             elif op == 'DATA':
+                   print line,"DATA",
+                   first = 1
+                   for v in instr[1]:
+                        if not first: print ",",
+                        first = 0
+                        print v,
+                   print
+
+    # Erase the current program
+    def new(self):
+         self.prog = {}
+
+    # Insert statements
+    def add_statements(self,prog):
+         for line,stat in prog.items():
+              self.prog[line] = stat
+
+    # Delete a statement
+    def del_line(self,lineno):
+         try:
+             del self.prog[lineno]
+         except KeyError:
+             pass
+
diff --git a/ext/ply/example/BASIC/basparse.py b/ext/ply/example/BASIC/basparse.py
new file mode 100644
index 000000000..930af9a22
--- /dev/null
+++ b/ext/ply/example/BASIC/basparse.py
@@ -0,0 +1,424 @@
+# An implementation of Dartmouth BASIC (1964)
+#
+
+from ply import *
+import basiclex
+
+tokens = basiclex.tokens
+
+precedence = (
+               ('left', 'PLUS','MINUS'),
+               ('left', 'TIMES','DIVIDE'),
+               ('left', 'POWER'),
+               ('right','UMINUS')
+)
+
+#### A BASIC program is a series of statements.  We represent the program as a
+#### dictionary of tuples indexed by line number.
+
+def p_program(p):
+    '''program : program statement
+               | statement'''
+
+    if len(p) == 2 and p[1]:
+       p[0] = { }
+       line,stat = p[1]
+       p[0][line] = stat
+    elif len(p) ==3:
+       p[0] = p[1]
+       if not p[0]: p[0] = { }
+       if p[2]:
+           line,stat = p[2]
+           p[0][line] = stat
+
+#### This catch-all rule is used for any catastrophic errors.  In this case,
+#### we simply return nothing
+
+def p_program_error(p):
+    '''program : error'''
+    p[0] = None
+    p.parser.error = 1
+
+#### Format of all BASIC statements.
+
+def p_statement(p):
+    '''statement : INTEGER command NEWLINE'''
+    if isinstance(p[2],str):
+        print p[2],"AT LINE", p[1]
+        p[0] = None
+        p.parser.error = 1
+    else:
+        lineno = int(p[1])
+        p[0] = (lineno,p[2])
+
+#### Interactive statements.
+
+def p_statement_interactive(p):
+    '''statement : RUN NEWLINE
+                 | LIST NEWLINE
+                 | NEW NEWLINE'''
+    p[0] = (0, (p[1],0))
+
+#### Blank line number
+def p_statement_blank(p):
+    '''statement : INTEGER NEWLINE'''
+    p[0] = (0,('BLANK',int(p[1])))
+
+#### Error handling for malformed statements
+
+def p_statement_bad(p):
+    '''statement : INTEGER error NEWLINE'''
+    print "MALFORMED STATEMENT AT LINE", p[1]
+    p[0] = None
+    p.parser.error = 1
+
+#### Blank line
+
+def p_statement_newline(p):
+    '''statement : NEWLINE'''
+    p[0] = None
+
+#### LET statement
+
+def p_command_let(p):
+    '''command : LET variable EQUALS expr'''
+    p[0] = ('LET',p[2],p[4])
+
+def p_command_let_bad(p):
+    '''command : LET variable EQUALS error'''
+    p[0] = "BAD EXPRESSION IN LET"
+
+#### READ statement
+
+def p_command_read(p):
+    '''command : READ varlist'''
+    p[0] = ('READ',p[2])
+
+def p_command_read_bad(p):
+    '''command : READ error'''
+    p[0] = "MALFORMED VARIABLE LIST IN READ"
+
+#### DATA statement
+
+def p_command_data(p):
+    '''command : DATA numlist'''
+    p[0] = ('DATA',p[2])
+
+def p_command_data_bad(p):
+    '''command : DATA error'''
+    p[0] = "MALFORMED NUMBER LIST IN DATA"
+
+#### PRINT statement
+
+def p_command_print(p):
+    '''command : PRINT plist optend'''
+    p[0] = ('PRINT',p[2],p[3])
+
+def p_command_print_bad(p):
+    '''command : PRINT error'''
+    p[0] = "MALFORMED PRINT STATEMENT"
+
+#### Optional ending on PRINT. Either a comma (,) or semicolon (;)
+
+def p_optend(p):
+    '''optend : COMMA
+              | SEMI
+              |'''
+    if len(p)  == 2:
+         p[0] = p[1]
+    else:
+         p[0] = None
+
+#### PRINT statement with no arguments
+
+def p_command_print_empty(p):
+    '''command : PRINT'''
+    p[0] = ('PRINT',[],None)
+
+#### GOTO statement
+
+def p_command_goto(p):
+    '''command : GOTO INTEGER'''
+    p[0] = ('GOTO',int(p[2]))
+
+def p_command_goto_bad(p):
+    '''command : GOTO error'''
+    p[0] = "INVALID LINE NUMBER IN GOTO"
+
+#### IF-THEN statement
+
+def p_command_if(p):
+    '''command : IF relexpr THEN INTEGER'''
+    p[0] = ('IF',p[2],int(p[4]))
+
+def p_command_if_bad(p):
+    '''command : IF error THEN INTEGER'''
+    p[0] = "BAD RELATIONAL EXPRESSION"
+
+def p_command_if_bad2(p):
+    '''command : IF relexpr THEN error'''
+    p[0] = "INVALID LINE NUMBER IN THEN"
+
+#### FOR statement
+
+def p_command_for(p):
+    '''command : FOR ID EQUALS expr TO expr optstep'''
+    p[0] = ('FOR',p[2],p[4],p[6],p[7])
+
+def p_command_for_bad_initial(p):
+    '''command : FOR ID EQUALS error TO expr optstep'''
+    p[0] = "BAD INITIAL VALUE IN FOR STATEMENT"
+
+def p_command_for_bad_final(p):
+    '''command : FOR ID EQUALS expr TO error optstep'''
+    p[0] = "BAD FINAL VALUE IN FOR STATEMENT"
+
+def p_command_for_bad_step(p):
+    '''command : FOR ID EQUALS expr TO expr STEP error'''
+    p[0] = "MALFORMED STEP IN FOR STATEMENT"
+
+#### Optional STEP qualifier on FOR statement
+
+def p_optstep(p):
+    '''optstep : STEP expr
+               | empty'''
+    if len(p) == 3:
+       p[0] = p[2]
+    else:
+       p[0] = None
+
+#### NEXT statement
+
+def p_command_next(p):
+    '''command : NEXT ID'''
+
+    p[0] = ('NEXT',p[2])
+
+def p_command_next_bad(p):
+    '''command : NEXT error'''
+    p[0] = "MALFORMED NEXT"
+
+#### END statement
+
+def p_command_end(p):
+    '''command : END'''
+    p[0] = ('END',)
+
+#### REM statement
+
+def p_command_rem(p):
+    '''command : REM'''
+    p[0] = ('REM',p[1])
+
+#### STOP statement
+
+def p_command_stop(p):
+    '''command : STOP'''
+    p[0] = ('STOP',)
+
+#### DEF statement
+
+def p_command_def(p):
+    '''command : DEF ID LPAREN ID RPAREN EQUALS expr'''
+    p[0] = ('FUNC',p[2],p[4],p[7])
+
+def p_command_def_bad_rhs(p):
+    '''command : DEF ID LPAREN ID RPAREN EQUALS error'''
+    p[0] = "BAD EXPRESSION IN DEF STATEMENT"
+
+def p_command_def_bad_arg(p):
+    '''command : DEF ID LPAREN error RPAREN EQUALS expr'''
+    p[0] = "BAD ARGUMENT IN DEF STATEMENT"
+
+#### GOSUB statement
+
+def p_command_gosub(p):
+    '''command : GOSUB INTEGER'''
+    p[0] = ('GOSUB',int(p[2]))
+
+def p_command_gosub_bad(p):
+    '''command : GOSUB error'''
+    p[0] = "INVALID LINE NUMBER IN GOSUB"
+
+#### RETURN statement
+
+def p_command_return(p):
+    '''command : RETURN'''
+    p[0] = ('RETURN',)
+
+#### DIM statement
+
+def p_command_dim(p):
+    '''command : DIM dimlist'''
+    p[0] = ('DIM',p[2])
+
+def p_command_dim_bad(p):
+    '''command : DIM error'''
+    p[0] = "MALFORMED VARIABLE LIST IN DIM"
+
+#### List of variables supplied to DIM statement
+
+def p_dimlist(p):
+    '''dimlist : dimlist COMMA dimitem
+               | dimitem'''
+    if len(p) == 4:
+        p[0] = p[1]
+        p[0].append(p[3])
+    else:
+        p[0] = [p[1]]
+
+#### DIM items
+
+def p_dimitem_single(p):
+    '''dimitem : ID LPAREN INTEGER RPAREN'''
+    p[0] = (p[1],eval(p[3]),0)
+
+def p_dimitem_double(p):
+    '''dimitem : ID LPAREN INTEGER COMMA INTEGER RPAREN'''
+    p[0] = (p[1],eval(p[3]),eval(p[5]))
+
+#### Arithmetic expressions
+
+def p_expr_binary(p):
+    '''expr : expr PLUS expr
+            | expr MINUS expr
+            | expr TIMES expr
+            | expr DIVIDE expr
+            | expr POWER expr'''
+
+    p[0] = ('BINOP',p[2],p[1],p[3])
+
+def p_expr_number(p):
+    '''expr : INTEGER
+            | FLOAT'''
+    p[0] = ('NUM',eval(p[1]))
+
+def p_expr_variable(p):
+    '''expr : variable'''
+    p[0] = ('VAR',p[1])
+
+def p_expr_group(p):
+    '''expr : LPAREN expr RPAREN'''
+    p[0] = ('GROUP',p[2])
+
+def p_expr_unary(p):
+    '''expr : MINUS expr %prec UMINUS'''
+    p[0] = ('UNARY','-',p[2])
+
+#### Relational expressions
+
+def p_relexpr(p):
+    '''relexpr : expr LT expr
+               | expr LE expr
+               | expr GT expr
+               | expr GE expr
+               | expr EQUALS expr
+               | expr NE expr'''
+    p[0] = ('RELOP',p[2],p[1],p[3])
+
+#### Variables
+
+def p_variable(p):
+    '''variable : ID
+              | ID LPAREN expr RPAREN
+              | ID LPAREN expr COMMA expr RPAREN'''
+    if len(p) == 2:
+       p[0] = (p[1],None,None)
+    elif len(p) == 5:
+       p[0] = (p[1],p[3],None)
+    else:
+       p[0] = (p[1],p[3],p[5])
+
+#### Builds a list of variable targets as a Python list
+
+def p_varlist(p):
+    '''varlist : varlist COMMA variable
+               | variable'''
+    if len(p) > 2:
+       p[0] = p[1]
+       p[0].append(p[3])
+    else:
+       p[0] = [p[1]]
+
+
+#### Builds a list of numbers as a Python list
+
+def p_numlist(p):
+    '''numlist : numlist COMMA number
+               | number'''
+
+    if len(p) > 2:
+       p[0] = p[1]
+       p[0].append(p[3])
+    else:
+       p[0] = [p[1]]
+
+#### A number. May be an integer or a float
+
+def p_number(p):
+    '''number  : INTEGER
+               | FLOAT'''
+    p[0] = eval(p[1])
+
+#### A signed number.
+
+def p_number_signed(p):
+    '''number  : MINUS INTEGER
+               | MINUS FLOAT'''
+    p[0] = eval("-"+p[2])
+
+#### List of targets for a print statement
+#### Returns a list of tuples (label,expr)
+
+def p_plist(p):
+    '''plist   : plist COMMA pitem
+               | pitem'''
+    if len(p) > 3:
+       p[0] = p[1]
+       p[0].append(p[3])
+    else:
+       p[0] = [p[1]]
+
+def p_item_string(p):
+    '''pitem : STRING'''
+    p[0] = (p[1][1:-1],None)
+
+def p_item_string_expr(p):
+    '''pitem : STRING expr'''
+    p[0] = (p[1][1:-1],p[2])
+
+def p_item_expr(p):
+    '''pitem : expr'''
+    p[0] = ("",p[1])
+
+#### Empty
+
+def p_empty(p):
+    '''empty : '''
+
+#### Catastrophic error handler
+def p_error(p):
+    if not p:
+        print "SYNTAX ERROR AT EOF"
+
+bparser = yacc.yacc()
+
+def parse(data):
+    bparser.error = 0
+    p = bparser.parse(data)
+    if bparser.error: return None
+    return p
+
+
+
+
+
+
+
+
+
+
+
+
+
+
diff --git a/ext/ply/example/BASIC/dim.bas b/ext/ply/example/BASIC/dim.bas
new file mode 100644
index 000000000..87bd95b32
--- /dev/null
+++ b/ext/ply/example/BASIC/dim.bas
@@ -0,0 +1,14 @@
+5 DIM A(50,15)
+10 FOR I = 1 TO 50
+20 FOR J = 1 TO 15
+30   LET A(I,J) = I + J
+35 REM  PRINT I,J, A(I,J)
+40 NEXT J
+50 NEXT I
+100 FOR I = 1 TO 50
+110 FOR J = 1 TO 15
+120   PRINT A(I,J),
+130 NEXT J
+140 PRINT 
+150 NEXT I
+999 END
diff --git a/ext/ply/example/BASIC/func.bas b/ext/ply/example/BASIC/func.bas
new file mode 100644
index 000000000..447ee16a9
--- /dev/null
+++ b/ext/ply/example/BASIC/func.bas
@@ -0,0 +1,5 @@
+10 DEF FDX(X) = 2*X
+20 FOR I = 0 TO 100
+30 PRINT FDX(I)
+40 NEXT I
+50 END
diff --git a/ext/ply/example/BASIC/gcd.bas b/ext/ply/example/BASIC/gcd.bas
new file mode 100644
index 000000000..d0b774608
--- /dev/null
+++ b/ext/ply/example/BASIC/gcd.bas
@@ -0,0 +1,22 @@
+10 PRINT "A","B","C","GCD"
+20 READ A,B,C
+30 LET X = A
+40 LET Y = B
+50 GOSUB 200
+60 LET X = G
+70 LET Y = C
+80 GOSUB 200
+90 PRINT A, B, C, G
+100 GOTO 20
+110 DATA 60, 90, 120
+120 DATA 38456, 64872, 98765
+130 DATA 32, 384, 72
+200 LET Q = INT(X/Y)
+210 LET R = X - Q*Y
+220 IF R = 0 THEN 300
+230 LET X = Y
+240 LET Y = R
+250 GOTO 200
+300 LET G = Y
+310 RETURN
+999 END
diff --git a/ext/ply/example/BASIC/gosub.bas b/ext/ply/example/BASIC/gosub.bas
new file mode 100644
index 000000000..99737b16f
--- /dev/null
+++ b/ext/ply/example/BASIC/gosub.bas
@@ -0,0 +1,13 @@
+100 LET X = 3
+110 GOSUB 400
+120 PRINT U, V, W
+200 LET X = 5
+210 GOSUB 400
+220 LET Z = U + 2*V + 3*W
+230 PRINT Z
+240 GOTO 999
+400 LET U = X*X
+410 LET V = X*X*X
+420 LET W = X*X*X*X + X*X*X + X*X + X
+430 RETURN
+999 END
diff --git a/ext/ply/example/BASIC/hello.bas b/ext/ply/example/BASIC/hello.bas
new file mode 100644
index 000000000..cc6f0b0b5
--- /dev/null
+++ b/ext/ply/example/BASIC/hello.bas
@@ -0,0 +1,4 @@
+5  REM HELLO WORLD PROGAM
+10 PRINT "HELLO WORLD"
+99 END
+
diff --git a/ext/ply/example/BASIC/linear.bas b/ext/ply/example/BASIC/linear.bas
new file mode 100644
index 000000000..56c08220b
--- /dev/null
+++ b/ext/ply/example/BASIC/linear.bas
@@ -0,0 +1,17 @@
+1  REM ::: SOLVE A SYSTEM OF LINEAR EQUATIONS
+2  REM ::: A1*X1 + A2*X2 = B1
+3  REM ::: A3*X1 + A4*X2 = B2
+4  REM --------------------------------------
+10 READ A1, A2, A3, A4
+15 LET D = A1 * A4 - A3 * A2
+20 IF D = 0 THEN 65
+30 READ B1, B2
+37 LET X1 = (B1*A4 - B2*A2) / D
+42 LET X2 = (A1*B2 - A3*B1) / D
+55 PRINT X1, X2
+60 GOTO 30
+65 PRINT "NO UNIQUE SOLUTION"
+70 DATA 1, 2, 4
+80 DATA 2, -7, 5
+85 DATA 1, 3, 4, -7
+90 END
diff --git a/ext/ply/example/BASIC/maxsin.bas b/ext/ply/example/BASIC/maxsin.bas
new file mode 100644
index 000000000..b96901530
--- /dev/null
+++ b/ext/ply/example/BASIC/maxsin.bas
@@ -0,0 +1,12 @@
+5 PRINT "X VALUE", "SINE", "RESOLUTION"
+10 READ D
+20 LET M = -1
+30 FOR X = 0 TO 3 STEP D
+40 IF SIN(X) <= M THEN 80
+50 LET X0 = X
+60 LET M = SIN(X)
+80 NEXT X
+85 PRINT X0, M, D
+90 GOTO 10
+100 DATA .1, .01, .001
+110 END
diff --git a/ext/ply/example/BASIC/powers.bas b/ext/ply/example/BASIC/powers.bas
new file mode 100644
index 000000000..a454dc3e2
--- /dev/null
+++ b/ext/ply/example/BASIC/powers.bas
@@ -0,0 +1,13 @@
+5 PRINT "THIS PROGRAM COMPUTES AND PRINTS THE NTH POWERS"
+6 PRINT "OF THE NUMBERS LESS THAN OR EQUAL TO N FOR VARIOUS"
+7 PRINT "N FROM 1 THROUGH 7"
+8 PRINT
+10 FOR N = 1 TO 7
+15 PRINT "N = "N
+20 FOR I = 1 TO N
+30 PRINT I^N,
+40 NEXT I
+50 PRINT
+60 PRINT
+70 NEXT N
+80 END
diff --git a/ext/ply/example/BASIC/rand.bas b/ext/ply/example/BASIC/rand.bas
new file mode 100644
index 000000000..4ff7a1467
--- /dev/null
+++ b/ext/ply/example/BASIC/rand.bas
@@ -0,0 +1,4 @@
+10 FOR I = 1 TO 20
+20 PRINT INT(10*RND(0))
+30 NEXT I
+40 END
diff --git a/ext/ply/example/BASIC/sales.bas b/ext/ply/example/BASIC/sales.bas
new file mode 100644
index 000000000..a39aefb76
--- /dev/null
+++ b/ext/ply/example/BASIC/sales.bas
@@ -0,0 +1,20 @@
+10 FOR I = 1 TO 3
+20 READ P(I)
+30 NEXT I
+40 FOR I = 1 TO 3
+50 FOR J = 1 TO 5
+60 READ S(I,J)
+70 NEXT J
+80 NEXT I
+90 FOR J = 1 TO 5
+100 LET S = 0
+110 FOR I = 1 TO 3
+120 LET S = S + P(I) * S(I,J)
+130 NEXT I
+140 PRINT "TOTAL SALES FOR SALESMAN"J, "$"S
+150 NEXT J
+200 DATA 1.25, 4.30, 2.50
+210 DATA 40, 20, 37, 29, 42
+220 DATA 10, 16, 3, 21, 8
+230 DATA 35, 47, 29, 16, 33
+300 END
diff --git a/ext/ply/example/BASIC/sears.bas b/ext/ply/example/BASIC/sears.bas
new file mode 100644
index 000000000..5ced3974e
--- /dev/null
+++ b/ext/ply/example/BASIC/sears.bas
@@ -0,0 +1,18 @@
+1 REM :: THIS PROGRAM COMPUTES HOW MANY TIMES YOU HAVE TO FOLD
+2 REM :: A PIECE OF PAPER SO THAT IT IS TALLER THAN THE
+3 REM :: SEARS TOWER.
+4 REM ::       S = HEIGHT OF TOWER (METERS)
+5 REM ::       T = THICKNESS OF PAPER (MILLIMETERS)
+10 LET S = 442
+20 LET T = 0.1
+30 REM CONVERT T TO METERS
+40 LET T = T * .001
+50 LET F = 1
+60 LET H = T
+100 IF H > S THEN 200
+120 LET H = 2 * H
+125 LET F = F + 1
+130 GOTO 100
+200 PRINT "NUMBER OF FOLDS ="F
+220 PRINT "FINAL HEIGHT    ="H
+999 END
diff --git a/ext/ply/example/BASIC/sqrt1.bas b/ext/ply/example/BASIC/sqrt1.bas
new file mode 100644
index 000000000..6673a9152
--- /dev/null
+++ b/ext/ply/example/BASIC/sqrt1.bas
@@ -0,0 +1,5 @@
+10 LET X = 0
+20 LET X = X + 1
+30 PRINT X, SQR(X)
+40 IF X < 100 THEN 20
+50 END
diff --git a/ext/ply/example/BASIC/sqrt2.bas b/ext/ply/example/BASIC/sqrt2.bas
new file mode 100644
index 000000000..862d85ef2
--- /dev/null
+++ b/ext/ply/example/BASIC/sqrt2.bas
@@ -0,0 +1,4 @@
+10 FOR X = 1 TO 100
+20 PRINT X, SQR(X)
+30 NEXT X
+40 END