ply: update PLY to version 3.2

1 files changed, 332 insertions, 0 deletions

diff --git a/ext/ply/CHANGES b/ext/ply/CHANGES
index d88f3e5d6..9d8b25d5a 100644
--- a/ext/ply/CHANGES
+++ b/ext/ply/CHANGES
@@ -1,3 +1,335 @@
+
+Version 3.2
+-----------------------------
+03/24/09: beazley
+          Added an extra check to not print duplicated warning messages
+          about reduce/reduce conflicts.
+
+03/24/09: beazley
+          Switched PLY over to a BSD-license.
+
+03/23/09: beazley
+          Performance optimization.  Discovered a few places to make
+          speedups in LR table generation.
+
+03/23/09: beazley
+          New warning message.  PLY now warns about rules never 
+          reduced due to reduce/reduce conflicts.  Suggested by
+          Bruce Frederiksen.
+
+03/23/09: beazley
+          Some clean-up of warning messages related to reduce/reduce errors.
+
+03/23/09: beazley
+          Added a new picklefile option to yacc() to write the parsing
+          tables to a filename using the pickle module.   Here is how
+          it works:
+
+              yacc(picklefile="parsetab.p")
+
+          This option can be used if the normal parsetab.py file is
+          extremely large.  For example, on jython, it is impossible
+          to read parsing tables if the parsetab.py exceeds a certain
+          threshold.
+
+          The filename supplied to the picklefile option is opened
+          relative to the current working directory of the Python 
+          interpreter.  If you need to refer to the file elsewhere,
+          you will need to supply an absolute or relative path.
+
+          For maximum portability, the pickle file is written
+          using protocol 0. 
+ 
+03/13/09: beazley
+          Fixed a bug in parser.out generation where the rule numbers
+          where off by one.
+
+03/13/09: beazley
+          Fixed a string formatting bug with one of the error messages.
+          Reported by Richard Reitmeyer
+
+Version 3.1
+-----------------------------
+02/28/09: beazley
+          Fixed broken start argument to yacc().  PLY-3.0 broke this
+          feature by accident.
+
+02/28/09: beazley
+          Fixed debugging output. yacc() no longer reports shift/reduce
+          or reduce/reduce conflicts if debugging is turned off.  This
+          restores similar behavior in PLY-2.5.   Reported by Andrew Waters.
+
+Version 3.0
+-----------------------------
+02/03/09: beazley
+          Fixed missing lexer attribute on certain tokens when
+          invoking the parser p_error() function.  Reported by 
+          Bart Whiteley.
+
+02/02/09: beazley
+          The lex() command now does all error-reporting and diagonistics
+          using the logging module interface.   Pass in a Logger object
+          using the errorlog parameter to specify a different logger.
+
+02/02/09: beazley
+          Refactored ply.lex to use a more object-oriented and organized
+          approach to collecting lexer information.
+
+02/01/09: beazley
+          Removed the nowarn option from lex().  All output is controlled
+          by passing in a logger object.   Just pass in a logger with a high
+          level setting to suppress output.   This argument was never 
+          documented to begin with so hopefully no one was relying upon it.
+
+02/01/09: beazley
+          Discovered and removed a dead if-statement in the lexer.  This
+          resulted in a 6-7% speedup in lexing when I tested it.
+
+01/13/09: beazley
+          Minor change to the procedure for signalling a syntax error in a
+          production rule.  A normal SyntaxError exception should be raised
+          instead of yacc.SyntaxError.
+
+01/13/09: beazley
+          Added a new method p.set_lineno(n,lineno) that can be used to set the
+          line number of symbol n in grammar rules.   This simplifies manual
+          tracking of line numbers.
+
+01/11/09: beazley
+          Vastly improved debugging support for yacc.parse().   Instead of passing
+          debug as an integer, you can supply a Logging object (see the logging
+          module). Messages will be generated at the ERROR, INFO, and DEBUG
+	  logging levels, each level providing progressively more information.
+          The debugging trace also shows states, grammar rule, values passed
+          into grammar rules, and the result of each reduction.
+
+01/09/09: beazley
+          The yacc() command now does all error-reporting and diagnostics using
+          the interface of the logging module.  Use the errorlog parameter to
+          specify a logging object for error messages.  Use the debuglog parameter
+          to specify a logging object for the 'parser.out' output.
+
+01/09/09: beazley
+          *HUGE* refactoring of the the ply.yacc() implementation.   The high-level
+	  user interface is backwards compatible, but the internals are completely
+          reorganized into classes.  No more global variables.    The internals
+          are also more extensible.  For example, you can use the classes to 
+          construct a LALR(1) parser in an entirely different manner than 
+          what is currently the case.  Documentation is forthcoming.
+
+01/07/09: beazley
+          Various cleanup and refactoring of yacc internals.  
+
+01/06/09: beazley
+          Fixed a bug with precedence assignment.  yacc was assigning the precedence
+          each rule based on the left-most token, when in fact, it should have been
+          using the right-most token.  Reported by Bruce Frederiksen.
+
+11/27/08: beazley
+          Numerous changes to support Python 3.0 including removal of deprecated 
+          statements (e.g., has_key) and the additional of compatibility code
+          to emulate features from Python 2 that have been removed, but which
+          are needed.   Fixed the unit testing suite to work with Python 3.0.
+          The code should be backwards compatible with Python 2.
+
+11/26/08: beazley
+          Loosened the rules on what kind of objects can be passed in as the
+          "module" parameter to lex() and yacc().  Previously, you could only use
+          a module or an instance.  Now, PLY just uses dir() to get a list of
+          symbols on whatever the object is without regard for its type.
+
+11/26/08: beazley
+          Changed all except: statements to be compatible with Python2.x/3.x syntax.
+
+11/26/08: beazley
+          Changed all raise Exception, value statements to raise Exception(value) for
+          forward compatibility.
+
+11/26/08: beazley
+          Removed all print statements from lex and yacc, using sys.stdout and sys.stderr
+          directly.  Preparation for Python 3.0 support.
+
+11/04/08: beazley
+          Fixed a bug with referring to symbols on the the parsing stack using negative
+          indices.
+
+05/29/08: beazley
+          Completely revamped the testing system to use the unittest module for everything.
+          Added additional tests to cover new errors/warnings.
+
+Version 2.5
+-----------------------------
+05/28/08: beazley
+          Fixed a bug with writing lex-tables in optimized mode and start states.
+          Reported by Kevin Henry.
+
+Version 2.4
+-----------------------------
+05/04/08: beazley
+          A version number is now embedded in the table file signature so that 
+          yacc can more gracefully accomodate changes to the output format
+          in the future.
+
+05/04/08: beazley
+          Removed undocumented .pushback() method on grammar productions.  I'm
+          not sure this ever worked and can't recall ever using it.  Might have
+          been an abandoned idea that never really got fleshed out.  This
+          feature was never described or tested so removing it is hopefully
+          harmless.
+
+05/04/08: beazley
+          Added extra error checking to yacc() to detect precedence rules defined
+          for undefined terminal symbols.   This allows yacc() to detect a potential
+          problem that can be really tricky to debug if no warning message or error
+          message is generated about it.
+
+05/04/08: beazley
+          lex() now has an outputdir that can specify the output directory for
+          tables when running in optimize mode.  For example:
+
+             lexer = lex.lex(optimize=True, lextab="ltab", outputdir="foo/bar")
+ 
+          The behavior of specifying a table module and output directory are
+          more aligned with the behavior of yacc().
+          
+05/04/08: beazley
+          [Issue 9]
+          Fixed filename bug in when specifying the modulename in lex() and yacc(). 
+          If you specified options such as the following:
+
+             parser = yacc.yacc(tabmodule="foo.bar.parsetab",outputdir="foo/bar")
+
+          yacc would create a file "foo.bar.parsetab.py" in the given directory.
+          Now, it simply generates a file "parsetab.py" in that directory. 
+          Bug reported by cptbinho.
+
+05/04/08: beazley
+          Slight modification to lex() and yacc() to allow their table files
+	  to be loaded from a previously loaded module.   This might make
+	  it easier to load the parsing tables from a complicated package
+          structure.  For example:
+
+	       import foo.bar.spam.parsetab as parsetab
+               parser = yacc.yacc(tabmodule=parsetab)
+
+          Note:  lex and yacc will never regenerate the table file if used
+          in the form---you will get a warning message instead. 
+          This idea suggested by Brian Clapper.
+
+
+04/28/08: beazley
+          Fixed a big with p_error() functions being picked up correctly
+          when running in yacc(optimize=1) mode.  Patch contributed by
+          Bart Whiteley.
+
+02/28/08: beazley
+          Fixed a bug with 'nonassoc' precedence rules.   Basically the
+          non-precedence was being ignored and not producing the correct
+          run-time behavior in the parser.
+
+02/16/08: beazley
+          Slight relaxation of what the input() method to a lexer will
+          accept as a string.   Instead of testing the input to see
+          if the input is a string or unicode string, it checks to see
+          if the input object looks like it contains string data.
+          This change makes it possible to pass string-like objects
+          in as input.  For example, the object returned by mmap.
+
+              import mmap, os
+              data = mmap.mmap(os.open(filename,os.O_RDONLY),
+                               os.path.getsize(filename),
+                               access=mmap.ACCESS_READ)
+              lexer.input(data)
+           
+
+11/29/07: beazley
+          Modification of ply.lex to allow token functions to aliased.
+          This is subtle, but it makes it easier to create libraries and
+          to reuse token specifications.  For example, suppose you defined
+          a function like this:
+
+               def number(t):
+                    r'\d+'
+                    t.value = int(t.value)
+                    return t
+
+          This change would allow you to define a token rule as follows:
+
+              t_NUMBER = number
+
+          In this case, the token type will be set to 'NUMBER' and use
+          the associated number() function to process tokens.   
+           
+11/28/07: beazley
+          Slight modification to lex and yacc to grab symbols from both
+          the local and global dictionaries of the caller.   This
+          modification allows lexers and parsers to be defined using
+          inner functions and closures.
+
+11/28/07: beazley
+          Performance optimization:  The lexer.lexmatch and t.lexer
+          attributes are no longer set for lexer tokens that are not
+          defined by functions.   The only normal use of these attributes
+          would be in lexer rules that need to perform some kind of 
+          special processing.  Thus, it doesn't make any sense to set
+          them on every token.
+
+          *** POTENTIAL INCOMPATIBILITY ***  This might break code
+          that is mucking around with internal lexer state in some
+          sort of magical way.
+
+11/27/07: beazley
+          Added the ability to put the parser into error-handling mode
+          from within a normal production.   To do this, simply raise
+          a yacc.SyntaxError exception like this:
+
+          def p_some_production(p):
+              'some_production : prod1 prod2'
+              ...
+              raise yacc.SyntaxError      # Signal an error
+
+          A number of things happen after this occurs:
+
+          - The last symbol shifted onto the symbol stack is discarded
+            and parser state backed up to what it was before the
+            the rule reduction.
+
+          - The current lookahead symbol is saved and replaced by
+            the 'error' symbol.
+
+          - The parser enters error recovery mode where it tries
+            to either reduce the 'error' rule or it starts 
+            discarding items off of the stack until the parser
+            resets.
+
+          When an error is manually set, the parser does *not* call 
+          the p_error() function (if any is defined).
+          *** NEW FEATURE *** Suggested on the mailing list
+
+11/27/07: beazley
+          Fixed structure bug in examples/ansic.  Reported by Dion Blazakis.
+
+11/27/07: beazley
+          Fixed a bug in the lexer related to start conditions and ignored
+          token rules.  If a rule was defined that changed state, but
+          returned no token, the lexer could be left in an inconsistent
+          state.  Reported by 
+          
+11/27/07: beazley
+          Modified setup.py to support Python Eggs.   Patch contributed by
+          Simon Cross.
+
+11/09/07: beazely
+          Fixed a bug in error handling in yacc.  If a syntax error occurred and the
+          parser rolled the entire parse stack back, the parser would be left in in
+          inconsistent state that would cause it to trigger incorrect actions on
+          subsequent input.  Reported by Ton Biegstraaten, Justin King, and others.
+
+11/09/07: beazley
+          Fixed a bug when passing empty input strings to yacc.parse().   This 
+          would result in an error message about "No input given".  Reported
+          by Andrew Dalke.
+
 Version 2.3
 -----------------------------
 02/20/07: beazley