| Age | Commit message (Collapse) | Author |
|---|
|
Use the PyBind11 wrapping infrastructure instead of SWIG to generate
wrappers for functionality that needs to be exported to Python. This
has several benefits:
* PyBind11 can be redistributed with gem5, which means that we have
full control of the version used. This avoid a large number of
hard-to-debug SWIG issues we have seen in the past.
* PyBind11 doesn't rely on a custom C++ parser, instead it relies on
wrappers being explicitly declared in C++. The leads to slightly
more boiler-plate code in manually created wrappers, but doesn't
doesn't increase the overall code size. A big benefit is that this
avoids strange compilation errors when SWIG doesn't understand
modern language features.
* Unlike SWIG, there is no risk that the wrapper code incorporates
incorrect type casts (this has happened on numerous occasions in
the past) since these will result in compile-time errors.
As a part of this change, the mechanism to define exported methods has
been redesigned slightly. New methods can be exported either by
declaring them in the SimObject declaration and decorating them with
the cxxMethod decorator or by adding an instance of
PyBindMethod/PyBindProperty to the cxx_exports class variable. The
decorator has the added benefit of making it possible to add a
docstring and naming the method's parameters.
The new wrappers have the following known issues:
* Global events can't be memory managed correctly. This was the
case in SWIG as well.
Change-Id: I88c5a95b6cf6c32fa9e1ad31dfc08b2e8199a763
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Andreas Hansson <andreas.hansson@arm.com>
Reviewed-by: Andrew Bardsley <andrew.bardsley@arm.com>
Reviewed-on: https://gem5-review.googlesource.com/2231
Reviewed-by: Tony Gutierrez <anthony.gutierrez@amd.com>
Reviewed-by: Pierre-Yves Péneau <pierre-yves.peneau@lirmm.fr>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
|
|
Swig wrappers for native objects currently share the _m5.internal name
space with Python code. This is undesirable if we ever want to switch
from Swig to some other framework for native binding (e.g., PyBind11
or Boost::Python). This changeset moves all of such wrappers to the
_m5 namespace, which is now reserved for native code.
Change-Id: I2d2bc12dbc05b57b7c5a75f072e08124413d77f3
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
Reviewed-by: Jason Lowe-Power <jason@lowepower.com>
|
|
The style refactor change (style: Refactor the style checker as a
Python package) moved region.py from src/python/m5/util/ to
util/style/. The SConscript update accidentally got lost in that
commit. This commit removes region.py from src/python/SConscript.
Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
--HG--
extra : amend_source : f69b75bf636dd4a4232af3e10c29f7eaa4d59dc8
|
|
Add the ability to build libgem5 without embedded Python or the
ability to configure with Python.
This is a prelude to a patch to allow config.ini files to be loaded
into libgem5 using only C++ which would make embedding gem5 within
other simulation systems easier.
This adds a few registration interfaces to things which cross
between Python and C++. Namely: stats dumping and SimObject resolving
|
|
This changeset adds a SWIG interface for the Serializable class, which
fixes a warning when compiling the SWIG interface for the event
queue. Currently, the only method exported is the name() method.
|
|
This patch moves the draining interface from SimObject to a separate
class that can be used by any object needing draining. However,
objects not visible to the Python code (i.e., objects not deriving
from SimObject) still depend on their parents informing them when to
drain. This patch also gets rid of the CountedDrainEvent (which isn't
really an event) and replaces it with a DrainManager.
|
|
Revised system visualization to reflect structure and memory hierarchy.
Improved visualization: less congested and cluttered; more colorful.
Nodes reflect components; directed edges reflect dirctional relation, from
a master port to a slave port. Requires pydot.
|
|
Replace the (broken as of previous changeset) swig_objdecl() method
that allowed/forced you to substitute a whole new C++ struct
definition for SWIG to wrap with a set of export_method* hooks
that let you just declare a set of C++ methods (or other declarations)
that get inserted in the auto-generated struct.
Restore the System get/setMemoryMode methods, and use this mechanism
to specialize SimObject as well, eliminating teh need for sim_object.i.
Needed bits of sim_object.i are moved to the new pyobject.i.
Also sucked a little SimObject specialization into cxx_param_decl()
allowing us to get rid of src/sim/sim_object_params.hh. Now the
generation and wrapping of the base SimObject param struct is more
in line with how derived objects are handled.
--HG--
rename : src/python/swig/sim_object.i => src/python/swig/pyobject.i
|
|
Build a python list and dict of all stats and expose flags properly.
--HG--
rename : src/python/m5/stats.py => src/python/m5/stats/__init__.py
|
|
This is basically like the range_map stuff in src/base (range already
exists in Python). This code is like a set of ranges. I'm using it
to keep track of changed lines in source code, but it could be use to
keep track of memory ranges and holes in memory regions. It could
also be used in memory allocation type stuff. (Though it's not at all
optimized.)
|
|
Thanks to swig this was interfering with the standard Python
random module. The only function in that module was seed(),
which erroneously called srand48(). Moved the function to
m5.internal.core, renamed it seedRandom(), and made it call
random_mt.init() instead.
|
|
I like the brevity of Ali's recent change, but the ambiguity of
sometimes showing the source and sometimes the target is a little
confusing. This patch makes scons typically list all sources and
all targets for each action, with the common path prefix factored
out for brevity. It's a little more verbose now but also more
informative.
Somehow Ali talked me into adding colors too, which is a whole
'nother story.
|
|
Move generated enums into internal.params, which gets
imported into object.params, restoring backward
compatibility for scripts that expect to find them there.
|
|
This is necessary because versions of swig older than 1.3.39 fail to
do the right thing and try to do relative imports for everything (even
with the package= option to %module). Instead of putting params in
the m5.internal.params package, put params in the m5.internal package
and make all param modules start with param_. Same thing for
m5.internal.enums.
Also, stop importing all generated params into m5.objects. They are
not necessary and now with everything using relative imports we wound
up with pollution of the namespace (where builtin-range got overridden).
--HG--
rename : src/python/m5/internal/enums/__init__.py => src/python/m5/internal/enums.py
rename : src/python/m5/internal/params/__init__.py => src/python/m5/internal/params.py
|
|
kill params.i and create a separate .i for each object (param, enums, etc.)
|
|
Instead of putting all object files into m5/object/__init__.py, interrogate
the importer to find out what should be imported.
Instead of creating a single file that lists all of the embedded python
modules, use static object construction to put those objects onto a list.
Do something similar for embedded swig (C++) code.
|
|
It would be nice if python had a tree class that would do this for real,
but since we don't, we'll just keep a sorted list of keys and update
it on demand.
|
|
Get rid of misc.py and just stick misc things in __init__.py
Move utility functions out of SCons files and into m5.util
Move utility type stuff from m5/__init__.py to m5/util/__init__.py
Remove buildEnv from m5 and allow access only from m5.defines
Rename AddToPath to addToPath while we're moving it to m5.util
Rename read_command to readCommand while we're moving it
Rename compare_versions to compareVersions while we're moving it.
--HG--
rename : src/python/m5/convert.py => src/python/m5/util/convert.py
rename : src/python/m5/smartdict.py => src/python/m5/util/smartdict.py
|
|
|
|
|
|
|
|
|
|
Also clean things up so that help strings can more easily be added.
Move the help function into trace.py
|
|
This should allow m5 to be more easily embedded into other simulators.
The m5 binary adds a simple main function which then calls into the m5
libarary to start the simulation. In order to make this work
correctly, it was necessary embed python code directly into the
library instead of the zipfile hack. This is because you can't just
append the zipfile to the end of a library the way you can a binary.
As a result, Python files that are part of the m5 simulator are now
compile, marshalled, compressed, and then inserted into the library's
data section with a certain symbol name. Additionally, a new Importer
was needed to allow python to get at the embedded python code.
Small additional changes include:
- Get rid of the PYTHONHOME stuff since I don't think anyone ever used
it, and it just confuses things. Easy enough to add back if I'm wrong.
- Create a few new functions that are key to initializing and running
the simulator: initSignals, initM5Python, m5Main.
The original code for creating libm5 was inspired by a patch Michael
Adler, though the code here was done by me.
|
|
|
|
main.py so things are a bit more obvious.
|
|
they're all in the same place. This also involves having just one
jobfile.py and moving it into the utils directory to avoid
duplication. Lots of improvements to the utility as well.
--HG--
rename : src/python/m5/attrdict.py => src/python/m5/util/attrdict.py
rename : util/pbs/jobfile.py => src/python/m5/util/jobfile.py
rename : src/python/m5/util.py => src/python/m5/util/misc.py
rename : src/python/m5/multidict.py => src/python/m5/util/multidict.py
rename : util/stats/orderdict.py => src/python/m5/util/orderdict.py
|
|
Add support for declaring SimObjects to swig so their members can be wrapped.
Make sim_object.i only contain declarations for SimObject.
Create system.i to contain declarations for System.
Update python code to properly call the C++ given the new changes.
--HG--
extra : convert_revision : 82076ee69e8122d56e91b92d6767e356baae420a
|
|
creation and initialization now happens in python. Parameter objects
are generated and initialized by python. The .ini file is now solely for
debugging purposes and is not used in construction of the objects in any
way.
--HG--
extra : convert_revision : 7e722873e417cb3d696f2e34c35ff488b7bff4ed
|
|
the SConscript files so that only the objects that are
actually available in a given build are compiled in.
Remove a bunch of files that aren't used anymore.
--HG--
rename : src/python/m5/objects/AlphaTLB.py => src/arch/alpha/AlphaTLB.py
rename : src/python/m5/objects/SparcTLB.py => src/arch/sparc/SparcTLB.py
rename : src/python/m5/objects/BaseCPU.py => src/cpu/BaseCPU.py
rename : src/python/m5/objects/FuncUnit.py => src/cpu/FuncUnit.py
rename : src/python/m5/objects/IntrControl.py => src/cpu/IntrControl.py
rename : src/python/m5/objects/MemTest.py => src/cpu/memtest/MemTest.py
rename : src/python/m5/objects/FUPool.py => src/cpu/o3/FUPool.py
rename : src/python/m5/objects/FuncUnitConfig.py => src/cpu/o3/FuncUnitConfig.py
rename : src/python/m5/objects/O3CPU.py => src/cpu/o3/O3CPU.py
rename : src/python/m5/objects/OzoneCPU.py => src/cpu/ozone/OzoneCPU.py
rename : src/python/m5/objects/SimpleOzoneCPU.py => src/cpu/ozone/SimpleOzoneCPU.py
rename : src/python/m5/objects/BadDevice.py => src/dev/BadDevice.py
rename : src/python/m5/objects/Device.py => src/dev/Device.py
rename : src/python/m5/objects/DiskImage.py => src/dev/DiskImage.py
rename : src/python/m5/objects/Ethernet.py => src/dev/Ethernet.py
rename : src/python/m5/objects/Ide.py => src/dev/Ide.py
rename : src/python/m5/objects/Pci.py => src/dev/Pci.py
rename : src/python/m5/objects/Platform.py => src/dev/Platform.py
rename : src/python/m5/objects/SimConsole.py => src/dev/SimConsole.py
rename : src/python/m5/objects/SimpleDisk.py => src/dev/SimpleDisk.py
rename : src/python/m5/objects/Uart.py => src/dev/Uart.py
rename : src/python/m5/objects/AlphaConsole.py => src/dev/alpha/AlphaConsole.py
rename : src/python/m5/objects/Tsunami.py => src/dev/alpha/Tsunami.py
rename : src/python/m5/objects/T1000.py => src/dev/sparc/T1000.py
rename : src/python/m5/objects/Bridge.py => src/mem/Bridge.py
rename : src/python/m5/objects/Bus.py => src/mem/Bus.py
rename : src/python/m5/objects/MemObject.py => src/mem/MemObject.py
rename : src/python/m5/objects/PhysicalMemory.py => src/mem/PhysicalMemory.py
rename : src/python/m5/objects/BaseCache.py => src/mem/cache/BaseCache.py
rename : src/python/m5/objects/CoherenceProtocol.py => src/mem/cache/coherence/CoherenceProtocol.py
rename : src/python/m5/objects/Repl.py => src/mem/cache/tags/Repl.py
rename : src/python/m5/objects/Process.py => src/sim/Process.py
rename : src/python/m5/objects/Root.py => src/sim/Root.py
rename : src/python/m5/objects/System.py => src/sim/System.py
extra : convert_revision : 173f8764bafa8ef899198438fa5573874e407321
|
|
--HG--
extra : convert_revision : 788068dd4f4994d0016dba7e8705359d45a3a45c
|
|
encumbered directory
--HG--
extra : convert_revision : 7062ce81183b989f0d922b00d02433633474a854
|
|
and python code into m5 to allow swig an python code to
easily added by any SConscript instead of just the one in
src/python. This provides SwigSource and PySource for
adding new files to m5 (similar to Source for C++). Also
provides SimObject for including files that contain SimObject
information and build the m5.objects __init__.py file.
--HG--
extra : convert_revision : 38b50a0629846ef451ed02f96fe3633947df23eb
|
|
to be created
--HG--
extra : convert_revision : 826cc692614528f987c80c3410cb025190f0a4e0
|
|
automatic. The point is that now a subdirectory can be added
to the build process just by creating a SConscript file in it.
The process has two passes. On the first pass, all subdirs
of the root of the tree are searched for SConsopts files.
These files contain any command line options that ought to be
added for a particular subdirectory. On the second pass,
all subdirs of the src directory are searched for SConscript
files. These files describe how to build any given subdirectory.
I have added a Source() function. Any file (relative to the
directory in which the SConscript resides) passed to that
function is added to the build. Clean up everything to take
advantage of Source().
function is added to the list of files to be built.
--HG--
extra : convert_revision : 103f6b490d2eb224436688c89cdc015211c4fd30
|
|
so things are organized in a more sensible manner. Take apart
finalInit and expose the individual functions which are now
called from python. Make checkpointing a bit easier to use.
--HG--
extra : convert_revision : f470ddabbb47103e7b4734ef753c40089f2dcd9d
|
|
based on the swig modules that we have
--HG--
extra : convert_revision : 2fd12db39d46608a62b9df36c2b36189f1d2bc30
|
|
expose all of the relevant functionality to python. Clean
up the mysql code while we're at it.
--HG--
extra : convert_revision : 5b711202a5a452b8875ebefb136a156b65c24279
|
|
the script is invoked as main. This allows us to import traceflags.py
if we just want the list of available flags.
Embed traceflags.py into the zipfile so it can be accessed from the
python side of things. With this, print an error on invalid flags and
add --trace-help option that will print out the list of trace flags
that are compiled in. If a flag is prefixed with a '-', now that flag
will be disabled.
--HG--
extra : convert_revision : 2260a596b07d127c582ff73474dbbdb0583db524
|
|
We don't currently use randomness much, so I didn't go too far, but
in the future, we may want to actually expose the random number values
themselves to python. For now, I'll at least let you seed it.
While we're at it, clean up a clearly bad way for generating random
doubles.
--HG--
extra : convert_revision : df2aa8b58dd0d9c2a7c771668a760b2df8db1e11
|
|
access to enabling/disabling tracing. Command line is
unchanged except for the removal of --trace-cycle since
it's not so clear what that means.
--HG--
extra : convert_revision : c0164d92d3615d76d0c6acaabaafd92a9278212a
|
|
m5.internal.event.create(). It takes a python object and a
Tick and calls process() when the Tick occurs.
--HG--
extra : convert_revision : 5e4c9728982b206163ff51e6850a1497d85ad7a3
|
|
--HG--
extra : convert_revision : 3aaf540a9e314a88a8945579398f0d79aa85d5cf
|
|
the Debug param context
--HG--
extra : convert_revision : 40e9dcfa9faedbe0c90a43f908f20a7c14ded6a4
|
|
Rename cc_main to internal.main
--HG--
extra : convert_revision : e938005f600fbf8a43435e29426a948f4501f072
|
|
configs/test/fs.py:
configs/test/test.py:
update for the new way that m5 deals with options
src/python/SConscript:
Compile AUTHORS, LICENSE, README, and RELEASE_NOTES into the
python stuff.
src/python/m5/__init__.py:
redo the way options work.
Move them all to main.py
src/sim/main.cc:
Migrate more functionality for main() into python.
Namely option parsing
src/python/m5/attrdict.py:
A dictionary object that overrides attribute access to
do item access.
src/python/m5/main.py:
The new location for M5's option parsing, and the main()
routine to set up the simulation.
--HG--
extra : convert_revision : c86b87a9f508bde1994088e23fd470c7753ee4c1
|
|
clear from the Python side that this is the
interface to C++.
src/SConscript:
main_wrap.cc -> cc_main_wrap.cc
src/python/SConscript:
src/python/m5/__init__.py:
src/sim/main.cc:
s/main/cc_main/
src/python/m5/config.py:
s/main/cc_main/
Also directly import cc_main so we don't need
to put the "m5." in front all the time.
--HG--
extra : convert_revision : 755552f70cf671881ff31e476c677b95ef12950d
|
|
User script now invokes initialization and
simulation loop after building configuration.
These functions are exported from C++ to Python
using SWIG.
SConstruct:
Set up SWIG builder & scanner.
Set up symlinking of source files into build directory
(by not disabling the default behavior).
configs/test/test.py:
Rewrite to use new script-driven interface.
Include a sample option.
src/SConscript:
Set up symlinking of source files into build directory
(by not disabling the default behavior).
Add SWIG-generated main_wrap.cc to source list.
src/arch/SConscript:
Set up symlinking of source files into build directory
(by not disabling the default behavior).
src/arch/alpha/ev5.cc:
src/arch/alpha/isa/decoder.isa:
src/cpu/o3/alpha_cpu_impl.hh:
src/cpu/trace/opt_cpu.cc:
src/cpu/trace/trace_cpu.cc:
src/sim/pseudo_inst.cc:
src/sim/root.cc:
src/sim/serialize.cc:
src/sim/syscall_emul.cc:
SimExit() is now exitSimLoop().
src/cpu/base.cc:
SimExitEvent is now SimLoopExitEvent
src/python/SConscript:
Add SWIG build command for main.i.
Use python/m5 in build dir as source for zip archive...
easy now with file duplication enabled.
src/python/m5/__init__.py:
- Move copyright notice back to C++ so we can print
it right away, even for interactive sessions.
- Get rid of argument parsing code; just provide default
option descriptors for user script to call optparse with.
- Don't clutter m5 namespace by sucking in all of m5.config
and m5.objects.
- Move instantiate() function here from config.py.
src/python/m5/config.py:
- Move instantiate() function to __init__.py.
- Param.Foo deferred type lookups must use m5.objects
namespace now (not m5).
src/python/m5/objects/AlphaConsole.py:
src/python/m5/objects/AlphaFullCPU.py:
src/python/m5/objects/AlphaTLB.py:
src/python/m5/objects/BadDevice.py:
src/python/m5/objects/BaseCPU.py:
src/python/m5/objects/BaseCache.py:
src/python/m5/objects/Bridge.py:
src/python/m5/objects/Bus.py:
src/python/m5/objects/CoherenceProtocol.py:
src/python/m5/objects/Device.py:
src/python/m5/objects/DiskImage.py:
src/python/m5/objects/Ethernet.py:
src/python/m5/objects/Ide.py:
src/python/m5/objects/IntrControl.py:
src/python/m5/objects/MemObject.py:
src/python/m5/objects/MemTest.py:
src/python/m5/objects/Pci.py:
src/python/m5/objects/PhysicalMemory.py:
src/python/m5/objects/Platform.py:
src/python/m5/objects/Process.py:
src/python/m5/objects/Repl.py:
src/python/m5/objects/Root.py:
src/python/m5/objects/SimConsole.py:
src/python/m5/objects/SimpleDisk.py:
src/python/m5/objects/System.py:
src/python/m5/objects/Tsunami.py:
src/python/m5/objects/Uart.py:
Fix up imports (m5 namespace no longer includes m5.config).
src/sim/eventq.cc:
src/sim/eventq.hh:
Support for Python-called simulate() function:
- Use IsExitEvent flag to signal events that want
to exit the simulation loop gracefully (instead of
calling exit() to terminate the process).
- Modify interface to hand exit event object back to
caller so it can be inspected for cause.
src/sim/host.hh:
Add MaxTick constant.
src/sim/main.cc:
Move copyright notice back to C++ so we can print
it right away, even for interactive sessions.
Use PYTHONPATH environment var to set module path
(instead of clunky code injection method).
Move main control from here into Python:
- Separate initialization code and simulation loop
into separate functions callable from Python.
- Make Python interpreter invocation more pure (more
like directly invoking interpreter).
Add -i and -p flags (only options on binary itself;
other options processed by Python).
Import readline package when using interactive mode.
src/sim/sim_events.cc:
SimExitEvent is now SimLoopExitEvent, and uses
IsSimExit flag to terminate loop (instead of
exiting simulator process).
src/sim/sim_events.hh:
SimExitEvent is now SimLoopExitEvent, and uses
IsSimExit flag to terminate loop (instead of
exiting simulator process).
Get rid of a few unused constructors.
src/sim/sim_exit.hh:
SimExit() is now exitSimLoop().
Get rid of unused functions.
Add comments.
--HG--
extra : convert_revision : 280b0d671516b25545a6f24cefa64a68319ff3d4
|
|
--HG--
extra : convert_revision : 77f475b156d81c03a2811818fa23593d5615c685
|
|
Use embedded zip archive to carry Python code instead
of homegrown embedded string/file mechanism.
Do argument parsing in Python instead of C++.
SConstruct:
Add Python interpreter include path & library.
Define two new simple builders which copy &
concatenate files, respectively, for use by
the Python embedded zipfile code.
src/SConscript:
Encapsulate environment creation in a function.
Add code to append Python zip archive to final executable.
Eliminate references to obsolete files.
src/python/SConscript:
Rewrite to generate embedded zip archive of Python code
(replacing old "embedded string" mechanism).
src/python/m5/__init__.py:
Move main arg-parsing loop here (out of C++ main()).
src/python/m5/config.py:
Minor fix (version incompatibility?).
src/sim/main.cc:
Invoke embedded Python interpreter to parse args
and generate config.ini, replacing C++ arg parsing code.
--HG--
extra : convert_revision : 72d21236b2bee139ff39ba4cf031a4a1f8560029
|
