summaryrefslogtreecommitdiff
path: root/tests
AgeCommit message (Collapse)Author
2016-08-02stats: update referencesCurtis Dunham
2016-07-22tests: Add regex-based ignore rules for ref filesAndreas Sandberg
There are cases where we need to ignore files with specific extensions (e.g., when Mercurial litters the file system with patch rejects). Implement this functionality using a helper class (FileIgnoreList) that supports both regular expressions and basic string comparisons. Change-Id: I34549754bd2e10ed230ffb2dc057403349f8fa78 Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
2016-07-21stats: update referencesCurtis Dunham
2016-07-01mem: tester for new HMC configurationAbdul Mutaal Ahmad
This patch provides the example test script to configure different HMC architecture and run traffic through traffic generator. Committed by Jason Lowe-Power <jason@lowepower.com>
2016-06-21stats: Update stats to reflect ARM changesAndreas Sandberg
2016-06-20tests: Add a test command to get test status as an exit codeAndreas Sandberg
Add a "test" command to tests.py that queries a test pickle file and returns different exit codes depending on the outcome of the tests in the file. The following exit codes can currently be returned: * 0: All tests were successful or skipped. * 1: General fault in the script such as incorrect parameters or failing to parse a pickle file. * 2: At least one test failed to run. This is what the summary formatter usually shows as a 'FAILED'. * 3: All tests ran correctly, but at least one failed to verify its output. When displaying test output using the summary formatter, such a test would show up as 'CHANGED'. The command can be invoked like this: ./tests/tests.py test `find build/ARM/tests/opt/ -name status.pickle` Change-Id: I7e6bc661516f38ff08dfda7c4359a1e10bf97864 Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com> Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
2016-06-20tests: Split test results into running and verificationAndreas Sandberg
The test base class already assumes that test cases consists of a run stage and a verification stage. Reflect this in the results class to make it possible to detect cases where a run was successful, but didn't verify. Change-Id: I31ef393e496671221c5408aca41649cd8dda74ca Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com> Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
2016-06-20mem: Resolve TrafficGen trace relative to the configAndreas Sandberg
The traffic generator currently resolves relative trace paths relative to gem5's current working directory. This can lead to surprising results for relative paths where the expectation would normally be that they are resolved relative to the configuration file. This changeset implements config-relative trace file lookups. The old behavior is kept as a fallback for configs that expect that behavior. Change-Id: I1bda4e16725842666ffc37dcb6838c23a6ff138c Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com> Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
2016-06-12stats: update EIO statsSteve Reinhardt
2016-06-06stats: Add power stats to test referencesAndreas Sandberg
Change-Id: Ic827213134b199446822f128b81d4a480e777fee
2016-06-06stats: update EIO statsSteve Reinhardt
2016-06-02stats: Update ref stats to match ARM TLB changesAndreas Sandberg
2016-06-02tests: Remove working dir assumption in tgen testsAndreas Sandberg
The traffic generator tests currently assume that they are run from the root of the source directory. This sometimes breaks tests when they are run using the new test framework. Change-Id: I6538a7902694c5d2c980295e076ea1c09acc4291 Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com> Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
2016-06-02stats: Update to match ARM ISA changesAndreas Sandberg
2016-06-02tests: add 'CHANGED' output to pickle viewerCurtis Dunham
Change-Id: I64c69fde8657c273adea69122877c5348a4f867a Reviewed-by: Andreas Sandberg <andreas.sandberg@arm.com>
2016-06-02tests: Only run Ruby tests when testing Ruby targetsAndreas Sandberg
Limit the test configs to Ruby-only configs when testing a Ruby target that isn't MI_example. This avoids re-running configs that has already been tested by the generic (non-Ruby) ISA target. This behavior was the expected behavior prior to switching to the new test framework. Change-Id: I3f138dbf9c7071ce862d1073aaec57c59afbc921 Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com> Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
2016-05-31stats: update and fix e273e86a873dCurtis Dunham
2016-05-31scons: Use the new test framework from sconsAndreas Sandberg
Rewrite the SCons script responsible for launching tests to use the new test framework. The biggest visible change after this changeset is that SCons no longer produces a "status" file in test build directories. Instead, it creates a status.pickle file. That file can be inspected using the new tests.py script. Another visible change is that timed out tests are now reported as failed rather than a using a separate summary message. Since the pickle file will remain in the build directory after running tests, it's now possible to convert the test results into other formats. For example: ./tests/tests.py show --format junit -o junit.xml \ `find build/ -name status.pickle` To better facilitate running outside of scons, there is now a set of targets that generate test listings that can be used by the test script. There are currently three targets, quick.list, long.list, and all.list. For example: scons build/ARM/tests/opt/all.list for F in `cat build/ARM/tests/opt/all.list`; do ./tests/tests.py run build/ARM/gem5.opt $F done Change-Id: I2c0f8ca0080d7af737362e198eda4cb3a72e6c36 Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com> Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
2016-05-31tests: Fix incorrect stat.txt ignore when updating refsAndreas Sandberg
ClassicTest was incorrectly ignoring stats.txt when updating reference statistics. This was caused by ignore rules being applied too aggressively when listing reference files. This changeset splits the ignore rules into two different lists: 1) diff_ignore_files that lists the files that shouldn't be diff:ed using the normal diff tool, and 2) ref_ignore_files which lists files that should be ignored by the test system. Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
2016-05-31stats: update for snoop filter tweakCurtis Dunham
--HG-- extra : source : 2323557eb4f4866fa1ea1575a9f5969e0022adc1
2016-05-27mem, config: Selective use of snoop filterStephan Diestelhorst
Disable the default snoop filter in the SystemXBar so that the typical membus does not have a snoop filter by default. Instead, add the snoop filter only when there are caches added to the system (with the caches / l2cache options). The underlying problem is that the snoop filter grows without bounds (for now) if there are no caches to tell it that lines have been evicted. This causes slow regression runs for all the atomic regressions. This patch fixes this behaviour. --HG-- extra : source : f97c20511828209757440839ed48d741d02d428f
2016-05-26tests: Add test infrastructure as a Python moduleAndreas Sandberg
Implement gem5's test infrastructure as a Python module and a run script that can be used without scons. The new implementation has several features that were lacking from the previous test infrastructure such as support for multiple output formats, automatic runtime tracking, and better support for being run in a cluster environment. Tests consist of one or more steps (TestUnit). Units are run in two stages, the first a run stage and then a verify stage. Units in the verify stage are automatically skipped if any unit run stage wasn't run. The library currently contains TestUnit implementations that run gem5, diff stat files, and diff output files. Existing tests are implemented by the ClassicTest class and "just work". New tests can that don't rely on the old "run gem5 once and diff output" strategy can be implemented by subclassing the Test base class or ClassicTest. Test results can be output in multiple formats. The module currently supports JUnit, text (short and verbose), and Python's pickle format. JUnit output allows CI systems to automatically get more information about test failures. The pickled output contains all state necessary to reconstruct a tests results object and is mainly intended for the build system and CI systems. Since many JUnit parsers parsers assume that test suite names look like Java package names. We currently output path-like names with slashes separating components. Test names are translated according to these rules: * '.' -> '-" * '/' -> '.' The test tool, tests.py, supports the following features: * Test listing. Example: ./tests.py list arm/quick * Running tests. Example: ./tests.py run -o output.pickle --format pickle \ ../build/ARM/gem5.opt \ quick/se/00.hello/arm/linux/simple-timing * Displaying pickled results. Example: ./tests.py show --format summary *.pickle Change-Id: I527164bd791237aacfc65e7d7c0b67b695c5d17c Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com> Reviewed-by: Curtis Dunham <curtis.dunham@arm.com> Reviewed-by: Joel Hestness <jthestness@gmail.com>
2016-05-09tests: Enable test running outside of gem5's source treeAndreas Sandberg
The learning gem5 scripts currently assumes that the current working directory is the root of gem5's source tree. This isn't necessarily the case when running the tests using gem5's new test runner. Change-Id: Ief569bbe77b1b3e2b0fb0e6c575fb0705bbba9b3 Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com> Reviewed-by: Curtis Dunham <curtis.dunham@arm.com>
2016-05-07tests: update EIO ref stats for removed cache statsSteve Reinhardt
Complaints about changes in EIO tests were due to reference files that still have removed cache stats from cset 11454:e55afadc4e19.
2016-04-28tests: Remove stale reference output filesAndreas Sandberg
Remove test reference files that are not generated any more: * chair.cook.ppm: This file should be generated by eon and not mcf, so it shouldn't be included as an output from mcf. * system.pc.terminal: The terminal device has been renamed so this file is no longer generated. Change-Id: I3962efe1ff25479ca276115f7564eccb5fac8cf9 Signed-off-by: Andreas Sandberg <andreas.sandberg@arm.com>
2016-04-25tests: Add a basic memcheck regressionAndreas Hansson
This patch adds a simple regression that calls the existing memcheck.py script. --HG-- rename : tests/configs/learning-gem5-p1-simple.py => tests/configs/memcheck.py rename : tests/quick/se/70.tgen/test.py => tests/quick/se/51.memcheck/test.py
2016-04-21tests: Update learning gem5 tests scripts with copyrightJason Power
2016-04-21stats: Update stats to reflect cache changesAndreas Hansson
Removed unused stats, now counting WriteLineReq, and changed how uncacheable writes are handled while responses are outstanding.
2016-04-09stats: Match current behaviourAndreas Hansson
Small changes to the branch predictor and BTB caused stats changes throughout.
2016-04-08stats: update stats for thermals, indirect BPCurtis Dunham
2016-03-17stats: update stats for ld.so supportSteve Reinhardt
Additional auxv entries leads to more instructions in start-up while walking the list, along with different cache conflicts wrt stack entries.
2016-03-17stats: update stats for mmap changesSteve Reinhardt
2016-03-17stats: update stats for mmap() change.Steve Reinhardt
SE O3 runs see an additional reg read per mmap() call.
2016-03-17stats: Bump stats to match cache changesAndreas Hansson
Update stats to match current behaviour. As a result of the earlier conflict check we are seeing a few prefetch requests being ignored before being sent as upward snoops.
2016-03-16stats: overdue updates to long regressionsSteve Reinhardt
2016-02-18stats: update gpu-ruby-GPU_RfO statsSteve Reinhardt
Output changed way back in this cset: changeset: 11345:b6a66a90e0a1 user: John Kalamatianos <john.kalamatianos@amd.com> summary: gpu: fix bugs with MemFence, Flat Instrs and Resource utilization
2015-07-19cpu: Fix LLSC atomic CPU wakeupKrishnendra Nathella
Writes to locked memory addresses (LLSC) did not wake up the locking CPU. This can lead to deadlocks on multi-core runs. In AtomicSimpleCPU, recvAtomicSnoop was checking if the incoming packet was an invalidation (isInvalidate) and only then handled a locked snoop. But, writes are seen instead of invalidates when running without caches (fast-forward configurations). As as simple fix, now handleLockedSnoop is also called even if the incoming snoop packet are from writes.
2016-02-24stats: Update stats to reflect forwarding of InvalidateReqAndreas Hansson
2016-02-24cpu: TraceGen fix for tick frequency checkMatteo Andreozzi
Bug fix for check on protobuf file frequency being different than global frequency. The ASCII encoder script is also fixed, and the example trace used in the regressions is updated.
2016-02-10stats: Update stats to reflect changes to cache and crossbarAndreas Hansson
2016-02-06style: fix missing spaces in control statementsSteve Reinhardt
Result of running 'hg m5style --skip-all --fix-control -a'.
2016-02-06style: remove trailing whitespaceSteve Reinhardt
Result of running 'hg m5style --skip-all --fix-white -a'.
2016-02-06stats: update EIO stats for recent changesSteve Reinhardt
2016-01-22stats: update stats to after GPU checkinTony Gutierrez
2016-01-22ruby: changed all references to numCPs to num-cpBrad Beckmann
2016-01-19gpu-compute: AMD's baseline GPU modelTony Gutierrez
2016-01-17stats: update SPARC FS statsSteve Reinhardt
The fs/80.solaris-boot/sparc/solaris/t1000-simple-atomic test was broken for so long that, now that it's working again, the stats output is out of date. This changeset updates the outputs, on the assumption that the stats changes are all valid differences due to other changes made while it was broken.
2015-12-30stats: more updates due to PCI changesSteve Reinhardt
A couple of the long regressions have been showing as CHANGED since 11244:a2af58a06c4e despite the updates in 11245:1c5102c0a7a9. The x86 regression looks like it was just missed, but it's not clear why the ARM one is giving different results (perhaps a non-determinism between zizzer and wherever the updated results were run?).
2015-12-28tests: update EIO reference outputsSteve Reinhardt
2015-12-12stats: bump stats to reflect ruby tester changesAnthony Gutierrez