summaryrefslogtreecommitdiff
path: root/testing/tools/run_corpus_tests.py
AgeCommit message (Collapse)Author
2015-11-03Revert "One test runner to rule them all."Dan Sinclair
This reverts commit a1215ba51a235fb7abcb995f0e768ea0176d9275. Revert "Return result of the test runner." This reverts commit d1579c9b92b7f9c1d9e0fac1ecb8e3cb23875fca. Reverting the new test_runner code as there appears to be something string happening on the non-Linux boxes on XFA. It looks like it ran the font_size test twice, once with the .in and once with the .pdf. The .in was suppressed, the .pdf faild and wasn't suppressed. Rendering PDF file /Volumes/data/b/build/slave/mac_xfa/build/pdfium/out/Debug/gen/pdfium/testing/pixel/font_size.pdf. Rendered 1 pages. Skipped 0 bad pages. Checking /Volumes/data/b/build/slave/mac_xfa/build/pdfium/out/Debug/gen/pdfium/testing/pixel/font_size.pdf.0.png FAILURE: font_size.in; Command '['/Volumes/data/b/build/slave/mac_xfa/build/pdfium/out/Debug/pdfium_diff', '/Volumes/data/b/build/slave/mac_xfa/build/pdfium/testing/resources/pixel/font_size_expected.pdf.0.png', '/Volumes/data/b/build/slave/mac_xfa/build/pdfium/out/Debug/gen/pdfium/testing/pixel/font_size.pdf.0.png']' returned non-zero exit status 1 font_size.in is suppressed, found in SUPPRESSIONS_mac file Rendering PDF file /Volumes/data/b/build/slave/mac_xfa/build/pdfium/out/Debug/gen/pdfium/testing/pixel/font_size.pdf. Rendered 1 pages. Skipped 0 bad pages. Checking /Volumes/data/b/build/slave/mac_xfa/build/pdfium/out/Debug/gen/pdfium/testing/pixel/font_size.pdf.0.png FAILURE: font_size.pdf; Command '['/Volumes/data/b/build/slave/mac_xfa/build/pdfium/out/Debug/pdfium_diff', '/Volumes/data/b/build/slave/mac_xfa/build/pdfium/testing/resources/pixel/font_size_expected.pdf.0.png', '/Volumes/data/b/build/slave/mac_xfa/build/pdfium/out/Debug/gen/pdfium/testing/pixel/font_size.pdf.0.png']' returned non-zero exit status 1 TBR=thestig@chromium.org Review URL: https://codereview.chromium.org/1410743008 .
2015-11-03Return result of the test runner.Dan Sinclair
We need to actually return the result of the test runner.... R=thestig@chromium.org Review URL: https://codereview.chromium.org/1413863011 .
2015-11-03One test runner to rule them all.Dan Sinclair
This CL takes the three test runners (corpus, javascript, pixel) and combines the code into a single test_runner file. Each of the individual runners still exists and calls the test runner with their data directory. With this change, the pixel and javascript test will now run in parallel if multiple processors are available. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1430623006 .
2015-10-28Allow running individual tests.Dan Sinclair
This CL adds the ability to run a given test from the corpus, javascript and pixel test runners. The filename to provide is relative to the testing directory in question. Because the directories for javascript and pixel are flat you just provide the filename (it will rewrite the .pdf to .in if .pdf is provided). For corpus tests you have to provide the path from the corpus directory. Development/pdfium/pdfium % ./testing/tools/run_javascript_tests.py apply.pdf Rendering PDF file /Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/javascript/apply.pdf. Non-linearized path... Rendered 1 pages. Skipped 0 bad pages. Development/pdfium/pdfium % ./testing/tools/run_pixel_tests.py bug_524043_1.pdf Rendering PDF file /Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/pixel/bug_524043_1.pdf. Linearized path... Rendered 1 pages. Skipped 0 bad pages. Checking /Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/pixel/bug_524043_1.pdf.0.png diff: 0.00% passed Development/pdfium/pdfium % ./testing/tools/run_corpus_tests.py third_party/tcpdf/example_065.pdf Rendering PDF file /Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/corpus/example_065.pdf. Non-linearized path... Rendered 1 pages. Skipped 0 bad pages. Checking /Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/corpus/example_065.pdf.0.png diff: 0.14% failed FAILURE: example_065.pdf; Command '['/Development/pdfium/pdfium/out/Debug/pdfium_diff', '/Development/pdfium/pdfium/testing/corpus/third_party/tcpdf/example_065_expected.pdf.0.png', '/Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/corpus/example_065.pdf.0.png']' returned non-zero exit status 1 Summary of Failures: /Development/pdfium/pdfium/testing/corpus/third_party/tcpdf/example_065.pdf R=thestig@chromium.org Review URL: https://codereview.chromium.org/1407913005 .
2015-10-22Make parallelized run_corpus_tests.py handle ctrl-c.Lei Zhang
Thanks Stack Overflow! R=tsepez@chromium.org Review URL: https://codereview.chromium.org/1421623002 .
2015-10-09Parallelize run_corpus_tests.py.Lei Zhang
- Use the number of cores as the default -j value - Fall back to old code for -j 1 R=nparker@chromium.org Review URL: https://codereview.chromium.org/1398793003 .
2015-09-09Remove existing generated images before running a test.Lei Zhang
R=tsepez@chromium.org Review URL: https://codereview.chromium.org/1314443007 .
2015-08-13Revert "Allow external font-path configuration from pdfium_test."Tom Sepez
This reverts commit d8b5e73d8609b74e6a995ee1768d20d47bd4b089. Broke corpus tests TBR=thestig@chromium.org Review URL: https://codereview.chromium.org/1292153002 .
2015-08-13Allow external font-path configuration from pdfium_test.Tom Sepez
R=thestig@chromium.org Review URL: https://codereview.chromium.org/1268323004 .
2015-07-16Fix typo from commit d3611f.Lei Zhang
TBR=tsepez@chromium.org Review URL: https://codereview.chromium.org/1242493003 .
2015-07-16Sort unexpected test results.Lei Zhang
R=tsepez@chromium.org Review URL: https://codereview.chromium.org/1232203003 .
2015-06-12Corpus tests check for unexpected successes.Tom Sepez
Update run_corpus_tests.py to always run all test cases, and just ignore the results that are supposed to fail. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1178393002.
2015-05-20Get test running scripts to detect and report common error.Bruce Dawson
testing/tools/run_corpus_tests.py assumes a debug build and will fail cryptically if only a release build is available. Arguably there shouldn't be a default because having one could lead to accidentally running a stale version, but that is probably too much of a change. R=tsepez@chromium.org Review URL: https://codereview.chromium.org/1150823003
2015-04-03Refactor PDFium python test utilities.Tom Sepez
Extract a common portions for determining suppressions and comparing pngs. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1057983003
2015-03-26Replace linux-specific code in test scripts.Tom Sepez
Now that there's a win bot, this needs to be more careful. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1036073002
2015-03-25Create per-platform pdfium test suppression files.Tom Sepez
This is required now that we have win/mac bots, which may produce different outputs. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1031203003
2015-03-23Support comments and blank lines in SUPPRESSIONS fileTom Sepez
The comment character is #. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1029193002
2015-03-23Create simple suppressions file in PDFium repository.Tom Sepez
This is needed to green the tree without continually reverting the pdfium_tests (corpus) repository. We will need to make this more sophisticated at some point, but for now, let's try to green the buildbot. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1016613004
2015-03-23Flush stdout before launching sub-processes.Tom Sepez
This should make the test logs more readable. Also give failure summary at end of tests, since searching through the log is tedious. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1026903002
2015-03-17Return OS status code from python test driver scripts.Tom Sepez
Consequently, some of the tests may diff but the waterfall remains green. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1014083003
2015-03-16Pull our new test case repository via deps.Tom Sepez
Add a run_corpus_tests.py script to run pdfium_test against the corpus tree. Note that this differs from the run_pixel_tests.py script, since pre-processing is not required. I'll work on unifying these in a subsequent CL. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1010833003
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-20 02:07:44 +0000