summaryrefslogtreecommitdiff
path: root/testing/tools
AgeCommit message (Collapse)Author
2015-11-23Enable Dr. Memory to run javascript, pixel, and corpus testschromium/2575chromium/2574Qin Zhao
- add DrMemoryWrapper in common.py to adjust Dr. Memory wrapper - add --wrapper option to run_*_tests.py for Dr. Mempry wrapper - update run_*_tests.py to handle Dr. Memory wrapper - add TestPDFiumTest in pdfium_tests.py to support run_*_tests.py - remove ValgrindTool in valgrind_tests.py R=thestig@chromium.org BUG=pdfium:238 Review URL: https://codereview.chromium.org/1464453003 .
2015-11-03Revert "One test runner to rule them all."Dan Sinclair
This reverts commit a1215ba51a235fb7abcb995f0e768ea0176d9275. Revert "Return result of the test runner." This reverts commit d1579c9b92b7f9c1d9e0fac1ecb8e3cb23875fca. Reverting the new test_runner code as there appears to be something string happening on the non-Linux boxes on XFA. It looks like it ran the font_size test twice, once with the .in and once with the .pdf. The .in was suppressed, the .pdf faild and wasn't suppressed. Rendering PDF file /Volumes/data/b/build/slave/mac_xfa/build/pdfium/out/Debug/gen/pdfium/testing/pixel/font_size.pdf. Rendered 1 pages. Skipped 0 bad pages. Checking /Volumes/data/b/build/slave/mac_xfa/build/pdfium/out/Debug/gen/pdfium/testing/pixel/font_size.pdf.0.png FAILURE: font_size.in; Command '['/Volumes/data/b/build/slave/mac_xfa/build/pdfium/out/Debug/pdfium_diff', '/Volumes/data/b/build/slave/mac_xfa/build/pdfium/testing/resources/pixel/font_size_expected.pdf.0.png', '/Volumes/data/b/build/slave/mac_xfa/build/pdfium/out/Debug/gen/pdfium/testing/pixel/font_size.pdf.0.png']' returned non-zero exit status 1 font_size.in is suppressed, found in SUPPRESSIONS_mac file Rendering PDF file /Volumes/data/b/build/slave/mac_xfa/build/pdfium/out/Debug/gen/pdfium/testing/pixel/font_size.pdf. Rendered 1 pages. Skipped 0 bad pages. Checking /Volumes/data/b/build/slave/mac_xfa/build/pdfium/out/Debug/gen/pdfium/testing/pixel/font_size.pdf.0.png FAILURE: font_size.pdf; Command '['/Volumes/data/b/build/slave/mac_xfa/build/pdfium/out/Debug/pdfium_diff', '/Volumes/data/b/build/slave/mac_xfa/build/pdfium/testing/resources/pixel/font_size_expected.pdf.0.png', '/Volumes/data/b/build/slave/mac_xfa/build/pdfium/out/Debug/gen/pdfium/testing/pixel/font_size.pdf.0.png']' returned non-zero exit status 1 TBR=thestig@chromium.org Review URL: https://codereview.chromium.org/1410743008 .
2015-11-03Return result of the test runner.Dan Sinclair
We need to actually return the result of the test runner.... R=thestig@chromium.org Review URL: https://codereview.chromium.org/1413863011 .
2015-11-03One test runner to rule them all.Dan Sinclair
This CL takes the three test runners (corpus, javascript, pixel) and combines the code into a single test_runner file. Each of the individual runners still exists and calls the test runner with their data directory. With this change, the pixel and javascript test will now run in parallel if multiple processors are available. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1430623006 .
2015-10-28Allow running individual tests.Dan Sinclair
This CL adds the ability to run a given test from the corpus, javascript and pixel test runners. The filename to provide is relative to the testing directory in question. Because the directories for javascript and pixel are flat you just provide the filename (it will rewrite the .pdf to .in if .pdf is provided). For corpus tests you have to provide the path from the corpus directory. Development/pdfium/pdfium % ./testing/tools/run_javascript_tests.py apply.pdf Rendering PDF file /Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/javascript/apply.pdf. Non-linearized path... Rendered 1 pages. Skipped 0 bad pages. Development/pdfium/pdfium % ./testing/tools/run_pixel_tests.py bug_524043_1.pdf Rendering PDF file /Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/pixel/bug_524043_1.pdf. Linearized path... Rendered 1 pages. Skipped 0 bad pages. Checking /Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/pixel/bug_524043_1.pdf.0.png diff: 0.00% passed Development/pdfium/pdfium % ./testing/tools/run_corpus_tests.py third_party/tcpdf/example_065.pdf Rendering PDF file /Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/corpus/example_065.pdf. Non-linearized path... Rendered 1 pages. Skipped 0 bad pages. Checking /Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/corpus/example_065.pdf.0.png diff: 0.14% failed FAILURE: example_065.pdf; Command '['/Development/pdfium/pdfium/out/Debug/pdfium_diff', '/Development/pdfium/pdfium/testing/corpus/third_party/tcpdf/example_065_expected.pdf.0.png', '/Development/pdfium/pdfium/out/Debug/gen/pdfium/testing/corpus/example_065.pdf.0.png']' returned non-zero exit status 1 Summary of Failures: /Development/pdfium/pdfium/testing/corpus/third_party/tcpdf/example_065.pdf R=thestig@chromium.org Review URL: https://codereview.chromium.org/1407913005 .
2015-10-22Make parallelized run_corpus_tests.py handle ctrl-c.Lei Zhang
Thanks Stack Overflow! R=tsepez@chromium.org Review URL: https://codereview.chromium.org/1421623002 .
2015-10-09Parallelize run_corpus_tests.py.Lei Zhang
- Use the number of cores as the default -j value - Fall back to old code for -j 1 R=nparker@chromium.org Review URL: https://codereview.chromium.org/1398793003 .
2015-09-09Remove existing generated images before running a test.Lei Zhang
R=tsepez@chromium.org Review URL: https://codereview.chromium.org/1314443007 .
2015-09-02Change fixup_pdf_template.py to open files in binary mode.Lei Zhang
Fix for the pixel test failure on Windows from commit 870b5b6793fa. Seems to work for me on Windows. TBR=tsepez@chromium.org Review URL: https://codereview.chromium.org/1312493006 .
2015-08-13Revert "Allow external font-path configuration from pdfium_test."Tom Sepez
This reverts commit d8b5e73d8609b74e6a995ee1768d20d47bd4b089. Broke corpus tests TBR=thestig@chromium.org Review URL: https://codereview.chromium.org/1292153002 .
2015-08-13Allow external font-path configuration from pdfium_test.Tom Sepez
R=thestig@chromium.org Review URL: https://codereview.chromium.org/1268323004 .
2015-08-04Add a license header for testing/tools/make_expected.sh.Lei Zhang
R=tsepez@chromium.org Review URL: https://codereview.chromium.org/1259963005 .
2015-07-30Add simple script to regenerate expected results.Tom Sepez
(I've been doing this by hand for far too long). R=thestig@chromium.org Review URL: https://codereview.chromium.org/1264873005 .
2015-07-16Fix typo from commit d3611f.Lei Zhang
TBR=tsepez@chromium.org Review URL: https://codereview.chromium.org/1242493003 .
2015-07-16Sort unexpected test results.Lei Zhang
R=tsepez@chromium.org Review URL: https://codereview.chromium.org/1232203003 .
2015-06-12Corpus tests check for unexpected successes.Tom Sepez
Update run_corpus_tests.py to always run all test cases, and just ignore the results that are supposed to fail. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1178393002.
2015-05-20Get test running scripts to detect and report common error.Bruce Dawson
testing/tools/run_corpus_tests.py assumes a debug build and will fail cryptically if only a release build is available. Arguably there shouldn't be a default because having one could lead to accidentally running a stale version, but that is probably too much of a change. R=tsepez@chromium.org Review URL: https://codereview.chromium.org/1150823003
2015-04-09Consider platform-specific expected .png files.Tom Sepez
Rolls DEPS to pull in the first windows-specific .png files, and unsupresses the corresponding tests. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1072613003
2015-04-03Refactor PDFium python test utilities.Tom Sepez
Extract a common portions for determining suppressions and comparing pngs. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1057983003
2015-03-26Replace linux-specific code in test scripts.Tom Sepez
Now that there's a win bot, this needs to be more careful. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1036073002
2015-03-25Suppress our "pixel" test on mac since it diffs.Tom Sepez
This involves bringing some of the suppressions file mechanism into run_pixel_tests.py (making a common module would be a nice follow-up) R=thestig@chromium.org Review URL: https://codereview.chromium.org/1032923002
2015-03-25Create per-platform pdfium test suppression files.Tom Sepez
This is required now that we have win/mac bots, which may produce different outputs. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1031203003
2015-03-23Support comments and blank lines in SUPPRESSIONS fileTom Sepez
The comment character is #. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1029193002
2015-03-23Create simple suppressions file in PDFium repository.Tom Sepez
This is needed to green the tree without continually reverting the pdfium_tests (corpus) repository. We will need to make this more sophisticated at some point, but for now, let's try to green the buildbot. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1016613004
2015-03-23Flush stdout before launching sub-processes.Tom Sepez
This should make the test logs more readable. Also give failure summary at end of tests, since searching through the log is tedious. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1026903002
2015-03-17Return OS status code from python test driver scripts.Tom Sepez
Consequently, some of the tests may diff but the waterfall remains green. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1014083003
2015-03-16Pull our new test case repository via deps.Tom Sepez
Add a run_corpus_tests.py script to run pdfium_test against the corpus tree. Note that this differs from the run_pixel_tests.py script, since pre-processing is not required. I'll work on unifying these in a subsequent CL. R=thestig@chromium.org Review URL: https://codereview.chromium.org/1010833003
2015-02-24Convert pdfium expected test results to PNG format.Tom Sepez
Required to save space vs. raw bitmap. Land prior to adding substantial number of tests. R=thestig@chromium.org Review URL: https://codereview.chromium.org/953723004
2015-02-13Run pixel tests via automated script.Tom Sepez
On origin/master, we only have .ppm format support, so the expected output files would take up a lot of space. Hence, this may not get going until XFA hits with its .png support. Nor is there a good way to diff these; XFA provides this for .png as well. But this will provide at least one automated test to ensure that we've got non-blank output, at least for one trivially simple case. R=thestig@chromium.org Review URL: https://codereview.chromium.org/926173002
2015-02-13run_javascript_tests.py: Be more flexible about directory layout.Tom Sepez
The top-level directory name isn't part of the repository, and we can't count on it always being "pdfium" (though many people will choose to do it this way). R=jam@chromium.org Review URL: https://codereview.chromium.org/921043005
2015-02-12Create run_javascript_tests.pyTom Sepez
Top level script driver for testing/resources/javascript. Converts each input template file in that directory to a .pdf file in the working directory (typically out/Debug/gen/pdfium/testing/resources/javascript), invokes pdfium_test on it to generate , and crudely diffs the expected output. We can now remove generated .pdfs from source control, keeping only the hand-editable .in templates. R=thestig@chromium.org Review URL: https://codereview.chromium.org/912803004
2015-02-05Fix segv in CPDF_DataAvail::CheckRoot() when /Root object is a string.Tom Sepez
Handles the case of this malformed PDF without crashing. Note that to get a reproducible test case, a small fix is applied to our .py script which results in some whitespace/numbering difs across the resources (down the road, we ought to generate them on the fly in an intermediate directory). BUG=454695 R=jun_fang@foxitsoftware.com, thestig@chromium.org Review URL: https://codereview.chromium.org/895933003
2015-01-21Add tests for GetNamedDests() API.Tom Sepez
Follow-on work from patch at https://codereview.chromium.org/845643008. This incorporates that patch, plus adds tests for it and similar conditions. A few changes are introduced to correct expected behaviour. This also incoprorates Deepak's fix in https://codereview.chromium.org/845643008/ This incorporates a formerly unlanded tool to generate small valid PDF files from template input, which is needed to cover all the error cases here. See https://codereview.chromium.org/791993006/ BUG=450133 R=bo_xu@foxitsoftware.com Review URL: https://codereview.chromium.org/837723009
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-15 02:43:57 +0000