summaryrefslogtreecommitdiff
path: root/testing/tools
AgeCommit message (Collapse)Author
2018-01-25Add timeout when downloading gold json.Henrique Nakashima
Short timeout for now as we don't yet rely on this json. Change-Id: I2dcbf714c04f39b120f2ad380dbd34e2569b8b78 Reviewed-on: https://pdfium-review.googlesource.com/24010 Reviewed-by: dsinclair <dsinclair@chromium.org> Commit-Queue: Henrique Nakashima <hnakashima@chromium.org>
2018-01-12Fix "the the" typos.Lei Zhang
TBR=hnakashima@chromium.org Change-Id: I50877f5a2ff3fb6f48630019d2481aa451945118 Reviewed-on: https://pdfium-review.googlesource.com/22774 Reviewed-by: Lei Zhang <thestig@chromium.org> Commit-Queue: Lei Zhang <thestig@chromium.org>
2018-01-08Make api_check.py warning more informative about what needs to be fixed.Henrique Nakashima
Currently the warning does not mention that fpdfview_c_api_test.c needs to be updated, and someone unfamiliar with the code needs to ask of dig into the warning script to find what went wrong. Change-Id: I56848438959ec64b0eab978d2e3638d4d4dc8e49 Reviewed-on: https://pdfium-review.googlesource.com/22430 Commit-Queue: Henrique Nakashima <hnakashima@chromium.org> Reviewed-by: Lei Zhang <thestig@chromium.org>
2017-12-19Add missing fpdfview_c_api_test.c entries.Lei Zhang
Add a presubmit check to prevent future missing entries. Also fix an erroneous header entry. Change-Id: I8aeafd820de984f5af90b3e4ea428f582e82f254 Reviewed-on: https://pdfium-review.googlesource.com/21571 Reviewed-by: Henrique Nakashima <hnakashima@chromium.org> Commit-Queue: Lei Zhang <thestig@chromium.org>
2017-11-29Read Skia gold json and compare to locally generated pngs.Henrique Nakashima
Change-Id: I73b6a477c1a66beec2b73072f23faab7db7d3342 Reviewed-on: https://pdfium-review.googlesource.com/18513 Commit-Queue: Henrique Nakashima <hnakashima@chromium.org> Reviewed-by: Lei Zhang <thestig@chromium.org>
2017-10-26Add option to regenerate only platform-specific expected pngs.Henrique Nakashima
Change-Id: Id4798fe9a4d297678a76d0511cde7fecbf130e3e Reviewed-on: https://pdfium-review.googlesource.com/16613 Commit-Queue: Henrique Nakashima <hnakashima@chromium.org> Reviewed-by: Nicolás Peña Moreno <npm@chromium.org>
2017-10-25Add --regenerate_expected option to test_runner.py.chromium/3250Henrique Nakashima
After a change in rendering, corpus and pixel test expected pngs need to be regenerated manually. This option reduces the toil by automating renaming and moving the new expected files over the old ones. Change-Id: I9d490369ccf946d4d4567e440b7b5252469eec46 Reviewed-on: https://pdfium-review.googlesource.com/16451 Commit-Queue: Henrique Nakashima <hnakashima@chromium.org> Reviewed-by: Nicolás Peña Moreno <npm@chromium.org>
2017-09-08Store raw json with run results when running safetynet_job.py.Henrique Nakashima
Storing the machine-readable data is a good idea in general, in this case allowing us to create graphs with the historical runs. The json now also contains: - a version number in case there are future changes in the format - the date and time when the run started - which profiler was used - which commits were being compared - whether a comparison was run or not, that is, if the commit hashes for before and after differ Change-Id: I2913beaef30b90669ac4ffe60a656d4b69030588 Reviewed-on: https://pdfium-review.googlesource.com/13370 Commit-Queue: Henrique Nakashima <hnakashima@chromium.org> Reviewed-by: Nicolás Peña <npm@chromium.org>
2017-08-31Implement FORM_OnFocus() API.Lei Zhang
Given a point, if there is an annotation at the point, give if focus if it is not focus already. If there is no annotation at the point, then remove the focus from the focused annotation. BUG=chromium:754594 Change-Id: Iec3070472bbbfbad9d86e517f25da560f82efd4e Reviewed-on: https://pdfium-review.googlesource.com/12530 Commit-Queue: Lei Zhang <thestig@chromium.org> Reviewed-by: dsinclair <dsinclair@chromium.org>
2017-08-22Prepare safetynet_job.py to have its output sent in emailsHenrique Nakashima
- Add --no-color option. - Fix exit code ambiguity (2 means bad usage) Change-Id: I04be9c3f21b5a71857b7847b3cc32bf595084994 Reviewed-on: https://pdfium-review.googlesource.com/11490 Commit-Queue: Henrique Nakashima <hnakashima@chromium.org> Reviewed-by: Nicolás Peña <npm@chromium.org>
2017-08-15Make errors in called processes more evident and easier to debug.Henrique Nakashima
Also trying to get unicode filenames right again. Change-Id: I501c94921b92b8a8cd6a10441aff1595fc6d878e Reviewed-on: https://pdfium-review.googlesource.com/10630 Commit-Queue: Henrique Nakashima <hnakashima@chromium.org> Reviewed-by: Nicolás Peña <npm@chromium.org> Reviewed-by: Ryan Harrison <rharrison@chromium.org>
2017-08-11Add 'testing' to a path that got missedRyan Harrison
This applies the proper configuration rules the end HTML generation. Bug: Change-Id: Ica159131e574fd805a1363db5b26a6581ecc0ac6 Reviewed-on: https://pdfium-review.googlesource.com/10810 Reviewed-by: Henrique Nakashima <hnakashima@chromium.org> Commit-Queue: Ryan Harrison <rharrison@chromium.org>
2017-08-10Add safetynet_job.py to run safetynet_compare.py periodically.chromium/3182Henrique Nakashima
safetynet.job.py verifies if there were performance changes since its last run. Its state is comprised only of the last checkpoint, and is kept in a directory passed by argument. Results of runs are written to this directory as well. Change-Id: I94e0e176b10fd1d2b18d84f82427f63be107ae17 Reviewed-on: https://pdfium-review.googlesource.com/10370 Commit-Queue: Henrique Nakashima <hnakashima@chromium.org> Reviewed-by: Lei Zhang <thestig@chromium.org>
2017-08-10Fixing utf-8 safetynet_conclusions problem when writing the json.Henrique Nakashima
The previous fix (encoding on print) breaks when running compare.py directly. The real problem was in writing the json. Change-Id: Ie4cf926b5c6153731c1ae187ec6a7cffa74ce67b Reviewed-on: https://pdfium-review.googlesource.com/10653 Reviewed-by: Ryan Harrison <rharrison@chromium.org> Commit-Queue: Henrique Nakashima <hnakashima@chromium.org>
2017-08-10Extracting GetBooleanGnArg from coverage_report.pyHenrique Nakashima
Change-Id: I5475b37aad69361e2e7fae4ce69faf011e3b5c40 Reviewed-on: https://pdfium-review.googlesource.com/10030 Reviewed-by: Ryan Harrison <rharrison@chromium.org> Commit-Queue: Henrique Nakashima <hnakashima@chromium.org>
2017-08-09Fixing PrintConclusionsDictHumanReadable for dicts loaded from string.Henrique Nakashima
json.loads() reads non-ascii filenames to a non-unicode string and an encode step is required before a format. Change-Id: I3145c7aa41e947f78898e73484c3cb1655418689 Reviewed-on: https://pdfium-review.googlesource.com/10410 Reviewed-by: Lei Zhang <thestig@chromium.org> Commit-Queue: Henrique Nakashima <hnakashima@chromium.org>
2017-08-09Move tools/coverage to testing/tools/coverageHenrique Nakashima
Change-Id: Icb1d85167a234c9862a4011dea5dfaa09299c71c Reviewed-on: https://pdfium-review.googlesource.com/10431 Reviewed-by: Ryan Harrison <rharrison@chromium.org> Commit-Queue: Henrique Nakashima <hnakashima@chromium.org>
2017-08-03Add script to compare performance of two versions of pdfium.Henrique Nakashima
Run from the pdfium root: $ testing/tools/safetynet_compare.py testing This compares the current branch with and without local changes. $ testing/tools/safetynet_compare.py testing/corpus --branch-before x This compares the current branch + local changes against branch x. It runs only the corpus tests. $ testing/tools/safetynet_compare.py testing --branch-before x --branch-after y --build-dir=~/output_compare This compares branch x and branch y. x and y can be revision hashes. The callgrind.out files of cases with significant changes will be created in ~/output_compare. $ testing/tools/safetynet_compare.py -h Print all options. Change-Id: I43aaf5fe890745db611fb3bc00a656ef799fdfef Reviewed-on: https://pdfium-review.googlesource.com/7390 Reviewed-by: Lei Zhang <thestig@chromium.org> Commit-Queue: Henrique Nakashima <hnakashima@chromium.org>
2017-07-21Clarify --build-dir help info in safetynet_measure.py.Lei Zhang
Change-Id: I35423437015ad9057ed862ab630c9a2f403a8e1f Reviewed-on: https://pdfium-review.googlesource.com/8710 Reviewed-by: Henrique Nakashima <hnakashima@chromium.org> Commit-Queue: Lei Zhang <thestig@chromium.org>
2017-07-21Add safetynet_measure.py to measure pdfium performance.Henrique Nakashima
This is a script that measures the performance of pdfium when loading and rendering a single pdf file. Supported profilers for now are callgrind (default) and perfstat. It will be used by the compare script, which will compare measurements this script takes. Change-Id: I9c78ec879a7f4839e60f217d5dd8ee35b1e9a051 Reviewed-on: https://pdfium-review.googlesource.com/7354 Reviewed-by: Lei Zhang <thestig@chromium.org> Reviewed-by: Nicolás Peña <npm@chromium.org> Commit-Queue: Henrique Nakashima <hnakashima@chromium.org>
2017-07-18Make progressive renderer default for pdfium_test.Dan Sinclair
When Chrome renders PDF pages it uses the progressive renderer. The one shot renderer is used to render print preview pages. This CL changes pdfium_test to use the progressive renderer by default to better match Chrome. This also adds a --render-oneshot flag which will use the non-progressive renderer. Bug: chromium:736695 Change-Id: If79333ae10a0ed1c9bce1290caad8d531e6709aa Reviewed-on: https://pdfium-review.googlesource.com/8130 Reviewed-by: Tom Sepez <tsepez@chromium.org> Commit-Queue: dsinclair <dsinclair@chromium.org>
2017-06-27Change terminology of image diff to differentiate from pixel tests.Henrique Nakashima
Change-Id: I957199b63f6049a586b8de2688501f5ae3449964 Reviewed-on: https://pdfium-review.googlesource.com/7038 Reviewed-by: Nicolás Peña <npm@chromium.org> Commit-Queue: Henrique Nakashima <hnakashima@chromium.org>
2017-06-27Fixed file matching regex in test_runner.Henrique Nakashima
It was not matching file names with characters like '-' and '&'. Also suppressed tests that are now matched by the regex and were not run. Change-Id: I70c0c8bebf4726748574a9e3ab17db64f1033a13 Reviewed-on: https://pdfium-review.googlesource.com/6835 Commit-Queue: Henrique Nakashima <hnakashima@chromium.org> Reviewed-by: dsinclair <dsinclair@chromium.org>
2017-06-27Fail pixel tests when they do not provide the expected image.Henrique Nakashima
Also showing stats on number of failures, successes, suppressions, etc. Change-Id: Id12faf504ea8d6b1c343d7a8b412d03dc787ca3a Reviewed-on: https://pdfium-review.googlesource.com/6834 Commit-Queue: Henrique Nakashima <hnakashima@chromium.org> Reviewed-by: dsinclair <dsinclair@chromium.org>
2017-06-26Fix the Size trailer entry in hand written PDFs.Lei Zhang
Change-Id: Ib84cc570c2ffaf9fdd49d32bc12c7e6197e130c1 Reviewed-on: https://pdfium-review.googlesource.com/6850 Commit-Queue: Lei Zhang <thestig@chromium.org> Reviewed-by: Henrique Nakashima <hnakashima@chromium.org> Reviewed-by: dsinclair <dsinclair@chromium.org>
2017-03-27Ensure empty output directory to avoid duplicate uploadStephan Altmueller
BUG= Change-Id: I491460db0b73e5f6fe0731611400af30e6458537 Reviewed-on: https://pdfium-review.googlesource.com/3121 Reviewed-by: Wei Li <weili@chromium.org> Reviewed-by: dsinclair <dsinclair@chromium.org> Commit-Queue: dsinclair <dsinclair@chromium.org>
2017-03-08Re-enable tests now that leak is fixedchromium/3036Dan Sinclair
The fixes to RTFBreak to convert to a vector have fixed the memory leak in these test files. Re-enable for asan and remove the asan hack from the suppressor. Change-Id: Id229d61101fdd2538b9bbc38b9364d694623da40 Reviewed-on: https://pdfium-review.googlesource.com/2937 Reviewed-by: Tom Sepez <tsepez@chromium.org> Reviewed-by: Nicolás Peña <npm@chromium.org> Commit-Queue: dsinclair <dsinclair@chromium.org>
2017-02-01Remove DrMemorychromium/3000dan sinclair
The DrMemory bots have been removed, remove the config and infrastructure. BUG=chromium:655521 Change-Id: I065d717b11d615a6dc981b79a8caefd8783b105b Reviewed-on: https://pdfium-review.googlesource.com/2494 Reviewed-by: Tom Sepez <tsepez@chromium.org> Commit-Queue: dsinclair <dsinclair@chromium.org>
2017-01-31Fixup test harnessdan sinclair
When the results of the test runner were converted to return a tuple the following code which checked the results for true/false were not updated to extract the success value from the tuple. This caused the code to think that the results always passed even when they failed. This CL fixes the two tests which were broken (both just minor image result differences) which slipped in during this breakage. Change-Id: I01b56dd7b05013c2c12c83543746cf59b145e561 Reviewed-on: https://pdfium-review.googlesource.com/2456 Reviewed-by: Tom Sepez <tsepez@chromium.org> Commit-Queue: dsinclair <dsinclair@chromium.org>
2017-01-26Add flag to ignore images by their MD5 digestchromium/2994stephana
BUG= Review-Url: https://codereview.chromium.org/2649313005
2017-01-13Ensures that the gold output directory existschromium/2983chromium/2982chromium/2981stephana
This will land after this CL https://chromium-review.googlesource.com/c/424575/ as a way of making sure it works as intended. TBR=borenet BUG=skia:5973 Review-Url: https://codereview.chromium.org/2629943002
2017-01-02Gold support in PDFiumchromium/2970stephana
Extends the PDFium tests to collect images and meta data to be uploaded to Gold. This feature is triggered by adding the --gold_* flags. It extends pdfium_test to output the MD5 hash of the underlying pixel buffer for each page it renders. That output is then processed by test_runner.py to generate the gold meta data. This behavior is modeled after the 'dm' tool in skia. See https://skia.googlesource.com/skia/+/master/dm/DM.cpp#1090 This should not cause any change in the current behavior of the tests, it will be trigger once we update the buildbot recipe. BUG=skia:5973 Review-Url: https://codereview.chromium.org/2578893004
2016-05-17Combine corpus runner into test_runner.pydsinclair
This CL combines the corpus runner into the test_runner.py. This also enables the javascript and pixel runners to do a multi test runner. Review-Url: https://codereview.chromium.org/1952923002
2016-05-04Copy the event definition file to the testing directory along with pdftsepez
data_binding.pdf is much closer now that events are sent. It still diffs due to some bg color issues, not that the box is unchecked. Review-Url: https://codereview.chromium.org/1952823002
2016-05-04Fix event sending in pdfium_test.tsepez
Passing bad args in pdfium_test -- all void* pointers look alike. Make output less verbose while we're here. Turn on event sending mode for corups tests. No effect until .evt files added to corpus. Review-Url: https://codereview.chromium.org/1946873004
2016-04-28Remove output redirect from corpus test runner.dsinclair
The corpus tests redirect output to a string but they just output it later anyway. This CL removes the output redirect and removes the redirect_output flag which will always be False now. Review-Url: https://codereview.chromium.org/1927633002
2016-04-25Combine the javascript and pixel test runners.dsinclair
These runners are very similar in their implementation. Combine them into a single test runner. Corpus will, hopefully, be combined into this test runner in the future. Review URL: https://codereview.chromium.org/1906803005
2016-04-20Revert of Re-Reland Combined test runner. (patchset #4 id:80001 of ↵dsinclair
https://codereview.chromium.org/1888673003/ ) Reason for revert: Dr.Memory code is wrong..... Original issue's description: > Re-Reland Combined test runner. > > Original Commit Message: > > This CL revives the old CL to combine the test runners [1] which was reverted > due to failing font_size tests. I've deleted the font_size.pdf as it is not > needed and for testing. Pixel tests are either .in or .pdf files, not both. > > Original description: > This CL takes the three test runners (corpus, javascript, pixel) and combines > the code into a single test_runner file. Each of the individual runners still > exists and calls the test runner with their data directory. > > With this change, the pixel and javascript test will now run in parallel if > multiple processors are available. > > 1-https://codereview.chromium.org/1430623006/ > > BUG= > > Committed: https://pdfium.googlesource.com/pdfium/+/84b26ec6ba8193235ce0df817dec5c0f1da29841 TBR=ochang@chromium.org,zhaoqin@chromium.org,tsepez@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true BUG= Review URL: https://codereview.chromium.org/1898353004
2016-04-20Re-Reland Combined test runner.dsinclair
Original Commit Message: This CL revives the old CL to combine the test runners [1] which was reverted due to failing font_size tests. I've deleted the font_size.pdf as it is not needed and for testing. Pixel tests are either .in or .pdf files, not both. Original description: This CL takes the three test runners (corpus, javascript, pixel) and combines the code into a single test_runner file. Each of the individual runners still exists and calls the test runner with their data directory. With this change, the pixel and javascript test will now run in parallel if multiple processors are available. 1-https://codereview.chromium.org/1430623006/ BUG= Review URL: https://codereview.chromium.org/1888673003
2016-04-18Exclude XFA-only corpus from non-xfa and roll corpus.tsepez
Test still need modification to process input events. For now, we just suppress anything that diffs. Review URL: https://codereview.chromium.org/1894083003
2016-04-15Use platform specific test results as fallback in difference testsweili
So far, we have only two Windows tests require to use platform specific test results due to platform idiosyncracies. However, VS2015 has fixed those issues. So in general, We only need to use platform based results if checking against the regular results failed. Review URL: https://codereview.chromium.org/1892013003
2016-04-13Revert of Reland Combined test runner. (patchset #4 id:60001 of ↵dsinclair
https://codereview.chromium.org/1886753002/ ) Reason for revert: Dr. Memory still broken: https://build.chromium.org/p/client.pdfium/builders/drm_win_xfa/builds/330/steps/pixel%20tests/logs/stdio Original issue's description: > Reland Combined test runner. > > This reverts commit 7a4374583efc0c41c826122aa26c1198c8d5cc37. > > Original Commit Message: > > This CL revives the old CL to combine the test runners [1] which was reverted > due to failing font_size tests. I've deleted the font_size.pdf as it is not > needed and for testing. Pixel tests are either .in or .pdf files, not both. > > Original description: > This CL takes the three test runners (corpus, javascript, pixel) and combines > the code into a single test_runner file. Each of the individual runners still > exists and calls the test runner with their data directory. > > With this change, the pixel and javascript test will now run in parallel if > multiple processors are available. > > 1-https://codereview.chromium.org/1430623006/ > > Committed: https://pdfium.googlesource.com/pdfium/+/9cec54ab0a5461b3075c585f8f233dbfd06c2cbd TBR=tsepez@chromium.org,ochang@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true Review URL: https://codereview.chromium.org/1890703002
2016-04-13Reland Combined test runner.dsinclair
This reverts commit 7a4374583efc0c41c826122aa26c1198c8d5cc37. Original Commit Message: This CL revives the old CL to combine the test runners [1] which was reverted due to failing font_size tests. I've deleted the font_size.pdf as it is not needed and for testing. Pixel tests are either .in or .pdf files, not both. Original description: This CL takes the three test runners (corpus, javascript, pixel) and combines the code into a single test_runner file. Each of the individual runners still exists and calls the test runner with their data directory. With this change, the pixel and javascript test will now run in parallel if multiple processors are available. 1-https://codereview.chromium.org/1430623006/ Review URL: https://codereview.chromium.org/1886753002
2016-04-13Add dummy -j flags to pixel and javascript test runnersdsinclair
This is an intermediate step to creating a unified test runner. We need to update Dr Memory to pass the -j 1 flag to the pixel and javascript tests. In order to do so, we need to have the flag available. The flag will be enabled for pixel and javascript tests when [1] lands. 1- https://codereview.chromium.org/1886753002/ Review URL: https://codereview.chromium.org/1882143005
2016-04-06Revert of Combined test runner. (patchset #2 id:20001 of ↵dsinclair
https://codereview.chromium.org/1850273003/ ) Reason for revert: Broke Dr. Memory Original issue's description: > Combined test runner. > > This CL revives the old CL to combine the test runners [1] which was reverted > due to failing font_size tests. I've deleted the font_size.pdf as it is not > needed and for testing. Pixel tests are either .in or .pdf files, not both. > > Original description: > This CL takes the three test runners (corpus, javascript, pixel) and combines > the code into a single test_runner file. Each of the individual runners still > exists and calls the test runner with their data directory. > > With this change, the pixel and javascript test will now run in parallel if > multiple processors are available. > > 1-https://codereview.chromium.org/1430623006/ > > Committed: https://pdfium.googlesource.com/pdfium/+/03e8f12cd3788038225d0376a164a9993f6cb720 TBR=tsepez@chromium.org,thestig@chromium.org # Skipping CQ checks because original CL landed less than 1 days ago. NOPRESUBMIT=true NOTREECHECKS=true NOTRY=true Review URL: https://codereview.chromium.org/1867643002
2016-04-06Combined test runner.dsinclair
This CL revives the old CL to combine the test runners [1] which was reverted due to failing font_size tests. I've deleted the font_size.pdf as it is not needed and for testing. Pixel tests are either .in or .pdf files, not both. Original description: This CL takes the three test runners (corpus, javascript, pixel) and combines the code into a single test_runner file. Each of the individual runners still exists and calls the test runner with their data directory. With this change, the pixel and javascript test will now run in parallel if multiple processors are available. 1-https://codereview.chromium.org/1430623006/ Review URL: https://codereview.chromium.org/1850273003
2016-01-19Merge to XFA: Hook up show-config to testing scripts.Tom Sepez
Review URL: https://codereview.chromium.org/1607923004 . (cherry picked from commit d40e189ba1ab515ef9b3df7928e215be150df336) TBR=ochang@chromium.org Review URL: https://codereview.chromium.org/1607113002 .
2016-01-19Merge to XFA: Unify suppressions files and add matcher syntax.Tom Sepez
Review URL: https://codereview.chromium.org/1586043006 . (cherry picked from commit 2cb9ae0309417222aa3555dabe4749119dc36d84) R=ochang@chromium.org, thestig@chromium.org Review URL: https://codereview.chromium.org/1582613012 .
2015-12-11XFA: Missing license in make_expected.shTom Sepez
TBR=thestig@chromium.org Review URL: https://codereview.chromium.org/1513393004 .
2015-11-23Merge to XFA: Enable Dr. Memory to run javascript, pixel, and corpus testsQin Zhao
- add DrMemoryWrapper in common.py to adjust Dr. Memory wrapper - add --wrapper option to run_*_tests.py for Dr. Mempry wrapper - update run_*_tests.py to handle Dr. Memory wrapper - add TestPDFiumTest in pdfium_tests.py to support run_*_tests.py - remove ValgrindTool in valgrind_tests.py R=thestig@chromium.org BUG=pdfium:238 Review URL: https://codereview.chromium.org/1464453003 . (cherry picked from commit 36476923ae5eb8e9283e605f3c85ee8811c86014) Review URL: https://codereview.chromium.org/1465333002 .
generated by cgit v1.2.3 (git 2.25.1) at 2024-11-18 22:22:08 +0000