Fix tile coverage calculations.

The code attempts to spot cases where a pattern tile is so large that
only 1 repeat is visible. Due to rounding errors, this test could
sometimes fail, and (on badly formed files) we'd attempt to allocate
huge pixmaps.

The fix is to allow for rounding errors.

1 files changed, 9 insertions, 4 deletions

diff --git a/pdf/pdf_interpret.c b/pdf/pdf_interpret.c
index 0720f782..6d312724 100644
--- a/pdf/pdf_interpret.c
+++ b/pdf/pdf_interpret.c
@@ -1164,10 +1164,15 @@ pdf_show_pattern(pdf_csi *csi, pdf_pattern *pat, fz_rect area, int what)
 	/* patterns are painted using the ctm in effect at the beginning of the content stream */
 	/* get bbox of shape in pattern space for stamping */
 	area = fz_transform_rect(invptm, area);
-	x0 = floorf(area.x0 / pat->xstep);
-	y0 = floorf(area.y0 / pat->ystep);
-	x1 = ceilf(area.x1 / pat->xstep);
-	y1 = ceilf(area.y1 / pat->ystep);
+
+	/* When calculating the number of tiles required, we adjust by a small
+	 * amount to allow for rounding errors. By choosing this amount to be
+	 * smaller than 1/256, we guarantee we won't cause problems that will
+	 * be visible even under our most extreme antialiasing. */
+	x0 = floorf(area.x0 / pat->xstep + 0.001);
+	y0 = floorf(area.y0 / pat->ystep + 0.001);
+	x1 = ceilf(area.x1 / pat->xstep - 0.001);
+	y1 = ceilf(area.y1 / pat->ystep - 0.001);
 
 	oldtopctm = csi->top_ctm;
 	oldtop = csi->gtop;