Switch to reading content streams on the fly during interpretation.

Previously, before interpreting a pages content stream we would
load it entirely into a buffer. Then we would interpret that
buffer. This has a cost in memory use.

Here, we update the code to read from a stream on the fly.

This has required changes in various different parts of the code.

Firstly, we have removed all use of the FILE lock - as stream
reads can now safely be interrupted by resource (or object) reads
from elsewhere in the file, the file lock becomes a very hard
thing to maintain, and doesn't actually benefit us at all. The
choices were to either use a recursive lock, or to remove it
entirely; I opted for the latter.

The file lock enum value remains as a placeholder for future use in
extendable data streams.

Secondly, we add a new 'concat' filter that concatenates a series of
streams together into one, optionally putting whitespace between each
stream (as the pdf parser requires this).

Finally, we change page/xobject/pattern content streams to work
on the fly, but we leave type3 glyphs using buffers (as presumably
these will be run repeatedly).

1 files changed, 4 insertions, 3 deletions

diff --git a/pdf/mupdf-internal.h b/pdf/mupdf-internal.h
index 3e4a4729..3396bb8a 100644
--- a/pdf/mupdf-internal.h
+++ b/pdf/mupdf-internal.h
@@ -198,6 +198,7 @@ fz_buffer *pdf_load_image_stream(pdf_document *doc, int num, int gen, pdf_image_
 fz_stream *pdf_open_image_stream(pdf_document *doc, int num, int gen, pdf_image_params *params);
 fz_stream *pdf_open_stream_with_offset(pdf_document *doc, int num, int gen, pdf_obj *dict, int stm_ofs);
 fz_stream *pdf_open_image_decomp_stream(fz_context *ctx, fz_buffer *, pdf_image_params *params, int *factor);
+fz_stream *pdf_open_contents_stream(pdf_document *xref, pdf_obj *obj);
 
 void pdf_repair_xref(pdf_document *doc, pdf_lexbuf *buf);
 void pdf_repair_obj_stms(pdf_document *doc);
@@ -257,7 +258,7 @@ struct pdf_pattern_s
 	fz_matrix matrix;
 	fz_rect bbox;
 	pdf_obj *resources;
-	fz_buffer *contents;
+	pdf_obj *contents;
 };
 
 pdf_pattern *pdf_load_pattern(pdf_document *doc, pdf_obj *obj);
@@ -280,7 +281,7 @@ struct pdf_xobject_s
 	int transparency;
 	fz_colorspace *colorspace;
 	pdf_obj *resources;
-	fz_buffer *contents;
+	pdf_obj *contents;
 	pdf_obj *me;
 };
 
@@ -513,7 +514,7 @@ struct pdf_page_s
 	int rotate;
 	int transparency;
 	pdf_obj *resources;
-	fz_buffer *contents;
+	pdf_obj *contents;
 	fz_link *links;
 	pdf_annot *annots;
 };