Switch to reading content streams on the fly during interpretation.

Previously, before interpreting a pages content stream we would
load it entirely into a buffer. Then we would interpret that
buffer. This has a cost in memory use.

Here, we update the code to read from a stream on the fly.

This has required changes in various different parts of the code.

Firstly, we have removed all use of the FILE lock - as stream
reads can now safely be interrupted by resource (or object) reads
from elsewhere in the file, the file lock becomes a very hard
thing to maintain, and doesn't actually benefit us at all. The
choices were to either use a recursive lock, or to remove it
entirely; I opted for the latter.

The file lock enum value remains as a placeholder for future use in
extendable data streams.

Secondly, we add a new 'concat' filter that concatenates a series of
streams together into one, optionally putting whitespace between each
stream (as the pdf parser requires this).

Finally, we change page/xobject/pattern content streams to work
on the fly, but we leave type3 glyphs using buffers (as presumably
these will be run repeatedly).

1 files changed, 2 insertions, 3 deletions

diff --git a/fitz/fitz-internal.h b/fitz/fitz-internal.h
index 197d0279..b3f7a73c 100644
--- a/fitz/fitz-internal.h
+++ b/fitz/fitz-internal.h
@@ -419,7 +419,6 @@ struct fz_stream_s
 	int pos;
 	int avail;
 	int bits;
-	int locked;
 	unsigned char *bp, *rp, *wp, *ep;
 	void *state;
 	int (*read)(fz_stream *stm, unsigned char *buf, int len);
@@ -428,8 +427,6 @@ struct fz_stream_s
 	unsigned char buf[4096];
 };
 
-void fz_lock_stream(fz_stream *stm);
-
 fz_stream *fz_new_stream(fz_context *ctx, void*, int(*)(fz_stream*, unsigned char*, int), void(*)(fz_context *, void *));
 fz_stream *fz_keep_stream(fz_stream *stm);
 void fz_fill_buffer(fz_stream *stm);
@@ -521,6 +518,8 @@ static inline int fz_is_eof_bits(fz_stream *stm)
 
 fz_stream *fz_open_copy(fz_stream *chain);
 fz_stream *fz_open_null(fz_stream *chain, int len, int offset);
+fz_stream *fz_open_concat(fz_context *ctx, int max, int pad);
+void fz_concat_push(fz_stream *concat, fz_stream *chain); /* Ownership of chain is passed in */
 fz_stream *fz_open_arc4(fz_stream *chain, unsigned char *key, unsigned keylen);
 fz_stream *fz_open_aesd(fz_stream *chain, unsigned char *key, unsigned keylen);
 fz_stream *fz_open_a85d(fz_stream *chain);