| Age | Commit message (Collapse) | Author |
|---|
|
|
|
|
|
|
|
|
|
Read the contents of a file into a fz_buffer in one go.
|
|
Find the first sibling, next sibling or first child matching tag name.
|
|
Currently each xref in the file results in an array from 0 to
num_objects. If we have a file that has been updated many times
this causes a huge waste of memory.
Instead we now hold each xref as a list of non-overlapping subsections
(exactly as the file holds them).
Lookup is therefore potentially slower, but only on files where the
xrefs are highly fragmented (i.e. where we would be saving in memory
terms).
Some parts of our code (notably the file writing code that does
garbage collection etc) assumes that lookups of object entry pointers
will not change previous object entry pointers that have been
looked up. To cope with this, and to cope with the case where we are
updating/creating new objects, we introduce the idea of a 'solid'
xref.
A solid xref is one where it has a single subsection record that spans
the entire range of valid object numbers for a file. Once we have
ensured that an xref is 'solid', we can safely work on the pointers
within it without fear of them moving.
We ensure that any 'incremental' xref is solid.
We also ensure that any non-incremental write makes the xref solid.
|
|
|
|
unused variables / functions / potential uninitialised variable usage
|
|
|
|
The dtoa function is for doubles (which is what MuJS uses) but for MuPDF
we only need and want float precision in our output formatting.
|
|
|
|
Add a whitespace preserving mode, for future use with XHTML.
Also parse XHTML entities. This is not strictly according to spec,
but for properly formed XML documents it should not matter.
|
|
pdf-tools.h is used so that we can call into pdfclean_main.
|
|
Add a new class of errors and use them to abort interpretation when
the test device detects a color page.
|
|
|
|
|
|
The original version of the test-device could characterise pages
as being grayscale/color purely based on the colorspaces used.
This could easily be upset by grayscale images or shadings that
happened to be specified in non-grayscale colorspaces however.
We now look at the actual shading and image color values, and use
a threshold value to allow for some measure of rounding errors in
color values that are in practice grayscale.
|
|
win32 supports tinting, but cannot change the color from the default.
|
|
Currently only tests for the presence of non-grayscale color.
|
|
|
|
If an iOS app uses too much memory, the OS asks it to free up some space.
If it doesn't do so in a timely manner, it will get a second warning before
being killed by the OS.
In other platforms, where malloc() return NULL in OOM, the store scavenger
releases memory when mallocs fail. In iOS, mallocs usually never return NULL
because the app is killed before this can happen. Therefore, we need to
initiate a scavenge from the low memory notification instead.
We evict the store to 50% of its current size when a memory warning occurs
when it is in the foreground, and 0% when a memory warning occurs whilst
it is in the background. Having said this, I didn't manage to get a background
warning to occur, presumably because we don't request background execution
Therefore, I think in practice the OS just kills the process. However, this
will be useful if we ever add background execution.
|
|
|
|
|
|
Increasing the existing data structure to 32-bit values would bloat the data
tables too much.
Simplify the data structure and use three separate range tables for lookups --
one with small 16-bit to 16-bit range lookups, one with 32-bit range lookups,
and a final one for one-to-many lookups.
This loses the range-to-table optimization we had before, but even with the
extra ranges this necessitates, the total size of the compiled binary CMap data
is smaller than if we were to extend the previous scheme to 32 bits.
|
|
NoExport (and ReadOnly) fields shouldn't mark the document for saving.
|
|
|
|
|
|
Split functions out of pdf-form.c that shouldn't be there, and make
javascript initialization explicit.
|
|
|
|
New routine to filter the content streams for pages, xobjects,
type3 charprocs, patterns etc. The filtered streams are guaranteed
to be properly matched with q/Q's, and to not have changed the top
level ctm. Additionally we remove (some) repeated settings of
colors etc. This filtering can be extended to be smarter later.
The idea of this is to both repair after editing, and to leave the
streams in a form that can be easily appended to.
This is preparatory to work on Bates numbering and Watermarking.
Currently the streams produced are uncompressed.
|
|
%q escapes using C syntax and wraps the string in double quotes.
%( escapes using PS/PDF syntax and wraps the string in parens.
|
|
The primary motivator for this is so that we can print floating point
values and get the full accuracy out, without having to print 1.5 as
1.5000000, and without getting 23e24 etc.
We only support %c, %f, %d, %o, %x and %s currently.
We only support the zero padding qualifier, for integers.
We do support some extensions:
%C turns values >=128 into UTF-8.
%M prints a fz_matrix.
%R prints a fz_rect.
%P prints a fz_point.
We also implement a fprintf variant on top of this to allow for
consistent results when using fz_output.
a
|
|
Previously pdf_process buffer did not understand inline images.
In order to make this work without needlessly duplicating complex code
from within pdf-op-run, the parsing of inline images has been moved to
happen in pdf-interpret.c. When the op_table entry for BI is called
it now expects the inline image to be in csi->img and the dictionary
object to be in csi->obj.
To make this work, we have had to improve the handling of inline images
in general. While non-inline images have been loaded and held in
memory in their compressed form and only decoded when required, until
now we have always loaded and decoded inline images immediately. This
has been due to the difficulty in knowing how many bytes of data to
read from the stream - we know the length of the stream once
uncompressed, but relating this to the compressed length is hard.
To cure this we introduce a new type of filter stream, a 'leecher'.
We insert a leecher stream before we build the filters required to
decode the image. We then read and discard the appropriate number
of uncompressed bytes from the filters. This pulls the compressed
data through the leecher stream, which stores it in an fz_buffer.
Thus images are now always held in their compressed forms in memory.
The pdf-op-run implementation is now trivial. The only real complexity
in the pdf-op-buffer implementation is the need to ensure that the
/Filter entry in the dictionary object matches the exact point at
which we backstopped the decompression.
|
|
Currently fz_streams have a 4K buffer within their header. The call
to read from a stream fills this buffer, resulting in more data being
pulled from any underlying stream than we might like. This causes
problems with the forthcoming 'leech' filter.
Here we simplify the fields available in the public stream header.
No specific buffer is given; simply the read and write pointers.
The underlying 'read' function is replaced by a 'next' function
that makes the next block of data available and returns the first
character of it (or EOF).
A caller to the 'next' function should supply the maximum number of
bytes that it knows it will need (possibly not now, but eventually).
This enables the underlying stream to efficiently decode just enough.
The underlying stream is free to return fewer, or a greater number
if it wants to.
The exact size of the 'block' of data returned will depend on the
filter in use and (possibly) the data therein.
Callers can get the currently available amount of data by calling
fz_available (but again should pass the maximum amount of data they know
they will need). The only time this will ever return 0 is if we have
hit EOF.
|
|
Pass in the 'tight' flag.
|
|
We add various facilities here, intended to allow us to efficiently
minimise the memory we use for holding cached pdf objects.
Firstly, we add the ability to 'mark' all the currently loaded objects.
Next we add the ability to 'clear the xref' - to drop all the currently
loaded objects that have no other references except the ones held by the
xref table itself.
Finally, we add the ability to 'clear the xref to the last mark' - to
drop all the currently loaded objects that have been created since the
last 'mark' operation and have no other references except the ones held
by the xref table.
We expose this to the user by adding a new device hint 'FZ_NO_CACHE'.
If set on the device, then the PDF interpreter will pdf_mark_xref before
starting and pdf_clear_xref_to_mark afterwards. Thus no additional
objects will be retained in memory after a given page is run, unless
someone else picks them up and takes a reference to them as part of
the run.
We amend our simple example app to set this device hint when loading
pages as part of a search.
|
|
The following changes allow font providers to make better choices WRT
what font to provide and under what circumstances:
* bold and italic flags are passed in so that implementors can decide
themselves whether to ask for simulated boldening/italicising
if a font claims not to be bold/italic
* is_substitute is replaced with needs_exact_metrics to make the
meaning of this argument hopefully clearer (that argument is set only
for PDF fonts without a FontDescriptor)
* the font name is always passed as requested by the document instead
of the cleaned name for the standard 14 fonts which allows
distinguishing e.g. Symbol and Symbol,Bold
|
|
|
|
|
|
Many times, the idiom p.x = x; p.y = y; fz_transform_point() is used.
This function should simplify that use case by both initializing and
transforming the point in one call.
|
|
|
|
This feature is being implemented mostly for the purpose of permitting
the addition to a page of invisible signatures.
Also change pdf_create_annot to make freshly created annotations
printable by default.
|
|
|
|
Patch from Thomas Fach-Pedersen. Many thanks!
Add a new format handler that copes with TIFF files. This replaces
the TIFF functionality within the image format handler, and is
better because this copes with multiple images (as one image per
page).
|
|
Patch from Thomas Fach-Pedersen. Many Thanks.
|
|
Useful utility missing from our arsenal.
|
|
Reuses the same internals as pdf_fprintf_obj etc.
|
|
When we find certain classes of flaw in the file while attempting to
read an object, we trigger an automatic repair of the file. This
leaves almost all objects unchanged; the sole exception is that of
the trailer object (and its sub objects) which can get dropped and
recreated.
To avoid leaving people holding handles to objects within the trailer
dict high and dry, we introduce a 'pre_repair_trailer' object to
each xref entry. On a repair, we copy the existing trailer object to
this. As we only ever repair once, this is safe.
The only known place where this is a problem is when setting up the
pdf_crypt for a document; we adapt the code here to allow for
potential problems.
The example file that shows this up is:
048d14d2f5f0ae31e9a2cde0be66f16a_asan_heap-uaf_86d4ed_3961_3661.pdf
Thanks to Mateusz Jurczyk and Gynvael Coldwind of the Google Security
Team for providing the fuzzing files.
|
|
For SumatraPDF, the following changes are required:
* fz_load_system_font is called from pdf_load_builtin_font as well so
that Arial, Courier New, etc. can be loaded from the system instead
of their Nimbus replacements. In order to distinguish between calls
from pdf_load_builtin_font and pdf_load_substitute_font, an
is_substitute argument is added.
* fz_load_system_cjk_font is added and called from
pdf_load_substitute_cjk_font so that a better replacement font can
be loaded instead of DroidSansFallback.
* Both fz_load_system_font and fz_load_system_cjk_font return fz_font*
instead of fz_buffer* so that implementers aren't required to load
fonts into memory (SumatraPDF uses fz_new_font_from_file for system
fonts).
In addition to that, fz_load_system_font_func is renamed to
fz_load_system_font_funcs since it now accepts two functions, and the
PDF_ROS_* constants are renamed to FZ_ADOBE_* (collection names aren't
passed as const char* so that implementers know which collections to
expect). For convenience, fz_load_*_font also never throws since
currently all callers have further fallbacks available.
|
