| Age | Commit message (Collapse) | Author |
|---|
|
Also tidy up the taking of fz_context *'s, and hide an unwanted indent
param.
|
|
Debug printing functions: debug -> print.
Accessors: get noun attribute -> noun attribute.
Find -> lookup when the returned value is not reference counted.
pixmap_with_rect -> pixmap_with_bbox.
We are reserving the word "find" to mean lookups that give ownership
of objects to the caller. Lookup is used in other places where the
ownership is not transferred, or simple values are returned.
The rename is done by the sed script in scripts/rename3.sed
|
|
Attempt to separate public API from internal functions.
|
|
I'd foolishly removed a check in the previous commit that was needed.
refs can be < 0 for static resources.
|
|
When we attempt to insert a key/value pair into the store, we have
to allow for the possibility that a racing thread may have already
inserted an equivalent key/value. We have special code in place to
handle this eventuality; if we spot an existing entry, we take the
existing one in preference to our new key/value pair.
This means that fz_store_item needs to take a new reference to any
existing thing it finds before returning it.
Currently the only store user that is exposed to this possibility
is pdf_image; it spots an existing tile being returned, and was
inadvertently double freeing the key.
|
|
Add docs for fz_store, fz_image, fz_halftones.
Move fz_item definition into res_store.c as it does not need to be
external.
Rename fz_store_context to fz_keep_store_context to be consistent.
|
|
When inserting an item into the store we check for an identically
keyed item being there already (for instance a pixmap created from
an image I at factor F may find that such a pixmap has already been
inserted). The correct thing to do is to return the old one so we
can use that in preference.
The code was attempting to do this, but was returning a pointer
to the fz_item rather than to the item->val. Fixed here.
|
|
Currently, we are in the slightly strange position of having
the PDF specific object types as part of fitz. Here we pull
them out into the pdf layer instead. This has been made possible
by the recent changes to make the store no longer be tied to
having fz_obj's as keys.
Most of this work is a simple huge rename; to help customers who
may have code that use such functions we have provided a sed
script to do the renaming; scripts/rename2.sed.
Various other small tweaks are required; the store used to have
some debugging code that still required knowledge of fz_obj
types - we extract that into a nicer 'type' based function
pointer. Also, the type 3 font handling used to have an fz_obj
pointer for type 3 resources, and therefore needed to know how
to free this; this has become a void * with a function to free
it.
|
|
Introduce a new 'fz_image' type; this type contains rudimentary
information about images (such as native, size, colorspace etc)
and a function to call to get a pixmap of that image (with a
size hint).
Instead of passing pixmaps through the device interface (and
holding pixmaps in the display list) we now pass images instead.
The rendering routines therefore call fz_image_to_pixmap to get
pixmaps to render, and fz_pixmap_drop those afterwards.
The file format handling routines therefore need to produce
images rather than pixmaps; xps and cbz currently just wrap
pixmaps as images. PDF is more involved.
The stream handling routines in PDF have been altered so that
they can recognise when the last stream entry in a filter
dictionary is an image decoding filter. Rather than applying
this filter, they read and store the parameters into a
pdf_image_params structure, and stop decoding at that point.
This allows us to read the compressed data for an image into
memory as a block. We can then restart the image decode process
later.
pdf_images therefore consist of the compressed image data for
images. When a pixmap is requested for such an image, the code
checks to see if we have one (of an appropriate size), and if
not, decodes it.
The size hint is used to determine whether it is possible to
subsample the image; currently this is only supported for
JPEGs, but we could add generic subsampling code later.
In order to handle caching the produced images, various changes
have been made to the store and the underlying hash table.
Previously the store was indexed purely by fz_obj keys; we don't
have an fz_obj key any more, so have extended the store by adding
a concept of a key 'type'. A key type is a pointer to a set of
functions that keep/drop/compare and make a hashable key from
a key pointer.
We make a pdf_store.c file that contains functions to offer the
existing fz_obj based functions, and add a new 'type' for keys
(based on the fz_image handle, and the subsample factor) in the
pdf_image.c file.
While working on this, a problem became apparent in the existing
store codel; fz_obj objects had no protection on their reference
counts, hence an interpreter thread could try to alter a ref count
at the same time as a malloc caused an eviction from the store.
This has been solved by using the alloc lock as protection. This in
turn requires some tweaks to the code to make sure we don't try
and keep/drop fz_obj's from the store code while the alloc lock is
held.
A side effect of this work is that when a hash table is created, we
inform it what lock should be used to protect its innards (if any).
If the alloc lock is used, the insert method knows to drop/retake it
to allow it to safely expand the hash table. Callers to the hash
functions have the responsibility of taking/dropping the appropriate
lock, and ensuring that they cope with the possibility that insert
might drop the alloc lock, causing race conditions.
|
|
Silly mistake in previous commit.
|
|
We only open one instance of freetype per document. We therefore
have to ensure that only 1 call to it takes place at a time. We
introduce a lock for this purpose (FZ_LOCK_FREETYPE), and arrange
to take/release it as required.
We also update the font context so it is properly shared.
|
|
This simplifies other locking issues (notably freetype).
|
|
This is a significant change to the use of locks in MuPDF.
Previously, the user had the option of passing us lock/unlock
functions for a single mutex as part of the allocation struct.
Now we remove these entries from the allocation struct, and
make a separate 'locks' struct. This enables people to use
fz_alloc_default with locking.
If multithreaded operation is required, then the user is
required to create FZ_LOCK_MAX mutexes, which will be locked
or unlocked by MuPDF calling the lock/unlock functions within
the new fz_locks_context structure passed in at context creation.
These mutexes are not required to be recursive (they may be, but
MuPDF should never call them in this way). MuPDF avoids deadlocks
by imposing a locking ordering on itself; a thread will never take
lock n, if it already holds any lock i for which 0 <= i <= n.
Currently, there are 4 locks used within MuPDF.
Lock 0: The alloc lock; taken around all calls to user supplied
(or default) allocation functions. Also taken around all accesses
to the refs field of storable items.
Lock 1: The store lock; taken whenever the store data structures
(specifically the linked list pointers) are accessed.
Lock 2: The file lock; taken whenever a thread is accessing the raw
file. We use the debugging macros to insist that this is held
whenever we do a file based seek or read. We also insist that this
is never held when we resolve an indirect reference, as this can
have the effect of moving the file pointer.
Lock 3: The glyphcache lock; taken whenever a thread calls freetype,
or accesses the glyphcache data structures. This introduces some
complexities w.r.t type3 fonts.
Locking can be hugely problematic, so to ease our minds as to
the correctness of this code, we introduce some debugging macros.
These compile away to nothing unless FITZ_DEBUG_LOCKING is defined.
fz_assert_lock_held(ctx, lock) checks that we hold lock.
fz_assert_lock_not_held(ctx, lock) checks that we do not hold lock.
In addition fz_lock_debug_lock and fz_lock_debug_unlock are used
on every fz_lock/fz_unlock to check the validity of the operation
we are performing - in particular it checks that we do/do not already
hold the lock we are trying to take/drop, and that by taking this
lock we are not violating our defined locking order.
The RESOLVE macro (used throughout the code to check whether we need
to resolve an indirect reference) calls fz_assert_lock_not_held to
ensure that we aren't about to resolve an indirect reference (and
hence move the stream pointer) when the file is locked.
In order to implement the file locking properly, pdf_open_stream
(and friends) now lock the file as a side effect (because they
fz_seek to the start of the stream). The lock is automatically
dropped on an fz_close of such streams.
Previously, the glyph cache was created in a context when it was first
required; this presents problems as it can be shared between several
contexts or not, depending on whether it is created before the
contexts are cloned. We now always create it at startup, so it is
always shared.
This means that we need reference counting for the glyph caches.
Added here.
In fz_render_glyph, we take the glyph cache lock, and check to see
whether the glyph is in the cache. If it is, we bump the refcount,
drop the lock and returned the cached character. If it is not, we
need to render the character.
For freetype based fonts we keep the lock throughout the rendering
process, thus ensuring that freetype is only called in a single
threaded manner.
For type3 fonts, however, we need to invoke the interpreter again
to render the glyph streams. This can require reentrance to this
routine. We therefore drop the glyph cache lock, call the
interpreter to render us our pixmap, and take the lock again.
This dropping and retaking of the lock introduces a possible race
condition; 2 threads may try to render the same character at the
same time. We therefore modify our hash table insert routines to
behave differently if it comes to insert an entry only to find
that an entry with the same key is already there.
We spot this case; if we have just rendered a type3 glyph and when
we try to insert it into the cache discover that someone has beaten
us to it, we just discard our entry and use the cached one.
Hopefully this will seldom be a problem in practise; to solve it
properly would require greater complexity (probably involving
spotting that another thread is already working on the desired
rendering, and sleeping on a semaphore until it completes).
|
|
|
|
Thanks for filmor for identifying this.
|
|
Thanks to filmor's help, fix some problems where locking was going
wrong; in 2 cases we failed to unlock, and in 2 cases we tried to
free (which relocks) while already locked.
All simple fixes.
|
|
When we moved over to a context based system, we laid the foundation
for a thread-safe mupdf. This commit should complete that process.
Firstly, fz_clone_context is properly implemented so that it
makes a new context, but shares certain sections (currently
just the allocator, and the store).
Secondly, we add locking (to parts of the code that have
previously just had placeholder LOCK/UNLOCK comments). Functions
to lock and unlock a mutex are added to the allocator structure;
omit these (as is the case today) and no multithreading is
(safely) possible. The context will refuse to clone if these are
not provided.
Finally we flesh out the LOCK/UNLOCK comments to be real calls of
the functions - unfortunately this requires us to plumb fz_context
into the fz_keep_storable function (and all the fz_keep_xxx
functions that call it). This is the largest section of the patch.
No changes expected to any test files.
|
|
Ensure we don't get a division by zero (when *phase goes to 16).
Also ensure we don't overflow the bounds of an unsigned int.
Thanks to Zeniko for spotting these problems.
|
|
|
|
The new fz_malloc_struct(A,B) macro allocates sizeof(B) bytes using
fz_malloc, and then passes the resultant pointer to Memento_label
to label it with "B".
This costs nothing in non-memento builds, but gives much nicer
listings of leaked blocks when memento is enabled.
|
|
Fixes for leaks (and SEGVs, division by zeros etc) seen when
Memsqueezing.
|
|
Fix warnings/errors thrown up by the last few commits (which were
only tested on windows).
|
|
When fz_malloc (etc) are about to fail, we try to scavenge memory
from the store and then retry. We repeatedly try to bin objects
from the store until the malloc succeeds, or until we have nothing
else to bin.
This means we no longer need the 'aging' of the store, so this is
removed.
|
|
Firstly, we rename pdf_store to fz_store, reflecting the fact that
there are no pdf specific dependencies on it.
Next, we rework it so that all the objects that can be stored in
the store start with an fz_storable structure. This consists of
a reference count, and a function used to free the object when
the reference count reaches zero.
All the keep/drop functions are then reimplemented by calling
fz_keep_sharable/fz_drop_sharable. The 'drop' functions as supplied
by the callers are thus now 'free' functions, only called if
the reference count drops to 0.
The store changes to keep all the items in the store in the linked
list (which becomes a doubly linked one). We still make use of
the hashtable to index into this list quickly, but we now have
the objects in an LRU ordering within the list.
Every object is put into the store, with a size record; this is
an estimate of how much memory would be freed by freeing that
object.
The store is moved into the context and given a maximum size;
when new things are inserted into the store, care is taken to
ensure that we do not expand beyond this size. We evict any
stored items (that are not in use) starting from the least
recently used.
Finding an object in the store now takes a reference to it already.
LOCK and UNLOCK comments are used to indicate where locks need to
be taken and released to ensure thread safety.
|
