| Age | Commit message (Collapse) | Author |
|---|
|
While running 'ensure_space', we can drop and retake the alloc lock.
This can enable other threads to perform store operations, which can
trigger reaps.
When reaps happen, fz_item's are discarded. This is particularly bad
when ensure_space happens to be holding onto a pointer to one (prev).
The fix, implemented here, is to move items out of the store list
(and hash table), and put them onto a local 'to_be_freed' list.
Once on this local list we can be sure that they won't be found
by other threads doing store operations, and they can be safely
freed from there (dropping/retaking the lock as required).
|
|
If Bitmap.lockPixels fails with an allocation error, scavenge in
the store.
|
|
Currently, our API uses static inlines for fz_lock and
fz_unlock, the definitions for which depend on whether
we build NDEBUG or not. This isn't ideal as it causes
problems when people link a release binary with a debug
lib (or vice versa).
We really want to continue to use static inlines for the
locking functions as used from MuPDF, as we hit them
hard in the keep/drop functions.
We therefore remove fz_lock/fz_unlock from the public API
entirely. Accordingly, we move the fz_lock/fz_unlock
static inlines into fitz-imp.h (an internal header),
together with the fz_keep_.../fz_drop_... functions.
We then have public fz_lock/fz_unlock functions for any
external callers to use that are free of compilications.
At the same time, to avoid another indirection, we change
from holding the locking functions as a pointer to a
struct to a struct itself.
|
|
Don't mess with conditional compilation with LARGEFILE -- always expose
64-bit file offsets in our public API.
|
|
|
|
No longer required, and causes leaks.
Also, fix some reference counting problems with colorspaces.
|
|
We call Memento_takeRef etc throughout the code so that each block
keeps a history of the events on which it's reference count changes.
I'd missed a case - fixed here.
|
|
key_storable objects can either be freed 'directly' by an API call
(such as fz_drop_image), or 'internally' by the store realising that
the sole thing holding onto a key_storable is a key.
The current code frees more structure in the direct call than it
does in the internal call. Clearly this is wrong and leads to
leaks.
The fix is to do ALL the freeing in the internal 'drop' function
within the key_storable.
This means we don't need (or want) either fz_drop_key_storable_key or
fz_drop_key_storable to return an int to tell us whether they were
actually dropped, so we make that change to simplify the code.
This shifts the responsibility for freeing the extra internal
structure into the innermost drop functions - fz_drop_image_imp and
fz_drop_image_gprf_imp. To avoid duplicating code, we put the
extra freeing logic into a fz_drop_image_base function.
|
|
Replace fz_print_hash with fz_hash_for_each iterator.
Use string formatting callback.
|
|
Don't use FMT_zu macro for fz_throw/fz_warn, since we can portably handle '%zu'
in our own printf formatting.
|
|
I can see no reason for having the reap lock now. We always
hold the ALLOC lock when we need it, so just live with that.
|
|
|
|
As seen with:
mutool draw -o out.ppm Bug697706.epub
|
|
|
|
Rename fz_write to fz_write_data.
Rename fz_write_buffer_* and fz_buffer_printf to fz_append_*.
Be consistent in naming:
fz_write_* calls write to fz_output.
fz_append_* calls append to fz_buffer.
Update documentation.
|
|
|
|
This allows us to prune the iteration functions.
We also remove a few other unused functions.
|
|
|
|
* Handle multiple calls to fz_drop_colorspace_context() and
fz_drop_font_context().
* Allow missing context in call to fz_drop_aa_context()
and fz_drop_glyph_cache_context().
* Only drop font context objects when dropping the last reference.
* Avoid unnecessary NULL checks.
|
|
Code MUST pass a non-null context to all functions.
Checking ctx for null and failing silently is no more useful
than segfaulting.
fz_keep_imp and fz_drop_imp handle NULL pointers safely, so
the NULL checks for this can also be dropped at the same time.
|
|
A few commits back, we introduced the fz_key_storable concept
to allow us to cope with objects that were used both as values
within the store and as parts of keys within the store.
This commit worked, but showed up performance problems; when the
store has several million PDF objects in it, bulk changes (such
as dropping a display list or document) could trigger many passes
across the store.
We therefore introduce a mechanism to ameliorate this. These
passes, now known as "reap passes", can be batched together using
fz_defer_reap_start and fz_defer_reap_end.
We trigger this start/end around display list dropping, and around
PDF content stream processing. This should be fine, as deferral
will be interrupted if we ever run our of memory during mallocing.
|
|
|
|
Now linear time rather than n^2.
|
|
The store is effectively a list of items, where each item is a
(key, value) pair. The design is such that we can easily get
into the state where the only reference to a value is that held
by the store. Subsequent references can then be generated by
things being 'found' from within the store.
While the only reference to an object is that held by it being
a value in the store, the store is free to evict it to save
memory.
Images present a complication to this design; images are stored
both as values within the store (by the pdf agent, so that we
do not regenerate images each time we meet them in the file),
and as parts of the keys within the store.
For example, once an image is decoded to give a pixmap, the
pixmap is cached in the store. The key to look that pixmap up
again includes a reference to the image from which the pixmap
was generated.
This means, that for document handlers such as gproof that do
not place images in the store, we can end up with images that
are kept around purely by dint of being used as references in
store keys. There is no chance of the value (the decoded pixmap)
ever being 'found' from the store as no one other than the
key is holding a reference to the image required. Thus the
images/pixmaps are never freed until the store is emptied.
This commit offers a fix for this situation.
Standard store items are based on an fz_storable type. Here we
introduce a new fz_key_storable type derived from that. As well
as keeping track of the number of references a given item has
to it, it keeps a separate count of the number of references a
given item has to it from keys in the store.
On dropping a reference, we check to see if the number of
references has become the same as the number of references from
keys in the store. If it has, then we know that these keys can
never be 'found' again. So we filter them out of the store,
which drops the items.
|
|
We call Memento_addRef etc in fz_keep_impXX functions, so
don't call them in the callers too.
|
|
|
|
This silences the many warnings we get when building for x64
in windows.
This does not address any of the warnings we get in thirdparty
libraries - in particular harfbuzz. These look (at a quick
glance) harmless though.
|
|
This has knock on effects in the store.
fix
|
|
Previously, we would refuse to store any object in the store that
was larger than the store limits. We'd also refuse to store any
object that took the total store size over the limit.
This was wrong.
Consider the case where we have a store of 1 byte, and a page that
repeatedly uses the same font. The first time we meet the font, we
look in the store, it isn't there, we load it, and we try to store
it. The current code refuses to store it, and we continue, putting
that font into the display list.
The next time we meet to the font, we look in the store, it still
isn't there, we load it, and we try to store it. Again we refuse to
store it, and that copy of the font goes into the display list.
The net effect of this is that we end up using far more memory in
total than we would have done had we stored the first one.
The code here, therefore, changes the store to always store objects
regardless of their size. Given that we have already loaded the
objects into memory before we store them, this doesn't actually
cost us any extra memory. If an object is dropped (bringing the
reference count down to 1, being the reference for the stores copy),
then the object is NOT freed instantly, but will be freed either
on the next attempt to store an object, or on the next scavenging
malloc.
|
|
|
|
|
|
In general, we should use 'const fz_blah' in device calls whenever
the callee should not alter the fz_blah.
Push this through. This shows up various places where we fz_keep
and fz_drop these const things.
I've updated the fz_keep and fz_drops with appropriate casts
to remove the consts. We may need to do the union dance to avoid
the consts for some compilers, but will only do that if required.
I think this is nicer overall, even allowing for the const<->no const
problems.
|
|
Use fz_output in debug printing functions.
Use fz_output in pdfshow.
Use fz_output in fz_trace_device instead of stdout.
Use fz_output in pdf-write.c.
Rename fz_new_output_to_filename to fz_new_output_with_path.
Add seek and tell to fz_output.
Remove unused functions like fz_fprintf.
Fix typo in pdf_print_obj.
|
|
|
|
Add locks around fz_path and fz_text reference counting.
|
|
Purge several embedded contexts:
Remove embedded context in fz_output.
Remove embedded context in fz_stream.
Remove embedded context in fz_device.
Remove fz_rebind_stream (since it is no longer necessary).
Remove embedded context in svg_device.
Remove embedded context in XML parser.
Add ctx argument to fz_document functions.
Remove embedded context in fz_document.
Remove embedded context in pdf_document.
Remove embedded context in pdf_obj.
Make fz_page independent of fz_document in the interface.
We shouldn't need to pass the document to all functions handling a page.
If a page is tied to the source document, it's redundant; otherwise it's
just pointless.
Fix reference counting oddity in fz_new_image_from_pixmap.
|
|
Rename fz_close to fz_drop_stream.
Rename fz_close_archive to fz_drop_archive.
Rename fz_close_output to fz_drop_output.
Rename fz_free_* to fz_drop_*.
Rename pdf_free_* to pdf_drop_*.
Rename xps_free_* to xps_drop_*.
|
|
|
|
|
|
If an iOS app uses too much memory, the OS asks it to free up some space.
If it doesn't do so in a timely manner, it will get a second warning before
being killed by the OS.
In other platforms, where malloc() return NULL in OOM, the store scavenger
releases memory when mallocs fail. In iOS, mallocs usually never return NULL
because the app is killed before this can happen. Therefore, we need to
initiate a scavenge from the low memory notification instead.
We evict the store to 50% of its current size when a memory warning occurs
when it is in the foreground, and 0% when a memory warning occurs whilst
it is in the background. Having said this, I didn't manage to get a background
warning to occur, presumably because we don't request background execution
Therefore, I think in practice the OS just kills the process. However, this
will be useful if we ever add background execution.
|
|
|
