summaryrefslogtreecommitdiff
path: root/source/fitz/store.c
AgeCommit message (Collapse)Author
2017-05-09Fix Memento 'history' of block actions.Robin Watts
We call Memento_takeRef etc throughout the code so that each block keeps a history of the events on which it's reference count changes. I'd missed a case - fixed here.
2017-05-09Fix key_storable operations leading to leaks of fz_image.Robin Watts
key_storable objects can either be freed 'directly' by an API call (such as fz_drop_image), or 'internally' by the store realising that the sole thing holding onto a key_storable is a key. The current code frees more structure in the direct call than it does in the internal call. Clearly this is wrong and leads to leaks. The fix is to do ALL the freeing in the internal 'drop' function within the key_storable. This means we don't need (or want) either fz_drop_key_storable_key or fz_drop_key_storable to return an int to tell us whether they were actually dropped, so we make that change to simplify the code. This shifts the responsibility for freeing the extra internal structure into the innermost drop functions - fz_drop_image_imp and fz_drop_image_gprf_imp. To avoid duplicating code, we put the extra freeing logic into a fz_drop_image_base function.
2017-04-27Clean up store debug printing.Tor Andersson
Replace fz_print_hash with fz_hash_for_each iterator. Use string formatting callback.
2017-04-27Rename FMT_zu to FZ_FMT_zu.Tor Andersson
Don't use FMT_zu macro for fz_throw/fz_warn, since we can portably handle '%zu' in our own printf formatting.
2017-03-31Remove Reap lock.Robin Watts
I can see no reason for having the reap lock now. We always hold the ALLOC lock when we need it, so just live with that.
2017-03-31Fix leaks of reaped object keys.Robin Watts
2017-03-31Fix mutex asserts seen during reaping.Robin Watts
As seen with: mutool draw -o out.ppm Bug697706.epub
2017-03-24Ensure fz_store_types are all static const.Robin Watts
2017-03-22Rename fz_putc/puts/printf to fz_write_*.Tor Andersson
Rename fz_write to fz_write_data. Rename fz_write_buffer_* and fz_buffer_printf to fz_append_*. Be consistent in naming: fz_write_* calls write to fz_output. fz_append_* calls append to fz_buffer. Update documentation.
2017-03-20Remove fz_drop_image_base.Robin Watts
2017-01-17Add value destructor callback to fz_hash_table.Tor Andersson
This allows us to prune the iteration functions. We also remove a few other unused functions.
2016-12-27Strip extraneous blank lines.Tor Andersson
2016-10-14Drop all contexts the same way.Sebastian Rasmussen
* Handle multiple calls to fz_drop_colorspace_context() and fz_drop_font_context(). * Allow missing context in call to fz_drop_aa_context() and fz_drop_glyph_cache_context(). * Only drop font context objects when dropping the last reference. * Avoid unnecessary NULL checks.
2016-10-12Remove superfluous context null checks.Tor Andersson
Code MUST pass a non-null context to all functions. Checking ctx for null and failing silently is no more useful than segfaulting. fz_keep_imp and fz_drop_imp handle NULL pointers safely, so the NULL checks for this can also be dropped at the same time.
2016-09-19fz_store: Reap passes.Robin Watts
A few commits back, we introduced the fz_key_storable concept to allow us to cope with objects that were used both as values within the store and as parts of keys within the store. This commit worked, but showed up performance problems; when the store has several million PDF objects in it, bulk changes (such as dropping a display list or document) could trigger many passes across the store. We therefore introduce a mechanism to ameliorate this. These passes, now known as "reap passes", can be batched together using fz_defer_reap_start and fz_defer_reap_end. We trigger this start/end around display list dropping, and around PDF content stream processing. This should be fine, as deferral will be interrupted if we ever run our of memory during mallocing.
2016-09-18Make printing empty hash table entries optional.Sebastian Rasmussen
2016-09-16Improve fz_filter_store speed.Robin Watts
Now linear time rather than n^2.
2016-09-16Extend store to cope with references used in keys.Robin Watts
The store is effectively a list of items, where each item is a (key, value) pair. The design is such that we can easily get into the state where the only reference to a value is that held by the store. Subsequent references can then be generated by things being 'found' from within the store. While the only reference to an object is that held by it being a value in the store, the store is free to evict it to save memory. Images present a complication to this design; images are stored both as values within the store (by the pdf agent, so that we do not regenerate images each time we meet them in the file), and as parts of the keys within the store. For example, once an image is decoded to give a pixmap, the pixmap is cached in the store. The key to look that pixmap up again includes a reference to the image from which the pixmap was generated. This means, that for document handlers such as gproof that do not place images in the store, we can end up with images that are kept around purely by dint of being used as references in store keys. There is no chance of the value (the decoded pixmap) ever being 'found' from the store as no one other than the key is holding a reference to the image required. Thus the images/pixmaps are never freed until the store is emptied. This commit offers a fix for this situation. Standard store items are based on an fz_storable type. Here we introduce a new fz_key_storable type derived from that. As well as keeping track of the number of references a given item has to it, it keeps a separate count of the number of references a given item has to it from keys in the store. On dropping a reference, we check to see if the number of references has become the same as the number of references from keys in the store. If it has, then we know that these keys can never be 'found' again. So we filter them out of the store, which drops the items.
2016-09-14Don't report addRef/dropRef events to Memento twice.Robin Watts
We call Memento_addRef etc in fz_keep_impXX functions, so don't call them in the callers too.
2016-07-08Use fz_keep_imp and fz_drop_imp for all reference counting.Tor Andersson
2016-06-17Use 'size_t' instead of int as appropriate.Robin Watts
This silences the many warnings we get when building for x64 in windows. This does not address any of the warnings we get in thirdparty libraries - in particular harfbuzz. These look (at a quick glance) harmless though.
2016-06-08Move to using size_t for all mallocs.Robin Watts
This has knock on effects in the store. fix
2016-04-26Fix fz_store thinko.Robin Watts
Previously, we would refuse to store any object in the store that was larger than the store limits. We'd also refuse to store any object that took the total store size over the limit. This was wrong. Consider the case where we have a store of 1 byte, and a page that repeatedly uses the same font. The first time we meet the font, we look in the store, it isn't there, we load it, and we try to store it. The current code refuses to store it, and we continue, putting that font into the display list. The next time we meet to the font, we look in the store, it still isn't there, we load it, and we try to store it. Again we refuse to store it, and that copy of the font goes into the display list. The net effect of this is that we end up using far more memory in total than we would have done had we stored the first one. The code here, therefore, changes the store to always store objects regardless of their size. Given that we have already loaded the objects into memory before we store them, this doesn't actually cost us any extra memory. If an object is dropped (bringing the reference count down to 1, being the reference for the stores copy), then the object is NOT freed instantly, but will be freed either on the next attempt to store an object, or on the next scavenging malloc.
2016-03-16Avoid unused var warnings in Memento ref counting code.Robin Watts
2016-03-15Make all ref changes on storable items Memento trackable.Robin Watts
2016-01-13Add lots of consts.Robin Watts
In general, we should use 'const fz_blah' in device calls whenever the callee should not alter the fz_blah. Push this through. This shows up various places where we fz_keep and fz_drop these const things. I've updated the fz_keep and fz_drops with appropriate casts to remove the consts. We may need to do the union dance to avoid the consts for some compilers, but will only do that if required. I think this is nicer overall, even allowing for the const<->no const problems.
2015-12-11Use fz_output instead of FILE* for most of our output needs.Tor Andersson
Use fz_output in debug printing functions. Use fz_output in pdfshow. Use fz_output in fz_trace_device instead of stdout. Use fz_output in pdf-write.c. Rename fz_new_output_to_filename to fz_new_output_with_path. Add seek and tell to fz_output. Remove unused functions like fz_fprintf. Fix typo in pdf_print_obj.
2015-04-07Fix whitespace.Tor Andersson
2015-02-17Add helper functions to keep/drop reference counts with locking.Tor Andersson
Add locks around fz_path and fz_text reference counting.
2015-02-17Add ctx parameter and remove embedded contexts for API regularity.Tor Andersson
Purge several embedded contexts: Remove embedded context in fz_output. Remove embedded context in fz_stream. Remove embedded context in fz_device. Remove fz_rebind_stream (since it is no longer necessary). Remove embedded context in svg_device. Remove embedded context in XML parser. Add ctx argument to fz_document functions. Remove embedded context in fz_document. Remove embedded context in pdf_document. Remove embedded context in pdf_obj. Make fz_page independent of fz_document in the interface. We shouldn't need to pass the document to all functions handling a page. If a page is tied to the source document, it's redundant; otherwise it's just pointless. Fix reference counting oddity in fz_new_image_from_pixmap.
2015-02-17Rename fz_close_* and fz_free_* to fz_drop_*.Tor Andersson
Rename fz_close to fz_drop_stream. Rename fz_close_archive to fz_drop_archive. Rename fz_close_output to fz_drop_output. Rename fz_free_* to fz_drop_*. Rename pdf_free_* to pdf_drop_*. Rename xps_free_* to xps_drop_*.
2014-06-30Fix coding style issues and remove C99 mid-block declaration.Matt Holgate
2014-06-19Fix compiler warning.Matt Holgate
2014-06-18Fix for bug #694405 - iOS Pdf CrashMatt Holgate
If an iOS app uses too much memory, the OS asks it to free up some space. If it doesn't do so in a timely manner, it will get a second warning before being killed by the OS. In other platforms, where malloc() return NULL in OOM, the store scavenger releases memory when mallocs fail. In iOS, mallocs usually never return NULL because the app is killed before this can happen. Therefore, we need to initiate a scavenge from the low memory notification instead. We evict the store to 50% of its current size when a memory warning occurs when it is in the foreground, and 0% when a memory warning occurs whilst it is in the background. Having said this, I didn't manage to get a background warning to occur, presumably because we don't request background execution Therefore, I think in practice the OS just kills the process. However, this will be useful if we ever add background execution.
2013-06-20Rearrange source files.Tor Andersson