summaryrefslogtreecommitdiff
path: root/scripts
AgeCommit message (Collapse)Author
2013-03-01Bug 693624: Ensure that windows copes with utf8 filenamesRobin Watts
When running under Windows, replace fopen with our own fopen_utf8 that converts from utf8 to unicode before calling the unicode version of fopen.
2012-11-26xps: Move XML parser into fitz namespace.Tor Andersson
2012-10-05Bug 693355: Fix cquote.c trying to write to input file.Robin Watts
Simple typo; trying to write to input file. Thanks to Moritz Lipp for pointing out the problem.
2012-07-17Fix a couple of bugs in c-string generatorPaul Gardiner
2012-07-12Separate out the Javascript utility functions and autogenerate C stringPaul Gardiner
2012-06-25Warning fixes and various clean ups:Sebastian Rasmussen
Remove unused variable, silencing compiler warning. No need to initialize variables twice. Remove initialization of unread variable. Remove unnecessary check for NULL. Close output file upon error in cmapdump.
2012-05-08Switch to reading content streams on the fly during interpretation.Robin Watts
Previously, before interpreting a pages content stream we would load it entirely into a buffer. Then we would interpret that buffer. This has a cost in memory use. Here, we update the code to read from a stream on the fly. This has required changes in various different parts of the code. Firstly, we have removed all use of the FILE lock - as stream reads can now safely be interrupted by resource (or object) reads from elsewhere in the file, the file lock becomes a very hard thing to maintain, and doesn't actually benefit us at all. The choices were to either use a recursive lock, or to remove it entirely; I opted for the latter. The file lock enum value remains as a placeholder for future use in extendable data streams. Secondly, we add a new 'concat' filter that concatenates a series of streams together into one, optionally putting whitespace between each stream (as the pdf parser requires this). Finally, we change page/xobject/pattern content streams to work on the fly, but we leave type3 glyphs using buffers (as presumably these will be run repeatedly).
2012-05-08Add scripts to generate hyperlinked source in HTML.Tor Andersson
2012-04-09Bug 692977: Stop harmless thread debugging messages during cmapdumpRobin Watts
If compiled with -DDEBUG, cmapdump throws a large number of warnings regarding thread locking. These are harmless and can be ignored, but are, nonetheless, not pretty. Fixed here. Thanks to Bas Weelinck for the report.
2012-03-13Rename some functions and accessors to be more consistent.Tor Andersson
Debug printing functions: debug -> print. Accessors: get noun attribute -> noun attribute. Find -> lookup when the returned value is not reference counted. pixmap_with_rect -> pixmap_with_bbox. We are reserving the word "find" to mean lookups that give ownership of objects to the caller. Lookup is used in other places where the ownership is not transferred, or simple values are returned. The rename is done by the sed script in scripts/rename3.sed
2012-03-12More API tidying.Robin Watts
Make fz_clone_context copy existing AA settings. Add accessor function for fz_bitmap. Add more documentation for various functions/types.
2012-03-06Split fitz.h/mupdf.h into internal/external headers.Robin Watts
Attempt to separate public API from internal functions.
2012-03-01Add some more docs to fitz.hRobin Watts
Add docs for fz_store, fz_image, fz_halftones. Move fz_item definition into res_store.c as it does not need to be external. Rename fz_store_context to fz_keep_store_context to be consistent.
2012-02-26Move fz_obj to be pdf_obj.Robin Watts
Currently, we are in the slightly strange position of having the PDF specific object types as part of fitz. Here we pull them out into the pdf layer instead. This has been made possible by the recent changes to make the store no longer be tied to having fz_obj's as keys. Most of this work is a simple huge rename; to help customers who may have code that use such functions we have provided a sed script to do the renaming; scripts/rename2.sed. Various other small tweaks are required; the store used to have some debugging code that still required knowledge of fz_obj types - we extract that into a nicer 'type' based function pointer. Also, the type 3 font handling used to have an fz_obj pointer for type 3 resources, and therefore needed to know how to free this; this has become a void * with a function to free it.
2012-02-13Add locking around freetype calls.Robin Watts
We only open one instance of freetype per document. We therefore have to ensure that only 1 call to it takes place at a time. We introduce a lock for this purpose (FZ_LOCK_FREETYPE), and arrange to take/release it as required. We also update the font context so it is properly shared.
2012-02-08Lock reworking.Robin Watts
This is a significant change to the use of locks in MuPDF. Previously, the user had the option of passing us lock/unlock functions for a single mutex as part of the allocation struct. Now we remove these entries from the allocation struct, and make a separate 'locks' struct. This enables people to use fz_alloc_default with locking. If multithreaded operation is required, then the user is required to create FZ_LOCK_MAX mutexes, which will be locked or unlocked by MuPDF calling the lock/unlock functions within the new fz_locks_context structure passed in at context creation. These mutexes are not required to be recursive (they may be, but MuPDF should never call them in this way). MuPDF avoids deadlocks by imposing a locking ordering on itself; a thread will never take lock n, if it already holds any lock i for which 0 <= i <= n. Currently, there are 4 locks used within MuPDF. Lock 0: The alloc lock; taken around all calls to user supplied (or default) allocation functions. Also taken around all accesses to the refs field of storable items. Lock 1: The store lock; taken whenever the store data structures (specifically the linked list pointers) are accessed. Lock 2: The file lock; taken whenever a thread is accessing the raw file. We use the debugging macros to insist that this is held whenever we do a file based seek or read. We also insist that this is never held when we resolve an indirect reference, as this can have the effect of moving the file pointer. Lock 3: The glyphcache lock; taken whenever a thread calls freetype, or accesses the glyphcache data structures. This introduces some complexities w.r.t type3 fonts. Locking can be hugely problematic, so to ease our minds as to the correctness of this code, we introduce some debugging macros. These compile away to nothing unless FITZ_DEBUG_LOCKING is defined. fz_assert_lock_held(ctx, lock) checks that we hold lock. fz_assert_lock_not_held(ctx, lock) checks that we do not hold lock. In addition fz_lock_debug_lock and fz_lock_debug_unlock are used on every fz_lock/fz_unlock to check the validity of the operation we are performing - in particular it checks that we do/do not already hold the lock we are trying to take/drop, and that by taking this lock we are not violating our defined locking order. The RESOLVE macro (used throughout the code to check whether we need to resolve an indirect reference) calls fz_assert_lock_not_held to ensure that we aren't about to resolve an indirect reference (and hence move the stream pointer) when the file is locked. In order to implement the file locking properly, pdf_open_stream (and friends) now lock the file as a side effect (because they fz_seek to the start of the stream). The lock is automatically dropped on an fz_close of such streams. Previously, the glyph cache was created in a context when it was first required; this presents problems as it can be shared between several contexts or not, depending on whether it is created before the contexts are cloned. We now always create it at startup, so it is always shared. This means that we need reference counting for the glyph caches. Added here. In fz_render_glyph, we take the glyph cache lock, and check to see whether the glyph is in the cache. If it is, we bump the refcount, drop the lock and returned the cached character. If it is not, we need to render the character. For freetype based fonts we keep the lock throughout the rendering process, thus ensuring that freetype is only called in a single threaded manner. For type3 fonts, however, we need to invoke the interpreter again to render the glyph streams. This can require reentrance to this routine. We therefore drop the glyph cache lock, call the interpreter to render us our pixmap, and take the lock again. This dropping and retaking of the lock introduces a possible race condition; 2 threads may try to render the same character at the same time. We therefore modify our hash table insert routines to behave differently if it comes to insert an entry only to find that an entry with the same key is already there. We spot this case; if we have just rendered a type3 glyph and when we try to insert it into the cache discover that someone has beaten us to it, we just discard our entry and use the cached one. Hopefully this will seldom be a problem in practise; to solve it properly would require greater complexity (probably involving spotting that another thread is already working on the desired rendering, and sleeping on a semaphore until it completes).
2012-02-06Pass context to cmap and font descriptor functions.Tor Andersson
2012-01-19Multi-threading support for MuPDFRobin Watts
When we moved over to a context based system, we laid the foundation for a thread-safe mupdf. This commit should complete that process. Firstly, fz_clone_context is properly implemented so that it makes a new context, but shares certain sections (currently just the allocator, and the store). Secondly, we add locking (to parts of the code that have previously just had placeholder LOCK/UNLOCK comments). Functions to lock and unlock a mutex are added to the allocator structure; omit these (as is the case today) and no multithreading is (safely) possible. The context will refuse to clone if these are not provided. Finally we flesh out the LOCK/UNLOCK comments to be real calls of the functions - unfortunately this requires us to plumb fz_context into the fz_keep_storable function (and all the fz_keep_xxx functions that call it). This is the largest section of the patch. No changes expected to any test files.
2012-01-11Use enum for FZ_STORE_DEFAULT default size.Tor Andersson
2012-01-11Hide glyph cache in context.Tor Andersson
2012-01-10Fix many spelling errors.Sebastian Rasmussen
2011-12-15Add scavenging functionality.Robin Watts
When fz_malloc (etc) are about to fail, we try to scavenge memory from the store and then retry. We repeatedly try to bin objects from the store until the malloc succeeds, or until we have nothing else to bin. This means we no longer need the 'aging' of the store, so this is removed.
2011-12-15Rework pdf_store to fz_store, a part of fz_context.Robin Watts
Firstly, we rename pdf_store to fz_store, reflecting the fact that there are no pdf specific dependencies on it. Next, we rework it so that all the objects that can be stored in the store start with an fz_storable structure. This consists of a reference count, and a function used to free the object when the reference count reaches zero. All the keep/drop functions are then reimplemented by calling fz_keep_sharable/fz_drop_sharable. The 'drop' functions as supplied by the callers are thus now 'free' functions, only called if the reference count drops to 0. The store changes to keep all the items in the store in the linked list (which becomes a doubly linked one). We still make use of the hashtable to index into this list quickly, but we now have the objects in an LRU ordering within the list. Every object is put into the store, with a size record; this is an estimate of how much memory would be freed by freeing that object. The store is moved into the context and given a maximum size; when new things are inserted into the store, care is taken to ensure that we do not expand beyond this size. We evict any stored items (that are not in use) starting from the least recently used. Finding an object in the store now takes a reference to it already. LOCK and UNLOCK comments are used to indicate where locks need to be taken and released to ensure thread safety.
2011-12-08Stylistic changes when testing pointer values for NULL.Tor Andersson
Also: use 'cannot' instead of 'failed to' in error messages.
2011-12-08Remove remaining fz_error_note calls in the pdf code.Tor Andersson
2011-12-06Move antialias levels into context.Robin Watts
In builds that support configurable layers of antialiasing, move the variables that control this into the context. This makes it possible to safely use different levels of antialiasing in different threads.
2011-11-28Move Freetype globals into context.Robin Watts
Freetype globals are not shared between threads currently - to do that we'll need to introduce a lock.
2011-11-15Merge branch 'master' into contextRobin Watts
Mostly redoing the xps_context to xps_document change and adding contexts to newly written code. Conflicts: apps/pdfapp.c apps/pdfapp.h apps/x11_main.c apps/xpsdraw.c draw/draw_device.c draw/draw_scale.c fitz/base_object.c fitz/fitz.h pdf/mupdf.h pdf/pdf_interpret.c pdf/pdf_outline.c pdf/pdf_page.c xps/muxps.h xps/xps_doc.c xps/xps_xml.c
2011-11-01Tweak build scripts for iOS viewer.Tor Andersson
2011-10-04Reintroduce alloc context section.Robin Watts
This was removed during a previous commit to make the editing easier. Now added back in.
2011-10-04Move to exception handling rather than error passing throughout.Robin Watts
This frees us from passing errors back everywhere, and hence enables us to pass results back as return values. Rather than having to explicitly check for errors everywhere and bubble them, we now allow exception handling to do the work for us; the downside to this is that we no longer emit as much debugging information as we did before (though this could be put back in). For now, the debugging information we have lost has been retained in comments with 'RJW:' at the start. This code needs fuller testing, but is being committed as a work in progress.
2011-09-21Rename malloc functions for arrays (fz_calloc and fz_realloc).Tor Andersson
2011-09-21Don't thread ctx through safe fz_obj functions.Tor Andersson
2011-09-20Reshuffle exception context code to fit Tor's aesthetic sense.Tor Andersson
2011-09-15Add context to mupdf.Robin Watts
Huge pervasive change to lots of files, adding a context for exception handling and allocation. In time we'll move more statics into there. Also fix some for(i = 0; i < function(...); i++) calls.
2011-09-14Initial import of exception handling codeRobin Watts
Import exception handling code from WSS, modified to fit into the fitz world. With this code we have 'real' fz_try/fz_catch/fz_rethrow functions, handling a fz_except type. We therefore rename the existing fz_throw/ fz_catch/fz_rethrow to be fz_error_make/fz_error_handle/fz_error_note. We don't actually use fz_try/fz_catch/fz_rethrow yet...
2011-09-14Add Memento build option.Robin Watts
Import memento files. Update Makefile to support: make build=memento (like debug, but with memento too). Tiny tweak to cmapdump to avoid memento building that utility.
2011-04-08Add special case non-aa scan converter with accompanying blit functions.Tor Andersson
Also turn on font hinting when rendering non-aa text.
2011-04-07Add text antialiasing too.Robin Watts
Update the text rendering code to use the raster renderer in freetype rather than the smooth one. No change to the hinting methods used yet.
2011-04-06pdf: Change how CMap and font data files are generated and compiled.Tor Andersson
2011-04-06pdf: Optimize storage for the Adobe Glyph List.Tor Andersson
2011-04-06Move scripts and config files into "scripts" directory.Tor Andersson