Lock the command channel while searching. (imap_body_contains): If

2000-12-24  Not Zed  <NotZed@HelixCode.com>

        * providers/imap/camel-imap-search.c (imap_body_contains): Lock
        the command channel while searching.
        (imap_body_contains): If performing a whole uid search, then add
        references to our own summary items, dont look it up in the
        folder.  This way they can't vanish unexpectedly.

        * providers/imap/camel-imap-folder.h (CamelImapFolder): Added a
        private field.

        * providers/imap/camel-imap-private.h: Added lock for imap
        searches.

        * Merge from camel-mt-branch.

        * providers/imap/camel-imap-folder.c (imap_update_summary): Merge
        fix, use the folder->summary.
        (imap_get_message_flags, imap_set_message_flags,
        imap_get_message_user_flag, imap_set_message_user_flag): Removed
        again.
        (camel_imap_folder_init): Setup private data/lock.
        (imap_finalize): Free private data/search lock.
        (imap_search_free): Lock the search_lock.
        (imap_search_by_expression): Lock the search lock when using the
        search object.  Also copy/ref hte summary, rather than getting it
        directly.
        (imap_refresh_info): Free any info lookups.  Use folder->summary
        not imap_folder->summary.  And lock around commands.

svn path=/trunk/; revision=7150

1 files changed, 171 insertions, 0 deletions

diff --git a/camel/README.mt b/camel/README.mt
new file mode 100644
index 0000000000..aeece1b0bb
--- /dev/null
+++ b/camel/README.mt
@@ -0,0 +1,171 @@
+
+This version of camel is working towards being multi-thread safe
+(MT-SAFE).  At least, for the important api's.
+
+This code has now been merged into the main head, but this file
+will remain here as a log of how it was done, incase any issues
+arise.  The ChangeLog of course has a much more detailed list
+of changes.
+
+Intended method
+===============
+
+I intend working on it in several stages:
+
+1. Making the api multi-threadable.  Basically removing some const-returns,
+and copying some data where it wasn't before.  The api should
+still continue to work if not being used in a multithreaded
+application.  There is not a significant amount of work here since
+this was more or less the intention all along.
+
+Some functions where references to objects are returned may have to be
+changed slightly, so that refcounts are incremented before return.
+This doesn't affect much though.
+
+camel_folder::get_message_info				done
+camel_folder_summary::uid				done
+camel_folder_summary::index				done
+camel_folder::get_summary
+	Needs to ref each summary item it points to.	done
+camel_folder::free_summary
+	Needs to unref each summary item it points to.	done
+camel_folder_get_message_tag
+	needs to copy the tag return
+camel_maildir_summary filename string
+	should not be able to modify the string
+	array contents after it has been added to
+	the summary.
+camel_folder						done
+	Make every camel-folder use a camel-folder-summary.
+	This just reduces some of the code duplication,
+	since everything but vee-folder does this already.
+
+2. Adding high level locks for proof of concept.  The locks will
+be stored in private or global data, so the api should remain the same for
+non-threaded applications.
+
+A per-folder lock which governs access to the folder
+	summary, the folder file or
+	communications socket, etc.			done
+Locking for exceptions.					done
+Per store locks for internal stuff.			done
+Per-service locks for various internal lists and
+	caches						done
+
+3. Further fine-grained locking where it can be done/is worthwhile.
+
+A per-index lock for libibex				done
+Locking for the search object				half done
+Internal lock for the folder_summary itself
+	So that searching can be detatched from other
+	folder operations, etc.				done
+Possibly a lock for access to parts of a mime-part
+	or message
+
+4. A method to cancel operations.
+
+Individual outstanding operations must be cancellable, and not just
+'all current operations'.  This will probably not use pthread_cancel
+type of cancelling.
+
+This will however, probably use a method for starting a new thread,
+through camel, that can then be cancelled, and/or some method of
+registering that a thread can be cancelled.  Blocking states within
+camel, within that thread, will then act as checkpoints for if the
+operation, and if it is cancelled, the operation will abort
+(i.e. fail, with an appropriate exception code).
+
+Operation cancelling should also function when the application is not
+multi-threaded.  Not sure of the api for this yet, probably a callback
+system.  Hopefully the api for both scenarios can be made the same.
+
+Other thoughts
+==============
+
+Basically much of the code in camel that does the actual work does NOT
+need to be thread safe to make it safely usable in an mt context.
+
+camel-folder, camel-summary, camel-imap-search, and the camel-service
+classes (at least) are the important ones to be made multithreaded.
+
+For other things, they are either resources that are created
+one-off (for example, camel-mime-message, and its associated
+parts, like camel-internet-address), or multithreadedness
+doesn't make a lot of sense - e.g. camel-stream, or camel-mime-parser.
+
+So basically the approach is a low-risk one.  Adding the minimum
+number of locks to start with, and providing further fine-grained
+locks as required.  The locks should not need to be particularly
+fine-grained in order to get reasonable results.
+
+Log of changes
+==============
+
+Changed CamelFolder:get_message_info() to return a ref'd copy, requiring
+all get_message_info()'s to have a matching free_message_info().
+
+Moved the CamelFolder frozen changelog data to a private structure.
+
+Added a mutex for CamelFolder frozen changelog stuff (it was just easy
+to do, although it isn't needed yet).
+
+Added a single mutex around all other CamelFolder functions that need
+it, this is just the first cut at mt'edness.
+
+Fixed all camel-folder implementations that call any other
+camel-folder functions to call via virtual methods, to bypass the locks.
+
+Added camel-store private data.
+
+Added a single mutex lock for camel-store's folder functions.
+
+Added camel-service private data.
+
+Added a single mutex lock for camel-service's connect stuff.
+
+Added a mutex for remote-store stream io stuff.
+
+Added a mutex for imap, so it can bracket a compound command
+exclusively.  Pop doesn't need this since you can only have a single
+folder per store, and the folder interface is already forced
+single-threaded.
+
+Added mutex for camel-session, most operations.
+
+Running the tests finds at least 1 deadlock so far.  Need to
+work on that.
+
+Fixed get_summary to ref/unref its items.
+
+Removed the global folder lock from the toplevel
+camel_folder_search(), each implementation must now handle locking.
+
+Fixed the local-folder implementation of searching.  imap-folder
+searching should already be mt-safe through the command lock.
+
+Fixed imap summary to ref/unref too.
+
+Built some test cases, and expanded the test framework library to
+handle multiple threads.  It works!
+
+Next, added a recursive mutex class, so that locking inside imap had
+any chance of working.  Got imap working.
+
+Moved the camel folder summary into the base folder class, and fixed
+everything to use it that way.
+
+Made the vfolder use a real camel-folder-summary rather than a
+hashtable + array that it was using, and probably fixed some problems
+which caused evolution-mail not to always catch flag updates.  Oh, and
+made it sync/expunge all its subfolders when sync/expungeing.
+
+Made the camel-folder summary completely mt-safe.
+
+Removed all of the locks on the folder functions dealing directly with
+the summary, so now for example all summary lookups will not be
+interupted by long operations.
+
+Made the nntp newsrc thing mt-safe, because of some unfortunate
+sideeffect of it being called from the summary interaction code in
+nntp-folder.
+