aboutsummaryrefslogtreecommitdiffstats
path: root/camel/README.mt
diff options
context:
space:
mode:
Diffstat (limited to 'camel/README.mt')
-rw-r--r--camel/README.mt171
1 files changed, 171 insertions, 0 deletions
diff --git a/camel/README.mt b/camel/README.mt
new file mode 100644
index 0000000000..aeece1b0bb
--- /dev/null
+++ b/camel/README.mt
@@ -0,0 +1,171 @@
+
+This version of camel is working towards being multi-thread safe
+(MT-SAFE). At least, for the important api's.
+
+This code has now been merged into the main head, but this file
+will remain here as a log of how it was done, incase any issues
+arise. The ChangeLog of course has a much more detailed list
+of changes.
+
+Intended method
+===============
+
+I intend working on it in several stages:
+
+1. Making the api multi-threadable. Basically removing some const-returns,
+and copying some data where it wasn't before. The api should
+still continue to work if not being used in a multithreaded
+application. There is not a significant amount of work here since
+this was more or less the intention all along.
+
+Some functions where references to objects are returned may have to be
+changed slightly, so that refcounts are incremented before return.
+This doesn't affect much though.
+
+camel_folder::get_message_info done
+camel_folder_summary::uid done
+camel_folder_summary::index done
+camel_folder::get_summary
+ Needs to ref each summary item it points to. done
+camel_folder::free_summary
+ Needs to unref each summary item it points to. done
+camel_folder_get_message_tag
+ needs to copy the tag return
+camel_maildir_summary filename string
+ should not be able to modify the string
+ array contents after it has been added to
+ the summary.
+camel_folder done
+ Make every camel-folder use a camel-folder-summary.
+ This just reduces some of the code duplication,
+ since everything but vee-folder does this already.
+
+2. Adding high level locks for proof of concept. The locks will
+be stored in private or global data, so the api should remain the same for
+non-threaded applications.
+
+A per-folder lock which governs access to the folder
+ summary, the folder file or
+ communications socket, etc. done
+Locking for exceptions. done
+Per store locks for internal stuff. done
+Per-service locks for various internal lists and
+ caches done
+
+3. Further fine-grained locking where it can be done/is worthwhile.
+
+A per-index lock for libibex done
+Locking for the search object half done
+Internal lock for the folder_summary itself
+ So that searching can be detatched from other
+ folder operations, etc. done
+Possibly a lock for access to parts of a mime-part
+ or message
+
+4. A method to cancel operations.
+
+Individual outstanding operations must be cancellable, and not just
+'all current operations'. This will probably not use pthread_cancel
+type of cancelling.
+
+This will however, probably use a method for starting a new thread,
+through camel, that can then be cancelled, and/or some method of
+registering that a thread can be cancelled. Blocking states within
+camel, within that thread, will then act as checkpoints for if the
+operation, and if it is cancelled, the operation will abort
+(i.e. fail, with an appropriate exception code).
+
+Operation cancelling should also function when the application is not
+multi-threaded. Not sure of the api for this yet, probably a callback
+system. Hopefully the api for both scenarios can be made the same.
+
+Other thoughts
+==============
+
+Basically much of the code in camel that does the actual work does NOT
+need to be thread safe to make it safely usable in an mt context.
+
+camel-folder, camel-summary, camel-imap-search, and the camel-service
+classes (at least) are the important ones to be made multithreaded.
+
+For other things, they are either resources that are created
+one-off (for example, camel-mime-message, and its associated
+parts, like camel-internet-address), or multithreadedness
+doesn't make a lot of sense - e.g. camel-stream, or camel-mime-parser.
+
+So basically the approach is a low-risk one. Adding the minimum
+number of locks to start with, and providing further fine-grained
+locks as required. The locks should not need to be particularly
+fine-grained in order to get reasonable results.
+
+Log of changes
+==============
+
+Changed CamelFolder:get_message_info() to return a ref'd copy, requiring
+all get_message_info()'s to have a matching free_message_info().
+
+Moved the CamelFolder frozen changelog data to a private structure.
+
+Added a mutex for CamelFolder frozen changelog stuff (it was just easy
+to do, although it isn't needed yet).
+
+Added a single mutex around all other CamelFolder functions that need
+it, this is just the first cut at mt'edness.
+
+Fixed all camel-folder implementations that call any other
+camel-folder functions to call via virtual methods, to bypass the locks.
+
+Added camel-store private data.
+
+Added a single mutex lock for camel-store's folder functions.
+
+Added camel-service private data.
+
+Added a single mutex lock for camel-service's connect stuff.
+
+Added a mutex for remote-store stream io stuff.
+
+Added a mutex for imap, so it can bracket a compound command
+exclusively. Pop doesn't need this since you can only have a single
+folder per store, and the folder interface is already forced
+single-threaded.
+
+Added mutex for camel-session, most operations.
+
+Running the tests finds at least 1 deadlock so far. Need to
+work on that.
+
+Fixed get_summary to ref/unref its items.
+
+Removed the global folder lock from the toplevel
+camel_folder_search(), each implementation must now handle locking.
+
+Fixed the local-folder implementation of searching. imap-folder
+searching should already be mt-safe through the command lock.
+
+Fixed imap summary to ref/unref too.
+
+Built some test cases, and expanded the test framework library to
+handle multiple threads. It works!
+
+Next, added a recursive mutex class, so that locking inside imap had
+any chance of working. Got imap working.
+
+Moved the camel folder summary into the base folder class, and fixed
+everything to use it that way.
+
+Made the vfolder use a real camel-folder-summary rather than a
+hashtable + array that it was using, and probably fixed some problems
+which caused evolution-mail not to always catch flag updates. Oh, and
+made it sync/expunge all its subfolders when sync/expungeing.
+
+Made the camel-folder summary completely mt-safe.
+
+Removed all of the locks on the folder functions dealing directly with
+the summary, so now for example all summary lookups will not be
+interupted by long operations.
+
+Made the nntp newsrc thing mt-safe, because of some unfortunate
+sideeffect of it being called from the summary interaction code in
+nntp-folder.
+