Use a bag instead of a hashtable to track the cache streams.

2002-12-07  Not Zed  <NotZed@Ximian.com>

	* camel-data-cache.c (data_cache_init): Use a bag instead of a
	hashtable to track the cache streams.
	(data_cache_finalise): Same.
	(free_busy): No longer needed.
	(data_cache_expire): use bag instead of hashtable.
	(stream_finalised): No longer required.
	(camel_data_cache_add): objectbagise
	(camel_data_cache_get): "
	(camel_data_cache_remove): "
	(data_cache_path): Set the now expired date before running expiry,
	so it plays better with multiple threads.  Still a couple of
	harmless races.

2002-12-06  Not Zed  <NotZed@Ximian.com>

	* providers/local/camel-spool-store.c (scan_dir): folders ->
	object bag.
	(get_folder_info_mbox): folders -> object bag.

	* providers/local/camel-mh-store.c (folder_info_new): folders ->
	object bag.

	* providers/local/camel-maildir-store.c (scan_dir): folders ->
	object bag.

	* providers/local/camel-local-store.c (rename_folder): folders ->
	object bag.

	* camel-private.h (CamelStorePrivate): Remove 'cache' lock,
	handled by the objectbag.

	* providers/imap/camel-imap-store.c (copy_folder): Removed.
	(imap_store_refresh_folders): folders -> object bag.
	(get_folder_counts): folders -> object bag.

	* camel-vee-store.c (vee_get_folder): changes for folders
	objectbag.
	(vee_get_folder_info): Change to use folders objectbag.  Also,
	dont refresh the base folder if we're in FAST mode.
	(build_info): Removed, no longer needed.
	(vee_rename_folder): Fixed for folders objectbag.

	* camel-store.c (camel_store_init): init the folders objectbag.
	(camel_store_finalize): Destroy the folders object bag.
	(folder_matches):
	(folder_finalize): Removed, now handled implicitly by the
	objectbag.
	(camel_store_get_folder): object bag changes.
	(camel_store_delete_folder): "
	(get_subfolders): Removed, now handled without a callback.
	(camel_store_rename_folder): Changed to use object bag of folders.
	(trash_add_folder): Removed.
	(init_trash): use folders object bag.
	(copy_folder_cache):
	(sync_folder): Removed, no longer needed.  Weird arsed code anyway.
	(store_sync): Use folder object bag instead of hashtable.
	(camel_store_unsubscribe_folder): "
	(camel_store_init): remove cache_lock init, no longer used.
	(camel_store_finalize): Same for cleanup.

2002-12-05  Not Zed  <NotZed@Ximian.com>

	* camel-store.h (struct _CamelStore): change folders from a
	hashtable into a CamelObjectBag.

	* camel-object.c (camel_object_ref): Use type_lock instead of
	class lock for ref counting.
	(camel_object_unref): Use type_lock instead of class lock for
	unref.
	(camel_object_unref): If the object is 'bagged', then also look
	hooks, and remove it from any bags.
	(camel_object_bag_new):
	(camel_object_bag_destroy):
	(camel_object_bag_add):
	(camel_object_bag_get):
	(camel_object_bag_remove_unlocked):
	(camel_object_bag_list):
	(camel_object_bag_abort):
	(camel_object_bag_remove): New functions to implement a utility
	object which can manage a 'bag' of weakly ref'd children in an
	atomic & threadsafe way.

svn path=/trunk/; revision=19056

13 files changed, 608 insertions, 449 deletions

diff --git a/camel/ChangeLog b/camel/ChangeLog
index 9db9d5ec0f..1e81abd4e3 100644
--- a/camel/ChangeLog
+++ b/camel/ChangeLog
@@ -1,3 +1,87 @@
+2002-12-07  Not Zed  <NotZed@Ximian.com>
+
+	* camel-data-cache.c (data_cache_init): Use a bag instead of a
+	hashtable to track the cache streams.
+	(data_cache_finalise): Same.
+	(free_busy): No longer needed.
+	(data_cache_expire): use bag instead of hashtable.
+	(stream_finalised): No longer required.
+	(camel_data_cache_add): objectbagise
+	(camel_data_cache_get): "
+	(camel_data_cache_remove): "
+	(data_cache_path): Set the now expired date before running expiry,
+	so it plays better with multiple threads.  Still a couple of
+	harmless races.
+
+2002-12-06  Not Zed  <NotZed@Ximian.com>
+
+	* providers/local/camel-spool-store.c (scan_dir): folders ->
+	object bag.
+	(get_folder_info_mbox): folders -> object bag.
+
+	* providers/local/camel-mh-store.c (folder_info_new): folders ->
+	object bag.
+
+	* providers/local/camel-maildir-store.c (scan_dir): folders ->
+	object bag.
+
+	* providers/local/camel-local-store.c (rename_folder): folders ->
+	object bag.
+
+	* camel-private.h (CamelStorePrivate): Remove 'cache' lock,
+	handled by the objectbag.
+
+	* providers/imap/camel-imap-store.c (copy_folder): Removed.
+	(imap_store_refresh_folders): folders -> object bag.
+	(get_folder_counts): folders -> object bag.
+
+	* camel-vee-store.c (vee_get_folder): changes for folders
+	objectbag.
+	(vee_get_folder_info): Change to use folders objectbag.  Also,
+	dont refresh the base folder if we're in FAST mode.
+	(build_info): Removed, no longer needed.
+	(vee_rename_folder): Fixed for folders objectbag.
+
+	* camel-store.c (camel_store_init): init the folders objectbag.
+	(camel_store_finalize): Destroy the folders object bag.
+	(folder_matches): 
+	(folder_finalize): Removed, now handled implicitly by the
+	objectbag.
+	(camel_store_get_folder): object bag changes.
+	(camel_store_delete_folder): "
+	(get_subfolders): Removed, now handled without a callback.
+	(camel_store_rename_folder): Changed to use object bag of folders.	
+	(trash_add_folder): Removed.
+	(init_trash): use folders object bag.
+	(copy_folder_cache): 
+	(sync_folder): Removed, no longer needed.  Weird arsed code anyway.
+	(store_sync): Use folder object bag instead of hashtable.
+	(camel_store_unsubscribe_folder): "
+	(camel_store_init): remove cache_lock init, no longer used.
+	(camel_store_finalize): Same for cleanup.
+
+2002-12-05  Not Zed  <NotZed@Ximian.com>
+
+	* camel-store.h (struct _CamelStore): change folders from a
+	hashtable into a CamelObjectBag.
+
+	* camel-object.c (camel_object_ref): Use type_lock instead of
+	class lock for ref counting.
+	(camel_object_unref): Use type_lock instead of class lock for
+	unref.
+	(camel_object_unref): If the object is 'bagged', then also look
+	hooks, and remove it from any bags.
+	(camel_object_bag_new):
+	(camel_object_bag_destroy):
+	(camel_object_bag_add):
+	(camel_object_bag_get):
+	(camel_object_bag_remove_unlocked):
+	(camel_object_bag_list): 
+	(camel_object_bag_abort): 
+	(camel_object_bag_remove): New functions to implement a utility
+	object which can manage a 'bag' of weakly ref'd children in an
+	atomic & threadsafe way.
+
 2002-12-04  Jeffrey Stedfast  <fejj@ximian.com>
 
 	* providers/imap/camel-imap-store-summary.c
diff --git a/camel/camel-data-cache.c b/camel/camel-data-cache.c
index 6147a888f1..c0d399bbff 100644
--- a/camel/camel-data-cache.c
+++ b/camel/camel-data-cache.c
@@ -55,27 +55,17 @@ static void stream_finalised(CamelObject *o, void *event_data, void *data);
 #define CAMEL_DATA_CACHE_CYCLE_TIME (60*60)
 
 struct _CamelDataCachePrivate {
-	GHashTable *busy_stream;
-	GHashTable *busy_path;
+	CamelObjectBag *busy_bag;
 
 	int expire_inc;
 	time_t expire_last[1<<CAMEL_DATA_CACHE_BITS];
-
-#ifdef ENABLE_THREADS
-	GMutex *lock;
-#define CDC_LOCK(c, l) g_mutex_lock(((CamelDataCache *)(c))->priv->l)
-#define CDC_UNLOCK(c, l) g_mutex_unlock(((CamelDataCache *)(c))->priv->l)
-#else
-#define CDC_LOCK(c, l)
-#define CDC_UNLOCK(c, l)
-#endif
 };
 
 static CamelObject *camel_data_cache_parent;
 
 static void data_cache_class_init(CamelDataCacheClass *klass)
 {
-	camel_data_cache_parent = (CamelObject *)camel_type_get_global_classfuncs (camel_object_get_type ());
+	camel_data_cache_parent = (CamelObject *)camel_object_get_type ();
 
 #if 0
 	klass->add = data_cache_add;
@@ -91,39 +81,17 @@ static void data_cache_init(CamelDataCache *cdc, CamelDataCacheClass *klass)
 	struct _CamelDataCachePrivate *p;
 
 	p = cdc->priv = g_malloc0(sizeof(*cdc->priv));
-
-	p->busy_stream = g_hash_table_new(NULL, NULL);
-	p->busy_path = g_hash_table_new(g_str_hash, g_str_equal);
-
-#ifdef ENABLE_THREADS
-	p->lock = g_mutex_new();
-#endif
-}
-
-static void
-free_busy(CamelStream *stream, char *path, CamelDataCache *cdc)
-{
-	d(printf(" Freeing busy stream %p path %s\n", stream, path));
-	camel_object_unhook_event((CamelObject *)stream, "finalize", stream_finalised, cdc);
-	camel_object_unref((CamelObject *)stream);
-	g_free(path);
+	p->busy_bag = camel_object_bag_new(g_str_hash, g_str_equal);
 }
 
 static void data_cache_finalise(CamelDataCache *cdc)
 {
 	struct _CamelDataCachePrivate *p;
+	GPtrArray *streams;
+	int i;
 
 	p = cdc->priv;
-
-	d(printf("cache finalised, %d (= %d?) streams reamining\n", g_hash_table_size(p->busy_stream), g_hash_table_size(p->busy_path)));
-
-	g_hash_table_foreach(p->busy_stream, (GHFunc)free_busy, cdc);
-	g_hash_table_destroy(p->busy_path);
-	g_hash_table_destroy(p->busy_stream);
-
-#ifdef ENABLE_THREADS
-	g_mutex_free(p->lock);
-#endif
+	camel_object_bag_destroy(p->busy_bag);
 	g_free(p);
 	
 	g_free (cdc->path);
@@ -249,10 +217,10 @@ data_cache_expire(CamelDataCache *cdc, const char *path, const char *keep, time_
 			|| (cdc->expire_access != -1 && st.st_atime + cdc->expire_access < now))) {
 			dd(printf("Has expired!  Removing!\n"));
 			unlink(s->str);
-			if (g_hash_table_lookup_extended(cdc->priv->busy_path, s->str, (void **)&oldpath, (void **)&stream)) {
-				g_hash_table_remove(cdc->priv->busy_path, oldpath);
-				g_hash_table_remove(cdc->priv->busy_stream, stream);
-				g_free(oldpath);
+			stream = camel_object_bag_get(cdc->priv->busy_bag, s->str);
+			if (stream) {
+				camel_object_bag_remove(cdc->priv->busy_bag, stream);
+				camel_object_unref(stream);
 			}
 		}
 	}
@@ -283,10 +251,11 @@ data_cache_path(CamelDataCache *cdc, int create, const char *path, const char *k
 
 		dd(printf("Checking expire cycle time on dir '%s'\n", dir));
 
+		/* This has a race, but at worst we re-run an expire cycle which is safe */
 		now = time(0);
 		if (cdc->priv->expire_last[hash] + CAMEL_DATA_CACHE_CYCLE_TIME < now) {
-			data_cache_expire(cdc, dir, key, now);
 			cdc->priv->expire_last[hash] = now;
+			data_cache_expire(cdc, dir, key, now);
 		}
 		cdc->priv->expire_inc = (cdc->priv->expire_inc + 1) & CAMEL_DATA_CACHE_MASK;
 	}
@@ -298,27 +267,6 @@ data_cache_path(CamelDataCache *cdc, int create, const char *path, const char *k
 	return real;
 }
 
-static void
-stream_finalised(CamelObject *o, void *event_data, void *data)
-{
-	CamelDataCache *cdc = data;
-	char *key;
-
-	d(printf("Stream finalised '%p'\n", data));
-
-	CDC_LOCK(cdc, lock);
-	key = g_hash_table_lookup(cdc->priv->busy_stream, o);
-	if (key) {
-		d(printf("  For path '%s'\n", key));
-		g_hash_table_remove(cdc->priv->busy_path, key);
-		g_hash_table_remove(cdc->priv->busy_stream, o);
-		g_free(key);
-	} else {
-		d(printf("  Unknown stream?!\n"));
-	}
-	CDC_UNLOCK(cdc, lock);
-}
-
 /**
  * camel_data_cache_add:
  * @cdc: 
@@ -340,29 +288,24 @@ stream_finalised(CamelObject *o, void *event_data, void *data)
 CamelStream *
 camel_data_cache_add(CamelDataCache *cdc, const char *path, const char *key, CamelException *ex)
 {
-	char *real, *oldpath;
+	char *real;
 	CamelStream *stream;
 
-	CDC_LOCK(cdc, lock);
-
 	real = data_cache_path(cdc, TRUE, path, key);
-	if (g_hash_table_lookup_extended(cdc->priv->busy_path, real, (void **)&oldpath, (void **)&stream)) {
-		g_hash_table_remove(cdc->priv->busy_path, oldpath);
-		g_hash_table_remove(cdc->priv->busy_stream, stream);
-		unlink(oldpath);
-		g_free(oldpath);
+	stream = camel_object_bag_reserve(cdc->priv->busy_bag, real);
+	if (stream) {
+		unlink(real);
+		camel_object_bag_remove(cdc->priv->busy_bag, stream);
+		camel_object_unref(stream);
 	}
 
 	stream = camel_stream_fs_new_with_name(real, O_RDWR|O_CREAT|O_TRUNC, 0600);
-	if (stream) {
-		camel_object_hook_event((CamelObject *)stream, "finalize", stream_finalised, cdc);
-		g_hash_table_insert(cdc->priv->busy_stream, stream, real);
-		g_hash_table_insert(cdc->priv->busy_path, real, stream);
-	} else {
-		g_free(real);
-	}
+	if (stream)
+		camel_object_bag_add(cdc->priv->busy_bag, real, stream);
+	else
+		camel_object_bag_abort(cdc->priv->busy_bag, real);
 
-	CDC_UNLOCK(cdc, lock);
+	g_free(real);
 
 	return stream;
 }
@@ -387,25 +330,16 @@ camel_data_cache_get(CamelDataCache *cdc, const char *path, const char *key, Cam
 	char *real;
 	CamelStream *stream;
 
-	CDC_LOCK(cdc, lock);
-
 	real = data_cache_path(cdc, FALSE, path, key);
-	stream = g_hash_table_lookup(cdc->priv->busy_path, real);
-	if (stream) {
-		camel_object_ref((CamelObject *)stream);
-		g_free(real);
-	} else {
+	stream = camel_object_bag_reserve(cdc->priv->busy_bag, real);
+	if (!stream) {
 		stream = camel_stream_fs_new_with_name(real, O_RDWR, 0600);
-		if (stream) {
-			camel_object_hook_event((CamelObject *)stream, "finalize", stream_finalised, cdc);
-			g_hash_table_insert(cdc->priv->busy_stream, stream, real);
-			g_hash_table_insert(cdc->priv->busy_path, real, stream);
-		} else {
-			g_free (real);
-		}
+		if (stream)
+			camel_object_bag_add(cdc->priv->busy_bag, real, stream);
+		else
+			camel_object_bag_abort(cdc->priv->busy_bag, real);
 	}
-
-	CDC_UNLOCK(cdc, lock);
+	g_free(real);
 
 	return stream;
 }
@@ -425,16 +359,14 @@ int
 camel_data_cache_remove(CamelDataCache *cdc, const char *path, const char *key, CamelException *ex)
 {
 	CamelStream *stream;
-	char *real, *oldpath;
+	char *real;
 	int ret;
 
-	CDC_LOCK(cdc, lock);
-
 	real = data_cache_path(cdc, FALSE, path, key);
-	if (g_hash_table_lookup_extended(cdc->priv->busy_path, real, (void **)&oldpath, (void **)&stream)) {
-		g_hash_table_remove(cdc->priv->busy_path, oldpath);
-		g_hash_table_remove(cdc->priv->busy_stream, stream);
-		g_free(oldpath);
+	stream = camel_object_bag_get(cdc->priv->busy_bag, real);
+	if (stream) {
+		camel_object_bag_remove(cdc->priv->busy_bag, stream);
+		camel_object_unref(stream);
 	}
 
 	/* maybe we were a mem stream */
@@ -449,8 +381,6 @@ camel_data_cache_remove(CamelDataCache *cdc, const char *path, const char *key,
 
 	g_free(real);
 
-	CDC_UNLOCK(cdc, lock);
-
 	return ret;
 }
 
diff --git a/camel/camel-object.c b/camel/camel-object.c
index 6e3b5ef54b..7214518839 100644
--- a/camel/camel-object.c
+++ b/camel/camel-object.c
@@ -35,6 +35,8 @@
 #include <e-util/e-msgport.h>
 #endif
 
+#define d(x)
+
 /* I just mashed the keyboard for these... */
 #define CAMEL_OBJECT_MAGIC           	 0x77A344ED
 #define CAMEL_OBJECT_CLASS_MAGIC     	 0xEE26A997
@@ -77,9 +79,32 @@ typedef struct _CamelHookPair
 	void *data;
 } CamelHookPair;
 
+struct _CamelObjectBag {
+	GHashTable *object_table; /* object by key */
+	GHashTable *key_table;	/* key by object */
+	guint32 flags;
+#ifdef ENABLE_THREADS
+	pthread_t owner;
+	pthread_cond_t cond;
+#endif
+};
+
+#define CAMEL_OBJECT_BAG_RESERVED (1)
+
+/* used to tag a bag hookpair */
+static const char *bag_name = "object:bag";
+
 /* ********************************************************************** */
 
+static CamelHookList *camel_object_get_hooks(CamelObject *o);
 static void camel_object_free_hooks(CamelObject *o);
+static void camel_object_bag_remove_unlocked(CamelObjectBag *inbag, CamelObject *o, CamelHookList *hooks);
+
+#ifdef ENABLE_THREADS
+#define camel_object_unget_hooks(o) (e_mutex_unlock((CAMEL_OBJECT(o)->hooks->lock)))
+#else
+#define camel_object_unget_hooks(o)
+#endif
 
 /* ********************************************************************** */
 
@@ -191,6 +216,8 @@ cobject_init (CamelObject *o, CamelObjectClass *klass)
 static void
 cobject_finalise(CamelObject *o)
 {
+	/*printf("%p: finalise %s\n", o, o->klass->name);*/
+
 	g_assert(o->ref_count == 0);
 
 	camel_object_free_hooks(o);
@@ -392,6 +419,7 @@ camel_object_new(CamelType type)
 		return NULL;
 
 	CLASS_LOCK(type);
+
 	o = e_memchunk_alloc0(type->instance_chunks);
 
 #ifdef CAMEL_OBJECT_TRACK_INSTANCES
@@ -403,8 +431,11 @@ camel_object_new(CamelType type)
 #endif
 
 	CLASS_UNLOCK(type);
+
 	camel_object_init(o, type, type);
 
+	d(printf("%p: new %s()\n", o, o->klass->name));
+
 	return o;
 }
 
@@ -415,9 +446,12 @@ camel_object_ref(void *vo)
 
 	g_return_if_fail(CAMEL_IS_OBJECT(o));
 
-	CLASS_LOCK(o->klass);
+	E_LOCK(type_lock);
+
 	o->ref_count++;
-	CLASS_UNLOCK(o->klass);
+	d(printf("%p: ref %s(%d)\n", o, o->klass->name, o->ref_count));
+
+	E_UNLOCK(type_lock);
 }
 
 void
@@ -425,22 +459,38 @@ camel_object_unref(void *vo)
 {
 	register CamelObject *o = vo;
 	register CamelObjectClass *klass, *k;
+	CamelHookList *hooks = NULL;
 
 	g_return_if_fail(CAMEL_IS_OBJECT(o));
 	
 	klass = o->klass;
 
-	CLASS_LOCK(klass);
+	if (o->hooks)
+		hooks = camel_object_get_hooks(o);
+
+	E_LOCK(type_lock);
+
 	o->ref_count--;
+
+	d(printf("%p: unref %s(%d)\n", o, o->klass->name, o->ref_count));
+
 	if (o->ref_count > 0
 	    || (o->flags & CAMEL_OBJECT_DESTROY)) {
-		CLASS_UNLOCK(klass);
+		E_UNLOCK(type_lock);
+		if (hooks)
+			camel_object_unget_hooks(o);
 		return;
 	}
 
 	o->flags |= CAMEL_OBJECT_DESTROY;
 
-	CLASS_UNLOCK(klass);
+	if (hooks)
+		camel_object_bag_remove_unlocked(NULL, o, hooks);
+
+	E_UNLOCK(type_lock);
+
+	if (hooks)
+		camel_object_unget_hooks(o);
 
 	camel_object_trigger_event(o, "finalize", NULL);
 
@@ -686,13 +736,6 @@ static CamelHookList *camel_object_get_hooks(CamelObject *o)
 	return o->hooks;	
 }
 
-/* unlock object hooks' list */
-#ifdef ENABLE_THREADS
-#define camel_object_unget_hooks(o) (e_mutex_unlock((CAMEL_OBJECT(o)->hooks->lock)))
-#else
-#define camel_object_unget_hooks(o)
-#endif
-
 unsigned int
 camel_object_hook_event(void *vo, const char * name, CamelObjectEventHookFunc func, void *data)
 {
@@ -1040,3 +1083,243 @@ camel_object_class_dump_tree(CamelType root)
 {
 	object_class_dump_tree_rec(root, 0);
 }
+
+CamelObjectBag *camel_object_bag_new(GHashFunc hash, GEqualFunc equal)
+{
+	CamelObjectBag *bag;
+
+	bag = g_malloc(sizeof(*bag));
+	bag->object_table = g_hash_table_new(hash, equal);
+	bag->key_table = g_hash_table_new(NULL, NULL);
+	bag->flags = 0;
+#ifdef ENABLE_THREADS
+	pthread_cond_init(&bag->cond, NULL);
+#endif
+	return bag;
+}
+
+static void
+remove_bag(char *key, CamelObject *o, CamelObjectBag *bag)
+{
+	camel_object_bag_remove(bag, o);
+	g_free(key);
+}
+
+void camel_object_bag_destroy(CamelObjectBag *bag)
+{
+	g_assert(bag->flags == 0);
+
+	g_hash_table_foreach(bag->object_table, (GHFunc)remove_bag, bag);
+	g_hash_table_destroy(bag->object_table);
+	g_hash_table_destroy(bag->key_table);
+	g_free(bag);
+}
+
+void camel_object_bag_add(CamelObjectBag *bag, const char *key, void *vo)
+{
+	CamelObject *o = vo;
+	CamelHookList *hooks;
+	CamelHookPair *pair;
+	char *k;
+
+	hooks = camel_object_get_hooks(o);
+	E_LOCK(type_lock);
+
+	pair = hooks->list;
+	while (pair) {
+		if (pair->name == bag_name && pair->data == bag) {
+			E_UNLOCK(type_lock);
+			camel_object_unget_hooks(o);
+			return;
+		}
+		pair = pair->next;
+	}
+
+	pair = pair_alloc();
+	pair->name = bag_name;
+	pair->data = bag;
+	pair->flags = 0;
+
+	pair->next = hooks->list;
+	hooks->list = pair;
+	hooks->list_length++;
+
+	k = g_strdup(key);
+	g_hash_table_insert(bag->object_table, k, vo);
+	g_hash_table_insert(bag->key_table, vo, k);
+
+#ifdef ENABLE_THREADS
+	if (bag->flags & CAMEL_OBJECT_BAG_RESERVED
+	    && bag->owner == pthread_self()) {
+		bag->flags &= ~CAMEL_OBJECT_BAG_RESERVED;
+		pthread_cond_broadcast(&bag->cond);
+	}
+#endif
+
+	E_UNLOCK(type_lock);
+	camel_object_unget_hooks(o);
+}
+
+void *camel_object_bag_get(CamelObjectBag *bag, const char *key)
+{
+	CamelObject *o;
+	int retry;
+
+	E_LOCK(type_lock);
+
+	do {
+		retry = FALSE;
+		o = g_hash_table_lookup(bag->object_table, key);
+		if (o) {
+			/* we use the same lock as the refcount */
+			o->ref_count++;
+		}
+#ifdef ENABLE_THREADS
+		else {
+			if (bag->flags & CAMEL_OBJECT_BAG_RESERVED) {
+				e_mutex_cond_wait(&bag->cond, type_lock);
+				retry = TRUE;
+			}
+		}
+#endif
+	} while (retry);
+	
+	E_UNLOCK(type_lock);
+
+	return o;
+}
+
+/* like get, but also reserve a spot for key if it isn't there */
+/* After calling reserve, you MUST call bag_abort or bag_add */
+/* Also note that currently you can only reserve a single key
+   at any one time in a given thread */
+void *camel_object_bag_reserve(CamelObjectBag *bag, const char *key)
+{
+	CamelObject *o;
+	int retry;
+
+	E_LOCK(type_lock);
+
+	do {
+		retry = FALSE;
+		o = g_hash_table_lookup(bag->object_table, key);
+		if (o) {
+			/* we use the same lock as the refcount */
+			o->ref_count++;
+		}
+#ifdef ENABLE_THREADS
+		else {
+			/* NOTE: We dont actually reserve the key, we just reserve
+			   the whole bag.  We could do either but this is easier */
+			if (bag->flags & CAMEL_OBJECT_BAG_RESERVED) {
+				e_mutex_cond_wait(&bag->cond, type_lock);
+				retry = TRUE;
+			} else {
+				bag->flags |= CAMEL_OBJECT_BAG_RESERVED;
+				bag->owner = pthread_self();
+			}
+		}
+#endif
+	} while (retry);
+	
+	E_UNLOCK(type_lock);
+
+	return o;
+}
+
+/* abort a reserved key */
+void camel_object_bag_abort(CamelObjectBag *bag, const char *key)
+{
+	CamelObject *o;
+	int retry;
+
+	E_LOCK(type_lock);
+
+	/* NOTE: We dont actually reserve the key, we just reserve
+	   the whole bag.  We could do either but this is easier */
+
+	g_assert(bag->owner == pthread_self());
+	g_assert(bag->flags & CAMEL_OBJECT_BAG_RESERVED);
+
+	bag->owner = 0;
+	bag->flags &= ~CAMEL_OBJECT_BAG_RESERVED;
+
+	pthread_cond_broadcast(&bag->cond);
+	
+	E_UNLOCK(type_lock);
+}
+
+static void
+save_bag(char *key, CamelObject *o, GPtrArray *list)
+{
+	/* we have the refcount lock already */
+	o->ref_count++;
+	g_ptr_array_add(list, o);
+}
+
+/* get a list of all objects in the bag, ref'd */
+GPtrArray *camel_object_bag_list(CamelObjectBag *bag)
+{
+	GPtrArray *list;
+
+	list = g_ptr_array_new();
+	E_LOCK(type_lock);
+
+	while (bag->flags & CAMEL_OBJECT_BAG_RESERVED)
+		e_mutex_cond_wait(&bag->cond, type_lock);
+
+	g_hash_table_foreach(bag->object_table, (GHFunc)save_bag, list);
+	E_UNLOCK(type_lock);
+
+	return list;
+}
+
+/* if bag is NULL, remove all bags from object */
+static void camel_object_bag_remove_unlocked(CamelObjectBag *inbag, CamelObject *o, CamelHookList *hooks)
+{
+	CamelHookPair *pair, *parent;
+	char *oldkey;
+	CamelObjectBag *bag;
+
+	parent = (CamelHookPair *)&hooks->list;
+	pair = parent->next;
+	while (pair) {
+		if (pair->name == bag_name
+		    && (inbag == NULL || inbag == pair->data)) {
+			bag = pair->data;
+			/* lookup object in table? */
+			oldkey = g_hash_table_lookup(bag->key_table, o);
+			if (oldkey) {
+				g_hash_table_remove(bag->key_table, o);
+				g_hash_table_remove(bag->object_table, oldkey);
+				g_free(oldkey);
+			}
+			parent->next = pair->next;
+			pair_free(pair);
+			hooks->list_length--;
+		} else {
+			parent = pair;
+		}
+		pair = parent->next;
+	}
+}
+
+void camel_object_bag_remove(CamelObjectBag *inbag, void *vo)
+{
+	CamelObject *o = vo;
+	CamelHookPair *pair, *parent;
+	CamelHookList *hooks;
+	char *oldkey;
+	CamelObjectBag *bag;
+
+	if (o->hooks == NULL)
+		return;
+
+	hooks = camel_object_get_hooks(o);
+	E_LOCK(type_lock);
+
+	camel_object_bag_remove_unlocked(inbag, o, hooks);
+		
+	E_UNLOCK(type_lock);
+	camel_object_unget_hooks(o);
+}
diff --git a/camel/camel-object.h b/camel/camel-object.h
index 785429b304..30da23c132 100644
--- a/camel/camel-object.h
+++ b/camel/camel-object.h
@@ -46,7 +46,7 @@ extern "C" {
 #endif
 
 /* turn on so that camel_object_class_dump_tree() dumps object instances as well */
-/*#define CAMEL_OBJECT_TRACK_INSTANCES*/
+#define CAMEL_OBJECT_TRACK_INSTANCES
 
 typedef struct _CamelObjectClass *CamelType;
 
@@ -214,6 +214,18 @@ int camel_object_getv(void *obj, struct _CamelException *ex, CamelArgGetV *);
 /* free a bunch of objects, list must be 0 terminated */
 void camel_object_free(void *vo, guint32 tag, void *value);
 
+/* for managing bags of weakly-ref'd 'child' objects */
+typedef struct _CamelObjectBag CamelObjectBag;
+
+CamelObjectBag *camel_object_bag_new(GHashFunc hash, GEqualFunc equal);
+void *camel_object_bag_get(CamelObjectBag *bag, const char *key);
+void *camel_object_bag_reserve(CamelObjectBag *bag, const char *key);
+void camel_object_bag_add(CamelObjectBag *bag, const char *key, void *o);
+void camel_object_bag_abort(CamelObjectBag *bag, const char *key);
+GPtrArray *camel_object_bag_list(CamelObjectBag *bag);
+void camel_object_bag_remove(CamelObjectBag *bag, void *o);
+void camel_object_bag_destroy(CamelObjectBag *bag);
+
 #ifdef __cplusplus
 }
 #endif /* __cplusplus */
diff --git a/camel/camel-private.h b/camel/camel-private.h
index 5165f31c8b..6de65b54eb 100644
--- a/camel/camel-private.h
+++ b/camel/camel-private.h
@@ -59,7 +59,6 @@ struct _CamelFolderPrivate {
 struct _CamelStorePrivate {
 #ifdef ENABLE_THREADS
 	EMutex *folder_lock;	/* for locking folder operations */
-	EMutex *cache_lock;	/* for locking access to the cache */
 #else
 	int dummy;
 #endif
diff --git a/camel/camel-store.c b/camel/camel-store.c
index d76af40e60..c6454fc8f5 100644
--- a/camel/camel-store.c
+++ b/camel/camel-store.c
@@ -126,8 +126,8 @@ camel_store_init (void *o)
 	CamelStoreClass *store_class = (CamelStoreClass *)CAMEL_OBJECT_GET_CLASS (o);
 
 	if (store_class->hash_folder_name) {
-		store->folders = g_hash_table_new (store_class->hash_folder_name,
-						   store_class->compare_folder_name);
+		store->folders = camel_object_bag_new(store_class->hash_folder_name,
+						      store_class->compare_folder_name);
 	} else
 		store->folders = NULL;
 	
@@ -139,7 +139,6 @@ camel_store_init (void *o)
 	store->priv = g_malloc0 (sizeof (*store->priv));
 #ifdef ENABLE_THREADS
 	store->priv->folder_lock = e_mutex_new (E_MUTEX_REC);
-	store->priv->cache_lock = e_mutex_new (E_MUTEX_SIMPLE);
 #endif
 }
 
@@ -148,18 +147,11 @@ camel_store_finalize (CamelObject *object)
 {
 	CamelStore *store = CAMEL_STORE (object);
 
-	if (store->folders) {
-		if (g_hash_table_size (store->folders) != 0) {
-			d(g_warning ("Folder cache for store %p contains "
-				     "%d folders at destruction.", store,
-				     g_hash_table_size (store->folders)));
-		}
-		g_hash_table_destroy (store->folders);
-	}
+	if (store->folders)
+		camel_object_bag_destroy(store->folders);
 	
 #ifdef ENABLE_THREADS
 	e_mutex_destroy (store->priv->folder_lock);
-	e_mutex_destroy (store->priv->cache_lock);
 #endif
 	g_free (store->priv);
 }
@@ -197,28 +189,6 @@ store_getv (CamelObject *object, CamelException *ex, CamelArgGetV *args)
 	return CAMEL_OBJECT_CLASS (parent_class)->getv (object, ex, args);
 }
 
-static gboolean
-folder_matches (gpointer key, gpointer value, gpointer user_data)
-{
-	if (value == user_data) {
-		g_free (key);
-		return TRUE;
-	} else
-		return FALSE;
-}
-
-static void
-folder_finalize (CamelObject *folder, gpointer event_data, gpointer user_data)
-{
-	CamelStore *store = CAMEL_STORE (user_data);
-
-	if (store->folders) {
-		CAMEL_STORE_LOCK(store, cache_lock);
-		g_hash_table_foreach_remove (store->folders, folder_matches, folder);
-		CAMEL_STORE_UNLOCK(store, cache_lock);
-	}
-}
-
 static void
 construct (CamelService *service, CamelSession *session,
 	   CamelProvider *provider, CamelURL *url,
@@ -264,14 +234,9 @@ camel_store_get_folder (CamelStore *store, const char *folder_name, guint32 flag
 	
 	CAMEL_STORE_LOCK(store, folder_lock);
 	
-	if (store->folders) {
+	if (store->folders)
 		/* Try cache first. */
-		CAMEL_STORE_LOCK(store, cache_lock);
-		folder = g_hash_table_lookup (store->folders, folder_name);
-		if (folder)
-			camel_object_ref (CAMEL_OBJECT (folder));
-		CAMEL_STORE_UNLOCK(store, cache_lock);
-	}
+		folder = camel_object_bag_reserve(store->folders, folder_name);
 	
 	if (!folder) {
 		folder = CS_CLASS (store)->get_folder (store, folder_name, flags, ex);
@@ -280,17 +245,14 @@ camel_store_get_folder (CamelStore *store, const char *folder_name, guint32 flag
 			if (store->vtrash)
 				camel_vee_folder_add_folder (CAMEL_VEE_FOLDER (store->vtrash), folder);
 			
-			if (store->folders) {
-				CAMEL_STORE_LOCK(store, cache_lock);
-				
-				g_hash_table_insert (store->folders, g_strdup (folder_name), folder);
-				
-				camel_object_hook_event (CAMEL_OBJECT (folder), "finalize", folder_finalize, store);
-				CAMEL_STORE_UNLOCK(store, cache_lock);
-			}
+			if (store->folders)
+				camel_object_bag_add(store->folders, folder_name, folder);
+		} else {
+			if (store->folders)
+				camel_object_bag_abort(store->folders, folder_name);
 		}
 	}
-	
+
 	CAMEL_STORE_UNLOCK(store, folder_lock);
 	return folder;
 }
@@ -362,12 +324,7 @@ camel_store_delete_folder (CamelStore *store, const char *folder_name, CamelExce
 	
 	/* if we deleted a folder, force it out of the cache, and also out of the vtrash if setup */
 	if (store->folders) {
-		CAMEL_STORE_LOCK(store, cache_lock);
-		folder = g_hash_table_lookup(store->folders, folder_name);
-		if (folder)
-			camel_object_ref((CamelObject *)folder);
-		CAMEL_STORE_UNLOCK(store, cache_lock);
-
+		folder = camel_object_bag_get(store->folders, folder_name);
 		if (folder) {
 			if (store->vtrash)
 				camel_vee_folder_remove_folder((CamelVeeFolder *)store->vtrash, folder);
@@ -377,17 +334,11 @@ camel_store_delete_folder (CamelStore *store, const char *folder_name, CamelExce
 
 	CS_CLASS (store)->delete_folder (store, folder_name, ex);
 
-	if (folder)
-		camel_object_unref((CamelObject *)folder);
+	if (store->folders)
+		camel_object_bag_remove(store->folders, folder);
 
-	if (store->folders) {
-		CAMEL_STORE_LOCK(store, cache_lock);
-		if (g_hash_table_lookup_extended(store->folders, folder_name, (void **)&key, (void **)&folder)) {
-			g_hash_table_remove (store->folders, key);
-			g_free (key);
-		}
-		CAMEL_STORE_UNLOCK(store, cache_lock);
-	}
+	if (folder)
+		camel_object_unref(folder);
 	
 	CAMEL_STORE_UNLOCK(store, folder_lock);
 }
@@ -399,34 +350,6 @@ rename_folder (CamelStore *store, const char *old_name, const char *new_name, Ca
 		     camel_type_to_name (CAMEL_OBJECT_GET_TYPE (store))));
 }
 
-struct _get_info {
-	CamelStore *store;
-	GPtrArray *folders;
-	const char *old;
-	const char *new;
-};
-
-static void
-get_subfolders(char *key, CamelFolder *folder, struct _get_info *info)
-{
-	int oldlen, namelen;
-
-	namelen = strlen(folder->full_name);
-	oldlen = strlen(info->old);
-
-	if ((namelen == oldlen &&
-	     strcmp(folder->full_name, info->old) == 0)
-	    || ((namelen > oldlen)
-		&& strncmp(folder->full_name, info->old, oldlen) == 0
-		&& folder->full_name[oldlen] == info->store->dir_sep)) {
-
-		d(printf("Found subfolder of '%s' == '%s'\n", info->old, folder->full_name));
-		camel_object_ref((CamelObject *)folder);
-		g_ptr_array_add(info->folders, folder);
-		CAMEL_FOLDER_LOCK(folder, lock);
-	}
-}
-
 /**
  * camel_store_rename_folder:
  * @store: a CamelStore
@@ -440,26 +363,39 @@ void
 camel_store_rename_folder (CamelStore *store, const char *old_name, const char *new_name, CamelException *ex)
 {
 	char *key;
-	CamelFolder *folder, *oldfolder;
-	struct _get_info info = { store, NULL, old_name, new_name };
-	int i;
+	CamelFolder *folder;
+	int i, oldlen, namelen;
+	GPtrArray *folders;
 
 	d(printf("store rename folder %s '%s' '%s'\n", ((CamelService *)store)->url->protocol, old_name, new_name));
 
 	if (strcmp(old_name, new_name) == 0)
 		return;
 
-	info.folders = g_ptr_array_new();
+	oldlen = strlen(old_name);
 
 	CAMEL_STORE_LOCK(store, folder_lock);
 
 	/* If the folder is open (or any subfolders of the open folder)
 	   We need to rename them atomically with renaming the actual folder path */
 	if (store->folders) {
-		CAMEL_STORE_LOCK(store, cache_lock);
-		/* Get all subfolders that are about to have their name changed */
-		g_hash_table_foreach(store->folders, (GHFunc)get_subfolders, &info);
-		CAMEL_STORE_UNLOCK(store, cache_lock);
+		folders = camel_object_bag_list(store->folders);
+		for (i=0;i<folders->len;i++) {
+			folder = folders->pdata[i];
+			namelen = strlen(folder->full_name);
+			if ((namelen == oldlen &&
+			     strcmp(folder->full_name, old_name) == 0)
+			    || ((namelen > oldlen)
+				&& strncmp(folder->full_name, old_name, oldlen) == 0
+				&& folder->full_name[oldlen] == store->dir_sep)) {
+				d(printf("Found subfolder of '%s' == '%s'\n", old_name, folder->full_name));
+				CAMEL_FOLDER_LOCK(folder, lock);
+			} else {
+				g_ptr_array_remove_index_fast(folders, i);
+				i--;
+				camel_object_unref(folder);
+			}
+		}
 	}
 
 	/* Now try the real rename (will emit renamed event) */
@@ -470,26 +406,19 @@ camel_store_rename_folder (CamelStore *store, const char *old_name, const char *
 		guint32 flags = CAMEL_STORE_FOLDER_INFO_RECURSIVE;
 		CamelRenameInfo reninfo;
 
-		CAMEL_STORE_LOCK(store, cache_lock);
-		for (i=0;i<info.folders->len;i++) {
+		for (i=0;i<folders->len;i++) {
 			char *new;
 
-			folder = info.folders->pdata[i];
+			folder = folders->pdata[i];
 
 			new = g_strdup_printf("%s%s", new_name, folder->full_name+strlen(old_name));
-
-			if (g_hash_table_lookup_extended(store->folders, folder->full_name, (void **)&key, (void **)&oldfolder)) {
-				g_hash_table_remove(store->folders, key);
-				g_free(key);
-				g_hash_table_insert(store->folders, new, oldfolder);
-			}
-
+			camel_object_bag_remove(store->folders, folder);
+			camel_object_bag_add(store->folders, new, folder);
 			camel_folder_rename(folder, new);
 
 			CAMEL_FOLDER_UNLOCK(folder, lock);
-			camel_object_unref((CamelObject *)folder);
+			camel_object_unref(folder);
 		}
-		CAMEL_STORE_UNLOCK(store, cache_lock);
 
 		/* Emit changed signal */
 		if (store->flags & CAMEL_STORE_SUBSCRIPTIONS)
@@ -503,16 +432,16 @@ camel_store_rename_folder (CamelStore *store, const char *old_name, const char *
 		}
 	} else {
 		/* Failed, just unlock our folders for re-use */
-		for (i=0;i<info.folders->len;i++) {
-			folder = info.folders->pdata[i];
+		for (i=0;i<folders->len;i++) {
+			folder = folders->pdata[i];
 			CAMEL_FOLDER_UNLOCK(folder, lock);
-			camel_object_unref((CamelObject *)folder);
+			camel_object_unref(folder);
 		}
 	}
 
 	CAMEL_STORE_UNLOCK(store, folder_lock);
 
-	g_ptr_array_free(info.folders, TRUE);
+	g_ptr_array_free(folders, TRUE);
 }
 
 
@@ -545,16 +474,6 @@ camel_store_get_inbox (CamelStore *store, CamelException *ex)
 	return folder;
 }
 
-
-static void
-trash_add_folder (gpointer key, gpointer value, gpointer data)
-{
-	CamelFolder *folder = CAMEL_FOLDER (value);
-	CamelStore *store = CAMEL_STORE (data);
-	
-	camel_vee_folder_add_folder (CAMEL_VEE_FOLDER (store->vtrash), folder);
-}
-
 static void
 trash_finalize (CamelObject *trash, gpointer event_data, gpointer user_data)
 {
@@ -572,15 +491,21 @@ init_trash (CamelStore *store)
 	store->vtrash = camel_vtrash_folder_new (store, CAMEL_VTRASH_NAME);
 	
 	if (store->vtrash) {
+		/* FIXME: this should probably use the object bag or another one ? ... */
 		/* attach to the finalise event of the vtrash */
 		camel_object_hook_event (CAMEL_OBJECT (store->vtrash), "finalize",
 					 trash_finalize, store);
 		
 		/* add all the pre-opened folders to the vtrash */
 		if (store->folders) {
-			CAMEL_STORE_LOCK(store, cache_lock);
-			g_hash_table_foreach (store->folders, trash_add_folder, store);
-			CAMEL_STORE_UNLOCK(store, cache_lock);
+			GPtrArray *folders = camel_object_bag_list(store->folders);
+			int i;
+
+			for (i=0;i<folders->len;i++) {
+				camel_vee_folder_add_folder (CAMEL_VEE_FOLDER (store->vtrash), (CamelFolder *)folders->pdata[i]);
+				camel_object_unref(folders->pdata[i]);
+			}
+			g_ptr_array_free(folders, TRUE);
 		}
 	}
 }
@@ -629,42 +554,24 @@ camel_store_get_trash (CamelStore *store, CamelException *ex)
 	return folder;
 }
 
-
-static void
-sync_folder (gpointer key, gpointer folder, gpointer ex)
-{
-	if (!camel_exception_is_set (ex))
-		camel_folder_sync (folder, FALSE, ex);
-	
-	camel_object_unref (CAMEL_OBJECT (folder));
-	g_free (key);
-}
-
-static void
-copy_folder_cache (gpointer key, gpointer folder, gpointer hash)
-{
-	g_hash_table_insert ((GHashTable *) hash, g_strdup (key), folder);
-	camel_object_ref (CAMEL_OBJECT (folder));
-}
-
 static void
 store_sync (CamelStore *store, CamelException *ex)
 {
 	if (store->folders) {
-		CamelException internal_ex;
-		GHashTable *hash;
-		
-		hash = g_hash_table_new (CS_CLASS (store)->hash_folder_name,
-					 CS_CLASS (store)->compare_folder_name);
-		
-		camel_exception_init (&internal_ex);
-		CAMEL_STORE_LOCK(store, cache_lock);
-		g_hash_table_foreach (store->folders, copy_folder_cache, hash);
-		CAMEL_STORE_UNLOCK(store, cache_lock);
-		camel_exception_xfer (ex, &internal_ex);
-		
-		g_hash_table_foreach (hash, sync_folder, &internal_ex);
-		g_hash_table_destroy (hash);
+		GPtrArray *folders;
+		CamelFolder *folder;
+		CamelException x;
+		int i;
+
+		folders = camel_object_bag_list(store->folders);
+		for (i=0;i<folders->len;i++) {
+			folder = folders->pdata[i];
+			if (!camel_exception_is_set(&x))
+				camel_folder_sync(folder, FALSE, &x);
+			camel_object_unref(folder);
+		}
+		camel_exception_xfer(ex, &x);
+		g_ptr_array_free(folders, TRUE);
 	}
 }
 
@@ -1102,12 +1009,7 @@ camel_store_unsubscribe_folder (CamelStore *store,
 
 	/* if we deleted a folder, force it out of the cache, and also out of the vtrash if setup */
 	if (store->folders) {
-		CAMEL_STORE_LOCK(store, cache_lock);
-		folder = g_hash_table_lookup(store->folders, folder_name);
-		if (folder)
-			camel_object_ref((CamelObject *)folder);
-		CAMEL_STORE_UNLOCK(store, cache_lock);
-
+		folder = camel_object_bag_get(store->folders, folder_name);
 		if (folder) {
 			if (store->vtrash)
 				camel_vee_folder_remove_folder((CamelVeeFolder *)store->vtrash, folder);
@@ -1117,17 +1019,11 @@ camel_store_unsubscribe_folder (CamelStore *store,
 
 	CS_CLASS (store)->unsubscribe_folder (store, folder_name, ex);
 
-	if (folder)
-		camel_object_unref((CamelObject *)folder);
+	if (store->folders)
+		camel_object_bag_remove(store->folders, folder);
 
-	if (store->folders) {
-		CAMEL_STORE_LOCK(store, cache_lock);
-		if (g_hash_table_lookup_extended(store->folders, folder_name, (void **)&key, (void **)&folder)) {
-			g_hash_table_remove (store->folders, key);
-			g_free (key);
-		}
-		CAMEL_STORE_UNLOCK(store, cache_lock);
-	}
+	if (folder)
+		camel_object_unref(folder);
 
 	CAMEL_STORE_UNLOCK(store, folder_lock);
 }
diff --git a/camel/camel-store.h b/camel/camel-store.h
index 29fb3cc95e..34627918e2 100644
--- a/camel/camel-store.h
+++ b/camel/camel-store.h
@@ -89,8 +89,7 @@ struct _CamelStore
 	
 	CamelFolder *vtrash;
 	
-	/* should have cache_lock when accessing this (priv->cache_lock) */
-	GHashTable *folders;
+	CamelObjectBag *folders;
 
 	int flags;
 
diff --git a/camel/camel-vee-store.c b/camel/camel-vee-store.c
index 4834ade7e1..9531d49fc0 100644
--- a/camel/camel-vee-store.c
+++ b/camel/camel-vee-store.c
@@ -74,7 +74,7 @@ camel_vee_store_class_init (CamelVeeStoreClass *klass)
 {
 	CamelStoreClass *store_class = (CamelStoreClass *) klass;
 	
-	camel_vee_store_parent = CAMEL_STORE_CLASS(camel_type_get_global_classfuncs (camel_store_get_type ()));
+	camel_vee_store_parent = (CamelStoreClass *)camel_store_get_type();
 
 	/* virtual method overload */
 	store_class->get_folder = vee_get_folder;
@@ -146,7 +146,7 @@ change_folder(CamelStore *store, const char *name, guint32 flags, int count)
 	fi->url = g_strdup_printf("vfolder:%s%s#%s", ((CamelService *)store)->url->path, (flags&CHANGE_NOSELECT)?";noselect=yes":"", name);
 	fi->unread_message_count = count;
 	camel_folder_info_build_path(fi, '/');
-	camel_object_trigger_event(CAMEL_OBJECT(store), (flags&CHANGE_DELETE)?"folder_deleted":"folder_created", fi);
+	camel_object_trigger_event(store, (flags&CHANGE_DELETE)?"folder_deleted":"folder_created", fi);
 	camel_folder_info_free(fi);
 }
 
@@ -154,6 +154,7 @@ static CamelFolder *
 vee_get_folder (CamelStore *store, const char *folder_name, guint32 flags, CamelException *ex)
 {
 	CamelVeeFolder *vf;
+	CamelFolder *folder;
 	char *name, *p;
 	int add;
 
@@ -166,12 +167,11 @@ vee_get_folder (CamelStore *store, const char *folder_name, guint32 flags, Camel
 		while ( (p = strchr(p, '/'))) {
 			*p = 0;
 
-			CAMEL_STORE_LOCK(store, cache_lock);
-			add = g_hash_table_lookup (store->folders, name) == NULL;
-			CAMEL_STORE_UNLOCK(store, cache_lock);
-
-			if (add)
+			folder = camel_object_bag_get(store->folders, name);
+			if (folder == NULL)
 				change_folder(store, name, CHANGE_ADD|CHANGE_NOSELECT, -1);
+			else
+				camel_object_unref(folder);
 			*p++='/';
 		}
 
@@ -194,78 +194,55 @@ vee_get_trash (CamelStore *store, CamelException *ex)
 	return NULL;
 }
 
-struct _build_info {
-	const char *top;
-	guint32 flags;
-	GPtrArray *infos;
-	GPtrArray *folders;
-};
-
-static void
-build_info(char *name, CamelVeeFolder *folder, struct _build_info *data)
-{
-	CamelFolderInfo *info;
-
-	/* check we have to include this one */
-	if (data->top) {
-		if (data->flags & CAMEL_STORE_FOLDER_INFO_RECURSIVE) {
-			int namelen = strlen(name);
-			int toplen = strlen(data->top);
-
-			if (!((namelen == toplen &&
-			       strcmp(name, data->top) == 0)
-			      || ((namelen > toplen)
-				  && strncmp(name, data->top, toplen) == 0
-				  && name[toplen] == '/')))
-				return;
-		} else {
-			if (strcmp(name, data->top))
-				return;
-		}
-	} else {
-		if ((data->flags & CAMEL_STORE_FOLDER_INFO_RECURSIVE) == 0) {
-			if (strchr(name, '/'))
-				return;
-		}
-	}
-
-	info = g_malloc0(sizeof(*info));
-	info->url = g_strdup_printf("vfolder:%s#%s", ((CamelService *)((CamelFolder *)folder)->parent_store)->url->path,
-				    ((CamelFolder *)folder)->full_name);
-	info->full_name = g_strdup(((CamelFolder *)folder)->full_name);
-	info->name = g_strdup(((CamelFolder *)folder)->name);
-	info->unread_message_count = -1;
-	g_ptr_array_add(data->infos, info);
-	camel_object_ref((CamelObject *)folder);
-	g_ptr_array_add(data->folders, folder);
-}
-
 static CamelFolderInfo *
 vee_get_folder_info(CamelStore *store, const char *top, guint32 flags, CamelException *ex)
 {
-	struct _build_info data;
 	CamelFolderInfo *info;
+	GPtrArray *folders, *infos;
 	int i;
 
-	/* first, build the info list */
-	data.top = top;
-	data.flags = flags;
-	data.infos = g_ptr_array_new();
-	data.folders = g_ptr_array_new();
-	CAMEL_STORE_LOCK(store, cache_lock);
-	g_hash_table_foreach(store->folders, (GHFunc)build_info, &data);
-	CAMEL_STORE_UNLOCK(store, cache_lock);
-
-	/* then make sure the unread counts are accurate */
-	for (i=0;i<data.infos->len;i++) {
-		CamelFolderInfo *info = data.infos->pdata[i];
-		CamelFolder *folder = data.folders->pdata[i];
-
-		camel_folder_refresh_info(folder, NULL);
-		info->unread_message_count = camel_folder_get_unread_message_count(folder);
-		camel_object_unref((CamelObject *)folder);
+	infos = g_ptr_array_new();
+	folders = camel_object_bag_list(store->folders);
+	for (i=0;i<folders->len;i++) {
+		CamelVeeFolder *folder = folders->pdata[i];
+		int add = FALSE;
+		char *name = ((CamelFolder *)folder)->full_name;
+
+		/* check we have to include this one */
+		if (top) {
+			if (flags & CAMEL_STORE_FOLDER_INFO_RECURSIVE) {
+				int namelen = strlen(name);
+				int toplen = strlen(top);
+
+				add = ((namelen == toplen &&
+					strcmp(name, top) == 0)
+				       || ((namelen > toplen)
+					   && strncmp(name, top, toplen) == 0
+					   && name[toplen] == '/'));
+			} else {
+				add = strcmp(name, top) == 0;
+			}
+		} else {
+			if ((flags & CAMEL_STORE_FOLDER_INFO_RECURSIVE) == 0)
+				add = strchr(name, '/') == NULL;
+		}
+
+		if (add) {
+			/* ensures unread is correct */
+			if ((flags & CAMEL_STORE_FOLDER_INFO_FAST) == 0)
+				camel_folder_refresh_info((CamelFolder *)folder, NULL);
+
+			info = g_malloc0(sizeof(*info));
+			info->url = g_strdup_printf("vfolder:%s#%s", ((CamelService *)((CamelFolder *)folder)->parent_store)->url->path,
+						    ((CamelFolder *)folder)->full_name);
+			info->full_name = g_strdup(((CamelFolder *)folder)->full_name);
+			info->name = g_strdup(((CamelFolder *)folder)->name);
+			info->unread_message_count = camel_folder_get_unread_message_count((CamelFolder *)folder);
+			g_ptr_array_add(infos, info);
+		}
+		camel_object_unref(folder);
 	}
-	g_ptr_array_free(data.folders, TRUE);
+	g_ptr_array_free(folders, TRUE);
 
 	/* and always add UNMATCHED, if scanning from top/etc */
 	if (top == NULL || top[0] == 0 || strncmp(top, CAMEL_UNMATCHED_NAME, strlen(CAMEL_UNMATCHED_NAME)) == 0) {
@@ -275,12 +252,12 @@ vee_get_folder_info(CamelStore *store, const char *top, guint32 flags, CamelExce
 		info->name = g_strdup(CAMEL_UNMATCHED_NAME);
 		info->unread_message_count = -1;
 		camel_folder_info_build_path(info, '/');
-		g_ptr_array_add(data.infos, info);
+		g_ptr_array_add(infos, info);
 	}
 		
 	/* convert it into a tree */
-	info = camel_folder_info_build(data.infos, (top&&top[0])?top:"", '/', TRUE);
-	g_ptr_array_free(data.infos, TRUE);
+	info = camel_folder_info_build(infos, (top&&top[0])?top:"", '/', TRUE);
+	g_ptr_array_free(infos, TRUE);
 
 	return info;
 }
@@ -289,7 +266,6 @@ static void
 vee_delete_folder(CamelStore *store, const char *folder_name, CamelException *ex)
 {
 	CamelFolder *folder;
-	char *key;
 
 	if (strcmp(folder_name, CAMEL_UNMATCHED_NAME) == 0) {
 		camel_exception_setv(ex, CAMEL_EXCEPTION_STORE_NO_FOLDER,
@@ -297,24 +273,20 @@ vee_delete_folder(CamelStore *store, const char *folder_name, CamelException *ex
 		return;
 	}
 
-	CAMEL_STORE_LOCK(store, cache_lock);
-	if (g_hash_table_lookup_extended(store->folders, folder_name, (void **)&key, (void **)&folder)) {
-		int update;
+	folder = camel_object_bag_get(store->folders, folder_name);
+	if (folder) {
+		camel_object_bag_remove(store->folders, folder);
 
-		update = (((CamelVeeFolder *)folder)->flags & CAMEL_STORE_FOLDER_PRIVATE) == 0;
-		g_hash_table_remove(store->folders, key);
-		CAMEL_STORE_UNLOCK(store, cache_lock);
 		if (store->vtrash)
 			camel_vee_folder_remove_folder((CamelVeeFolder *)store->vtrash, folder);
 
-		if (update) {
+		if ((((CamelVeeFolder *)folder)->flags & CAMEL_STORE_FOLDER_PRIVATE) == 0) {
 			/* what about now-empty parents?  ignore? */
-			change_folder(store, key, CHANGE_DELETE, -1);
+			change_folder(store, folder_name, CHANGE_DELETE, -1);
 		}
-		g_free(key);
-	} else {
-		CAMEL_STORE_UNLOCK(store, cache_lock);
 
+		camel_object_unref(folder);
+	} else {
 		camel_exception_setv(ex, CAMEL_EXCEPTION_STORE_NO_FOLDER,
 				     _("Cannot delete folder: %s: No such folder"), folder_name);
 	}
@@ -334,11 +306,11 @@ vee_rename_folder(CamelStore *store, const char *old, const char *new, CamelExce
 	}
 
 	/* See if it exists, for vfolders, all folders are in the folders hash */
-	CAMEL_STORE_LOCK(store, cache_lock);
-	if ((folder = g_hash_table_lookup(store->folders, old)) == NULL) {
+	folder = camel_object_bag_get(store->folders, old);
+	if (folder == NULL) {
 		camel_exception_setv(ex, CAMEL_EXCEPTION_STORE_NO_FOLDER,
 				     _("Cannot rename folder: %s: No such folder"), old);
+	} else {
+		camel_object_unref(folder);
 	}
-
-	CAMEL_STORE_UNLOCK(store, cache_lock);
 }
diff --git a/camel/providers/imap/camel-imap-store.c b/camel/providers/imap/camel-imap-store.c
index 0fba62fe36..01a9dcc79f 100644
--- a/camel/providers/imap/camel-imap-store.c
+++ b/camel/providers/imap/camel-imap-store.c
@@ -906,8 +906,8 @@ imap_forget_folder (CamelImapStore *imap_store, const char *folder_name, CamelEx
 	if (cache)
 		camel_imap_message_cache_clear (cache);
 	
-	camel_object_unref (CAMEL_OBJECT (cache));
-	camel_object_unref (CAMEL_OBJECT (summary));
+	camel_object_unref (cache);
+	camel_object_unref (summary);
 	
 	unlink (summary_file);
 	g_free (summary_file);
@@ -951,13 +951,6 @@ imap_check_folder_still_extant (CamelImapStore *imap_store, const char *full_nam
 	return TRUE;
 }
 
-static void
-copy_folder(char *key, CamelFolder *folder, GPtrArray *out)
-{
-	g_ptr_array_add(out, folder);
-	camel_object_ref((CamelObject *)folder);
-}
-
 /* This is a little 'hack' to avoid the deadlock conditions that would otherwise
    ensue when calling camel_folder_refresh_info from inside a lock */
 /* NB: on second thougts this is probably not entirely safe, but it'll do for now */
@@ -972,10 +965,7 @@ imap_store_refresh_folders (CamelImapStore *store, CamelException *ex)
 	GPtrArray *folders;
 	int i;
 	
-	folders = g_ptr_array_new();
-	CAMEL_STORE_LOCK(store, cache_lock);
-	g_hash_table_foreach (CAMEL_STORE (store)->folders, (GHFunc)copy_folder, folders);
-	CAMEL_STORE_UNLOCK(store, cache_lock);
+	folders = camel_object_bag_list(CAMEL_STORE (store)->folders);
 	
 	for (i = 0; i <folders->len; i++) {
 		CamelFolder *folder = folders->pdata[i];
@@ -992,12 +982,12 @@ imap_store_refresh_folders (CamelImapStore *store, CamelException *ex)
 			 * after being offline */
 			
 			namedup = g_strdup (folder->full_name);
-			camel_object_unref((CamelObject *)folder);
+			camel_object_unref(folder);
 			imap_folder_effectively_unsubscribed (store, namedup, ex);
 			imap_forget_folder (store, namedup, ex);
 			g_free (namedup);
 		} else
-			camel_object_unref((CamelObject *)folder);
+			camel_object_unref(folder);
 	}
 	
 	g_ptr_array_free (folders, TRUE);
@@ -2198,30 +2188,25 @@ get_folder_counts(CamelImapStore *imap_store, CamelFolderInfo *fi, CamelExceptio
 				} else {
 					fi->unread_message_count = get_folder_status (imap_store, fi->full_name, "UNSEEN");
 					/* if we have this folder open, and the unread count has changed, update */
-					CAMEL_STORE_LOCK(imap_store, cache_lock);
-					folder = g_hash_table_lookup(CAMEL_STORE(imap_store)->folders, fi->full_name);
-					if (folder && fi->unread_message_count != camel_folder_get_unread_message_count(folder))
-						camel_object_ref(folder);
-					else
-						folder = NULL;
-					CAMEL_STORE_UNLOCK(imap_store, cache_lock);
-					if (folder) {
+					folder = camel_object_bag_get(CAMEL_STORE(imap_store)->folders, fi->full_name);
+					if (folder && fi->unread_message_count != camel_folder_get_unread_message_count(folder)) {
 						CAMEL_FOLDER_CLASS (CAMEL_OBJECT_GET_CLASS(folder))->refresh_info(folder, ex);
 						fi->unread_message_count = camel_folder_get_unread_message_count(folder);
-						camel_object_unref(folder);
 					}
+					if (folder)
+						camel_object_unref(folder);
+
 				}
 		
 				CAMEL_SERVICE_UNLOCK (imap_store, connect_lock);
 			} else {
 				/* since its cheap, get it if they're open */
-				CAMEL_STORE_LOCK(imap_store, cache_lock);
-				folder = g_hash_table_lookup(CAMEL_STORE(imap_store)->folders, fi->full_name);
-				if (folder)
+				folder = camel_object_bag_get(CAMEL_STORE(imap_store)->folders, fi->full_name);
+				if (folder) {
 					fi->unread_message_count = camel_folder_get_unread_message_count(folder);
-				else
+					camel_object_unref(folder);
+				} else
 					fi->unread_message_count = -1;
-				CAMEL_STORE_UNLOCK(imap_store, cache_lock);
 			}
 
 			if (fi->child)
diff --git a/camel/providers/local/camel-local-store.c b/camel/providers/local/camel-local-store.c
index b325df769a..a794b778f3 100644
--- a/camel/providers/local/camel-local-store.c
+++ b/camel/providers/local/camel-local-store.c
@@ -319,7 +319,7 @@ static void
 rename_folder(CamelStore *store, const char *old, const char *new, CamelException *ex)
 {
 	char *path = CAMEL_LOCAL_STORE (store)->toplevel_dir;
-	CamelLocalFolder *folder;
+	CamelLocalFolder *folder = NULL;
 	char *newibex = g_strdup_printf("%s%s.ibex", path, new);
 	char *oldibex = g_strdup_printf("%s%s.ibex", path, old);
 
@@ -327,8 +327,7 @@ rename_folder(CamelStore *store, const char *old, const char *new, CamelExceptio
 
 	d(printf("local rename folder '%s' '%s'\n", old, new));
 
-	CAMEL_STORE_LOCK(store, cache_lock);
-	folder = g_hash_table_lookup(store->folders, old);
+	folder = camel_object_bag_get(store->folders, old);
 	if (folder && folder->index) {
 		if (camel_index_rename(folder->index, newibex) == -1)
 			goto ibex_failed;
@@ -344,11 +343,12 @@ rename_folder(CamelStore *store, const char *old, const char *new, CamelExceptio
 	if (xrename(old, new, path, "", FALSE, ex))
 		goto base_failed;
 
-	CAMEL_STORE_UNLOCK(store, cache_lock);
-
 	g_free(newibex);
 	g_free(oldibex);
 
+	if (folder)
+		camel_object_unref(folder);
+
 	return;
 
 base_failed:
@@ -365,9 +365,11 @@ ibex_failed:
 			      _("Could not rename '%s': %s"),
 			      old, g_strerror (errno));
 
-	CAMEL_STORE_UNLOCK(store, cache_lock);
 	g_free(newibex);
 	g_free(oldibex);
+
+	if (folder)
+		camel_object_unref(folder);
 }
 
 /* default implementation, only delete metadata */
diff --git a/camel/providers/local/camel-maildir-store.c b/camel/providers/local/camel-maildir-store.c
index 8e6cc1ee9d..c84c37158c 100644
--- a/camel/providers/local/camel-maildir-store.c
+++ b/camel/providers/local/camel-maildir-store.c
@@ -273,16 +273,15 @@ static int scan_dir(CamelStore *store, GHashTable *visited, char *root, const ch
 		base = path;
 
 	/* if we have this folder open, get the real unread count */
-	unread = -1;
-
-	CAMEL_STORE_LOCK(store, cache_lock);
-	folder = g_hash_table_lookup(store->folders, path);
+	folder = camel_object_bag_get(store->folders, path);
 	if (folder) {
 		if ((flags & CAMEL_STORE_FOLDER_INFO_FAST) == 0)
 			camel_folder_refresh_info(folder, NULL);
 		unread = camel_folder_get_unread_message_count(folder);
+		camel_object_unref(folder);
+	} else {
+		unread = -1;
 	}
-	CAMEL_STORE_UNLOCK(store, cache_lock);
 
 	/* if we dont have a folder, then scan the directory and get the unread
 	   count from there, which is reasonably cheap (on decent filesystem) */
diff --git a/camel/providers/local/camel-mh-store.c b/camel/providers/local/camel-mh-store.c
index 6c442d335b..7fe68ef4b5 100644
--- a/camel/providers/local/camel-mh-store.c
+++ b/camel/providers/local/camel-mh-store.c
@@ -300,14 +300,13 @@ static CamelFolderInfo *folder_info_new(CamelStore *store, const char *root, con
 	fi->unread_message_count = 0;
 
 	/* check unread count if open */
-	CAMEL_STORE_LOCK(store, cache_lock);
-	folder = g_hash_table_lookup(store->folders, path);
+	folder = camel_object_bag_get(store->folders, path);
 	if (folder) {
 		if ((((CamelMhStore *)store)->flags & CAMEL_STORE_FOLDER_INFO_FAST) == 0)
 			camel_folder_refresh_info(folder, NULL);
 		fi->unread_message_count = camel_folder_get_unread_message_count(folder);
+		camel_object_unref(folder);
 	}
-	CAMEL_STORE_UNLOCK(store, cache_lock);
 
 	/* We could: if we have no folder, and FAST isn't specified, perform a full
 	   scan of all messages for their status flags.  But its probably not worth
diff --git a/camel/providers/local/camel-spool-store.c b/camel/providers/local/camel-spool-store.c
index f2cbdd1e55..9419d897a8 100644
--- a/camel/providers/local/camel-spool-store.c
+++ b/camel/providers/local/camel-spool-store.c
@@ -285,13 +285,13 @@ static int scan_dir(CamelStore *store, GHashTable *visited, char *root, const ch
 	} else if (S_ISREG(st.st_mode)) {
 		/* incase we start scanning from a file.  messy duplication :-/ */
 		if (path) {
-			CAMEL_STORE_LOCK(store, cache_lock);
-			folder = g_hash_table_lookup(store->folders, path);
-			if (folder)
+			folder = camel_object_bag_get(store->folders, path);
+			if (folder) {
+				/* should this refresh if ! FAST? */
 				unread = camel_folder_get_unread_message_count(folder);
-			else
+				camel_object_unref(folder);
+			} else
 				unread = -1;
-			CAMEL_STORE_UNLOCK(store, cache_lock);
 			tmp = strrchr(path, '/');
 			if (tmp)
 				tmp++;
@@ -346,13 +346,13 @@ static int scan_dir(CamelStore *store, GHashTable *visited, char *root, const ch
 
 			if (S_ISREG(st.st_mode)) {
 				/* first, see if we already have it open */
-				CAMEL_STORE_LOCK(store, cache_lock);
-				folder = g_hash_table_lookup(store->folders, fname);
-				if (folder)
+				folder = camel_object_bag_get(store->folders, path);
+				if (folder) {
+					/* should this refresh if ! FAST? */
 					unread = camel_folder_get_unread_message_count(folder);
-				else
+					camel_object_unref(folder);
+				} else
 					unread = -1;
-				CAMEL_STORE_UNLOCK(store, cache_lock);
 
 				/* no?  check its content to see if its a folder or not */
 				if (folder == NULL) {
@@ -457,13 +457,12 @@ get_folder_info_mbox(CamelStore *store, const char *top, guint32 flags, CamelExc
 		fi->name = g_strdup("INBOX");
 		fi->url = g_strdup_printf("%s:%s#%s", service->url->protocol, service->url->path, fi->name);
 
-		CAMEL_STORE_LOCK(store, cache_lock);
-		folder = g_hash_table_lookup(store->folders, fi->full_name);
-		if (folder)
+		folder = camel_object_bag_get(store->folders, fi->full_name);
+		if (folder) {
 			fi->unread_message_count = camel_folder_get_unread_message_count(folder);
-		else
+			camel_object_unref(folder);
+		} else
 			fi->unread_message_count = -1;
-		CAMEL_STORE_UNLOCK(store, cache_lock);
 
 		camel_folder_info_build_path(fi, '/');
 	}