Fixes for the summary messageid changes. Hash the messageid and store it.

2000-11-20  Not Zed  <NotZed@HelixCode.com>

	* providers/nntp/camel-nntp-utils.c (get_XOVER_headers): Fixes for
	the summary messageid changes.  Hash the messageid and store it.
	(get_XOVER_headers): Use camel_folder_summary_info_new() to create
	the summary item before adding it.

	* camel-folder-summary.h (CamelMessageInfo): Changed the
	messgae-id to be an 8 byte md5 hash, and the references list to be
	an array of these.

	* providers/local/camel-mh-summary.c (mh_summary_sync_message):
	New function, sync out the message info stuff.  Only updates the
	X-Ev header if it can get away with it, otherwise writes out a
	whole new message.
	(mh_summary_sync): Added more functionality.  All summary info is
	now written to the X-Ev header, etc, and new messages re-written
	if required during the sync process.

	* providers/local/camel-local-folder.c
	(local_set_message_user_flag): Set the XEVCHANGE flag.
	(local_set_message_user_tag): And here too.

	* providers/local/camel-local-summary.h: New flag
	CAMEL_MESSAGE_FOLDER_XEVCHANGE to indicate the XEV header has
	probably changed size and needs to be rewritten in whole.

	* camel-folder-summary.c (next_uid_string): Want this static, not
	const.
	(message_info_new): Store the references and message-id values as
	64 bit, binary hashes.
	(message_info_load): fix for message-id/references changes.
	(message_info_save): Likewise.
	(camel_message_info_dup_to): And here.
	(camel_message_info_free): And here too.  No longer free
	message_id, and simple free for references array.
	(CAMEL_FOLDER_SUMMARY_VERSION): Bumped file revision.
	(camel_folder_summary_init): Init memchunk allocators to empty.
	(camel_folder_summary_finalize): Free memchunk allocators if
	there.
	(message_info_new): Use the chunk allocator to allocate message
	info's.
	(camel_folder_summary_info_new): New helper to allocate the
	message info, and setup the memchunk if required.
	(content_info_alloc): Likewise for content info's.
	(message_info_load): Use summary_info_new_empty.
	(content_info_new): Use content_info_alloc.
	(content_info_load): "
	(content_info_free): Free the content info as a memchunk.
	(message_info_free): Free everything directly and the base as a
	memchunk, rather than calling camel_message_info_free(), which
	assumes a malloc'd array.

	* providers/local/camel-local-summary.c: Include ctype.h, kill a
	warning.
	(local_summary_decode_x_evolution): If we get a NULL message info,
	then dont try and set anything, just check for validity.
	(camel_local_summary_write_headers): New function to write a set
	of headers to an fd.
	(camel_local_summary_check): Added some statistic generation
	stuff for memory profiling.

	* providers/local/camel-mbox-summary.c (header_write): Changed to
	use stdoi functions to write out the header to a buffered stream,
	instead of using writev, which is apparently slow (and writing
	each line separately is slow anyway).
	(mbox_summary_sync_full): New implementation.  Does things
	differently, doesn't use or require the content info stuff.
	(summary_rebuild): Dont return an error if we start scanning at
	the end of file.
	(mbox_summary_sync_full): If we are not writing out new headers,
	make sure we copy the From line as we go, and update frompos
	appropriately.
	(mbox_summary_sync_full): Always copy the From line from the
	existing one, rather than trying to make one up ourselves.
	(mbox_summary_sync): If we can get by with a quick-sync, then try
	it, if that fails, then try a full sync anyway.
	(mbox_summary_sync_quick): Quick sync.  Only update system flags,
	etc.
	(mbox_summary_sync_full): Use the proper local summary encode_xev
	function.
	(header_evolution_decode): Removed, no longer needed.
	(header_evolution_encode): Same.
	(copy_block): No longer needed, removed.
	(header_write): Removed, replaced with
	camel_local_summary_write_headers.
	(mbox_summary_sync_full): Fixed for header_write change.

	* camel-mime-parser.c (folder_scan_step): Implement the new
	optional parser state HSCAN_PRE_FROM, that returns the (currently
	unfiltered) input data.
	(folder_scan_drop_step): Do the right thing for the PRE_FROM
	state.
	(camel_mime_parser_scan_from): Update the doco.
	(camel_mime_parser_scan_pre_from): Ok, make this behaviour
	optional, it simplifies a lot of loops that dont otherwise need to
	know about it.
	(folder_scan_step): Made the PRE_FROM state optional.
	(struct _header_scan_state): Made the bool vars 1 bit.
	(folder_pull_part): Free the from_line buffer if it is there.
	(folder_scan_skip_line): Added a new arg, can save the skpped data
	to a byte_array, as we go.
	(folder_scan_step): Fixed calls to skip_line approrpiately.  Now
	we save the from line as we parse it.
	(camel_mime_parser_read): New function to read from the mime
	parser buffer directly.  Useful if you use the parser to read the
	first/some headers, then need to scan the rest of the data,
	without needing to use a seek(), or allocate your own buffers.

	* camel-mime-parser.h (struct _header_state): Added a new parser state,
	pre-from which returns any data found before a from line during
	parsing (all other data can be retrieved by the caller except
	this).

svn path=/trunk/; revision=6618

1 files changed, 20 insertions, 8 deletions

diff --git a/camel/providers/local/camel-mbox-folder.c b/camel/providers/local/camel-mbox-folder.c
index cfffbf7c16..d5e76f7cab 100644
--- a/camel/providers/local/camel-mbox-folder.c
+++ b/camel/providers/local/camel-mbox-folder.c
@@ -42,7 +42,7 @@
 #include "camel-mime-filter-from.h"
 #include "camel-exception.h"
 
-#define d(x) (printf("%s(%d): ", __FILE__, __LINE__),(x))
+#define d(x) /*(printf("%s(%d): ", __FILE__, __LINE__),(x))*/
 
 static CamelLocalFolderClass *parent_class = NULL;
 
@@ -118,7 +118,7 @@ camel_mbox_folder_new(CamelStore *parent_store, const char *full_name, guint32 f
 {
 	CamelFolder *folder;
 
-	d(printf("Creating mbox folder: %s\n", full_name));
+	d(printf("Creating mbox folder: %s in %s\n", full_name, camel_local_store_get_toplevel_dir((CamelLocalStore *)parent_store)));
 
 	folder = (CamelFolder *)camel_object_new(CAMEL_MBOX_FOLDER_TYPE);
 	folder = (CamelFolder *)camel_local_folder_construct((CamelLocalFolder *)folder,
@@ -169,7 +169,9 @@ mbox_append_message(CamelFolder *folder, CamelMimeMessage * message, const Camel
 	char *fromline = NULL;
 	int fd;
 	struct stat st;
-
+#if 0
+	char *xev;
+#endif
 	/* If we can't lock, dont do anything */
 	if (camel_local_folder_lock(lf, CAMEL_LOCK_WRITE, ex) == -1)
 		return;
@@ -186,9 +188,6 @@ mbox_append_message(CamelFolder *folder, CamelMimeMessage * message, const Camel
 	if (camel_exception_is_set(ex))
 		goto fail;
 
-	/* and we need to set the frompos explicitly */
-	((CamelMboxMessageInfo *)mi)->frompos = mbs->folder_size?mbs->folder_size+1:0;
-
 	d(printf("Appending message: uid is %s\n", mi->uid));
 
 	output_stream = camel_stream_fs_new_with_name(lf->folder_path, O_WRONLY|O_APPEND, 0600);
@@ -197,6 +196,18 @@ mbox_append_message(CamelFolder *folder, CamelMimeMessage * message, const Camel
 		goto fail;
 	}
 
+	/* and we need to set the frompos/XEV explicitly */
+	((CamelMboxMessageInfo *)mi)->frompos = mbs->folder_size?mbs->folder_size+1:0;
+#if 0
+	xev = camel_local_summary_encode_x_evolution(lf->summary, mi);
+	if (xev) {
+		/* the x-ev header should match the 'current' flags, no problem, so store as much */
+		camel_medium_set_header((CamelMedium *)message, "X-Evolution", xev);
+		mi->flags &= ~ CAMEL_MESSAGE_FOLDER_NOXEV|CAMEL_MESSAGE_FOLDER_FLAGGED;
+		g_free(xev);
+	}
+#endif
+
 	/* we must write this to the non-filtered stream ... prepend a \n if not at the start of the file */
 	fromline = camel_mbox_summary_build_from(((CamelMimePart *)message)->headers);
 	if (camel_stream_printf(output_stream, mbs->folder_size==0?"%s":"\n%s", fromline) == -1)
@@ -330,8 +341,9 @@ retry:
 	    || camel_mime_parser_tell_start_from(parser) != info->frompos) {
 
 		g_warning("Summary doesn't match the folder contents!  eek!\n"
-			  "  expecting offset %ld got %ld", (long int)info->frompos,
-			  (long int)camel_mime_parser_tell_start_from(parser));
+			  "  expecting offset %ld got %ld, state = %d", (long int)info->frompos,
+			  (long int)camel_mime_parser_tell_start_from(parser),
+			  camel_mime_parser_state(parser));
 
 		camel_object_unref((CamelObject *)parser);