Implement a complete() function, now we need one. (filter): Upgraded to

2000-11-07  Not Zed  <NotZed@HelixCode.com>

	* camel-mime-filter-bestenc.c (complete): Implement a complete()
	function, now we need one.
	(filter): Upgraded to match rfrc2045 properly.  Checks also for
	length of line and valid CRLF sequences.
	(camel_mime_filter_bestenc_get_best_encoding): Do the work of
	working out what is the best encoding given what we found about
	the stream.

	* camel-mime-part.c (camel_mime_part_encoding_to_string): Use a
	lookup table to get the encoding naem, and add the binary type.
	(camel_mime_part_encoding_from_string): Likewise for the reverse.

	* camel-mime-part.h: Added the binary encoding type, see rfc2045.

	* camel-mime-utils.c (header_param_list_format_append): Dont put a
	space before ;'s in parameter lists, makes them more
	readable/consistent.

	* camel-mime-message.c (multipart_has_8bit_parts): Cleaned up the
	old stuff, well removed it.
	(camel_mime_message_set_best_encoding): Added another argument
	that lets you select what you want to set the best of.  i.e. for
	smtp transport we only need 7 bit, and dont need to optimise the
	charset (although of course, we should always).
	(find_best_encoding): Implement this feature, if we are not
	getting the best charset, use the one we have.
	(best_encoding): Set the charset on the part appropriately.  Sigh,
	the interfaces for this are nonexistant.
	(find_best_encoding): Tell the bestenc filter that lf should be
	treated as crlf for the purposes of determining encodings.

2000-11-06  Not Zed  <NotZed@HelixCode.com>

	* camel-charset-map.c (camel_charset_init): Init function for an
	iterative charset determinator.
	(camel_charset_step): Iterate another buffer.
	(camel_charset_mask): Removed, since it couldn't have worked.
	(camel_charset_best): Use the iterative interface to do the work.
	(camel_charset_best_name): Get the best name for a charset so far.

	* camel-mime-filter-bestenc.c: New class, a stream
	filter that can be used to memory-efficiently determine the best
	encoding and/or charset to use for a given stream of bytes.

	* Makefile.am (libcamelinclude_HEADERS): Added stream-null*.
	(libcamel_la_SOURCES): Added bestenc*

	* camel-stream-null.c: New class, a null-stream, that always
	succeeds, and never has any contents.

	* camel-stream.c: Minor pointless changes.  Was going to do
	something else but changed my mind.  Added trivial default
	implementations for all callbacks.

	* camel-mime-message.h: Cleaned up some old cruft.

	* camel-folder-summary.c (camel_folder_summary_format_address):
	address_list_format() no longer encodes, so we dont need to decode
	it.

	* camel-address.c (camel_address_unformat): New function, attempts
	to reverse the formatting process on display addresses.
	(camel_address_length): New function to get the number of
	addresses, without having to peek the structure.

	* camel-mime-message.c (camel_mime_message_set_from): Fix a typo.
	(camel_mime_message_finalize): Only unref from/reply_to if we have
	it.
	(camel_mime_message_set_recipients): New function - set the
	recipients as a CamelInternetAddress.  This function effectively
	deprecates the older recipient setting functions.
	(camel_mime_message_add_recipient): What the hell, i'll bite the
	bullet.  Terminate this function.  The old api was ambiguious and
	inefficient and didn't work right anyway.
	(camel_mime_message_remove_recipient_address): And this one.
	(camel_mime_message_remove_recipient_name): And this one too.
	(camel_mime_message_set_recipients): If we set an empty header,
	then remove it from the header list.  Allow a null receipient
	object to clear a header.
	(camel_mime_message_set_from): Likewise, if setting an empty from
	address.
	(camel_mime_message_encode_8bit_parts): Eeek!!
	camel_stream_mem_new_with_byte_array owns the byte_array we give
	it, so make sure we dont free any of it!
	(camel_mime_message_encode_8bit_parts):  Infact, i'll just rewrite
	the whole lot, its a bit of a mess.  Should really rename it and
	make it a little more useful too, lets see ...
	(best_encoding): This has a string interface?  Oh boy.
	(camel_mime_message_foreach_part): New experimental function to
	iterate over all message parts.  Might not remain.
	(camel_mime_message_has_8bit_parts): New implementation using
	foreach_part.  Fixed a couple of problems.
	(find_best_encoding): New function, that finds the best encoding
	for a given part (will probably be moved to camel-mime-part), and
	also the best charset to use if it is a text part.  Since one
	affects the other it is a two pass process, but uses streams and
	not memory to achieve this.
	(camel_mime_message_set_best_encoding): Uses the function above to
	configure an entire message for the best encoding possible given
	transport constraints.
	(camel_mime_message_encode_8bit_parts): Reimplemented to use the
	function above to perform the work.

	* camel-internet-address.c
	(camel_internet_address_format_address): Dont put <> around a lone
	address with no real name.
	(camel_internet_address_encode_address): Similarly.
	(internet_decode): Actually return the count of decoded addresses.
	(internet_unformat): Implement the unformatting routine.

2000-11-05  Not Zed  <NotZed@HelixCode.com>

	* providers/smtp/camel-smtp-transport.c (_send_to): Changed to get
	the internetaddress directly, rather than having to parse it
	itself.

	* camel-address.c (camel_address_format): Added a new function
	which will format address, suitable for display.
	(camel_address_cat): Concatentate 1 camel address onto another.
	It is upto the caller to ensure the addresses are of compatible
	types.
	(camel_address_new_clone): New function to create a new address by
	copying an existing one of the same type.
	(camel_address_copy): New helper function to copy an address.

	* camel-mime-message.h (struct _CamelMimeMessage): Removed cached
	copy of date string.
	(struct _CamelMimeMessage): Added date_received info.

	* camel-mime-message.c (camel_mime_message_get_date_string):
	Removed.  Nothing uses it anyway, and it is redundant.
	(camel_mime_message_finalize): No more date_str.
	(camel_mime_message_init): No more date_str, initialise
	date_received*
	(write_to_stream): Change the check for a date header.
	(process_header): No longer track the date_str.
	(camel_mime_message_get_received_date): Removed.  totally invalid
	anyway.
	(camel_mime_message_get_sent_date): Removed.  Redundant.  The only
	'date' is the sent date, the received date is just made up.
	(camel_mime_message_get_date): Args changed to be more consistent
	with utility functions.
	(camel_mime_message_get_date): Dont set the date when we're asked
	for it (if its not set by the time its written, it'll be set
	then).
	(camel_mime_message_get_date_received): Actually do 'the right
	thing' here, if we have a received header, use that to determine
	the received date.  And return the data in the same format as
	get_date.
	(camel_mime_message_set_from): Changed the api to better match
	what we should be doing.  Pass a camelinternetaddress, etc.
	(camel_mime_message_set_reply_to): Cahnged similarly to take an
	internetaddress.
	(camel_mime_message_get_reply_to): Likewise.
	(camel_mime_message_finalize): Unref the from/reply_to objects.
	(format_address): Removed, no longer needed.
	(process_header): Changed to store the from/reply_to as
	internetaddress's.
	(write_to_stream): Set the from header directly to empty, if we
	dont have one.  Maybe we should just abort, and/or create one
	based on the current user.

	* camel-mime-utils.c (header_address_list_format): Renamed to
	header_address_list_encode, which is what it is actually doing.
	(header_address_list_format_append): Similarly.
	(encoding_map[]): Removed, no longer used.
	(header_address_list_encode_append): Take another arg, do we
	encode the address (for internet), or not (for display - utf8
	only).
	(header_address_list_format): Re-added this function, but now it
	generates a display version only.  Surprise surprise, that is all
	anythign needs to generate anyway.  Sigh.

	* camel-internet-address.c (camel_internet_address_get): Return
	false if we get an invalid index only.
	(camel_internet_address_encode_address): Helper function to encode
	a single address for mailing.
	(internet_encode): Use the above function to format it.
	(camel_internet_address_format_address): Format a single address
	for display.
	(internet_format): Implement the display version.
	(camel_internet_address_class_init): Init the internet_format
	virtual function.
	(internet_cat): Implement virtual function to concatenate
	addresses.

	* camel-folder-summary.c
	(camel_folder_summary_info_new_from_header): new function, only
	build the summary info, dont add it.
	(camel_folder_summary_info_new_from_parser): Likewise, for new
	info from parser.
	(camel_folder_summary_add_from_parser): Cahnged to call function
	above to build info.
	(camel_folder_summary_add_from_header): Changed to call function
	above, to build info.
	(camel_folder_summary_info_free): New function to free the summary
	message info.
	(camel_folder_summary_clear): Changed to clal above to free info.
	(camel_folder_summary_remove): Likewise.
	(camel_folder_summary_add): Cleaned up the clashing uid
	re-assignment logic a little bit.
	(camel_folder_summary_decode_uint32): Fixed a typo, 01 != -1.
	(camel_folder_summary_decode_time_t): Return -1 on error.
	(camel_folder_summary_encode_off_t): New function to encode an
	off_t type.
	(camel_folder_summary_decode_off_t): And likewise for the reverse.
	(CAMEL_FOLDER_SUMMARY_VERSION): Bumped the summary version, since
	we're now encoding time/off_t's right.
	(summary_header_save): Use time_t encoder to save the timestamp.
	(summary_header_load): Likewise for decoding the timestamp.
	(content_info_load): Decode off_t types directly, now we can.
	(content_info_save): And likewise for encoding.
	(camel_folder_summary_add_from_message): New function, create a
	summary item from an existing message and add it.
	(camel_folder_summary_info_new_from_message): New function, create
	a summary item from an existing message.
	(summary_build_content_info_message): New function to do the dirty
	work of building the conent info/indexing, from a message source.
	(format_recipients): Format an internetaddress suitable for the
	summary.
	(message_info_new_from_message): Build a new summary item from a
	mime message.
	(content_info_new_from_message): Build a new conent info from a
	mime part.
	(camel_folder_summary_class_init): Init the new class functions.
	(message_info_new_from_message): Fixed for message api change.

	Added documentation to the functions.

svn path=/trunk/; revision=6474

20 files changed, 1703 insertions, 543 deletions

diff --git a/camel/ChangeLog b/camel/ChangeLog
index c0e4ae2b38..258edef029 100644
--- a/camel/ChangeLog
+++ b/camel/ChangeLog
@@ -1,3 +1,234 @@
+2000-11-07  Not Zed  <NotZed@HelixCode.com>
+
+	* camel-mime-filter-bestenc.c (complete): Implement a complete()
+	function, now we need one.
+	(filter): Upgraded to match rfrc2045 properly.  Checks also for
+	length of line and valid CRLF sequences.
+	(camel_mime_filter_bestenc_get_best_encoding): Do the work of
+	working out what is the best encoding given what we found about
+	the stream.
+
+	* camel-mime-part.c (camel_mime_part_encoding_to_string): Use a
+	lookup table to get the encoding naem, and add the binary type.
+	(camel_mime_part_encoding_from_string): Likewise for the reverse.
+
+	* camel-mime-part.h: Added the binary encoding type, see rfc2045.
+
+	* camel-mime-utils.c (header_param_list_format_append): Dont put a
+	space before ;'s in parameter lists, makes them more
+	readable/consistent.
+
+	* camel-mime-message.c (multipart_has_8bit_parts): Cleaned up the
+	old stuff, well removed it.
+	(camel_mime_message_set_best_encoding): Added another argument
+	that lets you select what you want to set the best of.  i.e. for
+	smtp transport we only need 7 bit, and dont need to optimise the
+	charset (although of course, we should always).
+	(find_best_encoding): Implement this feature, if we are not
+	getting the best charset, use the one we have.
+	(best_encoding): Set the charset on the part appropriately.  Sigh,
+	the interfaces for this are nonexistant.
+	(find_best_encoding): Tell the bestenc filter that lf should be
+	treated as crlf for the purposes of determining encodings.
+
+2000-11-06  Not Zed  <NotZed@HelixCode.com>
+
+	* camel-charset-map.c (camel_charset_init): Init function for an
+	iterative charset determinator.
+	(camel_charset_step): Iterate another buffer.
+	(camel_charset_mask): Removed, since it couldn't have worked.
+	(camel_charset_best): Use the iterative interface to do the work.
+	(camel_charset_best_name): Get the best name for a charset so far.
+
+	* camel-mime-filter-bestenc.c: New class, a stream
+	filter that can be used to memory-efficiently determine the best
+	encoding and/or charset to use for a given stream of bytes.
+
+	* Makefile.am (libcamelinclude_HEADERS): Added stream-null*.
+	(libcamel_la_SOURCES): Added bestenc*
+
+	* camel-stream-null.c: New class, a null-stream, that always
+	succeeds, and never has any contents.
+
+	* camel-stream.c: Minor pointless changes.  Was going to do
+	something else but changed my mind.  Added trivial default
+	implementations for all callbacks.
+
+	* camel-mime-message.h: Cleaned up some old cruft.
+
+	* camel-folder-summary.c (camel_folder_summary_format_address):
+	address_list_format() no longer encodes, so we dont need to decode
+	it.
+
+	* camel-address.c (camel_address_unformat): New function, attempts
+	to reverse the formatting process on display addresses.
+	(camel_address_length): New function to get the number of
+	addresses, without having to peek the structure.
+
+	* camel-mime-message.c (camel_mime_message_set_from): Fix a typo.
+	(camel_mime_message_finalize): Only unref from/reply_to if we have
+	it.
+	(camel_mime_message_set_recipients): New function - set the
+	recipients as a CamelInternetAddress.  This function effectively
+	deprecates the older recipient setting functions.
+	(camel_mime_message_add_recipient): What the hell, i'll bite the
+	bullet.  Terminate this function.  The old api was ambiguious and
+	inefficient and didn't work right anyway.
+	(camel_mime_message_remove_recipient_address): And this one.
+	(camel_mime_message_remove_recipient_name): And this one too.
+	(camel_mime_message_set_recipients): If we set an empty header,
+	then remove it from the header list.  Allow a null receipient
+	object to clear a header.
+	(camel_mime_message_set_from): Likewise, if setting an empty from
+	address.
+	(camel_mime_message_encode_8bit_parts): Eeek!!
+	camel_stream_mem_new_with_byte_array owns the byte_array we give
+	it, so make sure we dont free any of it!
+	(camel_mime_message_encode_8bit_parts):  Infact, i'll just rewrite
+	the whole lot, its a bit of a mess.  Should really rename it and
+	make it a little more useful too, lets see ...
+	(best_encoding): This has a string interface?  Oh boy.
+	(camel_mime_message_foreach_part): New experimental function to
+	iterate over all message parts.  Might not remain.
+	(camel_mime_message_has_8bit_parts): New implementation using
+	foreach_part.  Fixed a couple of problems.
+	(find_best_encoding): New function, that finds the best encoding
+	for a given part (will probably be moved to camel-mime-part), and
+	also the best charset to use if it is a text part.  Since one
+	affects the other it is a two pass process, but uses streams and
+	not memory to achieve this.
+	(camel_mime_message_set_best_encoding): Uses the function above to
+	configure an entire message for the best encoding possible given
+	transport constraints.
+	(camel_mime_message_encode_8bit_parts): Reimplemented to use the
+	function above to perform the work.
+
+	* camel-internet-address.c
+	(camel_internet_address_format_address): Dont put <> around a lone
+	address with no real name.
+	(camel_internet_address_encode_address): Similarly.
+	(internet_decode): Actually return the count of decoded addresses.
+	(internet_unformat): Implement the unformatting routine.
+
+2000-11-05  Not Zed  <NotZed@HelixCode.com>
+
+	* providers/smtp/camel-smtp-transport.c (_send_to): Changed to get
+	the internetaddress directly, rather than having to parse it
+	itself.
+
+	* camel-address.c (camel_address_format): Added a new function
+	which will format address, suitable for display.
+	(camel_address_cat): Concatentate 1 camel address onto another.
+	It is upto the caller to ensure the addresses are of compatible
+	types.
+	(camel_address_new_clone): New function to create a new address by
+	copying an existing one of the same type.
+	(camel_address_copy): New helper function to copy an address.
+
+	* camel-mime-message.h (struct _CamelMimeMessage): Removed cached
+	copy of date string.
+	(struct _CamelMimeMessage): Added date_received info.
+
+	* camel-mime-message.c (camel_mime_message_get_date_string):
+	Removed.  Nothing uses it anyway, and it is redundant.
+	(camel_mime_message_finalize): No more date_str.
+	(camel_mime_message_init): No more date_str, initialise
+	date_received*
+	(write_to_stream): Change the check for a date header.
+	(process_header): No longer track the date_str.
+	(camel_mime_message_get_received_date): Removed.  totally invalid
+	anyway.
+	(camel_mime_message_get_sent_date): Removed.  Redundant.  The only
+	'date' is the sent date, the received date is just made up.
+	(camel_mime_message_get_date): Args changed to be more consistent
+	with utility functions.
+	(camel_mime_message_get_date): Dont set the date when we're asked
+	for it (if its not set by the time its written, it'll be set
+	then).
+	(camel_mime_message_get_date_received): Actually do 'the right
+	thing' here, if we have a received header, use that to determine
+	the received date.  And return the data in the same format as
+	get_date.
+	(camel_mime_message_set_from): Changed the api to better match
+	what we should be doing.  Pass a camelinternetaddress, etc.
+	(camel_mime_message_set_reply_to): Cahnged similarly to take an
+	internetaddress.
+	(camel_mime_message_get_reply_to): Likewise.
+	(camel_mime_message_finalize): Unref the from/reply_to objects.
+	(format_address): Removed, no longer needed.
+	(process_header): Changed to store the from/reply_to as
+	internetaddress's.
+	(write_to_stream): Set the from header directly to empty, if we
+	dont have one.  Maybe we should just abort, and/or create one
+	based on the current user.
+
+	* camel-mime-utils.c (header_address_list_format): Renamed to
+	header_address_list_encode, which is what it is actually doing.
+	(header_address_list_format_append): Similarly.
+	(encoding_map[]): Removed, no longer used.
+	(header_address_list_encode_append): Take another arg, do we
+	encode the address (for internet), or not (for display - utf8
+	only).
+	(header_address_list_format): Re-added this function, but now it
+	generates a display version only.  Surprise surprise, that is all
+	anythign needs to generate anyway.  Sigh.
+
+	* camel-internet-address.c (camel_internet_address_get): Return
+	false if we get an invalid index only.
+	(camel_internet_address_encode_address): Helper function to encode
+	a single address for mailing.
+	(internet_encode): Use the above function to format it.
+	(camel_internet_address_format_address): Format a single address
+	for display.
+	(internet_format): Implement the display version.
+	(camel_internet_address_class_init): Init the internet_format
+	virtual function.
+	(internet_cat): Implement virtual function to concatenate
+	addresses.
+
+	* camel-folder-summary.c
+	(camel_folder_summary_info_new_from_header): new function, only
+	build the summary info, dont add it.
+	(camel_folder_summary_info_new_from_parser): Likewise, for new
+	info from parser.
+	(camel_folder_summary_add_from_parser): Cahnged to call function
+	above to build info.
+	(camel_folder_summary_add_from_header): Changed to call function
+	above, to build info.
+	(camel_folder_summary_info_free): New function to free the summary
+	message info.
+	(camel_folder_summary_clear): Changed to clal above to free info.
+	(camel_folder_summary_remove): Likewise.
+	(camel_folder_summary_add): Cleaned up the clashing uid
+	re-assignment logic a little bit.
+	(camel_folder_summary_decode_uint32): Fixed a typo, 01 != -1.
+	(camel_folder_summary_decode_time_t): Return -1 on error.
+	(camel_folder_summary_encode_off_t): New function to encode an
+	off_t type.
+	(camel_folder_summary_decode_off_t): And likewise for the reverse.
+	(CAMEL_FOLDER_SUMMARY_VERSION): Bumped the summary version, since
+	we're now encoding time/off_t's right.
+	(summary_header_save): Use time_t encoder to save the timestamp.
+	(summary_header_load): Likewise for decoding the timestamp.
+	(content_info_load): Decode off_t types directly, now we can.
+	(content_info_save): And likewise for encoding.
+	(camel_folder_summary_add_from_message): New function, create a
+	summary item from an existing message and add it.
+	(camel_folder_summary_info_new_from_message): New function, create
+	a summary item from an existing message.
+	(summary_build_content_info_message): New function to do the dirty
+	work of building the conent info/indexing, from a message source.
+	(format_recipients): Format an internetaddress suitable for the
+	summary.
+	(message_info_new_from_message): Build a new summary item from a
+	mime message.
+	(content_info_new_from_message): Build a new conent info from a
+	mime part.
+	(camel_folder_summary_class_init): Init the new class functions.
+	(message_info_new_from_message): Fixed for message api change.
+
+	Added documentation to the functions.
+	
 2000-11-03  Radek Doulik  <rodo@helixcode.com>
 
 	* camel-mime-utils.c (header_msgid_generate): new function,
diff --git a/camel/Makefile.am b/camel/Makefile.am
index 67689b261a..8f487fdf68 100644
--- a/camel/Makefile.am
+++ b/camel/Makefile.am
@@ -25,6 +25,7 @@ libcamel_la_SOURCES = 				\
 	camel-folder.c				\
 	camel-internet-address.c		\
 	camel-medium.c				\
+	camel-mime-filter-bestenc.c		\
 	camel-mime-filter-basic.c		\
 	camel-mime-filter-charset.c		\
 	camel-mime-filter-crlf.c		\
@@ -52,6 +53,7 @@ libcamel_la_SOURCES = 				\
 	camel-stream-filter.c			\
 	camel-stream-fs.c			\
 	camel-stream-mem.c			\
+	camel-stream-null.c			\
 	camel-stream.c				\
 	camel-transport.c			\
 	camel-uid-cache.c			\
@@ -76,6 +78,7 @@ libcamelinclude_HEADERS =			\
 	camel-folder.h				\
 	camel-internet-address.h		\
 	camel-medium.h				\
+	camel-mime-filter-bestenc.h		\
 	camel-mime-filter-basic.h		\
 	camel-mime-filter-charset.h		\
 	camel-mime-filter-crlf.h		\
@@ -103,6 +106,7 @@ libcamelinclude_HEADERS =			\
 	camel-stream-filter.h			\
 	camel-stream-fs.h			\
 	camel-stream-mem.h			\
+	camel-stream-null.h			\
 	camel-stream.h				\
 	camel-transport.h			\
 	camel-types.h				\
diff --git a/camel/camel-address.c b/camel/camel-address.c
index 8f7cea3d67..392b0a69f9 100644
--- a/camel/camel-address.c
+++ b/camel/camel-address.c
@@ -73,10 +73,40 @@ camel_address_get_type (void)
 CamelAddress *
 camel_address_new (void)
 {
-	CamelAddress *new = CAMEL_ADDRESS ( camel_object_new (camel_address_get_type ()));
+	CamelAddress *new = CAMEL_ADDRESS(camel_object_new(camel_address_get_type()));
 	return new;
 }
 
+/**
+ * camel_address_new_clone:
+ * @in: 
+ * 
+ * Clone an existing address type.
+ * 
+ * Return value: 
+ **/
+CamelAddress *
+camel_address_new_clone(const CamelAddress *in)
+{
+	CamelAddress *new = CAMEL_ADDRESS(camel_object_new(CAMEL_OBJECT_GET_TYPE(in)));
+
+	camel_address_cat(new, in);
+	return new;
+}
+
+/**
+ * camel_address_length:
+ * @a: 
+ * 
+ * Return the number of addresses stored in the address @a.
+ * 
+ * Return value: 
+ **/
+int
+camel_address_length(CamelAddress *a)
+{
+	return a->addresses->len;
+}
 
 /**
  * camel_address_decode:
@@ -113,6 +143,82 @@ camel_address_encode	(CamelAddress *a)
 }
 
 /**
+ * camel_address_unformat:
+ * @a: 
+ * @raw: 
+ * 
+ * Attempt to convert a previously formatted and/or edited
+ * address back into internal form.
+ * 
+ * Return value: -1 if it could not be parsed, or the number
+ * of valid addresses found.
+ **/
+int
+camel_address_unformat(CamelAddress *a, const char *raw)
+{
+	g_return_val_if_fail(IS_CAMEL_ADDRESS(a), -1);
+
+	return CAMEL_ADDRESS_CLASS (CAMEL_OBJECT_GET_CLASS (a))->unformat(a, raw);
+}
+
+/**
+ * camel_address_format:
+ * @a: 
+ * 
+ * Format an address in a format suitable for display.
+ * 
+ * Return value: The formatted address.
+ **/
+char *
+camel_address_format	(CamelAddress *a)
+{
+	if (a == NULL)
+		return NULL;
+
+	g_return_val_if_fail(IS_CAMEL_ADDRESS(a), NULL);
+
+	return CAMEL_ADDRESS_CLASS (CAMEL_OBJECT_GET_CLASS (a))->format(a);
+}
+
+/**
+ * camel_address_cat:
+ * @dest: 
+ * @source: 
+ * 
+ * Concatenate one address onto another.  The addresses must
+ * be of the same type.
+ * 
+ * Return value: 
+ **/
+int
+camel_address_cat	(CamelAddress *dest, const CamelAddress *source)
+{
+	g_return_val_if_fail(IS_CAMEL_ADDRESS(dest), -1);
+	g_return_val_if_fail(IS_CAMEL_ADDRESS(source), -1);
+
+	return CAMEL_ADDRESS_CLASS(CAMEL_OBJECT_GET_CLASS(dest))->cat(dest, source);
+}
+
+/**
+ * camel_address_copy:
+ * @dest: 
+ * @source: 
+ * 
+ * Copy an address contents.
+ * 
+ * Return value: 
+ **/
+int
+camel_address_copy	(CamelAddress *dest, const CamelAddress *source)
+{
+	g_return_val_if_fail(IS_CAMEL_ADDRESS(dest), -1);
+	g_return_val_if_fail(IS_CAMEL_ADDRESS(source), -1);
+
+	camel_address_remove(dest, -1);
+	return camel_address_cat(dest, source);
+}
+
+/**
  * camel_address_remove:
  * @a: 
  * @index: The address to remove, use -1 to remove all address.
diff --git a/camel/camel-address.h b/camel/camel-address.h
index a2d6fe34dd..18cfbf8db7 100644
--- a/camel/camel-address.h
+++ b/camel/camel-address.h
@@ -43,14 +43,26 @@ struct _CamelAddressClass {
 	int   (*decode)		(CamelAddress *, const char *raw);
 	char *(*encode)		(CamelAddress *);
 
+	int   (*unformat)	(CamelAddress *, const char *raw);
+	char *(*format)		(CamelAddress *);
+
+	int   (*cat)		(CamelAddress *, const CamelAddress *);
+
 	void  (*remove)		(CamelAddress *, int index);
 };
 
 guint		camel_address_get_type	(void);
 CamelAddress   *camel_address_new	(void);
+CamelAddress   *camel_address_new_clone	(const CamelAddress *);
+int		camel_address_length	(CamelAddress *);
 
 int	        camel_address_decode	(CamelAddress *, const char *);
 char	       *camel_address_encode	(CamelAddress *);
+int	        camel_address_unformat	(CamelAddress *, const char *);
+char	       *camel_address_format	(CamelAddress *);
+
+int		camel_address_cat	(CamelAddress *, const CamelAddress *);
+int		camel_address_copy	(CamelAddress *, const CamelAddress *);
 
 void		camel_address_remove	(CamelAddress *, int index);
 
diff --git a/camel/camel-charset-map.c b/camel/camel-charset-map.c
index c0b697290b..fd18337aae 100644
--- a/camel/camel-charset-map.c
+++ b/camel/camel-charset-map.c
@@ -204,36 +204,22 @@ void main(void)
 #include <unicode.h>
 #include <glib.h>
 
-unsigned int
-camel_charset_mask(unsigned int c)
+void camel_charset_init(CamelCharset *c)
 {
-	if (c>0xffff)
-		return 0;
-
-	return charset_mask(c);
+	c->mask = ~0;
+	c->level = 0;
 }
 
-/* gets the best charset from the mask of chars in it */
-const char *
-camel_charset_best_mask(unsigned int mask)
+void
+camel_charset_step(CamelCharset *c, const char *in, int len)
 {
-	int i;
-
-	for (i=0;i<sizeof(camel_charinfo)/sizeof(camel_charinfo[0]);i++) {
-		if (camel_charinfo[i].bit & mask)
-			return camel_charinfo[i].name;
-	}
-	return "UTF-8";
-}
-
-/* finds the minimum charset for this string NULL means US-ASCII */
-const char *
-camel_charset_best(const char *in, int len)
-{
-	unsigned int mask = ~0;
-	int level = 0;
+	register unsigned int mask;
+	register int level;
 	const char *inptr = in, *inend = in+len;
 
+	mask = c->mask;
+	level = c->level;
+
 	/* check what charset a given string will fit in */
 	while (inptr < inend) {
 		unicode_char_t c;
@@ -245,7 +231,7 @@ camel_charset_best(const char *in, int len)
 		}
 		inptr = newinptr;
 		if (c<=0xffff) {
-			mask |= camel_charset_mask(c);
+			mask |= charset_mask(c);
 		
 			if (c>=128 && c<256)
 				level = MAX(level, 1);
@@ -257,12 +243,43 @@ camel_charset_best(const char *in, int len)
 		}
 	}
 
-	if (level == 1)
+	c->mask = mask;
+	c->level = level;
+}
+
+/* gets the best charset from the mask of chars in it */
+static const char *
+camel_charset_best_mask(unsigned int mask)
+{
+	int i;
+
+	for (i=0;i<sizeof(camel_charinfo)/sizeof(camel_charinfo[0]);i++) {
+		if (camel_charinfo[i].bit & mask)
+			return camel_charinfo[i].name;
+	}
+	return "UTF-8";
+}
+
+const char *camel_charset_best_name(CamelCharset *charset)
+{
+	if (charset->level == 1)
 		return "ISO-8859-1";
-	else if (level == 2)
-		return camel_charset_best_mask(mask);
+	else if (charset->level == 2)
+		return camel_charset_best_mask(charset->mask);
 	else
 		return NULL;
+
+}
+
+/* finds the minimum charset for this string NULL means US-ASCII */
+const char *
+camel_charset_best(const char *in, int len)
+{
+	CamelCharset charset;
+
+	camel_charset_init(&charset);
+	camel_charset_step(&charset, in, len);
+	return camel_charset_best_name(&charset);
 }
 
 
diff --git a/camel/camel-charset-map.h b/camel/camel-charset-map.h
index d5ce0799b8..53ba4af9d9 100644
--- a/camel/camel-charset-map.h
+++ b/camel/camel-charset-map.h
@@ -21,8 +21,18 @@
 #ifndef _CAMEL_CHARSET_MAP_H
 #define _CAMEL_CHARSET_MAP_H
 
-unsigned int camel_charset_mask(unsigned int c);
+typedef struct _CamelCharset CamelCharset;
+
+struct _CamelCharset {
+	unsigned int mask;
+	int level;
+};
+
+void camel_charset_init(CamelCharset *);
+void camel_charset_step(CamelCharset *, const char *in, int len);
+const char *camel_charset_best_name(CamelCharset *);
+
+/* helper function */
 const char *camel_charset_best(const char *in, int len);
-const char *camel_charset_best_mask(unsigned int mask);
 
 #endif /* ! _CAMEL_CHARSET_MAP_H */
diff --git a/camel/camel-folder-summary.c b/camel/camel-folder-summary.c
index f68c2a7970..2973083b5b 100644
--- a/camel/camel-folder-summary.c
+++ b/camel/camel-folder-summary.c
@@ -29,6 +29,7 @@
 #include "camel-folder-summary.h"
 
 #include <camel/camel-mime-message.h>
+#include <camel/camel-multipart.h>
 
 #include <camel/camel-mime-filter.h>
 #include <camel/camel-mime-filter-index.h>
@@ -36,6 +37,8 @@
 #include <camel/camel-mime-filter-save.h>
 #include <camel/camel-mime-filter-basic.h>
 #include <camel/camel-mime-message.h>
+#include <camel/camel-stream-mem.h>
+
 #include "hash-table-utils.h"
 
 /* this should probably be conditional on it existing */
@@ -48,7 +51,7 @@
 extern int strdup_count, malloc_count, free_count;
 #endif
 
-#define CAMEL_FOLDER_SUMMARY_VERSION (8)
+#define CAMEL_FOLDER_SUMMARY_VERSION (9)
 
 struct _CamelFolderSummaryPrivate {
 	GHashTable *filter_charset;	/* CamelMimeFilterCharset's indexed by source charset */
@@ -76,17 +79,20 @@ static int summary_header_save(CamelFolderSummary *, FILE *);
 
 static CamelMessageInfo * message_info_new(CamelFolderSummary *, struct _header_raw *);
 static CamelMessageInfo * message_info_new_from_parser(CamelFolderSummary *, CamelMimeParser *);
+static CamelMessageInfo * message_info_new_from_message(CamelFolderSummary *s, CamelMimeMessage *msg);
 static CamelMessageInfo * message_info_load(CamelFolderSummary *, FILE *);
 static int		  message_info_save(CamelFolderSummary *, FILE *, CamelMessageInfo *);
 static void		  message_info_free(CamelFolderSummary *, CamelMessageInfo *);
 
 static CamelMessageContentInfo * content_info_new(CamelFolderSummary *, struct _header_raw *);
 static CamelMessageContentInfo * content_info_new_from_parser(CamelFolderSummary *, CamelMimeParser *);
+static CamelMessageContentInfo * content_info_new_from_message(CamelFolderSummary *s, CamelMimePart *mp);
 static CamelMessageContentInfo * content_info_load(CamelFolderSummary *, FILE *);
 static int		         content_info_save(CamelFolderSummary *, FILE *, CamelMessageContentInfo *);
 static void		         content_info_free(CamelFolderSummary *, CamelMessageContentInfo *);
 
 static CamelMessageContentInfo * summary_build_content_info(CamelFolderSummary *s, CamelMimeParser *mp);
+static CamelMessageContentInfo * summary_build_content_info_message(CamelFolderSummary *s, CamelMessageInfo *msginfo, CamelMimePart *object);
 
 static void camel_folder_summary_class_init (CamelFolderSummaryClass *klass);
 static void camel_folder_summary_init       (CamelFolderSummary *obj);
@@ -104,12 +110,14 @@ camel_folder_summary_class_init (CamelFolderSummaryClass *klass)
 
 	klass->message_info_new  = message_info_new;
 	klass->message_info_new_from_parser = message_info_new_from_parser;
+	klass->message_info_new_from_message = message_info_new_from_message;
 	klass->message_info_load = message_info_load;
 	klass->message_info_save = message_info_save;
 	klass->message_info_free = message_info_free;
 
 	klass->content_info_new  = content_info_new;
 	klass->content_info_new_from_parser = content_info_new_from_parser;
+	klass->content_info_new_from_message = content_info_new_from_message;
 	klass->content_info_load = content_info_load;
 	klass->content_info_save = content_info_save;
 	klass->content_info_free = content_info_free;
@@ -204,12 +212,30 @@ camel_folder_summary_new (void)
 }
 
 
+/**
+ * camel_folder_summary_set_filename:
+ * @s: 
+ * @name: 
+ * 
+ * Set the filename where the summary will be loaded to/saved from.
+ **/
 void camel_folder_summary_set_filename(CamelFolderSummary *s, const char *name)
 {
 	g_free(s->summary_path);
 	s->summary_path = g_strdup(name);
 }
 
+/**
+ * camel_folder_summary_set_index:
+ * @s: 
+ * @index: 
+ * 
+ * Set the index used to index body content.  If the index is NULL, or
+ * not set (the default), no indexing of body content will take place.
+ *
+ * Also see :set_build_content(), which must be enabled to perform
+ * body indexing.
+ **/
 void camel_folder_summary_set_index(CamelFolderSummary *s, ibex *index)
 {
 	struct _CamelFolderSummaryPrivate *p = _PRIVATE(s);
@@ -217,17 +243,46 @@ void camel_folder_summary_set_index(CamelFolderSummary *s, ibex *index)
 	p->index = index;
 }
 
+/**
+ * camel_folder_summary_set_build_content:
+ * @s: 
+ * @state: 
+ * 
+ * Set a flag to tell the summary to build the content info summary
+ * (CamelMessageInfo.content).  The default is not to build content info
+ * summaries.
+ *
+ * This flag must also be set to on to perform indexing, see :set_index().
+ **/
 void camel_folder_summary_set_build_content(CamelFolderSummary *s, gboolean state)
 {
 	s->build_content = state;
 }
 
+/**
+ * camel_folder_summary_count:
+ * @s: 
+ * 
+ * Get the number of summary items stored in this summary.
+ * 
+ * Return value: The number of items int he summary.
+ **/
 int
 camel_folder_summary_count(CamelFolderSummary *s)
 {
 	return s->messages->len;
 }
 
+/**
+ * camel_folder_summary_index:
+ * @s: 
+ * @i: 
+ * 
+ * Retrieve a summary item by index number.
+ * 
+ * Return value: The summary item, or NULL if the index @i is out
+ * of range.
+ **/
 CamelMessageInfo *
 camel_folder_summary_index(CamelFolderSummary *s, int i)
 {
@@ -236,12 +291,31 @@ camel_folder_summary_index(CamelFolderSummary *s, int i)
 	return NULL;
 }
 
+/**
+ * camel_folder_summary_uid:
+ * @s: 
+ * @uid: 
+ * 
+ * Retrieve a summary item by uid.
+ * 
+ * Return value: The summary item, or NULL if the uid @uid
+ * is not available.
+ **/
 CamelMessageInfo *
 camel_folder_summary_uid(CamelFolderSummary *s, const char *uid)
 {
 	return g_hash_table_lookup(s->messages_uid, uid);
 }
 
+/**
+ * camel_folder_summary_next_uid:
+ * @s: 
+ * 
+ * Generate a new unique uid value as an integer.  This
+ * may be used to create a unique sequence of numbers.
+ * 
+ * Return value: The next unique uid value.
+ **/
 guint32 camel_folder_summary_next_uid(CamelFolderSummary *s)
 {
 	guint32 uid = s->nextuid++;
@@ -251,11 +325,30 @@ guint32 camel_folder_summary_next_uid(CamelFolderSummary *s)
 	return uid;
 }
 
+/**
+ * camel_folder_summary_set_uid:
+ * @s: 
+ * @uid: The next minimum uid to assign.  To avoid clashing
+ * uid's, set this to the uid of a given messages + 1.
+ * 
+ * Set the next minimum uid available.  This can be used to
+ * ensure new uid's do not clash with existing uid's.
+ **/
 void camel_folder_summary_set_uid(CamelFolderSummary *s, guint32 uid)
 {
+	/* TODO: sync to disk? */
 	s->nextuid = MAX(s->nextuid, uid);
 }
 
+/**
+ * camel_folder_summary_next_uid_string:
+ * @s: 
+ * 
+ * Retrieve the next uid, but as a formatted string.
+ * 
+ * Return value: The next uid as an unsigned integer string.
+ * This string must be freed by the caller.
+ **/
 char *
 camel_folder_summary_next_uid_string(CamelFolderSummary *s)
 {
@@ -339,6 +432,15 @@ perform_content_info_save(CamelFolderSummary *s, FILE *out, CamelMessageContentI
 	return 0;
 }
 
+/**
+ * camel_folder_summary_save:
+ * @s: 
+ * 
+ * Writes the summary to disk.  The summary is only written if changes
+ * have occured.
+ * 
+ * Return value: Returns -1 on error.
+ **/
 int
 camel_folder_summary_save(CamelFolderSummary *s)
 {
@@ -387,20 +489,31 @@ camel_folder_summary_save(CamelFolderSummary *s)
 	return 0;
 }
 
+/**
+ * camel_folder_summary_add:
+ * @s: 
+ * @info: 
+ * 
+ * Adds a new @info record to the summary.  If @info->uid is NULL, then a new
+ * uid is automatically re-assigned by calling :next_uid_string().
+ *
+ * The @info record should have been generated by calling one of the
+ * info_new_*() functions, as it will be free'd based on the summary
+ * class.
+ **/
 void camel_folder_summary_add(CamelFolderSummary *s, CamelMessageInfo *info)
 {
 	if (info == NULL)
 		return;
-retry:
-	if (info->uid == NULL) {
+
+	if (info->uid == NULL)
 		info->uid = camel_folder_summary_next_uid_string(s);
-	}
-	if (g_hash_table_lookup(s->messages_uid, info->uid)) {
+
+	while (g_hash_table_lookup(s->messages_uid, info->uid)) {
 		g_warning("Trying to insert message with clashing uid (%s).  new uid re-assigned", info->uid);
 		g_free(info->uid);
-		info->uid = NULL;
+		info->uid = camel_folder_summary_next_uid_string(s);
 		info->flags |= CAMEL_MESSAGE_FOLDER_FLAGGED;
-		goto retry;
 	}
 
 	g_ptr_array_add(s->messages, info);
@@ -408,18 +521,103 @@ retry:
 	s->flags |= CAMEL_SUMMARY_DIRTY;
 }
 
+/**
+ * camel_folder_summary_add_from_header:
+ * @s: 
+ * @h: 
+ * 
+ * Build a new info record based on a set of headers, and add it to the
+ * summary.
+ *
+ * Note that this function should not be used if build_content_info has
+ * been specified for this summary.
+ * 
+ * Return value: The newly added record.
+ **/
 CamelMessageInfo *camel_folder_summary_add_from_header(CamelFolderSummary *s, struct _header_raw *h)
 {
-	CamelMessageInfo *info = NULL;
+	CamelMessageInfo *info = camel_folder_summary_info_new_from_header(s, h);
 
-	info = ((CamelFolderSummaryClass *)(CAMEL_OBJECT_GET_CLASS(s))) -> message_info_new(s, h);
 	camel_folder_summary_add(s, info);
 
 	return info;
 }
 
+/**
+ * camel_folder_summary_add_from_parser:
+ * @s: 
+ * @mp: 
+ * 
+ * Build a new info record based on the current position of a CamelMimeParser.
+ *
+ * The parser should be positioned before the start of the message to summarise.
+ * This function may be used if build_contnet_info or an index has been
+ * specified for the summary.
+ * 
+ * Return value: The newly added record.
+ **/
 CamelMessageInfo *camel_folder_summary_add_from_parser(CamelFolderSummary *s, CamelMimeParser *mp)
 {
+	CamelMessageInfo *info = camel_folder_summary_info_new_from_parser(s, mp);
+
+	camel_folder_summary_add(s, info);
+
+	return info;
+}
+
+/**
+ * camel_folder_summary_add_from_message:
+ * @s: 
+ * @msg: 
+ * 
+ * Add a summary item from an existing message.
+ * 
+ * Return value: 
+ **/
+CamelMessageInfo *camel_folder_summary_add_from_message(CamelFolderSummary *s, CamelMimeMessage *msg)
+{
+	CamelMessageInfo *info = camel_folder_summary_info_new_from_message(s, msg);
+
+	camel_folder_summary_add(s, info);
+
+	return info;
+}
+
+/**
+ * camel_folder_summary_info_new_from_header:
+ * @s: 
+ * @h: 
+ * 
+ * Create a new info record from a header.
+ * 
+ * Return value: Guess?  This info record MUST be freed using
+ * camel_folder_summary_info_free(), camel_message_info_free() will not work.
+ **/
+CamelMessageInfo *camel_folder_summary_info_new_from_header(CamelFolderSummary *s, struct _header_raw *h)
+{
+	return ((CamelFolderSummaryClass *)(CAMEL_OBJECT_GET_CLASS(s))) -> message_info_new(s, h);
+}
+
+/**
+ * camel_folder_summary_info_new_from_parser:
+ * @s: 
+ * @mp: 
+ * 
+ * Create a new info record from a parser.  If the parser cannot
+ * determine a uid, then a new one is automatically assigned.
+ *
+ * If indexing is enabled, then the content will be indexed based
+ * on this new uid.  In this case, the message info MUST be
+ * added using :add().
+ *
+ * Once complete, the parser will be positioned at the end of
+ * the message.
+ *
+ * Return value: Guess?  This info record MUST be freed using
+ * camel_folder_summary_info_free(), camel_message_info_free() will not work.
+ **/
+CamelMessageInfo *camel_folder_summary_info_new_from_parser(CamelFolderSummary *s, CamelMimeParser *mp)
+{
 	CamelMessageInfo *info = NULL;
 	char *buffer;
 	int len;
@@ -432,7 +630,16 @@ CamelMessageInfo *camel_folder_summary_add_from_parser(CamelFolderSummary *s, Ca
 
 		camel_mime_parser_unstep(mp);
 
-		camel_folder_summary_add(s, info);
+		/* assign a unique uid, this is slightly 'wrong' as we do not really
+		 * know if we are going to store this in the summary, but no matter */
+		if (info->uid == NULL)
+			info->uid = camel_folder_summary_next_uid_string(s);
+
+		while (g_hash_table_lookup(s->messages_uid, info->uid)) {
+			g_free(info->uid);
+			info->uid = camel_folder_summary_next_uid_string(s);
+			info->flags |= CAMEL_MESSAGE_FOLDER_FLAGGED;
+		}
 
 		if (p->index) {
 			if (p->filter_index == NULL)
@@ -453,6 +660,46 @@ CamelMessageInfo *camel_folder_summary_add_from_parser(CamelFolderSummary *s, Ca
 	return info;
 }
 
+/**
+ * camel_folder_summary_info_new_from_message:
+ * @: 
+ * @: 
+ * 
+ * Create a summary item from a message.
+ * 
+ * Return value: 
+ **/
+CamelMessageInfo *camel_folder_summary_info_new_from_message(CamelFolderSummary *s, CamelMimeMessage *msg)
+{
+	CamelMessageInfo *info;
+	struct _CamelFolderSummaryPrivate *p = _PRIVATE(s);
+
+	info = ((CamelFolderSummaryClass *)(CAMEL_OBJECT_GET_CLASS(s)))->message_info_new_from_message(s, msg);
+
+	/* assign a unique uid, this is slightly 'wrong' as we do not really
+	 * know if we are going to store this in the summary, but no matter */
+	if (info->uid == NULL)
+		info->uid = camel_folder_summary_next_uid_string(s);
+
+	while (g_hash_table_lookup(s->messages_uid, info->uid)) {
+		g_free(info->uid);
+		info->uid = camel_folder_summary_next_uid_string(s);
+		info->flags |= CAMEL_MESSAGE_FOLDER_FLAGGED;
+	}
+
+	if (p->index)
+		ibex_unindex(p->index, info->uid);
+
+	/* build the content info, if we're supposed to */
+	if (s->build_content) {
+		info->content = summary_build_content_info_message(s, info, (CamelMimePart *)msg);
+		if (info->content->pos != -1)
+			info->size = info->content->endpos - info->content->pos;
+	}
+
+	return info;
+}
+
 static void
 perform_content_info_free(CamelFolderSummary *s, CamelMessageContentInfo *ci)
 {
@@ -467,12 +714,46 @@ perform_content_info_free(CamelFolderSummary *s, CamelMessageContentInfo *ci)
 	}
 }
 
+/**
+ * camel_folder_summary_info_free:
+ * @s: 
+ * @mi: 
+ * 
+ * Free the message info @mi, and all associated memory.
+ **/
+void camel_folder_summary_info_free(CamelFolderSummary *s, CamelMessageInfo *mi)
+{
+	CamelMessageContentInfo *ci;
+
+	g_assert(mi);
+	g_assert(s);
+
+	ci = mi->content;
+
+	((CamelFolderSummaryClass *)(CAMEL_OBJECT_GET_CLASS(s)))->message_info_free(s, mi);		
+	if (s->build_content && ci) {
+		perform_content_info_free(s, ci);
+	}
+}
+
+/**
+ * camel_folder_summary_touch:
+ * @s: 
+ * 
+ * Mark the summary as changed, so that a save will save it.
+ **/
 void
 camel_folder_summary_touch(CamelFolderSummary *s)
 {
 	s->flags |= CAMEL_SUMMARY_DIRTY;
 }
 
+/**
+ * camel_folder_summary_clear:
+ * @s: 
+ * 
+ * Empty the summary contents.
+ **/
 void
 camel_folder_summary_clear(CamelFolderSummary *s)
 {
@@ -481,15 +762,8 @@ camel_folder_summary_clear(CamelFolderSummary *s)
 	if (camel_folder_summary_count(s) == 0)
 		return;
 
-	for (i=0;i<camel_folder_summary_count(s);i++) {
-		CamelMessageInfo *mi = camel_folder_summary_index(s, i);
-		CamelMessageContentInfo *ci = mi->content;
-
-		((CamelFolderSummaryClass *)(CAMEL_OBJECT_GET_CLASS(s)))->message_info_free(s, mi);		
-		if (s->build_content && ci) {
-			perform_content_info_free(s, ci);
-		}
-	}
+	for (i=0;i<camel_folder_summary_count(s);i++)
+		camel_folder_summary_info_free(s, camel_folder_summary_index(s, i));
 
 	g_ptr_array_set_size(s->messages, 0);
 	g_hash_table_destroy(s->messages_uid);
@@ -497,19 +771,28 @@ camel_folder_summary_clear(CamelFolderSummary *s)
 	s->flags |= CAMEL_SUMMARY_DIRTY;
 }
 
+/**
+ * camel_folder_summary_remove:
+ * @s: 
+ * @info: 
+ * 
+ * Remove a specific @info record from the summary.
+ **/
 void camel_folder_summary_remove(CamelFolderSummary *s, CamelMessageInfo *info)
 {
-	CamelMessageContentInfo *ci = info->content;
-
 	g_hash_table_remove(s->messages_uid, info->uid);
 	g_ptr_array_remove(s->messages, info);
-	((CamelFolderSummaryClass *)(CAMEL_OBJECT_GET_CLASS(s)))->message_info_free(s, info);
-	if (s->build_content && ci) {
-		perform_content_info_free(s, ci);
-	}
+	camel_folder_summary_info_free(s, info);
 	s->flags |= CAMEL_SUMMARY_DIRTY;
 }
 
+/**
+ * camel_folder_summary_remove_uid:
+ * @s: 
+ * @uid: 
+ * 
+ * Remove a specific info record from the summary, by @uid.
+ **/
 void camel_folder_summary_remove_uid(CamelFolderSummary *s, const char *uid)
 {
         CamelMessageInfo *oldinfo;
@@ -521,6 +804,13 @@ void camel_folder_summary_remove_uid(CamelFolderSummary *s, const char *uid)
         }
 }
 
+/**
+ * camel_folder_summary_remove_index:
+ * @s: 
+ * @index: 
+ * 
+ * Remove a specific info record from the summary, by index.
+ **/
 void camel_folder_summary_remove_index(CamelFolderSummary *s, int index)
 {
         CamelMessageInfo *oldinfo;
@@ -530,6 +820,15 @@ void camel_folder_summary_remove_index(CamelFolderSummary *s, int index)
 		camel_folder_summary_remove(s, oldinfo);
 }
 
+/**
+ * camel_folder_summary_encode_uint32:
+ * @out: 
+ * @value: 
+ * 
+ * Utility function to save an uint32 to a file.
+ * 
+ * Return value: -1 on error.
+ **/
 int
 camel_folder_summary_encode_uint32(FILE *out, guint32 value)
 {
@@ -547,6 +846,16 @@ camel_folder_summary_encode_uint32(FILE *out, guint32 value)
 	return fputc(value | 0x80, out);
 }
 
+/**
+ * camel_folder_summary_decode_uint32:
+ * @in: 
+ * @dest: 
+ * 
+ * Retrieve an encoded uint32 from a file.
+ * 
+ * Return value: -1 on error.  @*dest will contain the
+ * decoded value.
+ **/
 int
 camel_folder_summary_decode_uint32(FILE *in, guint32 *dest)
 {
@@ -559,7 +868,7 @@ camel_folder_summary_decode_uint32(FILE *in, guint32 *dest)
         }
 	if (v == EOF) {
 		*dest = value>>7;
-		return 01;
+		return -1;
 	}
 	*dest = value | (v&0x7f);
 
@@ -568,6 +877,16 @@ camel_folder_summary_decode_uint32(FILE *in, guint32 *dest)
         return 0;
 }
 
+/**
+ * camel_folder_summary_encode_fixed_int32:
+ * @out: 
+ * @value: 
+ * 
+ * Encode a gint32, performing no compression, but converting
+ * to network order.
+ * 
+ * Return value: -1 on error.
+ **/
 int
 camel_folder_summary_encode_fixed_int32(FILE *out, gint32 value)
 {
@@ -579,6 +898,15 @@ camel_folder_summary_encode_fixed_int32(FILE *out, gint32 value)
 	return 0;
 }
 
+/**
+ * camel_folder_summary_decode_fixed_int32:
+ * @in: 
+ * @dest: 
+ * 
+ * Retrieve a gint32.
+ * 
+ * Return value: -1 on error.
+ **/
 int
 camel_folder_summary_decode_fixed_int32(FILE *in, gint32 *dest)
 {
@@ -592,6 +920,15 @@ camel_folder_summary_decode_fixed_int32(FILE *in, gint32 *dest)
 	}
 }
 
+/**
+ * camel_folder_summary_encode_time_t:
+ * @out: 
+ * @value: 
+ * 
+ * Encode a time_t value to the file.
+ * 
+ * Return value: -1 on error.
+ **/
 int
 camel_folder_summary_encode_time_t(FILE *out, time_t value)
 {
@@ -604,6 +941,15 @@ camel_folder_summary_encode_time_t(FILE *out, time_t value)
 	return 0;
 }
 
+/**
+ * camel_folder_summary_decode_time_t:
+ * @in: 
+ * @dest: 
+ * 
+ * Decode a time_t value.
+ * 
+ * Return value: -1 on error.
+ **/
 int
 camel_folder_summary_decode_time_t(FILE *in, time_t *dest)
 {
@@ -616,6 +962,55 @@ camel_folder_summary_decode_time_t(FILE *in, time_t *dest)
 		i--;
 	}
 	*dest = save;
+	if (v == EOF)
+		return -1;
+	return 0;
+}
+
+/**
+ * camel_folder_summary_encode_off_t:
+ * @out: 
+ * @value: 
+ * 
+ * Encode an off_t type.
+ * 
+ * Return value: 
+ **/
+int
+camel_folder_summary_encode_off_t(FILE *out, off_t value)
+{
+	int i;
+
+	for (i=sizeof(off_t)-1;i>=0;i--) {
+		if (fputc((value >> (i*8)) & 0xff, out) == -1)
+			return -1;
+	}
+	return 0;
+}
+
+/**
+ * camel_folder_summary_decode_off_t:
+ * @in: 
+ * @dest: 
+ * 
+ * Decode an off_t type.
+ * 
+ * Return value: 
+ **/
+int
+camel_folder_summary_decode_off_t(FILE *in, off_t *dest)
+{
+	off_t save = 0;
+	unsigned int v;
+	int i = sizeof(off_t) - 1;
+
+        while ( i>=0 && (v = fgetc(in)) != EOF) {
+		save |= v << (i*8);
+		i--;
+	}
+	*dest = save;
+	if (v == EOF)
+		return -1;
 	return 0;
 }
 
@@ -668,6 +1063,17 @@ token_search_cmp(char *key, char **index)
 }
 #endif
 
+/**
+ * camel_folder_summary_encode_token:
+ * @out: 
+ * @str: 
+ * 
+ * Encode a string value, but use tokenisation and compression
+ * to reduce the size taken for common mailer words.  This
+ * can still be used to encode normal strings as well.
+ * 
+ * Return value: -1 on error.
+ **/
 int
 camel_folder_summary_encode_token(FILE *out, char *str)
 {
@@ -711,6 +1117,15 @@ camel_folder_summary_encode_token(FILE *out, char *str)
 	return 0;
 }
 
+/**
+ * camel_folder_summary_decode_token:
+ * @in: 
+ * @str: 
+ * 
+ * Decode a token value.
+ * 
+ * Return value: -1 on error.
+ **/
 int
 camel_folder_summary_decode_token(FILE *in, char **str)
 {
@@ -756,6 +1171,15 @@ camel_folder_summary_decode_token(FILE *in, char **str)
 	return 0;
 }
 
+/**
+ * camel_folder_summary_encode_string:
+ * @out: 
+ * @str: 
+ * 
+ * Encode a normal string and save it in the output file.
+ * 
+ * Return value: -1 on error.
+ **/
 int
 camel_folder_summary_encode_string(FILE *out, char *str)
 {
@@ -775,6 +1199,15 @@ camel_folder_summary_encode_string(FILE *out, char *str)
 }
 
 
+/**
+ * camel_folder_summary_decode_string:
+ * @in: 
+ * @str: 
+ * 
+ * Decode a normal string from the input file.
+ * 
+ * Return value: -1 on error.
+ **/
 int
 camel_folder_summary_decode_string(FILE *in, char **str)
 {
@@ -809,6 +1242,13 @@ camel_folder_summary_decode_string(FILE *in, char **str)
 	return 0;
 }
 
+/**
+ * camel_folder_summary_offset_content:
+ * @content: 
+ * @offset: 
+ * 
+ * Offset the content info @content by @offset characters.
+ **/
 void
 camel_folder_summary_offset_content(CamelMessageContentInfo *content, off_t offset)
 {
@@ -848,7 +1288,8 @@ my_list_size(struct _node **list)
 static int
 summary_header_load(CamelFolderSummary *s, FILE *in)
 {
-	gint32 version, flags, nextuid, count, utime;
+	gint32 version, flags, nextuid, count;
+	time_t time;
 
 	fseek(in, 0, SEEK_SET);
 
@@ -857,14 +1298,14 @@ summary_header_load(CamelFolderSummary *s, FILE *in)
 	if (camel_folder_summary_decode_fixed_int32(in, &version) == -1
 	    || camel_folder_summary_decode_fixed_int32(in, &flags) == -1
 	    || camel_folder_summary_decode_fixed_int32(in, &nextuid) == -1
-	    || camel_folder_summary_decode_fixed_int32(in, &utime) == -1
+	    || camel_folder_summary_decode_time_t(in, &time) == -1
 	    || camel_folder_summary_decode_fixed_int32(in, &count) == -1) {
 		return -1;
 	}
 
 	s->nextuid = nextuid;
 	s->flags = flags;
-	s->time = (time_t) utime;
+	s->time = time;
 	s->saved_count = count;
 	if (s->version != version) {
 		g_warning("Summary header version mismatch");
@@ -883,7 +1324,7 @@ summary_header_save(CamelFolderSummary *s, FILE *out)
 	camel_folder_summary_encode_fixed_int32(out, s->version);
 	camel_folder_summary_encode_fixed_int32(out, s->flags);
 	camel_folder_summary_encode_fixed_int32(out, s->nextuid);
-	camel_folder_summary_encode_fixed_int32(out, s->time);
+	camel_folder_summary_encode_time_t(out, s->time);
 	return camel_folder_summary_encode_fixed_int32(out, camel_folder_summary_count(s));
 }
 
@@ -928,20 +1369,79 @@ static CamelMessageContentInfo * content_info_new_from_parser(CamelFolderSummary
 	return ci;
 }
 
+static char *
+format_recipients(const CamelInternetAddress *addr)
+{
+	const char *namep, *addrp;
+	char *ret;
+	int i;
+	GString *out;
+
+	if (addr == NULL)
+		return NULL;
+
+	out = g_string_new("");
+
+	for (i=0;camel_internet_address_get(addr, i, &namep, &addrp);i++) {
+		if (i>0)
+			g_string_append(out, ", ");
+		if (namep)
+			g_string_sprintfa(out, "%s <%s>", namep, addrp);
+		else
+			g_string_sprintfa(out, "<%s>", addrp);
+	}
+	/* well, this is probably more memory efficient, unfortunately */
+	ret = g_strdup(out->str);
+	g_string_free(out, TRUE);
+
+	return ret;
+}
+
+static CamelMessageInfo * message_info_new_from_message(CamelFolderSummary *s, CamelMimeMessage *msg)
+{
+	CamelMessageInfo *mi;
+
+	mi = g_malloc0(s->message_info_size);
+
+	mi->subject = g_strdup(camel_mime_message_get_subject(msg));
+	mi->from = camel_address_format((CamelAddress *)camel_mime_message_get_from(msg));
+	mi->to = format_recipients(camel_mime_message_get_recipients(msg, "to"));
+	mi->cc = format_recipients(camel_mime_message_get_recipients(msg, "cc"));
+	mi->user_flags = NULL;
+	mi->user_tags = NULL;
+	mi->date_sent = camel_mime_message_get_date(msg, NULL);
+	mi->date_received = camel_mime_message_get_date_received(msg, NULL);
+	mi->message_id = header_msgid_decode(camel_medium_get_header((CamelMedium *)msg, "message-id"));
+	/* if we have a references, use that, otherwise, see if we have an in-reply-to
+	   header, with parsable content, otherwise *shrug* */
+	mi->references = header_references_decode(camel_medium_get_header((CamelMedium *)msg, "message-id"));
+	if (mi->references == NULL)
+		mi->references = header_references_decode(camel_medium_get_header((CamelMedium *)msg, "message-id"));
+
+	return mi;
+}
+
+static CamelMessageContentInfo * content_info_new_from_message(CamelFolderSummary *s, CamelMimePart *mp)
+{
+	CamelMessageContentInfo *ci;
+
+	ci = ((CamelFolderSummaryClass *)(CAMEL_OBJECT_GET_CLASS(s)))->content_info_new(s, mp->headers);
+
+	return ci;
+}
+
+#warning "These should be made private again, easy to fix (used in filter-driver)"
 char *
 camel_folder_summary_format_address(struct _header_raw *h, const char *name)
 {
 	struct _header_address *addr;
 	const char *text;
-	char *ret, *tmp;
+	char *ret;
 
 	text = header_raw_find (&h, name, NULL);
 	addr = header_address_decode (text);
 	if (addr) {
-		/* FIXME: perhaps decoding would be best done in header_address_list_format */
-		tmp = header_address_list_format (addr);
-		ret = header_decode_string (tmp);
-		g_free (tmp);
+		ret = header_address_list_format (addr);
 		header_address_list_clear (&addr);
 	} else {
 		ret = g_strdup (text);
@@ -1127,20 +1627,16 @@ content_info_load(CamelFolderSummary *s, FILE *in)
 {
 	CamelMessageContentInfo *ci;
 	char *type, *subtype;
-	guint32 count, i, upos, ubodypos, uendpos;
+	guint32 count, i;
 	struct _header_content_type *ct;
 
 	io(printf("Loading content info\n"));
 
 	ci = g_malloc0(s->content_info_size);
 
-	camel_folder_summary_decode_uint32(in, &upos);
-	camel_folder_summary_decode_uint32(in, &ubodypos);
-	camel_folder_summary_decode_uint32(in, &uendpos);
-
-	ci->pos = (off_t) upos;
-        ci->bodypos = (off_t) ubodypos;
-	ci->endpos = (off_t) uendpos;
+	camel_folder_summary_decode_off_t(in, &ci->pos);
+	camel_folder_summary_decode_off_t(in, &ci->bodypos);
+	camel_folder_summary_decode_off_t(in, &ci->endpos);
 	
 	camel_folder_summary_decode_token(in, &type);
 	camel_folder_summary_decode_token(in, &subtype);
@@ -1175,9 +1671,9 @@ content_info_save(CamelFolderSummary *s, FILE *out, CamelMessageContentInfo *ci)
 
 	io(printf("Saving content info\n"));
 
-	camel_folder_summary_encode_uint32(out, ci->pos);
-	camel_folder_summary_encode_uint32(out, ci->bodypos);
-	camel_folder_summary_encode_uint32(out, ci->endpos);
+	camel_folder_summary_encode_off_t(out, ci->pos);
+	camel_folder_summary_encode_off_t(out, ci->bodypos);
+	camel_folder_summary_encode_off_t(out, ci->endpos);
 
 	ct = ci->type;
 	if (ct) {
@@ -1334,6 +1830,66 @@ summary_build_content_info(CamelFolderSummary *s, CamelMimeParser *mp)
 	return info;
 }
 
+/* build the content-info, from a message */
+static CamelMessageContentInfo *
+summary_build_content_info_message(CamelFolderSummary *s, CamelMessageInfo *msginfo, CamelMimePart *object)
+{
+	CamelDataWrapper *containee;
+	int parts, i;
+	struct _CamelFolderSummaryPrivate *p = _PRIVATE(s);
+	CamelMessageContentInfo *info = NULL, *child;
+
+	info = ((CamelFolderSummaryClass *)(CAMEL_OBJECT_GET_CLASS(s)))->content_info_new_from_message(s, object);
+	
+	containee = camel_medium_get_content_object(CAMEL_MEDIUM(object));
+
+	if (containee == NULL)
+		return info;
+
+	/* TODO: I find it odd that get_part and get_content_object do not
+	   add a reference, probably need fixing for multithreading */
+
+	/* using the object types is more accurate than using the mime/types */
+	if (CAMEL_IS_MULTIPART(containee)) {
+		parts = camel_multipart_get_number(CAMEL_MULTIPART(containee));
+		for (i=0;i<parts;i++) {
+			CamelMimePart *part = camel_multipart_get_part(CAMEL_MULTIPART(containee), i);
+			g_assert(part);
+			child = summary_build_content_info_message(s, msginfo, part);
+			if (child) {
+				child->parent = info;
+				my_list_append((struct _node **)&info->childs, (struct _node *)child);
+			}
+		}
+	} else if (CAMEL_IS_MIME_MESSAGE(containee)) {
+		/* for messages we only look at its contents */
+		child = summary_build_content_info_message(s, msginfo, (CamelMimePart *)containee);
+		if (child) {
+			child->parent = info;
+			my_list_append((struct _node **)&info->childs, (struct _node *)child);
+		}
+	} else if (p->index
+		   && gmime_content_field_is_type(CAMEL_DATA_WRAPPER(containee)->mime_type, "text", "*")) {
+		/* index all text parts if we're indexing */
+		CamelStreamMem *mem = (CamelStreamMem *)camel_stream_mem_new();
+
+		camel_data_wrapper_write_to_stream(containee, (CamelStream *)mem);
+		ibex_index_buffer(p->index, msginfo->uid, mem->buffer->data, mem->buffer->len, NULL);
+		camel_object_unref((CamelObject *)mem);
+	}
+
+	return info;
+}
+
+/**
+ * camel_flag_get:
+ * @list: 
+ * @name: 
+ * 
+ * Find the state of the flag @name in @list.
+ * 
+ * Return value: The state of the flag (TRUE or FALSE).
+ **/
 gboolean
 camel_flag_get(CamelFlag **list, const char *name)
 {
@@ -1347,6 +1903,14 @@ camel_flag_get(CamelFlag **list, const char *name)
 	return FALSE;
 }
 
+/**
+ * camel_flag_set:
+ * @list: 
+ * @name: 
+ * @value: 
+ * 
+ * Set the state of a flag @name in the list @list to @value.
+ **/
 void
 camel_flag_set(CamelFlag **list, const char *name, gboolean value)
 {
@@ -1374,6 +1938,14 @@ camel_flag_set(CamelFlag **list, const char *name, gboolean value)
 	}
 }
 
+/**
+ * camel_flag_list_size:
+ * @list: 
+ * 
+ * Get the length of the flag list.
+ * 
+ * Return value: The number of TRUE flags in the list.
+ **/
 int
 camel_flag_list_size(CamelFlag **list)
 {
@@ -1388,6 +1960,12 @@ camel_flag_list_size(CamelFlag **list)
 	return count;
 }
 
+/**
+ * camel_flag_list_free:
+ * @list: 
+ * 
+ * Free the memory associated with the flag list @list.
+ **/
 void
 camel_flag_list_free(CamelFlag **list)
 {
@@ -1414,6 +1992,14 @@ const char	*camel_tag_get(CamelTag **list, const char *name)
 	return NULL;
 }
 
+/**
+ * camel_tag_set:
+ * @list: 
+ * @name: 
+ * @value: 
+ * 
+ * Set the tag @name in the tag list @list to @value.
+ **/
 void		camel_tag_set(CamelTag **list, const char *name, const char *value)
 {
 	CamelTag *tag, *tmp;
@@ -1445,6 +2031,14 @@ void		camel_tag_set(CamelTag **list, const char *name, const char *value)
 	}
 }
 
+/**
+ * camel_tag_list_size:
+ * @list: 
+ * 
+ * Get the number of tags present in the tag list @list.
+ * 
+ * Return value: The number of tags.
+ **/
 int		camel_tag_list_size(CamelTag **list)
 {
 	int count=0;
@@ -1458,6 +2052,12 @@ int		camel_tag_list_size(CamelTag **list)
 	return count;
 }
 
+/**
+ * camel_tag_list_free:
+ * @list: 
+ * 
+ * Free the tag list @list.
+ **/
 void		camel_tag_list_free(CamelTag **list)
 {
 	CamelTag *tag, *tmp;
@@ -1515,6 +2115,7 @@ camel_message_info_dup_to(const CamelMessageInfo *from, CamelMessageInfo *to)
 		tag = tag->next;
 	}
 
+	/* No, this is impossible without knowing the class of summary we came from */
 	/* FIXME some day */
 	to->content = NULL;
 }
@@ -1524,6 +2125,10 @@ camel_message_info_dup_to(const CamelMessageInfo *from, CamelMessageInfo *to)
  * @mi: the message info
  *
  * Frees a CamelMessageInfo and its contents.
+ *
+ * Can only be used to free CamelMessageInfo's created with
+ * camel_message_info_dup_to.
+ *
  **/
 void
 camel_message_info_free(CamelMessageInfo *mi)
diff --git a/camel/camel-folder-summary.h b/camel/camel-folder-summary.h
index 95b394c69f..9908c1283e 100644
--- a/camel/camel-folder-summary.h
+++ b/camel/camel-folder-summary.h
@@ -141,6 +141,7 @@ struct _CamelFolderSummaryClass {
 	/* create/save/load an individual message info */
 	CamelMessageInfo * (*message_info_new)(CamelFolderSummary *, struct _header_raw *);
 	CamelMessageInfo * (*message_info_new_from_parser)(CamelFolderSummary *, CamelMimeParser *);
+	CamelMessageInfo * (*message_info_new_from_message)(CamelFolderSummary *, CamelMimeMessage *);
 	CamelMessageInfo * (*message_info_load)(CamelFolderSummary *, FILE *);
 	int		   (*message_info_save)(CamelFolderSummary *, FILE *, CamelMessageInfo *);
 	void		   (*message_info_free)(CamelFolderSummary *, CamelMessageInfo *);
@@ -148,6 +149,7 @@ struct _CamelFolderSummaryClass {
 	/* save/load individual content info's */
 	CamelMessageContentInfo * (*content_info_new)(CamelFolderSummary *, struct _header_raw *);
 	CamelMessageContentInfo * (*content_info_new_from_parser)(CamelFolderSummary *, CamelMimeParser *);
+	CamelMessageContentInfo * (*content_info_new_from_message)(CamelFolderSummary *, CamelMimePart *);
 	CamelMessageContentInfo * (*content_info_load)(CamelFolderSummary *, FILE *);
 	int		          (*content_info_save)(CamelFolderSummary *, FILE *, CamelMessageContentInfo *);
 	void		          (*content_info_free)(CamelFolderSummary *, CamelMessageContentInfo *);
@@ -177,6 +179,14 @@ void camel_folder_summary_add(CamelFolderSummary *, CamelMessageInfo *info);
 /* build/add raw summary items */
 CamelMessageInfo *camel_folder_summary_add_from_header(CamelFolderSummary *, struct _header_raw *);
 CamelMessageInfo *camel_folder_summary_add_from_parser(CamelFolderSummary *, CamelMimeParser *);
+CamelMessageInfo *camel_folder_summary_add_from_message(CamelFolderSummary *, CamelMimeMessage *);
+
+/* Just build raw summary items */
+CamelMessageInfo *camel_folder_summary_info_new_from_header(CamelFolderSummary *, struct _header_raw *);
+CamelMessageInfo *camel_folder_summary_info_new_from_parser(CamelFolderSummary *, CamelMimeParser *);
+CamelMessageInfo *camel_folder_summary_info_new_from_message(CamelFolderSummary *, CamelMimeMessage *);
+
+void camel_folder_summary_info_free(CamelFolderSummary *, CamelMessageInfo *);
 
 /* removes a summary item, doesn't fix content offsets */
 void camel_folder_summary_remove(CamelFolderSummary *s, CamelMessageInfo *info);
@@ -194,19 +204,18 @@ CamelMessageInfo *camel_folder_summary_uid(CamelFolderSummary *, const char *uid
 void camel_folder_summary_offset_content(CamelMessageContentInfo *content, off_t offset);
 
 /* summary formatting utils */
-char *camel_folder_summary_format_address (struct _header_raw *h, const char *name);
-char *camel_folder_summary_format_string (struct _header_raw *h, const char *name);
+char *camel_folder_summary_format_address(struct _header_raw *h, const char *name);
+char *camel_folder_summary_format_string(struct _header_raw *h, const char *name);
 
 /* summary file loading/saving helper functions */
 int camel_folder_summary_encode_fixed_int32(FILE *, gint32);
 int camel_folder_summary_decode_fixed_int32(FILE *, gint32 *);
-
 int camel_folder_summary_encode_uint32(FILE *, guint32);
 int camel_folder_summary_decode_uint32(FILE *, guint32 *);
-
 int camel_folder_summary_encode_time_t(FILE *out, time_t value);
 int camel_folder_summary_decode_time_t(FILE *in, time_t *dest);
-
+int camel_folder_summary_encode_off_t(FILE *out, off_t value);
+int camel_folder_summary_decode_off_t(FILE *in, off_t *dest);
 int camel_folder_summary_encode_string(FILE *, char *);
 int camel_folder_summary_decode_string(FILE *, char **);
 
diff --git a/camel/camel-internet-address.c b/camel/camel-internet-address.c
index 878ff1ca7b..e93fd581a1 100644
--- a/camel/camel-internet-address.c
+++ b/camel/camel-internet-address.c
@@ -21,8 +21,15 @@
 #include "camel-mime-utils.h"
 #include "camel-internet-address.h"
 
+#include <stdio.h>
+
+#define d(x) x
+
 static int    internet_decode		(CamelAddress *, const char *raw);
 static char * internet_encode		(CamelAddress *);
+static int    internet_unformat		(CamelAddress *, const char *raw);
+static char * internet_format		(CamelAddress *);
+static int    internet_cat		(CamelAddress *dest, const CamelAddress *source);
 static void   internet_remove		(CamelAddress *, int index);
 
 static void camel_internet_address_class_init (CamelInternetAddressClass *klass);
@@ -44,7 +51,10 @@ camel_internet_address_class_init(CamelInternetAddressClass *klass)
 
 	address->decode = internet_decode;
 	address->encode = internet_encode;
+	address->unformat = internet_unformat;
+	address->format = internet_format;
 	address->remove = internet_remove;
+	address->cat = internet_cat;
 }
 
 static void
@@ -74,7 +84,8 @@ static int
 internet_decode	(CamelAddress *a, const char *raw)
 {
 	struct _header_address *ha, *n;
-	
+	int count = a->addresses->len;
+
 	/* Should probably use its own decoder or something */
 	ha = header_address_decode(raw);
 	if (ha) {
@@ -96,7 +107,7 @@ internet_decode	(CamelAddress *a, const char *raw)
 		header_address_list_clear(&ha);
 	}
 	
-	return 0;
+	return a->addresses->len - count;
 }
 
 static char *
@@ -113,19 +124,104 @@ internet_encode	(CamelAddress *a)
 	
 	for (i = 0;i < a->addresses->len; i++) {
 		struct _address *addr = g_ptr_array_index(a->addresses, i);
-		char *name = header_encode_phrase(addr->name);
-		
+		char *enc;
+
+		if (i != 0)
+			g_string_append(out, ", ");
+
+		enc = camel_internet_address_encode_address(addr->name, addr->address);
+		g_string_sprintfa(out, "%s", enc);
+		g_free(enc);
+	}
+	
+	ret = out->str;
+	g_string_free(out, FALSE);
+	
+	return ret;
+}
+
+static int
+internet_unformat(CamelAddress *a, const char *raw)
+{
+	char *buffer, *p, *name, *addr;
+	int c;
+	int count = a->addresses->len;
+
+	if (raw == NULL)
+		return 0;
+
+	d(printf("unformatting address: %s\n", raw));
+
+	/* we copy, so we can modify as we go */
+	buffer = g_strdup(raw);
+
+	/* this can be simpler than decode, since there are much fewer rules */
+	p = buffer;
+	name = NULL;
+	addr = p;
+	do {
+		c = (unsigned char)*p++;
+		switch (c) {
+			/* HMMM.  Not sure we need this, we dont quote the names anyway ... */
+		case '"':
+			while (*p && *p != '"')
+				p++;
+			break;
+		case '<':
+			if (name == NULL)
+				name = addr;
+			addr = p;
+			addr[-1] = 0;
+			while (*p && *p != '>')
+				p++;
+			if (*p == 0)
+				break;
+			p++;
+			/* falls through */
+		case ',':
+			p[-1] = 0;
+			/* falls through */
+		case 0:
+			if (name)
+				name = g_strstrip(name);
+			addr = g_strstrip(addr);
+			if (addr[0]) {
+				d(printf("found address: %s <%s>\n", name, addr));
+				camel_internet_address_add((CamelInternetAddress *)a, name, addr);
+			}
+			name = NULL;
+			addr = p;
+			break;
+		}
+	} while (c);
+
+	g_free(buffer);
+
+	return a->addresses->len - count;
+}
+
+static char *
+internet_format	(CamelAddress *a)
+{
+	int i;
+	GString *out;
+	char *ret;
+	
+	if (a->addresses->len == 0)
+		return NULL;
+	
+	out = g_string_new("");
+	
+	for (i = 0;i < a->addresses->len; i++) {
+		struct _address *addr = g_ptr_array_index(a->addresses, i);
+		char *enc;
+
 		if (i != 0)
 			g_string_append(out, ", ");
-		
-		if (name) {
-			if (*name) {
-				g_string_sprintfa(out, "%s <%s>", name, addr->address);
-			} else if (addr->address)
-				g_string_sprintfa(out, "%s", addr->address);
-			g_free(name);
-		} else
-			g_string_sprintfa(out, "%s", addr->address);
+
+		enc = camel_internet_address_format_address(addr->name, addr->address);
+		g_string_sprintfa(out, "%s", enc);
+		g_free(enc);
 	}
 	
 	ret = out->str;
@@ -134,6 +230,20 @@ internet_encode	(CamelAddress *a)
 	return ret;
 }
 
+static int    internet_cat		(CamelAddress *dest, const CamelAddress *source)
+{
+	int i;
+
+	g_assert(IS_CAMEL_INTERNET_ADDRESS(source));
+
+	for (i=0;i<source->addresses->len;i++) {
+		struct _address *addr = g_ptr_array_index(source->addresses, i);
+		camel_internet_address_add((CamelInternetAddress *)dest, addr->name, addr->address);
+	}
+
+	return i;
+}
+
 static void
 internet_remove	(CamelAddress *a, int index)
 {
@@ -207,9 +317,8 @@ camel_internet_address_get	(const CamelInternetAddress *a, int index, const char
 	struct _address *addr;
 
 	g_assert(IS_CAMEL_INTERNET_ADDRESS(a));
-	g_return_val_if_fail(index >= 0, -1);
 
-	if (index >= ((CamelAddress *)a)->addresses->len)
+	if (index < 0 || index >= ((CamelAddress *)a)->addresses->len)
 		return FALSE;
 
 	addr = g_ptr_array_index( ((CamelAddress *)a)->addresses, index);
@@ -280,3 +389,55 @@ camel_internet_address_find_address(CamelInternetAddress *a, const char *address
 	}
 	return -1;
 }
+
+/**
+ * camel_internet_address_encode_address:
+ * @name: 
+ * @addr: 
+ * 
+ * Encode a single address ready for internet usage.
+ * 
+ * Return value: The encoded address.
+ **/
+char *
+camel_internet_address_encode_address(const char *real, const char *addr)
+{
+	char *name = header_encode_phrase(real);
+	char *ret = NULL;
+
+	g_assert(addr);
+
+	if (name && name[0])
+		ret = g_strdup_printf("%s <%s>", name, addr);
+	else
+		ret = g_strdup_printf("%s", addr);
+
+	g_free(name);
+
+	return ret;
+}
+
+/**
+ * camel_internet_address_format_address:
+ * @name: 
+ * @addr: 
+ * 
+ * Function to format a single address, suitable for display.
+ * 
+ * Return value: 
+ **/
+char *
+camel_internet_address_format_address(const char *name, const char *addr)
+{
+	char *ret = NULL;
+
+	g_assert(addr);
+
+#warning "If name contains a quote, then we're thrown for six ... "
+	if (name && name[0])
+		ret = g_strdup_printf("%s <%s>", name, addr);
+	else
+		ret = g_strdup_printf("%s", addr);
+
+	return ret;
+}
diff --git a/camel/camel-internet-address.h b/camel/camel-internet-address.h
index 6b303eef8f..63033e17e0 100644
--- a/camel/camel-internet-address.h
+++ b/camel/camel-internet-address.h
@@ -48,4 +48,8 @@ gboolean		camel_internet_address_get	(const CamelInternetAddress *, int, const c
 int			camel_internet_address_find_name(CamelInternetAddress *, const char *, const char **);
 int			camel_internet_address_find_address(CamelInternetAddress *, const char *, const char **);
 
+/* utility functions, for network/display formatting */
+char *			camel_internet_address_encode_address(const char *name, const char *addr);
+char *			camel_internet_address_format_address(const char *real, const char *addr);
+
 #endif /* ! _CAMEL_INTERNET_ADDRESS_H */
diff --git a/camel/camel-mime-message.c b/camel/camel-mime-message.c
index fd199f600d..6423072c91 100644
--- a/camel/camel-mime-message.c
+++ b/camel/camel-mime-message.c
@@ -32,7 +32,14 @@
 #include "camel-stream-mem.h"
 #include "string-utils.h"
 #include "hash-table-utils.h"
+
+#include "camel-stream-filter.h"
+#include "camel-stream-null.h"
+#include "camel-mime-filter-charset.h"
+#include "camel-mime-filter-bestenc.h"
+
 #include <stdio.h>
+#include <string.h>
 
 #define d(x)
 
@@ -116,7 +123,8 @@ camel_mime_message_init (gpointer object, gpointer klass)
 	mime_message->from = NULL;
 	mime_message->date = CAMEL_MESSAGE_DATE_CURRENT;
 	mime_message->date_offset = 0;
-	mime_message->date_str = NULL;
+	mime_message->date_received = CAMEL_MESSAGE_DATE_CURRENT;
+	mime_message->date_received_offset = 0;
 }
 
 static void           
@@ -124,13 +132,16 @@ camel_mime_message_finalize (CamelObject *object)
 {
 	CamelMimeMessage *message = CAMEL_MIME_MESSAGE (object);
 	
-	g_free (message->date_str);
-	g_free (message->subject);
-	g_free (message->reply_to);
-	g_free (message->from);
+	g_free(message->subject);
+
+	if (message->reply_to)
+		camel_object_unref((CamelObject *)message->reply_to);
 
-	g_hash_table_foreach (message->recipients, unref_recipient, NULL);
-	g_hash_table_destroy (message->recipients);
+	if (message->from)
+		camel_object_unref((CamelObject *)message->from);
+
+	g_hash_table_foreach(message->recipients, unref_recipient, NULL);
+	g_hash_table_destroy(message->recipients);
 }
 
 
@@ -157,8 +168,6 @@ static void unref_recipient (gpointer key, gpointer value, gpointer user_data)
 	camel_object_unref (CAMEL_OBJECT (value));
 }
 
-
-
 CamelMimeMessage *
 camel_mime_message_new (void) 
 {
@@ -168,12 +177,13 @@ camel_mime_message_new (void)
 	return mime_message;
 }
 
-
 /* **** Date: */
 
 void
 camel_mime_message_set_date(CamelMimeMessage *message,  time_t date, int offset)
 {
+	char *datestr;
+
 	g_assert(message);
 	if (date == CAMEL_MESSAGE_DATE_CURRENT) {
 		struct tm *local;
@@ -191,233 +201,176 @@ camel_mime_message_set_date(CamelMimeMessage *message,  time_t date, int offset)
 	}
 	message->date = date;
 	message->date_offset = offset;
-	g_free(message->date_str);
-	message->date_str = header_format_date(date, offset);
 
-	CAMEL_MEDIUM_CLASS(parent_class)->set_header((CamelMedium *)message, "Date", message->date_str);
+	datestr = header_format_date(date, offset);
+	CAMEL_MEDIUM_CLASS(parent_class)->set_header((CamelMedium *)message, "Date", datestr);
+	g_free(datestr);
 }
 
-void
-camel_mime_message_get_date(CamelMimeMessage *message,  time_t *date, int *offset)
+time_t
+camel_mime_message_get_date(CamelMimeMessage *msg, int *offset)
 {
-	if (message->date == CAMEL_MESSAGE_DATE_CURRENT)
-		camel_mime_message_set_date(message, CAMEL_MESSAGE_DATE_CURRENT, 0);
-	if (date)
-		*date = message->date;
 	if (offset)
-		*offset = message->date_offset;
-}
+		*offset = msg->date_offset;
 
-char *
-camel_mime_message_get_date_string (CamelMimeMessage *message)
-{
-	if (message->date == CAMEL_MESSAGE_DATE_CURRENT)
-		camel_mime_message_set_date (message, CAMEL_MESSAGE_DATE_CURRENT, 0);
-	return message->date_str;
+	return msg->date;
 }
 
-const gchar *
-camel_mime_message_get_received_date (CamelMimeMessage *message)
+time_t
+camel_mime_message_get_date_received(CamelMimeMessage *msg, int *offset)
 {
-	/* FIXME: is this the received date? and if so then get_sent_date must be wrong */
-	if (message->date == CAMEL_MESSAGE_DATE_CURRENT)
-		camel_mime_message_set_date (message, CAMEL_MESSAGE_DATE_CURRENT, 0);
-	return message->date_str;
-}
+	if (msg->date_received == CAMEL_MESSAGE_DATE_CURRENT) {
+		const char *received;
 
-const gchar *
-camel_mime_message_get_sent_date (CamelMimeMessage *message)
-{
-	/* FIXME: is this the sent date? and if so then get_received_date must be wrong */
-	if (message->date == CAMEL_MESSAGE_DATE_CURRENT)
-		camel_mime_message_set_date (message, CAMEL_MESSAGE_DATE_CURRENT, 0);
-	return message->date_str;
+		received = camel_medium_get_header((CamelMedium *)msg, "received");
+		if (received)
+			received = strrchr(received, ';');
+		if (received)
+			msg->date_received = header_decode_date(received + 1, &msg->date_received_offset);
+	}
+
+	if (offset)
+		*offset = msg->date_received_offset;
+
+	return msg->date_received;
 }
 
 /* **** Reply-To: */
 
 void
-camel_mime_message_set_reply_to (CamelMimeMessage *mime_message, const gchar *reply_to)
+camel_mime_message_set_reply_to (CamelMimeMessage *msg, const CamelInternetAddress *reply_to)
 {
-	CamelInternetAddress *cia;
 	char *addr;
 	
-	g_assert (mime_message);
-	
-	/* FIXME: check format of string, handle it nicer ... */
-	
-	g_free (mime_message->reply_to);
-	mime_message->reply_to = g_strstrip (g_strdup (reply_to));
-	
-	cia = camel_internet_address_new ();
-	camel_address_decode (CAMEL_ADDRESS (cia), mime_message->reply_to);
-	addr = camel_address_encode (CAMEL_ADDRESS (cia));
-	
-	CAMEL_MEDIUM_CLASS (parent_class)->set_header (CAMEL_MEDIUM (mime_message), "Reply-To", addr);
-	camel_object_unref (CAMEL_OBJECT (cia));
-	g_free (addr);
+	g_assert(msg);
+
+	if (msg->reply_to) {
+		camel_object_unref((CamelObject *)msg->reply_to);
+		msg->reply_to = NULL;
+	}
+
+	if (reply_to == NULL) {
+		CAMEL_MEDIUM_CLASS(parent_class)->remove_header(CAMEL_MEDIUM(msg), "Reply-To");
+		return;
+	}
+
+	msg->reply_to = (CamelInternetAddress *)camel_address_new_clone((CamelAddress *)reply_to);
+	addr = camel_address_encode((CamelAddress *)msg->reply_to);
+	CAMEL_MEDIUM_CLASS(parent_class)->set_header(CAMEL_MEDIUM(msg), "Reply-To", addr);
+	g_free(addr);
 }
 
-const gchar *
-camel_mime_message_get_reply_to (CamelMimeMessage *mime_message)
+const CamelInternetAddress *
+camel_mime_message_get_reply_to(CamelMimeMessage *mime_message)
 {
 	g_assert (mime_message);
 
+	/* TODO: ref for threading? */
+
 	return mime_message->reply_to;
 }
 
+/* **** Subject: */
+
 void
-camel_mime_message_set_subject (CamelMimeMessage *mime_message,
-				const gchar *subject)
+camel_mime_message_set_subject(CamelMimeMessage *mime_message, const char *subject)
 {
 	char *text;
 	
-	g_assert (mime_message);
+	g_assert(mime_message);
 	
-	g_free (mime_message->subject);
+	g_free(mime_message->subject);
 	mime_message->subject = g_strstrip (g_strdup (subject));
-	text = header_encode_string ((unsigned char *) mime_message->subject);
-	CAMEL_MEDIUM_CLASS (parent_class)->set_header (CAMEL_MEDIUM (mime_message), "Subject", text);
+	text = header_encode_string((unsigned char *)mime_message->subject);
+	CAMEL_MEDIUM_CLASS(parent_class)->set_header(CAMEL_MEDIUM (mime_message), "Subject", text);
 	g_free (text);
 }
 
-const gchar *
-camel_mime_message_get_subject (CamelMimeMessage *mime_message)
+const char *
+camel_mime_message_get_subject(CamelMimeMessage *mime_message)
 {
-	g_assert (mime_message);
+	g_assert(mime_message);
 
 	return mime_message->subject;
 }
 
 /* *** From: */
+
+/* Thought: Since get_from/set_from are so rarely called, it is probably not useful
+   to cache the from (and reply_to) addresses as InternetAddresses internally, we
+   could just get it from the headers and reprocess every time. */
 void
-camel_mime_message_set_from (CamelMimeMessage *mime_message, const gchar *from)
+camel_mime_message_set_from(CamelMimeMessage *msg, const CamelInternetAddress *from)
 {
-	CamelInternetAddress *cia;
 	char *addr;
 	
-	g_assert (mime_message);
-	
-	/* FIXME: check format of string, handle it nicer ... */
-	
-	g_free (mime_message->from);
-	mime_message->from = g_strstrip (g_strdup (from));
-	
-	cia = camel_internet_address_new ();
-	camel_address_decode (CAMEL_ADDRESS (cia), mime_message->from);
-	
-	addr = camel_address_encode (CAMEL_ADDRESS (cia));
-	CAMEL_MEDIUM_CLASS (parent_class)->set_header (CAMEL_MEDIUM (mime_message), "From", addr);
-	camel_object_unref (CAMEL_OBJECT (cia));
-	g_free (addr);
-}
-
-const gchar *
-camel_mime_message_get_from (CamelMimeMessage *mime_message)
-{
-	g_assert (mime_message);
-
-	return mime_message->from;
-}
-
-/*  ****  */
+	g_assert(msg);
 
-void
-camel_mime_message_add_recipient (CamelMimeMessage *mime_message, 
-				  const gchar *type, 
-				  const gchar *name, const char *address)
-{
-	CamelInternetAddress *addr;
-	char *text;
-
-	g_assert (mime_message);
+	if (msg->from) {
+		camel_object_unref((CamelObject *)msg->from);
+		msg->from = NULL;
+	}
 
-	addr = g_hash_table_lookup (mime_message->recipients, type);
-	if (addr == NULL) {
-		g_warning ("trying to add a non-valid receipient type: %s = %s %s", type, name, address);
+	if (from == NULL || camel_address_length((CamelAddress *)from) == 0) {
+		CAMEL_MEDIUM_CLASS(parent_class)->remove_header(CAMEL_MEDIUM(msg), "From");
 		return;
 	}
 
-	camel_internet_address_add (addr, name, address);
-
-	/* FIXME: maybe this should be delayed till we're ready to write out? */
-	text = camel_address_encode (CAMEL_ADDRESS (addr));
-	CAMEL_MEDIUM_CLASS (parent_class)->set_header (CAMEL_MEDIUM (mime_message), type, text);
-	g_free (text);
+	msg->from = (CamelInternetAddress *)camel_address_new_clone((CamelAddress *)from);
+	addr = camel_address_encode((CamelAddress *)msg->from);
+	CAMEL_MEDIUM_CLASS (parent_class)->set_header(CAMEL_MEDIUM(msg), "From", addr);
+	g_free(addr);
 }
 
-void
-camel_mime_message_remove_recipient_address (CamelMimeMessage *mime_message,
-					     const gchar *type,
-					     const gchar *address)
+const CamelInternetAddress *
+camel_mime_message_get_from(CamelMimeMessage *mime_message)
 {
-	CamelInternetAddress *addr;
-	int index;
-	char *text;
-
 	g_assert (mime_message);
 
-	addr = g_hash_table_lookup(mime_message->recipients, type);
-	if (addr == NULL) {
-		g_warning("trying to remove a non-valid receipient type: %s = %s", type, address);
-		return;
-	}
-	
-	index = camel_internet_address_find_address(addr, address, NULL);
-	if (index == -1) {
-		g_warning("trying to remove address for nonexistand address: %s", address);
-		return;
-	}
-
-	camel_address_remove (CAMEL_ADDRESS (addr), index);
+	/* TODO: we should really ref this for multi-threading to work */
 
-	/* FIXME: maybe this should be delayed till we're ready to write out? */
-	text = camel_address_encode (CAMEL_ADDRESS (addr));
-	CAMEL_MEDIUM_CLASS (parent_class)->set_header (CAMEL_MEDIUM (mime_message), type, text);
-	g_free (text);
+	return mime_message->from;
 }
 
+/*  **** To: Cc: Bcc: */
+
 void
-camel_mime_message_remove_recipient_name (CamelMimeMessage *mime_message,
-					  const gchar *type,
-					  const gchar *name)
+camel_mime_message_set_recipients(CamelMimeMessage *mime_message, const char *type, const CamelInternetAddress *r)
 {
-	CamelInternetAddress *addr;
-	int index;
 	char *text;
+	CamelInternetAddress *addr;
 
-	g_assert (mime_message);
+	g_assert(mime_message);
 
 	addr = g_hash_table_lookup(mime_message->recipients, type);
 	if (addr == NULL) {
-		g_warning("trying to remove a non-valid receipient type: %s = %s", type, name);
+		g_warning("trying to set a non-valid receipient type: %s", type);
 		return;
 	}
-	index = camel_internet_address_find_name(addr, name, NULL);
-	if (index == -1) {
-		g_warning("trying to remove address for nonexistand name: %s", name);
+
+	if (r == NULL || camel_address_length((CamelAddress *)r) == 0) {
+		camel_address_remove((CamelAddress *)addr, -1);
+		CAMEL_MEDIUM_CLASS(parent_class)->remove_header(CAMEL_MEDIUM(mime_message), type);
 		return;
 	}
 
-	camel_address_remove (CAMEL_ADDRESS (addr), index);
+	/* note this does copy, and not append (cat) */
+	camel_address_copy((CamelAddress *)addr, (const CamelAddress *)r);
 
-	/* FIXME: maybe this should be delayed till we're ready to write out? */
-	text = camel_address_encode (CAMEL_ADDRESS (addr));
-	CAMEL_MEDIUM_CLASS (parent_class)->set_header (CAMEL_MEDIUM (mime_message), type, text);
-	g_free (text);
+	/* and sync our headers */
+	text = camel_address_encode(CAMEL_ADDRESS(addr));
+	CAMEL_MEDIUM_CLASS(parent_class)->set_header(CAMEL_MEDIUM(mime_message), type, text);
+	g_free(text);
 }
 
 const CamelInternetAddress *
-camel_mime_message_get_recipients (CamelMimeMessage *mime_message, 
-				   const gchar *type)
+camel_mime_message_get_recipients(CamelMimeMessage *mime_message, const char *type)
 {
-	g_assert (mime_message);
+	g_assert(mime_message);
 	
 	return g_hash_table_lookup(mime_message->recipients, type);
 }
 
-
-
 /* mime_message */
 static int
 construct_from_parser(CamelMimePart *dw, CamelMimeParser *mp)
@@ -464,10 +417,11 @@ write_to_stream (CamelDataWrapper *data_wrapper, CamelStream *stream)
 
 	/* force mandatory headers ... */
 	if (mm->from == NULL) {
+		/* FIXME: should we just abort?  Should we make one up? */
 		g_warning("No from set for message");
-		camel_mime_message_set_from(mm, "");
+		camel_medium_set_header((CamelMedium *)mm, "From", "");
 	}
-	if (mm->date_str == NULL) {
+	if (!camel_medium_get_header((CamelMedium *)mm, "Date")) {
 		g_warning("Application did not set date, using 'now'");
 		camel_mime_message_set_date(mm, CAMEL_MESSAGE_DATE_CURRENT, 0);
 	}
@@ -484,23 +438,6 @@ write_to_stream (CamelDataWrapper *data_wrapper, CamelStream *stream)
 	return CAMEL_DATA_WRAPPER_CLASS (parent_class)->write_to_stream (data_wrapper, stream);
 }
 
-static char *
-format_address(const char *text)
-{
-	struct _header_address *addr;
-	char *ret;
-	
-	addr = header_address_decode (text);
-	if (addr) {
-		ret = header_address_list_format (addr);
-		header_address_list_clear (&addr);
-	} else {
-		ret = g_strdup (text);
-	}
-	
-	return ret;
-}
-
 /* FIXME: check format of fields. */
 static gboolean
 process_header (CamelMedium *medium, const char *header_name, const char *header_value)
@@ -509,36 +446,39 @@ process_header (CamelMedium *medium, const char *header_name, const char *header
 	CamelMimeMessage *message = CAMEL_MIME_MESSAGE (medium);
 	CamelInternetAddress *addr;
 
-	header_type = (CamelHeaderType) g_hash_table_lookup (header_name_table, header_name);
+	header_type = (CamelHeaderType)g_hash_table_lookup(header_name_table, header_name);
 	switch (header_type) {
 	case HEADER_FROM:
-		g_free (message->from);
-		message->from = format_address (header_value);
+		if (message->from)
+			camel_object_unref((CamelObject *)message->from);
+		message->from = camel_internet_address_new();
+		camel_address_decode((CamelAddress *)message->from, header_value);
 		break;
 	case HEADER_REPLY_TO:
-		g_free (message->reply_to);
-		message->reply_to = format_address (header_value);
+		if (message->reply_to)
+			camel_object_unref((CamelObject *)message->reply_to);
+		message->reply_to = camel_internet_address_new();
+		camel_address_decode((CamelAddress *)message->reply_to, header_value);
 		break;
 	case HEADER_SUBJECT:
-		g_free (message->subject);
-		message->subject = g_strstrip (header_decode_string (header_value));
+		g_free(message->subject);
+		message->subject = g_strstrip(header_decode_string(header_value));
 		break;
 	case HEADER_TO:
 	case HEADER_CC:
 	case HEADER_BCC:
 		addr = g_hash_table_lookup (message->recipients, header_name);
 		if (header_value)
-			camel_address_decode (CAMEL_ADDRESS (addr), header_value);
+			camel_address_decode(CAMEL_ADDRESS (addr), header_value);
 		else
-			camel_address_remove (CAMEL_ADDRESS (addr), -1);
+			camel_address_remove(CAMEL_ADDRESS (addr), -1);
 		break;
 	case HEADER_DATE:
-		g_free (message->date_str);
-		message->date_str = g_strdup (header_value);
 		if (header_value) {
-			message->date = header_decode_date (header_value, &message->date_offset);
+			message->date = header_decode_date(header_value, &message->date_offset);
 		} else {
 			message->date = CAMEL_MESSAGE_DATE_CURRENT;
+			message->date_offset = 0;
 		}
 		break;
 	default:
@@ -571,168 +511,243 @@ remove_header(CamelMedium *medium, const char *header_name)
 	parent_class->parent_class.remove_header (medium, header_name);
 }
 
+typedef gboolean (*CamelPartFunc)(CamelMimeMessage *, CamelMimePart *, void *data);
+
 static gboolean
-multipart_has_8bit_parts (CamelMultipart *multipart)
+message_foreach_part_rec(CamelMimeMessage *msg, CamelMimePart *part, CamelPartFunc callback, void *data)
 {
-	gboolean has_8bit = FALSE;
-	int i, nparts;
-	
-	nparts = camel_multipart_get_number (multipart);
-	
-	for (i = 0; i < nparts && !has_8bit; i++) {
-		GMimeContentField *content;
-		CamelMimePart *mime_part;
-		
-		mime_part = camel_multipart_get_part (multipart, i);
-		content = camel_mime_part_get_content_type (mime_part);
-		
-		if (gmime_content_field_is_type (content, "multipart", "*")) {
-			CamelDataWrapper *wrapper;
-			CamelMultipart *mpart;
-			
-			wrapper = camel_medium_get_content_object (CAMEL_MEDIUM (mime_part));
-			mpart = CAMEL_MULTIPART (wrapper);
-			
-			has_8bit = multipart_has_8bit_parts (mpart);
-		} else {
-			/* see if this part is 8bit */
-			has_8bit = camel_mime_part_get_encoding (mime_part) == CAMEL_MIME_PART_ENCODING_8BIT;
+	CamelDataWrapper *containee;
+	int parts, i;
+	int go = TRUE;
+
+	if (callback(msg, part, data) == FALSE)
+		return FALSE;
+
+	containee = camel_medium_get_content_object(CAMEL_MEDIUM(part));
+
+	if (containee == NULL)
+		return go;
+
+	/* using the object types is more accurate than using the mime/types */
+	if (CAMEL_IS_MULTIPART(containee)) {
+		parts = camel_multipart_get_number(CAMEL_MULTIPART(containee));
+		for (i=0;go && i<parts;i++) {
+			CamelMimePart *part = camel_multipart_get_part(CAMEL_MULTIPART(containee), i);
+
+			go = message_foreach_part_rec(msg, part, callback, data);
 		}
+	} else if (CAMEL_IS_MIME_MESSAGE(containee)) {
+		go = message_foreach_part_rec(msg, (CamelMimePart *)containee, callback, data);
 	}
-	
-	return has_8bit;
+
+	return go;
+}
+
+/* dont make this public yet, it might need some more thinking ... */
+/* MPZ */
+static void
+camel_mime_message_foreach_part(CamelMimeMessage *msg, CamelPartFunc callback, void *data)
+{
+	message_foreach_part_rec(msg, (CamelMimePart *)msg, callback, data);
+}
+
+static gboolean
+check_8bit(CamelMimeMessage *msg, CamelMimePart *part, void *data)
+{
+	int *has8bit = data;
+
+	/* check this part, and stop as soon as we are done */
+	*has8bit = camel_mime_part_get_encoding(part) == CAMEL_MIME_PART_ENCODING_8BIT;
+	return !(*has8bit);
 }
 
 gboolean
-camel_mime_message_has_8bit_parts (CamelMimeMessage *mime_message)
+camel_mime_message_has_8bit_parts(CamelMimeMessage *msg)
 {
-	GMimeContentField *content;
-	gboolean has_8bit = FALSE;
-	
-	content = camel_mime_part_get_content_type (CAMEL_MIME_PART (mime_message));
-	
-	if (gmime_content_field_is_type (content, "multipart", "*")) {
-		CamelDataWrapper *wrapper;
-		CamelMultipart *multipart;
-		
-		wrapper = camel_medium_get_content_object (CAMEL_MEDIUM (CAMEL_MIME_PART (mime_message)));
-		multipart = CAMEL_MULTIPART (wrapper);
-		
-		has_8bit = multipart_has_8bit_parts (multipart);
-	} else {
-		/* single-part message so just check this part */
-		has_8bit = camel_mime_part_get_encoding (CAMEL_MIME_PART (mime_message)) == CAMEL_MIME_PART_ENCODING_8BIT;
-	}
-	
-	return has_8bit;
+	int has8bit = FALSE;
+
+	camel_mime_message_foreach_part(msg, check_8bit, &has8bit);
+
+	return has8bit;
 }
 
-static int
-best_encoding (const guchar *text)
+/* finds the best charset and transfer encoding for a given part */
+static CamelMimePartEncodingType
+find_best_encoding(CamelMimePart *part, CamelBestencRequired required, CamelBestencEncoding enctype, char **charsetp)
 {
-	guchar *ch;
-	int count = 0;
-	int total;
-	
-	for (ch = (guchar *) text; *ch; ch++)
-		if (*ch > (guchar) 127)
-			count++;
-	
-	total = (int) (ch - text);
+	const char *charsetin = NULL;
+	char *charset = NULL;
+	CamelStream *null;
+	CamelStreamFilter *filter;
+	CamelMimeFilterCharset *charenc = NULL;
+	CamelMimeFilterBestenc *bestenc;
+	int idb, idc = -1;
+	gboolean istext;
+	unsigned int flags;
+	CamelMimePartEncodingType encoding;
+	CamelDataWrapper *content;
+
+	/* we use all these weird stream things so we can do it with streams, and
+	   not have to read the whole lot into memory - although i have a feeling
+	   it would make things a fair bit simpler to do so ... */
+
+	printf("starting to check part\n");
+
+	content = camel_medium_get_content_object((CamelMedium *)part);
+	if (content == NULL) {
+		/* charset might not be right here, but it'll get the right stuff
+		   if it is ever set */
+		*charsetp = NULL;
+		return CAMEL_MIME_PART_ENCODING_DEFAULT;
+	}
+
+	istext = gmime_content_field_is_type(part->content_type, "text", "*");
+	if (istext) {
+		flags = CAMEL_BESTENC_GET_CHARSET|CAMEL_BESTENC_GET_ENCODING;
+	} else {
+		flags = CAMEL_BESTENC_GET_ENCODING;
+	}
+
+	/* when building the message, any encoded parts are translated already */
+	flags |= CAMEL_BESTENC_LF_IS_CRLF;
+
+	/* first a null stream, so any filtering is thrown away; we only want the sideeffects */
+	null = (CamelStream *)camel_stream_null_new();
+	filter = camel_stream_filter_new_with_stream(null);
+
+	/* if we're not looking for the best charset, then use the one we have */
+	if (istext && (required & CAMEL_BESTENC_GET_CHARSET) == 0
+	    && (charsetin = gmime_content_field_get_parameter(part->content_type, "charset"))) {
+		/* if libunicode doesn't support it, we dont really have utf8 anyway, so
+		   we dont need a converter */
+		charenc = camel_mime_filter_charset_new_convert("UTF-8", charsetin);
+		if (charenc != NULL)
+			idc = camel_stream_filter_add(filter, (CamelMimeFilter *)charenc);
+		charsetin = NULL;
+	}
+
+	bestenc = camel_mime_filter_bestenc_new(flags);
+	idb = camel_stream_filter_add(filter, (CamelMimeFilter *)bestenc);
+	printf("writing to checking stream\n");
+	camel_data_wrapper_write_to_stream(content, (CamelStream *)filter);
+	camel_stream_filter_remove(filter, idb);
+	if (idc != -1) {
+		camel_stream_filter_remove(filter, idc);
+		camel_object_unref((CamelObject *)charenc);
+		charenc = NULL;
+	}
+
+	if (istext)
+		charsetin = camel_mime_filter_bestenc_get_best_charset(bestenc);
+
+	printf("charsetin = %s\n", charsetin);
+
+	/* if we have US-ASCII, or we're not doing text, we dont need to bother with the rest */
+	if (charsetin != NULL && (required & CAMEL_BESTENC_GET_CHARSET) != 0) {
+		charset = g_strdup(charsetin);
+
+		printf("have charset, trying conversion/etc\n");
+
+		/* now the 'bestenc' can has told us what the best encoding is, we can use that to create
+		   a charset conversion filter as well, and then re-add the bestenc to filter the
+		   result to find the best encoding to use as well */
+		
+		charenc = camel_mime_filter_charset_new_convert("UTF-8", charset);
+
+		/* eek, libunicode doesn't undertand this charset anyway, then the 'utf8' we
+		   thought we had is really the native format, in which case, we just treat
+		   it as binary data (and take the result we have so far) */
+		   
+		if (charenc != NULL) {
+
+			/* otherwise, try another pass, converting to the real charset */
+
+			camel_mime_filter_reset((CamelMimeFilter *)bestenc);
+			camel_mime_filter_bestenc_set_flags(bestenc, CAMEL_BESTENC_GET_ENCODING|CAMEL_BESTENC_LF_IS_CRLF);
+
+			camel_stream_filter_add(filter, (CamelMimeFilter *)charenc);
+			camel_stream_filter_add(filter, (CamelMimeFilter *)bestenc);
+
+			/* and write it to the new stream */
+			camel_data_wrapper_write_to_stream(content, (CamelStream *)filter);
+
+			camel_object_unref((CamelObject *)charenc);
+		}
+	}
 	
-	if ((float) count <= total * 0.17)
-		return CAMEL_MIME_PART_ENCODING_QUOTEDPRINTABLE;
+	encoding = camel_mime_filter_bestenc_get_best_encoding(bestenc, enctype);
+
+	camel_object_unref((CamelObject *)filter);
+	camel_object_unref((CamelObject *)bestenc);
+	camel_object_unref((CamelObject *)null);
+
+	printf("done, best encoding = %d\n", encoding);
+
+	if (charsetp)
+		*charsetp = charset;
 	else
-		return CAMEL_MIME_PART_ENCODING_BASE64;
+		g_free(charset);
+
+	return encoding;
 }
 
-static void
-multipart_encode_8bit_parts (CamelMultipart *multipart)
+struct _enc_data {
+	CamelBestencRequired required;
+	CamelBestencEncoding enctype;
+};
+
+static gboolean
+best_encoding(CamelMimeMessage *msg, CamelMimePart *part, void *datap)
 {
-	int i, nparts;
-	
-	nparts = camel_multipart_get_number (multipart);
-	
-	for (i = 0; i < nparts; i++) {
-		GMimeContentField *content;
-		CamelMimePart *mime_part;
-		
-		mime_part = camel_multipart_get_part (multipart, i);
-		content = camel_mime_part_get_content_type (mime_part);
-		
-		if (gmime_content_field_is_type (content, "multipart", "*")) {
-			/* ...and the search for Spock continues */
-			CamelDataWrapper *wrapper;
-			CamelMultipart *mpart;
-			
-			wrapper = camel_medium_get_content_object (CAMEL_MEDIUM (mime_part));
-			mpart = CAMEL_MULTIPART (wrapper);
-			
-			multipart_encode_8bit_parts (mpart);
-		} else {
-			/* re-encode this if necessary */
-			gboolean is_8bit;
-			
-			is_8bit = camel_mime_part_get_encoding (mime_part) == CAMEL_MIME_PART_ENCODING_8BIT;
-			if (is_8bit) {
-				CamelStream *stream;
-				GByteArray *array;
-				guchar *content;
-				
-				array = g_byte_array_new ();
-				stream = camel_stream_mem_new_with_byte_array (array);
-				camel_data_wrapper_write_to_stream (CAMEL_DATA_WRAPPER (mime_part), stream);
-				g_byte_array_append (array, "", 1);
-				
-				content = array->data;
-				g_byte_array_free (array, FALSE);
-				
-				camel_mime_part_set_encoding (mime_part, best_encoding (content));
-				g_free (content);
-				camel_object_unref (CAMEL_OBJECT (stream));
+	struct _enc_data *data = datap;
+	char *charset;
+	CamelMimePartEncodingType encoding;
+
+	/* we only care about actual content objects */
+	if (!CAMEL_IS_MULTIPART(part) && !CAMEL_IS_MIME_MESSAGE(part)) {
+
+		encoding = find_best_encoding(part, data->required, data->enctype, &charset);
+		/* we always set the encoding, if we got this far.  GET_CHARSET implies
+		   also GET_ENCODING */
+		camel_mime_part_set_encoding(part, encoding);
+
+		if ((data->required & CAMEL_BESTENC_GET_CHARSET) != 0) {
+			if (gmime_content_field_is_type(part->content_type, "text", "*")) {
+				char *newct;
+
+				/* FIXME: ick, the part content_type interface needs fixing bigtime */
+				gmime_content_field_set_parameter(part->content_type, "charset", charset?charset:"us-ascii");
+				newct = header_content_type_format(part->content_type->content_type);
+				if (newct) {
+					printf("Setting content-type to %s\n", newct);
+
+					camel_mime_part_set_content_type(part, newct);
+					g_free(newct);
+				}
 			}
 		}
 	}
+
+	return TRUE;
+}
+
+void
+camel_mime_message_set_best_encoding(CamelMimeMessage *msg, CamelBestencRequired required, CamelBestencEncoding enctype)
+{
+	struct _enc_data data;
+
+	if ((required & (CAMEL_BESTENC_GET_ENCODING|CAMEL_BESTENC_GET_CHARSET)) == 0)
+		return;
+
+	data.required = required;
+	data.enctype = enctype;
+
+	camel_mime_message_foreach_part(msg, best_encoding, &data);
 }
 
 void
 camel_mime_message_encode_8bit_parts (CamelMimeMessage *mime_message)
 {
-	GMimeContentField *content;
-	
-	content = camel_mime_part_get_content_type (CAMEL_MIME_PART (mime_message));
-	
-	if (gmime_content_field_is_type (content, "multipart", "*")) {
-		/* search for Spock */
-		CamelDataWrapper *wrapper;
-		CamelMultipart *multipart;
-		
-		wrapper = camel_medium_get_content_object (CAMEL_MEDIUM (CAMEL_MIME_PART (mime_message)));
-		multipart = CAMEL_MULTIPART (wrapper);
-		
-		multipart_encode_8bit_parts (multipart);
-	} else {
-		/* re-encode if we need to */
-		gboolean is_8bit;
-		
-		is_8bit = camel_mime_part_get_encoding (CAMEL_MIME_PART (mime_message)) == CAMEL_MIME_PART_ENCODING_8BIT;
-		if (is_8bit) {
-			/* FIXME: is there a better way of doing this? */
-			CamelStream *stream;
-			GByteArray *array;
-			guchar *content;
-			
-			array = g_byte_array_new ();
-			stream = camel_stream_mem_new_with_byte_array (array);
-			camel_data_wrapper_write_to_stream (CAMEL_DATA_WRAPPER (CAMEL_MIME_PART (mime_message)), stream);
-			g_byte_array_append (array, "", 1);
-			
-			content = array->data;
-			g_byte_array_free (array, FALSE);
-						
-			camel_mime_part_set_encoding (CAMEL_MIME_PART (mime_message), best_encoding (content));
-			g_free (content);
-			camel_object_unref (CAMEL_OBJECT (stream));
-		}
-	}
+	camel_mime_message_set_best_encoding(mime_message, CAMEL_BESTENC_GET_ENCODING, CAMEL_BESTENC_7BIT);
 }
+
diff --git a/camel/camel-mime-message.h b/camel/camel-mime-message.h
index 33436c6dad..4cf7e060d0 100644
--- a/camel/camel-mime-message.h
+++ b/camel/camel-mime-message.h
@@ -57,12 +57,15 @@ struct _CamelMimeMessage
 	/* header fields */
 	time_t date;
 	int date_offset;	/* GMT offset */
-	char *date_str;		/* cached copy of date string */
 
-	gchar *subject;
-	gchar *reply_to;
+	/* cached internal copy */
+	time_t date_received;
+	int date_received_offset;	/* GMT offset */
 
-	gchar *from;
+	char *subject;
+
+	CamelInternetAddress *reply_to;
+	CamelInternetAddress *from;
 
 	GHashTable *recipients;	/* hash table of CamelInternetAddress's */
 };
@@ -84,33 +87,21 @@ CamelType camel_mime_message_get_type (void);
 CamelMimeMessage * camel_mime_message_new                  (void);
 
 
-void               camel_mime_message_set_date		   (CamelMimeMessage *mime_message,  time_t date, int offset);
-void               camel_mime_message_get_date		   (CamelMimeMessage *mime_message,  time_t *date, int *offset);
-char		  *camel_mime_message_get_date_string	   (CamelMimeMessage *mime_message);
-
-const gchar *      camel_mime_message_get_received_date    (CamelMimeMessage *mime_message);
-const gchar *      camel_mime_message_get_sent_date        (CamelMimeMessage *mime_message);
-void               camel_mime_message_set_reply_to         (CamelMimeMessage *mime_message, 
-							    const gchar *reply_to);
-const gchar *      camel_mime_message_get_reply_to         (CamelMimeMessage *mime_message);
-void               camel_mime_message_set_subject          (CamelMimeMessage *mime_message, 
-							    const gchar *subject);
-const gchar *      camel_mime_message_get_subject          (CamelMimeMessage *mime_message);
-void               camel_mime_message_set_from             (CamelMimeMessage *mime_message, 
-							    const gchar *from);
-const gchar *      camel_mime_message_get_from             (CamelMimeMessage *mime_message);
-
+void            camel_mime_message_set_date		(CamelMimeMessage *mime_message, time_t date, int offset);
 
-void		camel_mime_message_add_recipient (CamelMimeMessage *mime_message,
-						  const char *type, const char *name, const char *address);
-void		camel_mime_message_remove_recipient_address (CamelMimeMessage *mime_message, 
-							     const char *type, const char *address);
-void		camel_mime_message_remove_recipient_name (CamelMimeMessage *mime_message, 
-							  const char *type, const char *name);
+time_t          camel_mime_message_get_date		(CamelMimeMessage *mime_message, int *offset);
+time_t          camel_mime_message_get_date_received	(CamelMimeMessage *mime_message, int *offset);
 
-const CamelInternetAddress *camel_mime_message_get_recipients (CamelMimeMessage *mime_message, 
-							       const char *type);
+void            camel_mime_message_set_reply_to         (CamelMimeMessage *mime_message, const CamelInternetAddress *reply_to);
+const CamelInternetAddress *camel_mime_message_get_reply_to         (CamelMimeMessage *mime_message);
+void        	camel_mime_message_set_subject          (CamelMimeMessage *mime_message, 
+							 const char *subject);
+const char *	camel_mime_message_get_subject          (CamelMimeMessage *mime_message);
+void        	camel_mime_message_set_from             (CamelMimeMessage *mime_message, const CamelInternetAddress *from);
+const CamelInternetAddress *camel_mime_message_get_from             (CamelMimeMessage *mime_message);
 
+const CamelInternetAddress *camel_mime_message_get_recipients (CamelMimeMessage *mime_message, const char *type);
+void			    camel_mime_message_set_recipients (CamelMimeMessage *mime_message, const char *type, const CamelInternetAddress *r);
 
 /* utility functions */
 gboolean camel_mime_message_has_8bit_parts    (CamelMimeMessage *mime_message);
diff --git a/camel/camel-mime-part.c b/camel/camel-mime-part.c
index e661e0caf5..61c0fb7f1f 100644
--- a/camel/camel-mime-part.c
+++ b/camel/camel-mime-part.c
@@ -653,45 +653,38 @@ construct_from_stream(CamelDataWrapper *dw, CamelStream *s)
 	return ret;
 }
 
-
-const gchar *
+/* this must be kept in sync with the header */
+static const char *encodings[] = {
+	"",
+	"7bit",
+	"8bit",
+	"base64",
+	"quoted-printable",
+	"binary"
+};
+
+const char *
 camel_mime_part_encoding_to_string (CamelMimePartEncodingType encoding)
 {
-	switch (encoding) {
-	case CAMEL_MIME_PART_ENCODING_DEFAULT:
-	case CAMEL_MIME_PART_ENCODING_7BIT:
-		return "7bit";
-	case CAMEL_MIME_PART_ENCODING_8BIT:
-		return "8bit";
-	case CAMEL_MIME_PART_ENCODING_BASE64:
-		return "base64";
-	case CAMEL_MIME_PART_ENCODING_QUOTEDPRINTABLE:
-		return "quoted-printable";
-	default:
-		break;
-	}
-	return "";
-}
-
+	if (encoding >= sizeof(encodings)/sizeof(encodings[0]))
+		encoding = 0;
 
+	return encodings[encoding];
+}
 
 /* FIXME I am not sure this is the correct way to do this.  */
 CamelMimePartEncodingType
 camel_mime_part_encoding_from_string (const gchar *string)
 {
-	if (string == NULL)
-		return CAMEL_MIME_PART_ENCODING_DEFAULT;
-	else if (strcasecmp (string, "7bit") == 0)
-		return CAMEL_MIME_PART_ENCODING_7BIT;
-	else if (strcasecmp (string, "8bit") == 0)
-		return CAMEL_MIME_PART_ENCODING_8BIT;
-	else if (strcasecmp (string, "base64") == 0)
-		return CAMEL_MIME_PART_ENCODING_BASE64;
-	else if (strcasecmp (string, "quoted-printable") == 0)
-		return CAMEL_MIME_PART_ENCODING_QUOTEDPRINTABLE;
-	else
-		/* FIXME?  Spit a warning?  */
-		return CAMEL_MIME_PART_ENCODING_DEFAULT;
+	int i;
+
+	if (string != NULL) {
+		for (i=0;i<sizeof(encodings)/sizeof(encodings[0]);i++)
+			if (!strcasecmp(string, encodings[i]))
+				return i;
+	}
+
+	return CAMEL_MIME_PART_ENCODING_DEFAULT;
 }
 
 
diff --git a/camel/camel-mime-part.h b/camel/camel-mime-part.h
index ed0df94a4b..4d8140c259 100644
--- a/camel/camel-mime-part.h
+++ b/camel/camel-mime-part.h
@@ -43,13 +43,14 @@ extern "C" {
 #define CAMEL_MIME_PART_CLASS(k) (CAMEL_CHECK_CLASS_CAST ((k), CAMEL_MIME_PART_TYPE, CamelMimePartClass))
 #define CAMEL_IS_MIME_PART(o)    (CAMEL_CHECK_TYPE((o), CAMEL_MIME_PART_TYPE))
 
-
+/* note, if you change this, make sure you change the 'encodings' array in camel-mime-part.c */
 enum _CamelMimePartEncodingType {
         CAMEL_MIME_PART_ENCODING_DEFAULT,
         CAMEL_MIME_PART_ENCODING_7BIT,
         CAMEL_MIME_PART_ENCODING_8BIT,
         CAMEL_MIME_PART_ENCODING_BASE64,
         CAMEL_MIME_PART_ENCODING_QUOTEDPRINTABLE,
+        CAMEL_MIME_PART_ENCODING_BINARY,
         CAMEL_MIME_PART_NUM_ENCODINGS
 };
 typedef enum _CamelMimePartEncodingType CamelMimePartEncodingType;
diff --git a/camel/camel-mime-utils.c b/camel/camel-mime-utils.c
index d2df6cac76..f1dbe6de4c 100644
--- a/camel/camel-mime-utils.c
+++ b/camel/camel-mime-utils.c
@@ -1081,12 +1081,6 @@ header_decode_string(const char *in)
 	return header_decode_text(in, strlen(in));
 }
 
-static char *encoding_map[] = {
-	"US-ASCII",
-	"ISO-8859-1",
-	"UTF-8"
-};
-
 /* FIXME: needs a way to cache iconv opens for different charsets? */
 static void
 rfc2047_encode_word(GString *outstring, const char *in, int len, const char *type, unsigned short safemask)
@@ -2215,7 +2209,7 @@ header_param_list_format_append(GString *out, struct _header_param *p)
 			len = 0;
 		}
 
-		g_string_sprintfa(out, " ; %s=", p->name);
+		g_string_sprintfa(out, "; %s=", p->name);
 
 		for (ch = p->value; *ch; ch++) {
 			if (is_tspecial(*ch))
@@ -2874,26 +2868,37 @@ void header_address_list_clear(struct _header_address **l)
 	*l = NULL;
 }
 
+/* if encode is true, then the result is suitable for mailing, otherwise
+   the result is suitable for display only (and may not even be re-parsable) */
 static void
-header_address_list_format_append(GString *out, struct _header_address *a)
+header_address_list_encode_append(GString *out, int encode, struct _header_address *a)
 {
 	char *text;
 
 	while (a) {
 		switch (a->type) {
 		case HEADER_ADDRESS_NAME:
-			text = header_encode_phrase (a->name);
+			if (encode)
+				text = header_encode_phrase (a->name);
+			else
+				text = a->name;
 			if (text && *text)
 				g_string_sprintfa(out, "%s <%s>", text, a->v.addr);
 			else
 				g_string_append(out, a->v.addr);
-			g_free (text);
+			if (encode)
+				g_free(text);
 			break;
 		case HEADER_ADDRESS_GROUP:
-			text = header_encode_string(a->name);
-			g_string_sprintfa(out, "%s:\n ", text);
-			header_address_list_format_append(out, a->v.members);
+			if (encode)
+				text = header_encode_phrase(a->name);
+			else
+				text = a->name;
+			g_string_sprintfa(out, "%s: ", text);
+			header_address_list_encode_append(out, encode, a->v.members);
 			g_string_sprintfa(out, ";");
+			if (encode)
+				g_free(text);
 			break;
 		default:
 			g_warning("Invalid address type");
@@ -2905,7 +2910,23 @@ header_address_list_format_append(GString *out, struct _header_address *a)
 	}
 }
 
-/* FIXME: need a 'display friendly' version, as well as a 'rfc friendly' version? */
+char *
+header_address_list_encode(struct _header_address *a)
+{
+	GString *out;
+	char *ret;
+
+	if (a == NULL)
+		return NULL;
+
+	out = g_string_new("");
+
+	header_address_list_encode_append(out, TRUE, a);
+	ret = out->str;
+	g_string_free(out, FALSE);
+	return ret;
+}
+
 char *
 header_address_list_format(struct _header_address *a)
 {
@@ -2917,7 +2938,7 @@ header_address_list_format(struct _header_address *a)
 
 	out = g_string_new("");
 
-	header_address_list_format_append(out, a);
+	header_address_list_encode_append(out, FALSE, a);
 	ret = out->str;
 	g_string_free(out, FALSE);
 	return ret;
diff --git a/camel/camel-mime-utils.h b/camel/camel-mime-utils.h
index a1f8bb549d..e829a17093 100644
--- a/camel/camel-mime-utils.h
+++ b/camel/camel-mime-utils.h
@@ -96,6 +96,9 @@ void header_address_list_clear(struct _header_address **);
 
 struct _header_address *header_address_decode(const char *in);
 struct _header_address *header_mailbox_decode(const char *in);
+/* for mailing */
+char *header_address_list_encode(struct _header_address *a);
+/* for display */
 char *header_address_list_format(struct _header_address *a);
 
 /* structured header prameters */
diff --git a/camel/camel-stream-filter.c b/camel/camel-stream-filter.c
index 211db36e7f..0a7d876159 100644
--- a/camel/camel-stream-filter.c
+++ b/camel/camel-stream-filter.c
@@ -21,8 +21,8 @@
 
 #include "camel-stream-filter.h"
 
-#define d(x)
-/*#include <stdio.h>*/
+#define d(x) x
+#include <stdio.h>
 
 struct _filter {
 	struct _filter *next;
@@ -106,7 +106,6 @@ camel_stream_filter_finalize(CamelObject *o)
 	camel_object_unref((CamelObject *)filter->source);
 }
 
-
 CamelType
 camel_stream_filter_get_type (void)
 {
@@ -125,7 +124,6 @@ camel_stream_filter_get_type (void)
 	return type;
 }
 
-
 /**
  * camel_stream_filter_new:
  *
@@ -144,7 +142,6 @@ camel_stream_filter_new_with_stream(CamelStream *stream)
 	return new;
 }
 
-
 /**
  * camel_stream_filter_add:
  * @filter: Initialised CamelStreamFilter.
diff --git a/camel/camel-stream.c b/camel/camel-stream.c
index f649494b2d..79fa07cc34 100644
--- a/camel/camel-stream.c
+++ b/camel/camel-stream.c
@@ -31,10 +31,13 @@ static CamelObjectClass *parent_class = NULL;
 /* Returns the class for a CamelStream */
 #define CS_CLASS(so) CAMEL_STREAM_CLASS(CAMEL_OBJECT_GET_CLASS(so))
 
-static int stream_flush   (CamelStream *stream);
-static int stream_close   (CamelStream *stream);
-static gboolean stream_eos (CamelStream *stream);
-
+/* default implementations, do very little */
+static ssize_t   stream_read       (CamelStream *stream, char *buffer, size_t n) { return 0; }
+static ssize_t   stream_write      (CamelStream *stream, const char *buffer, size_t n) { return n; }
+static int       stream_close      (CamelStream *stream) { return 0; }
+static int       stream_flush      (CamelStream *stream) { return 0; }
+static gboolean  stream_eos        (CamelStream *stream) { return stream->eos; }
+static int       stream_reset      (CamelStream *stream) { return 0; }
 
 static void
 camel_stream_class_init (CamelStreamClass *camel_stream_class)
@@ -42,9 +45,12 @@ camel_stream_class_init (CamelStreamClass *camel_stream_class)
 	parent_class = camel_type_get_global_classfuncs( CAMEL_OBJECT_TYPE );
 
 	/* virtual method definition */
-	camel_stream_class->flush = stream_flush;
+	camel_stream_class->read = stream_read;
+	camel_stream_class->write = stream_write;
 	camel_stream_class->close = stream_close;
+	camel_stream_class->flush = stream_flush;
 	camel_stream_class->eos = stream_eos;
+	camel_stream_class->reset = stream_reset;
 }
 
 CamelType
@@ -107,14 +113,6 @@ camel_stream_write (CamelStream *stream, const char *buffer, size_t n)
 	return CS_CLASS (stream)->write (stream, buffer, n);
 }
 
-
-static int
-stream_flush (CamelStream *stream)
-{
-	/* nothing */
-	return 0;
-}
-
 /**
  * camel_stream_flush:
  * @stream: a CamelStream object
@@ -132,14 +130,6 @@ camel_stream_flush (CamelStream *stream)
 	return CS_CLASS (stream)->flush (stream);
 }
 
-
-static int
-stream_close (CamelStream *stream)
-{
-	/* nothing */
-	return 0;
-}
-
 /**
  * camel_stream_close:
  * @stream: 
@@ -156,13 +146,6 @@ camel_stream_close (CamelStream *stream)
 	return CS_CLASS (stream)->close (stream);
 }
 
-
-static gboolean
-stream_eos (CamelStream *stream)
-{
-	return stream->eos;
-}
-
 /**
  * camel_stream_eos:
  * @stream: a CamelStream object
@@ -180,7 +163,6 @@ camel_stream_eos (CamelStream *stream)
 	return CS_CLASS (stream)->eos (stream);
 }
 
-
 /**
  * camel_stream_reset: reset a stream
  * @stream: the stream object
diff --git a/camel/camel-types.h b/camel/camel-types.h
index 79abc26a7d..bc3c226ed3 100644
--- a/camel/camel-types.h
+++ b/camel/camel-types.h
@@ -37,6 +37,7 @@ typedef struct _CamelInternetAddress CamelInternetAddress;
 typedef struct _CamelMedium CamelMedium;
 typedef struct _CamelMimeFilter CamelMimeFilter;
 typedef struct _CamelMimeFilterBasic CamelMimeFilterBasic;
+typedef struct _CamelMimeFilterBestenc CamelMimeFilterBestenc;
 typedef struct _CamelMimeFilterCharset CamelMimeFilterCharset;
 typedef struct _CamelMimeFilterIndex CamelMimeFilterIndex;
 typedef struct _CamelMimeFilterLinewrap CamelMimeFilterLinewrap;
@@ -54,6 +55,7 @@ typedef struct _CamelSession CamelSession;
 typedef struct _CamelSimpleDataWrapper CamelSimpleDataWrapper;
 typedef struct _CamelStore CamelStore;
 typedef struct _CamelStream CamelStream;
+typedef struct _CamelStreamNull CamelStreamNull;
 typedef struct _CamelStreamBuffer CamelStreamBuffer;
 typedef struct _CamelStreamDataWrapper CamelStreamDataWrapper;
 typedef struct _CamelStreamFilter CamelStreamFilter;
diff --git a/camel/providers/smtp/camel-smtp-transport.c b/camel/providers/smtp/camel-smtp-transport.c
index 6fa39bb930..df70d48baf 100644
--- a/camel/providers/smtp/camel-smtp-transport.c
+++ b/camel/providers/smtp/camel-smtp-transport.c
@@ -347,24 +347,20 @@ _send_to (CamelTransport *transport, CamelMedium *message,
 	  GList *recipients, CamelException *ex)
 {
 	CamelSmtpTransport *smtp_transport = CAMEL_SMTP_TRANSPORT (transport);
-	CamelInternetAddress *cia;
-	char *recipient, *sender;
+	const CamelInternetAddress *cia;
+	char *recipient;
 	const char *addr;
 	gboolean has_8bit_parts;
 	GList *r;
 	
-	sender = g_strdup (camel_mime_message_get_from (CAMEL_MIME_MESSAGE (message)));
-	if (!sender) {
+	cia = camel_mime_message_get_from(CAMEL_MIME_MESSAGE (message));
+	if (!cia) {
 		camel_exception_setv (ex, CAMEL_EXCEPTION_SYSTEM,
 				      "Cannot send message: "
 				      "sender address not defined.");
 		return FALSE;
 	}
 	
-	cia = camel_internet_address_new ();
-	camel_address_decode (CAMEL_ADDRESS (cia), sender);
-	g_free (sender);
-	
 	if (!camel_internet_address_get (cia, 0, NULL, &addr)) {
 		camel_exception_setv (ex, CAMEL_EXCEPTION_SYSTEM,
 				      "Cannot send message: "