new file. Document message UID and vfolder implementation.

1999-08-09  bertrand  <Bertrand.Guiheneuf@aful.org>

	* devel-docs/misc/ref_and_id_proposition.txt:
	new file. Document message UID and vfolder
	implementation.

svn path=/trunk/; revision=1100

2 files changed, 179 insertions, 36 deletions

diff --git a/ChangeLog b/ChangeLog
index 66794caa7a..51c20cec72 100644
--- a/ChangeLog
+++ b/ChangeLog
@@ -1,3 +1,9 @@
+1999-08-09  bertrand  <Bertrand.Guiheneuf@aful.org>
+
+	* devel-docs/misc/ref_and_id_proposition.txt:
+	new file. Document message UID and vfolder
+	implementation.
+
 1999-08-08  Robert Brady  <rwb197@ecs.soton.ac.uk>
 
 	* camel/gmime-rfc2047.c: more advanced RFC2047 encoder started.
diff --git a/devel-docs/misc/ref_and_id_proposition.txt b/devel-docs/misc/ref_and_id_proposition.txt
index 757fa7b2cd..961cdfb872 100644
--- a/devel-docs/misc/ref_and_id_proposition.txt
+++ b/devel-docs/misc/ref_and_id_proposition.txt
@@ -1,14 +1,14 @@
-Hi everyone,
+Author: Bertrand Guiheneuf <Bertrand.Guiheneuf@aful.org>
+Date: August 9th 1999
+Version: 0.1 
 
-This mail talks about problems related to message referencing in
-Camel. I would like to get people thoughts about two specific issues:
-
-1) How to identify reliably messages within folders. 
-2) How to handle references in folders to messages physically stored
-   in other folders.
+The last version of this document is always available in gnome CVS in
+the gnome-mailer module: devel-docs/misc/ref_and_id_proposition.txt
 
 
 
+A) Identifying messages within folders
+--------------------------------------
 
 Currently, in Camel there is only one way to retrieve a message from a
 mail store:
@@ -19,15 +19,17 @@ where number is an integer representing the message rank within its
 parent folder.
 
 This is a traditional method (JavaMail, MAPI) and it is very useful
-because this is often the only way to get a message from a classical
-store (pop3 for example).
+because this is often the only way to get a identify message in a
+classical store (pop3 for example).
 
 Moreover, various documents ([1], [2]) proposed to generalize the URL
-scheme used in Camel ([3]) to access mail stores to identify
-messages. For instance:
+scheme used in Camel ([3]) to access mail stores in order to identify
+messages. Such an URL would be, for instance:
 
 pop3://po.myisp.com:1
 
+Meaning: "Access message 1 on Pop3 server po.myisp.com"
+
  
 However, referencing a message with its number within a folder is a
 very unreliable method:
@@ -50,15 +52,15 @@ very unreliable method:
    
 These two scenarii show that it is quite impossible to use reliable
 folder caching or message referencing if messages are referenced only
-with their position within their parent folder.
+by their position within their parent folder.
    
 
 We thus have to find a general way to identify and retreive a message
 within its folder. One thing is sure, however: all folders
 implementation won't allow this method. Pop3 stores will always access
-messages using their rank on the server. MUA using will thus have to be
-prepared to access some stores providing only the old fashionned message 
-number access method. 
+messages using their rank on the server. MUA using Camel will thus
+have to be prepared to access some stores providing only the old
+fashionned message number access method.
 
 Basically, we have two choices:
 
@@ -66,34 +68,169 @@ Basically, we have two choices:
 
    A UID is a string identifier associated to a message, which is
    guaranteed to be unique within its parent folder and which will not
-   to change between sessions.
+   change between sessions.
 
 2) Accessing messages using Message ID
 
    A Message ID is a string identifier associated to a messages which
    is guaranteed to be unique in the world, that is, no other message
-   can have the same Message ID. The message ID is defined and RFC 822, and 
-   is stored as a message header
-   Message-id: ...
+   can have the same Message ID. The message ID is defined in RFC 822,
+   and is stored as the message header "Message-id"
+   
+Method (1) already exists in IMAP.  
+It is quite simple to define on local stores (MH, mbox, ....) but it
+may not resist to message modification by other MUA.
+Methods based on Message-id matching or message content-checksum seem
+to be the best one. Using an "X-" header is another possibility on
+non-read only headers. A combination of these three methods may be the
+most reliable solution.
+The UID is impossible to implement in a POP3 store provider.
+
+(2) Can be used with IMAP, but would be very ineficient. 
+The main issue with this method is its dependancy upon other MUAs and
+MTAs. Message-id is set before or during message transport. Moreover,
+some rfc822 compliant messages may not even have any Message-id
+header.
+These are major issues when accessing read-only stores.  
+The M-ID is also impossible to implement in a POP3 store provider.
    
-(1) Already exists in IMAP. It is quite simple to define on local
-stores (MH, mbox, ....) but may not resist to message modification by
-other MUA.  Methods based on Message-id matching or message
-content-checksum seem to be the best one. Using an "X-" header is
-another possibility on non-read only headers. A combination of these
-three methods may be the most reliable solution. 
-Impossible to implement on POP3
-
-(2) Can be used with IMAP, but would be very ineficient. The main
-issue with this method is its dependancy upon other MUAs and
-MTAs. Message-id is set during message transport. Moreover, some
-messages may even not have anay Message-id header. These are major
-issues when accessing read-only stores. 
-Impossible to implement on POP3
+
+We may not rely on external MUA and MTA to guarentee the uniqueness of
+the identifier . We may lose messages by never being able to read them
+if two had the same uid. It would be possible to find workarounds, but
+it could make Camel use a bit tricky.
+
+Given that most users will use IMAP or a database based store as their
+main mail store, and given that this stores allow UID very
+easily, I suggest that we use method (1). Discussion is still open,
+though.
+
+Here are the public methods I propose to add to CamelFolder:
+
+gboolean camel_folder_supports_uid (CamelFolder *folder)
+    returns true if the folder can get messages 
+    by their uid.
+
+gchar * camel_folder_get_uid_by_number (CamelFolder *folder, gint message_number)
+    return the uid of message which number in the folder
+    is %message_number.
+
+gchar * camel_folder_get_message_uid (CamelFolder *folder, CamelMimeMessage *message)
+    return the uid of the message within the folder.
+
+CamelMimeMessage *camel_folder_get_message_by_uid  (CamelFolder *folder, gchar *uid)
+    return the message which uid is %uid
+
+In addition, the CamelFolder Class will have a new public method
+
+gchar * camel_mime_message_get_uid (CamelMimeMessage *message) 
+    return the uid associated to the message in its physical parent
+    folder.
+
+
+
+B) Handling message references in (v)folders.
+---------------------------------------------
+ 
+
+We want the future Gnome mailer to be able to build (virtual) folders
+holding references to messages located physically in other
+folders. More generally, we would like folders to be able to hold:
+
+1) messages
+2) subfolders
+3) references to messages
+
+(1) and (2) are already implemented in Camel because most mail stores
+can hold messages and/or subfolders.
+
+(3) is a different issue, because no existing mail store can currently
+hold within folders references to messages in other folders.
+It will thus be a specific gnome-mailer extension. 
+
+
+One of the main issue is to determine what kind of behaviour we expect
+from folders holding references. Here is a possible API.
+
+( the world (v)folder is used to distinguish between the physical
+parent folder and the folder holding a reference to the message, when
+a confusion may arise)
+
+Addition to CamelFolder:
+
+gboolean camel_folder_can_hold_references (CamelFolder *folder)
+    return true if the folder can contain references
+
+void camel_folder_add_reference_by_uid (CamelFolder *folder, gchar *folder_url, gchar *message_uid)
+    add a reference into a folder. %folder_url is the url of 
+    the folder, %message_uid is the uid of the message within 
+    its physical parent folder.
+ 
+void camel_folder_add_reference_by_message (CamelFolder *folder, CamelMessage *message)
+    add a reference. The place where the reference points 
+    to is found using CamelMessage methods
+
+void camel_folder_remove_reference_by_uid (CamelFolder *folder, gchar *uid)
+    remove a message reference form a folder. Reference 
+    is identified using its uid within the folder.
+
+gboolean camel_folder_uid_is_reference (CamelFolder *folder, gchar *uid)
+    return true if the message corresponding to the uid is a reference.
+
+Then all usual operations on the folder act if the message was
+actually physically stored in this folder. For example, when the mailer
+uses camel_folder_get_message_by_uid onto the (v)folder, the actual
+message is retreived from its physical store.
+
+As you can see, the uid of the message within its physical parent
+folder is different than its uid within the (v)folder. This is because
+there is no way to guarantee that the uids of two messages in two
+different folders would be different. Using references on this two
+message in the same vfolder would break uniqueness of the uid in the
+(v)folder.
+
+A couple of other methods could be defined but all the basics are
+described here.  
+
+This draft API is far from complete nor perfect, and is described here
+only to stimulate discussions before the actual implementation.
+
+
+The question now is to know how we store references. There are basically 
+two ways:
+
+1) references are stored using the URL of the physical folder
+   and the uid of the message within the folder
+
+2) a list of reference is kept, and in this list, reference are stored
+as in (1). Folders would refer to the actual message using index in
+the list
+
+
    
+The main problem with (1) is that references get lost as soon
+as the actual message is moved. There is no way to find in which
+folders references to the message exist. 
+
+(2) is a way to solve this issue. When messages are used, Camel looks
+in the list to see if the message is refered somewhere, and actualize 
+the URL and the uid with their new values. 
+
+The problem with (2) is that we need to keep this information in a file
+and libraries writing automatically to files are generally a bad idea.
+
+As in additional remark, it is clear that Camel will only be able to
+hold references to messages on stores supporting UIDs.
+
+
+Thanks in advance for your comments and ideas,
+
+
+       Bertrand <Bertrand.Guiheneuf@aful.org>
 
 
+--
 
-[1] : http://www.selequa.com/%7epurp/gnomail/mail2db.html
-[2] : http://www.selequa.com/%7epurp/gnomail/dbRecFmt.html
-[3] : http://www.gnome.org/mailing-lists/archives/gnome-mailer-list/1999-April/0248.shtml
\ No newline at end of file
+[1] : http://www.selequa.com/%7epurp/gnomail/mail2db.html 
+[2] : http://www.selequa.com/%7epurp/gnomail/dbRecFmt.html 
+[3] : http://www.gnome.org/mailing-lists/archives/gnome-mailer-list/1999-April/0248.shtml