Bugfixes, performance improvemnts. Should scale up much better than

before, and be more bugfree than ever!

2000-10-25  Not Zed  <NotZed@HelixCode.com>

 	* ibex_internal.h (IBEX_VERSION): Bumped to another version.  The
 	file format hasn't changed, but earlier bugs may create invalid
 	files.

 	* block.c (ibex_block_read): Use the root data directly.
 	(ibex_block_cache_open): As well.
 	(ibex_block_get): And here too.
 	(ibex_block_cache_sync): Sync the root block directly here.

 	* block.h: Pad root block out to 1024 bytes.
 	Added root block to struct _memcache.

 	* disktail.c (tail_get): Dirty the root block.
 	(tail_get): Fix for changes to root access.
 	(disk_remove): And here too.

 	* wordindexmem.c (sync_cache_entry): Handle the case of not having
 	any files in the list, which can happen now.
 	(word_index_pre): Make sure we set the wordid on the new cache
 	entry.

 	* ibex_block.c (ibex_save): Sigh.  Pass the right argument to
 	index_post.

	* block.c (ibex_block_cache_open): Create a word_index_mem for
 	indexing the words, rather than a word_index.

 	* ibex_block.c (ibex_index_buffer): If we haven't called index_pre
 	yet, do it before indexing anything.
 	(ibex_save): If wehave called index_pre previously, call
 	index_post.
 	(ibex_close): And same for here.

 	* index.h: Added a cursor class, and cursor retrieval function for
 	iterating through an index's keys.

 	* wordindexmem.c (ibex_create_word_index_mem): New word class,
 	similar to wordindex, but meant to be faster for updates.
 	(word_index_pre): Implement.  We load all keys into memory.
 	(word_index_post): Implement.  We sync and free all keys.
 	(find): Remove lru code, its no longer a cache, but a lookup
 	table.
 	(add_index_cache): Remove lru code here too.
 	(find_name): And here.
 	(word_flush): Flush the hashtable direct.
 	(word_close): Call flush to flush, rather than doing it ourselves.
 	(add_index_cache): If we are in an index state, we can assume a
 	cache miss == a new word.
 	(word_index_post): Maintain whether or not we are in an index
 	state, and the depth of the state.
 	(word_index_pre): Likewise.  Dont reread the index if we have
 	already.
 	(cache_sanity): Fixed for struct changes.

 	* wordindex.h (IBEXWordClass): Added functions to prepare/cleanup
 	for lots of indexing.  i.e. can be used to optimise indexing speed
 	at the cost of extra memory usage during the indexing process.

	* hash.c (hash_cursor_create): Create a new cursor for iterating through a
 	hashtable.
 	(hash_cursor_close): 'close' the cursor. It is upto the
 	application to close any cursors it creates.
 	(hash_cursor_next): Goto the next key id.
 	(hash_cursor_next_key): Goto the next key, reutrn the key.
 	(hash_get_cursor): Return a cursor object.

	* wordindex.c (word_index_post):
 	(word_index_pre): Added (empty) callbacks for pre/post functions.

svn path=/trunk/; revision=6165

1 files changed, 17 insertions, 10 deletions

diff --git a/libibex/disktail.c b/libibex/disktail.c
index 90562889de..9f5a0dbea2 100644
--- a/libibex/disktail.c
+++ b/libibex/disktail.c
@@ -214,7 +214,6 @@ tail_dump(struct _memcache *blocks, blockid_t tailid)
 static blockid_t
 tail_get(struct _memcache *blocks, int size)
 {
-	struct _root *root = (struct _root *)ibex_block_read(blocks, 0);
 	blockid_t tailid;
 	struct _tailblock *tail;
 	int freeindex;
@@ -225,22 +224,30 @@ tail_get(struct _memcache *blocks, int size)
 
 	/* look for a node with enough space, if we dont find it fairly
 	   quickly, just quit.  needs a better free algorithm i think ... */
-	tailid = root->tail;
+	tailid = blocks->root.tail;
 	while (tailid && count<5) {
 		int space;
 
 		d(printf(" checking tail node %d\n", tailid));
 
 		tail = (struct _tailblock *)ibex_block_read(blocks, tailid);
+
 		if (tail->used == 0) {
 			/* assume its big enough ... */
 			tail->used = 1;
 			tail->tb_offset[0] = sizeof(tail->tb_data)/sizeof(tail->tb_data[0]) - size;
 			d(printf("allocated %d (%d), used %d\n", tailid, tailid, tail->used));
 			ibex_block_dirty((struct _block *)tail);
+
+			g_assert(&tail->tb_offset[tail->used-1]
+				 < &tail->tb_data[tail->tb_offset[tail->used-1]]);
+
 			return tailid;
 		}
 
+		g_assert(&tail->tb_offset[tail->used-1]
+			 < &tail->tb_data[tail->tb_offset[tail->used-1]]);
+
 		/* see if we have a free slot first */
 		freeindex = -1;
 		end = &tail->tb_data[sizeof(tail->tb_data)/sizeof(tail->tb_data[0])];
@@ -280,16 +287,18 @@ tail_get(struct _memcache *blocks, int size)
 	}
 
 	d(printf("allocating new data node for tail data\n"));
-	/* didn't find one, setup a new one */
-	root = (struct _root *)ibex_block_read(blocks, 0);
 	tailid = ibex_block_get(blocks);
 	tail = (struct _tailblock *)ibex_block_read(blocks, tailid);
-	tail->next = block_number(root->tail);
-	root->tail = tailid;
+	tail->next = block_number(blocks->root.tail);
+	blocks->root.tail = tailid;
 	tail->used = 1;
 	tail->tb_offset[0] = sizeof(tail->tb_data)/sizeof(tail->tb_data[0]) - size;
 	ibex_block_dirty((struct _block *)tail);
 	d(printf("allocated %d (%d), used %d\n", tailid, TAIL_KEY(tailid, 0), tail->used));
+
+	g_assert(&tail->tb_offset[tail->used-1]
+		 < &tail->tb_data[tail->tb_offset[tail->used-1]]);
+
 	return TAIL_KEY(tailid, 0);
 }
 
@@ -578,15 +587,13 @@ disk_remove(struct _IBEXStore *store, blockid_t *headptr, blockid_t *tailptr, na
 				start->used--;
 				block->bl_data[i] = start->bl_data[start->used];
 				if (start->used == 0) {
-					struct _root *root = (struct _root *)ibex_block_read(store->blocks, 0);
 					blockid_t new;
 
 					d(printf("dropping block %d, new head = %d\n", head, start->next));
 					new = block_location(start->next);
-					start->next = block_number(root->free);
-					root->free = head;
+					start->next = block_number(store->blocks->root.free);
+					store->blocks->root.free = head;
 					head = new;
-					ibex_block_dirty((struct _block *)root);
 				}
 				ibex_block_dirty(block);
 				ibex_block_dirty(start);