2000-11-17 Not Zed <NotZed@HelixCode.com>
* wordindexmem.c (add_list): If we have the namecache active, and
there is no name there, we add it directly and dont look it up
first.
* testindex.c: Some performance testing & stat gathering stuff.
2000-11-16 Not Zed <NotZed@HelixCode.com>
* wordindexmem.c (ibex_create_word_index_mem): Initialise nameinit
& namecache.
(contains_name): On first call, load all names into memory. We
usually do a whole lot of lookups in a row, and this saves a lot
of penalties on a big list, for not too much a memory hit.
(find_name): If we have the namelist in memory do a quick
short-circuit check to see if we have to do further processing.
(unindex_name): Cross check the namecache, if it is active.
Remove it there too/or exit (no work to do).
(word_flush): If we have the namecache active, destroy it now, as
it is not needed anymore (for now).
2000-10-30 Kjartan Maraas <kmaraas@gnome.org>
* hash.c: #include <stdlib.h> to remove warning.
* wordindex.c: #include <stdlib.h> and <string.h>.
2000-10-26 Not Zed <NotZed@HelixCode.com>
* block.c (ibex_block_cache_open): Use IBEX_VERSION rather than
hardcoded version string.
* ibex_internal.h (IBEX_VERSION): Bumped version again. This time
I did change the index format.
(IBEX_VERSION): moved into block.h
* hash.c (struct _hashroot): Add a linked list of keys to the table.
(struct _hashblock): Added a next pointer as a block number.
(hash_insert): Link new key blocks into the key block list.
(struct _HASHCursor): Renamed block to key and added a block item.
(hash_cursor_next): Changed to go through the linked list of all
hash items rather than through each hash chain separately. >>
faster.
(ibex_hash_dump_rec): Remove a warning.
2000-10-25 <jpr@helixcode.com>
* ibex_block.c: No longer include <db.h>
2000-10-25 Not Zed <NotZed@HelixCode.com>
* ibex_internal.h (IBEX_VERSION): Bumped to another version. The
file format hasn't changed, but earlier bugs may create invalid
files.
* block.c (ibex_block_read): Use the root data directly.
(ibex_block_cache_open): As well.
(ibex_block_get): And here too.
(ibex_block_cache_sync): Sync the root block directly here.
* block.h: Pad root block out to 1024 bytes.
Added root block to struct _memcache.
* disktail.c (tail_get): Dirty the root block.
(tail_get): Fix for changes to root access.
(disk_remove): And here too.
* wordindexmem.c (sync_cache_entry): Handle the case of not having
any files in the list, which can happen now.
(word_index_pre): Make sure we set the wordid on the new cache
entry.
* ibex_block.c (ibex_save): Sigh. Pass the right argument to
index_post.
2000-10-24 JP Rosevear <jpr@helixcode.com>
* .cvsignore: Shush
2000-10-24 Not Zed <NotZed@HelixCode.com>
* block.c (ibex_block_cache_open): Create a word_index_mem for
indexing the words, rather than a word_index.
* ibex_block.c (ibex_index_buffer): If we haven't called index_pre
yet, do it before indexing anything.
(ibex_save): If wehave called index_pre previously, call
index_post.
(ibex_close): And same for here.
* index.h: Added a cursor class, and cursor retrieval function for
iterating through an index's keys.
* wordindexmem.c (ibex_create_word_index_mem): New word class,
similar to wordindex, but meant to be faster for updates.
(word_index_pre): Implement. We load all keys into memory.
(word_index_post): Implement. We sync and free all keys.
(find): Remove lru code, its no longer a cache, but a lookup
table.
(add_index_cache): Remove lru code here too.
(find_name): And here.
(word_flush): Flush the hashtable direct.
(word_close): Call flush to flush, rather than doing it ourselves.
(add_index_cache): If we are in an index state, we can assume a
cache miss == a new word.
(word_index_post): Maintain whether or not we are in an index
state, and the depth of the state.
(word_index_pre): Likewise. Dont reread the index if we have
already.
(cache_sanity): Fixed for struct changes.
* wordindex.h (IBEXWordClass): Added functions to prepare/cleanup
for lots of indexing. i.e. can be used to optimise indexing speed
at the cost of extra memory usage during the indexing process.
* dumpindex.c: Dumps the contents of indexs.
* hash.c (ibex_hash_dump_rec): Also print the word count.
(hash_cursor_create): Create a new cursor for iterating through a
hashtable.
(hash_cursor_close): 'close' the cursor. It is upto the
application to close any cursors it creates.
(hash_cursor_next): Goto the next key id.
(hash_cursor_next_key): Goto the next key, reutrn the key.
(hash_get_cursor): Return a cursor object.
* wordindex.c (unindex_name): Cross-check the cache as well.
(word_index_post):
(word_index_pre): Added (empty) callbacks for pre/post functions.
2000-10-12 Not Zed <NotZed@HelixCode.com>
* ibex_internal.h (struct ibex): Bumped ibex rev.
* block.c (ibex_block_cache_open): Bumped the ibex file revision
because of the hash table size change.
* index.h: Added some stat stuff.
* wordindex.c (struct _wordcache): Changed files[] to be a pointer
to an allocated block/or an individual item.
(find): Fix for changes to struct.
(find_name): "
(sync_cache_entry): "
(add): "
(add_list): "
(add_index_cache): Free the cache file array if it was created.
(word_flush): And here.
(word_close): And here too.
(ibex_create_word_index): Double the size of the hashtables.
(word_flush): Make sure we reset the wordcount to 0 if we remove
the list items. DOH.
(add_index_cache): Use a slightly more sohpisticated aging
algorithm to remove expired nodes.
2000-10-10 Not Zed <NotZed@HelixCode.com>
* hash.c (hash_find):
(hash_remove):
(hash_insert): Truncate key if it is too big to fit in a
single block to MAX_KEYLEN bytes.
2000-09-28 Not Zed <NotZed@HelixCode.com>
* block.c (ibex_block_free): Make sure we map the 'free' block to
a block number when unlinking a block (fixes a lot of assertion
failures).
(ibex_block_cache_open): Initialise sync flag on root block. If
it is not set on open then the index could be in an invalid state,
and should be rescanned.
(ibex_block_cache_sync): Sync root block last, and set the sync
flag.
(ibex_block_cache_open): Mirror root block flags in block_cache
struct.
(ibex_block_cache_sync): Likewise.
(ibex_block_read): If we write a dirty block, then we clear the
sync flag if its still set; we are no longer synced.
2000-09-19 Not Zed <NotZed@HelixCode.com>
** Merged from IBEX_DISK branch to head.
* file.c:
* find.c:
* words.c:
* index.c: Removed unused files.
* block.h: Changed block to use only 24 bits for next and 8 for
used, and fixed all relevant code. Some cleanup.
* disktail.c (tail_get): If we use an empty tail node, then make
sure we make it dirty.
2000-09-15 Not Zed <NotZed@HelixCode.com>
* wordindex.c (word_close): Free hashtable on exit too.
* disktail.c: Implemented tail-node storage for the end of long
lists, or for short lists. Should save significant disk space
(5x?).
Implemented special case for 1-item lists, where the tailnode
pointer is used to store the index entry.
2000-09-14 Not Zed <NotZed@HelixCode.com>
* wordindex.c (add_index_key): Keys also handle tails.
* hash.c (hash_set_data_block): Added new parameter to keys - a
tail block (a full 32 bit block pointer).
(hash_get_data_block): And same here.
2000-09-12 Not Zed <NotZed@HelixCode.com>
* wordindex.c (word_close): Dont close namestore twice.
2000-09-11 Not Zed <NotZed@HelixCode.com>
** Redid almost everything, on-disk hash table to store an index
to index records, mroe on the way to modularisation (more to go),
now stores reverse indexes for deleting.
2000-08-31 Not Zed <NotZed@HelixCode.com>
* block.c (add_key_mem): Initialise a memory based array for newly
added index entries.
(add_record): Changed to cache updates in memory until we hit a
limit, and then flush them to disk.
(get_record): Merge in-memory records with disk records.
(remove_record): Remove from memory first, and if that fails, goto
disk.
(find_record): Check memory first, then disk if that fails.
(add_datum_list): oops, copy size * sizeof(blockid_t)
(add_indexed): Make sure we link in the head node when we create a
new one.
2000-08-09 Christopher James Lahey <clahey@helixcode.com>
* file.c, find.c: Fixed some warnings.
2000-05-11 NotZed <NotZed@HelixCode.com>
* index.c (ibex_unindex): Make sure we mark the ibex as dirty.
2000-05-07 NotZed <NotZed@HelixCode.com>
* file.c (ibex_save): New function, only write out the ibex if it
has changed.
2000-05-07 <notzed@helixcode.com>
* file.c (ibex_open): Also close the fd after we're done.
* find.c (ibex_contains_name): New function to find out if a file
is indexed.
2000-05-02 Matt Loper <matt@helixcode.com>
* Makefile.am: set G_LOG_DOMAIN.
2000-04-12 NotZed <NotZed@HelixCode.com>
* find.c (ibex_dump_all): Debug function to dump the whole index
to stdout.
* words.c (get_ibex_file): Use g_strdup(), not strdup().
2000-04-11 NotZed <NotZed@HelixCode.com>
* file.c (write_word): Always write out all words we have (even if
its 0 ... the file expects it). No longer check for removed files.
(store_word): Check for removed files here, and only add to the
ordered tree if we have references left to this word.
(ibex_write): First insert into the tree, to determine the
wordcount to be saved in the output file, and then write that.
(ibex_open): Remove some debug.
* words.c (ibex_index_buffer): Always set 'unread', if it is a
valid pointer (dont rely on caller to initialise it).
2000-03-26 NotZed <NotZed@HelixCode.com>
* lookup.c (main): Fixed call to ibex_open.
* mkindex.c (main): Fixed call to ibex_open.
* file.c (ibex_open): Changed to accept flags and mode equivalent
to open(2).
2000-02-25 Dan Winship <danw@helixcode.com>
* *.c: add gtk-doc-style comments
2000-02-21 Matt Loper <matt@helixcode.com>
* .cvsignore: Added mkindex.
2000-02-21 NotZed <NotZed@HelixCode.com>
* Makefile.am: change noinst_LIBRARIES to noinst_LTLIBRARIES, and
supply -static to LDFLAGS. Duh, and changed LDADD back to
libibex.la.
2000-02-20 Matt Loper <matt@helixcode.com>
* Makefile.am: changed mkindex_LDADD to libibex.a instead of
libibex.la.
2000-02-19 Matt Loper <matt@helixcode.com>
* .cvsignore: added lookup.
2000-02-18 Miguel de Icaza <miguel@nuclecu.unam.mx>
* Makefile.am (lookup_LDADD): For now. make a libibex.a library so
we can link it with the camel provider. I hate libtool
2000-02-16 Dan Winship <danw@helixcode.com>
* Makefile.am: automakify
2000-02-16 NotZed <NotZed@HelixCode.com>
* find.[ch] (ibex_find_name): Finds if a word is indexed under a
given name.
2000-02-14 NotZed <notzed@zedzone.helixcode.com>
* Makefile: Hack together a build using libtool. This should all
be auto*'d at some point I guess.
2000-02-13 NotZed <notzed@zedzone.helixcode.com>
* Added ChangeLog file.
