Interface Changes introduced in DB 2.1.0: 1. We have added a compatibility API for DB 1.85. We had not intended to do this, however, versions of the GNU gcc compiler in beta release trigger problems in DB 1.85 (and in previous releases of DB 2.0.N), causing them to drop core. As we're unwilling to do further work on DB 1.85, we're adding in a compatibility mode to the current DB package instead. The DB 1.85 compatibility API is optional (so folks using both DB 1.85 and DB 2.N in a single library aren't surprised). To include it in the DB library, use the configuration option --enable-compat185. For consistency, the configuration option --enable-build185, previously found in DB 2.N releases, has been renamed to be --enable-dump185. See the file build.unix/README in the DB distribution for further information. We have also made other changes in the DB 2.1.0 release so that historic DB 1.85 applications will generally perform better using the current release than they do using the historic release. Please note that the underlying databases MUST still be converted from the historic format to the current format. 2. We have changed the default behavior of db_appinit(3) so that it does NOT create the underlying shared memory regions by default. If the calling application wishes to create these regions, it must now specify the DB_CREATE flag when calling db_appinit(3). 3. Specifying transactions to the db_appinit(3) function now implies logging, i.e., DB_INIT_TXN now implies DB_INIT_LOG. 4. We have changed the memp_stat(3) interface. It previously took a pointer to a memory pool (DB_MPOOL *) and a file pointer (FILE *) and wrote memory pool statistics to the specified file pointer. The new version is: memp_stat(DB_MPOOL *, DB_MPOOL_STAT **, DB_MPOOL_FSTAT ***); and it returns memory pool statistics to the calling application, in two new structures. See the manual page db_mpool(3) in the DB distribution for more information. We expect to add similar statistical functions to the other DB subsystems in future releases. There is also a new utility, db_stat(1). This utility presents the statistics in human-readable form. 5. We have added new functionality to the mpool interface. There's a new field, mp_mmapsize, in the DB_ENV structure, and a new flag to the memp_fopen(3) structure (DB_NOMMAP). These features provide a finer level of control over which files will be mapped into the process address space. See the manual page db_mpool(3) in the DB distribution for more information. 6. Page-in and page-out functions specified for buffer pools must now return an errno value on failure and 0 on success, instead of the previous interface, which was non-zero on failure and 0 on success, setting errno. See the manual page db_mpool(3) in the DB distribution for more information. 7. There are a couple of major performance improvements in the new DB release for small, non-persistent buffer pools, e.g., applications using the traditional DB 1.85 interface or applications creating small, temporary databases. First, the shared memory regions are no longer fully populated on startup. Second, the backing file used for temporary buffer pools is no longer created until it is actually needed. These changes should cause no visible change for applications. Interface Changes introduced in DB 2.2.0: 1. In DB 2.2.0, the handles returned by the DB subsystems, e.g., the db_appinit() function's DB_ENV, and the db_open() function's DB, are free-threaded. If you specify the DB_THREAD flag to each subsystem, or globally using db_appinit(), you can then use the returned handle concurrently in any number of threads. There are some important caveats for using DB handles concurrently in multiple threads: a. The DB_THREAD flag must be specified for all subsystems either explicitly by calling the subsystems open() function, or via the db_appinit() function. Setting the DB_THREAD flag inconsistently may result in database corruption. b. Spinlocks must have be implemented for the compiler/architecture combination. Attempting to specify the DB_THREAD flag will fail with an EINVAL error if spinlocks are not available. c. Only a single thread may call the close function for a returned database or subsystem handle. See [1]db_open(3) and the appropriate subsystem manual pages for more information. d. Either the DB_DBT_MALLOC or DB_DBT_USERMEM flags must be set in a DBT used for key or data retrieval. See [2]db_open(3) for more information. e. The DB_CURRENT, DB_NEXT and DB_PREV flags to the log_get() function may not be used by a free-threaded handle. If such calls are necessary, a thread should explicitly create a unique DB_LOG handle by calling log_open(). See [3]db_log(3) for more information. f. Each database operation (i.e., any call to a function underlying the handles returned by db_open() and db_cursor()) is normally performed on behalf of a locker associated with the handle. If, within a single thread of control, multiple calls on behalf of the same locker are desired, then transactions must be used. See [4]db(3) for more information. g. Transactions may not span threads, i.e., each transaction must begin and end in the same thread, and each transaction may only be used by a single thread. See [5]db(3) and [6]db_appinit(3) for more information on using DB in the context of threads. 2. We've added spinlock support for OSF/1, HP, Solaris and UTS4, in all cases for the native compiler/architecture combination. In the case of Solaris, this means that all DB applications must be loaded using the -lthread library. 3. We've added a C++ API for DB. There are not yet any manual pages for the C++ API, and the interface is going to change! Please feel free to browse and send us comments, but please remember that the C++ interface is going to be different in the next release. The C++ files and directories in the source distribution are: db/cxx/ C++ API db/examples_cxx/ The example programs recoded in C++. db/include/cxx_int.h Internal C++ include file. db/include/db_cxx.h External C++ include file. C++ support is automatically built on Win32. To configure it under UNIX, specify --enable-cxx as a configuration argument (see the file db/build.unix/README for more information). 4. The functionality previously embodied in some of the support utilities ([7]db_archive(1), [8]db_checkpoint(1), [9]db_deadlock(1), and [10]db_recover(1)) is now available through the DB API as well. Utility Underlying API support: [11]db_archive(1) Log archival. See the log_archive() function in [12]db_log(3). [13]db_checkpoint(1) Transaction checkpoint. See the txn_checkpoint() function in [14]db_txn(3). [15]db_deadlock(1) Deadlock detection. See the lock_detect() function in [16]db_lock(3). [17]db_recover(1) Database recovery. See the DB_RECOVER and DB_RECOVER_FATAL flags for the db_appinit() function in [18]db_appinit(3). 5. We've added a new default hashing function, __ham_func5(), written by Glenn Fowler, Landon Curt Noll and Phong Vo, and integrated into DB by Ariel Faigon of SGI. We've also deleted the previous supplied function __ham_func1(). This change is NOT transparent to applications. We incremented the hash access method database version number, and the new hash function will only be used in newly created databases, which means that applications written using version DB 2.2.0 and greater will be able to share databases with applications written using previous versions of DB with a major number of 2. However, we now use the __ham_func5() hash function internally, in the log and lock subsystems, which means that applications written using version DB 2.2.0 and greater will NOT be able to share database environments, or read log files, written using previous versions of DB. 6. The interfaces that DB uses to export statistics have been enhanced and, in one case, modified: a. DB 2.2 exports statistical information via the DB handle for the access methods. Currently, the only access method for which this information is available is B+tree. See the dbp->db_stat() function in the [19]db_open(3) manual page for more information. b. DB 2.2 exports statistical information for the transaction region via a new function, txn_stat(). See [20]db_txn(3) for more information. c. The [21]db_stat(1) utility has two new options. The -d flag permits users to display the access method statistics. The -t flag permits users to display the transaction region statistics. d. The interface for shared memory buffer pool statistics in DB 2.2 has been revised to make it consistent with the interfaces provided for the transaction region and access methods. This change is NOT transparent to applications. See [22]db_mpool(3) for more information. 7. The interface to the shared memory buffer pool has been extended in DB 2.2 to permit applications to control the maximum size of read-only files that will be mapped into the application's address space instead of being read through the memory pool cache. See the DB_NOMMAP flag to the memp_open() function and the mp_mmapsize field in the DB_ENV structure, as described in [23]db_mpool(3). 8. The interface to the transaction subsystem has been extended in DB 2.2 to permit applications to specify that the log is not to be synchronously flushed on transaction commit. This potentially provides a significant performance improvement for applications that do not require database durability. See the DB_TXN_NOSYNC flag to the txn_open() function, as described in [24]db_txn(3). 9. There have been several changes to the process of creating a DB environment: a. By default, when a lock is unavailable to a DB thread (or process), the thread/process is put to sleep for a period of time, permitting other threads/processes to run. This may not be optimal in the presence of multiple threads in a single process. The DB_ENV structure has been extended to permit applications to specify a ``yield'' function, which is called when a DB thread has requested a lock which is unavailable. See [25]db_appinit(3) for more information. b. In previous versions of DB, the DB_CREATE flag was implied by calling the db_appinit() function, i.e., initializing the environment implied that the application wished to create the environment if it did not already exist. This version of DB no longer supports this semantic, and the DB_CREATE must be explicitly specified to db_appinit() if the application wishes to create the DB environment. This change is NOT transparent to applications. c. The flags that may be specified when creating the DB environment have been extended in DB 2.2 to allow the specification of the DB_MPOOL_PRIVATE flag, which was previously supported only by the underlying memory pool subsystem. The flags that may be specified when creating the DB environment have been extended in DB 2.2 to provide new functionality: the list of new flags includes DB_NOMMAP, DB_THREAD and DB_TXN_NOSYNC. d. The DB_DATA_DIR configuration argument to the db_appinit() function is now additive, permitting applications to specify multiple directories in which to search for database files. If multiple paths are specified, created data files will always be created in the first directory specified. e. The db_errbuf field of the DB_ENV structure has been deleted from the current release. In its place, we have added the db_errcall field, which specifies a function which is called with the information previously found in the db_errbuf buffer. This change is NOT transparent to applications. f. The default temporary file location list (used when no DB_TMP_DIR configuration argument was specified) has been extended to include any directory specified by the TempFolder environment variable, if it exists. In all cases, see [26]db_appinit(3) for more information. 10. There have been several changes to the logging subsystem: a. The log_get() and log_put() functions now support the standard DBT flags described by the [27]db_open(3) manual page. b. The interface to the log_flush() function has been extended to flush the entire log if a NULL LSN is specified. See [28]db_log(3) for more information. c. The interface to the log_file() function has been changed in DB 2.2 to eliminate the need for the library to return allocated memory that may never be freed. This change is NOT transparent to applications. See [29]db_log(3) for more information. d. The [30]db_checkpoint(1) and [31]db_deadlock(1) utilities have a new option, -L, in DB 2.2 to optionally log their process ID to a file. If they exit gracefully, or if they receive a SIGINT signal, the log file is removed before they exit. 11. There have been a couple of changes to the source code layout other than those specified above: a. Operating system specific functionality is separated out in DB 2.2 into a separate subdirectory in the source code, db/os. b. All of the include files in DB 2.2 have been moved into a single subdirectory, db/include, as part of the work to port DB to MacOS. Interface Changes introduced in DB 2.3.0: 1. The DB 2.3 btree access method has been enhanced to optionally support retrieval by record number. This functionality is based on a new DB_INFO flag, DB_RECNUM. If the btree is created using this flag, applications may retrieve records in the tree by record number, using the DB_SET_RECNO flag to either the dbp->get() or cursor->c_get() interfaces. This is another additional cursor->c_get() flag, DB_GET_RECNO which returns the record number of the record referenced by the cursor. Adding this feature required a change to the underlying btree database format. For this reason, databases created under previous versions of DB cannot be read by this release of DB, and vice-versa. If you have a support contract with Sleepycat Software, please contact us for further information and assistance in migrating applications and databases from previous releases of DB to this release. See [1]db_open(3) and [2]db_cursor(3) for more information, as well as the example program examples/ex_btrec.c. 2. In previous versions of the DB library, additions or deletions into recno (fixed and variable-length record) databases would automatically renumber all records logically after the add/delete point. This behavior is no longer the default behavior as of DB version 2.3. The new default behavior is that deleting records does not cause subsequent records to be renumbered, and it is an error to attempt to add new records between records already in the database. Attempting to retrieve deleted keys using either of the dbp->get() or cursor->c_get() functions will return DB_KEYEMPTY. The historic behavior is still available, based on a new DB_INFO flag, DB_RENUMBER. Applications depending on the historic recno access method semantics should specify the DB_RENUMBER flag, no other change should be necessary. In previous versions of the DB library, adding a record more than one logical record past the current last record in the recno database caused the creation of the intermediate missing records as existing records with zero-length data. While the intermediate missing records are still logically created in DB 2.3, it is an error to attempt to retrieve them, and the get() and c_get() functions will return DB_KEYEMPTY. In previous versions of the DB library, attempting to retrieve a deleted record using the same cursor with which it was deleted returned DB_NOTFOUND. For consistency with the new recno access method functionality, such attempts now return DB_KEYEMPTY. See [3]db_open(3) and [4]db_cursor(3) for more information. These changes are NOT transparent to applications. 3. A new flag, DB_APPEND, has been added to the db->put(3) function in DB 2.3. This flag is applicable only to the recno access method. It permits applications to append to the database without knowing how many records are in it or the current last record number. The DB example program examples/ex_tpcb has been modified to use this functionality. See [5]db_open(3) for more information. 4. An additional argument has been added to the memp_fopen(3) function in DB 2.3. The mpool functions must be able to uniquely identify files in order that multiple processes sharing a file will correctly share its underlying pages. Normally, the mpool functions use the file's device and inode numbers for this purpose. On some filesystems, e.g., FAT or NFS, file device and inode numbers are not necessarily unique across system reboots. To support applications wanting to maintain a shared memory buffer pool across system reboots, where the pool contains pages from files stored on such filesystems, it is now possible to specify a unique file identifier to the memp_fopen() call, which the memory pool functions will then use to identify shared files. The DB access method functions have been modified to use this functionality, and DB databases in 2.3 will work transparently on such filesystems. See [6]db_mpool(3) for more information. This change is NOT transparent to applications using the db_mpool(3) interfaces directly. 5. The interface to the DB access method stat function (dbp->stat()) has been changed. Per-thread statistics are no longer returned, only statistics for the entire database are returned. In addition, the stat function has been changed to take an additional ``flags'' parameter. The only legal value for this parameter in DB 2.3 is DB_RECORDCOUNT, which causes the stat function to return a count of records in the tree without collecting other statistics. A number of additional items of information have been added to the information returned by the stat function, notably database configuration flags and other information normally specified to the [7]db_open(3) function when the database is created. See [8]db_stat(1) and [9]db_open(3) for more information. This change is NOT transparent to applications. 6. A number of new configuration options have been added to the [10]db_load(1) utility, allowing databases to be dumped and reloaded in different configurations. See [11]db_load(1) for more information. 7. The C++ API for DB has been reworked, and is now believed to be fairly close to its final form. There are still no manual pages for the C++ API, although we expect to have them together and released within a couple of weeks. The C++ files and directories in the source distribution are: db/cxx/ C++ API db/examples_cxx/ The example programs recoded in C++. db/include/cxx_int.h Internal C++ include file. db/include/db_cxx.h External C++ include file. C++ support is automatically built on Win32. To configure it under UNIX, specify --enable-cxx as a configuration argument (see the file db/build.unix/README for more information). Interface Changes introduced in DB 2.3.5: 1. It has come to our attention that there are versions of the native Solaris compiler that order bit fields inside of structures differently than other compilers, e.g., gcc. Some of the on-page B+tree structures contained bit fields, which meant that databases built using these compilers were incompatible with databases built on other architectures, e.g., a Solaris database could not be read on an x86 architecture which used gcc as its compiler. The DB 2.3.5 release fixes this problem. Unfortunately, depending on the Solaris compiler used, the change may cause DB version 2.3.5 on Solaris to be unable to read databases written by earlier versions of DB. If you have a support contract with Sleepycat Software, please contact us for further information and assistance in migrating databases from previous releases of DB to this release. Interface Changes introduced in DB 2.3.10: 1. In order to simplify the DB Java API, the type of the flags argument to the three functions memp_fget(), memp_fput() and memp_fset() has been changed from ``unsigned long'' to ``int''. It's possible, although unlikely, that this change could result in compiler errors for DB applications on some systems, if the flags were stored in a local variable. To fix any such errors, change the type of the local variable from ``unsigned long'' to ``int'' and recompile the application. Interface Changes introduced in DB 2.3.11: 1. The values (but not the names) of some of the DB interface flags were changed in the 2.3.11 release. DB applications should be recompiled after upgrading to 2.3.11. Interface Changes introduced in DB 2.3.12: 1. A new DB interface, db_jump_set(), has been added. The db_jump_set function enables applications to replace underlying DB library functionality by replacing entries in a function call jump table. This interface is intended for applications needing to do run-time linking to specific underlying libraries, e.g., for systems where the malloc(3) that the application will use is not known until run-time. The db_yield field previously found in the DB_ENV structure has been removed, and its functionality incorporated into the db_jump_set function. The removal of the db_yield field is NOT transparent to applications. Applications should be converted from using the db_yield field to calling the db_jump_set() function with the DB_FUNC_YIELD flag. If you have a support contract with Sleepycat Software, please contact us for further information and assistance in migrating applications from previous releases of DB to this release. See [1]db_jump(3) for more information. 2. A new memory pool interface, memp_trickle(), has been added. The memp_trickle function permits an application to ensure that a certain percentage of the pages in the shared memory pool remain clean at all times, so that there are always buffers available for reading in new pages from the backing files. See [2]db_mpool(3) for more information. 3. A new log region interface, log_stat(), has been added. The log_stat function returns statistical information about the log region. The db_stat(1) utility has been updated to optionally display this information. See [3]db_log(3) for more information. 4. There have been various additions to the memory pool statistics structure, as returned by memp_stat() and displayed by db_stat(1). Applications using the memory pool statistics interface will need to be recompiled for this release. See [4]db_mpool(3) for more information. Interface Changes introduced in DB 2.3.14: 1. There was a mismatch in previous releases of DB between the documented interface to lock_detect(3) (or DbLockTab::detect()) and the DB software. This mismatch was resolved in favor of the source code. Any software that uses this interface should be checked to make sure that the second argument is the "flags" argument, and the third argument is the "atype" argument. This change may NOT be transparent to applications. 2. There is an new DB interface, db_value_set(), described in [1]db_internal(3). The only purpose of the db_value_set() interface in the DB 2.3.14 release is to permit applications to set the number of spins that mutexes will make before blocking, when a mutex is not immediately available. This is intended to permit applications on multi-CPU architectures a finer granularity of control over mutexes. As with db_jump_set(), db_value_set() is only available from the C API, and is expected to be used by few applications. 3. We've added a Java API for DB. The Java API should be considered a beta release, and therefore more subject to change than normal. We would very much appreciate email on any comments and/or experiences you have in using the DB Java API! The Java files and directories in the source distribution are: db/java/src/com/sleepycat/db The Java API db/java/README Information on building the Java API. db/man/man.html/index_java.html The Java API manual pages (available only in HTML). db/java/src/com/sleepycat/examples/ The example programs recoded in Java. db/libdb_java Sources to build the libdb_java shared library. Berkeley DB 2.3.16 Change Log Interface Changes Introduced in DB 2.3.16: 1. The interface to db_jump_set(3) for the DB_FUNC_IOINFO value has changed in order to avoid passing off_t types as interface arguments. Any software that uses this interface must be updated to use the new arguments. See [1]db_internal(3) for more information. This change is NOT transparent to applications. 2. The output of log_stat(3) has been enhanced to include the current log file number and offset within that file. See [2]db_log(3) for more information. 3. A new option, -T, has been added to the db_load(1) utility. This option simplifies using non-DB applications (or scripts) to create DB databases. See [3]db_load(1) for more information. 4. Applications wanting to us the dbm, ndbm or hsearch interfaces to the DB library must now specify a numeric value when they #define DB_DBM_HSEARCH. For example, instead of using: #define DB_DBM_HSEARCH #include "db.h" in the application, they must now use: #define DB_DBM_HSEARCH 1 #include "db.h" This change is NOT transparent to applications. 5. DB no longer intrudes on the historic dbm(3), ndbm(3) or hsearch(3) namespaces. This change permits applications to load the DB library and still load the dbm, ndbm or hsearch functions from another library. 6. The dbm(3) functions are no longer exported in the Win32 environment. The db_jump_set(3), db_value_set(3) and log_stat(3) interfaces are now exported in the Win32 environment. 7. In the Java DB API, Dbt's used to retrieve data must specify Db.DB_DBT_MALLOC or Db.DB_DBT_USERMEM in the Dbt flags. If Db.DB_DBT_USERMEM is used, the data field of the Dbt must be set to an appropriately sized byte array. Bug Fixes: 1. When transaction undo was performed during recovery, the Btree access method could incorrectly recover pages that were never created because of the system or application failure. 2. When the last log file did not contain a checkpoint, recovery could potentially fail. 3. When files were opened and closed between checkpoints, catastrophic recovery could potentially fail. 4. When log file #2 existed but log file #1 did not, recovery would fail. 5. Shared memory regions were not being explicitly initialized in the Win95 environment, potentially resulting in unexpected behavior. 6. An incorrect shared memory region offset was being specified in the shared memory buffer cache support for systems using fcntl(2) locking (rather than spinlock mutexes). This could potentially lead to incorrect locking behavior in that subsystem. 7. The shared memory buffer cache had a deadlock situation when a buffer writer and a thread or process syncing the pool attempted to write the same buffer at the same time, potentially leading to thread/process starvation. 8. The Java DB API did not correctly interact with the Db.DB_DBT_MALLOC and Db.DB_DBT_USERMEM flags, and a number of memory leaks have been eliminated. Additional Changes: 1. There are a large number of source changes to the DB 2.3.16 release, intended to clean up compiler warnings that could appear when various debugging and warning options were specified to the gcc and Solaris compilers. 2. DB 2.3.16 has been modified to never pass as arguments or store on disk variables of type off_t. This change is intended to increase DB's portability to compiler/architecture combinations where the DB library and the application may not have been compiled with the same size off_t. 3. DB now uses the -O2 compiler optimization flag when building on Linux systems. 4. The shared memory buffer cache subsystem has been enhanced to no longer hold mutex locks across calls to fsync(2). 5. The shared memory buffer cache subsystem now explicitly yields the processor when waiting on a buffer for which I/O was being performed, enhancing overall throughput. 6. The test suite has been enhanced to better test recovery and to increase dbm/ndbm interface coverage. Berkeley DB 2.4.10 Change Log Interface Changes Introduced in DB 2.4.10: 1. Support has been added for architectures with 16-bit integers, largely to port Berkeley DB to the Windows 3.1 platform. This required extensive code changes, although almost all of them were only semantic in nature. In almost all of the DB public interfaces, including any public structure elements, all variables of type "int" or "unsigned int" have been converted to be either type "int32_t" or "u_int32_t". Previously built binaries on any 32-bit machine should be backward compatible, however, previously built binaries on machines with 64-bit integers will not be binarily compatible. Applications should recompile correctly without warning or error, unless there are explicit casts in the application source code. This change may NOT be transparent to applications. 2. Support has been added to allow Berkeley DB to use shared memory backed by a paging file or swap space (as opposed to backed by the regular file system) on both UNIX and Win95/WNT platforms. This required a major rework of the shared memory support in DB, and, for this reason, the DB version 2.4.10 release should, perhaps, be tested more thoroughly than usual in your local environment before committing to the upgrade. This change caused the [1]db_jump_set(3) and [2]db_value_set(3) functions to change. The [3]db_jump_set(3) function interface specified by the DB_FUNC_MAP flag has changed. A new interface, specified by the DB_FUNC_RUNLINK flag has been added. Applications replacing these functions at run time will require additional care in upgrading to this release of DB. The [4]db_value_set(3) function interface has two additional flags: DB_REGION_ANON and DB_REGION_NAME, which allow applications to specify regions are to be created when using memory backed by a paging file. See the [5]db_internal(3) manual page for further information. 3. The [6]db_jump_set(3) interface no longer supports the DB_FUNC_CALLOC and DB_FUNC_STRDUP flags. Applications may continue to set the functions associated with those flags, but they are no longer needed and no longer have any effect. 4. The [7]db_value_set(3) interface has an additional flag: DB_REGION_INIT, which permits applications to specify that regions should be page-faulted into memory immediately upon creation. This fixes a problem where the additional time required to page-fault in region memory caused locks to convoy inside of the shared memory buffer pool. This additionally required an interface change to the DB_FUNC_SEEK interface as specified by the [8]db_jump_set(3) function interface. The "relative" argument is now a "u_int32_t" instead of a "u_long", and there is an additional argument "isrewind" in order to support seeking in the reverse direction. See the [9]db_internal(3) manual page for further information. This change may NOT be transparent to applications. 5. The [10]log_put(3) function has an additional flag: DB_CURLSN, which permits applications to determine the next log LSN value which will be used. This, coupled with changes to the transaction subsystem to no longer write transaction-begin records allows Berkeley DB to avoid writing any log records in the case of read-only transactions. 6. The interface the the [11]hcreate(3) function has been changed to take a "size_t" as an argument instead of an "unsigned int", for compatibility with vendor platforms, notably Solaris. This change may NOT be transparent to applications. 7. Specifying NULL or zero-length keys to DB functions is now explicitly disallowed by most of the DB interfaces, and EINVAL will be returned. This change may NOT be transparent to applications. 8. Performance measurements of Berkeley DB indicated that a significant amount of time was being spent clearing pages created in the shared memory buffer pool. In order to fix this, additional information has been added to the [12]memp_fopen(3) interface which allows callers to specify the number of bytes which need to be cleared when pages are created in the pool. The default behavior of the pool code is unchanged, and by default the entire page is cleared. However, the [13]memp_fopen(3) function interface has changed: 4 of the lesser-used historic arguments and the new argument have been moved to an information structure of type DB_MPOOL_FINFO which is passed to the memp_fget function as an argument. See the [14]db_mpool(3) man page for more information. This change may NOT be transparent to applications. 9. An interface has been added to the [15]db_checkpoint(1) function allowing database administrators to force a single checkpoint. See the [16]db_checkpoint(1) man page for more information. Configuration and Build Changes: 1. The environment variables used to specify a compiler, loader or compile and loader flags during DB configuration have been changed to be more in line with standard GNU autoconf practice. The changes are as follows: Old Name New Name ADDCPPFLAGS CPPFLAGS ADDCXXFLAGS CXXFLAGS ADDLDFLAGS LDFLAGS ADDLIBS LIBS See the file build.unix/README for further information. 2. The DB debugging support (previously configured by specifying --enable-debug during configuration) has been split into two separate pieces of functionality. To build DB with -g as a compiler flag and with DEBUG #defined during compilation, enter --enable-debug as an argument to configure. This will create DB with debugging symbols, as well as load various routines that can be called from a debugger to display pages, cursor queues and so forth. This flag should probably not be specified when configuring to build production binaries, although there shouldn't be any significant performance degradation. To build DB with diagnostic, run-time sanity checks and with DIAGNOSTIC #defined during compilation, enter --enable-diagnostic as an argument to configure. This will cause a number of special checks to be performed when DB is running. This flag should NOT be specified when configuring to build production binaries, as you will lose a significant amount of performance. 3. Some systems, notably versions of AIX, HP/UX and Solaris, require special compile-time options in order to create files larger than 2^32 bytes. These options are automatically enabled when DB is compiled. For this reason, binaries built on current versions of these systems may not run on earlier versions of the system, as the library and system calls necessary for large files are not available. To disable building with these compile-time options, enter --disable-bigfile. 4. The autoconf configure script has been enhanced to correctly support --disable-feature or --enable-feature=no options. 5. The Berkeley DB test suite has been re-ordered to run system tests before the tests of the individual access methods. 6. The Berkeley DB test suite has been largely ported to the Windows 3.1 environment. See the distribution file test/README.win32 for more information. 7. Mutex lock support for the GCC compiler and the PaRisc architecture has been added. 8. The db_printlog(1) debugging utility is now built by default, and the [17]db_stat(1) utility is now built on Win95/WNT platforms. 9. A number of C, C++ and Java compiler warnings have been fixed. 10. Support has been added for the Siemens-Nixdorf Informationssysteme AG ReliantUNIX platform. 11. Ports of Berkeley DB to the HP MPE/iX and Windows 3.1 environments have been begun. They are not yet complete. B+tree Access Method Bug Fixes: 1. When performing btree range searches (i.e., using the DB_SET_RANGE interface, or using the DB version 1.85 compatibility interface), the Btree access method could incorrectly return already deleted records. 2. When using a cursor to delete key/data pairs from a btree database, it was possible to leave empty pages in the database. This did not cause incorrect behavior, however, it could lead to unexpected growth in the database. 3. When doing a partial put into a off-page (i.e., "overflow") record, the Btree access method could incorrectly store the data item. 4. When an off-page (i.e., "overflow") record was promoted to an internal page of a Btree database, DB applications could dump core when walking the tree on machines where a certain stack variable was not automatically initialized to 0. 5. When an off-page (i.e., "overflow") item was promoted into an internal page in a Btree database, DB applications reading databases on machines with different byte orders could incorrectly walk the Btree, potentially dumping core. 6. When Btree databases had record numbering turned on (the DB_RECNUM option), it was possible for the record numbers to fail to stay synchronized with changes in the database. Recno Access Method Bug Fixes: 1. When iterating through Recno access method databases, DB could incorrectly return EINVAL instead of the next key/data pair. 2. Previous releases of DB did not permit Recno access method databases to be opened read-only. 3. When reading the last record in a Recno access method database, DB could incorrectly return DB_NOTFOUND instead of the key/data pair. 4. When a key/data item was deleted using the cursor interface to a Recno access method database, a subsequent put using DB_NOOVERWRITE set would fail. 5. Because DB version 2 lazily instantiates the underlying shared memory buffer pool files, the DB->fd call for Recno databases in the DB 1.85 compatibility interface did not always work. C++ and Java API Bug Fixes: 1. When the C++ interface [18]Db::get() function was called, DB incorrectly treated the DB_NOTFOUND case as an exception instead of returning DB_NOTFOUND. 2. The DB Java interface now sets the CLASSPATH variable instead of depending on -classpath. This allows DB to work with the SGI JDK 3.0.1. 3. Using Dbc.next with the DB_SET or DB_SET_RANGE options in the DB Java interface would not correctly position the cursor. 4. The DB Java interface now works correctly with the Solaris JDK 1.2. Additional Bug Fixes: 1. Shared regions now work correctly for multiple processes on Windows/95 platforms. 2. It was possible under rare conditions to return illegal addresses when read-only databases were mapped into the process address space. 3. When a DB environment had already been specified, setting the "db_cachesize" field of the DB_INFO structure would be silently ignored by [19]db_open(3). In DB version 2.4.10, specifying a DB environment as well as setting db_cachesize is considered an error, returning EINVAL. 4. Several boundary conditions were identified and fixed in database recovery. 5. Key-not-found and errno returns were being set incorrectly in the dbm/ndbm interface compatibility functions. 6. Under some conditions, database files could still be synced to disk on close, even if the DB_NOSYNC flag was specified to the DB->close function. 7. Under some conditions, the db_archive utility output could contain multiple slashes in pathnames. 8. The db_archive utility would always prepend the current working directory pathname when the -a option was specified, regardless of the DB environment information indicating that the database home was at an absolute pathname. 9. Page create statistics were not correctly maintained when the DB_MPOOL_NEW option was specified to the shared memory buffer pool subsystem. 10. Under rare conditions, log offsets were not maintained correctly when switching between log files. Additional Changes: 1. The transaction ID space in Berkeley DB is 2^31, or 2 billion entries. It is possible that some environments may need to be aware of this limitation. Consider an application performing 600 transactions a second for 15 hours a day. The transaction ID space will run out in roughly 66 days: 2^31 / (600 * 15 * 60 * 60) = 66 Doing only 100 transactions a second exhausts the transaction ID space in roughly one year. In order to decrease the likelihood of exhausting the transaction ID space, Berkeley DB now resets the transaction ID each time that recovery is run. See [20]db_txn(1) for more information. 2. The limitations on database lifetime imposed by the Berkeley DB logging subsystem has been documented. See [21]db_log(3) for more information. 3. Performance measurements of Berkeley DB indicated that a significant amount of time was spent in searching hash queues in the shared memory buffer pools. The internal DB hash bucket code has been enhanced to handle much larger core memories than was possible previously, which should result in increased performance when using large memory pool caches. 4. Performance measurements of Berkeley DB indicated that a significant amount of time was spent in searching lock queues, and, specifically, in calculating hash values of lock queue elements. Fast hash functions specifically for DB's use have been added to the lock subsystem to alleviate this problem. 5. Performance measurements of Berkeley DB indicated that a significant amount of time was spent in determining the number of processors on the Windows/NT platform. The value is now determined initially and then stored. 6. Historically, the [22]db_appinit(3) function returned EINVAL if an application attempted recovery, by specifying DB_RECOVER or DB_RECOVER_FATAL, and there were no log files currently in the database environment. In the 2.4.10 release, this is no longer an error. 7. The mutex locking code has been enhanced to no longer attempt full test-and-set instructions unless there is a strong probability of acquiring the mutex. This makes spinning on the mutex significantly less expensive. 8. The count of attempted mutex locks (mutex_set_wait) has been changed to be incremented every time a process/thread is refused the lock, instead of only once when it first requests the lock and is denied. 9. The [23]db2_dump185(1) utility has been enhanced to dump DB version 1.86 databases as well as DB version 1.85 databases. 10. The [24]db_stat(1) utility has been enhanced to display values larger than 10 million in a new format, "###M". 11. The [25]db_stat(1) utility has been enhanced to display lock region statistics. See [26]db_stat(1) and [27]db_lock(3) for more information. Berkeley DB 2.4.14 Change Log Interface Changes Introduced in DB 2.4.14: 1. Enhance the [1]db_load(1) utility so that the default behavior is to add data to existing databases (rather than to always create the database), and to correctly perform locking for databases running in Berkeley DB environments. This allows the use of [2]db_load(1) to add key/data pairs to existing, active databases. Add a -n option to the [3]db_load(1). This option causes [4]db_load(1) to refuse to overwrite already existing key/data pairs in databases. 2. Enhance the [5]db_stat(1) utility to display additional information about the shared memory buffer pool and lock regions. The new options are -C[Acflmo] for the lock region, and -M[Ahlm] for the shared memory buffer pool region. This information is normally intended only for Berkeley DB performance tuning. Add a -N option to the [6]db_stat(1) This option keeps [7]db_stat(1) from acquiring region locks when reading shared regions, which allows [8]db_stat(1) to be used to display information about regions where applications have crashed while holding region locks. Add region reference counts and sizes to all [9]db_stat(1) region displays. 3. Add a new flag, DB_MUTEXLOCKS to the [10]db_value_set(1) interface. This flag causes requests for mutual exclusion mutexes to be immediately granted without test. This flag is only intended for debugging purposes. B+tree Access Method Bug Fixes: 1. Fix bug in the B+tree access method where additional cursors could be incorrectly updated when key/data pairs were added or deleted. 2. Fix bug where an aborted transaction during a B+tree split could cause pages to be left pinned. 3. Fix possible core dump when freeing the (cursor referenced) last item in the last page in a duplicate chain. 4. Fix possible core dump when deleting overflow items from duplicate pages. 5. Fix bug where duplicate page splits could result in database corruption (common to both B+tree and Hash Access Methods). Hash Access Method Bug Fixes: 1. Fix bug where Hash access method recovery did not always correctly recover key/data pair addition or deletion. 2. Fix bug where deleting a key in a Hash access method database could cause a cursor sequentially walking the database to skip records. 3. Fix bug where duplicate page splits could result in database corruption (common to both B+tree and Hash Access Methods). Shared Memory Buffer Pool Bug Fixes: 1. Fix bug where a DB thread handle could be left locked if a read-only file descriptor could not be upgraded to read-write during a buffer pool sync to disk. Common Shared Region Support Bug Fixes: 1. Fix bug where private shared memory pool regions could not be created on architectures without spinlock mutex support. 2. Fix possible file descriptor leak on HP/UX architecture (reports #150, #158). 3. Fix a variety of inconsistencies where, after a failed attempt to open shared regions, the code recovery paths were incorrect. Additional Bug Fixes: 1. Fix possible core dump in [11]lock_open(1) when it was called without first creating a Berkeley DB database environment. Test Suite, Example Program and Build Procedure Bug Fixes and Changes: 1. Fix build procedures for a number of architectures: affected systems include VC++ 5.0 and various releases of SunOS, AIX, OSF/1, FreeBSD and Linux (reports #145, #156). 2. Fix the Berkeley DB test suite to correctly identify Windows systems in all cases (the previous method failed for some releases of the Tcl scripting language). 3. Fix the ex_thread example program to correctly set the buffer cache size. 4. Fix the Berkeley DB test suite to ignore errors where System V shared memory interfaces (shmget(2)) have not been configured correctly. 5. Add support for the BSD/OS 4.0 release. 6. Add support for the SCO OpenServer Release 5 (3.2.2). (The test suite has not yet run successfully on this architecture, although we currently believe that the problems are in the test suite, and not in the Berkeley DB library itself.) 7. Change configuration to ignore the FreeBSD shmget(2) interfaces when ftok(3) is not available from the C library. 8. Change configuration to fail on the Nextstep architecture. (Previously, Berkeley DB would configure successfully, but then not build on Nextstep.) Additional Changes: 1. Return EINVAL if the DB_RECOVER or DB_RECOVER_FATAL flags were specified to [12]db_appinit(3) without also specifying the DB_INIT_TXN flag. 2. Removed "Additional Change" #7 from the Berkeley DB 2.4.10 release: The mutex locking code has been enhanced to no longer attempt full test-and-set instructions unless there is a strong probability of acquiring the mutex. This makes spinning on the mutex significantly less expensive. Full test-and-set instructions are necessary on some architectures, and it this change actually decreased performance in some cases. Berkeley DB 2.5.9 Change Log Interface Additions in Berkeley DB 2.5.9: 1. A new flag is available in this release for the Berkeley DB cursor key/data retrieval interface: DB_NEXT_DUP. This flag causes the DBcursor->c_get routine to return the next duplicate in a list of duplicates, and DB_NOTFOUND if there are no additional duplicates to return. 2. A new flag is available in this release for the Berkeley DB cursor key/data retrieval interfaces: DB_GET_BOTH. This flag causes the DB->get and DBcursor->c_get routines to return success only if both the specified key and data items match the entry in the database. 3. A new flag is available in this release for the Berkeley DB key/data retrieval interfaces: DB_RMW. This flag causes the DB->get and DBcursor->c_get routines to acquire write locks instead of read locks when doing the retrieval. Setting this flag may decrease the likelihood of deadlock during a read-modify-write cycle by immediately acquiring the write lock during the read part of the cycle, so that another thread of control acquiring a read lock for the same item, in its own read-modify-write cycle, will not result in deadlock. 4. A new flag is available in this release: DB_DUPSORT. This flag causes duplicate records to be maintained in sorted order. By default, the sort order is the same default lexical sort used by the Btree access method. A new field, in the DB_INFO structure passed to the db_open routine, is available in this release as well: dup_compare. This field is a sort function that is optionally used to sort duplicate data items. It is intended to allow applications to maintain duplicates in a non-standard sort order. 5. A new interface is available in this release, DB->join. This interface takes a set of Berkeley DB cursors as arguments, and returns a specialized Berkeley DB cursor whose get function performs a database join on the records referenced by the set of cursors. 6. A new field is available in this release, set in the Berkeley DB structure returned by db_open, DB->byteswapped. This field is set if the underlying database was not in the native host byte order, and can be used by the application to determine if its stored data will require host-order cleanups before use. 7. The dbmclose() interface has been added to the Berkeley DB dbm/ndbm compatibility interface, for application compatibility with the Sun Microsystems Solaris and other dbm/ndbm interfaces. Interface Changes in Berkeley DB 2.5.9: 1. Previous Berkeley DB releases have been inconsistent with respect to which DBT structure flags may be specified to which Berkeley DB interfaces. For example, calling DB->put with the DB_DBT_MALLOC flag specified makes no sense, and DB has been inconsistent historically as to whether this was treated as an error or simply ignored. As of this release, inappropriate flags in the DBT structure will simply be ignored. This is intended to make it easy to, for example, retrieve a key/data pair and then use the data DBT as the key DBT for another database without having to reinitialize the flags in the DBT. Previous Berkeley DB releases also required that threaded applications always set the DB_DBT_MALLOC or DB_DBT_USERMEM flags in DBT structures when retrieving key/data items. As of this release, specifying DB_DBT_MALLOC or DB_DBT_USERMEM is only required when using the non-cursor DB interfaces (e.g., DB->get). When using cursor interfaces (e.g., DBcursor->c_get), the flags are not required, as memory in which the key/data items are returned is allocated and maintained on a per-cursor basis. 2. Berkeley DB log files are now named log.XXXXXXXXXX instead of log.XXXXX, in order to ensure that applications do not run out of log file name space. This change is transparent to applications, but may NOT be transparent to local shell scripts and utilities. 3. Previous Berkeley DB releases returned statistics for the Btree databases that were only valid for the lifetime of the handle with which they were requested, i.e., these statistics as returned for a particular Berkeley DB handle would only reflect database operations done by that Berkeley DB handle and any cursors associated with it. The following statistics have been removed from the returned Btree statistical information: bt_freed, bt_pfxsaved, bt_split, bt_rootsplit, bt_fastsplit, bt_added, bt_deleted, bt_get, bt_cache_hit, bt_cache_miss. If any of these are sufficiently useful to application writers that they should be put back into the system, please let us know. This change is NOT transparent to applications. 4. Previous Berkeley DB releases did not support embedded white space in Berkeley DB environment configuration strings. As of this release, configuration NAME/VALUE strings are still separated by one or more whitespace characters (which are discarded), but the VALUE string may contain embedded whitespace characters and is terminated by trailing whitespace characters and a newline character, both of which are also discarded. In addition, empty lines and lines whose first character is a whitespace or hash (#) character, in the Berkeley DB configuration, file are discarded. This change is potentially NOT transparent to applications. 5. The DB_REGION_INIT flag to the db_value_set interface has been enhanced to write a byte to each page in the region. This allows applications to use DB_REGION_INIT to ensure that there is sufficient disk space for the backing region file. Berkeley DB Environment failures: There exists a class of errors that Berkeley DB considers fatal to an entire Berkeley DB environment. An example of this type of error is a log write failure due to the disk being out of free space. The only way to recover from these failures is for the application to exit, run recovery of the Berkeley DB environment, and re-enter DB. (It is not strictly necessary that the application exit, although that is the only way to recover system resources, e.g., file descriptors and memory, currently allocated by Berkeley DB.) In previous Berkeley DB releases, the only way an application could determine that a fatal error had occurred was to monitor Berkeley DB function return values, looking for unexpected ones, such as ENOSPC, or EPERM (which has historically been returned by Berkeley DB to indicate a potential underlying database corruption). As of this release, we have added a new error return value, DB_RUNRECOVERY. This error can be returned by any Berkeley DB interface. If a fatal error occurs, DB_RUNRECOVERY will then be returned from all subsequent DB calls made by any threads or processes participating in the DB environment. The EPERM error return no longer has a special meaning in Berkeley DB. Optionally, applications may also specify a fatal-error callback function by setting the db_paniccall field of the DB_ENV structure before initializing the environment with db_appinit (DbEnv::appinit). This callback function will be called with two arguments: the DB_ENV structure associated with the environment and the errno value associated with the underlying error that caused the problem. Applications can handle fatal errors in one of two ways: by checking for DB_RUNRECOVERY as part of their normal Berkeley DB error return checking, or, in applications that have no cleanup processing of their own, by simply exiting the application when the callback function is called. We would be very interested in any comments that you'd care to make on this interface change, in particular, any comments on the sufficiency of the interface for your Berkeley DB application. This change is NOT transparent to applications. Documentation Changes: 1. The Berkeley DB documentation has been completely reworked. It is no longer available in flat text, UNIX roff or PostScript formats, but is now only available in HTML format. To use the Berkeley DB documentation, point your browser to the Berkeley DB distribution or installation directory db-2.5.9/docs/index.html. This release also includes the beginnings of the Berkeley DB Reference Guide, as well as the manual pages. B+tree Access Method Bug Fixes: 1. Cursor delete operations were not necessarily being undone after deadlock, potentially leading to incorrect data. 2. Deleted, off-page duplicate items could be recovered incorrectly, potentially leading to incorrect data. 3. Log records could be written outside of a transaction under some circumstances, potentially corrupting the log so that recovery would fail. 4. Completely emptying large trees could cause corruption of the database root page during the final reverse split. 5. Failure during page split could leave cursors referencing incorrect data. 6. Retrieving records based on logical record number could return incorrect data if logically adjacent records had previously been deleted. 7. The Btree access method is more aggressive in this release about discarding locks within transactions that are not needed for correctness. This change significantly decreases the probability of deadlock for some applications. Hash Access Method Bug Fixes: 1. Storing duplicate data items using the DB_CURRENT flag could result in incorrect data. 2. Cursors and their locks did not always return unchanged on operation failure. 3. Entering a sufficient number of duplicate data items into the database could result in incorrect cursor positioning and/or a corrupted database. Recno Access Method Bug Fixes: 1. The logical record number returned from DB_APPEND calls was stored into library memory instead of into the user-specified memory. 2. The memory in which the backing source file name was stored could be freed multiple times, potentially leading to application core dump. General Access Method Bug Fixes: 1. Incorrectly treated setting the database cachesize in the presence of a Berkeley DB environment to be an error, even if the environment didn't not initialize a shared memory buffer pool. 2. Using the DBcursor->c_get interface with the DB_KEYFIRST or DB_KEYLAST flags to insert a new key into the database would fail. 3. In previous Berkeley DB releases each cursor operation (when not part of a transaction) potentially used a different locker ID, making it possible for cursor operations to lock against themselves. In the 2.5.9 release, the cursor locker ID is maintained for the life of the cursor, instead. 4. The optional user-specified transaction recovery function was not being called during Berkeley DB recovery, and when using the db_printlog utility. 5. Pages of duplicate data items were incorrectly split and logged/recovered, potentially leading to database corruption. 6. During recovery, deleted database files could cause recovery to fail. C++ API Changes and Bug Fixes 1. A DbEnv::version method has been added to allow access to major, minor and patch numbers for the current version. 2. The DbEnv class has been cleaned up so that inappropriate get and set methods have been removed: DbEnv::get_data_cnt DbEnv::get_data_dir DbEnv::get_data_next DbEnv::get_flags DbEnv::get_home DbEnv::get_log_dir DbEnv::get_tmp_dir DbEnv::set_data_cnt DbEnv::set_data_dir DbEnv::set_data_next DbEnv::set_flags DbEnv::set_home DbEnv::set_log_dir DbEnv::set_tmp_dir These methods are unneeded because the constructor with arguments, or the appinit() method, can be used to set this information. DbEnv::get_errcall DbEnv::get_errfile DbEnv::get_error_model DbEnv::get_error_stream DbEnv::get_errpfx DbEnv::get_lg_max DbEnv::get_lk_conflicts DbEnv::get_lk_detect DbEnv::get_lk_max DbEnv::get_lk_modes DbEnv::get_lorder DbEnv::get_mp_mmapsize DbEnv::get_mp_size DbEnv::get_tx_max DbEnv::get_tx_recover DbEnv::get_verbose These get methods accessed information that was never set by Berkeley DB. 3. Remaining DbEnv::set_* methods may throw an exception if they are called after the environment has been initialized (either via appinit or the constructor with arguments). 4. The DbInfo class has been reworked so that inappropriate get methods have been removed. These get methods accessed information that was never set by Berkeley DB. DbInfo::get_bt_compare DbInfo::get_bt_maxkey DbInfo::get_bt_minkey DbInfo::get_bt_prefix DbInfo::get_cachesize DbInfo::get_flags DbInfo::get_h_ffactor DbInfo::get_h_hash DbInfo::get_h_nelem DbInfo::get_lorder DbInfo::get_malloc DbInfo::get_pagesize DbInfo::get_re_delim DbInfo::get_re_len DbInfo::get_re_pad DbInfo::get_re_source Methods to get and set underlying lock identifiers in a DbLock have been removed, as lock identifiers should be completely opaque to the application. Java API Changes and Bug Fixes 1. The DB_SET_RANGE flag did not correctly return data items. 2. Db.stat() is now declared to return an Object. The object returned is of type DbBtreeStat if the file was created using Db.DB_BTREE. In the future, this will return other types, e.g., DbHashStat. The DbBtreeStat, DbLockStat, DbMpoolFStat, DbMpoolStat and DbTxnStat classes have been changed to allow direct access to their data members. DbLogStat is a new class. 3. The DbEnv class has been reworked and all inappropriate get and set methods have been removed: DbEnv.get_data_cnt DbEnv.get_data_next DbEnv.get_flags DbEnv.get_home DbEnv.get_log_dir DbEnv.get_tmp_dir DbEnv.set_data_cnt DbEnv.set_data_next DbEnv.set_flags DbEnv.set_home DbEnv.set_log_dir DbEnv.set_tmp_dir These methods are unneeded because the constructor with arguments, or the appinit() method, can be used to set this information. DbEnv.get_errcall DbEnv.get_errpfx DbEnv.get_lg_max DbEnv.get_lk_conflicts DbEnv.get_lk_detect DbEnv.get_lk_max DbEnv.get_lk_modes DbEnv.get_lorder DbEnv.get_mp_mmapsize DbEnv.get_mp_size DbEnv.get_tx_max DbEnv.get_verbose These get methods used to access information that was never set by Berkeley DB. The DbEnv.get_java_version_string method has been removed, and the Java part of Berkeley DB no longer maintains its own version information. 4. Remaining DbEnv.set_* methods may throw a DbException if they are called after the environment has been initialized (either via appinit or the constructor with arguments). 5. The DbInfo class has been reworked so that inappropriate get methods have been removed. These get methods used to access information that was never set by Berkeley DB. DbInfo.get_bt_maxkey DbInfo.get_bt_minkey DbInfo.get_cachesize DbInfo.get_flags DbInfo.get_h_ffactor DbInfo.get_h_hash DbInfo.get_h_nelem DbInfo.get_lorder DbInfo.get_pagesize DbInfo.get_re_delim DbInfo.get_re_len DbInfo.get_re_pad DbInfo.get_re_source 6. Methods to get and set underlying lock identifiers in a DbLock have been removed, as lock identifiers should be completely opaque to the application. 7. The DbRunRecoveryException class has been added as a subclass of DbException. A DbRunRecoveryException object will be thrown when a fatal error occurs in Berkeley DB, requiring recovery to be performed. Shared Memory Buffer Pool Subsystem Bug Fixes: 1. It was possible for threads opening and closing databases in a fairly full buffer cache to free memory that was still in use, resulting in application failure. 2. If the memp_trickle() interface was unable to find a single buffer to flush in the entire buffer list, it would return with the shared memory region mutex locked. 3. Opening underlying files of certain sizes in the buffer pool would incorrectly fail. Locking Subsystem Bug Fixes: 1. If the a locker being forced to wait does not currently hold any locks, the deadlock detector is no longer run. Logging Subsystem Bug Fixes: 1. If the first log get call after database recovery used the DB_NEXT flag, it would fail. 2. If databases were opened multiple times without intervening closes, recovery could fail. 3. A memory leak in the log_archive interface has been fixed. Additional Bug Fixes: 1. Windows/NT: using shared anonymous memory did not work correctly between processes sharing the database. 2. Utilities: Signal handling and Berkeley DB region exit was incorrect. Among other issues, the db_load utility could exit holding a region mutex. 3. Dbm/Ndbm: System error values were not always being correctly returned. System Porting and Build Procedure Changes: 1. A alpha-release port to VMS has been added to the Berkeley DB distribution. The port has not yet run the Berkeley DB test suite, but there are no known problems. 2. Berkeley DB now uses the pstat_getdynamic(2) interface on Hewlett-Packard HP/UX systems to detect the presence of multiple processors. 3. For performance reasons, the Berkeley DB release now uses the Sun Microsystems Solaris pread(2) and pwrite(2) UNIX interfaces, if they are available. 4. Berkeley DB now compiles with the -D_THREAD_SAFE C preprocessor flag and loads with the libc_r.a C library by default on FreeBSD systems. 5. For portability reasons, shared memory segments allocated using the UNIX shmget(2) function are now allocated as IPC_PRIVATE. (Apparently, marking them as IPC_PRIVATE does not affect that they are available to other processes.) 6. The standard UNIX install for the Berkeley DB library now installs Berkeley DB into its own hierarchy instead of into separate local directories. By default, the install locations are: Location Contents /usr/local/BerkeleyDB/bin binaries /usr/local/BerkeleyDB/include include files /usr/local/BerkeleyDB/lib libraries /usr/local/BerkeleyDB/docs HTML documentation 7. For portability reasons, the standard UNIX Berkeley DB library archive is built with the -cr options in this release, instead of the -cq options as done previously. 8. The standard UNIX Berkeley DB configuration will now automatically detect and use gcc if no compiler named cc is found. 9. When the C pre-processor DIAGNOSTIC value is #defined, memory is overwritten with a 0xdb pattern instead of a 0xff pattern. Additional Changes: db_dump 1. The db_dump utility has a new option, -N. This option allows db_dump to be run without acquiring any shared region mutexes. This option is intended for debugging use only. 2. The db_dump utility now allows a Berkeley DB environment directory to be specified (the -h option) at the same time as the "debugging output" option (the -d option). 3. The db_dump utility now uses the shared memory buffer pool region if a Berkeley DB environment directory is specified, which allows users to see the current state of the database instead of only the database state that has already been flushed to disk. Berkeley DB 2.6.4 Change Log Interface Additions in Berkeley DB 2.6.4: 1. Berkeley DB now supports a new mode of operation: Concurrent Access Methods. This configuration supports multiple-reader, single-writer access without requiring deadlock detection and transaction protection. The new interface is specified by using the DB_INIT_CDB flag to the db_appinit() function. In addition, the DB->cursor interface has been modified to take a fourth argument, a flags argument. This change will require that all applications using the DB->cursor interface be modified and recompiled. (The necessary modification is trivial, simply add a final argument of 0 to the DB->cursor call.) This change is NOT transparent to applications. 2. In previous Berkeley DB releases, cursors in Recno databases with mutable record numbers had to be re-positioned each time they were used to delete a record. This is no longer the case, and all of the records in the database logically at and above the cursor may now be deleted by repeatedly calling the delete function after positioning the cursor once. This change causes Berkeley DB practice to conform to the documentation found in previous releases. This change is NOT transparent to applications. 3. Berkeley DB no longer permits applications to specify database page sizes that are not powers-of-two. This change fixes a bug which could cause application failure, as page alignments were not necessarily correct when files with page sizes other than powers-of-two were mapped into process memory. This change is NOT transparent to applications. B+tree Access Method Bug Fixes: 1. Fix a bug which could cause application failure or corrupted data when database operations would split a Btree page and then subsequently fail or attempt to physically delete records after being restarted. 2. Fix a bug which could cause application failure when deleting records failed due to a deadlock. 3. Fix a bug which could cause a NULL pointer dereference when the relinking of duplicate data item pages failed due to deadlock. 4. Fix a memory leak where an internal cursor could be left open when databases were opened. 5. Fix a single-byte memory leak when the standard DB->del() interface was used to delete key/data pairs with duplicate data items. Hash Access Method Bug Fixes: 1. Fix a bug which could cause application failure or data corruption when deleting on-page duplicate items. 2. Fix a memory leak when closing cursors. Recno Access Method Bug Fixes: 1. Cursors that referenced deleted records were not correctly updated when the record was overwritten with new data, and would continue to return that the record was still in a "deleted" state. 2. Fix a bug where application specified read-modify-write semantics were ignored. 3. Remove lock-coupling behavior in tiny databases to avoid perpetual deadlock. 4. Fix a bug where root pages were not always correctly write-locked. Java API Changes and Bug Fixes 1. Fix compile include directory ordering, which could cause Java to be compiled using the wrong db.h include file. General Access Method Bug Fixes: 1. Fix a bug which left the read-modify-write semantics in place after the operation for which the application specified it completed. 2. Fix a bug which caused application failure if a NULL key argument was specified for the DB_AFTER, DB_BEFORE or DB_CURRENT flags to the DBcursor->get() interface. 3. Change the initial mutex timer back-off from 10ms to 1ms. Shared Memory Buffer Pool Subsystem Bug Fixes: 1. Fix a bug in large database files (>4Gb), which could potentially cause data corruption when a database was closed, reopened and new pages were then allocated. 2. Modify the memp_sync() interface code to no longer hold Berkeley DB region locks across memory allocation calls (e.g., malloc(3)). This change significantly increases throughput in applications checkpointing large memory caches. 3. Modify the memory pool qsort(3) comparison function to work when called by badly written qsort(3) functions. Additional Bug Fixes: 1. Fix a bug in the Berkeley DB error functions where variable numbers of arguments were not correctly accessed on some architectures, e.g., the PowerPC. System Porting and Build Procedure Changes: 1. Support installation into prefix directories for which multiple directory components do not yet exist. 2. Support installation into different system directories on a per-type basis, e.g., all the binaries to one directory, and include files to another. 3. Fix a bug where the supplied portability routine memcpy() was not correctly compiled. 4. Modify auto-configuration to refuse to configure in the top-level distribution directory. Berkeley DB 2.6.5 Change Log Berkeley DB version 2.6.5 is the GA release of version 2.6.4. It is the new stable release of Berkeley DB, replacing version 2.4.14. Berkeley DB 2.7.1 Change Log New Features: 1. This release includes support for nested transactions. (No interface changes were required.) System Porting and Build Procedure Changes: 1. FreeBSD systems are now built using the flags -D_THREAD_SAFE and -pthread instead of directly loading with the -lc_r library. 2. Linux systems are now built using the -D_REENTRANT flag. Other Changes: 1. The default memory pool cache size has been increase from 128K to 256K. Berkeley DB 2.7.3 Change Log Interface Additions and Changes: 1. This release includes support for nested transactions. No interface changes were required. 2. This release includes support for Berkeley DB as an XA Resource Manager. There is an additional C interface, db_xa_open, and a corresponding C++ method, Db::xa_open. XA support is not yet fully tested. No Windows XA interface is included. Configuration and Build Changes: 1. FreeBSD systems are now built using the flags -D_THREAD_SAFE and -pthread instead of directly loading with the -lc_r library. 2. Linux systems are now built using the -D_REENTRANT flag. Additional Changes: 1. The default memory pool cache size has been increase from 128K to 256K. Additional Bug Fixes: 1. Freeing memory allocated by the TCL library historically failed on Windows platforms, due to a mismatch of malloc/free implementations between the TCL library and the Berkeley DB module. (This change only affects the Berkeley DB test suite.) Berkeley DB 2.7.7 Change Log Berkeley DB version 2.7.7 is version 2.7.5 with a set of specific bug fixes applied. There were no public interface changes or new features. Bug Fixes: 1. Fix potential races in updating checkpoint buffer counts that can cause checkpoint calls to never finish. 2. Fix for XA support, allows two-phase commit processing to work. 3. Fix a recovery bug when database files are opened/closed multiple times in the same session. 4. Change file descriptor usage to permit Sendmail's fcntl(2) locking scheme. 5. Fix ANSI C++ usage to avoid GNU gcc-2.95 warning messages. 6. If there are a sufficient number of threads competing for limited numbers of pages, it's possible to split Btree pages too many times, and cause a core dump. 7. Fix a case where it was possible for EAGAIN to not be returned from the database get-by-key interface. 8. Ignore log records not involved in transactions so that actions taken outside of transactions are not undone during recovery. 9. Fix a memory pool race during buffer discard in memory tight environments. Other Changes: 1. Update the Berkeley DB release version numbers from 2.7.5 to 2.7.7.