Database Module(from miGAPlib)

Database Module (DB)

Introduction

The database is a common repository for data. The most important application is the scene data, but it can keep auxiliary data such as function declarations and frame buffers. The DB module has calls to create a database item, to destroy an item, to get a pointer to an item (access), to give back (unpin) a pointer, and to write back changes into the distributed database after writing (flush). All items have a unique identifier, called a tag, that is assigned by the DB module when an item is created. All other functions reference items through tags. Tags, unlike pointers, are unique on the network and can be stored in the database as part of items, thus allowing users to grow trees of items.

An item is retrieved from the database by requesting a pointer to the data. The DB module makes sure that the requested item is in local memory, pins it, and returns a pointer to it. Pinning means that the DB module retains the item in memory until it is explicitly unpinned. One of the fundamental design goals is that data never exists twice in the same address space. All changes made to the item by using the returned pointer are immediately visible to all others in the local address space. It is the supervisor's responsibility to make sure that no conflicts occur, such as modules changing the scene while it is being rendered. (The supervisor is the module that started the whole operation, usually a translator).

Creating an item and requesting a pointer as described above implies pinning the item in memory, so the returned pointer is guaranteed to remain valid. While an item is pinned, the DB module will not destroy or move the item to make space for other items coming in from the network. If the pointer is used to change the item (which is usually the case after creating it), the item must be flushed explicitly after the change is complete. Flushing means that all other hosts on the network who have a copy of the item are notified to delete the item from their local memory, or re-read it if it was pinned locally. The DB module does not offer locking functions. If a write to the database must be done atomically, messages must be used to implement a lock.

DB Calls

char *mi_db_version (void): Return a version string.

void mi_db_init (void): Initialize the database on startup. This must be done before starting threads with mi_par_run because the database may only be initialized and exited once at any time.

void mi_db_exit (void): Delete the entire database. This is done before shutdown.

void *mi_db_create (miTag * const tag, const int type,: const int size) Create a data item with the specified size, and return a nonzero tag and a nonzero address (if size is 0, one byte is allocated to keep the item warm). Creating implies pinning. Calls mi_fatal and aborts on failure. The DB module always initializes new items with null bytes. The SCENE module offers calls to create certain database items, which must be used when available to create items with defaults. No module other than SCENE may call mi_db_create directly. The type must be specified for mi_db_type, and for debugging purposes. It must be between 1 and 255. See SCENE for type creation.

miTag mi_db_copy (const miTag tag): Create a duplicate of tag and return the tag of the duplicate. The duplicate has the same type and size, and the same contents as the source tag tag. The duplicate is not accessed, no unpin is required.

int mi_db_type (const miTag tag): Returns the type that was specified when the item was created. This is done when an item needs to be byte-swapped; every type is swapped in a different way. Type 0 is illegal. See the type list miScene_types in mi_scene.h for a list of predefined internal database item types. This call returns 0 if the given tag does not exist.

miInfo *mi_db_accessinfo (const miTag tag): Like mi_db_access, but returns a pointer to the miInfo structure of the item instead of the database item address. The item address can be read from the info structure. The miInfo structure describes the status of the database item and is shared between DB and SCENE, for efficiency reasons both use the same data structure. This call is intended for use by the SCENE module only to ``park'' its extra item information. The caller may read but not modify DB fields.

void *mi_db_access_type (int * const type,: const miTag tag) Like mi_db_access and mi_db_type rolled into one. This is useful in cases where both the address and the type are required and two separate calls would be too slow. The returned type actually has a data type miScene_type but is declared as an integer so mi_scene.h does not need to be included before mi_db.h.

void *mi_db_realloc (const miTag tag,: const int size) Changes the size of an existing database item. The resized items will be unchanged up to the lesser of the old and the new size. If the new size is greater than the old size, the remainder is zeroed. A pointer to the resized item is returned; the caller may not use the old pointer it may have gotten from an earlier mi_db_create or mi_db_access on the same database item. If size is 0, one byte is allocated and a valid pointer to it is returned. Returns 0 on failure. Like mi_db_access, reallocation increments the pin count, and there must be a mi_db_unpin for every reallocation. After finishing with the reallocated item, the caller must flush the item to inform other hosts of the change. This call is intended for the SCENE module only. All other modules should call mi_scene_growedit.

void mi_db_delete (const miTag tag): Delete the item referenced by the tag. Modules that have the item pinned are seriously out of luck because their pointers become invalid. The item is deleted on the local host and on all other hosts. The caller should make sure that no other host still has this item pinned; if one does that host generates a warning. This call is intended for the SCENE module only. All other modules should call mi_scene_delete.

void mi_db_delete_local (const miTag tag): Remove the item from local memory if it is a cached copy that is owned by another host. In this case, the next mi_db_access call is forced to re-read it over the network. This is intended to reduce flushing overhead: if an entire subtree has changed, the module that did the change could notify all hosts that the subtree has changed, rather than doing a mi_db_flush for every item in the subtree. The notified hosts would delete all their local copies of items in the subtree without further network traffic. Deleting locally implies unpinning.

void *mi_db_access (const miTag tag): Look up the tag in the database, pin it, and return a pointer to the referenced item. This is an extremely popular function that must execute as fast as possible. mi_fatal is called and the program aborts if the given tag does not exist. mi_db_access always returns a valid pointer. If an item is accessed twice, it must be unpinned twice; pinned is a counter, not a flag. Translators that intend to modify an item must call mi_scene_edit instead; it is equivalent but sets a dirty flag for later scene proprocessing, and may create overlay buffers for concurrent incremental updates.

void mi_db_flush (const miTag tag): Notify all other hosts that have a copy of the item to invalidate or re-read their copy. Flushing is necessary after the caller finished writing to an entry. If the item was just created using mi_db_create, it is not necessary to flush it because no other host can possibly have a copy, because the tag is only known to the creating routine. If tag is 0, no item is flushed, but all deferred flushes (if any) are propagated to other CPUs. Note that flushes of items created on another host are less efficient than flushes of items created locally. The mi_scene_edit_end call flushes implicitly.

void mi_db_deferredflush (const miTag tag): This call is equivalent to mi_db_flush, except that the tag may be buffered locally and sent as a block if either the buffer fills, or a regular mi_db_flush is called. If, for example, all materials are changed, it is far more efficient to call mi_db_deferredflush for each material followed by a single mi_db_flush(miNULLTAG) than to call mi_db_flush for each material.

void mi_db_unpin (const miTag tag): Unpin an item. The pointer returned by mi_db_access or mi_scene_edit may no longer be used. If the last pin is removed, the DB module is free to re-use the memory, provided that it is not the owner of the item (i.e., this is not the host where the item was created). This last restriction ensures that at least one copy of the item remains on the net of DBs. It is not allowed to unpin an item that is not pinned; development versions will abort with a failed assertion and production versions will silently ignore the extra unpin.

int mi_db_vpu_of_tag (const miTag tag): Extract the VPU from a tag. The returned VPU identifies the host where the tag was created initially, and where it still resides in local database memory. This call can be used to distribute tasks preferentially to hosts that already have the database items (eg., boxes) required by the task. The VPU number can directly be used for calls to the TASK module.

int mi_db_dump (const miModule module): This is a debugging function. It dumps all known database items to mi_info(), with host ID, tag number, address, type, number of pins, and the module that created the database item. If the listed address is 0, the item is not in local memory. If module is nonzero, the listing is restricted to database items created by that module; if module is 0, all database items are listed.

int mi_db_statistics (void): Print statistics about database activity since the last statistics printout, or since the program was started. In particular, time spent waiting for message replies from other hosts is reported. This call resets the statistics to 0 after reporting.

Examples

The first example assumes that there is a number of source materials to translate and store into a virgin database. The data types used do not necessarily agree with actual Scene module types, and errors must be handled differently than shown here. The flush is not strictly necessary because the tag has just been created and can not be known on other hosts yet, so other hosts do not have anything they need to flush.

    miTag            *tag;
    Source_material  *src;
    Raylib_material  *tar;

    for (src=first; src; src=src->next) {
            tar = mi_scene_create(&tag, miSCENE_MATERIAL);
            translate(tar, src);
            mi_db_deferredflush(tag);
            mi_db_unpin(tag);
            store_tag_in_parent_item(tag);
    }
    mi_db_flush(miNULLTAG);

The next example changes a material. It is assumed that the routine came into possession of the tag that identifies the material (it was probably found in another database item of type object):

    Raylib_material  *mtl;

    mtl = mi_scene_edit(tag);
    mtl->whatever = 42;
    mi_scene_edit_end(tag);

If the material had been read, not written, the call to mi_scene_edit_end would have been replaced with mi_scene_unpin.