I.R.I.S. Working Group

Interoperable Multimedia Retrieval in Distributed Systems

MPEG-7 Multimedia Database (MPEG-7 MMDB)


Multimedia Database Systems (MMDBMS) organize and store multimedia data for content retrieval. These systems rely on multimedia data models representing high- and low-level abstraction of media objects for facilitating various operations (e.g., insertion, indexing, querying or retrieval). Many models have been proposed in the past which reflect the needs of database users and developers. However, these models reveal important shortcomings: they are either limited by the use of one kind of multimedia data (e.g., only images are supported) or by the capacity of semantic modelling (e.g., only a keyword description of the content may be entered). This is as astonishing, as there has been recently published a new standard for describing the content of different types of multimedia data that offers richer semantics than existing systems. It is MPEG-7, the first standard of the Moving Picture Experts Group not dealing with coding exclusively. The reason for this re-orientation is quite straightforward, coding has reached a more or less satisfactory state (see MPEG-4), but yet we are not able to locate multimedia by standardized content. As mentioned above, the benefits of MPEG-7 have not yet reached MMDBMS. In this context, we present a Multimedia Data Cartridge (MDC) that maps the MPEG-7 standard into a database model.


Besides the more effective modelling of multimedia content, access and retrieval has to be considered. We need an efficient support for common multimedia retrieval functions, like similarity search, semantically meaningful queries and browsing in an MMDBMS. An important tool to guarantee efficient query processing is the indexing mechanism for commonly used datatypes. Innately, most database systems provide only a limited number of integrated access methods such as B-tree or hashing facilities. These techniques limit the use of database systems in multimedia. Available Object-Relational DBMSs (ORDBMS) provide some multimedia database extension packages such as DataBlades (IBM-Informix) or Extenders (IBM-DB2). They rarely handle indexing of d-dimensional data (d>2) and advanced similarity-search functionality, like a k-Nearest-Neighbor (k-NN) Search which is common for multimedia retrieval. Indexing feature vectors must be a core tool in a MMDBMS, as it is not unusual to index images with a feature vector dimensionality higher than 64 in. In this context several access methods for high dimensional data have been proposed, e.g., SS-tree, SR-tree, M-tree, X-tree or TV-tree. Unfortunately, these access methods are rarely available in common DBMS, nor are extensions proposed to integrate them. We are only aware of an extension package for Informix by M.Kornacker. This work integrates an adapted GiST framework into the Informix Dynamic Server with Universal Data Option (IDS/UDO). However, the framework does not supply a multimedia support (e.g., data model and index integration).

Being aware of the shortcomings of related approaches, we designed a methodology for enhancing extensible ORDBMS. In particular, we implemented an MPEG-7 Multimedia Data Cartridge (MDC)(see figure). Our system relies on the capacities of leading DBMSs (Oracle, IBM DB2 and IBM Informix, MSSQLServer) to provide services to extend their management system. Oracle's extension services were selected for the practical implementation, due to their availability (see http://technet.oracle.com). The multimedia extension created are, however, representatives for other DBMS extension services like DataBlades from Informix or Extenders from IBM. Our MDC enhances the Oracle type systems with an MPEG-7 compliant type system and integrates a new indexing system for multimedia retrieval and queries. This indexing system is supported by an external framework (Multimedia Indexing Framework (MIF)).

Multimedia Data Cartridge (MDC)

We have decided to use Oracle`s Data Cartridge technology for the implementation of our multimedia extension called Multimedia Data Cartridge (MDC). The MDC environment currently consists of three parts (see figure below).

The internal area (1) is composed of a multimedia database schema based on MPEG-7, the query optimizer and an indexer.

The second part (2) provides an internal set of libraries for incoming requests. These libraries support for instance inserting of MPEG-7 instance documents, their deletion or a query abstraction.

The third part (3) consists of several application libraries which are interfaces between applications and MDC. At the moment there exists following application libraries: BlobworldLib for a content based image retrieval application (Blobworld), AudioLib for a audio recognition tool and ProxyLib for a quality based intelligent MPEG-4 proxy (QBIX).
The fourth part (4) is the Multimedia Indexing Framework which provides an extensible indexing environment for multimedia retrieval. For instance, we may use an SS-tree or SR-tree for indexing a high-dimensional color histogram. Another possibility is the use of an LPC-file for an NN-search in high-dimensional spaces, or a V-tree for spatial queries. These are only few possibilities which are useful and possible.

Mdc Architecture Numbered New


SoftwareDescriptionAdditional Information
Multimedia Indexing Framework for Oracle (GistService) This framework enhances Oracle s indexing facilities by an adapted GiST framework and provides a rich set of access methods (SR-tree, M-tree, etc.) and predefined retrieval operations such as nearest neighbor or range queries for multidimensional feature vectors.Install Documentation (german)
Technical Documentation (english)

Team Members

Harald KoschUniversity KlagenfurtTeam Leader
Mario DoellerUniversity KlagenfurtMDC development (PhD Thesis)
Reinhold PaulitschUniversity KlagenfurtBlobworld integration (Master Thesis)
Holger CrysandtRWTH AachenAudio Recognition
Bernhard DoerflingerUniversity KlagenfurtInsertion of MPEG-7 documents into MDC (Master Thesis)
Alexander BachlechnerUniversity KlagenfurtMapping of MPEG-7 standard to a database model (Master Thesis)
Kerstin RennerUniversity PassauInstaller for the Multimedia Indexing Framework

Oracle Data Cartridge Technology

The extension package provided by Oracle is called Data Cartridge. Oracle databases are build as a modular architecture with the extensible services described previous. The usual way for using Oracle's Extensibility Services is to implement a Data Cartridge that extends the extensibility interface.

Oracle adds support for new types including user-defined objects, collections (e.g. VARRAY) or internal large object types (e.g., BLOB or XMLType).

Oracle's server execution environment provides two main advantages. First, the components of a data cartridge and other database procedures and functions can be implemented in any popular programming language such as PL/SQL, Java or external C language routines. Second, the type system decouples the implementation of an object's method from its specification. The specification just defines the head of the method with appropriate in and out parameters. It is not defined what kind of implementation is behind the object's method.

Further Oracle introduces the concept of an indextype for user-defined access methods. Each indextype has specific operators and uses an object that is responsible for the implementation. This object has to implement all necessary functions which are provided by ODCI (Oracle Data Cartridge Interface) for indexing e.g., ODCIIndexCreate or ODCIIndexInsert.

Data Cartridge Architecture