In addition to the standard numeric,
date and text data types, there are a number of data types that
are regarded as the basic building blocks of MM applications.
These data types, which are elements of more complex MM objects,
- different fonts and to produce special effects such as colour
- various audio file formats include Microsoft WAV (wave) and
MIDI, which is a more compact representation of sound.
- still images
- pixels can be 0 or 1 ('white' or 'black') or hi-res colour
images with 8, 16 or 24 bits per pixel.
- digital video
- usually stored as a sequence of frames. For realistic playback,
the transmission, compression, and decompression of digitized
continuous frames requires transfer rates of 30 frames per second.
If audio is required as well, the audio and video must be interleaved
so that the sequences can be timed properly. Microsoft's AVI
format can synchronize playback of audio and video.
- graphical objects
- such as 2- and 3-dimensional images.
Most of these data types require a lot of
storage space. An average page of text may require about 2 KB;
75 minutes of high-fidelity music may need 100MB; a full page,
still image varies from about 10KB for black-and-white to several
megabytes for colour; a video frame may require 1 MB of storage
so a video clip lasting a second needs something of the order
of 3 0MB. Compression techniques are available to reduce these
storage requirements but it will be clear that MM still needs
a lot of storage space.
MMDBs require all the basic attributes of
a database management system such as a transaction manager, query
optimizer, recovery manager etc. as well as special storage structures
and specialized search and querying modules.
Typical architecture of a MMDB application
Since existing relational and OO databases
comprise the basic requirements of any database, it is natural
that many multimedia and imaging DB applications are constructed
within such existing systems. In order to support such applications,
many DBMS vendors offer facilities suitable for MM. These include:
- long bit and byte strings
- paths or references of images where the
actual image stored elsewhere, such as on an optical storage
subsystem. The reasons for this are that document imaging systems
need on-line, near-line and off-line storage of images, including
archiving. This may be achieved by the use of optical jukeboxes
but most commercial DBMSs do not directly support optical storage
subsystems (Informix Online/optical is an exception).
- Content retrieval capabilities. In conventional
relational and OO DBs querying is based on the attributes of
objects. However, information retrieval and document imaging
systems require searching the content of documents. This ability
generalized to still images, audio and video.
Characteristics of Multimedia and Imaging
1. Support for imaging and multimedia data
2. the capacity to handle many MM objects plus
3. support for suitable storage management plus
4. database capabilities (transaction/concurrency control, integrity,
5. information retrieval capabilities (including exact match and
Take each of the above points in turn (except number 3 which will
be considered shortly) and describe briefly what is required
Hierarchical Storage Management
It has already been pointed out that MM
objects can require a lot of memory for storage. It will also
be necessary to have an appropriate storage mechanism so that
the system can keep track of objects that are swapped between
near-line and on-line and inform the user when an object is stored
off-line. In order to do this, the preferred mechanism is that
of hierarchical storage management. This is based on the idea
of managing a hierarchy of on-line, near-line and off-line storage
media. Each of these levels has a particular performance. capacity
- RAM - best performance, smallest capacity,
highest cost, little permanence.
- Hard-drive - good performance, reasonable
capacity. fairly high cost, some online storage capabilities.
- optical storage - on-line with a drive
or near-line with a jukebox. Acceptable performance when on-line
but slow when near-line. High capacity, reasonable cost (less
than preceding levels). Can be used for archiving which is permanent
e.g. WORMdevices, CD-ROM and recordable compact discs.
- optical media stored off line - stored
in cabinets, on shelves etc. Unlimited capacity. very cheap,
lasts much longer than magnetic media and therefore good for
archiving. Poor performance in the sense that the user has to
take the discs off the shelf and put them in the drive!
Requirements for a MMDBMS
A prime requirement for a multimedia
DBMS is to manage this hierarchy of devices. Through caching and
archival capabilities, objects that are no longer accessed are
automatically stored on slower media in the hierarchy and eventually
may be taken off-line. It is possible for advanced relational,
object-relational and object-oriented DBMSs to cope with these
requirements to some extent by assigning large segments or clusters
to store multimedia objects. The segments are then mapped onto
disc volumes and the application developer can use volumes on
the storage subsystem to manage the multimedia object. However,
there are problems with very large databases. Treating a volume
as a large storage space without considering the performance and
functionality of the storage subsystem will cause serious problems
or result in the system not working at all.
Searching for and retrieving data
Although there are similarities
with retrieving data from conventional DBs, in MMDBs it is often
necessary to find objects that satisfy the user's query as closely
as possible rather than finding an exact match. It should be possible
to rank the results of searches. Queries may involve record-attribute
searching as well as content-based searching and the query optimizer
must take this into account.
Spatial data types
In many MM systems elements may
have a spatial relationship with one another e.g. in GISs. The
user may wish to query the DB using spatial predicates such as
location, position with respect to others (object 1 is to left
of object2 or object 1 is
contained within object2, etc.). Such MMDBMSs require support
for queries of this type.
How do you query a MM object? If
you know that you want the whole of a particular named image or
a particular video clip then the query is straightforward. However,
frequently the query may be more complex and require some interactive
exchange between the user and the DB as the user attempts to refine
his query. In MMDBs it is common to have domains or pick lists
of various existing objects so that the user can construct a query
interactively. Query-by-example functions may also be available
where the user builds an example MM object from existing domain
elements. For example, the user may first ask to be shown images
of all the types of fabrics stored using a GUI. They might then
pick a fabric and say: show me all the fabrics of similar texture
and different colours.
Content retrieval is not as precise as querying records or object
attributes and various weights may be applied to extracted features.
For example, values of important features such as colour, texture
or shape are weighted and the distance between these sets of attributes
in the sample and returned images is computed. This can be used
as a measure of how closely the returned images fit the query
example image. Another approach is demonstrated by Excalibur,
a system which recognises patterns using a neural network. For
example, the user may provide an image, such as a fingerprint,
and ask the system to find those images that match it exactly
or are nearest to it.
Describe some other content-based searches that might be run on
a multi-media database.
Automatic feature extraction and indexing
When records are inserted into a
relational database the attribute values of the object must be
specified precisely. Indexes are normally specified by the user
or DB designer. However, for MM systems, tools may be available
to extract the important features of MM objects and even automatically
produce indexes. For example, with advanced document management
systems, paper documents may be converted into scanned digitized
images that are subsequently recognized by OCR products and the
contents may be automatically indexed. Hence, the bulk of the
attribute, or feature, data entry is performed automatically.
MMDBMSs must be able to provide
good performance for real-time querying and updating. Some of
the features that influence this are:
- most DBs use single key access structures such as B-trees which
can be used for retrieving ranges as well as precise matches.
For MM purposes, spatial and multidimensional indexes are also
useful. Two dimensional objects have X and Y co-ordinates ( multi-dimensional
objects will have more) and special structures such as R trees
or grid files will provide better access time.
- content-retrieval indexing - special indexes are required for this. For example,
the index for a video could contain the frame number of the start
of each clip or scene.
- organising BLOBs
- interfaces for BLOBs may allow the user to access and update
byte or bit streams and so positional indexes are required to
provide fast access to continuous streams of bytes or bits starting
at a certain position.
- query optimization - MMDBs are large and manage many complex objects.
Query optimization is vital for providing reasonable performance.
Relational Databases and Multi-media
It should be clear by now that, although
MMDBMSs require many of the features that are also possessed by
relational database management systems, they also have many requirements
that are not met by conventional systems. However, in view of
the current dominance of relational systems, it is reasonable
to consider whether relational DBMSs can be extended to provide
at least some MM capabilities.
Variable length fields
Most of the data types supported
by relational DBs are fixed length and so the length of each record
is also fixed However, many database vendors also provide variable
length fields with the aim of supporting at least some MM data
types - text, digitized audio, still images etc. (You are familiar
with Oracle's VARCHAR2, for example, and various vendors provide
data types such as BLOBs, IMAGE. CHARACTER VARYING). Unfortunately,
there is no uniformity about these. Even with varying length character
fields there are various maximum sizes - some may be as little
as 32 or 64 KB per field value which is not much use for most
MM purposes (see figures quoted earlier for storage requirements).
However, such small fields can be useful for memos or other simple
text additions to records.
SQL92 provides some support for variable
length attributes and also provides BIT and BIT VARYING for storing
bit-mapped graphics. SQL3 will provide more support for MM data
types when it finally appears.
What support is SQL3 likely to offer for MM data types?
Classical DB theory says that fields
in a relational database must be atomic. However, for large MM
data it is not reasonable to assume that all the bytes in the
long field will be read or updated as a whole. Although many vendors
provide support for BLOBS through various data types (BLOB, VARCHAR
VARYING. BIT VARYING) there is no single convention for manipulating
Embedding SQL statements in application
programming languages(e.g. C or C-H-). In these applications,
data is interchanged between SQL and variables set up in the host
programming language. There are several ways of doing this - for
example, rows of data may be processed one at a time using a cursor
operation or data may be inserted, updated, deleted without using
a cursor. An application programming interface (API) is required
that controls the exchange of information with the database server
that manages the SQL database.
One of the key features of APIs is that
they should be able to access BLOBs from a database and integrate
them with front-end applications that can manipulate large quantities
of space-intensive MM data. Clearly, it would be useful if substrings
of the MM data could be retrieved and updated - apart from anything
else, this would reduce the overhead (buffer or cache space) and
As mentioned above, various mechanisms are
used for retrieving values from a database to a host programming
language and there is no need to go into details here. The main
issues involved in manipulating these variable-length fields are:
- size is not known in advance
- the data stored in these field may be
quite large and cannot be read as a whole. So the data must be
managed a piece at a time and byte strings read as needed.
Examples of MM systems based
on Relational DBs
This relational database system
has built in support for BLOBs. The BLOBs are stored in collections
of 'segments' - the size of which can be specified by the user
and are basically a fixed length 'page' or I/O block.. InterBase
has a proprietary high-level language programme interface as well
as a standard SQL interface. The individual segments that comprise
a BLOB can be read and or updated.
Sybase SQL server allows users to declare
columns as TEXT and IMAGE data types which can be very large (2GB).
Sybase has an enhanced version of SQL called TransactSQL which
allows some manipulation of the TEXT and IMAGE data types such
as finding the first occurrence of a particular "pattern"
in the column. The column values of TEXT/IMAGE contain pointers
to the first page of the MM column and these pages are stored
separately from the tables for the database. The pages on which
the object is stored form a linked list.
XDP from Plexus
This is an imaging database engine
based on the INFORMIX Turbo relational DBMS. Unlike Sybase, this
system does support hierarchical storage subsystems and manages
magnetic discs, optical discs, and optical jukeboxes with on-line,
near-line and off-line facilities. Records and the images associated
with them are stored in different locations but both image/text
structures and records can be manipulated and updated consistently
in the same transaction.
Review the main limitations of using a relational database as
the basis for a
multimedia database system.