The Research Project is concerned with the retrieval of images and video sequences in large databases. The research group is :- Multimedia, Databases and Coding and my project supervisor is Prof. Josef Kittler

In many situations manual indexing and annotation of video and images in large databases is simply too expensive. For this reason current research has focused on Content Based Image Retrieval which uses computer vision techniques to index the images/video. The potential advantage of such systems is that they can be automated (at least in theory).

Until now the techniques for CBIR (Content Based Image Retrieval) have been mostly low level, based around low level descriptors (colour, texture etc). The problem with existing systems is that the user interfaces are non intuitive especially for the non-expert in computer vision. Often systems support the query by example paradigm, and in some cases systems attempt to allow the user to directly specify the low level descriptors.

While we acknowledge that visual information is very rich, and that humans are particularly good at making use of it, visual information is not an efficient means by which to describe content. We propose that future retrieval systems should use interfaces which are more intuitive the naive human operator (eg natural language). Towards this goal, we are investigating the concept of a 'Pictorial Dictionary' which would associate low level visual descriptors to higher level semantics. There are many problems that remain to be solved, and many potential areas for research.


Towards A High Level Interface For Content Based Retrieval And Querying

L.Gregory & J.Kittler

PROMS2000, Cracow, Poland. October 2000


The growing importance for content based retrieval and querying has become widely acknowledged. As computing technology and network capacity increases, more and more people will need to have efficient access to the vast archives of multimedia data. The techniques for retrieving this data must be improved, or it will remain largely inaccessible. The means by which humans interact with retrieval systems are currently unsatisfactory. Some of the issues are discussed in this paper. To alleviate these potential problems, a new paradigm, based on the concept of a pictorial dictionary, is proposed. This will allow high-level interaction between the user and the system by natural language. Experiments are conducted which demonstrate the proposed approach and show just one way in which retrieval performance using a pictorial dictionary can be improved.

gzipped postscript
pdf file
Word 200 format

Using contextual information for image retrieval

L.Gregory & J.Kittler

ICIAP 2001. Palermo, Italy


Visual Information Retrieval presents many challenges for the computer vision community. The terabytes of visual information stored in digital image and video libraries will remain inaccessible if the problems of indexing and retrieval are not addressed. In this paper, we present techniques for content based image retrieval using higher level contextual information. The content is represented and queried using attributed relational graphs, with colour attributes and relaxation labeling techniques. We present retrieval examples using both synthetic and real images of national flags. This, although a simplistic problem, highlights the shortcomings, and difficulties associated with content based retrieval systems.

Using graph search techniques for contextual colour retrieval

L.Gregory & J.Kittler

SSPR2002. Windsor, Canada.


We present a system for colour image retrieval which draws on higher level contextual information as well as low level colour descriptors. The system utilises matching through graph edit operations and optimal search methods. Examples are presented which show how the system can be used to label or retrieve images containing flags. The flag retrieval problem provides a good illustration for the technique due to the high potential for structural and attribute errors. The method is shown to improve on our previous research, in which probabilistic relaxation labelling was used.

Using a Pictorial Dictionary as a high level Interface for Visual Information Retrieval

L.Gregory & J.Kittler

ICIP 2003, Barcelona, Spain.


The need for efficient retrieval of visual information is now widely accepted across many research domains. While much progress has been made in the area of low level representation and matching, visual information retrieval systems are often limited by the users ability to express a given query. Current retrieval technology does not allow human operators to formulate queries by means of high level semantics. In this paper we propose a 'Pictorial Dictionary' scheme to address these problems.

Bibtex references and electronic copies can be found at the CVSSP publications archive.

Lee Gregory<>