Download: Movies Executables Publications

An Overview of How Jeremiah Worksfig1.jpg (23261 bytes)

Jeremiah is based around two subsystems, a graphics system which constitutes the head and a vision system which allows him to see. There is also a simple, built in emotion engine which allows him to respond to visual stimulus via expressions or emotions. I am currently working on a new head model called Saul (see fig 4,5) who is a deformable eigenHead models providing more lifelike appearance and motion and the ability to speak.

The system is perfectly capable of running on a single PC but for speed of operation each subsystem runs on a dedicated PC connected via a network, built into a flight case for portability.

fig2.jpg (8445 bytes)Jeremiah’s head is based upon the GeoFace (DECface) articulated bone model with prescripted expressions for key emotions. Linear Interpolation provides transitions between these expressions. He contains a simple Newtonian model of motion with random elements of movement, random blinking and ambient motion.

The vision system is based around a per pixel Gaussian Mixture model of colour distributions using expectation maximisation within the background segmentation framework proposed by Grimson et al with additional shadow suppression and noise removal algorithms. This allows static background scenes to be learnt dynamically, providing a robust segmentation of foreground objects. The system was developed, and is used for visual surveillance (Link to appear, see publications). Jeremiahs attention is randomly distributed between these objects weighted by their size and motion. Co-ordinates of objects within the field of view are sent to the head model for animation.

fig3.jpg (12485 bytes)The emotion engine probabilistically determines the current state of emotions from simple parameters extracted from objects of interest within the visual field. This simple set of rules allows chaotic behaviour in a similar fashion to flocking simulations where a simple set of rules in a complex environment can produce what appears to be emergent behaviour. Jeremiah likes visual stimulus - high rates of movement make him happy. Jeremiah likes company - no stimulus makes him sad. Jeremiah doesn’t like surprise - high rates of change in the size of objects make him surprised. Jeremiah doesn’t like to be ignored - if objects exist but don’t move then he assumes they are ignoring him and hence gets angry. Too much of a good thing spoils the experience - if Jeremiah experiences too much of any particular stimulus he will get bored with it and reduce its influence on himself.

Jeremiah in Operation

figA.jpg (11150 bytes)Jeremiah was originally designed to work on a single desktop computer where the act of waving at Jeremiah would stimulate a smile. During either a large scale installation or performance, Jeremiah works on the same principle and a typical set up can be seen in figure A. This figure demonstrates how a camera mounted above the projected head provides a wide field of view for Jeremiah (up to 180’). The projector mounted above either the performance space or audience, projects the head onto a black canvas. Within a few seconds Jeremiah is capable of learning a background sufficiently accurately to segment moving foreground objects. The model is constantly updated so subtle changes in scene structure or lighting are accommodated by the model and do not lead to failure. If a chair is placed in his field of view then he will see the chair. But if the chair remains static eventually he will get bored of the chair and will incorporate the chair into his model of the world. However, if the chair is then removed he will not get confused by the change in scene structure as he remembers what it looked like before the chair was present. Therefore, any object which enters Jeremiahs field of view can be seen by him. From all visible objects a specific object of interest is randomly selected weighted by its size and motion i.e. large fast objects are more likely to attract Jeremiahs attention. Once an object of interest has been selected the eyes of Jeremiah look in that direction. Depending upon the physical difference between the orientation of the head and eyes the head may follow to turn and face the object. Given multiple objects of interest this results in Jeremiah sharing his attention between objects. The simple emotions gained from this visual stimulus are reflected by the current facial expression of Jeremiah i.e. the more happy he is, the more he will smile. Figure 3 shows the basic expression of jeremiah (default, angry, happy, sad and surprised). All this results in an artificial system which exhibits similar behaviour to that of a child.

fig4.jpg (13266 bytes)Figures 1 & 2 see Jeremiah being used as part of a live performance ‘Blue Bloodshot Flowers’. Here he is one of the performers opposite a live performer who interacts with him. As he has both random elements and reacts directly to visual stimulus he never behaves in the same way twice so the performer is forced to interact with him to try and get the desired responses. Once the performance is over the audience is encouraged to enter his field of view and experience him for themselves. fig5.jpg (6747 bytes)However, the audience can be seen by him throughout the performance. Normally the performer is the most interesting object for Jeremiah to watch but should the performer remain inactive Jeremiah will respond to audience presence and movement (he gets bored very easily).


Downloadable movies :

Jeremiah shares his attention between people, Real Player
Trying to cheer up Jeremiah, Real Player
Sad to be left alone, Real Player

Download RealPlayer


Jeremiah was born from a security system

Jeremiahs visual system is actually based upon a system developed for visual surveillance, a system which is designed to watch people and vehicles performing their every day tasks. Waiting for something unusual to happen. Jeremiah has literally 'been born of this technology' and is the emotional face or the automated security guard who watches patiently for something to happen. Imagine now if we gave that system feelings. Boredom when nothing is happening, anger when people leave and he cant, sadness, joy and surprise when finally the event happens that he has been waiting for. This is Jeremiah, but his emotions and interest in the world and people make him more like a child than a computer system.

Download

Q: Can I have a copy of the source code ?
A: No

Q: Can I download Jeremiah and play with him ?
A: Yes (See below)

MSWindows Version6(build c) 8435KB (.zip)
MSWindows Version6(build d) 8474KB (.zip)

Publications and further information

Bowden R, KaewTraKulPong P, Lewin M, Jeremiah: The Face of Computer Vision, Smart Graphics'02, 2nd International Symposium on Smart Graphics, ACM Int. Conf. Proc. Series, pp 124-128, Hawthorn NY USA, June 2002

Bowden R, Jeremiah: Increasing Public Engagement with Science, Popular Science Poster at SET for Science 2002, House of Commons Reception, UK. 18 March 2002.

Press and Media Coverage

Exhibit in Minnesota Science Museum (Current)
naturalinteraction.org 19/5/03
Article in "Super Interessante" April 2003

Exhibit in British Science Museum Feb 2002
Radio Interview for South African National Radio SAfm "Future Watch" 25/2/02
Interview for Greek National Press "TA NEA" 21/2/02
Coverage in Swedish National Press "AFTONBLADET" 4/3/02
Science Photo Library
B
BC news 8/11/04 (pdf article, pdf front page)
kurzweilai.net 8/11/04
Tweakers.net 9/11/04 (Dutch News) (pdf article)

www.terra.com.mx 9/11/04 (Mexico news) (pdf article)
www.elmostrador.cl 9/11/04 (Colombia news) (pdf article)
www.oidossucious.com 9/11/04 (Latin American news) (pdf article)
BBC Southern Counties Radio Breakfast Show Surrey 10/11/04
BBC Southern Counties Radio Breakfast Show Sussex 10/11/04
BBC Southern Counties Radio Breakfast Show Brighton 10/11/04
BBC World Service Radio "Go Digital" (to appear)
BBC Radio 4 "All in the mind" (to appear)

NB: I know there is more, especially as I get called by all sorts of people asking for details but nobody ever tells me when and where these things appear. Ive got a few others I need to add but if youve seen Jeremiah appear somewhere then please let me know and Ill add it.

Richard Bowden
CVSSP, ECM
University of Surrey
Guildford, Surrey
GU2 7XH, UK
+44(0)1483 689838
r.bowden@surrey.ac.uk