Direct view Holographic Autostereoscopic Displays
David Trayner MA; Edwina Orr MA
6 Yorkton St. London E2 8NH
Tel: +44 (0)171 739 9700
Fax: +44 (0)171 739 9707
We describe a new autostereoscopic display technology based on direct view Liquid Crystal Display (LCD) and Holographic Optical Elements (HOEs)
The display uses a composite HOE to control the visibility of the pixels of a conventional LCD. One arrangement is described which uses horizontal line interlaced spatially multiplexed stereo images displayed on the LCD to provide an easy to viewed autostereoscopic (i.e. glasses-free real 3D) display. It is compatible with existing glasses-based stereo system using the field-sequential method coupled with a shutter system (e.g. LCD shuttered glasses).
We indicate how this new technology can provide backwards compatibility with existing 2D display technology through a full resolution 2D display option, thereby allowing it to perform as a 2D monitor when required.
We also describe the advanced capabilities of moving the stereo viewing zone to follow a moving viewer and we discuss and demonstrate recent experimental work showing full parallax (along in X,Y and Z axes) autostereoscopic 3D from computer generated images running on a PC. These experiments show that very convincing full parallax autostereoscopic images can be displayed to a viewer who has substantial freedom of movement.
Additional capabilities will be indicated including multiple viewer systems which will allow a number of static or independently mobile users to enjoy 3D autostereoscopic images simultaneously.
This display technology allows the manufacture of general purpose display with the added capability of being able to display autostereoscopic 3D when required.
Autostereoscopic, holography, HOE, stereoscopic, 3D, LCD, display.
The display technology described below is a direct view LCD-based device which as well as performing as a normal monitor can also display an autostereoscopic 3D image - i.e. the user can see a stereoscopic 3D image in front and behind the plane of the screen without the need for any special glasses. The technology is very new and makes use of a holographic illumination system combined with a conventional LCD panel.
2. Basic principles of operation.
2.1 Image holograms - background information.
Most people are familiar with image holograms where, under proper lighting conditions, very high resolution 3D images can be produced. Holograms are generally made by recording the fringe pattern produced by interference between a "reference beam" and light scattered by the object to be recorded. A coherent light source (i.e. a laser) and very high resolution ( 6000 line pairs per millimetre) photographic recording material are needed. It is very unlikely that real time holography will form the basis of a commercially viable 3D transmission and display system in the foreseeable future.
2.2 HOEs - background information.
Holographic techniques can, however, be used to manipulate and direct light in ways that are impossible using conventional optics. In an image hologram the illuminating light is focussed by diffraction in the hologram to produce a 3D image of an object. (We say that the wavefront of the original object light is reconstructed by the hologram.) In a HOE the "object" might be one or more points of light, in which case the HOE will perform in a way analogous to some combination of lenses, mirrors and diffraction gratings. HOEs are used e.g. in Head Up Displays, fibre optic interconnects and CD readers. They usually have properties such as multiple focal points, wavelength selectivity and transparent appearance which cannot be achieved with conventional optics. Their optical performance is not related to their physical shape but comes from diffraction which is most often produced by either a surface or volume phase grating.
2.2 Direct view autostereoscopic HOEs - principles of operation.
The HOE we have developed uses as its object a diffusely illuminated plane viewed through an array of apertures. It is a composite spatially multiplexed HOE originated using at least two similar laser exposures on one plate.
Its basic operation can be understood by first considering a simple hologram of a diffuser. The diffuser should be imagined as an evenly illuminated rectangular diffuse plane - like a white sheet of card.
Fig.1 illustrates such a hologram (or HOE) which is illuminated to produce a real image of the plane. Although it is not perceived as such, this is a image of a diffuser floating in front of the HOE. Consideration of the direction if the light forming this image shows that an eye (eye 1 in the figure) positioned so that the image lies in line with it and the HOE, will see the whole surface of the HOE brightly and evenly illuminated. Conversely a differently positioned eye (eye 2 in the figure) is not in line with the HOE and the image so the HOE appears dark. If a transparency is place on the HOE then for eye 1 it will be back lit and therefore visible; for eye 2, on the other hand, it will be dark and the image will be invisible.
Fig. 1. Reconstructed real image of a diffuse plane.
Figs. 2 and 3 show how we use this principle to create an autostereoscopic display.
Fig. 2 Composite HOE with two sets of HOE regions and two corresponding diffuse real images.
Figure 2 shows a composite HOE, it is divided into two set of regions, the members of one set diffract slight so as to form a real image of a diffuser in one position while those of the other set diffract it to form a real image in a different position. Consequently an eye positioned to look through one of these real images will see one set of regions brightly lit while the other set will be dark. The converse applies to the other eye.
To make an autostereoscopic display we make a HOE where there are two sets of regions in the form of interlaced horizontal lines with double the pitch of the pixels of the LCD. Odd numbered horizontal lines on the LCD then correspond to the members of one set of HOE regions while the even numbered ones correspond to those of the other set. This arrangement is illustrated in fig.3.
Fig. 3. Composite HOE and LCD.
We use transmissive LCDs - like the ones used on laptop computer and increasingly on the desktop. As they are transmissive devices they require a back light in order for the image to be visible - just like a photographic transparency. To make an autostereoscopic display we remove the conventional diffuse back light and replace it with a HOE. In making the HOE we position the real images for the diffusers so the left eye looks through one while the right eye looks through the other - typically at a distance of about 500 mm from the HOE and arranged so that the two real images abut each other. Consequently an image displayed on, say, the odd lines of the LCD will be visible to the left eye alone while the right eye will perceive an image displayed on the even numbered lines. Clearly, if each of these images is one of a stereo pair then a stereoscopic 3D image can be seen. Control of the stereo image translation and disparity can place the 3D image behind, straddling or in front of the LCD.
This is one of a variety of possible configurations.
3. Compatibility issues.
3.1 Hardware compatibility:
The displays are based on standard LCDs with standard drivers and interfaces, consequently they will plug directly into unmodified PCs without needing any special hardware or software modification.
3.2 Software compatibility - stereo mode.
The interlaced horizontal line configuration also has the advantage of being geometrically the same a field-sequential stereo display methods using shuttered glasses so our display is compatible with existing cameras and software that use this method - though ours is not operating in a time sequential mode and, of course, does not require shuttered glasses. It therefore operates with a range of existing material from stereo videotapes through stereo capable CAD, VR and modelling software.
3.3 Application compatibility - 2D operation mode.
A frequently underestimated requirement for a 3D display is that it should work well in 2D mode. The important issues here are (a) the full resolution of the display should be usable in 2D and (b) retinal rivalry must not be present in 2D mode. In effect this mean that 3D operation must be an option which can be turned on for viewing a 3D scene and turned off for tasks such as word processing or drawing. This is comparable with the need for a stereo sound system to play mono recordings or a colour display to be able to display black and white.
One way we can achieve this by exploiting the fact that the position of the left and right viewing zones is directly related to the position of the illuminating light source. Fig 4 illustrates a basic implementation where the use of two separate light sources causes the left and right viewing zones to overlap. In the overlapping region both eyes will see all the pixels giving full resolution 2D performance. The switch between 2 and 3D is achieved by lighting control which can itself be brought under software control so the appropriate display mode can be set automatically.
Fig.4 A 2D compatibility method
3.3 Commercial potential and performance targets.
This display technology allows the manufacture of general purpose display with the added capability of being able to display autostereoscopic 3D when required. Potentially this can be achieved at a similar cost to existing 2D LCD monitors. One target is to produce a display which has equal performance to existing 2D monitors in 2D mode but which will switch to 3D mode when needed - thereby enabling users to purchase a general purpose 3D capable display - as opposed to using either special glasses or having an additional monitor for 3D applications.
4. Advanced performance issues.
The basic technology provides a very easily viewed 3D image within a fixed viewing range. The display is intrinsically optimally aligned so the limitation on depth is the same as any well optimised stereo display system. In an interactive mode an image can be moved backwards and forwards by the user, interaction ensures that the user can fuse the stereo half images and in such a case the image location can be moved 1-2 metres behind the screen and to a point about halfway between the screen and the user's face. Such extremes involve substantial stereo disparity so in non-interactive situations the usable depth should be limited to around 2x the horizontal dimension of the screen, with 1/3 of this volume in front and 2/3 behind the plane of the display.
4.1 Mobile viewing zone.
The basic technology is capable of more than fixed viewing position 3D. The position of the light source controls the position of the viewing zones. If the source is moved then the stereo viewing position will also move. Consequently if the display is linked with an eye tracking device it is possible to provide stereo viewing for a mobile viewer. We have demonstrated this in an experimental implementation with a light source mounted on a translation stage (though in the future more elegant solutions without moving parts can be expected). The position of the light can then controlled by a PC and a commercial head tracking device. This provides substantial freedom of movement for the viewer.
4.2 Continuous parallax imagery - look around, over, under, nearer and further with perspective updating.
Normal stereo 3D consists of two perspective views on an object, if such a picture is viewed from different positions the image will appear to distort as there is no changing parallax corresponding to the viewer's changing position. This effect is not particularly disturbing but certainly reduces the sense of presence of the 3D image and its verisimilitude to a moving viewer. If, however, we introduce perspective updating it is possible to redraw the image with the perspective adjusted in real time to correspond to the viewer's changing position. This adjustment can be done for viewer movement in all three dimensions.
We have implemented this on our newest experimental display. We used Themekit's "Mindformer" VR package. The authors of this software introduced three advances: (a) the integration of the head tracker (b) software for the control of our stepper motor driven translation stage for the light in our display and (c) perspective updating to correspond to the viewer's position. Our display allows considerable freedom of movement in Z and Y directions without there being any need to move the light source, so it only needs moving horizontally side-to-side. The perspective updating, however, is done corresponding to viewer movement along all three axes.
The subjective effect is of an object actually being there - it is possible to look round it, under and over it as well as nearer and further from it. The software also allows for real time manipulation of the object and the provision of the strong sense of 3D space allows modelling in 3D without having constant recourse to the conventional 4-view (i.e. plan, front elevation, side elevation and perspective vie) 2D representations to locate the image in depth. This work is at an early stage but shows considerable promise. Fig.5 shows the layout of the display.
Fig.5. Schematic layout of experimental display with moving stereo viewing position.
4.3 Quasi-continuous parallax imagery by the multiple view method.
There is a second way of providing the impression of parallax in the horizontal direction. This involves the simultaneous display of multiple views of the object. Although it is perfectly easy for us to use this method in our invention we feel that the amount of views required in order for the effect to be usable for deep images and to approach the performance of a head tracked system is rather large (50 -200 separate for a monitor sized display). Such an arrangement would provide parallax effect in the x and z directions but still not vertically. The resolution and bandwidth overheads ( 50 times as many images to be displayed simultaneously) entailed by a multi view real time autostereoscopic display are much too heavy when compared with the 2-view head tracked system of similar performance. Multi-view systems with fewer views will restrict the depth of the scene that can be displayed and will also restrict the range of viewing distances, furthermore computer system integration and compatibility issues become clear problems - which is not the case with a 2-view head tracked system.
4.4 Simultaneous stereo viewing by multiple independently mobile viewers
The number of stereoscopic viewing positions can be increased by adding light sources and thereby adding one stereo viewing position per light. This is illustrated in fig.6. If both lights are independently mobile and there is a means of tracking two viewers then each viewer can move without loss of stereo viewing. The same principle can, of course, be extended to provide stereo viewing for more viewers. Some modification of the optical arrangement can also allow for tracking in distance - which will allow viewers to move in a substantial volume without loosing the 3D.
Fig.6 Display arrangement for 2 independently mobile viewers
5. Status of our work
This technology is covered by international patents, we have made three demonstrators, the first is based on a VGA resolution 8 inch TFT LCD. The second on a 10 inch S-VGA TFT LCD from Toshiba (pictured in fig.7). The third requires more work, it is based on a 13.3 inch Hitachi XGA resolution S-TFT LCD. is is provided with a head tracker and can demonstrate continuous parallax imagery.
Fig.7 Experimental holographic autostereoscopic display #2 (10 inch, S-VGA)
The authors would like to thank Dr.Robin Hollands for assistance with the mechanism and electronics to control the light source position and Martin Robinson of Themekit Ltd. for the integration of computer control of the light position and adaptation of Mindformer to provide real time perspective updating.