Leo's paper to VRSIG-97

CBR is a technology derived from Artificial Intelligence (AI) research which has its knowledge represented as cases previously used to solve problems (Kolodner, 1993). Thus, cases can be seen as a piece of memory representing a past experience, which is recorded for possible future re-use.

CBR research has focused mostly on two areas: the storage and indexing for case retrieval; and media and structures for case representation. The latter area represents the focus of this work, introducing VR as an interface for case representation. The integration of CBR and VR plays its role in keeping records of past cases represented in a virtual environment that simulates real situations. Thus, it allows users to have access to sites and experiences in a computer environment that contains the memories and actions of the human experts.

For example, one case in the prototype deals with a scaffold structure for the construction of a residential house. The description of the case refers to the information that differentiates it from the others held in the repository, such as the type of building and scaffold, the work to be provided, and the vicinity. The second part describes how the scaffold inspection was performed, and the sequence of items checked. The third part presents the situation found on this site and recommendations for future similar inspections.

This research uses VR to improve the interface for CBR systems in these three parts of a case. VR is a technology that handles representations of the physical world enabling the communication of ideas (Earnshaw, and Gigante, 1993). VR's capacity to handle objects and their properties, to walk-through the virtual environment in real time, and to emulate real situations in a 3D graphical display, makes it an ideal interface to describing world models.

Bringing together VR and CBR, thus generating a standard association for these technologies, is the main objective of this research. The hypothesis behind this work is that the research approach can help learning as it uses past cases to base its reasoning on, a natural process in human thinking. It presents both the advantages of CBT in an interactive environment and of learning by doing (Dean and Whitlock, 1992), which is one of the most recommended form to provide and stimulate learning (Schank, 1996).

Dearden (1995) stated that "the success of any interactive intelligent system, whether it is rule-based or case-based, is dependent not only on the quality or on the appropriateness of the knowledge encapsulated within the system but also on the quality of the interaction that the system supports". Thus, it can be inferred that the interface in CBR plays an important role in the quality of the support provided to users. Users have to at least interface with the CBR system: first to input the problem description; then to receive the information contained in the case retrieved.

This research does not present any improvement for problem description as it uses the selection of case features from menus, a technique existing already in many other CBR interfaces. However, VR can play a major role in presenting case information to users. Several CBRs can be found using multimedia techniques, such as sounds, pictures, image animation and even digitised films. Nonetheless, apart from VR, where images are represented in an object-oriented architecture, current visualisation techniques present "static" information. Note that the term "static" refers to the information only, and has no relation with the way information is displayed to users. This definition for instance applies to animated images where the sequence of animation is predefined. Conversely, VR is an "active" medium in the sense that users can interact with it and perform their own actions.

The VR interface for CBR also allows access to case features which are dynamically displayed on the screen and facilitate memory recall. Increasing CBR's potential to provide CBT applications as close to real situations as possible is the main reason to bring VR and CBR together. Further information about the capabilities of VR and its role in the process of memory recall is discussed in the following session.

"Images must be lively, active, striking, charged with emotional affects so that they may pass through the door of the storehouse of memory... however, we need to ask ourselves what would constitute the lively, active, striking and emotionally charged equivalents for our own time" (Yates, 1966)

This section draws upon Yates' (1966) book "Art of Memory" to support the use of VR for case representation. The author makes the assumption that people remember things in the context of place, even when there is no relevant connection between the thing remembered and the place where it happened. He provides an example where someone was asked to help name unrecognisable victims of a vehicle accident. This person was actually able to name them by recalling the places where they had been seated. Space can trigger memory recall and this power helps justify the use of VR for case representation as the issues below clearly support:

Virtual spaces have much to offer in assisting learners understand and memorise information. Although current VR technologies can support various aspects of memory recall, there are others that they cannot help with, such as temperature, aroma, or tactile textures. However, VR can work as a filter for space representation, allowing the display of information relevant to the domain. These and other key aspects concerning the design of past experiences in VR are the issues discussed in the following section.

"It may be that all human beings have the same perception of space at the biological level of perception. But certainly every society uses its space differently, both technologically and artistically" (Bolter, 1986)

In addition to the usual issues of case representation in CBR, such as feature selection and indexing cases for retrieval, this research also involves the construction of the virtual worlds within which the cases are held. This is a process of design with no universally accepted methodology to follow.

The comparison in Table 1 has only considered built-in functions of the VR packages, avoiding the need for programmers to include those features in the applications. Currently, most VR packages contain a programming language and thus, most of the features present in this table can be programmed in. However, if the aspect requires programming, it has not been included as "supported" in the comparison.

Aspects of Reality	VRT	WTK	IDS
Extent and scaling
height	yes	yes	yes
depth	yes	yes	yes
breadth	yes	yes	yes
Object's position and movement in 3D
linear velocities	yes	yes	yes
translation	yes	yes	yes
rotation	yes	yes	yes
non-linear velocities	yes	yes	yes
Lighting
light source	yes	yes	yes
distancing	yes	yes	yes
direction	yes	yes	yes
spread	yes	no	no
different colours	yes	yes	yes
Active colour properties
hue	yes	yes	yes
saturation	yes	yes	yes
Passive colour properties
transparency	yes	no	no
translucency	no	no	no
reflectivity	no	no	no
texture	yes	yes	yes
Viewpoint dynamics
3D free movement	yes	yes	yes
variable degrees of freedom	yes	yes	yes
fixing to objects	yes	no	no
not to penetrate some objects	yes	yes	no
dependable object constraints	no	no	no
hierarchical object constraints	yes	yes	yes
Object properties
mass	yes	yes	no
volume	yes	no	no
hardness	no	no	no
brittleness	no	yes	no
flexibility	no	no	no
Object behaviour
gravity	yes	no	no
change colour	yes	yes	yes
expand or contract	yes	yes	yes
reference to the viewpoint	yes	yes	yes
responsive sounds	yes	yes	no

Tab. 1 - aspects of reality supported by VR tools

In addition to the usual issues of case
representation in CBR, such as feature
selection and indexing cases for retrieval,
this research also involves the construction
of the virtual worlds within which the cases
are held. This is a process of design with
no universally accepted methodology to
follow.

The project development has shown that the
understanding of VR capabilities and their
influences over the human process of
perception and cognition can help in
deciding whether VR can be more
appropriate for case representation. In
designing VR cases, one should not just
focus on the possibilities VR offers to build
the cases, but should also carefully evaluate
other factors such as the users' interaction with
the virtual world, and the way it will be
displayed.

In order to help those interested in representing
cases in VR, Table 1 presents the results of a
comparative study of the representational
ability of three VR packages. Thus, developers
can match their needs to the VR package which
capabilities best suit their application. The VR
packages compared are:
Superscape VRT version 5 <http://www.superscape.com> Sense 8 WTK version 6;
<http://www.sense8.com>; and   Integrated Data Systems Inc. IDS VRealm
Builder <http://www.ids-net.com>.

The URL <http://146.87.176.38/postgrad/
Leopage.htm> holds the constantly updated version of Table 1.

Another comparison, involving technical issues in VR packages, can also be found at the URL <http://www8.zdnet.com/pcmag/iu/features/1519/ //buildsum.htm> (last visited in 29/05/96).

Developers should also consider the following prior to choosing a VR tool for case representation:

In fact, questions have been raised regarding the loss of abstraction that VR entails and its possible counter productive effect on understanding in certain domains. For instance, Satalich (1995) describes a study where the users of VR performed worse than a group who only worked on paper. The author cited several reasons supporting these results, such as: the amount of time users have been using the VR, considering the novelty of the technology; the issues and the subject evaluated; and the deficiencies of the hardware used. These results are worth keeping in mind. However, there is no reason to consider computer systems involving VR as necessarily inferior to traditional learning.

A prototype has been developed as part of this research to explore the issues involving VR as an interface to represent CBR’s cases and actions. The prototyping stage aimed at:

Three major reasons encouraged the task of prototyping, which were: (i) the availability of experts committed to supplying the knowledge; (ii) a background knowledge in object-oriented languages and Knowledge-Based Systems development, and (iii) a CBR application developed within the same department dealing with case representation using an object-oriented architecture. The following sub-sections provide further details on the methodology for the development of this application.

5.1- Development methodology

A major issue in this work was the development of a methodology combining CBT requirements with the CBR approach. The CBR paradigm focuses on the ability to represent and retrieve cases. On the other hand, CBT applications rely on methodologies to guide users’ understanding. Thus, developing an application of CBT based on CBR implies providing guidelines for case retrieval to assist users’ learning.

A literature review was performed to identify methodologies for the development of such systems as Knowledge-Based Systems (KBS), CBR, CBT, and Computer-Aided Learning (CAL) applications. The recommendations taken from this literature review have been cross-checked and the resulting methodology is summarised in Fig. 1. It presents the three main stages of the development of the prototype and aims at answering the following questions:

The VR package used for the development of this application was Superscape VRT version 4.0. It incorporates an environment for building VR worlds and a programming language that allowed the development of the CBR engine and the structure of guidelines for the training sessions. Further details about these issues are discussed in the following sections.

On the other hand, CBT applications rely on methodologies to guide users’ understanding. Thus, developing an application of CBT based on CBR implies providing guidelines for case retrieval to assist users’ learning.

Some of the issues concerning the application of VR to represent past experiences, which have been particularly important for the development of this work, are:

The structure for case representation in the prototype has been implemented using the concept of Memory Organisation Packets (MOPs) and Scripts", described in Shank (1996). This concept says, for instance, that a construction site with a scaffold structure serves as a MOP for an expert, and that the several activities involved in its inspection constitute what are called Scripts.

For example, one of the cases present in the prototype describes a site in which repairs will be done on the roof top of a three-storey building. In order to identify whether the scaffold complies with British Standards regulations, certain tasks have to be performed. One of these tasks is to check whether the vertical bars (technically called "standards") are well centred on top of soil plates. Any scaffold structure (which is not suspended), must have its bases well centred on top of soil plates. Thus, the task of inspecting if standards are being properly supported by the soil plates is an example of a Script which is common for several MOPs.

Each case (or MOP) in the prototype has been represented in a different VR file which also contains the Scripts belonging to each case. This approach was adopted due to the large number of objects involved in the representation of scaffold structures (due only to the characteristics of the VR representation of the domain) and this can really slow down the interaction process. The internal structure of the case files follows an object-oriented hierarchy built upon a "Root" object in a "tree" structure. This hierarchy is illustrated in Fig. 2 which also describes the framework for featuring cases for retrieval, which is the subject discussed in the following section.

Featuring cases in the prototype results from the combination of three main issues: (i) the CBR paradigm in terms of adopting features that differentiate the cases in the repository and properly address them for retrieval; (ii) the requirements of CBT applications in terms of methodologies to perform training and learning; and (iii) the capabilities of the object-oriented hierarchy used to represent the cases in VR.

Guidelines for case retrieval prescribe that case features should be useful in describing the case and allowing its proper recognition and retrieval (Kolodner, 1993; Burke and Kass, 1996). CBT requirements indicate that there is no unique way to provide training nor a single methodology leading to the best learning. Moreover, the process of designing the lessons can be one of the most problematic aspects of CBT development (Dean and Whitlock, 1992). A common approach goes from presenting general knowledge (e.g. general implications of Health & Safety on scaffolding) to describing more specific tasks (e.g. how to properly inspect scaffold foundations).

Redmond (1992) stated that CBR applications for training should also include two aspects, namely: presenting the same kind of situations users encounter on the job, as well as carrying a presentation that will be properly kept in the learners memory. The same author indicates that one of the greatest challenges in building such systems is the ability to provide features capable of proper case retrieval as well as helping users to access the cases knowledge in real situations.

The retrieval mechanism adopted in the prototype allows users to only search for MOPs, or for Scripts or for both together. This was made possible because cases and Scripts can be featured independently. The framework for featuring and retrieving cases and Scripts follows an object-oriented hierarchy as displayed in Fig 4. Case features are stored as properties attached to a child of the object at the top of the cases’ hierarchy (Global 1 to N). Script features are stored on a child of the object at the top of the Scripts’ hierarchy (Global S1 to SN).

At the very top of the hierarchy is a file called "Index", which holds the retrieval mechanism and all the information required for case retrieval, such as the names of the VR case files and the features describing all cases with their Scripts. Thus, cases and Scripts contained in the prototype repository have their features stored in the case files as well as in the Index file. The reason behind this redundancy is to avoid the need to open the various VR files searching for their contents, a time-consuming task which slowes down the retrieval process.

The retrieval mechanism performs a search using a weighted-nearest-neighbour approach (Kolodner, 1993) (Watson, 1997), which has been written in SCL (Superscape Control Language). Further details about the retrieval algorithm can be found in Oliveira and Watson (1997). The retrieval mechanism also plays a role in guiding the training sessions, which is the issue discussed in the following section.

On real sites, experts do not need to follow a pre-established sequence to inspect health & safety regulations on scaffold structures (though some prescriptions exist). The usual approach is based on checking key parts of the structure, which allows experts to identify whether the structure has been properly erected, will be safe to work on, and safe for anyone in the vicinity of the structure.

This freedom to choose the sequence of the tasks inspecting scaffolds is quite in accordance with the CBR paradigm. CBR allows users the freedom to retrieve the case they want, according to the inputted description of a case. The same approach has been included in this prototype which has also been designed to cater different levels of users, such as: beginners blindly following systems guidelines for the case retrieval sequence; users experienced with the domain retrieving the script they want to reinforce knowledge; and trainers illustrating their lessons with a virtual representation of past occurrences on site.

Fig. 3 shows one of the cases present in the system’s repository. The menu bar at the bottom prescribes a sequence of scripts associated to the inspection of health & safety regulations on this structure. When clicking the mouse over the script number, the system presents guidelines on how to properly perform the task associated with the script number. Them, the number will change its colour on the menu bar, indicating that the task has been performed.

For example, one of the scripts performs inspection on the overhang at the end of the scaffolding boards. Following system’s guidelines, the viewpoint will move around the structure, replicating the views that an expert’s would have on a real site (see Fig. 4). Theoretical information about this task is also provided by reading the menus or listening to an expert’s recorded advice.

Users can also freely walk-through the virtual case searching for irregularities on the structure, independently of system’s guidelines. This approach is specially important for trainers, who can use the system as a tool to illustrate site occurrences. For instance, viewpoints such as presented on Fig. 5, could be difficult or dangerous to access on real structures. Moreover, trainees would need to go to a site where this structure is present, and special supervision would be required.

Experiencing is central to using VR as a visualisation tool and is a key characteristic of this application. In fact, VR supports "learning by doing", a recommended form of learning. The physical world simulations achieved by VR provide an interactive environment that facilitates understanding of the lessons displayed. Designing virtual worlds can be difficult and time-consuming. However, the increasing availability of more powerful software for virtual world building and of third-party objects will ease the process of virtual world design. Modelling techniques making explicit an object’s behaviour, dimensions, positions, dependencies, and its links with other objects will help speed up world creation too.

CBR supports the retrieval of past cases that can help users learn by: (i) having the opportunity to access the structure and the contents of past experiences; (ii) understanding the content of a case and its relevance within the domain; (iii) accessing the actions and recommendations taken from the cases; (iv) solving new situations by comparing with similar cases, and (v) creating new cases by adapting from the existing ones.

Research shows there are very few domains, if any, in which CBT could not be used to assist learning. We understand that one of the main reason to choose CBT relies on providing good courseware at low cost. The experience taken from this project indicates that though it cannot be seen as a low cost option, subsequent updates and revisions should be possible at relatively low cost. CBT may not be an approach to bring best results in a short period of time.