Visual communication and visual knowledge are areas that can take important benefits from mature CG technologies. This paper demonstrates a methodology to give new insights into cultural heritage which is based on virtual reality techniques, Web navigation tools, advanced image analysis and photorealistic image synthesis methods. A VRML model of the ancient Roman Theatre of Aosta is the basis to explore multiple data types describing the current status of the building and to examine possible restoration and conservation choices. JAVA scripts increase the degree of interactivity with the data collected by the experts. Photometric and colorimetric data are derived by analysing stones. Image analysis and surface reconstruction algorithms are used to obtain a geometric representation of stones, necessary to produce photorealistic samples of both the present status and the past and future conditions of the building. The final rendering, using VRML tools, has been implemented by adopting texturing as well as photorealistic methods.

Keywords: image analysis, photorealistic images, virtual reality, Web tools, cultural heritage

INTRODUCTION

The documentation and examination of the large amounts of data describing the state and history of ancient monuments open complex problems to cultural heritage experts, whose background in computer technology is usually small, and who are interested in using computer methods to explore "what if" alternatives in order to select techniques for restoring and preserving monuments. Moreover, the richness of historical, artistic and technical information cannot be easily explained to non experts, to give them a better appreciation and understanding of the cultural heritage. Multimedia techniques are now available not only on off-line basis (e.g. CD ROM), but also on-line, for accessing remote repositories of data using Web technology. The best methods to convey a complex knowledge in the field of cultural heritage to experts and non experts are the visual representation and the visual interaction. The visual interaction has been extensively used in off-line CD ROM's, while only recently Internet technology has made advanced levels of visual representation available to the lay public.
In order to explore the feasibility and effectiveness of these new technologies, a prototypal and experimental data base has been used which collects a variety of data describing the ancient Roman Theatre of the town of Aosta. In particular, it includes geometric data, images, descriptive information collected by the cultural heritage experts and data generated by previous computations. These descriptive and pictorial data have been used to extract other data to further improve the degree of detail and accuracy of the ancient building description and representation. In particular, shape and visual appearance properties have been extracted by reconstructing 3-D models of stones, estimating their surface normal distribution and measuring in laboratory color parameters in different conditions (that is, after cutting and polishing the samples, bringing them to their original conditions and after a cleaning process on site). A 3-D reconstruction of the whole Theatre, exemplified in Fig.1, has been carried out to create a VRML model which becomes the basis for accessing the collected information and navigating by using Web browsers.

An Italian multi-disciplinary team is carrying out this project, which aims at developing methodologies and software tools for two main applications:

- a support for experts and specialists in conservation and restoration of ancient monuments;

- a platform to implement educational multimedia for the general public.

The underlying data-base is a fundamental professional help for specialists, who can collect data, analyse results and, also, keep track of on-going or past conservation activities. Advanced graphics rendering and image analysis tools are necessary for using the system as a kind of "what if" environment, to explore the effects of the simulation of conservation or restoration hypotheses, and to provide a feeling of the original aspect of the building. The better way to display a realistic representation of a complex object is to use photorealistic techniques, rather than impressionistic methods, which are based on textured pictures. The link between photorealistic rendering and optical and physical properties of materials can be used by the expert to control conservation and restoration processes better.

The visual interaction, made possible by Web navigation tools, proves itself to be extremely powerful, simplifying both the access to information and data and their maintenance and update. The same navigation tools are the major factors that can help to attract the general public to understand better the value and importance of an ancient building.
The programs have been implemented on SGI workstations, adopting Web Space navigation and authoring tools; the entire application has been integrated with standard HTML methods.
In the remainder of the paper, the available data and the methodologies that have been used are briefly described. Some details about Web integration and example images are also included. In the conclusions, lines of further development of this activity are summarized.

THE DATA

In order to test the methodologies developed in the project, a case study has been selected in cooperation with the ICR (Istituto Centrale di Restauro-Ministero per i Beni Culturali e Ambientali, Roma): the Roman Theatre of the town of Aosta. A significant amount of data was still available for the Theatre when the project started and other data were successively collected with the financial support of the project. A prototypal and experimental data base has been structured by experts working at the ITABC (Istituto per le Tecnologie Applicate ai Beni Culturali - CNR, Roma). A large variety of data types is included in the data base.

The main data types considered were derived from the following activities:

- conventional geometric survey and reconstruction made by a professional study;

- black and white images and geometric data of pudding-stone and travertine ashlars (material composing the Theatre) coming from to the photogrammetric survey made by a professional study;

- color images of pudding-stone and travertine ashlars acquired and chromatically corrected by DIE (Dipartimento Ingegneria Elettronica- Università di Firenze);

- descriptive information coming from a visual inspection made by cultural heritage experts (Soprintendenza per i Beni Culturali e Ambientali - Aosta) structured in a data base by ITABC;

- spectrometric and colorimetric analysis of pudding-stone and travertine ashlars samples, to get chemical composition and reflectivity curves, by DCOMA (Dipartimento di Chimica Organica, Metallorganica e Analitica- Universita' Milano); these analyses have been conducted on samples in their natural state and after cutting and polishing, and on cleaned samples to get reflectivity curves in different conditions.

THE METHODOLOGIES

The critical problem in realistic rendering of complex models, like an existing ancient building, is to control its visual appearance. A possible choice is to adopt texture mapping methods thus arriving at an "impressionistic" rendering. This is an efficient solution given the availability of display system with specific architectures to support real time navigation through textured data, but is limited to give only a rough idea of "their real aspect". In order to increase the degree of accuracy it is necessary to use photorealistic methods; ray tracing has been adopted. The limitation of this method is the impossibility of real time rendering, but the results are of great interest when a link between structural and chemical characteristic of the samples has to be maintained with their visual appearance.

Therefore, interactive navigation is implemented using impressionistic rendering with textured images, while accurate rendering is used for single viewpoint displays. The expert user can interactively choose to generate the rendering of a particular element or the entire building from a desired viewpoint. Ray tracing requires two main data: a description of the surface of the object and the computation of its reflectivity.

Surface Normal Estimation

Texture is a property characteristic of an image or sub-image and it can be used to extract elements belonging to a real or a synthetic scene. A texture descriptor is information derived from the scene, while texture mapping represents special properties that can be associated with surfaces. In particular, texture descriptors can be obtained by applying image analysis algorithms and then used to perform a reconstruction of initial graphic and pictorial properties.

A model to acquire images has been implemented to extract specific textural features of image homogeneous regions and finally insert the calculated parameters into geometric functional descriptors in order to reconstruct the source image. This model allows the user to evaluate the appearance or behavior of materials exposed to degradation by analyzing their samples (analysis phase) and studying their variations with respect to perturbing agents (synthesis phase).

Assuming this premise, an image can be considered as a representation of a model which can be mathematically extended by means of appropriate perturbation functions. Under certain constraints, the grey level of a pixel in an image can be related to the geometry of the object represented by the image, according to a law which states that the intensity of each pixel is directly proportional to the light source and the geometric normal vectors, that is I = L x N / |N| , where I = grey level, L = light source vector and N = normal vector. This relation states that the intensity of each pixel is directly proportional to the scalar product of the vectors L and N. According to this model, some region may be characterized by an intensity degree depending on the position of the pixels with respect to light sources illuminating the scene. In particular, this position is represented by the geometric normal at a certain point and the pixel intensity is closely linked to the orientation and direction of light sources. Therefore, given a vector L, the vector N can be modified and subsequently the surface under examination can be perturbed, by introducing a noise or a new property function. The relation between the original normal vector N and a new computed vector N' according to the above exposition is shown in Fig. 2:

where Pu, Pv are the projections of the normal vector N on the textural surface O(u,v)

A = N x Pu

B = N x Pv

E = noise or perturbation vector.

Ray tracing rendering of stone samples

In order to generate a photorealistic rendering, ray tracing has been applied to stones, whose general shape has been reconstructed from photogrammetric data. The choice of the illumination model used for the rendering is based on the availability of the necessary and suitable data. Chemical analysis allows to obtain data describing the chemical composition of the stones and, also, the spectral reflectivity of the samples in different conditions (in their present state and after cleaning and/or polishing). This information has been used to define color properties of the samples, by computing RGB values from spectral data of each chemical component and by averaging them with a weight distribution derived from the spectral composition of the samples. The RGB approximation of spectral data (sampled each 5 nanometers) has been computed by approximating X,Y,Z and converting the resulting triplet into RGB components by simple matrix transformation. The estimation of the surface normal allowed the application of a classical Whitted illumination model with normal perturbation; the perturbation rule derives from the above described normal estimation method. The ray tracing rendering has been applied on single stones, pudding-stone and travertine ashlars. The rendering can be used as a picture to evaluate the visual effect, but the RGB values and the image can be also used to support impressionistic rendering by texture mapping and material coloring under VRML representation. In Fig. 3 the surface normal distribution estimated from the image of a sample is shown and examples of the stones rendering before and after cleaning are displayed on colour Fig. 4.

WEB INTEGRATION

Web integration presents three main aspects: navigation within the descriptive information, visual navigation within the ancient building (keeping its appearance as it is at present, or changing its appearance according to different hypotheses), interactive access to programming tools for supporting the "what if" analysis. When the project started, the only available tools were CGI scripts, and the VRML 1.0. During the research project new languages and technologies have been developed, e.g. JAVA and remote data base access tools. Some of these new tools and techniques have been adopted.

Currently, the descriptive data base is implemented using the commercially available Archinfo system and the relevant data are exported and made usable by CGI scripts; the user access to the descriptive data is made through HTML pages that include the required information. By means of VRML 1.0 a general model of the Roman Theatre has been created by converting a DXF file and by dividing it into the most relevant parts. Texturing and material data description have been associated to these parts. The VRML model has also been used as the query interface to retrieve the descriptive data as well as an access point to exemplify "what if" results; for this purpose, simple anchor nodes have been associated to the parts.

A JAVA interface

Since VRML 1.0 does not allow the user to interactively modify the appearance of the stones and the building, a JAVA application has been implemented to assist the expert in exploring "what if" hypotheses of restoration and conservation. As described above, the results of "virtual" restoration (based mainly on cleaning and/or polishing the stones) can be displayed on single samples or applied to the whole building or only to some of its parts, by texture mapping or coloring the model components. The JAVA interface supports specific functions to: select a viewpoint, select stones or group of stones, apply colorimetric attributes (normal distribution and reflectivity) to the selected entities, define light sources, compute RGB values, compute ray tracing and generate a view of the selected entities.

The activation of ray tracing rendering is done by a simple system call to a separate process, which has not been ported to different architectures. To the contrary the JAVA interface is standard and portable. A snapshot of the JAVA interface is shown on Fig. 5.

CONCLUSIONS

Further developments of the research described in this paper will concentrate on a new implementation based on multimedia-oriented DBMS and on improvements of the algorithms for extracting necessary information and data from images to increase the quality of the photorealistic rendering. Moreover, in order to define characteristics and patterns "to search" into the images, a strong cooperation must be maintained with the cultural heritage expert which uses his/her proper own language (even if almost standardized but "sometimes foreign to informatics people") to describe the state of the monument. Such type of information has to become as more quantitative as possible in order to ensure its computational use.

ACKNOWLEDGEMENTS

This work is an extension of a research carried out within the Strategic Project "Knowledge through Images: an application to Cultural Heritage" supported by the Italian National Research Council (CNR). At present such activity is included within the CNR Finalized Project "Cultural Heritage". Thanks for precious cooperation is addressed to: Accardo G. (Istituto Centrale del Restauro, Roma), Appolonia L. (Soprintendenza Beni Culturali e Ambientali, Valle d'Aosta), Alparone L. (DIE, Università di Firenze), Salonia P. (ITABC, CNR, Roma), De Floriani L. (DISI, Università di Genova), Di Grazia and FOART Italian professional studies (Roma), Fantucci P. (Dipartimento di Chimica Organica, Metallorganica e Analitica, Università di Milano).

In addition a special thank to Francesco Falco, Mauro Guizzo, Paola Trapani and to collaborators of Laboratorio di Eidomatica.

BIBLIOGRAPHY

[1] Cuccu, F., Moltedo, L. 1993 Texture mapping for scientific visualization environments, Computer & Graphics, Vol. 17, n. 2, pp. 131-136.

[2] Evangelista, R., Salvetti, O. 1993 A morphometric and densitometric approach to image interpretation, Pattern Recognition and Image Analysis, Russian Academy of Sciences, Interperiodica Publ., 3, pp. 103-110.

[3] Glassner A.S. 1995 Principles of Digital Image Synthesis, Morgan Kauffmann, New York.

[4] Marini, D., Rizzi, A. 1993 Colour Constancy and Optical Illusions: a Computer Simulation with Retinex Theory, 7th ICIAP, Bari.

[5] Marini, D., Rossi, M. 1994 Parallelising Accelerated Ray Tracing for High Quality Image Synthesis, WTC94, Butterworths, London.

[6] Moltedo, L., Salvetti, O. 1995 A generalized model for texture coding and processing in applications of art conservation, Proc. ICIAM '95, Hamburg, ZAMM Vol.76, suppl. 2, pp. 449-452.