Heritage Enrichment using Spatial Object Relations (HENSOR)

The vast amount of knowledge that is contained in the RMFAB (digital) collections remains currently under-exploited globally. In order to cope with this issue, the FED-tWIN project will focus on the development of an end-to-end solution for the analysis, annotation and publication of the data contained in the collections and will try to offer a technical answer to a number of the challenges facing museum staff in the digital age. Fully understanding the end user’s needs/tasks, defining the most important features to describe an artwork (and more specifically its composition), opening the relevant metadata in a standardized manner, and being able to perform federated queries across collections using natural language are the challenges that will be tackled in this long-term collaboration between the RMFAB and Ghent University’s IDLab research group.

In the initial years of their FED-tWIN collaboration, the RMFAB and IDLab UGent will join forces in a pilot project that focuses on automatic detection and annotation of saints in paintings. The project aims to cover not just the purely descriptive level of the iconography, but also the hidden meaning (the actual subject) of what is depicted. Such automations can greatly speed up the registrars’ manual work process. In addition, these annotations will be linked with internationally standardised controlled vocabularies such as ICONCLASS within the RMFAB tailor-made wikibase. Both actions will help to increase the interoperability of the data contained in the collections and facilitate its digital dissemination and visibility among a large audience. In order to facilitate the retrieval of RMFAB collection items, end-user focused search mechanisms will also be investigated and evaluated. These tools will run on top of the wikibase and will support federated, intuitive querying in different languages.

In the second research phase (after 5 years) the scope will be extended to the other sub-collections and infrastructures within the RMFAB. A tool will be developed to search all existing digitized RMFAB sources (various carriers/formats via data, text and image recognition) in one internal search. Furthermore, multimodal cross-collection linking will be supported in this phase. This effort should be seen as the first step towards making future data crossing possible between the Federal Scientific Institutions, and later expanding this at a broader national, European and even global level. Once this has been achieved, the way will be open to connect to international projects and achieve unprecedented research results and presentations.

The proposed cooperation will raise the awareness of the usefulness of digital art history and of AI and the semantic web in the study and curation of digital heritage collections.