Visual Open Research Data VORD

Ana María Zapata Guzmán

The project addresses urgent open-accessibility issues concerning Visual Open Research Data (VORD). The action is field-specific to visual-art-related disciplines in the HEI (Higher-Education Institutions) and GLAM sectors (Galleries, Libraries, Archives, Museums). It has been proposed by the interdisciplinary research group Digital Visual Studies (DVS), a center based at the University of Zurich (UZH) and funded by the Max Planck Society (MPG) that explores cross-disciplinary problems with visual data.
There are two established practices to VORD. One (a) consists in manual database cataloging which, however, cannot deal with large digitized collections within a reasonable time. Moreover, it faces the issue of lacking interoperability across languages, ontologies, and database systems. The other (b), conversely, consists of automated methods developed to order, search, and display missing textual metadata for large image collections. Such Machine-Learning (ML) engines are based on visual similarity whose parameters are highly debatable. Moreover, they struggle to offer useful information for research and reliable metadata for annotating collections.
Instead, our proposed methodology constitutes a third alternative approach, exploring ways to introduce quantitative information about cultural heritage to enrich datasets with visual information derived from machine perception. Our objective is to automatically extract a set of diverse visual features (i.e., anatomic key points from body postures, histograms, or image embeddings) at scale on large artistic and visual cultural heritage databases, with the aid of different stacks of ML models. We consider this approach useful, necessary, and complementary, but distinct from the other strategies described above. Thus, our project does not aim at reconciling the issues of interoperability between legacy database systems and data ontologies as this is already addressed by projects like SARI and it clearly surpasses the scope of this call. On the contrary, we extract and share metadata in common file formats about visual features, which is agnostic to the data ontologies used, and, therefore, reusable and interoperable by design.
The expected results comprise a documented data science and machine learning pipeline to automate metadata generation based on quantitative visual descriptors. The project relies on a community of institutions and specialists from the HEI and GLAM sectors that share these challenges. In this manner, the iterative project management assures that the pipelines are community-led. Outreach is achieved through workshops, webinars, online publications, and teaching.