Digital Visual Studies
Profile
Dealing with questions that arise from developments in the Digital Humanities plays an important role in this Research Priority of the Department. Computer-aided processes increasingly determine the collection, management, and communication of research-related data – from sustained digitization of art historical sources and their automatized evaluation to formats of digital publishing. Considering the mediality of sources and tools, hitherto conventional conceptions of working in the humanities are changing. Apart from a reflection on these radical changes and their epistemological implications on the level of art history and theory, the Department encourages the acquisition and mediation of digital competences by introducing an extensive reorganization of the academic IT area at the Institute.
Digital Visual Studies is a Research Priority of the Department, as well as, and more precisely, the name-sake research group of its own: Digital Visual Studies (DVS) is a seven-year cooperation project funded by the Max Planck Society and hosted by the University of Zurich since January 2020. The project aims to establish Digital Visual Studies as a way of pushing Art History in the direction of the Digital Humanities, modernize its methodologies, and contribute to shaping the first generation of Digital Visual Humanists. Digital Visual Studies supports and funds Predoctoral, Postdoctoral, and Visiting Fellows who work in the areas of visual, textual, and spatiotemporal research. The project seeks to foster avant-garde transdisciplinary research focused on methodological innovation and critical reflection in the field between computer vision and art history. It is defined by its explorative nature and driven by the research interests of its members in a bottom-up approach.
Report 2022–2024
Since 2017, the Department has adopted a two-pronged strategy for digital innovation: at the Institute, it promotes the advancement of research infrastructures through the Digital Humanities Laboratory (DH Lab), which has been expanded by the addition of two positions, while it leads the avant-garde Max-Planck-funded research group Digital Visual Studies (DVS) consisting of about five pre- and postdoctoral researchers at the University of Zurich (UZH). Both initiatives are intimately connected and complementary, and both push forward the progress of the Institute in becoming a trendsetting institution and point of reference for Digital Art History. This is necessary to ensure the Institute’s international appeal and competitiveness as a scientific facility for academic careers, research questions, and scientific methods that are rapidly changing under the impact of new technologies, transdisciplinarity, and conflictual globalization, which challenge the traditional mission and profile of the BHMPI.
In Rome, the main institutional and infrastructural objective is to digitize physical resources on a large scale, publish through open access, standardize, link, and semantically enrich data, interconnect digital resources across the Institute’s services, and complete the information life cycle by developing digital publishing platforms and workflows in order to attract researchers beyond the established disciplinary and geographic boundaries. In Zurich, the scholarly goal is to develop avant-garde questions and methods at the intersection of Computer Science and Art History through experimental research projects in order to establish Digital Visual Studies as a new field, for which Zurich offers a competitive framework.
The DH Lab in Rome
Large institutions like the BHMPI provide Digital Art History with data, solutions, and infrastructures, which require a considerable long-term investment. Over the past three years, the Department has continued coordinating the DH Lab and its bi-weekly hybrid meetings in Rome. With the same rhythm, ongoing projects were discussed with international partner institutions in the 43 online meetings of the Consortium for Open Research Data in the Humanities (CORDH). The premiere projects of digitizing the whole Photographic Collection and the Library’s sections devoted to Rome, Naples, rare books, and Italian travel literature - initiatives launched by the Department - are now complete, while the data is being prepared for publication by the respective Scientific Services. In this regard, almost 5k volumes from the collections of the Institute, the KHI Florence, and the MPIWG Berlin, are now accessible with Transkribus’s AI-driven OCR on the new platform HumanitiesConnect Digital Library, funded by the Department. The latter has also supported the development of the digital infrastructure for edited volumes, which hosts the Hertziana Studies in Art History.
A major achievement in activating the interdisciplinary potential of the DH Lab was the development of the Staccioli Digital Archive, which links and publishes 28k archival items, and offers a model for how to build integrated digital research infrastructures. The integrative approach to research ideally consists of targeted digitization campaigns, data curation, Linked-Open-Data and Knowledge Graph technology, physical and online Research Exhibitions, digital scholarly publications on our platforms, and the Fellowships for researchers using the data. In Staccioli’s case, this approach generated a database, a Research Exhibition, a Workshop, and a Predoctoral Fellowship. The Department has also invited and sponsored two master’s students in DH, Hannah Casey (EPFL), who explored the data of the Library, and Marina Bertoldi (UZH), who as a computational linguist analyzed the urban spatiality in guidebooks of Rome. To explore new data-driven research perspectives across BHMPI’s resources and DVS’s methodologies, Maximilian Schich (Tallinn University) was invited as Wittkower Fellow 2023/24 to organize two international Workshops dedicated to a collaborative cultural analysis of the city of Rome (I and II).
The efforts undertaken over the past years culminated in the public presentation of the DH Lab’s coordinated initiatives and achievements in the service of digital research infrastructures for Art History in October 2024. This provided an occasion to reach out to the Institut national d’histoire de l’art (INHA) and the Getty Research Institute (GRI), Los Angeles, to compare respective DH initiatives and infrastructures. Similarly, the TEI publication platform, co-funded by the UZH, was inaugurated in November 2024 with the first two volumes of Heinrich Wölfflin’s writings in open access, representing one of the most comprehensive reusable digital infrastructures for digitally enriched critical editions to date. The Department runs an EasyDB instance that integrates the SARI CIDOC-CRM-based ontology and gives open, CC BY-NC 4.0 access to 13k photographs documenting 6k objects of art and architecture in Latin America.
Digital Visual Studies, Zurich
The development of new transdisciplinary questions and methods requires creative, experimental, agile, risky, and intense scientific teamwork. The goal of DVS is to critically analyze and understand how Large Multimodal Models encode culture in their Latent Spaces, and to develop, from there, new conceptual and computational approaches to the specific challenges of the Digital Humanities. In Zurich, the Digital Visual Studies (DVS) research group, funded for six years with 4M EUR by the MPG, looks back on a highly productive period since its launch in 2020. In this time, DVS has emerged as a unique, cutting-edge project that significantly contributes to opening a new scientific field at the intersection of Computer Science and Art History. The team, led and coordinated by Darío Negueruela, who studies historical and contemporary urban space with AI, as of December 2024, includes two Predoctoral Fellows: Pepe Ballesteros, who decodes light and shadow in early modern painting, and Ludovica Schaerf, who explores the aesthetic structures in AI’s Latent Spaces. Valentine Bernasconi, after a fellowship at the Swiss Institute in Rome, received her Ph.D. dedicated to a computational analysis of gestural expression in early modern Italian painting, while Jason Armitage submitted his thesis in December 2024, which explores embodied multimodality in virtual environments.
DVS’s former Postdoctoral Fellow Eva Cetinić, who analyzes how multimodal Machine Learning encodes visual culture, after securing a UZH Global Strategy and Partnerships Funding Scheme with Leonardo Impett at Cambridge University (20k EUR), obtained an SNSF Ambizione grant with the project The Canon of Latent Spaces (0.8M EUR), joining the UZH Digital Society Initiative (DSI) and Weddigen’s Chair at the University of Zurich with two Predoctoral Fellows, Laura Juliane Wagner and Maria-Teresa De Rosa Palmini, who work on the ethical and aesthetic problems of multimodal, generative AI models. Cetinić and Negueruela have edited the third volume in the Hertziana Studies Art History series, the open-access conference proceedings From Hype to Reality: Artificial Intelligence in the Study of Art and Culture, which document the most recent research in this new field. Another DVS Postdoctoral Fellow, Lucía García de Jalón Oyarzun, has analyzed sensibility implicit in AI-impacted urban space.
Apart from the collaborative project Mining Textual Images with the University of Geneva (8k EUR), DVS won a Swiss Open Research Data Grant of 75k EUR with the project New Methods for Visual Open Research Data in the Digital Humanities (VORD), which promises an innovative, AI-driven approach to extracting visual features from unlabeled art historical image collections. To work on this task and other projects, three researchers joined the team: Andrea Alfarano (Sapienza University of Rome) studying color understanding in LLMs, Iacopo Neri (Institute for Advanced Architecture of Catalonia, Barcelona) applying generative AI to urban space, and Ana María Zapata (Universidad de los Andes) exploring automatic enriching of visual heritage.
The very intense training, supervision, and collaboration was enriched by Peter Bell (Marburg University), Leonardo Impett (University of Cambridge), Fabian Offert (USC Santa Barbara), and Nuria Rodriguez Ortega (University of Málaga), who visited DVS in Zurich to discuss their and the team’s research. DVS organized academies for MA and Ph.D. students at the BHMPI (I, II, III), EPFL, and the université de Montréal, mostly dedicated to AI and Urban Studies.
A highlight was DVS’s participation in the Helsinki Biennial with the digital curatorial project Newly Formed City, relating multimodal data from the Helsinki Art Museum to urban space through generative AI. In the past three years, the DVS team has achieved a high research output by publishing one volume of conference proceedings, two Ph.D. theses, one book chapter, approximately 17 articles, 13 conference papers, and numerous presentations given internationally (in Amsterdam, Ghent, Hawaii, Lausanne, London, Madrid, Málaga, Mendrisio, Lecce, Milan, Montreal, Nicosia, Paris, Rome, San Francisco, Tokyo, Treviso, Zagreb etc.). The team also organized 15 Research Seminars and Workshops in Amsterdam, Madrid, Rome, Zurich etc. The DVS team also taught three BA and MA seminars on AI and Art History at UZH, training students in computational approaches to art and allowing members of the team to gain teaching experience.
As it offers technology and knowledge transfer to the DH Lab, the Swiss Art Research Infrastructure (SARI), presided over by Weddigen and directed by Thomas Hänsli, warrants mention. In the past three years, SARI has been validated as a Technology Platform (TPF) of the UZH, it successfully passed an international assessment, has completed several projects, most notable the Johannes Itten Linked Archive (JILA), Bilder der Schweiz online (BSO), and Open Research Data Environments for the Arts (ORDEA), and has processed a research funding turnover of about 1.8M EUR.
Outlook
In 2025, the DH Lab will shift its strategy to focus on one urgent common goal: building the Institute’s Knowledge Graph, including improvements to the Staccioli Digital Archive. Furthermore, Leonardo Impett (University of Cambridge), DVS Research Associate, will join the Institute as a Max Planck Research Group Leader. With the project Computational Visualities, the integration of Artificial Intelligence (AI) in Art History is imminent. In response to an invitation extended by Impett, Fabian Offert (USC Santa Barbara), a specialist in AI and Art History, will join the Institute as a Wittkower Fellow. As for scholarly publishing, the Department will develop an experimental digital platform for linked micro-publications with a new kind of open, seminar-like peer-reviewing process involving all the Department’s researchers, with the working title Atelier, a project that was conceived in 2017 but had to be postponed in favor of the development of the platforms for edited volumes and critical editions. Finally, the Department is funding the Library’s digitization of all travel literature beyond Italy to complete the online collection.
In 2025, DVS will explore funding avenues for an extension. To address a fundamental research problem and intensify the intellectual exchange with the Fellows at the BHMPI, DVS will focus on the notion of style, which will be discussed in communal online Reading Seminars, leading to a workshop at the end of the year. The team will use a residency at Museo de Arte Moderno de Bogotá (MAMBO) to expand and test the VORD prototype to a working tool for Digital Art History . In continuation of the Helsinki Biennial project, the team will co-organize with Núria Rodriguez an international symposium in Málaga focused on generative curation. The collaboration with Leonardo Impett, DVS Research Associate, will increase with the inauguration of his Research Group in Rome. Finally, two benchmarks will be developed for assessing Digital Art and Architectural History.
