The Vesuvius Challenge recently announced the winners of their 2023 Grand Prize. The winning team used an ensemble of AI models to read text from a scroll of papyrus that was buried in volcanic ash nearly 2,000 years ago.
The goal of the Vesuvius Challenge is to read the text of papyrus scrolls recovered from a villa in Herculaneum, a town buried by the eruption of Vesuvius in 79 C.E. Because the scrolls have been carbonized, they cannot be unrolled; instead, scientists have used x-ray scanning to create 3D point clouds of the scrolls. The winning team used several AI models, including TimeSformer, to detect ink in the x-ray images of the scrolls. Their code was able to decipher about 5% of the letters recorded in one of the scrolls, meeting the 2023 Grand Prize's challenge of "4 passages of 140 characters each, with at least 85% of characters recoverable." In fact, the team recovered more than 2,000 characters. According to historian Garrett Ryan:
The scrolls we have now may be just the beginning...We have not yet found the villa’s main library, which would have contained a much wider range of Greek and Latin literature. That library, with its thousands or even tens of thousands of scrolls, must still be buried. If those texts are discovered, and if even a small fraction can still be read, they will transform our knowledge of classical life and literature on a scale not seen since the Renaissance.
Although the scrolls were carbonized by the heat of the volcanic ash, this also preserved them. They were discovered in 1750, and a few were unrolled---and destroyed---in an attempt to read them. In 2015, researchers developed techniques for imaging ancient scrolls using x-ray tomography. In March of 2023 the Vesuvius Challenge launched, and soon competitors were able to detect ink and letters in the raw data.
Reading letters from the scrolls is a multi-stage process which involves AI models at several stages. First, the scroll is scanned by taking "hundreds to thousands" of x-ray images from various angles; the images are then combined into a virtual 3D reconstruction of the rolled-up scroll. This data is then segmented to detect the surfaces of the papyrus within the roll, and then the surfaces are virtually flattened to create a 3D image of the unrolled papyrus. The third dimension, height, is crucial for detecting ink, and thus the letters written on the papyrus.
The first and second place winners of the ink detection prize, Luke Farritor and Youssef Nader, teamed up with Julian Schilliger, who won prizes for improving the Challenge project's segmentation tooling. In addition to an improved ink detection pipeline, their submission included an improved segmentation algorithm. Their ink detection solution used three computer-vision models: TimeSformer, Resnet3D-101, and I3D.
In a discussion about the prize on X, one commenter asked whether the results from the models might be a "hallucination", referring to a common problem with generative AI. Schilliger replied:
All the top submissions independently confirm the findings. Just with some less precision compared to our Grand Prize submission!
The winning team's code is available on GitHub. The raw data of the scanned scrolls can be requested on the Vesuvius Challenge website.