All posts by jannakmueller

“Where AI meets historical documents”: Automatically transcribing historical prints with OCR and HTR

By Janna Katharina Müller

In my last blog post, I wrote about the source corpus for my master’s thesis – the journal “Monatliche Correspondenz zur Beförderung der Erd- und Himmelskunde” (MC) – and my plan to subject it to digital analysis. The main thing I needed for my analysis was a digital text. Thanks to the Thuringian University and State Library in Jena, the scanned originals of the MC are available online, but only as non-machine-readable PDF files. The first step towards usable data was thus to generate a text from image files.

However, it was essential to consider the type of writing used in the MC: As can be seen from the example page below, the MC was printed in a font that uses, among other things, the long s (“ſ”), an archaic form of the lower-case letter s. Unlike most German publications of the early 19th century, however, this is not a fractional font such as Fraktur, but rather an Antiqua font with serifs, which contains rounded arcs and was used primarily for Latin, Italian, and French texts, but was rather uncommon in German prints.

Figure 1: Example page from the MC (Monatliche Correspondenz zur Beförderung der Erd- und Himmelskunde, Juni-Heft (1801): 556)

OCR with Tesseract

One of the best-known ways to recognize text is Optical Character Recognition (OCR), the electronic or mechanical conversion of images into machine-coded text based on the recognition of individual characters. One software that uses OCR is Tesseract. This is available free of charge and an open-source program. Tesseract needs language add-ons for text recognition, in my case for German.  Here, I already encountered the first difficulties: The module for German text recognition “deu” is mainly intended for modern printing. However, as described above, my source is an older print. Therefore, I attempted text recognition with the additional module “deu” as well as “deu_frak” (for German Fraktur fonts).

If you are familiar with the use of the command line and have installed Tesseract, a simple short command is sufficient for transcription:

tesseract example_page.tif output_page.txt -l deu

Tesseract now provides the following text of my sample page using the German add-on:

556 Monatl. Correfp. 1801. $VNIVS.

die Glaskugel hätte machen können. Es [cheintauch,
dafs man leichter zwey Stahlcylinder einander voll-
kommen gleich wird machen köunen, als man zwey
Glaskugeln gleich grofs und gleich dick blafen kann:
fo dafs man allo bey Stahlcylindern leichter zwey voll-
kommen vergleichbare Thermometer wird erhalten
können,

 XLIV.

Über die

beym April-Hefte’ befindliche

Perfifche Karte.

 (Befchlufs zu S. sız des III B.)

 Mau hat mir den Vorwurf machen’ wollen, dals
ich die Anpreifungen und Empfehlungen Yadley’fcher
Spiegel-Sextanten übertreibe und bis zum Ekel wie-
derhole. Aber die vielfältigen Beyfpiele, welche mir
mehr, als jedem andern bekannt werden, beweilen
mir, dals ich lie bey weiten noch nicht genug em-
pfohlen habe; ich werde allo noch lange damit fort-
fahren. Dies kann doch ‚wol niemanden Verdrufs
oder Schaden bringen ? Ich habe erfahren, dafs [eit-
dem diefes nicht genug zu lobende Inftrument, durch
gegenwärtige Zeitfchrift, auch unter Laien, oder tın-
ter f[olchen Gelehrten bekannter geworden ift, deren
Beruf es nicht mit fich (bringt , aftronomifche Werke
zu lefen, der Gebrauch diefes nützlichen Werkzen-
ges

As you can see, this text is already quite readable, although full of errors. The long s “ſ” is not recognized as such, but transcribed as “f”, “l” or even “[“. There were also problems with the recognition of the word “JUNIUS” in italic capital letters, and the translation of “ſs” as “ß”.

Therefore, here is another try with the Fraktur module:

556 Umsatz«- coyrejp MU. FPNIZS

 die Glaskugel hätte machen können. Es scheint auch,
dass man leichter zwey stahlcylinder einander voll-
kommen gleich wird machen können . als man zwey
Glaskugeln gleich gross und gleich dick blasen kann-
so dass man also bey stahlcylindern leichter zwey voll-
kommen vergleichbare Thermometer wird erhalten
können. 

XLIV.

Über die

bcym April—Hefte’besindliche

Perlische Karte.

(Beschluss zu s. 513 des lll B.) 

Man hat mir den Vorwurf machen· wollen« das-
ich die Anpreisungen und Empfehlung-en flmlzey’scher
spiegel-sextanten übertreibe und bis zum Ekel wie-
derhole. Aber die vielfältigen Beyspiele . welche mir
mehr, als jedem andern bekannt werden, beweisen
mir, dass ich sie bey weiten noch nicht genug em-
Psohlen habe; ich werde also noch lange damit fort-
fahren. Dies kann doch wol niemanden Verdruss
oder schaden bringen ? lch habe erfahren, dass seit-
dem dieses nicht genug zu lobende Instrument , durch
gegenwärtige Zeitschrift, auch unter Laien, oder lin-
ter solchen Gelehrten bekannter geworden ist, deren
Beruf es nicht mit lich bringt . astronomische Werke
tu lesen, der Gebrauch dieses nützlichen Werkzeu—
ges

Here, the “ſ” was partially recognized, but all sorts of incomprehensible things were produced. This is explained by the fact that the “deu_frak” module contains the typical long s for fractional font, but the MC was printed in a rounder, non-fractional font, similar to Antiqua scripts. Since these are unusual for the German speaking area in the early 19th century, however, I could not find a German Antiqua module and thus a suitable module for my project – so I had to look for other ways. These difficulties in applying OCR to historical prints are not particularly uncommon, and I quickly came across another option:

HTR with Transkribus

Another method that was originally closely linked to the development of OCR is Handwritten Text Recognition (HTR). It combines pattern recognition methods with artificial intelligence to produce machine-readable text.  Originally developed for handwritten material, HTR can be applied just as well to printed text that, as in my case, is too complex for OCR processing.

One platform developed by the “READ” project (Recognition and Enrichment of Archival Documents) of the European Commission’s Seventh Framework Program that works with HTR is Transkribus [https://readcoop.eu/transkribus/?sc=Transkribus].  Here, Deep Neural Network Models are applied to manuscript material to decipher (historical) manuscripts or printed texts using digital images and transcriptions, and automatically generate transcripts of similar material.  Based on the input documents, users train an HTR model, which is thus language and script independent.  Baselines and corresponding text form the input of the learning algorithms of the neural network (see figure 2).

For full transparency, I should address the accessibility of the program: The software itself is free, and the use of the HTR was free for a long time. In October 2020, however, Transkribus switched to a partial payment model, so that users could still download the software for free and train an HTR model but had to pay for the application of this to subsequent materials with so-called “credits”, which have to be purchased. For the realization of my project, I received a scholarship of 3000 credits from the READ project, for which I am very grateful.

To feed Transkribus with baseline data (called ground truth), I uploaded the first issue of MC (January 1800). To mark the baselines, Transkribus offers an automatic layout analysis, where the images are segmented into single lines. These then have to be transcribed manually (see also Figure 2). To train an HTR model, about 15,000 words (approx. 75 pages) are required as a ground truth data set; for printed texts, 5000 words or 25 pages are sufficient.  With a few clicks and a little time, HTR models can be trained that achieve a character error rate (CER, average percentage of incorrectly transcribed characters) of less than 5% (handwritten texts), respectively 1–2% for printed texts.

Figure 2: View of a page in Transkribus

As a ground truth data set, I completely transcribed the first issue of MC and used it to train a first HTR model. In a second step, I let this model transcribe the February 1800 issue, corrected it manually and fed this data into my model as well. Thus, I ended up with my model “MC 1800 German” with a CER of 1.29% (see Figure 3). The transcriptions can then be exported as xml, pdf or txt files.

Figure 3: My HTR model “MC 1800 German”

Now, we can compare the transcriptions with my model with those of Tesseract and the example above:

Monatl. Corresp. 1800. JUNIUS.
556

die Glaskugel hätte machen können. Es scheint auch,
daß man leichter zwey Stahlcylinder einander voll¬
kommen gleich wird machen können, als man zwey
Glaskugeln gleich groß und gleich dick blasen kann:
so daß man also bey Stahlcylinder leichter zwey voll¬
kommen vergleichbare Thermometer wird erhalten
können.

XIIV.

Über die

beym April-Hefte’ befindliche

Persische Karte.

(Beschluß zu S. 513 des III B.)

Man hat mir den Vorwurf machenwollen, daß
ich die Anpreisungen und Empfehlungen Hadley’scher
Spiegel-Sextanten übertreibe und bis zum Ekel wie¬
derhole. Aber die vielfältigen Beyspiele, welche mir
mehr, als jedem andern bekannt werden, beweisen
mir, daß ich sie bey weiten noch nicht genug em¬
pfohlen habe; ich werde also noch lange damit fort¬
fahren. Dies kann doch wol niemanden Verdruß
oder Schaden bringen? Ich habe erfahren, daß seit¬
dem dieses nicht genug zu lobende Instrument, durch
gegenwärtige Zeitschrift, auch unter Laien, oder hn.
ter solchen Gelehrten bekannter geworden ist, deren
Beruf es nicht mit sich bringt, astronomische Werke
zu lesen, der Gebrauch dieses nützlichen Werkzeu¬
ges

This transcription is now almost flawless. However, minor corrections were necessary, especially for numbers and symbols. In particular, mathematical formulas and elaborate calculations as well as tables with observational data, which occur repeatedly in the MC, were particularly difficult to transcribe: Rare symbols were not recognized or were confused, fractions longer than two lines were segmented and thus deprived of their reference. The quality of the digitized material itself also led to errors: print-through of the backs, pale or blurred writing or stains affected the correct transcription. Nevertheless, and despite the relatively large amount of time that had to be invested in collecting ground truth data, Transkribus ultimately produced a text that was for the most part very close to the original text and machine-readable, and thus could form the basis for my further processing with digital tools.

 

Sources

Akhlaghi, Andrew. “OCR and Machine Translation.” Programming Historian 10 (2021). https://doi.org/10.46430/phen0091.

Eisenstein, Elizabeth L. The Printing Revolution in Early Modern Europe. Cambridge: Cambridge University Press, 1983.

Graves, Alex, and Jürgen Schmidhuber. “Offline Handwriting Recognition with Multidimensional Recurrent Neural Networks.” In Advances in Neural Information Processing Systems, Vol. 21, 2009. https://papers.nips.cc/paper/2008/hash/66368270ffd51418ec58bd793f2d9b1b-Abstract.html.

Mazal, Otto. Paläographie und Paläotypie. Zur Geschichte der Schrift im Zeitalter der Inkunabeln. Vol. 8. Bibliothek des Buchwesens. Stuttgart: Anton Hiersemann Verlag, 1984.

Muehlberger, Guenter, Louise Seaward, Melissa Terras, Sofia Ares Oliveira, Vincente Bosch, and Maximilian Bryan. “Transforming Scholarship in the Archives Through Handwritten Text Recognition.” Journal of Documentation 75, no. 5 (2019): 954–76.

Smith, Ray. “An Overview of the Tesseract OCR Engine.” Ninth International Conference on Document Analysis and Recognition (ICDAR 2007), 2007, 629–33.

Tappert, Charles C., Ching Yee Suen, and Toru Wakahara. “The State of the Art in Online Handwriting Recognition.” IEEE Trans. Pattern Anal. Mach. Intell. 12 (1990): 787–808.

The very first monthly astronomical journal in Germany: The Celestial Police and their structures of communication

By Janna Katharina Müller

Editorial note: Janna Katharina Müller studies the history and theory of science and technology [“Theorie und Geschichte der Wissenschaft und Technik”] at the Technische Universität Berlin. She’s currently working on her Master’s thesis focusing on the emergence and formation of a concept of the newly discovered asteroids between Mars and Jupiter in the first years after their discovery, 1801–1813. The title of her thesis is: “Von Planeto-Cometen und planetarischen Fragmenten. Die Himmels-Polizey und Asteroidenforschung im frühen 19. Jahrhundert.“ In the spring/summer 2021, she completed a remote internship at the GHI Washington, DC.

A while ago, I was preparing for an oral exam in one of my classes about the history of science during the Enlightenment and the early 19th century. We had to focus on one specific discipline or time period and give a short presentation about it. I decided to prepare a talk about the history of astronomy during that given time frame, and as I worked my way through historical surveys, an anecdote about a group of astronomers kept catching my eye: Six German astronomers founded an association in the early 19th century to track down undiscovered planets – and called themselves the “Himmels-Polizey”, the Celestial Police. They did not find any planets, but they did find the first asteroids – ahistorically speaking – and thus formed the prelude for the Minor Planet Research of the 19th and especially the 20th century.

Unfortunately, that was all the information I could initially find. The Celestial Police – what a name! – did not leave my mind completely, however. In fact, when I started considering possible Master’s thesis topics, I returned to the Celestial Police. In this post, I will introduce a digital source corpus that I located online that can be traced back to their work. In subsequent posts, I plan to reflect on my use of digital sources and tools.

One of these six astronomers involved was Franz Xaver von Zach, court astronomer to the Duke of Saxe-Gotha, in what is now Thuringia. Although he did not personally discover an asteroid, the literature attributes to him an important role in facilitating and coordinating the asteroid discoveries and exploration by organizing meetings among astronomers, initiating the formation of an astronomical society (our Celestial Police), and publishing journals, i.e., creating communicative infrastructures. After spending the early 1780s in Paris and London, von Zach became court astronomer to Duke Ernst II in 1786 and helped build the new observatory on the Seeberg near Gotha. In the following years, von Zach led this observatory to high reputation and, among other things, had many astronomers visiting his observatory again and again, so that he is credited with organizing the first ‘Astronomers’ Congress’ in 1798. Another important meeting took place in September 1800, when von Zach was joined by the German astronomers Schroeter, Olbers, and Harding, and the two state officials (and lay astronomers) von Ende and Gildemeister: During this meeting, they decided to establish a society that would be dedicated, among other things, to the search for previously unknown planets. In this society with the official name “Vereinigte Astronomische Gesellschaft” (United Astronomical Society) 18 additional astronomers from all over Europe were invited to divide the area around the ecliptic (apparent course of the sun in front of the background of the fixed stars) into 24 zones à 15° longitude and 7-8° northern and southern latitude, and divide these among the members for survey. Von Zach wrote in 1801 that such a

„[…] streng organisirte, in 24 Departements abgetheilte Himmels-Polizey hofften wir endlich, diesem, unsern Blicken sich so lange entzogenen Planeten, wenn er anders existirt und sich sichtbar zeigt, auf die Spuhr zu kommen.“

“[…] strictly organized Celestial Police, divided into 24 departments, we finally hoped to get to track down this planet, which has eluded our gaze for so long, if it exists otherwise and shows itself visibly.” (von Zach, Monthly Correspondence, June issue 1801)

Seal of the “United Astronomical Society” with its motto: “Non frustra signorum obitus speculamur et ortus” (Not in vain do we observe the sets of the signs of the zodiac and their rises), in Schroeter, 1805.

Von Zach maintained active contacts with astronomers throughout Europe, especially those in the Paris Académie, such as Lalande or Burckhardt. This correspondence, voluminous by the standards of the time, also formed the basis for von Zach’s publishing activities: In 1798, he founded the Allgemeine Geographische Ephemeriden (AGE), which, in contrast to Bode’s famous Astronomisches Jahrbuch, the most important German periodical publication up to that time, appeared monthly and covered not only geographical but also – closely related – astronomical topics. In addition to essays, book and map reviews, he also included excerpts from his correspondence on astronomical observations and calculations. Soon, however, he realized that dealing with both sets of topics in one journal was getting a bit out of hand and that the correspondence part was taking up more and more space. For this reason, he founded the Monatliche Correspondenz zur Beförderung der Erd- und Himmelskunde (Monthly Correspondence for the Advancement of Terrestrial and Celestial Science, short MC) in 1800. The MC, like the AGE, was issued monthly until December 1813. Fortunately, both sources have been digitized by the Thüringer Universitäts- und Landesbibliothek Jena (Thuringian University and State Library in Jena) and are accessible free of charge in their portal journals@UrMEL (here and here). Without this resource, it would not have been possible for me to access these sources in the last months, since not only the archives were closed due to the pandemic until recently, but also travel was discouraged, even within Germany.

In these blog posts, as in my master’s thesis, I will focus on the MC, as it is the more relevant source corpus for my research on the conceptual evolution of the newly discovered asteroids. The texts published in the periodical on mainly astronomical topics were partially (translated) treatises from other publications, and partially also – as the name suggests – complete letters from Zach’s correspondendents, compilations of excerpts from letters on a certain subject area, and summaries from correspondence whose content Zach communicated in a referential manner. It not only covers the period when the first asteroids were discovered and first studied, but also gathers among its authors many renowned astronomers who were interested in and studied the new celestial bodies, besides the founding members of the Celestial Police. The monthly and for this time remarkably short-period publication of the MC allowed von Zach to circulate news from the world of astronomy faster among an interested ‘expert audience’. At the same time, it provided for a speedy exchange of observations and calculations as well as conjectures and hypotheses, which were of course advantageous for astronomical positioning. These features offer the (incredible) opportunity of reconstructing and analyzing scholarly debates and discussions at the time.

Title page of the “Monatliche Correspondenz” from 1803.

Unfortunately, the source corpus of 28 volumes of six issues each is strikingly under-researched. Except for a book by Dieter Herrmann on astronomical journals in Germany and a work by Peter Brosche on the life of Zach, there are no in-depth analyses of the history of this journal. Yet it is full of exciting topics, aside from the detailed exploration of the asteroids: In addition to travel reports by the young Alexander von Humboldt from South America, geographical surveys of the world, reports on solar eclipses and comet phenomena, treatises on the new metric system, it includes all kinds of studies on the sun, the moon and the stars that are yet to be explored.

The fact that the MC merges two genres of textual sources, letters as well as the nascent journal, makes it particularly exciting for historians of science. On the one hand, it offers the opportunity to reflect on the emergence of the “Fachzeitschrift” (scholarly or scientific journal) as a medium in the German-speaking world. On the other, historians can follow disciplinary developments in the history of astronomy and related sciences on a case-study basis. The Monatliche Correspondenz is thus a very special source. In subsequent blog posts, I will explore its characteristics as a digital source.

Sources

Aubin, David, Charlotte Bigg, and H. Otto Sibum, eds. The Heavens on Earth. Observatories and Astronomy in Nineteenth-Century Science and Culture. Durham/London: Duke University Press, 2010.

Brosche, Peter. Der Astronom der Herzogin. Leben und Werk von Franz Xaver von Zach, 1754–1832. Acta Historica Astronomiae 12. Frankfurt a.M.: Harri Deutsch, 2001.

Herrmann, Dieter B. Die Entstehung der Astronomischen Fachzeitschriften in Deutschland (1798-1821). Veröffentlichungen der Archenhold-Sternwarte Berlin-Treptow 5. Berlin-Treptow: Archenhold-Sternwarte, 1972.

Schroeter, Johann Hieronymus. Lilienthalische Beobachtungen der neu entdeckten Planeten Ceres, Pallas und Juno, zur genauen und richtigen Kenntniß ihrer wahren Grössen, Atmosphären und übrigen merkwürdigen Naturverhältnisse im Sonnengebiete. Göttingen: Vandenhoeck-Ruprechtische Buchhandlung, 1805, HathiTrust, https://hdl.handle.net/2027/mdp.39015064586707.

Zach, Franz Xaver von, ed. Monatliche Correspondenz zur Beförderung der Erd- und Himmelskunde. Gotha: Beckersche Buchhandlung, 1800-1813, Urmel/ThuLB, https://zs.thulb.uni-jena.de/receive/jportal_jpvolume_00202775.