More href markers from Braunschweig, shared by Maret Nieländer, continuing our series on the Great Outdoors Web…
All photographs by Maret Nieländer.
Photo credit: Maret Nieländer
More href markers from Braunschweig, shared by Maret Nieländer, continuing our series on the Great Outdoors Web…
All photographs by Maret Nieländer.
Photo credit: Maret Nieländer
True, the digital space is vast and diverse, populated with code and tags, including href tags, which inspired the name of our blog. But there is also the great analog space, populated with its own code and tags, even analog hypertext references. Many thanks to Maret Nieländer, author of a recent href post, for sharing these href references from Braunschweig, created by an unknown graffiti artist. These markers are mysterious, yet intriguing, and maybe they carry the message that the seamless navigation between web pages, enabled by href tags, can also happen outdoors, in a urban space, where pedestrians and bicyclists navigate from land post to land post, href marker to href marker?
All photographs by Maret Nieländer.
Photo credit: Maret Nieländer
Editorial note: Hannah Voß has finished her undergraduate degree in History and English and American Studies at the University of Münster and is currently in the last semester of completing her master’s degree in history. Her bachelor’s thesis focused on the role of “New German Dance” in the period of National Socialism in Germany. Her current research interests include the history of National Socialism and the Holocaust as well as the history of social movements, Gender, and Queer History. In her master’s thesis, Hannah will analyze the women’s and lesbian movement in Germany in the 1970s and 1980s. She has also been a part of the curating team of the 2022 historic exhibition “Queer Münster. Eine andere Geschichte der Stadt / A Different History of the City.”
Born in 1848, Helene Lange became one of the most important representatives of the moderate wing of the German women’s movement in the late 19th and early 20th century. She immersed herself in the fight for women’s rights and education and became known as the author of the Yellow Brochure (1887), in which she sharply criticized the state of girls’ education. In 1890, she founded the General German Association of Women Teachers (ADLV). As a teacher, women’s rights activist, and politician, she made a decisive contribution to enabling German women from the middle classes to study and to opening up career opportunities for them.
Today, the Helene-Lange-Archiv (HLA), which is part of the Landesarchiv Berlin (Berlin State Archives), bears her name. In addition to parts of Helene Lange’s written legacy, the archive brings together the papers of a number of associations and societies advocating on behalf of women’s education, training, and careers. Large parts of the HLA have been digitized and are integrated into the Digital German Women’s Archive (DDF). The HLA focuses on the moderate wing of the middle-class women’s movement of the late 19th and the early 20th century as well as on two “radical” women’s associations of the movement. Most of the archive’s collections are now openly accessible online.
For the most part a digital archive, the HLA offers new possibilities for research on women’s and gender history and for public engagement with this history. Generally, having online access to sources means that researchers no longer need to travel inside or outside the country to review archival collections, which helps with facilitating historical research. Often, digitized sources are also searchable, either as full texts or based on their metadata. Research, which could take hours of flipping pages, can become a matter of seconds. The HLA is already pre-arranged online and organized by topic with the level of detail varying depending on the size of the collection. For example, anyone wishing to research Helene Lange’s archival legacy can retrieve digitized material under titles such as “Letters by and to Helene Lange”, “Writings by Helene Lange (Brochures, Journal Articles, Typescripts)”, “Journal Articles on the Women’s Movement, the Reform of Girls’ Secondary Schools and the Admission of Women to University Studies”, and more.[1]
However, online access to digitized material is not only about making the lives of established professional historians easier. It also gives those researchers access to sources who don’t have a strong institutional and financial support system. This is especially relevant for women’s and gender historians. In Germany, relevant gender history research is frequently carried out by doctoral and postdoctoral researchers who often work under precarious employment conditions.
The development of the HLA and the DDF also needs to be situated in the context of the development and institutionalization of women’s and gender history at universities in the United States, Western Europe, as well as other countries. In the 1960s in the USA and 1970s in many European countries, including Germany, women’s history emerged as its own field in the time of new social movements, especially the women’s movement. Building on work by women’s history pioneers, such as Mary Ritter Beard, Gerda Lerner created the first graduate program in women’s history in the US at Sarah Lawrence College in 1972. From the 1980s onwards, women’s and gender history gained acceptance and became more widespread at history faculties in the United States.[2] In Germany, the institutionalization of women’s and gender history took longer than in the US. In 1986, Annette Kuhn held the first chair in “Didactics of History, Medieval and Modern History, and Women’s History” in Bonn. In the 1990s, there were only three chairs at West German universities specializing in women’s or gender history. Still today, as women’s and gender history have become established fields of study at many German universities, there is still work to do. The field needs adequate funding for new research programs and could be more firmly anchored at universities.
From the beginning, women’s and gender historians aimed to discover new sources while reading existing sources through a different lens in order to counter the inherent male bias with which many sources had been interpreted before. The vast collection of sources on women’s associations and their politics, discussions, and individual members within the HLA makes this undertaking easier. Based on these collections, researchers can analyze a range of questions, including how women organized themselves during the Kaiserreich (German Empire) and the newly established Weimar Republic; how they formed different interest groups on local and state levels, how these groups interacted with each other and transformed in the wake of the Nazi movement and the NSDAP’s rise to power.
Online archives like the HLA also open up new possibilities for transnational research and comparative studies. What did women’s engagement look like under different circumstances in different countries in the early 20th century? How did women’s interest groups evolve in a newly established democracy? Questions like these can rarely be investigated based on source material from one’s own country of residence alone. And while driving or even flying to another country for archival visits might be possible for some researchers, it is easier (and cheaper) to visit an archive in a foreign country from the comfort of one’s own workplace. This can, in turn, spark the interests of more historians in transnational or comparative projects on women and gender, which makes the field even more diverse and thought-provoking.
Nevertheless, the HLA and other archives alike do not only benefit the academic historical profession. They are a great, accessible resource for teachers and other educators trying to incorporate women’s and gender history into common – often male-centered – historical narratives found in textbooks and curriculums. In Cornelsen’s Forum Geschichte for German high school students in North Rhine Westphalia, for example, there are 40 pages on the Weimar Republic, out of which two focus specifically on women’s rights.[3] In Westermann’s Zeiten und Menschen, there are none.[4] The HLA’s collection on the ADLV could, for example, provide first-hand accounts of women’s engagement in associations and politics on an individual as well as structural level in the Weimar Republic which can be connected to a variety of topics concerning the development of the first democratic German state.
Making archival material on women’s and gender history accessible online thus enables many small-scale developments: Historical research on the topic becomes more inclusive within and outside of the profession; historians can broaden the horizon of their research topics; women’s and gender history may receive more recognition in textbooks and education in general. What might appear as small-scale then has an impact on the bigger picture. In times where the legitimacy of women’s and gender history is still being questioned by some in- and outside of university campuses, archives like the HLA enable researchers and educators to anchor the field more equally in the historical canon. This acknowledges that women have always played a role in history no matter the time, place, or situation and that looking at the role of gender as a fundamental organizational category of society can give us new insights about past and present.
1 It would be desirable for the digitized collections of the HLA to be searchable in full text in the future.
2 In 1986, Joan W. Scott published her essay “Gender: A Useful Category of Historical Analysis”, now often considered a “classic”.
3 Forum Geschichte 3: Nordrhein-Westfalen Gymnasium G9. Berlin: Cornelsen, 2021, pp. 130-132.
4 Bethlehem, Siegfried et al. (eds.). Zeiten und Menschen: Geschichte: Qualifikationsphase Oberstufe Nordrhein-Westfalen. Braunschweig: Westermann, 2015.
Image References
1) Helene-Lange-Archiv. Verschiedenes Material über Helene Lange, in: B Rep. 235-11 – Nachlass Helene Lange. https://www.meta-katalog.eu/Record/BRep235116MFNr36003601VerschiedenesMathla/HierarchyTree?recordI D=BRep235116MFNr36003601VerschiedenesMathla#?#tree-BRep235116MFNr36003601VerschiedenesMathla &showDigitalObject=&c=&m=&s=&cv=&xywh=&r=.
2) Helene-Lange-Archiv. Glückwünsche, Würdigungen, Programme und Ansprachen von Feiern zu Helene Lange’s 80. Geburtstag (9. April 1928), in: B Rep. 235-11 – Nachlass Helene Lange 1928. https://www.meta-katalog.eu/Record/BRep235117MFNr36013607GlckwnscheWrdiguhla/HierarchyTree?recordID=BRep235117MFNr36013607GlckwnscheWrdiguhla#?#tree-BRep235117MFNr36013607GlckwnscheWrdiguhla&showDigitalObject=&c=&m=&s=&cv=&xywh=&r=.
3) Lange, Helene. “Steht die Frauenbewegung am Ziel oder am Anfang?” Die Frau. Sonderabdruck (1921): 1-14. Helene-Lange-Archiv. Schriften von Helene Lange (Broschüren, Zeitschriftenaufsätze, Typoskripte), in: B Rep. 235-11 – Nachlass Helene Lange. https://www.meta-katalog.eu/Record/ BRep235113MFNr35973599SchriftenvonHelehla/HierarchyTree?recordID=BRep235113MFNr35973599SchriftenvonHelehla#?#tree-BRep235113MFNr35973599SchriftenvonHelehla&showDigitalObject=&c=&m=&s=&cv=&xywh=&r =.
4) Beckmann, Emmy. “Abbau und weibliche Beamte.“ Deutsche Lehrerinnenzeitung: Organ des Allgemeinen Deutschen Lehrerinnenvereins 41, no. 17 (15 September 1924): 129-130. Helene-Lange-Archiv. Zeitschriftenartikel u.a. zu den Volksschulen, zum Personalabbau und Mädchenschulwesen, in: B Rep. 235-03 Allgemeiner Deutscher Lehrerinnen-Verein (ADLV) 1924–1925. https://www.meta-katalog.eu/Record/BRep2350341 Zeitschriftenarthla/HierarchyTree?recordID=BRep2350341Zeitschriftenarthla#?#tree-BRep2350341Zeitschriftenart hla&showDigitalObject=&c=&m=&s=&cv=&xywh=&r=.
5) Beckmann, Emmy. “Kampf gegen die Doppelverdiener und der Abbau der verheirateten Lehrerin.“ Allgemeiner Deutscher Lehrerinnenverein (A.D.L.V.). Helene-Lange-Archiv. Schriftwechsel mit weiblichen Abgeordneten und Eingaben, in: B Rep. 235-03 Allgemeiner Deutscher Lehrerinnen-Verein (ADLV) 1922-1933. https://www.meta-katalog.eu/Record/BRep2350337Schriftwechselmihla/HierarchyTree?recordID=BRep2350337Schriftwechselmihla#?#tree-BRep2350337Schriftwechselmihla&showDigitalObject=&c=&m=&s=&cv=&xywh=&r=.
Literature
Blom, Ida. “Global Women’s History: Organising Principles and Cross-Cultural Unterstandings.” In: Writing Women’s History: International Perspectives, edited by Karen M. Offen, Ruth Roach Pierson, and Jane Rendall, 135-149. London et al.: MacMillan, 1991.
Bock, Gisela. Frauen in der europäischen Geschichte: Vom Mittelalter bis zur Gegenwart. München: Beck, 2000.
Bridenthal, Renate, and Claudia Koonz. “Introduction“. In: Becoming Visible: Women in European History, edited by Renate Bridenthal, and Claudia Koonz, 1-10. Boston et al.: Houghton Mifflin Company, 1977.
Dayton, Cornelia H., and Lisa Levenstein. “The Big Tent of U.S. Women’s and Gender History: A State of the Field.” Journal of American History 99, no. 3 (December 2012): 793–817. https://doi.org/10.1093/jahist/jas454.
Frevert, Ute, Christina Vanja, and Heide Wunder. “Historical Research on Women in the Federal Republic of Germany.” In: Writing Women’s History: International Perspectives, edited by Karen M. Offen, Ruth Roach Pierson, and Jane Rendall, 291-331. London et al.: MacMillan, 1991.
“Geschlechtergeschichte. Herausforderungen und Chancen, Perspektiven und Strategien: Ein Positionspapier.“ H-Soz-Kult, 9/19/2023. <www.hsozkult.de/debate/id/fddebate-138660>.
Hagemann, Karen. “Gleichberechtigt? Frauen in der bundesdeutschen Geschichtswissenschaft.“ Zeithistorische Forschungen/Studies in Contemporary History (Online-Edition) 13, no. 1 (2016). https://doi.org/10.14765/zzf.dok-1418.
Heinsohn, Kirsten, and Claudia Kemper. “Geschlechtergeschichte, Version: 1.0.“ Docupedia-Zeitgeschichte, 12/4/2012. http://docupedia.de/zg/Geschlechtergeschichte.
Klarfeld, Roman Aron. “Gesammelter Feminismus: Das Berliner FFBIZ.“ Werkstatt Geschichte 75 (2017): 43–48. https://werkstattgeschichte.de/alle_ausgaben/in-bewegung/.
Kurvinen, Heidi. “Towards Digital Histories of Women’s Suffrage Movements: A Feminist Historian’s Journey to the World of Digital Humanities.” In: Digital Histories: Emergent Approaches within the New Digital History, edited by Mats Fridlund, Mila Oiva, and Petri Paju, 149–163. Helsinki: Helsinki University Press, 2020. https://doi.org/10.33134/HUP-5-9.
Offen, Karen M., Ruth Roach Pierson, and Jane Rendall. “Introduction.” In: Writing Women’s History: International Perspectives, edited by Karen M. Offen, Ruth Roach Pierson, and Jane Rendall, xix-xli. London et al.: MacMillan, 1991.
Schroeder, Hiltrud. “Helene Lange.“ Digitales Deutsches Frauenarchiv (2021). https://www.digitales-deutsches-frauenarchiv.de/akteurinnen/helene-lange. Last visited 12/12/2023.
Scott, Joan W. “Gender: A Useful Category of Historical Analysis.” The American Historical Review 91, no. 5 (1986): 1053–75. https://doi.org/10.2307/1864376.
By Simon Krahé
Editorial note: Simon Krahé is finishing his undergraduate studies in English and American Studies, History and Computer Science at the University of Wuppertal. He will use network analysis methods in his bachelor’s thesis. His research interests include digital humanities and digital history practices, especially the use of Natural Language Processing (NLP) methods on large corpora of cultural heritage data.
Network analysis can be a valuable tool for digital history research: it can lead to new insights into social structures of historical actors and organizations and can also track developments with a diachronic approach, i.e. comparing data at different points in time. In the past months, I have been doing network analysis as part of a larger project at the GHI in Washington, DC, by Prof. Dr. Simone Lässig analyzing the history of the Arnhold family. 1 The Arnholds were a German Jewish family that operated a family bank in Dresden since 1864. Members of the family were part of a larger business network in Germany and also served as board members at other companies. In the context of this project, network analysis offers the opportunity to examine changes in a network of board members and corresponding companies in Germany in the 1920s and 30s. In the following blog, I will demonstrate how structured print data can be efficiently, without manual transcription or correction, converted into a dataset that can be further used for such a network analysis. The basis for this is the 1932 “Adreßbuch der Direktoren und Aufsichtsräte (Band 1)” (address book of directors and boards, volume one),2 I have used a Python Jupyter notebook for the implementation.3
As a first step, I examined the source material visually to determine its capability for digitization. A typical page (see fig. 1) of the address book consists of five types of content: page numbers (red), name lines (blue) that include names, titles and often a listed occupation, addresses (green) with varying amount of specificity, position descriptions (purple), and the companies the descriptions relate to (yellow). Fortunately, at a first glance, all of these types can be distinguished based on their formal characteristics such as being aligned to the left or the right, the level of indentation and the use of italics, which makes these pages a promising case for efficient digitization (here meaning a transformation to a digital queryable format that preserves the semantic context of the print data).
Figure 1: Page 10 of the 1932 address book with the different data types highlighted 4
After assessing the page layout, the document had to be scanned. Luckily, the German Museum, a museum dedicated to the history of science and technology, has already digitized the address book and made it available online. The next step was to extract the text itself, which further requires the use of optical character recognition (OCR) techniques. While there are multiple software solutions for OCR, I used Transkribus with the Transkribus Print M1 model for this project due to its superior accuracy compared to Tesseract 5 and Abbyy FineReader 14, which performed poorer.5 In addition, Transkribus made it easier to work with its output data for the purpose of this network analysis.
Transkribus’ user interface structurally mimics the output of the format it provides, PRIMA PAGE XML. One can see a hierarchical structure that consists of regions, within which lines reside. Lines consist of smaller lines that are drawn between individual points, as can bee seen in fig. 2. The points’ coordinates are aligned to the top left of the overall area. The PAGE XML representation (fig. 3) also includes the coordinates of the line on a word level, but this has no further use here. Rather, the XML elements “TextLine” and its children “coords” and “TextEquiv”/”Unicode” contain the information on the coordinates of the text and the text itself. It becomes clear here that the text can be processed on a line-by-line basis.6 In the specific example shown in fig. 2 and 3, the original line “Agethen, Franz;” is split into multiple “Word” elements (namely “Agethen,” and “Franz,”) in the OCR output with each having specific coordinates. The entire line’s content, “Agethen, Franz,” is seen in line 58 of fig. 3, and its coordinates in line 43 respectively.
Figure 2: Transkribus user interface7
Figure 3: XML representation of the same excerpt8
Interfacing with the XML documents that contain the lines was done through the “xml” package for Python. For each “TextLine” element, I saved its coordinates and the associated text. I did not, however, require all coordinates since the maximum and minimum values for the x and y axes are enough to assess which area is covered by the line. These, alongside the coordinates for the maximum values of the text on the overall page, were saved.
Figure 4: XML representation of a line with the information that is saved or further processed highlighted
This information was then sufficient to classify each line. The following rules, applied in this order, constitute the basic algorithm applied. Some edge cases requiring extra consideration are introduced at a later point in this post.
If a line begins at the very left of a page, i.e., its leftmost point is close* to the left edge of the line positions, it is considered a name line, containing the name and potentially other information like the occupation of a person.
If a line is indented only once and its rightmost point is not close* to the right page edge, it is classified as the beginning of a new company name. Indentation can be measured with a constant that describes the starting position relative to the overall page width. Excluding lines that border the right edge of the page minimizes false positives in the form of long addresses that spread so far left into the line starts at the same point that company information lines do.
An exemption occurs if no position description (e.g. “board member of”) was read between the last read person and the given line; in this case it is not a company name, but a continuation of the person’s name on a second line. Here, the line’s content is added to the previous line and the current line itself is set to be ignored in further processing.
If a line is indented twice and does not reach the right margin of the page,* it is assumed to contain a position description.
A similar edge case to above applies, as long company names that continue on the following line are then indented again, starting on the same level. A distinction can be easily made by the human eye, as position descriptions are written in italics and company names are not. Unfortunately, neither is this information transcribed in any of the tested sufficiently well performing OCR tools, nor is the rule of using italics applied without exception by the original authors.9 However, there is still a way to distinguish the two fields as the position description text is thankfully quite predictable. By examining the source materials, certain keywords can be found with which the differentiation can be made.
If the line reaches the right edge of the page*, it contains an address.
If neither of the previous cases applies, the line is classified as unclear. This result is logged.
* tolerance margins apply
Once the initial classification process had been completed, I moved on to the next step and revisited all lines for the purpose of encoding the relationships contained within the data. The principal logic behind this step is quite simplistic as the following lines illustrate:
After a person entry is read:
After an address entry is read:
Add the address as that person’s address in their dataset
After a position information entry is read:
After a company is read:
Add the company to the person’s dataset under the above relationship
The stored data at this point then closely resembled the print version, as seen in fig. 5.
Figure 5: Excerpt of data saved internally in dictionaries and lists
In implementing and testing these algorithms, some edge cases can occur that have to be considered. That is, for example, the case for multi-line companies and names, which I have described above. Additional challenges may arise when Transkribus generates two separate lines in situations where a human reader would not. This situation typically occurs when there are extended pauses in the scanned text, which do not necessarily indicate a transition to a different content category.
Figure 6: One line is split in two, see the small line index in front of the word “Bergwerksdirektor”10
Only one out of two lines can be accurately classified based on its starting position. The content from the other line must be merged with that of its adjacent line, after which the original line will be disregarded in subsequent processing. Neighborhood is determined by the closeness of one line’s minimum x value and another’s maximum x value, while they are on a similar average y-value, i.e. when they are close together, which a human would perceive as a ‘normal’ line. While this sounds straightforward so far, the unintuitive case of the first line in the determined reading order being to the right of its neighbor needs to be considered as well. This occurs when the rightmost line is a bit higher than its neighbor. This behavior turns out to be an even bigger problem when it leads to addresses being listed right before the person they belong to, requiring special cases for such behavior in the above algorithm.
Figure 7: Constants, including keywords that are used in the classification and grouping processes
At the end of this process, we will need to consider the data points necessary to facilitate the implementation of a network analysis. The purpose of this specific network analysis was to identify relationships from person to person based on their mutual relationship to a company. This does, however, require that the company is recorded with the same name for both persons. At this point, errors caused by noisy OCR become visible. Without error correction, even a small mistake such as reading “in” as “m” could lead to associations not being made, which extends to network analysis metrics being imprecise as well. To find a solution, I had to check the company names that are being read against a list of already existing company names. If the string of a company’s name is close enough to another for it to be considered the same company with an OCR error in the transcript, the two names are automatically consolidated based on that criterion. Closeness in this case is measured with Levenshtein distance, i.e., the number of characters that need to be modified for string a to become string b.11
Once this error correction was completed, I exported the data. The use of simple structures to save it allowed for quick traversal. From the data, I created a NetworkX network in Python to export directly to the gexf format used in other network analysis tools such as Gephi. Furthermore, the data can also be exported to comma separated value (CSV) files (one for persons, companies, and their respective relation each) to facilitate reusing the dataset with other tools.
While the initial design and implementation of the algorithms for line classification and recreation of the hierarchy of the print source seems straightforward, troubleshooting, adjusting the constants and accounting for edge cases unsurprisingly proved to be very labor intensive. This was partially due to this specific project that required a low consistent error rate. I have successfully tested the script with a multi-page source sample. I was unable to process the entire address book straight away as some of the pages contain ads which need to be filtered out first.
To further improve the program, the processes of line merging (for lines with the same and with different heights), restructuring the data into hierarchical form, as well as cleaning and separating data within the now differentiated fields (such as a company name from the city of the company on the same line, a person’s occupation from their first and last name, etc.) will need to be improved. If we test the process with a larger test set, we can also expect additional edge cases.
Nevertheless, the working script proved to be much more efficient than manual transcription would have been. The potential reuse of the working script for address books from other years that follow the same stylistic guidelines makes it even more attractive. As a proof of concept, the process demonstrates the potential advantages of using digital methods to curate datasets for historical analyses of networks. Processing the full address book based on the workflow that I have outlined would be worthwhile, and promises to yield interesting insights into the shifting landscape of board memberships in 20th century Germany.
1 See the research overview of “Family and Enterprise in the Age of Industry” (https://www.ghi-dc.org/research/german-european-history/family-and-enterprise-in-the-age-of-industry), last accessed December 13, 2023, for more.
2 Mossner, Julius (ed.): Adreßbuch der Direktoren und Aufsichtsräte Band I. Jahrgang 1932., Berlin, 1932. https://digital.deutsches-museum.de/de/digital-catalogue/library-object/904368/, last accessed December 13, 2023.
3 At this point, credit should be given to Vanessa Tissen, whom I cooperated with on the wider effort of using network analysis methods to gain insights into the Arnholds’ network and with whom I developed the objectives of the script in the project context together.
4 Mossner: Adreßbuch, p. 10.
5 Nevertheless, OCR errors still occurred, such as the semicolon in the example given in the following figures being recognized as a comma due to the light print on its dot.
6 The underlying assumption for this process is that no two types of content intermingle on a single line. While, in case of a short name and a short address, these are printed at the same height, constituting a line in a human readable sense, the empty space between the two elements prompts Transkribus to create two distinct line elements.
7 The shown excerpt is from Mossner: Adreßbuch, p. 10.
8 The shown excerpt is from Mossner: Adreßbuch, p. 10.
9 See, for example, Mossner: Adreßbuch, p. 6, under the entries of the Cologne Mayor Conrad Adenauer.
10 The shown excerpt is from Mossner: Adreßbuch, p. 6.
11 As an example, the strings “abcd” and “abba” have a Levenshtein distance of two.
By Linus Lanfermann-Baumann
Editorial note: Having finished his undergraduate studies in History and English at Göttingen University, Linus Lanfermann-Baumann is in the final phase of his master’s degree in History at Heidelberg University. He also spent a virtual semester abroad at McMaster University and was a visiting graduate student at Yale University in 2022-23. His research on German migration to Canada during the Weimar Republic is forthcoming in the Oldenburger Jahrbuch, and his study of public memories of the Communist resistance against Nazism in Western Germany is forthcoming in the Jahrbuch für Historische Kommunismusforschung. His current interests include U.S. foreign relations during the Cold War and inter-American relations. Linus completed his internship at the GHI Washington in the summer of 2023.
The Foreign Relations of the United States series, prepared by the State Department’s Office of the Historian and commonly known under its abbreviation FRUS, is one of the longest-standing, most prolific, and most cited publication series of edited historical sources in the United States and around the world. As such, it plays a prominent role in researching and teaching the history of U.S. foreign affairs. In recent years, the Office of the Historian has digitized most of the more-than-500 individual volumes in what is by now a well-advanced (if still ongoing) process. The purpose of this blog entry is to introduce the series by briefly tracing its history, to evaluate the various ways of accessing FRUS materials online, and to offer a glimpse of digitized historical sources beyond FRUS that can also be used to study and teach U.S. foreign relations.
162 Years and Going
The Foreign Relations series was started in 1861, some eight months into the Civil War, under President Abraham Lincoln and Secretary of State William H. Seward. Since then, it has undergone major organizational and conceptual transformations that those using the series should be roughly aware of.[1] The first thing to note is that, essentially, FRUS is the executive branch of the U.S. government reporting on itself. This official nature has always entailed a fundamental tension between the government’s parallel interests in both transparency and secrecy – the first bestowing democratic legitimacy, the second protecting national security (or so it is argued).
Between 1861 and 1925, FRUS publications were largely a means for Congress, the legislative branch, to monitor the executive as part of the checks-and-balances system. Editing and review procedures were not highly formalized. This began to change in 1925, however, when Secretary of State Frank B. Kellogg issued an order that professionalized the production of future volumes, sparking the interest of academics in the series.
FRUS’s strong focus on State Department records shifted with the expansion of the national security state during the Cold War. Editors began to include documents from other departments and agencies involved in foreign policy and to draw on significant individuals’ private papers. Clearance procedures became more complicated with this documentary expansion, highlighting the ever-present tension between the contradictory needs for transparency and secrecy then shaped by Cold War objectives. Consequently, the “publication lag” (the time between events and their public documentation in the series) rose constantly, reaching more than thirty years by the late 1980s.
The first major FRUS controversy of the Cold War era was the publication under Republican congressional pressure of the “Yalta Papers” in 1955, only ten years after the famed conference attended by then-president Franklin D. Roosevelt, a Democrat.[2] A Historical Advisory Committee (HAC) consisting of academic historians was established as a consequence of this episode and still exists today. Strong criticism was also directed at the 1983 and 1989 volumes that covered U.S. activities in Iran and Guatemala but, hampered by the CIA and a new “Classification/Declassification Center” at the State Department, blatantly omitted much of Washington’s well-known role in overthrowing these countries’ elected leaders in 1953 and 1954.[3] Exacerbated by the resignation of the chair of the HAC, the ensuing crisis led to Congress passing a special FRUS statute in 1991, which demanded that documentation henceforth be “thorough, accurate, and reliable.” The statute thus reconfigured the balance between transparency and secrecy in favor of the former, even if the documentation of the Congo Crisis, the mass killings in Indonesia, and the proposed covert action in Greece during the 1960s continued to draw criticism in the 1990s and early 2000s.[4] Meanwhile, a strategic initiative advanced a plan that would make future volumes more flexible and accessible: electronic publication.
The Digitized FRUS
Since the “Future of FRUS” initiative introduced the idea in the late 1990s, efforts at electronic publication and digitization have come a long way.[5] Online access to the series is currently available via the websites of three different institutions: the State Department’s Office of the Historian; the University of Wisconsin-Madison Libraries; and the commercial database service HeinOnline, otherwise known for its legal materials. Although many purposes may be served by either of these three platforms, there are nevertheless important differences that make them more or less useful depending on the user’s objective. The following table summarizes these differences:
|
Office of the Historian (State Department) |
University of Wisconsin-Madison |
HeinOnline |
URL |
https://home.heinonline.org/content/foreign-relations-of-the-united-states/ |
||
Access |
free |
free |
subscription-based |
Completeness |
all volumes |
375 volumes covering 1861 to 1960 only |
all volumes |
Online view |
full web edition |
JPG facsimile preview |
PDF facsimile preview |
Download options |
full ebook download, limited PDF download (only vols. published in 2004 or later) |
full PDF download (of available vols., see above) |
limited PDF download (200 pages max. per download) |
Microfiche supplements |
yes |
no |
yes |
Perhaps the most intuitive place to look is the official website of the Office of the Historian – which, indeed, has a lot going for it. There is free access, for one, and the content will always be up to date. Now that all the main volumes are available online, the Office is currently publishing updated digital editions of the microfiche supplements compiled in the 1980s and the 1990s that documented the Eisenhower and Kennedy years despite budget restraints.[6] Users have two main options here: They can either use the FRUS web edition, which includes the option to search the full text either of a given volume or across all volumes, or they can download an ebook in either of the EPUB or mobi formats. OCR-searchable PDF versions are also available, but only for volumes published in 2004 or later and for the updated microfiche supplements. Finally, and in line with Barack Obama’s 2009 Open Government Initiative as well as the 2012 Digital Government Strategy, the source files are freely available on GitHub, even leading one scholar to use it as a test case for her proposal of “New Archival History.”[7]
The second option builds on a collaboration between the University of Wisconsin-Madison Libraries and the University of Illinois at Chicago Libraries and offers facsimiles of the printed volumes rather than a full-fledged web edition. Like the State Department’s website, the database is freely available and fully searchable, both within a given volume and across the entire series. No FRUS volumes are available beyond those covering the 1958-60 period, however. The key advantage of the University of Wisconsin facsimiles is that all available volumes can be downloaded freely as PDF versions.
Unlike these first two platforms, the HeinOnline database is not freely available but subscription-based. There are institutional subscriptions, however, so that affiliates of certain institutions would not have to pay out of their own pocket to gain access. HeinOnline has all the volumes available at the Office of the Historian website to view online, including full-text search options, and individual chapters can be downloaded as PDF files. Unfortunately, while there does not seem to be a cap on downloads overall, any individual download is limited to 200 pages.
In short: Those who prefer working online with a fully developed web edition rather than facsimiles, or offline with ebooks rather than PDF files, should use the digital materials offered by the Office of the Historian. Those who prefer PDF files and are not interested in the period after 1960 may want to use the University of Wisconsin-Madison database. And those who are looking for PDF files available neither at the Office of the Historian nor the University of Wisconsin Libraries (i.e., volumes published in print before 2004 that cover the post-1960 era) may want to consult HeinOnline.
Digitized Sources Beyond FRUS
Of course, there are many further options beyond the Foreign Relations series for studying and teaching U.S. foreign affairs with digitized primary sources. I cannot offer an exhaustive overview here but will merely refer to some of the most important resources.
For starters, the Office of the Historian offers a useful overview of various electronic resources on U.S. foreign affairs. Perhaps most important are the “electronic reading rooms,” a result of one of many amendments to the 1966 Freedom of Information Act. Signed by Bill Clinton in 1996, the so-called “e-FOIA” amendments provided for increased electronic access to government documents requested through FOIA and beyond. For example, electronic reading rooms are available on the CIA, the State Department, and the Department of Defense websites.[8] Declassified documents can also be obtained through the U.S. Declassified Sources Online archive run by the private publisher Gale, the WikiLeaks public library PlusD, and the National Security Archive, a nonprofit organization advocating government transparency.
HeinOnline’s above-mentioned FRUS database holds related subcollections, most of which are also freely available elsewhere. These include various government documents, such as Title 22 (Foreign Relations) of the Code of Federal Regulations, congressional reports, and official publication series of primary sources other than FRUS, such as American Foreign Policy: Current Documents.
Those interested in the role of U.S. presidents in foreign relations can consult the American Presidency Project at the University of California at Santa Barbara, which provides free access to the well-known Public Papers of the Presidents series, originally compiled by the Office of the Federal Register at the National Archives and Records Administration (NARA), and much more. The University of Virginia’s Miller Center runs a useful Presidential Recordings Program whose digital edition offers free access to at least some of the presidential tapes. NARA, furthermore, provides access to some of its electronic records through its Access to Archival Databases (AAD) resource, including State Department telegrams from the 1970s.
Overall, documentation of the historical role of the United States in the (analog) world has increasingly been made available in the digital world. While the FRUS series remains an important archive for many scholars and instructors, numerous other online resources have also facilitated and enriched research and teaching, and they will continue to do so in the foreseeable future as historians dive deeper into the digital age.
[1] The authoritative account of FRUS’s history and the basis for the following remarks is a meticulous self-portrayal by researchers at the Office of the Historian that largely supersedes two earlier article-length assessments by historian Richard W. Leopold. See William B. McAllister et al., Toward “Thorough, Accurate, and Reliable”: A History of the Foreign Relations of the United States Series (Washington, DC: U.S. Department of State, Office of the Historian, Bureau of Public Affairs, 2015); Richard W. Leopold, “The Foreign Relations Series: A Centennial Estimate,” The Mississippi Valley Historical Review 49, no. 4 (1963): 595–612; Richard W. Leopold, “The Foreign Relations Series Revisited: One Hundred Plus Ten,” The Journal of American History 59, no. 4 (1973): 935–57.
[2] Department of State, FRUS: Diplomatic Papers, Conferences at Malta and Yalta, 1945 (Washington, DC: Government Printing Office, 1955). The editing institution, the place of publication, and the publisher are the same for all FRUS volumes and omitted in further references.
[3] FRUS: 1952–1954, The American Republics, Volume IV (1983); FRUS: 1952–1954, Iran, 1951–1954, Volume X (1989). More complete FRUS volumes have since appeared on the U.S. role in these two coups, see FRUS: 1952–1954, Guatemala (2003) and FRUS: 1952–1954, Iran, 1951–1954 (2017), the latter of which was slightly expanded for a second edition in 2018.
[4] FRUS: 1958–1960, Africa, Volume XIV (1992); FRUS: 1961–1963, Volume XX, Congo Crisis (1994); FRUS: 1964–1968, Volume XXVI, Indonesia; Malaysia-Singapore; Philippines (2000); FRUS: 1964–1968, Volume XVI, Cyprus; Greece; Turkey (2002). Some of the more sensitive records on CIA covert action during the Congo Crisis were included much later, see FRUS: 1964–1968, Volume XXIII, Congo, 1960–1968 (2013). Besides the official FRUS history (footnote 1), see also Page Putnam Miller, “We Can’t Yet Read Our Own Mail: Access to the Records of the Department of State,” A Culture of Secrecy: The Government Versus the People’s Right to Know, ed. Athan G. Theoharis (Lawrence, KS: University Press of Kansas, 1998): 186–210. And see Matthew Jones and Paul McGarr, “‘Real Substance, Not Just Symbolism’? The CIA and the Representation of Covert Operations in the Foreign Relations of the United States Series,” Intelligence Studies in Britain and the US: Historiography since 1945, ed. Christopher R. Moran and Christopher J. Murphy (Edinburgh: Edinburgh University Press, 2013): 65–89.
[5] See McAllister et al., Toward “Thorough, Accurate, and Reliable, 323–24 for the original initiative.
[6] As of August 2023, four out of the thirteen microfiche supplements have been made available online.
[7] Eun Seo Jo, “Foreign Relations of the United States, 1860–1980: A Study in New Archival History” (Stanford University: Ph.D. dissertation, 2020), available at https://purl.stanford.edu/km610md5945. See https://github.com/HistoryAtState/frus for the TEI XML source files (many thanks to Simon Krahé at the University of Wuppertal for his helpful explanations). See also https://history.state.gov/open.
[8] The Department of Defense’s FOIA program is decentralized, see https://open.defense.gov/.