Overview

Text mining includes a number of applications, from tracking textual reuse and the fluctuation of certain words or themes over time to stylometry and the modeling of literary forms. These texts might represent a single author’s oeuvre, a periodical’s full print run, or a collection of texts from across multiple centuries. Important to any text analysis project is the curation of a dataset, which often entails digitization and optical character recognition (OCR), the process of turning words from images into searchable text.

Methods & Tools

Out-of-the-box tools that don’t require any custom programming include JSTOR Labs’s Text Analyzer, Voyant, AntConc, and Wordle.

Tools that require minimal command line usage include Stanford’s CoreNLP (which can identify people, locations, and even sentiment in unstructured text) and Andrew McCallum’s MALLET (which generates topic models).

Platforms that require some programming include Bookworm (for tracking a word’s frequency over time) and Andrew Goldstone’s dfrtopics (for visualizing topic models).

Popular packages that require more intensive levels of programming include the Natural Language Toolkit and the Python libraries: datasketch, NumPy, SciPy, and scikit-learn.

Datasets

Alan Liu’s curated collections

McGill .txtLab’s curated collections

Recommended Readings

“Seven Ways Humanists are Using Computers to Understand Text” by Ted Underwood

Stanford Literary Lab Pamphlets