Co-occurrence

Two entities (or terms, or concepts) co-occur when they appear together within some window — a sentence, a paragraph, a document. Counting and graphing co-occurrences surfaces relationships that are implicit in how the corpus talks about its subjects, without needing an explicit “relationship” annotation.

Why it matters for your research. Co-occurrence graphs are the classic DH visualisation — who appears in the same paragraph as whom, which places share political context, which concepts are argued together. They are interpretable because they come directly from the text, with no model fabrication in between.

In Archēglyph. On the project roadmap, under Next, scoped per cluster with a configurable window. Depends on NER landing first.

Not to be confused with. Correlation over time — diachronic — is a different question: when entities appear together. Co-occurrence is a structural measure within a chosen window.

A worked example. Suppose two merchants, named separately by NER across hundreds of letters in a correspondence archive, never appear in the same sentence but frequently appear in the same letter alongside a third name — a shipping agent. A co-occurrence graph scoped to the whole-letter window surfaces that triangle even though no single sentence states a relationship between the two merchants directly; the researcher still has to read the surrounding letters to decide whether the triangle means a business partnership, a rivalry routed through a common agent, or coincidence.

Why the window size changes the answer. A sentence-level window finds tight, usually deliberate pairings — two names mentioned in the same breath. A document-level window finds looser association — two names that share a subject without necessarily interacting. Neither window is “correct”; they answer different questions, and a co-occurrence graph built at the wrong granularity for the question being asked will look meaningful while measuring something else entirely, which is why the window has to be stated alongside any co-occurrence graph a researcher publishes.

Raw counts versus weighted association. Counting raw co-occurrences favours frequent entities: two names that each appear hundreds of times will co-occur often by sheer chance, even with no real relationship. Association measures like pointwise mutual information correct for this by comparing the observed co-occurrence rate against what chance alone would predict given each entity’s overall frequency, which is why a serious co-occurrence graph reports an association score alongside — or instead of — a raw count.

Depends on named-entity recognition landing first. A co-occurrence graph is only as good as the entity list feeding it: if NER misses a name or inconsistently tags variant spellings of the same person as different entities, the graph will show two disconnected nodes for what is really one person, or silently drop a real relationship entirely. That dependency is why co-occurrence sits later on the roadmap than NER itself.

Reading the graph rather than the count. A co-occurrence graph is most useful browsed visually, as a force-directed layout where dense clusters of interconnected names suggest a social or institutional group and isolated pairs suggest a one-off interaction. The raw counts behind it matter for deciding which edges are strong enough to draw at all — a graph that draws every co-occurrence regardless of strength tends to look like a solid mass of connections that tells a reader nothing.

A caveat worth stating alongside any graph. A co-occurrence graph describes what the surviving text says, not what happened historically — an archive with gaps, censorship, or selective preservation will under-represent relationships the original correspondence actually contained. Reading a co-occurrence graph as a complete social network rather than as a record of what a particular archive chose to keep is the most common way this kind of visualisation gets over-interpreted.

← Back to the glossary