Photo by Denny Müller on Unsplash

We can picture the body of literature in any research field as a river. Ideas spring up over time, start as a just trickle of papers, but then grow in number as others expand upon the earlier work. Tools or concepts introduced from different topic areas may then act as tributaries that further increase the flow of research. What results is an ever increasing volume of work that can completely overwhelm anyone who tries to wade in. So how can a new grad student safely navigate the river?

There are a number of different approaches for identifying literature relevant to your topic area. You can consult Wikipedia. You can search for keywords. You can scroll through tables of contents in journals specific to the field. It really doesn’t matter what technique you use initially, provided you can make your way somehow to the shoreline.

But once you’ve identified a relevant article, reference searches are a great tool for finding related work. Think of those initial documents as the base camp, the yellow tent on the shore, from which you can explore further up or down river.

Upstream: a reverse chronological search

Once you’ve pitched your tent, one option is to work your way upstream. We’ll refer to this as a reverse-chronological search because you’re looking back in time for earlier research. The nice thing about this type of search is that it is fully defined; the reference list in our base document serves as our map.

Let’s assume you’re interested in high-ankle sprains in hockey. In your initial searches, you find a 2004 article called “Ankle syndesmosis sprains in National Hockey League players” by Wright et al. in the American Journal of Sports Medicine. You decide to use it as your base document. Looking at the list of references in the article, you find a total of 19 cited works:

Tracking down 19 items is a reasonable task, but the process can quickly become intractable as the number grows. It is not uncommon for some documents, particularly book chapters or review articles, to have many hundreds of references. We need a way to reduce that number to something more manageable.

Thankfully, the text itself often gives you clues as to which of the cited works are particularly important. They may be referenced more frequently, they may be discussed in more detail, or the context in which they are cited may also give you some indication that they are particularly relevant. Even the titles may be helpful. For example, reference [16] in Wright et al. is about basketball and may be of less interest than reference [17]. If your focus is on hockey, it makes sense to prioritize the latter reference over the former.

Whatever strategy you use, you will end up with a series of earlier works that all contributed in some way to your base document. You may find that those references are themselves interrelated based on their own citations. Nevertheless, it is important to emphasize again that all the documents in a reverse-chronological search are known. There may be hundreds of documents in the reference list, and some of them may be difficult to obtain, but there is still only a finite number. Plus, you know exactly what you’re looking for because they are all listed in one place.

Once you’ve reviewed those earlier documents, you may want to repeat the process again by searching their references. You can search back multiple generations, if needed. Although there’s no hard and fast rule about how far back you need to go, eventually you will have to stop due to exhaustion, lack of interest, or the inability to access some documents.

A reverse-chronological search is helpful because it provides a historical perspective on how a field has evolved with time and helps to identify seminal papers. Only looking backwards, however, is insufficient. We must keep in mind that newer research may have expanded, altered, or disproved those earlier ideas.

Downstream: a forward-chronological search

Although an upstream search is helpful for finding the foundational research in an area, it can’t tell you what has happened since that time. To identify newer research, we need to explore downstream to find more recent documents that have cited it. We’ll call this a forward-chronological search, because we’re looking forward in time from our base document.

Doing so will require a citation index. As the name suggests, a citation index keeps track of documents that cite one another. Some are commercial services that require an institutional subscription like Web of Science or Scopus, whereas others are free services (e.g., Google Scholar, Semantic Scholar, or Microsoft Academic).

Each works in a similar way. For example, when we search for our base paper, we get the following results from four different services:

By clicking on the circled links, we’ll get a list of documents that have cited this paper by Wright et al. Unfortunately, there are no contextual clues to indicate which of these newer documents are particularly relevant. We have to rely on their titles and abstracts to prioritize those that we want to read.

The fact that the numbers of citations listed by the four different services vary between 81 and 163 highlights an important limitation. Citation indexes can only report information about documents included in their databases. There are likely some additional, unindexed works that still cite your base document. The greyed documents in the figure above are included as a reminder of this important shortcoming: a reverse-chronological search is fully defined, but a forward-chronological search is not. Because we are at the mercy of their coverage, it is always advisable to use two or more services to reduce the likelihood of missing something important.

It is also worth noting that the four screenshots of the different indexes were all made on the same day and within minutes of each other. If you were to repeat the searches today, however, you would probably find that the numbers of citations had grown. New material is constantly being published and then added to the indexes. Therefore, it is advisable to check back periodically for new work, or to set-up notifications (e.g., Google Scholar alerts), to avoid missing out on important developments in your field.

Conclusion

New research acknowledges previous work through references, and each citation creates a link back to the earlier ideas, techniques, or results on which it is built. It is only natural to exploit these relationships to explore the river of literature that develops. References searches are a powerful technique for searching upstream (through reference lists) for foundational work, and for looking downstream (through citation indexes) for finding the most current research in an area. Using these tools should help keep you from drowning in the information available.