OldNews USA contains features that make it easier to find articles about your ancestors in historic newspapers.  In part one of this article, I explained the different types of searches that OldNews USA provides, and how the app generates search suggestions.  In this article, I’ll show you my approach for greatly reducing the amount of time it takes to perform a thorough search for an article about a person.

The Problem With Newspapers

Experienced researchers know how difficult it can be to perform a reasonably exhaustive search for a person in the newspapers.  Consider the following difficulties:

  • You never know how a person’s name will appear in the newspaper. Will it be the person’s full name, first initial and surname, or Mr./Mrs. surname?
  • There can be many alternative spelling variations for a surname, since names were often spelled as they sounded.  Variations for a person’s given name are also common, especially when considering abbreviations or nicknames. For example, the name William could appear as Wm, Will, Willy, Bill, or Billy.
  • The name could be misspelled due to errors in the digitization process.  When you search historic newspapers, your search terms are being compared to text generated by Optical Character Recognition (OCR) software.  OCR converts the newspaper image into searchable text, but the accuracy of the conversion varies greatly, especially if the original image is faded or of poor quality.

The Hack

Clearly you need to try many different search variations before you conclude that the elusive article about your person just doesn’t exist.  You could manually enter each search variation into the Search By Newspaper and Search By State screens, but that is a lot of work, even if you always modify a previous search to create the next variation.

You can leverage the OldNews USAs ability to automatically generate search suggestions by modifying your Person Goal instead of modifying an individual search.  Every time you modify the goal, the app will generate up to 20 new search suggestions based on the modified data.  You can then quickly try each search variation, then modify the goal again, try the next set of searches, and so on.  How cool is that?

To modify the goal data, simply tap on the Current Goal card to display the Goal Details screen.  Make the your next spelling change, then click the Save button.  When you return to the main screen, the Search Suggestions card has your new search variations ready to go!

Touch the Current Goal card to edit your goal data

Case Study

Let’s see how I apply this search hacking technique using my gr-gr-grandmother Ann Doran as an example.  She lived in Minneapolis MN from about 1890 until her death, which was likely sometime before 1920.  My main objective is to find her obituary, since I am hoping to find a clue about her place of birth in Ireland.  This is a good example because her name is very common, and there are numerous variations of her first and last names.

 

A Person Goal for my search for articles about Ann Doran

Pro Tip:  If you enable research logging for your goal, the app will log every search that you try.  This can help you avoid repeating the same search, and makes it easier to pick up where you left off.  Just check the Enable Research Log checkbox at the bottom of the Goal Details screen to turn on logging.  You can access the log from the Research Log button on the bottom left of the Current Goal card.

Name Variations

I’ll start my search by entering Ann Doran as the Person Name in my goal.  OldNews USA will generate the following name variations as search suggestions:

  • Exact Phrase search for Ann Doran
  • All Within search with the words Doran and A. within 1 word of each other
  • All Within search with the words Doran and Mrs. within 1 word of each other
  • All Within search with the words Doran and Ann within 5 words of each other

 

Some of the generated search suggestions for my goal

It also generates a number of other search suggestions using the surname Doran in combination the name of the town and other optional keywords that I specified in the goal data.  I won’t list them all, but the searches vary the terms and level of specificity to maximize your chances for finding a match.  You just never know how the person’s name will appear in the newspaper.

Other goal variations worth trying are to use her maiden name Ann Mitchell, and also her deceased husbands name James Doran.  Since Ann was married long before moving Minneapolis, her maiden name is not very likely to appear in an article, except perhaps for her obituary.  However, she may very well appear in the papers as Mrs. James Doran or Mrs. J. Doran, both of which should be found by using her husband’s first name.

First Name Variations

I also need to consider variations of her first name, since her first name was often spelled Anne or Annie.

Before considering alternative surname spellings, I modified my Person Goal 5 times, resulting in a total of 100 search suggestions for each of the following name variations:

  • Ann Doran
  • Annie Doran
  • Anne Doran
  • James Doran
  • Ann Mitchell

Alternative Name Spellings

I have records that spell Ann’s surname Doran, Duran, Dorain, Dorrain, and Dorian.  I suspect that she pronounced her surname with a long A sound and an Irish accent, which led to the various alternative spellings in records.  These spelling variations give me 4 additional ways to hack my Person Goal data, resulting in the 12 more goal variations, and 120 new search suggestions:

  • Duran: Ann Duran, Annie Duran, Anne Doran
  • Dorain: Ann Dorain, Annie Dorain, Anne Dorain
  • Dorrain: Ann Dorrain, Annie Dorrain, Anne Dorrain
  • Dorian: Ann Dorian, Annie Dorian, Anne Dorian

Misspellings Due to OCR Errors

The Ancestor Hunt blog has a great article that explains some common OCR errors, and identifies letters that are most likely to be incorrectly translated by the OCR software: 8 Ways to Overcome OCR Errors when Searching Newspapers

Using the suggestions in the above article, a capital D could easily end up as a capital O in the OCR text, and if that happened, I would need to intentionally misspell Ann’s surname as “Ooran” in my search to find that article. Similarly, the letter n is often converted to ri.

The bad news is that there are many spelling variations to try.  The good news is that all of  searches are most likely to come back with no matches, so you can quickly check each variation.  Also, since a match of a misspelled surname such as Oorari is very unlikely, you probably only need to try one general search for the surname that covers the entire state for each misspelling.

For this hack, rather than modifying the goal data, I scrolled all the way to the bottom of the Search Suggestions list, selected the search-by-state suggestion that just contains just her surname, and modified that search for each of the following variations:

  • Doran -> Ooran, Dorari, Oorari
  • Duran -> Ouran, Durari, Ourari
  • Dorain -> Oorain, Dorairi, Oorairi
  • Dorrain -> Oorrain, Dorrairi, Oorrairi
  • Dorian -> Oorian, Doriari, Ooriari

Are Intentional Misspellings Necessary?

How much effort do you really need to put into searching for intentional misspellings due to OCR errors?  There are two ways to gauge the likelihood of errors.  The first way is by quality of the images that you look at in response to some of the search results with correctly spelled names.  If the images are faded, blurred, stained, or otherwise hard for you to read, the OCR software probably also had a hard time recognizing the letters.  The second way is to closely look at the text except that OldNews USA displays with each search result.  This text is a portion of the OCR-generated text, so if it is gibberish, the chances of OCR errors throughout the page is very high.

An example of a text excerpt with OCR errors. Notice how the character ‘s’ was incorrectly translated as a ‘3’ in two places.

Almost all of the articles I found about Ann Doran were published in The Irish Standard newspaper.  The quality of these images is excellent.  As the above example shows, the number of OCR errors is quite low, with 2 incorrect characters out of 200, or a 1% error rate.  Of course this is just one example, but many of the excerpts that I looked at didn’t have any errors. Given the quality, I didn’t spend a lot of time trying intentional misspellings, but I did try some.

Exhaustive or Exhausting?

If you are keeping score, I started by creating a Person Goal for gr-gr-Grandmother Ann Doran.  I then modified this goal 17 times, by changing just a few letters in her name for each alternative name spelling, include both first and last name variations.  Each time I modified the goal, OldNews USA automatically generated 20 new search variations that I could use without needing to enter any additional data.  That is 340 searches that I can execute with a single click.  I also created 15 more surname variations with intentional misspellings to try to find articles with OCR errors.

Did I really try all 355 search variations?  Of course not, and I don’t expect that you will either.  If you are looking for a common surname, many of the general searches will return too many results to reasonably search.  If you have an uncommon surname (lucky you!), the specific searches won’t be necessary.  You can decide which of the suggestions make the most sense to try based on your needs.

The Results

You may be wondering what articles I was able to find about Ann Doran using my technique.  I found a few brief social news articles that mentioned her, along with some articles about her sons.  I also found her obituary using the name Annie Doran.

Unfortunately, there was no mention of her birthplace in Ireland (sigh).  Newspaper research is like Forrest Gump’s box of chocolates.  You never know what you’re gonna get!

Download OldNews USA today and try my search hacking technique.  By modifying the goal data and using the automated search suggestions,  it possible to perform a reasonably exhaustive search for articles about a person with minimal data entry.  That elusive article that breaks down your brick wall may be out there!  You just might need a thorough search to find it.

 

Newspaper Search Hacking with OldNews USA

One thought on “Newspaper Search Hacking with OldNews USA

Leave a Reply

Your email address will not be published. Required fields are marked *