Indexing for FamilySearch

Our supervisor announced some changes to us in our work arrangements. He plans for us to travel to Berlin next month. The Berlin stake leaders have asked for training in matters relating to Family History, and he wants three couples to go to help them. Our part will be the FamilySearch Indexing program.

When we received our mission call we were informed only that we would be assigned to the area office of Family and Church History Support. While at the MTC in Provo we spent a week training to use FamilySearch. Our first six weeks in Germany were devoted mostly to training to provide support to users who were experiencing difficulties using the FamilySearch Indexing program, but at the end of that our supervisor would not allow that we be given assigned times to serve in that work, so we have been working on projects that he has assigned individually. A recent one was the assignment to review all of the KnowledgeBase documents in German relating to PAF to determine if they are still valid. That one could take a year to complete if we did all that could be done on it, though I doubt that such an effort would be worth it. More recently our supervisor has talked to us about providing the stakes training and motivation to become involved in FamilySearch use. Our particular slice of the pie is Indexing, which is for me one of the FamilySearch applications that is most interesting.
Here in Germany a lot of records have been filmed and are available through microfilm. They were cataloged as they were filmed - that means the Genealogical Society knows what kinds of records were filmed and where they came from, but that catalog can't tell you whose names are on the film. The research strategy for a genealogist must include determining as close as possible where the ancestors lived and at what times important events like births, marriage, and death would have taken place. They would then obtain access to the appropriate records and read them looking for the desired information. This used to require ordering microfilms, but the Church has been working hard for several years to "scan" the microfilm images and put them in digital format so they can be sent electronically to patrons anywhere in the world. No matter how the patron gets them, they must read the entries in the rocords looking for the names they want information about. They may have to read dozens or even hundreds of pages to find what they are looking for. Hopefully, the information they find will also include additional information on other family members to suggest how to move on to a subsequent step in the research.

What is really needed is a name index that would allow a researcher to determine the pages and entry lines where a given name appears. If it were designed properly the index would be able to list every instance of the name in all records from the selected location and time period. The researcher would then be able to select which instance of the name most likely represented the individual for whom they are looking, and exactly where on the record to look for it. In the case of digitized records, the program could even display the record on the researcher's computer screen.

The problem in getting to this point lies in the fact that the microfilm images are just photographs -- graphic images. Computers aren't yet able to efficiently search graphic images for textual information. Such a search does go extremely fast, however, if the data is in text format. What we need is a way to convert the necessary parts of the record into text - things like names, dates, etc., but no computer has yet been perfected that can read the handwriting of hundreds of thousands of different record keepers who wrote records in thousands of different hand-writing forms and hundreds of languages. For now, there is no way computers can do the job.
The only thing flexible enough to do the job is a live human being. Thus we have a need to have intelligent beings read the records and record the key information in text form into a database. Thus we get the reason for the Indexing effort of the church.

Germany is a very fruitful field for this type of work. There were many thousands of Germans who joined the church, and emigrated to America. The descendents of these people are interested in doing their genealogy. They face a double handicap, however: the records containing the necessary information are in a foreign language for the American-born, and the records were written in handwriting-styles that even most German-born people cannot read any more. Looking for a specific name on dozens of pages of German records could take an American researcher a year of spare time. By indexing these records, we can use the computer to identify the exact records on which family names appear, thus reducing the task to one of interpreting a single entry in a record that can be made available on your own computer screen in a matter of seconds.

During my years of work with computers I have repeatedly worked with the technology relating to doing these searches. The first year after I had to quite teaching seminary I worked with The CALICO organization at BYU, and my first assignment was to figure out how to create an index of publications that contained information about Computer-Assisted Language learning or Instruction (CALI). I partnered with a team from the BYU Library that wanted to develop such a search engine to be used in training librarians to use the commercial Dialog services. Later in my career with the Government we were always involved in either training people to use such databases to retrieve needed information or the implementation of such databases to achieve desired ends.
The particular type of index used for these types of searches is called an inverted index. The Church’s inverted indexes now run on hundreds of servers that have been installed in a huge data center in the Eastern part of the United States. Every server in use has another complete server installed to provide a backup. The complete system is huge. They recently moved the system to a new location, and the physical work for the project was supplied by the local stakes. The entire system was moved in such a way that the system continued to run flawlessly throughout the move.

Well that’s enough technical stuff about what the church is doing to support genealogy. Our little place in the machine is one of teaching stake leaders to understand the system so they can provide local leadership to not only use the system, but also provide help with the indexing. It has been found that people who get involved in indexing receive the Spirit of Elijah as a reward for their efforts. Some church leaders are now assigning Indexing work to their youth as one of their ways of keeping them spiritually active. New converts who spend a few hours a week Indexing or researching their genealogy receive the spirit of the work, and are much more likely to remain close to the church. Some bishops and stake presidents are assigning Indexing work as part of the repentance process because of the spirit it brings to the individuals. It has been fascinating to see what happens in the Church as people become involved with this part of the three-fold mission of the church. We are becoming aware that this part is as essential as the other two, and in fact can be the means of achieving the other two. We are even working to get non-members involved, for this could be the means of bringing them into "fellowship with the saints."

So on Friday the thirteenth of February we will be driving to Berlin, and on Valentines day we will give our first full-day training program to the stake leadership there. Between now and then we must plan the training program and prepare presentation materials so we can teach effectively. One important issue is that we need to teach in German. Have fun, Sister Howlett!

Also working its way onto the Spring calendar is another trip to work with local Genealogical groups and offer them the opportunity to help with the Indexing. That way we can get the best talent available on the task to the mutual advantage of all. The only obstacle is the fact that some loccal organizations are trying to do the same things in local areas, and maybe our best method will be to work out partnerships with them rather than operate in competition with them.