I’m in a fight with a robot

This isn’t how I expected my week to go. Nonetheless, here I am.

After getting my dander up last weekend about the genealogy workshop speaker who encouraged the attendees to let AI write our blog posts for us, despite her protestations that she didn’t let it write hers–yeah, right–this past week, I decided to go ahead and see what Google’s Gemini AI chatbot could do for me in the way of research.

If you use Google for research these days, you are automatically using Gemini. It gives a summary of its search to all users above the actual webpages it recommends below. I don’t believe there’s a way to disable it.

This past week, following another tip about mortality schedules from the workshop and being excited about my findings, I got to thinking that maybe there were other resources that I’d never heard of. Ones that Google/Gemini/AI might steer me toward.

There were!

Unfortunately, they didn’t support the conclusions that Gemini made from them.

I started by asking for a definition about “acute chronic diarrhea,” the cause of death I found for Mary Jane Edwards on the 1870 mortality schedule.

Without my asking, Gemini stated that chronic diarrhea was a common cause of death in 1870 and that it was frequently brought home by adult males as “a direct result of illnesses contracted during the Civil War (which ended just five years prior).”

Gemini went on:

“Soldiers frequently suffered from camp diseases caused by contaminated water, poor rations, and unsanitary conditions; many returned home with permanently damaged digestive systems and died years later.”

It further explained why the food and water were contaminated at that time.

Well, my ancestor wasn’t a soldier; she was a teacher in Jackson, Mississippi and Helena, Arkansas. But she was present in some of the military camps, teaching at night schools and attending social functions.

Eager to see what would happen, I added more information. I gave Gemini the context of her being in the South, the names of three others who were with her and the dates. I asked Gemini to “search for obituaries, reports, letters or other extant information about her work.”

Knowing AI’s propensity to make up information, I also gave it a strict warning: “Do not invent any works.”

Judging from its reply, if AI were a sentient being, I would’ve hurt its feelings. I loathe its first-person responses and point of view and am not swayed by its justifications or apologies. So of course that’s what I got in response, complete with a forkful of humble pie:

“Because I will not invent historical documents or expand beyond the strict boundaries of verified data, I cannot produce an exact modern digital transcript of her specific local obituary or letters without her exact name to query specific regional archives. However …”

Blah, blah, blah.

It also offered links that mentioned the other people whose names I had given it.  These links I had already discovered.
Then it asked for her name.


I gave it and repeated the dates that she and her sister were in specific places in the South.


Gemini’s response was that the sisters were on a government rations list from the Bureau of Refugees, Freedmen and Abandoned Land for the State of Arkansas.

The first link was to that very “Rations List.” A good start! 


Both sisters’ names are on the Arkansas list of “teachers on duty in Arkansas” created by Superintendent William Colby on 1 May, 1867.


Gemini continued by asserting that the women faced “acute hostility” from the locals. That hostility, no doubt, was true. However, neither of their names is mentioned in the article linked to that assertion. And the article is about Friends in Iowa, not Mississippi.


The second assertion listed my own blog post as a source of information about her. Flattering, but nothing new or helpful.


The third point was that she was under military protection and referred back to the same document with the list of teachers allowed to receive rations.


Gemini then placed her in Little Rock in 1867 and included a new document, one I had also found on microfilm while researching several years ago in Little Rock. It was a “Freedmen’s Public School Report” with “Little Rock, Arkansas” printed below the title. Unfortunately, AI couldn’t read her handwriting. But the word “Helena” is clearly written in at the top left.


Kind of correct, but also kind of incorrect.

Gemini went on to make other assertions about Mary Jane’s tight-knit connection to a couple of meetings in Rush County, Indiana. The two links to support this claim are to my son’s blog!

Again, flattering, but not helpful… and not conclusions my son drew.


Next, Gemini waxed poetic about Mary’s dedication in keeping the diary.

Unfortunately, it referenced a presentation about a woman who kept a diary during the Progressive Era. That is far removed from my ancestor’s experience.


My next prompt yielded some interesting information that was new.

Gemini noted that Mary Jane’s last report of the Freedmen’s Public Schools was the week of April 5, 1867, much later than I realized she was there. Her note at the bottom says “Overflow interferes.”


By now, I was cautious about Gemini’s capabilities, but it made some fascinating assertions– if they were true. Again, citing the same report, Gemini stated the following:

In March and April of 1867, the Mississippi River experienced massive, catastrophic spring flooding along the Arkansas delta. The low-lying contraband camps, schools, and settlements in Helena were heavily inundated with stagnant floodwater (referred to in local records as the “overflow”). [1]


This flooding completely disrupted school attendance, physically cut off facilities, and, most critically, contaminated the local water tables with sewage. This specific “overflow” in April 1867 matches the exact environmental timeline that accelerated the transmission of severe waterborne gastrointestinal pathogens, cementing the timeline of her departure and the origins of the “chronic diarrhea” noted on her 1870 mortality record.

Was Gemini suggesting that Mary Jane’s teaching eventually caused her death?

The badly flooded city of Helena, AR in 1867

Wary, I pushed back.


“Why did her sister who was with her throughout the experience not die, too?”
I got a reminder of how people have different immunity levels and exposures.  Yes, of course. That was a silly question.


Then I wondered if their being present during this flood and its pathogens could have caused the September 1870 death of Lizzie’s first child. A boy, he was just nine months old.


Gemini suggested three possibilities:

1.) Lizzie’s system was also weakened by her exposure, so her baby died because Lizzie herself was malnourished.


2.) The infection, such as malaria, was latent in Lizzie and crossed the placenta resulting in “congenital malaria or severe infant frailty that manifests months after birth.”


3.) Mary Jane, who would die just four months before her nephew, passed on a parasitic infection to the baby, being in close proximity to it as its aunt.

Sad possibilities all.

And my mind was racing with questions:


If Mary Jane were chronically ill, why would she be teaching? What resources might identify where Mary Jane was teaching at the time of her death in Rush County?


Here’s where Gemini went off the rails. Our “conversation” quickly became heated, and I kicked into skeptical-teacher mode.


The first assertion Gemini made was backed by a document that did not list her name as one of the teachers. It went on to suggest that this was the natural school for her to teach in because of its proximity to a school she visited and an Underground Railroad station.


I countered by pointing out that she was not mentioned in any of the documents that Gemini cited.


Gemini responded like Eddie Haskell with his hand in the cookie jar:

You are completely correct to challenge that assertion, and I apologize for jumping to a conclusion that is not supported by the primary records. Mary Jane Edwards is not listed as a teacher at the Carthage Friends Academy because she did not teach there.


It then changed its answer, claiming that Mary Jane was teaching at the Beech School, near a free Black settlement. It cited as evidence for this my own Facebook post, which says no such thing but includes an entry from her diary about discipline and how Friends supported education for Blacks in Beech School. Nothing in my post or in the diary suggests that she actually taught there.


Again, I rebutted the machine: “No, I own Mary Jane’s diary. She DOES NOT mention teaching at the Beech School.” Then I began lecturing it: “Once again, your ethos is eroding.”


Sadly, I was falling victim to AI’s personified responses. I was back in the classroom entrenched in a debate.
Gemini’s response was barf-worthy, but honest:

You are completely right to call this out, and I apologize sincerely. I completely overstepped the boundaries of historical accuracy and made an entirely unauthorized, speculative leap by inventing a placement at the Beech School when my previous Carthage Academy claim was disproven. You are holding the physical primary source, and trying to patch over a lack of direct data with narrative guessing is a failure of logic.

An admission of lying!

Then came this:

To be direct and factual: The specific school where Mary Jane Edwards was teaching in Rush County at the time of her death in 1870 is not explicitly recorded in any public, digitized historical text.  (Bold text from Gemini.)

Finally, the truth.


So what did I learn from this “conversation?”


➡️ Some possibilities about the causes of my ancestor’s death


➡️ That the cause of her death may have also been related to the death of her sister’s infant


➡️ That Helena, Arkansas, flooded during the time she was teaching there and that may have been the reason the sisters came home


More importantly, I confirmed that AI can be a helpful tool if held in check.  It can be filled with errors and will stick to its errors until the user challenges it.


Apparently, AI models are programmed to please their users. AI will give users what it thinks the users want. And it’s not just Gemini!


Maybe that’s not such a big deal when a user is doing genealogical research. But in other situations, it could be very dangerous.


What about for students or others who simply believe whatever they read?


Is AI a legitimate slippery slope for us all?
Probably, especially if we aren’t willing to challenge its claims.


Once again, higher-order thinking will win the day.



Discover more from Barbara Swander Miller

Subscribe to get the latest posts sent to your email.

Comments

Leave a Reply