My hype for AI-related tools combined with gospel study is very high right now.
Beep boop beep.
Despite the vast amounts of hype-vomit related to “AI” in recent months, a lot of which is unwarranted, there are some really exciting possibilities out there in technology land. I’ve gone back to AI and AI-adjacent tools to solve a few problems related to study and research in the gospel setting. What I found is surprising and thrilling. In fact, it sucked me in HARD the past few weeks.
I’m so excited about what is possible I want to write a super long post about it! Here is my report.
Writing a talk.
I was asked to speak in church recently and, on a whim, decided to “talk” with ChatGPT for part of that process. I asked it for advice on a certain subject, tried outlining, etc. The results were very boilerplate and uninspiring. I would say this could be a good tool to get started, brainstorming ideas, creating a first outline, or distilling points from a talk. I imagine youth speakers could get a lot of mileage out of using it! š
Use cases for talk writing:
- Brainstorm: Paste the text of your assigned conference talk and tell ChatGPT to distill it down to its main principles or points. See if any of those resonate with stories from your life.
- Outline: explain you are giving a talk on your given subject and ask it to provide an outline for a X minute talk. You can then use that as a starting point, cutting and replacing sections you don’t like.
- Expand ideas: Try asking the program to expand a concept. For example: “How could I explain Christ’s Healing Power in a talk? Are there scriptures that I could cite which relate?” If you do this in a conversation where you have already made it clear this is for a talk in the LDS church, the program will know to include our unique doctrines and scriptures in its response.
What it can’t do: Write a joke.
Eliminate clickbait from Latter-day Saint News? Check.
What is it?
For many years I have been aware of dozens, perhaps hundreds, of Latter-day Saint focused blogs, news sites, online magazines, etc. The problem, of course, is that ain’t nobody got time for that.
I’ve tried a few different solutions to this, relying on google alerts to find me relevant articles, but it often gets overwhelmed when something hits the news, or by false positives like obituaries, church building construction announcements in regional news outlets, or even ads. I tweaked and tweaked and finally gave up on that, resorting to just a simple feed aggregator.
The downsides of a feed aggregator are that A) many sites nowadays don’t have feeds, and B) It can still be way way way too much since you still have clickbait titles, unclear titles, and, well, a decades-old technology that isn’t really approachable.
So I tried an experiment. Could ChatGPT read the article for me and let me know if it’s something I actually care about?
The AI News Flow:
- A website posts an article.
- New articles are sent to ChatGPT with the instructions to summarize the article in the style of a news roundup email. That summary is saved in a google sheet.
- Once per day, all the new rows of the google sheet are put together as a single “post” and sent to Chat GPT with instructions to:
- Read the post and generate a fun, social-media style title based on the content.
- Read the post and generate a welcome paragraph or two based on the content.
- The daily roundup post is emailed to me with the title as the subject, and the body as the welcome plus the list of summaries.
This works pretty ok. It could work great if I was willing to pay for ChatGPT4 to do the work but that would cost me about 20 dollars per month instead of a buck or two in processing time. By using ChatGPT 3.5 it costs me next to nothing, but it often forgets to return the link to the original post or to format properly. I’m not too fussed about that. You can see the results of this experiment in /r/mormonism where I have MoroniBot post the results daily.
I really like this because I can skim the email or the post in /r/mormonism once per day and get a pretty good idea of what’s available to read without dealing with clickbaity headlines. I’d love your feedback.
What would be neat:
If I were feeling bold I would work on a system where ChatGPT actually ranks each article based on relevance and other factors. That ranking could then be used to eliminate false-positives, repetitive content, etc. and allow me to go back to using Google Alerts which would make the feed more news-like instead of just blog focused. The downside, of course, is that this would cost a lot (maybe an extra 20 – 30 bucks per month?) since GPT4 is really the best for this kind of qualitative task.
Second, while I can gather videos from youtube, if they don’t include a lot of details in the video description then there’s not much for ChatGPT to go on. It would be cool to send any new videos that hit the feed to Assemblyai or some other transcription tool before GPT analysis. Then even videos with blank descriptions could be included. Same with podcasts. This, of course, would add significantly to the cost for a casual user like me, but it’s crazy to think that the possibility is out there!
Tools used:
- Pipedream – for automation, cron, email, etc. Cost: free
- Google sheets – to store article summaries. I could also use pipedream for this, but eh. Cost: free
- OpenAI API – used to connect to ChatGPT. Cost: couple dollars per month based on processing time
- Reddit API – used to post on reddit. Cost: free
- FreshRSS – used to gather blog posts. Cost: free
This little experiment shows just how disruptive LLM enhanced algorithms will become in the future. Imagine if Youtube, which already HAS the transcripts for its countless videos, is able to implement GPT-style analysis at scale. It suddenly stops being a case of “here’s videos that people LIKE you also watched” but instead “Here are more videos which are like the ones you already enjoy.” Ideally this will eliminate those annoying false-positives like when searching for Latter-day Saint related subjects and you also get bombarded with all the anti-mormon content. (The scary truth side of that is: ideology bubbles would only be made stronger. You’d have to actually want to see a different perspective before finding it.)
Speaking of improved searches:
Semantic Search of Gospel Topics
What is it?
Underpinning the recent AI developments is a technology called “vectors.” This is not new tech. In fact it’s decades old. For our purposes, think of it this way: Instead of storing a word in a database, you convert that word into a shape. (That’s the vector) Every letter, word, sentence, and paragraph could be turned into a unique shape depending on how your program works.
Why is this useful? Well, computers are bad at comparing words, or making sense of natural language. But they are VERY good at math problems like comparing shapes. See where we’re going? So if you use vectors, you might be able to get a computer to identify when words are “pretty close” to other words.
Stick a bunch of words, phrases, and terms that mean “hate” in a vector database and a computer might be able to compare Yelp reviews to that database and figure out which reviewers hated their meals – even if they never used that word – simply because the shape of the review was mathematically similar or “close” to the shape of hate-related terms.
What makes the latest AI so powerful is that they are very very very good at generating vectors based on meaning or “semantics.” So, for example, the words “hot dog” and “mustard” will be more similar than “hot dog” and “dog.”
But what about in a gospel setting?
If this works right, it will mean much deeper studying and learning for us. Imagine l jump into the gospel library and search for “How can I access physical healing?” and get some great results in conference talks and articles which talk about that exact issue. But if I search “semantically” I might also get things like the gospel topic essay on death, and the one on adversity, since those are closely related semantically to the need for physical healing.
- “What is the purpose of suffering” will return not just articles about suffering but also topics like:
- Adversity
- Council in Heaven
- Atonement
- Conscience
- Heavenly Parents
- Original Sin
- Mortality
- Complex questions in conversation style can still get relevant results
- Well trained AI can translate your human-style question into a more targeted query if needed
- Well trained AI can examine all the returned results and summarize them for you
- Scriptures can be connected not just by footnote, but by meaning
My tests
I tested this out in an app using “streamlit” and a database of general conference talks, scriptures, and other documents. I fed large chunks of the talks to OpenAI which created a vector for each. Then the app uses a user query to search those vectors for similarity. You can try it out here: https://topical-guide.streamlit.app/ It’s not well done, and crashes often, (just try again if it happens to you) but I have had a great experience using it.
In the last few weeks I have used this tool to research reddit questions on gospel topics. Inevitably I find resources and insights I hadn’t thought of before.
What’s crazy is that I have ZERO programming and engineering experience and I was able to make something that I feel gets me some great results. A whole search engine made from scratch in a matter of a couple of days by a complete amateur!
I expect that real programmers and engineers will be able to create tools that will let members search through our vast gospel library MUCH more powerfully. Especially in the scriptures, where finding answers can be difficult, and meaning isn’t always clear. I look forward to see what people create.
What would be neat:
There are incredible libraries out there that we ought to be searching through. The reality is that we just don’t have the time to read the text, categorize it, and find places to include it in the existing gospel library. A semantic search engine fixes all of that!
I added BYU Studies and BYU Devotional texts to my app, but there’s so much more. It would not take much to create a database that searches through much more than we currently have in the gospel library, but still be focused on faith. I hope to add more documents, to my test and if you have any ideas of resources that should be in my database let me know. I’d also love to see an app that can return results based on date of the talk, maybe even sorting by weight for “authoritativeness” in some way. There’s some very interesting opportunities here.
Tools used:
- ChatGPT – to tell me how to write python scripts that will do what I want, and to generate vectors. – 20 / month for gpt4
- Python – you know, for doing things. – free
- Streamlit – creates an “app” from my hokey code -free
- Supabase – database for the text and vectors – free
Trained AI
What is it
A process called “Fine Tuning” is how you train an AI into acting the way you want. ChatGPT is a large language model that has been fine tuned by having tens of thousands of conversations with real humans who then correct it and give it examples of the “right” way to talk. While this cost millions of dollars, some companies are creating trained AI for as little as 600 dollars by simply letting the big AI do all the training instead of actual humans.
Ideally you will upload a large database of questions and responses to your model, thousands and thousands of examples of how you hope it will behave, the types of answers it should give, the temperament it should have, etc. But creating that database is a big challenge. For example, if I want to create a chat bot for a law firm, I’d have to spend months and months finding and inserting common legal questions and the right kind of non-legal-advice responses that would be appropriate. So these fine-tuned models are still only options for bleeding-edge tech companies.
Right?
The Gospel Angle
For decades, we’ve been chatting online, human-to-human, about gospel questions, needs and desires. I can think of two massive databases:
1. The Church Missionary Department, which ran live chats with missionaries for many years
2. FAIR Latter-day Saints, which has been offering email-based human responses to questions for decades.
I’d be very very interested to see how a Fine Tuned model could be adapted to answer simple questions for websites like FAIRLatterDaySaints.org and ChurchofJesusChrist.org. FAIR in particular could really power up their search function by combining semantic search and a Fine-Tuned model to summarize search results. And the crazy thing is it wouldn’t cost a lot. It might be a couple thousand to do the fine-tuning, but the database and the individual queries would be only a few dollars per month.
In essence, we could end up with a search function where the article you find isn’t the static subject matter page with additional resources, but a custom response written specifically for your need and presented in the style of the thousands of faithful members who have volunteered their time to help people just like you. It could express sympathy for your unique struggles, and offer encouragement tailored to your perspective.
Perhaps most importantly, it could be trained to offer a gentle prompt in the right direction for many, encouraging users to reach out for help.
Thanks for reading. Let me know your thoughts.