Over winter break, I decided to create a semantic search engine on the Bible, primarily for personal use but also to understand how semantic search is currently implemented. I often find myself wishing I could find passages about specific circumstances (ie concrete examples of faith) aside from the ones I already know by heart. While LLMs (ie ChatGPT) have been trained extensively on biblical text, I personally find it better to stay close to the text - to discover and read the passage in isolation, without external influence (ie an LLM’s explanation).

Originally, I designed the tool to be centered around user intent. The general flow would be: user inputs a question -> an LLM preprocesses the query to see if it’s biblically relevant (ie not about something like “Star Wars”) and suggests refinements if necessary -> take the query and run hybrid (keyword + vector) search on indexed biblical text.

I ultimately disabled the LLM preprocessing step because I didn’t have free API credits on hand, and it is mostly a safeguarding step (which probably isn’t very relevant, since I don’t expect many people to chance upon this tool anyway). An overview of the pipeline is as follows. All code is open sourced on the GitHub, and a working demo is deployed here (via Qdrant Cloud/Fly.io)

Data Preprocessing

We begin with a .json file containing verse-level Bible text. I chose to use the New Living Translation, and formatted the json file using the hierarchy "BookAbbreviation" : { "ChapterNumber" : { "VerseNumber" : "VerseText" } }.

Then, using Chonkie’s SemanticChunker with all-mpnet-base-v2, we group adjacent verses into semantic chunks based on topic similarity (cosine threshold 0.75). For me this resulted in ~17.6k chunks (“passages”). Here’s a sample of what the .json looked like:

{
    "text": "...",
    "chunk_type": "semantic",
    "verses": [
      {
        "book": "Gen",
        "chapter": 1,
        "verse": 1,
        "text": "..."
      },
      ...
    ],
    "reference_display": "Genesis 1:1-5",
    "book": "Gen",
    "chapter": 1,
    "start_verse": 1,
    "end_verse": 5,
    "semantic_coherence": 0.6204913854598999,
    "embedding_model": "all-mpnet-base-v2"
  },

Embeddings & Storage

After that, we encode the chunks. I chose to use intfloat/e5-base-v2 (768 dimensions), which generated asymmetric encodings. Asymmetric encodings are designed such that a questions (ie “summary of the gospel”) and answers (ie ”[text of john 3:16]”) will ideally be mapped closer together in the vector space (versus symmetric encodings, which purely map similar-sounding paragraphs together).

We can store the vectors in any cloud vector database (I chose Qdrant Cloud) with the full payload (text, verses, metadata).

Hybrid Search Pipeline

The general model is this: Query -> [Semantic Search || BM25 Search] -> Merge Results -> Cross-Encoder Rerank -> Deduplicate Overlap -> Results.

The query is given from the user, and is passed into pipelines for both semantic/vector search (using our Qdrant vectors) and lexical/keyword search (using the bm25 ranking algorithm). These results are normalized and merged into a preliminary ordering. We run this ordering through a cross-encoder reranker (essentially a tiny language model specialized for reranking results) for better results. Finally, we do some deduplication (ie if two chunks were surfaced but both contained similar verses) and display.

Optional: LLM Query Refinement

If enabled (and hooked up to an API key), we can have Claude analyze the query before search. The idea is to have the LLM extract biblical concepts, refine the query, classify confidence (HIGH/MEDIUM/LOW), and suggests alternative interpretations for ambiguous queries. Biblical concepts can be appended to the semantic query for richer matching. This is implemented in the codebase, but disabled from the frontend.