Semantic Search

Semantic Search is a type of search based on understanding the contextual meaning and intents rather than just keyword matching. Instead of merely looking for literal matches between search queries and indexed content, it aims to deliver more relevant search results by considering various factors, including the relationships between words, the searcher’s location, any previous searches, and the context of the search.

One of the good examples might be After this long day, I want to crash. Well if a machine interprets this it might be considering this like after this long day, I want to make an accident in a normal sense however here the author wants to mean that I want to sleep anywhere which is something semantic search will understand and give us answers based on the contextual meaning. The traditional search engine is focused on keywords and gives you search results based on the keyword matching. For instance, if you are looking to buy a car and you filter based on the keywords, a semantic search can find cars based on your intent. Machine Learning and Natural Language Processing are the backbones of the Semantic Search with the machine learning algorithms understanding the patterns and NLP understanding the human language.

Applications:

• E-commerce Sites
• Enterprises

Embedding: Embeddings are multi-dimensional vectors that help us represent words as a point in space and also establish relationships between similar blocks of text or tokens. Converting text into a vector is super handy because it's easier to do math with them rather than words, especially when wanting to compute the "distance" or similarities between two ideas. For example, if we were to represent the vectors for words like "Javascript" and "code", they would be placed relatively close to one another in our vector space as they are closely related. On the other hand, the vector for the word "coffee" would be further apart from those two, as it does not have such a close relationship with them. The cool thing is that you can get those vectors for individual words but also for entire sentences! Thus, we can get a mathematical representation of a given chunk of text from any piece of content on the blog, compare it with the vector of the user's query, and determine how close of a match those are. To find the similarity between the user query and text in the pdf we can use the cosine similarity. The idea is to calculate the cosine angle between two vectors and the smaller the angle the closer the sentences are and are more similar.

Fig- Example of Cosine Similarity

How embeddings are created from words?

Two popular techniques for creating word embeddings are Word2Vec and GloVe: Word2Vec: Word2Vec uses neural networks to create vector representations of words. It leverages the idea that words with similar contexts tend to have similar meanings. Word2Vec has two main approaches: Continuous Bag of Words (CBOW), where the model predicts a word based on its context, and Skip-gram, where the model predicts the context based on a word. GloVe (Global Vectors for Word Representation): GloVe is another method for creating word embeddings. It uses a matrix factorization approach to derive word vectors from global word co-occurrence statistics. GloVe aims to capture both local and global semantic relationships in a high-dimensional space. Word embeddings have become a cornerstone of NLP, enabling models to understand context, capture semantic relationships, and perform various tasks with greater accuracy.

Our Approach:

Tech Stack

• Backend – Fast API

• Frontend - Next Js

• Vector Database - FAISS

• AI/ML - Sentence Transformers, Ollama, Mistral AI.

Description: This diagram illustrates the end-to-end process of a semantic PDF search and Q&A system, built using Python Fast API, FAISS, and Ollama. It begins with a PDF file, which is passed through pdfplumber to extract all its text content. This raw text is then processed using NLTK, where it is split into meaningful sentence chunks. These chunks are grouped in a way that retains context by adding overlapping segments between them. Next, each chunk is converted into a numerical representation using a SentenceTransformer model. These embeddings — which are normalized to enable cosine similarity — are stored in a FAISS index. The same chunks are also saved in a pickle file so they can be easily loaded and used in another Python module. This entire part of the process — from PDF to indexed embeddings — is considered the indexing phase. When a user enters a question, the system enters the querying phase. The query is converted into an embedding using the same SentenceTransformer model. This embedding is then compared with all stored chunk embeddings in the FAISS index using cosine similarity (implemented through inner product on normalized vectors). FAISS returns the top-k most similar chunks, which are the most relevant segments of the PDF content based on the query. These relevant chunks are then passed to Ollama, a local language model, which takes both the chunks and the user's question to generate a natural-language response. The model rephrases the answer in a user-friendly, conversational tone. Optionally, the system can also highlight the matched chunks in the original PDF and show them alongside the chat, offering context to the user. All of this is handled in the backend using FAST API and sent to our next js frontend.

References:

Research Gate

Google Cloud