Vipul Singhal

Having worked with a wide variety of LLMs, this is a great roadmap to follow for anyone who wants to truly learn NLP

9

One of the biggest trade-offs in production RAG systems is retrieval quality vs latency. You can improve results by adding: • reranking • larger candidate sets • richer context But every additional step adds latency. A 2–3 second response may be fine for research assistants. But it can break the experience in customer-facing applications. The best AI architectures are not the most sophisticated ones. They’re the ones that fit within the latency budget of the product and the domain. #AIEngineering #GenAI #RAG #MachineLearning #ArtificialIntelligence

12

3 Comments

"Here is what 25 years actually teaches you: most AI fails not because the model is wrong. It fails because the problem definition was wrong."

1

I recently had the opportunity to sit with the team at Mentoria for a detailed conversation on the evolving landscape of AI and Machine Learning. The discussion examined how unexpected opportunities emerge in this rapidly developing domain and why certain underlying competencies often become decisive factors for recruiters evaluating talent. We explored foundational questions that many beginners overlook—what remains unsaid when entering the field, why successful careers extend far beyond coding alone, and which skills retain their relevance as the technological ecosystem shifts. The interview also includes a rapid-fire segment and a focused look at three specific CV issues that can lead to immediate rejection, underscoring the importance of precision and clarity when presenting one’s work. The full interview is available here: https://lnkd.in/gBV4J-TR #AI #MachineLearning #DataScience #CareerTips #Innovation #TechCareer #FutureSkills #Interview

7

"Gaurav, how can I fine tune an LLM on user activity data?" My first mistake was to answer his question. "Pick a base model like Llama, and fine tune it using question and answer data." He didn't seem impressed, so I moved to the fundamentals. "What are you trying to do?" "I am trying to recommend users products based on their activity. Past browsing, searches and current trends." "You mean, you want to recommend stuff based on what they have bought previously?" "Yes." "So...two tower approach?" "What's that?" "I mean collaborative filtering and XGboost." "I don't understand." "Search for 'How to build a recommendation engine' on ChatGPT". "Okay, thanks..." I don't think he was convinced. For most technical problems, you don't need fine-tuning. You need clarity.

225

28 Comments

While working on Airflow DAGs, I had a sudden realization; this feels very similar to what I used to do in Informatica with workflows, sessions, and the session monitor a decade back. The key differences: Informatica: Visual GUI, licensed enterprise tool, user-friendly but comes with a cost. Airflow: Open-source, code-driven (Python), highly flexible, and cloud-native. The shift from drag-and-drop to code-as-infrastructure takes a little getting used to, but it opens up a lot more control and customization, especially in modern data stack environments. Would I say Airflow is cheaper? Absolutely, from a licensing and flexibility standpoint, it is a major win, especially for teams building cloud-based data platforms. #DataEngineering #ApacheAirflow #Informatica #ETL #Orchestration #ModernDataStack #CareerJourney #Python #CloudComputing #AI #DataLife

20

Maths vs AI #1 : Question : If there are 3 non-collinear points in space and you wish to draw lines which are equidistant from all 3 points. How many such lines are possible? AI : Asks for clarification whether the space is 2D or 3D. For 2D it is 3 lines, and for 3D it is 1 line through circum-centre. It a WIN for AI 👏

1

Building an LLM - Concepts and Interview question series Topic 14 -Memory bandwidth is all you need - well almost!   As we start to build systems and think of 'scale', we are always worried about compute. When you actually implement and build systems the realization hits hard - Modern ML systems often don't have compute as their key bottleneck. it's memory bandwidth.   Energy reality: A single 32‑bit DRAM access is orders of magnitude more energy than a 32‑bit multiply; bulk movement (e.g., 1 GB) dwarfs compute energy ( Intro to ML Systems - Vijay Janapa Reddy) Performance reality: In memory‑bound regimes, kernels achieve a fraction of peak FLOPs; what matters is arithmetic intensity (FLOPs/byte) and achieved GB/s.   Putting this all together - what does it mean for builders. 1. Optimize data movement first Quantization, sparsity, KV‑cache reuse, activation checkpointing (trade compute for bytes) 2. Choose systems for bandwidth, not peak FLOPS 3. Measure the right metrics - Achieved GB/s vs theoretical, arithmetic intensity (FLOPs/byte) 4. Plan & procure accordingly - Optimize bandwidth/$ and memory/$ before TFLOPS - Profile end‑to‑end pipelines (I/O, pre/post‑processing), not just kernels Performance and energy scale with data movement, not FLOPs. Architect for bandwidth, locality, and interconnect first; measure GB/s, bytes/token, stall time; and procure on bandwidth/$ and memory/$ rather than TFLOPs. The memory wall is the system. Homework - Check Amdahl’s Law and its implications (or lack of it) in today's scenarios. #MLOps #GenAI #LLMs #FLOPS #MLSystems

20

1 Comment

Quantization in Large Language Models (LLMs) As LLMs scale, intelligence is no longer the bottleneck — memory, latency, and cost are. Quantization turns out to be one of the most powerful (and misunderstood) techniques enabling real-world deployment of large models. In this blog, I break down: • Why LLMs are surprisingly quantization-friendly • INT8 vs INT4 vs NF4 — what actually works in practice • How modern architectures absorb low-bit noise • Where quantization fails and how to design around it Quantization is no longer an optimization step — it’s foundational infrastructure for production LLM systems. Link: https://lnkd.in/gqGje75G #LLMs #AIEngineering #Quantization #MachineLearning #GenAI #MLOps

9

2 Comments

Few of the agentic AI implementations I am doing currently has some of the following learnings : 1. Agentic AI implementation is driven by a use case 2. while executing such implementation complexity resides in understanding the product and its UI/UX in terms of what fields and attributes we have to ask the agents and LLMs to populated automatically which ones will be the input to the agents 3. For the fields which are input to the agents we need the RAG to be created which is vector DB in this case pinecone from AWS and for each attribute you need descriptions to be put into the vector DB so that genAI can pick right understanding of the prompted texts 4. One agent not sufficient to do UI/UX integration and automation but minimum 5 agents master agent node agent build agent validator agent aggregator agent in some case we may have to device a listener agent 5. Use of crew.ai is good for agents creation is preferred over langchain due to better features 6. Claude sonnet is used as LLM 7. I have driven this project with lots of design documents which include scope screen designs, Agents mapping design and multi test case documents 8. On AWS we need to think about deployment options and regions. For claude sonnet we need to consider wider regions so that models gets exposed to larger public data and gives better responses This much a good write up for this saturday. See u next week.

18

3 Comments

A Thought Exercise : Context Engineering with LLMs is analogous to Feature Engineering in Classical ML What are the similatieties and differences ? Share in the thread #AImusings courtesy: Jim Hong :)

28

9 Comments

I used to think I understood how ML models learn… until I learned this. Back in my early days of exploring data science, there was this legendary subject that was mandatory for everyone to take — Pattern Recognition and Machine Learning. Sridhar Pappu kept telling us: “Learn Probablity probability first. You’ll need it.” At the time, I had no clue why probability mattered so much for ML. Much later, I realised something no one really spells out: 👉 Every single AI model — from simple classifiers to LLMs — is fundamentally just a probability problem. And once that clicked, ML suddenly became way more intuitive. A simple way to understand it Basically the core question behind ML,Forget neural networks for a moment. Imagine a researcher working in a biology lab. She’s studying a plant species and wants to understand,“What fraction of these seeds actually germinate?” She doesn’t know the true germination probability.But she can plant seeds and observe how many sprout. That’s essentially the heart of machine learning. Given some observations, what probability distribution best explains them? She collects data — maybe 70 out of 100 seeds sprout. Now she asks: “Which value of p (the germination probability) makes my observations most likely?” This leads directly to the idea of a likelihood function, It calculates how likely your data is for any assumed value of p. Maximising that likelihood — the famous Maximum Likelihood Estimation (MLE) — gives the best estimate. In this simple case, the answer is just, p = 0.70 Beautifully simple!! So how does this relate to LLMs? LLMs are doing the exact same thing, just at a much larger scale and with conditioning on input text. When you ask: “Who is the CEO of Apple?” The model pulls up the probability distribution of all the possible next tokens it has seen during training. Training the model is nothing more than: 📌 Estimating distributions that maximise the likelihood of the training data. That fancy “categorical cross-entropy” loss we hear about? It’s just MLE in disguise 😋 Strip away the jargon, and here’s what LLMs really do: They predict the next token by choosing what is statistically most likely, given everything they’ve seen before. Not magic — just probability at scale. Takeaway 🎁 : If you understand probability, you understand the foundation of AI. The entire field including LLMs rests on the simple idea of learning the probability distribution that best matches the data. Everything else is just engineering.

92

5 Comments

After going through endless posts on linkedin about RAG and reading about in different tech blogs. I thought about actually implementing it locally for a not so traditional use case. Every year about 11 to 13 lakh people appear for Preliminary exam of UPSC civil services in india, which has a general studies paper of about 100 questions, out of which as much as 50% questions are based on current affairs or current events based on static portion of subjects like polity, economics, geography, sci/tech, environment. There are so many coachings producing tonnes of materials trying to cover all the current events that can be asked in the exam. I tried to test if UPSC 2025 prelims paper was solveable using one of those through a RAG pipeline. Some of the text after this is LLM generated because I am too lazy to type and format. The Problem Statement 🛑 Every year, UPSC aspirants read ~1,800 pages of "Current Affairs" monthly magazines (Vision, Forum, Vajiram). The burning question has always been: How many actual Prelims questions does this material truly cover? Instead of guessing, I built a data pipeline to find out. The Solution: Local RAG Pipeline 🛠️ I engineered a Retrieval-Augmented Generation (RAG) system to perform a semantic audit of 12 months of PDFs against the 47 current affairs questions from the 2025 Prelims. Ingestion: Parsed & chunked 1,500+ pages using PyMuPDF & LangChain, preserving metadata (page numbers) for citations. Vector Search: Used ChromaDB with BAAI/bge-m3 embeddings for semantic retrieval (catching concepts, not just keywords). Precision Reranking: Implemented a Cross-Encoder (ms-marco-MiniLM) to score relevance (0-10) and filter out noise. The Verdict (N=47 Questions) 📉 I tested three major institutes. Here is how their material held up against the actual exam: 🏛️ ForumIAS • Broad Match: 53.2% (25/47) • High Confidence: 31.9% (15/47) 🏛️ Vajiram • Broad Match: 48.9% (23/47) • High Confidence: 29.8% (14/47) 🏛️ VisionIAS • Broad Match: 40.4% (19/47) • High Confidence: 25.5% (12/47) The analysis document generated has each question of prelims given a match score , the relevant document name mentioned, the page number in document, and the text chunk it has significant match with it. Needless to say this wont be true test whether someone can actually solve the question if the person had read than chunk. (Disclaimer: This is a personal "toy project" built for educational purposes to test AI retrieval pipelines. The data presented is the result of an automated algorithm and may contain errors or inaccuracies. It is not a definitive scientific study or a commercial review of these institutes.) Attaching the github link if anyone wants to test it out on a different corpus (or make it better by attaching a smart LLM judge that actually solves the question) : https://lnkd.in/gKnsg83U #RAG #NLP #upsc

35

3 Comments

Sarvam AI is outperforming large global LLMs on India-specific use cases. Across real-world India-focused tasks, Sarvam AI (fine-tuned for India) is delivering stronger performance than large general-purpose models — proving that targeted fine-tuning beats sheer parameter size. 🏆 Where Sarvam AI Outperformed: ✅ Vernacular Intent Detection (Hindi + regional mix) More accurate understanding of Hinglish, code-mixed queries, and regional phrasing. ✅ Regulatory & Compliance Q&A (India domain) Higher precision in interpreting India-specific policies and terminology. ✅ Localized Customer Support Responses More culturally aligned tone with fewer generic outputs. ✅ Cultural Context & Idiomatic Understanding Better handling of local expressions and socio-cultural nuances. Why This Is Happening Most global LLMs like those from OpenAI and Google (Gemini) are trained predominantly on large-scale English and Western-centric datasets. India’s linguistic landscape is fundamentally different: • 22+ official languages • Heavy code-mixing (Hinglish, Tanglish, etc.) • Deep cultural and regulatory context • Region-specific communication styles This gap is one reason why adoption of generic global LLMs in India can be slower for core business workflows. India needs models built for India’s data, languages, and users. Task-specific fine-tuning on curated India-centric datasets enables: • Higher accuracy • Lower latency • Better cost efficiency • Stronger real-world alignment 🧠 Key Insight: For country-specific and domain-heavy applications, alignment > size. Kudos to Sarvam for leading India’s LLM movement in the right direction. #AI #IndiaAI #LLM #MachineLearning #FineTuning #GenAI #DigitalIndia

20

User reviews are often treated as sentiment. But occasionally, they function more like qualitative validation. What’s interesting in the recent 5⭐ feedback isn’t the positivity—it’s the consistency of themes:  • structural clarity of data  • reduction of noise  • applicability across domains (research, risk, product) In other words, not just satisfaction, but fit. A system aligns with how people think about a problem. We compiled a set of these observations here: 👉 https://lnkd.in/dvHaG7XY Taken together, they form something close to a distributed product definition—written by users themselves. 👉 https://lnkd.in/esGQi9Yb

4