AI Hallucinations: Why You Shouldn’t Believe Everything Your AI Assistant Says

But recently, I learned a hard lesson that completely changed how I look at my screen: AI lies. And the scariest part is, it lies with absolute confidence.

In the tech world, they call this an “AI hallucination.” It’s basically what happens when an AI model confidently presents false, fabricated, or nonsensical information as if it were a universal truth.

A few weeks ago, I was deep into a data engineering side project, trying to automate the extraction of MHT-CET engineering cutoff data for 2023 and 2024. I was wrangling with hundreds of semi-structured PDF files, converting them into clean CSVs using pdfplumber and pandas. I hit a massive roadblock: I needed to filter specific category labels while excluding any entries missing both rank and percentile data. Frustrated, I pasted my prompt into my AI assistant, asking for the exact logic. Within seconds, it spat out a beautiful, heavily commented Python script. It looked perfect. I copied it, pasted it into my Jupyter Notebook, hit run, and immediately got a massive error. I spent the next hour debugging, only to realize that the AI had completely invented a built-in library function that didn’t exist. It sounded logical, it was named perfectly according to Python naming conventions, but it was pure fiction.

Because we are studying concepts like Machine Learning, Logistic Regression, and Support Vector Machines, it helps to understand why this happens. AI models like ChatGPT or Gemini aren’t thinking entities. They are advanced pattern-matching systems trained on vast amounts of text. They don’t actually “know” the MHT-CET cutoffs, nor do they understand Python documentation the way a human does. Instead, they predict the most statistically likely next words based on patterns in their training data and the context you provide. Most of the time, they get it right. But when they don’t know the answer, they rarely say, “I don’t know.” They simply generate what a confident answer would look like and serve it to you on a silver platter.

This doesn’t just happen with code. The consequences can be embarrassing if you aren’t paying attention. Imagine you’re working on an organizational development report—say, analyzing a small-scale, proprietor-driven trading firm in Pune using Weisbord’s Six-Box Model. Ask an AI for historical revenue trends or internal operational data for that specific local firm, and it will likely invent numbers, employee structures, and fictitious details just to fill the page. If your professor cross-checks your sources, you can’t exactly cite “ChatGPT” as your primary data analyst.

As engineering students, we are wired to look for the most efficient solution to a problem. But relying blindly on AI is like letting a highly enthusiastic but completely inexperienced intern do your final-year project. They want to please you so badly that they will just make things up to avoid looking incompetent.

So, how do we actually use AI without getting played?

First, treat AI as a brainstorming partner, not a final authority. If it gives you code, read the official documentation to verify the methods. If it gives you facts, dates, or citations, do a quick Google search to make sure they actually exist. Second, give it boundaries. When I write prompts now, I explicitly tell the AI things like, “Use only standard pandas functions” or “Do not invent data.”

AI is an incredible tool, and I definitely wouldn’t survive my engineering degree without it. But at the end of the day, when you’re sitting in a lab at VIT for your practicals, it’s your brain that gets evaluated, not your prompt engineering skills. Use the AI to get unstuck, but make sure you’re the one actually driving.

Trust, but verify.