Generative AI, LLM & Prompt Engineering Interview Questions (2026 Master Guide)

Q1. What is Generative AI and how does it differ from Traditional Discriminative AI?

Foundation • 0-2 Yrs

Discriminative AI learns decision boundaries between classes to classify or predict targets P(Y|X) (e.g., classifying an image as cat vs dog, churn prediction). Generative AI learns the underlying probability distribution of the data P(X) to generate entirely new, realistic content (text, images, audio, synthetic data) that resembles the training distribution.

Q2. What is a Large Language Model (LLM) and how does it generate text?

Foundation • 0-2 Yrs

An LLM is a deep neural network (usually billions of parameters) trained on massive text corpora to predict the next token in a sequence given preceding context. It generates text auto-regressively: sampling tokens sequentially based on probability distributions computed via softmax over its vocabulary.

Q3. Explain the core components of the Transformer Architecture (Vaswani et al., 2017).

Foundation • 0-2 Yrs

The Transformer replaces recurrence (RNNs) with Self-Attention. Components: Input Token & Positional Embeddings, Multi-Head Self-Attention layers (capturing contextual relationships in parallel), Feed-Forward Networks (FFN), Layer Normalization, and Residual Skip Connections.

Q4. What are Tokens in LLMs and how does Byte-Pair Encoding (BPE) work?

Foundation • 0-2 Yrs

Tokens are the fundamental semantic units processed by LLMs (words, sub-words, or characters; ~1 token = 0.75 words). Byte-Pair Encoding is a subword tokenization algorithm that iteratively replaces the most frequent pairs of consecutive bytes/characters with a new token, efficiently handling out-of-vocabulary words.

Q5. What is Prompt Engineering and what are Zero-Shot, One-Shot, and Few-Shot Prompting?

Foundation • 0-2 Yrs

Prompt Engineering is the practice of crafting, structuring, and optimizing inputs to guide an LLM toward accurate, deterministic outputs. Zero-Shot: Prompt provides no examples. One-Shot: Provides exactly one demonstration example. Few-Shot: Provides 2 to 5 demonstration examples establishing input-output formatting before the query.

Q6. Explain Chain-of-Thought (CoT) Prompting and why it improves reasoning.

Foundation • 0-2 Yrs

CoT prompting instructs the LLM to 'think step by step' and articulate intermediate logical steps before outputting the final answer. This breaks down complex multi-hop mathematical, logical, and algorithmic problems into manageable calculations, drastically reducing reasoning errors.

Q7. What is Temperature and Top-P (Nucleus Sampling) in LLM generation?

Foundation • 0-2 Yrs

Temperature (0 to 1): Controls randomness; lower values (<0.2) make outputs deterministic and focused; higher values (>0.7) increase creative diversity. Top-P (Nucleus Sampling): Samples only from the smallest set of tokens whose cumulative probability exceeds P (e.g., Top-P = 0.9), filtering out low-probability nonsensical tokens.

Q8. What are Vector Embeddings and how do Vector Databases work?

Foundation • 0-2 Yrs

Vector Embeddings are dense numerical array representations (e.g., 1536-dimensional vectors) capturing the semantic meaning of text chunks. Vector Databases (Pinecone, Chroma, Milvus) index these vectors using algorithms like HNSW to execute sub-second approximate nearest neighbor (ANN) cosine similarity searches.

Q9. What is Retrieval-Augmented Generation (RAG) and why is it preferred over fine-tuning for domain knowledge?

Foundation • 0-2 Yrs

RAG supplements prompts with external, verified document chunks retrieved from a knowledge base at runtime. Benefits over fine-tuning: Zero model retraining costs, immediate data updates (real-time data), elimination of hallucinations by grounding in verifiable citations, and strict access control.

Q10. What is Hallucination in LLMs and how can it be mitigated?

Foundation • 0-2 Yrs

Hallucination occurs when an LLM generates factually false, ungrounded, or nonsensical information with high confidence. Mitigate via: 1. Grounded RAG with strict citation instructions; 2. Lowering temperature to 0.0; 3. Few-shot validation; 4. Self-consistency checks; 5. Guardrails (NeMo, Llama Guard).

Q11. What is the difference between Fine-Tuning and In-Context Learning (Prompting)?

Foundation • 0-2 Yrs

In-Context Learning provides guidance and examples directly inside the prompt context window without altering model neural weights. Fine-Tuning updates the physical weight parameters of the model via backpropagation on a domain-specific dataset, adapting tone, specialized terminology, or task format permanently.

Q12. What is Parameter-Efficient Fine-Tuning (PEFT) and LoRA?

Foundation • 0-2 Yrs

Full fine-tuning updates all billions of parameters (computationally prohibitive). PEFT freezes the base model weights and trains a tiny fraction (<1%) of additional parameters. LoRA (Low-Rank Adaptation) decomposes weight updates into two low-rank matrices, reducing VRAM requirements by 90%.

Q13. What is RLHF (Reinforcement Learning from Human Feedback)?

Foundation • 0-2 Yrs

RLHF aligns LLMs with human values (helpful, harmless, honest). Process: 1. Pre-trained base model -> 2. Supervised Fine-Tuning (SFT) on prompt-response pairs -> 3. Train a Reward Model on human preference rankings -> 4. Optimize the model policy using Proximal Policy Optimization (PPO) against the reward model.

Q14. What are System Prompts, User Prompts, and Assistant Messages?

Foundation • 0-2 Yrs

System Prompt: High-level foundational instruction defining the AI's persona, boundaries, and formatting rules. User Prompt: The input query provided by the human end user. Assistant Message: The response generated by the LLM in multi-turn conversations.

Q15. What is Context Window length and what happens when it is exceeded?

Foundation • 0-2 Yrs

The Context Window is the maximum token capacity an LLM can process in a single interaction (prompt + completion, e.g., 8k, 32k, 128k, 1M tokens). Exceeding it results in truncated input, loss of early conversation context, or API context length errors.

Q16. Explain the Self-Attention formula and why multi-head attention is superior to single-head.

Experienced • 3-5+ Yrs

Attention(Q, K, V) = softmax((Q * K^T) / sqrt(d_k)) * V. Single-head attention averages all contextual features into a single representation. Multi-Head Attention projects Q, K, and V into multiple lower-dimensional subspaces, allowing the model to attend simultaneously to different aspects: syntactic, semantic, temporal, and entity relationships.

Q17. What is the 'Lost in the Middle' phenomenon in long-context LLMs?

Experienced • 3-5+ Yrs

Research shows LLM retrieval accuracy degrades significantly when relevant context is located in the middle of massive context windows, compared to when information is at the very beginning or end of the prompt. Solutions: Re-ranking retrieved chunks and putting critical instructions at the end.

Q18. How do you design an Advanced RAG pipeline with Hybrid Search and Re-ranking?

Experienced • 3-5+ Yrs

1. Ingest & Chunking (semantic or recursive chunking); 2. Hybrid Search (combines dense vector cosine similarity with sparse keyword BM25 retrieval to capture exact part numbers); 3. Cross-Encoder Re-ranking (Cohere Rerank re-evaluates top 50 chunks to pick top 5 most relevant); 4. Prompt assembly and synthesis.

Q19. Explain QLoRA (Quantized Low-Rank Adaptation) and 4-bit NormalFloat (NF4).

Experienced • 3-5+ Yrs

QLoRA quantizes pre-trained base model weights to 4-bit precision using the information-theoretically optimal NormalFloat (NF4) data type, keeping them frozen in VRAM, while backpropagating gradients through 16-bit LoRA adapter weights, enabling fine-tuning a 70B parameter model on a single consumer GPU.

Q20. What are AI Agents and how does the ReAct (Reason + Act) framework operate?

Experienced • 3-5+ Yrs

An AI Agent is an autonomous system leveraging an LLM as its central reasoning engine to accomplish multi-step goals. The ReAct pattern alternates between: Thought (reasoning about current state), Action (invoking an external tool: calculator, web search, SQL query), and Observation (analyzing tool output) until completion.

Q21. How do you evaluate RAG pipelines using the Ragas framework?

Experienced • 3-5+ Yrs

Ragas evaluates RAG without human annotations across 4 metrics: 1. Faithfulness (is the answer grounded in context?); 2. Answer Relevance (does the answer address the question?); 3. Context Precision (are relevant chunks ranked higher?); 4. Context Recall (did retrieval fetch all necessary facts?).

Q22. What is Direct Preference Optimization (DPO) and how does it replace PPO in alignment?

Experienced • 3-5+ Yrs

Traditional RLHF requires training a complex, unstable Reward Model followed by PPO reinforcement learning. DPO mathematically reformulates the objective, showing that the optimal policy can be derived analytically directly from human preference pairs (chosen vs rejected) using simple cross-entropy loss.

Q23. Explain KV Caching in Transformer inference and why it accelerates generation.

Experienced • 3-5+ Yrs

During auto-regressive generation, computing attention for each new token would normally recalculate Key and Value matrices for all preceding tokens (O(N²) redundant compute). KV Caching stores previously computed Keys and Values in GPU VRAM, reducing next-token generation to O(1) compute per step.

Q24. How do you prevent Prompt Injection attacks and Jailbreaking?

Experienced • 3-5+ Yrs

1. Input validation & sanitization; 2. Dual-LLM architecture (one unprivileged model sanitizes input; second executes task); 3. System prompt isolation using structural delimiters ( tags); 4. Llama Guard / NeMo Guardrails classifying input intent before generation.

Q25. What is Semantic Caching in production LLM applications?

Experienced • 3-5+ Yrs

Semantic Caching (e.g., GPTCache) embeds user queries and checks a vector database for previously answered questions with cosine similarity > 0.95. If matched, it returns the cached response instantly in 10ms with zero LLM API cost, saving up to 60% on cloud billing.

Q26. Explain FlashAttention and how it optimizes GPU memory bandwidth.

Experienced • 3-5+ Yrs

Standard attention materializes large intermediate N x N attention matrices in slow GPU High Bandwidth Memory (HBM). FlashAttention reorganizes attention computation into blocks using GPU on-chip SRAM (tiling) and computing softmax on-the-fly, reducing memory reads/writes by 5x and delivering 2-4x wall-clock speedup.

Q27. What is Mixture of Experts (MoE) architecture (e.g., Mixtral 8x7B)?

Experienced • 3-5+ Yrs

MoE replaces dense feed-forward layers with multiple independent 'expert' sub-networks. A learned Router/Gating network dynamically activates only a subset of experts (e.g., 2 out of 8) per token, providing the parameter capacity of a massive model with the inference speed and compute cost of a smaller model.

Q28. How does Speculative Decoding accelerate LLM inference latency?

Experienced • 3-5+ Yrs

Speculative decoding pairs a small, fast 'draft model' with a large 'target model'. The draft model speculatively generates K tokens quickly; the target model evaluates all K tokens in a single parallel forward pass, accepting valid tokens and regenerating on mismatches, yielding 2-3x faster inference without quality degradation.

Q29. What are Structured Outputs and JSON Mode in modern LLM APIs?

Experienced • 3-5+ Yrs

Structured Outputs enforce that the LLM completion strictly adheres to a developer-provided JSON Schema using grammar-constrained decoding (masking invalid tokens during generation), eliminating parsing exceptions when driving backend APIs or database writes.

Q30. Explain the difference between LangChain and LlamaIndex.

Experienced • 3-5+ Yrs

LangChain is a general-purpose orchestration framework focused on multi-step agent workflows, tool calling, chains, and conversational memory. LlamaIndex is specialized for data ingestion, indexing, and retrieval optimization for high-performance search and RAG pipelines.

Q31. Write a Python script using OpenAI API with Structured Outputs (Pydantic).

Live Coding Challenge

Use client.beta.chat.completions.parse with response_format=PydanticModel.

from pydantic import BaseModel
from openai import OpenAI

class LeadExtraction(BaseModel):
    name: str
    phone: str
    interest: str

client = OpenAI()
response = client.beta.chat.completions.parse(
    model="gpt-4o-mini",
    messages=[
        {"role": "system", "content": "Extract customer lead information."},
        {"role": "user", "content": "Hi, I am Rohan, reach me at 9876543210. Looking for Python training."}
    ],
    response_format=LeadExtraction
)
lead = response.choices[0].message.parsed
print(lead.name, lead.phone, lead.interest)

Q32. Write a RAG query function using LangChain and a Chroma vector store.

Live Coding Challenge

Initialize vector store retriever and combine with an LLM chain.

from langchain_community.vectorstores import Chroma
from langchain_openai import OpenAIEmbeddings, ChatOpenAI
from langchain.chains import create_retrieval_chain
from langchain.chains.combine_documents import create_stuff_documents_chain
from langchain_core.prompts import ChatPromptTemplate

def run_rag(query, db_path):
    vectorstore = Chroma(persist_directory=db_path, embedding_function=OpenAIEmbeddings())
    retriever = vectorstore.as_retriever(search_kwargs={"k": 3})
    llm = ChatOpenAI(model="gpt-4o-mini", temperature=0)
    prompt = ChatPromptTemplate.from_template(
        "Answer the question based only on context:\nContext: {context}\nQuestion: {input}"
    )
    combine_docs_chain = create_stuff_documents_chain(llm, prompt)
    rag_chain = create_retrieval_chain(retriever, combine_docs_chain)
    return rag_chain.invoke({"input": query})["answer"]

Q33. Write a Python function to compute cosine similarity between two text embeddings in NumPy.

Live Coding Challenge

Dot product divided by norms.

import numpy as np

def similarity(emb1, emb2):
    vec1, vec2 = np.array(emb1), np.array(emb2)
    return np.dot(vec1, vec2) / (np.linalg.norm(vec1) * np.linalg.norm(vec2))

Q34. Write a Python function to calculate token count using tiktoken.

Live Coding Challenge

Use tiktoken.encoding_for_model.

import tiktoken

def count_tokens(text: str, model="gpt-4o") -> int:
    enc = tiktoken.encoding_for_model(model)
    return len(enc.encode(text))

Q35. Write a Few-Shot Prompt template in Python for sentiment classification.

Live Coding Challenge

Template with 3 demonstration examples.

prompt_template = """
Classify the customer sentiment into POSITIVE, NEGATIVE, or NEUTRAL.

Review: The dashboard loaded instantly and saved 3 hours!
Sentiment: POSITIVE

Review: The support team never responded to my ticket.
Sentiment: NEGATIVE

Review: Package arrived on Tuesday.
Sentiment: NEUTRAL

Review: {user_review}
Sentiment:"""

Q36. Write a LoRA configuration in Python using Hugging Face PEFT.

Live Coding Challenge

Define LoraConfig targeting attention projections.

from peft import LoraConfig, TaskType

lora_config = LoraConfig(
    r=8,
    lora_alpha=16,
    target_modules=["q_proj", "v_proj"],
    lora_dropout=0.05,
    bias="none",
    task_type=TaskType.CAUSAL_LM
)

Q37. Write a Python function demonstrating Semantic Chunking using LangChain.

Live Coding Challenge

RecursiveCharacterTextSplitter with chunk_size and chunk_overlap.

from langchain.text_splitter import RecursiveCharacterTextSplitter

def split_documents(raw_text):
    splitter = RecursiveCharacterTextSplitter(
        chunk_size=500,
        chunk_overlap=50,
        separators=["\n\n", "\n", ". ", " ", ""]
    )
    return splitter.split_text(raw_text)

Q38. Write an OpenAI function calling / tool definition in Python for weather checking.

Live Coding Challenge

Define JSON schema tool for weather lookup.

tools = [{
    "type": "function",
    "function": {
        "name": "get_current_weather",
        "description": "Get current weather for a city",
        "parameters": {
            "type": "object",
            "properties": {
                "location": {"type": "string", "description": "City, e.g. Noida"},
                "unit": {"type": "string", "enum": ["celsius", "fahrenheit"]}
            },
            "required": ["location"]
        }
    }
}]

Q39. Write a prompt guardrail check in Python detecting system prompt leak attempts.

Live Coding Challenge

Regex or keyword detection on prompt extraction phrases.

import re

def check_prompt_injection(user_input: str) -> bool:
    patterns = [
        r"ignore previous instructions",
        r"reveal your system prompt",
        r"repeat the text above",
        r"you are now in developer mode"
    ]
    return any(re.search(p, user_input, re.IGNORECASE) for p in patterns)

Q40. Write a Python script to evaluate Faithfulness in RAG outputs.

Live Coding Challenge

Compare claims in generated answer against context statements.

def verify_groundedness(answer: str, context: str) -> bool:
    # Evaluates whether entities and numbers in answer exist in context
    answer_words = set(answer.lower().split())
    context_words = set(context.lower().split())
    overlap = len(answer_words.intersection(context_words)) / len(answer_words)
    return overlap > 0.6