NLP Interview Questions: From TF-IDF to Transformers

Why Classic NLP Still Gets Asked

Transformers may dominate production NLP, but interviews still lean heavily on the classics. There's a reason: TF-IDF and bag-of-words questions reveal whether you understand representing text as numbers — the core idea that embeddings later industrialized. At Amazon and Airbnb, an NLP screen usually starts here before it ever reaches attention mechanisms.

Text Preprocessing: The Unsexy Questions

"Walk me through preparing raw text for a model."

A complete answer covers normalization steps in order: lowercasing, removing or standardizing punctuation and unicode, handling numbers, then tokenizing. Practice: Text Normalization Steps.

Two follow-ups appear constantly:

• Stopwords — high-frequency, low-information words ("the", "is"). Removing them shrinks feature space for count-based models, but modern transformer pipelines usually keep them because they carry syntax. Know both sides: What Are Stopwords?
• Stemming vs lemmatization — stemming chops suffixes fast and crudely ("studies" → "studi"); lemmatization maps to dictionary forms using linguistic rules ("studies" → "study"). Speed versus accuracy. Practice: Stemming vs Lemmatization

Bag of Words and Its Limits

Bag of words represents a document as word counts, ignoring order entirely. It's the standard "explain the simplest text representation" question, and the interviewer is fishing for its two weaknesses: no word order ("not good" ≈ "good") and huge sparse vectors.

from sklearn.feature_extraction.text import CountVectorizer

docs = ["data science interview", "science of data"]
X = CountVectorizer().fit_transform(docs)  # sparse counts, order lost

Practice: Bag of Words Representation.

TF-IDF: The Most Common NLP Question

"How does TF-IDF improve on raw counts?"

Term frequency rewards words common in this document; inverse document frequency penalizes words common across all documents. The product surfaces words that characterize a document. A word like "the" scores near zero everywhere; "backpropagation" scores high in the one document about neural networks.

Interviewers often ask for the intuition behind the log in IDF — dampening, so a word appearing in 1 of 10,000 documents isn't absurdly overweighted. Practice: TF-IDF Intuition.

N-grams: Buying Back Word Order

N-grams extend bag of words to sequences of n consecutive tokens, capturing local phrases ("new york", "not good"). The trade-off is combinatorial explosion — bigrams and trigrams multiply your vocabulary. Practice: What Are N-grams?.

Connecting It to Transformers

The strongest interview move is drawing the through-line: bag of words → TF-IDF → static embeddings → contextual embeddings. Each step fixes the previous one's blindness — counts ignore meaning, static embeddings give every word one vector regardless of context, and attention finally lets "bank" mean different things in "river bank" and "bank account".

Practice

We have 24 NLP interview questions spanning this whole progression, each with an explanation of what a complete answer includes.