NLP Interview Questions: From TF-IDF to Transformers
Why Classic NLP Still Gets Asked
Transformers may dominate production NLP, but interviews still lean heavily on the classics. There's a reason: TF-IDF and bag-of-words questions reveal whether you understand representing text as numbers — the core idea that embeddings later industrialized. At Amazon and Airbnb, an NLP screen usually starts here before it ever reaches attention mechanisms.
Text Preprocessing: The Unsexy Questions
"Walk me through preparing raw text for a model."
A complete answer covers normalization steps in order: lowercasing, removing or standardizing punctuation and unicode, handling numbers, then tokenizing. Practice: Text Normalization Steps.
Two follow-ups appear constantly:
- Stopwords — high-frequency, low-information words ("the", "is"). Removing them shrinks feature space for count-based models, but modern transformer pipelines usually keep them because they carry syntax. Know both sides: What Are Stopwords?
- Stemming vs lemmatization — stemming chops suffixes fast and crudely ("studies" → "studi"); lemmatization maps to dictionary forms using linguistic rules ("studies" → "study"). Speed versus accuracy. Practice: Stemming vs Lemmatization
Bag of Words and Its Limits
Bag of words represents a document as word counts, ignoring order entirely. It's the standard "explain the simplest text representation" question, and the interviewer is fishing for its two weaknesses: no word order ("not good" ≈ "good") and huge sparse vectors.
from sklearn.feature_extraction.text import CountVectorizer
docs = ["data science interview", "science of data"]
X = CountVectorizer().fit_transform(docs) # sparse counts, order lost
Practice: Bag of Words Representation.
TF-IDF: The Most Common NLP Question
"How does TF-IDF improve on raw counts?"
Term frequency rewards words common in this document; inverse document frequency penalizes words common across all documents. The product surfaces words that characterize a document. A word like "the" scores near zero everywhere; "backpropagation" scores high in the one document about neural networks.
Interviewers often ask for the intuition behind the log in IDF — dampening, so a word appearing in 1 of 10,000 documents isn't absurdly overweighted. Practice: TF-IDF Intuition.
N-grams: Buying Back Word Order
N-grams extend bag of words to sequences of n consecutive tokens, capturing local phrases ("new york", "not good"). The trade-off is combinatorial explosion — bigrams and trigrams multiply your vocabulary. Practice: What Are N-grams?.
Connecting It to Transformers
The strongest interview move is drawing the through-line: bag of words → TF-IDF → static embeddings → contextual embeddings. Each step fixes the previous one's blindness — counts ignore meaning, static embeddings give every word one vector regardless of context, and attention finally lets "bank" mean different things in "river bank" and "bank account".
Practice
We have 24 NLP interview questions spanning this whole progression, each with an explanation of what a complete answer includes.
Frequently Asked Questions
What is the first step in an NLP interview question about preparing text?
The standard answer walks through text normalization in order: lowercasing, removing or standardizing punctuation and unicode characters, handling numbers, and then tokenizing the cleaned text. Interviewers listen for this ordered sequence rather than an ad hoc list of steps.
Should stopwords always be removed before building an NLP model?
Not always — removing high-frequency, low-information words like 'the' and 'is' shrinks the feature space and helps count-based models like bag-of-words or TF-IDF, but modern embedding-based and transformer models often perform better when stopwords are kept, since they carry useful grammatical context.
Why do NLP interviews still cover TF-IDF and bag-of-words when transformers dominate production?
TF-IDF and bag-of-words questions test whether you understand the core idea of representing text as numbers, which is the same underlying problem that embeddings and transformers later industrialized at scale. Interviewers use the classics as a foundation check before moving on to attention-based methods.
Ready to test your skills?
Practice real Nlp interview questions from top companies — with solutions.
Get interview tips in your inbox
Join data scientists preparing smarter. No spam, unsubscribe anytime.