What Is a Token in LLMs?

A complete word in the input text A sub-word unit from a learned vocabulary that serves as the model's basic processing element A single character of the input text A sentence boundary marker used during training