countTokens

open fun countTokens(text: String): Int

Token count. Equivalent to encode(text).size but implementations are encouraged to short-circuit when the underlying tokenizer offers a cheaper count path.