countTokens
Token count. Equivalent to encode(text).size but implementations are encouraged to short-circuit when the underlying tokenizer offers a cheaper count path.
Token count. Equivalent to encode(text).size but implementations are encouraged to short-circuit when the underlying tokenizer offers a cheaper count path.