Function tokenize

tokenize(sentence): string[]

Namespace

StringUtils

Tokenize a sentence into words.

Parameters

sentence: string

The input sentence to tokenize.

Returns string[]

An array of words in the sentence.

Example

const sentence = "The quick brown fox jumps over the lazy dog";
const words = StringUtils.tokenizeSentence(sentence);
console.log(words);
// Output: ["The", "quick", "brown", "fox", "jumps", "over", "the", "lazy", "dog"]

Generated using TypeDoc