Function tokenize

  • Namespace

    StringUtils

    Tokenize a sentence into words.

    Parameters

    • sentence: string

      The input sentence to tokenize.

    Returns string[]

    An array of words in the sentence.

    Example

    const sentence = "The quick brown fox jumps over the lazy dog";
    const words = StringUtils.tokenizeSentence(sentence);
    console.log(words);
    // Output: ["The", "quick", "brown", "fox", "jumps", "over", "the", "lazy", "dog"]

Generated using TypeDoc