Text Splitter

In short Splitting text breaks a long block into chunks by character count, word count, line count or a custom separator, and numbers the output. Character-count splitting is the one with a subtlety: cutting at exactly N characters lands mid-word, so a usable splitter backs up to the last word boundary and the chunks come out slightly under the limit rather than exactly at it. That is the correct behaviour for anything a person will read. The uses have grown rather than shrunk. Long messages have to be split for platforms with hard limits; documents have to be chunked before they are fed to a language model with a context window; subtitles have to be broken into readable lines. For the language-model case there is a rule worth knowing: splitting on paragraph or sentence boundaries preserves meaning far better than splitting on a character count, because a chunk cut mid-sentence loses the context that made it interpretable. Use the custom separator on a blank line where you can, and fall back to counts only when the text has no structure to split on.