Package uk.ac.starlink.ttools
Class Tokenizer
java.lang.Object
uk.ac.starlink.ttools.Tokenizer
Utility class to help with tokenizing strings.
- Since:
- 1 Sep 2005
- Author:
- Mark Taylor
-
Method Summary
Modifier and TypeMethodDescriptionstatic String[]
tokenizeLines
(String text) Splits a string up into lines, separated by semicolons or newlines.static String[]
tokenizeWords
(String line) Chops up a line of text into tokens.
-
Method Details
-
tokenizeLines
Splits a string up into lines, separated by semicolons or newlines. Semicolons may appear inside quoted strings without terminating a line.- Parameters:
text
- input string- Returns:
- array of lines
- Throws:
uk.ac.starlink.task.UsageException
-
tokenizeWords
Chops up a line of text into tokens. Works roughly like the shell, as regards quotes, whitespace and comments.- Parameters:
line
- line of text- Returns:
- array of words corresponding to
line
- Throws:
uk.ac.starlink.task.UsageException
-