Uses of Interface
opennlp.tools.tokenize.Detokenizer
Packages that use Detokenizer
Package
Description
Experimental package related to converting various corpora to OpenNLP Format.
Experimental package related to the
letsmt
corpus format.Package related to identifying sentence boundaries.
Contains classes related to finding token or words in a string.
-
Uses of Detokenizer in opennlp.tools.formats
Methods in opennlp.tools.formats that return DetokenizerModifier and TypeMethodDescriptionprotected Detokenizer
DetokenizerSampleStreamFactory.createDetokenizer
(DetokenizerParameter p) -
Uses of Detokenizer in opennlp.tools.formats.convert
Constructors in opennlp.tools.formats.convert with parameters of type DetokenizerModifierConstructorDescriptionNameToSentenceSampleStream
(Detokenizer detokenizer, ObjectStream<NameSample> samples, int chunkSize) Initializes aNameToSentenceSampleStream
.NameToTokenSampleStream
(Detokenizer detokenizer, ObjectStream<NameSample> samples) Initializes aNameToTokenSampleStream
.POSToSentenceSampleStream
(Detokenizer detokenizer, ObjectStream<POSSample> samples, int chunkSize) Initializes aPOSToSentenceSampleStream
.POSToTokenSampleStream
(Detokenizer detokenizer, ObjectStream<POSSample> samples) Initializes aPOSToTokenSampleStream
. -
Uses of Detokenizer in opennlp.tools.formats.letsmt
Constructors in opennlp.tools.formats.letsmt with parameters of type DetokenizerModifierConstructorDescriptionDetokenizeSentenceSampleStream
(Detokenizer detokenizer, ObjectStream<SentenceSample> samples) Initializes aDetokenizeSentenceSampleStream
. -
Uses of Detokenizer in opennlp.tools.sentdetect
Constructors in opennlp.tools.sentdetect with parameters of type Detokenizer -
Uses of Detokenizer in opennlp.tools.tokenize
Classes in opennlp.tools.tokenize that implement DetokenizerConstructors in opennlp.tools.tokenize with parameters of type DetokenizerModifierConstructorDescriptionDetokenizerEvaluator
(Detokenizer detokenizer, DetokenEvaluationErrorListener... listeners) Initializes the current instance with the givenDetokenizer
.TokenSample
(Detokenizer detokenizer, String[] tokens) Initializes ainstance
via aDetokenizer
.