Package nltk :: Package tokenize :: Module treebank
[hide private]
[frames] | no frames]

Module treebank

source code

A regular-expression based word tokenizer that tokenizes sentences using the conventions used by the Penn Treebank.

Classes [hide private]
    Regexp-based treebank tokenizer
TreebankWordTokenizer
A word tokenizer that tokenizes sentences using the conventions used by the Penn Treebank.