type BNCCorpusReader

Parameters:

• root - A path pointer identifying the root directory for this corpus. If a string is specified, then it will be converted to a PathPointer automatically.
• fileids - A list of the files that make up this corpus. This list can either be specified explicitly, as a list of strings; or implicitly, as a regular expression over file paths. The absolute path for each file will be constructed by joining the reader's root to each file name.
• encoding - The default unicode encoding for the files that make up the corpus. encoding's value can be any of the following:
  • A string: encoding is the encoding name for all files.
  • A dictionary: encoding[file_id] is the encoding name for the file whose identifier is file_id. If file_id is not in encoding, then the file contents will be processed using non-unicode byte strings.
  • A list: encoding should be a list of (regexp, encoding) tuples. The encoding for a file whose identifier is file_id will be the encoding value for the first tuple whose regexp matches the file_id. If no tuple's regexp matches the file_id, the file contents will be processed using non-unicode byte strings.
  • None: the file contents of all files will be processed using non-unicode byte strings.
• tag_mapping_function - A function for normalizing or simplifying the POS tags returned by the tagged_words() or tagged_sents() methods.

Overrides: api.CorpusReader.__init__

(inherited documentation)

Returns all of the words and punctuation symbols in the specified file that were in text nodes -- ie, tags are ignored. Like the xml() method, fileid can only specify one file.

Parameters:

• strip_space - If true, then strip trailing spaces from word tokens. Otherwise, leave the spaces on the tokens.
• stem - If true, then use word stems instead of word strings.

Returns: list of str

the given file(s) as a list of words and punctuation symbols.

Overrides: xmldocs.XMLCorpusReader.words

Parameters:

• c5 - If true, then the tags used will be the more detailed c5 tags. Otherwise, the simplified tags will be used.
• strip_space - If true, then strip trailing spaces from word tokens. Otherwise, leave the spaces on the tokens.
• stem - If true, then use word stems instead of word strings.

Returns: list of (str,str)

the given file(s) as a list of tagged words and punctuation symbols, encoded as tuples (word,tag).

Parameters:

• strip_space - If true, then strip trailing spaces from word tokens. Otherwise, leave the spaces on the tokens.
• stem - If true, then use word stems instead of word strings.

Returns: list of (list of str)

the given file(s) as a list of sentences or utterances, each encoded as a list of word strings.

Parameters:

• c5 - If true, then the tags used will be the more detailed c5 tags. Otherwise, the simplified tags will be used.
• strip_space - If true, then strip trailing spaces from word tokens. Otherwise, leave the spaces on the tokens.
• stem - If true, then use word stems instead of word strings.

Returns: list of (list of (str,str))

the given file(s) as a list of sentences, each encoded as a list of (word,tag) tuples.