Package nltk :: Package chunk :: Module util :: Class ChunkScore

type ChunkScore

object --+
         |
        ChunkScore

A utility class for scoring chunk parsers. ChunkScore can evaluate a chunk parser's output, based on a number of statistics (precision, recall, f-measure, misssed chunks, incorrect chunks). It can also combine the scores from the parsing of multiple texts; this makes it signifigantly easier to evaluate a chunk parser that operates one sentence at a time.

Texts are evaluated with the score method. The results of evaluation can be accessed via a number of accessor methods, such as precision and f_measure. A typical use of the ChunkScore class is:

   >>> chunkscore = ChunkScore()
   >>> for correct in correct_sentences:
   ...     guess = chunkparser.parse(correct.leaves())
   ...     chunkscore.score(correct, guess)
   >>> print 'F Measure:', chunkscore.f_measure()
   F Measure: 0.823

Instance Methods

[hide private]

__init__(self, **kwargs)

source code

_updateMeasures(self)

source code

score(self, correct, guessed)
Given a correctly chunked sentence, score another chunked version of the same sentence.

source code

accuracy(self)
Returns: The overall tag-based accuracy for all text that have been scored by this ChunkScore, using the IOB (conll2000) tag encoding. source code

float

precision(self)
Returns: the overall precision for all texts that have been scored by this ChunkScore. source code

float

recall(self)
Returns: the overall recall for all texts that have been scored by this ChunkScore. source code

float

f_measure(self, alpha=0.5)
Returns: the overall F measure for all texts that have been scored by this ChunkScore. source code

list of chunks

missed(self)
Returns: the chunks which were included in the correct chunk structures, but not in the guessed chunk structures, listed in input order.

source code

list of chunks

incorrect(self)
Returns: the chunks which were included in the guessed chunk structures, but not in the correct chunk structures, listed in input order.

source code

list of chunks

correct(self)
Returns: the chunks which were included in the correct chunk structures, listed in input order.

source code

list of chunks

guessed(self)
Returns: the chunks which were included in the guessed chunk structures, listed in input order.

source code

__len__(self)

source code

String

__repr__(self)
Returns: a concise representation of this ChunkScoring. source code

String

__str__(self)
Returns: a verbose representation of this ChunkScoring. source code

Instance Variables

[hide private]

list of Token _fn
List of false negatives

int _fn_num
Number of false negatives.

list of Token _fp
List of false positives

int _fp_num
Number of false positives

list of Token _tp
List of true positives

int _tp_num
Number of true positives

kwargs
Keyword arguments:

Method Details

[hide private]

init(self, **kwargs)
(Constructor)

source code

Overrides: object.__init__: (inherited documentation)

score(self, correct, guessed)

source code

Given a correctly chunked sentence, score another chunked version of the same sentence.

Parameters:

correct (chunk structure) - The known-correct ("gold standard") chunked sentence.
guessed (chunk structure) - The chunked sentence to be scored.

accuracy(self)

source code

Returns:: The overall tag-based accuracy for all text that have been scored by this ChunkScore, using the IOB (conll2000) tag encoding.

precision(self)

source code

Returns: float: the overall precision for all texts that have been scored by this ChunkScore.

recall(self)

source code

Returns: float: the overall recall for all texts that have been scored by this ChunkScore.

f_measure(self, alpha=0.5)

source code

Parameters:

alpha (float) - the relative weighting of precision and recall. Larger alpha biases the score towards the precision value, while smaller alpha biases the score towards the recall value. alpha should have a value in the range [0,1].

Returns: float

the overall F measure for all texts that have been scored by this ChunkScore.

missed(self)

source code

Returns: list of chunks: the chunks which were included in the correct chunk structures, but not in the guessed chunk structures, listed in input order.

incorrect(self)

source code

Returns: list of chunks: the chunks which were included in the guessed chunk structures, but not in the correct chunk structures, listed in input order.

correct(self)

source code

Returns: list of chunks: the chunks which were included in the correct chunk structures, listed in input order.

guessed(self)

source code

Returns: list of chunks: the chunks which were included in the guessed chunk structures, listed in input order.

repr(self)
(Representation operator)

source code

Returns: String: a concise representation of this ChunkScoring.
Overrides: object.__repr__

str(self)
(Informal representation operator)

source code

Returns: String: a verbose representation of this ChunkScoring. This representation includes the precision, recall, and f-measure scores. For other information about the score, use the accessor methods (e.g., missed() and incorrect()).
Overrides: object.__str__

Instance Variable Details

[hide private]

kwargs

Keyword arguments:

max_tp_examples: The maximum number actual examples of true positives to record. This affects the correct member function: correct will not return more than this number of true positive examples. This does *not* affect any of the numerical metrics (precision, recall, or f-measure)
max_fp_examples: The maximum number actual examples of false positives to record. This affects the incorrect member function and the guessed member function: incorrect will not return more than this number of examples, and guessed will not return more than this number of true positive examples. This does *not* affect any of the numerical metrics (precision, recall, or f-measure)
max_fn_examples: The maximum number actual examples of false negatives to record. This affects the missed member function and the correct member function: missed will not return more than this number of examples, and correct will not return more than this number of true negative examples. This does *not* affect any of the numerical metrics (precision, recall, or f-measure)
chunk_node: A regular expression indicating which chunks should be compared. Defaults to '.*' (i.e., all chunks).

type ChunkScore

__init__(self, **kwargs) (Constructor)

score(self, correct, guessed)

accuracy(self)

precision(self)

recall(self)

f_measure(self, alpha=0.5)

missed(self)

incorrect(self)

correct(self)

guessed(self)

__repr__(self) (Representation operator)

__str__(self) (Informal representation operator)

kwargs

init(self, **kwargs)
(Constructor)

repr(self)
(Representation operator)

str(self)
(Informal representation operator)