This box searches only this space. The box at the upper right searches the entire iPlant wiki.

# Taxonomy of Fitting Errors

In preparation of designing evaluation metrics for semantic reconstruction, below is a non-exhaustive list of ways fitting can fail.  The most interesting failure modes are those in which the raw structure matches well, appearing mostly correct using a purely geometric evaluation metric, but are still obviously incorrect to a human observer.

An ideal metric will both (a) identify these errors and (b) robustly evaluate good matches in the presence of these errors.   If identifying the error is too hard, the metric should at least be able to penalize it.

Key:

Green: Ground truth
Blue: Model

# Good match:

### Type 0: Unary Errors

Part correspondence failure

### Type 1: Pairwise Errors

Correct part correspondence

Incorrect internal correspondence

#### Subtype (c): Internal inclusion

Model part includes extra unnecessary internal structure

#### Subtype (d): Internal omission

Model omits internal structure from the ground-truth.
Complement of “Internal inclusion” subtype

#### Subtype (e): Routing error

Ordering of internal correspondences is not preserved.

• Types (a) through (d) are identified by structure that is unaccounted-for  in either the model or the ground-truth.  It could be argued that they are simply geometric errors

### Type 2: Correspondence errors

Partial incorrect part correspondence

#### Subtype (a): Many to one

(Many model to one GT)

#### Subtype (d): Node mis-location

Derivative subtypes:

• Many-to-many – Combo of subtypes (a) and (c)

• Need many-to-many correspondences to identify most of these explicitly.
• Geometry-level correspondence could do it
• One-to-one part correspondences will penalize these implicitly.
• Subtypes (a) and (b) will result in skip/insertion type Topological errors, as well as early start/stop pairwise errors.
• Subtype (c) becomes its own Topological error: generation reversal.
• Node mislocation may be considered a type of pairwise error?
• Or a separate category?

### Type 3: Topology errors

#### Subtype: Node skip

Model omits one or more intermediate nodes between matches.

#### Subtype: Node insertion

Model inserts one or more intermediate nodes.
(Complement of “Node Skip”)

#### Subtype: Late Birth

Model’s root node corresponds to non-root in GT

#### Subtype: Early Birth

Model’s root node corresponds to nothing in GT.
(Complement of “Late Birth”)

#### Subtype: Wrong Subtree

Child occurs in wrong subtree