Navigate space

Progress Report

This document will summarize my progress since the last update, covering approximately 12 weeks.

* 2D Ground Truth

* Branching Gaussian Process

* 3D Ground Truth from 2D Ground Truth

* 3D inference by data-association

2D Ground Truth

We've finished the first phase of 2D ground truth collection.  90 images have been hand-annotated, which provides us a gold standard agains which to evaluate our algorithm.

2D Annotation Tool (Complete)

Developed in-house with C++, OpenGL and GLUT.  Features:

• Users trace all plant stems using polybezier curves.  
• Curves are tracked between views.
• Users indicate branching topology ("curve #3 branches from curve #2", etc) 
• Prediction of partially-occluded curves by triangulating other views

Demo: http://youtu.be/S_7IkJBJFsQ

We have finished our inital phase of annotation, which includes a total 90 plant images from 10 different plants.

Branching Gaussian Process (Complete)

We've been doing research into Gaussian processes (GP) as a model for smooth stems in 3D.  Combining triangulation with traditional GP smoothing provides a model that (a) permits tractible inference, (b) has strong theoretical basis, and (c) provides visually satisfactory results.  In addition, we've adapted the traditional GP smoothing model to introduce geometric constraints that force stems to be attached to their parents.

3D Ground Truth from 2D Ground Truth (Complete)

Using the 2D ground-truth annotations we can we can reconstruct 3D stem structure.

1.  Use the dynamic-programming algorithm we developed over the summer, to estimate the point-to-point correspondences between views of corresponding curves.
2. Using the calibrated cameras from each view, triangulate the corresponding points to get rough 3d geometry
3. Smooth each curve using traditoinal GP smoothing.
4. Estimate branch points for each curve by finding the nearest point on each its parent.
5. Discard curves from step 3; Re-smooth the triangulated points using branching GP.

The resulting curves are (a) consistent with 2D annotations, (b) smooth, and (c) fully-connected.

We have performed 3D ground-truth reconstruction on the "phase 1" dataset (10 plants).

Demo: http://youtu.be/XJkPK_aS3Ic

3D Bayesian inference using data-association (In Progress)

We are in the process of developing a fully-automatic system for 3D plant reconstruction.  It will be an based on 3D ground-truth reconstruction algorithm, but instead of hand-annotations, curves will be extracted using an image-processing algorithm we developed over the summer.  This will require additional features not present in the ground-truth reconstruction:

1.  Estimating the unknown curve-to-curve correspondences.

2.  Better handling of noise, gaps, fragmentation, and false-positives that result from the image processing step.

We expect to have a first version running within the next week.