While many existing trained networks exist for common objects such as people, animals, cars and so on, little data exists for smaller features such as the tips and bases of wheat ears and leaves. One part of the research here focusing on deep learning specifically looks at smaller, custom, datasets. For example pollen, leaf and ears in wheat, and spikelet count. Convolutional nerual networks are used here, ranging from stacked hour glass networks to identify features, to object detection using YOLO3. The above diagram illustrates a simple deep learning process involving convolutions and pooling layers, with the final fully connected layer to classify the feature. The second part of the deep learning research places emphasis on driving a small remote car equipped with a standard webcam.
Previous research by Jonathon Gibbs looked at the modelling of plants in 3D (3D Reconstruction). Plants, however, are rarely still. They sway and deform in light to moderate wind on a daily basis such movement has a strong effect on light levels within the canopy, with strong implications for photosynthesis.
This research develops the part-based visual tracking and enhanced 3D reconstruction methods needed to measure and model canopy movement and dynamic photosynthesis in rice and wheat populations. The new 4D models will inform an experimental programme investigating the relationship between movement, photosynthetic properties and plant mechanical properties.
Frames of videos are used to produce a series of 2D images and 3D models foreach point in time. Features are detected in 2D using a Convolutional Neural Network. Epipolar geometry and a probability based algorithm are used to match these identified features across multiple images. The mapping is then triangulated to form 3D positions. Features for each time point can be identified to determine static movement.
Previous work, namely Jonathons PhD involved the recovery of 3D descriptions of viewed objects from multiple images, also known as 3D modelling. It is a longstanding problem in computer vision. While in recent years improvements with respect to both quality and performance have been made, the methods are often applied to trivial objects, those such as man-made objects, with symmetrical properties or curvature and straightness attributes, and relatively simple often convex objects. The objects in this researched constitute crowded scenes, in which multiple, closely-packed objects occlude each other in many views, at the time of writing these are beyond the current state-of-the-art. They constitute a more difficult challenge and are troublesome to accurately represent. Crowded scenes generate high levels of occlusion ¬- where part of the object is not visible from the current view - and parallax - the effect of the object appearing to differ when viewed at different angles, making accurate reconstruction exponentially harder than for simple convex objects.
The reconstruction pipeline here involves the acquistion of 2D images using an Active Vision system from which point cloud and volumetric data can be obtained. A novel merging algorithm is proposed which combines both volumetric and point clouds to produce a more faithful set of points, the merged model. From this a novel clustering and reconstruction algorithm have been proposed which manipulates PCA, applies level sets and cell merging to produce a 3D description improving current state-of-the-art.
Robotics and Active Vision
Jonathon A. Gibbs, Michael Pound, Darren M. Wells, Erik Murchie, Andrew French, Tony Pridmore, (2018). Plant Phenotyping: An Active Vision Cell for Three-Dimensional Plant Shoot Reconstruction. Plant Physiology [link]
Jonathon A. Gibbs, Michael Pound, Darren M. Wells, Erik Murchie, Andrew French, Tony Pridmore, (2016) Approaches to three-dimensional reconstruction of plant shoot topology and geometry. Functional Plant Biology ISSN 1445-4408 [link]
Jonathon Gibbs, (2016). The British Machine Vision Association and Society for Pattern Recognition: Plants in Computer Vision; A Fully Automated Active Vision Cell for 3D Reconstruction of Plant Shoots
Jonathon A. Gibbs, Michael Pound, Darren M. Wells, Erik Murchie, Andrew French, Tony Pridmore, Three-Dimensional Reconstruction of Plant Shoots from Multiple Images using an Active Vision System In G. Kootstra, Y Edan, E van Henten, and M Bergerman (Eds.), Proceedings of the IROS Workshop on Agri-Food Robotics. Hamburg, October 2, 2015 [link]
Jonathon Gibbs, Graham Kendall, and Ender Özcan. "Scheduling english football fixtures over the holiday period using hyper-heuristics." In International Conference on Parallel Problem Solving from Nature, pp. 496-505. Springer, Berlin, Heidelberg, 2010. [link]
IBM Award for most outstanding project [link]
PhD - The University of Nottingham (2014-2017)
BSc (First Class) - The University of Nottingham (2006-2010)