My current research interests are in Computer Vision and Machine Learning. See some of my current and past work below.

Leafsnap: an electronic field guide

Leafsnap logoLeafsnap is a user-friendly mobile app that identifies tree species from photos of their leaves, counting with over 1.5 million downloads. The recognition back end employs computer vision algorithms. Through the app, we have collected a large amount of data submitted by users, which allowed us to map the current geographic distribution of species. Leafsnap was born as a collaboration between Columbia University, University of Maryland and the Smithsonian Institution. See more information at the Leafsnap website.

Related publications:

  • Neeraj Kumar, Peter N. Belhumeur, Arijit Biswas, David W. Jacobs, W. John Kress, Ida C. Lopez, and João V.B. Soares. “Leafsnap: A computer vision system for automatic plant species identification.” In European Conference on Computer Vision (ECCV), pp. 502-516, 2012. [pdf]
  • João V.B. Soares, and David W. Jacobs. “Efficient segmentation of leaves in semi-controlled conditions.” Machine Vision and Applications 24, no. 8 (2013):1623-1643. [pdf]
  • W. John Kress, Carlos Garcia-Robledo, João V.B. Soares, David Jacobs, Katharine Wilson, Ida C. Lopez, and Peter N. Belhumeur. “Citizen Science and Climate Change: Mapping the Range Expansions of Native and Exotic Plants with the Mobile App Leafsnap.” BioScience 68, no. 5 (2018):348-358. [pdf]

Missing tooth detection in mine shovels

Missing tooth detectionWe developed a system for automatic detection of missing teeth in the buckets of mine shovels. This is a relevant industrial problem and at the same time very challenging, due to partial or complete occlusion of the teeth by earth, as well as changes in pose and lighting, including presence of cast shadows. I worked on this project during an internship at General Electric Global Research, were I was fortunate to collaborate with Ser Nam Lim and Ning Zhou.

Related publication:

  • Ser Nam Lim, João Soares, and Ning Zhou. “Tooth Guard: A Vision System for Detecting Missing Tooth in Rope Mine Shovel.” In Winter Conference on Applications of Computer Vision (WACV), 2016. [pdf]

2D shape attributes

Shape attributes: elongatednessWe developed a comprehensive set of attributes for 2D shape analysis. The attributes are designed to be intuitive and computationally efficient, so they may be applied to a range of problems where prior knowledge can be used. We experimented with the attributes on satellite images for discrimination of man-made objects (example here) and on photographs of leaves from Leafsnap for identification of tree species (example here).

Retinal vessel segmentation using wavelets and statistical classifiers

Retina project logoDuring my M.Sc., I worked on segmenting the blood vessel networks in retinal images. For this, we used 2D wavelets and supervised pixel classification. The segmentations obtained were then analyzed through automated shape analysis, with the main goal of assisting in automated diabetic retinopathy screening. See more at the project’s website. This work was done with the Creativision group of the Department of Computer Science at the University of São Paulo.

Related publications:

  • João V.B. Soares, Jorge J.G. Leandro, Roberto M. Cesar-Jr., Herbert F. Jelinek, and Michael J. Cree. “Retinal vessel segmentation using the 2-D Gabor wavelet and supervised classification.” IEEE Transactions on Medical Imaging 25, no. 9 (2006): 1214-1222. [pdf]
  • Herbert F. Jelinek, Michael J. Cree, Jorge J.G. Leandro, João V.B. Soares, Roberto M. Cesar-Jr., and A. Luckie. “Automated segmentation of retinal blood vessels and identification of proliferative diabetic retinopathy.” JOSA A 24, no. 5 (2007): 1448-1456. [pdf]
  • João V.B. Soares, and Roberto M. Cesar-Jr. Segmentation of Retinal Vasculature Using Wavelets and Supervised Classification: Theory and Implementation, in Automated Image Detection of Retinal Pathology, Herbert F. Jelinek and Michael J. Cree, Editors. CRC Press, 2009. [pdf]

%d bloggers like this: