  |
MATLAB Toolbox for the LabelMe Image Database LabelMe is a WEB-based image annotation tool that allows researchers to label images and share the annotations with the rest of the community. If you use this toolbox, we only ask you to contribute to the database, from time to time, by using the labeling tool. |
| Citation If you use this dataset, we would appreciate it if you cite: B. C. Russell, A. Torralba, K. P. Murphy, W. T. Freeman, LabelMe: a database and web-based tool for image annotation. MIT AI Lab Memo AIM-2005-025, September, 2005. (PDF) Contribute to the dataset If you find this dataset useful, you can help us to make it larger. Chose some unlabeled images from this folder and try to annotate as many objects as you can. Your annotations are made available for download inmediately. Here there are some examples of annotated images:
Content
1. Setup Download the toolbox and add it to the Matlab path. 2. A quick look into the dataset The toolbox allows you using the dataset online without needing to download
it first. Just execute the next lines to visualize the content of one
of the folders of the collection: 3. List of functions in the toolbox Introduction LabelMe tools for reading and ploting the annotations of individual files LMread - reads one image and the corresponding annotation XML file LMplot - image and polygons visualization LabelMe Database tools LMdatabase - loads all the annotations into a big database struct LMquery - performs a query on the database LMdbshowscenes - shows thumbnails for the all the images in the database LMdbshowobjects - shows crops of the objects in the database. LMobjectnames - returns a list with the name of all the objects in the database LMobjectindex - retuns the indices of an object class within the annotation struct LMcountobject - counts the number of instances of an object class in every image Image manipulation LMimread - reads one image from the database LMimscale - scales the image and the corresponding annotation LMimcrop - crops the image and the corresponding annotation LMimpad - pads an image with PADVAL and modifies the annotation Objects, polygons and segmentation LMobjectpolygon - returns all the polygons for an object class within an image LMobjectmask - returns the segmentation mask for all object instances of one class within an image LMobjectsinsideimage - removes polygons outside the image boundary Creation of training and test images and databases LMcookimage - reformat an image and annotation to fit certain requirements. LMcookdatabase - create tuned databases (you can control the difficulty, the object size, ...) Communication with online annotation tool LMphotoalbum - creates a web page with thumbnails and connected to LabelMe online. LMthumbnailsbar - creates a bar of thumbnails connected to LabelMe online Install and update images and annotations LMinstall - installs the database LMupdate - authomatic update of the annotations for specific files LMprogressiveinstall - will update your local copy of the images with only the new images and also download all of the annotations Translation from/to other formats PAS2LM - Translates PASCAL format to LabelMe LM2OpenCV - will output a query in format usable for OpenCV Haar Evaluation for object detection LMrecallPrecision - Precision-recall curve XML tools (translates the XML files into MATLAB struct arrays) writeXML - translates a struct array into a XML file loadXML - read XML file drawXML - shows the image and plots the polygons xml2struct - translates a XML string into a matlab struct struct2xml - translates a matlab struct into a XML string Object detection code A simple object detector using boosting - object detection with boosting trained with this dataset |
|
4. Using the toolbox 4.1 - Basic functions: indexing, queries First, you need to download the LabelMe
database or you need to provide the url of the images and annotations
online. If you are going to use the dataset frequently, it is better that
you get a local copy. 
% You can query the database looking for specific objects and visualize each instance in isolation: LMdbshowobjects(LMquery(database, 'object.name', 'bottle'), HOMEIMAGES); 
Creating training and test databases: the next lines show how you can create a database in which you can control the difficulty for detecting a specific object (or a set of objects). For instance, the next lines create a specialized database of images that contain only one computer screen, viewed from the front, keyboards and mouses. Furthermore, we want to normalize in size each image such that each screen has a size of 64x64 pixels, and each image has a maximal size of 128x128 pixels. % First, define the root folder for the new database NEWHOMEIMAGES = 'C:\yourpath\newdatabase\office\Images'; % you can set here your default folder NEWHOMEANNOTATIONS = 'C:\yourpath\newdatabase\office\Annotations'; % you can set here your default folder % Then, locate street scenes with frontal views of screens. [D,j] = LMquery(Dlabelme, 'object.name', 'screen+frontal'); % and now select the images with only one screen. counts = LMcountobject(Dlabelme(j), 'screen'); j = j(find(counts==1)); D = LMquery(Dlabelme(j), 'object.name', 'screen,keyboard,mouse-pad'); % Finally, cook database to fit our requirements LMcookdatabase(D, HOMEIMAGES, HOMEANNOTATIONS, NEWHOMEIMAGES, NEWHOMEANNOTATIONS, ... 'objectname', 'screen', 'objectsize', [64 64], 'objectlocation', 'original','maximagesize', [128 128]) % Load new database newdatabase = LMdatabase(NEWHOMEANNOTATIONS); % Show images: LMdbshowscenes(newdatabase, NEWHOMEIMAGES);  4.2 - Searching objects by viewpoint Some objects have information related to viewpoint in the annotation file. In the current setup, viewpoints are discretized in 12 different orientations. For instance, for cars we have:
In order to get all the frontal cars you can use the query: D = LMquery(Dlabelme, 'object.name', 'car+az270deg''); We have a tool that allows labeling viewpoints for new objects. If you want to add viewpoint information for new objects send us an email and we will send you a link to the tool. By using the online tool, the data that you will provide can be used in the future by other researchers. 4.3 - Depth ordering Frequently, an image will contain many partially overlapping polygons. This situation arises when users complete an occluded boundary or when labeling large regions containing small occluding objects. In these situations we need to know which polygon is on top in order to assign the image pixels to the correct object label. The function LMsortlayers will sort the polygons accorting to their relative depth ordering. The function uses some heuristics (see paper) and it will not be correct all the time.
4.4 - Object part hierarchies Despite that object parts are not explicitly labeled as such by the annotation tool, it is posible to authomatically discover object-part hierarchies. When two polygons have a high degree of overlap across many images, this provides evidence for an object-part hierarchy. The toolbox provides a set of functions to suggest parts for an object category. The function partsof will add a new field to each object.
The online query tool shows object parts whenever they are labeled. Here there are some examples:
In order to add parts to the object 'car', run the next line: D = partsof(Dlabelme, 'car', 'wheel,door,window,tire,mirror,license+plate,windshield,headlight,light'); 5 - Communication with the online tool You can look in the demo.m file, at the end there is a description that
explains how can you talk, using Matlab, with the online annotation tool
to label only the images that you want and how to get back just those
labels and update you annotation file. The effect is like if the annotation
tool was just working for you, but still all the annotations that you
will enter will be shared and stored online. 
|
