01. Load web datasets with GluonCV Auto Module

This tutorial introduces the basic dataset preprocesses that can be used to download and load arbitrary custom dataset as long as they follow certain supported data formats.

The current version supports loading datasets for - Image Classification(with csv lists and raw images, or folder separated raw images) - Object Detection(as in Pascal VOC format or COCO json annatations)

Stay tuned for new applications and formats, we are also looking forward to seeing contributions that brings new formats to GluonCV!

That’s enough introduction, let’s take a look at how web datasets can be loaded into a recommended formats supported in GluonCV auto module.

Image Classification

Managing the labels of an image classification dataset is pretty simple. In this example we show a few ways to organize them.

First of all, we could infer labels from nested folder structure automatically like:

root/car/0001.jpg
root/car/xxxa.jpg
root/car/yyyb.jpg
root/bus/123.png
root/bus/023.jpg
root/bus/wwww.jpg

or even more with train/val/test splits like:

root/train/car/0001.jpg
root/train/car/xxxa.jpg
root/train/bus/123.png
root/train/bus/023.jpg
root/test/car/yyyb.jpg
root/test/bus/wwww.jpg

where root is the root folder, car and bus categories are well organized in sub-directories, respectively

from gluoncv.auto.tasks import ImageClassification

We can use ImageClassification.Dataset to load dataset from a folder, here root can be a local path or url, if it’s a url, the archieve file will be downloaded and extracted automatically to ~/.gluoncv by default, to change the default behavior, you may edit ~/.gluoncv/config.yml

train, val, test = ImageClassification.Dataset.from_folders(
    'https://autogluon.s3.amazonaws.com/datasets/shopee-iet.zip',
    train='train', val='val', test='test', exts=('.jpg', '.jpeg', '.png'))

Out:

Downloading /root/.gluoncv/archive/shopee-iet.zip from https://autogluon.s3.amazonaws.com/datasets/shopee-iet.zip...

  0%|          | 0/40895 [00:00<?, ?KB/s]
  0%|          | 38/40895 [00:00<02:28, 275.59KB/s]
  0%|          | 157/40895 [00:00<01:05, 617.98KB/s]
  2%|1         | 701/40895 [00:00<00:18, 2126.19KB/s]
  6%|5         | 2350/40895 [00:00<00:06, 5977.17KB/s]
 13%|#2        | 5118/40895 [00:00<00:03, 11000.64KB/s]
 20%|##        | 8206/40895 [00:00<00:02, 14808.33KB/s]
 28%|##7       | 11278/40895 [00:00<00:01, 17172.29KB/s]
 35%|###5      | 14350/40895 [00:01<00:01, 18706.07KB/s]
 43%|####2     | 17438/40895 [00:01<00:01, 19779.10KB/s]
 50%|#####     | 20510/40895 [00:01<00:00, 20435.26KB/s]
 58%|#####7    | 23582/40895 [00:01<00:00, 20957.66KB/s]
 65%|######5   | 26654/40895 [00:01<00:00, 21417.17KB/s]
 73%|#######2  | 29742/40895 [00:01<00:00, 21483.40KB/s]
 80%|########  | 32814/40895 [00:01<00:00, 21619.13KB/s]
 88%|########7 | 35854/40895 [00:02<00:00, 22117.94KB/s]
 95%|#########5| 38885/40895 [00:02<00:00, 24077.29KB/s]
100%|##########| 40895/40895 [00:02<00:00, 17735.20KB/s]
data/
├── test/
└── train/

train split

print('train', train)

Out:

train                                                  image  label
0    /root/.gluoncv/datasets/shopee-iet/data/train/...      0
1    /root/.gluoncv/datasets/shopee-iet/data/train/...      0
2    /root/.gluoncv/datasets/shopee-iet/data/train/...      0
3    /root/.gluoncv/datasets/shopee-iet/data/train/...      0
4    /root/.gluoncv/datasets/shopee-iet/data/train/...      0
..                                                 ...    ...
795  /root/.gluoncv/datasets/shopee-iet/data/train/...      3
796  /root/.gluoncv/datasets/shopee-iet/data/train/...      3
797  /root/.gluoncv/datasets/shopee-iet/data/train/...      3
798  /root/.gluoncv/datasets/shopee-iet/data/train/...      3
799  /root/.gluoncv/datasets/shopee-iet/data/train/...      3

[800 rows x 2 columns]

test split

print('test', test)

Out:

test                                                 image  label
0   /root/.gluoncv/datasets/shopee-iet/data/test/B...      0
1   /root/.gluoncv/datasets/shopee-iet/data/test/B...      0
2   /root/.gluoncv/datasets/shopee-iet/data/test/B...      0
3   /root/.gluoncv/datasets/shopee-iet/data/test/B...      0
4   /root/.gluoncv/datasets/shopee-iet/data/test/B...      0
..                                                ...    ...
75  /root/.gluoncv/datasets/shopee-iet/data/test/w...      3
76  /root/.gluoncv/datasets/shopee-iet/data/test/w...      3
77  /root/.gluoncv/datasets/shopee-iet/data/test/w...      3
78  /root/.gluoncv/datasets/shopee-iet/data/test/w...      3
79  /root/.gluoncv/datasets/shopee-iet/data/test/w...      3

[80 rows x 2 columns]

you may notice that the dataset is a pandas DataFrame, which are handy and it’s okay that certain split is empty, as in this case, validation split is empty

print('validation', val)

Out:

validation Empty ImageClassificationDataset
Columns: [image, label]
Index: []

you may split the train set to train and val for training and validation

train, val, _ = train.random_split(val_size=0.1, test_size=0)
print(len(train), len(val))

Out:

711 89

In some cases, you may get a raw folder without splits, you may use from_folders instead:

dataset = ImageClassification.Dataset.from_folder('https://s3.amazonaws.com/fast-ai-imageclas/oxford-iiit-pet.tgz')

Out:

Downloading /root/.gluoncv/archive/oxford-iiit-pet.tgz from https://s3.amazonaws.com/fast-ai-imageclas/oxford-iiit-pet.tgz...

  0%|          | 0/792683 [00:00<?, ?KB/s]
  1%|1         | 8183/792683 [00:00<00:11, 69777.30KB/s]
  2%|1         | 15161/792683 [00:00<00:11, 66650.27KB/s]
  3%|3         | 24214/792683 [00:00<00:09, 76878.48KB/s]
  4%|4         | 33291/792683 [00:00<00:09, 82146.71KB/s]
  5%|5         | 42318/792683 [00:00<00:08, 84996.59KB/s]
  6%|6         | 51422/792683 [00:00<00:08, 87015.99KB/s]
  8%|7         | 60474/792683 [00:00<00:08, 88145.18KB/s]
  9%|8         | 69620/792683 [00:00<00:08, 89188.83KB/s]
 10%|9         | 78664/792683 [00:00<00:07, 89574.64KB/s]
 11%|#1        | 87832/792683 [00:01<00:07, 90220.82KB/s]
 12%|#2        | 96863/792683 [00:01<00:08, 86103.61KB/s]
 13%|#3        | 105518/792683 [00:01<00:10, 67290.34KB/s]
 14%|#4        | 112885/792683 [00:01<00:10, 62645.90KB/s]
 15%|#5        | 120381/792683 [00:01<00:10, 65634.13KB/s]
 16%|#6        | 128729/792683 [00:01<00:09, 70228.23KB/s]
 17%|#7        | 137973/792683 [00:01<00:08, 76200.86KB/s]
 18%|#8        | 145912/792683 [00:01<00:09, 67189.35KB/s]
 20%|#9        | 155589/792683 [00:02<00:08, 74785.76KB/s]
 21%|##        | 163472/792683 [00:02<00:08, 73410.56KB/s]
 22%|##1       | 172020/792683 [00:02<00:08, 75183.49KB/s]
 23%|##2       | 180639/792683 [00:02<00:07, 78207.77KB/s]
 24%|##3       | 188631/792683 [00:02<00:07, 76607.52KB/s]
 25%|##4       | 196412/792683 [00:02<00:08, 69899.30KB/s]
 26%|##5       | 203578/792683 [00:02<00:09, 61440.56KB/s]
 27%|##6       | 211117/792683 [00:02<00:08, 64930.44KB/s]
 27%|##7       | 217865/792683 [00:02<00:09, 58165.88KB/s]
 28%|##8       | 223952/792683 [00:03<00:10, 55290.20KB/s]
 29%|##8       | 229666/792683 [00:03<00:10, 52571.36KB/s]
 30%|###       | 238533/792683 [00:03<00:08, 61701.21KB/s]
 31%|###1      | 245751/792683 [00:03<00:09, 60526.52KB/s]
 32%|###2      | 253943/792683 [00:03<00:08, 62894.74KB/s]
 33%|###3      | 262134/792683 [00:03<00:08, 63502.78KB/s]
 34%|###4      | 270325/792683 [00:03<00:07, 68173.80KB/s]
 35%|###5      | 278515/792683 [00:03<00:07, 71785.76KB/s]
 36%|###6      | 287661/792683 [00:04<00:06, 77133.09KB/s]
 37%|###7      | 295505/792683 [00:04<00:08, 59772.86KB/s]
 38%|###8      | 303635/792683 [00:04<00:07, 64904.81KB/s]
 39%|###9      | 312423/792683 [00:04<00:06, 70741.11KB/s]
 41%|####      | 321135/792683 [00:04<00:06, 75099.56KB/s]
 42%|####1     | 329974/792683 [00:04<00:05, 78759.63KB/s]
 43%|####2     | 338178/792683 [00:04<00:08, 56095.36KB/s]
 44%|####3     | 344910/792683 [00:05<00:08, 54281.38KB/s]
 44%|####4     | 351110/792683 [00:05<00:07, 55939.77KB/s]
 45%|####5     | 357297/792683 [00:05<00:07, 57200.05KB/s]
 46%|####5     | 363459/792683 [00:05<00:08, 49961.92KB/s]
 47%|####6     | 368893/792683 [00:05<00:13, 31455.55KB/s]
 48%|####7     | 376822/792683 [00:05<00:10, 38557.78KB/s]
 49%|####8     | 385014/792683 [00:05<00:09, 44846.52KB/s]
 50%|####9     | 393205/792683 [00:06<00:07, 50093.33KB/s]
 50%|#####     | 399012/792683 [00:06<00:10, 35904.79KB/s]
 51%|#####     | 403657/792683 [00:06<00:10, 35688.60KB/s]
 52%|#####1    | 409588/792683 [00:06<00:09, 39846.55KB/s]
 53%|#####2    | 417779/792683 [00:06<00:07, 47723.07KB/s]
 54%|#####3    | 426291/792683 [00:06<00:06, 56378.37KB/s]
 55%|#####4    | 435302/792683 [00:06<00:05, 64759.35KB/s]
 56%|#####5    | 443716/792683 [00:07<00:05, 68401.20KB/s]
 57%|#####6    | 451077/792683 [00:07<00:05, 62716.94KB/s]
 58%|#####7    | 459264/792683 [00:07<00:04, 67620.53KB/s]
 59%|#####8    | 466415/792683 [00:07<00:04, 67668.44KB/s]
 60%|#####9    | 473455/792683 [00:07<00:04, 63909.03KB/s]
 61%|######    | 482818/792683 [00:07<00:04, 71861.65KB/s]
 62%|######1   | 490252/792683 [00:07<00:04, 62995.86KB/s]
 63%|######2   | 499101/792683 [00:07<00:04, 69507.54KB/s]
 64%|######4   | 507894/792683 [00:08<00:04, 67578.17KB/s]
 65%|######5   | 516085/792683 [00:08<00:03, 69846.69KB/s]
 66%|######6   | 523281/792683 [00:08<00:03, 69372.04KB/s]
 67%|######6   | 530362/792683 [00:08<00:04, 59163.40KB/s]
 68%|######7   | 536599/792683 [00:08<00:04, 58312.82KB/s]
 69%|######8   | 545458/792683 [00:08<00:03, 66062.07KB/s]
 70%|#######   | 555337/792683 [00:08<00:03, 63918.32KB/s]
 71%|#######   | 561970/792683 [00:08<00:04, 56030.31KB/s]
 72%|#######2  | 570852/792683 [00:09<00:03, 63704.14KB/s]
 73%|#######2  | 577630/792683 [00:09<00:03, 58870.61KB/s]
 74%|#######3  | 583830/792683 [00:09<00:03, 59475.52KB/s]
 75%|#######4  | 593373/792683 [00:09<00:02, 68794.67KB/s]
 76%|#######5  | 600562/792683 [00:09<00:02, 68720.57KB/s]
 77%|#######6  | 607650/792683 [00:09<00:03, 61204.98KB/s]
 78%|#######7  | 615118/792683 [00:09<00:02, 64680.91KB/s]
 79%|#######8  | 623749/792683 [00:09<00:02, 70411.09KB/s]
 80%|#######9  | 631028/792683 [00:09<00:02, 64457.50KB/s]
 81%|########  | 638829/792683 [00:10<00:02, 68040.74KB/s]
 82%|########1 | 647766/792683 [00:10<00:01, 73889.24KB/s]
 83%|########2 | 655363/792683 [00:10<00:01, 70362.57KB/s]
 84%|########3 | 664210/792683 [00:10<00:01, 75346.06KB/s]
 85%|########4 | 671908/792683 [00:10<00:02, 53465.98KB/s]
 86%|########5 | 680063/792683 [00:10<00:01, 58695.26KB/s]
 87%|########6 | 686760/792683 [00:10<00:01, 56469.61KB/s]
 87%|########7 | 692979/792683 [00:10<00:01, 56329.37KB/s]
 88%|########8 | 699010/792683 [00:11<00:01, 56123.76KB/s]
 89%|########8 | 704899/792683 [00:11<00:02, 41322.62KB/s]
 90%|######### | 713736/792683 [00:11<00:01, 51301.28KB/s]
 91%|#########1| 722219/792683 [00:11<00:01, 59037.17KB/s]
 92%|#########2| 731168/792683 [00:11<00:00, 66622.88KB/s]
 93%|#########3| 739666/792683 [00:11<00:00, 71440.50KB/s]
 94%|#########4| 748476/792683 [00:11<00:00, 75983.37KB/s]
 96%|#########5| 757548/792683 [00:11<00:00, 80112.95KB/s]
 97%|#########6| 766693/792683 [00:12<00:00, 83354.21KB/s]
 98%|#########7| 775292/792683 [00:12<00:00, 64826.11KB/s]
 99%|#########8| 782585/792683 [00:12<00:00, 63318.74KB/s]
100%|#########9| 789474/792683 [00:12<00:00, 61825.71KB/s]
100%|##########| 792683/792683 [00:12<00:00, 63296.21KB/s]
oxford-iiit-pet/
├── annotations/
└── images/
print(dataset)

Out:

                                                  image  label
0     /root/.gluoncv/datasets/oxford-iiit-pet/oxford...      1
1     /root/.gluoncv/datasets/oxford-iiit-pet/oxford...      1
2     /root/.gluoncv/datasets/oxford-iiit-pet/oxford...      1
3     /root/.gluoncv/datasets/oxford-iiit-pet/oxford...      1
4     /root/.gluoncv/datasets/oxford-iiit-pet/oxford...      1
...                                                 ...    ...
7385  /root/.gluoncv/datasets/oxford-iiit-pet/oxford...      1
7386  /root/.gluoncv/datasets/oxford-iiit-pet/oxford...      1
7387  /root/.gluoncv/datasets/oxford-iiit-pet/oxford...      1
7388  /root/.gluoncv/datasets/oxford-iiit-pet/oxford...      1
7389  /root/.gluoncv/datasets/oxford-iiit-pet/oxford...      1

[7390 rows x 2 columns]

Visualize Image Classification Dataset

you may plot the sample images with show_images, like:

train.show_images(nsample=16, ncol=4, shuffle=True, fontsize=64)
BabyShirt: 1, BabyPants: 0, womenchiffontop: 3, BabyPants: 0, BabyPants: 0, womencasualshoes: 2, BabyShirt: 1, BabyPants: 0, BabyPants: 0, womenchiffontop: 3, BabyShirt: 1, BabyShirt: 1, BabyPants: 0, womencasualshoes: 2, womencasualshoes: 2, BabyPants: 0

Object Detection

The labels for object detection is a little bit more complicated than image classification, addtional information such as bounding box coordinates have to be stored in certain formats.

In GluonCV we support loading from common Pascal VOC and COCO formats.

The key difference between VOC and COCO format is the way how annotations are stored.

For VOC, raw images and annotations are stored in unique directory, where annotations are usually per image basis, e.g., JPEGImages/0001.jpeg and Annotations/0001.xml is a valid image-label pair.

In contrast, COCO format stores all labels in a single annotation file, e.g., all training annotations are stored in instaces_train2017.json, validation annotations are stored in instances_val2017.json.

Other than identifying the valid format of desired dataset, there’s not so much different in loading the dataset into gluoncv

from gluoncv.auto.tasks import ObjectDetection

A subset of Pascal VOC

dataset = ObjectDetection.Dataset.from_voc('https://autogluon.s3.amazonaws.com/datasets/tiny_motorbike.zip')

Out:

Downloading /root/.gluoncv/archive/tiny_motorbike.zip from https://autogluon.s3.amazonaws.com/datasets/tiny_motorbike.zip...

  0%|          | 0/21272 [00:00<?, ?KB/s]
  0%|          | 26/21272 [00:00<01:41, 209.79KB/s]
  1%|          | 196/21272 [00:00<00:23, 886.82KB/s]
  4%|3         | 825/21272 [00:00<00:06, 3008.34KB/s]
 13%|#2        | 2693/21272 [00:00<00:02, 7922.61KB/s]
 25%|##5       | 5413/21272 [00:00<00:01, 13054.00KB/s]
 40%|####      | 8533/21272 [00:00<00:00, 17152.05KB/s]
 55%|#####4    | 11637/21272 [00:00<00:00, 19683.50KB/s]
 69%|######8   | 14677/21272 [00:00<00:00, 21133.46KB/s]
 83%|########3 | 17685/21272 [00:01<00:00, 22055.80KB/s]
 97%|#########7| 20709/21272 [00:01<00:00, 22730.98KB/s]
21273KB [00:01, 16978.07KB/s]
tiny_motorbike/
├── Annotations/
├── ImageSets/
└── JPEGImages/

The dataset is once again a pandas DataFrame

print(dataset)

Out:

                                                 image  ...                         image_attr
0    /root/.gluoncv/datasets/tiny_motorbike/tiny_mo...  ...  {'width': 500.0, 'height': 375.0}
1    /root/.gluoncv/datasets/tiny_motorbike/tiny_mo...  ...  {'width': 500.0, 'height': 375.0}
2    /root/.gluoncv/datasets/tiny_motorbike/tiny_mo...  ...  {'width': 500.0, 'height': 333.0}
3    /root/.gluoncv/datasets/tiny_motorbike/tiny_mo...  ...  {'width': 500.0, 'height': 375.0}
4    /root/.gluoncv/datasets/tiny_motorbike/tiny_mo...  ...  {'width': 333.0, 'height': 500.0}
..                                                 ...  ...                                ...
215  /root/.gluoncv/datasets/tiny_motorbike/tiny_mo...  ...  {'width': 500.0, 'height': 333.0}
216  /root/.gluoncv/datasets/tiny_motorbike/tiny_mo...  ...  {'width': 500.0, 'height': 375.0}
217  /root/.gluoncv/datasets/tiny_motorbike/tiny_mo...  ...  {'width': 500.0, 'height': 375.0}
218  /root/.gluoncv/datasets/tiny_motorbike/tiny_mo...  ...  {'width': 500.0, 'height': 375.0}
219  /root/.gluoncv/datasets/tiny_motorbike/tiny_mo...  ...  {'width': 500.0, 'height': 331.0}

[220 rows x 3 columns]

The dataset supports random split as well

train, val, test = dataset.random_split(val_size=0.1, test_size=0.1)
print('train', len(train), 'val', len(val), 'test', len(test))

Out:

train 185 val 16 test 19

For object detection, rois column is a list of bounding boxes in dict, ‘image_attr’ is optional attributes that can accelerate some image pre-processing functions, for example:

print(train.loc[0])

Out:

image         /root/.gluoncv/datasets/tiny_motorbike/tiny_mo...
rois          [{'class': 'bicycle', 'xmin': 0.316, 'ymin': 0...
image_attr                    {'width': 500.0, 'height': 375.0}
Name: 0, dtype: object

Visualize Object Detection Dataset

you may plot the sample images as well as bounding boxes with show_images, like:

train.show_images(nsample=16, ncol=4, shuffle=True, fontsize=64)
Image(76), Image(194), Image(98), Image(104), Image(89), Image(181), Image(102), Image(22), Image(27), Image(106), Image(8), Image(190), Image(57), Image(210), Image(73), Image(204)

Next step

You have access to arbitrary datasets, e.g., kaggle competition datasets, you can start training by looking at these tutorials: - 02. Train Image Classification with Auto Estimator - 03. Train classifier or detector with HPO using GluonCV Auto task You may also check out the`d8 dataset <http://preview.d2l.ai/d8/main/>`_ with built-in datasets. D8 datasets is fully compatible with gluoncv.auto, you can directly plug-in datasets loaded from d8 and train with fit functions.

Total running time of the script: ( 0 minutes 46.304 seconds)

Gallery generated by Sphinx-Gallery