Note
Click here to download the full example code
01. Load web datasets with GluonCV Auto Module¶
This tutorial introduces the basic dataset preprocesses that can be used to download and load arbitrary custom dataset as long as they follow certain supported data formats.
The current version supports loading datasets for - Image Classification(with csv lists and raw images, or folder separated raw images) - Object Detection(as in Pascal VOC format or COCO json annatations)
Stay tuned for new applications and formats, we are also looking forward to seeing contributions that brings new formats to GluonCV!
That’s enough introduction, let’s take a look at how web datasets can be loaded into a recommended formats supported in GluonCV auto module.
Image Classification¶
Managing the labels of an image classification dataset is pretty simple. In this example we show a few ways to organize them.
First of all, we could infer labels from nested folder structure automatically like:
root/car/0001.jpg
root/car/xxxa.jpg
root/car/yyyb.jpg
root/bus/123.png
root/bus/023.jpg
root/bus/wwww.jpg
or even more with train/val/test splits like:
root/train/car/0001.jpg
root/train/car/xxxa.jpg
root/train/bus/123.png
root/train/bus/023.jpg
root/test/car/yyyb.jpg
root/test/bus/wwww.jpg
where root is the root folder, car and bus categories are well organized in sub-directories, respectively
from gluoncv.auto.tasks import ImageClassification
We can use ImageClassification.Dataset to load dataset from a folder, here root can be a local path or url, if it’s a url, the archieve file will be downloaded and extracted automatically to ~/.gluoncv by default, to change the default behavior, you may edit ~/.gluoncv/config.yml
train, val, test = ImageClassification.Dataset.from_folders(
'https://autogluon.s3.amazonaws.com/datasets/shopee-iet.zip',
train='train', val='val', test='test', exts=('.jpg', '.jpeg', '.png'))
Out:
Downloading /root/.gluoncv/archive/shopee-iet.zip from https://autogluon.s3.amazonaws.com/datasets/shopee-iet.zip...
0%| | 0/40895 [00:00<?, ?KB/s]
0%| | 38/40895 [00:00<02:14, 303.84KB/s]
0%| | 157/40895 [00:00<00:59, 681.54KB/s]
2%|1 | 684/40895 [00:00<00:17, 2283.49KB/s]
6%|5 | 2324/40895 [00:00<00:05, 6527.56KB/s]
13%|#3 | 5380/40895 [00:00<00:02, 13488.17KB/s]
21%|## | 8452/40895 [00:00<00:01, 17305.61KB/s]
28%|##8 | 11476/40895 [00:00<00:01, 19539.71KB/s]
35%|###5 | 14500/40895 [00:00<00:01, 20988.99KB/s]
43%|####2 | 17508/40895 [00:01<00:01, 21904.01KB/s]
50%|##### | 20532/40895 [00:01<00:00, 22560.02KB/s]
58%|#####7 | 23540/40895 [00:01<00:00, 22978.17KB/s]
65%|######4 | 26548/40895 [00:01<00:00, 23263.02KB/s]
72%|#######2 | 29572/40895 [00:01<00:00, 23494.03KB/s]
80%|#######9 | 32564/40895 [00:01<00:00, 23576.38KB/s]
87%|########6 | 35572/40895 [00:01<00:00, 23673.10KB/s]
94%|#########4| 38612/40895 [00:01<00:00, 23819.69KB/s]
100%|##########| 40895/40895 [00:02<00:00, 19583.15KB/s]
data/
├── test/
└── train/
train split
print('train', train)
Out:
train image label
0 /root/.gluoncv/datasets/shopee-iet/data/train/... 0
1 /root/.gluoncv/datasets/shopee-iet/data/train/... 0
2 /root/.gluoncv/datasets/shopee-iet/data/train/... 0
3 /root/.gluoncv/datasets/shopee-iet/data/train/... 0
4 /root/.gluoncv/datasets/shopee-iet/data/train/... 0
.. ... ...
795 /root/.gluoncv/datasets/shopee-iet/data/train/... 3
796 /root/.gluoncv/datasets/shopee-iet/data/train/... 3
797 /root/.gluoncv/datasets/shopee-iet/data/train/... 3
798 /root/.gluoncv/datasets/shopee-iet/data/train/... 3
799 /root/.gluoncv/datasets/shopee-iet/data/train/... 3
[800 rows x 2 columns]
test split
print('test', test)
Out:
test image label
0 /root/.gluoncv/datasets/shopee-iet/data/test/B... 0
1 /root/.gluoncv/datasets/shopee-iet/data/test/B... 0
2 /root/.gluoncv/datasets/shopee-iet/data/test/B... 0
3 /root/.gluoncv/datasets/shopee-iet/data/test/B... 0
4 /root/.gluoncv/datasets/shopee-iet/data/test/B... 0
.. ... ...
75 /root/.gluoncv/datasets/shopee-iet/data/test/w... 3
76 /root/.gluoncv/datasets/shopee-iet/data/test/w... 3
77 /root/.gluoncv/datasets/shopee-iet/data/test/w... 3
78 /root/.gluoncv/datasets/shopee-iet/data/test/w... 3
79 /root/.gluoncv/datasets/shopee-iet/data/test/w... 3
[80 rows x 2 columns]
you may notice that the dataset is a pandas DataFrame, which are handy and it’s okay that certain split is empty, as in this case, validation split is empty
print('validation', val)
Out:
validation Empty ImageClassificationDataset
Columns: [image, label]
Index: []
you may split the train set to train and val for training and validation
train, val, _ = train.random_split(val_size=0.1, test_size=0)
print(len(train), len(val))
Out:
721 79
In some cases, you may get a raw folder without splits, you may use from_folders instead:
dataset = ImageClassification.Dataset.from_folder('https://s3.amazonaws.com/fast-ai-imageclas/oxford-iiit-pet.tgz')
Out:
Downloading /root/.gluoncv/archive/oxford-iiit-pet.tgz from https://s3.amazonaws.com/fast-ai-imageclas/oxford-iiit-pet.tgz...
0%| | 0/792683 [00:00<?, ?KB/s]
1%| | 7158/792683 [00:00<00:10, 71576.66KB/s]
2%|1 | 15079/792683 [00:00<00:10, 76058.60KB/s]
3%|2 | 22685/792683 [00:00<00:12, 63609.05KB/s]
4%|3 | 29249/792683 [00:00<00:19, 39424.62KB/s]
4%|4 | 34151/792683 [00:00<00:19, 39390.10KB/s]
5%|5 | 40948/792683 [00:00<00:17, 43162.66KB/s]
6%|6 | 49140/792683 [00:01<00:15, 48019.85KB/s]
7%|7 | 57331/792683 [00:01<00:13, 53479.97KB/s]
8%|8 | 65523/792683 [00:01<00:12, 58444.63KB/s]
9%|9 | 72009/792683 [00:01<00:12, 58760.55KB/s]
10%|9 | 78089/792683 [00:01<00:17, 39713.09KB/s]
10%|# | 82954/792683 [00:01<00:17, 40240.02KB/s]
11%|#1 | 90097/792683 [00:01<00:16, 42948.04KB/s]
12%|#2 | 98288/792683 [00:02<00:13, 49975.78KB/s]
13%|#3 | 106133/792683 [00:02<00:12, 56644.61KB/s]
14%|#4 | 112330/792683 [00:02<00:12, 54274.76KB/s]
15%|#4 | 118128/792683 [00:02<00:17, 38613.98KB/s]
15%|#5 | 122826/792683 [00:02<00:16, 39701.12KB/s]
16%|#6 | 127426/792683 [00:02<00:18, 35137.01KB/s]
17%|#6 | 131426/792683 [00:02<00:20, 32107.51KB/s]
18%|#7 | 139386/792683 [00:03<00:16, 39878.79KB/s]
18%|#8 | 143751/792683 [00:03<00:17, 37062.64KB/s]
19%|#8 | 147714/792683 [00:03<00:22, 29281.27KB/s]
19%|#9 | 152385/792683 [00:03<00:21, 30058.00KB/s]
20%|#9 | 155659/792683 [00:03<00:25, 24693.77KB/s]
21%|## | 163828/792683 [00:03<00:19, 32864.38KB/s]
21%|##1 | 169123/792683 [00:04<00:16, 36927.83KB/s]
22%|##2 | 174670/792683 [00:04<00:15, 41085.85KB/s]
23%|##2 | 180211/792683 [00:04<00:14, 42491.50KB/s]
24%|##3 | 188402/792683 [00:04<00:12, 47803.07KB/s]
25%|##4 | 196761/792683 [00:04<00:10, 56569.63KB/s]
26%|##5 | 202757/792683 [00:04<00:13, 44632.11KB/s]
26%|##6 | 207808/792683 [00:04<00:13, 43205.04KB/s]
27%|##6 | 213714/792683 [00:04<00:12, 46862.07KB/s]
28%|##7 | 221566/792683 [00:05<00:10, 54712.65KB/s]
29%|##8 | 227960/792683 [00:05<00:09, 57125.28KB/s]
30%|##9 | 234013/792683 [00:05<00:11, 46634.53KB/s]
30%|### | 239206/792683 [00:05<00:12, 45723.90KB/s]
31%|### | 245653/792683 [00:05<00:10, 50315.02KB/s]
32%|###1 | 251041/792683 [00:05<00:13, 41136.37KB/s]
32%|###2 | 255643/792683 [00:05<00:14, 38130.15KB/s]
33%|###2 | 261182/792683 [00:06<00:12, 42065.70KB/s]
34%|###3 | 265755/792683 [00:06<00:16, 32710.55KB/s]
34%|###4 | 269559/792683 [00:06<00:17, 30388.71KB/s]
34%|###4 | 273291/792683 [00:06<00:16, 31864.30KB/s]
35%|###5 | 279338/792683 [00:06<00:13, 38536.49KB/s]
36%|###6 | 286947/792683 [00:06<00:10, 47921.97KB/s]
37%|###7 | 294900/792683 [00:06<00:10, 47410.56KB/s]
38%|###8 | 303091/792683 [00:06<00:09, 53693.66KB/s]
39%|###8 | 308776/792683 [00:07<00:09, 53344.77KB/s]
40%|###9 | 314327/792683 [00:07<00:10, 45104.37KB/s]
40%|#### | 319162/792683 [00:07<00:12, 37476.48KB/s]
41%|#### | 323290/792683 [00:07<00:14, 33120.33KB/s]
41%|####1 | 328030/792683 [00:07<00:12, 36124.38KB/s]
42%|####1 | 331978/792683 [00:07<00:12, 36388.08KB/s]
42%|####2 | 335858/792683 [00:08<00:16, 26897.81KB/s]
43%|####3 | 341077/792683 [00:08<00:14, 32040.21KB/s]
44%|####3 | 344866/792683 [00:08<00:13, 32547.96KB/s]
44%|####4 | 350543/792683 [00:08<00:14, 31346.82KB/s]
45%|####4 | 353998/792683 [00:08<00:13, 31595.47KB/s]
46%|####5 | 361666/792683 [00:08<00:10, 42187.66KB/s]
47%|####6 | 368631/792683 [00:08<00:09, 45348.06KB/s]
48%|####7 | 376822/792683 [00:09<00:09, 44971.39KB/s]
49%|####8 | 385014/792683 [00:09<00:07, 52816.72KB/s]
49%|####9 | 391780/792683 [00:09<00:07, 56421.82KB/s]
50%|##### | 397769/792683 [00:09<00:07, 50322.65KB/s]
51%|##### | 403129/792683 [00:09<00:08, 47254.88KB/s]
52%|#####1 | 411223/792683 [00:09<00:06, 55470.13KB/s]
53%|#####2 | 417779/792683 [00:09<00:06, 57618.69KB/s]
54%|#####3 | 425971/792683 [00:09<00:05, 62763.53KB/s]
55%|#####4 | 434162/792683 [00:09<00:05, 61970.05KB/s]
56%|#####5 | 441934/792683 [00:10<00:05, 66095.00KB/s]
57%|#####6 | 449341/792683 [00:10<00:05, 63627.24KB/s]
58%|#####7 | 455843/792683 [00:10<00:05, 56807.10KB/s]
58%|#####8 | 461716/792683 [00:10<00:05, 56558.35KB/s]
59%|#####8 | 467503/792683 [00:10<00:07, 43232.77KB/s]
60%|#####9 | 475128/792683 [00:10<00:07, 41723.05KB/s]
61%|###### | 483319/792683 [00:10<00:06, 47217.21KB/s]
62%|######2 | 491674/792683 [00:11<00:05, 55146.03KB/s]
63%|######3 | 499702/792683 [00:11<00:04, 58800.95KB/s]
64%|######4 | 507894/792683 [00:11<00:04, 59827.45KB/s]
65%|######5 | 515534/792683 [00:11<00:04, 63903.70KB/s]
66%|######5 | 522228/792683 [00:11<00:04, 59733.61KB/s]
67%|######6 | 528436/792683 [00:11<00:05, 49851.88KB/s]
67%|######7 | 533803/792683 [00:11<00:06, 40158.20KB/s]
68%|######8 | 542565/792683 [00:12<00:05, 50009.03KB/s]
69%|######9 | 548851/792683 [00:12<00:05, 47033.79KB/s]
70%|####### | 555809/792683 [00:12<00:04, 52104.98KB/s]
71%|####### | 561583/792683 [00:12<00:04, 48189.43KB/s]
72%|#######1 | 567935/792683 [00:12<00:04, 51833.95KB/s]
73%|#######2 | 576986/792683 [00:12<00:03, 61655.62KB/s]
74%|#######3 | 583599/792683 [00:12<00:03, 59246.85KB/s]
74%|#######4 | 589844/792683 [00:12<00:03, 56548.26KB/s]
75%|#######5 | 598008/792683 [00:13<00:03, 59956.78KB/s]
76%|#######6 | 604167/792683 [00:13<00:04, 37737.18KB/s]
77%|#######6 | 609040/792683 [00:13<00:05, 35028.49KB/s]
78%|#######7 | 614391/792683 [00:13<00:04, 36278.51KB/s]
78%|#######8 | 620139/792683 [00:13<00:04, 40632.21KB/s]
79%|#######8 | 625245/792683 [00:13<00:04, 36054.91KB/s]
80%|#######9 | 630774/792683 [00:14<00:04, 37983.18KB/s]
81%|######## | 638965/792683 [00:14<00:03, 45033.11KB/s]
81%|########1 | 643808/792683 [00:14<00:03, 40072.53KB/s]
82%|########1 | 648096/792683 [00:14<00:03, 40382.64KB/s]
82%|########2 | 653643/792683 [00:14<00:03, 43482.03KB/s]
83%|########3 | 658197/792683 [00:14<00:04, 33268.16KB/s]
84%|########3 | 665742/792683 [00:14<00:02, 42347.98KB/s]
85%|########4 | 670869/792683 [00:15<00:02, 42681.40KB/s]
85%|########5 | 675618/792683 [00:15<00:02, 42232.94KB/s]
86%|########6 | 684217/792683 [00:15<00:02, 53224.41KB/s]
87%|########7 | 689985/792683 [00:15<00:02, 45496.09KB/s]
88%|########7 | 695443/792683 [00:15<00:02, 47224.87KB/s]
88%|########8 | 700540/792683 [00:15<00:02, 45979.63KB/s]
89%|########8 | 705395/792683 [00:15<00:02, 40859.55KB/s]
90%|########9 | 712696/792683 [00:15<00:01, 45587.80KB/s]
91%|######### | 720888/792683 [00:16<00:01, 50264.63KB/s]
92%|#########1| 729079/792683 [00:16<00:01, 57654.49KB/s]
93%|#########2| 735566/792683 [00:16<00:00, 57470.11KB/s]
94%|#########3| 741481/792683 [00:16<00:01, 45366.57KB/s]
94%|#########4| 747607/792683 [00:16<00:01, 43918.65KB/s]
95%|#########4| 752329/792683 [00:16<00:00, 42414.43KB/s]
95%|#########5| 756782/792683 [00:16<00:00, 39267.43KB/s]
97%|#########6| 765894/792683 [00:16<00:00, 51414.73KB/s]
97%|#########7| 771479/792683 [00:17<00:00, 28415.17KB/s]
98%|#########8| 778368/792683 [00:17<00:00, 34956.05KB/s]
99%|#########8| 783429/792683 [00:17<00:00, 36944.34KB/s]
99%|#########9| 788306/792683 [00:17<00:00, 34495.88KB/s]
100%|##########| 792683/792683 [00:17<00:00, 44310.06KB/s]
oxford-iiit-pet/
├── annotations/
└── images/
print(dataset)
Out:
image label
0 /root/.gluoncv/datasets/oxford-iiit-pet/oxford... 1
1 /root/.gluoncv/datasets/oxford-iiit-pet/oxford... 1
2 /root/.gluoncv/datasets/oxford-iiit-pet/oxford... 1
3 /root/.gluoncv/datasets/oxford-iiit-pet/oxford... 1
4 /root/.gluoncv/datasets/oxford-iiit-pet/oxford... 1
... ... ...
7385 /root/.gluoncv/datasets/oxford-iiit-pet/oxford... 1
7386 /root/.gluoncv/datasets/oxford-iiit-pet/oxford... 1
7387 /root/.gluoncv/datasets/oxford-iiit-pet/oxford... 1
7388 /root/.gluoncv/datasets/oxford-iiit-pet/oxford... 1
7389 /root/.gluoncv/datasets/oxford-iiit-pet/oxford... 1
[7390 rows x 2 columns]
Visualize Image Classification Dataset¶
you may plot the sample images with show_images, like:
train.show_images(nsample=16, ncol=4, shuffle=True, fontsize=64)
Object Detection¶
The labels for object detection is a little bit more complicated than image classification, addtional information such as bounding box coordinates have to be stored in certain formats.
In GluonCV we support loading from common Pascal VOC and COCO formats.
The key difference between VOC and COCO format is the way how annotations are stored.
For VOC, raw images and annotations are stored in unique directory, where annotations are usually per image basis, e.g., JPEGImages/0001.jpeg and Annotations/0001.xml is a valid image-label pair.
In contrast, COCO format stores all labels in a single annotation file, e.g., all training annotations are stored in instaces_train2017.json, validation annotations are stored in instances_val2017.json.
Other than identifying the valid format of desired dataset, there’s not so much different in loading the dataset into gluoncv
from gluoncv.auto.tasks import ObjectDetection
A subset of Pascal VOC
dataset = ObjectDetection.Dataset.from_voc('https://autogluon.s3.amazonaws.com/datasets/tiny_motorbike.zip')
Out:
Downloading /root/.gluoncv/archive/tiny_motorbike.zip from https://autogluon.s3.amazonaws.com/datasets/tiny_motorbike.zip...
0%| | 0/21272 [00:00<?, ?KB/s]
0%| | 33/21272 [00:00<01:32, 230.76KB/s]
1%| | 152/21272 [00:00<00:31, 665.92KB/s]
2%|1 | 407/21272 [00:00<00:14, 1407.20KB/s]
6%|6 | 1376/21272 [00:00<00:04, 4202.20KB/s]
15%|#5 | 3194/21272 [00:00<00:02, 8742.53KB/s]
27%|##7 | 5802/21272 [00:00<00:01, 13690.82KB/s]
36%|###6 | 7754/21272 [00:00<00:00, 15339.78KB/s]
49%|####8 | 10346/21272 [00:00<00:00, 17830.20KB/s]
58%|#####7 | 12266/21272 [00:01<00:00, 18086.12KB/s]
70%|######9 | 14874/21272 [00:01<00:00, 19678.12KB/s]
79%|#######9 | 16841/21272 [00:01<00:00, 19388.14KB/s]
91%|#########1| 19402/21272 [00:01<00:00, 20559.57KB/s]
21273KB [00:01, 15047.99KB/s]
tiny_motorbike/
├── Annotations/
├── ImageSets/
└── JPEGImages/
The dataset is once again a pandas DataFrame
print(dataset)
Out:
image ... image_attr
0 /root/.gluoncv/datasets/tiny_motorbike/tiny_mo... ... {'width': 500.0, 'height': 375.0}
1 /root/.gluoncv/datasets/tiny_motorbike/tiny_mo... ... {'width': 500.0, 'height': 375.0}
2 /root/.gluoncv/datasets/tiny_motorbike/tiny_mo... ... {'width': 500.0, 'height': 333.0}
3 /root/.gluoncv/datasets/tiny_motorbike/tiny_mo... ... {'width': 500.0, 'height': 375.0}
4 /root/.gluoncv/datasets/tiny_motorbike/tiny_mo... ... {'width': 333.0, 'height': 500.0}
.. ... ... ...
215 /root/.gluoncv/datasets/tiny_motorbike/tiny_mo... ... {'width': 500.0, 'height': 333.0}
216 /root/.gluoncv/datasets/tiny_motorbike/tiny_mo... ... {'width': 500.0, 'height': 375.0}
217 /root/.gluoncv/datasets/tiny_motorbike/tiny_mo... ... {'width': 500.0, 'height': 375.0}
218 /root/.gluoncv/datasets/tiny_motorbike/tiny_mo... ... {'width': 500.0, 'height': 375.0}
219 /root/.gluoncv/datasets/tiny_motorbike/tiny_mo... ... {'width': 500.0, 'height': 331.0}
[220 rows x 3 columns]
The dataset supports random split as well
train, val, test = dataset.random_split(val_size=0.1, test_size=0.1)
print('train', len(train), 'val', len(val), 'test', len(test))
Out:
train 170 val 23 test 27
For object detection, rois column is a list of bounding boxes in dict, ‘image_attr’ is optional attributes that can accelerate some image pre-processing functions, for example:
print(train.loc[0])
Out:
image /root/.gluoncv/datasets/tiny_motorbike/tiny_mo...
rois [{'class': 'bicycle', 'xmin': 0.316, 'ymin': 0...
image_attr {'width': 500.0, 'height': 375.0}
Name: 0, dtype: object
Visualize Object Detection Dataset¶
you may plot the sample images as well as bounding boxes with show_images, like:
train.show_images(nsample=16, ncol=4, shuffle=True, fontsize=64)
Next step¶
You have access to arbitrary datasets, e.g., kaggle competition datasets, you can start training by looking at these tutorials: - 02. Train Image Classification with Auto Estimator - 03. Train classifier or detector with HPO using GluonCV Auto task You may also check out the`d8 dataset <http://preview.d2l.ai/d8/main/>`_ with built-in datasets. D8 datasets is fully compatible with gluoncv.auto, you can directly plug-in datasets loaded from d8 and train with fit functions.
Total running time of the script: ( 0 minutes 52.567 seconds)