0
1
00:00:00,740 --> 00:00:07,370
So, in previous modules, you saw how we used CoreML in order to use a ready-made plug-and-play mlmodel
1

2
00:00:07,370 --> 00:00:12,900
to perform image recognition by adding it to our Xcode project.
2

3
00:00:12,920 --> 00:00:16,510
Now, in this tutorial, we're going to be going one step further,
3

4
00:00:16,550 --> 00:00:23,300
and we're going to show you how to convert a pre-trained Caffe model into the mlmodel format that you can
4

5
00:00:23,300 --> 00:00:25,380
use in your Xcode projects.
5

6
00:00:25,580 --> 00:00:31,790
So some of the code that you'll see in this tutorial will be more intermediate to advanced, but they
6

7
00:00:31,790 --> 00:00:35,030
have all been covered in the tutorials previously.
7

8
00:00:35,090 --> 00:00:40,750
So, ideally, you should have completed all of the modules leading up to this module.
8

9
00:00:40,760 --> 00:00:42,440
That way, you'll know all about
9

10
00:00:42,440 --> 00:00:48,260
"do catch" blocks, as well as "guard" statements and a lot of the things that we'll be using in order to
10

11
00:00:48,260 --> 00:00:49,120
make this app.
11

12
00:00:49,130 --> 00:00:54,560
So the Caffe model that we're using has already been trained on thousands upon thousands of images, and
12

13
00:00:54,590 --> 00:01:01,340
all that we're doing is taking that Caffe model and converting it into a .mlmodel which can be seen
13

14
00:01:01,430 --> 00:01:05,960
and used by our Swift files and which can be loaded into our app.
14

15
00:01:06,110 --> 00:01:12,380
So, as I mentioned previously, Apple has released open-source tools in Python that allow you to convert
15

16
00:01:12,470 --> 00:01:18,770
any pre-trained model that had been trained using Caffe, Keras, scikit-learn, and a whole list is available
16

17
00:01:18,860 --> 00:01:19,980
and a link to it.
17

18
00:01:20,090 --> 00:01:25,360
But the most important thing is that you need that .mlmodel file in order to incorporate it into
18

19
00:01:25,360 --> 00:01:26,660
your Xcode project,
19

20
00:01:26,660 --> 00:01:29,170
and we're going to show you how to do that in this tutorial.
20

21
00:01:29,240 --> 00:01:35,060
So when we were first talking about machine learning, I mentioned one of my favorite apps is an app called
21

22
00:01:35,180 --> 00:01:42,410
Garden, it allows you to take a picture of any flower and it will try to recognize what that flower
22

23
00:01:42,500 --> 00:01:43,440
is called.
23

24
00:01:43,460 --> 00:01:46,440
And in this mofule, we are going to make a clone of that app.
24

25
00:01:46,520 --> 00:01:51,920
So by the end of the tutorial, everything going well, you will have your very own flower recognition app,
25

26
00:01:52,010 --> 00:01:55,280
and you'll be able to recognize any flower that you see around you
26

27
00:01:55,430 --> 00:02:00,680
even if you don't have an internet connection, because our machine learning model is going to be preloaded
27

28
00:02:00,980 --> 00:02:03,580
onto the device and it won't need an API.
28

29
00:02:03,650 --> 00:02:09,280
So the data set that our model has been trained on is called the Oxford 102 Flower Dataset.
29

30
00:02:09,290 --> 00:02:15,860
So a bunch of researchers from Oxford did a very, very time -consuming task by labeling a whole bunch
30

31
00:02:15,860 --> 00:02:20,540
of images of flowers with the correct classification of the flower.
31

32
00:02:20,690 --> 00:02:26,720
And the reason why it's called Oxford 102 is because they have 102 categories for the flowers.
32

33
00:02:26,720 --> 00:02:33,500
And as you can see in their dataset, they've got anywhere between 400 and 200 images for each and every
33

34
00:02:33,500 --> 00:02:34,250
category.
34

35
00:02:34,370 --> 00:02:37,700
So for example, they have 71 images of buttercups.
35

36
00:02:37,700 --> 00:02:41,240
They have 102 images of Californian poppies,
36

37
00:02:41,330 --> 00:02:46,520
so images taken at different angles of the flower at different stages of life cycle of the flower.
37

38
00:02:46,640 --> 00:02:52,730
So that if you train a machine learning model on it, it should be able to classify with relative accuracy
38

39
00:02:53,060 --> 00:02:56,020
what flower is in the image that you've taken.
39

40
00:02:56,030 --> 00:03:02,120
So a very kind gentleman by the name of Jimmie Goode has used convolutional neural nets to train a machine
40

41
00:03:02,120 --> 00:03:07,850
learning model using this data set, and that is the Caffe model that we're going to be working with in
41

42
00:03:07,850 --> 00:03:08,710
this tutorial.
42

43
00:03:08,750 --> 00:03:09,000
All right.
43

44
00:03:09,000 --> 00:03:13,310
So I'm going to show you what the finished app will look like and what it will do.
44

45
00:03:13,310 --> 00:03:17,690
So our app is called WhatFlower, and it's got a nice little icon, and I'm going to show you how to create
45

46
00:03:17,690 --> 00:03:19,260
that step by step as well.
46

47
00:03:19,460 --> 00:03:25,430
And if you tap on it, you're able to hit the camera button and we're going to take a picture of some
47

48
00:03:25,490 --> 00:03:26,730
flowers I've got around here.
48

49
00:03:32,200 --> 00:03:32,430
All right.
49

50
00:03:32,440 --> 00:03:35,020
So there's our flower and we're going to hit
50

51
00:03:35,050 --> 00:03:36,040
Use Photo.
51

52
00:03:38,900 --> 00:03:42,870
And, voila, it tells us that this is called a Barberton Daisy.
52

53
00:03:42,910 --> 00:03:44,020
I had no clue.
53

54
00:03:44,490 --> 00:03:50,630
And this is the image that we pull up from the Wikipedia article for the Barberton Daisy, and there
54

55
00:03:50,650 --> 00:03:52,930
is a little bit of description about what it is
55

56
00:03:52,970 --> 00:03:54,140
from Wikipedia.
56

57
00:03:54,140 --> 00:03:55,230
Isn't that really, really cool?
57

58
00:03:55,280 --> 00:03:59,720
So this is what we're going to be building and I'm going to show you how to do it step by step.
58

59
00:03:59,720 --> 00:04:01,160
So let's get started.