1
00:00:00,570 --> 00:00:08,430
In this video, you learn how to scrape the comments off of Reddit posts, Reddit submission via Python.

2
00:00:08,610 --> 00:00:09,810
So what is Reddit?

3
00:00:10,200 --> 00:00:19,200
Reddit is this website and it consists of subreddits, so subreddits are like forums and each forum

4
00:00:19,200 --> 00:00:20,670
has a particular interest.

5
00:00:20,820 --> 00:00:26,010
For example, this one here is Reddit dot com slash or slash cats.

6
00:00:26,160 --> 00:00:32,729
This is a cat's subreddits where people talk about cats and postcards, videos and so on, and you have

7
00:00:32,729 --> 00:00:39,340
all sort of subreddits for every possible thing you can imagine on Reddit, so everyone can post.

8
00:00:39,360 --> 00:00:44,970
Everyone can comment, and everyone can use Python to do those things.

9
00:00:44,990 --> 00:00:50,760
So to post and comment and scrape comments from these posts and so on.

10
00:00:50,880 --> 00:00:55,830
So in this video, we're going to scrape the comments off this post here.

11
00:00:56,040 --> 00:01:00,330
For example, you can pick any posts that you like, so just go to Reddit.

12
00:01:01,840 --> 00:01:03,940
So that's the home page, for example.

13
00:01:04,330 --> 00:01:09,910
And you see different ones if you click on one that should take you to that post.

14
00:01:11,890 --> 00:01:12,970
So if I click.

15
00:01:14,140 --> 00:01:14,680
On this.

16
00:01:15,980 --> 00:01:25,100
Cat posts, it should take me to that post, so to access read it via Python, you need to create an

17
00:01:25,100 --> 00:01:26,870
app to do that.

18
00:01:26,900 --> 00:01:36,110
You should go to your profile here selected and then go to user settings, then go to safety and privacy

19
00:01:36,500 --> 00:01:37,880
and scroll down.

20
00:01:37,880 --> 00:01:41,690
And you should see here manage third party app authorization.

21
00:01:41,900 --> 00:01:47,630
If you don't see vets, you can also go directly to this page by entering Grady dot com slash prefs

22
00:01:47,810 --> 00:01:48,680
slash apps.

23
00:01:49,040 --> 00:01:54,410
But you should create a Reddit account first, and then you should go to this page.

24
00:01:54,710 --> 00:01:59,060
So if you scroll down here, you see these bots and create another app.

25
00:01:59,480 --> 00:02:09,289
So I'm going to click on vets, give a name to the app, automates with Python two, for example, and

26
00:02:09,289 --> 00:02:10,490
click on scripts.

27
00:02:11,360 --> 00:02:13,100
The description is optional.

28
00:02:13,400 --> 00:02:15,350
So is the airboats URL.

29
00:02:15,620 --> 00:02:16,790
For Redirects your URL.

30
00:02:16,790 --> 00:02:22,460
Just write something like HTP s colon slash slash example.

31
00:02:22,490 --> 00:02:25,050
Dot com and press on Create app.

32
00:02:26,270 --> 00:02:33,680
So that is the app I just created, and now I have this idea of the app, the secret of the app, and

33
00:02:33,680 --> 00:02:35,630
we should use them in Python.

34
00:02:35,960 --> 00:02:40,070
So let's go back to our Python script and import profile.

35
00:02:40,280 --> 00:02:48,770
So Pro is a Python third party library used to allow us accessing the Reddit's websites.

36
00:02:49,430 --> 00:02:55,070
So first of all, we should create a accredits instance using Proudlock to read it.

37
00:02:56,030 --> 00:02:57,380
So that is a class now.

38
00:02:58,070 --> 00:03:01,520
The first arguments you need to give is the user agent.

39
00:03:02,090 --> 00:03:09,500
Now this is some information that your script will send to the Reddit server, and this is not really

40
00:03:09,500 --> 00:03:12,020
important, so you can just write something like true.

41
00:03:12,770 --> 00:03:20,180
What is important is the client I.D., which consists of string, and you can get it.

42
00:03:21,560 --> 00:03:29,690
In your it's Abovitz, you created so that one on the personal use script that is the text you wanted,

43
00:03:29,690 --> 00:03:35,410
copied and pasted in here, then you need to give a client secret.

44
00:03:35,660 --> 00:03:39,620
Now this is getting long along the line, so I'm going to enter here.

45
00:03:39,800 --> 00:03:41,210
Two splits in the next line.

46
00:03:41,840 --> 00:03:46,010
The client's secret is again, the string this one.

47
00:03:47,080 --> 00:03:57,910
Copy that, place it there, your red, it's username, in my case, this is this one in here automates

48
00:03:57,910 --> 00:03:58,750
with Python.

49
00:04:00,070 --> 00:04:10,600
And it's also reflected in here, so automate with Python and the password of you ready to count.

50
00:04:11,890 --> 00:04:15,730
So this is just a throwaway account so that I created for this app.

51
00:04:16,420 --> 00:04:22,360
So these dots are will not work if you try them because I'm going to delete the app and also change

52
00:04:22,360 --> 00:04:22,960
the passwords.

53
00:04:23,230 --> 00:04:25,300
So please create your own account.

54
00:04:26,230 --> 00:04:28,150
So username and password is there.

55
00:04:28,510 --> 00:04:32,260
And therefore we do have this accredits instance now.

56
00:04:32,860 --> 00:04:36,460
So that is how you create a Reddit instance.

57
00:04:39,330 --> 00:04:44,340
I just shifted this a bit to the right to make them more visually appealing.

58
00:04:45,030 --> 00:04:47,640
So what is our purpose in this video?

59
00:04:48,360 --> 00:04:56,160
Our purpose is to get the comments off a post, so we need to give our Python program the URL of a post

60
00:04:56,250 --> 00:04:57,060
as a string.

61
00:04:58,050 --> 00:05:00,300
So that post was.

62
00:05:01,560 --> 00:05:04,020
Just pick whatever post the likes of this one.

63
00:05:04,440 --> 00:05:08,250
And you want to be inside that post.

64
00:05:08,250 --> 00:05:15,150
So if you see this view with comments on the posts, that means you are in that post page or submission.

65
00:05:15,600 --> 00:05:17,220
It's also called a submission.

66
00:05:17,490 --> 00:05:28,770
So you want to copy that you were l pasted there and then create a post variable Reddit submission.

67
00:05:29,400 --> 00:05:33,060
And this gets a useful argument which a sequel to the you variable.

68
00:05:33,240 --> 00:05:36,240
So that's one thing they know that we have.

69
00:05:36,240 --> 00:05:37,050
I paused.

70
00:05:37,680 --> 00:05:47,160
If you want to check what we have so far, you can print it out and see if things are working.

71
00:05:48,780 --> 00:05:50,040
So we don't get an error.

72
00:05:50,070 --> 00:05:53,430
However, we don't see any contents, either.

73
00:05:54,750 --> 00:05:57,210
That's just the idea of the post.

74
00:05:57,840 --> 00:06:05,010
And if you want to get the text the body of the post, you want to a polite self text.

75
00:06:06,000 --> 00:06:06,510
And run.

76
00:06:08,250 --> 00:06:11,850
Now, be aware of that for ports that don't have techs.

77
00:06:12,150 --> 00:06:14,260
You're going to get that empty streak.

78
00:06:14,280 --> 00:06:18,420
So for example, this only has a photo, a picture.

79
00:06:18,600 --> 00:06:25,110
So if you find a post with texts such as.

80
00:06:28,490 --> 00:06:29,330
This one in here.

81
00:06:29,900 --> 00:06:31,740
So that's the title.

82
00:06:31,790 --> 00:06:33,830
And that is the text of the post.

83
00:06:34,310 --> 00:06:40,730
If I copy that, you know and place it's in the you are a variable there.

84
00:06:41,750 --> 00:06:49,390
If I were doing this now, I'm going to get the body of the post printed out so they'd close off our

85
00:06:49,400 --> 00:06:49,940
everything.

86
00:06:50,270 --> 00:06:54,320
This one in here, if you want to get the title, you want to apply.

87
00:06:57,490 --> 00:06:59,350
That's title run.

88
00:07:01,050 --> 00:07:07,530
And that is the title, and that is the text of the post to get the comments.

89
00:07:08,780 --> 00:07:13,280
What he wants to do is he wants to iterate for comments in.

90
00:07:14,700 --> 00:07:16,440
Post those comments.

91
00:07:19,220 --> 00:07:26,390
And you want to print perhaps comment that body, so a comment.

92
00:07:27,140 --> 00:07:34,120
This is a list of comments and a comment as an object which has a body attributes.

93
00:07:34,820 --> 00:07:36,080
So if you wrong that.

94
00:07:38,470 --> 00:07:40,960
You should get all the comments.

95
00:07:42,510 --> 00:07:44,280
That's users who left.

96
00:07:45,390 --> 00:07:52,650
So if you want to see, for example, the length of the comments post that comments, you should apply

97
00:07:52,650 --> 00:07:53,040
that.

98
00:07:59,360 --> 00:08:04,880
And because there are too many comments here in the terminal, we don't see the length, so I'm going

99
00:08:04,880 --> 00:08:06,380
to comment that.

100
00:08:07,800 --> 00:08:11,160
Ron, again, so I see no.

101
00:08:13,090 --> 00:08:14,070
Down here.

102
00:08:14,830 --> 00:08:15,200
Yeah.

103
00:08:15,220 --> 00:08:17,870
So it's a hundred ninety three comments.

104
00:08:18,730 --> 00:08:26,770
But if you look at here, then you see that there are four hundred fifty three coalmines.

105
00:08:27,400 --> 00:08:32,590
But that is total comments which are nested comments as well, such as this one in here.

106
00:08:33,429 --> 00:08:37,590
But top level comments are these ones.

107
00:08:37,600 --> 00:08:44,200
So that's when it's a top level of comments and then goes on here.

108
00:08:45,010 --> 00:08:48,100
This one is a top level comment and so on.

109
00:08:48,220 --> 00:08:55,630
So there are 193 such top level comments, so that should give you the top level comments.

110
00:08:56,650 --> 00:09:03,280
So that is how you can access Reddit and scrape information.

111
00:09:04,060 --> 00:09:09,900
For more details, you should look up at the Pro Library documentation.

112
00:09:09,910 --> 00:09:13,330
It has more things you can do, so that's for now.

113
00:09:14,110 --> 00:09:23,140
In the next video, we're going to get the posts that were created during the last 24 hours on a particular

114
00:09:23,260 --> 00:09:24,310
subreddit.

115
00:09:24,700 --> 00:09:26,230
So let's do that in the next video.