1 00:00:00,150 --> 00:00:05,760 In the previous video, we wrote the script, which returns news on a given topic. 2 00:00:05,939 --> 00:00:08,460 And these news are in JSON format. 3 00:00:09,000 --> 00:00:12,970 Now in this video, we're going to go deeper and understand this. 4 00:00:12,990 --> 00:00:20,220 Jason data so that we can clean them and then then we can represent them to the user in a more user 5 00:00:20,220 --> 00:00:21,600 friendly format. 6 00:00:21,960 --> 00:00:24,840 So if you didn't follow of that video, that's OK. 7 00:00:25,080 --> 00:00:26,640 The script is pretty simple. 8 00:00:26,730 --> 00:00:28,680 All we do is we import requests. 9 00:00:28,800 --> 00:00:39,240 So requests allows us to perform HTP requests, so we request the contents of this URL using this method. 10 00:00:40,200 --> 00:00:41,580 So that's an A.P. request. 11 00:00:41,580 --> 00:00:42,630 You can do it with Python. 12 00:00:42,630 --> 00:00:44,280 You can do it from your web browser. 13 00:00:44,460 --> 00:00:47,940 So put the URL there and it gives you the contents all the time. 14 00:00:47,940 --> 00:00:49,140 We visit web pages. 15 00:00:49,440 --> 00:00:52,770 We do HDP requests, even though we don't realize that. 16 00:00:54,120 --> 00:00:59,170 So that will give you an R sort of a request object. 17 00:00:59,640 --> 00:01:07,110 And if you apply the JSON method from that request object, you will get a dictionary. 18 00:01:07,710 --> 00:01:10,680 So a dictionary is made of keys and values. 19 00:01:11,130 --> 00:01:18,480 This particular dictionary that you get from the new API has a nautical scheme, so that's the same 20 00:01:18,480 --> 00:01:18,960 dictionary. 21 00:01:18,960 --> 00:01:23,220 We can view this from Python and you see this article key here. 22 00:01:23,580 --> 00:01:32,360 So by doing that content and then we access the key articles that will give us the value of the article's 23 00:01:32,370 --> 00:01:33,050 key. 24 00:01:33,540 --> 00:01:35,790 The value of the article is that this one here? 25 00:01:38,070 --> 00:01:46,830 So it's a long list, therefore, if we run this, we're going to get that least. 26 00:01:47,550 --> 00:01:55,740 Now the first thing we want to do when we work with APIs is to understand for ourselves how these deals 27 00:01:55,740 --> 00:01:56,910 are structured. 28 00:01:57,390 --> 00:02:03,510 And it's not easy to do that from this results here on the command line. 29 00:02:03,990 --> 00:02:06,630 This is just a string is badly formatted. 30 00:02:07,350 --> 00:02:13,410 So what I prefer to use to understand the data for myself is to use a debugger. 31 00:02:14,400 --> 00:02:20,210 So if you're using a modern I.D., you have a debugger there on repo. 32 00:02:20,250 --> 00:02:23,010 There's also a debugger, so I'm going to use that debugger. 33 00:02:23,250 --> 00:02:27,990 If you don't know how to use your debugger in your I.D., then it's fine. 34 00:02:28,000 --> 00:02:34,110 You can just view the video for the next minute and see how the data are structured. 35 00:02:34,770 --> 00:02:43,440 If I click on the debugger here and then I can run this spot before I run, it's I want to set a breakpoints. 36 00:02:44,070 --> 00:02:50,430 For example, you set the breakpoint at Line six by clicking here and this dot shows. 37 00:02:51,060 --> 00:02:58,770 That means if I run the scripts using the run button over the debugger, what will happen is that the 38 00:02:58,770 --> 00:03:03,990 execution will freeze before the break points. 39 00:03:05,190 --> 00:03:12,600 That means here on this panel, I have access to the results of the variables in a nice format. 40 00:03:13,320 --> 00:03:22,740 So now the scripts is at this point and therefore I can see the content variable like can expand it. 41 00:03:24,510 --> 00:03:26,250 I can expand this window also. 42 00:03:26,670 --> 00:03:30,990 So you see that this is a dictionary format. 43 00:03:31,410 --> 00:03:35,100 So it has this article's key. 44 00:03:36,600 --> 00:03:40,080 And then you can also see the key in here. 45 00:03:40,830 --> 00:03:44,100 You can expand that and the value of that key has. 46 00:03:45,170 --> 00:03:45,980 Other values. 47 00:03:46,220 --> 00:03:49,370 So it's a hierarchy of dots on. 48 00:03:50,920 --> 00:03:58,180 So, for example, if I answer here zero, I will get the first article. 49 00:03:58,870 --> 00:04:02,140 And then this is made of all of the keys. 50 00:04:02,470 --> 00:04:04,450 So you have all three, you have the title. 51 00:04:04,630 --> 00:04:06,160 So if you add title there. 52 00:04:08,900 --> 00:04:13,160 You get the title of that article or the contents, if you like. 53 00:04:16,180 --> 00:04:19,000 So there's a content that's its value. 54 00:04:19,839 --> 00:04:21,760 So that's the first article, right? 55 00:04:24,690 --> 00:04:28,080 We have more we have the article with index one. 56 00:04:28,290 --> 00:04:33,750 So if you want to give the article the next one, you do that or what you could do. 57 00:04:33,840 --> 00:04:37,020 I'm going to stop the debugger now. 58 00:04:38,250 --> 00:04:41,070 What you could do is you could stall. 59 00:04:43,430 --> 00:04:44,450 All the articles. 60 00:04:47,520 --> 00:04:48,320 In a variable. 61 00:04:51,510 --> 00:04:58,720 And that will give you so type articles is a list. 62 00:04:59,880 --> 00:05:04,830 You see, it's a list of different dictionaries. 63 00:05:05,250 --> 00:05:13,830 So if you see the fall now for our article in articles Prince article. 64 00:05:17,500 --> 00:05:24,520 Title, for example, and that will give you only the titles for all the articles in the news feeds. 65 00:05:27,080 --> 00:05:29,690 So it's sourced from somewhere from here. 66 00:05:33,030 --> 00:05:35,490 That's one title for each line. 67 00:05:35,880 --> 00:05:37,590 So one title there, another there. 68 00:05:38,140 --> 00:05:48,750 And you can also get that the description along the title, so you can copy that and say Description, 69 00:05:51,150 --> 00:05:51,570 Ron. 70 00:05:53,640 --> 00:05:55,020 And so you get more data. 71 00:05:56,640 --> 00:05:59,460 Of course, you can format this as you want. 72 00:06:00,030 --> 00:06:02,430 You can do like title. 73 00:06:06,450 --> 00:06:09,240 Backslash and to create a brake line, for example. 74 00:06:13,750 --> 00:06:21,670 Description, backslash and and yeah, that's should improve, um. 75 00:06:22,380 --> 00:06:23,010 Here we go. 76 00:06:23,880 --> 00:06:28,050 This needs to be a comma and we have a comma here as well. 77 00:06:28,920 --> 00:06:29,700 Let's run again. 78 00:06:32,260 --> 00:06:34,180 Yes or no, it's formatted better. 79 00:06:34,420 --> 00:06:35,680 Here is the title. 80 00:06:36,590 --> 00:06:38,470 And then comes the description. 81 00:06:40,510 --> 00:06:46,240 You could also add a backslash and before the description to split it from the title. 82 00:06:46,660 --> 00:06:50,440 So that's the title, the description title and so on. 83 00:06:51,520 --> 00:06:54,430 So that's about the structure of the data itself. 84 00:06:54,700 --> 00:06:57,910 Now let's look at the structure of the URL again. 85 00:06:59,080 --> 00:07:02,530 And currently, this is a static URL. 86 00:07:02,740 --> 00:07:05,560 So I'd like to have a function. 87 00:07:05,560 --> 00:07:11,980 Is that where the user can pass values for? 88 00:07:13,570 --> 00:07:20,680 The key word they want to search for at this point, we only search for stock markets news. 89 00:07:20,980 --> 00:07:27,310 So we want to make that more dynamic and one way to do that is to write a function, define gets news 90 00:07:28,060 --> 00:07:30,010 and what parameters do we want to give? 91 00:07:30,040 --> 00:07:33,580 Well, um, let's say the topic. 92 00:07:36,730 --> 00:07:47,950 Then we have from date to date and then we have language, I'm going to leave this English by default 93 00:07:49,180 --> 00:07:50,320 and then we have more. 94 00:07:50,620 --> 00:07:52,420 Let me write, that's in the next line. 95 00:07:53,200 --> 00:07:54,700 It would be the API key. 96 00:07:55,780 --> 00:08:03,430 So you need to give the API key from the new API that all websites you need to go there and create an 97 00:08:03,430 --> 00:08:06,020 account to get the API key for this. 98 00:08:06,040 --> 00:08:14,170 I'm also going to leave it as a default value, my API key because this is not changing the language 99 00:08:14,170 --> 00:08:17,260 and the API key are not really changing much. 100 00:08:18,190 --> 00:08:25,420 In my case, at least the column and let's remove those spaces. 101 00:08:26,470 --> 00:08:29,710 And so what I want to do now is to construct the URL. 102 00:08:30,910 --> 00:08:32,020 So that's the URL. 103 00:08:32,020 --> 00:08:33,460 I'm going to get it from there. 104 00:08:37,340 --> 00:08:38,330 And places here. 105 00:08:38,600 --> 00:08:41,270 So this is going to be a live stream this time. 106 00:08:42,679 --> 00:08:45,890 Uh, it's F and yeah, like that's. 107 00:08:48,290 --> 00:08:49,220 So instead of. 108 00:08:50,890 --> 00:08:53,460 That's I'm going to place. 109 00:08:55,340 --> 00:08:56,480 A placeholder. 110 00:08:57,440 --> 00:08:58,580 There goes that topic. 111 00:09:00,560 --> 00:09:09,380 And from this date, instead of that static dates I want to place from the heat that variable. 112 00:09:18,300 --> 00:09:27,640 To date, sought by popularity, that's fine, I'm not going to move that language instead of English. 113 00:09:27,660 --> 00:09:33,150 I'm going to put language my parameter and the API key is. 114 00:09:36,010 --> 00:09:39,560 API key, that's parameter. 115 00:09:41,350 --> 00:09:44,830 So what I've have constructed the URL, then I want to. 116 00:09:48,260 --> 00:09:54,140 Apply or equal to requests, they'll get an answer to the URL that. 117 00:09:57,270 --> 00:09:59,310 Let me make some space in here. 118 00:10:01,800 --> 00:10:08,670 Requests that gets you are and then we have the content they see go to our Jason. 119 00:10:12,200 --> 00:10:20,060 Then we go the articles in here and then we apply the full loop. 120 00:10:22,240 --> 00:10:30,490 But instead of printing out these results, what I'm going to do instead is I will have an empty list. 121 00:10:30,880 --> 00:10:39,970 Let's see results, a sequel to an empty list and then I say here is salts that append and when I append 122 00:10:39,970 --> 00:10:47,890 the title and the description and the article there are, but this would be an f string. 123 00:10:48,190 --> 00:10:55,720 So f title and then article goes inside the place holder. 124 00:10:55,720 --> 00:10:59,650 But I'll change these single quotes here to double codes. 125 00:11:03,200 --> 00:11:10,670 Because we have single quotes in sides in here and do the same for the all other variable. 126 00:11:13,790 --> 00:11:16,910 And then finally return results. 127 00:11:18,430 --> 00:11:20,720 You know, let's see what areas we have. 128 00:11:24,080 --> 00:11:24,910 Let's run. 129 00:11:26,770 --> 00:11:29,470 So those are the results from these codes. 130 00:11:29,710 --> 00:11:33,550 But I'm going to come on this out because we don't need that anymore. 131 00:11:34,300 --> 00:11:41,260 And also call the function so it gets news. 132 00:11:43,090 --> 00:11:45,640 Topic is space. 133 00:11:45,670 --> 00:11:56,050 And from date is twenty twenty two to twenty seven and to date is. 134 00:12:01,410 --> 00:12:12,090 Twenty two to and 28 of the other parameters are default parameters, so that's all we need to do and 135 00:12:12,090 --> 00:12:12,390 run. 136 00:12:16,200 --> 00:12:20,260 And so these are the results, lists its items. 137 00:12:21,000 --> 00:12:33,450 So that's the list we wrote a function which gets the news, it gets the news, titles, avenues, descriptions 138 00:12:33,690 --> 00:12:36,510 and stores them in a Python list. 139 00:12:37,110 --> 00:12:40,500 Of course, you can store these results in an Excel file. 140 00:12:40,560 --> 00:12:47,370 See, as we there are videos in the course on how to work with CAC files, so that could be an exercise 141 00:12:47,370 --> 00:12:50,580 for you to do to improve the scripts even further. 142 00:12:51,120 --> 00:12:56,040 But you can also make a web app and show movies news on that web app. 143 00:12:56,700 --> 00:13:00,270 So that would be another exercise if you like to build with apps. 144 00:13:00,960 --> 00:13:02,610 And I thank you for filling out. 145 00:13:02,760 --> 00:13:03,480 Talk to you later.