How to Read Json Data From Web Api Python

JSON and APIs with Python

An Introduction to JSON and APIs using Python

Photo by Sean Lim on Unsplash

Introduction

In a different tutorial, nosotros discussed how to web scrape with python. The goal of web scraping was to access data from a website or webpage. Well, sometimes a website can make it easier for a user to accept direct admission to their data with the use of an API (Application Programming Interface). This basically means that the company has made a prepare of dedicated URLs that provide this data in a pure form (meaning without any presentation formatting). This pure information is often in a JSON (JavaScript Object Note) format, which nosotros can so parse through and extract what we need using python.

For this tutorial, we will employ the free API institute at covid19api.com that provides data on the coronavirus. We volition discover the total number of confirmed cases in each country and then we volition create a pandas dataframe that contains that information. So allow's begin!

Inspecting the API

If you go to the documentation folio of the API, this is what you'll encounter:

This shows us the unlike URLs in the API, the data they provide, and example requests/responses of those URLs on the right.

We can see that the information we are seeking is in the summary folio. We can click on view more than on the right and then we can see what the response would be from that URL:

This is a JSON object! As you tin see, it is very similar to a python lexicon and is fabricated up of key-value pairs. In fact, in society for us to parse through this and excerpt what we want from it, we will eventually plough it into a python dictionary object. Upon inspection, we can see that information technology looks like a nested dictionary. The outer dictionary has the keys 'Global' (with a value of a dictionary) and 'Countries' (with a value of a list that is fabricated up of dictionaries, with each dictionary corresponding to a specific country).

Making an HTTP Asking from the API

And then permit'southward open a jupyter notebook and asking the information from that URL. We volition use the requests library to make an HTTP request from that URL and save the response object's text under the variable response:

              response = requests.become('https://api.covid19api.com/summary').text            

This shows what the response is to our HTTP request from the API. As you can come across, information technology is a long python string that is in JSON format.

Creating a Python Dictionary

Since the response is in JSON format, we tin can load this string into python and convert it into a python dictionary. We start need to import the json library, and so nosotros can employ the loads method from the json library and pass information technology our string:

              response_info = json.loads(response)            

Annotation how the type of our response_info variable is at present a python lexicon!

Now that our response is in the form of a python dictionary, nosotros tin can employ what we know nearly python dictionaries to parse information technology and extract the data nosotros need!

Also note: The requests library has a congenital-in JSON decoder that we could have used instead of the json module that would have converted our JSON object to a python dictionary. Nonetheless, I used the above method to introduce the json module in this tutorial. Here'south what the code would accept looked like if nosotros instead used the JSON decoder inside the requests module:

              requests.get('https://api.covid19api.com/summary').json()            

Parsing the Dictionary

As previously mentioned, nosotros would like to brand a pandas dataframe that has two columns: countries, and the number of total confirmed cases for that country. Nosotros can do and so by looping through the values of the 'Countries' central of our outer dictionary:

Every bit you lot can run across, the value of our 'Countries' key is just a list of dictionaries, with each lexicon containing fundamental-value pairs corresponding to a specific country. So we need to loop through this list of dictionaries, extracting the values of the 'State' and 'TotalConfirmed' keys from each dictionary so appending them to a new list as follows:

              country_list = []              for country_info in response_info['Countries']:
                country_list.suspend([country_info['Country'], country_info['TotalConfirmed']])            

This will loop through the list of dictionaries, extracting the values from the 'Country' and 'TotalConfirmed' keys from each dictionary into a list, and so calculation this resulting list to our country_list. We will end upwards with a list of lists, with each list or element in the outer list containing the country name and the full confirmed cases for that specific country.

Creating a Pandas DataFrame

We will now create a pandas dataframe using this country_list and the pandas DataFrame constructor:

              country_df = pd.DataFrame(data=country_list, columns=['Country', 'Total_Confirmed'])            

Success! We now take a dataframe that contains 2 columns: Country and Total_Confirmed!

Determination

In this tutorial, nosotros had a cursory introduction to what APIs and JSON are. Nosotros and then made an HTTP request to a Coronavirus COVID19 API to get information on the number of total confirmed coronavirus cases in each land. We then converted this JSON response to our request into a python dictionary. We so parsed through this lexicon, extracting the information we were seeking, and then created a pandas dataframe containing this information.

browngiffer.blogspot.com

Source: https://towardsdatascience.com/json-and-apis-with-python-fba329ef6ef0

Share this post