How to Work With JSON Responses Without Documentation

What will you learn?

You need to use this REST API which gives JSON-response – the docs are missing?

What do you do?

Everybody knows you need to write your documentation – but even large enterprise companies like Microsoft have almost no documentation on their REST API – so what do you do? In this tutorial you will learn an easy way to figure out the structure of the JSON response you get from most REST APIs.

Step 1: Use a debugger to figure out the structure of the response

Most REST API’s give responses in JSON. While JSON is a structured way to represent data, it can be difficult to figure out how it is structured.

Many REST API’s are not well documented, or are not up to date. This can make it challenging to figure out how to interpret the result.

A great way to do that is to it is to use a debugger to figure out the structure. An IDE like PyCharm does a great job at that and works out of the box after installing it (download it here use the Community version, as it is free).

Step 2: Let’s try an example

The best way to learn is by trying it out yourself.

You can clone my GitHub repository here. After you clone it, make sure to have an virtual environment with Python and install the requirements (all explained in the video).

Alternatively, you just need to install the library yahoofinancials. This is done as follows.

pip install yahoofinancials

Then create a file with the following content.

import json
from yahoofinancials import YahooFinancials

def get_statement(ticker, frequency='quartely', statement='income'):
    yahoo_financials = YahooFinancials(ticker)
    return yahoo_financials.get_financial_stmts(frequency, statement)

my_ticker = 'AAPL'
income_statement = get_statement(my_ticker)
print(json.dumps(income_statement, indent=2))

This will serve as our example. As you see, we call YahooFinancial (which strictly speaking is not a REST API, but it returns JSON).

Then we print the JSON-response.

This is a simple example, but below we see the output, then we will dive into how get the data we want.

{
  "incomeStatementHistoryQuarterly": {
    "AAPL": [
      {
        "2022-06-25": {
          "researchDevelopment": 6797000000,
          "effectOfAccountingCharges": null,
          "incomeBeforeTax": 23066000000,
          "minorityInterest": null,

Step 3: Use the debugger to find the structure

Many JSON responses are more complex than the above (which is only a small sample of the output).

A great way to find the structure is to use the debugger, by setting a break point.

Mark a break-point at the print statement and press debugger.

As you see, you can unfold the structure of the JSON response.

This shows that you have a dictionary with the ticker name, then it follows a list of income statements. Each income statement has a structure too (not unfolded in the example).

Step 5: Extract the values the correct way

The best way to get values from a dictionary is by using get. This will give you the chance to give a default return value, if the key is not present in the dictionary. The best way is to return an empty dictionary {}, as the program can then proceed without failing.


Line 13 and 14 ensure the the code will run no matter if the keys are present in the JSON-response. This way the code on lines 16-18 will not fail either.

Conclusion

Using a debugger to understand the structure of a JSON response from a REST API is a great fast way to do it. Also, using the dictionary method get is a great way to avoid programs to crash, if the key is not present in a call.

Leave a Reply

%d bloggers like this: