HTTP Requests in Python (GET, POST, PUT, PATCH, DELETE)

John on October 06, 2020

Sending HTTP requests to a remote server has always been a tricky task due to the huge variety of different requirements that need to be fulfilled for each server and the complexity of implementing such features.

Performing HTTP GET, POST, PUT, PATCH .etc requests is made much easier with the Python requests module. With it, we can write simple or complex HTTP requests while maintaining clean, easy-to-read code.

In this tutorial, we will learn how to use the Python requests module for a variety of different request types with examples.

Installing and Importing the requests Module

Chances are your version of Python will have the requests packages preinstalled. If not, you can install it from the command-line like this:

pip install requests

Then import it into your program like this:

import requests

HTTP GET Requests

The easiest type of request to perform is a basic GET request. Let's say we wanted to get the JSON content from this URL:

https://www.skillsugar.com/examples/json/example_1.json

We could do this using the requests.get() method and storing the response in a variable.

import requests

url = 'https://www.skillsugar.com/examples/json/example_1.json'

result = requests.get(url)

The result variable in the above example is a Response object, which has properties containing different values from the response and methods for parsing data types. 

The type of content in the above example is JSON, which we can parse using the json() method:

print(result.json())

{'fruit': 'Apple', 'size': 'Large', 'color': 'Red'}

Here are some useful properties and methods available on the requests Response object to obtain data:

• result.encoding - the encoding that the document is using (UTF-8 .etc)
• result.headers - the response headers in JSON format
• result.cookies - the request cookies
• result.status_code - the status of the response in a code format (200, 500, 404 .etc.)
• result.content - the content of the response in binary format
• result.text - the decoded content of the response according to the response encoding header property
• result.is_redirect - True if the response status code is a redirect, False if not
• result.elapsed - the time between sending the request and getting a response
• result.url - the URL of the request
• result.json() - parse JSON content

Passing Parameters into a Request

It is common to send parameters in an HTTP request, this can be done by creating a dictionary and passing that as an argument with the value params= followed by the parameters.

import requests

url = 'https://www.skillsugar.com/search'

params = {'query': 'python', 'sort': 'date'}

result = requests.get(url, params=params)

print(result.url)

https://www.skillsugar.com/search?query=python&sort=date

It is much cleaner passing parameters in a request by using a dictionary as not only is it easier to read but you don't have to worry about joining the parameters manually.

HTTP POST Requests

An HTTP POST request is typically used to send a body of data to a server. We can create them with the requests package by using the requests.post() method.

import requests

url = 'https://www.skillsugar.com/contact'

params = {'name': 'john', 'subject' : 'Test', 'message': 'Hello', 'email': '[email protected]'}

result = requests.post(url, params=params)

print(result.headers)

Sending Data in the Request

To send a data payload in the request, use the data= argument. Let's imagine we have some JSON that should be sent across in a POST request, we could do it like this:

import requests

url = 'https://www.skillsugar.com/api'

json_data = {'fruit': 'Apple', 'size': 'Large', 'color': 'Red'}

result = requests.get(url, data=json_data)

Encode a JSON Data Payload

It is possible to encode a JSON payload by using the json= argument.

import requests

url = 'https://www.skillsugar.com/api'

payload = {'fruit': 'Apple', 'size': 'Large', 'color': 'Red'}

result = requests.get(url, json=payload)

note - the JSON argument will be ignored if either the data= or files= arguments are present.

How to Send Files in a Request

To send files with the requests package, create a dictionary of files to send with each value being a file object created with the open() function in 'rb' (read binary) mode. Then use the files= argument to pass the dictionary of files.

url = 'https://www.skillsugar.com/api'

files = {'file': open('test.txt', 'rb')}

result = requests.post(url, files=files)

print(result.status_code)

How to Add Headers and Cookies to a Request

As well as receiving cookies in the response, the requests package also allows you to send custom headers and cookies. We can create cookies and header properties in dictionaries before adding them to the request with the headers= and cookies= arguments.

import requests

url = 'https://www.skillsugar.com/api'

headers = {'user-agent': 'agent-name'}
cookies = {'content': 'Some content'}

result = requests.get(url, headers=headers, cookies=cookies)

A more comprehensive way of adding cookies can be done using the requests.cookies.RequestsCookieJar() function. First, create a RequestsCookieJar() object and then use the set() method to create as many cookies as needed before adding them to the request.

import requests

url = 'https://www.skillsugar.com/api'

jar = requests.cookies.RequestsCookieJar()
jar.set('cookie_1', 'first', domain='skillsugar.com', path='/cookies')
jar.set('cookie_1', 'second', domain='skillsugar.com', path='/somwhere')

result = requests.get(url, cookies=jar)

How to Send Authorization Token in a Request

To send a custom authorisation header token, create an Authorization key/value pair in a dictionary and pass it VIA the headers= argument.

import requests

url = 'https://www.skillsugar.com/api'

headers = {'Authorization': 'Bearer token_value'}

result = requests.get(url, headers=headers)

HTTP PUT Requests

To send an HTTP PUT request, use the requests.put() method.

result = requests.put('https://www.skillsugar.com/put', data = {'key':'value'})

HTTP PATCH Requests

To send an HTTP PATCH request, use the requests.patch() method.

result = requests.patch('https://www.skillsugar.com/patch', params = {'key':'value'})

HTTP DELETE Requests

To send an HTTP DELETE request, use the requests.put() method.

result = requests.delete('https://www.skillsugar.com/delete', params = {'key':'value'})

Conclusion

You now know how to use the Python requests package to send HTTP's requests in several different ways.