W hen making requests in Python, you may sometimes need to upload large files like videos, audio, or other binary data. Rather than loading the entire file contents into memory, you can upload data in chunks using a file-like object.

This saves memory and allows streaming uploads even for very large files.

Why Use File-Like Objects?

Uploading data using the files argument in Requests loads the full contents of the file into memory. This can cause problems for large files:

with open('large_video.mp4', 'rb') as f:
    requests.post('https://example.com/upload', files={'video': f}) 

Instead, we can create a file-like object that streams the data in chunks:

with open('large_video.mp4', 'rb') as f:
    requests.post('https://example.com/upload', data=f)

This streams the file data in memory-efficient chunks without loading the entire file.

Creating a File-Like Object in Python

Many built-in Python file objects already support file-like streaming uploads:

with open('data.bin', 'rb') as f:
    requests.post(url, data=f)

You can also create a custom file-like object by implementing Python's file-like API:

import io
class StreamingBody:
    def __init__(self, filepath):
        self.filepath = filepath
    def read(self, n):
        # custom read implementation
    def seek(self, offset):
        # custom seek implementation
stream = StreamingBody('video.mp4')
requests.post(url, data=stream)

with open('report.pdf', 'rb') as f:
   requests.post(url, files={'report': f})