Writing A File To S3 Using Lambda In Python With AWS

In AWS, I'm trying to save a file to S3 in Python using a Lambda function. While this works on my local computer, I am unable to get it to work in Lambda. I've been working on this

Solution 1:

Assuming Python 3.6. The way I usually do this is to wrap the bytes content in a BytesIO wrapper to create a file like object. And, per the boto3 docs you can use the-transfer-manager for a managed transfer:

from io import BytesIO
import boto3
s3 = boto3.client('s3')

fileobj = BytesIO(response.content)

s3.upload_fileobj(fileobj, 'mybucket', 'mykey')

If that doesn't work I'd double check all IAM permissions are correct.

Solution 2:

You have a writable stream that you're asking boto3 to use as a readable stream which won't work.

Write the file, and then simply use bucket.upload_file() afterwards, like so:

s3 = boto3.resource('s3')
bucket = s3.Bucket('transportation.manifests.parsed')
with open('/tmp/output2.csv', 'w') as data:
    data.write(response.content)

key = 'csv/' + key
bucket.upload_file('/tmp/output2.csv', key)