Is There A Way To Iterate Through S3 Object Content Using A Sql Expression?

I would like to iterate through each s3 bucket object and use a sql expression to find all the content that match the sql. I was able to create a python script that lists all the o

Solution 1:

Take a look at Amazon Athena – Interactive SQL Queries for Data in Amazon S3

Solution 2:

You may want to look at S3 Batch Operations which will allow you to execute your python code (deployed to lambda) on all the objects in your bucket(s)

With this solution, you won't need to list the objects in the bucket but can have AWS run your script on all of your objects.

Solution 3:

S3 Select is also an option but Athena would be easier.