Skip to content

Share pictures in a public S3 bucket

This tutorial explains how to use the upload_pics_to_s3_bucket.py script to share pictures in a public S3 bucket using static website hosting.

Danger

This page is under construction

Prerequisites

  1. AWS account with appropriate permissions:
  2. s3:PutPublicAccessBlock
  3. s3:PutBucketWebsite
  4. s3:PutBucketPolicy
  5. s3:PutObject

  6. s3:CreateBucket (if bucket doesn't exist)

  7. AWS credentials configured (either through environment variables, AWS CLI configuration, or IAM role)

  8. Python packages: 

    pip install boto3

Account-Level Settings

Before using the script, ensure that your AWS account allows public access to S3 buckets:

  1. Go to AWS Console > S3 > Block Public Access settings for this account
  2. Uncheck "Block all public access"
  3. Confirm your choice

Note: This is an account-level security setting that cannot be changed programmatically.

Using the Script

The script is located at python/scripts/upload_pics_to_s3_bucket.py. It handles: - Creating/configuring the S3 bucket - Setting up static website hosting - Uploading pictures with proper permissions - Maintaining a manifest of uploaded files

Basic Usage

# Upload pictures, using folder name as destination
python upload_pics_to_s3_bucket.py my-bucket-name /path/to/pics/of/butterflies

# Upload pictures to a specific destination
python upload_pics_to_s3_bucket.py my-bucket-name /path/to/pics/of/istanbul -d constantinople

What the Script Does

  1. Bucket Configuration
  2. Creates the bucket if it doesn't exist
  3. Disables public access blocks at the bucket level
  4. Configures static website hosting

  5. Sets a bucket policy allowing public read access

  6. File Management

  7. Scans the local directory for images
  8. Computes MD5 hashes to detect changes
  9. Only uploads new or modified files

  10. Maintains a manifest.json tracking uploaded files

  11. URL Generation

  12. Provides a website endpoint URL for the bucket
  13. Generates URLs for each uploaded file
  14. Creates a publicly accessible manifest

Example Output

Disabled block public access settings
Bucket website endpoint: http://my-bucket-name.s3-website-us-west-2.amazonaws.com
Uploaded butterfly1.jpg
Website URL: http://my-bucket-name.s3-website-us-west-2.amazonaws.com/butterflies/butterfly1.jpg
Uploaded butterfly2.jpg
Website URL: http://my-bucket-name.s3-website-us-west-2.amazonaws.com/butterflies/butterfly2.jpg
Updated manifest at: http://my-bucket-name.s3-website-us-west-2.amazonaws.com/butterflies/manifest.json

Supported File Types

The script supports common image formats: - .jpg, .jpeg - .png - .gif - .bmp - .tiff - .webp

Security Considerations

  1. Public Access
  2. All uploaded files will be publicly readable
  3. Anyone with the URL can access the files

  4. Consider using pre-signed URLs for sensitive content

  5. Bucket Policy

  6. The bucket policy allows read-only access
  7. Write operations still require AWS credentials

  8. Policy is scoped to objects only, not bucket configuration

  9. Website Hosting

  10. Uses HTTP, not HTTPS (consider CloudFront for HTTPS)
  11. Allows directory listing if index.html exists
  12. Provides a custom error page for 404s

Troubleshooting

  1. AccessDenied Errors
  2. Check IAM user permissions
  3. Verify account-level S3 public access settings

  4. Ensure bucket policy is not blocked

  5. Website Endpoint Not Working

  6. Wait a few minutes for DNS propagation
  7. Verify bucket name follows DNS naming rules

  8. Check if error.html exists for error pages

  9. Files Not Uploading

  10. Verify file extensions are supported
  11. Check local file permissions
  12. Ensure sufficient disk space

Next Steps

  • Set up CloudFront distribution for HTTPS
  • Configure custom domain names
  • Implement image resizing
  • Add metadata to the manifest
  • Set up monitoring and alerts

For more information, refer to: - AWS S3 Documentation - S3 Static Website Hosting - S3 Bucket Policies