Implement correct S3-compatible API usage patterns across S3, R2, Wasabi, MinIO, and GCS interop. Use when: S3 compatible storage, S3 API patterns, AWS SDK S3, R2 S3 compatible, Wasabi S3, MinIO S3, S3 client configuration, S3 multipart upload, S3 SDK usage, S3 API integration, S3 presigned URL SDK, boto3 S3, aws-sdk-go S3, S3 CORS configuration, S3 bucket policy, S3 compatible client.
| SDK | Language | Compatible Providers |
|---|---|---|
aws-sdk-go-v2 | Go | S3, R2, Wasabi, MinIO |
boto3 / aioboto3 | Python | S3, R2, Wasabi, MinIO |
@aws-sdk/client-s3 v3 |
| Node.js / TS |
| S3, R2, Wasabi, MinIO |
aws-sdk for Java | Java | S3, R2, Wasabi, MinIO |
| Presign via SDK | Any | S3, R2, Wasabi (R2 needs custom endpoint) |
For non-AWS providers, always require endpoint_url / endpoint override — the SDK defaults to s3.amazonaws.com and will silently fail.
aws-sdk-go-v2)cfg, _ := config.LoadDefaultConfig(ctx,
config.WithRegion(os.Getenv("STORAGE_REGION")), // e.g. "auto" for R2
config.WithCredentialsProvider(credentials.NewStaticCredentialsProvider(
os.Getenv("STORAGE_ACCESS_KEY_ID"),
os.Getenv("STORAGE_SECRET_ACCESS_KEY"),
"",
)),
config.WithEndpointResolverWithOptions(
aws.EndpointResolverWithOptionsFunc(func(service, region string, opts ...interface{}) (aws.Endpoint, error) {
return aws.Endpoint{URL: os.Getenv("STORAGE_ENDPOINT_URL"), SigningRegion: region}, nil
}),
),
)
client := s3.NewFromConfig(cfg, func(o *s3.Options) {
o.UsePathStyle = true // Required for MinIO and Wasabi
})
boto3)import boto3
client = boto3.client(
"s3",
region_name=os.environ["STORAGE_REGION"],
endpoint_url=os.environ.get("STORAGE_ENDPOINT_URL"), # None = default AWS
aws_access_key_id=os.environ["STORAGE_ACCESS_KEY_ID"],
aws_secret_access_key=os.environ["STORAGE_SECRET_ACCESS_KEY"],
)
@aws-sdk/client-s3 v3)import { S3Client } from "@aws-sdk/client-s3";
const s3 = new S3Client({
region: process.env.STORAGE_REGION,
endpoint: process.env.STORAGE_ENDPOINT_URL, // omit for AWS
credentials: {
accessKeyId: process.env.STORAGE_ACCESS_KEY_ID!,
secretAccessKey: process.env.STORAGE_SECRET_ACCESS_KEY!,
},
forcePathStyle: true, // Required for MinIO and Wasabi
});
Checklist:
endpoint_url / endpoint configurable per environmentuse_path_style / forcePathStyle set to true for MinIO and Wasabi"auto" for R2, actual AWS region for S3Use multipart upload for files >5 MB. Never buffer the entire file in memory.
Minimum part size: 5 MB (except the last part)
Maximum parts: 10,000
Maximum object: 5 TB
| File Size | Upload Method |
|---|---|
| < 5 MB | Single PutObject |
| 5 MB – 5 GB | Multipart upload (SDK managed, e.g., transfer.Upload in Go) |
| > 5 GB | Multipart upload (mandatory) |
Go — use the Transfer Manager:
uploader := manager.NewUploader(client, func(u *manager.Uploader) {
u.PartSize = 10 * 1024 * 1024 // 10 MB parts
u.Concurrency = 3
})
_, err := uploader.Upload(ctx, &s3.PutObjectInput{
Bucket: aws.String(bucket),
Key: aws.String(key),
Body: reader,
ContentType: aws.String(contentType),
})
io.ReadAll into memoryContentType on every upload — do not rely on auto-detection| Pattern | Example | Notes |
|---|---|---|
| UUID-based | uploads/a3f8b2c1-....pdf | Safest — no guessability |
{tenant}/{uuid} | t_abc123/uploads/uuid.pdf | Prefix-based isolation |
{date}/{uuid} | 2026/03/uuid.pdf | Partition by time (useful for lifecycle rules) |
Rules:
[a-z0-9_\-/.])Required for browser direct-to-storage uploads (presigned PUT).
{
"CORSRules": [
{
"AllowedOrigins": ["https://app.yourproduct.com"],
"AllowedMethods": ["PUT", "GET", "HEAD"],
"AllowedHeaders": ["Content-Type", "Content-Length", "x-amz-*"],
"ExposeHeaders": ["ETag"],
"MaxAgeSeconds": 3600
}
]
}
AllowedOrigins is an explicit allowlist — never "*" on a private bucketPUT and GET allowed — never DELETE from the browserETag exposed so the client can confirm multipart assembly| Provider | Known issues |
|---|---|
| R2 | Region must be "auto"; endpoint format: https://<account_id>.r2.cloudflarestorage.com |
| Wasabi | path_style required; endpoint per region (s3.us-east-1.wasabisys.com); no requester-pays |
| MinIO | path_style required; TLS cert may be self-signed in dev |
| GCS (S3 interop) | Enable via "Interoperability" in GCS console; HMAC keys required; endpoint: https://storage.googleapis.com |
forcePathStyle set correctly per providerProduce:
STORAGE_ENDPOINT_URL, STORAGE_ACCESS_KEY_ID, etc.)