s3

Store and fetch objects in Amazon S3 and S3-compatible services such as R2 and MinIO, and hand out time-limited signed links.

S3 is object storage: a bucket holds objects, each found by a key. Amazon S3 started it, and Cloudflare R2, MinIO, Backblaze and others speak the same protocol. std:s3 uploads, downloads and deletes objects, and makes presigned URLs that let someone upload or download one object without your credentials.

A client is made without touching the network, so creating one and signing a link runs here. Calls that move data are norun.

Make a client#

s3.new(options) returns a client for one bucket. endpoint and bucket are required. Credentials, region and a public URL are optional:

import "std:s3"

auto bucket = s3.new({
  endpoint: "https://s3.example.com",
  bucket: "photos",
  region: "eu-west-1",
  accessKeyId: "AKIAEXAMPLE",
  secretAccessKey: "secret"
})

log bucket.bucket()
log bucket.endpoint()
log bucket.region()

Leave out endpoint and run it: s3.new throws a ValueError that names the missing option, because a client that can't go anywhere is a mistake in your code. Read keys from env.

Public URLs#

If the bucket is served from a CDN or a public domain, publicURL lets client.public(key) build the link people can open, escaping the key:

import "std:s3"

auto bucket = s3.new({
  endpoint: "https://s3.example.com",
  bucket: "photos",
  publicURL: "https://cdn.example.com"
})

log bucket.public("holiday/beach day.png")

Upload and download#

upload(key, data) stores a string or bytes, uploadFile(key, path) stores a file, and download(key) returns bytes. downloadFile(key, path) saves one to disk. A missing object is a NotFoundError:

import "std:s3"
import "std:env"

auto bucket = s3.new({
  endpoint: "https://s3.example.com",
  bucket: "photos",
  accessKeyId: env.get("S3_KEY"),
  secretAccessKey: env.get("S3_SECRET")
})

bucket.upload("notes/hello.txt", "hello from OSL", {contentType: "text/plain"}).unwrap()
byte[] data = bucket.download("notes/hello.txt").unwrap()
log data.len

switch bucket.download("notes/missing.txt") (
  case ok(bytes)
    log "found"
  case err(e)
    log "no such object:" e.kind
)

contentType is stored with the object, so browsers treat it right. mime can pick one from a file name.

Delete#

delete(key) removes an object. Deleting a key that doesn't exist succeeds, as it does in S3, so you don't need to check first:

import "std:s3"

auto bucket = s3.new({
  endpoint: "https://s3.example.com",
  bucket: "photos",
  accessKeyId: "...",
  secretAccessKey: "..."
})
bucket.delete("notes/hello.txt").unwrap()
bucket.delete("notes/hello.txt").unwrap()

Presigned URLs#

A presigned URL gives whoever holds it one kind of access to one object for a limited time, with no credentials shared. Hand a browser a URL to upload straight to the bucket, so a big file never passes through your server. expires is in milliseconds:

import "std:s3"

auto bucket = s3.new({
  endpoint: "https://s3.example.com",
  bucket: "photos",
  accessKeyId: "AKIAEXAMPLE",
  secretAccessKey: "secret"
})

string download = bucket.presignURL("a/b.png", {expires: 600000})
log download.startsWith("https://s3.example.com/photos/a/b.png?")
log download.contains("X-Amz-Signature=")
log download.contains("X-Amz-Expires=600")

string upload = bucket.presignURL("a/new.png", {method: "PUT", expires: 60000})
log upload.contains("X-Amz-Expires=60")

The signature depends on the time, so the URL itself differs on every run. Try a different expires and see the number in the link change. The longest a link can live is seven days.

Errors#

Failures name their kind: PermissionError for bad credentials, NetworkError and TimeoutError for connection trouble, NotFoundError for a missing object. A service that is down shows up like this:

import "std:s3"

auto bucket = s3.new({
  endpoint: "http://127.0.0.1:1",
  bucket: "b",
  accessKeyId: "x",
  secretAccessKey: "y",
  timeout: 1000
})
log bucket.download("anything").unwrapErr().kind

import "std:s3"

log false

import "std:s3"

log false

true

API reference#