S3-compatible clients

S3

Use Boto3 with nuFS drives for uploads, downloads, listings and versioned reads.

Configure Boto3

Use your deployment's S3 origin, bucket identity and S3 signing credentials. The signing secret is different from a REST bearer token. Set the environment values before running the examples.

Python

import os
import boto3
from botocore.config import Config

s3 = boto3.client(
    "s3",
    endpoint_url=os.environ["NUFS_ENDPOINT"],
    aws_access_key_id=os.environ["NUFS_S3_ACCESS_KEY_ID"],
    aws_secret_access_key=os.environ["NUFS_S3_SECRET_ACCESS_KEY"],
    region_name="us-east-1",
    config=Config(
        signature_version="s3v4",
        s3={"addressing_style": "path"},
        request_checksum_calculation="when_required",
    ),
)
bucket = os.environ["NUFS_BUCKET"]

List files

Use a paginator to read bounded listing pages. Prefix filters the folder path.

Python

pages = s3.get_paginator("list_objects_v2")
for page in pages.paginate(Bucket=bucket, Prefix="assets/"):
    for item in page.get("Contents", []):
        print(item["Key"], item["Size"])

Upload and download

The managed transfer helpers support multipart uploads for larger files.

Python

s3.upload_file("./scene-01.mp4", bucket, "assets/scene-01.mp4")
s3.download_file(bucket, "assets/scene-01.mp4", "./downloaded.mp4")

s3.put_object(Bucket=bucket, Key="notes.txt", Body=b"hello")

Inspect a file and read a range

Range uses inclusive byte offsets. Close streaming bodies after reading.

Python

metadata = s3.head_object(Bucket=bucket, Key="notes.txt")
print(metadata["ContentLength"], metadata.get("VersionId"))

result = s3.get_object(Bucket=bucket, Key="notes.txt", Range="bytes=0-4")
try:
    data = result["Body"].read()
    print(data)
finally:
    result["Body"].close()

Read a saved version

Use the VersionId returned by a write, HEAD or version listing. This reads that version's retained contents.

Python

saved = s3.put_object(Bucket=bucket, Key="notes.txt", Body=b"original")
version_id = saved["VersionId"]
s3.put_object(Bucket=bucket, Key="notes.txt", Body=b"revised")

result = s3.get_object(Bucket=bucket, Key="notes.txt", VersionId=version_id)
try:
    print(result["Body"].read())
finally:
    result["Body"].close()

Generate temporary download and upload URLs

Presigned URLs are bearer capabilities. Keep them private and use the method they were signed for.

Python

download_url = s3.generate_presigned_url(
    "get_object",
    Params={"Bucket": bucket, "Key": "notes.txt"},
    ExpiresIn=300,
)
upload_url = s3.generate_presigned_url(
    "put_object",
    Params={"Bucket": bucket, "Key": "new-notes.txt"},
    ExpiresIn=300,
    HttpMethod="PUT",
)

Delete a current file

This deletes the current object. Current permissions also apply to historical reads.

Python

s3.delete_object(Bucket=bucket, Key="notes.txt")