Hosting media files on S3

Instead of hosting the media files of the family tree on the same server as the Gramps database, they can also be hosted on Amazon S3 or a different API-compatible object storage service. This strongly reduces storage and traffic requirements for the Gramps Web server.

Preparing credentials

In your AWS (or other object storage service) account, get an access key ID and secret access key (you may first want to create a separate user with a security policy allowing only access to S3).

Upload media files

To upload your existing media files to S3, on Linux you can use a local Gramps installation with the S3 Media Uploader Addon, that you can install directly from Gramps desktop. However, some additional configuration is necessary.

The addon uses the boto3 Python library under the hood. You need to install it first with

python3 -m pip install boto3

On the command line, export the access key ID and secret key to the appropriate environment variables:

export AWS_ACCESS_KEY_ID=my_access_key_id
export AWS_SECRET_ACCESS_KEY=my_secret_access_key

Now, you can start the uploader by runing the following command,

gramps -O "My Family Tree" -a tool \
    -p "name=s3uploader,bucket_name=my_bucket_name"

replacing "My Family Tree" with the name of your family tree and my_bucket_name with the complete name of your S3 bucket. You will need a dedicated S3 bucket for your Gramps Web instance. If the bucket does not exist yet, the S3 Uploader Addon will attempt to create it.

Configure Gramps Web

To use the new S3 bucket as media source for Gramps Web, simply set the MEDIA_BASE_DIR configuration option to s3://my_bucket_name.

When using Docker Compose, the easiest option is to add all environment variables to the env block:

env:
  AWS_ACCESS_KEY_ID: my_access_key_id
  AWS_SECRET_ACCESS_KEY: my_secret_access_key
  MEDIA_BASE_DIR: s3://my_bucket_name
  AWS_DEFAULT_REGION: eu-central-1

If you want to use an S3-compatible object storage other than AWS (e.g. GCP or for local testing), set the AWS_ENDPOINT_URL environment variable.

Synchronizing media files

To keep media files in sync between a local Gramps installation and S3-hosted Gramps Web media files, the Gramps Web Sync Addon supports file synchronization via the Web API.