# S3 Multisite

> This is a brief introduction to our replicated S3 Storage

---

Site [llms.txt](/llms.txt)

---

We offer Ceph RGW Multisite support for our S3 object storage service at [https://s3.uni-muenster.de](https://s3.uni-muenster.de).
The system is deployed across our current two locations. **Einsteinstraße** and **Schlossplatz**.

Your data is **asynchronously replicated** between both sites to ensure geographical redundancy and disaster recovery capabilities. In case of a failure at one site, we can switch access to the other site to maintain availability.

For technical details on how Ceph RGW Multisite works, see the official documentation: [Ceph RGW Multisite Guide](https://docs.ceph.com/en/latest/radosgw/multisite/)

## Object Count Limits per Bucket in Multisite Setup

Ceph RGW uses bucket index sharding to distribute object metadata across the cluster and can approximately store **100,000 objects per shard**.

If the number of objects exceeds the optimal number of shards, Ceph RGW can [dynamically reshard](https://docs.ceph.com/en/latest/radosgw/dynamicresharding/) the bucket index to increase the number of shards in that bucket.

Unfortunately, **dynamic resharding does not work for multisite setups** in our current Ceph version ([source](https://docs.ceph.com/en/latest/radosgw/dynamicresharding/)).

This means the number of shards — and therefore the maximum number of objects — needs to be known **beforehand**.

Our current default is **8 shards per bucket**, which translates to a maximum of **approximately 800,000 objects per bucket**.

If you plan to store more than 800,000 objects in a single bucket, please contact us **beforehand** with the bucket name and the expected maximum number of objects.

We will then create the bucket and configure the appropriate shard count manually.

**Please note:** Once a bucket is created, the **shard count cannot be changed later**.

### Why not just use a high shard count by default?

Using a high number of shards increases the complexity and resource usage of the system.
Each shard requires additional metadata storage and can increase the load on the cluster, potentially reducing overall performance for buckets with fewer objects.

This especially affects operations that need to process all shards, such as synchronization between sites (syncing) or listing objects (list operations), which become more resource-intensive and slower with more shards.

Therefore, setting a very high shard count by default is inefficient and can negatively impact performance, especially for buckets with a smaller number of objects.
This is why we prefer to configure shard counts based on your specific storage needs.